#python

posts tagged #python

0

Python keeps the proxy status that Chrome throws away, so don't retry every ProxyError

preview

Got ERR_TUNNEL_CONNECTION_FAILED in Chrome and no idea why? Try the same proxy from Python, because it’ll tell you what the browser won’t. Chromium drops every CONNECT reply except 200 and 407, so you can’t tell a policy block from a dead exit in there.

Python leaves the status in the exception text. You’ll see OSError: Tunnel connection failed: 502 Bad Gateway from http.client, and requests wraps that in a ProxyError with the same words inside.

I’d read that string before retrying anything, since a blanket except ProxyError plus a retry loop just burns time on errors that can’t heal. A 403 is usually policy, sometimes just a port they don’t allow. 407 means check your login. If you get 405 or 501 you aren’t talking to a forward proxy at all. Only 502, 503 and 504 are worth backing off on, those happened behind the proxy.

Disclosure: we run a proxy service, none of this depends on whose.

Got ERR_TUNNEL_CONNECTION_FAILED in Chrome and no idea why? Try the same proxy from Python, because it’ll tell you what the browser won’t. Chromium drops every CONNECT reply except 200 and 407, so you can’t tell a policy block from a dead exit in there.

Python leaves the status in the exception text. You’ll see OSError: Tunnel connection failed: 502 Bad Gateway from http.client, and requests wraps that in a ProxyError with the same words inside.

I’d read that string before retrying anything, since a blanket except ProxyError plus a retry loop just burns time on errors that can’t heal. A 403 is usually policy, sometimes just a port they don’t allow. 407 means check your login. If you get 405 or 501 you aren’t talking to a forward proxy at all. Only 502, 503 and 504 are worth backing off on, those happened behind the proxy.

Disclosure: we run a proxy service, none of this depends on whose.

0

Google's Custom Search JSON API shuts down on Jan 1, 2027 (and a free alternative for your own rankings)

preview

If you’ve got anything running on Google’s Custom Search JSON API, it’s on a clock. Google closed it to new customers and their overview page says it’s gone on January 1, 2027.

Easy to miss in the docs: one call returns 10 results and a query tops out at 100. So pulling the top 100 for a single keyword is 10 billable calls.

For checking your own site’s rankings there’s an easier path anyway. The Search Console API hands you query, page, country and device rows, up to 25,000 per request, and it’s all within Google’s rules.

Anyone already moved off Custom Search? What did you switch to?

If you’ve got anything running on Google’s Custom Search JSON API, it’s on a clock. Google closed it to new customers and their overview page says it’s gone on January 1, 2027.

Easy to miss in the docs: one call returns 10 results and a query tops out at 100. So pulling the top 100 for a single keyword is 10 billable calls.

For checking your own site’s rankings there’s an easier path anyway. The Search Console API hands you query, page, country and device rows, up to 25,000 per request, and it’s all within Google’s rules.

Anyone already moved off Custom Search? What did you switch to?

100

Broccoli2Shell: the Brotli decoder bug (CVE-2026-111497) - Affecting Node.js, Python, C library

Broccoli2Shell Broccoli2Shell: the Brotli decoder bug (CVE-2026-111497) broccoli2shell.com
preview

CVE-2026-111497 (Broccoli2Shell, aka Node2Shell): an out-of-bounds write in Google’s Brotli decoder affecting Node.js, Python Brotli/brotlicffi and anything built on the C decoder. Affected versions, how to check, and how to fix it.

CVE-2026-111497 (Broccoli2Shell, aka Node2Shell): an out-of-bounds write in Google’s Brotli decoder affecting Node.js, Python Brotli/brotlicffi and anything built on the C decoder. Affected versions, how to check, and how to fix it.

0

A plea not to use uv-build for Python

Michał Górny A plea not to use uv-build blogs.gentoo.org
preview

@mgorny@treehouse.systems:

After seeing yet another dozen packages lose keywords on Gentoo because of a single key dependency switching to uv-build, I would like to plea on behalf of Gentoo users: please don’t use uv-build as a build backend.

Just to be clear, this is not about uv itself or any other Astral tools; these are good tools with a nice and responsive upstream. The uv-build backend is one thing we have to disagree on. I’m not saying it’s bad; I’m saying that its benefits don’t outweigh the drawbacks.

@mgorny@treehouse.systems:

After seeing yet another dozen packages lose keywords on Gentoo because of a single key dependency switching to uv-build, I would like to plea on behalf of Gentoo users: please don’t use uv-build as a build backend.

Just to be clear, this is not about uv itself or any other Astral tools; these are good tools with a nice and responsive upstream. The uv-build backend is one thing we have to disagree on. I’m not saying it’s bad; I’m saying that its benefits don’t outweigh the drawbacks.

0

Hard Python Dictionary Exercise

preview

A hard exercise to help build the right mental model for Python data.

The “Solution” link visualizes execution and reveals what’s actually happening using 𝗺𝗲𝗺𝗼𝗿𝘆_𝗴𝗿𝗮𝗽𝗵: github.com/bterwijn/memory_graph

A hard exercise to help build the right mental model for Python data.

The “Solution” link visualizes execution and reveals what’s actually happening using 𝗺𝗲𝗺𝗼𝗿𝘆_𝗴𝗿𝗮𝗽𝗵: github.com/bterwijn/memory_graph

0

iceoryx2 0.10 released: full flatbuffer integration, zero copy ipc with unbounded data, robust event

ekxide Announcing iceoryx2 v0.10.0 ekxide.io
preview

geteilt von: programming.dev/post/56750254

geteilt von: programming.dev/post/56750248

Checkout the release notes: ekxide.io/blog/iceoryx2-0.10-release/

Git: github.com/eclipse-iceoryx/iceoryx2

iceoryx2 book: ekxide.github.io/iceoryx2-book/main/

geteilt von: programming.dev/post/56750254

geteilt von: programming.dev/post/56750248

Checkout the release notes: ekxide.io/blog/iceoryx2-0.10-release/

Git: github.com/eclipse-iceoryx/iceoryx2

iceoryx2 book: ekxide.github.io/iceoryx2-book/main/

0

How does Lazy Evaluation work in Python?

preview

In last week’s post we showed that a Python for-loop works through the Iterator Protocol.

This same protocol also enables values to be produced lazily.

In this new Lazy Evalution Example we show one sink reading values from five different sources:

  • source1: eagerly returns all values by list
  • source2: lazily by generator function using yield
  • source3: eagerly by list comprehension
  • source4: lazily by generator expression
  • source5: lazily by iterator protocol

The same for-loop consumes all five sources. The eager sources produce all values before the sink starts consuming them. In the lazy sources producer and consumer take turns:

  • produce → consume → produce → consume → …

Each value is produced only when the for-loop requests it.

Generator functions and generator expressions are concise, readable ways to create lazy iterables. The final source makes their underlying mechanism explicit:

  • iter() obtains an iterator.
  • next() requests the next value.
  • StopIteration signals that no values remain.

This clearly shows that generators achieve lazy iteration by implementing Python’s Iterator Protocol.

In last week’s post we showed that a Python for-loop works through the Iterator Protocol.

This same protocol also enables values to be produced lazily.

In this new Lazy Evalution Example we show one sink reading values from five different sources:

  • source1: eagerly returns all values by list
  • source2: lazily by generator function using yield
  • source3: eagerly by list comprehension
  • source4: lazily by generator expression
  • source5: lazily by iterator protocol

The same for-loop consumes all five sources. The eager sources produce all values before the sink starts consuming them. In the lazy sources producer and consumer take turns:

  • produce → consume → produce → consume → …

Each value is produced only when the for-loop requests it.

Generator functions and generator expressions are concise, readable ways to create lazy iterables. The final source makes their underlying mechanism explicit:

  • iter() obtains an iterator.
  • next() requests the next value.
  • StopIteration signals that no values remain.

This clearly shows that generators achieve lazy iteration by implementing Python’s Iterator Protocol.

0

IPython is All You Need

preview

“I use IPython as my terminal’s shell.”

“IPython in the shell?”

“No, IPython is the shell.”

“IPython? As the shell?”

“Only way to live.”

“I use IPython as my terminal’s shell.”

“IPython in the shell?”

“No, IPython is the shell.”

“IPython? As the shell?”

“Only way to live.”

0

Adding objects to set or dictionary: equality and hashing

preview

Yesterday my exercise got downvoted, probably because its point was unclear. I’d like to try again with this reworked exercise. Hope you’ll like this one better.

  • Solution
  • Explanation: “User-defined classes have __eq__() and __hash__() methods by default (inherited from the object class); with them, all objects compare unequal (except with themselves) and x.__hash__() returns an appropriate value such that x == y implies both that x is y and hash(x) == hash(y).”

#Python #memory_graph #Equality #Hashing

Yesterday my exercise got downvoted, probably because its point was unclear. I’d like to try again with this reworked exercise. Hope you’ll like this one better.

  • Solution
  • Explanation: “User-defined classes have __eq__() and __hash__() methods by default (inherited from the object class); with them, all objects compare unequal (except with themselves) and x.__hash__() returns an appropriate value such that x == y implies both that x is y and hash(x) == hash(y).”

#Python #memory_graph #Equality #Hashing

0

Adding objects to set or dictionary: equality and hashing

preview

An exercise to help build the right mental model for Python data.

  • Solution
  • Explanation: “User-defined classes have __eq__() and __hash__() methods by default (inherited from the object class); with them, all objects compare unequal (except with themselves) and x.__hash__() returns an appropriate value such that x == y implies both that x is y and hash(x) == hash(y).”

#Python #memory_graph #Equality #Hashing

An exercise to help build the right mental model for Python data.

  • Solution
  • Explanation: “User-defined classes have __eq__() and __hash__() methods by default (inherited from the object class); with them, all objects compare unequal (except with themselves) and x.__hash__() returns an appropriate value such that x == y implies both that x is y and hash(x) == hash(y).”

#Python #memory_graph #Equality #Hashing

0

5 tools that helped me modernize a legacy app

Marmelab From Legacy to Lightning: How To Modernize A Python App marmelab.com
preview

So I recently inherited a legacy application that was a nightmare to maintain (tooling was clearly lacking and the codebase was pretty outdated). I quickly realized that if I ever wanted to actually enjoy working on it, I would have to give it a proper overhaul.

These are the 5 changes that had some of the biggest impact IMO:

  1. uv: Being new to the Python ecosystem, I didn’t want to figure out pip vs poetry vs pyenv vs virtualenv, so I just used uv and let it handle all of that. One Rust-based tool that installs and pins Python versions, manages the venv automatically and locks deps in a uv.lock for reproducible builds. Installs are also a lot faster, which makes CI a lot less painful.

  2. Ruff:

Coming from JS, I really missed eslint --fix for automatically fixing linting issues and format code on save. Ruff brought that experience back. I know that there are plenty of linters and formatters in the Python ecosystem, but this one really stands out for me. Since it’s built in Rust, it’s super fast.

  1. Dependabot:

Instead of remembering to update dependencies every few months, I enabled Dependabot. It automatically opens PRs when updates are available and then CI tells me whether they’re safe to merge. It takes only a couple of minutes to set up but saves a lot of maintenance.

  1. Pylance:

Without it, VS Code gives generic completions and never warns you about passing the wrong type until runtime. Pylance provides proper type-aware autocompletion, jump-to-definition (even in third-party libraries), inline documentation, and real-time type checking. I personally keep it on “basic” mode for legacy codebases, since “strict” surfaced hundreds of errors (thank you, but no thank you lol).

  1. Pydantic:

Pydantic is basically Python’s Zod: you declare a model, pass your data in and get either a validated typed object or a ValidationError naming the exact field at fault. It really helped me keep the codebase clean, with one place defining the shape of the data instead of raw dicts floating around. I use it wherever data comes from outside, like API payloads, forms, and env vars with pydantic-settings, which fails at startup instead of mid-request.

None of these tools changed the application itself. But together they made working on it a lot less frustrating.

So I recently inherited a legacy application that was a nightmare to maintain (tooling was clearly lacking and the codebase was pretty outdated). I quickly realized that if I ever wanted to actually enjoy working on it, I would have to give it a proper overhaul.

These are the 5 changes that had some of the biggest impact IMO:

  1. uv: Being new to the Python ecosystem, I didn’t want to figure out pip vs poetry vs pyenv vs virtualenv, so I just used uv and let it handle all of that. One Rust-based tool that installs and pins Python versions, manages the venv automatically and locks deps in a uv.lock for reproducible builds. Installs are also a lot faster, which makes CI a lot less painful.

  2. Ruff:

Coming from JS, I really missed eslint --fix for automatically fixing linting issues and format code on save. Ruff brought that experience back. I know that there are plenty of linters and formatters in the Python ecosystem, but this one really stands out for me. Since it’s built in Rust, it’s super fast.

  1. Dependabot:

Instead of remembering to update dependencies every few months, I enabled Dependabot. It automatically opens PRs when updates are available and then CI tells me whether they’re safe to merge. It takes only a couple of minutes to set up but saves a lot of maintenance.

  1. Pylance:

Without it, VS Code gives generic completions and never warns you about passing the wrong type until runtime. Pylance provides proper type-aware autocompletion, jump-to-definition (even in third-party libraries), inline documentation, and real-time type checking. I personally keep it on “basic” mode for legacy codebases, since “strict” surfaced hundreds of errors (thank you, but no thank you lol).

  1. Pydantic:

Pydantic is basically Python’s Zod: you declare a model, pass your data in and get either a validated typed object or a ValidationError naming the exact field at fault. It really helped me keep the codebase clean, with one place defining the shape of the data instead of raw dicts floating around. I use it wherever data comes from outside, like API payloads, forms, and env vars with pydantic-settings, which fails at startup instead of mid-request.

None of these tools changed the application itself. But together they made working on it a lot less frustrating.

0

idemkit: runs your code once per key, even when two requests race or a worker dies

preview

I built idemkit after cleaning up duplicate charges one too many times.

The version everyone writes checks whether a key has been seen and replays the stored response. Two requests a millisecond apart both find nothing and both charge the card. And if the worker dies between charging and recording it, the retry charges again. Neither reproduces locally.

idemkit does it properly: an atomic claim instead of check-then-act, a lease that expires on the storage server’s clock, and a fencing token so a stalled worker can’t overwrite a good result.

from idemkit import idempotent, RedisBackend, MethodConfig 

@idempotent(
    backend=RedisBackend.from_url("redis://localhost:6379"), 
    config=MethodConfig(key_fields=["order_id"]), 
) 
async def charge(*, order_id, amount): 
    return await payments.charge(order_id, amount)

One core, three ways to use it: middleware for FastAPI/Flask/Django, a queue consumer wrapper or @idempotent on any function. Backends are Redis, Postgres, Mongo, DynamoDB, or in-memory for tests.

pip install idemkit, Apache-2.0: github.com/idemkit/idemkit

If you find it useful, I’d appreciate a star. It’s new, so visibility helps a lot right now.

I built idemkit after cleaning up duplicate charges one too many times.

The version everyone writes checks whether a key has been seen and replays the stored response. Two requests a millisecond apart both find nothing and both charge the card. And if the worker dies between charging and recording it, the retry charges again. Neither reproduces locally.

idemkit does it properly: an atomic claim instead of check-then-act, a lease that expires on the storage server’s clock, and a fencing token so a stalled worker can’t overwrite a good result.

from idemkit import idempotent, RedisBackend, MethodConfig 

@idempotent(
    backend=RedisBackend.from_url("redis://localhost:6379"), 
    config=MethodConfig(key_fields=["order_id"]), 
) 
async def charge(*, order_id, amount): 
    return await payments.charge(order_id, amount)

One core, three ways to use it: middleware for FastAPI/Flask/Django, a queue consumer wrapper or @idempotent on any function. Backends are Redis, Postgres, Mongo, DynamoDB, or in-memory for tests.

pip install idemkit, Apache-2.0: github.com/idemkit/idemkit

If you find it useful, I’d appreciate a star. It’s new, so visibility helps a lot right now.

0

Recursive Tower of Hanoi Solution Visualized

preview

Recursion becomes much easier once students have the right mental model. Visualization will help get them there.

Take the Tower of Hanoi problem. The recursive solution is beautifully short, to move n disks we:

  • first remove n-1 disks from the largest disk
  • then move the largest disk
  • and then move the n-1 disks back on top

But when students try to implement recursion, they often get stuck, and adding debug prints only adds to the confusion. That is where visualization can help to bring the right mental model. Here is the Tower of Hanoi problem solved recursively, visualized with 𝗶𝗻𝘃𝗼𝗰𝗮𝘁𝗶𝗼𝗻_𝘁𝗿𝗲𝗲: www.invocation-tree.com/#codeurl=https%3A%2F%2Fra…

Instead of thinking about “a function calling itself again and again” students can now see the depth-first execution of a tree of subproblems showing the divide-and-conquer strategy in action. Once a student can think in terms of a tree of subproblems, recursion becomes much easier to understand, explain, and debug.

#Python #invocation_tree #Recursion

Recursion becomes much easier once students have the right mental model. Visualization will help get them there.

Take the Tower of Hanoi problem. The recursive solution is beautifully short, to move n disks we:

  • first remove n-1 disks from the largest disk
  • then move the largest disk
  • and then move the n-1 disks back on top

But when students try to implement recursion, they often get stuck, and adding debug prints only adds to the confusion. That is where visualization can help to bring the right mental model. Here is the Tower of Hanoi problem solved recursively, visualized with 𝗶𝗻𝘃𝗼𝗰𝗮𝘁𝗶𝗼𝗻_𝘁𝗿𝗲𝗲: www.invocation-tree.com/#codeurl=https%3A%2F%2Fra…

Instead of thinking about “a function calling itself again and again” students can now see the depth-first execution of a tree of subproblems showing the divide-and-conquer strategy in action. Once a student can think in terms of a tree of subproblems, recursion becomes much easier to understand, explain, and debug.

#Python #invocation_tree #Recursion

0

Project structure for compat package with different dependency?

preview

So I have a Python package following the project structure outlined by the Python Packaging User Guide (Without the tests/ directory). Let’s call it Package A.

This package can be used alongside another package, Package B, however no proper compatibility exists in Package A to support Package B’s rendering system.

I want to make a compatibility package, Package AB, to make Package B’s systems work with Package A. It shares the same dependencies as Package A, however it has Package B as a dependency.

How should I setup my project structure for Package AB? Should it be its own project directory? Should it be nested inside Package A’s src directory?

Additionally, I have Sphinx /w Hatch setup with Package A for documentation. Depending on the setup chosen above, how should I setup documentation for Package AB?

Some things to note:

  • Package AB would use classes and methods from Package A and Package B
  • Package B’s license (MIT) is compatible with Package A (GPLv3). Package AB would be licensed under the same as Package A

If more details are required, let me know.

So I have a Python package following the project structure outlined by the Python Packaging User Guide (Without the tests/ directory). Let’s call it Package A.

This package can be used alongside another package, Package B, however no proper compatibility exists in Package A to support Package B’s rendering system.

I want to make a compatibility package, Package AB, to make Package B’s systems work with Package A. It shares the same dependencies as Package A, however it has Package B as a dependency.

How should I setup my project structure for Package AB? Should it be its own project directory? Should it be nested inside Package A’s src directory?

Additionally, I have Sphinx /w Hatch setup with Package A for documentation. Depending on the setup chosen above, how should I setup documentation for Package AB?

Some things to note:

  • Package AB would use classes and methods from Package A and Package B
  • Package B’s license (MIT) is compatible with Package A (GPLv3). Package AB would be licensed under the same as Package A

If more details are required, let me know.