AL+

Language: RO (română) EN (English)

Email us: paul@alplustech.com

Choose Python parallelism by what must cross the boundary

Python 3.14 interpreter pools enable parallel work with isolated state. Assess serialization, extension support and ownership before replacing shared threads.

Citiți articolul în română

A familiar executor interface can conceal a different data model. Python interpreter pools let work run in separate interpreters, but an application that depends on workers changing one shared dictionary cannot simply substitute a new executor and expect the same program. Choose the parallel execution model by the information that must cross between workers and the component responsible for combining their results.

Looking back from November 2026, the relevant release is Python 3.14.0 on 7 October 2025. PEP 734 was created on 6 November 2023 and identifies Python 3.14 as its target version. Its creation date is the beginning of a proposal, not the release of the standard-library feature. That separation matters when reconstructing what teams could actually adopt in October 2025.

A worker has its own interpreter

The Python 3.14 executor documentation, consulted in November 2026 as mutable documentation then labelled 3.14.10, describes concurrent.futures.InterpreterPoolExecutor as a ThreadPoolExecutor subclass. Each worker runs in its own thread and has its own interpreter. Each interpreter has its own GIL, allowing Python work in different interpreters to execute on different CPU cores.

The same reference describes isolated runtime state, separately imported module objects and no automatic sharing of mutable objects. The executor serializes submitted callables, arguments and return values using pickle. Immutable values have different sharing constraints; the lower-level interpreter APIs also provide managed communication mechanisms. These facts describe an exchange boundary. They do not promise that moving data across it is free, or that every object an application currently uses can cross it successfully.

The accepted PEP names the lower-level module concurrent.interpreters; the proposal text also retains discussion of the earlier proposed name. Keep the released module and the executor API distinct. A facility for creating interpreters and communicating between them is broader than the executor's submit-and-return model. The CPython 3.14.0 tagged reference independently records the executor contract at that release. Neither name means that ordinary mutable Python objects become a shared application database.

An illustrative document-processing job makes the distinction concrete. A coordinator can assign independent documents and combine returned summaries. Passing the document contents has a cost that the application would need to assess. Passing an identifier instead can reduce that transfer, but then each worker needs an authorized way to retrieve the corresponding document. Choosing smaller arguments moves responsibility; it does not remove it.

Decide who owns the final answer

Isolation can make ownership clearer. A worker can calculate a result from an explicit input, while the coordinator decides how to persist it. The design becomes harder when a task expects to modify an already shared cache, reuse a live connection from the caller or publish updates by changing a module-level variable. Those expectations need redesign rather than a change of executor name.

Separate imports also make initialization part of the plan. A module configured in the main interpreter does not imply an equivalent configured module in every worker. Define the necessary setup and what happens if it fails. An unavailable worker should not silently turn an incomplete collection of results into a successful report. The product needs to distinguish a failed item from an empty result.

Retries raise a related question. If a worker writes to an external service and the coordinator loses the result, repeating the task may repeat the write. Interpreter isolation does not establish idempotence for a database or remote API. Assign stable work identities and an explicit duplicate-handling rule wherever repeated effects would be harmful. These are application requirements even when workers share no mutable Python state.

The boundary is also operational. Interpreters remain within one process, so they should not be sold as process-level fault containment or a security sandbox for hostile code. Select a process or stronger isolation when that is the actual requirement. A private runtime state and a separately protected operating-system process solve different problems.

Parallelism is not a performance verdict

The strongest counterargument is the cost of making data independent. A large mutable dataset, frequent coordination or an extension that is not suitable for isolated interpreters may make another model more practical. Check extension compatibility against the actual installed version and its maintainers' guidance. A Python release announcement cannot certify every native dependency used by an application.

Threads can remain appropriate for waiting on external operations or for code whose relevant native work releases the GIL. Processes can be appropriate when a separate process boundary is valuable. Free-threaded Python is another choice with its own build and dependency considerations. Per-interpreter GILs and a free-threaded shared runtime are different designs; supporting one does not establish equivalent sharing behavior in the other.

This article runs no pool benchmark and makes no speedup claim. Any comparison would need the actual task size, serialization volume, initialization cost and installed extensions. It would also need to check correctness under partial failure. Measuring elapsed time for a result that silently omits failed documents would answer the wrong product question.

Write the boundary before selecting the executor

Start with one task described without concurrency vocabulary. State the input, the result, the external effects and the authority to repeat it. Then identify which information genuinely needs to be shared while work is in progress. If that list is long, an interpreter pool may expose a coupling problem worth solving, or it may simply be a poor fit for the task.

For custom software development, agree the completeness and retry rules alongside the data contract. Interpreter pools are worth considering when independent work and explicit exchange match those rules. The useful outcome is a program whose ownership remains understandable when work runs in parallel, rather than an executor change justified by the availability of a new release.