Threads & Processes

Concurrency starts feeling real the moment two workers touch the same value. Threads make waiting work overlap nicely; processes can use more CPU cores; both can also turn a clear program into a timing bug if you let shared state wander around unguarded.

This page shows desktop CPython patterns. The browser playground does not behave like your operating system for real threads and processes, so the worker examples below are static scripts you would run from a terminal.

Threads: light workers in one process

A thread is a line of execution inside a process. Multiple threads in the same process share memory, so they can see the same lists, dictionaries, objects, and module-level names.

Python's threading module gives you the basic pieces. A target is the function the thread will run. start() tells the operating system to begin that worker. join() makes your main thread wait until that worker finishes.

from threading import Thread
from time import sleep
 
 
def send_receipt(customer):
    print(f"start receipt for {customer}")
    sleep(0.05)
    print(f"sent receipt for {customer}")
 
 
threads = [
    Thread(target=send_receipt, args=("Mina",)),
    Thread(target=send_receipt, args=("Ravi",)),
    Thread(target=send_receipt, args=("Noor",)),
]
 
for thread in threads:
    thread.start()
 
for thread in threads:
    thread.join()
 
print("all receipts sent")

One possible desktop run:

$ python thread_receipts.py
start receipt for Mina
start receipt for Ravi
start receipt for Noor
sent receipt for Mina
sent receipt for Noor
sent receipt for Ravi
all receipts sent

The exact order of the middle lines can change. That is not a bug in the example; it is the point. Once workers overlap, scheduling belongs partly to the operating system.

Threads are most useful for I/O-bound work, where workers spend real time waiting on files, sockets, APIs, or databases. They are not the usual answer for CPU-bound Python loops because the GIL model from the concurrency landscape still applies in normal CPython.

The trap: shared state races

A race condition happens when the result depends on the timing of two or more workers. A critical section is a piece of code that must not be interrupted by another worker touching the same shared state.

The classic shape is a lost update. Two workers both read the same value, both calculate the next value, and one write silently overwrites the other.

Try it - simulate a lost update

This is not real threading. It is the same timing problem slowed down into visible steps.

Real threads can create the same wrong shape:

from threading import Thread
from time import sleep
 
orders_processed = 0
 
 
def record_orders(how_many):
    global orders_processed
    for _ in range(how_many):
        current = orders_processed
        sleep(0)  # give another thread a chance to run here
        orders_processed = current + 1
 
 
threads = [Thread(target=record_orders, args=(10_000,)) for _ in range(2)]
 
for thread in threads:
    thread.start()
 
for thread in threads:
    thread.join()
 
print("expected:", 20_000)
print("actual:", orders_processed)

One desktop run produced:

$ python race_counter.py
expected: 20000
actual: 10007

Locks and thread safety

A lock is an object that lets one worker enter a critical section at a time. In Python, threading.Lock() works as a context manager, so with counter_lock: acquires the lock before the block and releases it after the block.

Code is thread-safe when it still behaves correctly while multiple threads use it. Locking a shared counter is the small version of that idea:

from threading import Lock, Thread
from time import sleep
 
orders_processed = 0
counter_lock = Lock()
 
 
def record_orders(how_many):
    global orders_processed
    for _ in range(how_many):
        with counter_lock:
            current = orders_processed
            sleep(0)
            orders_processed = current + 1
 
 
threads = [Thread(target=record_orders, args=(10_000,)) for _ in range(2)]
 
for thread in threads:
    thread.start()
 
for thread in threads:
    thread.join()
 
print("expected:", 20_000)
print("actual:", orders_processed)

$ python locked_counter.py
expected: 20000
actual: 20000

Locks fix the lost update by making the read-calculate-write sequence one guarded move. They also introduce a design cost: workers can spend time waiting for the lock, and careless lock ordering can make programs freeze. The better habit is to share less mutable data in the first place.

Processes: separate interpreters, real CPU parallelism

A process is a separate running program with its own memory. The multiprocessing module starts extra Python processes, each with its own interpreter. That separation is heavier than threads, but it is the standard route for CPU-bound Python work that should run on multiple cores.

Because new processes need to import your script, put process-starting code under the if __name__ == "__main__": guard you met in modules. Run this kind of code from a real .py file, not the browser page or a pasted stdin session.

from multiprocessing import Process
 
 
def summarize(region, orders):
    print(f"{region}: {sum(orders)} orders")
 
 
if __name__ == "__main__":
    jobs = [
        Process(target=summarize, args=("North", [4, 7, 8])),
        Process(target=summarize, args=("South", [5, 2, 7])),
    ]
 
    for job in jobs:
        job.start()
 
    for job in jobs:
        job.join()
 
    print("all regions summarized")

One possible desktop run:

$ python process_regions.py
North: 19 orders
South: 14 orders
all regions summarized

Processes do not share ordinary Python objects the way threads do. If one process changes a list, another process does not see that same list mutate. That isolation is often a gift: fewer accidental shared-state bugs. It also means communication must be explicit.

Executors: worker pools with one clean shape

concurrent.futures gives you a higher-level API for worker pools. An executor manages workers for you. A future represents a result that will arrive later.

Use ThreadPoolExecutor when the jobs mostly wait. Use ProcessPoolExecutor when the jobs are CPU-bound and the work can be split into independent pieces.

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor
 
 
def fetch_status(order_id):
    return f"order {order_id}: ready"
 
 
def score_route(stops):
    return sum(stop * stop for stop in stops)
 
 
if __name__ == "__main__":
    with ThreadPoolExecutor(max_workers=3) as pool:
        for message in pool.map(fetch_status, ["A12", "B34", "C56"]):
            print(message)
 
    with ProcessPoolExecutor(max_workers=2) as pool:
        for score in pool.map(score_route, [[2, 4, 6], [1, 3, 5]]):
            print("route score:", score)

$ python executor_shapes.py
order A12: ready
order B34: ready
order C56: ready
route score: 56
route score: 35

map() preserves the input order when it yields results, even if workers finish internally in a different order. That makes it friendly for beginner code. When you need to react as soon as each job finishes, as_completed() is the next tool to learn.

Queues: pass work, not shared drawers

A queue is a first-in, first-out communication line. Instead of letting workers all reach into the same mutable list, the main thread puts work into a queue and workers take items out. In thread code, queue.Queue also handles the locking needed around the queue itself.

from queue import Queue
from threading import Thread
 
work_queue = Queue()
done_queue = Queue()
 
 
def pack_orders():
    while True:
        order = work_queue.get()
        if order is None:
            work_queue.task_done()
            break
 
        done_queue.put(f"packed {order}")
        work_queue.task_done()
 
 
worker = Thread(target=pack_orders)
worker.start()
 
for order in ["tea-104", "rice-208", "cake-309"]:
    work_queue.put(order)
 
work_queue.put(None)  # sentinel: tells the worker to stop
work_queue.join()
worker.join()
 
while not done_queue.empty():
    print(done_queue.get())

$ python queue_orders.py
packed tea-104
packed rice-208
packed cake-309

The None value is a sentinel, a special value that means "no more work." You have seen None as a reserved empty seat; here it is a clear stop signal.

For processes, multiprocessing.Queue fills the same role across process boundaries. The habit is the same: communicate by passing messages, not by pretending every worker can safely edit the same shared drawer.

Threads and processes are operating-system tools: powerful, blunt, and full of timing edges. Next, async gives you a different model: one event loop, many waiting tasks, and no need to create an OS thread for every little pause.