Iterators & Generators

Python's for loop looks like it magically knows how to walk through lists, strings, files, dictionaries, and your own objects. The magic is small: Python keeps asking for "one more item" until the object says it is done.

Meet the iterator protocol

An iterable is a value Python can ask for an item stream, and an iterator is the object that hands back one item at a time while remembering where it stopped. Think of an iterator like a ticket dispenser: each pull gives you the next ticket, and the dispenser does not rewind itself.

The iterator protocol is the pair of dunder methods Python looks for: __iter__ returns an iterator, and __next__ returns the next item or raises StopIteration. You saw protocols in lesson 11 as labeled buttons on a class blueprint. These are the two buttons behind iteration.

scores = [88, 91, 79]
reader = iter(scores)
 
print(next(reader))
print(next(reader))
print(list(reader))

88
91
[79]

The iter() function asks the list for an iterator. The next() function pulls one item. list(reader) pulls everything still waiting, which is why only 79 remains.

Consuming an iterator means pulling values from it so those values are no longer waiting in the stream.

StopIteration is the exception that means "this iterator has no more items." A for loop catches that exception for you, so you almost never see it directly.

Build a custom iterator

A custom iterator is a class you write that follows the iterator protocol. It needs state, which is remembered data the object carries between calls, because it must know what the next answer should be.

class Countdown:
    def __init__(self, start):
        self.next_number = start
 
    def __iter__(self):
        return self
 
    def __next__(self):
        if self.next_number <= 0:
            raise StopIteration
 
        current = self.next_number
        self.next_number = self.next_number - 1
        return current
 
 
launch = Countdown(3)
 
print(next(launch))
print(next(launch))
print(next(launch))
 
try:
    print(next(launch))
except StopIteration:
    print("lift off")
 
for number in Countdown(3):
    print(f"launch in {number}")

3
2
1
lift off
launch in 3
launch in 2
launch in 1

__iter__ returns self because the Countdown object is its own iterator. __next__ either returns one number or raises StopIteration. The for loop calls the same machinery, but it hides the try/except dance.

Custom iterator classes are useful when the state is rich: maybe pages from an API, chunks from a file, or turns in a game. For most everyday streams, Python gives you a shorter tool.

Use generators with yield

A generator is a special iterator created by a function that contains yield. The yield keyword returns one value and pauses the function, keeping its local names alive until the next pull.

The analogy is a paused recipe card. A normal function cooks the whole meal and returns once. A generator stops after each serving, keeps its place on the recipe, and continues when you ask again.

def daily_points():
    print("generator started")
    yield 10
    print("after first yield")
    yield 25
    print("after second yield")
 
 
points = daily_points()
 
print("made generator")
print(next(points))
print(next(points))
 
try:
    print(next(points))
except StopIteration:
    print("no more points")

made generator
generator started
10
after first yield
25
after second yield
no more points

Calling daily_points() does not run the body yet. It creates a generator object. The body starts on the first next(points), pauses at the first yield, then resumes from that exact spot on the next pull.

That pause is the whole power. You can write code that looks like a plain function, but behaves like a careful stream.

Try it — filter a score stream

Run this, then change the passing line from 70 to 85. Notice which names print before the first result appears.

Lazy evaluation saves memory

Lazy evaluation means Python delays work until a value is requested. Picture a sandwich counter that makes one sandwich when the next customer asks, instead of filling the whole counter with sandwiches before anyone arrives.

That matters when the stream could be huge. A list comprehension builds the whole list now. A generator produces one value, hands it over, and keeps only the small amount of state needed to continue.

scores = [62, 88, 71, 95]
bonus_points = (score + 5 for score in scores if score >= 70)
 
print(next(bonus_points))
print(list(bonus_points))

93
[76, 100]

A generator expression is a compact generator written like a list comprehension, but with parentheses instead of square brackets. (score + 5 for score in scores if score >= 70) does not build [93, 76, 100] all at once. It waits.

Delegate with yield from

yield from is syntax that passes values from another iterable through your generator. It is useful when one stream is built from smaller streams.

Without yield from, you would write a nested for loop and yield each item yourself. With it, your generator says, "hand out everything from this other iterable before continuing."

It is the same ticket counter temporarily pulling tickets from another roll. The person in line still receives one ticket at a time.

def warmup_tasks(player):
    yield f"{player}: stretch"
    yield f"{player}: breathe"
 
 
def main_tasks(player):
    yield f"{player}: solve puzzle"
    yield f"{player}: review notes"
 
 
def full_practice(player):
    yield "start practice"
    yield from warmup_tasks(player)
    yield from main_tasks(player)
    yield "finish practice"
 
 
for task in full_practice("Mina"):
    print(task)

start practice
Mina: stretch
Mina: breathe
Mina: solve puzzle
Mina: review notes
finish practice

The outer generator stays readable because each smaller generator owns one part of the story. yield from keeps the stream flat for the caller: the for loop still receives one task at a time.

You now know the machinery behind Python's smoothest loops: one item, one pause, one pull at a time. Next, decorators use another kind of language machinery to wrap functions without changing the code that calls them.