Python Iterators, Comprehensions and Generators: 10 Code-Along Examples

The article explains the through line: lazy evaluation, producing values only when they are asked for. Iterators are the mechanism, comprehensions are the concise way to build collections, and generators are how you do it without holding everything in memory. This workbook walks that path in ten runnable steps, from iter() and next() up to a generator that totals a million values using almost no memory. Run them in order and the connection between the three ideas becomes obvious.

1. Iterable versus iterator

An iterable is a collection you can loop over. An iterator is the object that actually walks through it, handing back one item at a time. You turn an iterable into an iterator with iter(), and you advance it with next(). When there is nothing left, next() raises StopIteration.

languages = ["Python", "Rust", "Go"]   # an iterable
it = iter(languages)   # ask the iterable for an iterator
print(next(it))        # Python
print(next(it))        # Rust
print(next(it))        # Go
# A further next(it) here would raise StopIteration

The list does not move; the iterator it carries the position. This split between the collection and the thing tracking progress through it is the foundation everything else is built on.

2. What a for loop really does

A for loop is syntactic sugar over exactly the pattern from Example 1: call iter() once, call next() repeatedly, and stop when StopIteration fires. Writing it out by hand once makes the loop feel less like magic.

languages = ["Python", "Rust", "Go"]
it = iter(languages)          # the loop calls this for you
while True:
    try:
        item = next(it)       # ...and this on every pass
    except StopIteration:
        break                 # ...and stops here automatically
    print(item)

This is precisely what for item in languages: does under the covers. Once you see it, you understand why anything that follows the iterator protocol can be used in a for loop, including the generators later in this workbook.

3. Index and item together with enumerate

Reaching for a manual counter is a common beginner habit. enumerate gives you the position and the item as a tuple on each pass, and start lets you choose where the count begins.

explorers = ["Ada", "Alan", "Grace"]
for position, name in enumerate(explorers, start=1):
    print(position, name)

You get 1 Ada, 2 Alan, 3 Grace. enumerate is itself lazy: it yields each (index, item) pair as the loop asks for it rather than building the whole list of pairs first.

4. Pairing and unpairing sequences with zip

zip walks several iterables in lockstep, pairing their elements. The same zip with the * unpacking operator reverses the process, splitting a list of pairs back into separate sequences.

names = ["Ada", "Alan", "Grace"]
fields = ["computing", "cryptography", "compilers"]
paired = list(zip(names, fields))     # pair them up
print(paired)                         # [('Ada', 'computing'), ...]
back_names, back_fields = zip(*paired)  # unzip back into two tuples
print(back_names)                       # ('Ada', 'Alan', 'Grace')
print(back_fields)                      # ('computing', 'cryptography', 'compilers')

zip stops at the shortest input, so mismatched lengths simply truncate rather than error. Like enumerate, it produces pairs lazily, which is why you wrap it in list() here to see them all at once.

5. List comprehensions, with and without a filter

A list comprehension builds a list in one expression: an output expression, a for clause, and an optional trailing if that filters which items make it through.

nums = range(1, 11)
squares = [n * n for n in nums]                  # transform every item
even_squares = [n * n for n in nums if n % 2 == 0]  # keep only some
print(squares)        # [1, 4, 9, ..., 100]
print(even_squares)   # [4, 16, 36, 64, 100]

The trailing if decides whether an item is included at all. Read it left to right: take n from nums, keep it only if it is even, then square it. This is the everyday workhorse of the three constructs.

6. A conditional expression inside a comprehension

There is a second, easily confused place to put an if. A trailing if filters; an if-else placed before the for chooses what value to produce for every item. It transforms rather than filters.

nums = range(1, 11)
labels = ["even" if n % 2 == 0 else "odd" for n in nums]
print(labels)   # ['odd', 'even', 'odd', 'even', ...]

Every number produces a label, so the result has ten elements, not a filtered subset. The rule worth memorising: if at the end removes items, if-else at the front changes each item.

7. Dictionary and set comprehensions

The same syntax builds dictionaries and sets, not just lists. Use curly braces, and for a dictionary give a key: value pair.

words = ["apple", "banana", "cherry", "apple"]
lengths = {w: len(w) for w in words}     # dict comprehension
initials = {w[0] for w in words}         # set comprehension
print(lengths)    # {'apple': 5, 'banana': 6, 'cherry': 6}
print(initials)   # {'a', 'b', 'c'}

The dict comprehension naturally deduplicates keys, so the repeated apple appears once. The set comprehension deduplicates values, collapsing the two a initials into one. Same shape, different container.

8. Nested comprehensions for grids and flattening

Comprehensions can nest. One inside another builds a two-dimensional structure like a grid, and two for clauses in a single comprehension flatten a nested list back to one dimension.

# Build a 3x3 grid: outer loop makes rows, inner loop fills columns
grid = [[row * 3 + col for col in range(3)] for row in range(3)]
for r in grid:
    print(r)        # [0, 1, 2] then [3, 4, 5] then [6, 7, 8]
# Flatten it back to a single list
flat = [value for r in grid for value in r]
print(flat)         # [0, 1, 2, 3, 4, 5, 6, 7, 8]

In the flattening version the clauses read in the same order you would write nested for loops: outer first, then inner. Nesting more than two levels deep usually hurts readability, so reach for ordinary loops when it gets dense.

9. Generator expressions and why memory matters

Swap a list comprehension’s square brackets for round ones and you get a generator expression. It looks almost identical but builds nothing up front; it produces each value only when asked. The memory difference is dramatic.

import sys
list_version = [n * n for n in range(10_000)]   # builds all 10,000 values now
gen_version = (n * n for n in range(10_000))     # builds nothing yet
print(sys.getsizeof(list_version), "bytes (list)")
print(sys.getsizeof(gen_version), "bytes (generator)")
print(sum(gen_version))   # consumes lazily, one value at a time

The list reports tens of thousands of bytes; the generator reports a small, constant size regardless of how many values it will eventually yield. One catch to remember: a generator is single-use. After sum drains it, looping over gen_version again yields nothing.

10. Generator functions with yield

A generator function uses yield instead of return. Each yield hands back a value and pauses the function, keeping all its local variables intact, then resumes from that exact spot on the next request. This is how you stream data too big to hold in memory.

def countdown(start):
    while start > 0:
        yield start      # pause here, remembering the value of `start`
        start -= 1       # resumes here on the next next() call
for n in countdown(3):
    print(n)             # 3, 2, 1
# Practical payoff: total a huge sequence without ever holding it all
def amounts():
    for i in range(1_000_000):
        yield i * 1.0
total = 0
for value in amounts():  # one value at a time, near-constant memory
    total += value
print(total)

The countdown function proves the pause-and-resume behaviour: start survives between yields without any saved state of your own. The amounts example shows the real reason generators matter, summing a million values while only one exists in memory at any moment.

These ten steps cover the whole article: the iterator protocol that powers every loop, comprehensions for building lists, dicts and sets concisely, and generators for doing it all lazily. The practical directive from the guide is worth pinning up: the moment your data might not fit in memory, switch to a generator. The same idea scales beyond these toy examples. You can yield lines from a huge file to process it without loading the whole thing, or pass chunksize to pandas.read_csv to walk an oversized CSV a block at a time. Once lazy evaluation clicks, you stop asking “can this fit in memory” and start streaming instead.

See you soon.