Generators and Iterators
What you'll build
A set of generators: fibonacci (infinite), chunker (split iterable into chunks), and a pipeline that filters/transforms a large data stream lazily.
Goals
- Use yield to produce values lazily
- Write an infinite generator
- Chain generators into a pipeline
- Compare memory usage vs list-based approaches
Prerequisites
- for loops and iterables
- Functions
- List comprehensions
Steps
- 1
Write a Fibonacci generator
A generator uses yield to produce values one at a time, pausing between each. Write an infinite Fibonacci generator — it never returns, but yields forever.
Codedef fibonacci(): a, b = 0, 1 while True: yield a a, b = b, a + b # Take the first 10 values fib = fibonacci() print([next(fib) for _ in range(10)]) # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]Expected result
First 10 Fibonacci numbers: [0, 1, 1, 2, 3, 5, 8, 13, 21, 34].
- 2
Use itertools.islice
Since the generator is infinite, use itertools.islice to take a finite number of values without list slicing (which would hang forever).
Codefrom itertools import islice print(list(islice(fibonacci(), 15))) # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377]Expected result
First 15 Fibonacci numbers as a list.
- 3
Write a chunker generator
Write a generator that yields chunks of size n from an iterable. This is useful for batch processing without loading everything into memory.
Codedef chunker(iterable, size): chunk = [] for item in iterable: chunk.append(item) if len(chunk) == size: yield chunk chunk = [] if chunk: yield chunk # yield the remaining partial chunk for chunk in chunker(range(10), 3): print(chunk) # [0, 1, 2], [3, 4, 5], [6, 7, 8], [9]Expected result
Four chunks: three of size 3 and one of size 1.
- 4
Build a processing pipeline
Chain generators: read numbers, filter evens, square them, take first 5. Each step is a generator, so the whole pipeline is lazy — nothing runs until you consume the output.
Codedef numbers(): n = 0 while True: yield n n += 1 def evens(src): for n in src: if n % 2 == 0: yield n def squared(src): for n in src: yield n * n pipeline = squared(evens(numbers())) print(list(islice(pipeline, 5))) # [0, 4, 16, 36, 64]Expected result
First 5 squares of even numbers: [0, 4, 16, 36, 64].
- 5
Compare memory with lists
Show that a generator processing 10M numbers uses constant memory, while a list would use ~80MB. Use sys.getsizeof to compare.
Codeimport sys # Generator: ~128 bytes regardless of how many it could produce gen = (x**2 for x in range(10_000_000)) print(f"Generator size: {sys.getsizeof(gen)} bytes") # List: would be ~80MB for 10M integers # lst = [x**2 for x in range(10_000_000)] # don't actually run this print("A list of 10M ints would be ~80MB")Expected result
Generator is ~128 bytes; a list would be megabytes.
Try it yourself
def fibonacci():
# TODO: yield Fibonacci numbers infinitely
passdef fibonacci():
a, b = 0, 1
while True:
yield a; a, b = b, a + bWhat you learned
You mastered generators — Python's way to process large or infinite data streams with constant memory. They power file iteration, streaming, and async pipelines.
Related Resources
Related Lessons
- Loops
Generators are iterables used in for loops.
Look up unfamiliar terms
A beginner-friendly glossary explains jargon in plain English — variables, functions, DOM, Promise, and more.
Decode error messages
Plain-English explanations of common errors — what they mean, why they happen, and how to fix them.