Generators
创建 lazy, memory-efficient 序列使用 generator 函数, yield, and generator 表达式.
你将学到
- 编写 generator 函数使用 yield
- 理解 lazy evaluation and memory efficiency
- Use generator 表达式 for inline generators
- 链 generators in 流水线
- Handle 无限 generators 使用 itertools
概念讲解
What Is a Generator?
generator 是一种 function that produces a sequence of values lazily — one at a time, on demand — instead of computing them all upfront and storing them in memory. You create one with the yield keyword:
def count_up_to(n):
i = 0
while i < n:
yield i
i += 1
# Create the generator — nothing runs yet
counter = count_up_to(5)
# Values are produced one at a time
print(next(counter)) # 0
print(next(counter)) # 1
print(next(counter)) # 2
When you call a generator function, it returns a generator object without running any code. Each call to 下一个() runs the function until it hits yield, produces that value, and pauses. The next 下一个() resumes from where it left off. When the function returns (or falls off the end), StopIteration is raised.
yield vs return
返回— returns a value and ends the functionyield— produces a value and pauses the function, to be resumed later
def squares(n):
for x in range(n):
yield x ** 2
for sq in squares(5):
print(sq) # 0, 1, 4, 9, 16
A generator function can have multiple yield statements. Each yield produces a value and suspends execution. The function resumes from the yield on the next 下一个() call.
Why Generators? Memory Efficiency
The key advantage is memory. Compare:
# List — all 1 million values in memory at once
squares_list = [x ** 2 for x in range(1_000_000)]
# Uses ~8 MB of memory
# Generator — only one value in memory at a time
squares_gen = (x ** 2 for x in range(1_000_000))
# Uses ~100 bytes of memory
For large or infinite sequences, generators are essential. You can process a 10 GB file line by line with a generator using only kilobytes of memory.
Generator Expressions
The parenthesized version of a list comprehension creates a generator:
# List comprehension — eager, all values in memory
squares_list = [x ** 2 for x in range(10)]
# Generator expression — lazy, one value at a time
squares_gen = (x ** 2 for x in range(10))
print(sum(squares_gen)) # 285 — computed without storing all values
Generator expressions are perfect for feeding into 总和(), 最大值(), 最小值(), or any function that consumes an iterable. They avoid building an intermediate list.
Sending Values with .send()
Generators can receive values from the caller using .发送():
def accumulator():
total = 0
while True:
value = yield total
if value is None:
break
total += value
acc = accumulator()
next(acc) # prime the generator — runs to first yield
print(acc.send(10)) # 10
print(acc.send(20)) # 30
print(acc.send(5)) # 35
This makes generators coroutines — they can both produce and consume values. This is an advanced feature used in async programming.
Chaining Generators
Generators compose beautifully — you can pipe them together:
def numbers():
for n in range(1, 20):
yield n
def squared(seq):
for n in seq:
yield n ** 2
def evens(seq):
for n in seq:
if n % 2 == 0:
yield n
# Pipeline: numbers → squared → evens
pipeline = evens(squared(numbers()))
print(list(pipeline)) # [4, 16, 36, 64, 100, 144, 196, 256, 324]
Each stage processes values lazily — no intermediate lists are created. This is the foundation of streaming data pipelines.
Infinite Generators
Generators can be infinite — they never run out of values:
def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
fib = fibonacci()
for _ in range(10):
print(next(fib)) # 0, 1, 1, 2, 3, 5, 8, 13, 21, 34
An infinite generator is safe as long as you only consume a finite number of values (with 下一个(), itertools.islice(), or a bounded loop). itertools provides many useful generators:
from itertools import islice, count, cycle
# count() — infinite counter: 0, 1, 2, 3, ...
for n in islice(count(), 5):
print(n) # 0, 1, 2, 3, 4
# cycle() — repeat a sequence forever
for item in islice(cycle(["", "B", "C"]), 7):
print(item) # A, B, C, A, B, C, A
yield from — Delegating to Sub-generators
yield 从 delegates to another generator, flattening nested generators:
def inner():
yield 1
yield 2
def outer():
yield 0
yield from inner() # yields all values from inner()
yield 3
print(list(outer())) # [0, 1, 2, 3]
This is cleaner than manually iterating and yielding each value from the sub-generator.
示例代码
# Basic generator with yield
def count_up_to(n):
i = 0
while i < n:
yield i
i += 1
print(list(count_up_to(5))) # [0, 1, 2, 3, 4]
# Generator vs list — memory efficiency
import sys
big_list = [x ** 2 for x in range(10000)]
big_gen = (x ** 2 for x in range(10000))
print(f"List size: {sys.getsizeof(big_list)} bytes")
print(f"Gen size: {sys.getsizeof(big_gen)} bytes")
# Using next() manually
counter = count_up_to(3)
print(next(counter)) # 0
print(next(counter)) # 1
print(next(counter)) # 2
# Infinite generator — Fibonacci
def fibonacci():
a, b = 0, 1
while True:
yield a
a, b = b, a + b
fib = fibonacci()
fib_10 = [next(fib) for _ in range(10)]
print(f"Fibonacci: {fib_10}")
# Chaining generators in a pipeline
def numbers():
for n in range(1, 10):
yield n
def squared(seq):
for n in seq:
yield n ** 2
def evens(seq):
for n in seq:
if n % 2 == 0:
yield n
pipeline = evens(squared(numbers()))
print(f"Pipeline: {list(pipeline)}")
# yield from — delegating
def inner():
yield 1
yield 2
def outer():
yield 0
yield from inner()
yield 3
print(f"yield from: {list(outer())}") # [0, 1, 2, 3]
# Reading a large file lazily
def read_lines(filename):
with open(filename) as f:
for line in f:
yield line.strip() # one line at a time, low memory
Shows 基础 generator, memory comparison 对比列表, manual 下一个() usage, 无限 Fibonacci generator, chained 流水线 (数字 → squared → evens), yield 从 delegation, and lazy file reader 模式.
动手试一试
- 文本计数器
Generators 是 perfect for streaming 文本处理行 通过行.
- JSON Formatter
Generators 能 stream 大 JSON 数组不使用加载一切到 memory.
常见错误
错误写法
Trying 到 reuse generator 那个有 是 exhausted
正确做法
Generators 是单个-use — 一次 consumed, 它们是 空 and 下一个() raises StopIteration. 到 reuse, recreate generator: gen = my_generator() 再次, or convert 到列表首先如果你 需要到 iterate 多个次数.
错误写法
创建无限 generator and iterating 使用 for 循环不使用 break
正确做法
无限 generator (虽然真: yield) 从不停止. 总是 use bounded consumer: 下一个() fixed 数字的 次数, itertools.islice(gen, n), or for 循环使用 break 条件. bare 'for x in infinite_gen:' hangs forever.
相关资源
相关速查表
- Python Cheatsheet
yield, generator 表达式, 发送(), and itertools 引用.
练习 Generators
4 个练习
查阅不认识的术语
新手友好的术语表用大白话解释编程术语——变量、函数、DOM、Promise 等等。
动手做项目
通过引导式项目应用所学,含起始代码与解决方案。
解读错误信息
用大白话解释常见错误——什么意思、为什么发生、如何修复。