Skip to content

Generators

创建 lazy, memory-efficient 序列使用 generator 函数, yield, and generator 表达式.

你将学到

  • 编写 generator 函数使用 yield
  • 理解 lazy evaluation and memory efficiency
  • Use generator 表达式 for inline generators
  • 链 generators in 流水线
  • Handle 无限 generators 使用 itertools

概念讲解

What Is a Generator?

generator 是一种 function that produces a sequence of values lazily — one at a time, on demand — instead of computing them all upfront and storing them in memory. You create one with the yield keyword:

def count_up_to(n):
    i = 0
    while i < n:
        yield i
        i += 1

# Create the generator — nothing runs yet
counter = count_up_to(5)

# Values are produced one at a time
print(next(counter))  # 0
print(next(counter))  # 1
print(next(counter))  # 2

When you call a generator function, it returns a generator object without running any code. Each call to 下一个() runs the function until it hits yield, produces that value, and pauses. The next 下一个() resumes from where it left off. When the function returns (or falls off the end), StopIteration is raised.

yield vs return

  • 返回 — returns a value and ends the function
  • yield — produces a value and pauses the function, to be resumed later
def squares(n):
    for x in range(n):
        yield x ** 2

for sq in squares(5):
    print(sq)  # 0, 1, 4, 9, 16

A generator function can have multiple yield statements. Each yield produces a value and suspends execution. The function resumes from the yield on the next 下一个() call.

Why Generators? Memory Efficiency

The key advantage is memory. Compare:

# List — all 1 million values in memory at once
squares_list = [x ** 2 for x in range(1_000_000)]
# Uses ~8 MB of memory

# Generator — only one value in memory at a time
squares_gen = (x ** 2 for x in range(1_000_000))
# Uses ~100 bytes of memory

For large or infinite sequences, generators are essential. You can process a 10 GB file line by line with a generator using only kilobytes of memory.

Generator Expressions

The parenthesized version of a list comprehension creates a generator:

# List comprehension — eager, all values in memory
squares_list = [x ** 2 for x in range(10)]

# Generator expression — lazy, one value at a time
squares_gen = (x ** 2 for x in range(10))

print(sum(squares_gen))  # 285 — computed without storing all values

Generator expressions are perfect for feeding into 总和(), 最大值(), 最小值(), or any function that consumes an iterable. They avoid building an intermediate list.

Sending Values with .send()

Generators can receive values from the caller using .发送():

def accumulator():
    total = 0
    while True:
        value = yield total
        if value is None:
            break
        total += value

acc = accumulator()
next(acc)           # prime the generator — runs to first yield
print(acc.send(10)) # 10
print(acc.send(20)) # 30
print(acc.send(5))  # 35

This makes generators coroutines — they can both produce and consume values. This is an advanced feature used in async programming.

Chaining Generators

Generators compose beautifully — you can pipe them together:

def numbers():
    for n in range(1, 20):
        yield n

def squared(seq):
    for n in seq:
        yield n ** 2

def evens(seq):
    for n in seq:
        if n % 2 == 0:
            yield n

# Pipeline: numbers → squared → evens
pipeline = evens(squared(numbers()))
print(list(pipeline))  # [4, 16, 36, 64, 100, 144, 196, 256, 324]

Each stage processes values lazily — no intermediate lists are created. This is the foundation of streaming data pipelines.

Infinite Generators

Generators can be infinite — they never run out of values:

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
for _ in range(10):
    print(next(fib))  # 0, 1, 1, 2, 3, 5, 8, 13, 21, 34

An infinite generator is safe as long as you only consume a finite number of values (with 下一个(), itertools.islice(), or a bounded loop). itertools provides many useful generators:

from itertools import islice, count, cycle

# count() — infinite counter: 0, 1, 2, 3, ...
for n in islice(count(), 5):
    print(n)  # 0, 1, 2, 3, 4

# cycle() — repeat a sequence forever
for item in islice(cycle(["", "B", "C"]), 7):
    print(item)  # A, B, C, A, B, C, A

yield from — Delegating to Sub-generators

yield 从 delegates to another generator, flattening nested generators:

def inner():
    yield 1
    yield 2

def outer():
    yield 0
    yield from inner()  # yields all values from inner()
    yield 3

print(list(outer()))  # [0, 1, 2, 3]

This is cleaner than manually iterating and yielding each value from the sub-generator.

示例代码

              # Basic generator with yield
def count_up_to(n):
    i = 0
    while i < n:
        yield i
        i += 1

print(list(count_up_to(5)))  # [0, 1, 2, 3, 4]

# Generator vs list — memory efficiency
import sys

big_list = [x ** 2 for x in range(10000)]
big_gen = (x ** 2 for x in range(10000))
print(f"List size: {sys.getsizeof(big_list)} bytes")
print(f"Gen size:  {sys.getsizeof(big_gen)} bytes")

# Using next() manually
counter = count_up_to(3)
print(next(counter))  # 0
print(next(counter))  # 1
print(next(counter))  # 2

# Infinite generator — Fibonacci
def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a + b

fib = fibonacci()
fib_10 = [next(fib) for _ in range(10)]
print(f"Fibonacci: {fib_10}")

# Chaining generators in a pipeline
def numbers():
    for n in range(1, 10):
        yield n

def squared(seq):
    for n in seq:
        yield n ** 2

def evens(seq):
    for n in seq:
        if n % 2 == 0:
            yield n

pipeline = evens(squared(numbers()))
print(f"Pipeline: {list(pipeline)}")

# yield from — delegating
def inner():
    yield 1
    yield 2

def outer():
    yield 0
    yield from inner()
    yield 3

print(f"yield from: {list(outer())}")  # [0, 1, 2, 3]

# Reading a large file lazily
def read_lines(filename):
    with open(filename) as f:
        for line in f:
            yield line.strip()  # one line at a time, low memory
            

Shows 基础 generator, memory comparison 对比列表, manual 下一个() usage, 无限 Fibonacci generator, chained 流水线 (数字 → squared → evens), yield 从 delegation, and lazy file reader 模式.

动手试一试

  • 文本计数器

    Generators 是 perfect for streaming 文本处理行 通过行.

  • JSON Formatter

    Generators 能 stream 大 JSON 数组不使用加载一切到 memory.

常见错误

错误写法

Trying 到 reuse generator 那个有 是 exhausted

正确做法

Generators 是单个-use — 一次 consumed, 它们是 空 and 下一个() raises StopIteration. 到 reuse, recreate generator: gen = my_generator() 再次, or convert 到列表首先如果你 需要到 iterate 多个次数.

错误写法

创建无限 generator and iterating 使用 for 循环不使用 break

正确做法

无限 generator (虽然真: yield) 从不停止. 总是 use bounded consumer: 下一个() fixed 数字的 次数, itertools.islice(gen, n), or for 循环使用 break 条件. bare 'for x in infinite_gen:' hangs forever.

相关速查表

练习 Generators

4 个练习

练习 Generators

查阅不认识的术语

新手友好的术语表用大白话解释编程术语——变量、函数、DOM、Promise 等等。

查看术语表

动手做项目

通过引导式项目应用所学,含起始代码与解决方案。

查看项目

解读错误信息

用大白话解释常见错误——什么意思、为什么发生、如何修复。

查看错误