List Comprehensions and Generator Expressions: Python’s Most Readable Shortcut

List comprehensions are one of Python’s most praised features — and one of its most abused. A well-written comprehension replaces five lines of loop code with one clear line. A badly written one turns straightforward logic into a puzzle. This guide covers both tools, when to use each, and when to reach for neither.

List Comprehensions

A list comprehension builds a new list by applying an expression to each item in an iterable, with an optional filter condition:

# Basic form: [expression for item in iterable]
squares = [x**2 for x in range(10)]
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

# With a filter: [expression for item in iterable if condition]
even_squares = [x**2 for x in range(10) if x % 2 == 0]
# [0, 4, 16, 36, 64]

Compare this to the equivalent loop:

# Loop version — same result, more code
even_squares = []
for x in range(10):
    if x % 2 == 0:
        even_squares.append(x**2)

The comprehension is more concise, and once you’re comfortable reading them, more immediately parseable too.

Real-World List Comprehension Examples

Transforming strings

raw_data = ["  Alice ", "  BOB", "charlie  "]
normalised = [name.strip().title() for name in raw_data]
# ['Alice', 'Bob', 'Charlie']

Filtering records

products = [
    {"name": "Widget", "price": 9.99, "in_stock": True},
    {"name": "Gadget", "price": 24.99, "in_stock": False},
    {"name": "Doohickey", "price": 4.99, "in_stock": True},
]

available = [p["name"] for p in products if p["in_stock"]]
# ['Widget', 'Doohickey']

Flattening nested lists

matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [cell for row in matrix for cell in row]
# [1, 2, 3, 4, 5, 6, 7, 8, 9]

The order in nested comprehensions matches the order of the for loops if written out longhand: outer loop first, inner loop second.

if-else inside a comprehension

When you need a conditional transform (not a filter), put the if-else in the expression position:

scores = [45, 82, 67, 90, 55, 78]
grades = ["pass" if s >= 60 else "fail" for s in scores]
# ['fail', 'pass', 'pass', 'pass', 'fail', 'pass']

When Comprehensions Get Too Complex

A comprehension with more than two levels of nesting or a complicated condition should be a loop instead. Readability comes first:

# Hard to follow — when to use a loop instead
result = [
    process(item)
    for sublist in nested_data
    for item in sublist
    if item is not None
    if validate(item)
]

# Same logic as a loop — easier to debug and extend
result = []
for sublist in nested_data:
    for item in sublist:
        if item is not None and validate(item):
            result.append(process(item))

The rule of thumb: if you need to explain the comprehension to someone else, consider rewriting it as a loop.

Generator Expressions

A generator expression looks exactly like a list comprehension but uses parentheses instead of square brackets:

# List comprehension — creates the entire list in memory
squares_list = [x**2 for x in range(1_000_000)]

# Generator expression — produces values one at a time on demand
squares_gen = (x**2 for x in range(1_000_000))

The difference is when and how the values are computed. A list comprehension runs immediately and stores all results. A generator expression stores the recipe, not the results, and computes each value only when you ask for it.

Lazy Evaluation

Generators are lazy — they only produce a value when the calling code requests the next one. This has two important consequences:

Memory efficiency: A generator over one million items uses the same memory as a generator over ten items — roughly none. The list comprehension equivalent would allocate memory for all one million values upfront.

Can represent infinite sequences:

def natural_numbers():
    n = 0
    while True:
        yield n
        n += 1

# Without a generator, this is impossible
first_five_evens = (n for n in natural_numbers() if n % 2 == 0)
for _ in range(5):
    print(next(first_five_evens))   # 0, 2, 4, 6, 8

Generator Expressions in Practice

Generators work naturally with functions that accept iterables:

# Sum without creating an intermediate list
total = sum(x**2 for x in range(1000))

# All/any short-circuit on first result
all_positive = all(x > 0 for x in [1, 2, 3, -1, 5])  # False
any_negative = any(x < 0 for x in [1, 2, 3, -1, 5])  # True

# Processing a large file line by line
long_lines = sum(1 for line in open("data.txt") if len(line) > 100)

When passed as the single argument to a function, the generator’s parentheses can double as the function’s parentheses:

total = sum(x**2 for x in range(100))  # not sum((x**2 for x in range(100)))

Converting Between Types

gen = (x**2 for x in range(5))

# Convert to list when you need to reuse or index the results
squares = list(gen)    # [0, 1, 4, 9, 16]

# Generator is exhausted after one pass
print(list(gen))       # [] — nothing left

A generator can only be iterated once. If you need to iterate over the same data multiple times, use a list comprehension instead.

Choosing Between the Two

Use a list comprehension when:

• You need to index or slice the result (result[0], result[2:5])
• You need to iterate over the result more than once
• The dataset is small enough that storing all results is fine
• You need len() on the result

Use a generator expression when:

• You’re passing the result directly to sum(), max(), any(), all(), or similar
• You’re processing a large dataset where storing all results would be wasteful
• You only need to iterate once
• You’re reading from a file or stream

When uncertain, generators are the safer default — you can always convert with list(), but you can’t unconvert a list back to a lazy sequence.