Python Iterators and Iterables: What’s Really Happening Inside a `for` Loop

Every time you write for item in something:, Python runs a specific sequence of steps behind the scenes. Understanding those steps — the iterator protocol — explains why for works on lists, strings, files, generators, and your own custom classes, and how to build objects that participate in the same protocol.

Two Distinct Concepts

An iterable is any object you can iterate over. It knows how to hand out an iterator when asked.

An iterator is the object that actually performs the iteration — it tracks position and returns the next item on request.

A list is an iterable. When you loop over a list, Python creates an iterator from it. The iterator remembers where in the list it is and advances one step at a time.

numbers = [10, 20, 30]

# Python does this internally when you write "for n in numbers:"
it = iter(numbers)          # calls numbers.__iter__()
print(next(it))             # 10 — calls it.__next__()
print(next(it))             # 20
print(next(it))             # 30
print(next(it))             # raises StopIteration — signals loop end

When next() raises StopIteration, the for loop catches it and exits cleanly.

The Iterator Protocol

An object is a valid iterator if it implements:

__iter__(self) — returns the iterator itself (allows iterators to be used where iterables are expected).
__next__(self) — returns the next value, or raises StopIteration when exhausted.

An object is a valid iterable if it implements:

__iter__(self) — returns an iterator (does not need to be itself).

Lists, tuples, and strings are iterables but not iterators. Calling iter() on them creates a separate iterator object each time, which means you can loop over a list multiple times.

An iterator is consumed: once exhausted, calling next() again raises StopIteration. You would need to create a new iterator to start from the beginning.

my_list = [1, 2, 3]

it1 = iter(my_list)
it2 = iter(my_list)  # independent iterator from the same list

print(next(it1))  # 1
print(next(it1))  # 2
print(next(it2))  # 1 — it2 started fresh

Building a Custom Iterator

Any class that implements __iter__ and __next__ becomes an iterator:

class NumberRange:
    """An iterator that yields integers from start to stop (inclusive)."""

    def __init__(self, start, stop, step=1):
        self.current = start
        self.stop = stop
        self.step = step

    def __iter__(self):
        return self  # this object is its own iterator

    def __next__(self):
        if self.current > self.stop:
            raise StopIteration
        value = self.current
        self.current += self.step
        return value

for n in NumberRange(1, 10, 2):
    print(n, end=" ")
# 1 3 5 7 9

Because NumberRange implements the iterator protocol, it works with for loops, list(), sum(), max(), and any other construct that iterates.

Separating the Iterable from the Iterator

For objects that should support multiple concurrent iterations (like a list), the iterable and the iterator should be separate objects. The iterable creates a fresh iterator each time __iter__ is called.

class Fibonacci:
    """Iterable that produces Fibonacci numbers up to a limit."""

    def __init__(self, limit):
        self.limit = limit

    def __iter__(self):
        # Returns a new FibonacciIterator each time
        return FibonacciIterator(self.limit)

class FibonacciIterator:
    def __init__(self, limit):
        self.limit = limit
        self.a, self.b = 0, 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.a > self.limit:
            raise StopIteration
        value = self.a
        self.a, self.b = self.b, self.a + self.b
        return value

fib = Fibonacci(100)

# Can iterate multiple times from the same object
print(list(fib))          # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
print(list(fib))          # [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]
print(sum(fib))           # 232

If Fibonacci.__iter__ returned self and Fibonacci tracked state directly, the second list(fib) would produce [] because the state was exhausted.

Generators: The Practical Alternative

Writing a full iterator class is correct but verbose. Python’s yield keyword creates a generator function that produces an iterator automatically, with much less code:

def fibonacci(limit):
    a, b = 0, 1
    while a <= limit:
        yield a
        a, b = b, a + b

print(list(fibonacci(100)))
# [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]

Every time the generator function executes yield, it pauses and hands the value to the caller. On the next next() call, it resumes from where it paused. The generator object produced by calling fibonacci(100) is itself an iterator.

Generators are lazy: they compute values one at a time on demand, making them memory-efficient for large or infinite sequences:

def count_up_from(n):
    while True:
        yield n
        n += 1

counter = count_up_from(0)
print(next(counter))   # 0
print(next(counter))   # 1
print(next(counter))   # 2
# Can run forever — only computes the next value when asked

When to Use Each Approach

Use a class-based iterator when the iteration state is complex, you need to support __len__ or __getitem__ alongside iteration, or when multiple iterators over the same data need to coexist.

Use a generator function for most custom iteration logic — it is simpler, requires less code, and handles StopIteration automatically.

Use a generator expression for simple one-shot sequences:

squares = (x ** 2 for x in range(1, 11))  # generator expression
print(sum(squares))  # 385

Practical Example: Lazy File Processing

def read_blocks(file_path, block_size=1024):
    """Read a file in fixed-size blocks rather than loading it all at once."""
    with open(file_path, "rb") as f:
        while True:
            block = f.read(block_size)
            if not block:
                break
            yield block

# Process a large file without loading it into memory
for block in read_blocks("large_file.bin"):
    process(block)  # handle one block at a time

The file’s contents are never all in memory at once. The for loop calls next() on the generator, which reads the next block.

Common Mistakes

Reusing an exhausted iterator. Once an iterator raises StopIteration, it is done. Call iter() again to get a fresh one, or redesign the class so __iter__ returns a new object.

Returning values from __iter__ instead of self. If the class is both the iterable and the iterator, __iter__ must return self, not create a new object.

Not raising StopIteration in __next__. Without this, for loops will hang or raise unexpected errors. Generators handle this automatically.

Python Iterators and Iterables: What’s Really Happening Inside a for Loop