Learning Python - Mark Lutz [280]
* * *
Why You Will Care: One-Shot Iterations
In Chapter 14, we saw how some built-ins (like map) support only a single traversal and are empty after it occurs, and I promised to show you an example of how that can become subtle but important in practice. Now that we’ve studied a few more iteration topics, I can make good on this promise. Consider the following clever alternative coding for this chapter’s zip emulation examples, adapted from one in Python’s manuals:
def myzip(*args):
iters = map(iter, args)
while iters:
res = [next(i) for i in iters]
yield tuple(res)
Because this code uses iter and next, it works on any type of iterable. Note that there is no reason to catch the StopIteration raised by the next(it) inside the comprehension here when any one of the arguments’ iterators is exhausted—allowing it to pass ends this generator function and has the same effect that a return statement would. The while iters: suffices to loop if at least one argument is passed, and avoids an infinite loop otherwise (the list comprehension would always return an empty list).
This code works fine in Python 2.6 as is:
>>> list(myzip('abc', 'lmnop'))
[('a', 'l'), ('b', 'm'), ('c', 'n')]
But it falls into an infinite loop and fails in Python 3.0, because the 3.0 map returns a one-shot iterable object instead of a list as in 2.6. In 3.0, as soon as we’ve run the list comprehension inside the loop once, iters will be empty (and res will be []) forever. To make this work in 3.0, we need to use the list built-in function to create an object that can support multiple iterations:
def myzip(*args):
iters = list(map(iter, args))
...rest as is...
Run this on your own to trace its operation. The lesson here: wrapping map calls in list calls in 3.0 is not just for display!
* * *
Value Generation in Built-in Types and Classes
Finally, although we’ve focused on coding value generators ourselves in this section, don’t forget that many built-in types behave in similar ways—as we saw in Chapter 14, for example, dictionaries have iterators that produce keys on each iteration:
>>> D = {'a':1, 'b':2, 'c':3}
>>> x = iter(D)
>>> next(x)
'a'
>>> next(x)
'c'
Like the values produced by handcoded generators, dictionary keys may be iterated over both manually and with automatic iteration tools including for loops, map calls, list comprehensions, and the many other contexts we met in Chapter 14:
>>> for key in D:
... print(key, D[key])
...
a 1
c 3
b 2
As we’ve also seen, for file iterators, Python simply loads lines from the file on demand:
>>> for line in open('temp.txt'):
... print(line, end='')
...
Tis but
a flesh wound.
While built-in type iterators are bound to a specific type of value generation, the concept is similar to generators we code with expressions and functions. Iteration contexts like for loops accept any iterable, whether user-defined or built-in.
Although beyond the scope of this chapter, it is also possible to implement arbitrary user-defined generator objects with classes that conform to the iteration protocol. Such classes define a special __iter__ method run by the iter built-in function that returns an object having a __next__ method run by the next built-in function (a __getitem__ indexing method is also available as a fallback option for iteration).
The instance objects created from such a class are considered iterable and may be used in for loops and all other iteration contexts. With classes, though, we have access to richer logic and data structuring options than other generator constructs can offer.
The iterator story won’t really be complete until we’ve seen how it maps to classes, too. For now, we’ll have to settle for postponing its conclusion until we study class-based iterators in Chapter 29.
* * *
[44] Interestingly, generator functions are also something of a “poor man’s” multithreading device—they interleave a function