Learning Python - Mark Lutz [283]
Timing Script
To time iteration tool speed, run the following script—it uses the timer module we wrote to time the relative speeds of the list construction techniques we’ve studied:
# File timeseqs.py
import sys, mytimer # Import timer function
reps = 10000
repslist = range(reps) # Hoist range out in 2.6
def forLoop():
res = []
for x in repslist:
res.append(abs(x))
return res
def listComp():
return [abs(x) for x in repslist]
def mapCall():
return list(map(abs, repslist)) # Use list in 3.0 only
def genExpr():
return list(abs(x) for x in repslist) # list forces results
def genFunc():
def gen():
for x in repslist:
yield abs(x)
return list(gen())
print(sys.version)
for test in (forLoop, listComp, mapCall, genExpr, genFunc):
elapsed, result = mytimer.timer(test)
print ('-' * 33)
print ('%-9s: %.5f => [%s...%s]' %
(test.__name__, elapsed, result[0], result[-1]))
This script tests five alternative ways to build lists of results and, as shown, executes on the order of 10 million steps for each—that is, each of the five tests builds a list of 10,000 items 1,000 times.
Notice how we have to run the generator expression and function results through the built-in list call to force them to yield all of their values; if we did not, we would just produce generators that never do any real work. In Python 3.0 (only) we must do the same for the map result, since it is now an iterable object as well. Also notice how the code at the bottom steps through a tuple of four function objects and prints the __name__ of each: as we’ve seen, this is a built-in attribute that gives a function’s name.[45]
Timing Results
When the script of the prior section is run under Python 3.0, I get these results on my Windows Vista laptop—map is slightly faster than list comprehensions, both are quicker than for loops, and generator expressions and functions place in the middle:
C:\misc> c:\python30\python timeseqs.py
3.0.1 (r301:69561, Feb 13 2009, 20:04:18) [MSC v.1500 32 bit (Intel)]
---------------------------------
forLoop : 2.64441 => [0...9999]
---------------------------------
listComp : 1.60110 => [0...9999]
---------------------------------
mapCall : 1.41977 => [0...9999]
---------------------------------
genExpr : 2.21758 => [0...9999]
---------------------------------
genFunc : 2.18696 => [0...9999]
If you study this code and its output long enough, you’ll notice that generator expressions run slower than list comprehensions. Although wrapping a generator expression in a list call makes it functionally equivalent to a square-bracketed list comprehension, the internal implementations of the two expressions appear to differ (though we’re also effectively timing the list call for the generator test):
return [abs(x) for x in range(size)] # 1.6 seconds
return list(abs(x) for x in range(size)) # 2.2 seconds: differs internally
Interestingly, when I ran this on Windows XP with Python 2.5 for the prior edition of this book, the results were relatively similar—list comprehensions were nearly twice as fast as equivalent for loop statements, and map was slightly quicker than list comprehensions when mapping a built-in function such as abs (absolute value). I didn’t test generator functions then, and the output format wasn’t quite as grandiose:
2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
forStatement => 6.10899996758
listComprehension => 3.51499986649
mapFunction => 2.73399996758
generatorExpression => 4.11600017548
The fact that the actual 2.5 test times listed here are over two times as slow as the output I showed earlier is likely due to my using a quicker laptop for the more recent test, not due to improvements in Python 3.0. In fact, all the 2.6 results for this script are slightly quicker than 3.0 on this same machine if the list call is removed from the map test to avoid creating the results list twice (try this on your own to verify).
Watch what happens, though, if we change this script to perform a real