Tuesday, July 14, 2015

Numpy str on integers extremely slow?

I found a weird performance difference today: create a numpy array of integers, and then convert it to an array of strings in a list comprehension. It's weirdly slow. Cast the numpy integers to python int's before calling str(), and you get nearly a 10x speedup. Observe:


$ python -m timeit 'import numpy as np; newX = [x for x in np.arange(4096)]; [str(x) for x in newX]'
100 loops, best of 3: 10.5 msec per loop
$ python -m timeit 'import numpy as np; newX = [x for x in np.arange(4096)]; [str(int(x)) for x in newX]'
1000 loops, best of 3: 1.23 msec per loop

Monday, July 13, 2015

Line by line python performance profiling (in particular, about where that @profile decorator comes from)

Here's the thing I didn't understand about rkern's line-by-line performance profiler: the kernprof executable contains information about the @profile decorator - no import is necessary. When you run your script without kernprof, you will get an error. But if you run kernprof -l [original python call], it works. Happy profiling!