In my code I have a directory of small python modules that act as plugins. They can be called by a main program as directed. Each plugin, therefore, has a standard interface and needs to work with the main program. I wanted to setup an automated test that would run the main program, load each… Read More Nose and test generators
In this post I take Python code to do a task (draw fractals), write it in the spirit of Python (a pleasure to write, read and maintain), use profiling to figure out the bottle necks and then speed up the most time consuming function using cython. Python is great for writing projects of all sizes.… Read More Cythonize!
Numpy has some great builtins for running unit tests under numpy.testing . There are routines for comparing arrays that come in handy for tests. Many of these have verbose settings that give useful details in a thoughtful format for chasing down why a test failed.
I use docopt to build simple, self-documenting command line interfaces to scripts. I was delighted when sphinx’s outdoc features seemed to interpret docopt’s options well: But it is apparent that the formatting of the usage part is a bit off and any option that does not begin with — or – messes up the options… Read More Making docopt and Sphinx work together
One majorly annoying thing for Pythonistas is the search path that the import command uses. When I was a beginner and wanted to get cracking I’d simply add whatever I was working on to PYTHONPATH, not wanting to get bogged down in silly details (that’s what we were using Python for, right?). I learned my… Read More Some notes on Python imports and project organization
An ideal code documentation system should allow you to write documentation once (when you are writing the code) and then allow you to display the documentation in different contexts, such as project manuals, inline help, command line help and so on. You shouldn’t need to duplicate docs – it is a waste of effort and… Read More Use numpydoc + sphinx
Rod Hilton has a blog post where he argues against using puzzler type questions during an interview. I agree with him in spirit, but feel that puzzler questions have a place in interviews, provided they are used correctly. Hilton’s piece is very well written and I recommend it, especially if you are in a position… Read More A note on “The Worst Programming Interview Question”
Just in case any of you, hearing about how slow and ponderous Python is, were worried about indexing into a list – it’s a O(1) operation: In : big_list = range(int(1e8))len(bIn : len(big_list)Out: 100000000In : %timeint big_list[-1]ERROR: Line magic function `%timeint` not found.In : %timeit big_list[-1]10000000 loops, best of 3: 66.1 ns per loopIn :… Read More Recap: Python list indexing is O(1)
So, sometimes you need to work at night and not light up the whole room (e.g. when there is a sleeping baby next to you). The dimmest setting on the mac screen is still annoyingly bright, and indeed after a while, begins to hurt my eyes. One native thing to try is “invert colors” under… Read More Mac OS X: Coding in the dark
A while ago a kindly reader pointed out that Python’s string .format method is, after all, a function, and carries with it some over head. I have some inner loop code that I could stand to run a little faster and I was looking for a way to speed things up without losing readability. In… Read More Python: To format or to concatenate
I had a torrid love affair with doctests. It was AMAZING! You mean you can write documentation AND test your code at the same time!? You can write a line of code, put down the answer next to it and the computer WILL CHECK IT FOR YOU? Well, this went on for several months. Then… Read More Doctests or nosetests?
Continuing with our timing and efficiency theme: What is the more efficient way of building up a string? from cStringIO import StringIOdef string_concat(size=int(1e6)): repeat = ‘A’ st = ” for n in range(size): st += repeat return stdef list_concat(size=int(1e6)): repeat = ‘A’ st =  for n in range(size): st += repeat return ”.join(st)def stringIO_concat(size=int(1e6)):… Read More Python string concatenation
Is it better to access a numpy array with an iterator or indexing? import numpy# This is how I’ve always done it – straight python idiomatic usagedef iterate1(m=100000): a = numpy.empty(m) sum = 0 for n in a: sum += n return sum# Didn’t know about this one, but it is the recommended one in… Read More numpy arrays and iterators
HDF5 is a good solution for storing large datasets on disk. Python’s h5py library makes it possible to pretend that data stored on disk is just like an in memory array. It is important to keep in mind that the data is really stored on disk and is read in every time a slice or… Read More HDF5 is not for fast access