TuningTricks


from email list optimizing histogram of numbers in huge file

 

- wrap your code in a function. Functions run much faster in python than module level code,

- look into using collections.defaultdict for histogram. A dictionary is a very appropriate way to store this data. numpy is good for processing numeric data once they are already in arrays, not for populating them.

 

Both Python "array.array" and "list" are implemented as dyn arrays, not as C-like linked lists.