'Is there a more efficient way to allocate lots of instances of a class in a loop?
I am working on a Python program, and there is an inner loop which is causing performance problems. I ran a profiler on it, and I found that 50% of the time is spent in object allocation. Basically, something like this:
result_list = []
for i in range(BIG_NUMBER):
obj = MyClass()
# do stuff with obj...
result_list.append(obj)
The line obj = MyClass() takes 50% of the time. Is there any way to efficiently allocate lots of these objects in Python? Perhaps some sort of bulk allocation?
Without copy / pasting the entire MyClass, it has the following characteristics:
- It is an ORM object, so it basically maps attributes to database values
- It has quite a few property functions, a meta class, etc. so it's a pretty "fat" class
The # do stuff with obj... line largely sets the various attributes, usually via setattr, but in a few cases via a custom function. So if there are 10 attributes to set, 9 will usually be via setattr, and 1 might be via a special function. This part takes ~40% of the time and I'm optimizing it separately.
Solution 1:[1]
If the object starts w an empty constructor call, could you not pull it out if the loop? Then copy.deepcopy your seed object?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | JL Peyret |
