'Using concurent.futures raise me a strange AttributeError on django python with channels
I'm facing an issue trying to use the concurrent.futures package in python. I'm working on a website using django and channels to perform asynchronous comunication.
I have to load data from many url and i want to be able to do it with multiprocessing.
Unfortunately i'm enable to make this code work as it always give me an AttributeError. Could it be related to the architecture of django or channels ? I'm using the ProcessPoolExecutor in a channels receive() function inside a WebSocket Object.
Here is a sample of my code :
class CreateValuesConsumer(WebsocketConsumer):
def connect(self):
self.accept()
def disconnect(self, close_code):
pass
def create_single_value(self, card):
print('start create_single_value')
condition = Condition.objects.filter(short_name='NM')[0]
card_language = Language.objects.filter(short_name='FR')[0]
seller_origin = Language.objects.filter(short_name='FR')[0]
value = card.create_value(condition, card_language, seller_origin)
if value is not None:
message = "models.value." + str(value.id) + ' : create | card_name = ' \
+ card.name_en.lower() + " | price = " + str(value.cheaper_price)
is_error = 0
else:
error = ParsingError.objects.filter(card=card)[0]
message = error.message
is_error = 1
self.send(text_data=json.dumps({
'message': message,
'is_error': is_error
}))
print('end')
def receive(self, text_data):
json_data = json.loads(text_data)
extension = Extension.objects.get(pk=json_data["id"])
cards_to_value = Card.objects.filter(extension=extension)
today = datetime.now()
for card in cards_to_value:
values = Value.objects.filter(card=card, date__year=today.year, date__month=today.month, date__day=today.day)
for value in values:
print('delete' + str(value.pk))
value.delete()
if len(cards_to_value) > 0:
futures = []
with concurrent.futures.ProcessPoolExecutor() as executor:
for card in cards_to_value:
futures.append(executor.submit(self.create_single_value, card=card))
for future in concurrent.futures.as_completed(futures):
print(str(future.result))
print(str(future.exception()))
And here are the errors that shows at execution :
<bound method Future.result of <Future at 0x1bafbc283a0 state=finished raised AttributeError>>
Can't pickle local object 'convert_exception_to_response.<locals>.inner'
I believe the 'convert_exception_to_response..inner' object is a django or a channels object but if anyone as an idea on how to solve i'll be happy to hear it :)
Thanks
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
