duplicated values multiprocessing python

Question

I'm trying to do multiprocessing with python but I have duplicated values in results files. Could you please help me to solve that? here is my code:

import itertools
from multiprocessing import Pool
from multiprocessing import Manager
import pandas as pd

PARAMS = {}
LPT_LIMIT = [4, 6, 8, 10]
HPT_LIMIT = [1.6, 1.8, 2.0]
NB_FLIGHT = [10, 30]
LPT_EXCEEDENCE = [1, 4]
HPT_EXCEEDENCE = [3, 4]

tmp = [LPT_LIMIT, HPT_LIMIT, NB_FLIGHT, LPT_EXCEEDENCE, HPT_EXCEEDENCE]
parameters = list(itertools.product(*tmp))

def toto(param):
    PARAMS['LPT_LMIT'] = param[0]
    PARAMS['HPT_LMIT'] = param[1] 
    PARAMS['NB_FLIGHT'] = param[2]
    PARAMS['LPT_EXCEEDENCE'] = param[3]
    PARAMS['HPT_EXCEEDENCE'] = param[4]
    return PARAMS

if __name__=='__main__':
    pool = Pool()
    manager = Manager()
    my_list = manager.list()
    my_list.append(pool.map(toto, parameters))
    flat_list = [item for sublist in my_list for item in sublist]
    pd.DataFrame(flat_list).to_excel('results.xlsx', index=False)
    

the results is that I have only value of 4 in HPT_EXCEEDENCE (please see attached file) enter image description here

I have only HPT_EXCEEDENCE = 4 but HPT_EXCEEDENCE is 3 or 4 So I don't know what's wrong with my code

Answer 1

Unless you are doing something more complex, you don't need Manager(). The problem specifically was the location of PARAMS = {}. See updated code below. This seems to get the result you want.

import itertools
from multiprocessing import Pool
from multiprocessing import Manager
import pandas as pd


LPT_LIMIT = [4, 6, 8, 10]
HPT_LIMIT = [1.6, 1.8, 2.0]
NB_FLIGHT = [10, 30]
LPT_EXCEEDENCE = [1, 4]
HPT_EXCEEDENCE = [3, 4]

tmp = [LPT_LIMIT, HPT_LIMIT, NB_FLIGHT, LPT_EXCEEDENCE, HPT_EXCEEDENCE]
parameters = list(itertools.product(*tmp))

def toto(param):
    PARAMS = {}
    PARAMS['LPT_LMIT'] = param[0]
    PARAMS['HPT_LMIT'] = param[1]
    PARAMS['NB_FLIGHT'] = param[2]
    PARAMS['LPT_EXCEEDENCE'] = param[3]
    PARAMS['HPT_EXCEEDENCE'] = param[4]
    return PARAMS

if __name__=='__main__':
    pool = Pool()
    my_list = pool.map(toto, parameters)
    pd.DataFrame(my_list).to_excel('results1.xlsx', index=False)