'Joblib TypeError: 'NoneType' object is not callable

I'm trying to run a function in parallel using joblib and concurrent. However it returns me an error after 49 iterations with 10 jobs (or 19 if i use 5 jobs). I cannot understand the problem because I used a very similar script and it worked perfectly.

This is the code and the error:

def fasta_for_each_cluster(i): 
    it=i[0]
    it_s=it.replace('.', 'X')
    protein_ids=list(i[1]['subject']) #i[0] group name
    thermo_id=np.unique(list(i[1]['query']))
    with open(str('all_group_id'+it_s),'w') as all:  
        for listitem in protein_ids:
            all.write('%s\n' % listitem)
        for id in thermo_id:
            all.write('%s\n' % id)
    cmd_removing_duplicated_id=[os.path.realpath(str(gawk_path+" -i inplace '!a[$0]++' "+ ('all_group_id'+it_s)))] #removing eventual ids duplicated
    procs=subprocess.call(cmd_removing_duplicated_id, shell=True) 
    

    number_file = str('all_group_id'+it_s) # Input ids, one per line
    result_file = str('all_group_fasta'+it_s+'.fasta') #fasta from previous ids list


    cmd=[os.path.realpath(str(grep_path + ' -w -A 1 -f '+ number_file + ' '+ fasta_file + "> "+ result_file))] 
    procs=subprocess.call(cmd, shell=True) 

    os.remove(str('all_group_id'+it_s)) 
    

if __name__ == '__main__':
    Parallel(n_jobs=n_jobs)(delayed(fasta_for_each_cluster(i))(i) for i in tqdm.tqdm(cluster_group))

File "/home/usr/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/home/usr/anaconda3/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
TypeError: 'NoneType' object is not callable
  0%|                        | 19/13751 [00:08<1:45:43,  2.16it/s]

I tried with python 3.9 and 3.7.

Thanks !



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source