How to use multiproccessing (multithreading in this example?)

Question

I have this piece of code. It calculates different sequences of symbols using itertools f.e. in this file symbols (x) = [34567QWERTYUIOPLKJHGFDSAZXCVBNM] and replacements (y) = 54. My task is to use all the powers of the machine to calc those sequences asaf. I wanna use multiprocessing. Can you show how?

import itertools
import time
import datetime

def calc():
    start_time = round(time.time())
    print('Date at the begin ', datetime.datetime.now())
    for i in itertools.combinations_with_replacement('234567QWERTYUIOPLKJHGFDSAZXCVBNM', 54):
        xs = ''.join(i)
        with open('hash.txt', 'a+') as writefile:
            writefile.write('\n' + xs)
    stop_time = round(time.time())
    diff_time = stop_time - start_time
    print("Took over ", diff_time)
    print('Date of the stop ', datetime.datetime.now())

What do I need? I want to use

1)multiprocessing (or multithreading) to run this solution on a single machine and it would compute the result much faster

2)Distribute this process via some machines using websockets? or something. So we have machines in different places via the world that work as one and compute that piece of code asaf. Can u suggest something?

Answer 1

You could do something like this. It will utilise your whole CPU, it won't be much faster tough because multiple processes accessing the same file won't be fast. Multiple threads won't do you any good here because of Python's Global Interpreter Lock (GIL). As to how you would do something like this on multiple PCs, I don't know the answer, but, as Lancelot du Lac pointed out, you will never find all those combinations.

import itertools
import time
import datetime
import multiprocessing

def write(i):
    xs = ''.join(i)
    with open('hash.txt', 'a+') as writefile:
        writefile.write('\n' + xs)

def calc():
    start_time = round(time.time())
    print('Date at the begin ', datetime.datetime.now())
    pool = multiprocessing.Pool()
    for i in itertools.combinations_with_replacement('234567QWERTYUIOPLKJHGFDSAZXCVBNM', 3):
        last_item = pool.apply_async(write, args=(i, ))
    last_item.wait()
    pool.terminate()
    stop_time = round(time.time())
    diff_time = stop_time - start_time
    print("Took over ", diff_time)
    print('Date of the stop ', datetime.datetime.now())


if __name__ == "__main__":
    calc()