Move multiple mice with Python

Question

Note: I am open to different solutions which achieve the desired capability

I am working on a project with many instances of the same game.

Therefore, I am sending keyboard and mouse instructions to each of theses processes, in parallel.

I am currently using win32ui as follows:

After finding the processes hwnd (windows handle) values from Get HWND of each Window?, so a hwnds_list with all the processes with a given name e.g. [788133, 723724, ...]

I am sending instructions to each of the processes, by creating a PyCWnd object:

PyCWnd = win32ui.CreateWindowFromHandle(hwnd)

Then, say I want to press the return key, I used:

def press_return(pycwnd):
    pycwnd.SendMessage(win32con.WM_KEYDOWN, win32con.VK_RETURN, 0)
    pycwnd.SendMessage(win32con.WM_KEYUP, win32con.VK_RETURN, 0)

Then I run this in parallel with:

def press_return_par(hwnds):
    # Get the Window from handle
    pycwnd = make_pycwnd(hwnds)
    time.sleep(0.1)
    press_return(pycwnd)

num_workers = len(hwnds_list)
with Pool(num_workers) as p:
    p.map(press_return_par, hwnds_list)

So, I have a good way of sending keyboard commands, and even scrolling with a mouse, but can't work out how to do this with mouse movements.

Ideally, I'd like to say, "Move to (x, y) coordinates over n time". This 'ideal' method needs to not effect the current cursor (or allow a locked cursor for each process/game), as I will want to do this across ~8 instances of the game.

I've looked through the official pywin32 docs http://timgolden.me.uk/pywin32-docs/contents.html, other answers that look bang on https://stackoverflow.com/a/3721198/11181287 but use win32api.mouse_event, so I don't know how to convert this to work with the multiple pycwnd objects.

• https://stackoverflow.com/a/3721053/11181287 looks close, but doesn't seem to move the mouse, it just does the right click, although I have made some guesses for the MAKELPARAM function which is not listed.
• In addition, https://github.com/oblitum/Interception could be helpful but haven't found good docs for how to apply this here.
• As the game is an FPS game, running multiple instances through nucleus-coop, using a VM etc... won't be fast enough (from my current research).
• PyAutoGUI is exactly the functionality I want, with the speed, but (as expected) I haven't been able to set it up to work for multiple mice/processes
• There could be something in sending DirectX inputs into the game (black ops 2)?

(I'm running windows 10, Python 3.7.11, and only know Python)

Answer 1

I have two possible solutions to your mice issue.

1. What if you used only one mouse to control all of the windows? With pyautogui you could tab into each window when necessary and control the mouse for that window. I'm not sure how efficient this would be and how fast the mouse control for each window would be, but it's still sort of a solution.

OR

2. You could control the mouse with the keyboard. See this article https://www.windowscentral.com/how-control-mouse-using-keyboard-windows-10

I apologize for not just commenting, unfortunately I don't have enough reputation.