'How do I automate a Java application with AutoHotkey?

I am using a Java application at work. I need to send clicks to buttons and fill in textboxes. I would like these actions to happen in the background. The ahk_class of the window is SunAwtFrame, and none of the controls are exposed to WindowSpy.

I have used JavaFerret to determine that the first button I want to press has an AccessibleAction

Version Information:
    Java virtual machine version: 1.7.0_25
    Access Bridge Java class version: 1.7.0_25
    Access Bridge Java DLL version: AccessBridge 2.0.2
    Access Bridge Windows DLL version: AccessBridge 2.0.2

AccessibleContext information:
    Name:  New Call
    Description:  Place a new call
    Role:  push button
    Role in en_US locale:  push button
    States:  enabled,focusable,visible,showing,opaque
    States in en_US locale:  enabled,focusable,visible,showing,opaque
    Index in parent:  1
    Children count:  0
    Bounding rectangle:  [288, 317, 385, 376]
    Top-level window name:  Phone Assistant: 
    Top-level window role:  frame
    Parent name:  
    Parent role:  panel
    Visible descendents count:  0

AccessibleIcons info:
    Number of icons:  1
    Icon 0 description: jar:http://proxy.m5net.com/vox/pa/receptioncenter.jar!/resources/phone.png
    Icon 0 height: 26
    Icon 0 width: 27

AccessibleActions info:
    Number of actions:  1
    Action 0 name: click

Accessible Value information:
    Current Value:  0
    Maximum Value:  1
    Minimum Value:  0

The accessibility docs tell me that I should be to tell the object to perform that action, but I don't know how to do this.

Preferably with AutoHotkey, how can this be done?

javaautomationaccessibilityautohotkey


Solution 1:[1]

One option is to set the CoordMode to Relative (Or Window which does the same thing), and then you can use Click or MouseMove with hard-coded pixels relative to the top left corner of the window. You can find the coordinates you need with the Window Spy.

Another option, if you're unsure the hard-coded pixels will be reliable, is you can use the FindText library which allows you to specify what the text/icon is within the window and automate clicks based off of a certain number of pixels nearby that. So for example, if there's always the text "Name:" and an input box to the right of it, you can build a search for "me: [" (left edge of the input box pixels) and then click to the right of that. The library is impressive and uses the built-in ImageSearch command in AutoHotkey.

The successor of this library is called AutomateMyTask, which I haven't tried yet, but it claims that you can build your whole click script within a gui that this tool provides. Check it out!

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Dean Householder

Related Questions