Pygame.MOUSEBUTTONDOWN coordinates are off unless i'm in xwindows

Question

If I load my pygame code from the console, it is reading the touchscreen coordinates wrong, but if I boot into xwindows, it reads them correctly.

I've gone in and calibrated the touch screen, and if I run evtest from the console, I am getting the correct coordinates back. It is only within python that it is returning the incorrect coordinates of the touch.

Is there something i'm missing to tell python how to calibrate?

import sys, pygame
pygame.init()

black = 0, 0, 0
size = width, height = 1280, 800
screen = pygame.display.set_mode(size)


while 1:
        for event in pygame.event.get():
                if event.type == pygame.QUIT: sys.exit()
                elif event.type == pygame.MOUSEBUTTONDOWN:
                        print(pygame.mouse.get_pos())
        screen.fill(black)

Answer 1

When not running under X11, PyGame will leverage SDL both for video (frame buffer) and mouse/touch input (input events). With a touchscreen (as opposed to a traditional mouse) SDL will usually report uncalibrated and (possibly?) un-rotated coordinates unless you make it use TSLIB.

Make sure you have all the needed bits and pieces:

apt-get install tslib libts-bin

Now tell SDL to use it by setting some environment variables before you start your Python script:

export SDL_FBDEV=/dev/fb1
export SDL_MOUSEDRV=TSLIB
export SDL_MOUSEDEV=/dev/input/touchscreen
./my_cool_program.py

Or you can do that "locally" in your Python script - this won't affect environment variables outside of the Python process:

import os
os.environ["SDL_FBDEV"] = "/dev/fb1"
os.environ["SDL_MOUSEDRV"] = "TSLIB"
os.environ["SDL_MOUSEDEV"] = "/dev/input/touchscreen"

This should take care of almost everything, including [touch]screen orientation. You should take care of checking the correctness of fbdev and input device paths for you system.

If you want to improve the touchscreen precision you can calibrate it:

TSLIB_FBDEVICE=/dev/fb1 TSLIB_TSDEVICE=/dev/input/touchscreen ts_calibrate

...and you can also test it:

TSLIB_FBDEVICE=/dev/fb1 TSLIB_TSDEVICE=/dev/input/touchscreen ts_test

Calibration data is saved to /etc/pointercal so you don't have to re-calibrate when you reboot the system.

Answer 2

I have a TSTP MTouch device and experienced the same problem but none of the solutions worked. I kept having accurate touch coordinates in X11, but the range of coordinates was much smaller if I used framebuffer outside X11. As it turned out, this was not a calibration issue but seems to be a pygame or SDL problem. I ended up directly polling the device and decoding the data in python.

My touchscreen showed up as follows in dmesg:

[ 6.008767] input: TSTP MTouch as /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0/usb1/1-1/1-1.4/1-1.4:1.0/0003:0416:C168.0001/input/input2 [ 6.009580] hid-multitouch 0003:0416:C168.0001: input,hidraw0: USB HID v1.11 Device [TSTP MTouch] on usb-0000:01:00.0-1.4/input0

I checked /dev/hidraw0 and found that it sends packets with 56 bytes of data for each touch interaction. I dumped the data using the command "xxd -c 56 /dev/hidraw0" and got the following structure:

Touch press:

000000e0: 0101 a903 5902 02ee 02b4 0200 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 000a 0000 0002

Touch release:

00000118: 0100 a903 5902 02ee 02b4 0200 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 0000 000a 0000 0002

As you can see, first byte is a header, second byte is a boolean, followed by two integers with the X/Y coordinates in little-endian, followed by a separator and then several X/Y multitouch points. Note that the touch coordinates were perfectly correct, there was no rotation or calibration needed.

I handle the data in python with the following function:

def handleTouchscreen():
    global lastCustomEvent
    previousClickedState = False
    device_file = "/dev/hidraw0"
    while True:
        try:
            with open(device_file, 'rb') as f:
                while True:
                    packet = f.read(packet_length)
                    if (touchDebug == True):
                        print("Received: %s" %(packet.hex()))
                    (tag, clicked, x, y) = struct.unpack_from('<c?HH', packet)
                    if (clicked == True and previousClickedState == False):
                        previousClickedState = True
                        if (touchDebug == True):
                            print("Pressed X=%d Y=%d" % (x, y))
                        lastCustomEvent = pygame.event.Event(pygame.JOYBUTTONDOWN, {'pos': (x, y), 'button': 1})
                    elif (clicked == False and previousClickedState == True):
                        previousClickedState = False
                        if (touchDebug == True):
                            print("Released X=%d Y=%d" % (x, y))
                        lastCustomEvent = pygame.event.Event(pygame.JOYBUTTONUP, {'pos': (x, y), 'button': 1})
                    time.sleep(0.01)

        except Exception as err:
            print(f'Error occured: {err}')

I used JOYBUTTONDOWN so I could handle them separate from the (still present) mouse events that had wrong coordinates.

In my actual pygame application I integrated it as follows:

event_list = pygame.event.get()
for event in event_list:
    if event.type == pygame.QUIT:
        run = False 
if (useCustomTouchscreenHandling == True and lastCustomEvent != False):
    event_list.append(lastCustomEvent)
    lastCustomEvent = False
mySpriteGroup.update(event_list)

This way I can enable/disable this as needed, and the incorrect mouse coordinates don't interfere with my program. I can now finally stop running this in X11 and save quite a lot of resources and processing power on my raspberry.