[gestures] Pinch In and Pinch Out are messy

[SAOMDVN]

• I tested it on latest raylib version from master branch
• I checked there is no similar issue already reported
• My code has no errors or misuse of raylib

Issue description

Input Gestures Example failed to recognise Pinch in and Pinch out, in some cases one appear instead of the expected other results, in other cases both are logged even when only one action was performed

Environment

Happened on HTML5, Safari, iPhone X, iOS 14.4
OpenGL ES 2.0

[mandar1jn]

I have the same issue on my IPhone XR

[raysan5]

Unfortunately, I don't have a proper touch screen for testing (my Surface Pro died some months ago) and using HTML5 through an Android phone is not the best environment...

In any case, current gestures processing system should be reviewed...

[tusharsingh09]

Works on my Razr but it's little fishy and janky, I guess that's because my screen but ot sure ah. But hard to get inputs I guess that's the prob

[raysan5]

Possibly related to this point: https://github.com/raysan5/raylib/blob/master/src/rcore.c#L5501

I think the touchEvent sets the Touch.pointCount to 1 every event, so, despite multiple touch-points could be in the screen, only one is processed by ProcessGestureEvent() and so pinch is no detected properly.

Despite not being the proper solution, I think removing the (i < gestureEvent.pointCount) condition from for (int i = 0; (i < gestureEvent.pointCount) && (i < MAX_TOUCH_POINTS); i++) could solve the issue. If anyone give it a try, please let me know!

[electronstudio]

OK just to confirm the line to change is now https://github.com/raysan5/raylib/blob/master/src/rcore.c#L5510 and I've changed it to for (int i = 0; i < MAX_TOUCH_POINTS; i++).

Testing on iPad...

Pinch-out is still almost always detected as pinch in. (Also there is something to with which finger goes down first as to whether it detects anything.)

Also swipes are always detected as swipe-left.

[electronstudio]

Upload.from.GitHub.for.iOS.MOV

Here is the same thing on a windows 11 laptop

[electronstudio]

Also weird: The first time you do a pinch-in, you get a pinch-in and a pinch-out.

[tusharsingh09]

You are getting something different from me, I rarely get touch. I tried it on my surface and works like a charm there but still no luck on Razr.
I guess the foldable screen creates an issue for WebGL and touch?

[raysan5]

@tusharsingh09 It probably works ok in Surface because that was the device I used for developing it and testing. Unfortunately my Surface Pro 3 just died some months ago.

[tusharsingh09]

Oh yeah Ray no issues on Surface but on Razr

[raysan5]

I don't have the required hardware/environment to solve this issue. Feel free to send me the required hardware. Closing for now.

[ghost]

After some debugging I believe that there are two issues causing the malfunction of the pinch gestures:

1. It's not properly saving the previous A and B positions to compare against the next ones;
2. The criteria to define if it was a Pinch In or Pinch Out.

I believe I managed to fix it by adding two Vector2 to the Touch struct and changing the criteria to define the Pinch In and Out. Will send a PR with the proposed changes.

Minimal reproduction code (modified the Input Gestures example examples/core/core_input_gestures.c) to test the issue more easily in a smartphone web browser:

// Modified the Input Gestures example (examples/core/core_input_gestures.c)
// to test the issue more easily in a smartphone web browser.

/*******************************************************************************************
*
*   raylib [core] example - Input Gestures Detection
*
*   Example originally created with raylib 1.4, last time updated with raylib 4.2
*
*   Example licensed under an unmodified zlib/libpng license, which is an OSI-certified,
*   BSD-like license that allows static linking with closed source software
*
*   Copyright (c) 2016-2023 Ramon Santamaria (@raysan5)
*
********************************************************************************************/

#include "raylib.h"

#if defined(PLATFORM_WEB)
    #include <emscripten/emscripten.h>
#endif

#define MAX_GESTURE_STRINGS 24

const int screenWidth = 400;
const int screenHeight = 500;

Vector2 touchPosition = { 0, 0 };
Rectangle touchArea = { 0, 0, screenWidth, screenHeight };
int gesturesCount = 0;
char gestureStrings[MAX_GESTURE_STRINGS][32];
int currentGesture = GESTURE_NONE;
int lastGesture = GESTURE_NONE;

void Update(void) {

    lastGesture = currentGesture;
    currentGesture = GetGestureDetected();
    touchPosition = GetTouchPosition(0);

    if (CheckCollisionPointRec(touchPosition, touchArea) && (currentGesture != GESTURE_NONE)) {
        if (currentGesture != lastGesture) {
            // Store gesture string
            switch (currentGesture) {
                case GESTURE_TAP: TextCopy(gestureStrings[gesturesCount], "TAP"); break;
                case GESTURE_DOUBLETAP: TextCopy(gestureStrings[gesturesCount], "DOUBLETAP"); break;
                case GESTURE_HOLD: TextCopy(gestureStrings[gesturesCount], "HOLD"); break;
                case GESTURE_DRAG: TextCopy(gestureStrings[gesturesCount], "DRAG"); break;
                case GESTURE_SWIPE_RIGHT: TextCopy(gestureStrings[gesturesCount], "SWIPE RIGHT"); break;
                case GESTURE_SWIPE_LEFT: TextCopy(gestureStrings[gesturesCount], "SWIPE LEFT"); break;
                case GESTURE_SWIPE_UP: TextCopy(gestureStrings[gesturesCount], "SWIPE UP"); break;
                case GESTURE_SWIPE_DOWN: TextCopy(gestureStrings[gesturesCount], "SWIPE DOWN"); break;
                case GESTURE_PINCH_IN: TextCopy(gestureStrings[gesturesCount], ">>> PINCH IN <<<"); break;
                case GESTURE_PINCH_OUT: TextCopy(gestureStrings[gesturesCount], "<<< PINCH OUT >>>"); break;
                default: break;
            }
            gesturesCount++;
            // Reset gestures strings
            if (gesturesCount >= MAX_GESTURE_STRINGS) {
                for (int i = 0; i < MAX_GESTURE_STRINGS; i++) { TextCopy(gestureStrings[i], "\0"); }
                gesturesCount = 0;
            }
        }
    }

    BeginDrawing();
    ClearBackground(RAYWHITE);
    DrawRectangleLinesEx({0, 0, screenWidth, screenHeight}, 2.0f, BLACK);
    for (int i = 0; i < gesturesCount; i++) {
        if (i < gesturesCount - 1) DrawText(gestureStrings[i], 10, 10 + 20*i, 20, BLUE);
        else DrawText(gestureStrings[i], 10, 10 + 20*i, 20, RED);
    }
    if (currentGesture != GESTURE_NONE) DrawCircleV(touchPosition, 30, RED);
    EndDrawing();
}

int main(void) {
    InitWindow(screenWidth, screenHeight, "Pinch Test");
    #if defined(PLATFORM_WEB)
        emscripten_set_main_loop(Update, 0, 1);
    #else
        SetTargetFPS(60);
        while (!WindowShouldClose()) { Update(); }
    #endif
    CloseWindow();
    return 0;
}

Screenshot of the test on Chrome (95.0.4638.74) on Android, after the proposed changes:

[raysan5]

@ubkp thank you very much for taking care of this issue!

[SAOMDVN]

@ubkp Thank you a lot!