Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing xpadneo and using Xbox One Controller over Bluetooth flips X+Y buttons and Start+Select buttons stop functioning until uninstalled. Over USB is fine. #303

Open
8 of 24 tasks
ipaqmaster opened this issue Jul 17, 2021 · 24 comments · Fixed by #320

Comments

@ipaqmaster
Copy link

ipaqmaster commented Jul 17, 2021

Important note

This is a continuation of my documented experience in this issue: ValveSoftware/steam-for-linux#7531

Continuing template..

Version of xpadneo

Experienced in:

  1. Installing directly from commit 45fef91
  2. Installing from AUR entry: https://aur.archlinux.org/packages/xpadneo-dkms-git/ which currently provides version 0.9.r83.g45fef91 (Appears to be same as above)

Controller Model

  • Xbox One S controller
  • Xbox Elite 2 controller
  • Xbox Series X|S controller
  • Other:

Connection mode

  • Bluetooth connection
  • Xbox Dongle connection (not yet supported)
  • USB cable (not yet supported) [Issue does not happen when I use USB]

Installed Software

  • Steam Input (enabled by default via Steam Desktop client)
  • Steam Link (usually via Raspberry Pi or other micro computers)
  • xow (alternative driver using the Xbox dongle)
  • Anti-Micro (may affect button mappings)
  • netstick (shares input devices via network similar to Steam Link)
  • xboxdrv (user-space gamepad driver)

Severity / Impact

  • I've read the docs and the bug reporting instructions
  • I've applied the latest firmware update to the controller
  • It does not work at all
  • It used to work in a previous version
  • It mostly works but sometimes it doesn't
  • I found a work-around
  • I probably didn't figure it all out but it's too early to give up
  • I don't know how to ...
  • It's too complicated
  • Fantastic work but ...
  • I can code and I want to help

Describe the Bug

As title. Installing xpadneo either directly from here or via an available AUR repo package for Archlinux and using my Xbox One Controller over Bluetooth flips the X+Y buttons in all titles. It seems my Start+Select buttons are also ignored. USB is fine but xpadneo claims it isn't supported yet, so using USB is likely just not using xpadneo.

This is only something I wanted to report as this all used to work just fine. Utilities like jstest /dev/input/js0 clearly show the buttons responding so I feel like it's a Steam or xpadneo thing.

I'm not sure if this is an issue of my system configuration, xpadneo, or something like Steam's controller recognition when xpadneo is active. Yet.

I've also noticed Steam will vibrate my controller indefinitely sometimes. I worked around this by disabling rumble for my controller under Big Picture mode in Steam, but ... upon installing xpadneo the infinite rumble bug I'm also experiencing has returned. Further making me wonder if Steam thinks it's a "different" controller.. or something.

Steps to Reproduce

  1. git clone https://github.com/atar-axis/xpadneo
  2. cd xpadneo
  3. sudo ./install.sh
    • Boot controller, connect over bluetooth
    • Open a game with Xbox controller support
    • Experience X+Y flippage and unresponsive Start+Select buttons.

Closing the game, uninstalling xpadneo and reconnecting the controller only to open game of choice again.. and the issue is gone.

Expected Behavior

  1. X and Y buttons not flipped.
  2. Start and Select functional to pause games and access other menus, etc.

(Not confident the infinite rumble bug while Steam running is xpadneo's problem)

Screenshots / GIFs / Videos

System Information

# uname -a
> Linux hostnamegoeshere 5.12.14-arch1-1 #1 SMP PREEMPT Thu, 01 Jul 2021 07:26:06 +0000 x86_64 GNU/Linux
# xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:*/report_descriptor | tee >(cksum)

# $ xxd -c20 -g1 /sys/module/hid_xpadneo/drivers/hid:xpadneo/0005:045E:*/report_descriptor | tee >(cksum)
00000000: 05 01 09 05 a1 01 85 01 09 01 a1 00 09 30 09 31 15 00 27 ff  .............0.1..'.
00000014: ff 00 00 95 02 75 10 81 02 c0 09 01 a1 00 09 33 09 34 15 00  .....u.........3.4..
00000028: 27 ff ff 00 00 95 02 75 10 81 02 c0 05 01 09 32 15 00 26 ff  '......u.......2..&.
0000003c: 03 95 01 75 0a 81 02 15 00 25 00 75 06 95 01 81 03 05 01 09  ...u.....%.u........
00000050: 35 15 00 26 ff 03 95 01 75 0a 81 02 15 00 25 00 75 06 95 01  5..&....u.....%.u...
00000064: 81 03 05 01 09 39 15 01 25 08 35 00 46 3b 01 66 14 00 75 04  .....9..%.5.F;.f..u.
00000078: 95 01 81 42 75 04 95 01 15 00 25 00 35 00 45 00 65 00 81 03  ...Bu.....%.5.E.e...
0000008c: 05 09 19 01 29 0c 15 00 25 01 75 01 95 0c 81 02 15 00 25 00  ....)...%.u.......%.
000000a0: 75 01 95 04 81 03 05 0c 0a 24 02 15 00 25 01 95 01 75 01 81  u........$...%...u..
000000b4: 02 15 00 25 00 75 07 95 01 81 03 05 0c 09 01 85 02 a1 01 05  ...%.u..............
000000c8: 0c 0a 23 02 15 00 25 01 95 01 75 01 81 02 15 00 25 00 75 07  ..#...%...u.....%.u.
000000dc: 95 01 81 03 c0 05 0f 09 21 85 03 a1 02 09 97 15 00 25 01 75  ........!........%.u
000000f0: 04 95 01 91 02 15 00 25 00 75 04 95 01 91 03 09 70 15 00 25  .......%.u......p..%
00000104: 64 75 08 95 04 91 02 09 50 66 01 10 55 0e 15 00 26 ff 00 75  du......Pf..U...&..u
00000118: 08 95 01 91 02 09 a7 15 00 26 ff 00 75 08 95 01 91 02 65 00  .........&..u.....e.
0000012c: 55 00 09 7c 15 00 26 ff 00 75 08 95 01 91 02 c0 05 06 09 20  U..|..&..u......... 
00000140: 85 04 15 00 26 ff 00 75 08 95 01 81 02 c0                    ....&..u......
3560615638 1558


Controller and Bluetooth Information

xpadneo-dmesg.txt
xpadneo-btmon.txt
xpadneo-lsusb.txt

Additional Context

Toggling Steam Beta Participation and restarting Steam changes nothing.

Unticking Xbox Controller Support in Big Picture just renders my controller unusable in games.

@kakra
Copy link
Collaborator

kakra commented Jul 17, 2021

That your SDL string starting with 0300 indicates that your controller is used via hidraw in SDL. This is currently known broken.

To confirm this problem, try the following steps:

  1. Connect the controller and watch dmesg
  2. Get the name/number of the hidraw device from the log, it will look like: xpadneo 0005:045E:0B13.0019: input,hidraw15: BLUETOOTH HID v9.03 Gamepad ... (hidraw15 in my case)
  3. Check the permissions of the file /dev/hidraw15 (from my example) and post them
  4. Revoke the permissions: sudo chmod a-rwx /dev/hidraw15 and try your games now, ensure the controller stays connected otherwise the device file will be recreated, usually with a new name

If this is fixes your problem, our work-around doesn't seem to apply. It used to work in previous versions then because SDL2 wasn't using the hidraw device back then.

Cross references:

@kakra
Copy link
Collaborator

kakra commented Jul 17, 2021

Unticking Xbox Controller Support in Big Picture just renders my controller unusable in games.

It works just fine for me. But well, the main problem may be the hidraw support in the first place. How does it render the controller unusable?

@ipaqmaster
Copy link
Author

ipaqmaster commented Jul 17, 2021

Thanks for getting back to me on this.

For clarification, when unticking Xbox Configuration Support in Big Picture it seems to be hit and miss for which titles will work. For example, Mortal Kombat 11 running through GloriousEggroll's Proton 6.1 works with it unticked in Big Picture but Linux-native game "One Step From Eden" can see the controller present but does not respond to controller input anymore.

As for your instructions above for the hidraw device:

[  +0.000095] microsoft 0005:045E:02FD.0026: input,hidraw11: BLUETOOTH HID v9.03 Gamepad [Xbox Wireless Controller] on 84:c5:a6:5a:5f:81
[  +0.000061] xpadneo 0005:045E:02FD.0026: input,hidraw11: BLUETOOTH HID v9.03 Gamepad [Xbox Wireless Controller] on 84:c5:a6:5a:5f:81
me@desktop ~ $ ls -lah /dev/hidraw11 
crw-rw----+ 1 root root 241, 11 Jul 17 20:44 /dev/hidraw11
me@desktop ~ $ sudo chmod a-rwx /dev/hidraw11
me@desktop ~ $ ls -lah /dev/hidraw11 
c---------+ 1 root root 241, 11 Jul 17 20:44 /dev/hidraw11

Trying OSFE again after revoking hidraw11's permissions with chmod the game accepted no controller input. I unticked Xbox Configuration Support in Big Picture again and still no change on relaunch.

MK11 was happy to use the controller after revoking the permissions of /dev/hidraw11 and also even after unticking Xbox Configuration Support in Big Picture and trying again.

It seems to be hit and miss? Though I wonder if Proton helps MK11 being a Windows-only title.

@kakra
Copy link
Collaborator

kakra commented Jul 17, 2021

There are a few titles which actually need Steam Input to properly work with all controllers. Usually, such titles come with an override option set by default in the game properties. I think Horizon Zero Dawn is such a title.

@c-x-berger
Copy link

I'm seeing something similar. Immediately after installing xpadneo, Steam's "Big Picture" mode has some interesting swaps for me:

  • Pressing Y registers as pressing X
  • Pressing LB registers as pressing Y
  • Start and Select map to the nearest bumper (I genuinely don't know which is Start and which is Select, sorry)
  • I have no idea where start and select have been moved to, but it's probably just as cursed.
  • A and B behave correctly.

If "Xbox Controller Support" is enabled in Big Picture mode, two things happen:

  • The mismatched buttons carry over into whatever game I start. Probably because something something Steam Input something.
  • Shortly after starting the game I've used for testing (Hyper Light Drifter), one of the rumble motors will begin going at max power and doesn't stop until Steam itself is closed. So far it's usually been the force feedback on the right trigger, but it has happened on the left trigger as well. (Haven't yet seen it on the "main" rumble motors.)

None of this happens when connected over USB.

If I remove the module with sudo dkms uninstall -m hid-xpadneo -v v0.9-85-g67585b3 (because uninstall.sh seems to be broken), buttons seem to map properly.

@kakra
Copy link
Collaborator

kakra commented Aug 31, 2021

@c-x-berger This is probably because SDL uses the hidraw device directly. Our HID report format is a bit different than what SDL expects (because xpadneo uses the same bitmap format across all controller models and SDL doesn't look at the report size to decide for a bitmap format but at the VID/PID instead, and then it even looks at the wrong one which we cannot work around). This also explains why the rumble goes crazy: xpadneo has a protection against a race condition in the controller firmware but we cannot apply that in the hidraw layer. If rumble data is sent too fast, the motors will stop responding to commands and spin for minutes before they eventually stop. The controller firmware may even reboot or not recover from that at all until you pull the batteries.

I quick fix would be to look at dmesg after connecting the controller to find the number of the hidraw device, then chmod a-rwx /dev/hidrawNUMBER, then see if the problem persists. If the controller is disconnected, the chmod change is lost and needs to be re-applied (the hidraw number usually changes at that point).

I'm currently planning on re-implementing the whole hidraw interface to be compatible with SDLs assumptions but we would probably still not be able to protect against the rumble race. But as far as I know that's fixed in newer SDL versions.

Buttons X and Y shift by one position because different Xbox controller models have different sparse bitmap formats, even within the same model depending on some strange firmware quirks. The original wired controller has bits A,B,X,Y,LB,... for the buttons, the wireless controllers have A,B,C,X,Y,Z,LB,... where C and Z are even not buttons on the gamepad. That's why the buttons move. The buttons that come after the initial bits may be even more messed up.

This has been documented here: #286

It's for historic reasons why xpadneo handles the device the way it does. With current SDL versions, this is probably no longer necessary. But I'm planning on adding wired and dongle support in HID mode and need to find a way to cover all cases.

Please report a uninstall bug separately - including the output you get.

@kakra
Copy link
Collaborator

kakra commented Sep 1, 2021

If "Xbox Controller Support" is enabled in Big Picture mode, two things happen:

One more idea: Update the controller to the latest firmware, SDL may apply wrong assumptions about the model if the firmware is too old (because MS messes with the HID report format between firmware versions, and SDL absolutely doesn't care about HID descriptors at all, it just assumes nothing ever changes in the format).

@kakra
Copy link
Collaborator

kakra commented Sep 1, 2021

  • I have no idea where start and select have been moved to, but it's probably just as cursed.

Probably to the thumb stick buttons and the guide button.

BTW: Select is left, start is right. As a memory hook: left goes back (to the selection menu), right goes forward (starting the game).

@tb936
Copy link

tb936 commented Sep 4, 2021

I am also experiencing this issue, have downloaded the latest firmware from the xbox app. The sticks don't map to start and select as I would expect.

Will use the hidraw device workaround until this is cleaned up

kakra added a commit to kakra/xpadneo that referenced this issue Nov 13, 2021
Also fixes Steam Input which apparently also uses SDL2.

The bug shows up when SDL identifies the controller as being on USB
instead of Bluetooth (SDL Gamepad ID starts with `03` instead of `05`)
which then messes with SDL's expectations for the contents of hidraw.

This fixes two SDL problems:

  * Wrong mappings in hidraw (buttons shifted to other buttons)
  * SDL2 doesn't properly throttle rumble commands (endless rumble)

See-also: atar-axis#286
Maybe-fixes: atar-axis#303
Maybe-fixes: atar-axis#311
Maybe-fixes: atar-axis#314
Signed-off-by: Kai Krakow <[email protected]>
kakra added a commit that referenced this issue Nov 13, 2021
Also fixes Steam Input which apparently also uses SDL2.

The bug shows up when SDL identifies the controller as being on USB
instead of Bluetooth (SDL Gamepad ID starts with `03` instead of `05`)
which then messes with SDL's expectations for the contents of hidraw.

This fixes two SDL problems:

  * Wrong mappings in hidraw (buttons shifted to other buttons)
  * SDL2 doesn't properly throttle rumble commands (endless rumble)

See-also: #286
Maybe-fixes: #303
Maybe-fixes: #311
Maybe-fixes: #314
Signed-off-by: Kai Krakow <[email protected]>
@kakra
Copy link
Collaborator

kakra commented Nov 15, 2021

This should actually be fixed, please confirm.

@kakra kakra reopened this Nov 15, 2021
kakra added a commit to kakra/xpadneo that referenced this issue Nov 17, 2021
Also fixes Steam Input which apparently also uses SDL2.

The bug shows up when SDL identifies the controller as being on USB
instead of Bluetooth (SDL Gamepad ID starts with `03` instead of `05`)
which then messes with SDL's expectations for the contents of hidraw.

This fixes two SDL problems:

  * Wrong mappings in hidraw (buttons shifted to other buttons)
  * SDL2 doesn't properly throttle rumble commands (endless rumble)

See-also: atar-axis#286
Maybe-fixes: atar-axis#303
Maybe-fixes: atar-axis#311
Maybe-fixes: atar-axis#314
Signed-off-by: Kai Krakow <[email protected]>
@Andrew-Fahmy
Copy link

Andrew-Fahmy commented Nov 17, 2021

I was having a similar issue with the buttons being mapped wrong in steam games. This was due to the hidraw device having the incorrect permissions which allowed SDL to read the device. This would fix my problem until the controller is disconnected:

I quick fix would be to look at dmesg after connecting the controller to find the number of the hidraw device, then chmod a-rwx /dev/hidrawNUMBER

I did some more digging and found out that there was a udev rule that was overriding the one that was installed by xpadneo. I had installed a rule into /etc/udev/rules.d/ which overrides any of the rules in /lib/udev/rules.d/. After removing that rule (or moving it to /lib/udev/rules.d/) my controller now works perfectly.

If anyone else if having issues, here is how I was able to find what rules where being loaded:

  1. Disconnect your controller
  2. Set udev log to debug and watch journal log
udevadm control --log-priority=debug
journalctl -f
  1. Reconnect controller

This should give you a little more info about what rules could be interfering.
Also If you do make changes to any rules make sure to reload the changes with udevadm control -R. Hope this helps :)

@kakra
Copy link
Collaborator

kakra commented Nov 18, 2021

If anyone else if having issues, here is how I was able to find what rules where being loaded:

@Andrew-Fahmy Can you remember which one actually has been the conflicting rule?

I think the following may happen:

Someone installs xpadneo from source, which installs rules to /etc prefix. Then, this installation is replaced by a distribution package without uninstalling the manual installation first: This will keep old /etc files but also install identically named files in /usr/lib prefix. Now, udev has a rule which allow rule file names to exist only once, and it prefers admin configuration in /etc over distribution configuration in /usr/lib (which makes sense). This ultimately puts you into the position to properly clean up a mess you installed as a local admin. ;-)

Maybe we'd need some xpadneo doctor which can check for this situation.

@Andrew-Fahmy
Copy link

Andrew-Fahmy commented Nov 19, 2021

In my case I manually installed the conflicting rule from qmk firmware as their qmk doctor command recommended. It has a rule that would set the mode of the all hidraw devices so that a tool called hid_listen could print out debug info coming from the keyboard with qmk firmware.

Do you think that I should open an issue about this with qmk?

Also I think an xpadneo doctor would be a nice addition.

@kakra
Copy link
Collaborator

kakra commented Nov 19, 2021

Actually, this line should not be there:

KERNEL=="hidraw*", MODE="0660", GROUP="plugdev", TAG+="uaccess", TAG+="udev-acl"

It's a security mess, don't make hidraw device just readable to every user in a group (plugdev) or every user logged into a local session (uaccess), and I'm not even sure what "udev-acl" is supposed to do. This rule is bad. It allows any local process in your session to spy on any hidraw device (even when unrelated to your session), which is almost every input device in the world, thus allowing to record keypresses, pin entries, or more generally: secrets. Don't do that.

Meanwhile, in Discord, we discovered that OpenRGB installs a similar problematic rule.

Access to hidraw devices MUST be guarded by conditionals for the specific devices which should get this ACL (uaccess).

@Andrew-Fahmy
Copy link

That is a really good point. I had not thought about it from the security perspective. I'll open an issue with qmk about this.

For now what do you think about adding some info about the udev rules to the troubleshooting page?

Also I want to thank you for all the work you have done on this project and responding to all my questions and concerns quickly and thoroughly!

@kakra
Copy link
Collaborator

kakra commented Nov 20, 2021

Actually, we can make it compatible even with exposed hidraw device. This conflict just uncovered questionable security issues in other software.

The change to re-enable xpadneo hidraw and make in compatible with SDL is planned for v0.10 or v0.11. Until then, xpadneo doesn't add uaccess rules to hidraw. But if other software does it unconditionally, it conflicts with our configuration - no matter whether it's a security problem. So it's bad already from two different views.

I'm pretty sure it's an oversight. It should be fixed or at least considered in some way, no matter if xpadneo managed to work around it or restored SDL-hidraw compatibility. OTOH, I don't like the hidraw approach of SDL very much, it re-implements logic that's already done at the kernel level, without proper checks if mapping fixups were already done (i.e., it doesn't look at the HID descriptor, it just assumes some format of the HID reports).

@kakra
Copy link
Collaborator

kakra commented Nov 20, 2021

For now what do you think about adding some info about the udev rules to the troubleshooting page?

I'll follow up on this with a PR when we fully understand the problem. I'm pretty sure we almost see the whole picture. I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing. It's a long standing bug, I don't know yet if it is the same problem in each issue because we have no documentation about what other software is installed, or which custom udev rules may exist. We are probably running races we cannot win, I'd need to fix our hidraw reports anyways (which makes controller mapping fixups much more complicated in xpadneo).

@Andrew-Fahmy
Copy link

It should be fixed or at least considered in some way, no matter if xpadneo managed to work around it or restored SDL-hidraw compatibility.

They have fixed it, they have another repo dedicated to generating the rules that I was not aware of

I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing.

For me removing those rules fixed all the problems

@kakra
Copy link
Collaborator

kakra commented Nov 20, 2021

This is exactly how OpenRGB should also fix their world-accessible hidraw rule.

@naynayll2
Copy link

For now what do you think about adding some info about the udev rules to the troubleshooting page?

I'll follow up on this with a PR when we fully understand the problem. I'm pretty sure we almost see the whole picture. I now need some confirmation that uninstalling those rules (aka the package that installed them) actually fixes what we are seeing. It's a long standing bug, I don't know yet if it is the same problem in each issue because we have no documentation about what other software is installed, or which custom udev rules may exist. We are probably running races we cannot win, I'd need to fix our hidraw reports anyways (which makes controller mapping fixups much more complicated in xpadneo).

I can confirm that uninstalling qmk rules fixed the problem for me as well.

@kakra
Copy link
Collaborator

kakra commented Mar 5, 2022

Deferring this to v0.11 as it is probably about hidraw interaction with OpenRGB and SDL. This can probably be solved by #286.

@Chr1sNo
Copy link

Chr1sNo commented May 14, 2022

This is exactly how OpenRGB should also fix their world-accessible hidraw rule.

Some feedback from upstream. The blanket hidraw rule was removed when udev rules were moved to compile time generation. This assumes that there are no xpadneo devices that would require an OpenRGB rule.

I would test to confirm this however I don't have a relevant XBone controller so please let me know if this is still an issue with OpenRGB.

kakra added a commit that referenced this issue May 26, 2022
Also fixes Steam Input which apparently also uses SDL2.

The bug shows up when SDL identifies the controller as being on USB
instead of Bluetooth (SDL Gamepad ID starts with `03` instead of `05`)
which then messes with SDL's expectations for the contents of hidraw.

This fixes two SDL problems:

  * Wrong mappings in hidraw (buttons shifted to other buttons)
  * SDL2 doesn't properly throttle rumble commands (endless rumble)

See-also: #286
Maybe-fixes: #303
Maybe-fixes: #311
Maybe-fixes: #314
Signed-off-by: Kai Krakow <[email protected]>
@carsoncall
Copy link

carsoncall commented May 31, 2022

In my case I manually installed the conflicting rule from qmk firmware as their qmk doctor command recommended. It has a rule that would set the mode of the all hidraw devices so that a tool called hid_listen could print out debug info coming from the keyboard with qmk firmware.

Came here to bump this, I have installed QMK relatively recently and the rule that Andrew-Fahmy mentioned was present in my /etc/udev/rules.d/50-qmk.rules though I do not recall manually installing it. I am led to believe that it is now a default in QMK, at least version 2022-02-26. Simply commenting it out fixed my issues.

It is worth mentioning that I am relatively inexperienced with these things and the documentation makes no mention of this, so I spent quite a bit of time messing with configuration and environmental variables before finally finding this bug report. If it is the right thing, I would be willing to open a separate bug report for docs or contribute the fix myself.

@kakra
Copy link
Collaborator

kakra commented Jun 2, 2022

Came here to bump this, I have installed QMK relatively recently and the rule that Andrew-Fahmy mentioned was present in my /etc/udev/rules.d/50-qmk.rules though I do not recall manually installing it. I am led to believe that it is now a default in QMK, at least version 2022-02-26. Simply commenting it out fixed my issues.

According to the commit discussions around that topic, this file should no longer be needed and there should be an identical named file in /usr/lib/udev/rules.d which does things properly. Maybe the needed commits didn't make it into a stable version already?

User settings for QMK should go to /etc/udev/rules.d/49-qmk.rules instead.

As far as I understand, the conflicting part of the rule is about some debugging console only, so if you're not a developer, you should not need it anyways, and commenting it out should not impact your usage of the software.

I suggest moving this part of the discussion to your distribution tracker or QMK itself, as I don't use that software and cannot support it. Alternatively, you may find users of QMK in our Discord channels. If you continue discussion somewhere else, feel free to leave a link here, so other users finding their way here can follow the link. I also accept PRs which add instructions to the docs. Thanks. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants