Raspberry Pi 4 accepts USB DATA packets with CRC error

[tmon-nordic]

Describe the bug

USB Device sent DATA0 packet containing 64 bytes ("real data"):

0000   d1 bd 0b b5 a9 6d 8e 94 b4 47 57 7e 64 10 90 42
0010   6a b5 0a 53 99 00 59 b1 bc 6f 07 b8 c4 6d e8 eb
0020   6c 43 43 2c 1b ce f2 c1 63 a2 0d 9e 23 3f 7b 62
0030   6c e0 74 b6 50 df 57 65 ce 91 0f 49 f5 86 f7 dd

with CRC 0xFEC1

Raspberry Pi apparently decoded the DATA0 packet as ("incorrectly received data"):

0000   d1 bd 0b b5 a9 6d 8e 94 b4 47 57 7e 64 10 90 42
0010   6a b5 0a 53 99 00 59 b1 bc 6f 07 b8 c4 6d e8 eb
0020   6c 43 43 2c 1b ce f2 c1 63 a2 0d fe 91 9f 3d 31
0030   36 70 3a 5b a8 ef ab 32 e7 c8 87 a4 7a c3 fb ee

User-space applications do not receive CRC information and thus application was unable to tell that the data was corrupted. The driver and/or Raspberry Pi 4 hardware should have rejected the packet.

Steps to reproduce the behaviour

1. Get a USB device with clock stability issues, ideally a Development Kit
2. Run some intensive tests, e.g. CDC ACM echo with xorshift pattern
3. Wait for data mismatch

Depending on the rate the USB device introduces glitches on the D+/D- lines the failure can happen within some seconds, minutes, hours or even days. My "best" device is able to reliably induce glitches (and thus CRC errors) in less than a minute.

Device (s)

Raspberry Pi 4 Mod. B

System

raspinfo.txt

Logs

To better describe the issue, following python code can be used to convert the data to bitstuffed binary stream and vice versa. The code is suitable for copy&pasting into python interactive window.

import textwrap

def byte_to_lsb_first_binary(byte):
    return bin(byte)[2:].zfill(8)[::-1]

def convert_to_bitstuffed_binary(data):
    binary = ''.join(map(byte_to_lsb_first_binary, data))
    bitstuffed = binary.replace('111111', '1111110')
    return bitstuffed

def convert_from_bitstuffed_binary(bitstuffed):
    binary = bitstuffed.replace('1111110', '111111')
    return bytes([int(''.join(reversed(bits)), 2) for bits in textwrap.wrap(binary, 8)]).hex()

The NRZI bitstream for real data is:

convert_to_bitstuffed_binary(bytes.fromhex("d1bd0bb5a96d8e94b447577e641090426ab50a53990059b1bc6f07b8c46de8eb6c43432c1bcef2c163a20d9e233f7b626ce074b650df5765ce910f49f586f7dd"))
'1000101110111101110100001010110110010101101101100111000100101001001011011110001011101010011111100001001100000100000001001010000100101011010101101010100001100101010011001000000001001101010001101001111011111011011100000000111010010001110110110000101111101011100110110110000101100001000110100110110000111001101001111100000111100011001000101101100000111100111000100111111000110111100100011000110110000001110010111001101101000010101111101111101010101001100111001110001001111100001001001010101111011000011110111110111011'

The NRZI bitstream for incorrectly received data is:

convert_to_bitstuffed_binary(bytes.fromhex("d1bd0bb5a96d8e94b447577e641090426ab50a53990059b1bc6f07b8c46de8eb6c43432c1bcef2c163a20dfe919f3d3136703a5ba8efab32e7c887a47ac3fbee"))
'10001011101111011101000010101101100101011011011001110001001010010010110111100010111010100111111000010011000001000000010010100001001010110101011010101000011001010100110010000000010011010100011010011110111110110111000000001110100100011101101100001011111010111001101101100001011000010001101001101100001110011010011111000001111000110010001011011000001111110110001001111110001101111001000110001101100000011100101110011011010000101011111011111010101010011001110011100010011111000010010010101011110110000111101111101110111'

The two bitstreams differ at column 352. Saleae capture (zipped because GitHub does not support attaching .sal files directly) saleae-IN-incorrectly-decoded-by-rpi4.sal.zip has marker put on where the Raspberry Pi 4 decoding disagrees with Saleae decoder.

The zoom in on the discrepancy is:

  Actual data:               00000111100111000
  Incorrectly decoded data:  00000111111011000

The NRZI bitstream for whole DATA0 packet with CRC is:

convert_to_bitstuffed_binary(bytes.fromhex("c3d1bd0bb5a96d8e94b447577e641090426ab50a53990059b1bc6f07b8c46de8eb6c43432c1bcef2c163a20d9e233f7b626ce074b650df5765ce910f49f586f7ddc1fe"))
'11000011100010111011110111010000101011011001010110110110011100010010100100101101111000101110101001111110000100110000010000000100101000010010101101010110101010000110010101001100100000000100110101000110100111101111101101110000000011101001000111011011000010111110101110011011011000010110000100011010011011000011100110100111110000011110001100100010110110000011110011100010011111100011011110010001100011011000000111001011100110110100001010111110111110101010100110011100111000100111110000100100101010111101100001111011111011101110000011011111101'

If the bitstream is altered to swap the actual data with incorrectly decoded data, the altered bitstream converts back to:

convert_from_bitstuffed_binary('11000011100010111011110111010000101011011001010110110110011100010010100100101101111000101110101001111110000100110000010000000100101000010010101101010110101010000110010101001100100000000100110101000110100111101111101101110000000011101001000111011011000010111110101110011011011000010110000100011010011011000011100110100111110000011110001100100010110110000011111101100010011111100011011110010001100011011000000111001011100110110100001010111110111110101010100110011100111000100111110000100100101010111101100001111011111011101110000011011111101')
'c3d1bd0bb5a96d8e94b447577e641090426ab50a53990059b1bc6f07b8c46de8eb6c43432c1bcef2c163a20dfe919f3d3136703a5ba8efab32e7c887a47ac3fbee607f'

C3 is DATA0 PID, and 60 7F are the CRC bytes (0x7F60) due to Little-Endian encoding. The altered bitstream converts back to usbmon captured data.

The CRC for the altered data would have been 9C 4D (0x4F9C), but the received was 60 7F. The receiver (here: Raspberry Pi 4) should have rejected the packet due to incorrect CRC.

pcaps-IN-incorrectly-decoded-by-rpi4.zip contains following pcap files:

• ov-IN-incorrectly-decoded-by-rpi4.pcap showing the transaction in question. OpenVizsla did agree on the decoding with Saleae. Both OpenVizsla and Saleae decoded the real data.
• usbmon-IN-incorrectly-decoded-by-rpi4.pcapng showing the transfer covering the transaction. The transfers during the capture session were purposely single-packet-only to make it easy to correlate bus-level captures with URB-level captures.
• reconstructed-packet-with-bad-crc.pcap shows the reconstructed transaction with the faulty data stream to illustrate that CRC error is expected in this case.

Additional context

Raspberry Pi 5 is working correctly with the same USB device, i.e. Raspberry Pi 5 is correctly rejecting (timing out handshake) after receiving glitched DATA packet. Other Raspberry Pi versions were not tested.

[JamesH65]

@P33M FYI.

[timg236]

@tmon-nordic Please can you confirm which USB host-controller you are using i.e.VL805, BCM-XHCI or DWC2 host-mode

[tmon-nordic]

@tmon-nordic Please can you confirm which USB host-controller you are using i.e.VL805, BCM-XHCI or DWC2 host-mode

I am plugging the device into the leftmost-bottom USB port (leftmost are the standard USB 2.0 ports, center is USB 3.0 and right is RJ45) on Raspberry Pi 4 Model B Rev 1.5.

dmesg log includes

[    2.218950] xhci_hcd 0000:01:00.0: xHCI Host Controller
[    2.218983] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
[    2.220818] xhci_hcd 0000:01:00.0: hcc params 0x002841eb hci version 0x100 quirks 0x0300240000000890
[    2.221743] xhci_hcd 0000:01:00.0: xHCI Host Controller
[    2.221762] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
[    2.221783] xhci_hcd 0000:01:00.0: Host supports USB 3.0 SuperSpeed

lsusb shows the device in question on bus 1. How do I tell which host controller it is?

$ cat /sys/bus/usb/devices/usb1/uevent 
MAJOR=189
MINOR=0
DEVNAME=bus/usb/001/001
DEVTYPE=usb_device
DRIVER=usb
OF_NAME=usb
OF_FULLNAME=/scb/pcie@7d500000/pci@0,0/usb@0,0
OF_COMPATIBLE_N=0
PRODUCT=1d6b/2/606
TYPE=9/0/1
BUSNUM=001
DEVNUM=001

$ lspci
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge (rev 20)
01:00.0 USB controller: VIA Technologies, Inc. VL805/806 xHCI USB 3.0 Controller (rev 01)

Therefore I believe it is VL805.