permanent crash loop (all bridged rooms down): error: 'Application service has not registered this user

[LordGrimmauld]

appserviceSilly.log
botconfig.txt
Matrix synapse setup: https://git.grimmauld.de/Grimmauld/grimm-nix-server/src/commit/981abe57ec42059f16a92f72bb43cefeef17ec5e/modules/matrix.nix
Appservice setup: https://git.grimmauld.de/Grimmauld/grimm-nix-server/src/commit/981abe57ec42059f16a92f72bb43cefeef17ec5e/modules/discord-matrix-bridge.nix

I am experiencing a crashloop of the appservice. Any time it tries to start the service, it soon after crashes. Attached is the log it generates, with the log level set to silly. It does however stll not produce helpful logs, at least as far as i understand. I am running matrix-appservice-discord version 4.0.0, synapse version 1.99.0.

As far as i can reconstruct what happened, the crash loop started happening when a greeter bot sent a welcome message to a newly joined member on discord and that welcome message wasn't correctly bridged. My current hypothesis: Not all bots on discord are notified of a new user join at the same time, resulting in a race condition between the two bots where a discord user is being mentioned by the welcome bot but the bridge can't resolve that mention yet as the bridge bot didn't yet receive the member join.

Problem: Even unbridging the room did not fix the crashloop. The crash loop happened in the bast too, but was resolved by the user writing a message. However, this most recent crash, the user did not write a message and instead left again, leaving my bot in permanent unfixable limbo.

Trying to debug this crash, it seems the crash happenes somewhere in matrix API, not even the code of this repo. However, this gets messy to a point where i am unqualified to dig further. If any additional info is required, i am happy to provide that. I just hope to get this resolved, i can also apply git patch sets to my installed appservice as i run essentially a source-derived version by nature of nixos, so if there is a fix that you are uncertain would fix it, i can try and report results.

[LordGrimmauld]

As to temporary fix: Modern synapse API complains about usernames starting with an underscore. What i did to unbrick the bot was INSERT INTO USERS VALUES ('@_discord_1194101579394060499:grimmauld.de', null, 1706267158, 0, null, 0, 'discord-bridge', null, null, null, 0, false, null, true, false); on the actual synapse backend database in postgres, but that can not be the encouraged solution to go fumble with the raw data base in a production environment.

[Doomsdayrs]

How can we change the username that the service generates?

I do not see it in the config

[Doomsdayrs]

I made a handy script to make this an easier task for me, adjust for your usecase.

#!/bin/bash

if [ -n "${1-}" ]; then
        sudo podman exec -it shosetsu-synapse-postgres psql -U synapse -c "INSERT INTO USERS VALUES ('$1', null, 1706267158, 0, null, 0, 'discord-bridge', null, null, null, 0, false, null, true, false);"
else
        echo "Please provide the username to insert"
fi

[LordGrimmauld]

ohj i already automated it, journalctl -b -u matrix-appservice-discord.service -o cat | grep -i "register" | grep -oEi '[0-9]{18}[0-9]*' | sort | uniq | awk '{print "INSERT INTO USERS VALUES (\"@_discord_"$1":grimmauld.de\", null, 1706267158, 0, null, 0, \"discord-bridge\", null, null, null, 0, false, null, true, false);"}' | sed "s/\"/'/g" | tee /tmp/fixdb.txt && sudo -u postgres psql -d synapse -f /tmp/fixdb.txt && sudo systemctl restart matrix-synapse.target if you are interested. Essentially lookign through log for all users it crashes on, then adding those and restarting the bridge.