Unicode is not displayed correctly in some places

[pastalian]

LRR Version and OS
dev-4172e92, debian container

Bug Details
There are some places where unicode is not displayed correctly.

• Logs
• Filename Parsing
  • When generating tags containing multibyte characters with this plugin in Edit Mode (only first time, save & reload will fix it)
  • Some characters like ★ (U+2605) and 々 (U+3005) will completely break the tag (reload will not fix it)

Matching Logs

[Minion] [info] Processing uploaded file /tmp/6LUeaFARLj/[�������] �������.tar...
[Archive] [debug] File Name: [�������] �������
[Archive] [debug] Filesystem Path: /LANraragi/content/[�������] �������.tar
[regexparse] [info] Sending the following tags to LRR: artist:�������
[regexparse] [info] Parsed title is �������

Screenshots

[pastalian]

I guess using redis_decode() will fix above 2 bugs?
Apparently file paths are intentionally not encoded, but I think the log output should be encoded (there's no point in an unreadable log).

[Difegue]

Yup, moving regexparse around from internal code to the plugin system adds an extra layer since we're accessing the file name from $lrr_info now. Forgot about that and added a decode there, should work fine in all cases.

The logs being garbled depend on where the newly uploaded file is coming from.
Files coming from Shinobu/file watching will show correctly, but files coming from Minion/web uploads won't. (Probably because of the necessary utf8::downgrade workaround)

redis_decode is basically the magic function for everything unicode now, but I don't like overusing it either since logging fully-decoded text can spew out wide character in say warnings from Perl, and it does overall muddy up the logic since what you're displaying isn't exactly what's being used in code...

I've added decodes to the Minion side of logging for now, so even if it garbles up a bit further down the chain users can still figure it out. Not sure why it does garble at all, since everything else seems to be working fine. 🤔

[pastalian]

Thanks, but there is one part that is forgotten 😢

LANraragi/lib/LANraragi/Utils/Database.pm

Lines 33 to 34 in 78fccb0

	$logger->debug("File Name: $name");
	$logger->debug("Filesystem Path: $file");

EDIT: Oh, I understand what you're saying now.