Can't install any mod in BCML

[imsl0wer]

Running on Arch Linux with python3.8 Log:

Looks like an older mod, let's upgrade it...
Converting meta file...
Identified mod: Linkle Title Screen Mod
VERBOSE{
  "disable": [],
  "options": {}
}
Scanning for modified files...
Finding modified files...
Found 1 modified files...
Found 8 modified files in SARCs...
VERBOSEMerger packs, #1 of 18
Finding modified SARCs...
VERBOSEMerger aamp, #2 of 18
Detecting general changes to AAMP files...
VERBOSEMerger drops, #3 of 18
VERBOSEMerger shop, #4 of 18
Detecting changes to shop files...
VERBOSEMerger aslist, #5 of 18
Detecting general changes to AS lists...
VERBOSEMerger texts, #6 of 18
Checking for modified languages...
VERBOSEMerger actors, #7 of 18
VERBOSEActorInfoMerger.generate_diff took 9.637e-06 seconds
VERBOSEMerger dungeonstatic, #8 of 18
VERBOSEMerger maps, #9 of 18
VERBOSEMerger mainstatic, #10 of 18
VERBOSEMerger gamedata, #11 of 18
VERBOSEMerger savedata, #12 of 18
VERBOSEMerger eventinfo, #13 of 18
VERBOSEMerger areadata, #14 of 18
VERBOSEMerger effects, #15 of 18
VERBOSEMerger residents, #16 of 18
VERBOSEMerger quests, #17 of 18
VERBOSEMerger rstb, #18 of 18
VERBOSE[ PosixPath('/tmp/tmpq6ojn_ma/BreathOfTheWild_!LinkleTitle/content/Pack/Title.pack'),
  'content/Pack/Title.pack//Layout/Title.sblarc',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBG_01^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBG_02^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBGMainA_01^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBGLink_00^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBGMainB_01^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/TitleBG_04^q.bflim',
  'content/Pack/Title.pack//Layout/Title.sblarc//timg/Logo_W_00^l.bflim']
Assigned mod priority of 100
Moving mod to /home/skylabhalo/.config/bcml/mods/0100_LinkleTitleScreenMod...
VERBOSEInstalled 1 mods
Installed 1 mods
Cleansing old merges...
Refreshing merged mods...
Loading modded SARC list...
Merging 1 SARC files...
Illegal instruction (core dumped)```

[NiceneNerd]

Does this happen with any mod?

[imsl0wer]

Yes :<

[NiceneNerd]

Hmm, very bizarre. I've never seen this before, so am not sure what could be different in your case, unless maybe something different about your system? (I also use Arch, though, so not just that.) What are your specs?

[imsl0wer]

It only seems to crash when it's trying to merge sarc files. I was able to install linkle's armor icons without a problem. My specs are: AMD FX8350 4 Ghz, Radeon 5700xt, 24 GB DDR3 RAM
also I tried removing every python installation and just installing python3.8 but that didn't work
I'm using the EU Version of the Game not sure if that could cause these issues

[v-fox]

Got same with my system. In dmesg it says: [26001.794600] traps: bcml[15462] trap invalid opcode ip:7f2acb932a0a sp:7f28f69f32e8 error:0 in bcml.cpython-39-x86_64-linux-gnu.so[7f2acb783000+229000].

I have Intel Xeon CPU E5-2697 v2 with these hardware extensions: "fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts md_clear flush_l1d".

I suspect that whatever binary files are distributed via pip are compiled with bad optimizations that are CPU-specific for the submitter's machine (non-common hardware extension is enabled). All distributable code must be compiled with generic optimizations, like -march=x86-64 -mtune=generic, maybe adding some "safe" extensions for post-2012 CPUs, like -msse4 -msse4.1 -msse4.2 -mavx -maes -mpclmul but no more than that, and certainly no "-march=native" which is 100% undistributable.

If it statically links with something else, the error may propagate from there, for example: it seems to want byml-v2 which wants syaz0 which wants its own static copy of outdared zlib-ng, maybe even patched one. This is already sketch and original outdated zlib-ng enables some extensions willy-nilly. Patched one just explicitly enables AVX2 which would definitively make both of our CPUs throw that error.

[GingerAvalanche]

The submitter's machine has nothing to do with any compiler optimizations, as the submitter's machine is not involved in publishing to PyPI. Linux publishing is handled via a Github workflow, the instructions of which are located here: https://github.com/NiceneNerd/BCML/blob/master/.github/workflows/publish.yml#L8 You can verify for yourself that there are no CPU-specific optimization flags explicitly set during those build commands - any such flags will have been set by the referenced action. I've been unable to determine what flags the action sets, if any.

byml-v2 is unused in BCML except in the case of the functions to upgrade v1 bnp files to v2. Those functions were run successfully, according to the console output from the first post. This rules byml-v2 and its dependencies out, as sources of this issue.

[imsl0wer]

I'm still not able to get it working which frustrates me. I think it has something to do with merging SARC Files

VERBOSE[]
VERBOSE{
  "mods": [
    "/home/skylabhalo/Downloads/linklemod_67229.bnp"
  ],
  "options": {
    "disable": [],
    "options": {}
  },
  "selects": {
    "/home/skylabhalo/Downloads/linklemod_67229.bnp": [
      "Cucco Gilder",
      "Royal Guard",
      "Ghirahim Sword"
    ]
  }
}
Opening mod...
Identified mod: Linkle Mod
Loading mod logs...
Assigned mod priority of 100
Moving mod to /home/skylabhalo/.config/bcml/mods/0100_LinkleMod...
VERBOSEInstalled 1 mods
Installed 1 mods
Cleansing old merges...
Refreshing merged mods...
Loading modded SARC list...
Merging 16 SARC files...
Illegal instruction (core dumped)

[eku]

Do we have a container build environment like other projects do? I don't want to pollute my production system with the build environment installation. I willing to debug the crash.

[NiceneNerd]

Alas, I don't.

[AftDawn]

Has anyone found and PR'ed a fix for this?

[GingerAvalanche]

Nope. Based on the logged lines, the illegal instruction is something in Rust code, but that makes it very hard to debug without access to the affected machine(s).

All operations performed by the Rust SARC merger are fairly simple, generic operations, so whatever is going wrong must be very low-level - low-level enough that I don't think it's something BCML is directly responsible for.

Someone would need to debug this directly, on a box that has the error.

[GingerAvalanche]

Deleted comments that were unrelated to this issue.