cpuinfo returns 0 for cpuinfo_get_l2_caches_count on some devices

[ScottTodd]

Discord discussion: https://discord.com/channels/689900678990135345/689906000043573354/804488403936215090

If cpuinfo_get_l2_caches_count() returns 0 on some machines, so our code that tries to pick the number of workers based on the l2 cache count bails to a single threaded fallback:

https://github.com/google/iree/blob/50d9823218605f1707abb375419df47c5d2ef28c/iree/task/topology.c#L356-L378

[benvanik]

Already checked: https://github.com/google/iree/blob/79684f2a96b749e2e5bc049b9185de9e77b47020/iree/task/topology.c#L347-L350

[benvanik]

Leaving this here for now as we are working around it by bailing to a fallback of 1 thread when this happens. If someone who has a machine that returns this they can debug on/file an upstream cpuinfo bug/etc that'd be helpful. Otherwise you get 1 thread :)

[bjacob]

Notes:

• It is common even for real CPUs (not just emulators) on the ARM architecture to have no L2 cache, either because they skip from L1 to L3 (typical on some phone-class cpus) or because they have nothing at all beyond L1 (for some microcontrolled CPUs).
• Of course, when running on emulator, it's even more unsurprising that there would be 0 L2 cache.

TLDR: this seems normal at both the qemu and cpuinfo level, and will happen not only on qemu.

[benvanik]

This is super useful information @bjacob - thanks! The current strategy (heh "strategy" is a stretch) is just a placeholder anyway - would love to sync more about reasonable defaults/specializations/etc and I was mostly just waiting until mahesh's queue is flushed with the linalg on tensors work so we have some parallel workloads and the CPU threading lands in the HAL rewrite so we'd have something concrete to talk about.

[bjacob]

no problem! whenever you want to get back to this, you could take a look at this code that I wrote with much help from Marat for ruy's needs --- It's not exactly the same as you're doing, but it does wrestle with a similar issue of using cpuinfo distinguishing shared vs non-shared ("local") caches:
https://github.com/google/ruy/blob/2887692065c38ef6617f423feafc6b69dd0a0681/ruy/cpuinfo.cc#L42-L83
the key part is this condition, which Marat wrote down for me:
https://github.com/google/ruy/blob/2887692065c38ef6617f423feafc6b69dd0a0681/ruy/cpuinfo.cc#L59-L63
With this logic, we detect which cache is effectively the last level of cache that is local to each processor, which would indeed be the L2 cache on a majority of current CPUs but would be the L1 cache when there is no L2 --- i.e. avoiding making any assumptions about level N cache being special for any particular value of N.

[benvanik]

That's fantastic code and effectively what I was reaching for when I originally wrote this and then punted on. I'll see if I can adapt that. Did you find any way to test that besides actually grabbing a device with certain characteristics? (it'd be cool if cpuinfo could mock out certain devices for testing - like "pretend you are X" - maybe it can?)

[bjacob]

Also note about ARM architecture CPUs: in addition to the above-mentioned case where there is no L2 cache, there are other CPUs with only L1 and L2, with the L2 cache being shared across cores!

Maybe ask Marat directly about cpuinfo mocking - I don't know personally.

[bjacob]

Ah, cpuinfo does support mocking: in its public include/ directory, right besides cpuinfo.h, you got cpuinfo-mock.h:
https://github.com/pytorch/cpuinfo/blob/master/include/cpuinfo-mock.h