Build broken by python error in aarch64/codec.py: No attribute 'split' on Tuple

[derekbruening]

Our internal build, once past the breakage from #6575, is still broken when trying to pull in the latest Github sources, on this error:

File "core/ir/aarch64/codec.py", line 73, in opnd_stem: No attribute 'split' on Tuple[List[str], List[str]] [attribute-error]
  In Union[Any, Tuple[List[str], List[str]], str]
Called from (traceback):
  line 782, in current file
  line 758, in main
  line 393, in generate_encoder

For more details, see [http://go/pytype-errors#attribute-error](https://www.google.com/url?q=http://go/pytype-errors%23attribute-error&sa=D)

[derekbruening]

I've locked the master branch for now to prevent further issues and reduce the amount of churn until the build issues are resolved

[derekbruening]

The culprit is PR #6535 960d1d7

[joshua-warburton]

I've had a quick look into this. I would expect this to fail if presented with a tuple, but it seems that the line it's failing on expects a string and would fail if it got a tuple even before the change. I've followed the code through and don't see anything in the patch itself that would cause this to fail as by this point we've re-written the opndsets with the function opndset_naming to remove the lists.

I can try to reproduce this if you have the python version you're using and which ref you're trying to build

[joshua-warburton]

Detail:
The build is failing because we attempt to run .split() on a tuple, pattern.opndset via the opnd_stem() function. However, if we remove that function we're attempting to encode a tuple as a string, which will fail still with:

>>> t = "%s"
>>> a = ("a", "b", "c")
>>> t %a
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not all arguments converted during string formatting

Tracing back, pattern.opndset is initially populated by either the highlighted arguments below

patterns.append(Pattern(pattern, opcode_bits, opnd_bits, high_soft_bits, opcode, ***(dsts, srcs)***, enum, feat))

patterns.append(Pattern(pattern, opcode_bits, opnd_bits, high_soft_bits, opcode, ***opndset***, enum, feat))

This is later flattened in the function opndset_naming on line 742. It takes both the string and non-string variants and rewrites these as something like "gen_04205000_001f07ff" and guarantees it will be a string from there forward.

Clearly, something is getting past this check but I'm not sure how and the patch mentioned above hasn't changed any of this.

[derekbruening]

This is trying to update from 0a6c057 to 3772e01. I only looked at the git blame for the split to identify the supposed culprit: did not do a bisect or look further to see if that PR was just moving stuff around. Looks like this error is actually from a static analyzer on codec.py rather than an actual runtime error: it's pytype https://google.github.io/pytype/

[joshua-warburton]

That would make sense Codec.py is structured in a way that's hard for static analysers to deal with. There isn't an easy way to restructure the code to remove the variable reuse that's likely causing the static analyser to dislike this.

We could selectively type annotate the functions, but I'd be hesitant about that breaking compatibility with older pythons.

I'd recommend adding a disabling comment for the checker:

    return opnd_name.split(".")[0] # pytype: disable=attribute-error

[derekbruening]

We may try to disable this pytype check temporarily to unblock internal and github progress so getting codec.py to not trigger pytype errors can be addressed separately.

[derekbruening]

I'd recommend adding a disabling comment for the checker:

    return opnd_name.split(".")[0] # pytype: disable=attribute-error

Thanks, that does seem to work.

[joshua-warburton]

I've spent a bit of time refactoring the code to remove the variable reuse and clean up some of the dead code:
#6595