Severe performance regression in GLES3/GLES2 renderer related to octahedral compression

[RevoluPowered]

Godot version

3.x (6304709)

System information

MacOS Big Sur 11.4 (AMD Radeon Pro 5500M 4GB)

Issue description

I have bisected, the bug was introduced by #46800

Screen.Recording.2021-08-03.at.12.55.42.mov

Without that PR I am getting solid FPS figures.

Steps to reproduce

1. Run MacOS with Godot 3.X branch
2. Open ferris wheel mesh
3. Move camera (huge fps drop down to 3-4 FPS)
4. Revert PR [3.x] Implement Octahedral Map Normal/Tangent Attribute Compression #46800
5. Repeat steps

Minimal reproduction project

FerrisWheel.zip

[akien-mga]

CC @The-O-King

[RevoluPowered]

I spent like 2 days looking into the cause and trying to find a solution, so am here to debug the issues too and can provide access to the macbook if you need it.

Some more info:

• using clang may make this reproducible on other platforms.
• functions added for compression don't handle Vector3(0,0,0), data (causes nan's to reach renderer, but fixing it won't solve the perf drop)

1. ASAN/UBSAN errors which might be a related symptom:
   
2. GL Draw Elements execution time with this PR seriously spikes (Shader compilation happening, or is OpenGL dropping to CPU rendering? unknown)

3. OpenGL profiling commands

Full debug file
profileone.txt

[The-O-King]

Should be fixed fully with #51376

[clayjohn]

Closed by #51376