[Discover] Improve doc viewer table performance

[kertal]

Follow up of #97649

When doc viewer table has to deal with lots of fields this leads to UI performance issues. Opening a doc in this view is no longer snappy. We initially intended to improve this by migrating to EuiInMemoryTable in #102149. But using the build in pagination leads to jumping UI issues when used in classic table (Because e.g. the page 2 can have a different height than page 1, and it's displayed inline in the doc table). It works better in the new table, which is using a flyout to display this view. So there are 2 ways to deal with this

1.) Wait for classic table to be deprecated, and use Pagination of EuiInMemoryTable
2.) Render e.g. 100 fields, display a "Show more fields" button at the end of the table, render more when the user clicks the button

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[timroes]

Talked with @kertal offline about this. We believe we should try to add a couple of improvements to this to make the flyout better for documents with many fields:

• Paginate the table, so we never show too many fields at the same time and block the browser by rendering a high amount of fields.
• Add a search bar to the top, that allows quickly filtering down by field name (frontend side).
• Potentially add a "Pin" button so you can temporarily (unsafed) pin specific fields, that will now always show at the top of the table while navigating through the documents. Ideally that temporary state is kept as long as the index pattern isn't changed (but kept when you close an reopen the flyout). Extra bonus points for storing the latest pinned fields per index pattern in localStorage, so we can reuse them even after switching index patterns back and forth.

The component is at the moment the same in Document Explorer and in the classical table. We only want those changes for the Document Explorer (due to the rendering in the flyout). Thus we should try to make a differentiation within that component using the discover:legacyTable uiSetting. If we feel that keeping both implementations together with that flag will be too troublesome we should consider cloning the implementation for now.

[kertal]

We only want those changes for the Document Explorer (due to the rendering in the flyout)

We might also have to consider that it should work
a) when rendered in the flyout
b) when rendered in the single doc view

For getting started, EUI doc page has some good examples (especially: In-memory table with search and external state):

https://elastic.github.io/eui/#/tabular-content/in-memory-tables