Add driver logs to Jobs page for submission jobs

[alanwguo]

Why are these changes needed?

• Add driver logs to Jobs page for submission jobs
• Adds a refresh button to the log viewer to reload the logs.
• Refactors the log viewer from the logs page into its own component
• Updates the look and feel of the jobs page to match the new IA style.
• Adds User-provided metadata to the job detail page. (fixes [Core|Dashboard] Support custom tags for jobs. #34187 )
• Updates the table icon
• Change "Tasks" to "Tasks/actor overview"
• Adds Node Count Card next to ray status cards

Job detail page

Other job pages (new look and feel)

Jobs user-provided metadata

Autoscaler section

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[rkooo567]

Is there any way to add a refresh button? Until we make it print last N lines & that's probably the best way to use this effectively.

[scottsun94]

1. The position of the "Expand" button is a bit weird. Can we put it always next to the text? (8px away)

2. Shall we say "User-provided metadata" since the metadata can be arbitrary data. "User metadata" feels like that they are related to the user.

3. It seems that the metadata is only available in jobs SDK. Is it something we should support in CLI?

4. for the title job id, the font size seems a bit too big. Can we change it to match this in the figma

[chaowanggg]

nit: do we need to add another else to catch the case

!driver_agent_http_address || (!driver_info || !ipLogMap[driver_info.node_ip_address])

[alanwguo]

No in that case, we do not show any logs so the default behavior of return undefined is what we want.

[chaowanggg]

Is this possible that driver_info.node_ip_address is undefined or driver_info.node_ip_address not exists in ipLogMap?

[alanwguo]

no for driver_info.node_ip_address, yes for the second thing. That's why we check for the value from the dictionary.

[alanwguo]

@scottsun94 , want to take another look? I think I added the autoscaler row to the job detail page since you last reviewed

[scottsun94]

I cannot seem to see the "User-provided metadata" field. Is that only available when it's not empty? Shall we just always show it? It might raise the awareness of this feature.

[alanwguo]

It's hidden for driver jobs because it's not possible to set metadata.

For submission jobs, it shows up as a "-" if empty.

[scottsun94]

oh. I see. This is the same for driver logs.
Just throwing a thought: shall we actually expose both of them and add message bars/tooltips whichever is appropriate to educate users why it's not available and the difference between driver jobs and submission jobs? Otherwise, the control logic here is just mystical to them because they may assume all ray jobs should work in the same way

NVM. Let's get this in first and wait for more info.

[scottsun94]

Everything LGTM. Thanks!