Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Incorrect display smooth feature #3837

Open
SoyGema opened this issue May 6, 2023 · 15 comments
Open

Incorrect display smooth feature #3837

SoyGema opened this issue May 6, 2023 · 15 comments
Assignees
@shcheklein
Labels
A: plots Area: plots webview, side panel and everything related bug Something isn't working priority-p1 Regular product backlog

Comments

@SoyGema
Copy link

SoyGema commented May 6, 2023 •
edited
Loading

CONTEXT :
Neural Machine Translation training scenario.

  • VSCode version 1.7.8
  • DVC Version 2.56.0
  • DVC VSCode extension Version 0.8.7

The smoothing feature shows two lines instead of one. The thicker green line should be transformed as the UI smoothing basr is moving through the bar but it´s not.

SmoothSmoothing.mp4

From conversation in Discord

Increasing plot size test seems to have the same issue
increasingplotsize

Machine Learning Epicstemics coming from the conversation in discord

The signal or number of datapoints can increase significantly when we change the batch size parameter during experiments : batch size controls the accuracy of the estimate of the error gradient when training Neural Networks. Too large batch sizes -less data points- may cause bad generalization. Many data points (smaller batch sizes ) take more training time but might have better generalization.

Iterating over this parameter can be a common case of experimentation scenario.

NOTE for machine learning practitioners: this parameter starts with 8 and increases by powers of 2. Experimenting with this parameter is key for better generalization by having other rational hypotheses in mind. This article gives good hints.

For developers

  • Test smoothing Scaling with respect XY axis ? Maybe might work changing the range and linearly interpolating in between values ?

Nice work !
@shcheklein shcheklein added the bug Something isn't working label May 6, 2023
@shcheklein
Copy link
Member

It's more related to dvc-render to be honest, but let's keep it here for now (cc @daavoo @dberenbaum ) . Some customers (Studio) also were mentioning broken and over aggressive smoothing which forced them to stay with TB. We need to got back and research the templates again I guess. Making p1, assigning to myself, David, and Dave. We'll try to get to it.

@shcheklein shcheklein added the A: plots Area: plots webview, side panel and everything related label May 6, 2023
@shcheklein shcheklein added the priority-p1 Regular product backlog label May 6, 2023
@SoyGema
Copy link
Author

SoyGema commented May 7, 2023

It's more related to dvc-render to be honest, but .... -->Hey thanks for this. From now on I´ll make the effort to explore codebases to report bugs more efficiently!
We'll try to get to it. --> You´ve got it!

@shcheklein shcheklein changed the title Incorrect Display Smooth feature Incorrect display smooth feature May 7, 2023
@mattseddon
Copy link
Member

My $0.02 is (even on a laptop) when the plot is given the screen's entire space that the experience isn't that bad:

Screen.Recording.2023-05-09.at.4.25.09.pm.mov

On a wide screen you get an even better sense of what is going on:

Screen.Recording.2023-05-09.at.4.29.15.pm.mov

@daavoo
Copy link
Contributor

daavoo commented May 9, 2023

I think the underlying problems here are:

  • We are incorrectly using the vega transformr fo smoothing linear charts.
    The transform we use is meant for scatter plots: https://vega.github.io/vega/examples/loess-regression/
    People is used to a behavior based in something like a exponential moving average (i.e. the one used in Tensorboard)

  • Related to above, displaying the original unconnected points make sense for the regression case but not for people expecting the "TensorBoard behavior"

@go2carter
Copy link

Generally, there seems to be some aggressive smoothing for what I'd expect, especially at the edges of plots. If there's interpolation with linear, where smoothness of 1 is linear, then I can definitely see why that is. I would not expect linear--I guess I'm someone expecting more like the "TensorBoard behavior". @daavoo , I think that first bullet is exactly where I'm coming from

For an example of what I'd expect, I'm attaching examples what tensorboard does

image
image
image

And this is studio:
image
image
image

@mattseddon mattseddon mentioned this issue May 9, 2023
Closed
@shcheklein
Copy link
Member

I'm looking into this (trying different Vega hacks). For the record, this is the Vega dump that can be used in the Vega editor:

{
  "$schema": "https://vega.github.io/schema/vega-lite/v5.json",
  "data": {
    "values": [
      { "step": 0, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 1, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 2, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 3, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 4, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 5, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 6, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 7, "accuracy_train": "0.0", "rev": "workspace" },
      { "step": 8, "accuracy_train": "0.0", "rev": "workspace" },
      {
        "step": 9,
        "accuracy_train": "7.027406536508352e-05",
        "rev": "workspace"
      },
      {
        "step": 10,
        "accuracy_train": "6.388551992131397e-05",
        "rev": "workspace"
      },
      {
        "step": 11,
        "accuracy_train": "0.00011732993880286813",
        "rev": "workspace"
      },
      {
        "step": 12,
        "accuracy_train": "0.00017272924014832824",
        "rev": "workspace"
      },
      {
        "step": 13,
        "accuracy_train": "0.0001603914424777031",
        "rev": "workspace"
      },
      {
        "step": 14,
        "accuracy_train": "0.0003600036143325269",
        "rev": "workspace"
      },
      {
        "step": 15,
        "accuracy_train": "0.0006608589319512248",
        "rev": "workspace"
      },
      {
        "step": 16,
        "accuracy_train": "0.0009273633477278054",
        "rev": "workspace"
      },
      {
        "step": 17,
        "accuracy_train": "0.001841690856963396",
        "rev": "workspace"
      },
      {
        "step": 18,
        "accuracy_train": "0.0036847058217972517",
        "rev": "workspace"
      },
      {
        "step": 19,
        "accuracy_train": "0.006334620527923107",
        "rev": "workspace"
      },
      {
        "step": 20,
        "accuracy_train": "0.011472896672785282",
        "rev": "workspace"
      },
      {
        "step": 21,
        "accuracy_train": "0.019665148109197617",
        "rev": "workspace"
      },
      {
        "step": 22,
        "accuracy_train": "0.030836043879389763",
        "rev": "workspace"
      },
      {
        "step": 23,
        "accuracy_train": "0.04692433401942253",
        "rev": "workspace"
      },
      {
        "step": 24,
        "accuracy_train": "0.06700370460748672",
        "rev": "workspace"
      },
      {
        "step": 25,
        "accuracy_train": "0.09210359305143356",
        "rev": "workspace"
      },
      {
        "step": 26,
        "accuracy_train": "0.1183219775557518",
        "rev": "workspace"
      },
      {
        "step": 27,
        "accuracy_train": "0.14470842480659485",
        "rev": "workspace"
      },
      {
        "step": 28,
        "accuracy_train": "0.17087672650814056",
        "rev": "workspace"
      },
      {
        "step": 29,
        "accuracy_train": "0.1962459832429886",
        "rev": "workspace"
      },
      {
        "step": 30,
        "accuracy_train": "0.22028383612632751",
        "rev": "workspace"
      },
      {
        "step": 31,
        "accuracy_train": "0.24271342158317566",
        "rev": "workspace"
      },
      {
        "step": 32,
        "accuracy_train": "0.26400768756866455",
        "rev": "workspace"
      },
      {
        "step": 33,
        "accuracy_train": "0.2843042016029358",
        "rev": "workspace"
      },
      {
        "step": 34,
        "accuracy_train": "0.30344653129577637",
        "rev": "workspace"
      },
      { "step": 35, "accuracy_train": "0.321421355009079", "rev": "workspace" },
      {
        "step": 36,
        "accuracy_train": "0.3384588062763214",
        "rev": "workspace"
      },
      {
        "step": 37,
        "accuracy_train": "0.35466134548187256",
        "rev": "workspace"
      },
      {
        "step": 38,
        "accuracy_train": "0.36987215280532837",
        "rev": "workspace"
      },
      {
        "step": 39,
        "accuracy_train": "0.3840927481651306",
        "rev": "workspace"
      },
      {
        "step": 40,
        "accuracy_train": "0.39798834919929504",
        "rev": "workspace"
      },
      {
        "step": 41,
        "accuracy_train": "0.41096988320350647",
        "rev": "workspace"
      },
      {
        "step": 42,
        "accuracy_train": "0.42340797185897827",
        "rev": "workspace"
      },
      {
        "step": 43,
        "accuracy_train": "0.4354696571826935",
        "rev": "workspace"
      },
      {
        "step": 44,
        "accuracy_train": "0.44676175713539124",
        "rev": "workspace"
      },
      {
        "step": 45,
        "accuracy_train": "0.45774802565574646",
        "rev": "workspace"
      },
      {
        "step": 46,
        "accuracy_train": "0.4681800305843353",
        "rev": "workspace"
      },
      {
        "step": 47,
        "accuracy_train": "0.4783846139907837",
        "rev": "workspace"
      },
      {
        "step": 48,
        "accuracy_train": "0.4879940450191498",
        "rev": "workspace"
      },
      {
        "step": 49,
        "accuracy_train": "0.49726226925849915",
        "rev": "workspace"
      },
      {
        "step": 50,
        "accuracy_train": "0.5061734914779663",
        "rev": "workspace"
      },
      {
        "step": 51,
        "accuracy_train": "0.5147313475608826",
        "rev": "workspace"
      },
      {
        "step": 52,
        "accuracy_train": "0.5229606032371521",
        "rev": "workspace"
      },
      {
        "step": 53,
        "accuracy_train": "0.5308094620704651",
        "rev": "workspace"
      },
      {
        "step": 54,
        "accuracy_train": "0.5384282469749451",
        "rev": "workspace"
      },
      {
        "step": 55,
        "accuracy_train": "0.5457383990287781",
        "rev": "workspace"
      },
      {
        "step": 56,
        "accuracy_train": "0.5528160929679871",
        "rev": "workspace"
      },
      {
        "step": 57,
        "accuracy_train": "0.5596035122871399",
        "rev": "workspace"
      },
      {
        "step": 58,
        "accuracy_train": "0.5662795305252075",
        "rev": "workspace"
      },
      {
        "step": 59,
        "accuracy_train": "0.5725557804107666",
        "rev": "workspace"
      },
      {
        "step": 60,
        "accuracy_train": "0.5788120627403259",
        "rev": "workspace"
      },
      {
        "step": 61,
        "accuracy_train": "0.5848563313484192",
        "rev": "workspace"
      },
      {
        "step": 62,
        "accuracy_train": "0.5907113552093506",
        "rev": "workspace"
      },
      {
        "step": 63,
        "accuracy_train": "0.5963535308837891",
        "rev": "workspace"
      },
      {
        "step": 64,
        "accuracy_train": "0.6018657088279724",
        "rev": "workspace"
      },
      {
        "step": 65,
        "accuracy_train": "0.6071356534957886",
        "rev": "workspace"
      },
      {
        "step": 66,
        "accuracy_train": "0.6122435927391052",
        "rev": "workspace"
      },
      {
        "step": 67,
        "accuracy_train": "0.6172387599945068",
        "rev": "workspace"
      },
      { "step": 68, "accuracy_train": "0.622154176235199", "rev": "workspace" },
      {
        "step": 69,
        "accuracy_train": "0.6268956065177917",
        "rev": "workspace"
      },
      {
        "step": 70,
        "accuracy_train": "0.6313748955726624",
        "rev": "workspace"
      },
      {
        "step": 71,
        "accuracy_train": "0.6357920169830322",
        "rev": "workspace"
      },
      {
        "step": 72,
        "accuracy_train": "0.6401164531707764",
        "rev": "workspace"
      },
      {
        "step": 73,
        "accuracy_train": "0.6443119049072266",
        "rev": "workspace"
      },
      {
        "step": 74,
        "accuracy_train": "0.6484841704368591",
        "rev": "workspace"
      },
      {
        "step": 75,
        "accuracy_train": "0.6524708867073059",
        "rev": "workspace"
      },
      {
        "step": 76,
        "accuracy_train": "0.6563584208488464",
        "rev": "workspace"
      },
      {
        "step": 77,
        "accuracy_train": "0.6601410508155823",
        "rev": "workspace"
      },
      {
        "step": 78,
        "accuracy_train": "0.6638056635856628",
        "rev": "workspace"
      },
      {
        "step": 79,
        "accuracy_train": "0.6673806309700012",
        "rev": "workspace"
      },
      {
        "step": 80,
        "accuracy_train": "0.6708938479423523",
        "rev": "workspace"
      },
      {
        "step": 81,
        "accuracy_train": "0.6743400692939758",
        "rev": "workspace"
      },
      {
        "step": 82,
        "accuracy_train": "0.6775490641593933",
        "rev": "workspace"
      },
      { "step": 83, "accuracy_train": "0.680792510509491", "rev": "workspace" },
      {
        "step": 84,
        "accuracy_train": "0.6839669346809387",
        "rev": "workspace"
      },
      {
        "step": 85,
        "accuracy_train": "0.6870697140693665",
        "rev": "workspace"
      },
      {
        "step": 86,
        "accuracy_train": "0.6900910139083862",
        "rev": "workspace"
      },
      { "step": 87, "accuracy_train": "0.693053662776947", "rev": "workspace" },
      {
        "step": 88,
        "accuracy_train": "0.6959678530693054",
        "rev": "workspace"
      },
      {
        "step": 89,
        "accuracy_train": "0.6988064646720886",
        "rev": "workspace"
      },
      {
        "step": 90,
        "accuracy_train": "0.7015607357025146",
        "rev": "workspace"
      },
      {
        "step": 91,
        "accuracy_train": "0.7042788863182068",
        "rev": "workspace"
      },
      {
        "step": 92,
        "accuracy_train": "0.7069124579429626",
        "rev": "workspace"
      },
      {
        "step": 93,
        "accuracy_train": "0.7095058560371399",
        "rev": "workspace"
      },
      { "step": 94, "accuracy_train": "0.712057888507843", "rev": "workspace" },
      {
        "step": 95,
        "accuracy_train": "0.7144908905029297",
        "rev": "workspace"
      },
      {
        "step": 96,
        "accuracy_train": "0.7168821096420288",
        "rev": "workspace"
      },
      {
        "step": 97,
        "accuracy_train": "0.7192414999008179",
        "rev": "workspace"
      },
      {
        "step": 98,
        "accuracy_train": "0.7215766310691833",
        "rev": "workspace"
      },
      {
        "step": 99,
        "accuracy_train": "0.7238864898681641",
        "rev": "workspace"
      },
      {
        "step": 100,
        "accuracy_train": "0.7261072397232056",
        "rev": "workspace"
      },
      {
        "step": 101,
        "accuracy_train": "0.7282916903495789",
        "rev": "workspace"
      },
      {
        "step": 102,
        "accuracy_train": "0.7304773926734924",
        "rev": "workspace"
      },
      {
        "step": 103,
        "accuracy_train": "0.7326055765151978",
        "rev": "workspace"
      },
      {
        "step": 104,
        "accuracy_train": "0.7347046732902527",
        "rev": "workspace"
      },
      {
        "step": 105,
        "accuracy_train": "0.7367346286773682",
        "rev": "workspace"
      },
      {
        "step": 106,
        "accuracy_train": "0.7387702465057373",
        "rev": "workspace"
      },
      {
        "step": 107,
        "accuracy_train": "0.7407589554786682",
        "rev": "workspace"
      },
      {
        "step": 108,
        "accuracy_train": "0.7427427172660828",
        "rev": "workspace"
      },
      {
        "step": 109,
        "accuracy_train": "0.7445874810218811",
        "rev": "workspace"
      },
      {
        "step": 110,
        "accuracy_train": "0.7464470267295837",
        "rev": "workspace"
      },
      {
        "step": 111,
        "accuracy_train": "0.748274028301239",
        "rev": "workspace"
      },
      {
        "step": 112,
        "accuracy_train": "0.750049889087677",
        "rev": "workspace"
      },
      {
        "step": 113,
        "accuracy_train": "0.7518092393875122",
        "rev": "workspace"
      },
      {
        "step": 114,
        "accuracy_train": "0.753516435623169",
        "rev": "workspace"
      },
      {
        "step": 115,
        "accuracy_train": "0.7552217245101929",
        "rev": "workspace"
      },
      {
        "step": 116,
        "accuracy_train": "0.7568762302398682",
        "rev": "workspace"
      },
      {
        "step": 117,
        "accuracy_train": "0.7585200071334839",
        "rev": "workspace"
      },
      {
        "step": 118,
        "accuracy_train": "0.7601910829544067",
        "rev": "workspace"
      },
      {
        "step": 119,
        "accuracy_train": "0.7616886496543884",
        "rev": "workspace"
      },
      {
        "step": 120,
        "accuracy_train": "0.763236939907074",
        "rev": "workspace"
      },
      {
        "step": 121,
        "accuracy_train": "0.7647789120674133",
        "rev": "workspace"
      },
      {
        "step": 122,
        "accuracy_train": "0.766293466091156",
        "rev": "workspace"
      },
      {
        "step": 123,
        "accuracy_train": "0.7678454518318176",
        "rev": "workspace"
      },
      {
        "step": 124,
        "accuracy_train": "0.7693033218383789",
        "rev": "workspace"
      },
      {
        "step": 125,
        "accuracy_train": "0.7707488536834717",
        "rev": "workspace"
      },
      {
        "step": 126,
        "accuracy_train": "0.772094190120697",
        "rev": "workspace"
      },
      {
        "step": 127,
        "accuracy_train": "0.7735177874565125",
        "rev": "workspace"
      },
      {
        "step": 128,
        "accuracy_train": "0.7748700976371765",
        "rev": "workspace"
      },
      {
        "step": 129,
        "accuracy_train": "0.776178240776062",
        "rev": "workspace"
      },
      {
        "step": 130,
        "accuracy_train": "0.7775197625160217",
        "rev": "workspace"
      },
      {
        "step": 131,
        "accuracy_train": "0.7787727117538452",
        "rev": "workspace"
      },
      {
        "step": 132,
        "accuracy_train": "0.7800529599189758",
        "rev": "workspace"
      },
      {
        "step": 133,
        "accuracy_train": "0.7812585234642029",
        "rev": "workspace"
      },
      {
        "step": 134,
        "accuracy_train": "0.7824690937995911",
        "rev": "workspace"
      },
      {
        "step": 135,
        "accuracy_train": "0.7836477160453796",
        "rev": "workspace"
      },
      {
        "step": 136,
        "accuracy_train": "0.7849162220954895",
        "rev": "workspace"
      },
      {
        "step": 137,
        "accuracy_train": "0.7861440181732178",
        "rev": "workspace"
      },
      {
        "step": 138,
        "accuracy_train": "0.7873712778091431",
        "rev": "workspace"
      },
      {
        "step": 139,
        "accuracy_train": "0.7885527014732361",
        "rev": "workspace"
      },
      {
        "step": 140,
        "accuracy_train": "0.7897130250930786",
        "rev": "workspace"
      },
      {
        "step": 141,
        "accuracy_train": "0.790870189666748",
        "rev": "workspace"
      },
      {
        "step": 142,
        "accuracy_train": "0.7920458316802979",
        "rev": "workspace"
      },
      {
        "step": 143,
        "accuracy_train": "0.7931103706359863",
        "rev": "workspace"
      },
      {
        "step": 144,
        "accuracy_train": "0.7942361235618591",
        "rev": "workspace"
      },
      {
        "step": 145,
        "accuracy_train": "0.7953284978866577",
        "rev": "workspace"
      },
      {
        "step": 146,
        "accuracy_train": "0.7963480353355408",
        "rev": "workspace"
      },
      {
        "step": 147,
        "accuracy_train": "0.7974063158035278",
        "rev": "workspace"
      },
      {
        "step": 148,
        "accuracy_train": "0.798450231552124",
        "rev": "workspace"
      },
      {
        "step": 149,
        "accuracy_train": "0.7994584441184998",
        "rev": "workspace"
      },
      {
        "step": 150,
        "accuracy_train": "0.8004546761512756",
        "rev": "workspace"
      },
      {
        "step": 151,
        "accuracy_train": "0.8013997077941895",
        "rev": "workspace"
      },
      {
        "step": 152,
        "accuracy_train": "0.8024122714996338",
        "rev": "workspace"
      },
      {
        "step": 153,
        "accuracy_train": "0.803358793258667",
        "rev": "workspace"
      },
      {
        "step": 154,
        "accuracy_train": "0.8043005466461182",
        "rev": "workspace"
      },
      {
        "step": 155,
        "accuracy_train": "0.8052694797515869",
        "rev": "workspace"
      },
      {
        "step": 156,
        "accuracy_train": "0.806199848651886",
        "rev": "workspace"
      },
      {
        "step": 157,
        "accuracy_train": "0.8070595264434814",
        "rev": "workspace"
      },
      {
        "step": 158,
        "accuracy_train": "0.8079778552055359",
        "rev": "workspace"
      },
      {
        "step": 159,
        "accuracy_train": "0.8088495135307312",
        "rev": "workspace"
      },
      {
        "step": 160,
        "accuracy_train": "0.8097116947174072",
        "rev": "workspace"
      },
      {
        "step": 161,
        "accuracy_train": "0.810590386390686",
        "rev": "workspace"
      },
      {
        "step": 162,
        "accuracy_train": "0.8114262819290161",
        "rev": "workspace"
      },
      {
        "step": 163,
        "accuracy_train": "0.8122937083244324",
        "rev": "workspace"
      },
      {
        "step": 164,
        "accuracy_train": "0.8131471872329712",
        "rev": "workspace"
      },
      {
        "step": 165,
        "accuracy_train": "0.8139777779579163",
        "rev": "workspace"
      },
      {
        "step": 166,
        "accuracy_train": "0.8147744536399841",
        "rev": "workspace"
      },
      {
        "step": 167,
        "accuracy_train": "0.8156265020370483",
        "rev": "workspace"
      },
      {
        "step": 168,
        "accuracy_train": "0.8164225220680237",
        "rev": "workspace"
      },
      {
        "step": 169,
        "accuracy_train": "0.8172279596328735",
        "rev": "workspace"
      },
      {
        "step": 170,
        "accuracy_train": "0.8180204033851624",
        "rev": "workspace"
      },
      {
        "step": 171,
        "accuracy_train": "0.8188337087631226",
        "rev": "workspace"
      },
      {
        "step": 172,
        "accuracy_train": "0.8196122646331787",
        "rev": "workspace"
      },
      {
        "step": 173,
        "accuracy_train": "0.8203689455986023",
        "rev": "workspace"
      },
      {
        "step": 174,
        "accuracy_train": "0.8211503624916077",
        "rev": "workspace"
      },
      {
        "step": 175,
        "accuracy_train": "0.821871817111969",
        "rev": "workspace"
      },
      {
        "step": 176,
        "accuracy_train": "0.8225604295730591",
        "rev": "workspace"
      },
      {
        "step": 177,
        "accuracy_train": "0.8232889771461487",
        "rev": "workspace"
      },
      {
        "step": 178,
        "accuracy_train": "0.8240391612052917",
        "rev": "workspace"
      },
      {
        "step": 179,
        "accuracy_train": "0.824738085269928",
        "rev": "workspace"
      },
      {
        "step": 180,
        "accuracy_train": "0.8254061341285706",
        "rev": "workspace"
      },
      {
        "step": 181,
        "accuracy_train": "0.8260559439659119",
        "rev": "workspace"
      },
      {
        "step": 182,
        "accuracy_train": "0.8267054557800293",
        "rev": "workspace"
      },
      {
        "step": 183,
        "accuracy_train": "0.8273881077766418",
        "rev": "workspace"
      },
      {
        "step": 184,
        "accuracy_train": "0.8280845284461975",
        "rev": "workspace"
      },
      {
        "step": 185,
        "accuracy_train": "0.828743040561676",
        "rev": "workspace"
      },
      {
        "step": 186,
        "accuracy_train": "0.8294104933738708",
        "rev": "workspace"
      },
      {
        "step": 187,
        "accuracy_train": "0.8300095200538635",
        "rev": "workspace"
      },
      {
        "step": 188,
        "accuracy_train": "0.8306493163108826",
        "rev": "workspace"
      },
      {
        "step": 189,
        "accuracy_train": "0.8312875628471375",
        "rev": "workspace"
      },
      {
        "step": 190,
        "accuracy_train": "0.8319101333618164",
        "rev": "workspace"
      },
      {
        "step": 191,
        "accuracy_train": "0.8325663208961487",
        "rev": "workspace"
      },
      {
        "step": 192,
        "accuracy_train": "0.8332136869430542",
        "rev": "workspace"
      },
      {
        "step": 193,
        "accuracy_train": "0.8338399529457092",
        "rev": "workspace"
      },
      {
        "step": 194,
        "accuracy_train": "0.8344071507453918",
        "rev": "workspace"
      },
      {
        "step": 195,
        "accuracy_train": "0.8350204229354858",
        "rev": "workspace"
      },
      {
        "step": 196,
        "accuracy_train": "0.8356167078018188",
        "rev": "workspace"
      },
      {
        "step": 197,
        "accuracy_train": "0.836176872253418",
        "rev": "workspace"
      },
      {
        "step": 198,
        "accuracy_train": "0.8368045091629028",
        "rev": "workspace"
      },
      {
        "step": 199,
        "accuracy_train": "0.8373478651046753",
        "rev": "workspace"
      },
      {
        "step": 200,
        "accuracy_train": "0.837903618812561",
        "rev": "workspace"
      },
      {
        "step": 201,
        "accuracy_train": "0.8384701609611511",
        "rev": "workspace"
      },
      {
        "step": 202,
        "accuracy_train": "0.8390499353408813",
        "rev": "workspace"
      },
      {
        "step": 203,
        "accuracy_train": "0.8396066427230835",
        "rev": "workspace"
      },
      {
        "step": 204,
        "accuracy_train": "0.8401467204093933",
        "rev": "workspace"
      },
      {
        "step": 205,
        "accuracy_train": "0.840649425983429",
        "rev": "workspace"
      },
      {
        "step": 206,
        "accuracy_train": "0.8411962985992432",
        "rev": "workspace"
      },
      {
        "step": 207,
        "accuracy_train": "0.8417292833328247",
        "rev": "workspace"
      },
      {
        "step": 208,
        "accuracy_train": "0.8422626256942749",
        "rev": "workspace"
      },
      {
        "step": 209,
        "accuracy_train": "0.8427655100822449",
        "rev": "workspace"
      },
      {
        "step": 210,
        "accuracy_train": "0.843235969543457",
        "rev": "workspace"
      },
      {
        "step": 211,
        "accuracy_train": "0.8437221646308899",
        "rev": "workspace"
      },
      {
        "step": 212,
        "accuracy_train": "0.8441882729530334",
        "rev": "workspace"
      },
      {
        "step": 213,
        "accuracy_train": "0.8446692228317261",
        "rev": "workspace"
      },
      {
        "step": 214,
        "accuracy_train": "0.845182478427887",
        "rev": "workspace"
      },
      {
        "step": 215,
        "accuracy_train": "0.8456679582595825",
        "rev": "workspace"
      },
      {
        "step": 216,
        "accuracy_train": "0.8461502194404602",
        "rev": "workspace"
      },
      {
        "step": 217,
        "accuracy_train": "0.8466311097145081",
        "rev": "workspace"
      },
      {
        "step": 218,
        "accuracy_train": "0.8471159338951111",
        "rev": "workspace"
      },
      {
        "step": 219,
        "accuracy_train": "0.8476173281669617",
        "rev": "workspace"
      },
      {
        "step": 220,
        "accuracy_train": "0.8480663299560547",
        "rev": "workspace"
      },
      {
        "step": 221,
        "accuracy_train": "0.8485521078109741",
        "rev": "workspace"
      },
      {
        "step": 222,
        "accuracy_train": "0.8490065336227417",
        "rev": "workspace"
      },
      { "step": 223, "accuracy_train": "0.84945148229599", "rev": "workspace" },
      {
        "step": 224,
        "accuracy_train": "0.8499021530151367",
        "rev": "workspace"
      },
      {
        "step": 225,
        "accuracy_train": "0.8503347039222717",
        "rev": "workspace"
      },
      {
        "step": 226,
        "accuracy_train": "0.8507705926895142",
        "rev": "workspace"
      },
      {
        "step": 227,
        "accuracy_train": "0.8512048125267029",
        "rev": "workspace"
      },
      {
        "step": 228,
        "accuracy_train": "0.8516327142715454",
        "rev": "workspace"
      },
      {
        "step": 229,
        "accuracy_train": "0.8520622849464417",
        "rev": "workspace"
      },
      {
        "step": 230,
        "accuracy_train": "0.8524938821792603",
        "rev": "workspace"
      },
      {
        "step": 231,
        "accuracy_train": "0.852929949760437",
        "rev": "workspace"
      },
      {
        "step": 232,
        "accuracy_train": "0.8533341288566589",
        "rev": "workspace"
      },
      {
        "step": 233,
        "accuracy_train": "0.8537390828132629",
        "rev": "workspace"
      },
      {
        "step": 234,
        "accuracy_train": "0.8540988564491272",
        "rev": "workspace"
      },
      {
        "step": 235,
        "accuracy_train": "0.8545152544975281",
        "rev": "workspace"
      },
      {
        "step": 236,
        "accuracy_train": "0.8549187779426575",
        "rev": "workspace"
      },
      {
        "step": 237,
        "accuracy_train": "0.8553181290626526",
        "rev": "workspace"
      },
      {
        "step": 238,
        "accuracy_train": "0.855717122554779",
        "rev": "workspace"
      },
      {
        "step": 239,
        "accuracy_train": "0.8561220765113831",
        "rev": "workspace"
      },
      {
        "step": 240,
        "accuracy_train": "0.8564921021461487",
        "rev": "workspace"
      },
      {
        "step": 241,
        "accuracy_train": "0.8568522930145264",
        "rev": "workspace"
      },
      {
        "step": 242,
        "accuracy_train": "0.8572503924369812",
        "rev": "workspace"
      },
      {
        "step": 243,
        "accuracy_train": "0.8576446771621704",
        "rev": "workspace"
      },
      {
        "step": 244,
        "accuracy_train": "0.8580135703086853",
        "rev": "workspace"
      },
      {
        "step": 245,
        "accuracy_train": "0.8583711981773376",
        "rev": "workspace"
      },
      {
        "step": 246,
        "accuracy_train": "0.8587556481361389",
        "rev": "workspace"
      },
      {
        "step": 247,
        "accuracy_train": "0.8591521978378296",
        "rev": "workspace"
      },
      {
        "step": 248,
        "accuracy_train": "0.859534502029419",
        "rev": "workspace"
      },
      {
        "step": 249,
        "accuracy_train": "0.859878659248352",
        "rev": "workspace"
      },
      {
        "step": 250,
        "accuracy_train": "0.8602045178413391",
        "rev": "workspace"
      },
      {
        "step": 251,
        "accuracy_train": "0.8605643510818481",
        "rev": "workspace"
      },
      {
        "step": 252,
        "accuracy_train": "0.8609074354171753",
        "rev": "workspace"
      },
      {
        "step": 253,
        "accuracy_train": "0.8612474799156189",
        "rev": "workspace"
      },
      {
        "step": 254,
        "accuracy_train": "0.8616113662719727",
        "rev": "workspace"
      },
      {
        "step": 255,
        "accuracy_train": "0.8619658946990967",
        "rev": "workspace"
      },
      {
        "step": 256,
        "accuracy_train": "0.8622974753379822",
        "rev": "workspace"
      },
      { "step": 257, "accuracy_train": "0.86265629529953", "rev": "workspace" },
      {
        "step": 258,
        "accuracy_train": "0.8630151748657227",
        "rev": "workspace"
      },
      {
        "step": 259,
        "accuracy_train": "0.8633338212966919",
        "rev": "workspace"
      },
      {
        "step": 260,
        "accuracy_train": "0.8636519312858582",
        "rev": "workspace"
      },
      {
        "step": 261,
        "accuracy_train": "0.8639867305755615",
        "rev": "workspace"
      },
      {
        "step": 262,
        "accuracy_train": "0.8643000721931458",
        "rev": "workspace"
      },
      {
        "step": 263,
        "accuracy_train": "0.8646564483642578",
        "rev": "workspace"
      },
      {
        "step": 264,
        "accuracy_train": "0.8649749159812927",
        "rev": "workspace"
      },
      {
        "step": 265,
        "accuracy_train": "0.8652567863464355",
        "rev": "workspace"
      },
      {
        "step": 266,
        "accuracy_train": "0.8655537366867065",
        "rev": "workspace"
      },
      {
        "step": 267,
        "accuracy_train": "0.8658890724182129",
        "rev": "workspace"
      },
      {
        "step": 268,
        "accuracy_train": "0.8662153482437134",
        "rev": "workspace"
      },
      {
        "step": 269,
        "accuracy_train": "0.8665201663970947",
        "rev": "workspace"
      },
      {
        "step": 270,
        "accuracy_train": "0.8668607473373413",
        "rev": "workspace"
      },
      {
        "step": 271,
        "accuracy_train": "0.8671742677688599",
        "rev": "workspace"
      },
      {
        "step": 272,
        "accuracy_train": "0.8674926161766052",
        "rev": "workspace"
      },
      {
        "step": 273,
        "accuracy_train": "0.8677963018417358",
        "rev": "workspace"
      },
      {
        "step": 274,
        "accuracy_train": "0.8681104779243469",
        "rev": "workspace"
      },
      {
        "step": 275,
        "accuracy_train": "0.8684048652648926",
        "rev": "workspace"
      },
      {
        "step": 276,
        "accuracy_train": "0.8686904311180115",
        "rev": "workspace"
      },
      {
        "step": 277,
        "accuracy_train": "0.8689703941345215",
        "rev": "workspace"
      },
      {
        "step": 278,
        "accuracy_train": "0.8692751526832581",
        "rev": "workspace"
      },
      {
        "step": 279,
        "accuracy_train": "0.869561493396759",
        "rev": "workspace"
      },
      {
        "step": 280,
        "accuracy_train": "0.8698569536209106",
        "rev": "workspace"
      },
      {
        "step": 281,
        "accuracy_train": "0.8701409101486206",
        "rev": "workspace"
      },
      {
        "step": 282,
        "accuracy_train": "0.870430052280426",
        "rev": "workspace"
      },
      { "step": 283, "accuracy_train": "0.87070631980896", "rev": "workspace" },
      {
        "step": 284,
        "accuracy_train": "0.8709831237792969",
        "rev": "workspace"
      },
      {
        "step": 285,
        "accuracy_train": "0.871252179145813",
        "rev": "workspace"
      },
      {
        "step": 286,
        "accuracy_train": "0.8715106248855591",
        "rev": "workspace"
      },
      {
        "step": 287,
        "accuracy_train": "0.8717894554138184",
        "rev": "workspace"
      },
      {
        "step": 288,
        "accuracy_train": "0.8720710873603821",
        "rev": "workspace"
      },
      {
        "step": 289,
        "accuracy_train": "0.8723511099815369",
        "rev": "workspace"
      },
      {
        "step": 290,
        "accuracy_train": "0.8726474046707153",
        "rev": "workspace"
      },
      {
        "step": 291,
        "accuracy_train": "0.8729016780853271",
        "rev": "workspace"
      },
      {
        "step": 292,
        "accuracy_train": "0.8731608390808105",
        "rev": "workspace"
      },
      {
        "step": 293,
        "accuracy_train": "0.8734003305435181",
        "rev": "workspace"
      },
      {
        "step": 294,
        "accuracy_train": "0.8736701607704163",
        "rev": "workspace"
      },
      {
        "step": 295,
        "accuracy_train": "0.873936653137207",
        "rev": "workspace"
      },
      {
        "step": 296,
        "accuracy_train": "0.8741785883903503",
        "rev": "workspace"
      },
      {
        "step": 297,
        "accuracy_train": "0.8744345307350159",
        "rev": "workspace"
      },
      {
        "step": 298,
        "accuracy_train": "0.8746727705001831",
        "rev": "workspace"
      },
      {
        "step": 299,
        "accuracy_train": "0.874925434589386",
        "rev": "workspace"
      },
      {
        "step": 300,
        "accuracy_train": "0.875191330909729",
        "rev": "workspace"
      },
      {
        "step": 301,
        "accuracy_train": "0.8754462599754333",
        "rev": "workspace"
      },
      {
        "step": 302,
        "accuracy_train": "0.8756964802742004",
        "rev": "workspace"
      },
      {
        "step": 303,
        "accuracy_train": "0.8759343028068542",
        "rev": "workspace"
      },
      {
        "step": 304,
        "accuracy_train": "0.8761361837387085",
        "rev": "workspace"
      },
      { "step": 305, "accuracy_train": "0.8763467669487", "rev": "workspace" },
      {
        "step": 306,
        "accuracy_train": "0.8765941858291626",
        "rev": "workspace"
      },
      {
        "step": 307,
        "accuracy_train": "0.8768274784088135",
        "rev": "workspace"
      },
      {
        "step": 308,
        "accuracy_train": "0.8770738840103149",
        "rev": "workspace"
      },
      {
        "step": 309,
        "accuracy_train": "0.8772940635681152",
        "rev": "workspace"
      },
      {
        "step": 310,
        "accuracy_train": "0.877560555934906",
        "rev": "workspace"
      },
      {
        "step": 311,
        "accuracy_train": "0.8778050541877747",
        "rev": "workspace"
      },
      {
        "step": 312,
        "accuracy_train": "0.8780496120452881",
        "rev": "workspace"
      },
      {
        "step": 313,
        "accuracy_train": "0.8782621622085571",
        "rev": "workspace"
      },
      {
        "step": 314,
        "accuracy_train": "0.8784927725791931",
        "rev": "workspace"
      },
      {
        "step": 315,
        "accuracy_train": "0.8787156343460083",
        "rev": "workspace"
      },
      {
        "step": 316,
        "accuracy_train": "0.8789414763450623",
        "rev": "workspace"
      },
      {
        "step": 317,
        "accuracy_train": "0.8791552782058716",
        "rev": "workspace"
      },
      {
        "step": 318,
        "accuracy_train": "0.8793788552284241",
        "rev": "workspace"
      },
      {
        "step": 319,
        "accuracy_train": "0.8796210289001465",
        "rev": "workspace"
      },
      {
        "step": 320,
        "accuracy_train": "0.8798565864562988",
        "rev": "workspace"
      },
      {
        "step": 321,
        "accuracy_train": "0.8800683617591858",
        "rev": "workspace"
      },
      {
        "step": 322,
        "accuracy_train": "0.8802956938743591",
        "rev": "workspace"
      },
      {
        "step": 323,
        "accuracy_train": "0.8805084228515625",
        "rev": "workspace"
      },
      {
        "step": 324,
        "accuracy_train": "0.8807024955749512",
        "rev": "workspace"
      },
      {
        "step": 325,
        "accuracy_train": "0.8809196352958679",
        "rev": "workspace"
      },
      {
        "step": 326,
        "accuracy_train": "0.8811274170875549",
        "rev": "workspace"
      },
      {
        "step": 327,
        "accuracy_train": "0.8813344240188599",
        "rev": "workspace"
      },
      {
        "step": 328,
        "accuracy_train": "0.8815332651138306",
        "rev": "workspace"
      },
      {
        "step": 329,
        "accuracy_train": "0.8817335367202759",
        "rev": "workspace"
      },
      {
        "step": 330,
        "accuracy_train": "0.8819234371185303",
        "rev": "workspace"
      },
      {
        "step": 331,
        "accuracy_train": "0.8820932507514954",
        "rev": "workspace"
      },
      {
        "step": 332,
        "accuracy_train": "0.8823164105415344",
        "rev": "workspace"
      },
      {
        "step": 333,
        "accuracy_train": "0.8825197219848633",
        "rev": "workspace"
      },
      {
        "step": 334,
        "accuracy_train": "0.8827186822891235",
        "rev": "workspace"
      },
      {
        "step": 335,
        "accuracy_train": "0.8829184174537659",
        "rev": "workspace"
      },
      {
        "step": 336,
        "accuracy_train": "0.8831091523170471",
        "rev": "workspace"
      },
      {
        "step": 337,
        "accuracy_train": "0.8833090662956238",
        "rev": "workspace"
      },
      {
        "step": 338,
        "accuracy_train": "0.8835080862045288",
        "rev": "workspace"
      },
      {
        "step": 339,
        "accuracy_train": "0.8837069272994995",
        "rev": "workspace"
      },
      {
        "step": 340,
        "accuracy_train": "0.883889377117157",
        "rev": "workspace"
      },
      {
        "step": 341,
        "accuracy_train": "0.8840998411178589",
        "rev": "workspace"
      },
      {
        "step": 342,
        "accuracy_train": "0.8843069672584534",
        "rev": "workspace"
      },
      {
        "step": 343,
        "accuracy_train": "0.884513795375824",
        "rev": "workspace"
      },
      {
        "step": 344,
        "accuracy_train": "0.884695291519165",
        "rev": "workspace"
      },
      {
        "step": 345,
        "accuracy_train": "0.8848926424980164",
        "rev": "workspace"
      },
      {
        "step": 346,
        "accuracy_train": "0.8850750923156738",
        "rev": "workspace"
      },
      {
        "step": 347,
        "accuracy_train": "0.8852613568305969",
        "rev": "workspace"
      },
      {
        "step": 348,
        "accuracy_train": "0.8854422569274902",
        "rev": "workspace"
      },
      {
        "step": 349,
        "accuracy_train": "0.8856532573699951",
        "rev": "workspace"
      },
      {
        "step": 350,
        "accuracy_train": "0.8858168125152588",
        "rev": "workspace"
      },
      {
        "step": 351,
        "accuracy_train": "0.8860132098197937",
        "rev": "workspace"
      },
      {
        "step": 352,
        "accuracy_train": "0.8862019181251526",
        "rev": "workspace"
      },
      {
        "step": 353,
        "accuracy_train": "0.8864020109176636",
        "rev": "workspace"
      },
      {
        "step": 354,
        "accuracy_train": "0.8865509629249573",
        "rev": "workspace"
      },
      {
        "step": 355,
        "accuracy_train": "0.8867299556732178",
        "rev": "workspace"
      },
      {
        "step": 356,
        "accuracy_train": "0.8868951797485352",
        "rev": "workspace"
      },
      {
        "step": 357,
        "accuracy_train": "0.8870682716369629",
        "rev": "workspace"
      },
      {
        "step": 358,
        "accuracy_train": "0.8872540593147278",
        "rev": "workspace"
      },
      {
        "step": 359,
        "accuracy_train": "0.8874168992042542",
        "rev": "workspace"
      },
      {
        "step": 360,
        "accuracy_train": "0.8875876069068909",
        "rev": "workspace"
      },
      {
        "step": 361,
        "accuracy_train": "0.8877578377723694",
        "rev": "workspace"
      },
      {
        "step": 362,
        "accuracy_train": "0.8879192471504211",
        "rev": "workspace"
      },
      {
        "step": 363,
        "accuracy_train": "0.8880944848060608",
        "rev": "workspace"
      },
      {
        "step": 364,
        "accuracy_train": "0.8882755041122437",
        "rev": "workspace"
      },
      {
        "step": 365,
        "accuracy_train": "0.8884318470954895",
        "rev": "workspace"
      },
      {
        "step": 366,
        "accuracy_train": "0.8886064887046814",
        "rev": "workspace"
      },
      {
        "step": 367,
        "accuracy_train": "0.8887869715690613",
        "rev": "workspace"
      },
      {
        "step": 368,
        "accuracy_train": "0.8889593482017517",
        "rev": "workspace"
      },
      {
        "step": 369,
        "accuracy_train": "0.8891066312789917",
        "rev": "workspace"
      },
      {
        "step": 370,
        "accuracy_train": "0.8892788290977478",
        "rev": "workspace"
      },
      {
        "step": 371,
        "accuracy_train": "0.8894485831260681",
        "rev": "workspace"
      },
      {
        "step": 372,
        "accuracy_train": "0.8896145820617676",
        "rev": "workspace"
      },
      {
        "step": 373,
        "accuracy_train": "0.8897762298583984",
        "rev": "workspace"
      },
      {
        "step": 374,
        "accuracy_train": "0.8899331092834473",
        "rev": "workspace"
      },
      {
        "step": 375,
        "accuracy_train": "0.8900969624519348",
        "rev": "workspace"
      },
      {
        "step": 376,
        "accuracy_train": "0.8902655243873596",
        "rev": "workspace"
      },
      {
        "step": 377,
        "accuracy_train": "0.8904131650924683",
        "rev": "workspace"
      },
      {
        "step": 378,
        "accuracy_train": "0.8906078338623047",
        "rev": "workspace"
      },
      {
        "step": 379,
        "accuracy_train": "0.8907554745674133",
        "rev": "workspace"
      },
      {
        "step": 380,
        "accuracy_train": "0.8909083604812622",
        "rev": "workspace"
      },
      {
        "step": 381,
        "accuracy_train": "0.8910605311393738",
        "rev": "workspace"
      },
      {
        "step": 382,
        "accuracy_train": "0.891206681728363",
        "rev": "workspace"
      },
      {
        "step": 383,
        "accuracy_train": "0.8913586139678955",
        "rev": "workspace"
      },
      {
        "step": 384,
        "accuracy_train": "0.8915366530418396",
        "rev": "workspace"
      },
      {
        "step": 385,
        "accuracy_train": "0.8917087912559509",
        "rev": "workspace"
      },
      {
        "step": 386,
        "accuracy_train": "0.8918344974517822",
        "rev": "workspace"
      },
      {
        "step": 387,
        "accuracy_train": "0.8919764161109924",
        "rev": "workspace"
      },
      {
        "step": 388,
        "accuracy_train": "0.8921183943748474",
        "rev": "workspace"
      },
      {
        "step": 389,
        "accuracy_train": "0.8922802805900574",
        "rev": "workspace"
      },
      {
        "step": 390,
        "accuracy_train": "0.8924030065536499",
        "rev": "workspace"
      },
      {
        "step": 391,
        "accuracy_train": "0.8925492167472839",
        "rev": "workspace"
      },
      {
        "step": 392,
        "accuracy_train": "0.8926982283592224",
        "rev": "workspace"
      },
      {
        "step": 393,
        "accuracy_train": "0.8928534984588623",
        "rev": "workspace"
      },
      {
        "step": 394,
        "accuracy_train": "0.8930209875106812",
        "rev": "workspace"
      },
      {
        "step": 395,
        "accuracy_train": "0.8931626081466675",
        "rev": "workspace"
      },
      {
        "step": 396,
        "accuracy_train": "0.8933201432228088",
        "rev": "workspace"
      },
      {
        "step": 397,
        "accuracy_train": "0.8934561610221863",
        "rev": "workspace"
      },
      {
        "step": 398,
        "accuracy_train": "0.8935997486114502",
        "rev": "workspace"
      },
      {
        "step": 399,
        "accuracy_train": "0.8937476873397827",
        "rev": "workspace"
      },
      {
        "step": 400,
        "accuracy_train": "0.893896758556366",
        "rev": "workspace"
      },
      {
        "step": 401,
        "accuracy_train": "0.8940339088439941",
        "rev": "workspace"
      },
      {
        "step": 402,
        "accuracy_train": "0.894163966178894",
        "rev": "workspace"
      },
      {
        "step": 403,
        "accuracy_train": "0.894299328327179",
        "rev": "workspace"
      },
      {
        "step": 404,
        "accuracy_train": "0.8944453597068787",
        "rev": "workspace"
      },
      {
        "step": 405,
        "accuracy_train": "0.8945977091789246",
        "rev": "workspace"
      },
      {
        "step": 406,
        "accuracy_train": "0.8947334289550781",
        "rev": "workspace"
      },
      {
        "step": 407,
        "accuracy_train": "0.8948788642883301",
        "rev": "workspace"
      },
      {
        "step": 408,
        "accuracy_train": "0.8950198292732239",
        "rev": "workspace"
      },
      {
        "step": 409,
        "accuracy_train": "0.8951461315155029",
        "rev": "workspace"
      },
      {
        "step": 410,
        "accuracy_train": "0.8952866196632385",
        "rev": "workspace"
      },
      {
        "step": 411,
        "accuracy_train": "0.8954126238822937",
        "rev": "workspace"
      },
      {
        "step": 412,
        "accuracy_train": "0.895540177822113",
        "rev": "workspace"
      },
      {
        "step": 413,
        "accuracy_train": "0.8956541419029236",
        "rev": "workspace"
      },
      {
        "step": 414,
        "accuracy_train": "0.8957872986793518",
        "rev": "workspace"
      },
      {
        "step": 415,
        "accuracy_train": "0.8959180116653442",
        "rev": "workspace"
      },
      {
        "step": 416,
        "accuracy_train": "0.8960731625556946",
        "rev": "workspace"
      },
      {
        "step": 417,
        "accuracy_train": "0.8962169885635376",
        "rev": "workspace"
      },
      {
        "step": 418,
        "accuracy_train": "0.8963183760643005",
        "rev": "workspace"
      },
      {
        "step": 419,
        "accuracy_train": "0.8964520692825317",
        "rev": "workspace"
      },
      {
        "step": 420,
        "accuracy_train": "0.8965981602668762",
        "rev": "workspace"
      },
      {
        "step": 421,
        "accuracy_train": "0.8967142701148987",
        "rev": "workspace"
      },
      {
        "step": 422,
        "accuracy_train": "0.8968445062637329",
        "rev": "workspace"
      },
      {
        "step": 423,
        "accuracy_train": "0.8969739079475403",
        "rev": "workspace"
      },
      {
        "step": 424,
        "accuracy_train": "0.8970954418182373",
        "rev": "workspace"
      },
      {
        "step": 425,
        "accuracy_train": "0.897201657295227",
        "rev": "workspace"
      },
      {
        "step": 426,
        "accuracy_train": "0.8973084688186646",
        "rev": "workspace"
      },
      {
        "step": 427,
        "accuracy_train": "0.8974169492721558",
        "rev": "workspace"
      },
      {
        "step": 428,
        "accuracy_train": "0.8975465893745422",
        "rev": "workspace"
      },
      {
        "step": 429,
        "accuracy_train": "0.8976767063140869",
        "rev": "workspace"
      },
      {
        "step": 430,
        "accuracy_train": "0.8977949023246765",
        "rev": "workspace"
      },
      {
        "step": 431,
        "accuracy_train": "0.8978936672210693",
        "rev": "workspace"
      },
      {
        "step": 432,
        "accuracy_train": "0.8980184197425842",
        "rev": "workspace"
      },
      {
        "step": 433,
        "accuracy_train": "0.8981432318687439",
        "rev": "workspace"
      },
      {
        "step": 434,
        "accuracy_train": "0.8982611298561096",
        "rev": "workspace"
      },
      {
        "step": 435,
        "accuracy_train": "0.898389458656311",
        "rev": "workspace"
      },
      {
        "step": 436,
        "accuracy_train": "0.8984837532043457",
        "rev": "workspace"
      },
      {
        "step": 437,
        "accuracy_train": "0.8986079096794128",
        "rev": "workspace"
      },
      {
        "step": 438,
        "accuracy_train": "0.8987410068511963",
        "rev": "workspace"
      },
      {
        "step": 439,
        "accuracy_train": "0.8988686800003052",
        "rev": "workspace"
      },
      {
        "step": 440,
        "accuracy_train": "0.899001955986023",
        "rev": "workspace"
      },
      {
        "step": 441,
        "accuracy_train": "0.8990991115570068",
        "rev": "workspace"
      },
      {
        "step": 442,
        "accuracy_train": "0.8992312550544739",
        "rev": "workspace"
      },
      {
        "step": 443,
        "accuracy_train": "0.8993203639984131",
        "rev": "workspace"
      },
      {
        "step": 444,
        "accuracy_train": "0.8994459509849548",
        "rev": "workspace"
      },
      {
        "step": 445,
        "accuracy_train": "0.8995696902275085",
        "rev": "workspace"
      },
      {
        "step": 446,
        "accuracy_train": "0.8996679782867432",
        "rev": "workspace"
      },
      {
        "step": 447,
        "accuracy_train": "0.8997864723205566",
        "rev": "workspace"
      },
      {
        "step": 448,
        "accuracy_train": "0.8999154567718506",
        "rev": "workspace"
      },
      {
        "step": 449,
        "accuracy_train": "0.9000263810157776",
        "rev": "workspace"
      },
      {
        "step": 450,
        "accuracy_train": "0.9001625180244446",
        "rev": "workspace"
      },
      {
        "step": 451,
        "accuracy_train": "0.9002816081047058",
        "rev": "workspace"
      },
      {
        "step": 452,
        "accuracy_train": "0.9003954529762268",
        "rev": "workspace"
      },
      {
        "step": 453,
        "accuracy_train": "0.9005227088928223",
        "rev": "workspace"
      },
      {
        "step": 454,
        "accuracy_train": "0.9006240963935852",
        "rev": "workspace"
      },
      {
        "step": 455,
        "accuracy_train": "0.900739848613739",
        "rev": "workspace"
      },
      {
        "step": 456,
        "accuracy_train": "0.9008573889732361",
        "rev": "workspace"
      },
      {
        "step": 457,
        "accuracy_train": "0.9009661078453064",
        "rev": "workspace"
      },
      {
        "step": 458,
        "accuracy_train": "0.9010756611824036",
        "rev": "workspace"
      },
      {
        "step": 459,
        "accuracy_train": "0.9011884331703186",
        "rev": "workspace"
      },
      {
        "step": 460,
        "accuracy_train": "0.9012945890426636",
        "rev": "workspace"
      },
      {
        "step": 461,
        "accuracy_train": "0.9014188647270203",
        "rev": "workspace"
      },
      {
        "step": 462,
        "accuracy_train": "0.901513934135437",
        "rev": "workspace"
      },
      {
        "step": 463,
        "accuracy_train": "0.9016255140304565",
        "rev": "workspace"
      },
      {
        "step": 464,
        "accuracy_train": "0.9017335176467896",
        "rev": "workspace"
      },
      {
        "step": 465,
        "accuracy_train": "0.9018502235412598",
        "rev": "workspace"
      },
      {
        "step": 466,
        "accuracy_train": "0.9019555449485779",
        "rev": "workspace"
      },
      {
        "step": 467,
        "accuracy_train": "0.9020495414733887",
        "rev": "workspace"
      },
      {
        "step": 468,
        "accuracy_train": "0.9021489024162292",
        "rev": "workspace"
      },
      {
        "step": 469,
        "accuracy_train": "0.9022411704063416",
        "rev": "workspace"
      },
      {
        "step": 470,
        "accuracy_train": "0.9023576378822327",
        "rev": "workspace"
      },
      {
        "step": 471,
        "accuracy_train": "0.9024544954299927",
        "rev": "workspace"
      },
      {
        "step": 472,
        "accuracy_train": "0.9025619029998779",
        "rev": "workspace"
      },
      {
        "step": 473,
        "accuracy_train": "0.9026703834533691",
        "rev": "workspace"
      },
      {
        "step": 474,
        "accuracy_train": "0.9027665257453918",
        "rev": "workspace"
      },
      {
        "step": 475,
        "accuracy_train": "0.9028506278991699",
        "rev": "workspace"
      },
      {
        "step": 476,
        "accuracy_train": "0.9029461145401001",
        "rev": "workspace"
      },
      {
        "step": 477,
        "accuracy_train": "0.9030681848526001",
        "rev": "workspace"
      },
      {
        "step": 478,
        "accuracy_train": "0.9031630754470825",
        "rev": "workspace"
      },
      {
        "step": 479,
        "accuracy_train": "0.9032744765281677",
        "rev": "workspace"
      },
      {
        "step": 480,
        "accuracy_train": "0.9033788442611694",
        "rev": "workspace"
      },
      {
        "step": 481,
        "accuracy_train": "0.9034688472747803",
        "rev": "workspace"
      },
      {
        "step": 482,
        "accuracy_train": "0.9035699963569641",
        "rev": "workspace"
      },
      {
        "step": 483,
        "accuracy_train": "0.903671145439148",
        "rev": "workspace"
      },
      {
        "step": 484,
        "accuracy_train": "0.903768002986908",
        "rev": "workspace"
      },
      {
        "step": 485,
        "accuracy_train": "0.9038483500480652",
        "rev": "workspace"
      },
      {
        "step": 486,
        "accuracy_train": "0.9039249420166016",
        "rev": "workspace"
      },
      {
        "step": 487,
        "accuracy_train": "0.9040051102638245",
        "rev": "workspace"
      },
      {
        "step": 488,
        "accuracy_train": "0.9040824770927429",
        "rev": "workspace"
      },
      {
        "step": 489,
        "accuracy_train": "0.9041761159896851",
        "rev": "workspace"
      },
      {
        "step": 490,
        "accuracy_train": "0.9042761325836182",
        "rev": "workspace"
      },
      {
        "step": 491,
        "accuracy_train": "0.9043554067611694",
        "rev": "workspace"
      },
      {
        "step": 492,
        "accuracy_train": "0.9044468402862549",
        "rev": "workspace"
      },
      {
        "step": 493,
        "accuracy_train": "0.9045285582542419",
        "rev": "workspace"
      },
      {
        "step": 494,
        "accuracy_train": "0.9046306014060974",
        "rev": "workspace"
      },
      {
        "step": 495,
        "accuracy_train": "0.9047051668167114",
        "rev": "workspace"
      },
      {
        "step": 496,
        "accuracy_train": "0.9047912955284119",
        "rev": "workspace"
      },
      {
        "step": 497,
        "accuracy_train": "0.9048776030540466",
        "rev": "workspace"
      },
      {
        "step": 498,
        "accuracy_train": "0.9049597978591919",
        "rev": "workspace"
      },
      {
        "step": 499,
        "accuracy_train": "0.9050392508506775",
        "rev": "workspace"
      },
      {
        "step": 500,
        "accuracy_train": "0.9051318168640137",
        "rev": "workspace"
      },
      {
        "step": 501,
        "accuracy_train": "0.9052268862724304",
        "rev": "workspace"
      },
      {
        "step": 502,
        "accuracy_train": "0.9053148627281189",
        "rev": "workspace"
      },
      {
        "step": 503,
        "accuracy_train": "0.9053894281387329",
        "rev": "workspace"
      },
      {
        "step": 504,
        "accuracy_train": "0.9054840803146362",
        "rev": "workspace"
      },
      {
        "step": 505,
        "accuracy_train": "0.905570387840271",
        "rev": "workspace"
      },
      { "step": 506, "accuracy_train": "0.90565025806427", "rev": "workspace" },
      {
        "step": 507,
        "accuracy_train": "0.905748188495636",
        "rev": "workspace"
      },
      {
        "step": 508,
        "accuracy_train": "0.9058336615562439",
        "rev": "workspace"
      },
      {
        "step": 509,
        "accuracy_train": "0.9059144854545593",
        "rev": "workspace"
      },
      {
        "step": 510,
        "accuracy_train": "0.9060183763504028",
        "rev": "workspace"
      },
      {
        "step": 511,
        "accuracy_train": "0.9060907363891602",
        "rev": "workspace"
      },
      {
        "step": 512,
        "accuracy_train": "0.9061748385429382",
        "rev": "workspace"
      },
      {
        "step": 513,
        "accuracy_train": "0.9062524437904358",
        "rev": "workspace"
      },
      {
        "step": 514,
        "accuracy_train": "0.9063463807106018",
        "rev": "workspace"
      },
      {
        "step": 515,
        "accuracy_train": "0.9064182043075562",
        "rev": "workspace"
      },
      {
        "step": 516,
        "accuracy_train": "0.9065081477165222",
        "rev": "workspace"
      },
      {
        "step": 517,
        "accuracy_train": "0.9065848588943481",
        "rev": "workspace"
      },
      {
        "step": 518,
        "accuracy_train": "0.9066889882087708",
        "rev": "workspace"
      },
      {
        "step": 519,
        "accuracy_train": "0.9067877531051636",
        "rev": "workspace"
      },
      {
        "step": 520,
        "accuracy_train": "0.9068773984909058",
        "rev": "workspace"
      },
      {
        "step": 521,
        "accuracy_train": "0.9069644212722778",
        "rev": "workspace"
      },
      {
        "step": 522,
        "accuracy_train": "0.9070423245429993",
        "rev": "workspace"
      },
      {
        "step": 523,
        "accuracy_train": "0.907119870185852",
        "rev": "workspace"
      },
      {
        "step": 524,
        "accuracy_train": "0.9071998000144958",
        "rev": "workspace"
      },
      {
        "step": 525,
        "accuracy_train": "0.9072876572608948",
        "rev": "workspace"
      },
      {
        "step": 526,
        "accuracy_train": "0.9073767066001892",
        "rev": "workspace"
      },
      {
        "step": 527,
        "accuracy_train": "0.9074490666389465",
        "rev": "workspace"
      },
      {
        "step": 528,
        "accuracy_train": "0.9075396656990051",
        "rev": "workspace"
      },
      {
        "step": 529,
        "accuracy_train": "0.9076550602912903",
        "rev": "workspace"
      },
      {
        "step": 530,
        "accuracy_train": "0.9077461361885071",
        "rev": "workspace"
      },
      {
        "step": 531,
        "accuracy_train": "0.9078185558319092",
        "rev": "workspace"
      },
      {
        "step": 532,
        "accuracy_train": "0.9079150557518005",
        "rev": "workspace"
      },
      {
        "step": 533,
        "accuracy_train": "0.9080250859260559",
        "rev": "workspace"
      },
      {
        "step": 534,
        "accuracy_train": "0.9081424474716187",
        "rev": "workspace"
      },
      {
        "step": 535,
        "accuracy_train": "0.9082441926002502",
        "rev": "workspace"
      },
      {
        "step": 536,
        "accuracy_train": "0.908315896987915",
        "rev": "workspace"
      },
      {
        "step": 537,
        "accuracy_train": "0.908398449420929",
        "rev": "workspace"
      },
      {
        "step": 538,
        "accuracy_train": "0.9084716439247131",
        "rev": "workspace"
      },
      {
        "step": 539,
        "accuracy_train": "0.908576250076294",
        "rev": "workspace"
      },
      {
        "step": 540,
        "accuracy_train": "0.9086793065071106",
        "rev": "workspace"
      },
      {
        "step": 541,
        "accuracy_train": "0.908767580986023",
        "rev": "workspace"
      },
      {
        "step": 542,
        "accuracy_train": "0.9088386297225952",
        "rev": "workspace"
      },
      {
        "step": 543,
        "accuracy_train": "0.9089210033416748",
        "rev": "workspace"
      },
      {
        "step": 544,
        "accuracy_train": "0.9090154767036438",
        "rev": "workspace"
      },
      {
        "step": 545,
        "accuracy_train": "0.9091333150863647",
        "rev": "workspace"
      },
      {
        "step": 546,
        "accuracy_train": "0.9092565178871155",
        "rev": "workspace"
      },
      {
        "step": 547,
        "accuracy_train": "0.9093694686889648",
        "rev": "workspace"
      },
      {
        "step": 548,
        "accuracy_train": "0.9094796776771545",
        "rev": "workspace"
      },
      {
        "step": 549,
        "accuracy_train": "0.909554123878479",
        "rev": "workspace"
      },
      {
        "step": 550,
        "accuracy_train": "0.9096507430076599",
        "rev": "workspace"
      },
      {
        "step": 551,
        "accuracy_train": "0.9097588062286377",
        "rev": "workspace"
      },
      {
        "step": 552,
        "accuracy_train": "0.909877598285675",
        "rev": "workspace"
      },
      {
        "step": 553,
        "accuracy_train": "0.9099975824356079",
        "rev": "workspace"
      },
      {
        "step": 554,
        "accuracy_train": "0.9101203680038452",
        "rev": "workspace"
      },
      {
        "step": 555,
        "accuracy_train": "0.9102339148521423",
        "rev": "workspace"
      },
      {
        "step": 556,
        "accuracy_train": "0.9103469848632812",
        "rev": "workspace"
      },
      {
        "step": 557,
        "accuracy_train": "0.910458505153656",
        "rev": "workspace"
      },
      {
        "step": 558,
        "accuracy_train": "0.9105759263038635",
        "rev": "workspace"
      },
      {
        "step": 559,
        "accuracy_train": "0.9106745719909668",
        "rev": "workspace"
      }
    ]
  },
  "title": "dvclive/plots/metrics/accuracy_train.tsv",
  "width": 300,
  "height": 300,
  "params": [
    {
      "name": "smooth",
      "value": 0.001,
      "bind": { "input": "range", "min": 0.001, "max": 1, "step": 0.001 }
    }
  ],
  "layer": [
    {
      "mark": "line",
      "encoding": {
        "x": { "field": "step", "type": "quantitative", "title": "step" },
        "y": {
          "field": "accuracy_train",
          "type": "quantitative",
          "title": "accuracy_train",
          "scale": { "zero": false }
        },
        "color": { "field": "rev", "type": "nominal" }
      },
      "transform": [
        {
          "loess": "accuracy_train",
          "on": "step",
          "groupby": ["rev", "filename", "field", "filename::field"],
          "bandwidth": { "signal": "smooth" }
        }
      ]
    },
    {
      "mark": { "type": "point", "tooltip": { "content": "data" } },
      "encoding": {
        "x": { "field": "step", "type": "quantitative", "title": "step" },
        "y": {
          "field": "accuracy_train",
          "type": "quantitative",
          "title": "accuracy_train",
          "scale": { "zero": false }
        },
        "color": { "field": "rev", "type": "nominal" }
      }
    }
  ]
}

@shcheklein shcheklein mentioned this issue May 11, 2023
Merged
@shcheklein
Copy link
Member

shcheklein commented May 11, 2023 •
edited
Loading

I think we need this vega/vega#3686 (+change the way we show lines a bit). + use window transform on top of the exponential avg https://stackoverflow.com/questions/55996589/how-to-layer-a-moving-average-on-line-chart-with-vega-lite

@shcheklein shcheklein mentioned this issue May 13, 2023
Merged
@dberenbaum dberenbaum mentioned this issue May 15, 2023
Open
@SoyGema
Copy link
Author

SoyGema commented Jun 18, 2023

This seems solved . Therefore closing it

@SoyGema SoyGema closed this as completed Jun 18, 2023
@shcheklein
Copy link
Member

@SoyGema hey, we've improved it, but it's not solved yet :( we need to fix the way smooth actually works via vega/vega#3686

@shcheklein shcheklein reopened this Jun 18, 2023
@shcheklein shcheklein added the blocked Issue or pull request blocked due to other dependencies or issues label Jun 18, 2023
@SoyGema
Copy link
Author

SoyGema commented Jun 18, 2023

Hey,
My apologies. Saw a Merged that looked good. :) keep it up!
Please, consider in the future a policy -the label is a great idea! thanks for adding it - to understand bug-reporting / issue impact / scope from the contributor perspective. :) Seems that the ownership/responsibility of the issue goes to Iterative team from the start, therefore I might not follow along with actions . Let me know if this hypothesis is correct.

Thanks ! Have a nice day!

@dberenbaum dberenbaum mentioned this issue Jun 19, 2023
Open
@dberenbaum
Copy link
Contributor

Opened iterative/dvc-render#135 since I don't think there's much VS Code can do.

Since we are waiting on vega/vega#3686 and it doesn't appear to be moving right now, is it worth considering other options?

The quickest fix is to move to a simple (non-exponential) moving average, which is similar to what we want except for having an unweighted, fixed window. Here's how it looks in comparison to the current smoothing (old smoothing shown by smooth; new moving average shown by rolling_window):

Screen.Recording.2023-06-19.at.4.03.08.PM.mov

Vega editor

We could also consider starting to move towards plotly (see iterative/dvc-render#7). It already has triangular moving average, which is probably close enough to exponential moving average.

@dberenbaum
Copy link
Contributor

Also note that the tensorboard example @SoyGema also looks off to me. In the last row with the max smoothing parameter, the smoothed line looks way below the actual trend of the points:

image

@mattseddon mattseddon mentioned this issue Sep 12, 2023
Closed
8 tasks
@daavoo daavoo removed their assignment Sep 29, 2023
@shcheklein shcheklein removed the blocked Issue or pull request blocked due to other dependencies or issues label Oct 3, 2023
@sroy3 sroy3 self-assigned this Oct 3, 2023
@sroy3
Copy link
Contributor

sroy3 commented Oct 18, 2023

This still hasn't propagated to Vega Lite and is blocked.

@sroy3 sroy3 added the blocked Issue or pull request blocked due to other dependencies or issues label Oct 18, 2023
@shcheklein
Copy link
Member

I think it's on us to propagate it. It seemed more or less straightforward the last time I checked and the vega-light is moving faster. Let's take a look please.

@sroy3
Copy link
Contributor

sroy3 commented Oct 18, 2023

I think it's on us to propagate it. It seemed more or less straightforward the last time I checked and the vega-light is moving faster. Let's take a look please.

In that case, I don't think I'm the person for that job as I am quite lost in all this. I would not know if I'm doing the right thing or not. I'll unassign myself.

@sroy3 sroy3 removed their assignment Oct 18, 2023
@shcheklein shcheklein removed the blocked Issue or pull request blocked due to other dependencies or issues label Oct 24, 2023
@julieg18 julieg18 self-assigned this Oct 31, 2023
@shcheklein shcheklein removed their assignment Jan 16, 2024
@shcheklein shcheklein assigned shcheklein and unassigned julieg18 Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shcheklein shcheklein

Labels
A: plots Area: plots webview, side panel and everything related bug Something isn't working priority-p1 Regular product backlog
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@dberenbaum @shcheklein @sroy3 @daavoo @go2carter @SoyGema @mattseddon @julieg18