Integration test fails even in clean checkout

[SethTisue]

sbt:scala-yaml> integration/test
org.virtuslab.yaml.K8sConfigSpec:
==> X org.virtuslab.yaml.K8sConfigSpec.initializationError  0.001s java.lang.NullPointerException: null
    at org.virtuslab.yaml.YamlRunnerSpec.<init>(YamlRunnerSpec.scala:14)
    at org.virtuslab.yaml.K8sConfigSpec.<init>(K8sConfigSpec.scala:3)
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/9DXL.tml
Events differ - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/RXY3.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/M5DY.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/SF5V.tml
Error in test - /Users/tisue/scala-yaml/integration-tests/target/scala-2.13/test-classes/yaml/test-suite/Q9WF.tml
...

this came up in the context of scala/community-build#1713

[SethTisue]

I think this was missed until now because the GitHub Actions config in this repo only runs core/test and does not run integration/test cc @tgodzik

[tgodzik]

I will take a look, thanks!

[lbialy]

@kpodsiad hey Kamil, while you're available - what was the highest percent of green tests in this suite, ever? I've seen 44% green in your older PR but current main is around 40%.

[kpodsiad]

@lbialy sorry, I don't remember. But some heads up - they yaml that almost everyone on daily basis is only a small fraction of yaml spec, spec itself is enormous and contains things that no sane person would use like #24. It might be good idea to determine subset of currently available tests / cases which needs to supported and leave the rest as a fun fact, something maybe nice to have, but absolutely non necessary for 99% of usecases.

[lbialy]

We should track the percent covered by the suite I guess.

[lbialy]

Uh, main today is at 36.65% :/

[lbialy]

And k8s spec is at 62.60%

[lbialy]

no improvement after merge of all bugfixes 😞 😭

[lbialy]

I have added outputYaml checking in official yaml test suite and now it's 14.60% from 36.65% ...