YAML support? #23
Comments
|
Thanks for your thorough analysis! In TOML, you can write inline tables such as: module = {
sharedCommon = {
root = "shared_common",
# ...
test = {
# ...
}
}
}With this syntax, the example is quite similar to your YAML file. I chose TOML because it has a simple specification and is used by other build tools like Cargo. I fear that having an additional configuration format will burden development, as we would have to offer the same codecs twice and make sure errors are handled in the same way. Also, there has to be a native Scala implementation such that we can support Scala Native in the future. As for Regarding #18, this will get implemented soon after I am done with some other improvements and once #17 has been merged in. |
|
Are you sure that's valid? The docs say
|
|
Have you considered using HOCON? That has the advantage of having ways to DRY things, as well as built in includes, support for environment variables, and other really useful features. And there are pure scala implementations: https://github.com/akka-js/shocon and https://github.com/ekrich/sconfig |
It appears that the TOML library implements a language extension that allows multi-line tables. The following build file loads fine: project = {
scalaVersion = "2.13.0",
scalaJsVersion = "0.6.28",
scalaOptions = [
"-encoding",
"UTF-8",
"-unchecked",
"-deprecation"
],
testFrameworks = ["minitest.runner.Framework"]
}
module = {
example = {
root = "shared",
sources = ["shared/src"],
targets = ["js"],
js = {
root = "js",
sources = ["js/src"]
},
test = {
js = {
jsdom = true,
sources = ["js/test"],
scalaDeps = [["io.monix", "minitest", "2.5.0"]]
}
}
}
}HOCON looks like a promising contender. I would have preferred if they offered fewer syntactic features to express the same concept, but the format is still much simpler than YAML. As I see it, HOCON has three main advantages over TOML which are JSON compatibility, value substitutions and access to environment variables. Alternatively, we could use JSON directly. JSON has the advantage that there are decent Scala libraries, is in wide use and can be easily manipulated with tools like If you would like to implement additional support to load HOCON or JSON configuration files, I'd be happy to provide some guidance. |
|
I'm playing with this locally. I basically just converted the circe HOCON and JSON could be done just by using circe-core or https://github.com/circe/circe-config instead of circe-yaml. |
|
That's a nice solution since it acts directly on the AST level, so we get to re-use all of the TOML codecs. However, I'd prefer if we used HOCON (e.g. via sconfig or circe-config) since this format is more common in the JVM ecosystem and has a simpler specification. |
|
I can do that when it comes to making a PR, like I said it doesn't change
much about the approach. However I would want to experiment with using
hocon first, however my experimenting is now kind of blocked on something,
which I opened a different issue for (how to go about compiling twirl
templates).
…On Tue, Aug 20, 2019, 6:04 AM Tim Nieradzik ***@***.***> wrote:
That's a nice solution since it acts directly on the AST level, so we get
to re-use all of the TOML codecs. However, I'd prefer if we used HOCON
(e.g. via sconfig or circe-config) since this format is more common in the
JVM ecosystem and has a simpler specification.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#23?email_source=notifications&email_token=AAAYAUBNHXTZU7KIX3A3LSTQFO6SPA5CNFSM4IDT7MP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4VYMPQ#issuecomment-522946110>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAYAUEONR6BAFVWNW3D4C3QFO6SPANCNFSM4IDT7MPQ>
.
|
|
My personal opinion: TOML is the way to go, friendly, simple and easy-to-learn. I'd see supporting YAML as a regression: The simpler this tool is, the better. That means, less configuration and more opinionated workflows (IMO) 😄 |
|
How heavily have you used Seed?
…On Thu, Sep 12, 2019, 5:47 AM Jorge ***@***.***> wrote:
My personal opinion: TOML is the way to go, friendly, simple and
easy-to-learn. I'd see supporting YAML as a regression:
TOML and YAML both emphasize human readability features, like comments that make it easier to understand the purpose of a given line. TOML differs in combining these, allowing comments (unlike JSON) but preserving simplicity (unlike YAML).
The simpler this tool is, the better. That means, less configuration and
more opinionated workflows (IMO) 😄
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#23?email_source=notifications&email_token=AAAYAUCWBJAIL76L4HX6SDTQJIF2VA5CNFSM4IDT7MP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6RJ6PQ#issuecomment-530751294>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAYAUE5YXAUPDSQFPITOELQJIF2VANCNFSM4IDT7MPQ>
.
|
|
Let me propose two more languages: DhallDhall appears to be a good fit for a build tool since it allows users to define 'variables' and functions. Furthermore, the language was designed not to be Turing-complete. The authors claim:
Dhall could allow us to avoid boilerplate as in #48. It supports importing Dhall code from URLs, so we can provide default configurations for common generators (such as Twirl templates or Play routes). Another advantage over the other formats so far is that it is fully typed. I experimented with it a bit here: https://github.com/tindzk/seed-dhall-experiments/ JsonnetJsonnet is already popular in DevOps. Its syntax is more approachable than Dhall's and there is a even a recent implementation for Scala. It comes with a standard library that contains common functions like Unlike Dhall it is not statically typed, but since we already perform type checking within Seed, I do not think this is a problem. ConclusionBoth languages look promising. They both generate JSON, so I think the best way forward is to add JSON support to Seed. While trying out Dhall, I did notice that our current schema is not the best fit for JSON, so we will need to optimise it without breaking compatibility with existing TOML builds. |
|
I really like how simple https://github.com/tindzk/seed-dhall-experiments/blob/master/build.json reads 👍 Both of those languages look a little bit complicated IMO, the benefit of using something like TOML and JSON out-of-the-box is that people should be more familiar with that syntax than Dhall's or Jsonnet's. I personally believe JSON is the way to go, trivially all JS build tools use it and they have proven how comfortable using JSON can be as configuration for any tool. However, if we want to be as similar to cargo as possible we should just continue using TOML and work on adding good VS Code support with completions. Possibly create a language server.
After giving it more thought, I'm no longer sure something like Dhall or Jsonnet would make the build tool easy to use for users. Edited: Changed my opinion on whether Dhall/Jsonnet is a good way forward, they seem too niche and too flexible. I would go for something simpler for which we can easily create editor support and that has a lower cognitive overhead (people should be able to change settings using their intuition instead of checking the docs). |
|
What JS build tools use it? There's package.json but that's usually very
simple. Webpack uses an actual JS file, not JSON (although typically it
consists mostly of JSON).
Almost all devops tools use YAML. Off the top of my head, almost all CIs,
docker-compose, kubernetes, ansible and salt.
If you've never used YAML before I admit there's a bit of a learning curve,
and some of the more advanced features like reuse features also have to be
learned, but at the end of the day it's the least noisy and results in the
simplest files.
Anyway it's great to experiment but can we discuss what are the metrics for
determining success?
Personally my goal would be what build file is the easiest to maintain,
which means easy to read, and as little code as possible.
…On Thu, Sep 12, 2019 at 8:53 AM Jorge ***@***.***> wrote:
I really like how simple
https://github.com/tindzk/seed-dhall-experiments/blob/master/build.json
reads 👍 Both of those languages look a little bit complicated IMO, the
benefit of using something like TOML and JSON out-of-the-box is that people
should be more familiar with that syntax than Dhall's or Jsonnet's.
I personally believe JSON is the way to go, trivially all JS build tools
use it and they have proven how comfortable using JSON can be as
configuration for any tool. That being said, if you believe something like
Dhall might be a good fit, I'm good with that: what's important for me is
that it generates JSON. However note that at the moment you say that you
can no longer call seed simple, at least from an aesthetics POV.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#23?email_source=notifications&email_token=AAAYAUBI2VR5Z6ZVYGFS2OTQJI3TRA5CNFSM4IDT7MP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6RYRXY#issuecomment-530811103>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAAYAUCFCZ3A2VHNXITIJRLQJI3TRANCNFSM4IDT7MPQ>
.
|
|
@tindzk I have edited my comment above to reflect some of my thoughts after sleeping on the Jsonnet/Dhall's proposal. I also left you some messages in Gitter in case you wanna chat 😄. |
|
@tindzk My Scala port of I think when you get down to enterprise grade configuration, value substitutions are a big deal. The library has a lot of history which could be considered legacy but also could be considered battle tested. I think the only user I have is |
|
Are we any closer to a decision here? |
|
For one thing I find IntelliJ's TOML support quite lacking... 😞 |
|
Sorry for the radio silence and thanks for all your suggestions! After discussing this issue more with @jvican, we came to the conclusion that the best option is to only support TOML for now. I do agree with some of the issues raised here, but feel that overall TOML is the most satisfying solution in terms of readability and simplicity. For the missing features like variables, I would be happy to implement them as language extensions in toml-scala. If the motivation for changing the configuration format is to avoid boilerplate, I propose we first explore how the schema could be improved. For instance, Play projects are not as well-supported as in sbt, but we can introduce abstractions to handle them better. If TOML support in IntelliJ is lacking, it would be best to improve the plug-in. This will benefit other projects too. Features particularly useful for Seed would be to enforce the schema and to auto-complete dependencies (see also #60). At the moment, my focus is to finalise Seed's core functionality (publishing artefacts, running test suites, Play support etc.) Afterwards, we can consider more elaborate use cases such as additional configuration formats. |
I really like Seed (although I can't use it without #18), however I'm not sure I can use it with TOML. Especially with #18 and #19 the configuration forms a deep tree, and especially with many modules (which the README says it's designed for) it can be very wide too. TOML is nice for some things but I can't see it scaling well for me.
As an example, GitLab CI uses TOML for configuring the runner binary, but YAML for defining a project's various jobs and their configuration, and I think they both make sense. The former is basically a few sections of flat config, with a bit of nesting. The latter is a much more complicated tree of settings. You can have many jobs, each of which can have many settings, each of which might itself need to be defined with sub-properties.
I admit the TOML generated by
seed initoverwhelmed me more than it might have, because some of what it generated may have been unnecessary.I found something online to convert TOML to YAML and was much happier with the result. So what I started doing is using
yaml2toml(on Ubuntu,sudo snap install marshal) to translate my config for seed. Specifically, I ranecho seedbuild.yml | entr -rs "yaml2toml --preserve-key-order seedbuild.yml build.toml && seed bloop && bloop compile -w (bloop projects)"(this will run the part in quotes wheneverseedbuild.ymlchanges via my shell, Fish).I didn't get all that far in converting my decently-sized sbt project to seed (I stopped mainly due to #18), but here is how the two ways of defining it compare:
TOML
YAML
The text was updated successfully, but these errors were encountered: