Add syntax for array of hashes.

[mojombo]

I'd like to propose this syntax for the creation of arrays of hashes. The mnemonic is an array (outside set of brackets) of hashes (inside set of brackets). Get it?

[[products]]
name = "Hammer"
sku = 738594937

[[products]]

[[products]]
name = "Nail"
sku = 284758393
color = "gray"

In JSON land, that would give you the following structure.

{
  "products": [
    { "name": "Hammer", "sku": 738594937 },
    { },
    { "name": "Nail", "sku": 284758393, "color": "gray" }
  ]
}

[mojombo]

Note: after looking through #50, this proposal still has the weakness of not accommodating nested hashes. cc @tnm.

[DouweM]

I like the syntax better than those proposed in #50, but that is a pretty big weakness.

[HersonHN]

A little bit confusing if you want to define nested arrays/hashes/arrays

{
    "artists": [
        {
            "name": "Frank Sinatra",
            "albums": [
                { "year": 1957, "title": "Come Fly Whit Me" },
                { "year": 1958, "title": "Come Dance With Me" },
                { "year": 1961, "title": "Swing Along With Me" }
            ]
        }, 
        {
            "name": "The Beatles",
            "albums": [
                { "year": 1969, "title": "Yellow Submarine" },
                { "year": 1969, "title": "Abbey Road" }
            ]
        }
    ]
}

[tnm]

The nested hashes thing is a problem. Going to think.

[BurntSushi]

Also, we should clarify the type of a hash. Is it just hash? Or more like a named tuple?

[rossipedia]

Are we operating under the requirement that any particular hash in an array of hashes can be composed of whatever it wants? We're not enforcing homogeneous hashes inside of arrays, correct?

[BurntSushi]

@rossipedia that's precisely what I'm asking :-) I suspect that having homogeneous hashes is too restrictive. Because the values must all be the same type. And it doesn't fit with the rest of TOML at all.

There are only two other alternatives I can think of.

The first is to declare complete heterogeneity like array is now and call its type hash. (In all honestly, it's probably how anonymous hashes should work too.)

The second alternative is to use a named tuple (a record type), then the semantics of an anonymous hash would be precisely the same as tuple, except the type would no longer be constrained by order. But there are dragons here. (In a hash all keys are optional, but that's not necessarily true in a record.) We can flesh them out if anyone is serious about pulling this thread, but I think we ought to just throw our hands up and claim the type as hash.

[rossipedia]

Agreed. My personal feeling on the spirit of TOML is to strive to keep things as simple as is possible, but without it being painful for use in real-world scenarios.

Not necessarily the easiest thing to do :)

[mbleigh]

No reason this syntax can't mostly accommodate nested hashes:

[outer.[inner]]
abc = 123

[outer.[inner]]
abc = 234

{
  "outer": {
    "inner": [{"abc":123},{"abc":234}]
  }
}

[DouweM]

@mbleigh The issue isn't with arrays in nested hashes, it's with nested hashes in arrays.

[pygy]

You could use a combinaision of arrays and tuples to represent the same data structures:

hash = ( ("keys"), [(values)] )

Example:

artists = (("name","albums"),
          [
            (
              "Frank Sinatra", 
              (("year", "title"),
              [ 
                (1957,  "Come Fly With Me"),
                (1958,  "Come Dance With Me"),
                (1957,  "Sing Along With Me") 
              ]
            )),
            (
              "The Beatles", 
              (("year", "title"),
              [
                (1969,  "Yellow Submarine"),
                (1969,  "Abbey Road") 
              ]
            )),
          ]

It is not very practical, though. The main drawback is that (assuming type homogeneity in arrays), if you want to add a key in a single dictionary, you must add it everywhere else, and add empty values in all corresponding fields.

[rossipedia]

Yeah, there's a lot of noise going on there. Not intending to be rude at all here, but if I wanted to maintain that kind of syntax, I'd just use JSON.

[ambv]

@rossipedia is unfortunately right. If you're coming up with a confusing sorta-like-INI sorta-like-JSON format, people will stick to the known predecessors. Or YAML really.

Don't go the SOAP route (S meant Simple once, you know). Pretty quickly you'll have to change the meaning of O in TOML from Obvious to Original.

[pygy]

I agree with both of you, it was just to show that it is possible (you can also do it with the current spec, but it is even more awkward, because each value must be enclosed in a single element array.

The most practical way to do this, IMO, would be to use a JSON-like syntax for hashes inside arrays: use braces as delimiters, but keep the TOML key = value\n pair syntax.

I don't think that looking like JSON is a liability.

[foo]
bar = [ 
    {
       key = 5
       key2 = "str"
    }, {
       key = ...
    }
]

[BurntSushi]

If we must have anonymous hashes (I don't think TOML needs them), then I like @pygy's syntax best.

[tnm]

Going through the various proposals for nested hashes — as good and even reasonable as they are — none of them feel both simple and obvious to me. If this was a full-blown data interchange format, I'd considered the lack of nested hashes a major weakness. But TOML is not that; it's a config format. Config formats ought to be simple, easy-to-read by sleepy ops people, and completely obvious. So after a good deal of thought, I'm ok with not being 100 percent complete in representing all the primitives in all contexts, in order to avoid the complexity that I think we'll see here.

[RichardVasquez]

I'm a little confused here with regards to my implementation in progress, specifically regarding nested hashes and keyname groups. I'm going to ask two a two parter, one as I understand it, then my problem.

Let's say my TOML file is something like this:

[[a]]
a=1
[[a]]
[[a]]
b=2

This should give me something like this, yes?

{
  "a" : [
    {"a": 1},
    {},
    {"b": 2}
  ]
}

Now let's say I add:

[a.b]
c=3

to my TOML file after the above original. What should my resulting JSON (or other format) look like then?

[pygy]

@tnm, what' not obvious and simple about my proposal?

[mojombo]

I believe we're overthinking this. I've been playing around with converting some real world complex config files to see what feels natural. Consider the following:

  # php/fastcgi
  [[http.server]]
    listen = 80
    server_name = ["domain1.com", "www.domain1.com"]
    access_log = ("logs/domain1.access.log", "main")
    root = "html"

    [http.server.upstream]
      server = ("127.0.0.3:8000", 5)

    # serve static files
    [[http.server.location]]
      match = "\.php$"
      fastcgi_pass = "127.0.0.1:1025"

    # pass requests
    [[http.server.location]]
      match = "\.php$"
      fastcgi_pass = "127.0.0.1:1025"

  # reverse proxy
  [[http.server]]
    listen = 80
    server_name = ["domain2.com", "www.domain2.com"]
    access_log = ("logs/domain2.access.log", "main")

    [[http.server.location]]
      match = "^/(images|javascript|js|css|flash|media|static)/"
      root = "/var/www/virtual/big.server.com/htdocs"
      expires = "30d"

Why not just consider the order in which nested keygroups appear to be important? Order of lines in TOML is already important (keys belong to the keygroup above them), so we can simply extend that to the notion of nested keygroups.

Specifically, if a nested keygroup appears, then it belongs to the last keygroup that was pushed to the keygroup array. Here's a condensed example and corresponding JSON:

[[fruit]]
  name = "apple"

  [fruit.physical]
    color = "red"
    shape = "round"

  [[fruit.variety]]
    name = "red delicious"

  [[fruit.variety]]
    name = "granny smith"

[[fruit]]
  name = "banana"

{
  "fruit": [
    {
      "name": "apple",
      "physical": {
        "color": "red",
        "shape": "round"
      },
      "variety": [
        { "name": "red delicious" },
        { "name": "granny smith" }
      ]
    },
    {
      "name": "banana"
    }
  ]
}

This approach makes it easy to understand what's going on, is easy to write, and should be simple enough to parse.

[DouweM]

Makes sense, 👍

[jprichardson]

I really like the double brackets syntax proposal.

[polymetis]

I quite like the syntax proposed here as well.

[rossipedia]

👍

[gamedevsam]

This syntax works, I need this feature. Is this ongoing? I still have to implement it in my parser.

[88Alex]

The only problem with arrays of hashes is that they might be sort of hard to implement in parsers.

[88Alex]

And, about the defining-every-key-value-pair-in-an-array-of-hashes problem, TOML can auto-initialize undefined keys: 0 for numeric values, false for booleans, and "" for strings.

The only problem is: what about date-time??

[skull-squadron]

Late 👍 The importance of Tom's approach is that right-hand values don't setup camp in your living room and proceed to invite their friends over. This keeps the syntax of TOML flatter and more like airbnb.

[kevinburke]

Any update on the status of this branch?

[mjemmeson]

The fruit example doesn't seem to clarify what happens with further nested levels.

What should happen with [[fruit.physical.foo]] or [[fruit.variety.bar]] ?

The following?

{ fruit: [
    {
        name: "apple",
        physical: {
            color: "red",
            shape: "round",
            foo: [ ... ]
        },
        variety: [
            { name: "red delicious" },
            { name: "granny smith", 
              bar: [ ... ]
            }
         ]
     }
]
}

The first seems logical, the second maybe less so...