World description DSL

[byorgey]

DSL for programming worlds, towards #1320 and #29 (and, indirectly, toward #50, since the world DSL should make a nice target for world saves) . Eventually this should be able to recreate all the world description/building features we have, though there is still a long way to go. But currently we can at least recreate the "classic" procedurally-generated world. I think this is a solid foundation we can merge as a first step, and then work on adding more features in subsequent PRs. Below are some notes that should help in reviewing. Note that the large number of files changed is due in large part to the elimination of the default field in scenario descriptions; see the "changed files" section below for an overview of the important/interesting changes.

Issues split off from this one: #1394 #1395 #1396 #1397

Major changes

• New data/worlds subdirectory
  • All .world files are parsed at load time and saved in a WorldMap which gets threaded through, similar to EntityMap (perhaps we should think about passing around a single record instead)
• Standard "classic" world
  • Used to be testWorld2, defined in Haskell code; now it is defined via the DSL in worlds/classic.world. This should make it much easier to experiment with variations.
  • We can now automatically extract entities mentioned in a world DSL term with extractEntities. There used to be an explicit list in testWorld2Entities, used to check pedagogy, generate documentation, etc., but it turns out it had (predictably) gotten out of date! This can't happen anymore.
  • It is now referenced in several tutorials (backstory, farming, world101, speedruns, etc.)
• The default field of world descriptions is no more: one can use dsl to just specify a constant
  • Note in Swarm.Game.State, dslWF and arrayWF are combined using the Monoid instance to create wf.
• Erasable
  • It used to be the case that if some kind of default terrain + entity was specified (e.g. stone + water), any map would completely override the default. However, we want to move towards combining everything with a Monoid instance. But by default this means the default entity would show through anywhere the map did not specify an entity. So we need a way to explicitly "erase" an entity from a lower layer.
  • If e is a Semigroup, then Maybe e is a Monoid where Nothing acts as an identity element. Likewise, Erasable e is a Monoid but adds two new elements: ENothing to be an identity, and EErase to be an annihilator. i.e. combining with EErase is like multiplying by zero.
  • We can now specify erase as an entity to override entity underneath.
  • There are several Haskell files with only changes related to Erasable, relating to e.g. the world editor, PCells, etc.; I'm not 100% sure I've always done the right thing here.

DSL overview

• Integer, float, and Boolean literals. Note that 3 is always an int, and 3.0 is a float. It makes things much easier to not have to deal with 3 possibly being either int or float, though it does make things slightly more annoying for programmers.
• Standard boolean, arithmetic, and comparison operators
• if ... then ... else ...
• <> operator for combining via Semigroup instance
• Cell literals are enclosed in curly braces. Unlike the previous awkward world description syntax with one, two, or three-element lists denoting terrain, terrain + entity, or terrain + entity + robot, there can now be any number of elements in any order.
  • {foo} will be resolved as either terrain, an entity, or a robot, whichever is successful. So if the names are unambiguous one can just write {tree} or {stone}.
  • It is possible to explicitly indicate the type of cell value with syntax like {entity: tree} or {terrain: stone}.
  • Multiple items separated by commas is syntax sugar for combining with <>. e.g. {tree, entity: boulder, stone} = {tree} <> {entity: boulder} <> {stone}.
• Ability to refer to the seed
• Refer to the current x or y coordinates or the hash of the current coordinates
• let-expressions for multiple variables: let x1 = e1, x2 = e2, ... in ...
• overlay [e1, e2, ...] layers e1 on the bottom, e2 on top of that, etc., using the Semigroup instance for world functions
• "foo" imports the DSL term in worlds/foo.world
• perlin function to generate perlin noise
• mask function to mask with a condition

Changed files

• Swarm.Util: moved the acquire function here and gave it a more descriptive name.
• Swarm.Doc.Gen: can now extract mentioned entities directly.
• Swarm.Game.Failure: added new failure modes
• Swarm.Game.Scenario.Topography.WorldDescription: get rid of defaultTerrain field, add worldProg for DSL.
• Swarm.Game.State: see comment.
• Swarm.Game.World: a bit of reorganization. Added a bunch of modules under this.
  • Swarm.Game.World.Coords: moved some code here from Swarm.Game.World.
  • Swarm.Game.World.Gen: moved some things here from Swarm.Game.WorldGen (also deleted a bunch of irrelevant code), and also added the extractEntities function to get all entities mentioned by a DSL term.
  • Swarm.Game.World.Syntax: raw, untyped syntax for world DSL terms.
  • Swarm.Game.World.Parse: parser for world DSL terms. Fairly standard.
  • Swarm.Game.World.Typecheck: takes raw, untyped terms produced by the parser and both typechecks and elaborates them into a simpler core language. An interesting feature is that the core language is type-indexed, so that the Haskell type system is actually ensuring that our typechecker is correct; every typechecked world DSL term value has a type which is indexed by a Haskell type corresponding to the type of the underlying DSL term. For example, {entity: tree} would have a type like TTerm [] (World CellVall) etc. Once terms make it through the typechecker, there cannot possibly be any bugs in the rest of the pipeline which would result in a crash, because the Haskell type system. (There could of course be semantic bugs.) Understanding exactly how the typechecker works is not too important. Of interest may be the resolveCell function, which determines how we decide what Cell is represented by a cell expression in curly braces.
  • Swarm.Game.World.Abstract: compile elaborated, typechecked world DSL terms down into an extremely simple core language with only constants and function application. This gives us very fast evaluation of world DSL terms. Understanding this module is not really necessary but there is a link to a blog post for those who are interested in how it works.
  • Swarm.Game.World.Compile: a further processing/compilation step after Swarm.Game.World.Abstract. Currently we don't actually use this, since it doesn't seem like it makes a big efficiency difference.
  • Swarm.Game.World.Interpret: interpreter for abstracted world DSL terms.
  • Swarm.Game.World.Eval: just puts together the pieces of the pipeline to evaluate a typechecked world DSL term.
  • Swarm.Game.World.Load: just loading world DSL terms from disk.

[xsebek]

@byorgey I will try to take a look at the code later. I like how the DSL looks nice and clean from your description though. 👍

Could you maybe copy the DLS description from the PR notes to some README?

[kostmo]

Fantastic! This opens up a lot of possibilities for scenario design.

Agree with @xsebek that the PR description is good and as much of it as possible should be copied into Haddocks.

[xsebek]

Impressive, I look forward to playing with this. 👍 🚀

[xsebek]

🗺️ 🥳