Migration guide 1.5 -> 2.0

[andreasabel]

The changelog isn't explicit about how to migrate from aeson-1.5 to -2.0.
Please add a migration guide with the most common fixes needed.

[phadej]

it's a changelog, not a migration guide. You can contribute your observations here.

You can pick ideas from

• haskell-github/github@dd3dfbc
• phadej/aeson-optics@ff31ca0
• clash-lang/clash-compiler@55ec75d

In summary, change import qualified Data.HashMap.Strict as HM to import qualified Data.Aeson.KeyMap as KM. Apply CPP to your liking.

[andreasabel]

Here is one more: agda/agda@8037346

[tfausak]

So essentially the official advice is "good luck"? That's disappointing.

Here's my summary of the changes and what you'll need to do to support them:

• Change Object to use an opaque KeyMap interface #866: Introduced the Data.Aeson.KeyMap.KeyMap type and replaced any HashMaps with KeyMaps. If your project depends on Aeson's Object type being a HashMap, then you'll have to update your code to use the new KeyMap instead. Since the interfaces are similar, usually you can use CPP to import either Data.HashMap.Strict or Data.Aeson.KeyMap depending on the version of Aeson.

• Data.Aeson.Key #868: Introduced the Data.Aeson.Key.Key type and replaced any Text keys with Keys. If your project depends on keys being Text, for example if you use the pair function with something other than an overloaded string, you'll have to update your code to use the new Key representation instead. Since Key's interface is minimal, you'll probably have to introduce fromKey and toKey functions that are defined with CPP so that you can continue to work with Text values and convert to Keys as late as possible.

• Drop GHC-7 support #872: Dropped support for GHC 7.10 and earlier. If you use an affected version of GHC, you can't upgrade to Aeson 2.

• Add ordered keymap option #871: Added the ordered-keymap flag. When set, this flag changes the representation of KeyMap from HashMap to Map. This is the change that will prevent a hash flood attack. The flag is not enabled by default with Aeson 2.0.0.0, but it is enabled by default with version 2.0.1.0. If it's important that you avoid hash flood attacks, you should explicitly enable this flag!

• Change +inf/-inf :: Double/Float serialization #873: Changed representation of infinite floating point values. Previously they were represented as null, but now they are "+inf" for positive infinity and "-inf" for negative infinity. This probably does not require any changes in your code, but be aware that JSON generated by Aeson may be different than before when it includes infinite floating point values.

• Accept all values in FromJSON () and FromJSON (Proxy tag) #874: Changed FromJSON instances for () and Proxy a to accept any value. Previously they only accepted an empty array. This probably does not require any changes in your code, but be aware that JSON parsed by Aeson will now accept other values for these types.

• If you do anything non-trivial with Aeson, you're probably going to need CPP (or something equivalent to it) in order to support these changes.

• Since this is a super major version upgrade (from 1.5.x.y to 2.0.x.y), be sure to get your version bounds correct! Don't do >= 1.5 && < 2.1, which would allow a hypothetical version 1.6, which would have breaking changes. Instead do >= 1.5 && < 1.6 || >= 2.0 && < 2.1, or something equivalent to it like ^>= 1.5 || ^>= 2.0.

[Bodigrim]

So essentially the official advice is "good luck"? That's disappointing.

Could we please stop bashing maintainers for doing free work? @phadej fixed the vulnerability (together with @Boarders), made a release, and marked this issue as "help wanted". I'm not sure what exactly is disappointing here.

[googleson78]

While I do strongly agree with the sentiment of "let's not bash people doing free labour", I'd also like to express thanks to @tfausak for the write-up. This was not expressible in a clear way with emojis, so I wrote a comment instead.

[frasertweedale]

Here's my diff for jose: frasertweedale/hs-jose@ffc0cde

Apart from updating Text -> Key there are some places where I filter maps by key, which required a bit of explicit converting between Map Text <-> KeyMap (these have the same representation with +ordered-keymap, so should be zero runtime overhead) Overall it was a non-trivial but definitely not painful update.

[googleson78]

So it might be worth extending the KeyMap interface with a filtering mechanism?

[frasertweedale]

@googleson78 not necessarily. How to filter efficiently can depend on the underlying representation (e.g. if you want to keep/drop keys in an existing Set or HashSet).

[jneira]

I would like to note that a possible way to make the code work across bot Map implementations could be use aeson lens like operators or directly lens combinators with the help of lens-aeson, if you are comfortable with lenses and are already using them.
I learnt about that from the wise @michaelpj who added support for lsp without any cpp:

• Aeson 2 compatibility lsp#360

-addNullField :: Text -> Value -> Value
-addNullField s (Object o) = Object $ HM.insertWith (\_new old -> old) s Null o
+addNullField :: String -> Value -> Value
+addNullField s (Object o) = Object $ o <> fromString s .= Null

• And helped to do it in hls: Add support for brittany (needs aeson-2) and floskell with ghc-9.0.1 haskell-language-server#2551 with neat code like

-  A.Object $
-    HMap.adjust
-      ( \(unsafeValueToObject -> o) ->
-          A.Object $ HMap.insert "plugin" elems o -- inplace the "plugin" section with our 'elems', leaving others unchanged
-      )
-      "haskell"
-      (unsafeValueToObject (A.toJSON defaultConfig))
+ -- Use 'ix' to look at all the "haskell" keys in the outer value (since we're not
+ -- setting it if missing), then we use '_Object' and 'at' to get at the "plugin" key
+ -- and actually set it.
+ A.toJSON defaultConfig & ix "haskell" . _Object . at "plugin" ?~ elems

Another example:

- "changes" `HM.member` editParams @? "Contains changes"
- not ("documentChanges" `HM.member` editParams) @? "Doesn't contain documentChanges"
+ (editParams & has (ix "changes"))  @? "Contains changes"
+ not (editParams & has (ix "documentChanges")) @? "Doesn't contain documentChanges"

[phadej]

@jneira except we'll probably change (or break if you prefer that term) lens-aeson/lens-optics later on, when aeson-2.0 catches up. See lens/lens-aeson#37 (comment)

EDIT: that example will work as at is overloaded, and "plugin" is OverloadedStrings literal; but still, using lens-aeson is not a fail proof solution.

[jneira]

Well at least we have a lens-aeson bridge version with support for pre and post 2.0, so we can leverage it to avoid cpp in client code for now (with a little bit of luck cpp will not needed later :-) )

[bergmark]

@phadej from my perpective, the changelog is a perfectly fine place to put a migration guide along with the changes. Let me emphasize that I don't exptect you to do the work of writing it. If someone else did the work, would you accept it into the changelog, or is there another place where you would find it more suitable?

[arobertn]

I'd just like to point out that many of the excellent suggestions provided here by @tfausak and others are in fact directed towards developers interested in building in both aeson-1 and aeson-2 environments. If you just want to migrate from 1.x to 2.x, like the title of this issue suggests, it is much easier. Switching out HashMap for KeyMap and utilizing the Key.{from,to}{String,Text} methods handles the lion's share. (KeyMap could use a ! method for more complete compatibility but that's not a major inconvenience.) No CPP needed!