Serializing Custom data types

Goose serializes data using taoensso/nippy. Encoding of custom data types like defrecord or Java class instances will not work out of the box. Nippy has provisions to extend serialization for such use cases. Follow these steps to extend Nippy:

• Choose a unique identifier for your data type, to serve as a bookmark for encoding and decoding.
• Convert your custom data into one of the many standard data types supported by Nippy.
• Use Nippy to encode and decode this transformed data.
• Convert the native data back to your custom data.
• Ensure that Nippy extension is initialized on both Client and Worker.

(ns my-ns
  (:require [taoensso.nippy :as nippy])
  ;; https://clojars.org/cnuernber/dtype-next
  (:import (tech.v3.datatype FastStruct)))

;;; Extending serialization for a `defrecord`.
(defrecord Person [fname lname])
(def my-person (Person. "Nick" "Bradshaw"))

(nippy/extend-freeze
  Person
  :my-ns.Person ; A unique identifier for Nippy.
  [x data-out]
  ;; Convert data into a map.
  (let [native-data {:fname (:fname x) :lname (:lname x)}]
    ;; Encode native data using nippy.
    (nippy/freeze-to-out! data-out native-data)))

(nippy/extend-thaw
  :my-ns.Person ; Reuse same identifier for decoding.
  [data-input]
  ;; Decode native data using nippy.
  (let [x (nippy/thaw-from-in! data-input)]
    ;; Create Person record from native data.
    (Person. (:fname x) (:lname x))))

;;; Extending serialization for a Java class instance.
(def faststruct-data
  (let [slots {:a 0}
        vals [1]]
    (FastStruct. slots vals)))

(nippy/extend-freeze
  tech.v3.datatype.FastStruct
  :tech.v3.datatype.FastStruct
  [x data-out]
  (->> x
       (into {})
       (nippy/freeze-to-out! data-out)))

(nippy/extend-thaw
  :tech.v3.datatype.FastStruct
  [data-input]
  (let [x (nippy/thaw-from-in! data-input)]
    (-> (FastStruct/createFactory (keys x))
        (.invoke (vec (vals x))))))

(comment
  (nippy/thaw (nippy/freeze my-person))
  (nippy/thaw (nippy/freeze faststruct-data)))

If you are unable to serialize your custom data types, feel free to create a query/usage issue on either Nippy or Goose's Github.
For immediate support, ping us on #goose @Clojurians slack.

---

Previous: Specs        Next: Production Readiness