Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: JSON decoder / encoder fns #403

Closed
wants to merge 5 commits into from
Closed

Feature: JSON decoder / encoder fns #403

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
373fb5f
Update malli to latest version
cap10morgan Feb 28, 2023
b5cdb5d
Update deps to latest compatible versions
cap10morgan Feb 28, 2023
ffbff42
Add transaction & query result schemas too
cap10morgan Feb 28, 2023
775623e
Swap expected / actual values in reverse-query-test ns
cap10morgan Feb 28, 2023
b686623
Make ::filter & ::bind non-recursive
cap10morgan Mar 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions deps.edn
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,14 @@
byte-streams/byte-streams {:mvn/version "0.2.4"}
cheshire/cheshire {:mvn/version "5.11.0"}
instaparse/instaparse {:mvn/version "1.4.12"}
metosin/malli {:mvn/version "0.9.2"}
metosin/malli {:mvn/version "0.10.1"}
com.fluree/json-ld {:git/url "https://github.com/fluree/json-ld.git"
:sha "a909330e33196504ef8a5411aaa0409ab72aaa35"}

;; logging
org.clojure/tools.logging {:mvn/version "1.2.4"}
ch.qos.logback/logback-classic {:mvn/version "1.4.5"}
org.slf4j/slf4j-api {:mvn/version "2.0.5"}
org.slf4j/slf4j-api {:mvn/version "2.0.6"}

;; Lucene
clucie/clucie {:mvn/version "0.4.2"}
Expand Down Expand Up @@ -44,8 +44,8 @@

:aliases
{:build
{:deps {io.github.clojure/tools.build {:git/tag "v0.8.5"
:git/sha "b73ff34"}
{:deps {io.github.clojure/tools.build {:git/tag "v0.9.3"
:git/sha "f37d64e"}
slipset/deps-deploy {:mvn/version "0.2.0"}}
:ns-default build}

Expand All @@ -54,12 +54,12 @@
:extra-deps {org.clojure/tools.namespace {:mvn/version "1.3.0"}
criterium/criterium {:mvn/version "0.4.6"}
figwheel-sidecar/figwheel-sidecar {:mvn/version "0.5.20"}
thheller/shadow-cljs {:mvn/version "2.20.12"}
thheller/shadow-cljs {:mvn/version "2.20.20"}
com.magnars/test-with-files {:mvn/version "2021-02-17"}}}

:cljtest
{:extra-paths ["test" "dev-resources"]
:extra-deps {lambdaisland/kaocha {:mvn/version "1.71.1119"}
:extra-deps {lambdaisland/kaocha {:mvn/version "1.77.1236"}
org.clojure/test.check {:mvn/version "1.1.1"}
io.github.cap10morgan/test-with-files {:git/tag "v1.0.0"
:git/sha "9181a2e"}}
Expand Down Expand Up @@ -107,5 +107,5 @@
:main-opts ["-m" "antq.core"]}

:clj-kondo
{:extra-deps {clj-kondo/clj-kondo {:mvn/version "2022.11.02"}}
{:extra-deps {clj-kondo/clj-kondo {:mvn/version "2023.01.20"}}
:main-opts ["-m" "clj-kondo.main" "--lint" "src" "--config" ".clj-kondo/config.edn"]}}}
19 changes: 18 additions & 1 deletion src/fluree/db/json_ld/transact.cljc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,28 @@
[fluree.db.policy.enforce-tx :as policy]
[fluree.db.dbproto :as dbproto]
[fluree.db.json-ld.credential :as cred]
[fluree.db.util.log :as log])
[fluree.db.util.log :as log]
[fluree.db.util.validation :as v]
[malli.core :as m])
(:refer-clojure :exclude [vswap!]))

#?(:clj (set! *warn-on-reflection* true))

(def registry
(merge
(m/predicate-schemas)
(m/class-schemas)
(m/comparator-schemas)
(m/type-schemas)
(m/sequence-schemas)
(m/base-schemas)
{::val [:fn v/value?]
::txn-map [:map-of v/iri-key [:orn [:iri v/iri]
[:val ::val]]]
::txn [:orn
[:single-map ::txn-map]
[:collection-of-maps [:sequential ::txn-map]]]}))

(declare json-ld-node->flakes)

(defn json-ld-type-data
Expand Down
47 changes: 31 additions & 16 deletions src/fluree/db/query/fql/syntax.cljc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
(ns fluree.db.query.fql.syntax
(:require [malli.core :as m]
[fluree.db.util.core :refer [pred-ident?]]))
(:require [clojure.string :as str]
[fluree.db.util.log :as log]
[malli.core :as m]
[fluree.db.util.core :refer [pred-ident?]]
[fluree.db.util.validation :as v]))

#?(:clj (set! *warn-on-reflection* true))

Expand All @@ -27,8 +30,6 @@
(and (or (string? x) (symbol? x) (keyword? x))
(-> x name first (= \?))))

(def value? (complement coll?))

(defn sid?
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

probably beyond the scope of this pr, but i think this function could go in the new validation util ns too.

[x]
(int? x))
Expand Down Expand Up @@ -77,25 +78,32 @@
[:string [:fn fn-string?]]
[:list [:fn fn-list?]]]
::wildcard [:fn wildcard?]
::var [:fn variable?]
::val [:fn value?]
::iri [:orn
[:keyword keyword?]
[:string string?]]
::var [:fn {:decode/json
(fn [v]
(log/debug "decoding var:" v)
(if (string? v)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the variable? fn already does a check for if this is a string or symbol iirc. why do we need to repeat that at the json decode level?

Copy link
Contributor Author

@cap10morgan cap10morgan Feb 28, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because the :decode/json fn runs before validation / coercion. It even runs on things that malli ultimately decides don't conform to that (e.g. if they're a part of an :or schema or similar). So you have to do a little sanity checking and just return the arg if it's not something you want to transform.

For example, the selector (i.e. value of the :select key in a query) schema looks like this:

[:orn
 [:var ::var]
 [:pred ::iri-pred]
 [:aggregate ::function]
 [:select-map ::select-map]]

...which means that this decoder fn will get passed the {?p [:*]} map when you have a query like {:select '{?p [:*]} ...} because we've told it it could be a ::var. So it has to know to ignore that.

(symbol v)
v))}
variable?]
::val [:fn v/value?]
::iri v/iri
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm glad we have tighter validation for these.

::iri-pred v/iri-key
::subject [:orn
[:sid [:fn sid?]]
[:iri ::iri]
[:ident [:fn pred-ident?]]]
::subselect-map [:map-of ::iri [:ref ::subselection]]
::subselect-map [:map-of
[:orn [:var ::var] [:iri ::iri]]
[:ref ::subselection]]
::subselection [:sequential [:orn
[:wildcard ::wildcard]
[:predicate ::iri]
[:predicate ::iri-pred]
[:subselect-map [:ref ::subselect-map]]]]
::select-map [:map-of {:max 1}
::var ::subselection]
::selector [:orn
[:var ::var]
[:pred ::iri]
[:pred ::iri-pred]
[:aggregate ::function]
[:select-map ::select-map]]
::select [:orn
Expand Down Expand Up @@ -135,14 +143,14 @@
[:val ::subject]]]
[:predicate [:orn
[:var ::var]
[:iri ::iri]]]
[:iri ::iri-pred]]]
[:object [:orn
[:var ::var]
[:iri ::iri]
[:ident [:fn pred-ident?]]
[:val :any]]]]
::where-tuple [:orn
[:triple ::triple]
[:binding [:sequential {:max 2} :any]]
[:remote [:sequential {:max 4} :any]]]
::where-pattern [:orn
[:where-map ::where-map]
Expand All @@ -152,7 +160,7 @@
[:collection [:sequential ::where-pattern]]]
::filter [:sequential ::function]
::union [:sequential [:sequential ::where-pattern]]
::bind [:map-of ::var :any]
::bind [:map-of ::var ::function]
::where [:sequential [:orn
[:where-map ::where-map]
[:tuple ::where-tuple]]]
Expand Down Expand Up @@ -201,7 +209,14 @@
::multi-query [:map-of [:or :string :keyword] ::analytical-query]
::query [:orn
[:single ::analytical-query]
[:multi ::multi-query]]}))
[:multi ::multi-query]]

::query-results [:sequential
[:map-of ::iri-pred [:or ::iri ::val
[:sequential
[:or ::iri ::val]]]]]

::multi-query-results [:map-of [:or :string :keyword] ::query-results]}))

(def query-validator
(m/validator ::query {:registry registry}))
Expand Down
Loading