Create dsl for search API

[dfarr]

Let's start with the most bare bones dsl, either:

• wildcard only with *
• prefix only search

[dfarr]

Once this is in place, we can reinstate SearchPromises API in DST.

DST

Deterministic simulation testing (DST) is fundamental to our development strategy at Resonate. Every four hours we run four iterations of DST given some seed value s.

1. A run with seed=s and store=sqlite, let's call this s1.
2. A run with seed=s and store=sqlite, let's call this s2.
3. A run with seed=s and store=postgres, let's call this p1.
4. A run with seed=s and store=postgres, let's call this p2.

Because our system is deterministic, that is, two systems that receive the same requests in the same order are guarenteed to be in the same state, we can now perform the following three diffs:

1. diff of s1 and s2
2. diff of p1 and p2
3. diff of s1 and p1

If any of the diffs fail, we know that we have broken our guarantee of determinism and have strong evidence that there is something more fundamentally incorrect that needs to be addressed in our codebase. See #104 for an example DST workflow run, download the logs and see for yourself if there is any difference!

Here is an example snippet from s1 starting at line 15616:

level=INFO msg=DST t=49 req="CreateSubscription(id=305, promiseId=379, url=https://resonatehq.io/408)" res="CreateSubscription(status=201, subscriptions=Subscription(id=305, promiseId=379, url=https://resonatehq.io/408, retryPolicy=RetryPolicy(delay=406, attempts=2)))" err=<nil>
level=INFO msg=DST t=49 req="ReadPromise(id=487)" res="ReadPromise(status=200, promise=Promise(id=487, state=PENDING, param=Value(ikey=<nil>, data=269), value=Value(ikey=<nil>, data=), timeout=386))" err=<nil>
level=INFO msg=DST t=49 req="RejectPromise(id=559, ikey=<nil>)" res="RejectPromise(status=404, promise=<nil>)" err=<nil>
level=INFO msg=DST t=49 req="DeleteSubscription(id=29, promiseId=315)" res="DeleteSubscription(status=404)" err=<nil>
level=INFO msg=DST t=49 req="RejectPromise(id=102, ikey=<nil>)" res="RejectPromise(status=201, promise=Promise(id=102, state=REJECTED, param=Value(ikey=42, data=158), value=Value(ikey=<nil>, data=), timeout=585))" err=<nil>

And here is the same snippet from p1:

level=INFO msg=DST t=49 req="CreateSubscription(id=305, promiseId=379, url=https://resonatehq.io/408)" res="CreateSubscription(status=201, subscriptions=Subscription(id=305, promiseId=379, url=https://resonatehq.io/408, retryPolicy=RetryPolicy(delay=406, attempts=2)))" err=<nil>
level=INFO msg=DST t=49 req="ReadPromise(id=487)" res="ReadPromise(status=200, promise=Promise(id=487, state=PENDING, param=Value(ikey=<nil>, data=269), value=Value(ikey=<nil>, data=), timeout=386))" err=<nil>
level=INFO msg=DST t=49 req="RejectPromise(id=559, ikey=<nil>)" res="RejectPromise(status=404, promise=<nil>)" err=<nil>
level=INFO msg=DST t=49 req="DeleteSubscription(id=29, promiseId=315)" res="DeleteSubscription(status=404)" err=<nil>
level=INFO msg=DST t=49 req="RejectPromise(id=102, ikey=<nil>)" res="RejectPromise(status=201, promise=Promise(id=102, state=REJECTED, param=Value(ikey=42, data=158), value=Value(ikey=<nil>, data=), timeout=585))" err=<nil>

Looking at the last RejectPromise request we can deduce that a promise with id 102 must have been created at some tick less than t=49, and in fact if we check both s1 and p1 we can find that this did indeed happen at t=23.

level=INFO msg=DST t=23 req="CreatePromise(id=102, ikey=42, timeout=585)" res="CreatePromise(status=201, promise=Promise(id=102, state=PENDING, param=Value(ikey=42, data=158), value=Value(ikey=<nil>, data=), timeout=585))" err=<nil>

DST and SearchPromises

Our SearchPromises implementation for sqlite uses the following query:

SELECT
	*
FROM
	promises
WHERE
	id GLOB ? AND state = ?
ORDER BY
	id

While glob is convenient, this produces different results than our postgres query:

SELECT
	*
FROM
	promises
WHERE
	id LIKE $1 AND state = $2
ORDER BY
	id

As a result, we have temporarily removed SearchedPromises from DST. Once the query logic is normalized across these stores by implementing our own dsl, we can re-enable SearchPromises in DST and verify the queries do in fact produce the same results.

More about our DST implementation

Every DST run creates a reproducible system from a single seed (an integer). From this seed every other system parameter is generated within stipulated bounds, and all requests are randomly generated and fed through the system. This allows our entire DST run to be reproducible - given a seed we can reproduce the exact same output.

Try running it locally, here we are limiting the number of ticks (iterations of the system) to 10 to keep the test snappy.

./resonate dst run --seed 0 --ticks 10

Try running a few DST examples yourself and see if you can find two runs that produce different results (and create an issue if you do!). By default, DST runs against sqlite but you can swap in postgres. Additionally you can fix parameters to a specific value by specifying the corresponding command line argument, you can specify either a single value (eg: --ids 100) or a range of values (eg: --ids 1:100).

You can see a list of all parameters by running:

./resonate dst run --help