You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To facilitate testing across the space of possible SDKv2 schemas, build a generator that creates tuples (s, v) of a schema s and a TF value that conforms to the schema v: s.
There are some important variations to cover
every possible combination of schema, up to depth 3, including maps, sets and objects
within the schema, every possible combination of Attribute, Block, and nesting modes (SingleNested, ListNested, etc)
every possible combination of attribute kind, Computed, Optional, Computed+Optional, Required
sets, including several distinct custom set hashes
objects need to have up to 2 attributes so Set[Obj] combination cover variations of attribute kind
when generating values, ensure to cover null, unknown, and empty values and 2 regular values, say for strings [null, unknown, "", "a", "b" ]
The text was updated successfully, but these errors were encountered:
Fixes#1790 by
building a rapid generator for schemas and associated values.
Large-ish problem 1: I do not have it figured out how to test unknown
values. TF literals as unknown values are forbidden and do not make
sense. We might need a helper resource so that testing unknown values
generates references to an output of the helper resource. This is logged
for future work.
Large-ish problem 2: iteration is pretty slow (x-proc). Normal n=100
rapid tests can take up to 10min. Could try batching so several
resources are tried in one shot say 100 resources.
Large-ish problem 3: I'm not sure if no-op Update and Create
implementations are acceptable. There is something to testing Computed
attributes where provider has to set values. Possibly Update also needs
to set values? Possibly not.
Small problems:
- [x] Using TF JSON syntax didn't handle null/empty correctly; that is
now discarded, using actual HCL syntax
- [x] TF representations are difficult to visualize in failing tests and
difficult to assert against
- [x] Lots of lost-in-translation papercuts possible between
representations (cty.Value, resource.PropertyValue, tftypes.Value)
- [x] this requires a change to providertest to abstract from testing.T
so we can pass rapid.T
- [x] it's very hard to disable annoying TF logging, using env vars for
now
We are starting to find bugs and discrepancies from this work:
- #1856 panic
corner-case
- #1852 need to
InternalValidate
- #1828
Future work:
- #1856
- #1857
- #1858
- #1859
- #1860
- #1861
- #1862
- #1863
- #1864
- #1865
- #1866
- #1867
To facilitate testing across the space of possible SDKv2 schemas, build a generator that creates tuples
(s, v)
of a schemas
and a TF value that conforms to the schemav: s
.There are some important variations to cover
The text was updated successfully, but these errors were encountered: