Request for help: typed Zod combinator with dynamic field name

[carlpaten]

I apologize, this is almost certainly not the correct place to ask, but I'm not sure where else.

I asked this question on Stack Overflow, reproducing it below.

---

My XML to JSON library emits {MyKey: T} for one-element lists, and {MyKey: T[]} for multi-element lists. The corresponding TypeScript type is type XmlJsonArray<T, element extends string> = Record<element, T | T[]>. I've used the following to implement it as a Zod schema:

const XmlJsonArray = <T, element extends string>(element: element, schema: z.Schema<T>) => {
  // TODO what is the idiomatic approach here?
  const outerSchema: Record<element, z.Schema<T | T[]>> = {} as any;
  outerSchema[element] = z.union([schema, z.array(schema)]);
  return z.object(outerSchema);
};

Is there a way to do this without using any?

[colinhacks]

Good question. I actually couldn't figure this out either. Trying to write generic methods that accept Zod schemas is tricky.

I ended up implementing a new method .setKey to allow for this use case. So you'll have to upgrade to [email protected] or higher. yarn add zod@beta will upgrade you to the newest version.

Here's how it's done:

const XmlJsonArray = <Key extends string, Schema extends z.Schema<any>>(
  element: Key,
  schema: Schema,
) => {
  return z.object({}).setKey(element, z.union([schema, z.array(schema)]));
};

const test = XmlJsonArray('asdf', z.string());

Parsing works as expected:

// both work
test.parse({ asdf: 'hello' });
test.parse({ asdf: ['hello'] });

And type inference works too:

Good question!

[carlpaten]

This is brilliant, thank you for your help!

[carlpaten]

Follow-up question: is there a reasonably concise way of representing the return type of XmlJsonArray?

const XmlJsonArray = <tag extends string, T>(
  tag: tag,
  schema: z.ZodType<T>
): z.ZodType<{[t in tag]: T | T[]}> => { // doesn't work
  return z.object({}).setKey(tag, z.union([schema, z.array(schema)]));
};

I suspect that I'm running into a hard limitation of TypeScript here, but you'd know better than I.

[colinhacks]

So you want to explicitly annotate the return type of the function? Any particular reason?

In general things get funky when you use z.ZodType as a return type since it's an abstract base class that isn't really intended for actual use. At some point I need to write up a whole guide about building generic functions on top of Zod that walk through some of these best practices.

The actual way to type this requires more intimate knowledge of Zod. You need to tell TypeScript the full structure of the schema that gets returned from your function. Here's how to do it:

import * as z from '.'

const XmlJsonArray = <T extends string, S extends z.ZodType<any>>(
  tag: T,
  schema: S,
): z.ZodObject<{ [k in T]: z.ZodUnion<[S, z.ZodArray<S>]> }> => {
  return z.object({}).setKey(tag, z.union([schema, z.array(schema)]));
};

const mySchema = XmlJsonArray('tuna', z.string());
type mySchema  = z.infer<typeof mySchema>
// => { tuna: string | string[]; }

But I wouldn't recommend this. If you change your implementation you'll need to update the type signature accordingly. Best to let the type inference engine work it's magic here.

Note: I made an important change to your code. When using generics, don't do this:

const myFunc = <T>(schema: z.Schema<T>)=>{
  // ...
}

instead do this:

const myFunc = <T extends z.Schema<any>>(schema: T)=>{
  // ...
}

This lets TypeScript infer the exact subclass of ZodType (e.g. ZodString, ZodObject, etc) instead of treating everything as an instance of ZodType (the base class for all Zod schemas).

[carlpaten]

So you want to explicitly annotate the return type of the function? Any particular reason?

I'm trying to feed the parsed output of XmlJsonArray to a function that converts it into a good old JavaScript array. I'm having some weird type mismatches that I'm trying to work out. From typing up the explanation below I think my problem is that setKey creates an optional field.

import * as z from 'zod';

const XmlJsonArray = <tag extends string, T, U extends z.Schema<T>>(tag: tag, schema: U) => {
  return z.object({}).setKey(tag, z.union([schema, z.array(schema)]));
  // or this syntax I just learned about
  return z.object({[tag]: z.union([schema, z.array(schema)])});
};

// TODO the field in a should NOT be optional!
const formatXmlJsonArray = <tag extends string, T>(tag: tag, a: {[v in tag]?: T | T[]}): T[] => {
  const elem: T | T[] = a[tag];
  if (elem instanceof Array) return elem;
  return [elem];
};

// Example usage

const numberParser = z.number();

// const zeroElements = '\r\n';
const oneElement = {Number: 1};
const manyElements = {Number: [1, 2, 3]};

// formatXmlJsonArray('Number', XmlJsonArray('Number', numberParser).parse(zeroElements)); // should return []
formatXmlJsonArray('Number', XmlJsonArray('Number', numberParser).parse(oneElement)); // should return [1]
formatXmlJsonArray('Number', XmlJsonArray('Number', numberParser).parse(manyElements)); // should return [1, 2, 3]

TL;DR two things of interest:

• The {[dynamicFieldName]: fieldValue} ES6 syntax for creating objects with dynamic field names
• The fact that setKey appears to create optional fields (by design? Bug?)

[colinhacks]

I'm having trouble duplicating the setKey creating optional fields error.

Make sure you have "strictNullChecks": true (or just "strict": true) in your tsconfig "compilerOptions" — you'll see strange behavior unless that setting is set.

I got this to work as (I think) you were intending. You need to use conditional types to get formatXmlJsonArray to have the output type signature vary based on the type of the input (which is what you were trying to achieve). Dive into the docs here to learn more: https://www.typescriptlang.org/docs/handbook/2/conditional-types.html 👍

import * as z from '.';

const XmlJsonArray = <tag extends string, T, U extends z.Schema<T>>(
  tag: tag,
  schema: U,
) => {
  return z.object({}).setKey(tag, z.union([schema, z.array(schema)]));
  // or this syntax I just learned about
  // return z.object({ [tag]: z.union([schema, z.array(schema)]) });
};

// TODO the field in a should NOT be optional!
const formatXmlJsonArray = <tag extends string, T>(
  tag: tag,
  a: { [v in tag]?: T },
): T extends Array<any> ? T : T[] => {
  const elem = a[tag] as any;
  if (elem instanceof Array) return elem as any;
  return [elem] as any;
};

// Example usage

const numberParser = z.number();

// const zeroElements = '\r\n';
const oneElement = { Number: 1 };
const manyElements = { Number: [1, 2, 3] };

// formatXmlJsonArray('Number', XmlJsonArray('Number', numberParser).parse(zeroElements)); // should return []
const sample1 = formatXmlJsonArray(
  'Number',
  XmlJsonArray('Number', numberParser).parse(oneElement),
); // should return [1]

const sample2 = formatXmlJsonArray(
  'Number',
  XmlJsonArray('Number', numberParser).parse(manyElements),
); // should return [1, 2, 3]

[carlpaten]

Make sure you have "strictNullChecks": true (or just "strict": true) in your tsconfig "compilerOptions" — you'll see strange behavior unless that setting is set.

Yep, that was it. Thank you so much for your time!

[carlpaten]

I apologize if this is silly, but just in case you missed it:

• The {[dynamicFieldName]: fieldValue} ES6 syntax for creating objects with dynamic field names

I suspect that this makes setKey redundant.

[colinhacks]

I was aware of this syntax but I couldn't get it to work regardless. If you consider this simple example:

const test = <T extends string>(key:T)=>{
  return {[key]: 25};
}

TypeScript isn't able to infer the return type ({ asdf: number}) using the dynamic field names syntax.

I was surprised by this actually. Anyway, I'm pretty sure .setKey is the only way to do this.

[carlpaten]

I opened an issue on the TypeScript repo. Cheers!

E: Ryan from the TypeScript team convinced me that this is working as intended, so I agree, setKey really is the way to go for the foreseeable future.

[nemonemi]

Could you please document the setKey?
There is no mention of it in the documentation at the moment.

For example, this works:

const validationSchema = z.object({}).setKey('name', z.string().max(10));

But this does not:

const validationSchema = z.object({})
for (const prop of someData) {
  validationSchema.setKey(prop.name, z.string().max(10));
}

And I need a way to loop through my data and define the schema. How can this be achieved?

[scotttrinh]

@nemonemi

The problem you're running into is that setKey does not mutate the schema. It returns a new schema, so you need to reassign your schema variable. This works:

CodeSandbox

const someData = [{ name: "foo" }, { name: "bar" }];

let schema = z.object({});
for (const prop of someData) {
  schema = schema.setKey(prop.name, z.string());
}

or if you prefer to avoid variable mutation:

const schema = someData.reduce(
  (schema, definition) => schema.setKey(definition.name, z.string()),
  z.object({})
);

[nemonemi]

Love it!
Thanks, @scotttrinh!
Why doesn't the documentation convey anything about the setKey method?

[scotttrinh]

No particular reason other than we just didn't write anything about it! I'll write something up later today, thanks for the encouragement.

[bitofbreeze]

@scotttrinh I just looked but couldn't find anything about setKey in the docs. I don't see a JSDoc comment either. How's the documentation coming along?

[k-kyler]

@nemonemi

The problem you're running into is that setKey does not mutate the schema. It returns a new schema, so you need to reassign your schema variable. This works:

CodeSandbox

const someData = [{ name: "foo" }, { name: "bar" }];

let schema = z.object({});
for (const prop of someData) {
  schema = schema.setKey(prop.name, z.string());
}

or if you prefer to avoid variable mutation:

const schema = someData.reduce(
  (schema, definition) => schema.setKey(definition.name, z.string()),
  z.object({})
);

I was looking around for a solution to loop through and create a dynamic schema.

Thank you for your solution @scotttrinh, I have solved my problem :D

[TimothyKrell]

@scotttrinh unfortunately, the inferred type from the schema ends up being an empty object. Is there another way to loop through the keys and have the correct inferred type?