Support for Pydantic

[patrick91]

Hi folks!

I'm thinking of starting using Pydantic for data validation in our work project and I'm consindering adding support for it into Strawberry[1]. I don't know yet how big this project will be, as I haven't seen much of Pydantic's source code.

The reason why I'd like to have support for Pydantic is to prevent duplication with strawberry input types and pydantic types.

In terms of API I was thinking at something like this:

class UserModel(BaseModel):
    name: str = Field(..., min_length=2)


@strawberry.pydantic.type(model=UserModel):
class User:
    pass


@strawberry.pydantic.input(model=UserModel):
class UserInput:
    pass



@strawberry.pydantic.error_type(model=UserModel):
class UserInputErrors:
    pass

let's go through each of these one by one, first we have the UserModel class, this is the model that we will be using to auto generate type, input type and an error type.

Our User class is a GraphQL object type, UserInput is a GraphQL input type, and, finally, UserInputError is another GraphQL object type that we can use to wrap validation errors.

Let's do a small digression on errors, why do we need a new type when GraphQL already supports errors?
Basically the idea is to not use GraphQL errors for expected errors (ie. validation errors, item not found, etc) but instead to return a new type that has information about the error (for example a string for each input field, some metadata, etc), this is all based from this blog post: https://medium.com/@sachee/200-ok-error-handling-in-graphql-7ec869aec9bc

So it would be nice to generate types to encapsulate errors. This is the GraphQL schema we'd get from the classes above:

type User {
    name: String!
}

input UserInput {
    name: String!
}

type UserInputFieldErrors {
    name: String
}

type UserInputErrors {
    fieldErrors: UserInputFieldErrors! 
}

The UserInputError has errors wrapped inside a fieldErrors to prevent name clashing with the output field (when returing Union[User, UserInputErrors]).

We might also have nonFieldErrors for errors that don't belong to a particular field.

This is how you'd use it:

@strawberry.type
class Mutation:
    @strawberry.mutation
    def create_user(self, input: UserInput) -> Union[User, UserInputErrors]:
        # here input would be a validated `UserInput`
        ...

we are hiding the data validation a bit, but maybe that's fine, any thoughts?

[1] I think we will do the same thing with Django in future.

[jkimbo]

+1 on modeling the errors explicitly. I'm not sure the error type needs to be tied to the UserModel though. It could be generic like this:

type PydanticInputValidationError {
    fieldName: String!
    errorMessage: String!
}

type UserInputErrors {
    nonFieldErrors: [String!]!
    fieldErrors: [PydanticInputValidationError!]! 
}

(names are TBD)

It's possible that it would be valuable to be able to define custom input error types per model but I can't think of a valid use case for that at the moment. Also it is always possible to just create the types by hand if they are needed.

Did you have a specific use case in mind @patrick91 ?

---

One thing to note is that Pydantic has support for quite complex validation on things like Lists so having just a plain fieldName type on the error type might not be enough. In the docs it represents the location of the error as a list that represents the "path" to the error so that might be a better way of modeling it:

[
  {
    "loc": [
      "friends",
      2
    ],
    "msg": "value is not a valid integer",
    "type": "type_error.integer"
  }
]

https://pydantic-docs.helpmanual.io/

---

Minor thing but in your python example @patrick91 I think you've referred to the wrong type in the return Union for create_user. I think it should be def create_user(self, input: UserInput) -> Union[User, UserInputErrors]:

[patrick91]

+1 on modeling the errors explicitly. I'm not sure the error type needs to be tied to the UserModel though. It could be generic like this:

type PydanticInputValidationError {
    fieldName: String!
    errorMessage: String!
}

type UserInputErrors {
    nonFieldErrors: [String!]!
    fieldErrors: [PydanticInputValidationError!]! 
}

That's another solution for sure 😊

It's possible that it would be valuable to be able to define custom input error types per model but I can't think of a valid use case for that at the moment. Also it is always possible to just create the types by hand if they are needed.

Did you have a specific use case in mind @patrick91 ?

I mainly wanted to type all the errors so that they'd map one to one to a field, to make clients life easier, like this:

input UserInput {
    name: String!
}

type UserInputFieldErrors {
    name: String
}

type UserInputErrors {
    fieldErrors: UserInputFieldErrors! 
}

type Mutation {
    createUser(input: UserInput): UserInputErrors | User
}

# ---------

const { data } = createUser(input);

if (data.__typename == "UserInputErrors") {
    console.log(data.fieldErrors.name)
}

# as opposed to 

const { data } = createUser(input);

if (data.__typename == "UserInputErrors") {
    console.log(data.fieldErrors.findOne(x => x.fieldName == 'name'))
}

It's not horrible, but I think if we can leverage GraphQL types, we should.

Maybe it is a bit too opinionated?

One thing to note is that Pydantic has support for quite complex validation on things like Lists so having just a plain fieldName type on the error type might not be enough. In the docs it represents the location of the error as a list that represents the "path" to the error so that might be a better way of modeling it:

[
  {
    "loc": [
      "friends",
      2
    ],
    "msg": "value is not a valid integer",
    "type": "type_error.integer"
  }
]

pydantic-docs.helpmanual.io

Yup, this mighe be solved with having one error type per input type :)

Minor thing but in your python example @patrick91 I think you've referred to the wrong type in the return Union for create_user. I think it should be def create_user(self, input: UserInput) -> Union[User, UserInputErrors]:

Correct, my bad! Will update 😊

[jkimbo]

I mainly wanted to type all the errors so that they'd map one to one to a field, to make clients life easier, like this:

Ah I see that makes sense. Sorry that part went over my head completely (I read the name field as being fieldName) and I completely agree that explicitly typing the errors would work much better.

How would you model the case where the error is for 1 particular element in a list though?

Also Pydantic validation can be infinitely nested so something like this is possible:

class UserModel(BaseModel):
    input_data: List[Dict[str, Dict[str, str]]]

In [38]: UserModel(input_data=[{"foo": {"bar": None}}])
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
<ipython-input-38-130a7a276650> in <module>
----> 1 UserModel(input_data=[{"foo": {"bar": None}}])

~/code/strawberry-django-test/.venv/lib/python3.8/site-packages/pydantic/main.cpython-38-darwin.so in pydantic.main.BaseModel.__init__()

ValidationError: 1 validation error for UserModel
input_data -> 0 -> foo -> bar
  none is not an allowed value (type=type_error.none.not_allowed)

[patrick91]

How would you model the case where the error is for 1 particular element in a list though?

Uhm, what about this:

input UserInput {
    names: [String!]!
    ages: [Int!]!
}

type UserInputFieldErrors {
    names: [String]
    ages: [String]
}

"""
{
    "names": [null, "Invalid name"]
}
"""

So it would basically use the same shape as the input, but converting the type to a string.
Does this make sense? Or we can do something like this:

input UserInput {
    names: [String!]!
    ages: [Int!]!
}

type PyDanticError { 
    message: String!
    index: Int
}

type UserInputFieldErrors {
    names: [PyDanticError!]
    ages: [PyDanticError!]
}

"""
{
    "names": [{ "message: "Invalid name", "index": 1}]
}
"""

so we can use this PyDanticError to add some metadata, but I think we can just use lists, from the first example, I also prefer it as it maps better.

Also Pydantic validation can be infinitely nested so something like this is possible:

class UserModel(BaseModel):
    input_data: List[Dict[str, Dict[str, str]]]

Another good point, I think for this we might be fine, since we don't really support something like this in plain strawberry, you'd need to create a specific type anyway. Here you could use a JSON scalar, and for the errors we can return the plain pydantic ones. What do you think?

[jkimbo]

Uhm, what about this:

input UserInput {
    names: [String!]!
    ages: [Int!]!
}

type UserInputFieldErrors {
    names: [String]
    ages: [String]
}

"""
{
    "names": [null, "Invalid name"]
}
"""

I like this, though we might want a PydanticError type as well to expose the validation type (value_error.missing etc)?

Another good point, I think for this we might be fine, since we don't really support something like this in plain strawberry, you'd need to create a specific type anyway. Here you could use a JSON scalar, and for the errors we can return the plain pydantic ones. What do you think?

Yeah I guess this could be solved with some good error messages and docs. Converting it to a JSON scalar sounds like a good idea too.

[patrick91]

I like this, though we might want a PydanticError type as well to expose the validation type (value_error.missing etc)?

Yes, we should do that, good point! 😊

[patrick91]

From #453 (comment)

Sorry if i've missed this being discussed elsewhere. Given the example above, assuming you have a field that should return UserType, presumably you can't just return an instance of User (which is good, because one of the big issues with graphene-django is confusion about what your return values should be). If this is the case, UserType probably also needs some way of constructing an instance from User, e.g UserType.from(user_instance).

@AndrewIngram we discussed this a bit on discord, and we were thinking of doing something like this:

user = UserType(instance=User(name="Jonathan Kim"), additional_attributes="Not my name")

Similar to django forms. I'm not 100% sure on this yet, because there's the additional complexity of input types.

For example if we wan't to access a user attribute I guess can do this:

user = UserType(instance=User(name="Jonathan Kim"), additional_attributes="Not my name")
user.instance.name

for input types it is different I guess, let's say that we do this:

class UserModel(BaseModel):
    name: str = Field(..., min_length=2)

@strawberry.pydantic.input(model=UserModel):
class UserInput:
    pass

@strawberry.type
class Mutation:
    @strawberry.mutation
    def create_user(self, input: UserInput) -> Union[User, UserInputErrors]:
        ...

how would I access the input data?

input.instance.name doesn't sounds good or intuitive, I was thinking of allowing this:
input.name, so proxy all the attributes. But I'm not 100% sure.

The other option is to just register pydantic types, ie making some sort of copies (for inputs and types). And then use them as if the were pydantic types, but I don't like that as you can't use strawberry.field for changing a fields name for example.

Do you have any suggestion?

[patrick91]

I'd caution against the latter even if it makes things slightly more convenient. When Strawberry resolvers start having to know about the existence of an instance, it starts to feel a bit off to me.

This make sense.

This probably means copying the fields from the User instance, but proxying might be okay if it's only getters and not setters.

I guess proxying is the easiest solution, but we'd still need to create a custom init method. So for example in this case:

class UserModel(BaseModel):
    name: str = Field(..., min_length=2)

@strawberry.pydantic.type(model=UserModel):
class UserType:
    @strawberry.field
    def name_with_prefix(self) -> str:
         return f'🔥 {self.name}'

you think it is ok to use self.name directly? I didn't like it at first, but after reading your suggestion I feel like it is ok.

@jkimbo @jaydenwindle what do you think?

The other thing that's not clear is what happens if the Pydantic type references another Pydantic type, does the entire relationship chain get followed -- building an entire mini graph of Strawberry types?

Right now I made so that you need to make a type for each referenced type, don't want to generate a strawberry type automatically, as it might lead to exposing fields that shouldn't be exposed.

[AndrewIngram]

To be honest, I prefer the idea of copying the instance field values onto the strawberry type over proxying. It's more expensive, but conceptually much cleaner and less likely to break a developer's mental model of what's going on. So in very rough code:

        # this is from the pydantic PR
        generated_type = _process_type(
            cls,
            name=name,
            is_input=is_input,
            is_interface=is_interface,
            description=description,
            federation=federation,
            _base_fields=base_fields,
        )

       # this is new
        def from_pydantic_instance(instance: model) -> generated_type:
            instance_kwargs = {}
            # Todo: iterate over instance fields, copy values across to 
            # instance_kwargs
            return generated_type(**instance_kwargs)

        generated_type.from_pydantic_instance = staticmethod(from_pydantic_instance)

[patrick91]

@AndrewIngram what if you add more fields to the type? would you have something like this:

        def from_pydantic_instance(instance: model, **kwargs) -> generated_type:
            instance_kwargs = {}
            # Todo: iterate over instance fields, copy values across to 
            # instance_kwargs
            return generated_type(**{**instance_kwargs, **kwargs})

        generated_type.from_pydantic_instance = staticmethod(from_pydantic_instance)

and what about the init method on the generated type? would it retain the validation feature of pydantic?

[AndrewIngram]

I suppose i'm focusing more on the meta-issue of how to avoid doing work multiple times when your Strawberry types are conceptually very close to your underlying domain models -- i.e. the work you're doing here for pydantic could be mirrored for Django, SQLAlchemy ORM etc. The second issue (which is the origin of this overall discussion) is about leveraging pydantic for input validation.

Beyond type-checking/query-validation (which will get more sophisticated once GraphQL supports unions on inputs), I think other validation can be deferred to the lower levels of the stack because they'd usually result in returning something like a modelled form error rather than throwing an exception. So it seems there are a few necessary conversion points, using pydantic here but could be any of the aforementioned alternatives:

pydantic model -> strawberry object type
pydantic model -> strawberry input type

pydantic model instance -> strawberry object type instance
strawberry input type instance -> pydantic model instance

In both of the model to strawberry type "factories", there's the opportunity to exclude fields, but also add new strawberry-only ones. So as you've rightly pointed out, there needs to be a way to incorporate similar customisation to the instance factory

and what about the init method on the generated type? would it retain the validation feature of pydantic?

For this particular question I think the cleanest (but most verbose) solution is to not retain the validation feature, but just make there's an easy way to convert the strawberry input instance to the pydantic model. But I'd need to think a bit more about whether the value of clean separation between layers applies equally to inputs and outputs, my assumption has always been yes but I've never really challenged it.

[patrick91]

Thanks for your inputs! I think I got some ideas for a first implementation. I wanted to make something a bit magic, so that it was easier to use, but for now I think I'll use your suggestions and make something where things are more explicit and see how that goes!