Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response validation: Automatically coerce date data type to ISO string #288

Closed
xeoneux opened this issue May 6, 2020 · 10 comments · Fixed by #499
Closed

Response validation: Automatically coerce date data type to ISO string #288

xeoneux opened this issue May 6, 2020 · 10 comments · Fixed by #499

Comments

@xeoneux
Copy link

xeoneux commented May 6, 2020

Is your feature request related to a problem? Please describe.
When validating responses, if the data type is a js Date object, the validation of type string with format date-time fails.

{
  updatedAt: new Date() // fails
}

{
  updatedAt: "2010-13-30T23:12:35Z" // passes
}

Describe the solution you'd like
The lib should try to coerce the type to ISO string and then try validation on it.

{
  updatedAt: new Date() // should pass
}

Describe alternatives you've considered
I have to programmatically search through all the object to find a date data type and call the toISOString method on it to pass the output validation.

Additional context
None

@cdimascio
Copy link
Owner

cdimascio commented May 7, 2020

@xeoneux is this occurring with response validation or request validation?

@xeoneux
Copy link
Author

xeoneux commented May 7, 2020

This is with response validation, didn't check the request part.

@cdimascio
Copy link
Owner

cdimascio commented May 7, 2020

@xeoneux I did some investigation here. This is a bit tricky to solve. I'm not sure a satisfactory solution exists within Ajv.

I tried to create a custom Ajv keyword. A keyword will enable me to manipulate the Date object and convert it to a string e.g. toISOString(), The problem is described here and here note:

The only caveat at the moment is that custom keywords will be validated AFTER standard keywords in the same schema object, ...

AFTER is the issue. Ultimately, validation will fail despite modifying the data because an object is being passed to a schema that expects a string. Ajv executes custom keywords after other validation. This means that the type validation will occur before the new custom keyword has a chance to modify the data i.e change it from Date to iso string. This means thetype validation still occurs against the original Date object and thus fails since the data is still an object (Date), not a string. There is no current way to modify the data prior to validation. :/

I certainly agree that using new Date() in this context is convenient, but there does not seem to be an obvious solution other than you explicitly assigning e.g. updatedAt to a string e.g. new Date().toISOString(), rather than new Date(). Arguably, this is correct anyway, it was declared in the spec to expect a string 🤷‍♂️

The following is the keyword I was experimenting with. I added this keyword to the response validator.

    // if this keyword is set, convert a javascript Date to its iso string
    // TODO validate the date, use Date.parse
    ajv.addKeyword('x-eov-date-as-iso-string', {
      modifying: true,
      compile: function (sch, parentSchema) {
        const match = sch && parentSchema.format === 'date-time';
        return match
          ? function(value, objectKey, object, key) {
              object[key] = new Date(value).toISOString();
              return true;
            }
          : function() => return true;
      },
    });

@cdimascio
Copy link
Owner

keeping this open, perhaps folks have alternative ideas

@electrotype
Copy link

electrotype commented May 12, 2020

I have the same problem and found this, which is related: #246

Still a workaround, and I still don't know if it works, but hey.

@cdimascio cdimascio changed the title Automatically coerce date data type to ISO string Response validation: Automatically coerce date data type to ISO string May 23, 2020
@kapouer
Copy link

kapouer commented Nov 9, 2020

@electrotype
Copy link

Do you accept bounties on issues? I could put 50$US for a fix on this one.

This is the only blocking issue for us. We have to JSON.parse(JSON.stringify(obj)) everything we send, simply for the date-time fields to pass the validation.

@cdimascio
Copy link
Owner

@pilerou has provided an option that will help solve this.
please have a look at #493 and provide feedback.
thanks!

@electrotype
Copy link

Feedback given. Thank you all for the effort on this issue.

@cdimascio
Copy link
Owner

this is solved in v4.10.0, please give it a try

pilerou added a commit to pilerou/express-openapi-validator that referenced this issue Dec 30, 2020
…ize` request parameters or fields. It could resolve : cdimascio#353 cdimascio#465 cdimascio#288 cdimascio#246

Unit tests validate Date and MongoDb ObjectID.
Developers have choice to :
- only serialize response contents
- also deserialize request strings to custom objects

Frequent SerDes are defined in base.serdes.ts (date and date-time).

Documentation updated with this setting
cdimascio pushed a commit that referenced this issue Feb 14, 2021
* `serDes` setting allows to `serialize` response objects and `deserialize` request parameters or fields. It could resolve : #353 #465 #288 #246
Unit tests validate Date and MongoDb ObjectID.
Developers have choice to :
- only serialize response contents
- also deserialize request strings to custom objects

Frequent SerDes are defined in base.serdes.ts (date and date-time).

Documentation updated with this setting

* I don't know why there was a cloneDeep but it seems to be necessary in all cases.

* Fix validation problems with oneOf and force default SerDes when no SerDes are defined (force DateTime and Date serialization).

* Delete old code comments

* New test : If I answer with an object which serialize fails (here because I answer with an ObjectID instead of Date and no toISOString function exists), a 500 error is thrown.

* Add Date and date-time serialization by default in addition to other serDes settings.
Custom settings can also override date and date-time formats in order to also deserialize date and/or date-time on requests

* Add Date and date-time serialization by default in addition to other serDes settings.
Custom settings can also override date and date-time formats in order to also deserialize date and/or date-time on requests

* `serDes` option adaptation to be more user friendly
Test OK
Documentation is modified
I also changed my https://github.com/pilerou/mongo-serdes-js with a 0.0.3 version which is compliant to the design :
```javascript
serDes: [
      OpenApiValidator.baseSerDes.date.serializer,
      OpenApiValidator.baseSerDes.dateTime,
      MongoSerDes.objectid, // this configuration if we want to deserialize objectid in request and serialize it in response
      MongoSerDes.uuid.serializer, // this configuration if we only want to serialize on response
    ],
```

* When we add custom formats in serDes, they are automatically added to unknownFormats

* Rename OpenApiValidator.baseSerDes to OpenApiValidator.serdes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants