Axios Safe JSON parsing

[NewDark90]

JSON.parse will throw on any input that isn't a string. Interceptors may change the data to return as a valid JS object/array/etc even if the client instance was controlled enough to ensure that every returned response data point is a JSON parseable string.

While not perfect, it will simply return the object it has in the case that it throws an error. This update also adds in the jsonParse function call for HandleReferences in case an override wants to be provided though code.

One mention of the issue here. 5d548d7#r37786449

[NewDark90]

@RicoSuter Any idea on when this can get looked at and implemented?

[RicoSuter]

@thijscrombeen @JyrkiHei @nunonux can you have a look?

[NewDark90]

Found the template directory functionality so I'm not hurting for this, and not wed to the implementation. Thanks for anyone that digs in. :)

[thijscrombeen]

I've checked your branch and was able to successfully generate a typescript file with your changes.
In my projects however I use the Class TypeStyle so I haven't actually run the generated code.

protected processSomeMethod(response: AxiosResponse): Promise<boolean> {
    const status = response.status;
    ...
    if (status === 200) {
      const _responseText = response.data;
      let result200: any = null;
      let resultData200 = _responseText;
      try {
        result200 = _responseText === '' ? null : <boolean>JSON.parse(resultData200, this.jsonParseReviver);
      } catch (err) {
        result200 = _responseText;
      }
      return result200;
    }
    ...

While above code doesn't give any compiler errors, it is not really correct.
'result200' is of type any and set to null while the method should actually return only boolean. (this is already currently wrong and got nothing to do with your PR)
The method can in this case return null but with this PR anything at all based on whatever is in '_responseText'

We could change the generated code to explain that it can return either boolean or null but then it no longer works with your change.

protected processSomeMethod(response: AxiosResponse): Promise<boolean | null> {
    const status = response.status;
    ...
    if (status === 200) {
      const _responseText = response.data;
      let result200: boolean | null = null;
      let resultData200 = _responseText;
      try {
        result200 = _responseText === '' ? null : <boolean>JSON.parse(resultData200, this.jsonParseReviver);
      } catch (err) {
        result200 = _responseText;
      }
      return result200;
    }
    ...

In which cases do you use interceptors that would make the JSON.parse fail?

[NewDark90]

So, given these parameters:
/TypeScriptVersion:3.8 /PromiseType:Promise /Template:Axios /GenerateClientClasses:true /GenerateClientInterfaces:true /TypeStyle:Interface /GenerateOptionalParameters:true /OperationGenerationMode:MultipleClientsFromFirstTagAndOperationId /MarkOptionalProperties:true /WrapResponses:true /ResponseClass:ThingClientResponse

Here's an example generation based on a pretty simple "get by id" type endpoint client:

export class ThingClient implements IThingClient {
    private instance: AxiosInstance;
    private baseUrl: string;
    protected jsonParseReviver: ((key: string, value: any) => any) | undefined = undefined;

    constructor(baseUrl?: string, instance?: AxiosInstance) {
        this.instance = instance ? instance : axios.create();
        this.baseUrl = baseUrl ? baseUrl : "";
    }

    /**
     * @return Success
     */
    get_api_thing_id(id: number): Promise<ThingClientResponse<ThingDto>> {
        let url_ = this.baseUrl + "/api/Thing/{id}";
        if (id === undefined || id === null)
            throw new Error("The parameter 'id' must be defined.");
        url_ = url_.replace("{id}", encodeURIComponent("" + id));
        url_ = url_.replace(/[?&]$/, "");

        let options_ = <AxiosRequestConfig>{
            validateStatus: () => true,
            method: "GET",
            url: url_,
            headers: {
                "Accept": "application/json"
            }
        };

        return this.instance.request(options_).then((_response: AxiosResponse) => {
            return this.processGet_api_thing_id(_response);
        });
    }

    protected processGet_api_thing_id(response: AxiosResponse): Promise<ThingClientResponse<ThingDto>> {
        const status = response.status;
        let _headers: any = {};
        if (response.headers && typeof response.headers === "object") {
            for (let k in response.headers) {
                if (response.headers.hasOwnProperty(k)) {
                    _headers[k] = response.headers[k];
                }
            }
        }
        if (status === 200) {
            const _responseText = response.data;
            let result200: any = null;
            let resultData200  = _responseText;
            if (typeof resultData200 !== "string") { 
                result200 = resultData200;
            }
            else {
                try { result200 = (_responseText === "" ? null : <ThingDto>JSON.parse(resultData200, this.jsonParseReviver)); } 
                catch(err) { result200 = resultData200; }
            }
            return Promise.resolve<ThingClientResponse<ThingDto>>(new ThingClientResponse<ThingDto>(status, _headers, result200));
        } else if (status === 404) {
            const _responseText = response.data;
            let result404: any = null;
            let resultData404  = _responseText;
            if (typeof resultData404 !== "string") { 
                result404 = resultData404;
            }
            else {
                try { result404 = (_responseText === "" ? null : <ProblemDetails>JSON.parse(resultData404, this.jsonParseReviver)); } 
                catch(err) { result404 = resultData404; }
            }
            return throwException("Not Found", status, _responseText, _headers, result404);
        } else if (status !== 200 && status !== 204) {
            const _responseText = response.data;
            return throwException("An unexpected server error occurred.", status, _responseText, _headers);
        }
        return Promise.resolve<ThingClientResponse<ThingDto>>(new ThingClientResponse(status, _headers, <any>null));
    }
}

Note, this stack overflow answer that basically says that Axios automatically parses json results, unless it specifically is fed transformers (or even, a lack of them, because it will remove the default one). https://stackoverflow.com/questions/41013082/disable-json-parsing-in-axios

Note this important bit of code from above:
return this.instance.request(options_).then((_response: AxiosResponse) => { return this.processGet_api_thing_id(_response); });

It's specifically modifying the response from the request above. The only valid thing that can come through as _response.data is a string representation of JSON. Currently there isn't anything that, for example, removes the transformers so that is the case all the time, and that seems kind of arbitrarily limiting doesn't it?

This is a super specific thing that it needs in order for it to function, and I'm halfway convinced it probably shouldn't even be trying to parse this data.

[thijscrombeen]

I just checked the Axios code and the default implementation of transformResponse is as follows:

transformResponse: [function transformResponse(data) {
    /*eslint no-param-reassign:0*/
    if (typeof data === 'string') {
      try {
        data = JSON.parse(data);
      } catch (e) { /* Ignore */ }
    }
    return data;
  }],

Would it be best to by default overwrite this so that axios itself does not do any parsing?
That doesn't fix the issue I mentioned above where we don't really type correctly currently.

[NewDark90]

@thijscrombeen I'd think we just don't try to parse it. Seems like it's trying to solve for a non-problem within this specific generation context. I think that would solve for all of it yeah?

[mohsenZaim]

At the moment (nswag 13.7.0.0), when /template:Axios and /typeStyle:Interface, then the output of the response process throws Json exception

SyntaxError: Unexpected token o in JSON at position 1

in this line (when trying to parse JSON)

let resultData200 = _responseText;
result200 = JSON.parse(resultData200);

guess this PR gonna fix this, right? @NewDark90
Any plan to merge or fix this issue? at the moment, it isnt possible to use axios template with type interfaces ? @RicoSuter

[dubzzz]

SyntaxError: Unexpected token o in JSON at position 1

I have exactly the same issue as the one reported by @mohsenZaim when toggling Interfaces with Axios.

Last week, I opened #2966 to suggest a fix for that issue (then closed it as I saw the current PR would be even better) but I believe that the current PR covers more edge cases.

[mohsenZaim]

SyntaxError: Unexpected token o in JSON at position 1

I have exactly the same issue as the one reported by @mohsenZaim when toggling Interfaces with Axios.

Last week, I opened #2966 to suggest a fix for that issue (then closed it as I saw the current PR would be even better) but I believe that the current PR covers more edge cases.

Is there any way to use the .liquid based templates locally and generate a correct output for the axios template? in swagger i used to modify .mustach files and use those instead of default templates to generate my code. here for this nswag, i havent found a way yet?

@RicoSuter do we have such feature to use locally modified .liquid templates to generate files, so we can use this axios based template ? or the only solution at the moment would be to wait for this bug to be fixed? :)

[RicoSuter]

You can test the changed tpl with an override:
https://github.com/RicoSuter/NSwag/wiki/Templates

As soon as i have the ok that its fine ill merge the pr.

[RicoSuter]

Not sure whether this is a good idea here.
The generated code should not require external transformations in general (leaky abstraction) and should work out-of-the-box and thus deserialization should work as expected and a JSON exception should be thrown, no?

[dubzzz]

In my suggested fix (see https://github.com/RicoSuter/NSwag/pull/2966/files), I just removed JSON.parse(...). It seems that it is not needed at all in the context of axios if I am not wrong. See explanation here: #2966 (comment) 🤔

[mohsenZaim]

You can test the changed tpl with an override:
https://github.com/RicoSuter/NSwag/wiki/Templates

As soon as i have the ok that its fine ill merge the pr.

Thanks @RicoSuter :)

[thijscrombeen]

@mohsenZaim @dubzzz
What kind of data are you returning where this throws the JSON error ?

Could you provide a swagger/open api json document and maybe the api controller implementation if the api is in c#?

[dubzzz]

@thijscrombeen The failure happens whenever I try to return a simple string in the end-point

[mohsenZaim]

@mohsenZaim @dubzzz
What kind of data are you returning where this throws the JSON error ?

Could you provide a swagger/open api json document and maybe the api controller implementation if the api is in c#?

Arrays for example.

Here is swagger contract sample:

"/api/Locations": {
"get": {
"tags": [
"Location"
],
"operationId": "Location_List",
"responses": {
"200": {
"x-nullable": false,
"description": "",
"schema": {
"type": "array",
"items": {
"$ref": "#/definitions/LocationListItem"
}
}

here is the output when use axios + type interface inside the process method.
the part which throws error is when status 200 oreturn for example
the data suppose to be array but it is trying to Json.parse it, therefore it throw error SyntaxError: Unexpected token o in JSON at position 1.

if (status === 200) {
const _responseText = response.data;
let result200: any = null;
let resultData200 = _responseText;
result200 = JSON.parse(resultData200);
return result200;
}

Not sure why the template is JSON parsing here ? and why not just return the response.body. or at least check if it is parsable.

and here is sample C# in the backend

    [ProducesResponseType(StatusCodes.Status404NotFound)]
    [ProducesResponseType(StatusCodes.Status200OK, Type = typeof(List<ApiModel.LocationListItem>))]
    [HttpGet]
    [Route("[controller]s", Name = "ListLocations")]
    public ActionResult<List<ApiModel.LocationListItem>> List()
    {
        // Call business logic
        var output = _locationService.List();

        if (output == null)
        {
            return NotFound();
        }

        // Convert output
        var mappedOutput = Mapper.Map<List<ApiModel.LocationListItem>>(output);

        return Ok(mappedOutput);
    }
}

[thijscrombeen]

As explained above Axios is also doing a JSON.parse. When using interface typestyle NSwag is also doing a JSON.parse which happens after the one from Axios. At that time it's no longer a string but it's a json object and it fails.

This can be overridden by setting the following on the axios client:

transformResponse: [function transformResponse(data) {
    return data;
  }],

For example in the constructor of the generated client:

constructor(baseUrl?: string, instance?: AxiosInstance) {
    this.instance = instance
      ? instance
      : axios.create({
          transformResponse: [
            function transformResponse(data: any) {
              return data;
            },
          ],
        });
    this.baseUrl = baseUrl ? baseUrl : "";
  }

If needed I can make a seperate PR for this change to add the transformresponse in the case of typestyle interface.

[mohsenZaim]

If needed I can make a seperate PR for this change to add the transformresponse in the case of typestyle interface.

Would be great if you can add this feature in a seperate PR that can be merged and release a new version, so we use that instead of overriding the template on our side.

[NewDark90]

@thijscrombeen But why override that functionality to stop parsing, just to do it ourselves anyway down the line? Doesn't make sense to me

[dubzzz]

Based on the discussion above, I recently tried to use the following instance of axios in my codebase and it worked like charm with interface mode on:

axios.create({
      transformResponse: [
        (data) => {
          // The output of this function is a string that can be passed to JSON.parse
          try {
            // types such as strings, numbers are already parsed so JSON.parse will fail on them
            // complex objects on the contrary come as strings
            JSON.parse(data);
            return data;
          } catch (err) {
            return JSON.stringify(data);
          }
        },
      ],
    });

I use it as a temporary workaround for the moment.

---

UPDATE: The trick above is incomplete. Date fields will not be Date anymore but raw strings.

[TorbjornHoltmon]

This issue is still here. I am not sure why the template is trying to parse the response as Axios already does that for us.

Generating a client from the pet store: https://petstore.swagger.io/v2/swagger.json with axios and interface DTOs will create a client that has this behavior.

Here is an example config:

{
  "runtime": "Net50",
  "defaultVariables": null,
  "documentGenerator": {
    "fromDocument": {
      "url": "https://petstore.swagger.io/v2/swagger.json",
      "output": null,
      "newLineBehavior": "Auto"
    }
  },
  "codeGenerators": {
    "openApiToTypeScriptClient": {
      "className": "{controller}Client",
      "moduleName": "",
      "namespace": "",
      "typeScriptVersion": 2.7,
      "template": "Axios",
      "promiseType": "Promise",
      "httpClass": "HttpClient",
      "withCredentials": false,
      "useSingletonProvider": false,
      "injectionTokenType": "OpaqueToken",
      "rxJsVersion": 6.0,
      "dateTimeType": "String",
      "nullValue": "Undefined",
      "generateClientClasses": true,
      "generateClientInterfaces": true,
      "generateOptionalParameters": false,
      "exportTypes": true,
      "wrapDtoExceptions": false,
      "exceptionClass": "ApiException",
      "clientBaseClass": null,
      "wrapResponses": false,
      "wrapResponseMethods": [],
      "generateResponseClasses": true,
      "responseClass": "SwaggerResponse",
      "protectedMethods": [],
      "configurationClass": null,
      "useTransformOptionsMethod": false,
      "useTransformResultMethod": false,
      "generateDtoTypes": true,
      "operationGenerationMode": "MultipleClientsFromFirstTagAndOperationId",
      "markOptionalProperties": true,
      "generateCloneMethod": false,
      "typeStyle": "Interface",
      "enumStyle": "Enum",
      "useLeafType": false,
      "classTypes": [],
      "extendedClasses": [],
      "extensionCode": null,
      "generateDefaultValues": true,
      "excludedTypeNames": [],
      "excludedParameterNames": [],
      "handleReferences": false,
      "generateConstructorInterface": true,
      "convertConstructorInterfaceData": false,
      "importRequiredTypes": true,
      "useGetBaseUrlMethod": false,
      "baseUrlTokenName": "API_BASE_URL",
      "queryNullValue": "",
      "useAbortSignal": false,
      "inlineNamedDictionaries": false,
      "inlineNamedAny": false,
      "templateDirectory": null,
      "typeNameGeneratorType": null,
      "propertyNameGeneratorType": null,
      "enumNameGeneratorType": null,
      "serviceHost": null,
      "serviceSchemes": null,
      "output": "index.ts",
      "newLineBehavior": "Auto"
    }
  }
}

[RicoSuter]

But why override that functionality to stop parsing, just to do it ourselves anyway down the line? Doesn't make sense to me

Because some properties (eg dates) need to be serialized differently, and sometimes the property names on the wire (in JSON) do not match the property names in the generated TypeScript classes (eg on the wire we have 'first-name' which cannot be used as a TS/JS property name and is renamed to "firstName").

If you do not care for date conversion or property rename then use interface instead of class output, in this case we could keep axios auto-deserialization enabled - otherwise we need to turn it off.

[RicoSuter]

I just removed JSON.parse(...). It seems that it is not needed at all in the context of axios if I am not wrong

In the case of $ref resolution it probably would be needed as axios does not pass the special reviver function in...

[RicoSuter]

I do not use Axios and I have no clue whether this here is fine or not. What's the current state?
We need to ensure it works with "Interface" and "Class" and ideally with $ref resolution.

[laurilarjo]

@thijscrombeen @RicoSuter is there any update on this?
I've uses nswag successfully with dotnet, but now when trying to use it in our typescript service and axios, I'm facing an error when our endpoint returns [] for empty result.
The suggested axios transformResponse: works, but is really not a performant solution.

I would love to take the nswag generated clients into proper use in our microservice environment, but I need it work well.

Update: I'll be using this without /TypeStyle:Interface so it works the way I need.

[flieks]

i also want to use axios with nswag to generate TS client
with fetch it works fine and it adds this when you check the "handle $values" option

but with axios it doesnt use the jsonParse at all

even though jsonParse is in the generated code (with axios)
but that function is never referenced.

[RicoSuter]

Is this PR still valid?

[aeslinger0]

@RicoSuter Yes, I believe the PR is still valid and would fix several open issues as described here: #5003