Adds option to specify character set in responses (with http adapter) #869

johntron · 2017-04-27T16:17:12Z

We have a service that returns responses encoded with the ISO-8859-1 character set, and needed a way to decode these correctly. This change allows us to decode correctly in our server:

axios.post('http://service-with-latin-encoding.com', {}, { charset: 'latin1' })
  .then(/* do something */);

coveralls · 2017-04-27T16:20:56Z

Coverage increased (+1.5%) to 93.719% when pulling 5c825a7 on johntron:master into f31317a on mzabriskie:master.

johntron · 2017-06-23T20:59:30Z

@nickuraltsev @rubennorte thanks for maintaining this project. Would you mind taking a look at this PR? It's a simple, backwards-compatible improvement for server-side requests - even comes with updated README :D

coveralls · 2017-06-23T21:00:37Z

Coverage remained the same at 93.734% when pulling 83406f8 on johntron:master into 46e275c on mzabriskie:master.

coveralls · 2017-06-23T21:07:06Z

Coverage remained the same at 93.734% when pulling f600705 on johntron:master into 46e275c on mzabriskie:master.

coveralls · 2017-06-23T21:08:19Z

Coverage remained the same at 93.734% when pulling f600705 on johntron:master into 46e275c on mzabriskie:master.

rubennorte · 2017-08-12T12:49:18Z

lib/adapters/http.js

@@ -185,7 +185,7 @@ module.exports = function httpAdapter(config) {
        stream.on('end', function handleStreamEnd() {
          var responseData = Buffer.concat(responseBuffer);
          if (config.responseType !== 'arraybuffer') {
-            responseData = responseData.toString('utf8');
+            responseData = responseData.toString(config.charset || 'utf-8');


The default is utf8 (without the dash) but as it's the Node Buffer default value you don't need to set the || 'utf8, you can just pass the config.charset.

rubennorte · 2017-08-12T12:50:48Z

README.md

@@ -280,6 +280,10 @@ These are the available config options for making requests. Only the `url` is re
  // options are 'arraybuffer', 'blob', 'document', 'json', 'text', 'stream'
  responseType: 'json', // default

+  // `charset` indicates charset to use for decoding responses
+  // Note: Ignored for `responseType` of 'stream' or client-side requests
+  charset: 'utf-8', // default


I'd use responseEncoding (adding the response prefix to make it clearer and using encoding to match the Node.js naming).

johntron · 2017-08-16T23:14:36Z

Thanks so much @rubennorte - I'll make those changes when I get home.

…pter

johntron · 2017-08-16T23:36:51Z

Updated

coveralls · 2017-08-16T23:39:05Z

Coverage remained the same at 93.812% when pulling 7340c5d on johntron:master into 07a7b7c on mzabriskie:master.

tnhu · 2017-10-05T23:51:31Z

I think it would be better to extract charset from response header content-type. This way you don't have to manually pass the charset per request.

johntron · 2017-10-08T20:06:28Z

I agree, but that can be done in a future PR based on the work I've done here. No reason we can't support both methods.

…

On Thu, Oct 5, 2017, 18:51 Tan Nhu ***@***.***> wrote: I think it would be better to extract charset from response header content-type. This way you don't have to manually pass the charset per request. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#869 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAIDeZ8yLgTrdZNJcpc3T1npR5aNmg1fks5spWuXgaJpZM4NKcyk> .

johntron · 2018-03-02T16:21:16Z

@rubennorte I made those changes in August of last year. Can this be merged yet?

mawrkus · 2018-03-10T11:15:52Z

Hi @rubennorte ! Any progress on this? I have a similar use case.

nickuraltsev · 2018-03-10T17:12:56Z

@johntron Sorry for the delay! I'm going to merge it now.

mawrkus · 2018-03-11T00:46:50Z

Thanks for the merge!

lepture · 2018-05-06T10:46:09Z

The implementation would be better in this way:

responseEncoding default value should be null
when responseEncoding has a value, use this encoding
otherwise use the charset in Content-Type header
fallback to utf-8

It seems that you need to use iconv-lite, since the default supported encodings in Node is very limited.

johntron · 2018-05-06T19:37:39Z

@lepture this PR has already been merged, but feel free to submit another PR. This one served my needs, and I would not like the added dependency for other encodings - I've never worked with an API that returned anything other than UTF-8 or latin-1.

johntron force-pushed the master branch 3 times, most recently from 83406f8 to f600705 Compare June 23, 2017 20:57

rubennorte requested changes Aug 12, 2017

View reviewed changes

Adds option to specify character set in responses when using http ada…

7340c5d

…pter

johntron force-pushed the master branch from f600705 to 7340c5d Compare August 16, 2017 23:36

neverendingqs mentioned this pull request Feb 11, 2018

Encoding issue with ISO-8859-1 #501

Closed

emilyemorehouse force-pushed the master branch 2 times, most recently from 2760755 to 48a7902 Compare February 19, 2018 23:22

nickuraltsev merged commit 9a6abd7 into axios:master Mar 10, 2018

jjeejj mentioned this pull request May 6, 2018

on the latest release 0.18.0 and npm package version 0.18.0 , lack of some functions #1519

Closed

JaakkoL mentioned this pull request May 7, 2018

File encoding with drive API broken googleapis/google-api-nodejs-client#1151

Closed

danielsalles mentioned this pull request May 24, 2018

charset latin1 not working #1561

Closed

ehrhart mentioned this pull request Aug 20, 2018

Adding responseEncoding to mergeConfig #1745

Merged

sdepold mentioned this pull request Nov 1, 2019

Problem with non-UTF8 encoded RSS feed sdepold/jquery-rss#141

Closed

axios locked and limited conversation to collaborators May 3, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds option to specify character set in responses (with http adapter) #869

Adds option to specify character set in responses (with http adapter) #869

johntron commented Apr 27, 2017

coveralls commented Apr 27, 2017

johntron commented Jun 23, 2017

coveralls commented Jun 23, 2017

coveralls commented Jun 23, 2017

coveralls commented Jun 23, 2017

rubennorte Aug 12, 2017

johntron Aug 16, 2017

rubennorte Aug 12, 2017

johntron Aug 16, 2017

johntron commented Aug 16, 2017

johntron commented Aug 16, 2017

coveralls commented Aug 16, 2017

tnhu commented Oct 5, 2017

johntron commented Oct 8, 2017 via email

johntron commented Mar 2, 2018

mawrkus commented Mar 10, 2018

nickuraltsev commented Mar 10, 2018

mawrkus commented Mar 11, 2018 •

edited

Loading

lepture commented May 6, 2018 •

edited

Loading

johntron commented May 6, 2018

Adds option to specify character set in responses (with http adapter) #869

Adds option to specify character set in responses (with http adapter) #869

Conversation

johntron commented Apr 27, 2017

coveralls commented Apr 27, 2017

johntron commented Jun 23, 2017

coveralls commented Jun 23, 2017

coveralls commented Jun 23, 2017

coveralls commented Jun 23, 2017

rubennorte Aug 12, 2017

Choose a reason for hiding this comment

johntron Aug 16, 2017

Choose a reason for hiding this comment

rubennorte Aug 12, 2017

Choose a reason for hiding this comment

johntron Aug 16, 2017

Choose a reason for hiding this comment

johntron commented Aug 16, 2017

johntron commented Aug 16, 2017

coveralls commented Aug 16, 2017

tnhu commented Oct 5, 2017

johntron commented Oct 8, 2017 via email

johntron commented Mar 2, 2018

mawrkus commented Mar 10, 2018

nickuraltsev commented Mar 10, 2018

mawrkus commented Mar 11, 2018 • edited Loading

lepture commented May 6, 2018 • edited Loading

johntron commented May 6, 2018

mawrkus commented Mar 11, 2018 •

edited

Loading

lepture commented May 6, 2018 •

edited

Loading