Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BOM not skipped when using json:parse(iterator) #602

Closed
jhayes-dev opened this issue Jun 1, 2017 · 4 comments
Closed

BOM not skipped when using json:parse(iterator) #602

jhayes-dev opened this issue Jun 1, 2017 · 4 comments
Assignees
Labels
confirmed kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Milestone

Comments

@jhayes-dev
Copy link

jhayes-dev commented Jun 1, 2017

The byte-order mark is not being skipped when I use json:parse(iterator) interface for example:
nlohmann::json::parse(static_cast<const char *>(buffer), static_cast<const char *>(buffer) + nbytes)

It looks like the json::parse(stream) interface does skip the BOM.

Here is a quick fix that seems to work for me:
json.txt

@jaredgrubb
Copy link
Contributor

Interesting question ... FWIW, Python's json parser does not support this directly either:
https://stackoverflow.com/questions/13156395/python-load-json-file-with-utf-8-bom-header

@nlohmann
Copy link
Owner

nlohmann commented Jun 2, 2017

I can confirm the observation. With the introduction of input adapters, I forgot to replicate the BOM skipping for all input types. Your fix seems valid. I shall add it as soon as I'm back from traveling.

@nlohmann nlohmann added this to the Release 3.0.0 milestone Jun 2, 2017
@nlohmann nlohmann self-assigned this Jun 2, 2017
nlohmann added a commit that referenced this issue Jun 2, 2017
I totally forgot about byte order marks in this scenario.
@nlohmann nlohmann added the solution: proposed fix a fix for the issue has been proposed and waits for confirmation label Jun 2, 2017
@nlohmann
Copy link
Owner

nlohmann commented Jun 2, 2017

Commit d19c5ce is a fix for the develop branch.

@red632nm It would be great if you could check if this fixes the problem.
@jaredgrubb I didn't know Python would not skip by default. I like the way RFC 7159 describes this:

In the interests of interoperability, implementations that parse JSON texts MAY ignore the presence of a byte order mark rather than treating it as an error.

@nlohmann nlohmann closed this as completed Jun 7, 2017
@jaredgrubb
Copy link
Contributor

@nlohmann : Thanks for the note! If it's a MAY, then totally agree that fixing is appropriate. Thanks for the good work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
confirmed kind: bug solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Projects
None yet
Development

No branches or pull requests

3 participants