Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem with Byte Order Mark #184

Closed
daschfg opened this issue Jul 3, 2017 · 4 comments
Closed

Problem with Byte Order Mark #184

daschfg opened this issue Jul 3, 2017 · 4 comments
Milestone

Comments

@daschfg
Copy link

daschfg commented Jul 3, 2017

Hi,

I just ran into a problem parsing a string beginning with a Byte Order Mark.
I'm not sure about it, but my understanding is that it remains a valid UTF-8 string even with BOM, therefore it should still be parsed. Or is this out of scope for this library and the caller should take care of it and then just supply the string without BOM?

@FSMaxB
Copy link
Collaborator

FSMaxB commented Jul 3, 2017

RFC 7159 states the following in section 8.1:

Implementations MUST NOT add a byte order mark to the beginning of a JSON text. In the interests of interoperability, implementations that parse JSON texts MAY ignore the presence of a byte order mark rather than treating it as an error.

Which means that I could choose to support parsing UTF-8 with BOM but I don't have to.

Since it is just a small change that doesn't hurt anyone I think I'll do it, although I'm not sure where BOMs for UTF-8 are actually used. (Haven't seen one so far).

@FSMaxB FSMaxB added this to the 1.6 milestone Jul 3, 2017
@daschfg
Copy link
Author

daschfg commented Jul 3, 2017

Ok, thank you for clearing that up for me.
I just stumbled upon them after editing a UTF-8 file in Microsofts Editor, apparently it adds a BOM by default. Besides that, I haven't seen them so far as well.

@FSMaxB
Copy link
Collaborator

FSMaxB commented Jul 3, 2017

This is now implemented on the develop branch in 5baa77f

@FSMaxB
Copy link
Collaborator

FSMaxB commented Oct 8, 2017

1.6 has been released with a fix for this

@FSMaxB FSMaxB closed this as completed Oct 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants