Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parsing multiple JSON objects from a string or stream #438

Closed
szikra opened this issue Jan 28, 2017 · 3 comments
Closed

Parsing multiple JSON objects from a string or stream #438

szikra opened this issue Jan 28, 2017 · 3 comments
Labels
confirmed kind: bug solution: duplicate the issue is a duplicate; refer to the linked issue instead

Comments

@szikra
Copy link

szikra commented Jan 28, 2017

Hi, I just started using your library, so far I like your design, but I run into this problem.

Example input (received from TCP socket):

{"data":[164,227,155,136],"dir":"rx"}{"data":[249,88,104,174],"dir":"rx"}{"data":[148,105,74,58],"dir":"rx"}

Code fragment:

        string smsg((char*)buff, len);
        if (debuglevel_&1) cout << "Raw data Str: " << smsg << endl;
        json jmsg = json::parse(smsg);

Alternative code:

        stringstream ss;
        ss<<smsg;
        while (! ss.eof()) {
            json jmsg;
            ss >> jmsg;

Output:

parse error - unexpected '{'; expected end of input

Expected result:
I would like to be able to parse multiple concatenated JSON strings.

A possible implementation could be if the result were an array of json objects. like ‘[{…},{…},{…}]’
This could be achieved by replacing ‘}{‘ with ‘},{‘ and inserting a ‘[‘ and a ‘]’ at the beginning and ending of the string. (My current workaround.) But this is not a very robust implementation, would need to replace all possible ‘}\r\n{‘ … variant, not to mention strings containing “}{”. Also not very efficient, needs to pseudo pre parse the input. (It would help somewhat if json could accept whitespace* separated objects in arrays not requiring comma, though that would not be valid JSON. This could be a relaxed parsing, instead of strict.)

Second possible implementation (preferred IMHO) for the parsing to be resumable. Meaning extract one JSON object from a stream, leave the read position and start from there the next time. Or in case if it’s a string provide a method to query how much characters were processed, so that part can be “removed” from the input string and the rest can be parsed again. (Like: string smsg ((char*)&buff[position], len-position); … position+=jmsg.??)

I have found these related issues:
#367
#210
The suggested workaround here (counting matching ‘{‘ ’}’ would not work if any string inside the JSON contained non matching ‘{‘,’}’ characters like "{"test3":"}"}").

Thank you for all your work and efforts.

@nlohmann nlohmann added confirmed kind: bug solution: duplicate the issue is a duplicate; refer to the linked issue instead labels Jan 28, 2017
@nlohmann
Copy link
Owner

Yes, this should be fixed with #367. I shall close this ticket.

When I added support for streams, I never thought about the fact that streams could be reused. I hope to be able to add this feature without too much refactoring.

@nlohmann
Copy link
Owner

@szikra Sorry if closing this seems rude - please feel free to comment on #367.

@pallas
Copy link

pallas commented Feb 1, 2019

I can still reproduce this on the latest development branch. The workaround is

    nlohmann::json j;
    while (!std::cin.eof() && std::cin >> j >> std::ws)
        std::cout << j << std::endl;

but it still feels like the following should not throw

    nlohmann::json j;
    while (std::cin >> j)
        std::cout << j << std::endl;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
confirmed kind: bug solution: duplicate the issue is a duplicate; refer to the linked issue instead
Projects
None yet
Development

No branches or pull requests

3 participants