srp_jsonX_parse

BarryStevens

I have an issue where sometimes users use [] in text and thus cause a parser error.
should this error really be occurring. If on first line it does not appear to get an error. I understand that [] is a json thing, but, i thought there must be someway the parser can distinguish the real thing.
Is there a workaround with maybe I have to substitute [] with ().

KevinFournier

You're saying that a value in the json is "swwss[222]"?

BarryStevens

yes

KevinFournier

Yeah, that seems wrong. Can you email me the json?

BarryStevens

From the HTTP log:
<29> HTTPPostString : {"new_job_details":["swwws
wswsssws
swwss[222]
swsws"],"long":"150.8278272","lat":"-34.4915968","date":"15/09/2022","end_time":"09:00AM","employee":[184],"job_completed_response":"No"}

KevinFournier

This is not valid json. You can't have hard line breaks in a string. You have to use \r\n in the string for line breaks.

BarryStevens

Ah, so the json should not have been returned to me like that.
So, I will have to convert at my end.
Can you tell me what the swap/convert command should look like please.
I assume I can apply it to the full HTTPPostString

BarryStevens

I will do swap \0A\ with '\r\n' in MyJsonString

Then I assume I will have to convert it back or will the parse do that?

KevinFournier

Are you creating the json using SRP_JsonX or manually? SRP_JsonX will escape line breaks for you. Otherwise you need to do something like:
Swap \0D0A\ with "\r\n" in Text

BarryStevens

A web developer is creating the json and I am getting it via a POST api (httpFrameworks)

KevinFournier

You can't just do a blanket swap on incoming json because any line breaks outside of the strings are legit. The web developer should know better. There is no web service that would accept this. Tools like jsonlint.com can be useful for validating json as proof.

DonBakke

@BarryStevens - Can you confirm that the JSON has line breaks in the original log file? I get the feeling that the line breaks got added when you pasted the JSON into the forum post. If you want, just email me the original log file.

BarryStevens

Yes, it does have a linefeed \0A\
this fixed the issue:

jsonbody = HTTP_Services('DecodePercentString', jsonbody) swap \0D0A\ with '\r\n' in jsonbody swap \0A\ with '\r\n' in jsonbody

DonBakke

Hmmm...something still smells off about this. I find it odd that a web developer would produce malformed JSON, especially if using a library that would prevent this from happening. I also think it is odd that this appeared to work when the [] characters were on the first line of the value. Do you think this was a red herring on your part?

BarryStevens

>>the [] characters were on the first line of the value.
only one line so no LineFeed

>>Do you think this was a red herring on your part?
No, definately \0A\ in the string when multilined

DonBakke

Then the [] issue was a red herring. It was never the problem, correct?

When I read this, "If on first line it does not appear to get an error", I took that to mean, "If the [] is on the first line but not on other lines, it does not appear to get an error". What you really meant was "I only have one line and I do not get an error".

Hence, the only problem was the multiline content. The location of [] had no relevance.

BarryStevens

Initialy the [] was an issue in another section that was displaying data that had [] in it.
The web developer fixed that.
I thought this could be the same issue, but, yes it was a red herring.
Btw: I think fixing the [] caused this error as I am sure it was working in a previous version, but I don't have a very good test bed to check it.

BarryStevens

@donbakke
Attached is a log file that has the multiline comments section in the "new_job_details" returned, with CRLF as line separator.

Can I assume then this is EXACTLY what was received and you have not formatted it, because the developer says he can see anything wrong, but, this is not his test data and maybe did not do mutli-line.
Time zone issue so am waiting on a response this afternoon from him.

BarryStevens

jsonbody = HTTP_Services('GetHTTPPostString')
What actual charactor should I be seeing at this point before the 'DecodePercentString'
Is it a %char

BarryStevens

Sorry, but doing a bit more investigating and now am baffled, so definitely need your help.

jsonbody = HTTP_Services('GetHTTPPostString')
debug
I see %0A as the LF, which I believe is correct
BUT
jsonbody = HTTP_Services('DecodePercentString', jsonbody)
comes back with \0A\

screenshots attached.

KevinFournier

It looks like the web developer is sending carriage returns in the body. OECGI likes to URL Encode a lot of data, so a carriage return would be encoded as %0A. That is not good. Just because the carriage return is nicely encoded in the html body does not mean the json is well formed. There is still a carriage return in the json. Instead of %0A, you want to see '\r' or '%5Cr'.

BarryStevens

Ok, I will do the conversion myself after HTTP_Services('GetHTTPPostString')

KevinFournier

This is not wise. You shouldn't have to do the conversion. The web developer is sending you json that no system on earth would accept. This is on them. If you choose to fix it, you should make sure not to swap out every instance of carriage return, only those that appear within strings, unless you are 100% certain you will never receive formatted json.

BarryStevens

thanks