Welcome to the SRP Forum! Please refer to the SRP Forum FAQ post if you have any questions regarding how the forum works.
getting what browser displays
Don, continued from RevForum:
url=https://epathway.wollongong.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/EnquiryDetailView.aspx?Id=155699
This is a similar topic that was discussed in this recent thread titled the same as yours. The simple answer is that browsers do magic that OLE_GetWebPage doesn't. OLE_GetWebPage will return the content available at the endpoint you specify (assuming, of course, other meta-data is available such as the proper request headers, cookies, etc.). However, browsers also perform actions such as redirects...which is what you are seeing happening with that URL you are trying to test.
OLE_GetWebPage is a capable routine for quick calls to an endpoint, but I recommend using a full featured API (or endpoint) tester like Postman or cURL so you can see the entire response and other important data such as the status code. In this case, the status code returned is 302 (which means Found) and a response header called Location which tells the client to navigate to the URL specified. Your browser does this automatically but OLE_GetWebPage is only designed to return the immediate information.
Is this an app that has the HTTP Framework installed? If so, you could also use the SendHTTPRequest service which does allow you to see the status code and response headers using the GetResponseStatusCode and GetResponseHeaderFields services.
Did:
responseBody = HTTPClient_Services('SendHTTPRequest', "GET", URL, HeaderList, Body, ProxyUser, ProxyPassword, UseAsynchronous, UseXMLHTTP)
GetResponseStatusCode=HTTPClient_Services('GetResponseStatusCode')
GetResponseHeaderFields=HTTPClient_Services('GetResponseHeaderFields')
Result:=
ResponseBody same as ole_getwebpage
GetResponseStatusCode = 200
Did a GET url in postman, same responses.
Did not see any 'Location' you mentioned.
I have absolutely no idea how to proceed from here to get the info that is browser rendered for the url
url=https://epathway.wollongong.nsw.gov.au/ePathway/Production/Web/GeneralEnquiry/EnquiryDetailView.aspx?Id=155699
This is a similar topic that was discussed in this recent thread titled the same as yours. The simple answer is that browsers do magic that OLE_GetWebPage doesn't. OLE_GetWebPage will return the content available at the endpoint you specify (assuming, of course, other meta-data is available such as the proper request headers, cookies, etc.). However, browsers also perform actions such as redirects...which is what you are seeing happening with that URL you are trying to test.
OLE_GetWebPage is a capable routine for quick calls to an endpoint, but I recommend using a full featured API (or endpoint) tester like Postman or cURL so you can see the entire response and other important data such as the status code. In this case, the status code returned is 302 (which means Found) and a response header called Location which tells the client to navigate to the URL specified. Your browser does this automatically but OLE_GetWebPage is only designed to return the immediate information.
Is this an app that has the HTTP Framework installed? If so, you could also use the SendHTTPRequest service which does allow you to see the status code and response headers using the GetResponseStatusCode and GetResponseHeaderFields services.
Did:
responseBody = HTTPClient_Services('SendHTTPRequest', "GET", URL, HeaderList, Body, ProxyUser, ProxyPassword, UseAsynchronous, UseXMLHTTP)
GetResponseStatusCode=HTTPClient_Services('GetResponseStatusCode')
GetResponseHeaderFields=HTTPClient_Services('GetResponseHeaderFields')
Result:=
ResponseBody same as ole_getwebpage
GetResponseStatusCode = 200
Did a GET url in postman, same responses.
Did not see any 'Location' you mentioned.
I have absolutely no idea how to proceed from here to get the info that is browser rendered for the url
Comments
I was wrong about the SendHTTPRequest service (along with OLE_GetWebPage). These will follow the redirect automatically, just as a browser does. Thus, the information that is returned by these routines should be the same as what you see in the browser. In my quick tests I'm seeing the same content.