Welcome to the SRP Forum! Please refer to the SRP Forum FAQ post if you have any questions regarding how the forum works.

SRP_Run_Command with multi-byte command characters

A customer has found a regression that appears to have come about since SRP Utilities v2.1.6 . I've tracked it down a difference in the way SRP_Run_Command() works, and suspect the changes made for "Fixed bug in SRP_Run_Command where output would not go into variable if it was too long"

We're using the ImageMagick DOS programs to create thumbnails. This now fails when processing an image that contains unicode multi-byte characters in the filename. As an example, when processing "F:\media\飞蛾.jpg" via SRP_Run_Command(), the response is an error "convert.exe: unable to open image 'F:\media\飞蛾.jpg': No such file or directory"

A simpler test would be a DOS COPY command. From v2.1.6, the command "COPY F:\Media\æ.jpg C:\TEMP" just returns "The system cannot find the file specified." - whereas before you'd get "1 file(s) copied."

It appears that the multi-byte characters are being interpreted as single-byte?

This still occurs in SRP Utilities version 2.2.0.3 32bit. However 64bit seems ok!

Any chance of a 32bit fix for this OI9? :)
Cheers, M@

Comments

  • edited September 2021
    Try 2.2.1.2. The problem was actually the Win32 API. I was calling CreateProcessA, which is supposed to convert everything to Unicode and call CreateProcessW. Apparently the 64-bit version does this, but not the 32-bit version. This version of SRP Utilities does the Unicode conversion manually and calls CreateProcessW directly.
  • Brilliant! - thanks for this. Yes, that fixes it.

    In testing this I found that it also fixed another quirk in that previous versions (even as far back as v1.6) would return multiple results for the command "dir f:\media\飞蛾.jpg /b", depending on what else was in that directory. It should only return one result, of course.

    But curiously, the 64bit version (still at v2.2.0.3) still returns multiple results for this command. Eg,
    run evalv "@ans = 'VAR'` call srp_run_command( 'dir f:\media\飞蛾.jpg /b', @ans)"
    73.jpg
    c.jpg
    ‘.jpg
    ??.jpg

    This doesn't affect us at the moment, but it's probably worth noting the difference ;).

    Cheers, M@
  • Here is 2.2.1.2 64-bit if you want to test if this quirk is resolved in OI10.
  • Hmm?, slightly different but not quite there..
    _run evalv "@ans = 'VAR'` Call SRP_Run_Command( 'dir f:\media\飞蛾.jpg /b', @ans)"
    73.jpg
    c.jpg
    �.jpg
    ??.jpg

    The �.jpg filename is for the 飞蛾.jpg requested.
    ??.jpg is for æ.jpg

    Actually, it doesn't recognise æ.jpg directly:
    _run evalv "@ans = 'VAR'` Call SRP_Run_Command( 'dir f:\media\æ.jpg /b', @ans)"
    File Not Found


    Works in 32bit !?
    Thanks, M@
  • These 2.2.1.3 RDKs should do the trick. Commands like Dir and Copy aren't actually EXE's, so they have to be routed through CMD.EXE, which was outputing everything as ASCII. A simple flag cleared that up. Sadly, though, other commands like IPCONFIG only output in ASCII, so I also had to add code to dynamically determine when output was Unicode or when it was ASCII. It should all look correct now.
  • Ah, thanks for the explanation Kevin!
    This is looking even better in 32bit with the correct unicode characters coming back in the output.

    The 64bit is still having problems though:
    _run evalv "@ans = 'VAR'` Call SRP_Run_Command( 'dir f:\media\飞蛾.jpg /b', @ans)"
    73.jpg
    c.jpg
    æ.jpg
    飞蛾.jpg


    Specifically for our case, I'm trying to extract the metadata of an image, eg for æ.jpg I get
    _run evalv "@ans = 'VAR'` c= 'C:\Program Files (x86)\Vernon Systems\ImageMagick\identify.exe -ping -verbose ': quote( 'F:\media\æ.jpg')` Call SRP_Run_Command( c, @ans)"
    identify.exe: UnableToOpenBlob 'F:\media\�.jpg': No such file or directory @ error/blob.c/OpenBlob/2691.


    It seems the problem now is with translation to the command line as well? I've tried this again with version 2.2.0.3 and it does recognise the file, even though the characters on output aren't right
    _run evalv "@ans = 'VAR'` c= 'C:\Program Files (x86)\Vernon Systems\ImageMagick\identify.exe -ping -verbose ': quote( 'F:\media\æ.jpg')` Call SRP_Run_Command( c, @ans)"
    Image: F:\media\æ.jpg
    Format: JPEG (Joint Photographic Experts Group JFIF format)
    Mime type: image/jpeg
    Class: DirectClass
    Geometry: 468x813+0+0
    Resolution: 300x300
    So there seems to be a slight regression somewhere between 2.2.0.3 and 2.2.1.3 for 64bit.

    Cheers, M@
  • I updated the DLL stub to use LPWSTR to force OI to do all the encoding/decoding. Version 2.2.1.4 should work.
  • Thanks for persisting with this Kevin ;).

    Ok, that works for æ.jpg - but 飞蛾.jpg is still being interpreted as multiple files at DOS. If I try to extract the metadata for this image, it gives me the metadata for 73.jpg as well (but not the other two!?). Perhaps there's just something very odd about the encoding of those chinese characters.
    And just confirming the 32bit is all ok.

    Cheers, M@
  • Give 2.2.1.5 a try. I confirmed that in OI10, the unicode is preserved both going into SRP_Run_Command and coming out, which can be confirmed using the DIR command. If this doesn't work, then I suspect that ImageMagick isn't handling the encoding correctly.
  • Hmm, I'm getting results similar to prior versions. v2.2.1.4 seems to behave the best out of all of these patches, certainly for 32bit. I think it must be a very rare 64bit quirk that's not worth worrying about - but I'd say 2.2.1.4 would be the better one going forward ;).

    Thanks for your time on this :)
    Cheers, M@
  • Can you to be more specific as to why 2.2.1.4 is better?
  • The output comes back to OI with the correct characters:

    In 2.2.1.4, 64bit
    _run evalv "c= 'dir F:\media\æ.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    æ.jpg

    _run evalv "c= 'dir F:\media\飞蛾.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    73.jpg
    c.jpg
    æ.jpg
    ??.jpg

    In 2.2.1.4, 32bit (perfect :)
    _run evalv "c= 'dir F:\media\æ.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    æ.jpg
    .
    _run evalv "c= 'dir F:\media\飞蛾.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    飞蛾.jpg


    But in 2.2.1.5, 64bit
    _run evalv "c= 'dir F:\media\æ.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    ‘.jpg

    _run evalv "c= 'dir F:\media\飞蛾.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    73.jpg
    c.jpg
    ‘.jpg
    ??.jpg

    and 2.2.1.5, 32bit
    _run evalv "c= 'dir F:\media\æ.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    ‘.jpg
    .
    _run evalv "c= 'dir F:\media\飞蛾.jpg /b'` @ans = 'VAR'` Call SRP_Run_Command( c, @ans)"
    ??.jpg
  • Oh dear, we've now discovered a different problem with v2.2.1.5 (and 4) 32bit - and this is quite weird! We're calling ImageMagic via SRP_Run_Command() just to return the dimensions of a thumbnail image, eg '75x114'. It seems that in cases where the width of the image is less than 100 pixels, the return value comes back as unicode characters.

    Eg, 2.2.1.4 32bit
    _run evalv "@ans = 'VAR'` c= 'C:\Program Files (x86)\Vernon Systems\ImageMagick\identify.exe -ping -format %G ': quote( 'C:\T\1993.1313.bmp')` call srp_run_command( c, @ans)"
    㔷ㅸ㐱

    Using 2.1.6, this returns correctly as '75x114'

    If I physically rotate the image 90 degrees, SRP_Run_Command() correctly returns '114x75'
    Rotate it again and it comes back as '㔷ㅸ㐱' instead of '75x114'

    This return value should just be a simple text string. Why the difference? Could ImageMagick be setting some kind of DOS return code that v2.2.1.5 is sensitive to??

    Hope you can help. I've already deployed this and might have to revert back to v2.1.6
    Cheers, M@.
  • I downloaded ImageMagick. Email me the image and all the steps (including the commands to rotate), and I'll see if I can't figure out what's happening.
  • Try SRP Utilities 2.2.2.0. I think I've finally settled on the best solution for handling Unicode.
  • Yes!! This works for all the tests above, 32 and 64 bit, and in our application.
    Thanks very much Kevin :)
Sign In or Register to comment.