1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Take Command console tab AND TCC are not Chinese friendly

Discussion in 'Support' started by roytam1, Apr 2, 2010.

  1. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    I download Take Command 11 Trial today and I found that Take Command console tab AND TCC are not Wide Char(Chinese/Japanese/Korean/etc.) friendly.

    (The upload function in the forum is too restricted I use xs.to instead.)

    The Chinese characters is showing half and the aligning is in a mess.
    [​IMG]

    While in console window works better.
    [​IMG]

    but the redirection/pipe from dir to external program or file is broken:
    [​IMG]
    Opening the output text file in hex editor, you can see that the redirected output use Unicode character count for truncating ANSI string, resulting truncated ANSI output. (This issue seems applied in TCC/LE 11.00.44)

    and the TYPE command is just inverted situation. Type command use ANSI byte count for Unicode output, resulting accessing uninitialized memory. (This issue seems *NOT* applied in TCC/LE 11.00.44)
    [​IMG]
     
  2. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    Take Command does not currently support wide character sets.

    Rex Conn
    JP Software
     
  3. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    But what about TCC and TCC/LE?
    I can just use the console window instead, but dir/type commands(maybe this issue affect all internal commands) corrupt the output which REALLY NEED fixing as soon as possible because they affect Asians from using TCC and TCC/LE very much.
     
  4. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    We cannot support that environment, as we do not have any in-house expertise
    with wide character sets, nor do we have help available in anything other
    than English, French, and German.

    Rex Conn
    JP Software
     
  5. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    Did you specify Unicode output (OPTION / Startup)? The default is ASCII
    output, and there's no way this would work with ASCII.


    Can you email me that text file? (And more details on your environment,
    such as the code page & font you're using.)

    Rex Conn
    JP Software
     
  6. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    Is this a Unicode (UTF-16) file (which should work) or a DBCS file (which
    definitely will not)?

    Rex Conn
    JP Software
     
  7. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    [​IMG]
    File:
    http://www.jpsoft.com/forums/attachment.php?attachmentid=125&stc=1&d=1270361530
    This is a Big5(ANSI/CP 950) encoded file.
    As I need compatibility with cmd.exe, I cannot use Unicode output.

    In cmd.exe, I can use Big5(ANSI/CP 950) and UTF-8(CP 65001). But in TCC and TCC/LE, UTF-8 dir output is also broken.
     

    Attached Files:

    • test.zip
      File size:
      132 bytes
      Views:
      2
  8. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    CMD.EXE also supports Unicode output.


    TCC does not support ANSI DBCS or UTF-8 (though I'm unaware of support for
    UTF-8 in CMD either).

    Rex Conn
    JP Software
     
  9. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    Related or not, the following behavior is odd. After "CHCP 65001", I see the following (these are not edited):

    Code:
    v:\> for /l %i in (125,1,135) echo %i %@char[%i]
    125 }
    126 ~
    127 
    135 ‡
    v:\> for /l %i in (125,1,136) echo %i %@char[%i]
    125 }
    126 ~
    127 
    136 ˆ
    The counted FOR loop stops at 127 and later prints the last number of the specified range.

    There is no problem echoing the missing ones separately:

    Code:
    v:\> echo 128 %@char[128]
    128 €
    v:\> echo 129 %@char[129]
    129 Â
    v:\> echo 130 %@char[130]
    130 ‚
     
  10. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    I see what's happening (I think). For some reason (double-byte character?) the newline is not echoed. So each line overwrites the previous one and I see only the last one.

     
  11. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    but external program doesn't support Unicode I/O, as a result I have to use ANSI.
     
  12. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    This has nothing to do with FOR, ECHO, %@CHAR, or TCC.

    The string is generated correctly, but the Windows WriteConsole API is
    choking (returning a "ERROR_GEN_FAILURE") when it contains those characters.

    Rex Conn
    JP Software
     
  13. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    Actually properly supporting so-called "wide character" is easy: take care the byte count after converting to target codepage.

    There is an very good example is posted in stackoverflow.com:
    http://stackoverflow.com/questions/215963/how-do-you-properly-use-widechartomultibyte

    for example, you can create a file called "«®©».txt", DIR outputing in UTF-8(CP 65001) which will be represented in "«®©».txt" in ANSI form.
    the original filename string is 8 wchar_t in length, but after converting to UTF-8 it will expended to 12 char in length.
    "—"(U+2014 EM Dash, 1 wchar_t in length) will be represented in "—"(3 char in length).

    So allocate/modify to enough size of the output buffer to length of converted string before writing to pipe/file, that's it.

    And so do TYPE command, doing the same action and resize/truncate with L'\0' of output buffer before writing to pipe/file, that's it. And this change also fixes an uninitialized memory access vulnerability when TYPE a MBCS file.

    I don't think JPSoft can't do such security bugfix for future version.
     
  14. roytam1

    Joined:
    Apr 1, 2010
    Messages:
    43
    Likes Received:
    0
    if you use True Type Font but not Bitmap Font, you will be able to have a complete loop.

    Code:
    [F:\TCCLE]for /l %i in (125,1,135) echo %i %@char[%i]
    125 }
    126 ~
    127 
    128 €
    129 
    130 ‚
    131 ƒ
    132 „
    133 …
    134 †
    135 ‡
    
    [F:\TCCLE]
    
    
     

Share This Page