Pasting Unicode data has different behavior on TCC and CMD

Aug 21, 2014
18
1
I just read about Microsoft's enhancements to the low level Unicode handling on command prompts on 1809, so I decided to try it. I copied some Unicode text that's not in Code Page 437 (example: ░▒▓) and tried pasting it into TCC.

On TCC running through Take Command, I get "°±²".
On TCC running as a command prompt, I also get "°±²".
On CMD, I get the correct value "░▒▓"

It seems to do this if I use Ctrl+V or paste via the system menu. It seems like there is some different behavior between the way Unicode text is handled.

TCC does have UTF-8 support enabled (via OPTION).
 

rconn

Administrator
Staff member
May 14, 2008
12,404
152
That's not quite what's happening -- none of those characters are Unicode. They're extended ASCII characters (8 bit) and their representation depends on the font & character set selected.

If you select a Unicode font and the Unicode character set, you will get °±². If you select a Unicode font and a DOS character set, you will get ░▒▓ . Which one is "correct"? (The characters are the same in either case - 0xB0, 0xB1, 0xB2.)

CMD is "sort of Unicode", except for file handling and some keyboard I/O. TCC is fully Unicode.
 
Aug 21, 2014
18
1
Interesting. I just checked other random Unicode characters that are represented in my console font (Consolas) and they do seem to work. It only seems like a problem with the characters that are in the classic DOS code page but not 437.

I copied those characters from the "Character Map" tool. I verified that the clipboard data (CF_UNICODETEXT) is U+2591, U+2592, U+2593 and the CF_TEXT is "???" since none of those characters are in my code page (437).
 
Similar threads
Thread starter Title Forum Replies Date
vefatica Pasting into TCMD via right-click menu? Support 2
Peter Murschall TEE cannot handle Unicode output Support 2
B Fullwidth Unicode forms display incorrectly Support 5
T @execstr unicode support Support 6
Peter Murschall TPIPE generate unicode on Piping or redirecting Support 3
vefatica TYPE goes crazy with no-BOM Unicode file Support 7
Charles Dye TCC smashing Unicode quotes Support 9
Peter Murschall UNICODE mixed with ANSI Code Support 11
Joe Caverly Unicode, Codepage 437, and line characters Support 3
B How to? Convert Unicode to ANSI Support 1
StarliteLemming Fileread fails on Unicode file Support 10
vefatica DO ... /P ... and Unicode? Support 3
vefatica Unicode ... I don't understand Support 1
jadaml Echo unicode characters from UTF-8 Batch files? Support 1
Charles Dye @ASCII vs. @UNICODE Support 5
A How to? Filter history list with unicode chars Support 0
vefatica TYPE, Unicode, installer Support 10
A WAD Limitations on display of unicode punctuation marks Support 11
A Include lists and Unicode Support 1
M How to? How do I read a Unicode file through standard-input? Support 4
M WAD A bit of strangeness related to Unicode-marked file not being Unicode Support 2
M @CHAR and UNICODE Support 4
D LIST command wierdness with empty Unicode file Support 1
B Unicode/dword issue in TCC12 Support 4
J dir failure with some unicode characters Support 6
M TCC Unicode support? Support 7
vefatica BOMs in [dir]history / TAIL with Unicode Support 2
vefatica Unicode screw-up in IDE Support 4
vefatica Unicode anomaly Support 0
vefatica Debugger now Unicode? Support 1
vefatica TYPE /X and Unicode files? Support 0
dcantor Convert ASCII to Unicode or vice versa? Support 6
H HISTORY and DIRHISTORY /R can't handle Unicode Support 0
R Reading an Unicode file with more than 8191 lines Support 1
S How to? best way to compose data for command to use out of presets/variables Support 1
vefatica Finding files with alternate data streams? Support 4
J WAD Output redirection in tcexit looses data Support 31
C Win7 Program Data.... Support 2
vefatica Fixed TYPE http://... incomplete data Support 40
T Lua sharing data with batch files Support 2
D ftps gives "Unable to build data connection" Support 5
T @ping: response code, data size and documentation incorrect Support 2
p.f.moore Best way of embedding data in a BTM file Support 12
A GPF when copying large amount of data on busy USB Drive Support 3

Similar threads