Problem with UnicodeOutput

May 20, 2008
603
0
Sammamish, WA
I recently set UnicodeOutout=Yes and then got some very funny results.

I'm using "cmd /?" simply as a test program that generates text output.

This exists at least back to 4NT 6, and I don't understand why it
would behave this way. Appending to the clipboard appears to be
different than just writing to it. Using "LIST CLIP:" shows that the
first part of the file is Unicode (16 bit wide characters), but the
second half is ASCII (8 bit wide characters).

D:\>Option //UnicodeOutput=No
D:\>cmd /? > clip:
D:\>cmd /? >>clip:
D:\>*tail clip:

The completion code deals correctly with file names that contain spaces
or other special characters by placing quotes around the matching path.
Also, if you back up, then invoke completion from within a line, the
text to the right of the cursor at the point completion was invoked is
discarded.

The special characters that require quotes are:
<space>
&()[]{}^=;!'+,`~

However, when I set UnicodeOutput:
D:\>Option //UnicodeOutput=Yes
D:\>cmd /? > clip:
D:\>cmd /? >>clip:
D:\>*tail clip:
The completion code deals correctly with file names that contain spaces
or other special characters by placing quotes around the matching path.
Also, if you back up, then invoke completion from within a line, the
text to the right of the cursor at the point completion was invoked is
discarded.

The special characters that require quotes are:
<space>
&()[]{}^=;!'+,`~
??????????????????????????????????????????????????????????????????????+???????????????+++?????4????????????????????????????
++??????????????????????????????+++????????????????????????????????++?????????/++??????????????????????????????????+++?????
???4???????????????????????????++?????????????????????????????????+++????????+?????????????????????????????????????+???????
?????
etc.

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
 

rconn

Administrator
Staff member
May 14, 2008
12,350
150
>I recently set UnicodeOutout=Yes and then got some very funny results.
>
> I'm using "cmd /?" simply as a test program that generates text output.
>
> This exists at least back to 4NT 6, and I don't understand why it
> would behave this way. Appending to the clipboard appears to be
> different than just writing to it. Using "LIST CLIP:" shows that the
> first part of the file is Unicode (16 bit wide characters), but the
> second half is ASCII (8 bit wide characters).
>
> D:\>Option //UnicodeOutput=No
> D:\>cmd /? > clip:
> D:\>cmd /? >>clip:
> D:\>*tail clip:

UnicodeOutput only affects the output of TCC's internal commands, not the
output of external apps (see the help).

If you do something like this:

dir /? > clip:
dir /? >> clip:

you'll see that everything is saved as expected.

You'll have to ask Microsoft why CMD behaves that way.

Rex Conn
JP Software
 
May 20, 2008
603
0
Sammamish, WA
>>I recently set UnicodeOutout=Yes and then got some very funny results.


> UnicodeOutput only affects the output of TCC's internal commands, not the
> output of external apps (see the help).
> dir /? > clip:
> dir /? >> clip:

I disagree with the statement that UnicodeOutput does not affect the
output of external apps.

As evidence, I used the simplest "Hello world" program:

#include <stdio.h>
int main(int argc, char * argv[])
{
printf("Hello world\n");
}

Then I ran it like this
Option //UnicodeOutput=Yes
helloworld.exe > clip:
helloworld.exe >>clip:
list /x clip:

Touching the U key once shows this output:

0000 ff fe 48 00 65 00 6c 00 6c 00 6f 00 20 00 77 00 ”H.e.l.l.o. .w.
0010 6f 00 72 00 6c 00 64 00 0d 00 0a 00 48 65 6c 6c o.r.l.d.....Hell
0020 6f 20 77 6f 72 6c 64 0d 0a 00 o world...

Running the same test with //UnicodeOutput=No does what I would expect:

0000 48 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a 48 65 6c Hello world..Hel
0010 6c 6f 20 77 6f 72 6c 64 0d 0a lo world..

I submit that the only difference is the UnicodeOutput setting.

If I've missed something here, I'm more than happy to admit my
ignorance and be educated.

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
 
May 20, 2008
603
0
Sammamish, WA
>>I recently set UnicodeOutout=Yes and then got some very funny results.


> UnicodeOutput only affects the output of TCC's internal commands, not the
> output of external apps (see the help).
> dir /? > clip:
> dir /? >> clip:

I disagree with the statement that UnicodeOutput does not affect the
output of external apps.

As evidence, I used the simplest "Hello world" program:

#include <stdio.h>
int main(int argc, char * argv[])
{
printf("Hello world\n");
}

Then I ran it like this
Option //UnicodeOutput=Yes
helloworld.exe > clip:
helloworld.exe >>clip:
list /x clip:

Touching the U key once shows this output:

0000 ff fe 48 00 65 00 6c 00 6c 00 6f 00 20 00 77 00 ”H.e.l.l.o. .w.
0010 6f 00 72 00 6c 00 64 00 0d 00 0a 00 48 65 6c 6c o.r.l.d.....Hell
0020 6f 20 77 6f 72 6c 64 0d 0a 00 o world...

Running the same test with //UnicodeOutput=No does what I would expect:

0000 48 65 6c 6c 6f 20 77 6f 72 6c 64 0d 0a 48 65 6c Hello world..Hel
0010 6c 6f 20 77 6f 72 6c 64 0d 0a lo world..

I submit that the only difference is the UnicodeOutput setting.

If I've missed something here, I'm more than happy to admit my
ignorance and be educated.

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
 
Similar threads
Thread starter Title Forum Replies Date
Dick Johnson Weird Color Problem Support 8
fishman@panix.com Problem with 27.15 Support 2
M Problem with VSDevCmd.bat in VS 16.7.3 Support 0
R Problem with @INT[ value] in V26 Support 9
M Selecting test "off by one" problem in Take command Support 4
Alpengreis UTF-8 problem in TCC related to Python Support 7
K_Meinhard Small problem in german IDE 26 Support 3
B Problem with color in nested shells Support 1
Joe Caverly Problem creating and switching to a DESKTOP Support 9
vefatica Another popup problem Support 10
Alpengreis ffind dialog (/W) problem Support 4
Alpengreis [TCMD v25.00.24] Small space problem with the DE translation in Prefs-GUI Support 1
Alpengreis [TCMD v25.00.24] Problem with copy and paste and the # char via mouse in TCC Support 6
A Problem with functions @int @decimal and identifying Powershell as a shell. Support 12
B IF command problem in tcexit.btm Support 9
fishman@panix.com Problem at Startup of TCC Support 3
P Problem with SFTP copies Support 7
P Problem with FTP copies Support 10
Jay Sage Problem with Context Menu Copy+Paste+Run Key Assignment Support 7
R Problem with %_do_loop in nested do loops Support 2
fishman@panix.com New Problem with later Windows 10 Support 10
Peter Murschall A little problem with LEAVE and COMMENT Support 5
M Handling of %~I problem Support 4
WinLanEm FOR problem Support 18
K Problem With SCRPUT /u Support 3
vefatica What's TCMD's problem with ^e[0m? Support 13
Peter Murschall IDE: RTL with non-English resources-problem is back in Build 28 Support 4
rps Regex problem: \xnn not recognized as a hex character Support 0
rps PRIORITY LOW problem Support 1
old coot Regex problem: \xnn not recognized as a hex character Support 12
vefatica Another problem with build 22. Support 3
Alpengreis [23.x] Download-Problem Support 2
S Problem with " Support 3
C Problem navigating forums Support 11
x13 Problem listing repository files using DIR http(s)://... Support 8
Joe Caverly Problem with TEE in v22 Support 2
Alpengreis Problem with thousands delimiter and colors Support 17
M Take Command 18, migrating to another machine, license problem. Support 1
B TCC 21.01.50 Problem with ALIAS /r and SET /r Support 2
rps How to? @search problem Support 2
WinLanEm @SELECT problem Support 6
T Fixed Problem with use of Batch parameters in the IDE Support 1
Oz Solomon Problem with "list" Support 14
S V21.24 theme problem Support 2
Alpengreis Problem if command prompt is not legacy Support 4
Alpengreis TCMD.INI: The "super hidden" problem ... Support 6
G Odd problem - screen brightness Support 4
rps V20 In-process pipes problem Support 8
Alpengreis Again theme problem [v20.0.21]? Support 11
WinLanEm INPUT problem Support 1

Similar threads