UnicodeOutput question

May 30, 2008
235
2
I have this command sequence:

(echo commandSequence | externalProgram) > file.txt


I would like "file.txt" to be Unicode so this should seem to work:

OPTION //UnicodeOutput=yes
(echo commandSequence | externalProgram) > file.txt
OPTION //UnicodeOutput=no

However, this causes externalProgram to also get its input in Unicode which it does not like too much.


This works as I want, but it requires a temporary file:

(echo commandSequence | externalProgram) > tmpfile.txt
OPTION //UnicodeOutput=yes
type tmpfile.txt > file.txt
OPTION //UnicodeOutput=no


Is there a way to change the UnicodeOutput option after the pipe, but before the redirection to the result file?

I tried this:

((echo commandSequence | externalProgram) & (OPTION //UnicodeOutput=yes)) > file.txt

It produced a file, but it was not in Unicode format.

Should it work, and in that case, how?
 
May 20, 2008
603
0
Sammamish, WA
I believe the Unicode output only applies to internal commands.

Sent from Cookie's iPhone
Jim Cook

On Dec 15, 2010, at 0:10, nikbackm <> wrote:


> I have this command sequence:
>
> (echo commandSequence | externalProgram) > file.txt
>
>
> I would like "file.txt" to be Unicode so this should seem to work:
>
> OPTION //UnicodeOutput=yes
> (echo commandSequence | externalProgram) > file.txt
> OPTION //UnicodeOutput=no
>
> However, this causes externalProgram to also get its input in Unicode which it does not like too much.
>
>
> This works as I want, but it requires a temporary file:
>
> (echo commandSequence | externalProgram) > tmpfile.txt
> OPTION //UnicodeOutput=yes
> type tmpfile.txt > file.txt
> OPTION //UnicodeOutput=no
>
>
> Is there a way to change the UnicodeOutput option after the pipe, but before the redirection to the result file?
>
> I tried this:
>
> ((echo commandSequence | externalProgram) & (OPTION //UnicodeOutput=yes)) > file.txt
>
> It produced a file, but it was not in Unicode format.
>
> Should it work, and in that case, how?
>
>
>
>
 
May 30, 2008
235
2
I believe the Unicode output only applies to internal commands.

Sent from Cookie's iPhone
Jim Cook

On Dec 15, 2010, at 0:10, nikbackm <> wrote:

:o You're right of course.

I guess I expected UnicodeOutput to do some magic. It should be up to the externalProgram to output Unicode or not, TCC (or any other command processor) would likely not interfere when its output is redirected to a file.

But the TYPE seems to handle the job, so all's good.
 
May 20, 2008
3,515
4
Elkridge, MD, USA
---- Original Message ----
From: nikbackm
To: ESFabian@comcast.net
Sent: Wednesday, 2010. December 15. 03:10
Subject: [Support-t-2486] UnicodeOutput question

| I have this command sequence:
|
| (echo commandSequence | externalProgram) > file.txt
|
|
| I would like "file.txt" to be Unicode so this should seem to work:
|
| OPTION //UnicodeOutput=yes
| (echo commandSequence | externalProgram) > file.txt
| OPTION //UnicodeOutput=no
|
| However, this causes externalProgram to also get its input in Unicode
| which it does not like too much.
|
|
| This works as I want, but it requires a temporary file:
|
| (echo commandSequence | externalProgram) > tmpfile.txt
| OPTION //UnicodeOutput=yes
| type tmpfile.txt > file.txt
| OPTION //UnicodeOutput=no
|
|
| Is there a way to change the UnicodeOutput option after the pipe, but
| before the redirection to the result file?
|
| I tried this:
|
| ((echo commandSequence | externalProgram) & (OPTION
| //UnicodeOutput=yes)) > file.txt
|
| It produced a file, but it was not in Unicode format.
|
| Should it work, and in that case, how?
---- End of Original Message ----

You are trying to use ASCII input to a program, and want to save its output in Unicode. Unless that program is prepared to do that, you need to work around it. AFAIK the only way to achieve your goal is the one you already found, using an
temporary ASCII file, and translating it to Unicode.
--
Steve
 
Oct 29, 2008
88
0
You are trying to use ASCII input to a program, and want to save its output in Unicode. Unless that program is prepared to do that, you need to work around it. AFAIK the only way to achieve your goal is the one you already found, using an temporary ASCII file, and translating it to Unicode.
Or iconv if you don't insist on a TCC-only solution. You can make aliases to use in pipes, e.g.
Code:
utf8=iconv --binary -f UTF-16LE -t UTF-8
utf16=iconv --binary -f UTF-8 -t UTF-16LE
then your batch file becomes
Code:
OPTION //UnicodeOutput=yes
(echo commandSequence | utf8 | externalProgram | utf16) > file.txt
OPTION //UnicodeOutput=no
which should do what you want.
 
May 30, 2008
235
2
Or iconv if you don't insist on a TCC-only solution. You can make aliases to use in pipes, e.g.
Code:
utf8=iconv --binary -f UTF-16LE -t UTF-8
utf16=iconv --binary -f UTF-8 -t UTF-16LE
then your batch file becomes
Code:
OPTION //UnicodeOutput=yes
(echo commandSequence | utf8 | externalProgram | utf16) > file.txt
OPTION //UnicodeOutput=no
which should do what you want.

Thank you!

That does work very well indeed.

Also seems to be more reliable than using CHCP 65001 overall, which does not always display UTF-8 output from external programs correctly. The same output formatted in a different way causes it to produce mangled characters in some cases.

"ICONV -f UTF-8 -t CP1252" handles this better for some reason. Using codepage 65001 in the console is probably an hack that is not fully supported by Microsoft, or at least I've heard so a few times.
 
Similar threads
Thread starter Title Forum Replies Date
Charles Dye BOM when UnicodeOutput=Yes Support 11
Charles Dye Strange output, here-doc redirection, TYPE, //UnicodeOutput=Yes Support 6
J Problem with UnicodeOutput Support 3
C forum Posting Question... Support 3
C question re: Move Support 3
S How to? Upgrade Question: What supporting documentation is required at time of purchase? Support 2
Peter Murschall Documentation Question to %@PSHELL Support 4
Fross Tab Question Support 6
Fross Quick Function Question Support 17
Dick Johnson Question about the Touch command Support 0
rps Documentation "Copy+Paste+run" question Support 2
vefatica SFTP question Support 17
vefatica Question about IPWorks Support 0
C How to? SHORTCUT question..... Support 6
vefatica TPIPE, crash and question Support 1
C Question / Suggestion Support 1
MickeyF Another TPIPE question Support 6
vefatica Another @EVERYTHING question Support 4
mikea How to? %@everything[] question Support 10
A License Question for Single User Support 5
Jay Sage Help Correction (and Related Question) Support 0
S Elapsed time in TCC prompt question Support 0
H command line parsing question Support 5
C Forum question Support 2
B Newbie here, I have a question. Support 4
J Dumb Question ... Support 4
Timothy Byrd Updating from TC 9 to TC 17 - TCTOOLBAR question Support 2
M A possibly stupid question about the "del" command... Support 7
M A just out of curiosity question re @DiskTotal Support 4
Fross Status Bar Time Question Support 1
A Stupid, obvious question - launch a BAT with TC Support 10
C How to? GLOBAL question Support 5
vefatica OT: VisualStudio question Support 1
vefatica Wildcard question Support 4
Dan Glynhampton v15 installer question Support 2
C Move question with Encrypted file... Support 6
M Maybe Maybe I'm being stupid (again), but a simple question... Support 4
C Message Board question Support 1
James Miller How to? filename completion question Support 1
C coding question Support 3
M A just-out-of-curiosity question because it's ... Support 6
Frank question to FOLDERMONITOR Support 14
C Dumb titleprompt question Support 4
M Upgrade Question Support 2
S option /u question Support 19
M How to? A probably stupid question re "@Files" and Include Lists... Support 2
M Just an out-of-curiosity question re. "Do ... /P ..." Support 31
M How to? An admittedly somewhat silly question regarding elevation... Support 2
M Just an out-of-curiosity question the List command vs. the @Lines function... Support 13
M PDir format question... Support 3

Similar threads