Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

TPIPE generate unicode on Piping or redirecting

Sep
134
1
I've some ANSI-CSV-Files (bank statements) which I'm glued together with a BTM-Script using TAIL.
Now I want to sort the content, remove duplicate lines and some unnecassary chars (¤ ).
Code:
OPTION //UNICODEOUTPUT=NO
Set TempFile=%@UNIQUE[%_CWD]
::Copy Headerline
HEAD /n1 %outputfile% >%tempfile%
:: %tempfile% is ANSI
::get the rest, sort it, remove all "¤ " , remove duplicate lines with TPIPE
TAIL /n+1 /n %@LINES[%outputfile%] %outputfile% | TPIPE /replace=4,1,0,0,0,0,0,0,0,"\xA4\x20","" /SORT=2,1,0,1,0 >>%tempfile%
:: %tempfile% is Mixed ANSI and Unicode !!!
Move %tempfile% %outputfile%
The first line is ANSI, but the rest is Unicode :
2320


The question is, what happens here ?

Meanwhile I use TPIPE instead TAIL with redirection, this works:
Code:
TPIPE /output=%outputfile% /OutputAppend=1 /replace=4,1,0,0,0,0,0,0,0,"\xA4\x20","" /Sort=2,0,0,1,0
 
Yes, my default is to output Unicode. I've chosen the switch to Unicode, because I get more and more problems with the output of OEM-Characters, in IDE/BTM also with CMD/Notepad etc..

Since I'm working in Unicode I save my BTM-Scripts to Unicode, and the IDE and TCCs .BTM have no problems anymore.
For example trying this:
Code:
Echo %0
Echo UTF8Output=%@OPTION[UTF8Output] UnicodeOutput==%@OPTION[UnicodeOutput]
Echo Testing OEMs: ÄÖÜßäöü§
Echo Überlauf ! Größe reduzieren, Änderungen speichern und schließen ?

will produce this, if it saved as ANSI:
2323


saved as Unicode, the output is ok
2324


So I'm working consequent with Unicode and BTMs.
 

Similar threads

Back
Top