TPIPE generate unicode on Piping or redirecting

I've some ANSI-CSV-Files (bank statements) which I'm glued together with a BTM-Script using TAIL.
Now I want to sort the content, remove duplicate lines and some unnecassary chars (¤ ).
Code:
OPTION //UNICODEOUTPUT=NO
Set TempFile=%@UNIQUE[%_CWD]
::Copy Headerline
HEAD /n1 %outputfile% >%tempfile%
:: %tempfile% is ANSI
::get the rest, sort it, remove all "¤ " , remove duplicate lines with TPIPE
TAIL /n+1 /n %@LINES[%outputfile%] %outputfile% | TPIPE /replace=4,1,0,0,0,0,0,0,0,"\xA4\x20","" /SORT=2,1,0,1,0 >>%tempfile%
:: %tempfile% is Mixed ANSI and Unicode !!!
Move %tempfile% %outputfile%
The first line is ANSI, but the rest is Unicode :
2320


The question is, what happens here ?

Meanwhile I use TPIPE instead TAIL with redirection, this works:
Code:
TPIPE /output=%outputfile% /OutputAppend=1 /replace=4,1,0,0,0,0,0,0,0,"\xA4\x20","" /Sort=2,0,0,1,0
 
Yes, my default is to output Unicode. I've chosen the switch to Unicode, because I get more and more problems with the output of OEM-Characters, in IDE/BTM also with CMD/Notepad etc..

Since I'm working in Unicode I save my BTM-Scripts to Unicode, and the IDE and TCCs .BTM have no problems anymore.
For example trying this:
Code:
Echo %0
Echo UTF8Output=%@OPTION[UTF8Output] UnicodeOutput==%@OPTION[UnicodeOutput]
Echo Testing OEMs: ÄÖÜßäöü§
Echo Überlauf ! Größe reduzieren, Änderungen speichern und schließen ?

will produce this, if it saved as ANSI:
2323


saved as Unicode, the output is ok
2324


So I'm working consequent with Unicode and BTMs.
 
Similar threads
Thread starter Title Forum Replies Date
samintz How to? TPIPE /simple=61 Support 6
R Bug TPIPE's pdf to text conversions don't work Support 2
Kachupp How to? tpipe Support 4
H TPIPE /simple=4 Support 12
D TPIPE produces unwanted logs. Support 1
Kachupp How to? tpipe Support 8
vefatica Documentation TPIPE, /perl, DotMatchesNewLines Support 1
vefatica Stop TPIPE /EOL removing bad EOLs (and TPIPE changine EOLs)? Support 2
vefatica TPIPE in an alias? Support 2
C How to? TPIPE ?'s Support 2
H History and TPipe Oddness Support 3
Jay Sage Baffling Behavior With TPIPE Support 5
vefatica TPIPE: unbalanced escaped quotes in a regex? Support 5
Joe Caverly Multiple Text Searches at once using FFIND or TPIPE Support 4
Kachupp FFIND TPIPE Support 9
vefatica TPIPE and avarice? Support 5
x13 TPIPE /REPLACE - Remove line (including EOL) Support 3
Charles Dye Documentation TPIPE /REPLACELIST Support 0
Joe Caverly TPIPE and /inputstring Support 3
vefatica How to? Add real <ESC> characters with TPIPE Support 2
vefatica How to? TPIPE ... see output right away? Support 4
vefatica TPIPE.EXE crashes on Ctrl-C Support 9
O TPIPE: Error -Can´t access textpipeengine.dll Support 24
vefatica Extract fields with TPIPE? Support 10
vefatica TPIPE, crash and question Support 1
MickeyF Another TPIPE question Support 6
MickeyF Can't get my TPIPE /replace to work Support 4
Joe Caverly Printing TPIPE documentation Support 3
C WAD tpipe /simple=28 Support 8
vefatica Remove double-quotes with TPIPE Support 4
vefatica TPIPE crashes on Ctrl-C Support 1
C How to? Add date/time to custom tpipe based BTM Support 1
C tpipe /simple=28 error Support 5
C TPipe /dup Support 5
T TPipe: Replacing String with Double Quotes Support 3
C tpipe and extract email addresses Support 1
vefatica How do I do this with TPIPE? Support 1
vefatica TPIPE's /perl option? Support 1
D TPIPE help typo Support 0
vefatica TPIPE access violations are back Support 2
Joe Caverly Word Count using TPIPE Support 10
Dan Glynhampton TPIPE in Build 41 Support 4
Phileosophos TPIPE doesn't work at all with v16 Support 3
vefatica TPIPE access violation again Support 1
David McClelland TPIPE matching problem Support 3
vefatica TPIPE: Access violation Support 5
jbanaszczyk TCMD 16 - TPIPE Support 4
P TPipe relaunches TCC window and 4start, etc Support 7
Phileosophos Documentation TPIPE /string type code confusion Support 4
C TPIPE questions.... Support 5

Similar threads