@replace and @xreplace problems

Jan 16, 2009
Unless I am missing something, there seems to be a bug (or maybe its WAD !ugh!) in @replace when the end of the file has CRLF or CRLFCRLF.Or maybe I don't understand how variables are handled internally.

The many text files I want to process have a CRLF (Carriage Return Line Feed) at the end of every line. A new paragraph has CRLFCRLF at its beginning. Some files end with CRLF; some end with normal characters, such as "." or " ".
The processing I want to do is to: remove every single CRLF and retain every CRLFCRLF.
I am using @REPLACE. My method is to 1.replace every CRLFCRLF with Hex 01 01 01 01. 2.Replace every CRLF with " ". 3.Replace every Hex 01 01 01 01 with CRLFCRLF. 4.Write out the new file.
All works fine except when the original file ends with CRLF. Then: Step 1 partially works. The CRLFCRLF are replaced properly, but the final CRLF is replaced with 00 00. (If the original file ends in CRLFCRLF, that is replaced with 00 00 00 00.) Step 2 works fine, except that the final 00 00 (or 00 00 00 00) of step 1 is now 00. Step 3 works fine and doesn't change any file 00 00. Step 4 changes the final 00 in the output file to Hex 00 00.
After testing on a few files, I tested on 20 files. In a test file, the output truncated 2 chars of the original file. The original file ended in CRLF. It seems that TCC treats variables as having a length that may vary depending on how they are processed. This implicit length is hidden, and unclear. I am frustrated!
Update: I have gotten the program to work reliably using a combination of @replace and @xreplace. I can find no logic to why one works in one particular step and the other doesn't.
Functions used: Knowing the file's length, I read the file using @saferead (from safechars plugin). Works fine.
I write the file out using @filewriteb, using for the final write a length reduced by the number of CRLF's I have replaced with spaces.


Scott Mintz
May 20, 2008
Solon, OH, USA
Since you are only doing CRLF processing, you could just read every line and write it back out again. And replace blank lines (double EOL) with the EOL sequence.
setdos /x-45678
set in=%@fileopen["%1",r,t]
set out=%@fileopen["%2",w,b]
set r=%@truncate[%out]
set r=%@fileread[%in]
do while %r. != **EOF**.
  set len=%@len[%r]
  iff %len == 0 then
    set s=%@filewriteb[%out,-1,13 10]
    set s=%@filewriteb[%out,%len,%r]
  set r=%@fileread[%in]
set in=%@fileclose[%in]
set out=%@fileclose[%out]

I'm sure there is a TPIPE filter you could use too.

I created a test.txt file:
line 1
line 2

line 5
line 6

in binary:
00000000  6C 69 6E 65 20 31 0D 0A 6C 69 6E 65 20 32 0D 0A    line 1..line 2..
00000010  0D 0A 0D 0A 6C 69 6E 65 20 35 0D 0A 6C 69 6E 65    ....line 5..line
00000020  20 36 0D 0A                                         6..

And after running the script:
test test.txt test1.txt

The binary of text1.txt is:
00000000  6C 69 6E 65 20 31 6C 69 6E 65 20 32 0D 0A 0D 0A    line 1line 2....
00000010  6C 69 6E 65 20 35 6C 69 6E 65 20 36                line 5line 6
Last edited:


Scott Mintz
May 20, 2008
Solon, OH, USA
One other thing I wanted to point out is that the sequence of EOL chars by your description wasn't exactly clear.
For example:
line 1<EOL>
line 2<EOL>
line 3<EOL>
In that example every line ends with <EOL> and there are 2 blank lines. But that sequence is actually 3 <EOL>'s in a row.
line 1<EOL>
line 2<EOL>
line 3<EOL>
In this example, there is a sequence of 2 <EOL>'s but one of them is the EOL marker for line 2.

The test script I wrote works based on the assumption that every line ends with EOL and blank lines keep their EOL. Either way, I think it does what you wanted.


Scott Mintz
May 20, 2008
Solon, OH, USA
Actually, I take that back. If your input data is structured with the 2 EOL's, the test script will reduce that to 1 EOL. If you always want 2, then change the line
set s=%@filewriteb[%out,-1,13 10]
set s=%@filewriteb[%out,-1,13 10 13 10]
May 20, 2008
Syracuse, NY, USA
How about something more suited to the task?
v:\> type dups.txt





v:\> tpipe /input=dups.txt /output=dups2.txt /replace=4,0,0,0,0,0,0,0,0,"\r\n\r\n","\001" /replace=4,0,0,0,0,0,0,0,0,"\r\n","" /replace=4,0,0,0,0,0,0,0,0,"\001","\r\n\r\n"

v:\> type dups2.txt




Similar threads
Thread starter Title Forum Replies Date
Joe Caverly Documentation Replace internal command with LIBRARY command Support 0
I how to make TCC default/replace cmd.exe? Support 9
x13 TPIPE /REPLACE - Remove line (including EOL) Support 3
T @replace -- what is wrong here Support 2
C Replace 1st argument on command line Support 4
R How to? Replace in Rename Support 9
C How to? @replace[a comma,_,filename] Support 5
MickeyF Can't get my TPIPE /replace to work Support 4
C How to? Replace &'s in filenames / FixNames.dll Support 2
M I'm unable to use @replace[] with a % character in string2 Support 12
MikeBaas Strange prob with %@replace.. Support 4
vefatica Back-refs in TPIPE's /replace's replacement string? Support 2
Frank @replace works case sensitive Support 5
J @replace weirdness Support 1
vefatica TPIPE /replace? Support 3
Stefano Piccardi replace tcc process space? Support 8
vefatica @SUBST ... how replace character with a caret? Support 9
mdwyer @replace - global use in a text file Support 3
B Regex and Replace Support 6
B REPLACE is good news Support 0
J Problem with %var:find=replace% syntax Support 5
Charles S. Roaten Problems with TCSTART.BTM Support 1
vefatica Office problems better Support 0
Peter Murschall TCC V26.01.3x Debugging has problems with Alias /R Support 1
Peter Murschall CMDebug 26.01.37 has problems with Alias Support 21
C unzip problems Support 2
C TCC 26 BATCOMP BTM problems? Support 5
Alpengreis WAD [v26 Beta RC3]: MSGBOX color problems Support 4
Alpengreis QUERYBOX problems Support 5
R New Cloud Storage Tab in TCC options causing problems Support 0
Craig Fitzgerald Problems using tcstart.btm Support 3
M Problems with @REGBREAD on remote system Support 4
K_Meinhard Help problems Support 3
S Problems with dir command in the debugger Support 5
D How to? Path problems in TCC20 Support 1
G Version 20.11 installed on Windows 10 problems Support 9
M WAD Set statements in () causes problems Support 8
M Fixed Set "VAR=VAL" format causes problems Support 12
H Problems with PSUBST Support 3
Alpengreis The TCMD Display problems and font size ... Support 2
cgunhouse Problems With IFTP Support 3
Gamegod Fixed TCC identify Chinese problems Support 30
vefatica 21-22 Update problems Support 8
T Problems with the new help format Support 18
cgunhouse Event Plugin Problems With 19.10.43 Support 2
Alpengreis Forum problems? Support 0
N Fixed Problems with SHA256/384/512 Support 5
Alpengreis [TC 19.0 Beta 22/23] Install/Update-"Problems" Support 4
P Problems registering Support 5
R Parser problems after build 44 Support 5

Similar threads