Is there an 8MB limit somewhere?

[vefatica]

Using this alias (to reverse output lines):

v:\> alias tac
ffind /e"\r\n" /r /v /m /k

this command correctly produces 8388 1000-byte lines (just under 8MB) and the alias correctly reverses those lines

v:\> (do x=1000001 to 1008388 ( echo %x %@repeat[x,990] ) ) | tac > tac1.txt

This command attempts the same for 8389 such lines (just over 8MB) and somehow fails

v:\> (do x=1000001 to 1008389 ( echo %x %@repeat[x,990] ) ) | tac > tac2.txt

The file sizes and the tails of the files show something went awry:

v:\> dir /k /m tac*
2012-06-18  21:51      8,388,000  tac1.txt
2012-06-18  21:52      8,388,698  tac2.txt
 
v:\> tail /n1 tac1.txt
1000001 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
 
v:\> tail /n2 tac2.txt
1004196 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

 

[vefatica]

Using this alias (to reverse output lines):

v:\> alias tac
ffind /e"\r\n" /r /v /m /k

this command correctly produces 8388 1000-byte lines (just under 8MB) and the alias correctly reverses those lines

v:\> (do x=1000001 to 1008388 ( echo %x %@repeat[x,990] ) ) | tac > tac1.txt

This command attempts the same for 8389 such lines (just over 8MB) and somehow fails

v:\> (do x=1000001 to 1008389 ( echo %x %@repeat[x,990] ) ) | tac > tac2.txt

The file sizes and the tails of the files show something went awry:

v:\> dir /k /m tac*
2012-06-18  21:51      8,388,000  tac1.txt
2012-06-18  21:52      8,388,698  tac2.txt

It's a little interesting how the bad file is corrupt. The first 4194 lines (lines numbered 1008389 through 1004196 from DO's output are correct. The next line of the file is 1008389 (again!) and has two extra x's. Then the lines then go in order (with the correct number of x's) from 1008388 down to 1004196 (again) with the next line, the last line of the file, having no numerical prefix and 694 x's followed by a CRLF.

 

[JohnQSmith]

Upgrade to a new version of tac. Mine works fine. Even for much bigger numbers.

z:\>(do x=1000001 to 1128389 ( echo %x %@repeat[x,990] ) ) | tac > tac2.txt
 
z:\> *dir /k /m tac*
6/19/2012  08:34      8,388,000  tac1.txt
6/19/2012  08:39    128,389,000  tac2.txt

 

[Stefano Piccardi]

his tac is an alias

 

[Stefano Piccardi]

Vince, getting rid of pipes in your example results in an apparently infinite loop in FFIND - see size of tac2.txt after breaking out of the apparently stuck FFIND with Ctrl+C.
Windows 7 SP1 x86 English TCC 14.00.23

C:\>(do x=1000001 to 1008388 ( echo %x %@repeat[x,990] ) ) >itac1.txt
 
C:\>copy itac1.txt itac2.txt
C:\itac1.txt => C:\itac2.txt
1 file copied
 
C:\>(do x=1008389 to 1008389 ( echo %x %@repeat[x,990] ) ) >>itac2.txt
 
C:\>dir /k /m itac*
6/19/2012 16:16 8,388,000 itac1.txt
6/19/2012 16:22 8,389,000 itac2.txt
 
C:\>ffind /e"\r\n" /r /v /m /k itac1.txt >tac1.txt
 
C:\>ffind /e"\r\n" /r /v /m /k itac2.txt >tac2.txt
^C
 
C:\>dir /k /m tac*
6/19/2012 16:22 8,388,000 tac1.txt
6/19/2012 16:24 454,461,000 tac2.txt

 

[vefatica]

Vince, getting rid of pipes in your example results in an apparently infinite loop in FFINDC:\>ffind /e"\r\n" /r /v /m /k itac1.txt >tac1.txt

C:\>ffind /e"\r\n" /r /v /m /k itac2.txt >tac2.txt
^C

C:\>dir /k /m tac*
6/19/2012 16:22 8,388,000 tac1.txt
6/19/2012 16:24 454,461,000 tac2.txt
[/code]

I noticed that also (but I didn't let it go to half a GB). I figured it came from the same limit/problem.

 

[vefatica]

Upgrade to a new version of tac. Mine works fine. Even for much bigger numbers.

z:\>(do x=1000001 to 1128389 ( echo %x %@repeat[x,990] ) ) | tac > tac2.txt
 
z:\> *dir /k /m tac*
6/19/2012  08:34      8,388,000  tac1.txt
6/19/2012  08:39    128,389,000  tac2.txt

I've been unable to find a TAC.EXE that works at all on WIN7. What's yours?

 

[JohnQSmith]

I've been unable to find a TAC.EXE that works at all on WIN7. What's yours?

I use Cygwin and its assorted myriad of utility goodness.

 

[JohnQSmith]

his tac is an alias

I missed that part of tac being an alias to ffind.

 

[vefatica]

Vince, getting rid of pipes in your example results in an apparently infinite loop in FFIND - see size of tac2.txt after breaking out of the apparently stuck FFIND with Ctrl+C.

When I use, instead "ffind /t"^r^n" /r /v /m /k" ("/r" is documented to work with "/t") the endless loop problem does not happen (and the order of the lines in the file *is* reversed).

v:\> (do x=1000001 to 1008388 ( echo %x %@repeat[x,990] ) ) > tac1.txt
 
v:\> (do x=1000001 to 1008389 ( echo %x %@repeat[x,990] ) ) > tac2.txt
 
v:\> d tac*
2012-06-21  00:38      8,388,000  tac1.txt
2012-06-21  00:38      8,389,000  tac2.txt
 
v:\> ffind /t"^r^n" /r /v /m /k tac1.txt > tac1rev.txt
 
v:\> ffind /t"^r^n" /r /v /m /k tac2.txt > tac2rev.txt
 
v:\> d tac*
2012-06-21  00:38      8,388,000  tac1.txt
2012-06-21  00:39      8,388,000  tac1rev.txt
2012-06-21  00:38      8,389,000  tac2.txt
2012-06-21  00:40      8,389,000  tac2rev.txt

But when the same FFIND args are used in a pipe, the same (as noted earlier) corruption occurs:

v:\> (do x=1000001 to 1008388 ( echo %x %@repeat[x,990] ) ) | ffind /t"^r^n" /r /v /m /k > tac1.txt
 
v:\> (do x=1000001 to 1008389 ( echo %x %@repeat[x,990] ) ) | ffind /t"^r^n" /r /v /m /k > tac2.txt
 
v:\> d tac[12].txt
2012-06-21  00:47      8,388,000  tac1.txt
2012-06-21  00:48      8,388,698  tac2.txt

The file tac2.txt is corrupted in the same way as described earlier (when I was using "/e" instead of "/t").

 

[rconn]

The FFIND (and LIST) buffer is 8mb. I'm not too worried about this case, as it seems to be a somewhat grotesque misuse of FFIND to do something far more easily done with (many) other tools.

 

[Steve Fabian]

The FFIND (and LIST) buffer is 8mb. I'm not too worried about this case, as it seems to be a somewhat grotesque misuse of FFIND to do something far more easily done with (many) other tools.

Is there any other simple method to do it with only TCC internal commands?

 

[rconn]

Not at this time. Why is that important?

 

[Steve Fabian]

Not at this time. Why is that important?

Only because of your earlier statement:

...a somewhat grotesque misuse of FFIND to do something far more easily done with (many) other tools...

Cf: My only tool for most work is TCC (even when it is not always as simple as using a HLL).

 

[Charles Dye]

Steve, did you try UPEND?

 

[Steve Fabian]

Sorry, I have been very busy away from my computers in the past week.

 

[rconn]

Cf: My only tool for most work is TCC (even when it is not always as simple as using a HLL).

I don't think there's a compelling need for every possible obscure tool to be built into TCC.

 

[Steve Fabian]

I don't think there's a compelling need for every possible obscure tool to be built into TCC.

I agree, though I have needed to reverse the order of elements in a file too often to consider it an obscure requirement.

 

[Steve Fabian]

The FFIND (and LIST) buffer is 8mb.

I think this limit should be listed in HELP topic "Limits", along with any other limits of this kind. The more obscure the limit, the more important that it be documented. Most of this thread would not have been posted if that had been done.

 

[Frank]

>dir /sfh d:\temp\recurse\ | logparser -i:textline -q:on "select text from stdin order by text desc"
D:\temp\recurse\dir3\a\b\c
D:\temp\recurse\dir3\a\b
D:\temp\recurse\dir3\a
D:\temp\recurse\dir3
D:\temp\recurse\dir2\a\b\c
D:\temp\recurse\dir2\a\b
D:\temp\recurse\dir2\a
D:\temp\recurse\dir2
D:\temp\recurse\dir1\a\b\c
D:\temp\recurse\dir1\a\b
D:\temp\recurse\dir1\a
D:\temp\recurse\dir1
 
>alias reverse=logparser -i:textline -q:on "select text from stdin order by text desc"
 
>dir /sfh d:\temp\recurse\ | reverse
D:\temp\recurse\dir3\a\b\c
D:\temp\recurse\dir3\a\b
D:\temp\recurse\dir3\a
D:\temp\recurse\dir3
D:\temp\recurse\dir2\a\b\c
D:\temp\recurse\dir2\a\b
D:\temp\recurse\dir2\a
D:\temp\recurse\dir2
D:\temp\recurse\dir1\a\b\c
D:\temp\recurse\dir1\a\b
D:\temp\recurse\dir1\a
D:\temp\recurse\dir1

 

[samintz]

Is there a reason you can't use good ol' fashioned SORT?

> ((do x=1000001 to 1008389 ( echo %x %@repeat[x,990] ) ) | sort /r) > tac2.txt
 
> dir
6/26/2012  15:46       8,389,000  tac2.txt
 
> tail /n1 tac2.txt
1000001 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxx

-Scott

 

[vefatica]

Is there a reason you can't use good ol' fashioned SORT?

The whole point was to produce the lines in reverse order (unsorted). SORT /R will only do that if the file is already sorted. This is the desired behavior:

v:\> echo 2^r^n1^r^n3 | tac
3
1
2

Not this:

v:\> echo 2^r^n1^r^n3 | sort /r
3
2
1

 

[Frank]

The whole point was to produce the lines in reverse order (unsorted). SORT /R will only do that if the file is already sorted. This is the desired behavior:

v:\> echo 2^r^n1^r^n3 | tac
3
1
2

Not this:

v:\> echo 2^r^n1^r^n3 | sort /r
3
2
1

oops, then my example with logparser was rubbish.

 

[dcantor]

The whole point was to produce the lines in reverse order (unsorted). SORT /R will only do that if the file is already sorted. This is the desired behavior:

v:\> echo 2^r^n1^r^n3 | tac
3
1
2

Not this:

v:\> echo 2^r^n1^r^n3 | sort /r
3
2
1

How about

TPIPE /input=infile.txt /output=outfile.txt /simple=34

? That works for me.

 

[JohnQSmith]

How about

TPIPE /input=infile.txt /output=outfile.txt /simple=34

? That works for me.

There you go... making sense again.

edit: yeah, that works like a champ.

C:\temp> (do x=1000001 to 1128389 ( echo %x %@repeat[x,990] ) ) | tpipe /simple=34 > tac.txt
 
C:\temp> dir tac.txt
 
Volume in drive C is unlabeled      Serial number is 58b8:a853
Directory of  C:\temp\tac.txt
 
6/27/2012  15:08    128,389,000  tac.txt
      128,389,000 bytes in 1 file and 0 dirs    128,389,120 bytes allocated
    24,191,352,832 bytes free

 

[vefatica]

I did the same test as JQS with the same satisfactory result. It's quite fast too. Using 8388 lines, which FFIND /R can handle, it took about 30 seconds. With TPIPE it took about 3 seconds.