Why is the TEE command 120 times slower?

[dumbenis]

I have written this code:

@ECHO OFF
TIMER /1 ON
DO i in /S *.*
    ECHO %i
    ECHO %_date %_time %i >> %tmp\logbook1.txt
ENDDO
TIMER /1 OFF

It executes in 1 minute and 17 seconds.

Because ECHO is used twice, I have written this code:

@ECHO OFF
TIMER /1 ON
DO i in /S *.*
    ECHO %i | TEE /a /d /t  %tmp\logbook2.txt
ENDDO
TIMER /1 OFF

It executes in 2 hours 45 minutes and 11 seconds.

Question: why is the TEE command 120 times slower?

 

[RogerB]

I suspect it's because every time you use the pipe a new instance of TCC is started which carries a processing overhead. Do you have

iff %_pipe EQ 1 then...

in your TCCstart to ensure only the necessary parts of the startup files is processed in a pipe?

If you haven't already, you might wish to review the TCC help for In process pipes, they might be quicker in this case.

 

[rconn]

Redirection just changes the output handle to point to a file instead of the console. A pipe starts a second copy of TCC and connects the output of the parent with the input of the child. So you're starting a new TCC every time through the DO loop.

You should be using a command group and piping that, or piping the output of the entire batch file.

 

[vefatica]

I suspect it's because every time you use the pipe a new instance of TCC is started which carries a processing overhead. Do you have

iff %_pipe EQ 1 then...

in your TCCstart to ensure only the necessary parts of the startup files is processed in a pipe?

If you haven't already, you might wish to review the TCC help for In process pipes, they might be quicker in this case.

The first line of my TCSTART.BTM is

if %_pipe == 1 .or. %_transient == 1 quit

Here, the time difference between dumbenis's two experiments is a factor of 10. I suppose the loading/saving of a history file (which I don't do) could also make a significant difference.

 

[vefatica]

You should be using a command group and piping that, or piping the output of the entire batch file.

Amen!

Here, this

TIMER /1 ON
DO i in /S *.* | TEE /a /d /t %tmp\logbook2.txt
    ECHO %i
ENDDO
TIMER /1 OFF

is about 3 times as fast as

TIMER /1 ON
DO i in /S *.*
    ECHO %_date %_time %i
    ECHO %_date %_time %i >> %tmp\logbook1.txt
ENDDO
TIMER /1 OFF

 

[AnrDaemon]

The construction's behavior ain't very clear, I must say.

 

[vefatica]

The construction's behavior ain't very clear, I must say.

If you mean
[/code]
DO i in /S . | TEE /a /d /t %tmp\logbook2.txt
ECHO %i
ENDDO[/code]
I agree. But I don't know another way to do it with a multi-line DO without a subroutine or another batch file. Fortunately, it works. I'm not crazy about the way this (below) looks, but it also works.

TIMER /1 ON
GOSUB doit | TEE /a /d /t %tmp\logbook2.txt 
TIMER /1 OFF
QUIT

:doit
DO i in /S *.*
    ECHO %i
ENDDO
RETURN

 

[AnrDaemon]

Yes, I mean redirection in the first line.
In all other shells, the redirection normally placed at the end of structural block.

 

[NetMage]

Redirection just changes the output handle to point to a file instead of the console. A pipe starts a second copy of TCC and connects the output of the parent with the input of the child. So you're starting a new TCC every time through the DO loop.

You should be using a command group and piping that, or piping the output of the entire batch file.

Perhaps TCC could handle the "| TEE" construct specially ?

 

[Charles Dye]

Is there a significant speed difference if you use an in-process pipe instead?

 

[rconn]

Perhaps TCC could handle the "| TEE" construct specially ?

Three problems with that:

1) At the time of the redirection, TCC doesn't know (or care) what the next command is going to be. That would require a major effort to write a new "look ahead" parser.

2) I think it's a bad idea in general to change the behavior based on isolated cases. It tends to surprise the users, usually unpleasantly.

3) Changing the pipe behavior when it's going to a TEE would actually slow down the output if you are using the appropriate syntax for piping -- i.e., with a command group or piping the batch file output.