Speed question

May 20, 2008
11,530
102
Syracuse, NY, USA
I'm testing a pair of plugin functions. The command VARNAMES outputs the names of the environment variables, one per line. The variable function @VARNAMES[] returns an =-separated list of the environment variable names. The two use nearly identical code, the only difference being

Code:
if ( !bUseRegex || (rx = regex_match(szRegex, szVarName)) > 0 )
    Printf(L"%s\r\n", szVarName);
for VARNAMES, vs.

Code:
if ( !bUseRegex || (rx = regex_match(szRegex, szVarName)) > 0 )
    psz += Sprintf(psz, L"%s=", szVarName);
for @VARNAMES.

I'd expect them to be nearly equally fast. But they're not even close (see below). Each will accept a regular expression on which to match variable names but below, none is specified, so regex_match() is not called.

Code:
v:\> timer & for /l %i in (1,1,10000) (varnames > nul) & timer
Timer 1 on: 19:51:00
Timer 1 off: 19:51:14  Elapsed: 0:00:13.14

v:\> timer & for /l %i in (1,1,10000) (echo %@varnames[] > nul) & timer
Timer 1 on: 19:51:15
Timer 1 off: 19:51:18  Elapsed: 0:00:02.94
Rex, can you guess what accounts for the huge difference in times? I have no complaints; I'm just very curious.
 
May 20, 2008
11,530
102
Syracuse, NY, USA
It's always a lot faster to display one big string rather than a bunch of
little ones. (STDOUT is not buffered, and you've got a lot more API
overhead.)

Thanks. I hadn't thought of that (and if I had, probably wouldn't have realized it was significant). But a little test exemplifies it:

Code:
v:\> timer & for /l %i in (1,1,5000) ((echo a & echo b & echo c & echo d) > NUL) & timer
Timer 1 on: 22:20:48
Timer 1 off: 22:20:52  Elapsed: 0:00:03.86

v:\> timer & for /l %i in (1,1,5000) (echo a^r^nb^r^nc^r^nd > NUL) & timer
Timer 1 on: 22:20:58
Timer 1 off: 22:20:59  Elapsed: 0:00:01.25
 
May 20, 2008
11,530
102
Syracuse, NY, USA
It's always a lot faster to display one big string rather than a bunch of little ones. (STDOUT is not buffered, and you've got a lot more API overhead.)

So I added simple buffering to the command version (VARNAMES).

Code:
if ( bList ) // @VARNAMES
    r += Sprintf(r, L"%s=", szVarName);
else        // VARNAMES command
{
    r += Sprintf(r, L"%s\r\n", szVarName);
    if ( r-psz >= 1024 )
    {
        Printf(L"%s", psz);
        *psz = 0;
        r = psz;
    }
}
The command went from about 1/4 the speed of the variable function to **twice** the speed. That (above) is pretty much the difference between the two (except for the ECHO needed to output the variable function version).

So now I must ask if there's that much more overhead involved in calling the variable function compared to calling the command.m They're both under .0005 seconds so I'm not complaining.

I also noticed that I can "ECHO @VARNAMES[]" when there's over 24000 characters in the string (varname1=varname2=...). What are the limits these days?
 

rconn

Administrator
Staff member
May 14, 2008
12,404
152
> So now I must ask if there's that much more overhead involved in
> calling the variable function compared to calling the command.m They're
> both under .0005 seconds so I'm not complaining.

Yes, there's that much more overhead in the (very, very complex) variable
substitution.


> I also noticed that I can "ECHO @VARNAMES[]" when there's over 24000
> characters in the string (varname1=varname2=...). What are the limits
> these days?

32K for an input line, 64K expanded.

Rex Conn
JP Software
 
May 20, 2008
11,530
102
Syracuse, NY, USA
On Fri, 27 Aug 2010 22:17:43 -0400, rconn <>
wrote:

|---Quote---
|> I also noticed that I can "ECHO @VARNAMES[]" when there's over 24000
|> characters in the string (varname1=varname2=...). What are the limits
|> these days?
|---End Quote---
|32K for an input line, 64K expanded.

I see. And the size of any one token in the expanded line doesn't
seem to matter. You might up the 8191 limit on @REPEAT, then.
 
May 20, 2008
3,515
4
Elkridge, MD, USA
| Code:
| ---------
| if ( bList ) // @VARNAMES
| r += Sprintf(r, L"%s=", szVarName);
| else // VARNAMES command
| {
| r += Sprintf(r, L"%s\r\n", szVarName);
| if ( r-psz >= 1024 )
| {
| Printf(L"%s", psz);
| *psz = 0;
| r = psz;
| }
| }
| ---------

Since you are so interested in speed, did you consider other speed-up
methods, e.g., eliminating all the format-string parsing and
pseudoformatting done by Sprintf and instead using memcpy, strcpy, or their
Unicode equivalent to build your output string, and to precalculate the
upper limit of your buffer so you don't need to add 1024 each time? BTW, you
WILL overflow a 1024-element buffer if the total is more than 1024 elements,
because you test AFTER appending; resetting the first element of the buffer
to NUL is not neeeded, either - the next szVarName will overwrite it,
anyway.
--
Steve
 
May 20, 2008
11,530
102
Syracuse, NY, USA
On Sat, 28 Aug 2010 12:25:33 -0400, Steve Fábián
<> wrote:

|| Code:
|| ---------
|| if ( bList ) // @VARNAMES
|| r += Sprintf(r, L"%s=", szVarName);
|| else // VARNAMES command
|| {
|| r += Sprintf(r, L"%s\r\n", szVarName);
|| if ( r-psz >= 1024 )
|| {
|| Printf(L"%s", psz);
|| *psz = 0;
|| r = psz;
|| }
|| }
|| ---------
|
| Since you are so interested in speed, did you consider other speed-up
|methods, e.g., eliminating all the format-string parsing and
|pseudoformatting done by Sprintf and instead using memcpy, strcpy, or their
|Unicode equivalent to build your output string, and to precalculate the
|upper limit of your buffer so you don't need to add 1024 each time? BTW, you
|WILL overflow a 1024-element buffer if the total is more than 1024 elements,
|because you test AFTER appending; resetting the first element of the buffer
|to NUL is not neeeded, either - the next szVarName will overwrite it,
|anyway.

The buffer (psz) is far bigger than 1024; it's the buffer passed to my
function (as in "INT WINAPI VARNAMES (WCHAR *psz));

I suspect Sprintf() is quite fast; after seeing "%s" it knows it's
copying a nul-terminated string and probably does something akin to
the code below (which is what memcpy, wcscpy, et c. would do). With
the method you suggested, I'd have to (separately) get the length of
szVarName to properly increment r. So instead of Sprintf, I tried
this (probably about as fast as it'll get, and adding 32 bytes to the
".text" segment and making anyone reading the code have to think a
little more).

Code:
WCHAR *z = szVarName;
while ( *z )
	*r++ = *z++;
*r++ = L'\r';
*r++ = L'\n';
*r = 0;

The difference:

Code:
v:\> timer & for /l %i in (1,1,10000) (varnames > nul) & timer
Timer 1 on: 13:33:57
Timer 1 off: 13:34:00  Elapsed: 0:00:02.80

v:\> timer & for /l %i in (1,1,10000) (varnames > nul) & timer
Timer 1 on: 13:38:06
Timer 1 off: 13:38:09  Elapsed: 0:00:02.80

That's why sprintf, strcpy, memcpy (and friends and relatives) exist
... to prevent continual re-invention of the wheel. I think the
overhead in such functions is small compared to the job they do.
 
Similar threads
Thread starter Title Forum Replies Date
cgunhouse Typing Speed Support 9
deleyd Speed up startup of TCMD? Support 2
Gisle Vanem How to? 4NT "detach" question Support 2
D PDIR question Support 0
C forum Posting Question... Support 3
C question re: Move Support 3
S How to? Upgrade Question: What supporting documentation is required at time of purchase? Support 2
Peter Murschall Documentation Question to %@PSHELL Support 4
Fross Tab Question Support 6
Fross Quick Function Question Support 17
Dick Johnson Question about the Touch command Support 0
rps Documentation "Copy+Paste+run" question Support 2
vefatica SFTP question Support 17
vefatica Question about IPWorks Support 0
C How to? SHORTCUT question..... Support 6
vefatica TPIPE, crash and question Support 1
C Question / Suggestion Support 1
MickeyF Another TPIPE question Support 6
vefatica Another @EVERYTHING question Support 4
mikea How to? %@everything[] question Support 10
A License Question for Single User Support 5
Jay Sage Help Correction (and Related Question) Support 0
S Elapsed time in TCC prompt question Support 0
H command line parsing question Support 5
C Forum question Support 2
B Newbie here, I have a question. Support 4
J Dumb Question ... Support 4
Timothy Byrd Updating from TC 9 to TC 17 - TCTOOLBAR question Support 2
M A possibly stupid question about the "del" command... Support 7
M A just out of curiosity question re @DiskTotal Support 4
Fross Status Bar Time Question Support 1
A Stupid, obvious question - launch a BAT with TC Support 10
C How to? GLOBAL question Support 5
vefatica OT: VisualStudio question Support 1
vefatica Wildcard question Support 4
Dan Glynhampton v15 installer question Support 2
C Move question with Encrypted file... Support 6
M Maybe Maybe I'm being stupid (again), but a simple question... Support 4
C Message Board question Support 1
James Miller How to? filename completion question Support 1
C coding question Support 3
M A just-out-of-curiosity question because it's ... Support 6
Frank question to FOLDERMONITOR Support 14
C Dumb titleprompt question Support 4
M Upgrade Question Support 2
S option /u question Support 19
M How to? A probably stupid question re "@Files" and Include Lists... Support 2
M Just an out-of-curiosity question re. "Do ... /P ..." Support 31
M How to? An admittedly somewhat silly question regarding elevation... Support 2
M Just an out-of-curiosity question the List command vs. the @Lines function... Support 13

Similar threads