TCC/LE Causing 99% CPU Spikes and DIR Delays

[PAB]

I am using TCC/LE 9.02.154 on Windows XP SP3 (though the problem I'm about to describe affects previous versions and other platforms too).

My primary reason for using TCC/LE is to show certain file types in various colors, but I am seeing major CPU spikes and long delays when trying to associate colors with a large number of extensions.

I am using a TCSTART.BAT file (renamed and attached) in the TCC/LE program directory and defining my color choices with "set colordir=". But the more extensions I assign colors to, the worse TCC/LE behaves.

For example, if I assign multiple various colors to a total of 710 extensions and then issue a DIR command within C:\WINDOWS\system32\, there is a 20 second pause before TCC/LE starts listing the files - and during that time, tcc.exe will be utilizing 99% of the CPU.

If I drop down to 130 extensions, however, the same command then takes 3 seconds to start - so it's obviously just an issue with the number of extensions being used.

Naturally the more files there are in a directory, the longer the delay before listing will be. And even after a listing, any relisting will take just as long - implying there is no form of caching going on. I've even seen larger directories take one minute or longer to start listing.

Is there any chance this is a bug which can be fixed or maybe a code issue which can be compensated for in some way?

Or is this problem a design limitation of the software?

 

[rconn]

I am using TCC/LE 9.02.154 on Windows XP SP3 (though the problem I'm about to describe affects previous versions and other platforms too).

My primary reason for using TCC/LE is to show certain file types in various colors, but I am seeing major CPU spikes and long delays when trying to associate colors with a large number of extensions.

I am using a TCSTART.BAT file (renamed and attached) in the TCC/LE program directory and defining my color choices with "set colordir=". But the more extensions I assign colors to, the worse TCC/LE behaves.

For example, if I assign multiple various colors to a total of 710 extensions and then issue a DIR command within C:\WINDOWS\system32\, there is a 20 second pause before TCC/LE starts listing the files - and during that time, tcc.exe will be utilizing 99% of the CPU.

If I drop down to 130 extensions, however, the same command then takes 3 seconds to start - so it's obviously just an issue with the number of extensions being used.

Naturally the more files there are in a directory, the longer the delay before listing will be. And even after a listing, any relisting will take just as long - implying there is no form of caching going on. I've even seen larger directories take one minute or longer to start listing.

Is there any chance this is a bug which can be fixed or maybe a code issue which can be compensated for in some way?

Or is this problem a design limitation of the software?

710 extensions is pretty extravagant; I don't think I've ever seen anybody using 1/10 as many before.

I'm afraid that's the way it's designed -- the colorization code has to check for a lot of options (extension, attributes, and boolean operators) for each extension. For a large directory, there's a whole lot of crunching going on.

Note that you can use wildcards in the extensions, so you ought to be able to reduce your total somewhat.

 

[PAB]

710 extensions is pretty extravagant; I don't think I've ever seen anybody using 1/10 as many before.

I'm afraid that's the way it's designed -- the colorization code has to check for a lot of options (extension, attributes, and boolean operators) for each extension. For a large directory, there's a whole lot of crunching going on.

Note that you can use wildcards in the extensions, so you ought to be able to reduce your total somewhat.

OK, thanks very much for responding Rex. I appreciate it.

 

[Jim Cook]

> I am using a TCSTART.BAT file (renamed and attached) in the TCC/LE program directory and defining my color choices with "set colordir=". But the more extensions I assign colors to, the worse TCC/LE behaves.

> Or is this problem a design limitation of the software?

On my machine, it takes 9.71 seconds to do the DIR using your file.
Curiously, using my DIR replacement and the same colorization, it only
takes 0.89 seconds. I'm not using anything more clever than a linked
list to lookup, so I can't imagine what Rex might be doing.

Without any colorization, 4NT takes 0.25 seconds.

--
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
Measure wealth by the things you have for which you would not take money.

 

[logic]

From: JP Software Forums [mailto:[email protected]]
Sent: Sunday, November 09, 2008 11:59 PM
Subject: [Support-t-631] TCC/LE Causing 99% CPU Spikes and DIR Delays

>
> For example, if I assign multiple various colors to a total of 710
> extensions and then issue a DIR command within C:\WINDOWS\system32\,
> there is a 20 second pause before TCC/LE starts listing the files - and
> during that time, tcc.exe will be utilizing 99% of the CPU.
>
> If I drop down to 130 extensions, however, the same command then takes 3
> seconds to start - so it's obviously just an issue with the number of
> extensions being used.

I have a feeling Rex doesn't believe in hash tables. Similar slowdowns occur
in 4NT/TCC when you have large (even moderate) numbers of aliases or
functions, even if they're provided by plugins.

Jonathan Gilbert

 

[Jim Cook]

>> My primary reason for using TCC/LE is to show certain file types in various colors, but I am seeing major CPU spikes and long delays when trying to associate colors with a large number of extensions.

> Rex said:
> I'm afraid that's the way it's designed -- the colorization code has to check for a lot of options (extension, attributes, and boolean operators) for each extension. For a large directory, there's a whole lot of crunching going on.

If you are just trying to get a colorized output, and can live with a
program that will only match on extension, not the extensive set that
Rex supports, you could try my timedir program at
http://jcook.net/timedir

I tried to have most options behave like 4NT's DIR/PDIR do. It has
some things that DIR doesn't, and doesn't have some things that DIR
does.

Anyway, in my own tests for the one colorization file you sent, the
syntax is very close to the same and timedir performed very much
quicker. Perhaps it'll be useful. The "hidden" attribute you use is
not supported by timedir at this time, and will simply look for an
extension of ".hidden"

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.

 

[Jim Cook]

> If you are just trying to get a colorized output, and can live with a
> program that will only match on extension, not the extensive set that
> Rex supports, you could try my timedir program at
> http://jcook.net/timedir

I updated version 5.73 to handle attributes as well.

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.

 

[PAB]

I updated version 5.73 to handle attributes as well.

Belated thanks Jim, I just downloaded TIMEDIR version 5.80 and this seems to work well for me. Even with all of the extensions I have defined there are no noticable DIR delays or CPU spikes.

Two questions though:

1. Would it be possible to add support for "InputColors" (for input text color) and "ErrorColors" (for error message text color)?

After my original post, I also added the following to my TCSTART.BAT file:

option //InputColors=bright white on black
option //ErrorColors=black on bright red

and find this to be extremely useful. I realize these are general CMD shell colors whereas TIMEDIR is strictly specific to the DIR command, but I figure it doesn't hurt to ask. :)

2. How would I configure a system to (a) always use TIMEDIR instead of DIR whenever I issue the DIR command and (b) always have TIMEDIR automatically start with some necessary parameters (in my case, "/R:C:\PROGRA~1\TimeDir\TDCOLORS.CFG /A")?

Right now, I add the "DIRCMD" system variable set to "/o:gn /a" within my Environment Variables (under the System Properties panel). But as far as I can tell, TIMEDIR isn't honoring that global /A parameter used by DIR (and only appears to honor /O:GN because that also happens to be a default in TIMEDIR).

Just FYI, I did also find one minor incompatibility issue. Bracketed wildcards are not processed the way TCC/LE handles them. For example, ra[r01] should equal rar, ra0, and ra1 and r[012]* should equal r0*, r1*, and r2*. But such "[ ]" combinations do not work with TIMEDIR. This is no big deal, however, since you can just explicitly define each individual extension instead - but it is a compatibility difference worth noting.

Thanks very much.

 

[Jim Cook]

> 2. How would I configure a system to (a) always use TIMEDIR instead of DIR

If you want to replace the default behavior of any command, built-in
or external, you should use an alias.

For the rest of the TIMEDIR specific questions, please email me at jim
at jcook dot net and I'll gladly discuss details. I couldn't see your
direct email address in any of the headers.


--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.

 

[Jim Cook]

> Right now, I add the "DIRCMD" system variable set to "/o:gn /a" within my Environment Variables (under the System Properties panel). But as far as I can tell,

According to the DIRCMD help and my testing, the DIRCMD system
variable is not honored in 4NT.

Topic "dircmd.htm" last edited 1/27/2008

"The DIRCMD variable is used by some versions of CMD.EXE to hold
default options for the DIR command. TCC does not directly support
this variable."

--
Jim Cook
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.

 

[PAB]

Duh... Nevermind... I should have gotten some sleep before posting. :)

After waking up, I realized that all I really needed to do was add this:

@alias dir=*C:\PROGRA~1\TimeDir\timedir /R:C:\PROGRA~1\TimeDir\TDCOLORS.CFG /A
​

to my TCSTART.BAT file and remove all the "COLORDIR" entries.

Now I get TCC/LE with all the colors I want and no performance issues... :D

Thanks again Jim!

PS: I was only using DIRCMD with the OS Command Prompt. I also had to use an alias entry in TCSTART.BAT for the same support in TCC/LE. Sorry for not being more clear about that.

 

[Steve Fabian]

PAB wrote:
| ...
| to my TCSTART.BAT file

Note that using either the .BAT or the .CMD extension results in slower
execution than using .BTM. .BTM files are copied into internal storage in
their entirety, and executed thence. .BAT and .CMD files are read one line
at a time to support self-modifying batch files.
--
HTH, Steve