WAD TC 15.0.1.58 x64 crasches with a simple dir command

Oct 14, 2008
6
0
I have some (big) file server (about 2TB data) where need a list of off line files using ...

dir \\fileservername\D$\*.* /s /ao /b >> Offline-Files.txt

... but TC disappears without any error message after a certain number of files listed in Offline-Files.txt (this run 60528).

It looks like a bug (buffer issue??).
Any idea how to get this fixed?
 
Oct 14, 2008
6
0
Accidentally this did not help.
Again 60528 files where identified and than TC disappeared
(not even to be found in the task list anymore).
I am afraid we have a bug here.
I'll try the same procedure on another file server and will post the result soon.
 
Oct 14, 2008
6
0
On another file server it was even faster and TC disappeared after 1319 found files.
Still I am afraid we have a bug here.
 
Dec 2, 2008
224
2
Canada
I tried the following:

md \dfasdfasdfasdfasdf
md \dfasdfasdfasdfasdf\fasdfasdfasdf
:
md \dfasdfasdfasdfasdf\fasdfasdfasdf\...\fasdfasdfas

where I kept adding more and more directories until TCMD 16.00.40 crashed. I also tried to delete this direcory using:

rmdir /s \dfasdfasdfasdfasdf

It also crashed.

I think somewhere on your file server is a directory that is to long for TCC to handle and the "dir" commnand just stops.

This on a XP SP3 Pro Workstation
 
Last edited:
Oct 14, 2008
6
0
Is there a limitation of the directory depth?
FYI:
I used to use Ghislers Total Commander find functionality (also with the offline attribute)
And Total Commander came back without issues (1.670.072 files found).
The only issue with Total Commander is to produce a proper list of the files found.
Still looking for a simple solution.
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
This is WAD, not a bug. (And it's been this way for many years.)

DIR loads all of the files in any given directory into memory before displaying the directory and then processing the subdirectories. Directories with a very large number of files (and with a limited amount of memory for TCC) will exhaust the system heap and/or stack space.

For something like this, you'd be a lot better off using a DO loop (which will be slower but only process one file at a time).
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
I tried the following:

md \dfasdfasdfasdfasdf
md \dfasdfasdfasdfasdf\fasdfasdfasdf
:
md \dfasdfasdfasdfasdf\fasdfasdfasdf\...\fasdfasdfas

where I kept adding more and more directories until TCMD 16.00.40 crashed. I also tried to delete this direcory using:

rmdir /s \dfasdfasdfasdfasdf

It also crashed.

This is a known Windows API bug.
 
May 26, 2008
537
4
I think somewhere on your file server is a directory that is to long for TCC to handle and the "dir" commnand just stops.
I really hope TCC doesn't crash because of this. CMD handles it without crashing and just outputs the message "Directory too long" or something like that.
 
Oct 14, 2008
6
0
Apologize to say but I thought TC is even better than CMD.
That is why I would comment WAD as NGD (no good design).
I would expect that TC reports an error at least instead of disappearing at all.
Is there a chance to get something else within TC?
Will try the proposed loop you mentioned before.
Do you know another command line based tool that could provide me the list?
If you like in a privat email.
Thanks
 
May 20, 2008
3,515
4
Elkridge, MD, USA
Is there a limitation of the directory depth?
For any specific disk volume the limit is MAX_PATH (260 characters) for the full path name, including the terminating NUL character. However, this can be circumvented by mapping a volume to a subdirectory of the parent volume, e.g., by the SUBST or the NET USE commands.

MSDN articles also mention that it is possible to use up to 32767 Unicode characters, and even more, under special circumstances.
 
May 20, 2008
3,515
4
Elkridge, MD, USA
This is WAD, not a bug. (And it's been this way for many years.)

DIR loads all of the files in any given directory into memory before displaying the directory and then processing the subdirectories. Directories with a very large number of files (and with a limited amount of memory for TCC) will exhaust the system heap and/or stack space.

For something like this, you'd be a lot better off using a DO loop (which will be slower but only process one file at a time).
Could this be documented in Limitations?
Could the impending heap / stack overflow be detected before TCC crashes?
 
Apr 13, 2010
304
7
61
The Hague
No exeception, out-of-heap, stack overflow, crash or sudden-death could ever be WAD.
A decent intelligible error message is the minimal respons for any condition to be considered "handled" as designed.
There is point calling something a design if the design says "do nothing".
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
Apologize to say but I thought TC is even better than CMD.
That is why I would comment WAD as NGD (no good design).
I would expect that TC reports an error at least instead of disappearing at all.
Is there a chance to get something else within TC?
Will try the proposed loop you mentioned before.

This issue comes up only once every two or three years. I could trap it in TCC, but (1) it would slow down DIR processing significantly (adversely affecting the 99.999999999999999% of DIRs that don't have 60K+ files in a directory); and (2) it wouldn't do anything useful anyway, other than saying "out of memory".
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
Could this be documented in Limitations?
Could the impending heap / stack overflow be detected before TCC crashes?

Sure. Are you willing to accept a significant slowdown in all of your directory accesses (i.e., nearly everything you do in TCC) so TCC can intercept something that you've never encountered, and are extremely unlikely to ever encounter?

If so, add it to the feedback forum. If enough people say they want TCC to be slower, I will implement it.
 
May 20, 2008
3,515
4
Elkridge, MD, USA
The occasion I encountered the issue is typically with using a single PDIR command to accumulate a catalog of a whole primary drive, including CRC, inode and link count information. Certainly a very, very infrequent operation, not worth slowing down TCC. (I presume PDIR is sharing relevant code with DIR). OTOH documenting that it is conceivable that DIR runs out of resources when more than some number of files are in a single directory, or whatever the restriction is, both in the Limitations as well as the DIR topic (and PDIR if applicable) would be a cheap warning.
 
Jun 28, 2008
41
0
I doubt I will ever run into this, so, honestly, I don't honestly have a strong opinion on this. Yet, if CMD is capable of detecting a problem, and I've never found CMD's dir to be intolerably slow, I would tend to think that TCC could too.

I'm a programmer myself. Though, obviously, you know your product and your coding better than I do. But, it just seems to me that when you call malloc() or whatever memory allocation call you use, that a simple test if it is valid shouldn't be a significant overhead, and generally considered good programming practice. But, again, I know you know your product better than I do, but, just from my own experience, it doesn't seem like it should be a significant overhead.

But, again, since I doubt I will run into it, I will let the people who have stronger opinions discuss it in more detail.

The only reason I even opened this thread was I found the "WAD" tag on a thread entitled TCC crashes on dir to be a bizarre WAD.
 

rconn

Administrator
Staff member
May 14, 2008
12,340
149
I'm a programmer myself. Though, obviously, you know your product and your coding better than I do. But, it just seems to me that when you call malloc() or whatever memory allocation call you use, that a simple test if it is valid shouldn't be a significant overhead, and generally considered good programming practice. But, again, I know you know your product better than I do, but, just from my own experience, it doesn't seem like it should be a significant overhead.

TCC is running out of stack space, not heap space.
 
Feb 1, 2010
38
0
TCC is running out of stack space, not heap space.
Is it really __try/__except is so much overhead? I would imagine entering __try block it's much less time than the kernel calls needed to actually read a directory content.
 
Similar threads
Thread starter Title Forum Replies Date
S Issues with version v26.02.42 x64 Support 0
fishman@panix.com 27.00.17 x64 will not install for me!! Support 4
K Fixed Prompt display will be shifted after use dir to display a filename with Chinese. (v25.00.28 x64) Support 18
DrusTheAxe TCMD 24.02.49 x64 crashed due to DivideByZero :-( Support 4
Joe Caverly Help not launching in CMDebug v23.00.14 x64 Support 1
C Del /W999 2gbFile.ext / latest build / Win7 x64 / MSE / Everything Support 5
thorntonpg 21.00.15 x64 to .16 failed Support 4
Joe Caverly Windows x64 Support 4
C sendmail in v20.00.25 x64 Support 7
Craig Fitzgerald problem with executable extensons with TCC version 19.10.51 x64 Support 3
gentzel Fixed dirs +n bug in 20.0.12 x64 Support 2
J How to? 20beta: How to install x86 on x64 Support 2
Alpengreis [v19.0.32 (x64)] Problem with [Options, Tabbed Toolbar, Add Button] in german language Support 1
bwawsc2 Can't install updates to TCMD x64 Support 5
Alpengreis [Bug?] View (V) Prefs Error (TC 18.00.27 x64) Support 4
jbarnes1967 TC 18.00 x64 issue with lua io.popen() Support 2
I Take Command 18 x64 Install Hang Support 2
P MS VS2013 vsdevcmd.bat fails to run with tcmd 17 x64 Support 10
B Documentation Reference/Windows X64: Redundant text at the end Support 0
S Install TC 17.00.37 x64 expires immediately & invalidates registered TCmd v16.03.55 Support 3
G How to? Trying to create TCC shortcut that opens with blue background (on Windows 8.1 x64) Support 1
C SafeChars x64 plugin under Win7 x64 Support 2
C WAD @index not working in v16 x64 Support 14
MickeyF problem using COM object in VBScript from v16 x64 TCC but not from v15 32-bit TCC Support 4
B Take Command x64 15.01.55 Error: Update installation failed Support 7
samintz Can't install 15.01.54 on Win7 x64 Support 6
C Latest TCMDx64 fails on XP Pro x64 Support 1
gschizas Installation folder for TCMD x64 15.1 is wrong Support 5
B tpipe and MSVCR110.dll TCC x64 Build 44?? Support 19
greyfairer Very slow startup since last Windows 7 Update (14.03.59 x64) Support 9
W /g option (%) on MOVE cmd appears broken TCC 14.03.57 x64 Support 8
7 TCC startup crash in Windows 8 Pro x64 Support 20
Frank problem with environment variable x86 vs. x64 Support 2
W Custom colors not saved in TCC 14 / Windows 7 x64 Support 4
S Fixed View and V do not work in new 13.04.52 x64 installation Support 11
P WAD Bug in TC v 13.03 build 39 x64 Support 1
G Problem with TC 13 x64 Support 7
jason404 I would like to buy TCC x64 Support 1
K_Meinhard x64 Installation Support 0
S Location of tcmd.ini in x64 Support 4
fromano Invalid parameter 12.0.34 x64 Support 1
G TCMD x64 Build 28 Crashing Support 5
mscheuner WINDOW doesn't work under TCMD v11 x64 b40 ? Support 4
Juanma Barranquero list /c fails for attached TCC on TCMD 11 x64 Support 0
Juanma Barranquero Several nitpicks and a crash in TCMD 11 x64 / Windows 7 Support 17
fpefpe plugin and x64 Support 2
mscheuner Background color not "sticking" in TCMD v11 x64 on Win7 Support 3
rconn v11.0.28 x64 reuploaded Support 1
Fross Win7 x64 Dialog Size Issue Support 3
rconn Seeking Take Command x64 beta testers Support 0

Similar threads