DO /S ... ignore errors?

[vefatica]

After several other "Access is denied" errors, the command below terminates when c:\Recovery triggers that error. It doesn't try to process further directories in the root. Is that WAD? Can I change that behavior?

c:\> do dir in /s /a:d * ( echo %@full[%dir] )
<snip>
C:\ProgramData\Microsoft\Network\Connections
TCC: (Sys) Access is denied.
 "C:\ProgramData\Microsoft\Network\Downloader\"
<snip>
C:\ProgramData\VS\vs10sp1\SetupCache\Graphics
C:\ProgramData\Windows Genuine Advantage\Data
TCC: (Sys) Access is denied.
 "C:\Recovery\"
********** output stopped here ************
c:\>

 

[vefatica]

Similarly, here:

TCC: (Sys) Access is denied.
 "C:\ProgramData\Microsoft\OfficeSoftwareProtectionPlatform\"
C:\ProgramData\NVIDIA\SHIMGen

After the error, DO /S processed none of the (further) siblings of "OfficeSoftwareProtectionPlatform" inside "C:\ProgramData\Microsoft\".

 

[rconn]

DO does not terminate the loop when a command returns an error. DO is terminating because DO itself cannot access the next directory. There is no way to ignore that error.

 

[vefatica]

DO does not terminate the loop when a command returns an error. DO is terminating because DO itself cannot access the next directory. There is no way to ignore that error.

DO is not terminating! It's skipping the (further) siblings of the directory which caused the error and continuing with the next sibling of the parent (unlike any other directory recursion routine I have seen).
Look at this test carefully. It tries to look/go inside v:\test\dir1\subdir1 and can't. Then it (wrongly) skips the remaining subdirectories of v:\test\dir1 (which could be entered/listed) and continues with v:\test\dir2. Below, I think it should have found v:\test\dir1\subdir2\subsubdir1.

v:\test> tree

V:\test
├──dir1
│  ├──subdir1
│  ├──subdir2
│  │  └──subsubdir1
│  └──subdir3
└──dir2
  ├──subdir1
  │  └──subsubdir1
  ├──subdir2
  └──subdir3

v:\test> do dir in /s /a:d * ( echo %@full[%dir] )
V:\test\dir1
V:\test\dir2
V:\test\dir1\subdir1
V:\test\dir1\subdir2
V:\test\dir1\subdir3
TCC: (Sys) Access is denied.
 "V:\test\dir1\subdir1\"
V:\test\dir2\subdir1
V:\test\dir2\subdir2
V:\test\dir2\subdir3
V:\test\dir2\subdir1\subsubdir1

 

[rconn]

WAD.

DO is terminating the current directory processing loop. (It has behaved this way since day 1.) If DO cannot access something in the current directory, it does not try to continue processing other directory entries, as it's far more likely to do damage by blindly continuing than by giving up on this directory.

This will not change. If you want to request a new switch to continue even though something in the loop is broken, you know where to submit it.

 

[vefatica]

How foolish of me ... to think that DO /S would process the same directories as GLOBAL.

 

[rconn]

How foolish of me ... to think that DO /S would process the same directories as GLOBAL.

And in fact, the default behavior for GLOBAL and DO is exactly the same.

But GLOBAL also has a /I switch to ignore errors accessing directories, and continue processing from that subdirectory. Nobody (including you!) ever asked for a similar switch for DO.

 

[vefatica]

And in fact, the default behavior for GLOBAL and DO is exactly the same.

But GLOBAL also has a /I switch to ignore errors accessing directories, and continue processing from that subdirectory. Nobody (including you!) ever asked for a similar switch for DO.

Without /I, GLOBAL stops dead in its tracks when it can't process a directory. DO /S skips some directories it could have processed and continues. If DO /S stopped dead as GLOBAL does, I'm confident there would have been, by now, a request (more likely a public outcry) for a /I(gnore errors) option. As far as I know, GLOBAL always had that option; I can only verify back to 4DOS/1995.

My interest came from another forum thread in which the user wanted a list of directories containing .OBJ files. He was using GLOBAL, so I suggested "GLOBAL /I /Q IF EXIST *.OBJ ECHO %_CWD" which apparently works. Had he been using DO /S he (and I) might have thought the results were correct and that there were a few directories that couldn't be processed. There's no indication at all that siblings of unprocessed directories were just plain skipped.

IMHO, continuing at the next sibling should be the default behavior of DO /S. If that takes a new option and a request for it, I hereby make that request.

P.S. That's my whole beef ... that DO /S unceremoniously skips directories it could have processed.

 

[Steve Fabian]

Doesn't %_do_errors make a correct report? Can ON ERROR be used in a desirable fashion?

 

[rconn]

IMHO, continuing at the next sibling should be the default behavior of DO /S. If that takes a new option and a request for it, I hereby make that request.

We don't save (or implement) feature requests made in this forum, only in the feedback forum.

 

[vefatica]

We don't save (or implement) feature requests made in this forum, only in the feedback forum.

I think of it as a bug ... DO /S skips directories that it could process. And the last time I tried to use the feedback forum, it wouldn't let me post because I have third-party cookies disabled. It's the only site that ever complained about that browser/privacy setting.

 

[rconn]

I think of it as a bug ... DO /S skips directories that it could process.

Then you should have mentioned something 15 years ago! Any change now in the default behavior would IMO be for the worse (and probably break existing code).

And the last time I tried to use the feedback forum, it wouldn't let me post because I have third-party cookies disabled. It's the only site that ever complained about that browser/privacy setting.

WAD. We had some users who were upset that nobody liked their ideas, so they connected anonymously and voted for themselves. Over and over.

 

[vefatica]

Then you should have mentioned something 15 years ago! Any change now in the default behavior would IMO be for the worse (and probably break existing code).

I wonder how many times in those 15 years a user accepted the results of DO as correct when in fact it had skipped the siblings of directories it couldn't access. There's no indication/documentation at all that that's what happens. It is behavior that's unlike any other directory recursion in TCC. I doubt users are even aware of it.

And if that aint bad enough, FOR /R finds only 4 directories containing the string "Microsoft" in the c:\ProgramData tree. This one's really screwy. It doesn't find . and .. inside C:\ProgramData\Microsoft and it otherwise doesn't seem to recurse at all.

c:\programdata> (for /r /a:d %dir in ( * ) echo %dir) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft
C:\ProgramData\Microsoft Help
C:\ProgramData\Microsoft Help\.
C:\ProgramData\Microsoft Help\..

  4 lines in  1 file

GLOBAL finds 1.

c:\programdata> (global /i /q  echo %_cwd) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft Help

  1 line in  1 file

DIR /S finds 225.

c:\programdata> dir /s /a:d /b | ffind /v /t"Microsoft"
<snip>

  225 lines in  1 file

c:\programdata>

DO /S finds 68.

c:\programdata> (do dir in /s /a:d * ( echo %@full[%dir] )) | ffind /v /t"Microsoft"
<snip>

  68 lines in  1 file

c:\programdata>

 

[Steve Fabian]

What are the causes preventing accessing a directory?

I agree with Vince that when DO /S is used to enumerate files or directories, and there is no indication that not all matches are reported, it is a BUG, not subject to the FEEDBACK mechanism.

Vince: the /U (summary only) option of FFIND would make snipping unnecessary in your last post.

 

[vefatica]

What are the causes preventing accessing a directory?

There are a few. I think lack of permission to "Traverse folder / execute file" is the one involved here.

 

[rconn]

First, Many of those "directories" are not directories (they're junctions). Second, everything other than DIR tries to access the directories / junctions and fail, because you don't have permission. DIR just lists them, which doesn't require permission.

 

[vefatica]

First, Many of those "directories" are not directories (they're junctions). Second, everything other than DIR tries to access the directories / junctions and fail, because you don't have permission. DIR just lists them, which doesn't require permission.

C:\ProgramData\Microsoft and all of it's immediate subdirectories are truly directories. Many (that I've tried) can be entered and listed by an ordinary user. The counts given in my second-last post are way off for FOR /R and GLOBAL (and we've already discussed DO /S). When I do the same 4 tests in an elevated session, FOR /R and GLOBAL produce exactly the same results (way off, and otherwise screwy) as in an unelevated session. In an elevated session the counts from DIR and DO /S agree (at 245). I still think the behavior of DO /S should be changed. And I'm pretty sure there's something wrong with FOR /R and GLOBAL. Here are those two tests again, this time in an elevated session.

c:\programdata> echo %_elevated
1

c:\programdata> (for /r /a:d %dir in ( * ) echo %dir) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft
C:\ProgramData\Microsoft Help
C:\ProgramData\Microsoft Help\.
C:\ProgramData\Microsoft Help\..

  4 lines in  1 file

c:\programdata> (global /i /q  echo %_cwd) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft Help

  1 line in  1 file

c:\programdata>

 

[vefatica]

And for added emphasis, here are the FOR /R and GLOBAL results when run as SYSTEM. The FOR /R one gives one fewer than when run in an elevated session (?).

c:\programdata> echo %_winuser
SYSTEM

c:\programdata> (for /r /a:d %dir in ( * ) echo %@full[%dir]) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft
C:\ProgramData\Microsoft Help
C:\ProgramData\Microsoft Help\

  3 lines in  1 file

c:\programdata> (global /i /q echo %_cwd) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft Help

  1 line in  1 file

c:\programdata> whoami
nt authority\system

c:\programdata>

 

[vefatica]

Apparently, FOR /R and GLOBAL are ignoring directories with the S(ystem) attribute (why? ... documented for FOR, not for GLOBAL). When I remove the S(ystem) attribute from C:\ProgramData\Microsoft. The counts obtained by my FOR /R and GLOBAL examples agree perfectly with the count gotten from DIR (and that's how it should be).

And then there's the question of DO /S skipping stuff!

 

[Steve Fabian]

First, Many of those "directories" are not directories (they're junctions). Second, everything other than DIR tries to access the directories / junctions and fail, because you don't have permission. DIR just lists them, which doesn't require permission.

Whether or not junctions are to be listed should be taken care of the relevant options , e.g., the /J or /NJ options of GLOBAL. For most commands the option /A: when running elevated is supposed to grant permission to access ALL directory entries (both files and directories), including junctions. Does it, or is it impossible to list the whole content of a volume?

 

[vefatica]

Whether or not junctions are to be listed should be taken care of the relevant options , e.g., the /J or /NJ options of GLOBAL. For most commands the option /A: when running elevated is supposed to grant permission to access ALL directory entries (both files and directories), including junctions. Does it, or is it impossible to list the whole content of a volume?

GLOBAL doesn't have an attribute switch. FOR does, but it apparently won't recurse into directories with the S attribute. It changes the output very slightly but makes it nowhere near correct. The results below are the same elevated or not.

c:\programdata> (for /r /d %dir in ( * ) echo %dir) | ffind /v /t"Microsoft"

---- CON
C:\ProgramData\Microsoft Help

  1 line in  1 file

c:\programdata> (for /r /d /a: %dir in ( * ) echo %dir) | ffind /v /t"Microsoft"


---- CON
C:\ProgramData\Microsoft
C:\ProgramData\Microsoft Help

  2 lines in  1 file

 

[rconn]

The attribute switch in FOR is for retrieving the files in the directory, not for filtering the directories it recurses into.

FOR and GLOBAL share the same code for traversing directories, so it's not surprising they would return the same results.

FOR and GLOBAL don't recurse into hidden / system directories because (1) 99.99999% of the time, it's a bad idea (there is usually a reason those directories were hidden), and (2) FOR exists solely for CMD compatibility. If you can find a syntax where CMD will do what you want with FOR, I will add it to TCC. I will not add anything to FOR that breaks CMD compatibility, even in minor ways.

 

[Steve Fabian]

The attribute switch in FOR is for retrieving the files in the directory, not for filtering the directories it recurses into.

Please elucidate.

FOR and GLOBAL don't recurse into hidden / system directories because (1) 99.99999% of the time, it's a bad idea (there is usually a reason those directories were hidden), and (2) FOR exists solely for CMD compatibility. If you can find a syntax where CMD will do what you want with FOR, I will add it to TCC. I will not add anything to FOR that breaks CMD compatibility, even in minor ways.

FOR's documentation explicitly (but with the qualification that they apply only if wildcard is included) includes the /A: option, and since V6 states that it supports attribute switches. Your statement above contradicts it!

IMHO it is grossly derogatory to the sophistication of those who progressed from CMD to TCC to make such sweeping claims what constitutes a bad idea.

 

[Steve Fabian]

BTW, if such a large discrepancy existed for all these versions between operation and documentation, probably all registered users ought to be notified

 

[vefatica]

Windows always had a "show things with the S attribute". Enforcing a restriction on such things today seems like an anachronism given Windows much-enhanced security. I don't care about FOR, but how about GLOBAL and DO /S?

 

[rconn]

Please elucidate.

There are two directory search loops occurring with FOR /R. The first one searches for matching files in the current directory. When that has completed, FOR begins a second loop looking for subdirectories in the current directory, CD's into each one, and calls itself recursively to run the first loop again. The FOR attribute switches only apply to the first loop.

FOR's documentation explicitly (but with the qualification that they apply only if wildcard is included) includes the /A: option, and since V6 states that it supports attribute switches. Your statement above contradicts it!

No -- see above.

IMHO it is grossly derogatory to the sophistication of those who progressed from CMD to TCC to make such sweeping claims what constitutes a bad idea.

IMHO this entire thread is nonsensical -- if you haven't noticed fundamental behavior for 20+ years, I doubt you actually have a need. I suspect you're simply arguing for the sake of arguing. But I'm willing to be convinced -- post it in the feedback forum (something neither you nor Vince seem willing to do, which doesn't imply a strong desire!) and we'll see if anybody else shows any interest.

 

[rconn]

Windows always had a "show things with the S attribute". Enforcing a restriction on such things today seems like an anachronism given Windows much-enhanced security. I don't care about FOR, but how about GLOBAL and DO /S?

As I've said, "Show" is completely different from "Go To". Commands like DIR simply show the content of directories. DO, FOR, and GLOBAL actually have to CD into each directory before running their command arguments. It's the CD access that's being blocked (by Windows).