DIR /: /u ... streams not counted?

May 20, 2008
11,400
99
Syracuse, NY, USA
The stream wasn't counted in the byte total when I added "/u". WAD?

Code:
v:\streamtest> dir /k /h /:
2013-06-10  21:43        223,424  du.exe
                              26    Zone.Identifier:$DATA
          223,450 bytes in 1 file and 0 dirs    225,280 bytes allocated
    6,174,920,704 bytes free
 
v:\streamtest> dir /k /h /: /u
          223,424 bytes in 1 file and 0 dirs    225,280 bytes allocated
    6,174,920,704 bytes free
 
May 20, 2008
11,400
99
Syracuse, NY, USA
WAD. The file size of the file includes the sizes of all of its streams.
It doesn't appear so. Here, du2.exe is a copy of du.exe made before any streams were created. Neither byte total seems to include the stream. And the byte totals disagree; the one from "/U" seems correct (but doesn't count the stream) and the other is 4 bytes (?) more.
Code:
v:\streamtest> dir /: /k /h
2013-07-02  22:28        223,424  du.exe
                            4,202    stream:$DATA
2013-07-02  22:27        223,424  du2.exe
          446,852 bytes in 2 files and 0 dirs    450,560 bytes allocated
    6,249,238,528 bytes free
 
v:\streamtest> dir /: /u /k /h
          446,848 bytes in 2 files and 0 dirs    450,560 bytes allocated
    6,249,238,528 bytes free
 
May 20, 2008
11,400
99
Syracuse, NY, USA
Here's another test. Try this BTM file. The two totals differ by 4 and don't include the stream.
Code:
v:\stream> type streamtest.btm
del /q file*
do i=1 to 50 (echo %@repeat[x,4000] >> file1.txt)
copy /q file1.txt file2.txt
echo %@repeat[x,4200] > file1.txt:stream
echo DIR /: /K /H
dir /: /k /h
echo ^nDIR /: /K /H /U
dir /: /k /h /u
 
v:\stream> streamtest.btm
DIR /: /K /H
2013-07-02  23:02        200,100  file1.txt
                            4,202    stream:$DATA
2013-07-02  23:02        200,100  file2.txt
2013-07-02  23:01            210  streamtest.btm
          400,414 bytes in 3 files and 0 dirs    405,504 bytes allocated
    6,248,828,928 bytes free
 
DIR /: /K /H /U
          400,410 bytes in 3 files and 0 dirs    405,504 bytes allocated
    6,248,828,928 bytes free
 

rconn

Administrator
Staff member
May 14, 2008
12,345
150
I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?
 
May 20, 2008
11,400
99
Syracuse, NY, USA
I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?
Doesn't sound worth it. But even without /U, stream bytes are counted wrongly depending on the stream size. In the first example below it seems the stream (998 bytes) was counted and the byte total is correct. In the second example, it seems the stream (now 1998 bytes) contributed only 3 bytes to the total!
Code:
v:\stream> dir /:
 
Directory of  V:\stream\*
 
2013-07-03  10:13        200,100  file1.txt
                              998    stream:$DATA
2013-07-03  10:13        200,100  file2.txt
2013-07-03  10:13            209  streamtest.btm
          401,407 bytes in 3 files and 2 dirs
 
v:\stream> dir /:
 
Directory of  V:\stream\*
 
2013-07-05  14:56        200,100  file1.txt
                            1,998    stream:$DATA
2013-07-05  14:56        200,100  file2.txt
2013-07-05  14:56            210  streamtest.btm
          400,411 bytes in 3 files and 2 dirs
 
May 20, 2008
3,515
4
Elkridge, MD, USA
I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?
When I need to know whether or not the suite of files will fit on another disk volume, the stream size information is critical - but differences in allocation unit (cluster) sizes are also significant, and so is the issue whether or not the target file system supports streams. It would be too much for the DIR command to do. I propose a novel solution: a new TCC feature to determine whether or not the specified suite of files would fit on a target. This could be a new command, a new conditional expression, or a variable function. It would need the target volume to be accessible and explicitly specified, or - a less reliable situation - its relevant properties available.
 
May 20, 2008
11,400
99
Syracuse, NY, USA
Steve, do you really think it is likely to make a critical difference? On my system drive (63,313) files there are only 37 alternate streams, totalling 44,770 bytes. No doubt nearly all of them are so small that they don't get any (extra) allocated space, but instead fit into the file's MFT record. The most common stream is :Zone.Identifier:$DATA ... tiny ... almost never is there any space outside the MFT allocated for them. Another common stream is :favicon:$DATA. They're often a few hundred bytes and often fit in the MFT entry.

:-) A notable exception ... I have an internet shortcut to the Wall Street Journal (puzzle page). The .URL file has a favicon stream of 34,494 bytes! The icon isn't even particularly impressive. ... kinda makes you wonder!
 
May 20, 2008
3,515
4
Elkridge, MD, USA
As mentioned in HELP -> NTFS File Streams, you can save many versions of a file in its streams. As long as you deal only with stream-supporting file systems (AFAIK that means NTFS) that's convenient, but makes the directory totals meaningless for target allocation purposes. NOTE that I did not suggest that DIR /: /U deal with the issue, rather, that it be a new feature (function or command), used on the rare occasion when it is able to save you an incomplete backup - and such situations warrant all the extra time Rex mentioned. BTW, I don't use streams, not even for descriptions, since my primary backup devices are not NTFS.
 
Similar threads
Thread starter Title Forum Replies Date
vefatica DIR /F and streams? Support 7
vefatica DIR, streams, and wildcards? Support 1
E Fixed Bug with DIR /Z displaying descriptions Support 8
J Paths shown in DIR /B Support 2
K Fixed Prompt display will be shifted after use dir to display a filename with Chinese. (v25.00.28 x64) Support 18
Jesse Heines How to? How to display picture creation date with dir command Support 6
vefatica WAD DIR.BTM? Support 11
DrusTheAxe DIR reports meaningless SYMLINK information Support 14
C show file description? with dir? Support 8
rps Multi-column DIR /v not displaying all files. Support 5
R How to? Dir specific file search patterns with spaces in the pathnames? Support 6
rps Dir /Nfv -> Alt-F2 Support 2
rps @FILESIZE[....,a] allocated size not matching Dir results Support 8
A TCMD - Dir Command puts out blank lines? Support 16
S Problems with dir command in the debugger Support 5
M TCC incorrect dir output since Windows 1803 Support 6
x13 Problem listing repository files using DIR http(s)://... Support 8
cxxl dir /s works in mysterious ways :( Support 4
vefatica Help nit (FFIND and DIR with /S) Support 0
N Fixed Strange dir behavior Support 6
JohnQSmith Weird DIR output (missing lines) Support 1
C 7zip with date range .vs. filelist created with dir and daterange Support 0
D Towards shared (dir-)history lists Support 3
vefatica WAD DIR /HL still gets names wrong Support 16
vefatica DIR /S /HL? Support 4
H Fixed DIR /G returns wrong sizes Support 2
nickles WAD dir.htm Support 2
vefatica DO dir in /s /a:+d /d"g:\" * ( ... ) Support 26
vefatica DIR \\.\...? Support 4
M Fixed DIR /S /B1 ignores "/S" Support 5
C tcmd.ini not loading from program dir? Support 5
D Fixed Dir /Nm:n has changed Support 1
rps How to? dir /s unexpected results Support 10
vefatica Update to current install dir? Support 8
cgunhouse Problem with "dir /=" Support 4
T dir /h error in empty directory Support 22
P WAD TC 15.0.1.58 x64 crasches with a simple dir command Support 18
CWBillow dir /4 strange Support 2
samintz WAD DIR /B1 and /X Support 2
nickles dir behaves inconsistently Support 5
vefatica Documentation DIR /B /S /: Support 2
samintz How to? DIR listing for exact match Support 1
dcantor WAD dir "ftp:// ..." fails in TCC 15 Support 7
T How to? dir/pdir - 2nd level down only Support 7
MikeBaas How to? DIR: supress extensions? Support 5
old coot dir /s dies on my C: drive Support 2
A WAD Dir daterange + multiple path wildcards crashes tcc Support 2
old coot TC DIR command has trouble on my SSD Support 2
M Fixed character set in dir/copy Support 3
C odd behavior of "dir" Support 0

Similar threads