DIR /: /u ... streams not counted?

[vefatica]

The stream wasn't counted in the byte total when I added "/u". WAD?

v:\streamtest> dir /k /h /:
2013-06-10  21:43        223,424  du.exe
                              26    Zone.Identifier:$DATA
          223,450 bytes in 1 file and 0 dirs    225,280 bytes allocated
    6,174,920,704 bytes free
 
v:\streamtest> dir /k /h /: /u
          223,424 bytes in 1 file and 0 dirs    225,280 bytes allocated
    6,174,920,704 bytes free

 

[vefatica]

WAD. The file size of the file includes the sizes of all of its streams.

It doesn't appear so. Here, du2.exe is a copy of du.exe made before any streams were created. Neither byte total seems to include the stream. And the byte totals disagree; the one from "/U" seems correct (but doesn't count the stream) and the other is 4 bytes (?) more.

v:\streamtest> dir /: /k /h
2013-07-02  22:28        223,424  du.exe
                            4,202    stream:$DATA
2013-07-02  22:27        223,424  du2.exe
          446,852 bytes in 2 files and 0 dirs    450,560 bytes allocated
    6,249,238,528 bytes free
 
v:\streamtest> dir /: /u /k /h
          446,848 bytes in 2 files and 0 dirs    450,560 bytes allocated
    6,249,238,528 bytes free

 

[vefatica]

Here's another test. Try this BTM file. The two totals differ by 4 and don't include the stream.

v:\stream> type streamtest.btm
del /q file*
do i=1 to 50 (echo %@repeat[x,4000] >> file1.txt)
copy /q file1.txt file2.txt
echo %@repeat[x,4200] > file1.txt:stream
echo DIR /: /K /H
dir /: /k /h
echo ^nDIR /: /K /H /U
dir /: /k /h /u
 
v:\stream> streamtest.btm
DIR /: /K /H
2013-07-02  23:02        200,100  file1.txt
                            4,202    stream:$DATA
2013-07-02  23:02        200,100  file2.txt
2013-07-02  23:01            210  streamtest.btm
          400,414 bytes in 3 files and 0 dirs    405,504 bytes allocated
    6,248,828,928 bytes free
 
DIR /: /K /H /U
          400,410 bytes in 3 files and 0 dirs    405,504 bytes allocated
    6,248,828,928 bytes free

 

[rconn]

I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?

 

[vefatica]

I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?

Doesn't sound worth it. But even without /U, stream bytes are counted wrongly depending on the stream size. In the first example below it seems the stream (998 bytes) was counted and the byte total is correct. In the second example, it seems the stream (now 1998 bytes) contributed only 3 bytes to the total!

v:\stream> dir /:
 
Directory of  V:\stream\*
 
2013-07-03  10:13        200,100  file1.txt
                              998    stream:$DATA
2013-07-03  10:13        200,100  file2.txt
2013-07-03  10:13            209  streamtest.btm
          401,407 bytes in 3 files and 2 dirs
 
v:\stream> dir /:
 
Directory of  V:\stream\*
 
2013-07-05  14:56        200,100  file1.txt
                            1,998    stream:$DATA
2013-07-05  14:56        200,100  file2.txt
2013-07-05  14:56            210  streamtest.btm
          400,411 bytes in 3 files and 2 dirs

 

[Steve Fabian]

I can add stream counting to /U. The disadvantage is that it will slow DIR by at least an order of magnitude, even if you don't have any files with streams attached. If you actually do have some streams, add another 200-500%.

Is it important enough to you to slow DIR by a factor of anywhere from 20 to 50 times?

When I need to know whether or not the suite of files will fit on another disk volume, the stream size information is critical - but differences in allocation unit (cluster) sizes are also significant, and so is the issue whether or not the target file system supports streams. It would be too much for the DIR command to do. I propose a novel solution: a new TCC feature to determine whether or not the specified suite of files would fit on a target. This could be a new command, a new conditional expression, or a variable function. It would need the target volume to be accessible and explicitly specified, or - a less reliable situation - its relevant properties available.

 

[vefatica]

Steve, do you really think it is likely to make a critical difference? On my system drive (63,313) files there are only 37 alternate streams, totalling 44,770 bytes. No doubt nearly all of them are so small that they don't get any (extra) allocated space, but instead fit into the file's MFT record. The most common stream is :Zone.Identifier:$DATA ... tiny ... almost never is there any space outside the MFT allocated for them. Another common stream is :favicon:$DATA. They're often a few hundred bytes and often fit in the MFT entry.

:-) A notable exception ... I have an internet shortcut to the Wall Street Journal (puzzle page). The .URL file has a favicon stream of 34,494 bytes! The icon isn't even particularly impressive. ... kinda makes you wonder!

 

[Steve Fabian]

As mentioned in HELP -> NTFS File Streams, you can save many versions of a file in its streams. As long as you deal only with stream-supporting file systems (AFAIK that means NTFS) that's convenient, but makes the directory totals meaningless for target allocation purposes. NOTE that I did not suggest that DIR /: /U deal with the issue, rather, that it be a new feature (function or command), used on the rare occasion when it is able to save you an incomplete backup - and such situations warrant all the extra time Rex mentioned. BTW, I don't use streams, not even for descriptions, since my primary backup devices are not NTFS.