How to? Batch file problem

[SteveW]

Please assist me in evaluating whether I can use Take Command to solve the following problem using DOS Batch (BAT) files.

I have a directory tree as show below containing four files of same name index.txt.

The goal is to read each line in the first file c:\index.txt to obtain the next directory path that points to the next index.txt file and repeat until all index.txt files are found empty. Next, create a file named ListFile.txt containing all directory paths found but in the reverse order as follows:

C:\b\index.txt
C:\b\c\index.txt
C:\a\index.txt
C:\index

Note: c:\b\c\index.txt is in the directory tree twice and should not be appear twice in the Listfile.txt file.

                    +-------- c:\b\index.txt ---<  --- empty
                    |
            +---------- c:\a\index.txt ---+ 
            |                              |
C:\index.txt   +                       +--------- c:\b\c\index.txt ---<  --- empty
            |
           +----------- c:\b\c\index.txt ---empty

Do not assume any restrictions on the depth of the directory tree or number of entries in a given index.txt files.

 

[vefatica]

I don't understand your diagram. You say c:\b\index.txt is empty. What will guide you to c:\b\c\index.txt?

What's in index.txt ... directory names (only)? ... fully-qualified? ... a list of fully-qualified index.txt names?

If the index.txt files are guaranteed to be correct, why not simply look for all the index.txt files and print their names and sizes?

pdir /s /(fpn z) index.txt

To reverse the order in which they are printed use

pdir /s /(fpn z) index.txt | tpipe /simple=34

To get all that into a file ...

pdir /s /(fpn z) index.txt | tpipe /simple=34 > ListFile.txt

 

[Steve Fabian]

As Vince wrote, what you want to do is not difficult using TCMD, but you ought to show the acutal contents of the top level index.txt file and at least of one other; preferably all for the small sample problem, and also show the desired result.

BTW: How can a file be in the directory tree twice? My guess is that the file is referenced in more than one index file. I guess that your hierarchy may contain files named index.txt which are not part of the set.

I tried to guess at the contents of your sample files, but could not. Your diagram, as displayed on my computer, does not provide a clear set of connections. You probably used a variable width font when drawing it, but tagged it as code, which results in it being displayed in fixed-width font, breaking vertical lines. Using the more options choice under the reply box displays your post with another choice: preview, which allows you to see what others reading your post will see.

 

[vefatica]

                                            +-------- c:\b\index.txt
                                            |
              +---------- c:\a\index.txt ---+
              |                             |
C:\index.txt  +                             +--------- c:\b\c\index.txt
              |
              +----------- c:\b\c\index.txt

With considerable difficulty, I reformatted that tree diagram (try it) and it doesn't make much sense to me. I doesn't look like a directory (or tree) structure. Can any index.txt point to any other? Can an index.txt point to itself (which would cause problems if you tried to do process them recursively)?

 

[vefatica]

P.S. I had to manually, after the fact, change the <code> font to Courier New (and then do some tweaking). Doesn't code-tagged text automatically get a monospaced font? It should! This (below) was cut right out of a console and pasted into the code insertion dialog. It doesn't look very good. Rex, can you make the default code font monospaced?

V:\tcsh\
└──tcsh-6.18.01
  ├──config
  ├──cygwin
  ├──nls
  │  ├──C
  │  ├──et
  │  ├──finnish
  │  ├──french
  │  ├──german
  │  ├──greek
  │  ├──italian
  │  ├──ja
  │  ├──pl
  │  ├──russian
  │  ├──spanish
  │  └──ukrainian
  ├──tests
  └──win32
      └──msg

 

[SteveW]

I apologize for the confusion so I have removed the diagram and have the original problem statement given to me.

PROBLEM STATEMENT

We have a set of files named "index.txt" in various directories on the C: drive with the following property:

Each file is either empty or contains a list of full directory paths to other index.txt files, which we call its children.

Your script will be invoked from a directory that has one of these index.txt files and starts parsing the entire tree.

It should then print the full pathname to the visited index.txt files in a bottom-up manner displaying each entry only once.

So here are the conditions:

1. Full pathname to any index.txt file can only printed once. Any reappearance of the same index.txt file in the tree should be skipped

2. Full pathname to empty index.txt files can be immediately printed if they have not been printed before.

3. Full pathname of the remaining index.txt files can be printed if they have not been printed before but the names of all of their children have already been printed.

Here is an example of how you would test your script:

Create a text file C:\index.txt that contains the following two lines:

C:\a\index.txt
C:\b\c\index.txt

Where C:\a\index.txt is another text file which contains the line:

C:\b\index.txt
C:\b\c\index.txt

And C:\b\c\index.txt and C:\b\index.txt are empty files.

If you start your program at C:\ it should first print:

C:\b\index.txt
C:\b\c\index.txt

in any order because they are empty followed by:

C:\a\index.txt

because its children have already been printed and finally:

C:\index

Do not assume any restrictions on the depth of the directory tree or number of entries in a given index.txt files.

 

[Charles Dye]

So you have to remember all filenames processed to prevent recurrence, and also do your processing recursively (for the bottom-up order)? With no size limits on the input files, and no depth limit on the recursion?

 

[SteveW]

So you have to remember all filenames processed to prevent recurrence, and also do your processing recursively (for the bottom-up order)? With no size limits on the input files, and no depth limit on the recursion?

Yes, thank you. I am open to all idea's how to solve this in DOS batch files.

 

[SteveW]

My general idea is to start with c:\index.txt and save each line to a new file resulting in 1.txt and 2.txt.
Then use path in 1.txt to change dir and repeat this resulting in 3.txt, ect....until all index.txt files are found empty.
Eliminate duplicate files keep the later files using FC file compare.
Find a way to create a ListFile.txt starting with N.txt,....3.txt, 2.txt, 1.txt so file in reverse order as required.

I am thinking that using files as stated allows for the requirement "With no size limits on the input files, and no depth limit"

Any better ideas will be welcomed!

 

[Steve Fabian]

What is the definition of "DOS batch files"? TCMD works only on MS-Windows platforms, XP or later. An earlier product, 4DOS, works on the PC-DOS/MS-DOS/FreeDOS platform (also on 32b Windows platforms up to XP). However, some people completely ignore that DOS is an IBM-coined acronym for "disk operating system" predating Windows by a decade, and refer incorrectly but exclusively to the Microsoft command line processors (shells in terminology adopted from Unix) COMMAND.COM and CMD.EXE as DOS. The file extension .BAT is associated with COMMAND.COM. If you must use batch files that can be executed by COMMAND.COM, this forum is not your best choice, it is like a child learning to read using Einstein's works. If you do not have such a restriction, the command processor part of TCMD, TCC, is quite capable of performing the task. Is this a school assignment?

 

[vefatica]

I'd try a recursive subroutine approach. This is off the top of my head and very sketchy. Since the OP didn't describe a strict tree structure, I'd be wary of infinite loops. The order, duplicates, and getting output into a file are further considerations.

:follow [ifile]
echo %ifile
iff %@filesize[%ifile] != 0
    do line in @%ifile
        iff "%@filename[%line]" == "index.txt" then
            gosub follow %line
        endiff
    enddo
endiff
return

 

[SteveW]

I'm using Windows 7, 64-bit and other pc's can be Windows XP, 32-bit. So TCMD should not be a problem, right? Thanks.

 

[SteveW]

I'd try a recursive subroutine approach. This is off the top of my head and very sketchy. Since the OP didn't describe a strict tree structure, I'd be wary of infinite loops. The order, duplicates, and getting output into a file are further considerations.

:follow [ifile]
echo %ifile
iff %@filesize[%ifile] != 0
    do line in @%ifile
        iff "%@filename[%line]" == "index.txt" then
            gosub follow %line
        endiff
    enddo
endiff
return

I'd try a recursive subroutine approach. This is off the top of my head and very sketchy. Since the OP didn't describe a strict tree structure, I'd be wary of infinite loops. The order, duplicates, and getting output into a file are further considerations.

:follow [ifile]
echo %ifile
iff %@filesize[%ifile] != 0
    do line in @%ifile
        iff "%@filename[%line]" == "index.txt" then
            gosub follow %line
        endiff
    enddo
endiff
return

What is follow % line? directory tree is unlimited as I said in a previous post by Charles Dye

 

[vefatica]

It's "gosub follow %line". That is, recursively call the "follow" subroutine.

How can the tree be unlimited? Do you have a hard drive with an infinite amount of space?

To do what you want it would seem that some nesting will be required. Whether or not you exceed the nesting level of the tool you're using will depend on the tool and the depth.

Hmmm ... you could first find all the index.txt files, index them 1, 2, ..., N, make an NxN incidence array (Aij = 1 if file i points to file j, else 0), then process the array ... but wait! if the tree is unlimited, you might need an infinite amount of memory. Darn!

 

[SteveW]

It's "gosub follow %line". That is, recursively call the "follow" subroutine.

How can the tree be unlimited? ............. if the tree is unlimited, you might need an infinite amount of memory. Darn!

LOL..............Darn! Thank you for posting the code in #11 Vince.

Please refer to the problem that was given to me in post #6 which specifies there are "With no size limits on the input files, and no depth limit". No worry of this situation occurring!

Please give feedback to my idea in post #9? It proposes to read each line and save to a new file 1.txt, 2.txt,.....after all index.txt files are found you can merge all the files into one file ListFile.txt in reverse order as specified in the problem statement in post #6. ListFile.txt can be printed or viewed in the editor. How would you implement this in DOS?

 

[vefatica]

Please give feedback to my idea in post #9? It proposes to read each line and save to a new file 1.txt, 2.txt,.....after all index.txt files are found you can merge all the files into one file ListFile.txt in reverse order as specified in the problem statement in post #6. ListFile.txt can be printed or viewed in the editor. How would you implement this in DOS?

With that strategy ...

What about duplicates (you mentioned they could exist)?

Forgetting about reversing the order for a minute ... is that the order you want (later to be reversed)? ... all the files mentioned in the first-visited index.txt listed, then all the files mentioned in the next-visited index you visit ... et c.? If so, then you don't need the N.txt files. Use a recursive subroutine that outputs every filename to a single file, with each filename prefixed by a number. Roughly ...

set top_file=%@full[%1]
set index=0
echo %@format[6,%index] top_file > filelist.txt
set index=1
gosub process top_file
quit

:process [file]
do line in @file 
    echo %@format[6,%index] %line >> filelist.txt
    set index=%@inc[%index]
enddo
do line in @file
    gosub process %line
enddo
return

Later you can reverse sort listfile.txt (they'll sort by index) and get rid of the indices if you want. I made all the indices 6 characters wide; that might make it easier to discard them (say, with TPIPE) when you're done.

Another strategy ... find them all and rename each with a prefixing index

index.txt --> %N-index.txt
set N=%@inc[%N]

Later find all such (N-index.txt) files (with DIR /S) output to a file ... sort ... et c.

- Vince

 

[SteveW]

Vince,
Forgetting about reversing the order for a minute ... is that the order you want (later to be reversed)? The last dir path found in the last index.txt file should be first and the first dir path found in c:\index.txt should be last, so yes REVERSED.
... all the files mentioned in the first-visited index.txt listed, then all the files mentioned in the next-visited index you visit ... et c.? If so, then you don't need the N.txt files I used N to express that we do not know in advance how many index.txt files will be encountered.

Create a text file C:\index.txt that contains the following two lines:
C:\a\index.txt

C:\b\c\index.txt

Where C:\a\index.txt is another text file which contains the line:
C:\b\index.txt
C:\b\c\index.txt

And C:\b\c\index.txt and C:\b\index.txt are empty files.


If you start your program at C:\ it should first print:

C:\b\index.txt

C:\b\c\index.txt
in any order because they are empty followed by:

C:\a\index.txt

because its children have already been printed and finally:


C:\index


So ListFile.txt should contain 4 lines in this order:
C:\b\index.txt
C:\b\c\index.txt
C:\a\index.txt
C:\index

Do not assume any restrictions on the depth of the directory tree or number of entries in a given index.txt files.

 

[vefatica]

What you want is a bit bizarre ... not a normal way of traversing a tree. And I'm not going to spend much more time on it. Asuming the the actual existence of the index.txt files coincides with their contents, then this seems to do what you want (without looking inside the files). I hope it gives you some ideas.

:: BIZARRE.BTM
set maxdepth=1

:: find out how deep they go
global /q /i gosub finddepth

:: output the empty ones
dir /b /s /[s0,0] index.txt

:: output the non-empty ones going up the directory tree
do i=%maxdepth to 1 by -1
   global /q /i if exist index.txt .and. %@index[%@full[index.txt],\,0] == %i .and. %@filesize[index.txt] != 0 echo %@full[index.txt]
enddo
quit

:finddepth
iff exist index.txt then
   :: how many \'s
   set maxdepth=%@max[%maxdepth,%@index[%@full[index.txt],\,0]]
endiff
return

Here it is in action.

v:\> bizarre.btm

V:\b\index.txt
V:\b\c\index.txt
V:\a\index.txt
V:\index.txt

 

[vefatica]

P.S. What I posted won't work if the existence of index.txt files doesn't match their contents, or if the tree is not strictly downward oriented. In either of those cases ... good luck!

 

[SteveW]

Thanks Vince! I will try it out.

 

[vefatica]

This one does the same thing but is 20-25% faster.

:: BIZARRE.BTM

:: find out how deep they go
set maxdepth=1
global /q /i if exist index.txt set maxdepth=%@max[%maxdepth,%@index[%@full[index.txt],\,0]]

:: output the empty ones
dir /b /s /[s0,0] index.txt

:: output the non-empty ones going up the directory tree
do i=%maxdepth to 1 by -1
    global /q /i gosub output %i
enddo
quit

:output [level]
iff exist index.txt then
    iff %@index[%@full[index.txt],\,0] == %level then
        iff %@filesize[index.txt] != 0 then
            echo %@full[index.txt]
        endiff
    endiff
endiff
return

 

[vefatica]

I'm wondering if that could be made a little faster with a plugin internal variable ...

_DLEVEL = 0-based depth of %_CWD

... essentially, %@EVAL[%@INDEX[%_CWDS,\,0]-1] (but done in C, not counting the leading "\\" of network dirs).

Any comments?

 

[SteveW]

Vince,
I just installed Take Command on my laptop to test the DOS script problem.
Here is the procedure I have tried using Take Command debugger:

1. open TC 15.01
   
2. click the Edit/Debug button
3. paste BIZARRE.BTM code into edit window of IDE 15.01
4. Select Debug menu to Start program
5. Click the Step-over icon a few times until it TCC prompt window displays "quit"

Where does the program put the output data as in post 18?
V:\b\index.txt
V:\b\c\index.txt
V:\a\index.txt
V:\index.txt

Thanks!

 

[SteveW]

What I see in the TCC Prompt window is this:

set maxdepth=1
global /q /i if exist index.txt set maxdepth=%@max[%maxdepth,%@index[%@full[inde
x.txt],\,0]]
dir /b /s /[s0,0] index.txt
do i=%maxdepth to 1 by -1
global /q /i gosub output %i
iff exist index.txt then
return
quit

 

[Steve Fabian]

I'm wondering if that could be made a little faster with a plugin internal variable ...

_DLEVEL = 0-based depth of %_CWD

... essentially, %@EVAL[%@INDEX[%_CWDS,\,0]-1] (but done in C, not counting the leading "\\" of network dirs).

Any comments?

If you do it, please build both an internal variable %_dlevel for %_cwd, and also a variable function for arbitrary directory or file: %@dlevel[xxx] where xxx is a directory entry, either a directory or a file (with the usual path resolution, including such relative path indicators as .. or .... (1 or 3 levels up from %_cwd, resp.). Naturally the values of %@dlevel[] and %@dlevel[%_cwd] and %_dlevel are to be the same.

BTW, in your sample code, why did you use IF EXIST rather than IF ISFILE?

 

[Steve Fabian]

Where does the program put the output data as in post 18?
V:\b\index.txt
V:\b\c\index.txt
V:\a\index.txt
V:\index.txt

When the debugger is started, it typically is displayed on top of the right side of the TCC window; the STDOUT of the program it executes (displays from commands such as directories, outputs of the TEXT and ECHO commands, etc.) are displayed on the left side of the window, unless STDOUT itself is redirected.

 

[SteveW]

My laptop has this directory tree with four index files.

c:\index.txt (this file has two lines: c:\a\index.tx, c:\b\c\index.txt)
c:\a\index.txt (this file has two lines: c:\b\index.tx, c:\b\c\index.txt)
c:\b\index.txt (this file is empty)
c:\b\c\index.txt (this file is empty)

How does the BIZARRE.BTM program find these index.txt files? I didn't see any reference to the C:\ drive or the V:\ in the program.

 

[SteveW]

BIZARRE.BTM doesn't appear to read the first c:\index.txt file to get the path to the next index.txt file.......and so forth? Because an index.txt file could have any number of lines, and each line can be a directory path to any folder on the c:\ drive. So you would have to read each line of the file in order to find the next index.txt file. So the Problem Statement in post #6 has this requirement.

 

[vefatica]

My laptop has this directory tree with four index files.

c:\index.txt (this file has two lines: c:\a\index.tx, c:\b\c\index.txt)
c:\a\index.txt (this file has two lines: c:\b\index.tx, c:\b\c\index.txt)
c:\b\index.txt (this file is empty)
c:\b\c\index.txt (this file is empty)

How does the BIZARRE.BTM program find these index.txt files? I didn't see any reference to the C:\ drive or the V:\ in the program.

It starts in TCC's current directory and looks down the directory tree from there. My TCC current directory was v:\; yours, I suppose, is c:\. It finds the empty ones with the DIR command. It finds the others with"GLOBAL ... if exist index.txt ...".

 

[vefatica]

BIZARRE.BTM doesn't appear to read the first c:\index.txt file to get the path to the next index.txt file.......and so forth? Because an index.txt file could have any number of lines, and each line can be a directory path to any folder on the c:\ drive. So you would have to read each line of the file in order to find the next index.txt file. So the Problem Statement in post #6 has this requirement.

Do you mean c:\a\b\c\d\index.txt might have a line that points to c:\a\index.txt? If so, you are on your own. That's one god-awful system (in need of re-design) and I want nothing to do with it.