1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

move_all.btm

Discussion in 'Support' started by Charles G, Jan 27, 2014.

  1. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    I have the following snippet of code:

    Code:
     :DoFldr
        rem for each file in this fldr
        rem   compare extension to each s* variable abouve
        rem     if match is fouind = move file to folder named above - depending on type of file....
        rem     else move to C:\Z_NewAll\z_NewDown\
    
        echo ``
        echo Processing %_CWD
    
        for %fn in (*) (
          echos .
          IFF isfile "%fn" then
            set pTGT=%sUnkn%
            set cExt=%@upper[%@ext["%fn"]]
    
    The problem is that %fn can have chars that setdos /x..... would help process correctly, but I don't want to use setdos at all but want to iterate over all those badly named files. Is there a way to do this? At the very least I thought the IFF isfile "%fn" then .... ELSE ... ENDIFF would help trap those files....
     
  2. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Charles Dye's SafeChars.dll plug-in should be able to assist in processing your ill-named files.

    I cannot scroll below the line starting with set scExt. Is this intentional?

    Suggest you use do / enddo instead of multiline command group with FOR - there have been many reasons listed in the Forum. Also it is my guess that switch would be a a simpler method for selecting between multiple enumerated alternatives than iff/elseiff/elseiff/else.
     
  3. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    I said it was just a snippet - not full statements. But just for grins I'll post the full BTM here as an attachment.
     

    Attached Files:

  4. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    How would you suggest I use the safechars plugin?
     
  5. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Request help from its author... The issue about which I defer to Charles Dye is that while safechars provides a unique mapping for the special characters into "nonspecials", for the MOVE command ultimately you need the reverse translation as well, and I am not at this time prepared to do so.
     
  6. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    All I want to do is to leave the files that have special chars or that would if isfile "%fn" ... would generate an error.....

    Will contact the author shortly.... thanks S.F.
     
  7. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,307
    Likes Received:
    39
    Which characters are you having trouble with?

    Double-quoting the filename will help with some problem characters (spaces, ampersands.) You can deal with the escape character easily enough by remapping it to something weird, e.g. SETDOS /E0XFFF8. But if your filenames contain grave accents ("backquotes") or percent signs, you're pretty much out of luck. I don't think SafeChars will help with such filenames; sorry....

    I do have a plugin FixNames that looks for some of the most egregiously awful filenames (graves, percent signs, carets, and so on) and attempts to rename them. You don't get much control over the new filenames, though. In most cases, the problem characters are just replaced with underscores.
     
  8. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Charles, I guess the best way to find out the chars I am having trouple with is to do:

    [path] dir /a:-d /b /s /f * > outfile.lst

    remove the full path of each file in outfile.lst, leaving just the full name.

    then remove chars I know not to cause problems, namely, 0-9,A-F,a-f then the punctuation chars.

    upload the file ......

    Am I correct?
     
  9. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,307
    Likes Received:
    39
    Probably would be a good idea to OPTION //UNICODEOUTPUT=YES first. (NTFS uses UTF-16 internally; Windows allows you to put all kinds of awful things in filenames.)
     
  10. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Actually, PDIR is your friend here:
    [path]pdir /a:-d /s /(fn) > outfile.lst
    which gives you the whole list without path names.
    One could then take the whole file, and read it character by character (beware of Unicode vs. ASCII), e.g., using @filereadb or @bread, and mark those characters which occur in the file. The result will be which characters occur...
     
  11. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Would I need to turn Unicode on before the PDIR?

    So use a loop to @filereadb or @bread then write to a new file one char per line then sort and remove duplicates?
     
  12. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,307
    Likes Received:
    39
    Before, so any oddball characters aren't squished down to code page 437 (or whatever you use).
     
  13. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    and I guess Unicode needs to be on when I create the 2nd output file? Will create and send the file later tonioght - going out for dinnere now...
     
  14. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Using this short BTM to create the 2nd file, and UNICODE is ON

    Code:
    goto :here
    
    outfile_unicode.lst created with unicode ON and PDIR command earlier in message thread.
    
    :here
    set h=%@fileopen[outfile_Unicode.lst,r]
    set o=%@fileopen[outfile_out.lst,write]
    do forever
      set b=%@filereadb[%h,1]
      if "%b" EQ **EOF** LEAVE
      set z=%@filewrite[%o,%b]
      echos .
    enddo
    echo closing handle #%h: %@fileclose[%h]
    echo closing handle #%o: %@fileclose[%o]
    
     
  15. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    More like this:
    Code:
    :here
    setarray codes[65536]
    set h=%@fileopen[outfile_Unicode.lst,r]
    set n=%@fileseek[%h,0,2]
    do while %@fileseek[%h,0,1] LT %n
      set uc=%@filereadb[%h,2]
      set codes[%uc]=1
    enddo
    echo closing handle #%h: %@fileclose[%h]
    
    Now all Unicodes used are set to 1, unused Unicodes are undefined. The next step is to find those Unicodes which are not "nice" to have in filenames...
     
  16. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    The next step would be to go thru codes[] and write out the index for entries that are set to TRUE?

    then from there see what are not good items for filenames?

    Thank you for your help!!!
     
  17. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
  18. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Code:
    setlocal
    
      timer on
      set cOutFile=c:\z_Unicode\Out\outfile_unicode.lst
    
      goto :here
    
      set Unicode=%@option[UnicodeOutput]
      option //UnicodeOutput=Yes
      pdir /a:-d /s /(fn) c:\Z_NewAll\* c:\Z_UserFiles\* >! %cOutFile
      option //UnicodeOutput=%Unicode
    
      :here
      unsetarray /q codes
        setarray    codes[65536]
      for /l %nIndex in (0,1,65535) set codes[%nIndex]=0
      set h=%@fileopen[%cOutFile,r]
      echo File handle: %h
      set n=%@fileseek[%h,0,2]
      echo File Size: %@comma[%n]
      set nAt=0
      set n=%@fileseek[%h,0,0]
      do while %@fileseek[%h,0,1] LT %n
        set uc=%@filereadb[%h,2]
        set nAt=%@inc[%nAt]
        set nAt=%@inc[%nAt]
        set codes[%uc]=1
      enddo
      echo closing handle #%h: %@fileclose[%h]
      echo Reads: %@comma[%nAt]
    
      rem for /l %nIndex in (0,1,65535] if codes[%nIndex] EQ 1 echo %@format[5,%nIndex]
    
      timer off
    
    endlocal
    
    Gives:

    Code:
    [C:\Program Files\JPSoft\TCMD16x64]c:\z_Unicode\WriteFile.btm
    Timer 1 on: 16:34:01
    File handle: 520
    File Size: 6,988,998
    closing handle #520: 0
    Reads: 0
    Timer 1 off: 16:34:04  Elapsed: 0:00:02.96
    I don't know about fileread or fileseek so why is it not going inside the DO WHILE .... ENDDO
     
  19. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    A comment: it is not necessary to initialize the array. The value of each element is either 1 or it is not.
    I made a mistake which you caught - I should just have used @filesize. This would be the correct logic:

    echo File handle: %h
    set n=%@filesize[%cOutFile]
    echo File Size: %@comma[%n]
    set nAt=0
    do while %@fileseek[%h,0,1] LT %n
     
  20. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Steve,

    %uc is 2 numbers, yet codes array is one column. What needs to happen. It gives correct output except for the set codes[%uc]=1
     
  21. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    I ended up:

    echo [%uc] >>! aOutputFile
    sort aOutputFile /output aOutputFile.out
    remove duplicate lines in aOutputFile.out

    Now what do I need to do?
     

    Attached Files:

  22. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Apparently the data is little-endian, as is typical of the Intel-like architectures. The sort itself is based on the decimal representation of the low order digit being more significant than the high order. One of your alternatives is to just postprocess the listing and sort them in true numeric order; the other is to rebuild the list as numeric.

    I am just writing a postprocessing program for the data you collected to display it in a more reasonable order.
     
  23. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    OK, I massaged your data with the following BTM:
    Code:
    @echo off
    setlocal
    do l in @c:\work\cg\codes.txt
      set nobrkt=%@trim[%@strip[[],%l]]
      set high=%@word[1,%nobrkt]
      set low=%@word[0,%nobrkt]
      set num=%@eval[ 256 * %high + %low ]
      echo %@fz3[%high] %@fz3[%low] %@f6[%num] %@fz4[%@convert[10,16,%num]]   %l
    enddo
    
    (fz3=%@format[03,%&] and f6=%@format[6,%&])

    The BTM puts in leading zeros as needed, puts the high order byte before the low order byte, and adds both decimal and hexadecimal columns. Probably the hexadecimal column will be the most meaningful.
     

    Attached Files:

  24. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Steve,

    What is next please? Remember this work is just to see what chars in the orginal list TCMD/TCC has problems with - so that the file names can be changed...
     
  25. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,307
    Likes Received:
    39
    Perhaps you could run FIXNAMES /N and see which files that dislikes? The plugin is here.
     
  26. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Here is the redirected output and error files for Fixnames /N /S * in both directories....
     

    Attached Files:

  27. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,307
    Likes Received:
    39
    Most of your problem filenames seem to have carets. Easy enough to deal with by temporarily changing your escape character, e.g. SETDOS /E0XFFFE. One contains backquotes, which are harder to deal with. (Try an external like FIXNAMES, or run REM in a transient CMD.EXE shell?)

    And one just contains some 0xFF characters -- not illegal and not really a problem at all, just potentially confusing if you happen to use one of those code pages where it's a non-breaking space.
     
  28. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Code:
    [C:\Z_UserFiles]mode
    
    Status for device CON:
    ----------------------
        Lines:          5000
        Columns:        188
        Keyboard rate:  31
        Keyboard delay: 1
        Code page:      437
    
    
    [C:\Z_UserFiles]
     
  29. Charles G

    Joined:
    Apr 2, 2011
    Messages:
    1,021
    Likes Received:
    0
    Code:
    setlocal
    
      timer on
    
      set cOutFile=c:\z_Unicode\Out\outfile_unicode.txt
    
      set cCodesFile=c:\z_Unicode\Out\outfile_codes.txt
    
      set cSortedFile=c:\z_Unicode\Out\outfile_Sorted.txt
    
      set cSortedUnique=c:\z_Unicode\Out\outfile_Sorted.txt.unq
    
      set cFinalFile=c:\z_Unicode\Out\FinalFile.txt
    
      del /q /n:e /w %cOutFile %cCodesFile %cSortedFile %cFinalFile
    
      echo Making PDIR list....
    
      set Unicode=%@option[UnicodeOutput]
      option //UnicodeOutput=Yes
      pdir /a:-d /s /(fn) c:\Z_NewAll\* c:\Z_UserFiles\* >! %cOutFile
      option //UnicodeOutput=%Unicode
    
      :here
    
      echo Reading Unicode chars from list file...
    
      set h=%@fileopen[%cOutFile,r]
      echo File handle: %h
      set nSize=%@filesize[%cOutFile]
      echo File Size: %@comma[%nSize]
      set nAt=0
      do while %@fileseek[%h,0,1] LT %nSize
        set uc=%@filereadb[%h,2]
        set nAt=%@inc[%nAt]
        set nAt=%@inc[%nAt]
        echo [%uc] >>! %cCodesFile
      enddo
      echo closing handle #%h: %@fileclose[%h]
      echo Reads: %@comma[%nAt]
    
      del /q /n:e /w "%cOutFile"
    
      rem for /l %nIndex in (0,1,65535] if codes[%nIndex] EQ 1 echo %@format[5,%nIndex]
    
      rem =================================================================================
    
      echo SORTing Unicode codes file
    
      sort %cCodesFile /Output %cSortedFile
    
      del /q /n:e /w "%cCodesFile"
    
      echo Loading MEW32
    
      rem Next line creates "%cSortedUnique"
      start /pgm mew32 /snSorted "%cSortedfile"
    
      del /q /n:e /w "%cSortedFile"
    
      rem =================================================================================
    
      :Here1
    
      echo making Final File.....
    
      do l in @%cSortedUnique
        set nobrkt=%@trim[%@strip[[],%l]]
        set high=%@word[1,%nobrkt]
        set low=%@word[0,%nobrkt]
        set num=%@eval[ 256 * %high + %low ]
        echo %@format[03,%high] %@format[03,%low] %@format[6,%num] %@format[04,%@convert[10,16,%num]]   %l >>! %cFinalFile
      enddo
    
      del /q /n:e /w "%cSortedUnique"
    
      echo Final file: %cFinalFile
    
      timer off
    
    endlocal
    
    gives the attached results.

    How can I see the actual chars in the %cFinalFile?
     

    Attached Files:

  30. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    Use the @char[] function for each entry. You can modify my little program (after :Here1) by inserting "%@char[%num]" before the %l.

    The program attached does the whole job, but has not been tested.
     

    Attached Files:

Share This Page