1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

FOLDERMONITOR missing events??

Discussion in 'Support' started by Steve Pitts, Jun 11, 2011.

  1. Steve Pitts

    Joined:
    Jul 7, 2008
    Messages:
    159
    Likes Received:
    0
    I've been messing around with FOLDERMONITOR (prompted by Jang's thread about his problems with it), trying to understand how I might make use of it for handling the synchronisation of folders between the local HD and the NAS, and I came across an anomalous situation that I suspect might be a bug (sorry, undesirable feature).

    The sequence of activities that triggered the output below was
    • set up duff foldermonitor (the first line of output)
    • clear that and set it up properly
    • change name of mp3stick.not folder using Windows Explorer
    • rename that folder twice more using an elevated command prompt
    • edit meminfo_full_dump.out using Kedit for Windows and immediately save it (which you can see produces a .BAK version as well as updating the original)
    • create empty_example.txt using an ECHO from the elevated prompt
    • edit that empty_example.txt using Kedit again, add some text and then save. This created a backup and updated the original but FOLDERMONITOR only saw the first of those actions
    • delete both empty_example.txt and the .bak version from Windows Explorer but FOLDERMONITOR only registered one of the two deletions

    Code:
    I:\websites\Badgers\new>A= C= N= 1= 2=
                            foldermonitor /c
    
    I:\websites\Badgers\new>foldermonitor /s d:\temp\ created deleted modified renamed forever echo A=%%_folderaction C=%%_foldercount N=%%_foldername 1=%
    %_folderfile1 2=%%_folderfile2
    
    I:\websites\Badgers\new>foldermonitor
    d:\temp\   created deleted modified renamed   (0/FOREVER)   echo A=%_folderaction C=%_foldercount N=%_foldername 1=%_folderfile1 2=%_folderfile2
    
    I:\websites\Badgers\new>A=RENAMED C=1 N=d:\temp\ 1=mp3stick.not 2=mp3stick
                            elevate
    
    I:\websites\Badgers\new>foldermonitor
    d:\temp\   created deleted modified renamed   (1/FOREVER)   echo A=%_folderaction C=%_foldercount N=%_foldername 1=%_folderfile1 2=%_folderfile2
    
    I:\websites\Badgers\new>A=RENAMED C=2 N=d:\temp\ 1=mp3stick 2=mp3sticky
    A=RENAMED C=3 N=d:\temp\ 1=mp3sticky 2=mp3stick
    A=RENAMED C=4 N=d:\temp\ 1=meminfo_full_dump.out 2=meminfo_full_dump.bak
    A=MODIFIED C=5 N=d:\temp\ 1=meminfo_full_dump.out 2=
    A=CREATED C=6 N=d:\temp\ 1=empty_example.txt 2=
    A=RENAMED C=7 N=d:\temp\ 1=empty_example.txt 2=empty_example.bak
    A=DELETED C=8 N=d:\temp\ 1=empty_example.txt 2=
                            foldermonitor
    d:\temp\   created deleted modified renamed   (8/FOREVER)   echo A=%_folderaction C=%_foldercount N=%_foldername 1=%_folderfile1 2=%_folderfile2
    
    I:\websites\Badgers\new>
    It might be worth noting that the meminfo file is over 180MB in size, whereas the empty text file started as 1 byte and was only a dozen or so when I saved it, so I wonder whether there is a timing issue here with events happening too close together??

    I know I'm one build behind the curve and will attempt to re-test this later, but for now I have a cricket match to go and umpire :)

    TCC 12.10.66 x64 Windows 7 [Version 6.1.7601]
    TCC Build 66 Windows 7 Build 7601 Service Pack 1
     
  2. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,855
    Likes Received:
    83
    WAD -- FOLDERMONITOR (like all of the other monitoring commands) does not queue multiple rapid events. All of the monitoring commands are handled by a thread that wakes up every 200 ms and checks to see if anything has been triggered; if so, it processes those commands and then goes back to sleep.

    This was chosen as the most reasonable compromise between handling events quickly while not overloading the rest of the system.

    In your example, although FOLDERMONITOR is seeing all of the events, it's only notifying you about the most recent ones when you fire them off rapidly. IMO that's acceptable, given that in the real world you'd never expect to be able to handle potentially thousands of FOLDERMONITOR notifications per second. (Nor could you expect to actually DO anything with every notification if you did get it.)
     
  3. Steve Pitts

    Joined:
    Jul 7, 2008
    Messages:
    159
    Likes Received:
    0
    No problem with that decision, although I'm obviously paying the price for having a beast of a machine and therefore being much more likely to hit this situation, but perhaps the behaviour should be documented in the help for each of the monitoring commands??
     
  4. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,941
    Likes Received:
    30
    Steve, I revived an old plugin which provides WATCHDIR. When I build that project, WATCHDIR logs 76 events (compared to FOLDERMONITOR's 8, I don't think it misses any). I've got to do some work on it, but I think I can make one available this evening. Are you interested? So far I have only built it for 32-bit Windows. Do you need x64? Here's it's log file for the build. It can also output these lines to the console.

    1 20110613123003.470 DELETE later.obj
    2 20110613123003.486 DELETE sysutil.obj
    3 20110613123003.486 DELETE utility.obj
    4 20110613123003.486 DELETE watchdir.obj
    5 20110613123003.486 DELETE watchwin.obj
    6 20110613123003.486 DELETE 4threads.dll
    7 20110613123003.486 DELETE vc90.pdb
    8 20110613123003.486 DELETE vc90.idb
    9 20110613123003.486 DELETE 4threads.lib
    10 20110613123003.486 DELETE 4threads.exp
    11 20110613123003.486 DELETE 4threads.dll.intermediate.manifest
    12 20110613123003.486 DELETE mt.dep
    13 20110613123003.486 MODIFY BuildLog.htm
    14 20110613123003.486 CREATE RSP00001922885200.rsp
    15 20110613123003.486 MODIFY RSP00001922885200.rsp
    16 20110613123003.548 CREATE vc90.pdb
    17 20110613123003.548 MODIFY vc90.pdb
    18 20110613123003.548 CREATE vc90.idb
    19 20110613123003.548 MODIFY vc90.idb
    20 20110613123004.502 MODIFY vc90.idb
    21 20110613123004.502 MODIFY vc90.pdb
    22 20110613123005.673 MODIFY vc90.pdb
    23 20110613123005.689 MODIFY vc90.idb
    24 20110613123005.689 CREATE later.obj
    25 20110613123005.705 MODIFY later.obj
    26 20110613123005.705 CREATE sysutil.obj
    27 20110613123005.720 MODIFY sysutil.obj
    28 20110613123005.720 CREATE utility.obj
    29 20110613123005.720 MODIFY utility.obj
    30 20110613123005.720 CREATE watchdir.obj
    31 20110613123005.861 MODIFY watchdir.obj
    32 20110613123005.861 CREATE watchwin.obj
    33 20110613123005.877 MODIFY watchwin.obj
    34 20110613123005.892 CREATE RSP00001A22885200.rsp
    35 20110613123005.892 MODIFY RSP00001A22885200.rsp
    36 20110613123005.908 CREATE 4threads.dll
    37 20110613123005.908 MODIFY 4threads.dll
    38 20110613123005.955 CREATE 4threads.lib
    39 20110613123005.955 MODIFY 4threads.lib
    40 20110613123005.955 CREATE 4threads.exp
    41 20110613123005.955 MODIFY 4threads.exp
    42 20110613123005.955 MODIFY 4threads.exp
    43 20110613123005.955 MODIFY 4threads.exp
    44 20110613123005.955 MODIFY 4threads.lib
    45 20110613123005.955 MODIFY 4threads.lib
    46 20110613123005.970 CREATE 4threads.dll.intermediate.manifest
    47 20110613123005.970 MODIFY 4threads.dll.intermediate.manifest
    48 20110613123005.970 MODIFY 4threads.dll.intermediate.manifest
    49 20110613123005.970 MODIFY 4threads.dll.intermediate.manifest
    50 20110613123005.970 MODIFY 4threads.dll
    51 20110613123005.970 MODIFY 4threads.dll
    52 20110613123005.970 CREATE RSP00001B22885200.rsp
    53 20110613123005.970 MODIFY RSP00001B22885200.rsp
    54 20110613123006.002 CREATE RCX33A4.tmp
    55 20110613123006.002 MODIFY RCX33A4.tmp
    56 20110613123006.002 MODIFY RCX33A4.tmp
    57 20110613123006.002 MODIFY RCX33A4.tmp
    58 20110613123006.002 MODIFY RCX33A4.tmp
    59 20110613123006.002 MODIFY RCX33A4.tmp
    60 20110613123006.002 MODIFY RCX33A4.tmp
    61 20110613123006.002 DELETE 4threads.dll
    62 20110613123006.002 OLDNAME RCX33A4.tmp
    63 20110613123006.002 NEWNAME 4threads.dll
    64 20110613123006.002 MODIFY 4threads.dll
    65 20110613123006.002 MODIFY 4threads.dll
    66 20110613123006.002 MODIFY 4threads.dll
    67 20110613123006.002 MODIFY 4threads.dll
    68 20110613123006.002 CREATE BAT00001C22885200.bat
    69 20110613123006.002 MODIFY BAT00001C22885200.bat
    70 20110613123006.017 CREATE mt.dep
    71 20110613123006.017 MODIFY mt.dep
    72 20110613123006.033 DELETE RSP00001922885200.rsp
    73 20110613123006.033 DELETE RSP00001A22885200.rsp
    74 20110613123006.033 DELETE RSP00001B22885200.rsp
    75 20110613123006.033 DELETE BAT00001C22885200.bat
    76 20110613123006.033 MODIFY BuildLog.htm
    **EOF**
     
  5. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    vefatica wrote:
    | Steve, I revived an old plugin which provides WATCHDIR. When I build
    | that project, WATCHDIR logs 76 events (compared to FOLDERMONITOR's 8,
    | I don't think it misses any). I've got to do some work on it, but I
    | think I can make one available this evening. Are you interested?

    Yes please.
    --
    Steve
     
  6. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,941
    Likes Received:
    30
    I uploaded ftp://lucky.syr.edu/4plugins/4threads.zip and ftp://lucky.syr.edu/4plugins/x64/4threads64.zip. They contain WATCHDIR and WATCHWIN (both are old, WATCHDIR got some recent attention, WATCHWIN no attention in years.

    WATCHDIR tips:

    Use a log file (that's the default) and if it's in the directory being watched, exclude it from the watch with the "/X" option. The log file is opened with FILE_SHARE_READ so you should be able to read it while watching; apps that insist on opening it for writing will choke.

    Console output (/O) is messy, like FOLDERMONITOR's.

    I have no idea how well TCC can keep up with rapid changes if "/E command" is specified.

    To watch the build I mentioned earlier, I used the command

    Code:
    g:\projects\4threads\release> watchdir id1 . /* /i * /x id1.log
    . Watch the current dir
    id1 a name for the watch object (for /List or /Kill)
    /* use all filters (look up "ReadDirectoryChanges" at MS)
    /i * include all files (except ...)
    /x id1.log exclude the (default-named) logfile

    Here's the help message.

    Code:
    ATCHDIR id path filter(s) [/S] [/N n] [/I spec(s)] [/X spec(s)] [/R]
    /B] [/O] [/L file] [/E "command"]
    
       id          string ID, case-insensitive, 15 char limit
                   also names variable to get last action, time and filename
       path        directory to watch
       /S          watch subdirectories
       /N n        stop after n changes; n < 65536 (0 = unspec'd = no limit)
       /I specs    include names matching any spec
       /X specs    exclude names matching any spec
                   specs = spec[;spec]... wildcards cause wildcard comparison of
                   target name only, else case-insensitive substring search on
                   \path\name. If /I and /X are both spec'd the one spec'd second
                   is seen only as exceptions to the one spec'd first
       /R          use case-insensitive regular expression searches (on
                   \path\name) instead of those described above (/I and /X)
       /B          block: wait for changes
       /O          console output (can be messy without /B)
       /L file     log file; default: .\[watch_id]<id>.log - use "/L NUL" for no log
       /E command  quote entire command; use \" for interior quotes
    
     Filters       (at least one)
    
       /F          filename
       /D          dirname
       /Z          size
       /W          write time
       /C          create time
       /A          access time
       /*          all of the above
    
    ATCHDIR /K [spec] - kill watch(es); wildcard id spec; default: *
    
    ATCHDIR /P [spec] - pause/resume watch(es), a toggle; ignore events
    
    ATCHDIR /L [spec] - list watch(es); spec as above
    </id>
     
  7. Steve Pitts

    Joined:
    Jul 7, 2008
    Messages:
    159
    Likes Received:
    0
    I would certainly be willing to give it a look, although I see Mr. Fabian got in before me so you've gone ahead with it anyway :) All I'm looking for is something that monitors each of half a dozen directories (and all of their subdirectories) and records changes made to an activity file. The intention is to post-process that file to update related directories on a NAS and thus keep those handful of directories in sync across three different PCs (I've been trying to find a tool that makes this easy but all seem to founder on what to do when a file gets updated on two different source machines between syncs).

    I am a virgin when it comes to plugins (despite being an active 4whatever user for nearly two decades) but I'm sure it is easy enough to set up.

    As things stand I have two 32 bit Windows XP machines running v7 of 4NT/TC (will your plugin work with v7 under XP??) and the beast running 64 bit Windows 7. If I could have made FOLDERMONITOR work then I would have upgraded the two XP machines to v12 of TC, and I guess if your plugin gives me the ability that I'm looking for but only runs in v12 then that would still be an option.
     
  8. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,941
    Likes Received:
    30
    On Tue, 14 Jun 2011 04:15:37 -0400, Steve Pitts <> wrote:

    |---Quote (Originally by vefatica)---
    |So far I have only built it for 32-bit Windows. Do you need x64?
    |---End Quote---
    |As things stand I have two 32 bit Windows XP machines running v7 of 4NT/TC (will your plugin work with v7 under XP??) and the beast running 64 bit Windows 7. If I could have made FOLDERMONITOR work then I would have upgraded the two XP machines to v12 of TC, and I guess if your plugin gives me the ability that I'm looking for but only runs in v12 then that would still be an option.

    I doubt that it will work with 4NT v7. Since v7 there have been changes to the
    basic 4NT/TCC interaction and to a few of the functions provided by TakeCmd.dll
    for plugins to use. XP is no problem; it's the platform on which I develop
    plugins.
     

Share This Page