FOR command and regular expressions

[TLis]

Hi,

I have tried the following command:

FOR %x in ( ::tcc.+\.btm ) do echo %x

but it did not show the tcc*.btm files I expected to see, just the pattern itself. Obviously, my final pattern would be more complex - the above was just a test of the TCC abilities in processing files.

Am I doing something wrong, or the FOR command does not support regular expressions?

 

[TLis]

Partial success. The /W switch proved to be essential. It still puzzles me though why the following:

for /W %x in ( ::\d+f\.rar ) do echo %x​

does not work, but

for /W %x in ( ::[0-9]+f.rar ) do echo %x​

does ... I tried both Perl and Ruby regular expression syntax options. In the last case, unfortunately, the file 12fxrar (no extension) is also processed. It seems, that the pattern ::[0-9]+f[.]rar (correct, rejects 12fxrar) works, but it is rather awkward, why the \d placeholder nor the \. escape do not work. Any hints?

Update: This is even more weird after noticing that the command

dir ::\d+f\.rar​

displays the expected files, whereas

for /W %x in ( ::\d+f\.rar ) do echo %x​

does not

 

[TLis]

So, what? Is this a bug, that the FOR command does not fully support the regular expressions, or I am missing something obvious?

 

[rconn]

So, what? Is this a bug, that the FOR command does not fully support the regular expressions, or I am missing something obvious?

Not a bug, and you're missing something obvious. :)

The FOR command exists for compatibility with CMD, so it inherits all of CMD's bizarre and contradictory syntax. FOR arguments are not filenames, they're strings. Only if the string contains wildcard characters (* or ?) is it treated as a filename. Your regular expression doesn't contain any wildcard characters, so FOR simply treats it as a string to pass to ECHO.

However, FOR in TCC does have a workaround -- the /W option tells FOR that the arguments really are filenames. See the help for details.

(Or, you could use DO, which actually behaves in a sensible and understandable manner ...)

 

[TLis]

As you can see in my examples, I HAVE used the /W switch, but still the FOR loop could not handle my regular expressions. And it is not it did not handle them at all ... Simple regular expressions, without \d placeholders or \ escapes worked fine, but as soon as I used these features of regex, it failed to recognise the syntax. So it looks that the FOR loop does support some very basic regex syntax, but it has nothing to do with the standard as established in the options.

I will follow your advice and use the DO loop, but still I find the implementation of the FOR loop to be flawed. Shouldn't it either support the full syntax as defined by the options or not support regular expressions at all?

 

[vefatica]

About "/W" the help says "This is useful if you want to use regular expressions with FOR". That suggests that REs should work. But I can't get a fairly simple RE to work. The RE below is (perl) equivalent to "*.*"; no output is produced.

v:\> for /w %f in ( ::.*\..* ) echo %f
 
v:\>

Some others work, but as in TLis's examples, anything with a (perl) escape character (e.g., the common "\.") fails.

 

[vefatica]

About "/W" the help says "This is useful if you want to use regular expressions with FOR". That suggests that REs should work. But I can't get a fairly simple RE to work. The RE below is (perl) equivalent to "*.*"; no output is produced.

v:\> for /w %f in ( ::.*\..* ) echo %f
 
v:\>

Some others work, but as in TLis's examples, anything with a (perl) escape character (e.g., the common "\.") fails.

And DO is no better; again no output

v:\> do f in ::.*\..* (echo %f)
 
v:\>

 

[rconn]

There is only one regular expression parser in TCC; every command uses the same one.

 

[vefatica]

There is only one regular expression parser in TCC; every command uses the same one.

Why don't the examples given work with FOR and DO? The regex "::.*\..*" works with DIR.

 

[TLis]

My suspicion is that the parser gets confused, if it should consider the "\" character as the regex escape character or the path separator :-) Apparently it attempts to interpret it as the path separator character and not as the regex syntax metacharacter ...

The above may be true, but then indeed it cannot be explained, why DIR works, but FOR /W doesn't with the same regex pattern ...

 

[TLis]

Until the FOR and DO loops get fixed to support fully the regular expressions, I have found a workaround:

PDIR /(f) ::\d+f\.rar >%temp%\list.txt
FOR /W %x in (@%temp%\list.txt) echo %x

It works, since PDIR supports the regex syntax as expected, but it costs the creation of a temporary file ...

 

[Steve Fabian]

You could simplify it thus:

PDIR /(f) ::\d+f\.rar | FOR %x in (@CON) echo %x

This does not create a temporary file. Note that the /W option is not needed in FOR, because PDIR passed only files to the pipe.

 

[vefatica]

Or

You could simplify it thus:

PDIR /(f) ::\d+f\.rar | FOR %x in (@CON) echo %x

This does not create a temporary file. Note that the /W option is not needed in FOR, because PDIR passed only files to the pipe.

Yipes! How about

DO f IN /P DIR /B ::\d+f\.rar (ECHO %f)

 

[TLis]

It is good to see the creativity at works :) Thanks for these alternatives - they are much cleaner.

I sincerely hope, though, that the Author will put the fixing of support of regular expressions in the FOR and DO loops in his plans for future releases. I guess I am going to rely heavily on regular expressions in my scripts written for TCC, so I need them working correctly, at least as far as the Perl syntax is concerned.

Thanks in advance ;-)

 

[rconn]

Why don't the examples given work with FOR and DO? The regex "::.*\..*" works with DIR.

DIR passes a fully qualified filename; FOR doesn't.

 

[rconn]

As you can see in my examples, I HAVE used the /W switch, but still the FOR loop could not handle my regular expressions. And it is not it did not handle them at all ... Simple regular expressions, without \d placeholders or \ escapes worked fine, but as soon as I used these features of regex, it failed to recognise the syntax. So it looks that the FOR loop does support some very basic regex syntax, but it has nothing to do with the standard as established in the options.

I will follow your advice and use the DO loop, but still I find the implementation of the FOR loop to be flawed. Shouldn't it either support the full syntax as defined by the options or not support regular expressions at all?

Use the correct syntax (regular expressions should be quoted) and it should work fine (it works here):

for /w %f in ( "::.*\..*" ) echo %f

 

[TLis]

Use the correct syntax (regular expressions should be quoted) and it should work fine (it works here):

for /w %f in ( "::.*\..*" ) echo %f

Confirmed! Thanks a lot :-)

 

[TLis]

I thought that the issue has been resolved :-( See, however, the following output in the TCMD console window (TCC):

[D:\Tomek\Documents\Projects\tcmd]for /W %x in ("::\d+\.rar") echo %x
D:\Tomek\Documents\Projects\tcmd\001.rar
D:\Tomek\Documents\Projects\tcmd\001_001.rar
D:\Tomek\Documents\Projects\tcmd\001_002.rar
D:\Tomek\Documents\Projects\tcmd\002.rar
D:\Tomek\Documents\Projects\tcmd\002_001.rar
D:\Tomek\Documents\Projects\tcmd\002_002.rar
D:\Tomek\Documents\Projects\tcmd\012.rar

Why are the nnn_mmm.rar files listed?

Update: The same problem have the DIR / PDIR commands and the @FILES function. In options I have Perl syntax selected. Version used 14.03.53 x64 ...

Update 2: It seems, that adding the "^" and "$" metacharacters achieves again the required result:

[D:\Tomek\Documents\Projects\tcmd]for /W %x in ("::^\d+\.rar$") echo %x
D:\Tomek\Documents\Projects\tcmd\001.rar
D:\Tomek\Documents\Projects\tcmd\002.rar
D:\Tomek\Documents\Projects\tcmd\012.rar

So, it looks like the ::regex specification is a SEARCH pattern and not the MATCH pattern, i.e. looks for a match ANYWHERE in the file name. Sigh ...

 

[vefatica]

So, it looks like the ::regex specification is a SEARCH pattern and not the MATCH pattern, i.e. looks for a match ANYWHERE in the file name. Sigh ...

Why the sigh? You can turn a search pattern into a match pattern easily enough (as you pointed out). But you can't turn a match pattern into a search pattern. As it is we can have either.

 

[TLis]

Why the sigh? You can turn a search pattern into a match pattern easily enough (as you pointed out). But you can't turn a match pattern into a search pattern. As it is we can have either.

Vefatica,

I am just a new user of TCMD and being quite familiar with the regular expressions engine used in Python or .Net I simply tried a few features using TCMD and became stuck at a few fairly simple points, when following my "natural instincts" in this area :)

Workarounds have been found (e.g. double quoting the regex), but they soon proved to be not quite complete and some other cases I tried did not work as expected. Then another workaround ... I hope you can feel me becoming more and more annoyed in this process ;-) I felt more like a tester, not as an end-user ...

Eventually, I am glad there is a way to make regular expressions working, but it wasn't a smooth exercise. Are there any How-Tos or documentation that describe the use of regexes in TCMD in such a way, that someone like me could start using them without so much hassle? ;-) Even the examples in the help file do not suggest double-quoting the regexes as a rule (only when space characters are present)! The help file also does not mention clearly the fact, that a regex pattern does not specify the full name (as wildcards do, e.g. *ab*.txt), but any part of it! (hence the SEARCH not MATCH). This can be nasty to find out later. I think I was lucky to have discovered it, as my unwanted file names had digits followed by '.rar' after the underscore, so when testing I could easily see, that both 001.rar and 001_001.rar are returned. What if my script decided to remove the files found?

I strongly believe the documentation should be updated in this area, as otherwise, I completely appreciate the power offered by TCC enhanced facilities.