FOR command and regular expressions

Dec 9, 2012
20
0
Hi,

I have tried the following command:

FOR %x in ( ::tcc.+\.btm ) do echo %x

but it did not show the tcc*.btm files I expected to see, just the pattern itself. Obviously, my final pattern would be more complex - the above was just a test of the TCC abilities in processing files.

Am I doing something wrong, or the FOR command does not support regular expressions?
 
Dec 9, 2012
20
0
Partial success. The /W switch proved to be essential. It still puzzles me though why the following:
for /W %x in ( ::\d+f\.rar ) do echo %x​
does not work, but
for /W %x in ( ::[0-9]+f.rar ) do echo %x​
does ... I tried both Perl and Ruby regular expression syntax options. In the last case, unfortunately, the file 12fxrar (no extension) is also processed. It seems, that the pattern ::[0-9]+f[.]rar (correct, rejects 12fxrar) works, but it is rather awkward, why the \d placeholder nor the \. escape do not work. Any hints?

Update: This is even more weird after noticing that the command
dir ::\d+f\.rar​
displays the expected files, whereas
for /W %x in ( ::\d+f\.rar ) do echo %x​
does not
 
Dec 9, 2012
20
0
So, what? Is this a bug, that the FOR command does not fully support the regular expressions, or I am missing something obvious?
 

rconn

Administrator
Staff member
May 14, 2008
12,316
148
So, what? Is this a bug, that the FOR command does not fully support the regular expressions, or I am missing something obvious?

Not a bug, and you're missing something obvious. :)

The FOR command exists for compatibility with CMD, so it inherits all of CMD's bizarre and contradictory syntax. FOR arguments are not filenames, they're strings. Only if the string contains wildcard characters (* or ?) is it treated as a filename. Your regular expression doesn't contain any wildcard characters, so FOR simply treats it as a string to pass to ECHO.

However, FOR in TCC does have a workaround -- the /W option tells FOR that the arguments really are filenames. See the help for details.

(Or, you could use DO, which actually behaves in a sensible and understandable manner ...)
 
Dec 9, 2012
20
0
As you can see in my examples, I HAVE used the /W switch, but still the FOR loop could not handle my regular expressions. And it is not it did not handle them at all ... Simple regular expressions, without \d placeholders or \ escapes worked fine, but as soon as I used these features of regex, it failed to recognise the syntax. So it looks that the FOR loop does support some very basic regex syntax, but it has nothing to do with the standard as established in the options.

I will follow your advice and use the DO loop, but still I find the implementation of the FOR loop to be flawed. Shouldn't it either support the full syntax as defined by the options or not support regular expressions at all?
 
May 20, 2008
11,288
95
Syracuse, NY, USA
About "/W" the help says "This is useful if you want to use regular expressions with FOR". That suggests that REs should work. But I can't get a fairly simple RE to work. The RE below is (perl) equivalent to "*.*"; no output is produced.
Code:
v:\> for /w %f in ( ::.*\..* ) echo %f
 
v:\>
Some others work, but as in TLis's examples, anything with a (perl) escape character (e.g., the common "\.") fails.
 
May 20, 2008
11,288
95
Syracuse, NY, USA
About "/W" the help says "This is useful if you want to use regular expressions with FOR". That suggests that REs should work. But I can't get a fairly simple RE to work. The RE below is (perl) equivalent to "*.*"; no output is produced.
Code:
v:\> for /w %f in ( ::.*\..* ) echo %f
 
v:\>
Some others work, but as in TLis's examples, anything with a (perl) escape character (e.g., the common "\.") fails.
And DO is no better; again no output
Code:
v:\> do f in ::.*\..* (echo %f)
 
v:\>
 
Dec 9, 2012
20
0
My suspicion is that the parser gets confused, if it should consider the "\" character as the regex escape character or the path separator :-) Apparently it attempts to interpret it as the path separator character and not as the regex syntax metacharacter ...

The above may be true, but then indeed it cannot be explained, why DIR works, but FOR /W doesn't with the same regex pattern ...
 
Dec 9, 2012
20
0
Until the FOR and DO loops get fixed to support fully the regular expressions, I have found a workaround:

PDIR /(f) ::\d+f\.rar >%temp%\list.txt
FOR /W %x in (@%temp%\list.txt) echo %x

It works, since PDIR supports the regex syntax as expected, but it costs the creation of a temporary file ...
 
May 20, 2008
3,515
4
Elkridge, MD, USA
You could simplify it thus:
Code:
PDIR /(f) ::\d+f\.rar | FOR %x in (@CON) echo %x

This does not create a temporary file. Note that the /W option is not needed in FOR, because PDIR passed only files to the pipe.
 
May 20, 2008
11,288
95
Syracuse, NY, USA
Or
You could simplify it thus:
Code:
PDIR /(f) ::\d+f\.rar | FOR %x in (@CON) echo %x

This does not create a temporary file. Note that the /W option is not needed in FOR, because PDIR passed only files to the pipe.
Yipes! How about
Code:
DO f IN /P DIR /B ::\d+f\.rar (ECHO %f)
 
Dec 9, 2012
20
0
It is good to see the creativity at works :) Thanks for these alternatives - they are much cleaner.

I sincerely hope, though, that the Author will put the fixing of support of regular expressions in the FOR and DO loops in his plans for future releases. I guess I am going to rely heavily on regular expressions in my scripts written for TCC, so I need them working correctly, at least as far as the Perl syntax is concerned.

Thanks in advance ;-)
 

rconn

Administrator
Staff member
May 14, 2008
12,316
148
As you can see in my examples, I HAVE used the /W switch, but still the FOR loop could not handle my regular expressions. And it is not it did not handle them at all ... Simple regular expressions, without \d placeholders or \ escapes worked fine, but as soon as I used these features of regex, it failed to recognise the syntax. So it looks that the FOR loop does support some very basic regex syntax, but it has nothing to do with the standard as established in the options.

I will follow your advice and use the DO loop, but still I find the implementation of the FOR loop to be flawed. Shouldn't it either support the full syntax as defined by the options or not support regular expressions at all?

Use the correct syntax (regular expressions should be quoted) and it should work fine (it works here):

for /w %f in ( "::.*\..*" ) echo %f
 
Dec 9, 2012
20
0
I thought that the issue has been resolved :-( See, however, the following output in the TCMD console window (TCC):

[D:\Tomek\Documents\Projects\tcmd]for /W %x in ("::\d+\.rar") echo %x
D:\Tomek\Documents\Projects\tcmd\001.rar
D:\Tomek\Documents\Projects\tcmd\001_001.rar
D:\Tomek\Documents\Projects\tcmd\001_002.rar
D:\Tomek\Documents\Projects\tcmd\002.rar
D:\Tomek\Documents\Projects\tcmd\002_001.rar
D:\Tomek\Documents\Projects\tcmd\002_002.rar
D:\Tomek\Documents\Projects\tcmd\012.rar

Why are the nnn_mmm.rar files listed?

Update: The same problem have the DIR / PDIR commands and the @FILES function. In options I have Perl syntax selected. Version used 14.03.53 x64 ...

Update 2: It seems, that adding the "^" and "$" metacharacters achieves again the required result:

[D:\Tomek\Documents\Projects\tcmd]for /W %x in ("::^\d+\.rar$") echo %x
D:\Tomek\Documents\Projects\tcmd\001.rar
D:\Tomek\Documents\Projects\tcmd\002.rar
D:\Tomek\Documents\Projects\tcmd\012.rar

So, it looks like the ::regex specification is a SEARCH pattern and not the MATCH pattern, i.e. looks for a match ANYWHERE in the file name. Sigh ...
 
May 20, 2008
11,288
95
Syracuse, NY, USA
So, it looks like the ::regex specification is a SEARCH pattern and not the MATCH pattern, i.e. looks for a match ANYWHERE in the file name. Sigh ...
Why the sigh? You can turn a search pattern into a match pattern easily enough (as you pointed out). But you can't turn a match pattern into a search pattern. As it is we can have either.
 
Dec 9, 2012
20
0
Why the sigh? You can turn a search pattern into a match pattern easily enough (as you pointed out). But you can't turn a match pattern into a search pattern. As it is we can have either.
Vefatica,

I am just a new user of TCMD and being quite familiar with the regular expressions engine used in Python or .Net I simply tried a few features using TCMD and became stuck at a few fairly simple points, when following my "natural instincts" in this area :)

Workarounds have been found (e.g. double quoting the regex), but they soon proved to be not quite complete and some other cases I tried did not work as expected. Then another workaround ... I hope you can feel me becoming more and more annoyed in this process ;-) I felt more like a tester, not as an end-user ...

Eventually, I am glad there is a way to make regular expressions working, but it wasn't a smooth exercise. Are there any How-Tos or documentation that describe the use of regexes in TCMD in such a way, that someone like me could start using them without so much hassle? ;-) Even the examples in the help file do not suggest double-quoting the regexes as a rule (only when space characters are present)! The help file also does not mention clearly the fact, that a regex pattern does not specify the full name (as wildcards do, e.g. *ab*.txt), but any part of it! (hence the SEARCH not MATCH). This can be nasty to find out later. I think I was lucky to have discovered it, as my unwanted file names had digits followed by '.rar' after the underscore, so when testing I could easily see, that both 001.rar and 001_001.rar are returned. What if my script decided to remove the files found?

I strongly believe the documentation should be updated in this area, as otherwise, I completely appreciate the power offered by TCC enhanced facilities.
 
Similar threads
Thread starter Title Forum Replies Date
T VIEW only works from command line Support 7
rconn News Take Command / TCC / CMDebug / TCC-RT v28 Released Support 0
T Take Command 28: Lua reports version 5.4.0 (5.4.3 expected) Support 0
Jay Sage WAD Possible Bug With OPTION Command With @FILE Support 5
rconn News Take Command / TCC / CMDebug / TCC-RT 27.01.24 uploaded Support 0
J Take command does not start on Windows 10 Support 3
vefatica `Back quotes` - command line vs. batch file Support 5
Jay Sage WAD Failure of Alias Command with /Z Option Support 14
Phileosophos Is there any way to open the Take Command options dialog from the command line? Support 7
Phileosophos Command-line Editing Shortkeys That Fail Support 6
fpefpe How to? free (internal) command Support 0
rconn News Take Command / TCC / CMDebug / TCC-RT 27.01.23 uploaded Support 0
T How to? retain command history across reboot? Support 6
rconn News Take Command / TCC / CMDebug / TCC-RT v27.01 Build 22 Uploaded Support 0
Jay Sage Bug With Flow Control and/or Command Grouping Support 2
GermanDirk How to? EVENTLOG Command with simple User Rights not functional Support 4
Jay Sage Take Command Is Too Fast! Support 4
Jay Sage Comparison of IF with Command Group and IFF Support 6
D Open Windows 10 Photos app from command line Support 11
rconn News Take Command / TCC / CMDebug / TCC-RT 27.0.21 uploaded Support 0
rconn News Take Command / TCC / CMDebug / TCC-RT 27.0.20 uploaded Support 0
U Can not get/start Take Command gui Support 1
rconn News Take Command / TCC / CMDebug / TCC-RT 27.0.19 Support 0
S Increasing Font Point Size in Command Tabs Support 2
mfarah Fixed The DATE command produces the wrong error message. Support 1
vefatica Command name mix-up Support 3
FreezerBurnt "Missing SETLOCAL" when initializing Visual Studio Command Prompt. Support 9
E How to? Closing desktop created by DESKTOP command Support 10
rconn News Take Command / TCC / CMDebug / TCC-RT v27.0.18 Support 0
K Take Command fox XP? Support 7
Joe Caverly Documentation Replace internal command with LIBRARY command Support 0
fpefpe How to? command echo in title line/bar Support 2
U Command help on file names Support 5
vefatica Command line DO with no closing parenthesis? Support 9
vefatica Command history search is off Support 7
H How to? How can I say "if the command separator is not ~ then setdos /C~" Support 8
N Command execution slows down in TCC Prompt after a while Support 6
R Long commands retrieved from command history have blank line when wrapped Support 20
jpeni Take Command Language Examples Support 1
rconn News Take Command / TCC / CMDebug / TCC-RT 26.02.41 Uploaded Support 0
fpefpe winfr command Support 4
Dmitry L. Kobyakov Fixed The SETARRAY /R command works incorrectly with cyrillic files Support 2
M Selecting test "off by one" problem in Take command Support 4
Dmitry L. Kobyakov Documentation Error in the Help: the WINDOW command Support 0
R How to? Open Take Command at current folder Support 7
rconn News Take Command / TCC / CMDebug / TCC-RT 26.01.40 uploaded Support 0
C TEE command appending null characters to output Support 6
vefatica IDE external command syntax color? Support 1
rconn News Take Command / TCC / CMDebug / TCC-RT 26.0.37 Uploaded Support 0
Alpengreis List command: text truncated in find box (german) Support 9

Similar threads