Command line evaluation question - variable inside parenthesis

May 30, 2008
235
2
Hi,

I stumbled upon a (to me) strange thing when I generated some SQL for piping to sqlite3.exe.

These two statements where called in a loop:

Code:
set extraWhereCondParsed=%extraWhereCondParsed% OR (%thisCondParsed%)
echo "%extraWhereCondParsed%"
Two outputs from the echo command were:
Code:
"("Last Name" LIKE 'b*') OR (%thisCondParsed%)"
"("Last Name" LIKE 'b*') OR (%thisCondParsed%) OR (%thisCondParsed%)"
So the environment variable inside the parenthesis was not evaluated. (But other variables inside parenthesis elsewhere were evaluated, just not this one.

Adding `` around the parenthesis:es fixed the problem.

Code:
set extraWhereCondParsed=%extraWhereCondParsed% OR `(`%thisCondParsed%`)`
echo "%extraWhereCondParsed%"

"("Last Name" LIKE 'b*') OR ("First Name" LIKE 'a*')"
"("Last Name" LIKE 'b*') OR ("First Name" LIKE 'a*') OR ("First Name" LIKE 'a*')"
Any idea what's going on? Would it always be a good idea (or even necessary in some cases) to use `` around literal (and not meant for e.g. command grouping) parenthesis:es?
 

rconn

Administrator
Staff member
May 14, 2008
12,345
150
I cannot duplicate your results (I suspect you've simplified things in the message to the point that it doesn't show the complete steps to reproduce it).

TCC will not expand variables inside a command group (this is a feature!), but with what you've shown TCC shouldn't think this *is* a command group.

If you think you have a case where it is not expanding variables where it should please try to reduce it to a simple batch file.
 
May 30, 2008
235
2
Here is a simplied batch to demonstrate the problem.

Well, as simplified as I could make it with not too much effort. :)

To trigger the (possible) error just run:
Code:
err bb -wfn.a* -wfn.b*

Output "will" be:
Code:
extraWhereCondParsed: ("First Name" LIKE 'b*')
extraWhereCondParsed: ("First Name" LIKE 'b*') OR (%thisCondParsed%)
Active code page: 1252
.header on
.mode column
.width 6 40 15 15 10 3 3
SELECT Books."BookID","Title","Last Name","First Name","Date read","Owned","Bought Ebook" FROM Books
INNER JOIN DatesRead USING(BookID)
        INNER JOIN AuthorBooks USING(BookID)
        INNER JOIN Authors USING(AuthorID)
WHERE (Title LIKE '%%') AND (("First Name" LIKE 'b%%') OR ())
ORDER BY "Last Name","First Name","Title";

But it "should" be:
Code:
extraWhereCondParsed: ("First Name" LIKE 'b*')
extraWhereCondParsed: ("First Name" LIKE 'b*') OR ("First Name" LIKE 'a*')
Active code page: 1252
.header on
.mode column
.width 6 40 15 15 10 3 3
SELECT Books."BookID","Title","Last Name","First Name","Date read","Owned","Bought Ebook" FROM Books
INNER JOIN DatesRead USING(BookID)
        INNER JOIN AuthorBooks USING(BookID)
        INNER JOIN Authors USING(AuthorID)
WHERE (Title LIKE '%%') AND (("First Name" LIKE 'b%%') OR ("First Name" LIKE 'a%%'))
ORDER BY "Last Name","First Name","Title";

"First Name" LIKE 'a*' is simply not expanded from the variable.

Error happens in routine :extractExtraWhereCond [whereCond].
I added some "!!!" markers where it happens.

I
 

Attachments

  • err.btm
    7.6 KB · Views: 98

rconn

Administrator
Staff member
May 14, 2008
12,345
150
Re: Here is a simplied batch to demonstrate the problem.

Well, as simplified as I could make it with not too much effort. :)

Not a bug.

You're running into a problem with nested variable expansion -- your first argument is being expanded to "("First Name" Like 'b*')". TCC then looks to see if it should expand this argument again, and finds that it's now looking at a command group -- so it skips further expansion on the line.

If you do a "setdos /x-4" it should work as you expect.
 
May 30, 2008
235
2
Re: Here is a simplied batch to demonstrate the problem.

Not a bug.

You're running into a problem with nested variable expansion -- your first argument is being expanded to "("First Name" Like 'b*')". TCC then looks to see if it should expand this argument again, and finds that it's now looking at a command group -- so it skips further expansion on the line.

If you do a "setdos /x-4" it should work as you expect.

Hm, I don't quite understand why the variable expansion would be nested.

Code:
iff defined extraWhereCondParsed then
    set extraWhereCondParsed=%extraWhereCondParsed% OR (%thisCondParsed%)
else
    set extraWhereCondParsed=(%thisCondParsed%)
endiff

echo extraWhereCondParsed: %extraWhereCondParsed%
The gosub routine is called two times.

The first time the else branch is taken and extraWhereCondParsed is set to "(%thisCondParsed%)" The following echo also shows that "thisCondParsed" was expanded correctly and not "got stuck" inside extraWhereCondParsed.

The next time the if-branch is taken, but this time the same "(%thisCondParsed%)" expression fails to expand. Is that just because it came after a preceding (not nested) variable expansion of "%extraWhereCondParsed%" on the same line?

setdos /x-4 DID help, I just don't understand where the nesting comes in. Does it really matter (apparently) where in the line something is?

Changing the order like this also makes it expand correctly.

Code:
iff defined extraWhereCondParsed then
    set extraWhereCondParsed=(%thisCondParsed%) OR %extraWhereCondParsed%
else
    set extraWhereCondParsed=(%thisCondParsed%)
endiff

echo extraWhereCondParsed: %extraWhereCondParsed%

I think I prefer using `` around the parenthesis since this expresses the intent best (assuming it's always guaranteed to disable what happens with command grouping here).
 

rconn

Administrator
Staff member
May 14, 2008
12,345
150
Re: Here is a simplied batch to demonstrate the problem.

Hm, I don't quite understand why the variable expansion would be nested.

*All* variable expansion is nested unless disabled with SETDOS /X-4. Every time a variable is expanded, the parser runs it through expansion again until it's sure there aren't any more variables. In your case, it expanded the variable, found a command group, and aborted further expansion on the entire command.
 
May 30, 2008
235
2
Re: Here is a simplied batch to demonstrate the problem.

*All* variable expansion is nested unless disabled with SETDOS /X-4. Every time a variable is expanded, the parser runs it through expansion again until it's sure there aren't any more variables. In your case, it expanded the variable, found a command group, and aborted further expansion on the entire command.

Ok, I'm starting to get it. Must be a bit slow today. :)

Thanks for taking the time to explain!


So, the second gosub call, when

Code:
set extraWhereCondParsed=%extraWhereCondParsed% OR (%thisCondParsed%)
is expanded, then "%extraWhereCondParsed%" will expand to a command group (as set by the else-branch the first call) and this will stop further expansion of variables on the same line. (Adding back quotes around "(" and ")" made them not behave like a command group so in that case it worked as wanted).

Is this behavior documented? The topic "Command Parsing" - "3. Expanding variables" does mention that variable expansion works differently for some internal commands like IF and GLOBAL, but does not mention command groups.
Or maybe that's implied by:
These commands are always followed by another command, so variable expansion takes place separately for the original command and the command that follows it.
Still don't understand how SETDOS /x-4 helped (as well) as that disables nested variable expansion. Which sounds like the opposite of what would be needed. And the variables WERE still expanded.
 
May 30, 2008
235
2
Re: Here is a simplied batch to demonstrate the problem.

Ok, I'm starting to get it. Must be a bit slow today. :)

Still don't understand how SETDOS /x-4 helped (as well) as that disables nested variable expansion. Which sounds like the opposite of what would be needed. And the variables WERE still expanded.

Wait, think I just got it.

Disabling nested variable expansion means that "the expander" just expands %extraWhereCondParsed% without further looking at the expanded value. So it does not see that it expands to a command group which would disable further variable expansions on the rest of the command line (and not only of the newly expanded value as setdos /X-4 does).

Initially I just thought of disabling nested variable expansion as having to effect of not further expanding hidden variables becoming visible as a result of expanding an "outer" variable.

Seems it can have other (more subtle) effects too! Well, at least I'm aware of this one now.
 
Similar threads
Thread starter Title Forum Replies Date
T VIEW only works from command line Support 14
vefatica `Back quotes` - command line vs. batch file Support 5
Phileosophos Is there any way to open the Take Command options dialog from the command line? Support 8
Phileosophos Command-line Editing Shortkeys That Fail Support 6
D Open Windows 10 Photos app from command line Support 11
fpefpe How to? command echo in title line/bar Support 2
vefatica Command line DO with no closing parenthesis? Support 9
R Long commands retrieved from command history have blank line when wrapped Support 20
D Double Quote in TC Tool Bar Button Command line Support 3
Joe Caverly Expand all aliases on the command line Support 2
Jay Sage Cannot Postion Cursor in Command Line Using Mouse Support 7
vefatica IDE.EXE's command line? Support 19
D btm file command line augments comparing for string or math. Support 12
P Escape key no longer clears command line in v24 Support 21
J File explorer handling from command line? Support 2
M Ctrl-C when using command line history does nothing Support 2
M Command line parsing differences between cmd.exe and TCC Support 6
C Replace 1st argument on command line Support 4
K Command Line Selection Not Working Support 12
J Error running Python at command line Support 9
rps How to? Alias/Command line use of *log on/off Support 2
x13 //directive ignored on TCC command-line Support 4
J How to? max. practical command line length? Support 3
mfarah WAD Odd behaviour with command-line arguments on TCMD. Support 7
thorntonpg F1 on command line The webpage cannot be found Support 5
J Python: TCC command line parsing removes '=' equal sign characters Support 4
L Fixed console line length in Take Command 19? Support 3
C LOG entry for TCMD/TCC command line Support 15
H command line parsing question Support 5
Joe Caverly How to? Date Range in command line for 7zip /= Support 1
cgunhouse One of the Focuses of V17 Beta Testing Should Be Command Line Parsing Support 1
cgunhouse Jabber Command Line Option Being Transmitted -- Fixed with Build 42 Support 2
M Hotkey-mechanism, for doing something with the current command-line Support 4
J Passing % character through command line to .btm Support 5
vefatica Command line SETLOCAL? Support 11
tmaynard How to? Control-U alias for Escape (command line editing) Support 5
BobK How to? Handling filenames with blanks at command line Support 3
CWBillow Send Command-line with prompt? Support 6
M Missing PERL command-line arguments Support 1
fpefpe WAD Python support does not process command line Support 14
vefatica SENDMAIL, command line override SMTP server? Support 2
vefatica (OT) Show desktop from command line? Support 20
M Starting Windows GUI programs from the command line... Support 10
A How to? Remove last argument on command line Support 4
nickles Reload tcmd.ini from the command line Support 11
A How to? Invoke View Clipboard from command line Support 8
ehab aboudaya How to? activate tab with same directory in command line Support 6
A minor documentation fix - command line editing Support 0
A Minor documentation fix ["-" on command line] Support 3
A WAD BOM printed on command line from BAT file Support 5

Similar threads