Functions acting directly on variables

[rfajman]

How about a way to have a function act directly on the contents of a variable or the output of another function (e.g., @function{variable} and @function{@function{variable}})? The point is that the values of the variables and functions would not pass through the command line. There are a number of possibilities. I haven’t worked out the details, but it seems to me that it could permit processing of text without all the problems of having it go through the command line (with all the problems that various special characters cause).

 

[Rick Reinckens]

echo %@left[3,%@right[7,%_isodate]]

That takes the right-most 7 characters of isodate, e.g., 2019-04-25 returns 9-04-25, then it returns the left-most 3 characters, so the result is 9-0.

Having the function work directly without a SET command would be a major departure from established syntax and could easily lead to unintentionally changing variable values.

 

[Charles Dye]

I see no reason why a command couldn't, say, accept the name of an environment variable on the command line; read data from the environment variable, and store its result back in the same variable. Wouldn't require any special new syntax or funky changes to the parser.

 

[rfajman]

My concern is that it is just as much or maybe even more of a nightmare to process arbitrary text with TCC than with CMD, because it has more special characters to worry about. It comes up very often with command output. Here's one example, a command, dig, which queries the DNS:

dig @107.6.45.4 s3.wasabisys.com soa -4 +timeout=5 +tries=1
; <<>> DiG 9.14.0 <<>> @107.6.45.4 s3.wasabisys.com soa -4 +timeout=5 +tries=1
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 58233
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;s3.wasabisys.com. IN SOA

;; AUTHORITY SECTION:
wasabisys.com. 119 IN SOA dns1.p05.nsone.net. hostmaster.nsone.net. 1550853901 14400 3600 604800 300

;; Query time: 129 msec
;; SERVER: 107.6.45.4#53(107.6.45.4)
;; WHEN: Sun Apr 28 00:00:40 Eastern Daylight Time 2019
;; MSG SIZE rcvd: 110

Notice all the < and > characters, plus the parentheses. Other things have parentheses and brackets too.

I have to remember to quote parentheses in an ECHO command when I issue a message to the user. If the echo command is part of another command file, I may have to do multiple quoting of parentheses, which means I have to know how many levels there are. I have written various subroutines for issuing a message to the user, which provide additional functions. That leads to the need for quoting.

There are can be problems just capturing the output of a command. For example, to capture the output of the command

dig ... | grep ...

the | character has to be quoted. I have also needed to pass input to a command using < and capture the output. Obviously, the < has to be quoted.

P.S. - If you are familiar with dig, which was ported from Unix, you may want to tell me that dig has options for suppressing parts of the output. I do know that, obviously. Some other ports from Unix that I have found useful are grep and sed.

 

[AnrDaemon]

Use bash already.

 

[Charles Dye]

Some time back, I wrote a plugin to help handle problematic text. Basically it remaps troublesome characters to similar-looking characters in the Unicode Fullwidth block, then changes them back later. The whole thing is kind of a bodge, but it seems to work. I use it to e.g. chop up HTML doc files for conversion to .CHM. If you're interested, the documentation is here: SafeChars plugin

Other possible approaches include dedicated text-handling tools (sed, awk, grep) and actual programming languages (C++, Java, Basic, Pascal). I think any command shell is going to have similar issues -- though as you say, they may be worse in TCC, simply because TCC has more characters with syntactic significance.

 

[vefatica]

There are can be problems just capturing the output of a command. For example, to capture the output of the command

dig ... | grep ...

the | character has to be quoted.

Please give an example of what you mean.

If you are familiar with dig, which was ported from Unix, you may want to tell me that dig has options for suppressing parts of the output.

Ordinarily, I get the verbose output from DIG.EXE that you quoted (17 lines, in a recent test). My DIG.EXE has many options for suppressing output. Here are two.

v:\> dig myip.opendns.com +nocomments +noquestion +noauthority +nostats +noqr +nocmd @resolver1.opendns.com
myip.opendns.com.       0       IN      A       71.176.103.102

v:\> dig myip.opendns.com +short @resolver1.opendns.com
71.176.103.102

 

[Rick Reinckens]

I see no reason why a command couldn't, say, accept the name of an environment variable on the command line; read data from the environment variable, and store its result back in the same variable. Wouldn't require any special new syntax or funky changes to the parser.

He didn't say command, he said function. I do know that some languages (if I remember correctly, FoxPro) could use the syntax:

= JoinNames(FirstName, MiddleName, LastName, GenSfx)

That would do whatever the function was designed to do (for instance, combine them, write that to a file and return it as a string) but the output would be ignored--so you don't have to define a variable just to have something to accept the function's return value.

 

[rfajman]

Let me try to explain what I mean more clearly. Now there is the @eval function which takes an expression consisting of variables, function calls, and arithmetic operators (e.g., @eval[a + 1]). There are lots of operators and, most importantly, variables and functions are not preceded by "%", which means that there is no substitution, but rather the values of the variables and functions are processed without being substituted, as in most programming languages. Let's extend it to do string operations too, with operators such as "." for concatenation. Then we can invent a new construct using braces, perhaps, to enclose an expression or multiple expressions, as in {expression1,expression2,...}. TCC does not seem to use braces, as far as I can tell, but if it does, something else could be used. Then we could have constructs like @function{expression1,expression2,...} for function calls with no substitution in the arguments or maybe %{expression} for a computed value substituted into a command.

I'm not trying to work out all the details here, but I hope this gives you a better idea about what I am thinking of. I don't know the internals of TCC, but it seems to me that it wouldn't require a complete rewrite.

 

[rfajman]

Some time back, I wrote a plugin to help handle problematic text. Basically it remaps troublesome characters to similar-looking characters in the Unicode Fullwidth block, then changes them back later. The whole thing is kind of a bodge, but it seems to work. I use it to e.g. chop up HTML doc files for conversion to .CHM. If you're interested, the documentation is here: SafeChars plugin

Other possible approaches include dedicated text-handling tools (sed, awk, grep) and actual programming languages (C++, Java, Basic, Pascal). I think any command shell is going to have similar issues -- though as you say, they may be worse in TCC, simply because TCC has more characters with syntactic significance.

SafeChars looks very useful. Thanks!

 

[AnrDaemon]

@rfajman , this is an inherent issue of CMD (or, more specifically, WinAPI, and consequently TCC) comparing to POSIX shells.
POSIX demands that all unquoting is performed by the shell, and parameters passed to the program as already parsed list.
CMD has no such possibility, WinAPI requires that all parameters are passed to the program via single unparsed string.