Observation/Question about ISSPACE Function

[Jay Sage]

Is it WAD that escape characters are not recognized by the ISSPACE function?

The expression %@isspace[string] returns 0 for strings ^t, ^n, ^f, ^r, and ^s but 1 for strings %@char[9], %@char[10], %@char[12], %@char[13,] and %@char[32].

The same difference in behavior applies to the ISCNTRL function as well.

On the other hand, the function ISASCII recognizes the escape characters as ASCII.

 

[vefatica]

I wouldn't be surprised if, for example, %@isascii[^n] == 1 because the string is entirely composed of ascii characters (namely, '^' and 'n'). I tried several @functions and their parameters do not seem to be escaped. I reckon @isspace[^s] looks at two characters, neither of which is whitespace.

 

[Jay Sage]

There certainly are functions that recognize the escape characters. For example, %@ascii[^t] returns 9, while %@ascii[^^t] returns 94 116.

I think that most string functions recognize the escape characters. Thus %@insert[3,^s,onetwo] returns one two. Oddly, "%@insert[3,^s,onetwo]" returns one^stwo. Quoting has an effect that I was not expecting.

C:]echo 1^s2
1 2

C:]echo "1^s2"
"1^s2"

C:]echo %@quote[1^s2]
"1^s2"

I was not expecting that quoting would prevent ^s from being expanded, and I certainly was not expecting the third result. In %@quote[1^s2] I would definitely expect the string 1^s2 to be parsed as '1' space '2' and the QUOTE function to turn that into "1 2".

Compare the above results to the following:

C:]echo %@ascii[1^s2]
49 32 50

C:]echo %@ascii["1^s2"]
34 49 32 50 34

C:]echo %@ascii[%@quote[1^s2]]
34 49 32 50 34

​

Maybe you can explain why the above makes sense. If you can't, undoubtedly Rex will have a full explanation.

 

[Jay Sage]

Here's some more playing around.

The function @ASCII turns off the processing of special characters in its argument, but it does expand the escape characters.

C:]echo %@ascii[1&2]
49 38 50

C:]echo %@ascii[1^s2]
49 32 50

However, the function @LEN turns off the processing of special characters AND turns off interpretation of escape characters.

C:]echo %@len[1&2]
3

C:]echo %@len[1^s2]
4

Here's another interesting set. The escape character ^s is recognized in both of the string arguments.

C:]echo %@index[one two,^s]
3

C:]echo %@index[one^stwo,^s]
3

C:]echo %@index[one^stwo, ]
3

Here's another really interesting set.

C:]echo !%@format[10,one^stwo]!
!  one two!

C:]echo !%@format[10,one two]!
!   one two!

C:]echo !%@format[010,one two]!
!000one two!

C:]echo !%@format[010,one^stwo]!
!00one two!

Although the string one^stwo is interpreted and displayed as one two, the length is not calculated correctly, and thus the formatting is one character off. The FORMAT function thinks that one^stwo is 8 characters long, so it inserts only two spaces or zeroes to pad out the string. My guess is that the FORMAT function is using the LEN function to get the length of the string, and we saw above that it counts ^s as two characters.

 

[vefatica]

I can't explain it all. Obviously, @ASCII and @FORMAT escape the string arguments. Many (most?) other variable functions don't. If plugin variable functions are any indication, TCC does not escape the arguments before giving them to the function. I've written hundreds of variable functions and, looking back, have only used the EscapeLine() function a handful of times. I have no particular strategy. If Rex does have a strategy, I hope he chimes in and enlightens us.

It is a bit mysterious ... when troublesome characters are [not] protected inside function arguments ... when processing escapes does [not] happen.

For @LEN, all the troublesome characters are protected and escapes are not processed.

v:\> echo %@len[a>b|c<d&e^f]
11

For @UPPER, the escapes are processed (first below). The special characters are not protected.

v:\> echo %@upper[a^>b]
A>B

v:\> echo %@upper[a>b]  <===========  A wound up in the file "b"

v:\> echo %@upper[a&b]  <===========  echoed A then executed my 'b' alias (cdd-)
A

c:\users\vefatica\desktop>

 

[vefatica]

I guess those are expected. They turn into echo A>B and echo A&B.

 

[Jay Sage]

My initial thought is that escape characters in string arguments would always be processed first. On the other hand, I can imagine that the issue is much more complicated than it appears at first, and Rex has probably had to deal with it.

The one case I described above that I think really needs to be corrected is with the FORMAT function. If the escape characters are going to be expanded, then the length calculation has to be performed on that expanded result.

I've been wrestling all day with a script I have that lists files with their size, date, and description, with the description wrapped with hanging indent. I managed to get the script to handle descriptions with the special characters & | < and >. Then I ran into problems with commas because the string was processed by a complex function that got its arguments confused when a string contained a comma. I think I've finally licked that problem, too.

 

[rconn]

Escape processing is done AFTER variable expansion. (It obviously has to be done that way - unless you never want to have an escape in a variable or alias.)

There are a few special cases where variable functions will look for escapes, but they are definitely the exception.