More SafeChars type issues...

  • This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn more.
#1
As a rather-long introduction, as I have stated multiple times previously, it is my goal to write .btm scripts rather than C++ programs (well more then 300 of them at this moment from many years of coding, most of which were one-time-only programs or are now "obsolete" for various reasons; but there are 53 of them that are still in relatively "active" use - I know this because the executables are all consolidated into one directory and I just listed that directory excluding all of the ones that I do not to consider to be in "active" use), and "invalid characters" (to TCC) are a continual problem because I often have no control whatsoever over what characters appear in the input files/streams to these programs (and am usually not even interested in the lines that contain these "invalid characters" although there is no "general" "rule" that can be used such as the "Find /V" command to eliminate these lines). The "SafeChars" plugin (by Charles Dye; thanks again Charles!) has been a real aid in these matters, but as far as I know/can determine there is no way to make it work directly on the output of programs/commands as "captured" by @ExecStr and @ExecArray. So, to determine how much of an impact this would have, I wrote a quite-simple .btm program that could test this out, and the result I got from this .btm file was at least close to what I was expecting.

So, on to the point.

The code for this batch file is simple enough to include it (excluding "overhead" like SetLocal, EndLocal, Quit, etc.) directly here. And this batch file (named Loop.btm) just "Echoed" all of the characters in the ASCII character set with some "identification" information to standard output:

Code:
...
On Error Gosub ReportChar
Set I=1
Do While %I LT 256
   @Echo %I 0x%@Right[2,0%@Convert[10,16,%I]] --%@Char[%I]--
   Set I=%@Inc[%I]
EndDo
...
:ReportChar
   @EchoErr I: %I 0x%@Right[2,0%@Convert[10,16,%I]
   Return
So here are the not quite "unexpected" things:

1. I can not even "run" Loop.btm to completion in a "standalone" fashion writing its output directly to the console - the TCC window in which I run it "disappears" without notice. And note that the "On Error" statement does not "capture" any error(s).

2. Exactly the same thing happens if I "execute" it writing its output to a file, the command "Loop >&Loop.txt" also terminates that TCC session without notice and looking at the Loop.txt file (from a "new" TCC session) indicates that the last character that was "successfully" written to it was ASCII 37 (0x25), which did produce somewhat "weird" output which was not too surprising because that is the "%" character; although why it apparently "crashes" TCC on ASCII 38 (I can't quite figure out from an ASCII table what that character even is - a single quote?) is beyond me (maybe my bad eyes are an issue here).

3. Slightly modifying "Loop.btm" ("If %I != 38 ...") so that it does not write any output for ASCII 38 causes it to "continue" until ASCII 62, which is the greater than (">") symbol, not completely surprising.

4. Why ASCII 60 (the less-than symbol ("<")) is not much of a problem I can't quite figure out, although nothing is displayed for that character.

5. Slightly modifying "Loop.btm" again ("If %I != 38 .AND. %I != 62 ...") so it also does not write any output for ASCII 62 (again, the "greater than" symbol), it is able to proceed "successfully" until ASCII 96, almost certainly the back quote as far as I can tell.

6. Slightly modifying "Loop.btm" yet again ("If %I != 38 .AND. %I != 62 .AND. %I != 96 ...") so it also does not write any output for ASCII 96 (again, the back quote symbol), allows it to proceed "successfully" to ASCII 255 (the last "valid" ASCII value, of course).

8. However ASCII 124 (the OR bar ("|")) produces 'TCC: Z:\Loop.btm [7] Unknown command "--"', which is not too surprising.

So the bottom line is that I definitely need "SafeChars"-like functionality for the @ExecArray (and possibly @ExecStr) function(s), but looking at the "plugin" SDK I don't even see any way for it to even "see" and therefore process an "array" variable. Does this mean I am simply out of luck and have to go back to writing C++ programs to do this kind of thing? (Although maybe writing a C++ program that will execute the desired program "capturing" its output, "altering" the "invalid" characters (as does "SafeChars"), and then returning said output to @ExecArray and therefor the array will get the job done - but that seems to me to be a substantial amount of work - although it might be worth it given no other alternatives.)

- Dan Mathews
 
#2
From: mathewsdw
...
| Code:
| ...
| On Error Gosub ReportChar
| Set I=1
| Do While %I LT 256
| @Echo %I 0x%@Right[2,0%@Convert[10,16,%I]] --%@Char[%I]--
| Set I=%@Inc[%I]
| EndDo
| ...
|| ReportChar
| @EchoErr I: %I 0x%@Right[2,0%@Convert[10,16,%I]
| Return
|
| So here are the not quite "unexpected" things:
|
| 1. I can not even "run" Loop.btm to completion in a "standalone"
| fashion writing its output directly to the console - the TCC window
| in which I run it "disappears" without notice. And note that the "On
| Error" statement does not "capture" any error(s).
|
| 2. Exactly the same thing happens if I "execute" it writing its
| output to a file, the command "Loop >&Loop.txt" also terminates that
| TCC session without notice and looking at the Loop.txt file (from a
| "new" TCC session) indicates that the last character that was
| "successfully" written to it was ASCII 37 (0x25), which did produce
| somewhat "weird" output which was not too surprising because that is
| the "%" character; although why it apparently "crashes" TCC on ASCII
| 38 (I can't quite figure out from an ASCII table what that character
| even is - a single quote?) is beyond me (maybe my bad eyes are an
| issue here).

You problem for both 1/ and 2/ (and also for 3/ through 8/) is the visual delimiter you chose for the characters. If the character to be displayed has syntactical significance for TCC, as does the ampersand (&, ASCII: 38) (and the characters mentioned later) it needs to be escaped in some form. The loop below worked as expected (note that I used DO's built-in indexing capability):

do i = 1 to 255
echo %@f3[%i] 0x%@right[2,0%@convert[10,16,%i]] "%@char[%i]"
enddo

Notice that I enclosed the character to be displayed in quotation marks!

| So the bottom line is that I definitely need "SafeChars"-like
| functionality for the @ExecArray (and possibly @ExecStr) function(s),
| but looking at the "plugin" SDK I don't even see any way for it to
| even "see" and therefore process an "array" variable. Does this mean
| I am simply out of luck and have to go back to writing C++ programs
| to do this kind of thing? (Although maybe writing a C++ program that
| will execute the desired program "capturing" its output, "altering"
| the "invalid" characters (as does "SafeChars"), and then returning
| said output to @ExecArray and therefor the array will get the job
| done - but that seems to me to be a substantial amount of work -
| although it might be worth it given no other alternatives.)

There is a great difference between @EXECARRAY - which sets the values of array elements and @EXECSTR, which returns a string result from a command. The @SAFEEXP[] function of the SafeChars.DLL works with both.

Assume there is an array MyArray which has been set by @EXECARRAY. You access its element at offest N using SafeChars thus:
%@SafeExp[MyArray[%nN]]

To handle the string returned by @EXECSTR:
%@SafeExp[@ExecStr[command]]

Notice a peculiarity of @SafeExp: you do not prefix the array name or function name with the percent sign % normally required.
--
HTH, Steve
 
#3
From: mathewsdw
...

There is a great difference between @EXECARRAY - which sets the values of array elements and @EXECSTR, which returns a string result from a command. The @SAFEEXP[] function of the SafeChars.DLL works with both.

Assume there is an array MyArray which has been set by @EXECARRAY. You access its element at offest N using SafeChars thus:
%@SafeExp[MyArray[%nN]]

To handle the string returned by @EXECSTR:
%@SafeExp[@ExecStr[command]]

Notice a peculiarity of @SafeExp: you do not prefix the array name or function name with the percent sign % normally required.

...
--
HTH, Steve
Thanks, Steve! I either didn't know or completely forgot about @SafeExp (could be my oft-mentioned bad memory again). And for some now-unknown reason I didn't think that double quotes effectively "escaped" "illegal" characters.

However, that did not at all fix the underlying problem. Here is the code where I was "executing" Loop.btm
Code:
SetArray Junque[300]
Set I=0
@Echo %@ExecArray[Junque,Z:\Loop] >NUL:
@Echo %%_ExecArray: %_ExecArray
Do While %I LT %_ExecArray
   @Echo %I 0x%@Convert[10,16,%I] %@SafeExp[Junque[%I]]
   Set I=%@Inc[%I]
EndDo
UnsetArray Junque
And, without bothering to post it here (you can run it yourself if you have any doubts, but I don't think that will be necessary) the output as read by "@ExecArray" entirely stops at character decimal 28. But this turns out to be the "File Separator" character (whatever that is exactly). While I have some doubt that there is any "workaround" for this, I have great doubt that I'll ever "see" that character in the output of any program/batch file that I am "executing".

Just out of curiosity, why did you think that I didn't know both of and about @ExecStr? I didn't reference it in my posting because it was exactly the same issue as with @ExecArray except that @ExecArray might be more difficult to handle because of the array.

- Dan
 
#4
---- Original Message ----
From: mathewsdw
...
Just out of curiosity, why did you think that I didn't know both of and about @ExecStr? I didn't reference it in my posting because it was exactly the same issue as with @ExecArray except that @ExecArray might be more difficult to handle because of the array.
---- End Original Message ----
You mentioned both @EXECSTR and @EXECARRAY in the same sentence. While it was clear from the text you understood the nature of TCC arrays, the reference to @EXECSTR was very generic. I thought that pointing out the difference is harmless if you likewise understood the nature of the @EXECSTR function, but useful if it had been otherwise.
--
Steve
 
#5
---- Original Message ----
You mentioned both @EXECSTR and @EXECARRAY in the same sentence. While it was clear from the text you understood the nature of TCC arrays, the reference to @EXECSTR was very generic. I thought that pointing out the difference is harmless if you likewise understood the nature of the @EXECSTR function, but useful if it had been otherwise.
--
Steve
Thank you for your answer, Steve! (And you were right about it being "harmless", other tha piquing my curiosity. As I said, I was curious.)

- Dan