How to? Character transfer among @BALLOC buffers, command tails, and possibly redirected standard streams

[LarryBrasfield]

After contemplating the difficulties and impediments associated with trying to combine command grouping with collection of command tails for (likely redirected) output, I am looking into an alternative which relies upon explicitly created pipes. [a] However, I am still stymied by one part of the alternative, which is getting command tails or possibly non-interpreted text into a buffer created by @BALLOC[]. I hope to find a solution to that problem which does not require protecting the text content from further TCC interpretation when the @BWRITE[] function is called. (Such protection partially defeats the purpose of the alternative.)

[a. An "explicitly created pipe" is one created by @FILEOPEN["\\.\pipe\whatever",...] rather than a consequent of redirection. ]
[b. By "non-interpreted text" I mean what is defined within TEXT/ENDTEXT constructs or by "here-document" syntax. ]

It would suffice if there existed internal commands, say BPUTLITERAL and BPUTCOOKED, which could be invoked thusly:
BPUTLITERAL %myBufferHandle {text subject to no interpretation other than end-of-line termination}
or
BPUTCOOKED %myBufferHandle {text subject to all the normal TCC pre-execution expansions}
. As far as I can see, there is no way to synthesize these commands from TCC internal commands and functions. Nor do I see a way to implement them as normal .exe (or other executable forms) because there is no way to pass one of TCC's binary buffer handles to another process where they would have any meaning. Nor is there a way to get a non-TCC-interpreted command tail to such an external program on its command line.

If my above @BALLOC buffer loading scheme must be a bust, a good alternative would be a way to redirect output to a named pipe with an important difference from normal redirection processing. That difference is that execution of the command(s) would not be wrapped by opening and closing the pipe. This would be similar in concept to use of the standard handles, which the shell (TCC or any other) uses without having to open and close them. It could, logically, be worked into the common redirection syntax, 1> or 2>, except that there would be a way to make the handle of a pipe to be written known as 3 (or 4 or ...), so that redirection would take the form 3> (or 4> or ...).

Is there a way to synthesize BPUTLITERAL and BPUTCOOKED in TCC?

Alternatively, is there a way to directly redirect output to an already-open named pipe without TCC closing it afterward?

 

[Joe Caverly]

I'm probably not heading in the correct direction with this suggestion, but what about the SafeChars Plugin that Charles Dye developed?

Joe

 

[Charles Dye]

I'm probably not heading in the correct direction with this suggestion, but what about the SafeChars Plugin that Charles Dye developed?

It knows nothing about binary buffers, I'm afraid.

 

[LarryBrasfield]

Thanks, Joe. That looked promising, in part. (The other part is that I still saw no way in the SafeChars doc to get some text given to a plugin function or command until late in the pre-execution processing (which is what I wish to avoid.)

My reading of the help topic, TCC/Command Line/Command Parsing, indicates that no plugin (or any other command implementation [a]) gets a chance to execute prior to expanding env-vars, redirection processing, and escape processing.

[a. Apparently, then, the ECHOX command gets special cased so that its command tail is not subject to the usual pre-execution processing. It would be great if a plugin could designate itself for the same treatment. ]

I am going to experiment with combinations of command grouping and gosub. I am hoping that a gosub call avoids the side-effects of the glomming used to implement command grouping.

Has Charles published source for his SafeChars plugin?

 

[Charles Dye]

My reading of the help topic, TCC/Command Line/Command Parsing, indicates that no plugin (or any other command implementation [a]) gets a chance to execute prior to expanding env-vars, redirection processing, and escape processing.

That's right. Variable expansion and redirection happen before the command is called. (For plugin commands the same as externals.)

Has Charles published source for his SafeChars plugin?

Not in any formal way, but if you poke around the FTP site you'll find a SRC directory. Fair warning: my code is probably not for the weak of stomach.

 

[samintz]

Is there a reason you can't use SETDOS /X?

 

[vefatica]

[a. Apparently, then, the ECHOX command gets special cased so that its command tail is not subject to the usual pre-execution processing. It would be great if a plugin could designate itself for the same treatment. ]

Would that really help? The argument to such a plugin (command or variable function) could only come from one place ... hard-coding. And if it has to be hard-coded you might as well protect all the troublesome characters in the first place.

I am curious about the "handle" returned by @BALLOC. Is it really a handle? From such a handle, could a plugin determine the size of the buffer and its location?

 

[Charles Dye]

re: special-handling flags for plugin commands

Would that really help? The argument to such a plugin (command or variable function) could only come from one place ... hard-coding. And if it has to be hard-coded you might as well protect all the troublesome characters in the first place.

The command line could name an environment variable, which the command could read and parse itself. I've occasionally wished for this ability myself, but it seems like it be useful in only a very few places. Like, SafeChars and nothing else. Not worth complicating the plugin interface over.

 

[Charles Dye]

I am curious about the "handle" returned by @BALLOC. Is it really a handle? From such a handle, could a plugin determine the size of the buffer and its location?

The last time I looked, the 'handle' is just a pointer to the data. There's no header or anything; it just points to byte #0. You can get the allocated size with QueryMemSize(). I figure that if QueryMemSize() returns zero or MAXUINT_PTR, then you have an invalid handle.

Of course, this is all undocumented internal stuff. Rex could change it tomorrow.

 

[vefatica]

re: special-handling flags for plugin commands


The command line could name an environment variable, which the command could read and parse itself. I've occasionally wished for this ability myself, but it seems like it be useful in only a very few places. Like, SafeChars and nothing else. Not worth complicating the plugin interface over.

If the plugin were expecting the name of an envvar ... just pass it. But what about getting some nasty string into the variable in the first place.

I was fooling around with SETDOS, putting a nasty string into an envvar, constructing an ECHOX command (in an envvar) and executing that command. I met with some success but noticed an oddity ... an extra space. Compare the two below.

v:\> set command=echo foo

v:\> %command
foo

v:\> set command=echox foo

v:\> %command
 foo

 

[LarryBrasfield]

Is there a reason you can't use SETDOS /X?

Well, there was a reason: I forgot about that. Thanks.

Time for some experimentation. For example, variations on:

@echo off
setlocal
(
gosub putStuff
gosub putStuff
) > junk.txt

endlocal
quit

:putStuff
  setdos /X-135789A
  echo. Whatever I !@#$%Want
  setdos /X0
return

 

[LarryBrasfield]

Would that really help? The argument to such a plugin (command or variable function) could only come from one place ... hard-coding. And if it has to be hard-coded you might as well protect all the troublesome characters in the first place.

I frequently need the functionality or ease-of-use of some language other than TCC batch language. Producing separate little snippets in separate files is clumsy when what I want to submit to another interpreter resembles:
# Snippet 1, many lines needing no TCC expansion and likely to suffer from it
...
# Snippet 2, one or a few lines which usefully incorporate text resulting from TCC expansions
...
where the whole sequence needs to be emitted into a stream, uninterrupted by I/O handle closes.

Of course, it is conceivable that all of it could be carefully escaped, with TCC's expansion and escaping rules used to get the desired result. I know, from having gone that route (many times), that it is tedious, easy to get wrong, produces code that is hard to read, and can create bugs which are hard (and tedious) to isolate. That is why I seek a way to more directly express what is being emitted. I manage to deal with specific language rules (such as for SQL or Perl), but when constructs for another language have to be combined with defense against modification by TCC, the results are a distraction from getting the other language code right. I usually spend more time fighting TCC's depredations than getting what survives them correct.