True. Indeed, because of the limitations of CMD, I used to do all my complex file operations with Visual C++ code using Windows API functions. TCC has saved me hours upon hours! The precise reason that we all use TCC is because it can do what CMD does not let us do. And that's why we wish that it would be even more complete, to be able to handle all of our files without getting stuck on an odd filename that someone created.
I used to use all kinds of other tools, until I started to use 4DOS. After that I abandoned most of them!
1] Regarding the "user-definable" point - well, even the current escape chars are user-definable as well, but in practice it doesn't matter because almost everybody uses the default. Similarly here - we can pick an undefined unicode char from a fairly marginal unicode range, and then we'll also have virtually everybody sticking with it, except for a rare exception.
Disagree. Many of us who started with 4DOS still use its set of special characters (CommandSep, EscapeChar, ParameterChar directives; SETDOS options /C, /E, /P; resp.) both to maintain functionality of old batch programs and aliases, and because of habit; although I only use %+ and %= in new or revised aliases and batch programs.
2] Regarding the unicode/ASCII point. First of all, the world is moving towards unicode more and more, especially power users (and users of TCC tend to be power users!). Secondly, this wouldn't modify existing behavior, but rather it is an added capability. So if someone is still on ASCII, they could either continue as they do now, or they would have the option of moving to unicode to gain this very useful added capability. And for someone who really needs this capability, moving to unicode fonts would be a very small move to make in order to gain a huge advantage. Finally, you could allow the character to be set to an ASCII char as well, so that users who are on ASCII systems could have the option of setting it to a low-ASCII or high-ASCII char that they never use in filenames.
There are many old ASCII-only utilities which I continue to use for lack of similar capabilities in newer products (or in some cases affordability), including the 147 home-brew BRIEF macros. So ASCII is still viable. But you realized yourself that the solution to this issue need not be a strictly Unicode one.
All in all, I think that it would be a very useful and appreciated feature. I know it would completely perfect TCC in my mind, because it would mean that I'd be able to run batches on all filenames, regardless of oddball chars. I gather that Steve Fabian would agree, as well as many other users. Would you prefer that I post the request on the uservoice forum?
Yes, I agrrrrrreeeee. But the real issue is not with filenames only, it is also with strings that contain data, e.g., read from a file. The problem is that the LANGUAGE used by TCC, inherited from the COMMAND.COM of PC-DOS 3 (or earlier), does not distinguish between command and data. All the enhancements 4DOS, 4NT and TCC brought required using many more characters as syntactical delimiters. In the days of FAT file systems these delimiters were not legal in file names, and rare in data. This allowed to continue to use the old system of totally untyped strings. Based on my own theoretical knowledge of programming languages, this lack of types will never allow unrestricted use of all characters. The POSIX-based shells make the distinction between command token and data.
If you go back in the history of the JPsoft support, you will find that others and I had requested a new syntactic form making a strict distinction between command and data many years ago. Of course, it would not really be CMD compatible, in some ways it would be closed to POSIX shells... But as long as it is a superset, or a startup option, possibly using a new file extension for its command procedures, CMD and backward compatibility can be retained.