1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

A problem with the SafeChars Plugin...

Discussion in 'Plugins' started by mathewsdw, Nov 22, 2011.

  1. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    As a preface just to be honest, this is so strange and unexpected that I almost can't believe it myself, and I just downloaded and installed the latest version of the plugin ("SafeChars plugin v1.5.7 loaded."). So, since this is really quite simple, the complete listing of a very short batch file which illustrates the problem:

    Code:
    @Echo Off
    SetLocal
    UnSafe /E:, >NUL:
    Set RandomInformation=2011-07-21 And Other Stuff...
    @Echo Without SafeExp: "%RandomInformation"
    @Echo With SafeExp:"%@SafeExp[%RandomInformation]"
    UnSafe /Z >NUL:
    EndLocal
    Quit 0
    
    and the output of the above batch file:

    Code:
    Without SafeExp: "2011-07-21 And Other Stuff..."
       With SafeExp:"-07-21 And Other Stuff..."
    
    I somehow doubt that I have to give any significant information as to what exactly is wrong. I do have a "workaround" for now, but I consider it to be "dangerous" because it makes some assumptions about the input data and that is something I try to avoid if at all possible.
     
  2. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    From: mathewsdw
    ...
    | Code:
    | @Echo Off
    | SetLocal
    | UnSafe /E:, >NUL:
    | Set RandomInformation 11-07-21 And Other Stuff...
    | @Echo Without SafeExp: "%RandomInformation"
    | @Echo With SafeExp:"%@SafeExp[%RandomInformation]"
    | UnSafe /Z >NUL:
    | EndLocal
    | Quit 0
    |
    | and the output of the above batch file:
    |
    | Code:
    | Without SafeExp: "2011-07-21 And Other Stuff..."
    | With SafeExp:"-07-21 And Other Stuff..."
    |
    | I somehow doubt that I
    | have to give any significant information as to what exactly is wrong.
    | I do have a "workaround" for now, but I consider it to be "dangerous"
    | because it makes some assumptions about the input data and that is
    | something I try to avoid if at all possible.

    There are two separate issues with your code.

    1/ UnSafe is not a command in TCC nor in any plugin, esp. not in SafeChars.dll - what it does might affect SafeChars' operation.

    2/ @SefeExp is not intended for ENVIRONMENT VARIABLES - you should use @SafeEnv instead. However, as the example below shows, @SafeExp also works correctly (probably @SafeEnv is a stripped down version of it). More importantly, both @SafeExp and @SafeEnv use unusual syntax: you do not prefix names with the percent sign "%". This allows the plugin to evaluate the environment variable (in @SafeEnv) or the internal variable or function or array element (in @SafeExp) WITHOUT the TCC parser already doing so.

    Here is my example, which shows that using the correct function with the correct syntax everything works as documented:

    @Echo Off
    SetLocal
    Set RandomInformation 11-07-21 And Other Stuff...
    @Echo Without SafeChars: "%RandomInformation"
    @Echo With SafeEnv:"%@SafeEnv[RandomInformation]"
    @Echo With SafeExp:"%@SafeExp[RandomInformation]"

    Here is its output:

    Without SafeChars: "2011-07-21 And Other Stuff..."
    With SafeEnv:"2011-07-21 And Other Stuff..."
    With SafeExp:"2011-07-21 And Other Stuff..."
    --
    HTH, Steve
     
  3. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Don't put a percent sign before the variable (function, etc.) name. If you do, TCC will expand it before my code ever gets a chance to see it. (The same goes for @SAFEENV, too.)

    Code:
    @Echo With SafeExp:"%@SafeExp[RandomInformation]"
    
    In your case, TCC expands %RandomInformation to "2011-07-21 And Other Stuff", then hands that to @SAFEEXP, which expands %2011 to nothing... unless you are also passing at least two thousand and eleven command-line parameters to your batch file....
     
  4. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Thanks, Steve!

    Actually, I added that command a few months back to allow users to customize the list of problematic characters.
     
  5. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    Thank you for reminding me of that!!! My often-mentioned bad memory makes remembering things that are outside of the norm (just slightly, in this case) very hard for me to do. And the lack of the "%" makes perfect sense (in hindsight!).

    And also thank you for correcting Steve's response, it really confused me! (Yes, I'm easily confused.) The first thing I did after reading Steve's reply was to look for it in the SafeChar's documentation, and there it was! At that moment I had no clue as to what was going on. And, just as an aside, because of my bad memory I am heavily dependent on (good!) documentation, so if somebody adds or changes something in something I a regularly use, I will see it (and I won't even be aware that it is "new"!).
     
  6. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    New question re: A problem with the SafeChars Plugin...

    Charles, unfortunately the major problem that started this whole thing kind of got lost in the shuffle, so to speak, and I still don't know quite how to handle it. (I've found two things that "seem" to work, but I really don't trust them.) And I will warn you in advance that unfortunately this is kind of long.

    So here's the problem. I'm getting data from an external "source" via the "@ExecArray" function. Said data may contain "dangerous" characters, which, of course, will be "remapped" to their "safe" equivalents using "%@SafeExp". But the issue is that I often want to search individual elements of the array for (constant!) string(s) that may, themselves, contain "unsafe" characters.

    So an example of what I was trying to do is to look for an "&" character in a particular element of that array.

    So, some code fragments that I think will "illustrate" the problems I'm having.

    First, the following code is contained in a .btm file that begins with:
    Code:
    @Echo Off
    SetLocal
    UnSafe /E:,
    
    and ends with:
    Code:
    UnSafe /Z >NUL:
    EndLocal
    Quit 0
    
    This is, of course, all pretty much the standard prologue and epilogue code of a "safe" batch file.

    Now the first actual line of "useful" code is, for this example:
    Code:
    Set ABC="Ampersand: &  Caret: ^  OR bar: |  Comma: , "
    
    Note that this is a quoted string, so the ampersand, caret, OR bar, and comma are all "valid".

    So, executing:
    Code:
    @Echo String as entered: %ABC
    
    produces:
    Code:
    String as entered: "Ampersand: &  Caret: ^  OR bar: |  Comma: , "
    
    which is what you would expect, and is OK other than the fact that it is enclosed in double quotes, which is generally not acceptable in the situations where I am trying to process this data.

    So executing:
    Code:
    @Echo Unquoted string: %@UnQuote[%ABC]
    
    produces:
    Code:
    Unquoted string: Ampersand:
    TCC: Z:\SafeCharTests.btm [6]  Unknown command "Caret:"
    TCC: Z:\SafeCharTests.btm [6]  Unknown command "Comma:"
    
    which is also, somewhat unfortunately, what you would expect.

    So, executing:
    Code:
    @Echo Index of the ampersand in the "unsafe" environment variable with the ampersand "escaped":
    @Echo %@Index[%ABC,^&]
    
    produces:
    Code:
    Index of the ampersand in the "unsafe" environment variable with the ampersand "escaped":
    12
    
    which is "almost" acceptable if you remember that the opening quote of the environment variable is included in the index.

    So, if you execute:
    Code:
    @Echo "Safe" environment Variable: %@SafeEnv[ABC]
    
    produces pretty much what you would expect given that the "unsafe" characters have been replaced by close equivalents that are visually similar but not identical to the "original" characters.

    However, executing:
    Code:
    @Echo Unquoted "safe"  environment variable:
    @Echo %@UnQuote[%SafeEnv[ABC]]
    
    produces:
    Code:
    Unquoted "safe" environment variable:
    "Ampersand: &  Caret: ^  OR bar: |  Comma: , "
    
    Note that the "quotes" are still present because the "remapped" quotes are not really ASCII quotes, of course.

    And executing:
    Code:
    @Echo Index of the ampersand in the "safe" environment variable:
    %@Index[%@SafeEnv[ABC],^&]
    
    produces:
    Code:
    -1
    
    because the "remapped" ampersand is no longer an ASCII ampersand, of course.

    And, rather strangely in my opinion, executing:
    Code:
    @Echo  Index of the caret in the "Unsafe" environment  variable:
    @Echo %@Index[%ABC,^^]
    
    produces:
    Code:
    -1
    
    for reasons I don't really understand.

    However, in researching this, the command sequence:
    Code:
    Set ABC="^"
    Echo %@Ascii[%ABC]
    
    produces:
    Code:
    34 34
    
    Only the double quotes come up, no sign of the caret. And, just for verification:
    Code:
    Echo %ABC
    
    produces,
    Code:
    "^"
    
    meaning that the caret is still there, somewhere. Evidently some parsing that is taking place is "discarding" it in both this and other possibly similar situations.

    Well, to search for a string containing an ampersand, for example, I was creating and then using a search term that was defined similar to:
    Code:
    Set SearchTerm1=%@SafeExp[This is an ampersand: &]
    
    Now, in the "real" world situation the environment variable being searched would not be set by a "Set" command, but rather would be one of the results of the "@ExecArray" function. However, if you then:
    Code:
    @Echo %SearchTerm1
    
    you get:
    Code:
     is an ampersand: &
    
    What happened to the first word is kind of a mystery to me other than to say I was not supplying an internal variable, function, or array element to the "@SafeExp" function, which means, I suppose, that the the "results" of this statement are essentially undefined.

    And I also don't have a clue as to a "supported" way to set an environment variable the "safe" equivalent of an "unsafe" character.

    Now, there are two things that both appear to work, the first for completely mysterious to me reasons meaning I don't trust them at all, and the second one where, for whatever reason, the "undefined" behavior appears to have a "useful" result, but is it reliable?

    So, the first, "mysterious", technique is to do something like:
    Code:
    Set SearchTerm1=%@SafeExp[Dropped This is an ampersand: &]
    
    followed by:
    Code:
    Set SearchTerm1=%@LTrim[" ",%SearchTerm1]
    
    which, when executing
    Code:
    echo %Searchterm1
    
    produces:
    Code:
    This is an ampersand: &
    
    Which is what I want. However, I think that you will agree with me when I say that the above is kind of a klutzy kludge (or maybe it's kluge, looking up both words online would seem that they are very similar if not in fact identical)

    Another, slightly less odd, "solution" is:
    Code:
    Set SearchTerm1=This is an ampersand %@SafeExp[&] isn't it.
    
    which produces, when echoed:
    Code:
    This is an ampersand & isn't it.
    
    which is what I want but also depends on, at best, "undocumented" behavior.

    So the question is, of course, how should this be handled?
     
  7. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    New question re: A problem with the SafeChars Plugin...

    Matthew:

    First, I apologize for the comment relating to UNSAFE - indeed I had an earlier version of the plug-in,and I should have checked my version against the version you specified.

    Rather than trying to answer your latest post point by point, I just want to point out that Charles made available "internal variables" in the plugin that contain the representations of the "safe" versions of the characters which the plugin translates, for the very purpose you have - to be able to search the safe version of a string for the special characters. These are copied below from the "Internal Variables" topic:

    Variable Returns
    %_AMP safe ampersand &
    %_BQ safe backquote `
    %_CARET safe caret ^
    %_CLOSEBRK close bracket ]
    %_CLOSEPAT close parenthesis )
    %_GT safe greater-than sign >
    %_LT safe less-than sign <
    %_OPENBRK open bracket [
    %_OPENPAT open parenthesis (
    %_PCT safe percent sign %
    %_QUOTE safe double quotes "
    %_VBAR safe vertical bar ¦



    Note that only the default unsafe characters have corresponding internal variables. Any additional character you declare unsafe using UNSAFE /E do not have corresponding internal variables.


    Charles:

    It would be nice if you would make the above into a TABLE, with two additional columns - the unsafe and safe codes. Another possible enhancement to go along with the UNSAFE command would be to allow the user to specify not only the characters to be "safed" but also their "safecode"-s. For non-alphanumerics it would be nice for you to create entries in the above "safe" table (i.e., internal variable, description, glyph, ASCII and safecode) in a separate "safeable" section. For decimal digits I'd suggest _SAFE0 .. _SAFE9, for lower case letters _SLCA .. _SLCZ, and for upper case letters _SCAPA .. _SCAPZ. (Not _sucX - not all sound OK). It would be conceivable to have the plugin register all of them with TCC when loaded even if not activated. This would match the role of e.g. _quote when the quotation mark is declared safe.
    --
    Steve
     
  8. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    From: Charles Dye
    | Thanks, Steve!
    |
    |
    | Quote:
    | Originally Posted by Steve Fabian
    | 1/ UnSafe is not a command in TCC nor in any plugin, esp. not in
    | SafeChars.dll - what it does might affect SafeChars' operation.
    |
    | Actually, I added that command a few months back to allow users to
    | customize the list of problematic characters.

    Sorry, I did not have the latest version. Is your stuff still accessible by FTP, which allows me to download only things that are newer than my copies, or do I have to keep looking for announcements?
    --
    Steve
     
  9. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    In further defense of my not having the latest version(s), the last specific announcement in the PLUGINS forum of a newer version of SafeChars.dll was in October of last year (2010). The 1.4.0 version of 2011-01-03 was not announced (I did not check possible announcements in the SUPPORT forum), but it still did not have the UNSAFE command documented.
    --
    Steve
     
  10. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    I don't generally announce uploads here -- this is Rex's forum, not my vanity press, and most new builds tend to be fairly trivial tweaks. You can find the current builds here: Plugins for Take Command

    Yes, I have toyed with the idea of setting up my own private FTP server, but honestly I don't see any benefit beyond allowing automated scraping.
     
  11. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    In the docs, you mean? There is a table near the top which gives decimal and hex codes for both. I could replicate those columns in the table under "Internal Variables", but it seems to me that the primary benefit of having the internal variables is that you don't have to think about the codes....

    If you want more variables, it's easy to define them yourself. Just add an offset of 0xFEE0:

    Code:
    set safecomma=%@char[%@eval[%@ascii[,] + 0xfee0]]
    
    While it's possible to declare alphanumerics as 'unsafe', I can't imagine why anybody would actually want to do it.
    [/QUOTE]
     
  12. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    From: Charles Dye
    | I don't generally announce uploads here -- this is Rex's forum, not
    | my vanity press, and most new builds tend to be fairly trivial
    | tweaks. You can find the current builds here: Plugins for Take
    | Command

    Actually, IIRC this is the forum where users may announce new and improved plugins, request new or improved plugins, comment on existing ones, and request plugin support they cannot create.

    Rex, could you create a "Plugin Announcements" forum?

    | Yes, I have toyed with the idea of setting up my own private FTP
    | server, but honestly I don't see any benefit beyond allowing
    | automated scraping.

    A basic difference btw. FTP and HTTP is that FTP provides file timestamp information, which can be used to determine whether or not the last downloaded version is still up to date. I will in the near future be on a download budget (I am switching from a cable ISP to a HotSpot device, with lower speed than cable, and even lower after I downloaded a certain amount of data in the billing month), and I do not like to browse. "Automatic scraping" is what I do at jpsoft.com - copy anything newer than in my last download, but only when I am aware there is something likely to be gained. Actually I use TCC command "copy/s/[dxxx]/uf" - which I presume is an implementation of "automatic scraping".

    Theoretically Rex provided a method to upload plugins to his FTP server, whence users can download them, guaranteed to be the current version. If all plugin authors would upload released versions of their plugins, in the absence of announcements this would require browsing only a single location, instead of keeping track of the websites of all plugin authors... The latter would be really difficult, as at least two plugin authors have disappeared from the forum, though both have plugins at ftp://jpsoft.com/plugins/.

    --
    Steve
     
  13. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    Daniel:

    I won't dissect your post in detail here, but I'll respond to a few points. First, as Steve points out, there are internal variables corresponding to 'safe' versions of the more troublesome characters. So if, for example, you want to search for a 'safe' ampersand, you can use %_AMP in %@INDEX.

    There's nothing cosmic about these variables; you can create your own easily. Just add 0xFEE0 (65248 decimal) to the ASCII value of the character you want to make safe. The comma has an ASCII value of 44, so the 'safe' comma would be character 65248+44 or 65292, and you can

    Code:
       set safecomma=%@char[65292]
    

    What you're trying to do with your @SAFEEXP hack is to create a function which automatically 'safes' a string. It would be trivial for me to add such a function to the plugin, but not so easy for you to use, since TCC expands arguments before the function sees them; you'd still have to escape or quote significant characters, or use SETDOS....

    When you do this:

    Code:
       Set SearchTerm1=%@SafeExp[This is an ampersand: &]
    
    the function expands the variable THIS. Since THIS probably isn't defined, the result is an empty string and your THIS vanishes. You could hack around it by doing something like this:

    Code:
       Set SearchTerm1=%@SafeExp[[]This is an ampersand: &]
    
    which expands nothing into nothing -- nothing lost, nothing gained.

    I don't make the point very hard in the docs, but you can also use @SAFEEXP to expand array variables. To get a safed version of FOO[5,2], you can use %@SAFEEXP[FOO[5,2]]. You could store it back by doing

    Code:
      set foo[5,2]=%@safeexp[foo[5,2]]
    
    Or fix an arbitrary value with:

    Code:
      set foo[%i,%j]=%@safeexp[foo[%i,%j]]
    
    Do that in a loop, and you can declaw the entire array. (I'm hoping that Rex will add an array API to some future version of TCC, so I can do neat stuff like that in a plugin without calling SET x number of times....)
     
  14. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Re: New question re: A problem with the SafeChars Plugin...

    From: Charles Dye
    | Originally Posted by Steve Fabian
    || It would be nice if you would make the above into a TABLE, with two
    || additional columns - the unsafe and safe codes.
    |
    | In the docs, you mean? There is a table near the top which gives
    | decimal and hex codes for both. I could replicate those columns in
    | the table under "Internal Variables", but it seems to me that the
    | primary benefit of having the internal variables is that you don't
    | have to think about the codes....

    In the "Theory" topic you could just provide a hyperlink to the "Internal Variables" topic, stating that the list is to be found there. In the "Internal Variables" topic you could add the codes to the existing table, and thus provide all the information in a single location.

    | If you want more variables, it's easy to define them yourself. Just
    | add an offset of 0xFEE0:
    |
    | Code:
    | set safecomma=%@char[%@eval[%@ascii[,] + 0xfee0]]

    The reason I suggested that you provide them as internal variables a priori is because the UNSAFE command does not provide a method to do it.

    | While it's possible to declare alphanumerics as 'unsafe', I can't imagine why anybody
    | would actually want to do it.

    Nor do I, but - if I understand the UNSAFE command - it is possible to do so...

    BTW - and this is way off topic - a question about a subject related to @UQUOTE. Do you know how to "type" (i.e., enter on the keyboard) the special "opening quote mark" which is a descender instead of an ascender? Not intended for use with TCC or TCMD...
    --
    Steve
     
  15. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    You mean this one: ‟ ?

    In a console or Take Command tab window, it's Alt-08223. In other Windows programs, it's not so easy. Alt-+201f might work, if you have the hex keypad option turned on, and the program permits it, and the font contains that character, and the wind is out of the west on alternate Thursdays....
     
  16. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Re: New question re: A problem with the SafeChars Plugin...

    From: Charles Dye
    | Originally Posted by Steve Fabian
    || BTW - and this is way off topic - a question about a subject related
    || to @UQUOTE. Do you know how to "type" (i.e., enter on the keyboard)
    || the special "opening quote mark" which is a descender instead of an
    || ascender? Not intended for use with TCC or TCMD...
    |
    | You mean this one: ‟ ?

    No, its the opening quotation mark here: „arra a napra†This is the Hungarian style of quoting.

    | In a console or Take Command tab window, it's Alt-08223. In other
    | Windows programs, it's not so easy. Alt-+201f might work, if you have
    | the hex keypad option turned on, and the program permits it, and the
    | font contains that character, and the wind is out of the west on
    | alternate Thursdays....

    hex keypad option? how do I turn it on in WinXP?
    --
    TIA, Steve
     
  17. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    Something got badly UTF8ized there; low double-9 quotes? Probably Alt-08222, or Alt-+201e.

    Code:
    echo %@regset["HKCU\Control Panel\Input Method\EnableHexNumpad",REG_SZ,1]
    
    I think you may need to reboot after doing that.
     
  18. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    Re: New question re: A problem with the SafeChars Plugin...

    Thank you Charles, you precisely answered my questions.

    This is long and hopefully the last time if I've already stated much of this in the past: Possibly again (you'll understand the "possibly" in a moment) I have multiple disabilities, by far the worst of which is my bad memory (that's why the "possibly"). If it was not for my cell phone (my chief "memory") and this computer (for those things that are more "extensive"), I would have to be institutionalized as of this moment because I would not be able to survive on my own. (I had lunch with a close friend last Friday and she told me that she thought that the accommodations I had made - virtually all of them involving technology in some way - for both this and my other disabilities was truly amazing to her and she thought that just about anybody who had as many and as severe disabilities as I have would have been institutionalized a long time ago.) But the bottom line, and this does not embarrass me in any way, is that the sum total of my disabilities makes me, simply put, incompetent. (Just as an example, you would not believe how reliant I am on a spell-checker in "composing" documents such as this - I have probably had more than two dozen spelling errors up to this moment - fortunately this software highlights them if I manage to see them - poor eyesight - and I am totally reliant on both Microsoft Word and Google because they are essentially my "saviors" in these situations).

    So, the bottom line is that I remembered the existence of those "internal variables" and what they were only after you reminded me of them, and I just spent about 10 minutes trying to look up/figure out what they were/were called just now - because of my bad memory I had forgotten that already so I had to search the forums and originally I came up with nothing because I searched the "support" forum instead of this forum, but for whatever reason I wasn't able to supply the correct "key words" to even search this forum - I finally resorted to directly accessing the "SafeChars.chm" help file - for reasons I don't understand (probably my stupidity at this moment), some times SafeChars stuff "shows up" in the standard help system, most of the time it does (like just now) not. A mystery that's not important enough to track down and solve.

    And, as a side note, as of this moment I have 59 programming projects in the queue, so to speak, and I know this because I have to keep a list of them in a file that I have handy that I add to as new "projects" come up and delete from when I actually manage to complete a project, and I'm very slow at this time so that doesn't happen very often. And every single one of these "projects" is related to my disabilities in some way or another, poor finances because I do not get much money from my Social Security Disability Income, poor memory in one way or another related to projects that help me "compensate" for that in one way or another, and (many!) projects to automate things that I do on a regular basis to compensate for one or more of my disabilities and because I am rather paranoid that I will make substantial mistakes in doing manually (as well as the time taken to do them). I used to be a very, very, good C++ programmer (in fact, I tied for the second-highest score up to that point on a Microsoft C++ "competency" exam that they used to administer), but that is effectively no longer the case.

    So the bottom line here is that I will probably often ask dumb questions here; I am truly sorry about that and am very appreciative of your help in answering those questions.

    Thank you very much!!!
     
  19. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Re: New question re: A problem with the SafeChars Plugin...

    From: Charles Dye
    | Originally Posted by Steve Fabian
    || No, its the opening quotation mark here: „arra a napra†This is the
    || Hungarian style of quoting.
    |
    | Something got badly UTF8ized there; low double-9 quotes? Probably
    | Alt-08222, or Alt-+201e.

    It went through all the manipulations from my posting by email to receiving by email as the correct character, matching your guess of "low double-9 quotes". However, when I am in TCC window (not a TCMD tab), none of the numbers in the alt-08222 vicinity displays it correctly (shows a comma), in LIST the hex is 201e (dec. 8222), as you predicted. So the code is correct, the display is not - not a big surprise. BTW, I know "alt-08222" means hold ALT key down why entering 08222 on the numeric keypad, but what keystroke sequence is "alt-+201e"?

    || hex keypad option? how do I turn it on in WinXP?
    |
    | Code:
    | echo %@regset["HKCU\Control Panel\Input
    | Method\EnableHexNumpad",REG_SZ,1]

    | I think you may need to reboot after doing that.

    Probably I need to do that.
    --
    Thanks, Steve
     
  20. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    Using the default "Raster Fonts"? Try Consolas or Lucida Console; I don't think the default console font includes that character.

    Hold down ALT while typing +201 (on the numeric keypad) and E (on the alphabetic keys, of course.) This doesn't work in a console window -- ask Microsoft -- and it doesn't work in Take Command, either. TC uses Alt-+ for "Zoom In", but even if you disable that, it still doesn't work, perhaps because the underlying console window still doesn't support it...?
     
  21. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Re: New question re: A problem with the SafeChars Plugin...

    From: Charles Dye
    | Using the default "Raster Fonts"? Try Consolas or Lucida Console; I
    | don't think the default console font includes that character.

    Indeed, Lucida Console displays it, but my WHICH.BTM displays alias definitions incorrectly, they work in raster fonts...
    I succeeded just using alt-08222. Hex mode not tested...

    Thanks!
    --
    Steve
     
  22. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,280
    Likes Received:
    38
    Re: New question re: A problem with the SafeChars Plugin...

    Aw, quit apologizing. I can name more than a few people with no apparent disabilities, who communicate far worse than you do. It's what you do with your strengths that counts.
     
  23. mfarah

    Joined:
    Nov 2, 2009
    Messages:
    226
    Likes Received:
    5
    Re: New question re: A problem with the SafeChars Plugin...

    If you have a need to frequently type certain [Unicode] characters that aren't available in your regular keyboard layout(like, say, ಠ_ಠ ʃ ʘ ₩ ₭ ₮ ₱ ⱥ Ƶ Ȕ ṍ , etc.), you're better off creating a custom one. I did that and I'm a happy camper. :-)

    Take a look at Keyboard layouts for Windows. - my "Latin American Extended" layout is a great example of this.
     
  24. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,523
    Likes Received:
    4
    Re: New question re: A problem with the SafeChars Plugin...

    From: mfarah
    | If you have a need to frequently type certain [Unicode] characters
    | that aren't available in your regular keyboard layout(like, say, à²*_à²*
    | ʃ ʘ â‚© â‚* â‚® ₱ â±¥ Ƶ È” á¹ , etc.), you're better off creating a custom one.
    | I did that and I'm a happy camper. :-)
    |
    | Take a look at Keyboard layouts for Windows. - my "Latin American
    | Extended" layout is a great example of this.

    Thanks Miguel. I already have Hungarian keyboard available (alt-shift-1 for EN, alt-shift-2 for HU), but only the Hungarian edition of Word supports the "opening quotation mark", the only character I needed, and need it only very rarely. I will probably modify the HU keyboard layout to make available this extra character. And then I will spend hours going back to actually using it...

    Happy thanksgiving to US readers!
    --
    Steve
     
  25. Stefano Piccardi

    Joined:
    May 31, 2008
    Messages:
    376
    Likes Received:
    2
    Re: New question re: A problem with the SafeChars Plugin...

    In dealing with SafeChars strings I often find myself needing to unquote strings, so I developed a simple idiom:

    Code:
    rem make %1 safe as %s
    set s=%@SAFEEXP[1]
    ...
    rem safe @unquotes applied to s
    if "%[_QUOTE]%_QUOTE" == "%@left[1,%s]%@right[1,%s]" set s=%@left[-1,%@right[-1,%s]]
    
    I memorized this idiom as
     
  26. Stefano Piccardi

    Joined:
    May 31, 2008
    Messages:
    376
    Likes Received:
    2
    update:
    To unquote safechar strings, I use these two:
    Code:
    @echo off
    rem Topic: @unquotes[] for %[_QUOTE]s
    set local
    rem x must be a safechars string, in particular it must not include real double quotes
    set x=%[_QUOTE]C:\a \b,,,d.e%[_QUOTE]
    rem plugin-less version
    echo %@execstr[if "%[_QUOTE]%[_QUOTE]" == "%@left[1,%x]%@right[1,%x]" (echo %@left[-1,%@right[-1,%x]]) else (echo %x)]
    rem needs 4UTILS plugin
    echo %@XREPLACE[\A(%[_QUOTE])(.*)\1$,\2,%x]
    
     
  27. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    Stefano, you are certainly correct regarding the 4Utils "XREPLACE", which I was not at all aware of (and I have that plugin). Thank you!

    And this may fall into the category of "just being picky", I suppose, but your "@ExecStr" solution, while perfectly "valid", I think, doesn't directly address the issue as well as invoking the "%@ExecStr" function, which I tend to believe might have some performance impact. (Maybe I'm incorrect in that belief, Rex!)

    So, I had already "solved" this problem by writing a function that can do it directly:
    Code:
    Function UnSafeQuote=`%@If["%_QUOTE%%_QUOTE" == "%@Left[1,%@UnQuoteS[%1]]%@Right[1,%@UnQuoteS[%1]]",%@Left[-1,%@Right[-1,%@UnQuoteS[%1]]],%@UnQuoteS[%1]]`
    
    Now, this function does have one slight disadvantage in that it appears to me to be "contrary" to your solution in that if the argument contains white space and/or commas it must be enclosed in double-quotes ("real" double quotes, not %_Quote's; no white space, no commas, no need for the double quotes). This is because an argument containing white space and/or commas will be effectively "translated" into multiple arguments [%1, %2, %3...] by the TCC parser, and because of that you will not get what you would hope; and using "%$" instead of "%1" in the function definition does not completely solve the problem (I don't know exactly why nor do I care).

    To exactly "demonstrate" that, the below:
    Code:
    @Echo Off
    SetLocal
    Function UnSafeQuote=`%@If["%_QUOTE%%_QUOTE" == "%@Left[1,%@UnQuoteS[%1]]%@Right[1,%@UnQuoteS[%1]]",%@Left[-1,%@Right[-1,%@UnQuoteS[%1]]],%@UnQuoteS[%1]]`
    UnSafe /E:, >NUL:
    ::
    :: Without quotes, white space, commas or invalid characters
    ::
    @Echo ^:^: 1^: Without quotes and without white space, commas or invalid characters
    Set Value=AStringWithNoLeadingQuoteAndNoTrailingQuoteAndNoWhiteSpaceOrCommas
    @Echo      Before @SafeExp: %Value
    Set Value=%@SafeExp[Value]
    @Echo Before UnSafeQuoting: %Value
    @Echo  After UnSafeQuoting: %@UnSafeQuote[%Value]
    @Echo ^n^n
    ::
    :: Extraneous white space or commas but and a (previously) invalid character not enclosed in
    :: double quotes when invoking @UnSafeQuote
    ::
    @Echo ^:^: 2^: Extraneous white space and commas and an invalid character enclosed in double quotes
    @Echo      when invoking @UnSafeQuote
    Set Value="A    string    with  extraneous  white  space, commas, & an invalid character enclosed in double quotes"
    Set Value=%@SafeExp[%Value]
    @Echo Before UnSafeQuoting: %Value
    @Echo  After UnSafeQuoting: %@UnSafeQuote["%Value"]
    @Echo ^n^n
    ::
    :: Extraneous white space or commas and an invalid character not enclosed in double quotes
    :: when invoking @UnSafeQuote
    ::
    @Echo ^:^: 3^: White space and commas and an invalid character not in double quotes when invoking
    @Echo        @UnSafeQuote
    Set Value="A  string with white  space, commas, & an invalid character enclosed in double quotes"
    Set Value=%@SafeExp[Value]
    @Echo Before UnSafeQuoting: %Value
    @Echo      Calling @UnSafeQuote without the safe-char'd value enclosed in double quotes
    @Echo  After UnSafeQuoting: %@UnSafeQuote[%Value]
    UnSafe /Z >NUL:
    Function UnSafeQuote=
    EndLocal
    Quit 0
    
    (It's not too long despite the fact that I "left in" all of the necessary "overhead".)

    And the results of running the above:
    Code:
    :: 1: Without quotes and without white space, commas or invalid characters
          Before @SafeExp: AStringWithNoLeadingQuoteAndNoTrailingQuoteAndNoWhiteSpaceOrCommas
    Before UnSafeQuoting: AStringWithNoLeadingQuoteAndNoTrailingQuoteAndNoWhiteSpaceOrCommas
    After UnSafeQuoting: AStringWithNoLeadingQuoteAndNoTrailingQuoteAndNoWhiteSpaceOrCommas
     
    :: 2: Extraneous white space and commas and an invalid character enclosed in double quotes
          when invoking @UnSafeQuote
    Before UnSafeQuoting: "A    string    with  extraneous  white  space, commas, & an invalid character enclosed in double quotes"
    After UnSafeQuoting: A    string    with  extraneous  white  space, commas, & an invalid character enclosed in double quotes
     
    :: 3: White space and commas and an invalid character not in double quotes when invoking
          @UnSafeQuote
    Before UnSafeQuoting: "A  string with white  space, commas, & an invalid character enclosed in double quotes"
        Calling @UnSafeQuote without the safe-char'd value enclosed in double quotes
    After UnSafeQuoting: "A
    
    I tend to believe that the comments in the above code are sufficient.

    And, as far as "@XReplace" goes, due to my aversion to regular expressions due to my bad memory and since I have (and have had for a couple of days) the above function written and thoroughly tested, I think I'll just "stick to" what I've already done.

    And I really have to ask what is a purely academic question here: Why the "%[_Quote]" rather than just "%_Quote"? I really don't see any difference, in theory, between "%[XXX]" and "%XXX", and in at least this case there doesn't appear to be any difference, either. Specifically,
    Code:
    [Z:\]Echo ===%_Quote=== %@Len[%_Quote] :: ===%[_Quote]=== %@Len[%[_Quote]]
    
    produces
    Code:
    ==="=== 1 :: ==="=== 1
    
    exactly what I would expect.

    - Dan
     
  28. David Marcus

    Joined:
    Jun 4, 2008
    Messages:
    646
    Likes Received:
    1
    It relieves you from remembering to double the percent signs in something like %_Quote%%_Quote.
     
  29. mathewsdw

    Joined:
    May 24, 2010
    Messages:
    855
    Likes Received:
    0
    David,

    That makes total sense although that has never been a problem for me despite my bad memory. Since I know, almost instinctively by this point in time, that you want to follow a variable by a single percent sign in order to concatenate an alpha-numeric string to it (the fact is that virtually nothing else would make any sense given the nature of the language), if I were to code a single percent sign (which may actually happen "momentarily"), I immediately see that that effectively means just concatenation of a constant character string and that if I want to concatenate the contents of another variable I need to double the percent signs. (This is in no sense a criticism of any kind; you do what works for you just as I do what works for me and there is no significant downside to doing things the way you do them.) I was just curious and thank you for clearing up the (small!) mystery!

    - Dan
     
  30. Stefano Piccardi

    Joined:
    May 31, 2008
    Messages:
    376
    Likes Received:
    2
    Mathew, sorry to report that your version doesn't correctly unquote the test string that I indicated in my previous post. It works fine with more standard strings, but commas and possibly other characters confuse your version. As for disliking regular expressions, I understand your aversion, just let me explain what mine does.
    Code:
    rem below _ marks a safe quote character
    %@XREPLACE[\A(%[_QUOTE])(.*)\1$,\2,%x]      rewrite for brevity as %@XREPLACE[\A(_)(.*)\1$,\2,%x])
     
    \A(_) looks for _ and, if it finds it, sets \1 = _ (\1 is a variable name)
     
    \1$ looks for what's inside \1 which is _ if \A(_) found a _
     
    (.*) looks for anything, including the null string, and sets \2 = anything
     
    Why \A and $ ? Because they anchor a valid match to be respectively at the start and
    at the end of the input string, which ensures that we are matching SURROUNDING safe
    quotes vs. embedded quotes.
     
    So, if the match is successful XREPLACE changes the input string to the contents of
    \2, which is the input string without surrounding safe quotes.
     
    If the match isn't successful XREPLACE returns the input string unchanged.
    
    Test
    Code:
    rem below _ marks a safe quote character
    E:\>set x=%[_QUOTE]C:\a \b,,,d.e%[_QUOTE]
     
    E:\>echo (%x)^n(%@XREPLACE[\A(%[_QUOTE])(.*)\1$,\2,%x])
    (_C:\a \b,,,d.e_)
    (C:\a \b,,,d.e)
     
    E:\>echo (%@UnsafeQuote[%x%])
    (_C:\a)
     
    E:\>echo ("%@UnsafeQuote[%x%]")
    ("_C:\a")
     
    E:\>echo (%@UnsafeQuote["%x%"])
    (C:\a \b)
    
     

Share This Page