1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

WAD Leiningen broken with TCC (but OK with CMD)

Discussion in 'Support' started by tmaynard, Mar 5, 2013.

  1. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    The command 'lein' (https://github.com/technomancy/leiningen) fails with TCC but works okay with CMD, like so...

    TCC:
    [C:\Users\Tom\Projects\command-line-args]lein version
    TCC: C:\Users\Tom\bin\lein.bat [96] Unknown command ")"
    TCC: C:\Users\Tom\bin\lein.bat [96] Unknown command ")"
    TCC: C:\Users\Tom\bin\lein.bat [261] Unknown command "-client"

    CMD:
    C:\Users\Tom\Projects\command-line-args>lein version
    Leiningen 2.0.0 on Java 1.7.0_11 Java HotSpot(TM) 64-Bit Server VM

    Leiningen is the de facto Clojure build tool (BAT file attached for your perusal), and works just fine with CMD, bash, zsh, and probably plenty of other shells.

    There seems to be a problem with "backward compatibility."
     

    Attached Files:

  2. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,277
    Likes Received:
    38
    Does it work as expected if you remove the parentheses from lines 94 and 96?
     
  3. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    Almost. The error on line 261 persists:
    [C:\Users\Tom]leinx version
    TCC: C:\Users\Tom\bin\leinx.bat [261] Unknown command "-client"
     
  4. Charles Dye

    Charles Dye Super Moderator
    Staff Member

    Joined:
    May 20, 2008
    Messages:
    3,277
    Likes Received:
    38
    Try removing or commenting out lines 84 and 94 as well.

    (The peculiar parentheses in lines 94 and 96 appear to be an attempt to export the variables _VAR and _RESULT from the SETLOCAL/ENDLOCAL block -- a task which CMD.EXE provides no documented way to perform. But, if they're going to export the changed variables anyway, why even use SETLOCAL? Weird.)
     
  5. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    Same error from line 261, or (to put it another way) no difference.
     
  6. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,730
    Likes Received:
    80
    What are the minimal steps I need to take in order to reproduce this? Just running "lein version" returns:

    Code:
    C:\Users\Rex\.lein\self-installs\leiningen-2.0.0-standalone.jar can not be found
    .
    You can try running "lein self-install"
    or change LEIN_JAR environment variable
    or edit lein.bat to set appropriate LEIN_JAR path.
    That last part seems a bit dubious, since bash & zsh can't run any BAT files.
     
  7. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    You could follow the instructions: "lein self-install" and that should work, or -- you may need Java installed (just the JRE, but that's almost standard these days).

    Ahem. Let's not get overly literal, okay? The lein command (in whatever flavor ... but identical code logic) runs on a multitude of platforms, everywhere that the JVM and Clojure will run. [Clojure is an alternative language for the JVM, like Scala, Groovy, NetRexx, and others.]

    Edit: Deobfuscation -- BAT and shell files are different code but identical intent.
     
  8. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,730
    Likes Received:
    80
    I had to install some other things as well (like wget), but I can now reproduce the problem.

    The problem lies in the somewhat obscure line 91:

    Code:
    set _result=%_result:"=%
    I assume the author is trying to remove quotes, but it's not going to work as-is in TCC. TCC is going to break the variable name at the ':', resulting in "set _result to the value of _result, and append ":=%" ". This is not likely to be useful, and it also means the following command group closing ) will be seen as inside a double quoted string, and therefore not a real command group closure.

    There are several possible workarounds in TCC, but I don't know if you are interested in modifying the batch file to work with TCC (trivial), or if you want to run it as-is. (Which is not possible now; I *might* be able to come up with a parser change to support this syntax, but it's not going to be today.) Generally, for highly kludged CMD scripts like this one, we recommend that you just run it in CMD to avoid having to set too many restrictive TCC options.



    I don't think there's any other way to deal with software issues; my ESP isn't working ... ;)
     
  9. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    My primary goal is the combination of (A) TCC, and (B) successful execution of LEIN.BAT. If that is accomplished by tweaking the file, then so be it. I can report the required changes (and, ideally, their reason(s)) to the Leiningen developers.

    Since Leiningen's use is truly widespread (I don't know a single Clojure hacker that doesn't use it, and they are legion), perhaps you could pursue a fix from the TCC side. Admittedly, most of those are Linux and/or OSX folks so the Venn diagram overlap of TCC and Clojure users is going to be small -- whether it's worth the effort is up to you to decide.

    Possibly, but I moved up to TCC to avoid the excruciating limitations of CMD (versus zsh) ... and to get "native" Rexx support. In fact, that's how I discovered the error in the first place: simply by trying the same command under TCC and CMD. I'd rather tweak Leiningen than go back to CMD.

    Fair enough. I was a programmer and a systems programmer for close to 20 years, and I know that when you talk to a compiler every day you learn to be literal. The compiler cuts you no slack at all. I'll try to be more specific in the future -- if there is a future!

    Just let me know what needs to change in LEIN.BAT and I'll do it, post-haste ... and I'll open a ticket with Leiningen support (if you can instruct me on the whys and wherefores, I'll appreciate that info as well -- and it will make my ticket much more instructive, I hope).

    Thanks for chasing this one down.
     
  10. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,730
    Likes Received:
    80
    There are two problematic lines in the batch file, which are the result of an odd decision on CMD line parsing. CMD expands the variables on a line *before* it does any parsing on the line; this means that if you have compound commands on a line, the variables will be expanded before any of the commands are executed. This can lead to some rather strange results -- for example:

    Code:
    set a=12
    set a=6 & echo %a% & set a=0 & echo %a%
    will return in CMD:

    Code:
    12
    12
    
    The TCC parser was designed to be more rational, and we decided not to emulate this CMD "unfeature". TCC will return:

    Code:
    6
    0
    
    This is rarely an issue with CMD batch files, and it certainly makes scripts written for TCC easier to understand and debug. On rare occasions (when the CMD batch programmer is especially deranged or determined to hammer the square peg into the round hole), it creates an incompatibility with TCC.

    In LEIN.BAT, the lines causing problems are 91:

    Code:
    set _result=%_result:"=%
    and 95:

    Code:
      if "x%_result%" == "x" (set %_var%=%2) ELSE (set %_var%="%_result%")
    Line 91 confuses the TCC parser with the unmatched " in the command group, which causes the trailing ) to be ignored while TCC looks for a matching trailing ".

    The problem with line 95 is the "set %_var%=" syntax -- the %= is interpreted as an escape metacharacter (causing the following " to be ignored, causing the trailing " to be interpreted as a leading ", causing the trailing ) to be ignored again). Line 95 is apparently an attempt by the batch programmer to take advantage of the CMD parsing unfeature to export a variable from a SETLOCAL, something which is otherwise not supported in CMD. So that command group is constructed as:

    ( ENDLOCAL & if "x%_result%" == "x" (set %_var%=%2) ELSE (set %_var%="%_result%") )

    The variables on the line are expanded before ENDLOCAL is executed, and the IF is executed after ENDLOCAL has restored the environment, so the SET is outside the SETLOCAL / ENDLOCAL block. This is either a clever workaround for a CMD limitation, or an unholy alliance with the forces of darkness. Perhaps both ... :banghead:

    Fortunately, TCC doesn't require any of this unpleasantness, and it's simple to make the batch file work in both environments.

    Continued in the next message ...
     
  11. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,730
    Likes Received:
    80
    To make LEIN.BAT work in TCC, we need to change the ":EnsureIsSet" (pseudo)subroutine. What the programmer is apparently trying to do is remove double quotes from the argument, and then add surrounding double quotes if the argument is already defined. (Otherwise, just return the specified default value.)

    Code:
    :EnsureIsSet 
    rem parameters: Variable DefaultValue
    rem Variable's value can be empty, if it is then it will be set to the DefaultValue(which is not modified/stripped)
    rem it will strip all encountered double quotes from Variable and from Variable's value
    rem this script will fail if Variable contains characters like >, <, |, & or even parentheses or even number of double quotes
    rem the Variable's value will be surrounded by double quotes (no inner double quotes though, they're all stripped)
    rem  except in the case of DefaultValue which is set as it is
     
    SETLOCAL
    set _var=%~1
     
    rem Are we running TCC?
    if %@eval[1+1]==2 (
    set _result=%@unquote[%[%_var]]
    if "x%_result%" == "x" (set %_var=%2) ELSE (set %_var="%_result%")
    ENDLOCAL %_var
    goto :eof
    )
     
    call set _result=%%%_var%%%
     
    for /f "useback tokens=*" %%a in ('%_result%') do (
    set _result=%%~a
    set _result=%_result:"=%
    )
     
    ( ENDLOCAL
      if "x%_result%" == "x" (set %_var%=%2) ELSE (set %_var%="%_result%")
    )
     
    goto :eof
    
    (The TCC code could be smaller, but I was trying to make easily understandable.)

    The "if %@eval" statement is testing to see if TCC is running the script. If so, it removes the quotes from the argument, sets the result value, and exports that value from the SETLOCAL / ENDLOCAL block. If CMD is running the script, the "if %@eval" statement will fail and CMD will execute its original code (destroying a few more brain cells on the way).
     
  12. tmaynard

    Joined:
    May 2, 2012
    Messages:
    25
    Likes Received:
    0
    My apologies for the late response: other tasks assumed priority.

    However: I got a fresh copy of LEIN.BAT, deleted everything from :EnsureIsSet through goto EOF and replaced it with the code you supplied above, and finally saved it as LEINX.BAT.

    At least on my system, the new version still fails, namely

    TCC:
    [C:\Users\Tom]
    $ leinx version
    Usage: java [-options] class [args...]
    (to execute a class)
    or java [-options] -jar jarfile [args...]
    (to execute a jar file)
    where options include:
    -d32 use a 32-bit data model if available
    ...many lines elided for brevity...
    -splash:<imagepath>
    show splash screen with specified image
    See http://www.oracle.com/technetwork/java/javase/documentation/index.html for more details.
    TCC: C:\Users\Tom\bin\leinx.bat [104] Unknown command ")"
    TCC: C:\Users\Tom\bin\leinx.bat [104] Unknown command ")"
    TCC: C:\Users\Tom\bin\leinx.bat [270] Unknown command "-client"
    CMD:
    [C:\Users\Tom]
    $ leinx version
    The syntax of the command is incorrect.

    (The original LEIN.BAT still works properly under CMD.) Since I presume you tested the new version on your own system -- and that it worked, of course -- perhaps I am missing something?
     
  13. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,730
    Likes Received:
    80
    I'm attaching the LEIN.BAT I've been testing, which works under both TCC and CMD.
     

    Attached Files:

Share This Page