BDEBUGGER vs not inconsistency

#1
It's almost 2am, so maybe I'm seeing something, but something weird is happening whether I run a BTM in BDEBUGGER.

I'm migrating some scripts from a 2008 R2 TC 15 server that I am decommissioning and am testing the scripts on a new 2012R2 system running TC 16, this BTM previously worked on v15, but seems to fail oddly in v16.

TCC 16.00.41 x64 Windows Server 2012R2 [Version 6.3.9600]
TCC Build 41 Windows Server 2012R2 Build 9600

Code:
set regex=\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$
set pdffile=4000000000000000-2014Jan21-2014Feb20.pdf

echo %%@regex[%%[regex],%%[pdffile]]
echo %%@regex[%[regex],%[pdffile]]
echo Regex: %@regex[%[regex],%[pdffile]]

set regexout=%@regex[%[regex],%[pdffile]]
set regexout
When run in BDEBUGGER:

%@regex[%[regex],%[pdffile]]
%@regex[\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$,4000000000000000-2014Jan21-2014Feb20.pdf]
TCC: C:\Users\DAVEAD~1\AppData\Local\Temp\test.btm [13] Not in environment "regexout*
When run in TCMD directly:

%@regex[%[regex],%[pdffile]]
%@regex[\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$,4000000000000000-2014Jan21-2014Feb20.pdf]
Regex: 1
1
When I step through in BDEBUGGER, it completely skips over the %@regex lines. Even weirder, if I comment out the UNSET and SET commands at the top and leave the regex and pdffile variables set from a previous execution, the output of BDEUBGGER seems to be correct.

What am I missing here?

(And yes, the regex is a bit more complicated than it needs to be -- I intend on extracting the various groups and doing further matches once I determine whether we have a match or not)
 

rconn

Administrator
Staff member
May 14, 2008
10,742
97
#2
Not reproducible here, at least using the snippet you provided.

BDEBUGGER is running the same code as TCC, so it would be surprising if it behaved differently. However, you might be invoking BDEBUGGER differently -- are you using a custom TCMD.INI configuration in your TCC window that isn't also being used in your BDEBUGGER window?
 
#3
Here (Win7/32/SP1) the very same code either crashes IDE (quite randomly, details below) or it runs correctly. The most likely way for me to get it to crash IDE is to start a new TCC, "BDEBUGGER ...", then "Start without debugging".

Code:
TCC  16.00.41
Module=G:\TC16\IDE.EXE
Address=00E10155
Exception=C0000005
EAX=00000000  EBX=0F87A610  ECX=033B9DF8  EDX=0273F2D8
ESI=0F883AD0  EDI=7612B225  EBP=FFFFFFFF  ESP=0273F2C4
CS=0000001B  DS=00000023  ES=00000023  SS=00000023
Flags=00010246

Stack:
1 :  0000:00000000
Code:
Stack:
1 : IDE.EXE 0001:0000f155
Problem signature:
  Problem Event Name:   BEX
  Application Name:   IDE.EXE
  Application Version:   16.0.41.0
  Application Timestamp:   52eaf9c0
  Fault Module Name:   StackHash_0864
  Fault Module Version:   0.0.0.0
  Fault Module Timestamp:   00000000
  Exception Offset:   e07010ec
  Exception Code:   c0000005
  Exception Data:   00000008
  OS Version:   6.1.7601.2.1.0.256.48
  Locale ID:   1033
  Additional Information 1:   0864
  Additional Information 2:   0864d224a5c100309cff114a52297350
  Additional Information 3:   4885
  Additional Information 4:   48852ebe81dde95428a2a2cf5536c40f
Here's another one in which case I started a new TCC, "BDEBUGGER ...", and pressed "Start".
Code:
TCC  16.00.41
Module=G:\TC16\IDE.EXE
Address=012E7C36
Exception=C0000005
EAX=FFFFFFF0  EBX=0BA5BD20  ECX=00000000  EDX=777F70F4
ESI=FFFFFFF0  EDI=00F23A08  EBP=02BCF0F8  ESP=02BCF0E0
CS=0000001B  DS=00000023  ES=00000023  SS=00000023
Flags=00010287

Stack:
1 : IDE.EXE 0001:00006c36
2 : IDE.EXE 0001:0015c269
 
#4
Not reproducible here, at least using the snippet you provided.

BDEBUGGER is running the same code as TCC, so it would be surprising if it behaved differently. However, you might be invoking BDEBUGGER differently -- are you using a custom TCMD.INI configuration in your TCC window that isn't also being used in your BDEBUGGER window?
The snippet is the complete BTM that I'm using for testing (with an @ECHO OFF, although I dropped near the end -- The missing command literally never shows up, nor does any other command containing that @regex. In the original BTM, it was part of a complicated IFF statement which was skipped completely, causing one of the conditions to run before an error was thrown, but I reduced the code to just the offending @REGEX and built up a sample that was code-complete and 100% reproducible on two machines here), I'm running in one window, one time prepending the BTM with "BDEBUGGER" and the next, without, in an otherwise identical environment.

Try adding "unset regex pdffile regexout" to both the top and bottom of your BTM, this will ensure that the variable doesn't persist from one execution to another, which seems to make a difference in BDEBUGGER (in other words, if you already ran this BTM from the command line before firing up BDEBUGGER, and the variables are already set, you won't be able to reproduce a problem)
 
#5
I can reproduce those GPF (with equal ease/difficulty/randomness) with other BTMs. I used a shortcut like this (below, also with Ctrl-F5) to test. The timing is not critical; I've used KEYSTACK waits from 3 to 36 with similar results (i.e., IDE randomly crashes). REGGIE.BTM is exactly thedave's BTM.
Code:
c:\users\vefatica\desktop> shortcut TCC16.lnk
Command=G:\TC16\tcc.exe
Arguments=/q keystack /w36 f5 & bdebugger reggie.btm
Directory=v:\
Description=
Link=TCC16.lnk
Icon=
Offset=0
Mode=1
Hotkey=
 
#6
It's not just that one regex either, I moved on to validating additional BTMs, and found that "iff %@regex["\d+\.\d+\.\d+","%version%"] eq 1 then" also fails, where %version% is 2.0.9 or similar.

So it's not specific to that one regex, but it seems to happy with many/most @REGEX commands.
 

rconn

Administrator
Staff member
May 14, 2008
10,742
97
#7
Here (Win7/32/SP1) the very same code either crashes IDE (quite randomly, details below) or it runs correctly. The most likely way for me to get it to crash IDE is to start a new TCC, "BDEBUGGER ...", then "Start without debugging".
Not reproducible here.

Your first crash is in the "On Tabify" function (which seems unlikely); your second one is in the "Save All Modified Files" function (which seems more unlikely).
 
#8
Not reproducible here.

Your first crash is in the "On Tabify" function (which seems unlikely); your second one is in the "Save All Modified Files" function (which seems more unlikely).
I can't explain it. I ran the shortcut below 20 times and it GPF'd 6 times, each time prodicing the GPF file further below.
Code:
c:\users\vefatica\desktop> shortcut TCC16.lnk
Command=G:\TC16\tcc.exe
Arguments=/q keystack /w36 f5 & bdebugger v:\reggie.btm
Directory=v:\
Description=
Link=TCC16.lnk
Icon=
Offset=0
Mode=1
Code:
TCC  16.00.41
Module=G:\TC16\IDE.EXE
Address=011E7C36
Exception=C0000005
EAX=FFFFFFF0  EBX=0F5B0248  ECX=00000000  EDX=777F70F4
ESI=FFFFFFF0  EDI=00D9D9F0  EBP=02A9F274  ESP=02A9F25C
CS=0000001B  DS=00000023  ES=00000023  SS=00000023
Flags=00010287

Stack:
1 : IDE.EXE 0001:00006c36
2 : IDE.EXE 0001:0015c269
 
#9
Not to harp on this point too much, but I finished validating my scripts operate normally outside of BDEBUGGER by using the tried-and-true method of turning any dangerous command into a ECHO, and they work perfectly. However, as soon as I run them within BDEBUGGER, nearly every line with a @REGEX is skipped, which of course causes havoc when it's a IFF...THEN statement.

I don't see a crash, but is it possible that 32-bit crashes while 64-bit just skips the offending line?
 

rconn

Administrator
Staff member
May 14, 2008
10,742
97
#10
Try adding "unset regex pdffile regexout" to both the top and bottom of your BTM, this will ensure that the variable doesn't persist from one execution to another, which seems to make a difference in BDEBUGGER (in other words, if you already ran this BTM from the command line before firing up BDEBUGGER, and the variables are already set, you won't be able to reproduce a problem)
Still not reproducible.

I created this batch file from your example:

Code:
unset regex pdffile regexout
set regex=\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$
set pdffile=4000000000000000-2014Jan21-2014Feb20.pdf
echo %%@regex[%%[regex],%%[pdffile]]
echo %%@regex[%[regex],%[pdffile]]
echo Regex: %@regex[%[regex],%[pdffile]]
set regexout=%@regex[%[regex],%[pdffile]]
set regexout
unset regex pdffile regexout
and ran it 20 times without any errors.

Are you loading any plugins? Any aliases?
 

rconn

Administrator
Staff member
May 14, 2008
10,742
97
#11
I can't explain it. I ran the shortcut below 20 times and it GPF'd 6 times, each time prodicing the GPF file further below.
Still not reproducible here. I ran your shortcut 50 times and it never generated an error or crashed.

The crash you're reporting now (which appears totally unrelated to the crashes you were reporting earlier?) is in the message processing loop when TCC is notified that the user is closing the app. (And apparently the global variable list is trashed.)

I doubt that such a major problem as you're reporting (with any batch file!) would have gone unnoticed by everybody else, so it's likely to be something unique to your system. Did you try it without any plugins, TCSTART, and TCEXIT?
 
Last edited:
#12
Are you loading any plugins? Any aliases?
Code:
@ECHO OFF
dir "%_ininame" "%_tcstart"
alias
plugin

unset regex pdffile regexout

set regex=\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$
set pdffile=4000000000000000-2014Jan21-2014Feb20.pdf
echo %%@regex[%%[regex],%%[pdffile]]
echo %%@regex[%[regex],%[pdffile]]
echo Regex: %@regex[%[regex],%[pdffile]]
set regexout=%@regex[%[regex],%[pdffile]]
set regexout
No aliases, no INI, no TCSTART, nothing:

TCC: (Sys) C:\Users\daveadmin\AppData\Local\Temp\test.btm [2] The system cannot find the file specified.
""
TCC: (Sys) C:\Users\daveadmin\AppData\Local\Temp\test.btm [2] The system cannot find the file specified.
""
TCC: C:\Users\daveadmin\AppData\Local\Temp\test.btm [3] No aliases defined
TCC: C:\Users\daveadmin\AppData\Local\Temp\test.btm [4] No plugins loaded
%@regex[%[regex],%[pdffile]]
%@regex[\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$,4000000000000000-2014Jan21-2014Feb20.pdf]
TCC: C:\Users\daveadmin\AppData\Local\Temp\test.btm [14] Not in environment "regexout*"
No antivirus or anything else of particular interest on this system, it's an almost fresh Windows 2012 R2 x64 server that was just built within the last few weeks, so it's a nice clean environment, running as an elevated administrator.
 
#13
Not reproducible here.

Your first crash is in the "On Tabify" function (which seems unlikely); your second one is in the "Save All Modified Files" function (which seems more unlikely).
It happens (identically) with no plugins (all renamed *.DLX) and no automatic BTMs. It never happens on my computer at work. When I see that discrepancy, I think multiprocessor (home) vs. uniprocessor (work) but I never found solid evidence of any relationship.
 

rconn

Administrator
Staff member
May 14, 2008
10,742
97
#14
Code:
@ECHO OFF
dir "%_ininame" "%_tcstart"
alias
plugin

unset regex pdffile regexout

set regex=\A(\d+)-\d\d\d\d\w\w\w\d\d-(\d\d\d\d)(\w\w\w)(\d\d)\.pdf$
set pdffile=4000000000000000-2014Jan21-2014Feb20.pdf
echo %%@regex[%%[regex],%%[pdffile]]
echo %%@regex[%[regex],%[pdffile]]
echo Regex: %@regex[%[regex],%[pdffile]]
set regexout=%@regex[%[regex],%[pdffile]]
set regexout
The problem you're experiencing is because your regular expression is causing the regex parser (Oniguruma) to throw an internal exception. The exception is caught and the line processing canceled -- which is why it appears that those two lines are being skipped. They're not, they're just not completing.

I don't know yet if the problem is with your regular expression or something internal to Oniguruma, but the parser ought to display an error message there. It was never added because nobody had managed to make Oniguruma choke until now.