Unicode screw-up in IDE

May 20, 2008
12,175
133
Syracuse, NY, USA
Try this.

ECHO foo > foofoo.bat
BDEBUGGER foofoo.bat

In the editor, I see 5 "unprintables".

If I change that experiment to

ECHO foooooooooooooo > foofoo.bat
BDEBUGGER foofoo.bat

it appears correctly in the editor. I'm guessing that in the first case, the file is wrongly identified as Unicode (which is odd since QueryIsFileUnicode() seems to get it right).
 
May 20, 2008
12,175
133
Syracuse, NY, USA
Sorry, make that experiment

Code:
ECHO echo foo > foofoo.bat
bdebugger v:\test.bat
Try this.

ECHO foo > foofoo.bat
BDEBUGGER foofoo.bat

In the editor, I see 5 "unprintables".

If I change that experiment to

ECHO foooooooooooooo > foofoo.bat
BDEBUGGER foofoo.bat

it appears correctly in the editor. I'm guessing that in the first case, the file is wrongly identified as Unicode (which is odd since QueryIsFileUnicode() seems to get it right).
 

rconn

Administrator
Staff member
May 14, 2008
12,557
167
Try this.

ECHO foo > foofoo.bat
BDEBUGGER foofoo.bat

In the editor, I see 5 "unprintables".

If I change that experiment to

ECHO foooooooooooooo > foofoo.bat
BDEBUGGER foofoo.bat

it appears correctly in the editor. I'm guessing that in the first case, the file is wrongly identified as Unicode (which is odd since QueryIsFileUnicode() seems to get it right).

The debugger uses QueryIsFileUnicode to determine if the file is Unicode. As we've discussed several times in the past, very short (and in this case useless?) files are not reliably determined by the Windows IsTextUnicode API.
 
May 20, 2008
12,175
133
Syracuse, NY, USA
On Wed, 30 Dec 2009 20:04:03 -0600, rconn <> wrote:

|The debugger uses QueryIsFileUnicode to determine if the file is Unicode. As we've discussed several times in the past, very short (and in this case useless?) files are not reliably determined by the Windows IsTextUnicode API.

Did you see the corrected test? ... "ECHO echo foo > foofoo.bat".

I have a plugin FCONVERT (to/from Unicode). It only test is QueryIsFileUnicode,
and it apparently gets it right (while the IDE apparently doesn't). It would
seem there has to be more to the story.

v:\> echo echo foo > test.bat

v:\> type /x test.bat
0000 0000 65 63 68 6f 20 66 6f 6f 0d 0a echo foo..

v:\> fconvert test.bat testu.bat

v:\> type /x testu.bat
0000 0000 65 00 63 00 68 00 6f 00 20 00 66 00 6f 00 6f 00 e c h o f o o
0000 0010 0d 00 0a 00 . .

v:\> fconvert testu.bat test2.bat

v:\> type /x test2.bat
0000 0000 65 63 68 6f 20 66 6f 6f 0d 0a echo foo..
--
- Vince
 

rconn

Administrator
Staff member
May 14, 2008
12,557
167
> Did you see the corrected test? ... "ECHO echo foo > foofoo.bat".

Yes.


> I have a plugin FCONVERT (to/from Unicode). It only test is
> QueryIsFileUnicode,
> and it apparently gets it right (while the IDE apparently doesn't). It
> would seem there has to be more to the story.

Trust me, the IDE calls QueryIsFileUnicode. But your results are guaranteed
(by MS) to be highly erratic for anything < 16 bytes.

Having a CR/LF on the end of the file will change everything.

Rex Conn
JP Software
 

Similar threads