Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

"externally altered" error message

Peter Bratton

Peter Bratton

Jul
81
0
I'd appreciate some insight on the error message below. It started showing up after I installed v13, in btm files that had been running well, in some cases for years.

TCC: (Sys) C:\Btm\stat.btm [189] Le fichier ouvert n'est plus valide car le volume qui le contient a été endommagé de manière externe.
"%@lines[c:\wrk\stat\reg.txt]"
TCC: (Sys) C:\Btm\stat.btm [192] Le fichier ouvert n'est plus valide car le volume qui le contient a été endommagé de manière externe.
"%@line[c:\wrk\stat\reg.txt,4]"

TCC: (Sys) C:\Btm\stat.btm [189] The volume for a file has been externally altered so that the opened file is no longer valid.
"%@lines[c:\wrk\stat\reg.txt]"
TCC: (Sys) C:\Btm\stat.btm [192] The volume for a file has been externally altered so that the opened file is no longer valid.
"%@line[c:\wrk\stat\reg.txt,4]"

There is a pattern, the error pops up when a batch directs the output from some command to a file, then processes each line in the file. This snippet is representative, the "offending" lines are 189 and 192, corresponding to the messages quoted above:

Code: 
185 do root in @%wrk\registry.lst
186   set abbr=HK%@if[%@index[%root,_LOCAL_] ge 0,LM,CU]
187
188   reg query "%root" >&> nul > %wrk\reg.txt
189   set ll=%@lines[%wrk\reg.txt]
190   iff %ll ge 4 then
191     do cnt = 4 to %ll
192       set txt=%@line[%wrk\reg.txt,%cnt]
193       echo %abbr!%@trim[%@word["%=t",0,%txt]]!%@word["%=t",2-9,%txt] >> %wrk\df.txt
194     enddo
195   endiff
196 enddo

In passing, the English error message says "altered", the French says "damaged"
--
Peter
 
Peter Bratton said:
I'd appreciate some insight on the error message below. It started showing up after I installed v13, in btm files that had been running well, in some cases for years.

TCC: (Sys) C:\Btm\stat.btm [189] Le fichier ouvert n'est plus valide car le volume qui le contient a été endommagé de manière externe.
"%@lines[c:\wrk\stat\reg.txt]"
TCC: (Sys) C:\Btm\stat.btm [192] Le fichier ouvert n'est plus valide car le volume qui le contient a été endommagé de manière externe.
"%@line[c:\wrk\stat\reg.txt,4]"

TCC: (Sys) C:\Btm\stat.btm [189] The volume for a file has been externally altered so that the opened file is no longer valid.
"%@lines[c:\wrk\stat\reg.txt]"
TCC: (Sys) C:\Btm\stat.btm [192] The volume for a file has been externally altered so that the opened file is no longer valid.
"%@line[c:\wrk\stat\reg.txt,4]"

There is a pattern, the error pops up when a batch directs the output from some command to a file, then processes each line in the file. This snippet is representative, the "offending" lines are 189 and 192, corresponding to the messages quoted above:

Code: 
185 do root in @%wrk\registry.lst
186   set abbr=HK%@if[%@index[%root,_LOCAL_] ge 0,LM,CU]
187
188   reg query "%root" >&> nul > %wrk\reg.txt
189   set ll=%@lines[%wrk\reg.txt]
190   iff %ll ge 4 then
191     do cnt = 4 to %ll
192       set txt=%@line[%wrk\reg.txt,%cnt]
193       echo %abbr!%@trim[%@word["%=t",0,%txt]]!%@word["%=t",2-9,%txt] >> %wrk\df.txt
194     enddo
195   endiff
196 enddo
In passing, the English error message says "altered", the French says "damaged"
--
Peter
Click to expand...

I've been getting the same error using @LINE and @LINES. I've been trying to produce the error in isolation from the batch files where it occurs but haven't succeeded in isolating it yet. From an interactive session, the error doesn't seem to occur; it only seems to occur in batch files (both .BAT and .BTM).

Under TCC v12 (any build) this does not happen, but in TCC v13, it does.
 
I don't know what's happening but it's easy enough to reproduce.

Code: 
v:\> echos > empty.txt

v:\> echo %@lines[empty.txt]
TCC: (Sys) The volume for a file has been externally altered so that the opened
file is no longer valid.
 "%@lines[empty.txt]"

v:\>
That seems to happen whenever redirection results in a 0-byte file.

While Rex should look into this, the OP should check his REG command.
 
On Tue, 04 Oct 2011 22:02:43 -0400, vefatica <> wrote:

|That seems to happen whenever redirection results in a 0-byte file.

Actually, @LINES seems to do that with any 0-byte file. This one was created
months ago:

Code: 
v:\> echo %@filesize[cwatch.log]
0

v:\> echo %@lines[cwatch.log]
TCC: (Sys) The volume for a file has been externally altered so that the opened
file is no longer valid.
 "%@lines[cwatch.log]"

v:\>
 
And (coincidentally?) my plugin LC (linecount) has a problem with 0-byte files.

Code: 
v:\> lc cwatch.log
TCC: (Sys) Invalid access to memory location.
 "LC"

v:\>

My problem (I think) is trying to read the character before EOF to see if it's a
newline. Rex, I'm curious about the problem with @LINES.

On Tue, 04 Oct 2011 22:16:10 -0400, vefatica <> wrote:

||That seems to happen whenever redirection results in a 0-byte file.
|
|Actually, @LINES seems to do that with any 0-byte file. This one was created
|months ago:
|
|
|Code:
|---------
|v:\> echo %@filesize[cwatch.log]
|0
|
|v:\> echo %@lines[cwatch.log]
|TCC: (Sys) The volume for a file has been externally altered so that the opened
|file is no longer valid.
| "%@lines[cwatch.log]"
|
|v:\>
|---------
 
vefatica said:
I don't know what's happening but it's easy enough to reproduce.

Code: 
v:\> echos > empty.txt

v:\> echo %@lines[empty.txt]
TCC: (Sys) The volume for a file has been externally altered so that the opened
file is no longer valid.
 "%@lines[empty.txt]"

v:\>
That seems to happen whenever redirection results in a 0-byte file.

While Rex should look into this, the OP should check his REG command.
Click to expand...

Thanks for narrowing that down, Vince. Fortunately for me, I'm testing for zero or more so I can work around this by using @FILESIZE, but will have to test for existence with IF EXIST first.
 
vefatica said:
I don't know what's happening but it's easy enough to reproduce.

Code: 
v:\> echos > empty.txt

... the OP should check his REG command.[/QUOTE]
reg query always produces at least a file header, so there is never an empty output file.  Moreover I can run the batch from the debugger and view the contents of the output file immediately after the first error is reported.  And this doesn't change the fact that the batch has worked well as is under several previous versions of TCC.
--
Peter
Click to expand...
 
dcantor said:
Thanks for narrowing that down, Vince. Fortunately for me, I'm testing for zero or more so I can work around this by using @FILESIZE, but will have to test for existence with IF EXIST first.
Click to expand...

Adding a pause as short as 1 millisecond right before the @lines[] function call quashes the error, and the function returns an expected value. Presumably that's enough time for the redirection to complete and its output file to become accessible.

The redirection change and the message are definitely new to TCC v13. I think the function should return -1 for a zero-byte file, as it used to, instead of an error message.

Thanks guys for your insights
--
Peter
 
On Wed, 05 Oct 2011 20:09:46 -0400, Peter Bratton <> wrote:

|---Quote (Originally by vefatica)---
|I don't know what's happening but it's easy enough to reproduce.
|
|
Code: 
v:\> echos > empty.txt
|
|... the OP should check his REG command.
|---End Quote---
|reg query always produces at least a file header, so there is never an empty output file.  Moreover I can run the batch from the debugger and view the contents of the output file immediately after the first error is reported.  And this doesn't change the fact that the batch has worked well as is under several previous versions of TCC.

Well, the error in question (1006) is a very odd one.  I have no idea how to
cause it using "C" or with a plugin using the functions in the TCC API.  But
being reproducible (@LINES[0-byte file]) will let Rex track it down.  As you
say, it happens in other circumstances too.  I'm not surprised ... just really
curious about what causes it.
 
On Wed, 05 Oct 2011 21:17:23 -0400, Peter Bratton <> wrote:

|Adding a pause as short as 1 millisecond right before the @lines[] function call quashes the error, and the function returns an expected value. Presumably that's enough time for the redirection to complete and its output file to become accessible.

I am very surprised. In a later post I pointed out that it seemed to have
nothing to do with the redirection (or being in a batch file). Right at the
command line, using a previously-existing 0-byte file, I get

Code: 
v:\> echo %@filesize[cwatch.log]
0

v:\> echo %@lines[cwatch.log]
TCC: (Sys) The volume for a file has been externally altered so that the opened
file is no longer valid.
 "%@lines[cwatch.log]"
 
From: vefatica
| Peter Bratton wrote:
|
|| Adding a pause as short as 1 millisecond right before the @lines[]
|| function call quashes the error, and the function returns an
|| expected value. Presumably that's enough time for the redirection to
|| complete and its output file to become accessible.
|
| I am very surprised. In a later post I pointed out that it seemed to have
| nothing to do with the redirection (or being in a batch file). Right at the
| command line, using a previously-existing 0-byte file, I get
|
|
| Code:
| v:\> echo %@filesize[cwatch.log]
| 0
|
| v:\> echo %@lines[cwatch.log]
| TCC: (Sys) The volume for a file has been externally altered so that
| the opened file is no longer valid.
| %@lines[cwatch.log]

Same here. Tested on years-old empty file. V8 and V12 return -1, as expected, using the same command line.
--
Steve
 
On Wed, 05 Oct 2011 23:45:11 -0400, rconn <> wrote:

|---Quote---
|> Well, the error in question (1006) is a very odd one.
|---End Quote---
|The error is returned by CreateFileMapping, which fails if the file size is
|0. (Probably not unreasonably, but it's undocumented behavior.)
|
|I've added a minor kludge in build 25 to test for a 0-byte file.

Thanks for the explanation.

I read a lot of Google hits for "1006" and the text of the message but didn't
read far enough to find any about CreateFileMapping. Many dealt with particular
pieces of software, Python, installers, and USB/network drives.

Given the chance I would have done it too. I'll have to look through my
plugins.
 
From: rconn
| The error is returned by CreateFileMapping, which fails if the file size is
| 0. (Probably not unreasonably, but it's undocumented behavior.)

I think it is unreasonable unless the mapping is created in read-only mode. However, I guess @LINE[] and @LINES[] would create a read-only map (and probably not with exclusive access), so in the instance it is reasonable. Documentation? Standard for MS.

I presume use of this API call to implement @LINE[] and @LINES[] is new for V13, and is the reason for their faster operation.

| I've added a minor kludge in build 25 to test for a 0-byte file.

Thanks for the quick fix. Expected from no other vendor but you.

I presume the new test also applies to @LINES[], which in V12 returns ***EOF***, but in V13 the same error message for an empty file.
--
Steve
 
rconn said:
Not so - nonexistent files have always returned an error. -1 is only
returned for empty files.
Click to expand...

Yes, sorry. I realized that a few minutes after I posted it and I edited my post to remove that statement a few minutes after I wrote it.

-1 for empty files.
 
On Thu, 06 Oct 2011 11:03:43 -0400, Steve Fabian <> wrote:

|From: rconn
|| The error is returned by CreateFileMapping, which fails if the file size is
|| 0. (Probably not unreasonably, but it's undocumented behavior.)
|
|I think it is unreasonable unless the mapping is created in read-only mode. However, I guess @LINE[] and @LINES[] would create a read-only map (and probably not with exclusive access), so in the instance it is reasonable. Documentation? Standard for MS.

I wonder about the speed advantage. In a quick test comparing @LINES to my @LC
(line count, which does not use a MM file) they performed the same. The test
was flawed, though, because it used the same collection of files repeatedly ...
the files were probably cached (Win7 is cache-crazy!), and after that, all
operations will be fast.

I have in mind to more carefully devise a statistical test. My thoughts are
(suggestions welcome) ... With DIR /B create a large list of text files (here,
g:\*.cpp;*.h ought to come up with a few thousand). Then time routines (for
%LINES and %LC) which randomly select files from that list for line counting;
they should generally be uncached.

But there are other considerations. One is simply resolving the name of the
function (%LINES vs. @LC). I suspect TCC looks through the names of plugin
functions before looking through the names of built-in functions (right, Rex?).
This could affect the results. Another thing I have always wanted to ask Rex:
When trying to resolve the name of something (_variable, @function, or command
... internal or plugin, alias, UDF) is the search a linear one, a binary one,
something else?

Long ago I found that BTM's run significantly faster if you unset all
unnecessary variables, unalias all unnecessary aliases, unfunction any
unnecessary UDFs, unload all unneseccary plugins, and use very short names for
environment variables. All of these must affect the speed at which TCC resolves
the names of things.
 
vefatica said:
I suspect TCC looks through the names of plugin
functions before looking through the names of built-in functions (right, Rex?).
Click to expand...

Yes. And TCC also does a lot of generic processing of the arguments before passing it to the specific @ function. (This slows things down slightly but vastly reduces the amount of code necessary.

Another thing I have always wanted to ask Rex:
When trying to resolve the name of something (_variable, @function, or command ... internal or plugin, alias, UDF) is the search a linear one, a binary one, something else?
Click to expand...

Command, variable, and @function are all binary searches. The alias and UDF searches are sequential (can't really do it any other way without the overhead making it slower anyway).

Plugin searches are sequential based on the name passed by the plugin. (Again, it can't be done any other way without requiring the plugin authors to order everything.)

Long ago I found that BTM's run significantly faster if you unset all unnecessary variables, unalias all unnecessary aliases, unfunction any
unnecessary UDFs, unload all unneseccary plugins, and use very short names for environment variables.
Click to expand...

True, but that's becoming less relevant over time with the increases in system speeds.
 
From: vefatica
| I wonder about the speed advantage. In a quick test comparing @LINES to my @LC
| (line count, which does not use a MM file) they performed the same. The test
| was flawed, though, because it used the same collection of files repeatedly ...
| the files were probably cached (Win7 is cache-crazy!), and after that, all
| operations will be fast.
|
| I have in mind to more carefully devise a statistical test. My thoughts are
| (suggestions welcome) ... With DIR /B create a large list of text files (here,
| g:\*.cpp;*.h ought to come up with a few thousand). Then time routines (for
| %LINES and %LC) which randomly select files from that list for line counting;
| they should generally be uncached.

Caching and release from cache are often not controllable, esp. that done by the disk drives themselves. One alternative is to use files so big they will not fit into all your internal memory (and onboard disk cache) together, and accessing locations too far from one another in the file to be read in one access. This would mean multigigabyte size! The other alternative is to use files already cached because of prior access. This would mean the execution time of the first call to each function (@LINE and @LC) is not counted, only the subsequent several hundred execution times. E.g.:

set dummy=%@lines[xxx]
timer on /1
do 1000 (set dummy=%@lines[xxx])
set lines=%@timer[1,s]
timer off /1
set dummy=%@lc[xxx]
timer on /1
do 1000 (set dummy=%@lc[xxx])
set lc=%@timer[1,s]
timer off /1
echo LINES: %lines s
echo LC: %lc s

| But there are other considerations. One is simply resolving the name of the
| function (%LINES vs. @LC). I suspect TCC looks through the names of plugin
| functions before looking through the names of built-in functions (right, Rex?).
| This could affect the results. Another thing I have always wanted to ask Rex:
| When trying to resolve the name of something (_variable, @function, or command
| ... internal or plugin, alias, UDF) is the search a linear one, a binary one,
| something else?

The fastest search is hash search, esp. when the list is constant, e.g., internals, when a perfect hash is possible, thus totally avoiding collisions. In such case the hash algorithm is unique to the table, but the computed hash code is the exact index into the table for the string (though you must still verify the input string matching the table entry to eliminate invalid strings): 1 match is all. Of course, whenever the list changes (e.g., a command is added), the perfect hash algorithm itself changes, and thus the order of entires in table itself. OTOH a single table for all internals (commands, functions and environment variables) would suffice. Programs for developing a perfect hash algorithm automatically exist. For a random set of strings with possible collisions the average is 2.5 matches if the table is not more than half full. See Donald Knuth: The Art of Computer Programming, vol. 3, Sorting and Searching.

I suspect that if Rex uses the Windows API version of the Standard-C library function getenv() for resolving environmental variables, it ends up with a linear search - MS is not known for speedy algorithms.

| Long ago I found that BTM's run significantly faster if you unset all
| unnecessary variables, unalias all unnecessary aliases, unfunction any
| unnecessary UDFs, unload all unneseccary plugins, and use very short
| names for environment variables. All of these must affect the speed at
| which TCC resolves the names of things.

I suspect the presence or absence of a category of non-built-ins is more important than the number of members in the category.

BTW, just as the asterisk (*) prefix for a command name eliminates alias search, it would be useful to have a syntax that selectively restricts function, variable and command searches to built-ins only (ignore plugins and UD entries), or to built-ins and plugins only (ignore aliases and UDFs). OTOH it would be extremely useful to allow UDFs which currently require executing a batch file (typically via @EXECSTR[]) to be able to load the batch file into the UDF space, as is done in some *nix shells.

NOTE: This post was almost complete when an area-wide power loss killed the original version a while ago. Some of its contents may no longer be current.
--
Steve
 
vefatica said:
I wonder about the speed advantage. In a quick test comparing @LINES to my @LC (line count, which does not use a MM file) they performed the same.
Click to expand...

If you want a fairer test, in addition to all of the preprocessing the parser does for the @ functions, you'll also need to add support to @LC for Unicode files, CON:, and CLIP: (all of which @LINE / @LINES support).
 
vefatica said:
On Wed, 05 Oct 2011 21:17:23 -0400, Peter Bratton <> wrote:

|Adding a pause as short as 1 millisecond right before the @lines[] function call quashes the error, and the function returns an expected value. Presumably that's enough time for the redirection to complete and its output file to become accessible.

I am very surprised. In a later post I pointed out that it seemed to have
nothing to do with the redirection (or being in a batch file).
Click to expand...
Correct. The @lines[] function behaves differently with no redirection in the mix, to wit
Code: 
touch /c file.txt
echo %@lines[file.txt]
triggers the error.

But I think redirection is behaving differently too. In previous versions of TCC, there was no need for a pause. I tried replacing the pause with a couple of other commands (unsetarray, setarray, echo something), the code still produced the error, so it seems the pause is needed.
--
Peter
 
You must log in or register to reply here.
Back
Top
[FOX] Ultimate Translator
Translate