unzip issues in v12

[nikbackm]

zip/unzip issues in v12

TCC 12,00,35 Windows Vista [Version 6,0,6002]

Seems to be two problems, but it might be me misunderstanding how to use UNZIP.

First, unzip does not ask before overwriting files, which it should according to the documentation. Second, UNZIP /D does not restore the directory structure correctly. Sub-directories are not unzipped even if the output says so).

If you specify /D then UNZIP does ask before overwriting first, but not when run with only the zipfile as argument.

A question as well, is there some way to test the CRC of the generated zip-file (equivalent of "7z t" for example)?


Example test run:

20:41 (2010-11-13) C:\Users\niklas\test>dir /s /b
C:\Users\niklas\test\aa
C:\Users\niklas\test\aa\bb
C:\Users\niklas\test\aa\f1
C:\Users\niklas\test\aa\f2
C:\Users\niklas\test\aa\bb\f3
C:\Users\niklas\test\aa\bb\f4

20:41 (2010-11-13) C:\Users\niklas\test>zip /r testzip.zip aa
<= C:\Users\niklas\test\aa\*

20:41 (2010-11-13) C:\Users\niklas\test>zip /v testzip.zip
2010-11-13  20:41              1 100%  aa\
2010-11-13  20:41              1 100%  aa\bb\
2010-11-13  20:41             13 -15%  aa\bb\f3
2010-11-13  20:41             13 -15%  aa\bb\f4
2010-11-13  20:41             13 -15%  aa\f1
2010-11-13  20:41             13 -15%  aa\f2

20:41 (2010-11-13) C:\Users\niklas\test>unzip testzip.zip
=>
=>
=> C:\Users\niklas\test\f3
=> C:\Users\niklas\test\f4
=> C:\Users\niklas\test\f1
=> C:\Users\niklas\test\f2

20:42 (2010-11-13) C:\Users\niklas\test>unzip /d testzip.zip
=> C:\Users\niklas\test\aa\
=> C:\Users\niklas\test\aa\bb\
C:\Users\niklas\test\aa\bb\f3 (Replace) (Y/N/A/R)? Y
=> C:\Users\niklas\test\aa\bb\f3
C:\Users\niklas\test\aa\bb\f4 (Replace) (Y/N/A/R)? Y
=> C:\Users\niklas\test\aa\bb\f4
C:\Users\niklas\test\aa\f1 (Replace) (Y/N/A/R)? Y
=> C:\Users\niklas\test\aa\f1
C:\Users\niklas\test\aa\f2 (Replace) (Y/N/A/R)? Y
=> C:\Users\niklas\test\aa\f2
Indexing C:\Users\niklas\test

20:42 (2010-11-13) C:\Users\niklas\test>deltree aa
Delete C:\Users\niklas\test\aa\f1 (Y/N/A/R)? R
Deleting C:\Users\niklas\test\aa\f2
Deleting C:\Users\niklas\test\aa\bb\f3
Deleting C:\Users\niklas\test\aa\bb\f4
Removing C:\Users\niklas\test\aa\bb\
Removing C:\Users\niklas\test\aa\
     4 files deleted              4 096 bytes freed

20:42 (2010-11-13) C:\Users\niklas\test>unzip /d testzip.zip
=> C:\Users\niklas\test\aa\
=> C:\Users\niklas\test\aa\bb\
=> C:\Users\niklas\test\aa\bb\f3
=> C:\Users\niklas\test\aa\bb\f4
=> C:\Users\niklas\test\aa\f1
=> C:\Users\niklas\test\aa\f2
Indexing C:\Users\niklas\test

20:42 (2010-11-13) C:\Users\niklas\test>dir aa

 Volume in drive C is unlabeled      Serial number is 28b9:6305
 Directory of  C:\Users\niklas\test\aa\*

13.11.2010  20:42         <dir>    .
13.11.2010  20:42         <dir>    ..
13.11.2010  20:41              13  f1
13.11.2010  20:41              13  f2
                26 bytes in 2 files and 2 dirs    8 192 bytes allocated
   176 564 928 512 bytes free

20:43 (2010-11-13) C:\Users\niklas\test>

</dir></dir>

 

[nikbackm]

Also some ZIP issues

If you specify ZIP /U you get an error if the zip-file does not exists.

21:15 (2010-11-13) C:\Users\niklas\test>zip /U test2.zip f*
TCC: Error in archive C:\Users\niklas\test\test2.zip :
  Could not open archive file "C:\Users\niklas\test\test2.zip". (error 0x2).

21:15 (2010-11-13) C:\Users\niklas\test>zip /a test2.zip f*
<= C:\Users\niklas\test\f1
<= C:\Users\niklas\test\f12

It would be preferable if ZIP simply behaved as if /A had been specified in this case. That is how most other zippers I've used does it. Having to check if the zip-file exists first and then using either /U or /A seems extra work for no benefit. (The target zip-file might be a regular backup-target file, and deleting it is a way to remove old data. This should not cause backup-scripts using /U to break the next time they are run)

Specifying ZIP /U /R causes ZIP to needlessly complain about sub-directories.
And it seems the files in the sub-directory (old in this case) is not updated to the target zipfile either.

echo Backing up batch files
zip /U /R d:\backup\batchfiles.zip "c:\bin\bat\*" 

Backing up batch files
<= C:\bin\bat\backup_jpsoft.btm
<= C:\bin\bat\old\*
TCC: C:\bin\bat\backup_jpsoft.btm [15]  Error in archive D:\backup\batchfiles.zip :
  File already exists in the archive. "C:\bin\bat\old"
<= C:\bin\bat\old_getcbrtvtoclip.btm

 

[rconn]

TCC 12,00,35 Windows Vista [Version 6,0,6002]

Seems to be two problems, but it might be me misunderstanding how to use UNZIP.

First, unzip does not ask before overwriting files, which it should according to the documentation. Second, UNZIP /D does not restore the directory structure correctly. Sub-directories are not unzipped even if the output says so).

If you specify /D then UNZIP does ask before overwriting first, but not when run with only the zipfile as argument.

A question as well, is there some way to test the CRC of the generated zip-file (equivalent of "7z t" for example)?

In your first example, UNZIP isn't asking because the files are all being put in the current ("test") directory, not the subdirectories (because you didn't specify /D) -- so nothing is actually being overwritten.


The /D is creating the first subdirectory but not the second; this appears to be a bug in the zip library. I'll pass it on to the developers and let you know what they say.

I don't know what you're looking for regarding the CRC -- can you expand a bit?

 

[nikbackm]

(I include a listing as attachment too, as the editor seems to mess up my CODE segments)

In your first example, UNZIP isn't asking because the files are all being put in the current ("test") directory, not the subdirectories (because you didn't specify /D) -- so nothing is actually being overwritten.


The /D is creating the first subdirectory but not the second; this appears to be a bug in the zip library. I'll pass it on to the developers and let you know what they say.

I don't know what you're looking for regarding the CRC -- can you expand a bit?

I just tested UNZIP again, it both did not confirm before overwriting existing files, and I had modified them before unzipping again, so it overwrote a newer version. It also restored the directory structure without being given /D parameter.

21:43 (2010-11-15) C:\Users\niklas\test>dir

 Volume in drive C is unlabeled      Serial number is 28b9:6305
 Directory of  C:\Users\niklas\test\*

15.11.2010  21:43         <dir>    .
15.11.2010  21:43         <dir>    ..
13.11.2010  23:20             710  test2.zip
               710 bytes in 1 file and 2 dirs    4 096 bytes allocated
   173 858 377 728 bytes free

21:43 (2010-11-15) C:\Users\niklas\test>7z l test2.zip

7-Zip 4.65  Copyright (c) 1999-2009 Igor Pavlov  2009-02-03

Listing archive: test2.zip


   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2010-11-13 23:19:48 D....            0            0  aa
2010-11-13 20:41:14 ....A           13           13  aa\f1
2010-11-13 20:41:16 ....A           13           13  aa\f2
2010-11-13 23:20:04 D....            0            0  aa\rr
2010-11-13 23:20:08 D....            0            0  aa\rr\aa
2010-11-13 23:20:08 ....A           13           13  aa\rr\aa\aaaa
2010-11-13 23:19:54 ....A           13           13  aa\rr\gdfgf
------------------- ----- ------------ ------------  ------------------------
                                    52           52  4 files, 3 folders

21:43 (2010-11-15) C:\Users\niklas\test>unzip test2.zip
=>
=> C:\Users\niklas\test\f1
=> C:\Users\niklas\test\f2
=>
=>
=> C:\Users\niklas\test\aaaa
=> C:\Users\niklas\test\gdfgf

21:44 (2010-11-15) C:\Users\niklas\test>unzip test2.zip
=>
=> C:\Users\niklas\test\f1
=> C:\Users\niklas\test\f2
=>
=>
=> C:\Users\niklas\test\aaaa
=> C:\Users\niklas\test\gdfgf

21:44 (2010-11-15) C:\Users\niklas\test>echo test > aa\f1

21:44 (2010-11-15) C:\Users\niklas\test>type aa\f1
test

21:44 (2010-11-15) C:\Users\niklas\test>unzip test2.zip
=>
=> C:\Users\niklas\test\f1
=> C:\Users\niklas\test\f2
=>
=>
=> C:\Users\niklas\test\aaaa
=> C:\Users\niklas\test\gdfgf

21:44 (2010-11-15) C:\Users\niklas\test>type aa\f1
ECHO is OFF

21:44 (2010-11-15) C:\Users\niklas\test>dir

 Volume in drive C is unlabeled      Serial number is 28b9:6305
 Directory of  C:\Users\niklas\test\*

15.11.2010  21:44         <dir>    .
15.11.2010  21:44         <dir>    ..
15.11.2010  21:44         <dir>    aa
13.11.2010  23:20             710  test2.zip
               710 bytes in 1 file and 3 dirs    4 096 bytes allocated
   173 858 377 728 bytes free

21:45 (2010-11-15) C:\Users\niklas\test>dir aa

 Volume in drive C is unlabeled      Serial number is 28b9:6305
 Directory of  C:\Users\niklas\test\aa\*

15.11.2010  21:44         <dir>    .
15.11.2010  21:44         <dir>    ..
13.11.2010  20:41              13  f1
13.11.2010  20:41              13  f2
                26 bytes in 2 files and 2 dirs    8 192 bytes allocated
   173 858 377 728 bytes free

Specifying UNZIP /D make it behave the same, except it queries before overwriting.

21:46 (2010-11-15) C:\Users\niklas\test>7z t test2.zip

7-Zip 4.65  Copyright (c) 1999-2009 Igor Pavlov  2009-02-03

Processing archive: test2.zip

Testing     aa
Testing     aa\f1
Testing     aa\f2
Testing     aa\rr
Testing     aa\rr\aa
Testing     aa\rr\aa\aaaa
Testing     aa\rr\gdfgf

Everything is Ok

Folders: 3
Files: 4
Size:       52
Compressed: 710

As for checking the CRC. Something like this. Verify that the zip-file contents matches the stored checksums. Seems like something which should be part of the zip-library already.

 

[rconn]

As for checking the CRC. Something like this. Verify that the zip-file contents matches the stored checksums. Seems like something which should be part of the zip-library already.

I don't see the point -- if the contents don't match the stored CRCs, you'll get an error message when you view or unzip the file.

And no, it's not already part of the zip library.

 

[Steve Fabian]

rconn:
| nikbackm:
|| As for checking the CRC. Something like this. Verify that the
|| zip-file contents matches the stored checksums. Seems like something
|| which should be part of the zip-library already.
|
| I don't see the point -- if the contents don't match the stored CRCs,
| you'll get an error message when you view or unzip the file.

One purpose is to check whether or not the archived file is identical to the one in the file system already, regardless of dates.

| And no, it's not already part of the zip library.

It ought to be, the feature was available in PKZIP since its first version, and it is also available in WinZip, InfoZip and 7z. IIRC a CRC was also available in the old ARC program, though it may have been a 16-bit version...
--
Steve

 

[rconn]

One purpose is to check whether or not the archived file is identical to the one in the file system already, regardless of dates.

But that wasn't what he was asking for -- he wanted an internal check.

| And no, it's not already part of the zip library.

It ought to be, the feature was available in PKZIP since its first version, and it is also available in WinZip, InfoZip and 7z. IIRC a CRC was also available in the old ARC program, though it may have been a 16-bit version...

The internal CRC *is* available, what's not available is the check for the contents against the internally stored CRC.

 

[nikbackm]

But that wasn't what he was asking for -- he wanted an internal check.

Yep, but this is no big deal. UNZIP does check the CRCs when it unzips after all, and I have plenty of other zippers to simply test zipfile integrity. Just a suggestion to make the ZIP interface more complete.

My main interest in TCC ZIP/UNZIP is getting a command interface more consistent with other TCC commands, like the file selection options. Another plus would be the sparse output; just added/extracted file names and no annoying "headers" as used by 3rd party zippers.

So hopefully these other issues can be resolved soon. :)

 

[nikbackm]

Retesting with 12.00.36

Ok, UNZIP now seems to behave as it should. Always asks before overwriting (unless don't ask argument given) and unzips the directory structure as requested (/D included or not) too.

ZIP /U works as it should if the file does not exists.

However, seems to be some issues when using ZIP /U /D. First the added/updates files in the sub-dirs are not shown, only the sub-dir that contains the files.

And if you add files to sub-dirs after running ZIP /U /D then the added files are not added to the target zip if you run ZIP /U /D again.

See included listing file.

14:18 (2010-11-19) C:\User\testzip>dir /s /b
C:\User\testzip\sub1
C:\User\testzip\file1.txt
C:\User\testzip\file2.txt
C:\User\testzip\file3.txt
C:\User\testzip\file4.txt
C:\User\testzip\sub1\sub2
C:\User\testzip\sub1\tile1.txt
C:\User\testzip\sub1\tile2.txt
C:\User\testzip\sub1\sub2\zile1.txt
C:\User\testzip\sub1\sub2\zile2.txt

14:18 (2010-11-19) C:\User\testzip>zip /U /R myzip.zip *
<= C:\User\testzip\file1.txt
<= C:\User\testzip\file2.txt
<= C:\User\testzip\file3.txt
<= C:\User\testzip\file4.txt
<= C:\User\testzip\sub1\*

14:18 (2010-11-19) C:\User\testzip>echo add1 > sub1\add1.txt

14:18 (2010-11-19) C:\User\testzip>echo add2 > sub1\sub2\zadd2.txt

14:18 (2010-11-19) C:\User\testzip>zip /U /R myzip.zip *
<= C:\User\testzip\sub1

14:18 (2010-11-19) C:\User\testzip>dir /s /b
C:\User\testzip\sub1
C:\User\testzip\file1.txt
C:\User\testzip\file2.txt
C:\User\testzip\file3.txt
C:\User\testzip\file4.txt
C:\User\testzip\myzip.zip
C:\User\testzip\sub1\sub2
C:\User\testzip\sub1\add1.txt
C:\User\testzip\sub1\tile1.txt
C:\User\testzip\sub1\tile2.txt
C:\User\testzip\sub1\sub2\zadd2.txt
C:\User\testzip\sub1\sub2\zile1.txt
C:\User\testzip\sub1\sub2\zile2.txt

14:18 (2010-11-19) C:\User\testzip>zip /v myzip.zip
2010-11-19  14:12              4 -50%  file1.txt
2010-11-19  14:12              4 -50%  file2.txt
2010-11-19  14:13              4 -50%  file3.txt
2010-11-19  14:12              4 -50%  file4.txt
2010-11-19  14:18              1 100%  sub1\
2010-11-19  14:18              1 100%  sub1\sub2\
2010-11-19  14:13              4 -50%  sub1\sub2\zile1.txt
2010-11-19  14:13              4 -50%  sub1\sub2\zile2.txt
2010-11-19  14:13              4 -50%  sub1\tile1.txt
2010-11-19  14:13              4 -50%  sub1\tile2.txt

14:19 (2010-11-19) C:\User\testzip>

 

[rconn]

Re: Retesting with 12.00.36

However, seems to be some issues when using ZIP /U /D. First the added/updates files in the sub-dirs are not shown, only the sub-dir that contains the files.

And if you add files to sub-dirs after running ZIP /U /D then the added files are not added to the target zip if you run ZIP /U /D again.

There is only limited support for combining /U and /D.

When you do a /D, ZIP passes the directory name to the zip library, and the library archives everything in that directory & its subdirectories. So the zip library is only checking whether the directory date is modified, not whether any individual files in a subdirectory have been modified.

I had tried the alternative (TCC doing all of the directory parsing itself archiving each file individually), but it was *really* slow and had generated complaints from the beta testers.

(Note that a /D always only shows the root directory name; combining it with /U will not make it show individual files.)

 

[nikbackm]

Re: Retesting with 12.00.36

There is only limited support for combining /U and /D.

When you do a /D, ZIP passes the directory name to the zip library, and the library archives everything in that directory & its subdirectories. So the zip library is only checking whether the directory date is modified, not whether any individual files in a subdirectory have been modified.

I had tried the alternative (TCC doing all of the directory parsing itself archiving each file individually), but it was *really* slow and had generated complaints from the beta testers.

(Note that a /D always only shows the root directory name; combining it with /U will not make it show individual files.)

But doesn't this make it impossible to (correctly) update a directory tree to a zip-file (using built-in ZIP) then?

I'd rather it be slow (and correct) than not available at all (or buggy). Otherwise the /U option is pretty much useless (at least for me). Far too many directories tend to have sub-directories. :)

 

[rconn]

Re: Retesting with 12.00.36

But doesn't this make it impossible to (correctly) update a directory tree to a zip-file (using built-in ZIP) then?

I'd rather it be slow (and correct) than not available at all (or buggy). Otherwise the /U option is pretty much useless (at least for me). Far too many directories tend to have sub-directories. :)

Just leave off the /U. It's faster to (re)zip everything than updating 1 file at a time.

 

[nikbackm]

Re: Retesting with 12.00.36

Just leave off the /U. It's faster to (re)zip everything than updating 1 file at a time.

Might be an option, but not for my intended use.

If there are no changes when you use /U, then the zip-file will not be modified and no further actions are needed. If some source files have changed, then the zip-file will be modified as well (A attribute cleared) and it can be "sent forward" for further processing (copying to FTP server in my case). So always re-creating the whole zip-file is not acceptable.

I can of course always keep using my current zip-program. :)

I would still hope that TCC-ZIP would support /U + /D correctly, surely slow is better than not supported at all? And for speed /U can always be skipped. I would assume that is what the beta testers who complained do anyway, since /U + /D does not work! Even if it will not be supported (since it requires extra work outside of using the zip-library) I think it would be better for ZIP to outright reject that combination instead of possibly making the user believe all of his files got zipped even though some possibly were not.