zip wrong central directory signature

#1
Windows Server 2003
TCC 12.1

zip /a /u /r zipfile1.zip directory1
zip /a /u /r zipfile2.zip directory2
zip /a zipfile3.zip zipfile1.zip zipfile2.zip

produces the error message: wrong central directory signature
when the 3rd command line is completed and no zipfile2.zip produced.

I have never seen this error and my internet searches have yielded nothing of benefit.

Thanks

JH
 
#2
No problem on Win7.
Code:
v:\> zip /a /u /r zip1.zip p:\4SysUtils\
<= P:\4SysUtils\*
 
v:\> zip /a /u /r zip2.zip p:\4threads\
<= P:\4threads\*
 
v:\> zip /a zip3.zip zip1.zip zip2.zip
<= V:\zip1.zip
<= V:\zip2.zip
 
v:\> d *.zip
8/29/2012  14:26      7,972,134  zip1.zip
8/29/2012  14:26      5,723,885  zip2.zip
8/29/2012  14:26      13,534,691  zip3.zip
 
#3
What's weird is that it all worked fine till a month or so ago and then kaput.
It is something with Windows Server 2003 because this only shows up on that one machine. Unfortuantely I have no other Windows Server box to try it on.

JH
 
#4
Looking at your original posting, John, I have a lot to say here regarding both tools and techniques because I have a lot of experience in this area because there was a time when my disk space was at a real premium. So, first as an overview:

A. If you're going to double-zip, the first zip should be done with no compression. This is simply because zip keeps a "dictionary" for each file in the zip when there is compression, and these "dictionaries" are not very compressible. (If they were compressible, it would mean that the zip compression algorithm wasn't doing a very good job.) If you're really looking for maximum compression, don't use zip at all. This is because there is a much better program out there in terms of doing compression named 7-zip (or just 7z), and this program is free and readily available on the web - just Google "7zip". The primary reason for this is that, unlike zip, it uses a single dictionary in the 7z file for all of the files that are 7z'd at one time, and often a lot more compression can be obtained than what zip can do by consolidating what would be all of the dictionaries in the archive into a single dictionary, particularly if the files being archived are similar.

B. 7z does a better job of compressing even a single file with a single dictionary.

And the proof of reason "A." (and this is extensive) done using "binary" files (executables and the Take Command "Help" file, which is binary but contains a lot of straight text, as well as HTML files (which are straight text) is as follows (note that the first size column is just size, the second size column, if present, is allocated size):
Code:
        73,616 D's.7z
        89,088 DelDups32.exe
        90,765 D's with max cmprssn.zip
        126,976 DCalc.exe
        216,572 D's with no cmprssn.zip
        232,252 TCMD.chw
      1,527,664 TCMD's.7z
      1,597,698 All binaries.7z
      2,009,707 TCMD's with max cmprssn.zip
      2,100,046 All zips with max cmprssn.zip
      2,100,450 All binaries with max cmprssn.zip
      2,101,766 All Zips with no cmprssn.zip
      3,949,016 TCMD.exe
      4,181,764 TCMD's with no cmprssn.zip
 
 11,470  12,288 Set2 HTML 7z'd.7z
 12,824  16,384 Set1 HTML 7z'd.7z
 16,378  16,384 Both Sets of HTML Files 7z'd.7z
 16,851  20,480 7z of both sets of HTML files zppd with no cmpressn.7z
 24,569  24,576 Both Sets of 7z'd HTML Files.7z
 33,777  36,864 Zip with max cmprssn of both HTML sets zppd with no cmprssn.zip
 41,718  45,056 Set2 HTML zipped with maximum compression.zip
 43,300  45,056 Set1 HTML zipped with maximum compression.zip
 79,382  81,920 7z of both HTML sets zipped with max compression.7z
 82,863  86,016 Zip of both HTML sets zipped with maximum compression.zip
 82,863  86,016 Zip with max cmprssn of both sets of HTML files zppd with max compr
 84,996  86,016 Both HTML sets with maximum compression.zip
144,366 147,456 Set2 HTML zip with no compression.zip
150,260 151,552 Set1 HTML zip with no compression.zip
And just to completely illustrate what exactly is in each zip and 7z file:
Code:
Listing archive: D's.7z
--
Physical Size = 73,616
Allocated Size = 73,728
Headers Size = 192

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2011-04-09 22:31:19 ....A       126976        73424  DCalc.exe
2011-04-10 20:42:49 ....A        89088               DelDups32.exe
------------------- ----- ------------ ------------  ------------------------
                                216064        73424  2 files, 0 folders


Archive:  D's with max cmprssn.zip
Physical Size = 90,765
Allocated Size = 94,208

  Length     Date   Time    Name
 --------    ----   ----    ----
   126976  04/09/11 22:31   DCalc.exe
    89088  04/10/11 20:42   DelDups32.exe
 --------                   -------
   216064                   2 files


Archive:  D's with no cmprssn.zip
Physical Size = 216,572
Allocated Size = 217,088

  Length     Date   Time    Name
 --------    ----   ----    ----
   126976  04/09/11 22:31   DCalc.exe
    89088  04/10/11 20:42   DelDups32.exe
 --------                   -------
   216064                   2 files


Listing archive: TCMD's.7z
--
Physical Size = 1,527,664
Allocated Size = 1,527,808
Headers Size = 187

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2011-09-02 16:05:55 ....A       232252        21273  TCMD.chw
2012-02-24 22:36:52 ....A      3949016      1506204  TCMD.exe
------------------- ----- ------------ ------------  ------------------------
                               4181268      1527477  2 files, 0 folders


Listing archive: All binaries.7z
--
Physical Size = 1,597,698
Allocated Size = 1,601,536
Headers Size = 250

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2011-09-02 16:05:55 ....A       232252        21273  TCMD.chw
2011-04-09 22:31:19 ....A       126976      1576175  DCalc.exe
2011-04-10 20:42:49 ....A        89088               DelDups32.exe
2012-02-24 22:36:52 ....A      3949016               TCMD.exe
------------------- ----- ------------ ------------  ------------------------
                               4397332      1597448  4 files, 0 folders


Archive:  TCMD's with max cmprssn.zip
Physical Size = 2,009,707
Allocated Size = 2,011,136

  Length     Date   Time    Name
 --------    ----   ----    ----
   232252  09/02/11 16:05   TCMD.chw
  3949016  02/24/12 21:36   TCMD.exe
 --------                   -------
  4181268                   2 files


Archive:  All zips zipped with max cmprssn.zip
Physical Size = 2,100,046
Allocated Size = 2,101,248

  Length     Date   Time    Name
 --------    ----   ----    ----
    90765  08/30/12 02:16   D's with max cmprssn.zip
  2009707  08/30/12 02:20   TCMD's with max cmprssn.zip
 --------                   -------
  2100472                   2 files
 
 
Archive:  All binaries with max cmprssn.zip
Physical Size = 2,100,450
Allocated Size = 2,101,248
  Length     Date   Time    Name
 --------    ----   ----    ----
   126976  04/09/11 22:31   DCalc.exe
    89088  04/10/11 20:42   DelDups32.exe
  3949016  02/24/12 21:36   TCMD.exe
   232252  09/02/11 16:05   TCMD.chw
 --------                   -------
  4397332                   4 files
 
 
Archive:  TCMD's with no cmprssn.zip
Physical Size = 4,181,764
Allocated Size = 4,182,016

  Length     Date   Time    Name
 --------    ----   ----    ----
   232252  09/02/11 16:05   TCMD.chw
  3949016  02/24/12 21:36   TCMD.exe
 --------                   -------
  4181268                   2 files
 
==========================================================
 
Set2 HTML 7z'd.7z
--
Physical Size = 11,470
Allocated Size = 12,288
Headers Size = 233

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-29 23:54:36 ....A        26156        11237  Set2.Page1.html
2012-08-29 23:52:42 ....A        26715               Set2.Page2.html
2012-08-29 23:56:32 ....A        27926               Set2.Page3.html
2012-08-29 23:55:21 ....A        27842               Set2.Page4.html
2012-08-29 23:53:48 ....A        34430               Set2.Page5.html
------------------- ----- ------------ ------------  ------------------------
                                143069        11237  5 files, 0 folders
 
 
Set1 HTML 7z'd.7z
--
Physical Size = 12,824
Allocated Size = 16,384
Headers Size = 234

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-30 00:01:00 ....A        26779        12590  Set1.Page1.html
2012-08-29 23:57:21 ....A        28338               Set1.Page2.html
2012-08-30 00:01:28 ....A        32205               Set1.Page3.html
2012-08-30 00:00:03 ....A        28587               Set1.Page4.html
2012-08-29 23:58:03 ....A        33054               Set1.Page5.html
------------------- ----- ------------ ------------  ------------------------
                                148963        12590  5 files, 0 folders
 
 
Both Sets of HTML Files 7z'd.7z
--
Physical Size = 16378
Allocated Size = 16,384
Headers Size = 312

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-30 00:01:00 ....A        26779        16066  Set1.Page1.html
2012-08-29 23:57:21 ....A        28338               Set1.Page2.html
2012-08-30 00:01:28 ....A        32205               Set1.Page3.html
2012-08-30 00:00:03 ....A        28587               Set1.Page4.html
2012-08-29 23:58:03 ....A        33054               Set1.Page5.html
2012-08-29 23:54:36 ....A        26156               Set2.Page1.html
2012-08-29 23:52:42 ....A        26715               Set2.Page2.html
2012-08-29 23:56:32 ....A        27926               Set2.Page3.html
2012-08-29 23:55:21 ....A        27842               Set2.Page4.html
2012-08-29 23:53:48 ....A        34430               Set2.Page5.html
------------------- ----- ------------ ------------  ------------------------
                                292032        16066  10 files, 0 folders
 
 
7z of both sets of HTML files zppd with no cmpressn.7z
--
Physical Size = 16,851
Allocated Size = 20,480
Headers Size = 203

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-30 00:25:39 ....A       150260        16648  Set1 zip with no compression.zip
2012-08-30 00:25:23 ....A       144366               Set2 zip with no compression.zip
------------------- ----- ------------ ------------  ------------------------
                                294626        16648  2 files, 0 folders
 
 
Both Sets of 7z'd HTML Files.7z
--
Physical Size = 24,569
Allocated Size = 24,576
Headers Size = 176

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-30 00:34:50 ....A        12824        24393  Set1 7z'd.7z
2012-08-30 00:35:00 ....A        11470               Set2 7z'd.7z
------------------- ----- ------------ ------------  ------------------------
                                 24294        24393  2 files, 0 folders
 
 
Zip with max cmprssn of both HTML sets zppd with no cmprssn.zip
--
Physical Size = 33,777
Allocated Size = 36,864

  Length     Date   Time    Name
 --------    ----   ----    ----
   150260  08/30/12 00:25   Set1 zip with no compression.zip
   144366  08/30/12 00:25   Set2 zip with no compression.zip
 --------                   -------
   294626                   2 files
 
 
Set2 HTML zipped with maximum compression.zip
--
Physical Size: 41,718
Allocated Size: 45,056
 
  Length     Date   Time    Name
 --------    ----   ----    ----
    26156  08/29/12 23:54   Set2.Page1.html
    26715  08/29/12 23:52   Set2.Page2.html
    27926  08/29/12 23:56   Set2.Page3.html
    27842  08/29/12 23:55   Set2.Page4.html
    34430  08/29/12 23:53   Set2.Page5.html
 --------                   -------
   143069                   5 files
 
 
Set1 HTML zipped with maximum compression.zip
--
Physical Size = 43,300
Allocated Size = 45,056
 
  Length     Date   Time    Name
 --------    ----   ----    ----
    26779  08/30/12 00:01   Set1.Page1.html
    28338  08/29/12 23:57   Set1.Page2.html
    32205  08/30/12 00:01   Set1.Page3.html
    28587  08/30/12 00:00   Set1.Page4.html
    33054  08/29/12 23:58   Set1.Page5.html
 --------                   -------
   148963                   5 files
 
 
7z of both HTML sets zipped with max compression.7z
--
Physical Size = 79382
Allocated Size = 81,920
Headers Size = 213

   Date      Time    Attr         Size   Compressed  Name
------------------- ----- ------------ ------------  ------------------------
2012-08-30 00:24:38 ....A        43300        79169  Set1 zip with maximum compression.zip
2012-08-30 00:24:56 ....A        41718               Set2 zip with maximum compression.zip
------------------- ----- ------------ ------------  ------------------------
                                 85018        79169  2 files, 0 folders
 
 
Zip of both HTML sets zipped with maximum compression.zip
--
Physical Size = 82,863
Allocated Size = 86,016

  Length     Date   Time    Name
 --------    ----   ----    ----
    43300  08/30/12 00:24   Set1 zip with maximum compression.zip
    41718  08/30/12 00:24   Set2 zip with maximum compression.zip
 --------                   -------
    85018                   2 files
 
 
Zip with max cmprssn of both sets of HTML files zppd with max comprssn.zip
--
Physical Size = 82,863
Allocated Size = 86,016

  Length     Date   Time    Name
 --------    ----   ----    ----
    43300  08/30/12 00:24   Set1 zip with maximum compression.zip
    41718  08/30/12 00:24   Set2 zip with maximum compression.zip
 --------                   -------
    85018                   2 files
 
 
Both HTML sets with maximum compression.zip
--
Physical Size = 84,996
Allocated Size = 86,016

  Length     Date   Time    Name
 --------    ----   ----    ----
    26779  08/30/12 00:01   Set1.Page1.html
    28338  08/29/12 23:57   Set1.Page2.html
    32205  08/30/12 00:01   Set1.Page3.html
    28587  08/30/12 00:00   Set1.Page4.html
    33054  08/29/12 23:58   Set1.Page5.html
    26156  08/29/12 23:54   Set2.Page1.html
    26715  08/29/12 23:52   Set2.Page2.html
    27926  08/29/12 23:56   Set2.Page3.html
    27842  08/29/12 23:55   Set2.Page4.html
    34430  08/29/12 23:53   Set2.Page5.html
 --------                   -------
   292032                   10 files
 
 
Archive:  Set2 HTML zip with no compression.zip
Physical Size = 144,366
Allocated Size = 147,456

  Length     Date   Time    Name
 --------    ----   ----    ----
    26156  08/29/12 23:54   Set2.Page1.html
    26715  08/29/12 23:52   Set2.Page2.html
    27926  08/29/12 23:56   Set2.Page3.html
    27842  08/29/12 23:55   Set2.Page4.html
    34430  08/29/12 23:53   Set2.Page5.html
 --------                   -------
   143069                   5 files
 
 
Set1 HTML zip with no compression.zip
Physical Size = 150,260
Allocated Size = 151,552

  Length     Date   Time    Name
 --------    ----   ----    ----
    26779  08/30/12 00:01   Set1.Page1.html
    28338  08/29/12 23:57   Set1.Page2.html
    32205  08/30/12 00:01   Set1.Page3.html
    28587  08/30/12 00:00   Set1.Page4.html
    33054  08/29/12 23:58   Set1.Page5.html
 --------                   -------
   148963                   5 files
Bottom lines are:

1. Double-zipping only buys you something if the first zip(s) are done no compression.
2. 7zip does a better job than zip, period, end of sentence.

Proof of reason B. for a series of .html files that were both 7'd and zipped:

Note that in the following I'm using 2 sets of "related" HTML files downloaded from the web; each "set" of web pages cover the same "subject" so that all of the pages in Set1 are related and have similarities and the same goes for Set2.
Code:
 7,369  Set2.Page2.html.7z
 7,399  Set2.Page4.html.7z
 7,490  Set1.Page4.html.7z
 7,648  Set1.Page2.html.7z
 7,801  Set2.Page1.html.7z
 7,829  Set2.Page3.html.7z
 7,885  Set2.Page2.html.zip
 7,907  Set2.Page4.html.zip
 7,986  Set1.Page4.html.zip
 8,104  Set1.Page1.html.7z
 8,200  Set1.Page2.html.zip
 8,320  Set2.Page1.html.zip
 8,355  Set2.Page3.html.zip
 8,547  Set1.Page5.html.7z
 8,653  Set1.Page1.html.zip
 8,708  Set2.Page5.html.7z
 8,803  Set1.Page3.html.7z
 9,176  Set1.Page5.html.zip
 9,339  Set2.Page5.html.zip
 9,373  Set1.Page3.html.zip
26,156  Set2.Page1.html
26,715  Set2.Page2.html
26,779  Set1.Page1.html
27,842  Set2.Page4.html
27,926  Set2.Page3.html
28,338  Set1.Page2.html
28,587  Set1.Page4.html
32,205  Set1.Page3.html
33,054  Set1.Page5.html
34,430  Set2.Page5.html
I'm not going to cover this in any detail because you can do that for yourself if you are interested, it's completely straightforward. The bottom line is that 7z always does a better job than zip for HTML files, and I've never encountered a case where it didn't do a better job irregardless of the file type(s).

- Dan
 
#5
Thank you Dan for your reply. I did not know about the directory stored in .zip files so that bit of info illuminated my thinking a bit. Not enough to actually understand exactly what is amiss on this specific server problem BUT enough to work around the problem. I was going to test alternate zip programs so your suggestion about 7-Zip was a good one. Things are working like a charm and for the time being my curiosity is on a back burner. I may not learn anything specific about this "wrong central directory signature" problem but the system is doing the backups properly now.
Again thanks.
 
#6
John, an addendum just in case it's relevant. If you want to have an archive that contains two archives as you were doing in your original question, first create the two contained archives using zip with no compression, then 7z the two created zip files. Again, the idea is that you want to avoid trying to compress data that is already compressed.

- Dan
 
#8
Just a note on a couple of points. Double zipping in the manner Dan suggested is more efficient not only because the first level zip files do not contain virtually incompressible dictionaries, but also because the one dictionary for the compound is optimized for all of the data to be compressed, instead of a different one for each constituent file. In fact the first level could be TAR instead of zip. It is also the reason why 7zip can do better compression - its dictionary is based on all of the data. Plaugher's book "The Standard C Library" included it s software in the double-zip mode, thus fitting on a single diskette. Normal zipping would not have made it...
 
#9
And Steve is, of course, correct on all points. I just didn't go to that level of detail because my focus was on actually demonstrating it by real-world example(s). And I used zip with no compression rather than TAR because I figured that that would be a universally-known program which TAR might not be without a Unix background and the differences between using the two would be minor in all circumstances. I originally used to TAR and then zip when I was actually still using zip.

And Steve, I'll note that the reason why I had the zipped and 7z'd single-file examples was to show that the unified dictionary in 7z while major is not the only reason 7z is better - it uses a better straight compression algorithm than does zip. Dictionary consolidation is, of course, irrelevant when 7-zipping a single file.

- Dan
 
#12
Dan, I am sorry to say I have no idea what the "last minute update (of your previous post)" is that you refer to in your latest post. If you refer to your statement that 7zip's compression algorithm is better than PKZIP's, I was not aware of it, but do not find it surprising - it is a newer algorithm, and would not be used if it weren't better.

BTW, one sometimes responds to a post which was the latest at the time of starting the response, but isn't the latest by the time the response is posted. This is especially true for long responses. In the traditional USENET format it is always possible to determine, even without including a quotation, which post a new one is directly responding to. Unfortunately in the browser-based formats this is not the case - if my response is posted after your response, there is no way to know which post I am responding to without bloating my response by quoting all of the post I am responding to.
 
#13
Steve, you are right, it was nothing more than noting that the compression algorithm was better. And I agree with you about updating postings after the fact, it's just that that was such a quick update that making a brand new posting for it seemed a little bit silly. I was wrong. - Dan