ZIP/U compares wrong timestamp

[Christian Albaret]

ZIP offsets the internal timestamp of the file it stores (since TCMD V20) (looks like time-zone handling). ZIP/U seems to compare the timestamps without accounting for the time offset; it keeps adding the file even if the timestamp did not change.

> touch/c asd
2017-11-28 14:50:39.695 D:\testzip\asd

>zip /a testzip.zip asd
<= D:\testzip\asd

> zip /v testzip.zip
2017-11-28 13:50 1 -100% asd <<<=== time offset by 1 hour

> zip /u testzip.zip asd
<= D:\testzip\asd <<<=== file is added again

 

[vefatica]

It happens the other way around on this side of the prime meridian ... make the file newer and it's not updated.

v:\database> touch /c abc
2017-11-28 11:19:34.752  V:\database\abc

v:\database> filetimes abc
Created:  2017-11-28 11:19:34
Accessed: 2017-11-28 11:19:34
Modified: 2017-11-28 11:19:34

v:\database> zip /a test.zip abc
<= V:\database\abc

v:\database> zip /v test.zip
2017-11-28  16:19              1 -100%  abc

v:\database> touch /ta12:13:51.26 abc
2017-11-28 12:13:51.260  V:\database\abc

v:\database> touch /tc12:13:51.26 abc
2017-11-28 12:13:51.260  V:\database\abc

v:\database> touch /tw12:13:51.26 abc
2017-11-28 12:13:51.260  V:\database\abc

v:\database> filetimes abc
Created:  2017-11-28 12:13:51
Accessed: 2017-11-28 12:13:51
Modified: 2017-11-28 12:13:51

v:\database> zip /u test.zip abc

v:\database> zip /v test.zip
2017-11-28  16:19              1 -100%  abc

v:\database>

 

[vefatica]

And a small gripe ... TOUCH shows thousandths of a second but does not let you specify thousandths of a second.

v:\database> touch /tw12:13:51.26 abc
2017-11-28 12:13:51.260  V:\database\abc

v:\database> touch /tw13:14:51.260 abc
TCC: Invalid time "13:14:51.260"

 

[rconn]

This pops up every couple of years (Steve F. used to rant about it constantly).

It's Windows behavior, not TCC. TCC is querying the two filetimes without doing any adjustments; the problem is that Windows changes the internal filetimes for things like Daylight Savings Time (and doesn't change back when DST ends). I have several times tried to come up with a workaround for this, but it always gets much too messy trying to outguess Windows internals (and trying to determine where the computer is, as some states don't have DST).

 

[rconn]

And a small gripe ... TOUCH shows thousandths of a second but does not let you specify thousandths of a second.

WAD. TOUCH used to only show hh:mm:ss, and some users complained that it looked like files had the same times when they were slightly different. So I added thousandths for the display, but not for setting, because IMO nobody actually needs that.

And even the thousandths display is useless, because the time resolution is 100ns,

 

[vefatica]

This pops up every couple of years (Steve F. used to rant about it constantly).

It's Windows behavior, not TCC. TCC is querying the two filetimes without doing any adjustments; the problem is that Windows changes the internal filetimes for things like Daylight Savings Time (and doesn't change back when DST ends). I have several times tried to come up with a workaround for this, but it always gets much too messy trying to outguess Windows internals (and trying to determine where the computer is, as some states don't have DST).

I tried my earlier experiment again, this time changing the timestamps to THREE hours later. That should be enough to overcome a DST change. The file, now 3 hours newer was not updated by "ZIP /U". It should be pointed out that this experiment uses a newly-created file and lasts but 5 minutes ... no time for any DST change. [When I make it another 3 hours newer (now a total of 6 hours, enough to overcome the EST/UTC difference) it is updated by "ZIP /U".]

v:\database> touch /c abc
2017-11-29 16:44:05.070  V:\database\abc

v:\database> filetimes abc /u
Created:  2017-11-29 21:44:05
Accessed: 2017-11-29 21:44:05
Modified: 2017-11-29 21:44:05

v:\database> zip /a test.zip abc
<= V:\database\abc

v:\database> zip /v test.zip
2017-11-29  21:44              1 -100%  abc

v:\database> touch /ta19:24:35 abc
2017-11-29 19:24:35.000  V:\database\abc

v:\database> touch /tc19:24:35 abc
2017-11-29 19:24:35.000  V:\database\abc

v:\database> touch /tw19:24:35 abc <=============  it's 3 hours newer
2017-11-29 19:24:35.000  V:\database\abc

v:\database> filetimes abc /u
Created:  2017-11-30 00:24:35
Accessed: 2017-11-30 00:24:35
Modified: 2017-11-30 00:24:35

v:\database> zip /u test.zip abc

v:\database> zip /v test.zip
2017-11-29  21:44              1 -100%  abc <============== not updated

 

[vefatica]

Here it is again,a little simpler. If I add 1, 2, 3, or 4 hours, it's not updated by "ZIP /U". When I add 5 hours (enough to beat the EST/UTC difference by 1 minute), it is updated. I'm using this alias.

v:\database> alias touchall
do z in /L a c w ( touch /t%[z]%1 %2 )

v:\database> touch /c abc
2017-11-29 17:09:43.987  V:\database\abc

v:\database> zip /a test.zip abc
<= V:\database\abc

v:\database> touchall 18:09:45 abc
2017-11-29 18:09:45.000  V:\database\abc
2017-11-29 18:09:45.000  V:\database\abc
2017-11-29 18:09:45.000  V:\database\abc

v:\database> zip /u test.zip abc

v:\database> touchall 19:09:45 abc
2017-11-29 19:09:45.000  V:\database\abc
2017-11-29 19:09:45.000  V:\database\abc
2017-11-29 19:09:45.000  V:\database\abc

v:\database> zip /u test.zip abc

v:\database> touchall 20:09:45 abc
2017-11-29 20:09:45.000  V:\database\abc
2017-11-29 20:09:45.000  V:\database\abc
2017-11-29 20:09:45.000  V:\database\abc

v:\database> zip /u test.zip abc

v:\database> touchall 21:09:45 abc
2017-11-29 21:09:45.000  V:\database\abc
2017-11-29 21:09:45.000  V:\database\abc
2017-11-29 21:09:45.000  V:\database\abc

v:\database> zip /u test.zip abc

v:\database> touchall 22:09:45 abc
2017-11-29 22:09:45.000  V:\database\abc
2017-11-29 22:09:45.000  V:\database\abc
2017-11-29 22:09:45.000  V:\database\abc

v:\database> zip /u test.zip abc
<= V:\database\abc

 

[Christian Albaret]

This pops up every couple of years (Steve F. used to rant about it constantly).

It's Windows behavior, not TCC. …

I wasn't aware of this fact.
This makes the /U and /F options virtually unusable (unless there would be some way to shift the timestamps). As said, ZIP did not shift the timestamps in V19.
I will work with /A:A and then reset the Archive attribute (there is no /X option as for COPY) (I call ZIP from a batch script).

 

[rconn]

I made a change a few builds ago to do the comparisons using UTC rather than local time.

 

[Christian Albaret]

I tested on TCC 22.00.29 x64 Windows 7 [Version 6.1.7601].
ZIP/U seems to correctly select the files based on their timestamp.
The timestamp (as seen with ZIP/V) in some cases shows identical to DIR and in some cases still shows offset by 1h (files in a directory in the C:\Users\… Windows profile; don't know if a Windows trickery about the profile could explain that)
On a test UNZIP/U also correctly selects the files based on the timestamp; however on the two directories I process with ZIP+UNZIP, UNZIP/U always extracts some files (one directory is the one in the C:\Users\… profile, the other one is simply on D:) (it logs that the files are extracted (lines beginning with =>), and I also get an error (logged in red, the color I use for error output) if I make a file read-only) (it always extracts 75 in 125 files in one case, 24 files in 58; the files that are extracted are always the same ones; the proportion is about ½; looks to me like a rounding effect on the timestamp).

 

[Charles Dye]

The timestamp (as seen with ZIP/V) in some cases shows identical to DIR and in some cases still shows offset by 1h (files in a directory in the C:\Users\… Windows profile; don't know if a Windows trickery about the profile could explain that)

More like SystemTimeToFileTime() behavior. It converts UTC to local time using the daylight savings offset in effect now, rather than at the time specified in the timestamp. This is long-standing Windows behavior -- wrong, but too well-established for Microsoft to ever change.

Rex: Have you considered using SystemTimeToTzSpecificLocalTime() instead?

 

[Charles Dye]

... (and trying to determine where the computer is, as some states don't have DST).

Hell, some parts of states. Arizona doesn't observe DST, except for the Navajo nation, which does. And I have no idea what the deal is with Indiana.
But even if you implemented everything perfectly, Congress would just change the rules next year. Or some state legislature, or a tribal government....

 

[dcantor]

Assuming the user has somehow told Windows what the correct timezone settings are, you can find out whether DST is in effect and whether DST is even observed and what the offsets from UTC are.

~\Work> w32tm /tz
Time zone: Current:TIME_ZONE_ID_STANDARD Bias: 300min (UTC=LocalTime+Bias)
  [Standard Name:"Eastern Standard Time" Bias:0min Date:(M:11 D:1 DoW:0)]
  [Daylight Name:"Eastern Daylight Time" Bias:-60min Date:(M:3 D:2 DoW:0)]

These values are also in the registry at HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\TimeZoneInformation .

 

[Christian Albaret]

ZIP now compares UTC times, which don't vary with DST. I notice a 1h offset for files in the user-profile, but this seems to be handled correctly with UTC times too.
The last quirk I can see is that some files are extracted by UNZIP although the times look identical (when comparing ZIP/V and DIR output). I suspect there is a minor time difference, maybe due to rounding. If the matter is a small time difference, it could be handled by a /UF option (update 2s+).