Find the Last Modified Date/Time of a file via download link

Joe Caverly · 2021-10-27

Code:

     _x64: 1
   _admin: 1
_elevated: 1

TCC  28.01.14 x64   Windows 10 [Version 10.0.19043.1288]

Here's an alias that allows one to find the last modified date/time of a file on a website;

Code:

last-modified=*pshell /s "$http = new-object System.Net.WebClient; $http.OpenRead('%1').Close(); $http.ResponseHeaders['Last-Modified']"

Proof;

Code:

e:\utils>last-modified https://jpsoft.com/downloads/v28/tcc.exe
Sun, 22 Aug 2021 02:59:54 GMT

Joe

Ref: Determine size of file via download link

vefatica · 2021-10-27

Hmmm! I have one much like it.

Code:

v:\> last-modified https://jpsoft.com/downloads/v28/tcmd.exe
Sun, 22 Aug 2021 02:59:09 GMT

v:\> echo %@ifiletime[jpsoft.com/downloads/v28/tcmd.exe]
Sun, 22 Aug 2021 02:59:09 GMT

v:\> which @ifiletime
@ifiletime is a plugin variable (4UTILS)

Should it work for ftp://? It just fails silently. Mine fails with a timeout, no doubt because it's trying https://.

Kachupp · 2021-10-27

Joe Caverly said:
Code:

_x64: 1 _admin: 1 _elevated: 1 TCC 28.01.14 x64 Windows 10 [Version 10.0.19043.1288]

Here's an alias that allows one to find the last modified date/time of a file on a website;

Code:

last-modified=*pshell /s "$http = new-object System.Net.WebClient; $http.OpenRead('%1').Close(); $http.ResponseHeaders['Last-Modified']"

Proof;

Code:

e:\utils>last-modified https://jpsoft.com/downloads/v28/tcc.exe Sun, 22 Aug 2021 02:59:54 GMT

Joe

This is very useful, I have one problem the file I'm after has GUID.xlsx which would be hard to damn near impossible to guess. Any thoughts

Joe Caverly · 2021-10-28

vefatica said:
Should it work for ftp://? It just fails silently. Mine fails with a timeout, no doubt because it's trying https://.

Hey @vefatica,
No, that is an HTTP only alias.

Here's one for FTP;

Code:

alias last-modified=`pshell /s "$request = [System.Net.WebRequest]::Create('%1'); $request.Method = [System.Net.WebRequestMethods+FTP]::GetDateTimeStamp; $response=$request.GetResponse();$response.LastModified.tostring()"`

Proof;

Code:

e:\utils>last-modified ftp://prospero.unm.edu/textutils.zip
2021-10-20 6:21:18 AM

There may be a .NET method that handles both HTTP and FTP, but I have yet to discover it.

Joe

Joe Caverly · 2021-10-28

Hey @Kachupp,

Kachupp said:
This is very useful, I have one problem the file I'm after has GUID.xlsx which would be hard to damn near impossible to guess. Any thoughts

Does the site that has GUID.xlsx allow one to get a listing of the files that can be downloaded?

If it does, here is how I do it, using Windows 10 with WSL and Lynx.

As an example;

Code:

jlc@DESKTOP-H2JFFTF:/mnt/e/Utils$ lynx -dump -listonly https://downloads.powerbasic.com/inc/

References

   1. https://downloads.powerbasic.com/inc/PBCC40_INC.zip
   2. https://downloads.powerbasic.com/inc/PBCC50_INC.zip
   3. https://downloads.powerbasic.com/inc/PBCC60_INC.zip
   4. https://downloads.powerbasic.com/inc/PBWIN80_INC.zip
   5. https://downloads.powerbasic.com/inc/PBWIN90_INC.zip
   6. https://downloads.powerbasic.com/inc/PBWIN10_INC.zip

You will find that most sites are secured from allowing folks to do this.

Joe

drwtsn32 · 2021-10-28

This is pretty cool! But won't this download the entire file?

Code:

$http.OpenRead('%1')

Maybe it would be better to use the HEAD method. I don't know if that is possible with WebClient, but you can do it with WebRequest:

Code:

$http = [System.Net.WebRequest]::Create('%1'); $http.Method = 'HEAD'; $http.GetResponse().LastModified

Kachupp · 2021-10-29

Joe Caverly said:
Hey @Kachupp,

Does the site that has GUID.xlsx allow one to get a listing of the files that can be downloaded?

If it does, here is how I do it, using Windows 10 with WSL and Lynx.

As an example;

Code:

jlc@DESKTOP-H2JFFTF:/mnt/e/Utils$ lynx -dump -listonly https://downloads.powerbasic.com/inc/ References 1. https://downloads.powerbasic.com/inc/PBCC40_INC.zip 2. https://downloads.powerbasic.com/inc/PBCC50_INC.zip 3. https://downloads.powerbasic.com/inc/PBCC60_INC.zip 4. https://downloads.powerbasic.com/inc/PBWIN80_INC.zip 5. https://downloads.powerbasic.com/inc/PBWIN90_INC.zip 6. https://downloads.powerbasic.com/inc/PBWIN10_INC.zip

You will find that most sites are secured from allowing folks to do this.

Joe

Sadily no. Its downloadable only when you hover over a link

Kachupp · 2021-10-29

drwtsn32 said:
This is pretty cool! But won't this download the entire file?

Nope its an alias where the cmdline would pass a filename.ext as `%1` to look for on the website

drwtsn32 · 2021-10-29

Kachupp said:
Nope its an alias where the cmdline would pass a filename.ext as `%1` to look for on the website

Yeah, I understand it's an alias. But the question is if that function triggers a full download or not, just to get the last modified timestamp from the header. The HTTP HEAD command ensures you're only getting the header.

Joe Caverly · 2021-10-29

drwtsn32 said:
Yeah, I understand it's an alias. But the question is if that function triggers a full download or not, just to get the last modified timestamp from the header. The HTTP HEAD command ensures you're only getting the header.

Doing an .OpenRead with the System.Net.WebClient is just opening the file for reading.

If one wanted the contents of the file, they would have to create a StreamReader, and read it to the end of the stream, to download any or all of the file.

Joe

drwtsn32 · 2021-10-29

Joe Caverly said:
... is just opening the file for reading.

We're talking about the HTTP protocol, and it's not the same as traditional file open/close calls on a regular filesystem.

With HTTP you can do a GET on a url and it will initiate the transfer. I was afraid that's when "OpenRead" might be doing. But maybe that doesn't occur until - as you point out - you actually read the stream. (I'm not that familiar with the WebClient class.)

I just wanted to point out that doing an HTTP HEAD call would definitely avoid any file download for sure. If it's a non-issue then nothing to worry about. :)

vefatica · 2021-10-30

Joe Caverly said:

No, that is an HTTP only alias.

Here's one for FTP;

Code:

alias last-modified=`pshell /s "$request = [System.Net.WebRequest]::Create('%1'); $request.Method = [System.Net.WebRequestMethods+FTP]::GetDateTimeStamp; $response=$request.GetResponse();$response.LastModified.tostring()"`

That one you can do with TCC only, but you'd prpbably have to monkey with the output a bit.

Code:

v:\> iftp /v vefatica.net > nul & iftp /s mdtm pset3264.zip & iftp /c > nul
mdtm pset3264.zip
213 20211001155825.387

v:\> pshell /s "$request = [System.Net.WebRequest]::Create('ftp://vefatica.net/pset3264.zip'); $request.Method = [System.Net.WebRequestM
ethods+FTP]::GetDateTimeStamp; $response=$request.GetResponse();$response.LastModified.tostring()"
2021-10-01 11:58:25

vefatica · 2021-10-30

And beside the less-than-optimal output, the TCC command in my last post gives a UTC time while the System.Net.WebRequest method gives a local time

vefatica · 2021-10-30

drwtsn32 said:
Code:
$http = [System.Net.WebRequest]::Create('%1'); $http.Method = 'HEAD'; $http.GetResponse().LastModified

I get

I use HEAD in my @IFILETIME, but it's a "verb", a string parameter to HttpOpenRequest.

Joe Caverly · 2021-10-30

Hey @vefatica,
I think that HTTP supports the following methods;

Code:

GET, PUT, POST, DELETE, TRACE, HEAD or OPTIONS

So far, the only method I found that FTP supports is;

Code:

RETR

I'm sure that more info is contained in the MSDN files.

Joe

Joe Caverly · 2021-10-30

vefatica said:
And beside the less-than-optimal output, the TCC command in my last post gives a UTC time while the System.Net.WebRequest method gives a local time

Hey @vefatica,
Modified to return UTC time;

Code:

pshell /s "$request = [System.Net.WebRequest]::Create('ftp://vefatica.net/pset3264.zip'); $request.Method = [System.Net.WebRequestMethods+FTP]::GetDateTimeStamp; $response=$request.GetResponse();$response.LastModified.ToUniversalTime().ToString()"

Proof;

Code:

e:\utils>pshell /s "$request = [System.Net.WebRequest]::Create('ftp://vefatica.net/pset3264.zip'); $request.Method = [System.Net.WebRequestMethods+FTP]::GetDateTimeStamp; $response=$request.GetResponse();$response.LastModified.ToUniversalTime().ToString()"
2021-10-01 3:58:25 PM

Joe

vefatica · 2021-10-30

Those may be methods for System.Net.WebRequest but the WinInet APIs don't use classes at all. My plugin does this

Code:

if    (        !(hInet = InternetOpen(L"JPUser", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0))
            ||    !(hConnect = InternetConnect(hInet, szHost, INTERNET_DEFAULT_HTTP_PORT, NULL, NULL, INTERNET_SERVICE_HTTP, 0, 0))
            ||    !(hRequest = HttpOpenRequest(hConnect, L"HEAD", szResource, NULL, NULL, NULL, INTERNET_FLAG_RELOAD | INTERNET_FLAG_NO_CACHE_WRITE, 0))
            ||    !HttpSendRequest(hRequest, NULL, 0, NULL, 0)
            ||    !HttpQueryInfo(hRequest, nInfoLevel, psz, &dwBufLen, &dwIndex)        
        )
...

where nInfoLevel is HTTP_QUERY_LAST_MODIFIED. In another plugin (@IFILESIZE) the same code uses nInfoLevel equal to HTTP_QUERY_CONTENT_LENGTH.

Joe Caverly · 2021-10-31

Here's the C# source code for WebRequest.cs, FtpWebRequest.cs and HttpWebRequest.cs for .Net Framework 4.8, all of which are used by PowerShell to perform the necessary tasks to get the last modified date/time of a file on a website.

Majority of this is beyond my abilities, but passing on for those that might be interested.

Joe

vefatica · 2021-11-24

This works, too. [I must admit that when it comes to VBS, I don't know what I'm doing. But if I put together enoubh Google results and do some experimenting, it sometimes works.

]

Does anyone know if I can run this with TCC's internal SCRIPT command?

Code:

'ifiletime.vbs
Dim h
Dim lastmod
Set h = CreateObject("MSXML2.ServerXMLHTTP")
    h.Open "HEAD", "https://jpsoft.com/downloads/v28/tcmd.exe", False 
    h.send
    lastmod = h.getResponseHeader("Last-Modified")
WScript.Echo(lastmod)

Code:

v:\> cscript ifiletime.vbs
Sun, 21 Nov 2021 20:18:22 GMT

vefatica · 2021-11-26

I modified my script (far below) to get a specified header (default all headers). The distribution TCMD.EXE has an "Age" header but when I get it the results are wacky. I might get any of several values. When I get a value "similar" (i.e., roughly equal) to a previous one, the value is incremented appropriately (apparently in seconds); for example, the ones below marked with '*'. Any idea what's going on? Is there a way to see the headers with a browser?

Code:

v:\> do i=1 to 10 (iheader.vbs "https://jpsoft.com/downloads/v28/tcmd.exe" Age & delay 1)
5531
308060 *
285224
308063 *
308064 *
204314
308066 *
2830
5541
285232

Code:

'IHEADER.VBS

If WScript.Arguments.Count = 0 Or WScript.Arguments.Count > 2 then
    WScript.Echo("Syntax: IHEADER.VBS URL [header_name]" & vbNewLine & "Default: all headers")
    WScript.Quit
End If

Dim h

Set h = CreateObject("MSXML2.ServerXMLHTTP")
h.Open "HEAD", WScript.Arguments.Item(0), False 
h.send

If WScript.Arguments.Count = 2 then
    result = h.GetResponseHeader(WScript.Arguments.Item(1))
Else
    result = h.GetAllResponseHeaders()
End If

If Len(result) = 0 then
    WScript.Echo("Header not found")
Else
    Wscript.Echo(result)
End If

vefatica · 2021-11-26

Hmmm! I changed my plugin @IFILETIME so that it used HTTP_QUERY_AGE instead of HTTP_QUERY_LAST_MODIFIED. I get more consistent results (sort of).

Code:

v:\> do i=1 to 10 (echo %@ifiletime[jpsoft.com/downloads/v28/tcmd.exe] & delay 1)
8519
8520
8521
8522
8523
8525
8526
8527
8528
8529

Moments later,

Code:

v:\> do i=1 to 10 (echo %@ifiletime[jpsoft.com/downloads/v28/tcmd.exe] & delay 1)
206321
206323
206324
206325
206326
206327
206328
206329
206330
206331

And here are two TCCs doing the same thing at the same time. ???????????????

vefatica · 2021-11-26

I figured out how to see the headers with Firefox. The "age" header is all over the place!

Joe Caverly · 2022-11-23

Joe Caverly said:
Code:

_x64: 1 _admin: 1 _elevated: 1 TCC 28.01.14 x64 Windows 10 [Version 10.0.19043.1288]

Here's an alias that allows one to find the last modified date/time of a file on a website;

Code:

last-modified=*pshell /s "$http = new-object System.Net.WebClient; $http.OpenRead('%1').Close(); $http.ResponseHeaders['Last-Modified']"

Proof;

Code:

e:\utils>last-modified https://jpsoft.com/downloads/v28/tcc.exe Sun, 22 Aug 2021 02:59:54 GMT

Joe

Ref: Determine size of file via download link

Thanks for the update in v29 for @FILETIME and @FILEDATE;

Code:

E:\Utils>echo %@filedate[https://jpsoft.com/downloads/sdk/sdk.zip]
2022-11-23

Joe

drwtsn32 · 2022-11-23

vefatica said:
I get [error]

Sorry, just noticed your reply. It works for me on both powershell 5 and 7.3. What version are you using?

Search

Welcome!

Find the Last Modified Date/Time of a file via download link

Joe Caverly

vefatica

Kachupp

Joe Caverly

Joe Caverly

drwtsn32

Kachupp

Kachupp

drwtsn32

Joe Caverly

drwtsn32

vefatica

vefatica

vefatica

Joe Caverly

Joe Caverly

vefatica

Joe Caverly

vefatica

vefatica

vefatica

vefatica

Joe Caverly

drwtsn32