Fixed TYPE http://... incomplete data

#1
On my computer at work, with TCCD16, I get this (below). It's quite incomplete, both at the beginning and at the end. View the page in a browser to see all that I should be getting (a lot more). It works correctly on the same computer with TCC v15, v14, v13, ...
Code:
v:\> type http://w1.weather.gov/data/obhistory/KSYR.html
 size="2" noshade></td>  </tr><tr valign="top"><td class="gray" align="left">National Weather Service<br>Southern Region Headquarters<br>
  Fort Worth, Texas<br><ahref="http://weather.gov/disclaimer">Disclaimer</a><br></td>
  <td><form><input type="button" value="Back to previous page" onClick="history.back()"></form></td><td align="right" class="gray">
  Last Modified: Febuary,7 2012<br><a href="http://weather.gov/privacy">Privacy Policy</a></td>
  </tr></table></body></html>
 
#3
Here's another example showing that "TYPE http://..." is undependable.
Code:
v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  2041

v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  2049
 
#4
I presume wc is the POSIX "word count" with the option -c instructing it to report the character count of the file, at the report sometimes returns the actual file size, and sometimes a value just over 2000 characters. If this is the correct interpretation of your post, it is exactly the same issue - well, exactly the same symptoms - that I have been struggling with "copy http://jpsoft.com/downloads/v16/tcmdx64.exe" (and also tcmd.exe). However, I wonder that the "root" of the website is unchanged between the tests?
 
#6
No, I tried both ways (with and without www) in v14, v15 and v16, all the same size, but 6 different CRC32s. I suspect (not being a web maven) that the root has a code (current time? viewer count?) that is not displayed, which would explain this. What is not explained is when instead of 23,327 characters only a bit more than 2,000 characters showed in Vince's OP. One would need to create a known, unalterable web page, and downloaded many times to validate the issue. Sorry, no can do, but would like to learn the results.
 
#7
Here's another, missing data at both ends. V15 gets it right every time.
Code:
v:\> type http://www.google.com
|0<=a.indexOf("#q="))&&(a=a.substring(1),-1==a.indexOf("#"))){for(var d=0;d<a.le
ngth;){var b=d;"&"==a.charAt(b)&&++b;var c=a.indexOf("&",b);-1==c&&(c=a.length);
b=a.substring(b,c);if(0==b.indexOf("fp="))a=a.substring(0,d)+a.substring(c,a.len
gth),c=d;else if("cad=h"==b)return 0;d=c}_gjwl.href="/search?"+a+"&cad=h";return
1}return 0}
function _gjh(){!_gjuc()&&window.google&&google.x&&google.x({id:"GJH"},function(
){google.nav&&google.nav.gjh&&google.nav.gjh()})};
window._gjh&&_gjh();</script><style>#gbar,#guser{font-size:13px;padding-top:1px
!important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:ri
ght}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:
absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;verti
cal-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important
}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{
color:#900 !important}</style><style>body,td,a,p,.h{font-family:arial,sans-serif
}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac
_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts t
d{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.
lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17p
x arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;
margin-left:4px}input{font-family:inherit}a.gb1,a.gb2,a.gb3,a.gb4{color:#11c !im
portant}body{background:#fff;color:black}a{color:#11c;text-decoration:none}a:hov
er,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}a
.gb1,a.gb4{text-decoration:underline}a.gb3:hover{text-decoration:none}#ghead a.g
b2:hover{color:#fff !important}.sblc{padding-top:5px}.sblc a{display:block;margi
n:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;b
order-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{disp
lay:inline-block;margin:0 12px}.lsb{background:url(/images/srpr/
v:\>
 
#8
FWIW, when v16 uses the ipw9 DLLs from the v15 distribution, there are no errors.
And when v15 uses the ipw9 DLLs from the v16 distribution, the errors mentioned in my earlier posts occur.
 
#9
No, I tried both ways (with and without www) in v14, v15 and v16, all the same size, but 6 different CRC32s. I suspect (not being a web maven) that the root has a code (current time? viewer count?) that is not displayed, which would explain this. What is not explained is when instead of 23,327 characters only a bit more than 2,000 characters showed in Vince's OP. One would need to create a known, unalterable web page, and downloaded many times to validate the issue. Sorry, no can do, but would like to learn the results.
You could try http://lucky.syr.edu (an HTTP front-end for the FTP server). I don't think it changes. Here, the v16/v15 difference is striking:
Code:
d:\tc16> ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  16.01.45  Windows 7 [Version 6.1.7601]
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
Code:
d:\tc15> ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  15.01.57  Windows 7 [Version 6.1.7601]
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,586
46
Albuquerque, NM
prospero.unm.edu
#10
Less consistent here:

Code:
C:\>ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  16.01.45  Windows XP [Version 5.1.2600]
  37861
  37861
  2048
  37861
  2048
  2048
  37861
  1986
  2048
  37861

C:\>
I think Steve's right. This isn't an issue with TYPE, it's an IPWorks issue which also affects COPY, and possibly other commands as well.
 
#11
I recently encountered a similar issue with ftp:. Copying in some RAR archive from our file server left me with an unreadable file (using ftp.exe worked with no problems).
Seems there's a bug somewhere.
 
#15
Install Cygwin. It's part of the base load.
I prefer UnxUtils. In the absence of either, do it with your bare hands!
Code:
v:\> ver & do i=1 to 5 (copy /q http://lucky.syr.edu tmp.tmp & echo %@filesize[tmp.tmp]) & del /q tmp.tmp

TCC  16.01.45  Windows 7 [Version 6.1.7601]
2048
2048
2048
2048
2048
Code:
d:\tc15> ver & do i=1 to 5 (copy /q http://lucky.syr.edu tmp.tmp & echo %@filesize[tmp.tmp]) & del /q tmp.tmp

TCC  15.01.57  Windows 7 [Version 6.1.7601]
37861
37861
37861
37861
37861
 
#17
I prefer UnxUtils.
I find the UnxUtils to be several versions out of date.
In the absence of either, do it with your bare hands!
I tried doing that using the @LEN[] function, but I guess I need a little assistance figuring that out. I tried things like
Code:
echo %@LEN[type http://lucky.syr.edu]
which gave me "25" and then
Code:
echo %@LEN[%@EXEC[type http://lucky.syr.edu]]
which spewed out the source of the website, but then returned "1" since it returned the length of the return value from the @EXEC function instead of the length of the site code.

There's gotta be a TCC command/function I'm missing.
 
#18
I find the UnxUtils to be several versions out of date.

I tried doing that using the @LEN[] function, but I guess I need a little assistance figuring that out. I tried things like
Code:
echo %@LEN[type http://lucky.syr.edu]
which gave me "25" and then
Code:
echo %@LEN[%@EXEC[type http://lucky.syr.edu]]
which spewed out the source of the website, but then returned "1" since it returned the length of the return value from the @EXEC function instead of the length of the site code.

There's gotta be a TCC command/function I'm missing.
I think you're thinking of @LEN[@EXECSTR[ ... but that @EXECSTR gives only one line of the command's output (and @EXEC gives only the command's return value).
 
#19
I tried @EXECSTR[] too. But, as you pointed out, it only returns the first line, which doesn't include the whole site.
 
#20
Looking through the TCC documentation, I found in the Reference > Limits... then under "Miscellaneous Limits (characters)"... "character count in any function" = "20,000". Does that mean the @LEN function would fail anyway since the website is 37k? Or is that talking about the length of user defined functions rather than the input to functions?

Edit: Answered my own question. The @LEN function would fail.
Code:
C:\temp> echo %@LEN[%@REPEAT[x,20000]]
20000

C:\temp> echo %@LEN[%@REPEAT[x,20010]]
TCC: (Sys) The parameter is incorrect.
"%@REPEAT[x,20010]"
20003 characters seems to work, but it fails at 20004 or higher.

Edit 2: Actually, it's the @REPEAT function that's failing above 20003. The @LEN function seems to be able to handle it. I'm not sure if it will work in the end since I got the following
Code:
C:\temp> echo %@LEN[%@REPEAT[x,20000] %@REPEAT[y,20000]]
TCC: Command line too long
TCC has a 32k command line length limit.
 
Last edited:
#26
But it is possible to copy the whole file just to find out its size and immediately discard the copy... One can even go fancier and just count the bytes without saving them as you download them. Of course, unless one has infinite bandwidth it is a lot of wasted effort. What would be totally impossible is to obtain file age. After all, HyperText Transfer Protocol is designed for text, not files. Any tricks to verify content, for example, installer integrity and software updating. is non-standard, custom grafted onto the standard itself.
 
#27
Don't even need to ask. HTTP doesn't provide any way to report file sizes.
I'm not so sure about that. I found a few examples (none in C/C++) that uses Wininet to get the size of **something** (maybe it's the file size). They involve:
hInet = InternetOpen(szMyName, INTERNET_OPEN_TYPE_PRECONFIG, ...)
hConnect = InternetConnect(szServerName, INTERNET_DEFAULT_HTTP_PORT, ...)
hRequest = HttpOpenRequest(hInternet, L"HEAD", szResourceName, ...)
HttpSendRequest(hRequest, ...)
HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, pResult, ...)

I'll play with it a bit.
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
3,586
46
Albuquerque, NM
prospero.unm.edu
#28
I'm not so sure about that. I found a few examples (none in C/C++) that uses Wininet to get the size of **something** (maybe it's the file size). They involve:
hInet = InternetOpen(szMyName, INTERNET_OPEN_TYPE_PRECONFIG, ...)
hConnect = InternetConnect(szServerName, INTERNET_DEFAULT_HTTP_PORT, ...)
hRequest = HttpOpenRequest(hInternet, L"HEAD", szResourceName, ...)
HttpSendRequest(hRequest, ...)
HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, pResult, ...)

I'll play with it a bit.
I look forward to being proven wrong....
 
#29
I look forward to being proven wrong....
OK! :-)
The very sketchy test code (far) below (used in place of 4UTILS's _DESK function) tells me that the size of "http://jpsoft.com/downloads/tcmd.exe" is ...
Code:
l:\projects\4utils\release> timer echo %_desk
Timer 1 on: 22:24:38
21534064
Timer 1 off: 22:24:38  Elapsed: 0:00:00.00
And that's correct. And I'm pretty sure it didn't download the file in that brief time (I was watching with a sniffer anyway ... and it didn't). I won't have any time soon to try to make a robust plugin out of this.
Code:
#include <wininet.h>
INT WINAPI _DESK ( WCHAR *psz )
{
   HINTERNET hInet = InternetOpen(L"JPUser", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
   HINTERNET hConnect = InternetConnect(hInet, L"jpsoft.com", INTERNET_DEFAULT_HTTP_PORT, NULL, NULL, INTERNET_SERVICE_HTTP, 0, 0);
   HINTERNET hRequest = HttpOpenRequest(hConnect, L"HEAD", L"/downloads/v16/tcmd.exe", NULL, NULL, NULL, 0, 0);
   HttpSendRequest(hRequest, NULL, 0, NULL, 0);
   WCHAR Buffer[32];
   DWORD Len = 32 * sizeof(WCHAR), Index = 0;
   HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, Buffer, &Len, &Index);
   Sprintf(psz, L"%s\r\n", Buffer);
   return 0;
}
 
#30
That is apparently limited to **files**. I don't think it will tell you how much data would be sent if you "TYPE http://www.jpsoft.com".
And (obviously) there's no error checking in that code. And I didn't close any of the HINTERNETs. And I didn't need the "\r\n" in the result (it was left over from a Printf()).
 
Last edited: