Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

Fixed TYPE http://... incomplete data

vefatica

vefatica

May
13,117
180
On my computer at work, with TCCD16, I get this (below). It's quite incomplete, both at the beginning and at the end. View the page in a browser to see all that I should be getting (a lot more). It works correctly on the same computer with TCC v15, v14, v13, ...
Code: 
v:\> type http://w1.weather.gov/data/obhistory/KSYR.html
 size="2" noshade></td>  </tr><tr valign="top"><td class="gray" align="left">National Weather Service<br>Southern Region Headquarters<br>
  Fort Worth, Texas<br><ahref="http://weather.gov/disclaimer">Disclaimer</a><br></td>
  <td><form><input type="button" value="Back to previous page" onClick="history.back()"></form></td><td align="right" class="gray">
  Last Modified: Febuary,7 2012<br><a href="http://weather.gov/privacy">Privacy Policy</a></td>
  </tr></table></body></html>
 
Here's a PNG showing the same command twice in a row ... with different and incomplete results. It works fine with earlier versions of TCC.
weather.png
 
Here's another example showing that "TYPE http://..." is undependable.
Code: 
v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  2041

v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  23327

v:\> type http://www.jpsoft.com | wc -c
  2049
 
I presume wc is the POSIX "word count" with the option -c instructing it to report the character count of the file, at the report sometimes returns the actual file size, and sometimes a value just over 2000 characters. If this is the correct interpretation of your post, it is exactly the same issue - well, exactly the same symptoms - that I have been struggling with "copy http://jpsoft.com/downloads/v16/tcmdx64.exe" (and also tcmd.exe). However, I wonder that the "root" of the website is unchanged between the tests?
 
No, I tried both ways (with and without www) in v14, v15 and v16, all the same size, but 6 different CRC32s. I suspect (not being a web maven) that the root has a code (current time? viewer count?) that is not displayed, which would explain this. What is not explained is when instead of 23,327 characters only a bit more than 2,000 characters showed in Vince's OP. One would need to create a known, unalterable web page, and downloaded many times to validate the issue. Sorry, no can do, but would like to learn the results.
 
Here's another, missing data at both ends. V15 gets it right every time.
Code: 
v:\> type http://www.google.com
|0<=a.indexOf("#q="))&&(a=a.substring(1),-1==a.indexOf("#"))){for(var d=0;d<a.le
ngth;){var b=d;"&"==a.charAt(b)&&++b;var c=a.indexOf("&",b);-1==c&&(c=a.length);
b=a.substring(b,c);if(0==b.indexOf("fp="))a=a.substring(0,d)+a.substring(c,a.len
gth),c=d;else if("cad=h"==b)return 0;d=c}_gjwl.href="/search?"+a+"&cad=h";return
1}return 0}
function _gjh(){!_gjuc()&&window.google&&google.x&&google.x({id:"GJH"},function(
){google.nav&&google.nav.gjh&&google.nav.gjh()})};
window._gjh&&_gjh();</script><style>#gbar,#guser{font-size:13px;padding-top:1px
!important;}#gbar{height:22px}#guser{padding-bottom:7px !important;text-align:ri
ght}.gbh,.gbd{border-top:1px solid #c9d7f1;font-size:1px}.gbh{height:0;position:
absolute;top:24px;width:100%}@media all{.gb1{height:22px;margin-right:.5em;verti
cal-align:top}#gbar{float:left}}a.gb1,a.gb4{text-decoration:underline !important
}a.gb1,a.gb4{color:#00c !important}.gbi .gb4{color:#dd8e27 !important}.gbf .gb4{
color:#900 !important}</style><style>body,td,a,p,.h{font-family:arial,sans-serif
}body{margin:0;overflow-y:scroll}#gog{padding:3px 8px 0}td{line-height:.8em}.gac
_m td{line-height:17px}form{margin-bottom:20px}.h{color:#36c}.q{color:#00c}.ts t
d{padding:0}.ts{border-collapse:collapse}em{font-weight:bold;font-style:normal}.
lst{height:25px;width:496px}.gsfi,.lst{font:18px arial,sans-serif}.gsfs{font:17p
x arial,sans-serif}.ds{display:inline-box;display:inline-block;margin:3px 0 4px;
margin-left:4px}input{font-family:inherit}a.gb1,a.gb2,a.gb3,a.gb4{color:#11c !im
portant}body{background:#fff;color:black}a{color:#11c;text-decoration:none}a:hov
er,a:active{text-decoration:underline}.fl a{color:#36c}a:visited{color:#551a8b}a
.gb1,a.gb4{text-decoration:underline}a.gb3:hover{text-decoration:none}#ghead a.g
b2:hover{color:#fff !important}.sblc{padding-top:5px}.sblc a{display:block;margi
n:2px 0;margin-left:13px;font-size:11px}.lsbb{background:#eee;border:solid 1px;b
order-color:#ccc #999 #999 #ccc;height:30px}.lsbb{display:block}.ftl,#fll a{disp
lay:inline-block;margin:0 12px}.lsb{background:url(/images/srpr/
v:\>
 
FWIW, when v16 uses the ipw9 DLLs from the v15 distribution, there are no errors.
And when v15 uses the ipw9 DLLs from the v16 distribution, the errors mentioned in my earlier posts occur.
 
Steve Fabian said:
No, I tried both ways (with and without www) in v14, v15 and v16, all the same size, but 6 different CRC32s. I suspect (not being a web maven) that the root has a code (current time? viewer count?) that is not displayed, which would explain this. What is not explained is when instead of 23,327 characters only a bit more than 2,000 characters showed in Vince's OP. One would need to create a known, unalterable web page, and downloaded many times to validate the issue. Sorry, no can do, but would like to learn the results.
Click to expand...
You could try http://lucky.syr.edu (an HTTP front-end for the FTP server). I don't think it changes. Here, the v16/v15 difference is striking:
Code: 
d:\tc16> ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  16.01.45  Windows 7 [Version 6.1.7601]
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
  2049
Code: 
d:\tc15> ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  15.01.57  Windows 7 [Version 6.1.7601]
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
  37861
 
Less consistent here:

Code: 
C:\>ver & do i=1 to 10 (type http://lucky.syr.edu | wc -c)

TCC  16.01.45  Windows XP [Version 5.1.2600]
  37861
  37861
  2048
  37861
  2048
  2048
  37861
  1986
  2048
  37861

C:\>

I think Steve's right. This isn't an issue with TYPE, it's an IPWorks issue which also affects COPY, and possibly other commands as well.
 
I recently encountered a similar issue with ftp:. Copying in some RAR archive from our file server left me with an unreadable file (using ftp.exe worked with no problems).
Seems there's a bug somewhere.
 
LIST seems more reliable, but after trying list /w http://lucky.syr.edu numerous times, I have gotten a partial file on a few transfers. Oddly, the start of the file seems to be lost at least as often as the end.
 
I'd like to try the test above but don't have anything similar to wc - so would someone please send me a copy....
 
Install Cygwin. It's part of the base load.
 
JohnQSmith said:
Install Cygwin. It's part of the base load.
Click to expand...
I prefer UnxUtils. In the absence of either, do it with your bare hands!
Code: 
v:\> ver & do i=1 to 5 (copy /q http://lucky.syr.edu tmp.tmp & echo %@filesize[tmp.tmp]) & del /q tmp.tmp

TCC  16.01.45  Windows 7 [Version 6.1.7601]
2048
2048
2048
2048
2048
Code: 
d:\tc15> ver & do i=1 to 5 (copy /q http://lucky.syr.edu tmp.tmp & echo %@filesize[tmp.tmp]) & del /q tmp.tmp

TCC  15.01.57  Windows 7 [Version 6.1.7601]
37861
37861
37861
37861
37861
 
vefatica said:
I prefer UnxUtils.
Click to expand...
I find the UnxUtils to be several versions out of date.
vefatica said:
In the absence of either, do it with your bare hands!
Click to expand...
I tried doing that using the @LEN[] function, but I guess I need a little assistance figuring that out. I tried things like
Code: 
echo %@LEN[type http://lucky.syr.edu]
which gave me "25" and then
Code: 
echo %@LEN[%@EXEC[type http://lucky.syr.edu]]
which spewed out the source of the website, but then returned "1" since it returned the length of the return value from the @EXEC function instead of the length of the site code.

There's gotta be a TCC command/function I'm missing.
 
JohnQSmith said:
I find the UnxUtils to be several versions out of date.

I tried doing that using the @LEN[] function, but I guess I need a little assistance figuring that out. I tried things like
Code: 
echo %@LEN[type http://lucky.syr.edu]
which gave me "25" and then
Code: 
echo %@LEN[%@EXEC[type http://lucky.syr.edu]]
which spewed out the source of the website, but then returned "1" since it returned the length of the return value from the @EXEC function instead of the length of the site code.

There's gotta be a TCC command/function I'm missing.
Click to expand...
I think you're thinking of @LEN[@EXECSTR[ ... but that @EXECSTR gives only one line of the command's output (and @EXEC gives only the command's return value).
 
I tried @EXECSTR[] too. But, as you pointed out, it only returns the first line, which doesn't include the whole site.
 
Looking through the TCC documentation, I found in the Reference > Limits... then under "Miscellaneous Limits (characters)"... "character count in any function" = "20,000". Does that mean the @LEN function would fail anyway since the website is 37k? Or is that talking about the length of user defined functions rather than the input to functions?

Edit: Answered my own question. The @LEN function would fail.
Code: 
C:\temp> echo %@LEN[%@REPEAT[x,20000]]
20000

C:\temp> echo %@LEN[%@REPEAT[x,20010]]
TCC: (Sys) The parameter is incorrect.
"%@REPEAT[x,20010]"

20003 characters seems to work, but it fails at 20004 or higher.

Edit 2: Actually, it's the @REPEAT function that's failing above 20003. The @LEN function seems to be able to handle it. I'm not sure if it will work in the end since I got the following
Code: 
C:\temp> echo %@LEN[%@REPEAT[x,20000] %@REPEAT[y,20000]]
TCC: Command line too long
TCC has a 32k command line length limit.
 
Last edited:
@REPEAT has a documented count-limit of 32K. So something ain't right!

That's a bug in either the function or the documentation. You ought to start a new "bug" thread about it.
 
vefatica said:
@REPEAT has a documented count-limit of 32K. So something ain't right!

That's a bug in either the function or the documentation. You ought to start a new "bug" thread about it.
Click to expand...
P.S. It seems OK in v14 (but not in v15).
 
But it is possible to copy the whole file just to find out its size and immediately discard the copy... One can even go fancier and just count the bytes without saving them as you download them. Of course, unless one has infinite bandwidth it is a lot of wasted effort. What would be totally impossible is to obtain file age. After all, HyperText Transfer Protocol is designed for text, not files. Any tricks to verify content, for example, installer integrity and software updating. is non-standard, custom grafted onto the standard itself.
 
Charles Dye said:
Don't even need to ask. HTTP doesn't provide any way to report file sizes.
Click to expand...
I'm not so sure about that. I found a few examples (none in C/C++) that uses Wininet to get the size of **something** (maybe it's the file size). They involve:
hInet = InternetOpen(szMyName, INTERNET_OPEN_TYPE_PRECONFIG, ...)
hConnect = InternetConnect(szServerName, INTERNET_DEFAULT_HTTP_PORT, ...)
hRequest = HttpOpenRequest(hInternet, L"HEAD", szResourceName, ...)
HttpSendRequest(hRequest, ...)
HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, pResult, ...)

I'll play with it a bit.
 
vefatica said:
I'm not so sure about that. I found a few examples (none in C/C++) that uses Wininet to get the size of **something** (maybe it's the file size). They involve:
hInet = InternetOpen(szMyName, INTERNET_OPEN_TYPE_PRECONFIG, ...)
hConnect = InternetConnect(szServerName, INTERNET_DEFAULT_HTTP_PORT, ...)
hRequest = HttpOpenRequest(hInternet, L"HEAD", szResourceName, ...)
HttpSendRequest(hRequest, ...)
HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, pResult, ...)

I'll play with it a bit.
Click to expand...

I look forward to being proven wrong....
 
Charles Dye said:
I look forward to being proven wrong....
Click to expand...
OK! :-)
The very sketchy test code (far) below (used in place of 4UTILS's _DESK function) tells me that the size of "http://jpsoft.com/downloads/tcmd.exe" is ...
Code: 
l:\projects\4utils\release> timer echo %_desk
Timer 1 on: 22:24:38
21534064
Timer 1 off: 22:24:38  Elapsed: 0:00:00.00
And that's correct. And I'm pretty sure it didn't download the file in that brief time (I was watching with a sniffer anyway ... and it didn't). I won't have any time soon to try to make a robust plugin out of this.
Code: 
#include <wininet.h>
INT WINAPI _DESK ( WCHAR *psz )
{
   HINTERNET hInet = InternetOpen(L"JPUser", INTERNET_OPEN_TYPE_PRECONFIG, NULL, NULL, 0);
   HINTERNET hConnect = InternetConnect(hInet, L"jpsoft.com", INTERNET_DEFAULT_HTTP_PORT, NULL, NULL, INTERNET_SERVICE_HTTP, 0, 0);
   HINTERNET hRequest = HttpOpenRequest(hConnect, L"HEAD", L"/downloads/v16/tcmd.exe", NULL, NULL, NULL, 0, 0);
   HttpSendRequest(hRequest, NULL, 0, NULL, 0);
   WCHAR Buffer[32];
   DWORD Len = 32 * sizeof(WCHAR), Index = 0;
   HttpQueryInfo(hRequest, HTTP_QUERY_CONTENT_LENGTH, Buffer, &Len, &Index);
   Sprintf(psz, L"%s\r\n", Buffer);
   return 0;
}
 
That is apparently limited to **files**. I don't think it will tell you how much data would be sent if you "TYPE http://www.jpsoft.com".
And (obviously) there's no error checking in that code. And I didn't close any of the HINTERNETs. And I didn't need the "\r\n" in the result (it was left over from a Printf()).
 
Last edited:
You must log in or register to reply here.
Back
Top
[FOX] Ultimate Translator
Translate