1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

LIST /X and TYPE /X give different results

Discussion in 'Support' started by dcantor, Mar 14, 2009.

  1. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3
    I have a file 00ff.bin which is 256 bytes long and each byte's 8-bit value is its own offset from the beginning of the file; i.e., byte 0 contains 0, byte 65 contains 0x41, byte 255 contains 0xFF.

    LIST /X 00ff.bin results in

    Code:
    0000 0000 00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  .☺☻♥♦♣♠•◘.....♫☼
    0000 0010 10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  ►◄↕‼¶§▬↨↑↓→.∟↔▲▼
    0000 0020 20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f   !"#$%&'()*+,-./
    0000 0030 30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  0123456789:;<=>?
    0000 0040 40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  @ABCDEFGHIJKLMNO
    0000 0050 50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  PQRSTUVWXYZ[\]^_
    0000 0060 60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  `abcdefghijklmno
    0000 0070 70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  pqrstuvwxyz{|}~⌂
    0000 0080 80 81 82 83 84 85 86 87  88 89 8a 8b 8c 8d 8e 8f  ÇüéâäàåçêëèïîìÄÅ
    0000 0090 90 91 92 93 94 95 96 97  98 99 9a 9b 9c 9d 9e 9f  ÉæÆôöòûùÿÖÜ¢£¥₧ƒ
    0000 00a0 a0 a1 a2 a3 a4 a5 a6 a7  a8 a9 aa ab ac ad ae af  áíóúñѪº¿⌐¬½¼¡«»
    0000 00b0 b0 b1 b2 b3 b4 b5 b6 b7  b8 b9 ba bb bc bd be bf  ░▒▓│┤╡╢╖╕╣║╗╝╜╛┐
    0000 00c0 c0 c1 c2 c3 c4 c5 c6 c7  c8 c9 ca cb cc cd ce cf  └┴┬├─┼╞╟╚╔╩╦╠═╬╧
    0000 00d0 d0 d1 d2 d3 d4 d5 d6 d7  d8 d9 da db dc dd de df  ╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀
    0000 00e0 e0 e1 e2 e3 e4 e5 e6 e7  e8 e9 ea eb ec ed ee ef  αßΓπΣσµτΦΘΩδ∞φε∩
    0000 00f0 f0 f1 f2 f3 f4 f5 f6 f7  f8 f9 fa fb fc fd fe ff  ≡±≥≤⌠⌡÷≈°∙·√ⁿ²■ 
    
    as expected. However, TYPE /X 00FF.BIN results in

    Code:
    0000 0000 00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f  ......¤   
    0000 0010 10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f  ¶§.   
    0000 0020 20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f   !"#$%&'()*+,-./   
    0000 0030 30 31 32 33 34 35 36 37  38 39 3a 3b 3c 3d 3e 3f  0123456789:;<=>?   
    0000 0040 40 41 42 43 44 45 46 47  48 49 4a 4b 4c 4d 4e 4f  @ABCDEFGHIJKLMNO   
    0000 0050 50 51 52 53 54 55 56 57  58 59 5a 5b 5c 5d 5e 5f  PQRSTUVWXYZ[\]^_   
    0000 0060 60 61 62 63 64 65 66 67  68 69 6a 6b 6c 6d 6e 6f  `abcdefghijklmno   
    0000 0070 70 71 72 73 74 75 76 77  78 79 7a 7b 7c 7d 7e 7f  pqrstuvwxyz{|}~   
    0000 0080 c7 fc e9 e2 e4 e0 e5 e7  ea eb e8 ef ee ec c4 c5  ¦nTGSastOdFne8-+   
    0000 0090 c9 e6 c6 f4 f6 f2 fb f9  ff d6 dc a2 a3 a5 20 19  +µ¦(÷=v· +_óúÑºÆ   
    0000 00a0 e1 ed f3 fa f1 d1 aa ba  bf 23 ac bd bc a1 ab bb  ßf=·±-¬¦+¼++í½+   
    0000 00b0 25 25 25 25 25 25 25 25  25 25 25 25 25 25 25 25  æÆô$abVUcQW]\[   
    0000 00c0 25 25 25 25 25 25 25 25  25 25 25 25 25 25 25 25  ¶4,
    0000 00d0 25 25 25 25 25 25 25 25  25 25 25 25 25 25 25 25  hdeYXRSkjêäîÉÇ   
    0000 00e0 3b df 39 3c 3a 3c b5 3c  3a 39 3a 3b 22 3c 3b 22  ¦¯ô+ú+¦-ªÿ¬¦¦¦)   
    0000 00f0 22 b1 22 22 23 23 f7 22  b0 22 b7 22 20 b2 25 a0  a¦ed !˜H¦+¦áá   
    
    Why the difference, or more importantly, why does TYPE /X give incorrect results?
     
  2. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    Was your output from TYPE meant to be truncated or is that all you got (32
    bytes)? When I try it, I get full (256 bytes) output from both LIST and TYPE,
    but it looks like they use different code pages (I don't know how well that will
    show up below). Here, it seems LIST uses code page 437 (the active code page)
    and TYPE uses code page 1252.

    v:\> type /x 00FF.BIN
    0000 0000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f .????.....?¤
    0000 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f ????¶§?????.????
    0000 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f !"#$%&'()*+,-./
    0000 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 0123456789:;<=>?
    0000 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f @ABCDEFGHIJKLMNO
    0000 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f PQRSTUVWXYZ[\]^_
    0000 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f `abcdefghijklmno
    0000 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f pqrstuvwxyz{|}~¦
    0000 0080 c7 fc e9 e2 e4 e0 e5 e7 ea eb e8 ef ee ec c4 c5 ¦nTGSastOdFne8-+
    0000 0090 c9 e6 c6 f4 f6 f2 fb f9 ff d6 dc a2 a3 a5 20 19 +µ¦(÷=v·*+_óúѺÆ
    0000 00a0 e1 ed f3 fa f1 d1 aa ba bf 23 ac bd bc a1 ab bb ßf=·±-¬¦+?¼++í½+
    0000 00b0 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 æÆô?$abVUcQW]\[?
    0000 00c0 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 ¶4,?
    0000 00d0 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 25 hdeYXRSkj??êäîÉÇ
    0000 00e0 3b df 39 3c 3a 3c b5 3c 3a 39 3a 3b 22 3c 3b 22 ¦¯ô+ú+¦-ªÿ¬¦?¦¦)
    0000 00f0 22 b1 22 22 23 23 f7 22 b0 22 b7 22 20 b2 25 a0 a¦ed !˜H¦?+?¦¦áá

    v:\> list /x 00FF.BIN
    00FF.BIN ¦ F1 Help ¦ Col 0 Line 1
    0000 0000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f .??????•?.....?¤
    0000 0010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f ????¶§?????.????
    0000 0020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f !"#$%&'()*+,-./
    0000 0030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 0123456789:;<=>?
    0000 0040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f @ABCDEFGHIJKLMNO
    0000 0050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f PQRSTUVWXYZ[\]^_
    0000 0060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f `abcdefghijklmno
    0000 0070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f pqrstuvwxyz{|}~¦
    0000 0080 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f ÇüéâäàåçêëèïîìÄÅ
    0000 0090 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f ÉæÆôöòûùÿÖÜ¢£¥Pƒ
    0000 00a0 a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af áíóúñѪº¿¬¬½¼¡«»
    0000 00b0 b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf ¦¦¦¦¦¦¦++¦¦+++++
    0000 00c0 c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf +--+-+¦¦++--¦-+-
    0000 00d0 d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df ---++++++++¦_¦¦¯
    0000 00e0 e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef aßGpSsµtFTOd8fen
    0000 00f0 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff =±==()÷˜°··vn²¦*

    --
    - Vince
     
  3. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    vefatica wrote:
    | Was your output from TYPE meant to be truncated or is that all you
    | got (32 bytes)? When I try it, I get full (256 bytes) output from
    | both LIST and TYPE, but it looks like they use different code pages
    | (I don't know how well that will show up below). Here, it seems LIST
    | uses code page 437 (the active code page) and TYPE uses code page
    | 1252.

    All issues below affect the glyphs displayed:
    - is TCC in its own window, or a TCMD tab
    - codepage
    - Unicode option

    On my WinXP SP3 system running TCC in its own window, %@option[unicode]=no,
    %_codepage=437, LIST displays boxes, and TYPE shows the proper glyphs, both
    for control codes (0-31) and non-ASCII codes (128-255).
    --
    Steve
     
  4. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3

    I got all 256 bytes with both TYPE /X and LIST /X, and I can see them in my own original post.

    I don't know what the code page would have to do with anything. Binary is binary.
     
  5. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3
    I'm not complaining about the glyphs of the characters; I'm talking about the binary values of the bytes (i.e., the hexadecimal codes).


    The purpose of both LIST /X and TYPE /X are to see the hexadecimal values, not the character glyphs.
     
  6. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    On Sat, 14 Mar 2009 11:54:43 -0500, dcantor <> wrote:

    |The purpose of both LIST /X and TYPE /X are to see the hexadecimal values, not the character glyphs.

    TYPE /X certainly doesn't do that correctly.
    --
    - Vince
     
  7. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3
    Yes. That's my point.
     
  8. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    First, TYPE (as documented) is intended only for text files, and will
    invariably have problems with binary files. Do not use TYPE with binary
    files!!

    Second, the reason for the difference (*not* incorrect, just diferent -- at
    least according to Windows) is because TYPE is converting the ASCII text to
    Unicode prior to display (necessary in order to display it on the console),
    and Windows is responsible for all of the interesting conversion. (Windows
    is converting it according to the default code page.)

    Rex Conn
    JP Software
     
  9. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3
    Thanks for the response.

    Ah, so sorry. I misunderstood. I thought that TYPE /X was intended so one could see the hex values of the individual bytes. I thought you had implemented a way to get the equivalent of LIST /X out to a file.

    Back to using XXD, I guess.
     
  10. rconn

    rconn Administrator
    Staff Member

    Joined:
    May 14, 2008
    Messages:
    9,870
    Likes Received:
    83
    dcantor wrote:


    That's not possible without removing support for redirection, piping,
    and environment variables. (Not to mention having to rewrite TCC in a
    different language ...)

    Rex Conn
    JP Software
     
  11. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    It would be easy to do this in a plugin command (looks better in a console):

    g:\projects\toh\release> toh.exe hw.txt

    00000000 48 65 6C 6C 6F 20 57 6F 72 6C 64 21 0D 0A 48 65 Hello World!..He
    00000010 6C 6C 6F 20 57 6F 72 6C 64 21 0D 0A 48 65 6C 6C llo World!..Hell
    00000020 6F 20 57 6F 72 6C 64 21 0D 0A 48 65 6C 6C 6F 20 o World!..Hello
    00000030 57 6F 72 6C 64 21 0D 0A World!..

    The hex would be accurate and the text displayed (32-127) would be at the mercy of TCC's Printf() function. You could redirect that to a file. ... any interest? Suggestions are welcomed.
     
  12. dcantor

    Joined:
    May 29, 2008
    Messages:
    507
    Likes Received:
    3

    Okay, Rex. I understand that it's a restriction.

    That must be the reason that @FILEWRITEB[%handle,-1,217] writes a 55 instead of 217. (That value is the current year (2009) mod 256.)

    The help page on TYPE doesn't say that it needs to be a text file (that's kind of obvious when typing the text as text, but not for /X because it says "Display the file in hex" and there's no restriction documented. I suggest changing type.htm to read:

    <table style="line-height: normal;" border="0" cellpadding="0" cellspacing="0"> <tbody> <tr style="vertical-align: baseline;" valign="baseline"> <td width="72">Purpose:</td> <td>Display the contents of the specified text file(s).

    </td></tr></tbody></table>and

    file The text file or list of text files that you want to display


    So short of writing a bat-file or using an external image like XXD, there is still currently no way to get a byte-by-byte hex dump of a file to anywhere but the screen. Redirecting LIST /X output results in nothing. Please consider that a wish for the next version.










     
  13. Jim Cook

    Joined:
    May 20, 2008
    Messages:
    604
    Likes Received:
    0
    As implemented, TYPE /X is useless to me. The only time I want to see
    the hex values is when I suspect something funny is going on, and I
    want to know exactly what bytes are present. If the ASCII column shows
    only 0x20 to 0x7F well, that won't matter to me, as long as the hex
    bytes are represented accurately. I do regularly use third party tools
    to dump in hex the contents of arbitrary files. I look at more core
    dumps from embedded systems than I wish for.

    If I must only use TYPE /X with ASCII files, how do I make that
    determination? TYPE /X can not be used to tell, since it does not
    display the actual contents of the file. What characters' presence
    makes TYPE /X fail?

    If TYPE /X misrepresents the bytes by design, then it can not be
    trusted. For me, that makes it useless.

    In my testing, with a file 64K words long, with each little endian
    word in order, I see that TYPE /X does not faithfully represent the
    contents of the file.

    For example, early in the file, I see the words 0x0070 to 0x007F
    followed by 0x00C7 0x00FC, and later by a string of many 0x0025.

    TYPE /X output:
    0000 00e0 70 00 71 00 72 00 73 00 74 00 75 00 76 00 77 00
    0000 00f0 78 00 79 00 7a 00 7b 00 7c 00 7d 00 7e 00 7f 00
    0000 0100 c7 00 fc 00 e9 00 e2 00 e4 00 e0 00 e5 00 e7 00
    0000 0110 ea 00 eb 00 e8 00 ef 00 ee 00 ec 00 c4 00 c5 00
    0000 0120 c9 00 e6 00 c6 00 f4 00 f6 00 f2 00 fb 00 f9 00
    0000 0130 ff 00 d6 00 dc 00 a2 00 a3 00 a5 00 20 00 19 00
    0000 0140 e1 00 ed 00 f3 00 fa 00 f1 00 d1 00 aa 00 ba 00
    0000 0150 bf 00 23 00 ac 00 bd 00 bc 00 a1 00 ab 00 bb 00
    0000 0160 25 00 25 00 25 00 25 00 25 00 25 00 25 00 25 00
    0000 0170 25 00 25 00 25 00 25 00 25 00 25 00 25 00 25 00
    0000 0180 25 00 25 00 25 00 25 00 25 00 25 00 25 00 25 00

    Whereas a clip from LIST /X shows that same portion to be as expected:
    0000 00e0 70 00 71 00 72 00 73 00 74 00 75 00 76 00 77 00
    0000 00f0 78 00 79 00 7a 00 7b 00 7c 00 7d 00 7e 00 7f 00
    0000 0100 80 00 81 00 82 00 83 00 84 00 85 00 86 00 87 00
    0000 0110 88 00 89 00 8a 00 8b 00 8c 00 8d 00 8e 00 8f 00
    etc.


    On Sun, Mar 15, 2009 at 3:28 PM, rconn <> wrote:



    --
    Jim Cook
    2009 Saturdays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
    Next year they're Sunday.
     
  14. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    I suspect that 100% of users, seeing

    00000000 XX XX XX ...

    will take it to mean that these are hex representations of the bytes in the file. If they're not that, they're useless.
     
  15. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30
    My TCMD.INI is a Unicode **text** file. "TYPE TCMD.INI" shows it nicely.

    type d:\tcmd10\TCMD.INI
    [4NT]
    PauseOnError=No
    DirWinOpen=PgDn
    MailServer=smtp-server
    [snip]

    "TYPE /X" shows garbage; it seems to be skipping every second character (and
    it's printing spaces instead of dots, which I did not ask it to do).

    type /x d:\tcmd10\TCMD.INI
    0000 0000 5b 00 4e 00 5d 00 0a 00 61 00 73 00 4f 00 45 00 [ N ] . a s O E
    0000 0010 72 00 72 00 4e 00 0d 00 44 00 72 00 69 00 4f 00 r r N . D r i O
    0000 0020 65 00 3d 00 67 00 6e 00 0a 00 61 00 6c 00 65 00 e = g n . a l e
    0000 0030 76 00 72 00 73 00 74 00 2d 00 65 00 76 00 72 00 v r s t - e v r
    0000 0040 0a 00 61 00 6c 00 64 00 72 00 73 00 3d 00 65 00 . a l d r s = e
    0000 0050 61 00 69 00 61 00 76 00 66 00 74 00 63 00 2e 00 a i a v f t c .
    0000 0060 65 00 0d 00 3b 00 6f 00 6d 00 6c 00 64 00 74 00 e . ; o m l d t
    [snip]
    --
    - Vince
     
  16. samintz

    samintz Scott Mintz

    Joined:
    May 20, 2008
    Messages:
    1,191
    Likes Received:
    11
    HEXDUMP.BTM:

    Code:
    setlocal
    setdos /x-45678
    set fh=%@fileopen[%1,r,b]
    set r=%@filereadb[%fh,16]
    set aset ofs=0
    do while %r != **EOF**
            echos %@format[08,%@convert[10,16,%ofs]]:
            do i = 0 to %@dec[%@words[%r]]
                    set w=%@format[02,%@convert[10,16,%@word[%i,%r]]]
                    echos. %[w]
                    set a=%[a]%@if[0x%w lt 0x20,.,%@if[0x%w == 
    0x20,%@char[1],%@char[0x%w]]]
            enddo
            echo. %@repeat[ ,%@eval[3 * (16-%@words[%r])]] 
    %@replace[%@char[1], ,%a]
            set a        set r=%@filereadb[%fh,16]
            set /a ofs+=16
    enddo
    set fh=%@fileclose[%fh]
    endlocal
    
    -Scott

    vefatica <> wrote on 03/15/2009 11:14:43 PM:


    file.

    console):

    World!..He

    World!..Hell

    World!..Hello

     
  17. vefatica

    Joined:
    May 20, 2008
    Messages:
    7,972
    Likes Received:
    30

    That sure got mangled! Is it anywhere near complete? What about the line that says "set a set r=%@filereadb[%fh,16]"?
     
  18. samintz

    samintz Scott Mintz

    Joined:
    May 20, 2008
    Messages:
    1,191
    Likes Received:
    11
    It looks like 2 lines got mangled. The line "set aset ofs=0" is supposed
    to be 2 lines. I originally had "set a=" on one line and "set ofs=0" on
    another.

    if I change the line to "unset a" it would probably avoid the mangling.
    Apparently, you can't end a line with an equals sign and expect it to get
    through.

    I modified the code below to use UNSET.

    vefatica <> wrote on 03/16/2009 07:52:22 PM:


    > ---End Quote---
    >
    > That sure got mangled! Is it anywhere near complete? What about
    > the line that says "set a set r=%@filereadb[%fh,16]"?
    >
    >
    >
    -Scott
     
  19. samintz

    samintz Scott Mintz

    Joined:
    May 20, 2008
    Messages:
    1,191
    Likes Received:
    11
    I attached my HEXDUMP.BTM to this thread.

    -Scott
     

    Attached Files:

  20. Steve Fabian

    Joined:
    May 20, 2008
    Messages:
    3,520
    Likes Received:
    4
    samintz wrote:
    | I attached my HEXDUMP.BTM to this thread.
    |
    | -Scott
    |
    | Attached to this message is hexdump.btm
    | (http://jpsoft.com/forums/attachment.php?attachmentid=51&d=1237299571)

    NOTE: to view or download attachments you have to use the browser access,
    for some reason they are not made part of the email delivery (as they ought
    to be).
    --
    Steve
     
  21. TEA-Time

    Joined:
    Jun 2, 2008
    Messages:
    282
    Likes Received:
    1
    Personally, I'd rather not be forced to "download" everyone's attachments. And I think I recall Rex stating a while back that it would put a big load on the mail server.
     

Share This Page