How to? swap endianness in a binary file

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
Since tpipe /simple=61 doesn't work for my needs, what is the easiest way to do it?
I have a long list of 32-bit hexadecimal numbers:
f8000008
f80225fc
f88b4d37
f89689e3
f877aeeb
(snip)

And I want to change the endianness of it to:
080000f8
fc2502f8
374d8bf8
e38996f8
ebae77f8

So it's not just a reversal of the bytes - that would cause f8 to become 8f. In this case I want 2 bytes kept together but the groups of 4x2 to be reversed.

I could use a DO loop and @INSTR. I'm not sure how efficient that is. Is it possible to use TPIPE to do this?
 
May 20, 2008
11,423
99
Syracuse, NY, USA
I just did this.

Code:
tpipe /input=xavtemp.btm /output=xavtemp.rev /simple=61

The BTM has many 0x0D characters. The output file is the same siza as the input.

2021-01-12 23:43 1,764 xavtemp.btm
2021-06-24 21:29 1,764 xavtemp.rev

Here's a bit of comparison. It looks like TPIPE did exactly what you're asking for.

1624584799166.png


1624584838070.png


40 65 63 68 became 68 63 65 40 and so on.

And F8 is one whole byte. TPIPE will put it in another position but it won't turn it into 8F
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
In my case, the binary data being corrupted was
00000500 D0 0D 04 00 22 00 02 26 40 00 05 00 42 00 EC 00 Ð..."..&@...B.ì.
And it got turned into this:
00000500 04 0A 0D D0 02 00 22 00 05 00 40 26 EC 00 42 00 ...Ð.."...@&ì.B.

So it acts like the 0D was turned into 0D 0A - so the D0 0D 04 00 became D0 0D 0A 04 00 and then got reversed. But the trailing 00 ends up getting pushed into the next group of 4 bytes so the whole sequence is off from that point forward.

Using Charles' suggestion of adding the /EOL switch fixed it.
 
May 20, 2008
11,423
99
Syracuse, NY, USA
40 65 63 68 became 68 63 65 40 and so on.
And that's NOT how simple=61 is documented. If the help is correct, it should have turned 40 65 63 68 into 63 68 45 65. From the help simple=61)

TPIPE - Text filtering, search, and substitution
This filter swaps groups of 2-byte words

Byte number12345678
Input FileFFFE000000310000
Output File0000FEFF00003100
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
I think it does what it is supposed to - 32 bit endian swap. This text "This filter swaps groups of 2-byte words" is a little misleading. But the example in the table is correct.
 

Charles Dye

Super Moderator
Staff member
May 20, 2008
4,469
88
Albuquerque, NM
prospero.unm.edu
So this really is a binary file — not a text file containing hexadecimal digits in ASCII.

You're a brave man. I wouldn't use TPIPE for that. I'd probably write a custom executable, unless I was in a real hurry.
 
May 20, 2008
11,423
99
Syracuse, NY, USA
I think it does what it is supposed to - 32 bit endian swap. This text "This filter swaps groups of 2-byte words" is a little misleading. But the example in the table is correct.
But that's not what it did. Look at the two hex edit pics. It's swapping the endian of 32-bit fields (4-byte DWORDs). 01020304 becomes 04030201.
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
I have it as hex ascii originally. I pasted it into HxD and saved the binary file. I tried the reverse the line stuff but it reverses characters not bytes - i.e. F8BC becomes CB8F instead of BCF8.

This works to do the hex ascii reversal:
Code:
do l in @pka_fw.hex (echo %@instr[6,2,%l]%@instr[4,2,%l]%@instr[2,2,%l]%@instr[0,2,%l])
 
May 20, 2008
11,423
99
Syracuse, NY, USA
Scott, in ASCII, characters are bytes. It's hard to figure out what you want. The ASCII string ABCD (4bytes) is 65666768 when written in hex notation. Apparently you want it to become CDAB, or, in hex, 67686566. That's swapping WORDs (2-byte pairs). Is that right? The help says simple=61 is supposed to do that, but it doesn't. It swaps two words and also swaps the two bytes within each word, effectively reversing each consecutive 4-bytes.
 

samintz

Scott Mintz
May 20, 2008
1,503
16
Solon, OH, USA
Well I'm obviously not doing a good job explaining myself. I have hex ascii data that is in big endian order and I need it in little endian order. The data is uint32_t.
I essentially have: uint32_t data[1024];
And each element of data needs to be changed from big to little endian.

One approach to is to reverse the hex ascii. Another is to import the hex ascii and convert it to binary and reverse the binary.

I have tried both approaches. The DO loop with @INSTR works on the hex ascii. Tpipe /simple=61 works with the /EOL switch set to ignore EOL's.
 
May 20, 2008
11,423
99
Syracuse, NY, USA
So you have 8-byte hex representations of 4-byte numbers. That is, you have, say, 3ADE68B1 (an 8-byte character string representing the decimal number 987654321).

... and you want to turn that into B168DE3A (an 8-byte character string representing the decimal number 2976439866).

You could do

Code:
echo %@revbytes[3ADE68B1]
B168DE3A

where

Code:
v:\> function revbytes
%@eval[((0x%1 AND 0xFF000000) SHR 24) OR ((0x%1 AND 0x00FF0000) SHR 8) OR ((0x%1 AND 0xFF00) SHL 8) OR ((0x%1 AND 0xFF) SHL 24)=H]

But I doubt it's an improvement on your @INSTR method.
 

Similar threads