BTM Base64

Rex Clark · 2008-09-01

I'm messing around with Base64 encoding in BTM flavour I have a basic working
model. I have come against a brick wall mainly because mathematics wasn't really
my strong point. anyone done this before ?

If anyone interested I could post what I have. you may be able to suggest a fix
for my problem.

Jim Cook · 2008-09-02

Feel free to post, or email to me directly. I'll help if I can.

On Mon, Sep 1, 2008 at 8:08 PM, Rex Clark <> wrote:

> I'm messing around with Base64 encoding in BTM flavour I have a basic
> working
> model. I have come against a brick wall mainly because mathematics wasn't
> really
> my strong point. anyone done this before ?
>
> If anyone interested I could post what I have. you may be able to suggest a
> fix
> for my problem.
>
>
>
>
>

--
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
Measure wealth by the things you have for which you would not take money.

Jim Cook · 2008-09-02

Feel free to post, or email to me directly. I'll help if I can.

On Mon, Sep 1, 2008 at 8:08 PM, Rex Clark <> wrote:

> I'm messing around with Base64 encoding in BTM flavour I have a basic
> working
> model. I have come against a brick wall mainly because mathematics wasn't
> really
> my strong point. anyone done this before ?
>
> If anyone interested I could post what I have. you may be able to suggest a
> fix
> for my problem.
>
>
>
>
>

--
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
Measure wealth by the things you have for which you would not take money.

Rex Clark · 2008-09-02

Thank you Jim

if {%1} == {} quit
if exist %@name[%1].B64 del /kq %@name[%1].b64

SetLocal
(unset /q * %+ unalias /q * %+ unfunction /q *)

:Functions

function be64=`0%@instr[0,,%@convert[10,2,%@unicode[%1]]]`
function bits=`%@instr[%@convert[2,10,%_bin],1,%1]`

:Base64Alphabet

SET BASE64= ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/

set open_0=%@fileopen[%1,r,b]
set open_1=%@fileopen[%@name[%1].B64,w,b]
set open_2=%@fileopen[%@name[%1].BIN,w,b]

do read=0 to %@filesize[%1,b]
if %_kbhit == 1 leave
set _bit=%@be64[%@fileread[%open_0,1]]
if %_bit == 00 set _bit=%@format[08,0]
set write=%@filewriteb[%open_2,8,%_bit]
enddo

unset /q _bit write read
gosub writebase64
gosub close

unfunction /q *
endlocal
quit

:Close

set open_ > close.all
set _ca=%@lines[close.all]
if {%_ca} == {-1} return
do _xa=0 to %_ca
set close=%@fileclose[%@field["=",1,1%@line[close.all,%_xa]]]
enddo
del /kq close.all %@name[%1].bin
return

:WriteBase64

set max=%@fileseek[%open_2,0,2]
set jmp=%@fileseekl[%open_2,0]

do read=0 to %max by 5
set _bin=%@fileread[%open_2,6]
if %_bin == **EOF** leave
set _pin=%@eval[%_pin + %write]
if %_pin ge 72 set write=%@filewrite[%open_1,] && unset /q _pin
if %read lt %max .and. %@len[%_bin] lt 6 gosub pad && leave
set write=%@filewriteb[%open_1,1,%@bits[%base64]]
enddo
return

:Pad

bdebugger /ne

set pad=0%@fileread[%open_2,6]

do _xr=0 to %@len[%pad] by 6
set _bin=%@instr[%_xr,6,%pad]
if %@len[%_bin] lt 6 set _bin=%@format[06,0]
set write=%@filewriteb[%open_1,1,%@bits[%base64]]
enddo

unset /q pad _xr write
return

Jim Cook · 2008-09-02

> do read=0 to %@filesize[%1,b]
> if %_kbhit == 1 leave
> set _bit=%@be64[%@fileread[%open_0,1]]
> if %_bit == 00 set _bit=%@format[08,0]
> set write=%@filewriteb[%open_2,8,%_bit]
> enddo

If you get rid of your be64 function and replace your loop with the
following, I think the program will work.

There are much more efficient means, but I think this is the smallest
perturbation to your algorithm.

do read=0 to %@filesize[%1,b]
if %_kbhit == 1 leave
set _bit=%@format[08,%@convert[10,2,%@filereadb[%open_0,1]]]
set write=%@filewriteb[%open_2,8,%_bit]
enddo

To make the program much more efficient, instead of converting to
binary and back to BASE64, you could construct a number with something
like the following. Note that I just typed this into the email without
actually trying it. You'll need to test for EOF in the filereadb.

set number=%@filereadb[%open_0,1]
set number=%@eval[(number SHL 8) + %@filereadb[%open_0,1]]
set number=%@eval[(number SHL 8) + %@filereadb[%open_0,1]]

Then deconstruct the three bytes (24 bits) into the four 6-bit chunks with
set sixbit=%@eval[(%number SHR 18) AND 0x3F]
set wrote=%@filewriteb[%open_1,1,%sixbit]
set sixbit=%@eval[(%number SHR 12) AND 0x3F]
set wrote=%@filewriteb[%open_1,1,%sixbit]
set sixbit=%@eval[(%number SHR 6) AND 0x3F]
set wrote=%@filewriteb[%open_1,1,%sixbit]
set sixbit=%@eval[(%number SHR 0) AND 0x3F]
set wrote=%@filewriteb[%open_1,1,%sixbit]

--
2008 Fridays: 4/4, 6/6, 8/8, 10/10, 12/12 and 5/9, 9/5, 7/11, 11/7.
Next year they're Saturday.
Measure wealth by the things you have for which you would not take money.

Rex Clark · 2008-09-02

Jim the post has an error .. in the alphabet section remove the space after
BASE64=

Rex Clark · 2008-09-02

Thank you jim worked like a charm..

do read=0 to %@filesize[%1,b]
if %_kbhit == 1 leave
set _bit=%@format[08,%@convert[10,2,%@unicode[%@fileread[%open_0,1]]]]
set write=%@filewriteb[%open_2,8,%_bit]
enddo

Rex Clark · 2008-09-08

From: "Jim Cook" <>
| set number=%@filereadb[%open_0,1]
| set number=%@eval[(number SHL 8) + %@filereadb[%open_0,1]]
| set number=%@eval[(number SHL 8) + %@filereadb[%open_0,1]]
|
| Then deconstruct the three bytes (24 bits) into the four 6-bit chunks with
| set sixbit=%@eval[(%number SHR 18) AND 0x3F]
| set wrote=%@filewriteb[%open_1,1,%sixbit]
| set sixbit=%@eval[(%number SHR 12) AND 0x3F]
| set wrote=%@filewriteb[%open_1,1,%sixbit]
| set sixbit=%@eval[(%number SHR 6) AND 0x3F]
| set wrote=%@filewriteb[%open_1,1,%sixbit]
| set sixbit=%@eval[(%number SHR 0) AND 0x3F]
| set wrote=%@filewriteb[%open_1,1,%sixbit]

I Changed my version to reflect your suggestion. 100% faster than previous

Code:

do read=0 to %@filesize[%1,b] by 3
  if %_kbhit == 1 leave
   set _bin=%@unicode[%@fileread[%open_0,1]]
   set _bin=%@eval[(%_bin SHL 8) + %@unicode[%@fileread[%open_0,1]]]
   set _bin=%@eval[(%_bin SHL 8) + %@unicode[%@fileread[%open_0,1]]]
   set stepdown=18
   do d=0 to 3
    set sixbit=%@eval[(%_bin SHR %stepdown) AND 0x3f]
    set out=%@filewriteb[%open_1,1,%@instr[%sixbit,1,%base64]]
    (set /a stepdown=stepdown - 6 %+ set /a _eol=%_eol + %out)
   enddo
  if %_eol == 72 set crlf=%@filewrite[%open_1,] && unset /q _eol
 enddo

Thanks again

samintz · 2010-02-18

Rex,

I like to keep things simple. Here's my take on your Base64 encoder:

Code:

setlocal
setdos /x-45678
set fh=%@fileopen[%1,r,b]
set r=%@filereadb[%fh,3]
set base64=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
set ofs=0
do while %r != **EOF**
    do i = 0 to %@dec[%@words[%r]] by 3
        set w=%@word[%i,%r]
        set w=%@eval[(%w SHL 8) + %@word[%@eval[%i+1],%r]]
        set w=%@eval[(%w SHL 8) + %@word[%@eval[%i+2],%r]]

        rem Deconstruct the three bytes (24 bits) into the four 6-bit chunks
        set sixbit=%@eval[(%w SHR 18) AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
        set sixbit=%@eval[(%w SHR 12) AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
        set sixbit=%@eval[(%w SHR 6) AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
        set sixbit=%@eval[%w AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
        set /a ofs+=3
    enddo
    set r=%@filereadb[%fh,3]
    iff %ofs ge 72 then
        echo. 
        set ofs=0
    endiff
enddo
set fh=%@fileclose[%fh]
echo.
endlocal

You would use it like this:

Code:

Base64 file.bin > file.b64

-Scott

samintz · 2010-02-18

I published that last one a tad too soon. I actually looked up how Base64 is supposed to work after I published it. The new version correctly appends the right number of ='s at the end.

Here is a corrected version. I also eliminated an internal loop that I copied and pasted that wasn't supposed to be there.

Code:

setlocal
setdos /x-45678
set fh=%@fileopen[%1,r,b]
set r=%@filereadb[%fh,3]
set base64=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
set ofs=0
do while %r != **EOF**
    set wds=%@words[%r]
    set w=%@word[0,%r]
    set w=%@eval[(%w SHL 8) + %@word[1,%r]]
    set w=%@eval[(%w SHL 8) + %@word[2,%r]]

    rem Deconstruct the three bytes (24 bits) into the four 6-bit chunks
    set sixbit=%@eval[(%w SHR 18) AND 0x3F]
    echos %@instr[%sixbit,1,%base64]
    set sixbit=%@eval[(%w SHR 12) AND 0x3F]
    echos %@instr[%sixbit,1,%base64]
    iff %wds ge 2 then
        set sixbit=%@eval[(%w SHR 6) AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
    else
        echos =
    endiff
    iff %wds == 3 then
        set sixbit=%@eval[%w AND 0x3F]
        echos %@instr[%sixbit,1,%base64]
    else
        echos =
    endiff
    set /a ofs+=3
    iff %ofs ge 72 then
        echo. 
        set ofs=0
    endiff
    set r=%@filereadb[%fh,3]
enddo
set fh=%@fileclose[%fh]
if %ofs != 0 echo.
endlocal

-Scott

Kachupp · 2010-02-18

Character length should be 76 base-64 characters in length
Your version is 96!
My last version I never posted incorporated chunk deconstruction shown to me by Jim Cook
It work alot faster, I also wrote the original one in 4nt ver8.
I'm messing with the decoder now (have been for a while) lack of interest on my part has
slowed it somewhat

---- Original Message -----
From: "samintz" <>

Sent: Friday, February 19, 2010 12:12 PM
Subject: RE: [T&T - Scripting-t-426] Re: BTM Base64

>I published that last one a tad too soon. I actually looked up how Base64 is supposed to
>work after I published it. The new version correctly appends the right number of ='s at
>the end.
>
> Here is a corrected version. I also eliminated an internal loop that I copied and
> pasted that wasn't supposed to be there.
>
>
> Code:
> ---------
> setlocal
> setdos /x-45678
> set fh=%@fileopen[%1,r,b]
> set r=%@filereadb[%fh,3]
> set base64=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/
> set ofs=0
> do while %r != **EOF**
> set wds=%@words[%r]
> set w=%@word[0,%r]
> set w=%@eval[(%w SHL 8) + %@word[1,%r]]
> set w=%@eval[(%w SHL 8) + %@word[2,%r]]
>
> rem Deconstruct the three bytes (24 bits) into the four 6-bit chunks
> set sixbit=%@eval[(%w SHR 18) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> set sixbit=%@eval[(%w SHR 12) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> iff %wds ge 2 then
> set sixbit=%@eval[(%w SHR 6) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> else
> echos > endiff
> iff %wds == 3 then
> set sixbit=%@eval[%w AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> else
> echos > endiff
> set /a ofs+=3
> iff %ofs ge 72 then
> echo.
> set ofs=0
> endiff
> set r=%@filereadb[%fh,3]
> enddo
> set fh=%@fileclose[%fh]
> if %ofs != 0 echo.
> endlocal
> ---------
> -Scott
>
>
>
>

samintz · 2010-02-19

Doh! I added 3 to ofs instead of 4 in the loop! It was supposed to
insert cr-lf after 72 chars.

change the line: set /a ofs+=3
to: set /a ofs+=4

-Scott

Kachupp <>
02/18/2010 10:16 PM
Please respond to

To
[email protected]
cc

Subject
RE: [T&T - Scripting-t-426] Re: BTM Base64

Character length should be 76 base-64 characters in length
Your version is 96!
My last version I never posted incorporated chunk deconstruction shown to
me by Jim Cook
It work alot faster, I also wrote the original one in 4nt ver8.
I'm messing with the decoder now (have been for a while) lack of interest
on my part has
slowed it somewhat

---- Original Message -----
From: "samintz" <>

Sent: Friday, February 19, 2010 12:12 PM
Subject: RE: [T&T - Scripting-t-426] Re: BTM Base64

---Quote---

>I published that last one a tad too soon. I actually looked up how

Base64 is supposed to

>work after I published it. The new version correctly appends the right

number of ='s at

>the end.
>
> Here is a corrected version. I also eliminated an internal loop that I

copied and

> pasted that wasn't supposed to be there.
>
>
> Code:
> ---------
> setlocal
> setdos /x-45678
> set fh=%@fileopen[%1,r,b]
> set r=%@filereadb[%fh,3]
> set

base64=ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/

> set ofs=0
> do while %r != **EOF**
> set wds=%@words[%r]
> set w=%@word[0,%r]
> set w=%@eval[(%w SHL 8) + %@word[1,%r]]
> set w=%@eval[(%w SHL 8) + %@word[2,%r]]
>
> rem Deconstruct the three bytes (24 bits) into the four 6-bit chunks
> set sixbit=%@eval[(%w SHR 18) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> set sixbit=%@eval[(%w SHR 12) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> iff %wds ge 2 then
> set sixbit=%@eval[(%w SHR 6) AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> else
> echos > endiff
> iff %wds == 3 then
> set sixbit=%@eval[%w AND 0x3F]
> echos %@instr[%sixbit,1,%base64]
> else
> echos > endiff
> set /a ofs+=3
> iff %ofs ge 72 then
> echo.
> set ofs=0
> endiff
> set r=%@filereadb[%fh,3]
> enddo
> set fh=%@fileclose[%fh]
> if %ofs != 0 echo.
> endlocal
> ---------
> -Scott
>
>
>
>

---End Quote---

Steve Fabian · 2010-02-19

| set /a ofs+=4

I cannot find any HELP documentation of the availability of C's fancy
"selfmodifying" operators.
--
Steve

samintz · 2010-02-19

Copied from CMD /C SET /?

SET /A expression

The /A switch specifies that the string to the right of the equal sign
is a numerical expression that is evaluated. The expression evaluator
is pretty simple and supports the following operations, in decreasing
order of precedence:

() - grouping
! ~ - - unary operators
* / % - arithmetic operators
+ - - arithmetic operators
<< >> - logical shift
& - bitwise and
^ - bitwise exclusive or
| - bitwise or
= *= /= %= += -= - assignment
&= ^= |= <<= >> , - expression separator

If you use any of the logical or modulus operators, you will need to
enclose the expression string in quotes. Any non-numeric strings in the
expression are treated as environment variable names whose values are
converted to numbers before using them. If an environment variable name
is specified but is not defined in the current environment, then a value
of zero is used. This allows you to do arithmetic with environment
variable values without having to type all those % signs to get their
values. If SET /A is executed from the command line outside of a
command script, then it displays the final value of the expression. The
assignment operator requires an environment variable name to the left of
the assignment operator. Numeric values are decimal numbers, unless
the assignment operator. Numeric values are decimal numbers, unless
prefixed by 0x for hexadecimal numbers, and 0 for octal numbers.
So 0x12 is the same as 18 is the same as 022. Please note that the octal
notation can be confusing: 08 and 09 are not valid numbers because 8 and
9 are not valid octal digits.

-Scott

Steve F$BaC(Bi$BaO(B <> wrote on 02/19/2010 11:25:50
AM:

> | set /a ofs+=4
>
> I cannot find any HELP documentation of the availability of C's fancy
> "selfmodifying" operators.
> --
> Steve
>
>
>
>

Steve Fabian · 2010-02-19

| Copied from CMD /C SET /?
...
Thanks for the information. My complaint was that it is not documented
anywhere in TCMD.CHM. I never use CMD and know nothing about its syntax. By
the time I acquired my first WinNT system I already had 4nt, and had no need
to take a step (actually, a giant leap) backward. This leaves my out when
TCC emulates some feature of CMD without documentation, and I am sure there
are many like me.
--
Steve

rconn · 2010-02-21

> | Copied from CMD /C SET /?
> ...
> Thanks for the information. My complaint was that it is not documented
> anywhere in TCMD.CHM. I never use CMD and know nothing about its syntax.
> By the time I acquired my first WinNT system I already had 4nt, and had no
> need
> to take a step (actually, a giant leap) backward. This leaves my out when
> TCC emulates some feature of CMD without documentation, and I am sure
> there are many like me.

I don't generally document useless CMD features (like this one). If you
want to know all of the CMD features, see the non-existent Microsoft
documentation.

Rex Conn
JP Software

Steve Fabian · 2010-02-21

| I don't generally document useless CMD features (like this one). If
| you
| want to know all of the CMD features, see the non-existent Microsoft
| documentation.

One of my favorite features in C is the set of operators in the += *= /=
category, and ever since I discovered them I wished all HLLs had it. Until
Mr. Mintz's post I had no idea that they are available in TCC! They'd have
simplified hundreds of lines in my batch programs. I consider them extremely
useful TCC features, regardless of the implication you consider them
imitations of useless CMD features. And no, I'm not at all interested in CMD
features, but any TCC features that are not documented (unless you plan to
remove them in the near future) potentially reduce TCMD sales. Documenting a
CMD feature which is emulated by referring to its MS documentation is fine.
When a feature is documented neither by MS nor by JPsoft it might as well
not exist.
--
Steve

Charles Dye · 2010-02-21

rconn said:
I don't generally document useless CMD features (like this one). If you want to know all of the CMD features, see the non-existent Microsoft documentation.

But be aware that not all of the useless features documented by Microsoft are supported by TCC. E.g. cmd.exe's evil behavior of assuming that all numbers beginning in '0' should be interpreted as octal is not emulated -- thank goodness and Rex!

Search

Welcome!

BTM Base64

Rex Clark

Jim Cook

Jim Cook

Rex Clark

Jim Cook

Rex Clark

Rex Clark

Rex Clark

samintz

Scott Mintz

samintz

Scott Mintz

Kachupp

samintz

Scott Mintz

Steve Fabian

samintz

Scott Mintz

Steve Fabian

rconn

Administrator

Steve Fabian

Charles Dye

Super Moderator