Welcome!

By registering with us, you'll be able to discuss, share and private message with other members of our community.

SignUp Now!

Checksum functions

May
238
2
Is there anyone using the various built-in checksum functions like @crc32, @md5, @sha512 etc. for generating and maintaining checksums of backup files (on e.g. external HDs) for verification purposes?

I have considered doing so, but maybe a specialized tool like checksum for Windows or similar would be better? (haven't tried that one yet)

Anyone has any experience or (even better!) good BTM files to share?
 
Yes, I use @CRC32 extensively for such purposes.

I have one .btm, CRC32TAG, which adds a "CRC: 0123CDEF" (example)
description to any file(s) I specify. It's a little smarter than
that, of course. It first checks to see if a description begins
with "CRC: " and if that's followed by a hex string. If it finds
such a match, it skips that file and moves on to the next. If it
doesn't find that, it adds the string to an undescribed file or
prepends it to any existing description.

Its complement is CRC32CHK, which looks for that same specific
format of string in a description. If one is found, the CRC is
computed anew and compared to the stored value. Results are
displayed in green for a match or red for a mismatch, giving a
visual clue that's very easy to spot. I like and use both .btms on
a regular basis. Writing them was time well spent. I almost never
have any need to create .md5, .sfv or .par2 files for myself
anymore. And since I nearly always use TCC to perform copy and move
operations, the descriptions travel around nicely from directory to
directory or drive to drive.

I also have a few PDIR commands that integrate just that portion of
a description into their output.

I realize that @CRC32 is probably the weakest of my available
choices in TCC but it produces the shortest output and is therefore
best suited to my needs.


-- Dan McMullin
 
Thank you for sharing! That was a very interesting approach.

I had considered storing the checksums in some way (planned to use MD5 rather than CRC perhaps) but updating and querying the stored values in a single file seemed a little troublesome. (I'd rather avoid having the overhead (and clutter) of an extra .md5 file for every archived file)

Never thought to have DESCRIBE and @DESCRIPT do the heavy lifting! Much easier and should hopefully also be faster than parsing custom txt or xml files manually with a btm script. I currently don't use file descriptions at all (never saw the need with long file names) so it will not collide with any existing uses either in my case.

I've since also discovered Microsoft's FCIV tool to generate and compare checksums (MD5 or SHA) to an XML file for either single files or whole directory trees, but it seems it does not support incremental updates of the stored checksums, and I would rather avoid regenerating the checksums for several gigabytes just for adding a few small files! This should be no problem at all for a TCC script on the other hand.
 
From: nikbackm
| I had considered storing the checksums in some way (planned to use
| MD5 rather than CRC perhaps) but updating and querying the stored
| values in a single file seemed a little troublesome. (I'd rather
| avoid having the overhead (and clutter) of an extra .md5 file for
| every archived file)
On CRC32 vs. MD5 - I have read, but cannot cite a source, that MD5 - despite its more complex algorithm and longer hash code - has more "collisions" (files that are different but have the same hash code) than CRC32. Since the sole purpose is to detect unwanted changes, I'd go for CRC32.
BTW, the PDIR command's output options include field r - the same CRC32 generated by the @CRC32[] function. You may benefit from its use in the program to generate or to verify the hash codes.
--
Steve
 
One of my PDIR aliases places a portion of any description just
before the "r" field so that I get this type of output:

... CRC: DEADBEEF DEADBEEF ...
... CRC: F00DFACE 00C0FFEE ...
... <Undescribed> 01020304 ...

This also provides a convenient way to verify that the files have or
haven't changed but has the "drawback" of always generating the
CRC. In addition to having color-coded output to catch my eye, the
.btm approach allows me to skip generating the CRC on any file(s)
which don't already contain it in the description, which would be
wasted effort for my purposes.


-- Dan McMullin
 

Similar threads

Back
Top