Archive for the ‘checksum’ Category

MD5 Tools on Mac OS and Windows 7

Friday, October 18th, 2013

The md5 checksum algorithm is very old and nowadays regarded as “breakable”, to such an extent that it is not recommended for use in data transmission (through interfering environments) protocols.  However it is handy, well-established. and hence warm/fuzzy…

On Mac OS, the ‘md5’ command is inbult to the OS:

  •  Compute an MD5 checksum:
    • $ md5 test.mp4
      • MD5 (test1.mp4) = f3ed1559874599d26fddc5802d65266e
    • $ md5 *
      • MD5 (test1.mp4) = f3ed1559874599d26fddc5802d65266e
      • MD5 (test2.mp4) = b9bfa87a6a126911f2246c7a615bff27
  • For help: [ man md5 ]
  • To check a file against an expected checksum value:
    • md5 -c CHECKSUM.MD5
    • Where CHECKSUM.MD5 has (only) multiple entries (lines) as returned by MD5 command, e.g.:
      • MD5 (test1.mp4) = f3ed1559874599d26fddc5802d65266e
      • MD5 (test2.mp4) = b9bfa87a6a126911f2246c7a615bff27
  • ???

On Windows 7:

  • There is nothing inbuilt to the OS…
  • Third-party applications are available:
    • Probably best to use a non-Microsoft one, e.g. a linux-like one…
    • There are also many user-friendly/convenience (GUI-based checksum applications, offering additional checksum varieties, beyond md5.
    • http://support.microsoft.com/kb/841290
      • The FCIV utility runs on Microsoft Windows 2000, Windows XP, and Windows Server 2003
        • No mention of Windows 7 but that was after its time(2004) so maybe still worth a try?
      • Download link:
      • The Microsoft File Checksum Integrity Verifier (FCIV) utility is an unsupported command-line utility that computes MD5 or SHA1 cryptographic hashes for files. Microsoft does not provide support for this utility. Use this utility at your own risk. Microsoft Product Support Services (PSS) cannot answer questions about the File Checksum Integrity Verifier utility.
        • Weird!  Why would they maintain such a …discouraging… attitude?
      • For help: [ fciv.exe /? ]
      • Capabilities:
        • Supports MD5 or SHA1 hash algorithms (The default is MD5.)
        • Can output hash values to the console or store the hash value and file name in an XML file
        • Can recursively generate hash values for all files in a directory and in all subdirectories (for example, fciv.exe c:\ -r)
        • Supplies an exception list to specify files or directories to hash
        • Can store hash values for a file with or without the full path of the file
    • ?

Posted in checksum, Mac, Windows, Windows 7 | No Comments »

Checksum-based Backup Methodology – Thoughts

Friday, December 10th, 2010

Looks like I’ve evolved a hybrid system:

  1. Proper backup / synch tool: SuperFlexibleFileSynchronizer (abbrev to “SuperFlex”).
  2. Procedural with lower-level tools: File drag-copy combined with MD5Summer.

The idea is that normally I would use SuperFlex, but for occasions where I already have manually-created (supposed) mirrors, I can retrospectively check consistency at content (not just datetime and size) level.  Have yet to experiment with SuperFlex to see if it can verify existing copies of files (as opposed to copies that it is making).

Posted in backup, checksum, file mgt, Mac, md5, Windows | No Comments »

md5 Check-Sum Apps for Windows

Thursday, December 9th, 2010

md5 Checksumming in Windows:

  • There is no in-built checksumming tool I know of in Windows, but they are standard in linux etc.
  • The best third-party md5 checksum tool I have discovered for Windows is MD5Summer (V. 1.2.0.11)  [http://www.md5summer.org/download.html].
    • When you run it, it first checks file associations and offers to set it as default for [.md5] files.  In W7 an attempt to do so may be denied (admin status required).
    • You first select a root-folder then files within it (or all, recursively).  To select files, it’s not sufficient to highlight them, you have to double-click them (or click the Add button), so that they appear in the right-hand pane.  Only then will the OK button be enabled.
    • By running it recursively from a root folder, rather than on individual folders, a single [.md5] file is created that does not pollute the subfolders (e.g. BPAV folders as recorded by XDCAM-EX).
    • Had a slight issue once where I manually copied from XP to W7 machines, then generated MD5 on source machine (XP) and verified on target machine (W7).  At first, the verifier immediately returned all mismatches.  So immediately in fact that it seems likely it was not performing any computation.  Later on it worked as expected.  Not known what changed to cause this, maybe simply the act of opening the md5 file in NotePad..

Concepts (to the best of my understanding):

  • A check-sum of some data is a “fingerprint” numeric value that is probably-unique to that data.  A fingerprint is useful when it is much smaller than the data it represents yet (virtually) uniquely identifies the content of that data.  A checksum of a file reflects that file’s data-contents but is not affected by the file name etc.
  • Various checksum schemes/algorithms exist, one of the most popular being md5.   An md5 fingerprint is extremely unlikely to be the same for any other file and will (for that reason) almost certainly change in value if the contents of a file is changed by even the tiniest degree.
  • A typical md5 tool will take one or more specified files and generate their fingerprints as corresponding entries (each entry being a filename and a fingerprint) in a [.md5] file.  Correspondingly it will take a given [.md5] file and report whether or not the stated and (re-) computed fingerprints agree.
  • An [.md5] file is a text file with one or more lines consisting of a checksum value (in hexadecimal) then a space then an asterisk then a file name, possibly preceded by a folder path (with respect to the folder containing the [.md5] file).  It can also have comment-lines, each beginning with a hash (#) character.  Example entries:
    • eb574b236133e60c989c6f472f07827b *fred.exe
    • [eb574b236133e60c989c6f472f07827b *tmp/fred.exe].
  • Some download sites include [ .md5] files alongside or along with their associated  files.  Some sites just display the fingerprint itself on the webpage.  Typically the purpose is to allow the user to check whether a download was complete or was corrupted.
  • The fingerprint computed by a typical checksumming application is not affected by a file’s name or read-only status etc., only by its data contents.  Thus it is not a total basis for consistency-checking of system configuration.

(more…)

Posted in checksum, file mgt, md5, util | No Comments »