md5 Checksumming in Windows:
- There is no in-built checksumming tool I know of in Windows, but they are standard in linux etc.
- The best third-party md5 checksum tool I have discovered for Windows is MD5Summer (V. 1.2.0.11) [http://www.md5summer.org/download.html].
- When you run it, it first checks file associations and offers to set it as default for [.md5] files. In W7 an attempt to do so may be denied (admin status required).
- You first select a root-folder then files within it (or all, recursively). To select files, it’s not sufficient to highlight them, you have to double-click them (or click the Add button), so that they appear in the right-hand pane. Only then will the OK button be enabled.
- By running it recursively from a root folder, rather than on individual folders, a single [.md5] file is created that does not pollute the subfolders (e.g. BPAV folders as recorded by XDCAM-EX).
- Had a slight issue once where I manually copied from XP to W7 machines, then generated MD5 on source machine (XP) and verified on target machine (W7). At first, the verifier immediately returned all mismatches. So immediately in fact that it seems likely it was not performing any computation. Later on it worked as expected. Not known what changed to cause this, maybe simply the act of opening the md5 file in NotePad..
Concepts (to the best of my understanding):
- A check-sum of some data is a “fingerprint” numeric value that is probably-unique to that data. A fingerprint is useful when it is much smaller than the data it represents yet (virtually) uniquely identifies the content of that data. A checksum of a file reflects that file’s data-contents but is not affected by the file name etc.
- Various checksum schemes/algorithms exist, one of the most popular being md5. An md5 fingerprint is extremely unlikely to be the same for any other file and will (for that reason) almost certainly change in value if the contents of a file is changed by even the tiniest degree.
- A typical md5 tool will take one or more specified files and generate their fingerprints as corresponding entries (each entry being a filename and a fingerprint) in a [.md5] file. Correspondingly it will take a given [.md5] file and report whether or not the stated and (re-) computed fingerprints agree.
- An [.md5] file is a text file with one or more lines consisting of a checksum value (in hexadecimal) then a space then an asterisk then a file name, possibly preceded by a folder path (with respect to the folder containing the [.md5] file). It can also have comment-lines, each beginning with a hash (#) character. Example entries:
- eb574b236133e60c989c6f472f07827b *fred.exe
- [eb574b236133e60c989c6f472f07827b *tmp/fred.exe].
- Some download sites include [ .md5] files alongside or along with their associated files. Some sites just display the fingerprint itself on the webpage. Typically the purpose is to allow the user to check whether a download was complete or was corrupted.
- The fingerprint computed by a typical checksumming application is not affected by a file’s name or read-only status etc., only by its data contents. Thus it is not a total basis for consistency-checking of system configuration.
More Tools:
- Toast442.org MD5 GUI from [www.toast442.org/md5] is old software designed for Windows 95/98NT. It appears to work OK for files (but not folders) under XP. It is mentioned here as it is the incumbent md5 app I last used, several years ago, on an old machine. Not recommended, just noted here for posterity.
- In Check Files mode you drag an [.md5] file (only) there and if the checksums and files (stated in that [.md5] file) agree then it shows a green blob. Otherwise it shows a red exclamation mark. Also it shows the stated and computed checksums themselves.
- In Generate Checksums mode you drag a file there and its computed checksum value is displayed. A SaveAs button then lets you save it to any filename; a (manually-entered) [.md5] extension being recommended. If you repeat this, e.g. for further files, saving to the same file, then the corresponding further checksums get appended to that file. It acts as write-append, not write-replace.
- Issues
- It looks at first like you can drag folders into it but there appears to be a problem – it generates a red exclamation mark and [.md5] files subsequently saved from it do not have the expected list of files (folder contents). Maybe an aspect of XP that this app doesn’t like?
- It doesn’t appear to like file paths, only file names.
- WinMD5Free (v1.20) is one I discovered by Googling for XP-compatible md5 apps. It seems designed for the web-downloader (type of user) in mind. It has a browser to select the downloaded file, a text-slot where the checksum (e.g. from a webpage or [.md5] file) can be pasted, and a Verify button. Doesn’t look like it generates [.md5] files though.
- MD5Summer ( 1.2.0.11): On one of my systems (MP-BC-XP) there is installed MD5Summer which adds a Verify with MD5Summer option to Windows Explorer‘s folder Context menu. It not only does md5 but also sha1.
- FCIV: Microsoft’s File Checksum Integrity Verifier tool: A commandline utility discovered from [http://www.walkernews.net/2009/12/26/how-to-check-md5-checksum-in-windows-7]. Can both generate and verify, recursively. Downloaded from [http://www.microsoft.com/downloads/details.aspx?FamilyID=B3C93558-31B7-47E2-A663-7365C1686C08&displaylang=en]
Snippets:
- MD5Deep
- [http://en.wikipedia.org/wiki/Md5deep]
- md5deep is a software package used in the computer security, system administration and computer forensics communities for purposes of running large numbers of files through any of several different cryptographic digests (not just md5, despite the name) .
- [http://md5deep.sourceforge.net/]
- Recursive operation – md5deep is able to recursive examine an entire directory tree. That is, compute the MD5 for every file in a directory and for every file in every subdirectory.
- Time estimation – md5deep can produce a time estimate when it’s processing very large files.
- The latest stable versions of md5deep (and hashdeep) are version 3.6 and was released on 23 Mar 2010
- [http://jessekornblum.livejournal.com/tag/md5deep]
- It is … trivial to compute a set of hashes for these files using md5deep, md5sum, or even a single (complicated) line of shell code. But the magic of md5deep is that we can now search for these hashes, recursively.
- First, we compute a set of hashes: [$ md5deep -rl interesting_files/ > known.txt]
- And then we search for those hashes: [$ md5deep -rwm known.txt /home /usr]
- It is … trivial to compute a set of hashes for these files using md5deep, md5sum, or even a single (complicated) line of shell code. But the magic of md5deep is that we can now search for these hashes, recursively.
- [http://en.wikipedia.org/wiki/Md5deep]
- [http://www.afterdawn.com/glossary/term.cfm/md5]
- MD5 data is most commonly distributed as separate .md5 files alongside the main file download files and can be used to check whether the download was complete and whether the data remained uncorrupted during the download process.
- [ http://en.wikipedia.org/wiki/MD5 ] paraphrases / extracts / quotes:
- md5 commandline: e.g. [ MD5(“The quick brown fox”) ]
- The 128-bit (16-byte) MD5 hashes (also termed message digests) are typically represented as a sequence of 32 hexadecimal digits.
- …file servers often provide a pre-computed MD5 (known as Md5sum) checksum for the files, so that a user can compare the checksum of the downloaded file to it. Unix-based operating systems include MD5 sum utilities in their distribution packages, whereas Windows users use third-party applications.
Thoughts:
- For a folder-structure, one could imagine checksumming the unified contents of all its files to a single fingerprint value as opposed to checksumming all of its files individually. Given a folder-structure, it is presumably advisable to follow the latter method, since both generation and checking of all files can be automated, and this method will identify the specific files that differ.