A checksum file is the least interesting part of a release until the day a download is corrupt, and then it is the only part that matters. On Linux the tool that writes one and the tool that reads it back are the same binary: sha256sum prints digests when you give it files, and verifies them when you give it -c and a manifest. Most people only ever use the first half, copy the digest into a terminal and compare it by eye.
The second half is where the detail lives. The manifest has a format, that format has a field almost nobody knows is a field, and the commands that read it disagree about what belongs in it. md5sum, sha1sum and sha256sum share one interface and differ only in the digest, shasum is a separate program that produces the same shape of line, and a manifest written by one of them can fail under another in a way that looks like a missing file rather than a format problem.
Every block below was run on GNU coreutils 9.4 and shasum 6.04 (Perl Digest::SHA), against the four file folder the prerequisite section creates, so the digests and the exit codes reproduce. Where a block measures something else, the sentence above it says what.
Applies to: GNU coreutils on any Linux distribution, and shasum wherever Perl is installed
Quick answer
Write the manifest from inside the folder the names should be relative to, then verify it the same way and read the exit status rather than the OK lines.
# write a manifest for everything in the current folder
sha256sum * > SHA256SUMS
# verify it: one line per file, and an exit status that is 0 only if all of them passed
sha256sum -c SHA256SUMS
echo $?
# in a script, say nothing and test the status
sha256sum -c --status SHA256SUMS || echo 'verification failed'
Before the first example
Four things have to be true for the blocks below to produce the output shown. The first two are almost certainly already true on your machine.
1. GNU coreutils provides the commands. They ship in the same package, so they report the same version.
sha256sum --version | head -1
md5sum --version | head -1
sha256sum (GNU coreutils) 9.4
md5sum (GNU coreutils) 9.4
2. shasum is present for the cross tool section. It ships with Perl rather than with coreutils, so a minimal container may not have it.
shasum --version
perl -v | sed -n '2p'
6.04
This is perl 5, version 38, subversion 2 (v5.38.2) built for x86_64-linux-gnu-thread-multi
3. A folder with four files of known content. printf is used rather than echo because the bytes have to be exact: a shell that adds a different line ending produces a different digest, and then none of the values below match.
mkdir -p ~/hashlab/release && cd ~/hashlab
printf 'zaur.it sample payload v1.4.0\n' > release/app-1.4.0.tar.gz
printf 'zaur.it sample installer v1.4.0\n' > release/app-1.4.0.exe
printf 'release notes for 1.4.0\n' > release/README.txt
printf 'scratch\n' > release/notes.txt
4. Every block runs from ~/hashlab. That matters more than it looks: the names in a manifest are resolved against the working directory at check time, which has a section of its own below.
What one line of output actually is
Hash a file twice, once in the default mode and once with -b, and look at the two lines with cat -A so that nothing is hidden. The $ at the end is the line ending that cat -A draws, not part of the output.
sha256sum release/app-1.4.0.tar.gz > SHA256SUMS
sha256sum -b release/app-1.4.0.tar.gz >> SHA256SUMS
cat -A SHA256SUMS
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 release/app-1.4.0.tar.gz$
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 *release/app-1.4.0.tar.gz$
The digest is identical. What changed is the single character sitting between the digest and the file name. The GNU manual describes the untagged format in one sentence, and it is worth reading slowly:
A space, then a flag, then the name. The flag is a field of its own, one character wide, and the manual says what goes in it: binary mode is indicated with * and text mode with a space. Text mode is the default on Linux, which is why the usual line looks as though it has two spaces in it. It does not. It has one space and one empty looking field.
| Part of the line | Width | What it holds |
|---|---|---|
| Digest | 64 characters for SHA-256 | lower case hexadecimal |
| Separator | 1 character | always a space |
| Mode flag | 1 character | * for binary, a space for text |
| File name | the rest of the line | exactly the argument that was given |
-b description says so outright: this option merely flags each input mode as binary, and the checksum is unaffected. The distinction exists for the systems the manual calls MS-DOS like, which do treat the two kinds of file differently.
The second space is not a space
That one character field is the reason hand written manifest parsers break, and they break on exactly the lines that were produced carefully. Here are four idioms a shell script reaches for, each run against a text mode line and a binary mode line of the same file.
cd release
sha256sum app-1.4.0.tar.gz > SHA256SUMS
sha256sum -b app-1.4.0.tar.gz >> SHA256SUMS
text=$(head -1 SHA256SUMS)
bin=$(tail -1 SHA256SUMS)
want=app-1.4.0.tar.gz
hits=0
row() { # row "idiom" "line type" "result"
[ "$3" = "$want" ] && hits=$((hits + 1))
printf '%-34s%-10s%s\n' "$1" "$2" "$3"
}
printf '%-34s%-10s%s\n' 'idiom' 'line' 'the file name it returns'
row "awk '{print \$2}'" text "$(printf '%s\n' "$text" | awk '{print $2}')"
row '' binary "$(printf '%s\n' "$bin" | awk '{print $2}')"
row "cut -d' ' -f3" text "$(printf '%s\n' "$text" | cut -d' ' -f3)"
row '' binary "$(printf '%s\n' "$bin" | cut -d' ' -f3)"
row "cut -c67-" text "$(printf '%s\n' "$text" | cut -c67-)"
row '' binary "$(printf '%s\n' "$bin" | cut -c67-)"
row "sed 's/^[0-9a-f]\{64\} .//'" text "$(printf '%s\n' "$text" | sed 's/^[0-9a-f]\{64\} .//')"
row '' binary "$(printf '%s\n' "$bin" | sed 's/^[0-9a-f]\{64\} .//')"
echo
printf '%-34s%s\n' 'parses tested' 8
printf '%-34s%s\n' 'returning the file name' "$hits"
idiom line the file name it returns
awk '{print $2}' text app-1.4.0.tar.gz
binary *app-1.4.0.tar.gz
cut -d' ' -f3 text app-1.4.0.tar.gz
binary
cut -c67- text app-1.4.0.tar.gz
binary app-1.4.0.tar.gz
sed 's/^[0-9a-f]\{64\} .//' text app-1.4.0.tar.gz
binary app-1.4.0.tar.gz
parses tested 8
returning the file name 6
Six of the eight return the file name. The two that do not are both reading a binary mode line: awk hands back the name with the flag glued to the front, and cut -d' ' -f3 hands back nothing at all, because in a binary line there is no third space separated field. Neither reports an error.
-b. The two idioms that survive both lines are the ones that treat position 66 as a field rather than as part of a separator.
If a script has to read one of these files, the honest version of the pattern is the sed form above: sixty four hexadecimal characters, one space, any one character, then the name. The better version is not to parse it at all, which the next sections are about.
A file name can rewrite the whole line
There is a second rule in the same format, and it changes the line before the digest. When a file name contains a backslash, a newline or a carriage return, the manual says the line is started with a backslash and each problematic character in the name is escaped.
cd release
touch 'back\slash.txt'
sha256sum 'back\slash.txt'
sha256sum 'back\slash.txt' > ESCAPED
sha256sum -c ESCAPED
\e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 back\\slash.txt
\back\\slash.txt: OK
The line now begins with a backslash, so the digest starts at character two, and the single backslash in the name is written as two. The -c run reads it back correctly and prints the escaped form in its report. The parsers that survived the binary flag do not survive this.
cd release
want='back\slash.txt'
a=$(cut -c67- ESCAPED)
b=$(cut -c68- ESCAPED)
c=$(sed 's/^[0-9a-f]\{64\} .//' ESCAPED)
hits=0
for got in "$a" "$b" "$c"; do
[ "$got" = "$want" ] && hits=$((hits + 1))
done
printf '%-36s%s\n' 'the first character of the line' "$(cut -c1 ESCAPED)"
printf '%-36s%s\n' 'the name on disk' "$want"
echo
printf '%-36s%s\n' 'cut -c67- returns' "$a"
printf '%-36s%s\n' 'cut -c68- returns' "$b"
printf '%-36s%s\n' 'the sed above returns' "$(printf '%s' "$c" | cut -c1-30)"
echo
printf '%-36s%s\n' 'parses tested' 3
printf '%-36s%s\n' 'returning the name on disk' "$hits"
the first character of the line \
the name on disk back\slash.txt
cut -c67- returns back\\slash.txt
cut -c68- returns back\\slash.txt
the sed above returns \e3b0c44298fc1c149afbf4c8996fb
parses tested 3
returning the name on disk 0
None of the three returns the name that is on disk, and the sed form fails the hardest: the leading backslash stops the digest matching at all, so the pattern does not fire and the whole line comes back untouched.
Example 1: write a manifest for a release folder
The names recorded in the file are the arguments as given, so the folder you stand in when you write it decides what the manifest says. Writing it from inside the folder keeps the names short and keeps them portable.
cd release
sha256sum app-1.4.0.tar.gz app-1.4.0.exe README.txt > SHA256SUMS
cat SHA256SUMS
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 app-1.4.0.tar.gz
c63677527d625de6ea2aca66e81f2bf0917f90654188a263a48245c113d6f7ef app-1.4.0.exe
13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930 README.txt
Three files named, three lines, no header and no algorithm name anywhere in the file. That last point is worth holding on to: nothing in a manifest says which digest produced it. The reader has to know, or has to guess from the length of the first field, which is the only clue there is. The four lengths are measured further down, in the section on the commands that agree.
| Command | Hex characters per line | What the file is usually called |
|---|---|---|
md5sum | 32 | MD5SUMS, or .md5 beside each file |
sha1sum | 40 | SHA1SUMS |
sha256sum | 64 | SHA256SUMS |
sha512sum | 128 | SHA512SUMS |
Example 2: check it, and read the exit status
The half of the tool that gets skipped is -c. It reads the manifest, hashes each named file, prints a verdict per line and sets an exit status for the run as a whole. Five scenarios run in sequence, each one damaging the folder a little more, with stdout, stderr and the status reported separately.
cd release
report() { # report "label" command...
label=$1; shift
out=$("$@" 2>stderr.txt); rc=$?
printf '%-34s%s\n' 'scenario' "$label"
printf '%-34s%s\n' 'stdout' "$(printf '%s' "$out" | tr '\n' '|')"
printf '%-34s%s\n' 'stderr' "$(tr '\n' '|' < stderr.txt)"
printf '%-34s%s\n\n' 'exit status' "$rc"
}
sha256sum app-1.4.0.tar.gz app-1.4.0.exe README.txt > SHA256SUMS
report 'every file matches' sha256sum -c SHA256SUMS
printf 'one more byte\n' >> app-1.4.0.exe
report 'one file altered' sha256sum -c SHA256SUMS
mv app-1.4.0.exe elsewhere.exe
report 'one file not there' sha256sum -c SHA256SUMS
report 'one file not there, ignore-missing' sha256sum -c --ignore-missing SHA256SUMS
mv app-1.4.0.tar.gz elsewhere.tar.gz
mv README.txt elsewhere.txt
report 'no listed file is there' sha256sum -c --ignore-missing SHA256SUMS
scenario every file matches
stdout app-1.4.0.tar.gz: OK|app-1.4.0.exe: OK|README.txt: OK
stderr
exit status 0
scenario one file altered
stdout app-1.4.0.tar.gz: OK|app-1.4.0.exe: FAILED|README.txt: OK
stderr sha256sum: WARNING: 1 computed checksum did NOT match|
exit status 1
scenario one file not there
stdout app-1.4.0.tar.gz: OK|app-1.4.0.exe: FAILED open or read|README.txt: OK
stderr sha256sum: app-1.4.0.exe: No such file or directory|sha256sum: WARNING: 1 listed file could not be read|
exit status 1
scenario one file not there, ignore-missing
stdout app-1.4.0.tar.gz: OK|README.txt: OK
stderr
exit status 0
scenario no listed file is there
stdout
stderr sha256sum: SHA256SUMS: no file was verified|
exit status 1
The first three behave the way the output reads. The last two are the ones that cost an afternoon.
| Scenario | Printed per file | Exit status |
|---|---|---|
| Everything matches | OK | 0 |
| A file was altered | FAILED | 1 |
| A file is not there | FAILED open or read | 1 |
A file is not there, with --ignore-missing | nothing for that file | 0 |
None of the listed files is there, with --ignore-missing | nothing at all | 1 |
--ignore-missing exists so you can verify a subset of a large manifest, and it does that job well. If the download step silently produced nothing, the same option turns a folder with no files in it into a run that prints nothing on stdout. The only thing separating it from a clean pass is the exit status and one line on stderr reading that no file was verified.
The manual states the rule plainly, and all three clauses matter: if any listed file cannot be opened or read, if any valid line has a checksum inconsistent with the associated file, or if no valid line is found, the command exits with nonzero status.
Example 3: quiet and status suppress different things
Two options reduce the noise and they are not interchangeable. Both are run here against a manifest of two files where one of them has been altered.
cd release
sha256sum app-1.4.0.exe README.txt > SHA256SUMS
printf 'one more byte\n' >> app-1.4.0.exe
oneline() { tr '\n' '|' | sed 's/|$//'; }
report() {
label=$1; shift
out=$("$@" 2>stderr.txt); rc=$?
printf '%-12s%-40s%-56s%s\n' "$label" \
"$(printf '%s' "$out" | oneline)" "$(oneline < stderr.txt)" "$rc"
}
printf '%-12s%-40s%-56s%s\n' 'option' 'stdout' 'stderr' 'exit'
report 'none' sha256sum -c SHA256SUMS
report '--quiet' sha256sum -c --quiet SHA256SUMS
report '--status' sha256sum -c --status SHA256SUMS
option stdout stderr exit
none app-1.4.0.exe: FAILED|README.txt: OK sha256sum: WARNING: 1 computed checksum did NOT match 1
--quiet app-1.4.0.exe: FAILED sha256sum: WARNING: 1 computed checksum did NOT match 1
--status 1
--quiet drops the OK lines and keeps the failures, so a passing run prints nothing on stdout and a failing run prints only what went wrong. --status prints nothing on either stream and leaves the exit status as the entire result, which is what a script wants.
--quiet and disappears under --status. If a cron job is mailing you its stderr, that is the difference between hearing about a bad file and not.
Example 4: the manifest the other tool cannot read
Nothing in the file says which algorithm wrote it, so the obvious question is what happens when the wrong command reads it. Three manifests, all handed to sha256sum -c: one written by md5sum, one empty, and one written by shasum in its Universal Newlines mode.
cd release
report() {
label=$1; shift
out=$("$@" 2>stderr.txt); rc=$?
printf '%-34s%s\n' 'scenario' "$label"
printf '%-34s%s\n' 'stdout' "$(printf '%s' "$out" | tr '\n' '|')"
printf '%-34s%s\n' 'stderr' "$(tr '\n' '|' < stderr.txt)"
printf '%-34s%s\n\n' 'exit status' "$rc"
}
md5sum app-1.4.0.tar.gz > MD5SUMS
report 'md5 manifest, sha256sum -c' sha256sum -c MD5SUMS
: > EMPTY
report 'empty manifest' sha256sum -c EMPTY
shasum -a 256 -U app-1.4.0.tar.gz > UNIVERSAL
cat UNIVERSAL
report 'shasum -U manifest, sha256sum -c' sha256sum -c UNIVERSAL
report 'shasum -U manifest, shasum -a 256 -c' shasum -a 256 -c UNIVERSAL
scenario md5 manifest, sha256sum -c
stdout
stderr sha256sum: MD5SUMS: no properly formatted checksum lines found|
exit status 1
scenario empty manifest
stdout
stderr sha256sum: EMPTY: no properly formatted checksum lines found|
exit status 1
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 Uapp-1.4.0.tar.gz
scenario shasum -U manifest, sha256sum -c
stdout Uapp-1.4.0.tar.gz: FAILED open or read
stderr sha256sum: Uapp-1.4.0.tar.gz: No such file or directory|sha256sum: WARNING: 1 listed file could not be read|
exit status 1
scenario shasum -U manifest, shasum -a 256 -c
stdout app-1.4.0.tar.gz: OK
stderr
exit status 0
The first two fail the same way and the message is accurate: there is no line in the file that this command recognises. The third is the interesting one. It is not rejected. shasum writes U in the mode flag position for Universal Newlines, sha256sum has no such mode, and rather than complaining it reads the U as the first character of the file name and reports a file called Uapp-1.4.0.tar.gz that could not be read.
shasum -a 256 -c, which is the last line of the block, so the two tools disagree about a file that is sitting right there.
The mode flag field has four possible values under shasum and two under coreutils, and that asymmetry is the whole of the incompatibility.
| Mode flag | Written by | Understood by coreutils |
|---|---|---|
| a space | both, for text mode | yes |
* | both, for binary mode | yes |
U | shasum -U, Universal Newlines | no, it becomes part of the name |
^ | shasum -0, BITS mode | no, it becomes part of the name |
shasum extensions, documented in its own manual page. Nothing is broken on either side. The format simply has one field whose vocabulary is larger in one implementation than in the other.
Example 5: the folder that lists itself
The shortest way to write a manifest is sha256sum * > SHA256SUMS, and it is correct exactly once. Run it twice and the second run behaves differently from the first.
cd release
sha256sum * > SHA256SUMS
printf '%-34s%s\n' 'files in the folder' "$(ls | wc -l)"
printf '%-34s%s\n' 'lines after the first run' "$(wc -l < SHA256SUMS)"
sha256sum * > SHA256SUMS
printf '%-34s%s\n' 'lines after the second run' "$(wc -l < SHA256SUMS)"
printf '%-34s%s\n' 'lines naming the manifest' "$(grep -c SHA256SUMS SHA256SUMS)"
echo
sha256sum -c SHA256SUMS; printf '%-34s%s\n' 'exit status' "$?"
files in the folder 5
lines after the first run 4
lines after the second run 5
lines naming the manifest 1
README.txt: OK
SHA256SUMS: FAILED
app-1.4.0.exe: OK
app-1.4.0.tar.gz: OK
notes.txt: OK
sha256sum: WARNING: 1 computed checksum did NOT match
exit status 1
On the first run the shell expands * before the redirection creates the file, so the manifest covers four files and not itself. On the second run the file already exists, the glob picks it up, and the command ends up hashing a file that is being truncated and rewritten underneath it. Whatever value lands in that line, it will not be the digest of the finished manifest, so -c reports SHA256SUMS: FAILED from then on.
The fix is not to exclude the name with a second glob, which brings its own quoting problems on a folder whose file names you do not control. It is to build the list with find, which is the general answer and the subject of the next example.
Example 6: a manifest for a whole tree
There is no recursive mode. sha256sum -r is not an option and a directory argument is refused.
cd ~/hashlab
sha256sum -r release
sha256sum release
sha256sum: invalid option -- 'r'
Try 'sha256sum --help' for more information.
sha256sum: release: Is a directory
Hashing a tree therefore means producing a list and feeding it in. find with -print0 and xargs -0 is the form that survives spaces and newlines in names, and sorting keeps the manifest stable between runs on different filesystems.
cd release
find . -type f ! -name SHA256SUMS -print0 | sort -z | xargs -0 sha256sum > SHA256SUMS
cat SHA256SUMS
printf '%-34s%s\n' 'lines in the manifest' "$(wc -l < SHA256SUMS)"
sha256sum -c --quiet SHA256SUMS; printf '%-34s%s\n' 'exit status' "$?"
13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930 ./README.txt
c63677527d625de6ea2aca66e81f2bf0917f90654188a263a48245c113d6f7ef ./app-1.4.0.exe
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 ./app-1.4.0.tar.gz
85b0b6789bdd49d88b8d851fe4eb9e46fdb2c57b8ce7a571d727cd6cb5fb0a0e ./docs/guide.txt
a27110a155b1dd079db5ea8fee149a2b80019f48b359a7852f281a7720fe15a8 ./notes.txt
lines in the manifest 5
exit status 0
The subfolder appears as part of the name, the manifest excludes itself by name, and --quiet reduces a clean verification to an exit status. The sort is worth keeping: nothing guarantees the order find walks a tree, so without it two manifests of the same files can list them differently and a diff between them becomes unreadable.
./ because that is what was passed to find. Use find . consistently and the manifest verifies from the same folder it was written in, which is the subject of the next section.
The working directory decides what gets checked
A manifest holds names, not paths from the root, and they are resolved against the directory the check runs in. Moving the file somewhere tidier breaks it.
cd release
sha256sum * > ../SHA256SUMS
cd ..
printf '%-34s%s\n' 'manifest written in' 'release/'
printf '%-34s%s\n' 'manifest stored in' './'
sha256sum -c SHA256SUMS; printf '%-34s%s\n' 'exit status from hashlab/' "$?"
echo
cd release
sha256sum -c ../SHA256SUMS > /dev/null 2>&1; printf '%-34s%s\n' 'exit status from release/' "$?"
manifest written in release/
manifest stored in ./
sha256sum: README.txt: No such file or directory
README.txt: FAILED open or read
sha256sum: app-1.4.0.exe: No such file or directory
app-1.4.0.exe: FAILED open or read
sha256sum: app-1.4.0.tar.gz: No such file or directory
app-1.4.0.tar.gz: FAILED open or read
sha256sum: notes.txt: No such file or directory
notes.txt: FAILED open or read
sha256sum: WARNING: 4 listed files could not be read
exit status from hashlab/ 1
exit status from release/ 0
The same manifest and the same unmodified files, verified twice: a status of 1 and four failures from one directory, and a status of 0 from the other. Nothing is wrong with the files, and the message does not say that the manifest is in the wrong place, it says that the files are missing.
Four commands that agree, and one that does not
Four commands produce the SHA-256 of a file and a fifth looks as though it does. Only the first sixteen characters are printed, which is enough to tell agreement from disagreement.
cd release
printf '%-30s%s\n' 'sha256sum' "$(sha256sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'shasum -a 256' "$(shasum -a 256 app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'cksum -a sha256 --untagged' "$(cksum -a sha256 --untagged app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'openssl dgst -sha256' "$(openssl dgst -sha256 app-1.4.0.tar.gz | sed 's/.*= //' | cut -c1-16)"
printf '%-30s%s\n' 'shasum, no -a option' "$(shasum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'sha1sum' "$(sha1sum app-1.4.0.tar.gz | cut -c1-16)"
echo
for cmd in md5sum sha1sum sha256sum sha512sum; do
printf '%-30s%s\n' "hex characters, $cmd" "$($cmd app-1.4.0.tar.gz | cut -d' ' -f1 | tr -d '\n' | wc -c)"
done
printf '%-30s%s\n' 'hex characters, shasum' "$(shasum app-1.4.0.tar.gz | cut -d' ' -f1 | tr -d '\n' | wc -c)"
sha256sum 0bae767bc2f54d4d
shasum -a 256 0bae767bc2f54d4d
cksum -a sha256 --untagged 0bae767bc2f54d4d
openssl dgst -sha256 0bae767bc2f54d4d
shasum, no -a option 00eaf458cac61c68
sha1sum 00eaf458cac61c68
hex characters, md5sum 32
hex characters, sha1sum 40
hex characters, sha256sum 64
hex characters, sha512sum 128
hex characters, shasum 40
The first four agree. shasum with no -a does not, and the reason is in its own help text: the algorithm list reads 1 (default), 224, 256, 384, 512, 512224, 512256. Its default is SHA-1, so a bare shasum produces exactly what sha1sum produces, forty hexadecimal characters instead of sixty four.
| Spelling | Default algorithm | Comes from |
|---|---|---|
sha256sum | SHA-256, the name says so | GNU coreutils |
cksum -a sha256 | the algorithm is an argument | GNU coreutils |
shasum | SHA-1 | Perl, the Digest::SHA distribution |
openssl dgst -sha256 | the algorithm is an argument | OpenSSL |
Get-FileHash | SHA-256, documented | PowerShell |
shasum. Everything else either names the algorithm in the command or takes it as an argument you cannot forget. Of the five, it is the only one where leaving the option off silently changes the algorithm.
The output formats line up well enough that the manifests interoperate, as long as the mode flag stays inside the two values coreutils knows. A manifest written by shasum -a 256 verifies under sha256sum -c and the other way round.
Reading a Linux manifest on Windows
A release built on Linux is often verified on Windows, where there is no -c. The check has to be written by hand, which means parsing the format, which means the mode flag field again.
Get-Content ./SHA256SUMS | ForEach-Object {
if ($_ -match '^(?<hash>[0-9a-f]{64}) (?<mode>.)(?<file>.+)$') {
$actual = (Get-FileHash -LiteralPath $Matches.file -Algorithm SHA256).Hash
'{0,-22}{1}' -f $Matches.file, $(if ($actual -eq $Matches.hash) { 'OK' } else { 'FAILED' })
}
}
app-1.4.0.tar.gz OK
README.txt OK
app-1.4.0.exe FAILED
The pattern takes sixty four hexadecimal characters, one space and one character of any kind, which is the documented shape, and the regular expression is the same length whether the manifest came from a text mode or a binary mode run. The comparison is -eq, which is case insensitive in PowerShell, so the lower case digests in the file match the upper case ones Get-FileHash returns without any conversion.
What md5sum still proves
md5sum is not dead, it is scoped, and the manual scopes it in one paragraph rather than leaving it to opinion. The MD5 digest is described as more reliable than a simple CRC for detecting accidental file corruption, followed by the clause that decides where it may be used: however, it should not be considered secure against malicious tampering.
Accidental corruption is still the common case, and MD5 catches it. One character changed in a file of the same length, through both commands:
cd release
cp app-1.4.0.tar.gz transferred.tar.gz
printf '%-34s%s\n' 'bytes in the original' "$(wc -c < app-1.4.0.tar.gz)"
# change one character in the copy, keeping the length the same
printf 'zaur.it sample payload v1.4.1\n' > transferred.tar.gz
printf '%-34s%s\n' 'bytes in the copy' "$(wc -c < transferred.tar.gz)"
printf '%-34s%s\n' 'characters that differ' "$(cmp -l app-1.4.0.tar.gz transferred.tar.gz | wc -l)"
echo
printf '%-34s%s\n' 'md5sum, original' "$(md5sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'md5sum, copy' "$(md5sum transferred.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'sha256sum, original' "$(sha256sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'sha256sum, copy' "$(sha256sum transferred.tar.gz | cut -c1-16)"
bytes in the original 30
bytes in the copy 30
characters that differ 1
md5sum, original 654c6242b08254d5
md5sum, copy 6487a4bb1a5e51d6
sha256sum, original 0bae767bc2f54d4d
sha256sum, copy 904070f4afc65999
Both digests change completely for a one character difference, which is the property that makes either of them useful for a transfer check. The difference between them is not sensitivity, it is whether someone can construct a second file that produces the same value on purpose.
| Question being asked | md5sum is enough | Use SHA-256 |
|---|---|---|
| Did this copy survive the network | yes | also fine |
| Did this tape restore correctly | yes | also fine |
| Are these two local files identical | yes | also fine |
| Is this the file the vendor signed | no | yes |
| Did anyone replace this download | no | yes |
What the manual actually says about these commands
Two claims in this article come straight out of the manual rather than out of habit, and both can be re-derived in a few seconds. The source of the GNU coreutils 9.4 manual is one file.
curl -sO https://raw.githubusercontent.com/coreutils/coreutils/v9.4/doc/coreutils.texi
printf '%-44s%s\n' 'lines in the 9.4 manual source' "$(wc -l < coreutils.texi)"
printf '%-44s%s\n' 'times it says "two spaces"' "$(grep -c 'two spaces' coreutils.texi)"
printf '%-44s%s\n' 'times it says "a space, a flag"' "$(grep -c 'a space, a flag' coreutils.texi)"
echo
echo 'the one "two spaces" and the command it belongs to:'
grep -n 'two spaces' coreutils.texi | cut -c1-62
grep -n '@node fmt invocation' coreutils.texi
echo
echo 'the sections carrying the "not secure against tampering" note:'
grep -n '@weakHash{' coreutils.texi
lines in the 9.4 manual source 20122
times it says "two spaces" 1
times it says "a space, a flag" 1
the one "two spaces" and the command it belongs to:
2521:between sentences to two spaces.
2447:@node fmt invocation
the sections carrying the "not secure against tampering" note:
4301:@weakHash{MD5}
4357:@weakHash{SHA-1}
The phrase "two spaces" appears once in twenty thousand lines of manual, in the section about fmt and sentence spacing, and nowhere near a checksum. The sentence that does describe the format says a space and a flag, exactly once, in the section on output modes.
The second grep finds the warning paragraph. It is a macro in the source, invoked twice, and the two call sites are the whole answer to which digests the manual qualifies.
| Manual section | Carries the tampering warning |
|---|---|
md5sum invocation | yes, naming MD5 |
sha1sum invocation | yes, naming SHA-1 |
| sha2 utilities | no |
b2sum invocation | no |
Hidden gems
Two options change what a manifest is worth, and neither is in anybody's muscle memory.
cd release
sha256sum --tag app-1.4.0.tar.gz README.txt > TAGGED
cat TAGGED
{ cat TAGGED; echo 'a line that is not a checksum'; } > MIXED
report() {
label=$1; shift
"$@" > /dev/null 2>stderr.txt; rc=$?
printf '%-32s%-42s%s\n' "$label" "$(sed 's/^sha256sum: //' stderr.txt | tr '\n' '|' | sed 's/|$//')" "$rc"
}
echo
printf '%-32s%-42s%s\n' 'command' 'stderr, without the program name' 'exit'
report 'cksum -c TAGGED' cksum -c TAGGED
report 'sha256sum -c TAGGED' sha256sum -c TAGGED
report 'sha256sum -c MIXED' sha256sum -c MIXED
report 'sha256sum -c --strict MIXED' sha256sum -c --strict MIXED
SHA256 (app-1.4.0.tar.gz) = 0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753
SHA256 (README.txt) = 13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930
command stderr, without the program name exit
cksum -c TAGGED 0
sha256sum -c TAGGED 0
sha256sum -c MIXED WARNING: 1 line is improperly formatted 0
sha256sum -c --strict MIXED WARNING: 1 line is improperly formatted 1
--tag writes the BSD style line, which puts the algorithm name in the file and removes the guessing the format otherwise requires. Both cksum -c and sha256sum -c read it back. The second half of the block is the one that should change how a script is written: a line that is not a checksum at all is a warning on stderr and an exit status of zero. --strict turns it into a failure.
--strict with --status.
sha256sum -and a baresha256sumread standard input, soprintf 'abc' | sha256sumhashes a string without a temporary file. Useprintfrather thanecho, which adds a newline and changes the answer.--ignore-missingis the right option for verifying part of a large manifest, which is what a published file list usually is. Pair it with a check that something was actually verified, because an empty folder also produces no failures.-wnames the line number of a malformed entry on stderr, which is faster than reading a long manifest looking for the broken one.- The sample certificates page on this site publishes a real
SHA256SUMSfile beside its downloads, which makes it somewhere to practise-con a manifest nobody wrote for a tutorial.
Where this matters
- An installer or image that was downloaded over a flaky link. The vendor publishes a digest, and checking it is cheaper than debugging an install that fails halfway through.
- A release pipeline that ships artefacts. Writing a manifest at build time and verifying it after transfer turns a class of silent corruption into a failed job, provided the job reads the exit status and uses
--strict. - A restore from backup. A manifest taken before the backup and verified after the restore answers the only question that matters about the restore, and it answers it without comparing file sizes by eye.
- A migration between two filesystems or two hosts. Two manifests written the same way, with the same
findand the samesort, diff cleanly against each other. - Any script that reads somebody else's checksum file. This is where the mode flag and the escaping rules stop being trivia, because the file was written by a tool you do not control.
Tips and limitations
- The manifest format records the argument you gave, so write it from inside the folder the names should be relative to and keep the file there.
- The character at position 66 is a mode flag and not part of the separator. On Linux it never changes the digest, only the line.
- A file name containing a backslash or a newline changes the whole line, which starts with a backslash when it happens. Hand written parsers do not survive it and
-cdoes. - Nothing in an untagged manifest names the algorithm.
--tagfixes that, at the cost of a format older scripts may not read. - A malformed line is a warning and not a failure unless
--strictis given, and a missing file is not a failure if--ignore-missingis given. Both defaults are reasonable and both can hide a broken run. shasumdefaults to SHA-1 and writes two mode flags coreutils does not understand. If a manifest has to cross tools, name the algorithm and leave the exotic modes alone.md5sumremains a sound corruption check and is not an integrity check against somebody who wants to fool it, which is the distinction the manual draws for MD5 and SHA-1 and does not draw for the SHA-2 family.- None of this says anything about who produced the file. A digest published beside a download proves the bytes arrived intact, and a signature is a different subject.
Official documentation
- cksum output modes: the untagged line format | GNU coreutils
- cksum common options: check, quiet, status, strict | GNU coreutils
- md5sum invocation | GNU coreutils
- sha1sum invocation | GNU coreutils
- sha2 utilities: sha224sum, sha256sum, sha384sum, sha512sum | GNU coreutils
- b2sum invocation | GNU coreutils
- shasum: print or check SHA checksums | MetaCPAN
Related tools
- Hash generator: computes MD5, SHA1, SHA256 and SHA512 in the browser and compares the result against a value you paste, which settles a single file without a shell.
Related guides
- Computing and comparing file hashes on Windows: the same job in PowerShell, where the digest comes back in upper case and the comparison operator decides the answer.
- Linux commands cheat sheet: the one line version of this page, alongside the rest of the daily set.
- OpenSSL cheat sheet: carries the
openssl dgstform, which is the fallback on a host with neither coreutils nor Perl. - Sample certificates for testing: ships a real
SHA256SUMSfile with its downloads, so-chas something honest to read. - A hands-on vi practice walk-through: ends by checking the edited file against a published digest, which is this command in its smallest form.
- A hands-on nano practice walk-through: the same one line check at the end of a different editor exercise.
Five cheat sheets, one PDF
Subnet masks, PowerShell, Linux commands, HTTP status codes and the ESXi command line - one page each, free to keep. Leave an address and it arrives in a minute.