sha256sum -c: the checksum line that is not two spaces

A checksum file is the least interesting part of a release until the day a download is corrupt, and then it is the only part that matters. On Linux the tool that writes one and the tool that reads it back are the same binary: sha256sum prints digests when you give it files, and verifies them when you give it -c and a manifest. Most people only ever use the first half, copy the digest into a terminal and compare it by eye.

The second half is where the detail lives. The manifest has a format, that format has a field almost nobody knows is a field, and the commands that read it disagree about what belongs in it. md5sum, sha1sum and sha256sum share one interface and differ only in the digest, shasum is a separate program that produces the same shape of line, and a manifest written by one of them can fail under another in a way that looks like a missing file rather than a format problem.

Every block below was run on GNU coreutils 9.4 and shasum 6.04 (Perl Digest::SHA), against the four file folder the prerequisite section creates, so the digests and the exit codes reproduce. Where a block measures something else, the sentence above it says what.

Applies to: GNU coreutils on any Linux distribution, and shasum wherever Perl is installed


Quick answer

Write the manifest from inside the folder the names should be relative to, then verify it the same way and read the exit status rather than the OK lines.

# write a manifest for everything in the current folder
sha256sum * > SHA256SUMS

# verify it: one line per file, and an exit status that is 0 only if all of them passed
sha256sum -c SHA256SUMS
echo $?

# in a script, say nothing and test the status
sha256sum -c --status SHA256SUMS || echo 'verification failed'
Warning: A hash proves two copies carry the same bytes. It proves nothing about who produced them. If the manifest came down the same link as the file, anyone who could replace one could replace the other, so this is a corruption check and not a signature.

Before the first example

Four things have to be true for the blocks below to produce the output shown. The first two are almost certainly already true on your machine.

1. GNU coreutils provides the commands. They ship in the same package, so they report the same version.

sha256sum --version | head -1
md5sum --version | head -1
sha256sum (GNU coreutils) 9.4
md5sum (GNU coreutils) 9.4

2. shasum is present for the cross tool section. It ships with Perl rather than with coreutils, so a minimal container may not have it.

shasum --version
perl -v | sed -n '2p'
6.04
This is perl 5, version 38, subversion 2 (v5.38.2) built for x86_64-linux-gnu-thread-multi

3. A folder with four files of known content. printf is used rather than echo because the bytes have to be exact: a shell that adds a different line ending produces a different digest, and then none of the values below match.

mkdir -p ~/hashlab/release && cd ~/hashlab
printf 'zaur.it sample payload v1.4.0\n'   > release/app-1.4.0.tar.gz
printf 'zaur.it sample installer v1.4.0\n' > release/app-1.4.0.exe
printf 'release notes for 1.4.0\n'         > release/README.txt
printf 'scratch\n'                         > release/notes.txt

4. Every block runs from ~/hashlab. That matters more than it looks: the names in a manifest are resolved against the working directory at check time, which has a section of its own below.

Note: The two files with Windows extensions are plain text a few bytes long. Nothing here depends on their contents being a real archive, only on their being the same bytes every time.

What one line of output actually is

Hash a file twice, once in the default mode and once with -b, and look at the two lines with cat -A so that nothing is hidden. The $ at the end is the line ending that cat -A draws, not part of the output.

sha256sum release/app-1.4.0.tar.gz > SHA256SUMS
sha256sum -b release/app-1.4.0.tar.gz >> SHA256SUMS
cat -A SHA256SUMS
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753  release/app-1.4.0.tar.gz$
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 *release/app-1.4.0.tar.gz$

The digest is identical. What changed is the single character sitting between the digest and the file name. The GNU manual describes the untagged format in one sentence, and it is worth reading slowly:

From the manual: For each file, we print the checksum, a space, a flag indicating binary or text input mode, and the file name.

A space, then a flag, then the name. The flag is a field of its own, one character wide, and the manual says what goes in it: binary mode is indicated with * and text mode with a space. Text mode is the default on Linux, which is why the usual line looks as though it has two spaces in it. It does not. It has one space and one empty looking field.

Part of the lineWidthWhat it holds
Digest64 characters for SHA-256lower case hexadecimal
Separator1 characteralways a space
Mode flag1 character* for binary, a space for text
File namethe rest of the lineexactly the argument that was given
Note: On a GNU system the flag is a label and nothing more. The -b description says so outright: this option merely flags each input mode as binary, and the checksum is unaffected. The distinction exists for the systems the manual calls MS-DOS like, which do treat the two kinds of file differently.

The second space is not a space

That one character field is the reason hand written manifest parsers break, and they break on exactly the lines that were produced carefully. Here are four idioms a shell script reaches for, each run against a text mode line and a binary mode line of the same file.

cd release
sha256sum    app-1.4.0.tar.gz >  SHA256SUMS
sha256sum -b app-1.4.0.tar.gz >> SHA256SUMS
text=$(head -1 SHA256SUMS)
bin=$(tail -1 SHA256SUMS)
want=app-1.4.0.tar.gz
hits=0
row() {                         # row "idiom" "line type" "result"
  [ "$3" = "$want" ] && hits=$((hits + 1))
  printf '%-34s%-10s%s\n' "$1" "$2" "$3"
}
printf '%-34s%-10s%s\n' 'idiom' 'line' 'the file name it returns'
row "awk '{print \$2}'"           text   "$(printf '%s\n' "$text" | awk '{print $2}')"
row ''                            binary "$(printf '%s\n' "$bin"  | awk '{print $2}')"
row "cut -d' ' -f3"               text   "$(printf '%s\n' "$text" | cut -d' ' -f3)"
row ''                            binary "$(printf '%s\n' "$bin"  | cut -d' ' -f3)"
row "cut -c67-"                   text   "$(printf '%s\n' "$text" | cut -c67-)"
row ''                            binary "$(printf '%s\n' "$bin"  | cut -c67-)"
row "sed 's/^[0-9a-f]\{64\} .//'" text   "$(printf '%s\n' "$text" | sed 's/^[0-9a-f]\{64\} .//')"
row ''                            binary "$(printf '%s\n' "$bin"  | sed 's/^[0-9a-f]\{64\} .//')"
echo
printf '%-34s%s\n' 'parses tested' 8
printf '%-34s%s\n' 'returning the file name' "$hits"
idiom                             line      the file name it returns
awk '{print $2}'                  text      app-1.4.0.tar.gz
                                  binary    *app-1.4.0.tar.gz
cut -d' ' -f3                     text      app-1.4.0.tar.gz
                                  binary    
cut -c67-                         text      app-1.4.0.tar.gz
                                  binary    app-1.4.0.tar.gz
sed 's/^[0-9a-f]\{64\} .//'       text      app-1.4.0.tar.gz
                                  binary    app-1.4.0.tar.gz

parses tested                     8
returning the file name           6

Six of the eight return the file name. The two that do not are both reading a binary mode line: awk hands back the name with the flag glued to the front, and cut -d' ' -f3 hands back nothing at all, because in a binary line there is no third space separated field. Neither reports an error.

Common mistake: Splitting on whitespace is the natural thing to write and it is wrong in a way that only shows up on manifests published by careful people, since the flag is a space until somebody passes -b. The two idioms that survive both lines are the ones that treat position 66 as a field rather than as part of a separator.

If a script has to read one of these files, the honest version of the pattern is the sed form above: sixty four hexadecimal characters, one space, any one character, then the name. The better version is not to parse it at all, which the next sections are about.


A file name can rewrite the whole line

There is a second rule in the same format, and it changes the line before the digest. When a file name contains a backslash, a newline or a carriage return, the manual says the line is started with a backslash and each problematic character in the name is escaped.

cd release
touch 'back\slash.txt'
sha256sum 'back\slash.txt'
sha256sum 'back\slash.txt' > ESCAPED
sha256sum -c ESCAPED
\e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  back\\slash.txt
\back\\slash.txt: OK

The line now begins with a backslash, so the digest starts at character two, and the single backslash in the name is written as two. The -c run reads it back correctly and prints the escaped form in its report. The parsers that survived the binary flag do not survive this.

cd release
want='back\slash.txt'
a=$(cut -c67- ESCAPED)
b=$(cut -c68- ESCAPED)
c=$(sed 's/^[0-9a-f]\{64\} .//' ESCAPED)
hits=0
for got in "$a" "$b" "$c"; do
  [ "$got" = "$want" ] && hits=$((hits + 1))
done
printf '%-36s%s\n' 'the first character of the line' "$(cut -c1 ESCAPED)"
printf '%-36s%s\n' 'the name on disk' "$want"
echo
printf '%-36s%s\n' 'cut -c67- returns' "$a"
printf '%-36s%s\n' 'cut -c68- returns' "$b"
printf '%-36s%s\n' 'the sed above returns' "$(printf '%s' "$c" | cut -c1-30)"
echo
printf '%-36s%s\n' 'parses tested' 3
printf '%-36s%s\n' 'returning the name on disk' "$hits"
the first character of the line     \
the name on disk                    back\slash.txt

cut -c67- returns                    back\\slash.txt
cut -c68- returns                   back\\slash.txt
the sed above returns               \e3b0c44298fc1c149afbf4c8996fb

parses tested                       3
returning the name on disk          0

None of the three returns the name that is on disk, and the sed form fails the hardest: the leading backslash stops the digest matching at all, so the pattern does not fire and the whole line comes back untouched.

Warning: This is not an exotic case. A file arriving from a Windows share, an extracted archive that kept a path separator in a name, or any name a user typed can trigger it, and a manifest with one escaped line among many ordinary ones looks completely normal in an editor.

Example 1: write a manifest for a release folder

The names recorded in the file are the arguments as given, so the folder you stand in when you write it decides what the manifest says. Writing it from inside the folder keeps the names short and keeps them portable.

cd release
sha256sum app-1.4.0.tar.gz app-1.4.0.exe README.txt > SHA256SUMS
cat SHA256SUMS
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753  app-1.4.0.tar.gz
c63677527d625de6ea2aca66e81f2bf0917f90654188a263a48245c113d6f7ef  app-1.4.0.exe
13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930  README.txt

Three files named, three lines, no header and no algorithm name anywhere in the file. That last point is worth holding on to: nothing in a manifest says which digest produced it. The reader has to know, or has to guess from the length of the first field, which is the only clue there is. The four lengths are measured further down, in the section on the commands that agree.

CommandHex characters per lineWhat the file is usually called
md5sum32MD5SUMS, or .md5 beside each file
sha1sum40SHA1SUMS
sha256sum64SHA256SUMS
sha512sum128SHA512SUMS
Note: Those names are convention and nothing enforces them. The length of the first field is the only thing in the file that identifies the algorithm, which is why feeding the wrong manifest to the wrong command produces the error it does, two sections below.

Example 2: check it, and read the exit status

The half of the tool that gets skipped is -c. It reads the manifest, hashes each named file, prints a verdict per line and sets an exit status for the run as a whole. Five scenarios run in sequence, each one damaging the folder a little more, with stdout, stderr and the status reported separately.

cd release
report() {                      # report "label" command...
  label=$1; shift
  out=$("$@" 2>stderr.txt); rc=$?
  printf '%-34s%s\n' 'scenario' "$label"
  printf '%-34s%s\n' 'stdout' "$(printf '%s' "$out" | tr '\n' '|')"
  printf '%-34s%s\n' 'stderr' "$(tr '\n' '|' < stderr.txt)"
  printf '%-34s%s\n\n' 'exit status' "$rc"
}

sha256sum app-1.4.0.tar.gz app-1.4.0.exe README.txt > SHA256SUMS
report 'every file matches' sha256sum -c SHA256SUMS

printf 'one more byte\n' >> app-1.4.0.exe
report 'one file altered' sha256sum -c SHA256SUMS

mv app-1.4.0.exe elsewhere.exe
report 'one file not there' sha256sum -c SHA256SUMS
report 'one file not there, ignore-missing' sha256sum -c --ignore-missing SHA256SUMS

mv app-1.4.0.tar.gz elsewhere.tar.gz
mv README.txt elsewhere.txt
report 'no listed file is there' sha256sum -c --ignore-missing SHA256SUMS
scenario                          every file matches
stdout                            app-1.4.0.tar.gz: OK|app-1.4.0.exe: OK|README.txt: OK
stderr                            
exit status                       0

scenario                          one file altered
stdout                            app-1.4.0.tar.gz: OK|app-1.4.0.exe: FAILED|README.txt: OK
stderr                            sha256sum: WARNING: 1 computed checksum did NOT match|
exit status                       1

scenario                          one file not there
stdout                            app-1.4.0.tar.gz: OK|app-1.4.0.exe: FAILED open or read|README.txt: OK
stderr                            sha256sum: app-1.4.0.exe: No such file or directory|sha256sum: WARNING: 1 listed file could not be read|
exit status                       1

scenario                          one file not there, ignore-missing
stdout                            app-1.4.0.tar.gz: OK|README.txt: OK
stderr                            
exit status                       0

scenario                          no listed file is there
stdout                            
stderr                            sha256sum: SHA256SUMS: no file was verified|
exit status                       1

The first three behave the way the output reads. The last two are the ones that cost an afternoon.

ScenarioPrinted per fileExit status
Everything matchesOK0
A file was alteredFAILED1
A file is not thereFAILED open or read1
A file is not there, with --ignore-missingnothing for that file0
None of the listed files is there, with --ignore-missingnothing at all1
Common mistake: The last row is the trap. --ignore-missing exists so you can verify a subset of a large manifest, and it does that job well. If the download step silently produced nothing, the same option turns a folder with no files in it into a run that prints nothing on stdout. The only thing separating it from a clean pass is the exit status and one line on stderr reading that no file was verified.

The manual states the rule plainly, and all three clauses matter: if any listed file cannot be opened or read, if any valid line has a checksum inconsistent with the associated file, or if no valid line is found, the command exits with nonzero status.


Example 3: quiet and status suppress different things

Two options reduce the noise and they are not interchangeable. Both are run here against a manifest of two files where one of them has been altered.

cd release
sha256sum app-1.4.0.exe README.txt > SHA256SUMS
printf 'one more byte\n' >> app-1.4.0.exe

oneline() { tr '\n' '|' | sed 's/|$//'; }
report() {
  label=$1; shift
  out=$("$@" 2>stderr.txt); rc=$?
  printf '%-12s%-40s%-56s%s\n' "$label" \
    "$(printf '%s' "$out" | oneline)" "$(oneline < stderr.txt)" "$rc"
}
printf '%-12s%-40s%-56s%s\n' 'option' 'stdout' 'stderr' 'exit'
report 'none'      sha256sum -c SHA256SUMS
report '--quiet'   sha256sum -c --quiet SHA256SUMS
report '--status'  sha256sum -c --status SHA256SUMS
option      stdout                                  stderr                                                  exit
none        app-1.4.0.exe: FAILED|README.txt: OK    sha256sum: WARNING: 1 computed checksum did NOT match   1
--quiet     app-1.4.0.exe: FAILED                   sha256sum: WARNING: 1 computed checksum did NOT match   1
--status                                                                                                    1

--quiet drops the OK lines and keeps the failures, so a passing run prints nothing on stdout and a failing run prints only what went wrong. --status prints nothing on either stream and leaves the exit status as the entire result, which is what a script wants.

Note: The warning line on stderr survives --quiet and disappears under --status. If a cron job is mailing you its stderr, that is the difference between hearing about a bad file and not.

Example 4: the manifest the other tool cannot read

Nothing in the file says which algorithm wrote it, so the obvious question is what happens when the wrong command reads it. Three manifests, all handed to sha256sum -c: one written by md5sum, one empty, and one written by shasum in its Universal Newlines mode.

cd release
report() {
  label=$1; shift
  out=$("$@" 2>stderr.txt); rc=$?
  printf '%-34s%s\n' 'scenario' "$label"
  printf '%-34s%s\n' 'stdout' "$(printf '%s' "$out" | tr '\n' '|')"
  printf '%-34s%s\n' 'stderr' "$(tr '\n' '|' < stderr.txt)"
  printf '%-34s%s\n\n' 'exit status' "$rc"
}

md5sum app-1.4.0.tar.gz > MD5SUMS
report 'md5 manifest, sha256sum -c' sha256sum -c MD5SUMS

: > EMPTY
report 'empty manifest' sha256sum -c EMPTY

shasum -a 256 -U app-1.4.0.tar.gz > UNIVERSAL
cat UNIVERSAL
report 'shasum -U manifest, sha256sum -c' sha256sum -c UNIVERSAL
report 'shasum -U manifest, shasum -a 256 -c' shasum -a 256 -c UNIVERSAL
scenario                          md5 manifest, sha256sum -c
stdout                            
stderr                            sha256sum: MD5SUMS: no properly formatted checksum lines found|
exit status                       1

scenario                          empty manifest
stdout                            
stderr                            sha256sum: EMPTY: no properly formatted checksum lines found|
exit status                       1

0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753 Uapp-1.4.0.tar.gz
scenario                          shasum -U manifest, sha256sum -c
stdout                            Uapp-1.4.0.tar.gz: FAILED open or read
stderr                            sha256sum: Uapp-1.4.0.tar.gz: No such file or directory|sha256sum: WARNING: 1 listed file could not be read|
exit status                       1

scenario                          shasum -U manifest, shasum -a 256 -c
stdout                            app-1.4.0.tar.gz: OK
stderr                            
exit status                       0

The first two fail the same way and the message is accurate: there is no line in the file that this command recognises. The third is the interesting one. It is not rejected. shasum writes U in the mode flag position for Universal Newlines, sha256sum has no such mode, and rather than complaining it reads the U as the first character of the file name and reports a file called Uapp-1.4.0.tar.gz that could not be read.

Common mistake: A format problem arrives disguised as a missing file. The same manifest verifies cleanly under shasum -a 256 -c, which is the last line of the block, so the two tools disagree about a file that is sitting right there.

The mode flag field has four possible values under shasum and two under coreutils, and that asymmetry is the whole of the incompatibility.

Mode flagWritten byUnderstood by coreutils
a spaceboth, for text modeyes
*both, for binary modeyes
Ushasum -U, Universal Newlinesno, it becomes part of the name
^shasum -0, BITS modeno, it becomes part of the name
Note: The two rows coreutils does not understand are both shasum extensions, documented in its own manual page. Nothing is broken on either side. The format simply has one field whose vocabulary is larger in one implementation than in the other.

Example 5: the folder that lists itself

The shortest way to write a manifest is sha256sum * > SHA256SUMS, and it is correct exactly once. Run it twice and the second run behaves differently from the first.

cd release
sha256sum * > SHA256SUMS
printf '%-34s%s\n' 'files in the folder' "$(ls | wc -l)"
printf '%-34s%s\n' 'lines after the first run' "$(wc -l < SHA256SUMS)"

sha256sum * > SHA256SUMS
printf '%-34s%s\n' 'lines after the second run' "$(wc -l < SHA256SUMS)"
printf '%-34s%s\n' 'lines naming the manifest' "$(grep -c SHA256SUMS SHA256SUMS)"
echo
sha256sum -c SHA256SUMS; printf '%-34s%s\n' 'exit status' "$?"
files in the folder               5
lines after the first run         4
lines after the second run        5
lines naming the manifest         1

README.txt: OK
SHA256SUMS: FAILED
app-1.4.0.exe: OK
app-1.4.0.tar.gz: OK
notes.txt: OK
sha256sum: WARNING: 1 computed checksum did NOT match
exit status                       1

On the first run the shell expands * before the redirection creates the file, so the manifest covers four files and not itself. On the second run the file already exists, the glob picks it up, and the command ends up hashing a file that is being truncated and rewritten underneath it. Whatever value lands in that line, it will not be the digest of the finished manifest, so -c reports SHA256SUMS: FAILED from then on.

Warning: The value in that line is an artefact of how much had been written when the file was read, so it is not worth quoting and will differ on another machine. The reproducible part is the verdict: a manifest built this way fails against itself for ever.

The fix is not to exclude the name with a second glob, which brings its own quoting problems on a folder whose file names you do not control. It is to build the list with find, which is the general answer and the subject of the next example.


Example 6: a manifest for a whole tree

There is no recursive mode. sha256sum -r is not an option and a directory argument is refused.

cd ~/hashlab
sha256sum -r release
sha256sum release
sha256sum: invalid option -- 'r'
Try 'sha256sum --help' for more information.
sha256sum: release: Is a directory

Hashing a tree therefore means producing a list and feeding it in. find with -print0 and xargs -0 is the form that survives spaces and newlines in names, and sorting keeps the manifest stable between runs on different filesystems.

cd release
find . -type f ! -name SHA256SUMS -print0 | sort -z | xargs -0 sha256sum > SHA256SUMS
cat SHA256SUMS
printf '%-34s%s\n' 'lines in the manifest' "$(wc -l < SHA256SUMS)"
sha256sum -c --quiet SHA256SUMS; printf '%-34s%s\n' 'exit status' "$?"
13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930  ./README.txt
c63677527d625de6ea2aca66e81f2bf0917f90654188a263a48245c113d6f7ef  ./app-1.4.0.exe
0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753  ./app-1.4.0.tar.gz
85b0b6789bdd49d88b8d851fe4eb9e46fdb2c57b8ce7a571d727cd6cb5fb0a0e  ./docs/guide.txt
a27110a155b1dd079db5ea8fee149a2b80019f48b359a7852f281a7720fe15a8  ./notes.txt
lines in the manifest             5
exit status                       0

The subfolder appears as part of the name, the manifest excludes itself by name, and --quiet reduces a clean verification to an exit status. The sort is worth keeping: nothing guarantees the order find walks a tree, so without it two manifests of the same files can list them differently and a diff between them becomes unreadable.

Note: The names begin with ./ because that is what was passed to find. Use find . consistently and the manifest verifies from the same folder it was written in, which is the subject of the next section.

The working directory decides what gets checked

A manifest holds names, not paths from the root, and they are resolved against the directory the check runs in. Moving the file somewhere tidier breaks it.

cd release
sha256sum * > ../SHA256SUMS
cd ..
printf '%-34s%s\n' 'manifest written in' 'release/'
printf '%-34s%s\n' 'manifest stored in'  './'
sha256sum -c SHA256SUMS; printf '%-34s%s\n' 'exit status from hashlab/' "$?"
echo
cd release
sha256sum -c ../SHA256SUMS > /dev/null 2>&1; printf '%-34s%s\n' 'exit status from release/' "$?"
manifest written in               release/
manifest stored in                ./
sha256sum: README.txt: No such file or directory
README.txt: FAILED open or read
sha256sum: app-1.4.0.exe: No such file or directory
app-1.4.0.exe: FAILED open or read
sha256sum: app-1.4.0.tar.gz: No such file or directory
app-1.4.0.tar.gz: FAILED open or read
sha256sum: notes.txt: No such file or directory
notes.txt: FAILED open or read
sha256sum: WARNING: 4 listed files could not be read
exit status from hashlab/         1

exit status from release/         0

The same manifest and the same unmodified files, verified twice: a status of 1 and four failures from one directory, and a status of 0 from the other. Nothing is wrong with the files, and the message does not say that the manifest is in the wrong place, it says that the files are missing.

Warning: This is a common reason a published checksum file looks broken when nothing is wrong with it. The manifest belongs beside the files it names, and it is checked from there.

Four commands that agree, and one that does not

Four commands produce the SHA-256 of a file and a fifth looks as though it does. Only the first sixteen characters are printed, which is enough to tell agreement from disagreement.

cd release
printf '%-30s%s\n' 'sha256sum'                   "$(sha256sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'shasum -a 256'               "$(shasum -a 256 app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'cksum -a sha256 --untagged'  "$(cksum -a sha256 --untagged app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'openssl dgst -sha256'        "$(openssl dgst -sha256 app-1.4.0.tar.gz | sed 's/.*= //' | cut -c1-16)"
printf '%-30s%s\n' 'shasum, no -a option'        "$(shasum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-30s%s\n' 'sha1sum'                     "$(sha1sum app-1.4.0.tar.gz | cut -c1-16)"
echo
for cmd in md5sum sha1sum sha256sum sha512sum; do
  printf '%-30s%s\n' "hex characters, $cmd" "$($cmd app-1.4.0.tar.gz | cut -d' ' -f1 | tr -d '\n' | wc -c)"
done
printf '%-30s%s\n' 'hex characters, shasum'      "$(shasum app-1.4.0.tar.gz | cut -d' ' -f1 | tr -d '\n' | wc -c)"
sha256sum                     0bae767bc2f54d4d
shasum -a 256                 0bae767bc2f54d4d
cksum -a sha256 --untagged    0bae767bc2f54d4d
openssl dgst -sha256          0bae767bc2f54d4d
shasum, no -a option          00eaf458cac61c68
sha1sum                       00eaf458cac61c68

hex characters, md5sum        32
hex characters, sha1sum       40
hex characters, sha256sum     64
hex characters, sha512sum     128
hex characters, shasum        40

The first four agree. shasum with no -a does not, and the reason is in its own help text: the algorithm list reads 1 (default), 224, 256, 384, 512, 512224, 512256. Its default is SHA-1, so a bare shasum produces exactly what sha1sum produces, forty hexadecimal characters instead of sixty four.

SpellingDefault algorithmComes from
sha256sumSHA-256, the name says soGNU coreutils
cksum -a sha256the algorithm is an argumentGNU coreutils
shasumSHA-1Perl, the Digest::SHA distribution
openssl dgst -sha256the algorithm is an argumentOpenSSL
Get-FileHashSHA-256, documentedPowerShell
Warning: The odd row is shasum. Everything else either names the algorithm in the command or takes it as an argument you cannot forget. Of the five, it is the only one where leaving the option off silently changes the algorithm.

The output formats line up well enough that the manifests interoperate, as long as the mode flag stays inside the two values coreutils knows. A manifest written by shasum -a 256 verifies under sha256sum -c and the other way round.


Reading a Linux manifest on Windows

A release built on Linux is often verified on Windows, where there is no -c. The check has to be written by hand, which means parsing the format, which means the mode flag field again.

Get-Content ./SHA256SUMS | ForEach-Object {
    if ($_ -match '^(?<hash>[0-9a-f]{64}) (?<mode>.)(?<file>.+)$') {
        $actual = (Get-FileHash -LiteralPath $Matches.file -Algorithm SHA256).Hash
        '{0,-22}{1}' -f $Matches.file, $(if ($actual -eq $Matches.hash) { 'OK' } else { 'FAILED' })
    }
}
app-1.4.0.tar.gz      OK
README.txt            OK
app-1.4.0.exe         FAILED

The pattern takes sixty four hexadecimal characters, one space and one character of any kind, which is the documented shape, and the regular expression is the same length whether the manifest came from a text mode or a binary mode run. The comparison is -eq, which is case insensitive in PowerShell, so the lower case digests in the file match the upper case ones Get-FileHash returns without any conversion.

Note: Lines for escaped file names start with a backslash and will not match that pattern, which is a feature rather than an oversight. A manifest containing them should be verified with the tool that wrote it.

What md5sum still proves

md5sum is not dead, it is scoped, and the manual scopes it in one paragraph rather than leaving it to opinion. The MD5 digest is described as more reliable than a simple CRC for detecting accidental file corruption, followed by the clause that decides where it may be used: however, it should not be considered secure against malicious tampering.

Accidental corruption is still the common case, and MD5 catches it. One character changed in a file of the same length, through both commands:

cd release
cp app-1.4.0.tar.gz transferred.tar.gz
printf '%-34s%s\n' 'bytes in the original'    "$(wc -c < app-1.4.0.tar.gz)"

# change one character in the copy, keeping the length the same
printf 'zaur.it sample payload v1.4.1\n' > transferred.tar.gz
printf '%-34s%s\n' 'bytes in the copy'        "$(wc -c < transferred.tar.gz)"
printf '%-34s%s\n' 'characters that differ'   "$(cmp -l app-1.4.0.tar.gz transferred.tar.gz | wc -l)"
echo
printf '%-34s%s\n' 'md5sum, original'  "$(md5sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'md5sum, copy'      "$(md5sum transferred.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'sha256sum, original' "$(sha256sum app-1.4.0.tar.gz | cut -c1-16)"
printf '%-34s%s\n' 'sha256sum, copy'     "$(sha256sum transferred.tar.gz | cut -c1-16)"
bytes in the original             30
bytes in the copy                 30
characters that differ            1

md5sum, original                  654c6242b08254d5
md5sum, copy                      6487a4bb1a5e51d6
sha256sum, original               0bae767bc2f54d4d
sha256sum, copy                   904070f4afc65999

Both digests change completely for a one character difference, which is the property that makes either of them useful for a transfer check. The difference between them is not sensitivity, it is whether someone can construct a second file that produces the same value on purpose.

Question being askedmd5sum is enoughUse SHA-256
Did this copy survive the networkyesalso fine
Did this tape restore correctlyyesalso fine
Are these two local files identicalyesalso fine
Is this the file the vendor signednoyes
Did anyone replace this downloadnoyes
Note: Plenty of vendors publish both values. Where both are there, the SHA-256 one is the one to check, and the MD5 one still answers the question of whether the copy arrived intact.

What the manual actually says about these commands

Two claims in this article come straight out of the manual rather than out of habit, and both can be re-derived in a few seconds. The source of the GNU coreutils 9.4 manual is one file.

curl -sO https://raw.githubusercontent.com/coreutils/coreutils/v9.4/doc/coreutils.texi
printf '%-44s%s\n' 'lines in the 9.4 manual source' "$(wc -l < coreutils.texi)"
printf '%-44s%s\n' 'times it says "two spaces"' "$(grep -c 'two spaces' coreutils.texi)"
printf '%-44s%s\n' 'times it says "a space, a flag"' "$(grep -c 'a space, a flag' coreutils.texi)"
echo
echo 'the one "two spaces" and the command it belongs to:'
grep -n 'two spaces' coreutils.texi | cut -c1-62
grep -n '@node fmt invocation' coreutils.texi
echo
echo 'the sections carrying the "not secure against tampering" note:'
grep -n '@weakHash{' coreutils.texi
lines in the 9.4 manual source              20122
times it says "two spaces"                  1
times it says "a space, a flag"             1

the one "two spaces" and the command it belongs to:
2521:between sentences to two spaces.
2447:@node fmt invocation

the sections carrying the "not secure against tampering" note:
4301:@weakHash{MD5}
4357:@weakHash{SHA-1}

The phrase "two spaces" appears once in twenty thousand lines of manual, in the section about fmt and sentence spacing, and nowhere near a checksum. The sentence that does describe the format says a space and a flag, exactly once, in the section on output modes.

The second grep finds the warning paragraph. It is a macro in the source, invoked twice, and the two call sites are the whole answer to which digests the manual qualifies.

Manual sectionCarries the tampering warning
md5sum invocationyes, naming MD5
sha1sum invocationyes, naming SHA-1
sha2 utilitiesno
b2sum invocationno
Result: The set of digests GNU qualifies is exactly MD5 and SHA-1. The pages for the SHA-2 family and for BLAKE2 carry no such sentence, which is a clearer answer than any blog post about whether MD5 is still acceptable.

Hidden gems

Two options change what a manifest is worth, and neither is in anybody's muscle memory.

cd release
sha256sum --tag app-1.4.0.tar.gz README.txt > TAGGED
cat TAGGED
{ cat TAGGED; echo 'a line that is not a checksum'; } > MIXED

report() {
  label=$1; shift
  "$@" > /dev/null 2>stderr.txt; rc=$?
  printf '%-32s%-42s%s\n' "$label" "$(sed 's/^sha256sum: //' stderr.txt | tr '\n' '|' | sed 's/|$//')" "$rc"
}
echo
printf '%-32s%-42s%s\n' 'command' 'stderr, without the program name' 'exit'
report 'cksum -c TAGGED'            cksum -c TAGGED
report 'sha256sum -c TAGGED'        sha256sum -c TAGGED
report 'sha256sum -c MIXED'         sha256sum -c MIXED
report 'sha256sum -c --strict MIXED' sha256sum -c --strict MIXED
SHA256 (app-1.4.0.tar.gz) = 0bae767bc2f54d4dd1b2b80a59dc6135c893483f25728d02df40d708108be753
SHA256 (README.txt) = 13cc4bcdcd87ecb2851a65bf7ee01e9545aabf95624afccabad9bb62d3bab930

command                         stderr, without the program name          exit
cksum -c TAGGED                                                           0
sha256sum -c TAGGED                                                       0
sha256sum -c MIXED              WARNING: 1 line is improperly formatted   0
sha256sum -c --strict MIXED     WARNING: 1 line is improperly formatted   1

--tag writes the BSD style line, which puts the algorithm name in the file and removes the guessing the format otherwise requires. Both cksum -c and sha256sum -c read it back. The second half of the block is the one that should change how a script is written: a line that is not a checksum at all is a warning on stderr and an exit status of zero. --strict turns it into a failure.

Common mistake: A manifest truncated by a failed download, or one with a stray line from an editor, passes by default. Every verified line still verified, so the run is a success as far as the exit status is concerned. In a pipeline, use --strict with --status.
  • sha256sum - and a bare sha256sum read standard input, so printf 'abc' | sha256sum hashes a string without a temporary file. Use printf rather than echo, which adds a newline and changes the answer.
  • --ignore-missing is the right option for verifying part of a large manifest, which is what a published file list usually is. Pair it with a check that something was actually verified, because an empty folder also produces no failures.
  • -w names the line number of a malformed entry on stderr, which is faster than reading a long manifest looking for the broken one.
  • The sample certificates page on this site publishes a real SHA256SUMS file beside its downloads, which makes it somewhere to practise -c on a manifest nobody wrote for a tutorial.

Where this matters

  • An installer or image that was downloaded over a flaky link. The vendor publishes a digest, and checking it is cheaper than debugging an install that fails halfway through.
  • A release pipeline that ships artefacts. Writing a manifest at build time and verifying it after transfer turns a class of silent corruption into a failed job, provided the job reads the exit status and uses --strict.
  • A restore from backup. A manifest taken before the backup and verified after the restore answers the only question that matters about the restore, and it answers it without comparing file sizes by eye.
  • A migration between two filesystems or two hosts. Two manifests written the same way, with the same find and the same sort, diff cleanly against each other.
  • Any script that reads somebody else's checksum file. This is where the mode flag and the escaping rules stop being trivia, because the file was written by a tool you do not control.

Tips and limitations

  • The manifest format records the argument you gave, so write it from inside the folder the names should be relative to and keep the file there.
  • The character at position 66 is a mode flag and not part of the separator. On Linux it never changes the digest, only the line.
  • A file name containing a backslash or a newline changes the whole line, which starts with a backslash when it happens. Hand written parsers do not survive it and -c does.
  • Nothing in an untagged manifest names the algorithm. --tag fixes that, at the cost of a format older scripts may not read.
  • A malformed line is a warning and not a failure unless --strict is given, and a missing file is not a failure if --ignore-missing is given. Both defaults are reasonable and both can hide a broken run.
  • shasum defaults to SHA-1 and writes two mode flags coreutils does not understand. If a manifest has to cross tools, name the algorithm and leave the exotic modes alone.
  • md5sum remains a sound corruption check and is not an integrity check against somebody who wants to fool it, which is the distinction the manual draws for MD5 and SHA-1 and does not draw for the SHA-2 family.
  • None of this says anything about who produced the file. A digest published beside a download proves the bytes arrived intact, and a signature is a different subject.

Official documentation


  • Hash generator: computes MD5, SHA1, SHA256 and SHA512 in the browser and compares the result against a value you paste, which settles a single file without a shell.

Five cheat sheets, one PDF

Subnet masks, PowerShell, Linux commands, HTTP status codes and the ESXi command line - one page each, free to keep. Leave an address and it arrives in a minute.