Generate AVIF, WebP, JPEG XL data (bash script)

The recommended way of generating image codec performance data to be displayed with Codec-Compare is using https://github.com/webmproject/codec-compare. This is an alternative solution for legacy and demo purposes.

The bash script below can be used to generate encoded images, decoded images and JSON batches to be displayed by Codec-Compare for the codec implementations libavif, libwebp and libjxl. Unix only.

Instructions:

Go to an empty folder. Copy codec_compare_gen.sh inside it (code below). Make codec_compare_gen.sh an executable.
```
cd $(mktemp -d)
cp /path/to/codec_compare_gen.sh ./codec_compare_gen.sh
chmod +x codec_compare_gen.sh
```
Create a folder called "images" next to codec_compare_gen.sh. Put all input images in the "images" folder (just one here for demo and quick output).
```
mkdir images
wget -O images/img.png https://raw.githubusercontent.com/test-images/png/main/202105/pg-couplevn.png
```
Build the codecs (cmake, meson, ninja and clang must be installed). Encode and decode the images. Generate JSON batches.
```
./codec_compare_gen.sh
```

Download the Codec-Compare framework source.

git clone https://github.com/webmproject/codec-compare.git

Copy the batches to the assets folder.

cp -R images codec-compare/assets/images
mv encoded codec-compare/assets/encoded
mv decoded codec-compare/assets/decoded
mv json codec-compare/assets/json
mv batches.json codec-compare/assets/demo_batches.json

Build and launch Codec-Compare (npm must be installed).
```
cd codec-compare
npm i
npm run dev
```

codec_compare_gen.sh

#!/bin/bash

set -e

# Path to the directory containing the input PNG images.
IMAGES="./images"
# Path to the directory containing the encoded images.
ENCODED="./encoded"
# Path to the directory containing the decoded images.
DECODED="./decoded"
# Path to the directory containing the JSON entries.
JSON="./json"

echo "Download and build the codecs"

LIBAVIF_COMMIT="995f89fc2074e7fa6a8633cc5305d9017362b847"
LIBWEBP_COMMIT="d7a0506dcc9b7cf75742e16fd2707835d7ce003d"
LIBWEBP2_COMMIT="e0c6533107649063c236b5666f2cc4c10b9b7591"
LIBJXL_COMMIT="3e3a706755dee2c7af7bed34980484489ab6b123"

mkdir third_party
pushd third_party
  git clone https://github.com/AOMediaCodec/libavif.git
  pushd libavif
    git checkout "${LIBAVIF_COMMIT}"
    pushd ext
      ./aom.cmd
      ./dav1d.cmd
      ./libyuv.cmd
    popd
    cmake -S . -B build \
      -DAVIF_BUILD_APPS=ON -DAVIF_BUILD_EXAMPLES=OFF -DAVIF_BUILD_TESTS=OFF \
      -DAVIF_CODEC_AOM=LOCAL -DAVIF_CODEC_DAV1D=LOCAL -DAVIF_LIBYUV=LOCAL \
      -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
    cmake --build build --parallel
  popd

  git clone https://chromium.googlesource.com/webm/libwebp
  pushd libwebp
    git checkout "${LIBWEBP_COMMIT}"
    cmake -S . -B build \
      -DWEBP_BUILD_CWEBP=ON -DWEBP_BUILD_DWEBP=ON -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
    cmake --build build --parallel
  popd

  # For PSNR and SSIM metrics.
  git clone https://chromium.googlesource.com/codecs/libwebp2
  pushd libwebp2
    git checkout "${LIBWEBP2_COMMIT}"
    cmake -S . -B build \
      -DWP2_BUILD_TESTS=OFF -DWP2_BUILD_EXTRAS=OFF -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON \
      -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
    cmake --build build --parallel
  popd

  # DEVTOOLS=ON for Butteraugli and SSIMULACRA2 metrics binaries. See
  # https://github.com/cloudinary/ssimulacra2/blob/d2be72505ddc5c92aeb30f4a7f3ab53db45b314b/build_ssimulacra_from_libjxl_repo
  git clone https://github.com/libjxl/libjxl.git
  pushd libjxl
    git checkout "${LIBJXL_COMMIT}"
    ./deps.sh
    cmake -S . -B build \
      -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF -DJPEGXL_ENABLE_EXAMPLES=OFF \
      -DJPEGXL_ENABLE_JPEGLI=OFF -DJPEGXL_ENABLE_OPENEXR=OFF -DJPEGXL_ENABLE_DEVTOOLS=ON \
      -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON
    cmake --build build --parallel
  popd
popd

AVIFENC="third_party/libavif/build/avifenc"
AVIFDEC="third_party/libavif/build/avifdec"
CJXL="third_party/libjxl/build/tools/cjxl"
DJXL="third_party/libjxl/build/tools/djxl"
CWEBP="third_party/libwebp/build/cwebp"
DWEBP="third_party/libwebp/build/dwebp"
CWP2="third_party/libwebp2/build/cwp2"
DWP2="third_party/libwebp2/build/dwp2"
GET_DISTO="third_party/libwebp2/build/get_disto"
SSIMULACRA="third_party/libjxl/build/tools/ssimulacra_main"
SSIMULACRA2="third_party/libjxl/build/tools/ssimulacra2"
BUTTERAUGLI="third_party/libjxl/build/tools/butteraugli_main"

rm -rf "${ENCODED}"
rm -rf "${DECODED}"
rm -rf "${JSON}"

mkdir "${ENCODED}"
mkdir "${DECODED}"
mkdir "${JSON}"

echo "Generate AVIF images"

for SPEED in 9 6; do
  JSON_PATH="${JSON}/avif_s${SPEED}.json"
  cat <<EOT > "${JSON_PATH}"
{
  "constant_descriptions": [
    {"name": "Name of this batch"},
    {"codec": "Name of the codec used to generate this data"},
    {"repository": "URL to the codec implementation source"},
    {"version": "Version of the codec used to generate this data"},
    {"time": "Timestamp of when this data was generated"},
    {"original_path": "Path to the original image"},
    {"encoded_path": "Path to the encoded image"}
  ],
  "constant_values": [
    "AVIF speed ${SPEED}",
    "avif",
    "https://github.com/AOMediaCodec/libavif.git",
    "${LIBAVIF_COMMIT}",
    "$(date +%Y-%m-%dT%H:%M:%S)",
    "images/\${original_name}",
    "encoded/\${encoded_name}"
  ],
  "field_descriptions": [
    {"original_name": "Original image file name"},
    {"effort": "Compression effort parameter"},
    {"quality": "Compression quality parameter"},
    {"encoded_name": "Encoded image file name"},
    {"encoded_size": "Size of the encoded image file in bytes"},
    {"encoding_time": "Encoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"decoding_time": "Decoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"psnr": "Quality metric Peak Signal-to-Noise Ratio (libwebp2 implementation). See https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssim": "Quality metric Structural Similarity Index Measure (libwebp2 implementation). See https://en.wikipedia.org/wiki/Structural_similarity. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"butteraugli": "Quality metric Butteraugli (libjxl implementation). See https://en.wikipedia.org/wiki/Guetzli#Butteraugli. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra": "Quality metric SSIMULACRA (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra2": "Quality metric SSIMULACRA2 (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"p3norm": "Quality metric P3-norm (libjxl implementation). See https://en.wikipedia.org/wiki/Norm_(mathematics)#p-norm. Warning: There is no scientific consensus on which objective quality metric to use."}
  ],
  "field_values": [
EOT

  for IMAGE in "${IMAGES}"/*.png; do
    # libavif maps the quality range [0..100] to the discrete quantization range [63..0], so there is no need to encode 101 images.
    # Not doing quality 100 because it is lossless (inefficient and logs a warning).
    for QUALITY in 0 2 3 5 6 8 10 11 13 14 16 17 19 21 22 24 25 27 29 30 32 33 35 37 38 40 41 43 44 46 48 49 51 52 54 56 57 59 60 62 63 65 67 68 70 71 73 75 76 78 79 81 83 84 86 87 89 90 92 94 95 97 98; do
      IMAGE_NAME="$(basename "${IMAGE}")"
      ENCODED_NAME="$(basename "${IMAGE}" ".png")_avif_s${SPEED}_q${QUALITY}.avif"
      ENCODED_PATH="${ENCODED}/${ENCODED_NAME}"
      DECODED_PATH="${DECODED}/${ENCODED_NAME}.png"

      ENC_S="$(date +%s%N)"
      "${AVIFENC}" --speed ${SPEED} --qcolor ${QUALITY} --jobs 1 --ignore-exif --ignore-xmp --ignore-icc --cicp 2/2/2 \
        "${IMAGE}" -o "${ENCODED_PATH}" &> /dev/null
      ENC_S="$(($(date +%s%N)-ENC_S))"
      BYTES=$(stat -c%s "$ENCODED_PATH")
      DEC_S="$(date +%s%N)"
      "${AVIFDEC}" "${ENCODED_PATH}" "${DECODED_PATH}" &> /dev/null
      DEC_S="$(($(date +%s%N)-DEC_S))"

      METRIC_PSNR=( $("${GET_DISTO}" -psnr "${DECODED_PATH}" "${IMAGE}") )
      METRIC_SSIM=( $("${GET_DISTO}" -ssim "${DECODED_PATH}" "${IMAGE}") )
      METRIC_BUTTERAUGLI_P3NORM=( $("${BUTTERAUGLI}" "${IMAGE}" "${DECODED_PATH}" | tr '\n' ' ') )
      METRIC_SSIMULACRA=$("${SSIMULACRA}" "${IMAGE}" "${DECODED_PATH}")
      METRIC_SSIMULACRA2=$("${SSIMULACRA2}" "${IMAGE}" "${DECODED_PATH}")

      echo "    [\"${IMAGE_NAME}\", ${SPEED}, ${QUALITY}, \"${ENCODED_NAME}\", ${BYTES}, ${ENC_S}, ${DEC_S}, \
${METRIC_PSNR[1]}, ${METRIC_SSIM[1]}, ${METRIC_BUTTERAUGLI_P3NORM[0]}, ${METRIC_SSIMULACRA}, ${METRIC_SSIMULACRA2}, ${METRIC_BUTTERAUGLI_P3NORM[2]}]," >> "${JSON_PATH}"
    done
  done

  truncate -s-2 "${JSON_PATH}" # Remove comma
  echo "" >> "${JSON_PATH}" # New line
  echo "  ]" >> "${JSON_PATH}"
  echo "}" >> "${JSON_PATH}"
done

echo "Generate WebP images"

for METHOD in 1 4; do
  JSON_PATH="${JSON}/webp_m${METHOD}.json"
  cat <<EOT > "${JSON_PATH}"
{
  "constant_descriptions": [
    {"name": "Name of this batch"},
    {"codec": "Name of the codec used to generate this data"},
    {"repository": "URL to the codec implementation source"},
    {"version": "Version of the codec used to generate this data"},
    {"time": "Timestamp of when this data was generated"},
    {"original_path": "Path to the original image"},
    {"encoded_path": "Path to the encoded image"}
  ],
  "constant_values": [
    "WebP method ${METHOD}",
    "webp",
    "https://chromium.googlesource.com/webm/libwebp",
    "${LIBWEBP_COMMIT}",
    "$(date +%Y-%m-%dT%H:%M:%S)",
    "images/\${original_name}",
    "encoded/\${encoded_name}"
  ],
  "field_descriptions": [
    {"original_name": "Original image file name"},
    {"effort": "Compression effort parameter"},
    {"quality": "Compression quality parameter"},
    {"encoded_name": "Encoded image file name"},
    {"encoded_size": "Size of the encoded image file in bytes"},
    {"encoding_time": "Encoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"decoding_time": "Decoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"psnr": "Quality metric Peak Signal-to-Noise Ratio (libwebp2 implementation). See https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssim": "Quality metric Structural Similarity Index Measure (libwebp2 implementation). See https://en.wikipedia.org/wiki/Structural_similarity. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"butteraugli": "Quality metric Butteraugli (libjxl implementation). See https://en.wikipedia.org/wiki/Guetzli#Butteraugli. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra": "Quality metric SSIMULACRA (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra2": "Quality metric SSIMULACRA2 (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"p3norm": "Quality metric P3-norm (libjxl implementation). See https://en.wikipedia.org/wiki/Norm_(mathematics)#p-norm. Warning: There is no scientific consensus on which objective quality metric to use."}
  ],
  "field_values": [
EOT

  for IMAGE in "${IMAGES}"/*.png; do
    for QUALITY in {0..100}; do
      IMAGE_NAME="$(basename "${IMAGE}")"
      ENCODED_NAME="$(basename "${IMAGE}" ".png")_webp_m${METHOD}_q${QUALITY}.webp"
      ENCODED_PATH="${ENCODED}/${ENCODED_NAME}"
      DECODED_PATH="${DECODED}/${ENCODED_NAME}.png"

      ENC_S="$(date +%s%N)"
      "${CWEBP}" -m ${METHOD} -q ${QUALITY} "${IMAGE}" -o "${ENCODED_PATH}" &> /dev/null
      ENC_S="$(($(date +%s%N)-ENC_S))"
      BYTES=$(stat -c%s "$ENCODED_PATH")
      DEC_S="$(date +%s%N)"
      "${DWEBP}" "${ENCODED_PATH}" -o "${DECODED_PATH}" &> /dev/null
      DEC_S="$(($(date +%s%N)-DEC_S))"

      METRIC_PSNR=( $("${GET_DISTO}" -psnr "${DECODED_PATH}" "${IMAGE}") )
      METRIC_SSIM=( $("${GET_DISTO}" -ssim "${DECODED_PATH}" "${IMAGE}") )
      METRIC_BUTTERAUGLI_P3NORM=( $("${BUTTERAUGLI}" "${IMAGE}" "${DECODED_PATH}" | tr '\n' ' ') )
      METRIC_SSIMULACRA=$("${SSIMULACRA}" "${IMAGE}" "${DECODED_PATH}")
      METRIC_SSIMULACRA2=$("${SSIMULACRA2}" "${IMAGE}" "${DECODED_PATH}")

      echo "    [\"${IMAGE_NAME}\", ${SPEED}, ${QUALITY}, \"${ENCODED_NAME}\", ${BYTES}, ${ENC_S}, ${DEC_S}, \
${METRIC_PSNR[1]}, ${METRIC_SSIM[1]}, ${METRIC_BUTTERAUGLI_P3NORM[0]}, ${METRIC_SSIMULACRA}, ${METRIC_SSIMULACRA2}, ${METRIC_BUTTERAUGLI_P3NORM[2]}]," >> "${JSON_PATH}"
    done
  done

  truncate -s-2 "${JSON_PATH}" # Remove comma
  echo "" >> "${JSON_PATH}" # New line
  echo "  ]" >> "${JSON_PATH}"
  echo "}" >> "${JSON_PATH}"
done

echo "Generate JPEG XL images"

for EFFORT in 2 7; do
  JSON_PATH="${JSON}/jxl_e${EFFORT}.json"
  cat <<EOT > "${JSON_PATH}"
{
  "constant_descriptions": [
    {"name": "Name of this batch"},
    {"codec": "Name of the codec used to generate this data"},
    {"repository": "URL to the codec implementation source"},
    {"version": "Version of the codec used to generate this data"},
    {"time": "Timestamp of when this data was generated"},
    {"original_path": "Path to the original image"},
    {"encoded_path": "Path to the encoded image"},
    {"decoded_path": "Path to the decoded image"}
  ],
  "constant_values": [
    "JPEG XL effort ${EFFORT}",
    "jpegxl",
    "https://github.com/libjxl/libjxl.git",
    "${LIBJXL_COMMIT}",
    "$(date +%Y-%m-%dT%H:%M:%S)",
    "images/\${original_name}",
    "encoded/\${encoded_name}",
    "decoded/\${encoded_name}.png"
  ],
  "field_descriptions": [
    {"original_name": "Original image file name"},
    {"effort": "Compression effort parameter"},
    {"quality": "Compression quality parameter"},
    {"encoded_name": "Encoded image file name"},
    {"encoded_size": "Size of the encoded image file in bytes"},
    {"encoding_time": "Encoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"decoding_time": "Decoding duration in nanoseconds. Warning: Timings are environment-dependent and inaccurate."},
    {"psnr": "Quality metric Peak Signal-to-Noise Ratio (libwebp2 implementation). See https://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssim": "Quality metric Structural Similarity Index Measure (libwebp2 implementation). See https://en.wikipedia.org/wiki/Structural_similarity. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"butteraugli": "Quality metric Butteraugli (libjxl implementation). See https://en.wikipedia.org/wiki/Guetzli#Butteraugli. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra": "Quality metric SSIMULACRA (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"ssimulacra2": "Quality metric SSIMULACRA2 (libjxl implementation). See https://en.wikipedia.org/wiki/Structural_similarity#SSIMULACRA. Warning: There is no scientific consensus on which objective quality metric to use."},
    {"p3norm": "Quality metric P3-norm (libjxl implementation). See https://en.wikipedia.org/wiki/Norm_(mathematics)#p-norm. Warning: There is no scientific consensus on which objective quality metric to use."}
  ],
  "field_values": [
EOT

  for IMAGE in "${IMAGES}"/*.png; do
    # Quality 100 is lossless.
    for QUALITY in {0..99}; do
      IMAGE_NAME="$(basename "${IMAGE}")"
      ENCODED_NAME="$(basename "${IMAGE}" ".png")_jxl_e${EFFORT}_q${QUALITY}.jxl"
      ENCODED_PATH="${ENCODED}/${ENCODED_NAME}"
      DECODED_PATH="${DECODED}/${ENCODED_NAME}.png"

      ENC_S="$(date +%s%N)"
      "${CJXL}" --effort ${EFFORT} --quality ${QUALITY} --num_threads 0 -x strip=exif -x strip=xmp -x strip=iptc -x color_space=RGB_D65_SRG_Per_SRG \
        "${IMAGE}" "${ENCODED_PATH}" &> /dev/null
      ENC_S="$(($(date +%s%N)-ENC_S))"
      BYTES=$(stat -c%s "$ENCODED_PATH")
      DEC_S="$(date +%s%N)"
      "${DJXL}" "${ENCODED_PATH}" "${DECODED_PATH}" &> /dev/null
      DEC_S="$(($(date +%s%N)-DEC_S))"

      METRIC_PSNR=( $("${GET_DISTO}" -psnr "${DECODED_PATH}" "${IMAGE}") )
      METRIC_SSIM=( $("${GET_DISTO}" -ssim "${DECODED_PATH}" "${IMAGE}") )
      METRIC_BUTTERAUGLI_P3NORM=( $("${BUTTERAUGLI}" "${IMAGE}" "${DECODED_PATH}" | tr '\n' ' ') )
      METRIC_SSIMULACRA=$("${SSIMULACRA}" "${IMAGE}" "${DECODED_PATH}")
      METRIC_SSIMULACRA2=$("${SSIMULACRA2}" "${IMAGE}" "${DECODED_PATH}")

      echo "    [\"${IMAGE_NAME}\", ${SPEED}, ${QUALITY}, \"${ENCODED_NAME}\", ${BYTES}, ${ENC_S}, ${DEC_S}, \
${METRIC_PSNR[1]}, ${METRIC_SSIM[1]}, ${METRIC_BUTTERAUGLI_P3NORM[0]}, ${METRIC_SSIMULACRA}, ${METRIC_SSIMULACRA2}, ${METRIC_BUTTERAUGLI_P3NORM[2]}]," >> "${JSON_PATH}"
    done
  done

  truncate -s-2 "${JSON_PATH}" # Remove comma
  echo "" >> "${JSON_PATH}" # New line
  echo "  ]" >> "${JSON_PATH}"
  echo "}" >> "${JSON_PATH}"
done

echo "Reference all codec batches from top-level JSON"

echo "[" > batches.json
for BATCH in "${JSON}"/*.json; do
  echo "  [\"${BATCH}\"]," >> batches.json
done
truncate -s-2 batches.json
echo "" >> batches.json
echo "]" >> batches.json

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generate AVIF, WebP, JPEG XL data (bash script)

Clone this wiki locally