制作avifenc-butteraugli docker镜像

WARNING: This article may be obsolete
This post was published in 2022-05-19. Obviously, expired content is less useful to users if it has already pasted its expiration date.

This article is categorized as "Garbage" . It should NEVER be appeared in your search engine's results.

Table of Contents

参考资料

向依赖地狱投降

经验总结：绝大部分场合下，编译butteraugli相关的内容时一定要使用最新git repo

所有参考资料：

🔗 [Enable tune butteraugli support · Issue #622 · AOMediaCodec/libavif] https://github.com/AOMediaCodec/libavif/issues/622

🔗 [Error while loading shared libraries · Issue #49 · gmarcais/Jellyfish] https://github.com/gmarcais/Jellyfish/issues/49

🔗 [relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC · Issue #2171 · BVLC/caffe] https://github.com/BVLC/caffe/issues/2171

🔗 [Exception for sh1 dot not found · Issue #13 · beyondcode/laravel-er-diagram-generator] https://github.com/beyondcode/laravel-er-diagram-generator/issues/13

🔗 [High quality encoding of AVIF images using aomenc-av1: a small guide : AV1] https://www.reddit.com/r/AV1/comments/o7s8hk/high_quality_encoding_of_avif_images_using/

🔗 [Is there any suggest about the butteraugli compare result value? · Issue #22 · google/butteraugli] https://github.com/google/butteraugli/issues/22

（有关butteraugli的编译）🔗 [butteraugli | Wener Live & Life] https://wener.me/notes/service/media/butteraugli

🔗 [Using AVIF to compress images on your site] https://web.dev/compress-images-avif/#building-command-line-encoder-avifenc

准备

Debian的Docker（命令行版本，不是Docker-Desktop）安装笔记见：🔗 [Wordpress docker - Truxton's blog] https://truxton2blog.com/wordpress-docker/

先拉一个Debian 12（以前用的是Debian 11，2023年6月换成了Debian 12），安装一堆包：

# 在此之前最好检查一下有没有历史遗留的docker debian images

docker pull docker.io/debian

执行 $ docker run --name debian_container_avif_webp -it debian /bin/bash 以后，在docker-debian-bash下：

apt update;apt upgrade


apt install -y clang vim wget git sudo ninja-build nasm cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev

方法1（成功）

汇总

版本：

libwebp: v1.4.0

libavif: v1.1.1

aom: v3.10.0

libjxl: ~~默认latest. 因为只会用到butteraugli相关的代码，所以对版本稳定性没有特殊要求。~~ 很长一段时间都用latest，但2023年6月突然发现latest有点问题（butteraugli.h装不上去），所以暂时用最新的stable release v0.8.2 . 再后来我发现libjxl开发组已经把butteraugli api从项目中永久移除了（见：🔗 [remove Butteraugli from the libjxl API by jonsneyers · Pull Request #2576 · libjxl/libjxl] https://github.com/libjxl/libjxl/pull/2576），但鉴于reddit的很多人认为Butteraugli仍然是最佳，所以本项目暂时还在用Butteraugli，v0.8.2可能要一直这么用下去了。

但一直用v0.8.2总觉得有点不舒服，所以还在密切关注Butteraugli的候选算法（比如VMAF）。

2024-08-30 编译通过：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev


# webp
cd /root

wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.4.0.tar.gz

tar -xzvf libwebp-1.4.0.tar.gz

cd libwebp-1.4.0
./configure
make -j$(nproc)
sudo make install

rm -rf /root/libwebp-1.4.0

rm /root/libwebp-1.4.0.tar.gz


# avif

cd /root

git clone -b v0.8.2 https://github.com/libjxl/libjxl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev

cd libjxl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone -b v3.10.0 https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j$(nproc)
sudo make install


cd /root

git clone -b v1.1.1 https://github.com/AOMediaCodec/libavif

cd libavif/ext
bash libyuv.cmd
bash libsharpyuv.cmd
bash libjpeg.cmd
bash zlibpng.cmd

cd ..

cmake -S . -B build -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=SYSTEM -DAVIF_LIBYUV=LOCAL -DAVIF_LIBSHARPYUV=LOCAL -DAVIF_JPEG=LOCAL -DAVIF_ZLIBPNG=LOCAL -DAVIF_BUILD_APPS=ON
cmake --build build --parallel

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

接下来会删除多余文件，如果以上编译环境需要保存并反复调试则需要在接下来的命令开始之前保存一次docker：

# 删除多余文件
apt --purge remove clang vim wget git ninja-build nasm cmake
apt autoremove

# 有的时候还是会误伤libatomic，需要补回去
# 最近（2023年9月以后）发现已经不需要这么做了
# apt install libatomic1

rm -rf /root/aom
rm -rf /root/libjxl

# 检查
avifenc --version
avifdec --version
cwebp -version
dwebp -version

然后导出Docker镜像：

docker commit debian_container_avif_webp debian:avif_webp_modified

docker save debian:avif_webp_modified | gzip > avif_webp.tar.gz

历史版本

2022-05，不包含GoogleTest的版本：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev


cd /root

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install


cd /root

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DCMAKE_BUILD_TYPE=Release

ninja

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

2022-05，包含GoogleTest的版本：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev


cd /root

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install


cd /root

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DAVIF_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release

ninja

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

分段代码（软件版本已经过期）

建议直接参考汇总

直接照搬了下面链接的构建代码：

🔗 [Enable tune butteraugli support · Issue #622 · AOMediaCodec/libavif] https://github.com/AOMediaCodec/libavif/issues/622

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install

如果拉取的libavif在commit 44b086a31433e63f84d9a4e6cca49e311d7c9833 之后，那么就需要去掉google test的构建；否则可以继续使用google test.

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DAVIF_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release

ninja

在启动指令前，需要

export LD_LIBRARY_PATH=/usr/local/lib

在Docker中运行

首先后台开一个docker容器：

docker run  -v /xxxxx/wp-content/:/xxxxx/wp-content:Z --name avif --rm -it -d  -e LD_LIBRARY_PATH=/usr/local/lib [docker image ID]

可以再次检查一遍avif/webp的版本：

docker exec avif cwebp -version

docker exec avif dwebp -version

docker exec avif avifenc --version

docker exec avif avifdec --version

然后执行转换命令（每转换一张图都要使用这个命令）：

docker exec avif avifenc [-parameters] input.jpg output.jpg.avif

方法2（目前是废案，无法通过butteraugli的编译）

开始构建jpegXL

# 主要参考https://github.com/libjxl/libjxl的教程
# 一定要使用最新版


git clone https://github.com/libjxl/libjxl.git --recursive --shallow-submodules


export CC=clang CXX=clang++


#  apt install libavif-dev libavif9


cd libjxl
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF ..
cmake --build . -- -j$(nproc)


sudo cmake --install .


#  apt remove libavif-dev libavif9

然后开始处理libavif-ext

git clone https://github.com/AOMediaCodec/libavif


cd libavif/ext

处理aom，首先clone git repo：

git clone --depth=1 https://aomedia.googlesource.com/aom

cd aom
mkdir build.libavif
cd build.libavif

# 选择一个cmake语句
# cmake xxxxxxxx

ninja

可选cmake：

# 原始cmake（不含DBUILD_SHARED_LIBS）：
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..

# 添加DBUILD_SHARED_LIBS：
cmake -G Ninja -DBUILD_SHARED_LIBS=1 -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0  ..

# 添加DCONFIG_TUNE_BUTTERAUGLI：
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0 -DCONFIG_TUNE_BUTTERAUGLI=1 ..

然后处理libyuv

修改libyuv.cmd的内容如下：

git clone https://chromium.googlesource.com/libyuv/libyuv

cd libyuv

mkdir build
cd build

# 选择一个cmake语句
# cmake xxxxxxx

ninja yuv

可选的cmake语句：

# 原始cmake
cmake -G Ninja -DBUILD_SHARED_LIBS=0 -DCMAKE_BUILD_TYPE=Release ..

# 修改DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS
cmake -G Ninja -DBUILD_SHARED_LIBS=1 -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_BUILD_TYPE=Release ..

然后编译libavif

cd libavif
mkdir build
cd build


# 选择一个cmake语句
# cmake xxxxxxx

make

可选的cmake语句：

# 原始语句，https://web.dev/compress-images-avif/#building-command-line-encoder-avifenc
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DAVIF_CODEC_AOM=1 -DAVIF_LOCAL_AOM=1 -DAVIF_BUILD_APPS=1 ..

# 使用本地libyuv:
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DAVIF_CODEC_AOM=1 -DAVIF_LOCAL_AOM=1 -DAVIF_BUILD_APPS=1 -DAVIF_LOCAL_LIBYUV=1 ..

参数组合

aom	libyuv	libavif	结果
原班人马: 不显式声明-DBUILD_SHARED_LIBS	原班人马：显式声明-DBUILD_SHARED_LIBS=0	原班人马：显式声明-DBUILD_SHARED_LIBS=0	（原班人马）可行

不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	显式声明-DBUILD_SHARED_LIBS=1	55%以后爆炸报错
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	显式声明-DBUILD_SHARED_LIBS=0	88%以后爆炸报错
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=0	88%以后爆炸报错，同时还有报错“ undefined reference to `JxlButteraugliApiSetHFAsymmetry' ”
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	59%以后爆炸报错，结尾没有关于Butteraugli的报错
显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1	报错：recompile with -fPIC，需要修改libyuv的cmake选项（修改结果见下一行）
显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	修好了上一行的问题：可行
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	52%以后爆炸报错
显式声明-DBUILD_SHARED_LIBS=0，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	88%以后爆炸报错，同时还有报错“ undefined reference to `JxlButteraugliApiSetHFAsymmetry' ”

去掉本地yuv，专门负责处理aom

aom	libavif	结果
不显式声明-DBUILD_SHARED_LIBS	显式声明-DBUILD_SHARED_LIBS=0	能编译

不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	92%以后出现错误： undefined reference to `JxlButteraugli*'
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1	62%以后爆炸错误
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	92%以后出现错误： undefined reference to `JxlButteraugli*'
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1	92%以后出现错误： undefined reference to `JxlButteraugli*'
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	去掉-DBUILD_SHARED_LIBS	92%以后出现错误： undefined reference to `JxlButteraugli*'

libyuv废案：

git clone https://chromium.googlesource.com/libyuv/libyuv
cd libyuv
mkdir out
cd out
cmake -DCMAKE_INSTALL_PREFIX="/usr/local" -DCMAKE_BUILD_TYPE="Release" ..
cmake --build . --config Release
sudo cmake --build . --target install --config Release

Last Modified in 2024-08-30

参考资料

准备

方法1（成功）

汇总

分段代码（软件版本已经过期）

在Docker中运行

方法2（目前是废案，无法通过butteraugli的编译）

Leave a Comment Anonymous comment is allowed / 允许匿名评论 Cancel reply