制作avifenc-butteraugli docker镜像

WARNING: This article may be obsolete
This post was published in 2022-05-19. Obviously, expired content is less useful to users if it has already pasted its expiration date.

This article is categorized as "Garbage" . It should NEVER be appeared in your search engine's results.

Table of Contents

参考资料

向依赖地狱投降

经验总结：绝大部分场合下，编译butteraugli相关的内容时一定要使用最新git repo

所有参考资料：

🔗 [Enable tune butteraugli support · Issue #622 · AOMediaCodec/libavif] https://github.com/AOMediaCodec/libavif/issues/622

🔗 [Error while loading shared libraries · Issue #49 · gmarcais/Jellyfish] https://github.com/gmarcais/Jellyfish/issues/49

🔗 [relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC · Issue #2171 · BVLC/caffe] https://github.com/BVLC/caffe/issues/2171

🔗 [Exception for sh1 dot not found · Issue #13 · beyondcode/laravel-er-diagram-generator] https://github.com/beyondcode/laravel-er-diagram-generator/issues/13

🔗 [High quality encoding of AVIF images using aomenc-av1: a small guide : AV1] https://www.reddit.com/r/AV1/comments/o7s8hk/high_quality_encoding_of_avif_images_using/

🔗 [Is there any suggest about the butteraugli compare result value? · Issue #22 · google/butteraugli] https://github.com/google/butteraugli/issues/22

（有关butteraugli的编译）🔗 [butteraugli | Wener Live & Life] https://wener.me/notes/service/media/butteraugli

🔗 [Using AVIF to compress images on your site] https://web.dev/compress-images-avif/#building-command-line-encoder-avifenc

准备

Debian的Docker（命令行版本，不是Docker-Desktop）安装笔记见：🔗 [Wordpress docker - Truxton's blog] https://truxton2blog.com/wordpress-docker/

先拉一个Debian 12（以前用的是Debian 11，2023年6月换成了Debian 12），安装一堆包：

# 在此之前最好检查一下有没有历史遗留的docker debian images

docker pull docker.io/debian

执行 $ docker run --name debian_container_avif_webp -it debian /bin/bash 以后，在docker-debian-bash下：

apt update;apt upgrade


apt install -y clang vim wget git sudo ninja-build nasm cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev

方法1（成功）

汇总

版本：

libwebp: v1.4.0

libavif: v1.0.4

aom: v3.9.1

libjxl: ~~默认latest. 因为只会用到butteraugli相关的代码，所以对版本稳定性没有特殊要求。~~ 很长一段时间都用latest，但2023年6月突然发现latest有点问题（butteraugli.h装不上去），所以暂时用最新的stable release v0.8.2 . 再后来我发现libjxl开发组已经把butteraugli api从项目中永久移除了（见：🔗 [remove Butteraugli from the libjxl API by jonsneyers · Pull Request #2576 · libjxl/libjxl] https://github.com/libjxl/libjxl/pull/2576），但鉴于reddit的很多人认为Butteraugli仍然是最佳，所以本项目暂时还在用Butteraugli，v0.8.2可能要一直这么用下去了。

但一直用v0.8.2总觉得有点不舒服，所以还在密切关注Butteraugli的候选算法。

2024-06-10 编译通过：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev


# webp
cd /root

wget https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.4.0.tar.gz

tar -xzvf libwebp-1.4.0.tar.gz

cd libwebp-1.4.0
./configure
make -j$(nproc)
sudo make install

rm -rf /root/libwebp-1.4.0

rm /root/libwebp-1.4.0.tar.gz


# avif

cd /root

git clone -b v0.8.2 https://github.com/libjxl/libjxl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev

cd libjxl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone -b v3.9.1 https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j$(nproc)
sudo make install


cd /root

git clone -b v1.0.4 https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DCMAKE_BUILD_TYPE=Release

ninja

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

保存docker以后可以考虑删除多余文件：

apt --purge remove clang vim wget git ninja-build nasm cmake
apt autoremove

# 有的时候还是会误伤libatomic，需要补回去
# 最近（2023年9月以后）发现已经不需要这么做了
# apt install libatomic1

rm -rf /root/aom
rm -rf /root/libjxl

# 检查
avifenc --version
avifdec --version
cwebp -version
dwebp -version

然后导出Docker镜像：

docker commit debian_container_avif_webp debian:avif_webp_modified

docker save debian:avif_webp_modified | gzip > avif_webp.tar.gz

历史版本

2022-05，不包含GoogleTest的版本：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev


cd /root

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install


cd /root

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DCMAKE_BUILD_TYPE=Release

ninja

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

2022-05，包含GoogleTest的版本：

apt update;apt -y upgrade


apt install -y clang vim wget git sudo ninja-build nasm


apt install -y  cmake pkg-config libbrotli-dev libgflags-dev libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev


cd /root

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

cd /root
git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install


cd /root

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DAVIF_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release

ninja

ln -s /root/libavif/build/avifenc /usr/bin/avifenc
ln -s /root/libavif/build/avifdec /usr/bin/avifdec

分段代码（软件版本已经过期）

建议直接参考汇总

直接照搬了下面链接的构建代码：

🔗 [Enable tune butteraugli support · Issue #622 · AOMediaCodec/libavif] https://github.com/AOMediaCodec/libavif/issues/622

git clone https://gitlab.com/wg1/jpeg-xl.git --recursive
sudo apt install cmake pkg-config libbrotli-dev
sudo apt install libgif-dev libjpeg-dev libopenexr-dev libpng-dev libwebp-dev

cd jpeg-xl
mkdir build
cd build

apt --purge remove -y gcc

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF -DJPEGXL_ENABLE_BENCHMARK=OFF ..
cmake --build . -- -j$(nproc)

sudo cmake --install .

git clone https://aomedia.googlesource.com/aom
cd aom

# If you want to use libaom v3.1.0, run the command:
#     git checkout v3.1.0
# and then apply the following four patches:
#     https://aomedia-review.googlesource.com/c/aom/+/136044
#     https://aomedia-review.googlesource.com/c/aom/+/134506
#     https://aomedia-review.googlesource.com/c/aom/+/135521
#     https://aomedia-review.googlesource.com/c/aom/+/136082

mkdir build.libavif
cd build.libavif
cmake -DCONFIG_TUNE_BUTTERAUGLI=1 -DCMAKE_BUILD_TYPE=Release \
-DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 \
-DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..
make -j
sudo make install

如果拉取的libavif在commit 44b086a31433e63f84d9a4e6cca49e311d7c9833 之后，那么就需要去掉google test的构建；否则可以继续使用google test.

git clone https://github.com/AOMediaCodec/libavif
cd libavif
mkdir build
cd build


cmake -G Ninja .. -DBUILD_SHARED_LIBS=OFF -DAVIF_CODEC_AOM=ON \
-DAVIF_LOCAL_AOM=OFF -DAVIF_BUILD_EXAMPLES=ON -DAVIF_BUILD_APPS=ON \
-DAVIF_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Release

ninja

在启动指令前，需要

export LD_LIBRARY_PATH=/usr/local/lib

在Docker中运行

首先后台开一个docker容器：

docker run  -v /xxxxx/wp-content/:/xxxxx/wp-content:Z --name avif --rm -it -d  -e LD_LIBRARY_PATH=/usr/local/lib [docker image ID]

可以再次检查一遍avif/webp的版本：

docker exec avif cwebp -version

docker exec avif dwebp -version

docker exec avif avifenc --version

docker exec avif avifdec --version

然后执行转换命令（每转换一张图都要使用这个命令）：

docker exec avif avifenc [-parameters] input.jpg output.jpg.avif

方法2（目前是废案，无法通过butteraugli的编译）

开始构建jpegXL

# 主要参考https://github.com/libjxl/libjxl的教程
# 一定要使用最新版


git clone https://github.com/libjxl/libjxl.git --recursive --shallow-submodules


export CC=clang CXX=clang++


#  apt install libavif-dev libavif9


cd libjxl
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF ..
cmake --build . -- -j$(nproc)


sudo cmake --install .


#  apt remove libavif-dev libavif9

然后开始处理libavif-ext

git clone https://github.com/AOMediaCodec/libavif


cd libavif/ext

处理aom，首先clone git repo：

git clone --depth=1 https://aomedia.googlesource.com/aom

cd aom
mkdir build.libavif
cd build.libavif

# 选择一个cmake语句
# cmake xxxxxxxx

ninja

可选cmake：

# 原始cmake（不含DBUILD_SHARED_LIBS）：
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0 ..

# 添加DBUILD_SHARED_LIBS：
cmake -G Ninja -DBUILD_SHARED_LIBS=1 -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0  ..

# 添加DCONFIG_TUNE_BUTTERAUGLI：
cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DENABLE_DOCS=0 -DENABLE_EXAMPLES=0 -DENABLE_TESTDATA=0 -DENABLE_TESTS=0 -DENABLE_TOOLS=0 -DCONFIG_TUNE_BUTTERAUGLI=1 ..

然后处理libyuv

修改libyuv.cmd的内容如下：

git clone https://chromium.googlesource.com/libyuv/libyuv

cd libyuv

mkdir build
cd build

# 选择一个cmake语句
# cmake xxxxxxx

ninja yuv

可选的cmake语句：

# 原始cmake
cmake -G Ninja -DBUILD_SHARED_LIBS=0 -DCMAKE_BUILD_TYPE=Release ..

# 修改DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS
cmake -G Ninja -DBUILD_SHARED_LIBS=1 -DCMAKE_CXX_FLAGS=-fPIC -DCMAKE_BUILD_TYPE=Release ..

然后编译libavif

cd libavif
mkdir build
cd build


# 选择一个cmake语句
# cmake xxxxxxx

make

可选的cmake语句：

# 原始语句，https://web.dev/compress-images-avif/#building-command-line-encoder-avifenc
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DAVIF_CODEC_AOM=1 -DAVIF_LOCAL_AOM=1 -DAVIF_BUILD_APPS=1 ..

# 使用本地libyuv:
cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=0 -DAVIF_CODEC_AOM=1 -DAVIF_LOCAL_AOM=1 -DAVIF_BUILD_APPS=1 -DAVIF_LOCAL_LIBYUV=1 ..

参数组合

aom	libyuv	libavif	结果
原班人马: 不显式声明-DBUILD_SHARED_LIBS	原班人马：显式声明-DBUILD_SHARED_LIBS=0	原班人马：显式声明-DBUILD_SHARED_LIBS=0	（原班人马）可行

不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	显式声明-DBUILD_SHARED_LIBS=1	55%以后爆炸报错
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	显式声明-DBUILD_SHARED_LIBS=0	88%以后爆炸报错
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=0	88%以后爆炸报错，同时还有报错“ undefined reference to `JxlButteraugliApiSetHFAsymmetry' ”
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	59%以后爆炸报错，结尾没有关于Butteraugli的报错
显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1	报错：recompile with -fPIC，需要修改libyuv的cmake选项（修改结果见下一行）
显式声明-DBUILD_SHARED_LIBS=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	修好了上一行的问题：可行
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	52%以后爆炸报错
显式声明-DBUILD_SHARED_LIBS=0，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1，同时添加CMAKE_CXX_FLAGS=-fPIC	显式声明-DBUILD_SHARED_LIBS=1	88%以后爆炸报错，同时还有报错“ undefined reference to `JxlButteraugliApiSetHFAsymmetry' ”

去掉本地yuv，专门负责处理aom

aom	libavif	结果
不显式声明-DBUILD_SHARED_LIBS	显式声明-DBUILD_SHARED_LIBS=0	能编译

不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	92%以后出现错误： undefined reference to `JxlButteraugli*'
不显式声明-DBUILD_SHARED_LIBS，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1	62%以后爆炸错误
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=0	92%以后出现错误： undefined reference to `JxlButteraugli*'
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	显式声明-DBUILD_SHARED_LIBS=1	92%以后出现错误： undefined reference to `JxlButteraugli*'
显式声明-DBUILD_SHARED_LIBS=1，附带-DCONFIG_TUNE_BUTTERAUGLI=1	去掉-DBUILD_SHARED_LIBS	92%以后出现错误： undefined reference to `JxlButteraugli*'

libyuv废案：

git clone https://chromium.googlesource.com/libyuv/libyuv
cd libyuv
mkdir out
cd out
cmake -DCMAKE_INSTALL_PREFIX="/usr/local" -DCMAKE_BUILD_TYPE="Release" ..
cmake --build . --config Release
sudo cmake --build . --target install --config Release

Last Modified in 2024-06-10

参考资料

准备

方法1（成功）

汇总

分段代码（软件版本已经过期）

在Docker中运行

方法2（目前是废案，无法通过butteraugli的编译）

Leave a Comment Anonymous comment is allowed / 允许匿名评论 Cancel reply