Using Aurora5 with FFmpeg

To use Aurora5 with FFmpeg, you need to invoke the encoder using the parameter libwz265 in your command line.

  • ./ffmpeg -h encoder=libwz265
Encoder libwz265 [libwz265 H.265 / HEVC]:
    General capabilities: delay threads
    Threading capabilities: other
    Supported pixel formats: yuv420p yuvj420p yuv420p10le gray gray10le yuv422p yuv422p10le yuv444p yuv444p10le

libwz265 AVOptions:
  -aq-mode           <int>        E..V....... AQ method (from -1 to INT_MAX) (default -1)
  -aq-strength       <float>      E..V....... AQ strength. Reduces blocking and blurring in flat and textured areas. (from -1 to FLT_MAX) (default -1)
  -aq-smooth         <float>      E..V....... AQ smooth.  (from -1 to FLT_MAX) (default -1)
  -psy-rd            <float>      E..V....... Strength of psychovisual optimization (from -1 to FLT_MAX) (default -1)
  -crf               <float>      E..V....... Set the wz265 crf (from -1 to FLT_MAX) (default -1)
  -preset            <string>     E..V....... Set the wz265 preset (default "medium")
  -tune              <string>     E..V....... set the wz265 tune parameter
  -udu_sei           <boolean>    E..V....... Use user data unregistered SEI if available (default false)
  -a53cc             <boolean>    E..V....... Use A53 Closed Captions (if available) (default true)
  -wz265-params      <string>     E..V....... Set the wz265 configuration using a :-separated list of key=value parameters

Note:

  • A list of all the parameters that can be passed using the configuration string wz265-params can be found here.
  • Further explanations for -preset and -tune in the table above are provided in the Presets and Tuning Options

Use of Aurora5 with FFmpeg and stereoscopic video

Stereoscopic video can be efficiently compressed using the multiview extension of the HEVC standard (MV-HEVC). This would require to specify via the wz265-params string the following:

./ffmpeg encoder=libwz265 -wz265-params use-mv-hevc=1

As mentioned in the API Definitions page, the left and right views for each multiview frame need to be submitted in an interleaved fashion to the Aurora5 encoder. Multiview content can be often be organised in a single compressed file where views are arranged in a single frame having twice the amount of pixels horizontally (if the arrangement is left/right) or vertically (if the arrangement if top/bottom). Alternatively, two separate compressed files, each associated with a monocular view, can be provided. The following command lines, provide the user with some examples on how to handle multiview input formats with FFmpeg so that the underlying Aurora5 encoder will receive the views according to the order expected.

Input multiview video organized as stacked top/bottom views

In this case, the `stereo3d video filter from FFmpeg is used to unpack the views and create in the internal FFmpeg's pipeline a sequence of alternated monocular views:

./ffmpeg -i <input_2D_video> -vf "stereo3d=abl:al,format=yuv420p" -c:v libwz265 -crf <quality> -wz265-params "use-mv-hevc=1", -preset <preset_chosen> <mv_hevc_stream.265>

Input multiview video organized as side-by-side views

Again we use the `stereo3d video filter from FFmpeg:

./ffmpeg -i <input_2D_video> -vf "stereo3d=sbsl:al,format=yuv420p" -c:v libwz265 -crf <quality> -wz265-params "use-mv-hevc=1", -preset <preset_chosen> <mv_hevc_stream.265>

Input multiview video organized as two separated files

In this case the interleave FFmpeg video filter is used to create in the internal FFmpeg's pipeline an interleaved sequence of monocular views as expected by Aurora5:

./ffmpeg -r <fps/2> -i <input_2D_video1>  -r <fps/2> -i <input_2D_video2> -filter_complex "[0:v][1:v] interleave" -c:v libwz265 -crf <quality> -wz265-params "use-mv-hevc=1", -preset <preset_chosen> <mv_hevc_stream.265>

It should be noted the specification in the command line of half of the input frame rate fps/2. This is because the interleaved video filter would create in output a video with twice the input frame rate of both videos.

Note:

  • Due to the recent interest around MV-HEVC for applications using the Apple Vision Pro headset, the output file format wrapper from FFmpeg (e.g. .mov or .mp4) might not be complaint with the Apple HEVC Stereo Video Profile. It is recommended to simply output from FFmpeg the elementary stream (so-called AnnexB stream, i.e. the .265 file) and then wrap it using MP4Box.

Integrating Aurora5's SDK to FFmpeg

Here are build instructions taking FFmpeg n7.0.2 version as an example (assume $HOME is the working directory),

  • Step 1: Checkout FFmpeg codebase to tag n7.0.2 and copy files from aurora5-sdk/ffmpeg_src to FFmpeg works. Or applying aurora5-sdk/ffmpeg_src/FFmpeg-7.0.2.diff to FFmpeg works as well. For example,
    • cd $HOME && git clone https://git.ffmpeg.org/ffmpeg.git
    • cd $HOME/ffmpeg && git checkout n7.0.2
    • cp -r $HOME/aurora5-sdk/ffmpeg_src/* $HOME/ffmpeg/
  • Step 2: Create a "dep" directory under the FFmpeg project, and copy the whole "aurora5-sdk" directory into it.
    • mkdir -p ./dep
    • cp -r $HOME/aurora5-sdk/ ./dep/
  • Step 3: Run build.sh.
    • chmod +x ./build.sh
    • ./build.sh
  • Step 4: You can find your own ffmpeg (binary file) under $HOME/ffmpeg/out/bin/

Note:

When integrating without using the build.sh script, a couple of extra steps need to be executed.

  • Modify the prefix value in the pc files (lib/pkgconfig/wz265.pc) to the full path of Aurora5 SDKs directory and set the environment named PKG_CONFIG_PATH. For example,
sed -i "s@^prefix=\(\S\+\)@prefix=$HOME/aurora-sdk@g" $HOME/aurora5-sdk/lib/pkgconfig/*.pc
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:$HOME/aurora5-sdk/lib/pkgconfig
  • Set the lib and include directory with the environment variables named LD_LIBRARY_PATH/LIBRARY_PATH/C_INCLUDE_PATH. For example,
  • export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/aurora5-sdk/lib
  • export LIBRARY_PATH=$LIBRARY_PATH:$HOME/aurora5-sdk/lib
  • export C_INCLUDE_PATH=$C_INCLUDE_PATH:$HOME/aurora5-sdk/lib
  • Add necessary settings to configure options as follows, and note that --enable-libwz265 must be set at the same time:
 --extra-libs="-lpthread -lstdc++ -lm" \
 --extra-cflags="-msse4 -std=c99 -D_GNU_SOURCE" \
 --enable-libwz265 \