Film Grain Synthesis

Introduction

Aurora1 supports two methods for film grain synthesis:

Integrated Approach: Noise reduction and estimation are performed during the encoding process.
Combined Approach: A separate denoiser and noise modeler are used to generate a denoised video and a grain table. The grain table is then passed to the encoder as a parameter.

Integrated Approach

To enable Film Grain Synthesis (FGS) in Aurora1, add the --denoise-noise-level=25 option to the command line.

For example:

./aomenc {INPUT} --cpu-used={SPEED} --threads={THREADS} --kf-min-dist=0 --kf-max-dist={GOP} --passes=2 --lag-in-frames=25 --end-usage=q --cq-level={CRF}  -o {OUTPUT} --denoise-noise-level=25 --deltaq-mode=2 --deltaq-strength=100

When using the API, configure WZAV1E_SET_DENOISE_NOISE_LEVEL. For example:

  // 5. Set more options
  if (wz_codec_control(&codec, WZE_SET_CPUUSED, speed))
    die_codec(&codec, "Failed to set cpu-used");

  // To enable film grain synthesis, set the Noise_level parameter to a value
  // greater than 0.
  const int noise_level = 0;
  if (wz_codec_control(&codec, WZAV1E_SET_DENOISE_NOISE_LEVEL, noise_level))
    die_codec(&codec, "Failed to set denoise_level");

This approach eliminates the need to store intermediate denoised results. However, it requires performing denoising and noise modeling for each encoding pass during ABR ladder creation, resulting in higher computational complexity.

The --denoise-noise-level parameter acts as a switch for Film Grain Synthesis (FGS). While the noise level itself is estimated internally, setting this parameter to 0 disables FGS, while any value greater than 0 enables it.

Combined Approach

To perform film grain synthesis separately, use the following command lines:

# 1. Denoise first
./video_denoiser --input=$FILE --output=$FILE\_denoised_weak.yuv --arnr-maxframes=5 --threads=8 --degrain-strength=10 --tf-degrain-strength=7 
./video_denoiser --input=$FILE --output=$FILE\_denoised_strong.yuv --arnr-maxframes=5 --threads=8 --degrain-strength=15 --tf-spatial-ctx=2 --tf-degrain-strength=20 

# 2. Noise modeling
./noise_model  --input-denoised=$FILE\_denoised_strong.yuv --input=$FILE --output-grain-table=$FILE\_grain-params_$TAG.tbl --skip-frames=1 --threads=8 --boost-ratio=10

# 3. Encoding the denoised video with film grain table
./aomenc $FILE\_denoised_weak.yuv  --passes=2 --cpu-used={SPEED} --threads={THREADS} --kf-min-dist=0 --kf-max-dist={GOP} --passes=2 --lag-in-frames=25 --end-usage=q --cq-level={CRF} -o {OUTPUT} --film-grain-table=grain-params.tbl --width=1920 --height=1080 --fps=24000/1001

Note that the video_denoiser creates two distinct denoised videos: one optimized for the video encoder and the other for the noise modeler. This dual approach allows for improved detail preservation in the encoded video while simultaneously enabling more accurate grain estimation. To reduce computational cost, you can optionally create a single denoised video to be used by both the noise modeler and the video encoder.

When using the API, configure the grain parameter table path by setting WZAV1E_SET_FILM_GRAIN_TABLE. For example:

  // 5. Set more options
  if (wz_codec_control(&codec, WZE_SET_CPUUSED, speed))
    die_codec(&codec, "Failed to set cpu-used");

  // Setting external film grain table.
  const char *grain_param_tbl = "input-grain-param.tbl";
  if (wz_codec_control(&codec, WZAV1E_SET_FILM_GRAIN_TABLE, grain_param_tbl))
    die_codec(&codec, "Failed to set film grain table path");

This method allows for offline processing using various denoisers and noise modelers, avoiding redundant denoising and noise modeling steps. The drawback is the necessity to store the intermediate denoised video.

Video Denoiser

The video_denoiser tool is designed to reduce noise in video files by applying spatial and temporal denoising techniques, supporting raw YUV and Y4M formats. Users can specify parameters like resolution, bit depth, pixel format, and thread count, along with additional settings to control the strength of both spatial and temporal denoising. This tool provides flexible options for adjusting denoise modes, degrain strength, and spatial context to optimize video quality.

Usage

./video_denoiser -i input_filename -o dst_filename -w width -h height -b bitdepth -t threads --pix-fmt=format

Explanation of the parameters

Short form	Long form	Description
`--help`		Show usage options and exit
`-i <arg>`	`--input=<arg>`	Input file name (Only raw YUV file or Y4M file are supported)
`-o <arg>`	`--output=<arg>`	Output file name (Only raw YUV file are supported)
`-w <arg>`	`--width=<arg>`	Input width, valid range [32, 7680]
`-h <arg>`	`--height=<arg>`	Input height, valid range [32, 7680]
`-b <arg>`	`--bitdepth=<arg>`	Bit depth of input file, only 8/10 bit are supported
	`--pix-fmt=<arg>`	Pixel format, only supports 420/422/444
`-t <arg>`	`--threads=<arg>`	Maximum threads to be used, valid range [1, 16]
	`--degrain-strength`	Strength of spatial denoising, default 10, range [10, 40]
	`--arnr-maxframes`	Number of frames used in temporal denoising, valid range [3, 9]
	`--limit=<arg>`	Maximum threads to be processed
	`--denoise-mode=<arg>`	1: spatial denoise only; 2: temporal denoise only; 3: spatial + temporal denoise
	`--tf-degrain-strength=<arg>`	Strength for temporal denoise, range [1, 40], default is 10
	`--tf-spatial-ctx=<arg>`	Size of spatial support used in temporal filter, default 1

Noise Modeler

The noise_model tool is used to generate film grain parameters for video files, specifically supporting raw YUV and Y4M formats. It takes an original and a denoised input file to generate an output grain table file, with customizable options for resolution, bit depth, frame rate, and pixel format. This tool allows control over frame skipping, thread usage, and grain boost ratio to fine-tune the synthesis of film grain for enhanced visual quality.

Usage

./noise_model --input=source.yuv --input-denoised=denoised.yuv --output-grain-table=grain-params.tbl -w width -h height -b bitdepth -t threads --pix-fmt=format --fps=rate

Explanation of the parameters

Short form	Long form	Description
`--help`		Show usage options and exit
`-i <arg>`	`--input=<arg>`	Input file name (Only raw YUV file/Y4M are supported)
`-d <arg>`	`--input-denoised=<arg>`	Input denoised file name (Only raw YUV file are supported)
`-o <arg>`	`--output-grain-table=<arg>`	Output film grain table file path
`-w <arg>`	`--width=<arg>`	Input width, valid range [32, 7680]
`-h <arg>`	`--height=<arg>`	Input height, valid range [32, 7680]
`-b <arg>`	`--bitdepth=<arg>`	Bit depth of input file, only 8/10 bit are supported
	`--fps=<arg>`	Frame rate (rate/scale), Mandatory
	`--pix-fmt=<arg>`	Pixel format, only support 420/422/444
`-s <arg>`	`--skip-frames=<arg>`	(Number of frames - 1) to be skipped, suggest 8
	`--limit=<arg>`	Maximum threads to be processed
	`--boost-ratio=<arg>`	Control the ratio to boost the strength of synthesized grain, real ratio is boost_arg / 10, default 10