ProRes The Definitive FFmpeg Guide

A Definitive FFmpeg Guide By Harry Munday

Do you hate FFmpeg?

Life?

Lazy Software Developers?

Well I cant help you with that, but I can fix your ProRes encodes before you lose your mind or a job in some cases.

What is a Codec & Container?

Just watch this YouTube video:

https://www.youtube.com/watch?v=-4NXxY4maYc

What is ProRes

An capture format for cameras (Mostly External Recorders as it requires a licence)

Used widely in editing and professional distribution of masters or proxies of media.

It goes from lossy compressed to visually lossless compressed to 4444XQ / ProRes RAW / ProRes RAW HQ

ProRes Raw is not supported by FFmpeg for encoding as of writing this - July 2023

Tools You Need

Media Info

AMCDX Video Patcher / Archive Download

FFmpeg (Best Installed as path by apt get / )

StaxRip Which is a Avisynth / Vapoursynth/ FFmpeg Graphical Scripting Tool.

Getting FFmpeg

You want it installed via path (i.e system wide on any terminal, also makes scripts just work)

Linux apt get ffmpeg
MacOS brew install ffmpeg
Windows choco install ffmpeg

Brew on MacOS follow your applicable install guide here

How to install choco on windows?

Open a PowerShell Terminal as administrator.

Install Choco (chocolatey package manager)

Set-ExecutionPolicy Bypass -Scope Process -Force; [System.Net.ServicePointManager]::SecurityProtocol = [System.Net.ServicePointManager]::SecurityProtocol -bor 3072; iex ((New-Object System.Net.WebClient).DownloadString('https://community.chocolatey.org/install.ps1'))

FFMPEG ProRes HQ Encoding (Interlaced)

Premade commands to save time and suffering!

This list includes 2 command sets, passthrough witch will use whatever input resolution the source clip is, and affixed scaled output for use with standard playback equipment or standard definition mastering.

Rec 709 - HDTV (Passthough)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pixel_format yuv422p10lep -vendor apl0 -bits_per_mb 8000 -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -vf setdar=4/3,setfield=tff output.mov

bt470bg - PAL (Passthough)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt470bg -colorspace bt470bg -vf setdar=4/3,setfield=tff output.mov

smpte170m - NTSC (Passthough)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -vf setdar=4/3,setfield=tff output.mov

bt470bg PAL to Rec 709 HDTV (Passthough)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -vf scale=in_range=tv:out_range=tv:in_color_matrix=bt470bg:out_color_matrix=bt709 setdar=4/3,setfield=tff output.mov

smpte170m NTSC to Rec 709 HDTV (Passthough)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709 -vf scale=in_range=tv:out_range=tv:in_color_matrix=smpte170m:out_color_matrix=bt709,setdar=4/3,setfield=tff output.mov

Equipment SD Standard

bt470bg - PAL (720x576)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt470bg -colorspace bt470bg -vf setdar=4/3,setfield=tff,scale=720:576 output.mov

smpte170m - NTSC (720x480)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m -vf setdar=4/3,setfield=tff,scale=720:576 output.mov

bt470bg PAL to Rec 709 HDTV (720x576)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt709 -color_trc bt470bg -colorspace bt709 -vf scale=in_range=tv:out_range=tv:in_color_matrix=bt470bg:out_color_matrix=bt709:720x576,setdar=4/3,setfield=tff output.mov

smpte170m NTSC to Rec 709 HDTV (720x480)

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores -profile:v 3 -vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pix_fmt yuv422p10le -color_range tv -color_primaries bt709 -color_trc smpte170m -colorspace bt709 -vf scale=in_range=tv:out_range=tv:in_color_matrix=smpte170m:out_color_matrix=bt709:720x480,setdar=4/3,setfield=tff output.mov

ProRes HQ StaxRip De-intelacing

Use StaxRip or AviSynth Directly.

Source Filter: FFVideoSource

Field Filter: Deinterlace --> QTGMC --> Placebo

Then we edit it to Sharpness 0.3 (and you can manually override threads if you have a high core chip i.e 16-64 cores etc)

Click Edit Code:

QTGMC(preset="Placebo", InputType=0, sourceMatch=3, Lossless=2, sharpness=0.3, tr2=2, ediThreads=8)

Profile: ProRes

Container: QuickTime .mov / Material Exchange Format .mxf

Encoder Options:

-vendor apl0 -bits_per_mb 8000 -quant_mat hq -mbs_per_slice 8 -pixel_format yuv422p10lep -vendor apl0 -bits_per_mb 8000 -color_range tv -color_primaries bt709 -color_trc bt709 -colorspace bt709

Output Container can be the following

Generally with ProRes its recommended to stick to .mov

.mov (QuickTime)
.mxf (Material eXchange Format)
.mkv (Matroska)

Profiles can be the following

10-bit 4:2:2 Profiles

-profile:v 0 -pixel_format yuv422p10lep - ProRes Proxy

-profile:v 1 -pixel_format yuv422p10lep - ProRes LT

-profile:v 2 -pixel_format yuv422p10lep - ProRes 422 (Standard)

-profile:v 3 -pixel_format yuv422p10lep - ProRes HQ

10-bit 4:4:4 Profiles

-profile:v 4 -pix_fmt yuva444p10le ProRes 4444 (10-bit)

-profile:v 5 -pix_fmt yuva444p10le ProRes 4444QX (10-bit)

12-bit 4:4:4 Profiles

(Well when FFmpeg finally pushes a fix for ProRes 12-bit...)

-profile:v 4 -pix_fmt yuva444p12le ProRes 4444 (12-bit)

-profile:v 5 -pix_fmt yuva444p12le ProRes 4444QX (12-bit)

Flagging interlaced `field store method` properly.

FFmpeg supports interlacing flagging but is very clunky and command orders need to be right or it just ignores things, after 2 weeks this is what worked.

4:3 Media

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores Add your Profile and other arguments -vf setdar=4/3,setfield=tff output.mov

16:9 Media

ffmpeg -i input.mkv -vf setfield=tff -flags +ilme+ildct -c:v prores Add your Profile and other arguments here -vf setdar=16/9,setfield=tff output.mov

ProRes Specific Arguments

The FourCC codec IDs for the different Apple ProRes variants:

Apple ProRes 422 High Quality: ‘apch‘ (‘hcpa’ in little-endian)
Apple ProRes 422 Standard Definition: ‘apcn‘ (‘ncpa’ in little-endian)
Apple ProRes 422 LT: ‘apcs‘ (‘scpa’ in little-endian)
Apple ProRes 422 Proxy: ‘apco‘ (‘ocpa’ in little-endian)
Apple ProRes 4444: ‘ap4h’ (‘h4pa‘ in little-endian)

`-vf setdar=K:M`

Sets DAR or Aspect Ratio K:M can be 4:3 or 16:9

`-r <integer>`

Sets frame rate value, 24/25/30/29.97/50/59.97/60 etc

`-bits_per_mb <integer>`

Bitrate control defines how many bits to allot for coding one macroblock or MB. Different profiles use between 200 and 2400 bits per macroblock, the maximum is 8000.

The expected bitrate is capped by this math!

(frame_width/16) * (frame_height/16) * fps * bits_per_mb

`-mbs_per_slice <integer>`

Number of macroblocks in each slice (1-8); the default value (8) should be good in almost all situations.

`-vendor <string>`

Override the 4-byte vendor ID A custom vendor ID like the following:

apl0 Apple Encoder

atms ATMOS Recorders

adb0 Adobe Suite

fmpg FFmpeg

`-alpha_bits <integer>`

Specify number of bits for alpha component. Possible values are 0, 8 and 16. Use 0 to disable alpha plane coding.

`-quant_mat <integer>`

Selects quantization matrix. If this option is set to auto, the matrix matching the profile will be picked. If not set, the matrix providing the highest quality, default, will be picked.

auto / default / proxy / lt / standard / hq

`-codec:v prores_videotoolbox`

Apple Silicone Encoder, now in the 2020s you can encode compliant ProRes on M1 and newer ARM based macs with FFmpeg.

-codec:v prores_videotoolbox enables the hardware encoder

-profile:v hq etc

Clean Aperture & Colr Data Atom

This applies to anything using the QuickTime MOV Container, MP4, and MXF Material eXchange Format.

This has been the biggest pain in the ass during the development of this book then I had the solution handed to me by Wrapscallion (Ty) on the DD86 discord so all the credit in the world for sharing this tool thats practically lost to low google SEO scores.

AMCDX Video Patcher / Archive Download is a wonderful little tool that visually allows you to click your issues away with things like clean aperture or clap settings and full colour flagging.

Metadata-Editor

File-to-File-Editor

Frame-Editor

`qtff-parameter-editor` Tool Link

This tool provided by the BBC allows for the editing and correction of ProRes files in the .MOV container mainly the colr Atom allowing you to have correct colour parameters for your media type.

Several post-production tools and utilities are now aware of the colour and transfer function parameters specified in ITU-R BT.2100-0. However, some tools are unable to correctly signal the correct parameters, and may result in a file with the incorrect video parameters. Subsequent tools or displays may then look at these video parameters and render the image incorrectly, for instance, video that this signalled as ITU-R BT.709 colour primaries and in fact is ITU-R BT.2020 colour primary and then displayed on a monitor will look desaturated when the display is interpreting the signalling contained within the file. Incorrect signalling may also result in unnecessary and incorrect transcoding between colour spaces and transfer functions.

This document introduces a series of tools to allow editing of the colour primaries, colour matrix and transfer function characteristics in a QuickTime File Format (MOV) using a ProRes video codec.

QuickTime File Format (qtff)

The QuickTime File Format (qtff) is a container file supporting a wide range of video, audio and other data formats. The format itself is object-orientated, consisting of a collection of objects that can be parsed and expanded.

The basic data unit is known as an Atom. The Atom that defines the relevant information required to define the colour primaries, colour matrix and transfer function are found in the "colr" data Atom, which is located inside the Video Media Atom. The structure of the "colr" Atom is as follows:

Colour Atom	Bytes
Atom Size	4
Type = "colr"	4
Colour Parameter type = "nclc"	4
Primary index = 1	2
Transfer Function index = 1	2
Colour Matrix index = 1	2

SMPTE RDD 36:2015 - Apple ProRes Bitstream Syntax and Decoding Process

SMPTE RDD 36 describes the syntax and decoding process for the Apple ProRes video compression scheme. It is an intra-frame codec, specifically designed for high-quality workflows and supports a variety of video formats, and is common usage.

In addition to the colour information carried within the Color Atom, information regarding the transfer function, colour matrix and primaries are also stored within the frame header information of the ProRes elementary stream, alongside other parameters, such as frame rate, spatial resolution and chroma format. This header is repeated throughout the bitstream. Full details of the header layout can be found in the SMPTE specification.

To avoid any ambiguity in any workflows, it is imperative that the the information in the header for the ProRes stream match that of the qtff colr Atom.

Video Characteristics

The colour primaries can be selected from the list:

No.	Colour Primaries
0	Reserved
1	ITU-R BT.709
2	Unspecified
3	Reserved
4	ITU-R BT.470M
5	ITU-R BT.470BG
6	SMPTE 170M
7	SMPTE 240M
8	FILM
9	ITU-R BT.2020
10	SMPTE ST 428-1
11	DCI P3
12	P3 D65

The transfer function can be selected from the list:

No.	Transfer Function
0	Reserved
1	ITU-R BT.709
2	Unspecified
3	Reserved
4	Gamma 2.2 curve
5	Gamma 2.8 curve
6	SMPTE 170M
7	SMPTE 240M
8	Linear
9	Log
10	Log Sqrt
11	IEC 61966-2-4
12	ITU-R BT.1361 Extended Colour Gamut
13	IEC 61966-2-1
14	ITU-R BT.2020 10 bit
15	ITU-R BT.2020 12 bit
16	SMPTE ST 2084 (PQ)
17	SMPTE ST 428-1
18	ARIB STD-B67 (HLG)