Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Allow for lowering completeness threshold on ffmpeg adapter #114

Merged
merged 1 commit into from
Nov 9, 2022
Merged

Allow for lowering completeness threshold on ffmpeg adapter #114

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion lib/active_encode/engine_adapters/ffmpeg_adapter.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ class FfmpegAdapter
MEDIAINFO_PATH = ENV["MEDIAINFO_PATH"] || "mediainfo"
FFMPEG_PATH = ENV["FFMPEG_PATH"] || "ffmpeg"

class_attribute :completeness_threshold

def create(input_url, options = {})
# Decode file uris for ffmpeg (mediainfo works either way)
case input_url
Expand Down Expand Up @@ -110,7 +112,7 @@ def find(id, opts = {})
encode.state = :completed
elsif cancelled? encode.id
encode.state = :cancelled
elsif encode.percent_complete < 100
elsif encode.percent_complete < (completeness_threshold || 100)
encode.errors.prepend("Encoding has completed but the output duration is shorter than the input")
encode.state = :failed
end
Expand Down
Empty file added spec/fixtures/ffmpeg/incomplete-id/error.log
View file
Open in desktop
Empty file.
1 change: 1 addition & 0 deletions spec/fixtures/ffmpeg/incomplete-id/exit_status.code
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
0
102 changes: 102 additions & 0 deletions spec/fixtures/ffmpeg/incomplete-id/input_metadata
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
<?xml version="1.0" encoding="UTF-8"?>
<MediaInfo
xmlns="https://mediaarea.net/mediainfo"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://mediaarea.net/mediainfo https://mediaarea.net/mediainfo/mediainfo_2_0.xsd"
version="2.0">
<creatingLibrary version="18.05" url="https://mediaarea.net/MediaInfo">MediaInfoLib</creatingLibrary>
<media ref="/home/pdinh/Downloads/videoshort.mp4">
<track type="General">
<VideoCount>1</VideoCount>
<AudioCount>1</AudioCount>
<FileExtension>mp4</FileExtension>
<Format>MPEG-4</Format>
<Format_Profile>Base Media</Format_Profile>
<CodecID>mp42</CodecID>
<FileSize>199160</FileSize>
<Duration>6.315</Duration>
<OverallBitRate_Mode>VBR</OverallBitRate_Mode>
<OverallBitRate>252301</OverallBitRate>
<FrameRate>23.719</FrameRate>
<FrameCount>149</FrameCount>
<StreamSize>5679</StreamSize>
<HeaderSize>160</HeaderSize>
<DataSize>193489</DataSize>
<FooterSize>5511</FooterSize>
<IsStreamable>No</IsStreamable>
<Encoded_Date>UTC 2010-09-23 00:37:25</Encoded_Date>
<Tagged_Date>UTC 2010-09-23 00:37:27</Tagged_Date>
<File_Modified_Date>UTC 2017-12-14 19:29:35</File_Modified_Date>
<File_Modified_Date_Local>2017-12-14 14:29:35</File_Modified_Date_Local>
<Encoded_Application>HandBrake 0.9.4 2009112300</Encoded_Application>
</track>
<track type="Video">
<StreamOrder>0</StreamOrder>
<ID>1</ID>
<Format>AVC</Format>
<Format_Profile>Main</Format_Profile>
<Format_Level>1.1</Format_Level>
<Format_Settings_CABAC>Yes</Format_Settings_CABAC>
<Format_Settings_RefFrames>2</Format_Settings_RefFrames>
<CodecID>avc1</CodecID>
<Duration>6.282</Duration>
<BitRate>74477</BitRate>
<Width>200</Width>
<Height>110</Height>
<Stored_Width>208</Stored_Width>
<Stored_Height>112</Stored_Height>
<Sampled_Width>200</Sampled_Width>
<Sampled_Height>110</Sampled_Height>
<PixelAspectRatio>1.000</PixelAspectRatio>
<DisplayAspectRatio>1.818</DisplayAspectRatio>
<Rotation>0.000</Rotation>
<FrameRate_Mode>VFR</FrameRate_Mode>
<FrameRate>23.719</FrameRate>
<FrameRate_Minimum>12.500</FrameRate_Minimum>
<FrameRate_Maximum>24.390</FrameRate_Maximum>
<FrameRate_Original>24.000</FrameRate_Original>
<FrameCount>149</FrameCount>
<ColorSpace>YUV</ColorSpace>
<ChromaSubsampling>4:2:0</ChromaSubsampling>
<BitDepth>8</BitDepth>
<ScanType>Progressive</ScanType>
<StreamSize>58482</StreamSize>
<Encoded_Library>x264 - core 79 r1347 5ddd61b</Encoded_Library>
<Encoded_Library_Name>x264</Encoded_Library_Name>
<Encoded_Library_Version>core 79 r1347 5ddd61b</Encoded_Library_Version>
<Encoded_Library_Settings>cabac=1 / ref=2 / deblock=1:0:0 / analyse=0x1:0x111 / me=hex / subme=6 / psy=1 / psy_rd=1.0:0.0 / mixed_ref=0 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=0 / cqm=0 / deadzone=21,11 / chroma_qp_offset=-2 / threads=3 / nr=0 / decimate=1 / mbaff=0 / constrained_intra=0 / bframes=2 / b_pyramid=0 / b_adapt=1 / b_bias=0 / direct=1 / wpredb=0 / wpredp=2 / keyint=240 / keyint_min=24 / scenecut=40 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=25.5 / qcomp=0.60 / qpmin=10 / qpmax=51 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00</Encoded_Library_Settings>
<Encoded_Date>UTC 2010-09-23 00:37:25</Encoded_Date>
<Tagged_Date>UTC 2010-09-23 00:37:27</Tagged_Date>
<colour_range>Limited</colour_range>
<colour_description_present>Yes</colour_description_present>
<colour_primaries>BT.601 NTSC</colour_primaries>
<transfer_characteristics>BT.709</transfer_characteristics>
<matrix_coefficients>BT.601</matrix_coefficients>
</track>
<track type="Audio">
<StreamOrder>1</StreamOrder>
<ID>2</ID>
<Format>AAC</Format>
<Format_Profile>LC</Format_Profile>
<CodecID>mp4a-40-2</CodecID>
<Duration>6.315</Duration>
<BitRate_Mode>VBR</BitRate_Mode>
<BitRate>171030</BitRate>
<BitRate_Maximum>201736</BitRate_Maximum>
<Channels>1</Channels>
<ChannelPositions>Front: C</ChannelPositions>
<ChannelLayout>C</ChannelLayout>
<SamplesPerFrame>1024</SamplesPerFrame>
<SamplingRate>48000</SamplingRate>
<SamplingCount>303120</SamplingCount>
<FrameRate>46.875</FrameRate>
<FrameCount>296</FrameCount>
<Compression_Mode>Lossy</Compression_Mode>
<StreamSize>134999</StreamSize>
<StreamSize_Proportion>0.67784</StreamSize_Proportion>
<Title>Stereo</Title>
<Encoded_Date>UTC 2010-09-23 00:37:25</Encoded_Date>
<Tagged_Date>UTC 2010-09-23 00:37:27</Tagged_Date>
</track>
</media>
</MediaInfo>
90 changes: 90 additions & 0 deletions spec/fixtures/ffmpeg/incomplete-id/output_metadata-high
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<?xml version="1.0" encoding="UTF-8"?>
<MediaInfo
xmlns="https://mediaarea.net/mediainfo"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://mediaarea.net/mediainfo https://mediaarea.net/mediainfo/mediainfo_2_0.xsd"
version="2.0">
<creatingLibrary version="18.05" url="https://mediaarea.net/MediaInfo">MediaInfoLib</creatingLibrary>
<media ref="spec/fixtures/ffmpeg/completed-id/low.mp4">
<track type="General">
<VideoCount>1</VideoCount>
<AudioCount>1</AudioCount>
<FileExtension>mp4</FileExtension>
<Format>MPEG-4</Format>
<Format_Profile>Base Media</Format_Profile>
<CodecID>isom</CodecID>
<FileSize>125403</FileSize>
<Duration>6.336</Duration>
<OverallBitRate>158337</OverallBitRate>
<FrameRate>24.000</FrameRate>
<FrameCount>150</FrameCount>
<StreamSize>5648</StreamSize>
<HeaderSize>40</HeaderSize>
<DataSize>119763</DataSize>
<FooterSize>5600</FooterSize>
<IsStreamable>No</IsStreamable>
<File_Modified_Date>UTC 2018-09-07 17:36:26</File_Modified_Date>
<File_Modified_Date_Local>2018-09-07 13:36:26</File_Modified_Date_Local>
<Encoded_Application>Lavf58.12.100</Encoded_Application>
</track>
<track type="Video">
<StreamOrder>0</StreamOrder>
<ID>1</ID>
<Format>AVC</Format>
<Format_Profile>High</Format_Profile>
<Format_Level>1.1</Format_Level>
<Format_Settings_CABAC>Yes</Format_Settings_CABAC>
<Format_Settings_RefFrames>4</Format_Settings_RefFrames>
<CodecID>avc1</CodecID>
<Duration>6.250</Duration>
<BitRate>79302</BitRate>
<Width>200</Width>
<Height>110</Height>
<Stored_Width>208</Stored_Width>
<Stored_Height>112</Stored_Height>
<Sampled_Width>200</Sampled_Width>
<Sampled_Height>110</Sampled_Height>
<PixelAspectRatio>1.000</PixelAspectRatio>
<DisplayAspectRatio>1.818</DisplayAspectRatio>
<Rotation>0.000</Rotation>
<FrameRate_Mode>CFR</FrameRate_Mode>
<FrameRate_Mode_Original>VFR</FrameRate_Mode_Original>
<FrameRate>24.000</FrameRate>
<FrameCount>150</FrameCount>
<ColorSpace>YUV</ColorSpace>
<ChromaSubsampling>4:2:0</ChromaSubsampling>
<BitDepth>8</BitDepth>
<ScanType>Progressive</ScanType>
<StreamSize>61955</StreamSize>
<Encoded_Library>x264 - core 152 r2854 e9a5903</Encoded_Library>
<Encoded_Library_Name>x264</Encoded_Library_Name>
<Encoded_Library_Version>core 152 r2854 e9a5903</Encoded_Library_Version>
<Encoded_Library_Settings>cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00</Encoded_Library_Settings>
</track>
<track type="Audio">
<StreamOrder>1</StreamOrder>
<ID>2</ID>
<Format>AAC</Format>
<Format_Profile>LC</Format_Profile>
<Format_Settings_SBR>No (Explicit)</Format_Settings_SBR>
<CodecID>mp4a-40-2</CodecID>
<Duration>6.336</Duration>
<BitRate_Mode>CBR</BitRate_Mode>
<BitRate>72000</BitRate>
<Channels>2</Channels>
<Channels_Original>1</Channels_Original>
<ChannelPositions>Front: C</ChannelPositions>
<ChannelLayout>C</ChannelLayout>
<SamplesPerFrame>1024</SamplesPerFrame>
<SamplingRate>48000</SamplingRate>
<SamplingCount>304128</SamplingCount>
<FrameRate>46.875</FrameRate>
<FrameCount>297</FrameCount>
<Compression_Mode>Lossy</Compression_Mode>
<StreamSize>57800</StreamSize>
<StreamSize_Proportion>0.46091</StreamSize_Proportion>
<Default>Yes</Default>
<AlternateGroup>1</AlternateGroup>
</track>
</media>
</MediaInfo>
90 changes: 90 additions & 0 deletions spec/fixtures/ffmpeg/incomplete-id/output_metadata-low
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
<?xml version="1.0" encoding="UTF-8"?>
<MediaInfo
xmlns="https://mediaarea.net/mediainfo"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://mediaarea.net/mediainfo https://mediaarea.net/mediainfo/mediainfo_2_0.xsd"
version="2.0">
<creatingLibrary version="18.05" url="https://mediaarea.net/MediaInfo">MediaInfoLib</creatingLibrary>
<media ref="spec/fixtures/ffmpeg/completed-id/low.mp4">
<track type="General">
<VideoCount>1</VideoCount>
<AudioCount>1</AudioCount>
<FileExtension>mp4</FileExtension>
<Format>MPEG-4</Format>
<Format_Profile>Base Media</Format_Profile>
<CodecID>isom</CodecID>
<FileSize>125403</FileSize>
<Duration>6.336</Duration>
<OverallBitRate>158337</OverallBitRate>
<FrameRate>24.000</FrameRate>
<FrameCount>150</FrameCount>
<StreamSize>5648</StreamSize>
<HeaderSize>40</HeaderSize>
<DataSize>119763</DataSize>
<FooterSize>5600</FooterSize>
<IsStreamable>No</IsStreamable>
<File_Modified_Date>UTC 2018-09-07 17:36:26</File_Modified_Date>
<File_Modified_Date_Local>2018-09-07 13:36:26</File_Modified_Date_Local>
<Encoded_Application>Lavf58.12.100</Encoded_Application>
</track>
<track type="Video">
<StreamOrder>0</StreamOrder>
<ID>1</ID>
<Format>AVC</Format>
<Format_Profile>High</Format_Profile>
<Format_Level>1.1</Format_Level>
<Format_Settings_CABAC>Yes</Format_Settings_CABAC>
<Format_Settings_RefFrames>4</Format_Settings_RefFrames>
<CodecID>avc1</CodecID>
<Duration>6.250</Duration>
<BitRate>79302</BitRate>
<Width>200</Width>
<Height>110</Height>
<Stored_Width>208</Stored_Width>
<Stored_Height>112</Stored_Height>
<Sampled_Width>200</Sampled_Width>
<Sampled_Height>110</Sampled_Height>
<PixelAspectRatio>1.000</PixelAspectRatio>
<DisplayAspectRatio>1.818</DisplayAspectRatio>
<Rotation>0.000</Rotation>
<FrameRate_Mode>CFR</FrameRate_Mode>
<FrameRate_Mode_Original>VFR</FrameRate_Mode_Original>
<FrameRate>24.000</FrameRate>
<FrameCount>150</FrameCount>
<ColorSpace>YUV</ColorSpace>
<ChromaSubsampling>4:2:0</ChromaSubsampling>
<BitDepth>8</BitDepth>
<ScanType>Progressive</ScanType>
<StreamSize>61955</StreamSize>
<Encoded_Library>x264 - core 152 r2854 e9a5903</Encoded_Library>
<Encoded_Library_Name>x264</Encoded_Library_Name>
<Encoded_Library_Version>core 152 r2854 e9a5903</Encoded_Library_Version>
<Encoded_Library_Settings>cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x113 / me=hex / subme=7 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=1 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-2 / threads=3 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=1 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=24 / scenecut=40 / intra_refresh=0 / rc_lookahead=40 / rc=crf / mbtree=1 / crf=23.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00</Encoded_Library_Settings>
</track>
<track type="Audio">
<StreamOrder>1</StreamOrder>
<ID>2</ID>
<Format>AAC</Format>
<Format_Profile>LC</Format_Profile>
<Format_Settings_SBR>No (Explicit)</Format_Settings_SBR>
<CodecID>mp4a-40-2</CodecID>
<Duration>6.336</Duration>
<BitRate_Mode>CBR</BitRate_Mode>
<BitRate>72000</BitRate>
<Channels>2</Channels>
<Channels_Original>1</Channels_Original>
<ChannelPositions>Front: C</ChannelPositions>
<ChannelLayout>C</ChannelLayout>
<SamplesPerFrame>1024</SamplesPerFrame>
<SamplingRate>48000</SamplingRate>
<SamplingCount>304128</SamplingCount>
<FrameRate>46.875</FrameRate>
<FrameCount>297</FrameCount>
<Compression_Mode>Lossy</Compression_Mode>
<StreamSize>57800</StreamSize>
<StreamSize_Proportion>0.46091</StreamSize_Proportion>
<Default>Yes</Default>
<AlternateGroup>1</AlternateGroup>
</track>
</media>
</MediaInfo>
1 change: 1 addition & 0 deletions spec/fixtures/ffmpeg/incomplete-id/pid
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
99999
11 changes: 11 additions & 0 deletions spec/fixtures/ffmpeg/incomplete-id/progress
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
frame=150
fps=0.0
stream_0_0_q=-1.0
bitrate= 158.9kbits/s
total_size=125403
out_time_ms=6000000
out_time=00:00:06.000000
dup_frames=1
drop_frames=0
speed=43.1x
progress=end
Binary file added spec/fixtures/ffmpeg/incomplete-id/video-high.mp4
View file
Open in desktop
Binary file not shown.
Binary file added spec/fixtures/ffmpeg/incomplete-id/video-low.mp4
View file
Open in desktop
Binary file not shown.
16 changes: 16 additions & 0 deletions spec/integration/ffmpeg_adapter_spec.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
end
let(:completed_job) { find_encode "completed-id" }
let(:completed_with_warnings_job) { find_encode "completed-with-warnings-id" }
let(:incomplete_job) { find_encode "incomplete-id" }
let(:failed_job) { find_encode 'failed-id' }
let(:completed_tech_metadata) do
{
Expand Down Expand Up @@ -212,6 +213,21 @@ def find_encode(id)
expect(File).to exist("#{work_dir}/#{subject.id}/exit_status.code")
expect(File.read("#{work_dir}/#{subject.id}/exit_status.code").to_i).to eq(-22)
end

context 'with less than 100 percent completeness' do
subject { incomplete_job }

it { is_expected.to be_failed }
it 'has an error' do
expect(incomplete_job.errors).to include "Encoding has completed but the output duration is shorter than the input"
end

it 'succeeds with a configured completeness threshold' do
allow(ActiveEncode::EngineAdapters::FfmpegAdapter).to receive(:completeness_threshold).and_return(95)
expect(incomplete_job).not_to be_failed
expect(incomplete_job.errors).to be_empty
end
end
end
end

Expand Down