FS#53698 - [ffmpeg] 3.3 reliably muxes corrupted files

Attached to Project: Arch Linux
Opened by c (c) - Monday, 17 April 2017, 18:39 GMT
Last edited by Maxime Gauduin (Alucryd) - Friday, 19 May 2017, 12:27 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Maxime Gauduin (Alucryd)
Architecture All
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 0
Private No


ffmpeg muxing corrupts file

Additional info:
* 1:3.3-1

* I've build ffmpeg n2.8.11 and that doesn't show the regression.

Steps to reproduce:
ffmpeg -i original.mp4 -c:a libopus -c:v copy new.mkv
ffmpeg -i original.mp4 -c:a aac -c:v copy new.mkv
or anything that remuxes for that matter.

Play original.mp4 and new.mkv to see the corruption. When not using hw-decoding (vaapi) some of the corruption is papered over and invisible but that just means the software decoders is more forgiving towards the corrupted stream.
This task depends upon

Closed by  Maxime Gauduin (Alucryd)
Friday, 19 May 2017, 12:27 GMT
Reason for closing:  Fixed
Additional comments about closing:  3.3.1
Comment by c (c) - Monday, 17 April 2017, 18:45 GMT
I'm lucky I noticed this today or else I'd have to redo many more files.
Comment by c (c) - Monday, 17 April 2017, 19:27 GMT
Forgot to set severity but I don't consider this Low. It's critical because it doesn't crash but actually produces corrupt output.
Comment by c (c) - Monday, 17 April 2017, 23:10 GMT
Errors you might see when playing a corruptedly muxed by ffmpeg 3.3 file:

[ffmpeg/video] h264: co located POCs unavailable
WARNING: Invalid RefPicListX[] entry!!! It is not included in DPB

The same file muxed by ffmpeg 2.8.11 gives no such error and doesn't start off with visible artifacts.

It doesn't have to be mp4 -> mkv. Muxing mp4 to mp4 and just recoding the audio track will have the same effect.
Comment by c (c) - Monday, 24 April 2017, 22:13 GMT Comment by Maxime Gauduin (Alucryd) - Tuesday, 25 April 2017, 18:14 GMT
Thanks, I was able to reproduce it as well with a few files, but didn't have time to look into it further. I'll stalk upstream for a fix.
Comment by c (c) - Wednesday, 26 April 2017, 16:45 GMT
Cool. Can you please force 3.2.4 or 2.8.11 in place of 3.3? The corruption is irreversible and any file corruption is unmistakably a grave regression from my perspective. I've already noticed streaming sites (not youtube) that serve such corrupted files.
Comment by Maxime Gauduin (Alucryd) - Wednesday, 26 April 2017, 17:08 GMT
I won't downgrade unless upstream takes too long, in the meantime please build 3.2.4 and add ffmpeg to the IgnorePkg entry in pacman.conf. Note that I'll be upgrading x265 in a few hours, which contains a soname bump.
Comment by c (c) - Wednesday, 26 April 2017, 18:21 GMT
I understand your point and don't know what process arch packages maintainers usually follow in such a situation. However please allow me to make a last appeal before I stop mentioning it.

Corruption of files is a severe regression and even more so since recoding media files is heavy on cpu time and maybe impossible because the source material is not available. Since there's a simple fix in the form of promoting 3.2.4/2.8.11 in arch stable index until upstream releases a fix, I'm unable to understand why users' files should knowingly be corrupted anytime they remux while changing the audio track (volume adjustments, codec changes, filters etc.).

If arch testing was ffmpeg 3.3 and it didn't land in stable yet, I wouldn't ask, but I think silent corruption is the worst bug. If it crashed, users would notice something is up.

Admittedly Arch Linux is not an LTS distro, but we have a fix for a serious regression, and waiting for upstream which has been aware of the bug long before 3.3 has been taged, if you read the duplicate upstream tickets, doesn't seem like the logical decision. The earliest bug report is https://trac.ffmpeg.org/ticket/6227 (7 weeks ago) (which I didn't find before submitting my ticket since I didn't search for dropped packets).

To be clear, I'm not mad about this and I would just like to protect the rest of the arch users.

At the very least, I would appreciate a news post on arch announce or the homepage, letting users know that they should install 2.8.11 from archive.archlinux.org and pacman ignore ffmpeg in the meantime. This could work since arch users can be expected to follow.
Comment by Maxime Gauduin (Alucryd) - Wednesday, 26 April 2017, 19:16 GMT
If that issue affected a lot of people I'm sure it would have had a higher priority, an annoucement for such a thing seems like overkill to me. If you wish to speed up its resolution you can start by upvoting the upstream bug report to get the devs' attention (I already did it), once a patch is available I'll backport it in our ffmpeg package.

FYI, I just pushed a new ffmpeg2.8 to [testing], this one includes binaries which you can use instead of the 3.3 ones.
Comment by c (c) - Thursday, 27 April 2017, 13:13 GMT
I disagree on the severity of the issue and stand by my point that any silent corruption of files is the worst bug and demands resolving especially if there is a clear remedy available. But as promised I won't be asking for you to downgrade and I do accept your decision. I solved the problem for myself and hope not many other users are affected by it.

Please, if you can spare a moment, I'd like to know the implications of a downgrade which make it a hard decision to take. It's possible I don't know the problems (of a downgrade) and have been suggesting it too quickly.
Comment by DrZaius (DrZaius) - Sunday, 14 May 2017, 18:36 GMT
This was fixed 12 days ago upstream. Will be included in the 3.3.1 point update which will likely be released within a day or two.