FS#45186 - [ffmpeg] Please include libsoxr support in build

Attached to Project: Arch Linux
Opened by Luis Quiles (lquiles) - Tuesday, 02 June 2015, 23:36 GMT
Last edited by Maxime Gauduin (Alucryd) - Tuesday, 28 July 2015, 22:20 GMT
Task Type Feature Request
Category Packages: Extra
Status Closed
Assigned To Ionut Biru (wonder)
Bartłomiej Piotrowski (Barthalion)
Maxime Gauduin (Alucryd)
Architecture All
Severity Low
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:

FFmpeg's own resampler (SWR) is of notoriously poor quality. It can use SoX resampler (soxr), though. Please add --enable-libsoxr to PKGBUILD.
This task depends upon

Closed by  Maxime Gauduin (Alucryd)
Tuesday, 28 July 2015, 22:20 GMT
Reason for closing:  Fixed
Additional comments about closing:  1:2.7.2-2
Comment by Bartłomiej Piotrowski (Barthalion) - Saturday, 06 June 2015, 18:50 GMT
Soxr development has stalled. I'd like to avoid introducing dead packages as new dependencies to ffmpeg. Please report your quality issues upstream.
Comment by Luis Quiles (lquiles) - Sunday, 07 June 2015, 02:11 GMT
  • Field changed: Percent Complete (100% → 0%)
There is nothing new that libsoxr is supposed to do. Its development has not stalled. It has matured.

It does its job perfectly fine as it is. Audacity, for example, uses it internally.

It was complaints about quality issues of swr in the first place that got FFmpeg developers to include soxr support.
Comment by Bartłomiej Piotrowski (Barthalion) - Sunday, 07 June 2015, 08:30 GMT
Let this request mature as well and see how many votes can you get.
Comment by Luis Quiles (lquiles) - Sunday, 07 June 2015, 13:15 GMT
Apologies for the repeated re-open requests. It seems refreshing the browser tab to see if things had changed had that effect and I was unaware of it. It was not my intent.
Comment by James Almer (jamrial) - Sunday, 21 June 2015, 02:21 GMT
Luis, could you report your quality concern/findings to http://trac.ffmpeg.org/? Swr is developed and maintained by the leader of the project, who's always open to suggestions about how to improve certain parts of the code.
Explaining what needs work (resampling? mixing? format conversion?) and some comparisons of the output from other resamplers like Soxr would greatly help improve the library.
Comment by Bartłomiej Piotrowski (Barthalion) - Sunday, 21 June 2015, 08:46 GMT
I have exchanged few emails with Carl Hoyos and Michael Niedermayer and this is definitely way to go. As Michael stated, libsoxr support was added because "someone has sent the patch". Please report your concerns upstream.
Comment by Luis Quiles (lquiles) - Saturday, 18 July 2015, 14:58 GMT
  • Field changed: Percent Complete (100% → 0%)
Read here: https://trac.ffmpeg.org/wiki/FFmpeg%20and%20the%20SoX%20Resampler

There is no upstream issue. No one upstream is making a fuss about having libsoxr support in FFmpeg or contesting its better quality compared to the computationally cheaper swr. Should FFmpeg developers be interested in improving swr they have at least two different open-source resamplers of proven higher quality (src, soxr) at their disposal to consider.

FFmpeg currently supports two resamplers: swr and soxr. One is internal, the other external. Both are equally relevant to FFmpeg's functionality. One does not come at the expense of the other.

I requested the addition of --enable-libsoxr flag so that the higher quality resampler which FFmpeg documentation clearly refers to, see ffmpeg-resampler(1), would be available in Arch's default FFmpeg build. That is all. Otherwise, most self-respecting users of FFmpeg can just go ahead and do a local build all the same with this switch added.

What exactly is "the way to go," Barthalion? If you don't wish to add this flag for any odd reason you are of course the sole decision maker on this package and can do so. It must be understood, however, that this is not an upstream issue in any way, shape, or form.
Comment by Maxime Gauduin (Alucryd) - Tuesday, 21 July 2015, 14:09 GMT
I personally have no qualms about enabling libsoxr in ffmpeg, the more options the better, as long as it doesn't introduce any issue. I am curious though, is soxr being superior to swr still true? The last tests we could find are years old and surely ffmpeg has improved since then. TBH I can't hear any difference between the two, then again my ears are not the keenest. Have you got any recent source to support your claim?
Comment by Luis Quiles (lquiles) - Tuesday, 21 July 2015, 20:52 GMT
Libswresample is the part of FFmpeg responsible for abstracting internal and external resamplers. The resource (src.infinitewave.ca) I've linked earlier, also linked from Audacity's resampler wiki page, are for FFmpeg 1.1.1 released on 2013-01-20. The resource itself is regularly updated for all its competing entries; latest on July 1, 2015. The reason the results for FFmpeg are not updated to newer versions is visible in libswresample development history: https://github.com/FFmpeg/FFmpeg/commits/master/libswresample/resample.c

There have been no core changes to swr since late 2012 and addition of libsoxr support. The algorithm has not changed, of course, and changes affecting quality have not been added. There are bug fixes and peformance optimizations such as SSE and AVX support. The mathematics of transformations applied does not change either. FFmpeg's swr is made to be fast and computationally cheap which is important to playback software, like mpv, that rely on FFmpeg. While libsoxr (derived from SoX) aims for high quality conversion; at a computational cost. The importance of being fast and cheap has rapidly declined. Even a low-end recent machine can comfortably use libsoxr conversion in playback, let alone transcoding.

Anecdote: In my personal A/V playback setup I use upsampling (for most audio which comes in 44.1 kHz or 48 kHz) to 96 kHz because I have a D/A converter that's happier with 96 kHz. It outputs to a pair of studio monitors. I use mpv to play multimedia. Prior to learning of libsoxr support in FFmpeg and enabling it in a local build I heard occasional transient clicks generated in the audio output. I assumed it was a source defect or I/O latency of transport but disabling upsampling at the price of the D/A converter not being as happy revealed the source was clean. I could probably have decided that based on unrepeatability of the clicks that it was not a source problem. I still had to rule out latency issues. It turned out simply playing back the media at its native sample rate resolved the click issue. So I looked further into mpv resampling and traced the problem to FFmpeg and its default resampler. Doing the local build and passing the right lavfi bridge parameters to mpv to serve to FFmpeg resolved the click issue in upsampling. I had different quality and performance issues with MPD's internal resampler that were also resolved by using its support for libsoxr. I added a request for inclusion of libsoxr support in Arch's MPD build and the maintainer kindly agreed to it. No complaints so far. (Plus, others and I are spared remembering to do a local build at every update.)
Comment by James Almer (jamrial) - Tuesday, 21 July 2015, 21:25 GMT
You can see what commits changed the quality of audio output as tested by "FFmpeg Automated Testing Environment" here https://git.videolan.org/?p=ffmpeg.git;a=history;f=tests/fate/libswresample.mak;h=HEAD
There have been a some changes since 2012 that affected the quality of the audio generated by the library, and probably more than those listed there since the tests were only added in late 2013.

I'm not against the inclusion of libsoxr for that matter. It wouldn't affect the default behavior of using swr's internal resampling unless explicitly requested, and since it's now been moved to the extra repository it's certainly guaranteed to be maintained. The more options available the better.
But as i said before if you have reproducible examples where the swr's quality is below other resampling libraries you should either open a ticket in ffmpeg's trac, send an email to the ffmpeg-devel list or directly to Michael Niedermayer (the maintainer). He's always open to suggestions and feedback and countless times fixed and improved code this way.
Comment by Luis Quiles (lquiles) - Tuesday, 21 July 2015, 21:41 GMT
That's a useful resource. Thanks. I looked at the list, however, and there're clearly no core changes. What's listed is fixes to the implementation of the existing algorithm. I have also reviewed resample.c's change history since 2012 on FFmpeg's GitHub repo. There are only bug fixes and performance improvements as far as I can tell.

I agree it would have been best if I had examples but I was motivated to open this task due to the problem I mentioned in my earlier comment and then looking into quality of resamplers. The problem was nuisome. I had nothing but an elusive hope that changing to soxr would fix it. Thankfully, it did. Even the resource that compares qualities does not look for random clicks in the output audible on high quality speakers. They're not supposed to be there regardless of how computationally cheap the resampler is, as long as it is sound in theory and implementation. It is almost certainly a bug and not a characteristic of swr. On hand, I have nothing but that anecdote. It is possible if some FFmpeg people have a similar audio setup they may be able to observe similar degradation.
Comment by Maxime Gauduin (Alucryd) - Wednesday, 22 July 2015, 08:39 GMT
Alright then, I'll include libsoxr sometime later this week. Thx for the input guys.

Loading...