FS#19211 - [x264] Regression (bitrate option doesn't seem to apply)

Attached to Project: Arch Linux
Opened by Karthik Periagaram (karthikp) - Thursday, 22 April 2010, 06:35 GMT
Last edited by Ionut Biru (wonder) - Wednesday, 26 May 2010, 14:06 GMT
Task Type Bug Report
Category Packages: Extra
Status Closed
Assigned To Ionut Biru (wonder)
Architecture x86_64
Severity Medium
Priority Normal
Reported Version
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 1
Private No

Details

Description:

About April 11, the x264 package was updated from 20100312-1 -> 20100410-1. Since this update, there seems to be a problem encoding video using the following command I use. This is the first pass of a two-pass process:

mencoder movie.vob -vf pullup,softskip,crop=$CROP,harddup -oac copy -ovc x264 -x264encopts bitrate=2400:subq=4:bframes=3:b_pyramid=normal:weight_b:turbo=1:threads=auto:pass=1 -of rawvideo -o /dev/null -msglevel all=0:statusline=5 || exit 1

I'd like to draw your attention to the "bitrate=2400". When I use the older 20100312-1 package, the status line from mencoder will look something like this:

Pos:1234.5s 12345f (12%) 27.65 fps Trem: 30min 1800mb A-V: 0.018 [2452:448]

The numbers at the tail end of this line represent the bitrate of the video and audio stream. As you can see, the bitrate for the video stream matches the bitrate I selected in the command. In addition, "Trem: 30" min indicates that about 30 minutes remain till the end of the encoding and the "1800mb" represents its guess of the final file size.

Now, with the newer 20100410-1 package, here's what I see:

x264 [warning]: non-strictly-monotonic PTS 11min 22mb A-V:-0.015 [29:448]

The "x264 warning" is obscuring a portion of the status line. The things to note are

* The "Trem" value is much smaller than I usually see for this step.
* The estimated file size is very small. I don't expect a movie to be encoded to a 22 MB file!
* The video bitrate is being shown as 29 kbps. This is inspite of specifying a target bitrate of 2400 kbps.

After this, the second pass fails with the error message:
x264 [error]: requested bitrate is too low. estimated minimum is 8 kbps

Now, I dug deeper.

The first pass produces a divx2pass.log file that contains data for the second pass. The first line of this log file lists the options the encoder is using. Here's the older package's options line (this is the package that seems to work):

#options: 720x352 fps=30000/1001 timebase=1001/30000 cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x13 me=hex subme=2 psy=1 psy_rd=0.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 sliced_threads=0 nr=0 decimate=1 interlaced=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2400 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00

Now, here's the options line from the same vob file using the new x264 package:

#options: 720x352 fps=30000/1001 cabac=1 ref=1 deblock=1:0:0 analyse=0x3:0x13 me=hex subme=2 psy=1 psy_rd=0.00:0.00 mixed_ref=0 me_range=16 chroma_me=1 trellis=0 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=0 threads=12 sliced_threads=0 nr=0 decimate=1 mbaff=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 wpredb=1 wpredp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=abr mbtree=1 bitrate=2400 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=51 qpstep=4 ip_ratio=1.40 aq=1:1.00

Things to note:
Both option lines are almost identical. The bitrate option is clearly passed to the encoder and set at 2400. The only difference is that the older package has a "timebase=1001/30000" option which is missing from the new package. This is probably either the cause or at least indicative of the cause of this problem.

For now, the fix is to downgrade the following packages:
mplayer(31040-1 -> 30886-1)
ffmpeg (22837-2 -> 22511-1)
x264 (20100410-1 -> 20100312-1)

in that order. This isn't a perfect fix, however. I notice that this older package doesn't seem to use all the cores of my processor for encoding and ends up taking longer to encode. But it results in a properly encoded file, so it's a workaround for now.

I hope this info helps in tracking the bug down. It's possible that this is an upstream bug in x264, but I couldn't find any references to this on the x264-devel mailing list (I assume somebody would have noticed this...) - so this could just be a bug in the Arch packages.

Additional info:
* package version(s)
x264-20100410-1

* config and/or log files etc.
-none-

Steps to reproduce:
1. Use the following commands to dump any DVD to a vob file and encode the video with x264 (uses mplayer/mencoder). In what follows, set the DVD, TRACK and CROP variables - usually, to "dvd", "1" and "" (or "720:480:0:0") respectively.

# Dump track to vob
mplayer -dvd-device /dev/$DVD dvd://$TRACK -dumpstream -dumpfile movie.vob -msglevel all=0
# Encode video (first pass)
mencoder movie.vob -vf pullup,softskip,crop=$CROP,harddup -oac copy -ovc x264 -x264encopts bitrate=2400:subq=4:bframes=3:b_pyramid=normal:weight_b:turbo=1:threads=auto:pass=1 -of rawvideo -o /dev/null -msglevel all=0:statusline=5
# Encode video (second pass)
mencoder movie.vob -vf pullup,softskip,crop=$CROP,harddup -oac copy -ovc x264 -x264encopts bitrate=2400:subq=5:8x8dct:frameref=2:bframes=3:b_pyramid=normal:weight_b:threads=auto:pass=2 -of rawvideo -o video.264 -msglevel all=0:statusline=5

2. You should be an abnormally small bitrate and estimated file size for the first pass and the second pass will fail with an error message from x264.
This task depends upon

Closed by  Ionut Biru (wonder)
Wednesday, 26 May 2010, 14:06 GMT
Reason for closing:  Fixed
Comment by Ionut Biru (wonder) - Thursday, 22 April 2010, 09:25 GMT
nice bug report, but please inform upstream.

before doing that please upstream x264 to the latest snapshot, mplayer to the latest revision(do svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer and tar xJf mplayer-revision.tar.xz) and compile it against the newest x264.
Comment by Ionut Biru (wonder) - Thursday, 22 April 2010, 09:36 GMT Comment by Ionut Biru (wonder) - Friday, 23 April 2010, 12:15 GMT
r31049 | lorenm | 2010-04-20 12:14:54 +0300 (Tue, 20 Apr 2010) | 3 lines

Tell x264 that we aren't going to give it timestamps.
Fixes some warnings starting in x264-r1480.

update fully and please try: http://dev.archlinux.org/~ibiru/mplayer-31059-1-x86_64.pkg.tar.xz
Comment by Karthik Periagaram (karthikp) - Tuesday, 27 April 2010, 06:56 GMT
Sorry for the delay, I was waiting to get my hands on a DVD to try this out again. Thanks for the references. It appears that both projects (x264 and mplayer) are aware of this.

Status:
So far, no luck. Here's what I did.

1. First, I updated mplayer using your package and tried. No luck, the video bitrate was really low. However, there were no error messages from x264.

2. So, I built the x264-mt-git and ffmpeg-mt-git packages from AUR. I tried again, but mplayer-31059 was linked against the older libx264.so, and gave an error regarding that. The error is:

mencoder: error while loading shared libraries: libx264.so.92: cannot open shared object file: No such file or directory

(libx264.so.94 is the new file.)

3. So I built mplayer-mt-git as well. With these packages (recent as of April 27), I tried and still got the small bitrates.

4. The mplayer-mt-git package isn't from the official svn repo from mplayer, so I figured it may be out dated. So, I tried building the mplayer-svn package from AUR, but I got a make error.

The make error is at this point:

cc -MD -MP -Wstrict-prototypes -Wmissing-prototypes -Wundef -Wdisabled-optimization -Wno-pointer-sign -Wdeclaration-after-statement -std=gnu99 -Wall -Wno-switch -Wpointer-arith -Wredundant-decls -O4 -march=native -mtune=native -pipe -ffast-math -fomit-frame-pointer -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -D__STDC_LIMIT_MACROS -Ilibdvdread4 -I. -D_REENTRANT -I/usr/include/ -I/usr/include/freetype2 -I/usr/lib/live/liveMedia/include -I/usr/lib/live/UsageEnvironment/include -I/usr/lib/live/BasicUsageEnvironment/include -I/usr/lib/live/groupsock/include -c -o libmpdemux/demux_rtp.o libmpdemux/demux_rtp.cpp
cc1plus: warning: command line option "-Wstrict-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wmissing-prototypes" is valid for Ada/C/ObjC but not for C++
cc1plus: warning: command line option "-Wno-pointer-sign" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-Wdeclaration-after-statement" is valid for C/ObjC but not for C++
cc1plus: warning: command line option "-std=gnu99" is valid for C/ObjC but not for C++
In file included from libmpdemux/aviheader.h:25:0,
from libmpdemux/stheader.h:23,
from libmpdemux/demux_rtp.cpp:28:
./libavutil/common.h: In function 'int32_t av_clipl_int32(int64_t)':
./libavutil/common.h:154:47: error: 'UINT64_C' was not declared in this scope
libmpdemux/demux_rtp.cpp: At global scope:
libmpdemux/demux_rtp.cpp:89:1: warning: 'typedef' was ignored in this declaration
make: *** [libmpdemux/demux_rtp.o] Error 1
==> ERROR: Build Failed.
Aborting...

[I also wanted to add that my downgrading solution works perfectly. I previously reported that when I downgraded, mencoder didn't use all the cores of my CPU. That was incorrect. When I rebooted, it worked perfectly fine.]

Any thoughts on how I should proceed?
Comment by Ionut Biru (wonder) - Tuesday, 27 April 2010, 09:35 GMT
report on mplayer. here is the arhive of april month. i see a threa about x264 and 2pass encoding.

http://lists.mplayerhq.hu/pipermail/mencoder-users/2010-April/thread.html

i really apreciate if you open a discussion there and point the thread to me to follow it and maybe i can reply if is needed.
Comment by Ionut Biru (wonder) - Wednesday, 26 May 2010, 10:33 GMT
still valid with the latest x264? 20100524
Comment by Karthik Periagaram (karthikp) - Wednesday, 26 May 2010, 14:02 GMT
I tested this two days ago with the following versions and the bug was apparently resolved (note that the x264 package version is unchanged from when I filed this bug):

x264-20100410-1
ffmpeg-23065-1
mplayer-31147-1

I was going to update this bug report for those packages. I have the new updates now, with the following packages:

x264-20100524-1
ffmpeg-23328-1
mplayer-31217-1

and I haven't tested this with them yet - I don't have a DVD handy to play around with. I guess you could go ahead and mark this bug as resolved since the last time I checked, it worked flawlessly. If there is a regression again, I'll post here/open a new report on mplayer/ffmpeg (since the bug wasn't technically in x264, but rather in the interoperability of x264 with mplayer).

Thanks for the fix. Much appreciated. :)

Loading...