FS#24850 - [pacman] makepkg does not use distcc but make does
Attached to Project:
Pacman
Opened by John (graysky) - Thursday, 23 June 2011, 20:02 GMT
Last edited by Dan McGee (toofishes) - Friday, 24 June 2011, 20:44 GMT
Opened by John (graysky) - Thursday, 23 June 2011, 20:02 GMT
Last edited by Dan McGee (toofishes) - Friday, 24 June 2011, 20:44 GMT
|
Details
pacman 3.5.3-1
distcc-3.1-3 Summary and Info: I configured /etc/makepkg.conf to use distcc but as I monitor the build in distccmon-text, none of the work is being alloted to my other server when queued via makepkg. It does however from the shell using make (manually). Test package I am compiling is kernel26 from ABS which is know to work with distcc. Setup Details: *Both machines have distcc from [community] installed and distccd daemon is running on both machines. *Both machines have the following entry in /etc/hosts.allow: distccd: 192.168.0. Server PC (dual core on 192.168.0.3): ------------------ /etc/conf.d/distccd contains DISTCC_ARGS="--user nobody --allow 192.168.0.2" Daemon PC (quad core on 192.168.0.2): ------------------ /etc/makepkg.conf has been modified follows: MAKEFLAGS="-j7" BUILDENV=(distcc fakeroot color !ccache check) DISTCC_HOSTS="192.168.0.3" # this is the server PCs IP I start a makepkg -s and look at distccmon-text as it is compiling to see: $ distccmon-text 586 Blocked [0] 576 Blocked [0] 573 Blocked [0] 920 Compile localhost[0] 953 Compile localhost[1] 936 Compile localhost[2] 944 Compile localhost[3] This is in stark contrast to when I do the following from a shell: $ export DISTCC_HOSTS='localhost/5 192.168.0.32/3' $ make bzImage -j7 CC=distcc Now once the compile starts going: $ distccmon-text 17826 Blocked [0] 17747 Compile skbuff.c 192.168.0.3[2] 17800 Compile scatterlist.c 192.168.0.3[4] 17784 Compile compat.c localhost[0] 17810 Compile array.c localhost[1] 17692 Compile erst.c localhost[2] 17815 Compile random.c localhost[3] |
This task depends upon
$ export DISTCC_HOSTS='localhost/5 192.168.0.32/3' SHOULD BE
$ export DISTCC_HOSTS='localhost/5 192.168.0.3/3'
But you get the idea.
For using the distcc compiler, we simply add /usr/lib/distcc/bin to the front of the PATH, and do not override the compiler setting. What happens if you try to do it this way manually?
[[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
export DISTCC_HOSTS
then make as normal, no compiler specified.
So I just did this:
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ export DISTCC_HOSTS
But when I confirmed the variable, it was null:
$ echo $DISTCC_HOSTS
I then run the make line but the output was full of errors complaining:
$ time make bzImage -j7
CHK include/linux/version.h
HOSTCC scripts/basic/fixdep
CHK include/generated/utsrelease.h
HOSTCC scripts/basic/docproc
CC kernel/bounds.s
distcc[21490] (dcc_parse_hosts) Warning: /etc/distcc/hosts contained no hosts; can't distribute work
distcc[21490] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ . /etc/makepkg.conf
$ time make bzImage -j7
I get the same errors.
However, if I modify the line in /etc/makepkg.conf from this:
DISTCC_HOSTS="192.168.0.3"
to this:
DISTCC_HOSTS="192.168.0.2/5 192.168.0.3/3"
The makepkg results is the same as before EXCEPT there are way more than 5 threads for it and the three that are supposed to go to 192.168.0.3 are still "blocked":
$ distccmon-text
18361 Blocked [0]
17119 Blocked [0]
17113 Blocked [0]
18598 Compile localhost[0]
18575 Compile localhost[0]
18539 Compile localhost[0]
18526 Compile localhost[0]
18525 Compile localhost[0]
18597 Compile localhost[1]
18538 Compile localhost[2]
18574 Compile localhost[3]
Interestingly, if I open a fresh shell, and compile manually as you suggested + sourcing same /etc/makepkg.conf as I just used, everything works! See:
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ . /etc/makepkg.conf
$ echo $DISTCC_HOSTS
192.168.0.2/5 192.168.0.3/3
$time make bzImage -j7
$ distccmon-text
22286 Preprocess probe_64.c 192.168.0.2[0]
22276 Compile apic_noop.c 192.168.0.2[0]
22250 Compile kfifo.c 192.168.0.2[0]
22256 Compile blk-core.c 192.168.0.2[1]
22195 Compile block_dev.c 192.168.0.3[1]
22263 Compile compat.c 192.168.0.3[2]
22091 Compile hugetlb.c 192.168.0.3[3]
Seems like the problem is somehow inside /usr/bin/makepkg
?
$ distccmon-text
7370 Compile conflict.c 10.0.2.121[2]
7277 Compile alpm.c 10.0.2.121[2]
7279 Compile backup.c 10.0.2.121[4]
7387 Compile be_local.c 10.0.2.148[0]
7363 Compile add.c 10.0.2.148[1]
7377 Compile be_sync.c 10.0.2.148[3]
Note that I'm setting the --allow parameter to be my entire subnet (--allow 10.0.2.0/24), and im always specifying the LAN IP (never localhost) in DISTCC_HOSTS.
/etc/hosts.allow
/etc/conf.d/distccd
/etc/makepkg.conf
I'm going a bit crazy with this not working :)
Also, what package did you compile in your example above? I will attempt to repeat with it.
hosts.allow doesn't come into this as tcp wrappers is not involved.
Can you post your /etc/makepkg.conf
# makepkg.conf (from the master)
MAKEFLAGS="-j8"
BUILDENV=(fakeroot distcc color !ccache check)
DISTCC_HOSTS="10.0.2.121/4 10.0.2.148/4"
# /etc/conf.d/distccd (from BOTH machines)
DISTCC_ARGS="--user nobody --allow 10.0.2.0/24"