Historical bug tracker for the Pacman package manager.
The pacman bug tracker has moved to gitlab:
https://gitlab.archlinux.org/pacman/pacman/-/issues
This tracker remains open for interaction with historical bugs during the transition period. Any new bugs reports will be closed without further action.
The pacman bug tracker has moved to gitlab:
https://gitlab.archlinux.org/pacman/pacman/-/issues
This tracker remains open for interaction with historical bugs during the transition period. Any new bugs reports will be closed without further action.
FS#24850 - [pacman] makepkg does not use distcc but make does
Attached to Project:
Pacman
Opened by John (graysky) - Thursday, 23 June 2011, 20:02 GMT
Last edited by Dan McGee (toofishes) - Friday, 24 June 2011, 20:44 GMT
Opened by John (graysky) - Thursday, 23 June 2011, 20:02 GMT
Last edited by Dan McGee (toofishes) - Friday, 24 June 2011, 20:44 GMT
|
Detailspacman 3.5.3-1
distcc-3.1-3 Summary and Info: I configured /etc/makepkg.conf to use distcc but as I monitor the build in distccmon-text, none of the work is being alloted to my other server when queued via makepkg. It does however from the shell using make (manually). Test package I am compiling is kernel26 from ABS which is know to work with distcc. Setup Details: *Both machines have distcc from [community] installed and distccd daemon is running on both machines. *Both machines have the following entry in /etc/hosts.allow: distccd: 192.168.0. Server PC (dual core on 192.168.0.3): ------------------ /etc/conf.d/distccd contains DISTCC_ARGS="--user nobody --allow 192.168.0.2" Daemon PC (quad core on 192.168.0.2): ------------------ /etc/makepkg.conf has been modified follows: MAKEFLAGS="-j7" BUILDENV=(distcc fakeroot color !ccache check) DISTCC_HOSTS="192.168.0.3" # this is the server PCs IP I start a makepkg -s and look at distccmon-text as it is compiling to see: $ distccmon-text 586 Blocked [0] 576 Blocked [0] 573 Blocked [0] 920 Compile localhost[0] 953 Compile localhost[1] 936 Compile localhost[2] 944 Compile localhost[3] This is in stark contrast to when I do the following from a shell: $ export DISTCC_HOSTS='localhost/5 192.168.0.32/3' $ make bzImage -j7 CC=distcc Now once the compile starts going: $ distccmon-text 17826 Blocked [0] 17747 Compile skbuff.c 192.168.0.3[2] 17800 Compile scatterlist.c 192.168.0.3[4] 17784 Compile compat.c localhost[0] 17810 Compile array.c localhost[1] 17692 Compile erst.c localhost[2] 17815 Compile random.c localhost[3] |
This task depends upon
$ export DISTCC_HOSTS='localhost/5 192.168.0.32/3' SHOULD BE
$ export DISTCC_HOSTS='localhost/5 192.168.0.3/3'
But you get the idea.
For using the distcc compiler, we simply add /usr/lib/distcc/bin to the front of the PATH, and do not override the compiler setting. What happens if you try to do it this way manually?
[[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
export DISTCC_HOSTS
then make as normal, no compiler specified.
So I just did this:
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ export DISTCC_HOSTS
But when I confirmed the variable, it was null:
$ echo $DISTCC_HOSTS
I then run the make line but the output was full of errors complaining:
$ time make bzImage -j7
CHK include/linux/version.h
HOSTCC scripts/basic/fixdep
CHK include/generated/utsrelease.h
HOSTCC scripts/basic/docproc
CC kernel/bounds.s
distcc[21490] (dcc_parse_hosts) Warning: /etc/distcc/hosts contained no hosts; can't distribute work
distcc[21490] (dcc_build_somewhere) Warning: failed to distribute, running locally instead
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ . /etc/makepkg.conf
$ time make bzImage -j7
I get the same errors.
However, if I modify the line in /etc/makepkg.conf from this:
DISTCC_HOSTS="192.168.0.3"
to this:
DISTCC_HOSTS="192.168.0.2/5 192.168.0.3/3"
The makepkg results is the same as before EXCEPT there are way more than 5 threads for it and the three that are supposed to go to 192.168.0.3 are still "blocked":
$ distccmon-text
18361 Blocked [0]
17119 Blocked [0]
17113 Blocked [0]
18598 Compile localhost[0]
18575 Compile localhost[0]
18539 Compile localhost[0]
18526 Compile localhost[0]
18525 Compile localhost[0]
18597 Compile localhost[1]
18538 Compile localhost[2]
18574 Compile localhost[3]
Interestingly, if I open a fresh shell, and compile manually as you suggested + sourcing same /etc/makepkg.conf as I just used, everything works! See:
$ [[ -d /usr/lib/distcc/bin ]] && export PATH="/usr/lib/distcc/bin:$PATH"
$ . /etc/makepkg.conf
$ echo $DISTCC_HOSTS
192.168.0.2/5 192.168.0.3/3
$time make bzImage -j7
$ distccmon-text
22286 Preprocess probe_64.c 192.168.0.2[0]
22276 Compile apic_noop.c 192.168.0.2[0]
22250 Compile kfifo.c 192.168.0.2[0]
22256 Compile blk-core.c 192.168.0.2[1]
22195 Compile block_dev.c 192.168.0.3[1]
22263 Compile compat.c 192.168.0.3[2]
22091 Compile hugetlb.c 192.168.0.3[3]
Seems like the problem is somehow inside /usr/bin/makepkg
?
$ distccmon-text
7370 Compile conflict.c 10.0.2.121[2]
7277 Compile alpm.c 10.0.2.121[2]
7279 Compile backup.c 10.0.2.121[4]
7387 Compile be_local.c 10.0.2.148[0]
7363 Compile add.c 10.0.2.148[1]
7377 Compile be_sync.c 10.0.2.148[3]
Note that I'm setting the --allow parameter to be my entire subnet (--allow 10.0.2.0/24), and im always specifying the LAN IP (never localhost) in DISTCC_HOSTS.
/etc/hosts.allow
/etc/conf.d/distccd
/etc/makepkg.conf
I'm going a bit crazy with this not working :)
Also, what package did you compile in your example above? I will attempt to repeat with it.
hosts.allow doesn't come into this as tcp wrappers is not involved.
Can you post your /etc/makepkg.conf
# makepkg.conf (from the master)
MAKEFLAGS="-j8"
BUILDENV=(fakeroot distcc color !ccache check)
DISTCC_HOSTS="10.0.2.121/4 10.0.2.148/4"
# /etc/conf.d/distccd (from BOTH machines)
DISTCC_ARGS="--user nobody --allow 10.0.2.0/24"