Distcc itself is not a compiler, it's a distributed front-end for GCC. It's running on the client machine to provide a "pool" of machine that will take part in the compilation. I will start with the preparation step up-to my first experiment.
Preparation
1. The first thing to do, is read some tutorial over the net. I found this IBM developer works tute helpful, even if it's not up-to-date. I'll talk more about it in the upcoming section.
2. The next thing is to download the latest distcc from distcc official site.
3. Configure and install distcc. I'm using the following approach in compiling and installing the system. Note that my system is x86_64 system with AMD Turion64 1.8GHz and AMD Athlon64 X2 4000+ cpus. That's why you'll see an AMD64-centric optimization flags. These are the command to configure and install the package. Also, don't forget that I'm using SLES 10 SP1 with cooked-up kernel 2.6.24.3
export CFLAGS=" -m64 -O3 -mtune=athlon64 -funroll-loops -fexpensive-optimizations "
export CXXFLAGS=${CFLAGS}
./configure --prefix=/usr --libdir=/usr/lib64 --sysconfdir=/etc --mandir=/usr/share/man --with-gtk
make -j4
su
make install
The --with-gtk switch is used to inform the distcc sources that you want a gtk-based front-end for the distcc daemon. You'll see a screenshot later when I'm working on the compilation test.
My First distcc Run a.k.a Preliminary Test
Once distcc installed, I'm ready to run the test. First, I need to export the available clients
using the DISTCC_HOSTS environment variable in the machine where I will carry-out the compilation. In this case, it's my Turion64 laptop.
darmawan@opunaga:~/download/unpack/xine-lib-1.1.10.1> export DISTCC_HOSTS="167.205.22.189 127.0.0.1"
The order of the available distcc hosts in the command above is important because the first one will be given the task to compile the file(s) first. You have to specify the fastest machine in the "pool" in this position. That's why my desktop machine comes first in the command above.
Then, run the distccd in the "pool", i.e. my desktop machine because I only have one machine beside the machine where the compilation invoked from:
darmawan@opusera:/home/sources/unpack/distcc-2.18.3> /usr/bin/distccd --daemon --allow 167.205.22.187
This is where the difference between the IBM developer works' tute and the current state of distccd (distccd version 2.18.3). The current version requires you to specify explicitly to run the distccd server as daemon process and also the client(s) that will be allowed to use the distccd server, if you run distccd as a stand-alone server process. Note that the distccd comand above runs on my Athlon64 X2 desktop.
Now, from the client, i.e. my Turion Laptop, I'm testing the distcc by using it to compile the xine library:
darmawan@opunaga:~/download/unpack/xine-lib-1.1.10.1> CC=distcc ./configure
darmawan@opunaga:~/download/unpack/xine-lib-1.1.10.1> make -j6
During the build process, I monitor the compilation process from my laptop using distccmon-gnome. This is the screenshot.
This is only preliminary test. My next target would be cross-compilation setup and other advanced distcc usage along with quantitative benchmark against single machine compilation. Stay tune :-).
Benchmarks and Fine Tuning
After several fine tuning attempts, I found that the most usable configuration is to run distcc locally in the laptop as ordinary user with the following parameters.
distccd --daemon -a 127.0.0.1 -N20 -j2
and also setting up the DISTCC_HOSTS environment variable to:
export DISTCC_HOSTS="167.205.22.189 127.0.0.1"
This way, the distccd won't disturb me working on the laptop and will squeeze every ounce of performance from the Athlon64 X2 desktop. The distccd runs on the desktop on ordinary user account with the following parameters.
distccd --daemon -a 167.205.22.187 -N0 -j6
Now, let's see how the single machine compilation compares to offloading the task to two machines. To see the difference, I'm doing a linux kernel 2.6.24.3 compilation benchmark. The following is the result.
Platform Compile Time
Turion64 laptop (single core 1.8GHz, 1GB RAM)
GCC 4.1.2 x86_64 multilib 17 min 26 sec
Distcc (GCC 4.1.2 x86_64 multilib back-end)
Turion64 laptop (single core 1.8GHz, 1GB RAM) +
Athlon64 X2 desktop (dual core 2.1GHz, 2GB RAM) 7 min 26 sec
The kernel compilation is timed with the following scripts:
#!/bin/bash
#
# This script is used to build the linux kernel and provide
# build-time information for distributed compilation
#
#
make distclean
cat /proc/config.gz > .config.gz
gzip -dv .config.gz
make silentoldconfig
date +%T > timing_info.txt
make CC="distcc gcc" -j10
date +%T >> timing_info.txt
#!/bin/bash
#
# This script is used to build the linux kernel and provide
# build-time information for single machine compilation
#
#
make distclean
cat /proc/config.gz > .config.gz
gzip -dv .config.gz
make silentoldconfig
date +%T >> timing_info.txt
make -j2
date +%T >> timing_info.txt
This is the screenshot of the distcc during the kernel compilation benchmark.
As you can see, distcc improves my productivity more than two folds due to task distribution to the much more powerful machine.
A Glimpse Over Distributed Cross-Compilation
I've tried distributed cross-compilation as well. But, not as fine-grained as the previous kernel compilation benchmark. To carry-out distributed cross compilation, what you need are:
- The same cross-compiler in all the machine that will participate. Make sure that the path to the cross-compiler and its associated tools is placed in the system-wide PATH environment variable. You can echo ${PATH} to ensure it has been setup correctly.
- Distcc installed in all of the machine that will participate.
To do the cross-compilation, invoke the specific cross compiler when you run make, for example, to distribute the mips cross-compilation task, you would invoke:
make CC="distcc mips-uclibc-gcc"
Or another way is to edit the corresponding Makefile(s) in the source code that going to be compiled.
In the next update of this post, I will show how to do cross compilation in more detail, with some benchmarks of course.
