Working with Data Structure in IDA Pro

I've been working with data structure in IDA Pro for a while and I found this article to be very helpful. It helps you a lot in reading the disassembled binary, particularly those generated from C/C++.

Wine Font Rendering Hiccups and Fixes

I have to admit that the default font rendering in Wine sucks. That is due to the disabled Sub Pixel Font Rendering in default Wine installation. Before we get into the fix, I'd like to highlight my system configuration first. Here it is:

1. Operating System: Slamd64 12.1 with customized kernel 2.6.28.10


2. Installed font rendering-related packages: FreeType 2.3.5, Fontconfig 2.4.2, Fontforge 20090923


3. Wine version 1.1.33
   

I built Fontforge with the following slackbuild script:

#!/bin/sh

# Slackware build script for fontforge

# Copyright 2006-2008  Robby Workman  Northport, Alabama, USA
# All rights reserved.
#
# Redistribution and use of this script, with or without modification, is
# permitted provided that the following conditions are met:
#
# 1. Redistributions of this script must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ''AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
# EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# Thanks to Eric Hameleers for some pointers on getting the fontforge.pc
# file right without too much manual intervention :-)

PRGNAM=fontforge
VERSION=20090923
ARCH=${ARCH:-x86_64}
BUILD=${BUILD:-1}
TAG=${TAG:-_SB64}

CWD=$(pwd)
TMP=${TMP:-/tmp/SB64}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

if [ "$ARCH" = "i486" ]; then
SLKCFLAGS="-O2 -march=i486 -mtune=i686"
elif [ "$ARCH" = "i686" ]; then
SLKCFLAGS="-O2 -march=i686 -mtune=i686"
elif [ "$ARCH" = "x86_64" ]; then
SLKCFLAGS="-O2 -fPIC"
fi

set -e

rm -rf $PKG
mkdir -p $TMP $PKG $OUTPUT
cd $TMP
rm -rf $PRGNAM-$VERSION
tar xvf $CWD/${PRGNAM}_full-${VERSION}.tar.bz2
cd $PRGNAM-$VERSION
chown -R root:root .
chmod -R u+w,go+r-w,a-s .

CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
--prefix=/usr \
--libdir=/usr/lib64 \
--mandir=/usr/man \
--with-x \
--with-pic \
--with-devicetables \
--with-freetype-src=/usr/include/freetype \
--without-python \
--disable-pyextension \
--enable-libff \
--enable-type3 \
--enable-tilepath \
--disable-debug \
--enable-static=no \
--enable-shared=yes \
--build=$ARCH-slamd64-linux

make
make prefix=$PKG/usr libdir=$PKG/usr/lib64 install

# Let's add the cidmaps if the user downloaded them
if [ -e $CWD/cidmaps.tgz ]; then
mkdir -p $PKG/usr/share/fontforge
tar xvf $CWD/cidmaps.tgz -C $PKG/usr/share/fontforge || exit 1
chmod 0644 $PKG/usr/share/fontforge/*.cidmap
chown root:root $PKG/usr/share/fontforge/*.cidmap
fi

( cd $PKG
find . | xargs file | grep "executable" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true
find . | xargs file | grep "shared object" | grep ELF | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null
)

( cd $PKG/usr/man
find . -type f -exec gzip -9 {} \;
for i in $( find . -type l ); do ln -s $( readlink $i ).gz $i.gz ; rm $i ; done
)

mkdir -p $PKG/usr/doc/$PRGNAM-$VERSION
cp -a AUTHORS INSTALL LICENSE README-Unix.html README-unix VERSION \
$PKG/usr/doc/$PRGNAM-$VERSION
cat $CWD/$PRGNAM.SlackBuild > $PKG/usr/doc/$PRGNAM-$VERSION/$PRGNAM.SlackBuild
find $PKG/usr/doc -type f -exec chmod 644 {} \;

mkdir -p $PKG/install
cat $CWD/slack-desc > $PKG/install/slack-desc

cd $PKG
/sbin/makepkg -l y -c n $OUTPUT/$PRGNAM-$VERSION-$ARCH-$BUILD$TAG.tgz

The Wine installation is also custom compiled with the following script:

#!/bin/sh
# $Id: wine.SlackBuild,v 1.54 2009/09/14 20:51:17 root Exp root $
# Copyright 2006-2009  Eric Hameleers, Eindhoven, NL
# All rights reserved.
#
#   Permission to use, copy, modify, and distribute this software for
#   any purpose with or without fee is hereby granted, provided that
#   the above copyright notice and this permission notice appear in all
#   copies.
#
#   THIS SOFTWARE IS PROVIDED ``AS IS'' AND ANY EXPRESSED OR IMPLIED
#   WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
#   MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
#   IN NO EVENT SHALL THE AUTHORS AND COPYRIGHT HOLDERS AND THEIR
#   CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
#   USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
#   ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
#   OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT
#   OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
#   SUCH DAMAGE.
# -----------------------------------------------------------------------------
# Slackware SlackBuild script
# ===========================
# By:        Eric Hameleers 
# For:       wine
# Descr:     Wine Is Not an Emulator
# URL:       http://winehq.com/
# Needs:     fontforge (builds better fonts when compiling wine)
# 1.1.29-1:   14/sep/2009 by Eric Hameleers 
#
# Run 'sh wine.SlackBuild' to build a Slackware package.
# The package (.tgz) plus descriptive .txt file are created in /tmp .
# Install using 'installpkg'.
#
# -----------------------------------------------------------------------------

# Set initial variables:

PRGNAM=wine
SRCVER=${SRCVER:-"1.1.33"}
VERSION=$(echo $SRCVER | tr '-' '.')
ARCH=${ARCH:-"x86_64"}
BUILD=${BUILD:-1}
TAG=${TAG:-alien}

DOCS="ANNOUNCE AUTHORS COPYING.LIB ChangeLog LICENSE* README VERSION"

# Set the variable OPENGL to "NO" if you don't have a card that
# supports hardware accelerated OpenGL:
OPENGL=${OPENGL:-"YES"}    # Use 'YES' not 'yes' : case-sensitive!

# If you set REQUIRE_FONTFORGE to "NO" then the script won't refuse to build
#   wine in case you don't have fontforge installed (it is needed to
#   generate the required base fonts).
REQUIRE_FONTFORGE=${REQUIRE_FONTFORGE:-"YES"}

# Where do we look for sources?
SRCDIR=$(cd $(dirname $0); pwd)

# Place to build (TMP) package (PKG) and output (OUTPUT) the program:
TMP=${TMP:-/tmp/build}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

SOURCE="$SRCDIR/${PRGNAM}-${SRCVER}.tar.bz2"
SRCURL="http://ibiblio.org/pub/linux/system/emulators/${PRGNAM}/${PRGNAM}-${SRCVER}.tar.bz2"

##
## --- with a little luck, you won't have to edit below this point --- ##
##

# Exit the script on errors:
set -e
trap 'echo "$0 FAILED on line $LINENO!" | tee $OUTPUT/error-${PRGNAM}.log' ERR
# Catch unitialized variables:
set -u
P1=${1:-1}

case "$ARCH" in
i486)      SLKCFLAGS="-O2 -march=i486 -mtune=i686"
;;
s390)      SLKCFLAGS="-O2"
;;
powerpc)   SLKCFLAGS="-O2"
;;
x86_64)    SLKCFLAGS="-O2 -fPIC"
;;
esac

# Prepare a sane build environment:
mkdir -p $TMP/tmp-$PRGNAM # location to build the source
rm -rf $TMP/tmp-$PRGNAM/* # remove the remnants of previous build and continue
mkdir -p $PKG             # place for the package to be built
rm -rf $PKG/*             # We always erase old package's contents
mkdir -p $OUTPUT          # place for the package to be saved

# Source files availability:
if ! [ -f ${SOURCE} ]; then
if ! [ "x${SRCURL}" == "x" ]; then
# Check if the $SRCDIR is writable at all - if not, download to $OUTPUT
[ -w "$SRCDIR" ] || SOURCE="$OUTPUT/$(basename $SOURCE)"
echo "Source '$(basename ${SOURCE})' not available yet..."
echo "Will download file to $(dirname $SOURCE)"
wget -nv -T 20 -O "${SOURCE}" "${SRCURL}" || true
if [ $? -ne 0 -o ! -s "${SOURCE}" ]; then
echo "Downloading '$(basename ${SOURCE})' failed... aborting the build."
mv -f "${SOURCE}" "${SOURCE}".FAIL
exit 1
fi
else
echo "File '$(basename ${SOURCE})' not available... aborting the build."
exit 1
fi
fi

if [ "$P1" == "--download" ]; then
echo "Download complete."
exit 0
fi

# --- PACKAGE BUILDING ---

echo "++"
echo "|| $PRGNAM-$VERSION"
echo "++"

if ! which fontforge >/dev/null 2>&1 ; then
echo "##"
echo "## The 'fontforge' program does not seem to be installed."
echo "## Wine uses fontforge to generate several TTF fonts (tahoma,tahomabd,marlett)"
echo "##   that your Windows programs may want to use!"
if [ "$REQUIRE_FONTFORGE" != "YES" ]; then
echo "##"
echo "## Continuing the build anyway, but you were warned..."
echo "## Sleeping for 5 seconds, press -C if you want to abort now."
echo "##"
sleep 5
else
echo "##"
echo "## Aborting the build - set the internal script variable:"
echo "##"
echo "##   'REQUIRE_FONTFORGE' to a value of 'NO'"
echo "##"
echo "## if you don't want to use fontforge to generate TTF fonts."
echo "##"
exit 1
fi
fi

cd $TMP/tmp-$PRGNAM

echo "Extracting the source archive(s) for $PRGNAM..."
tar -xvf ${SOURCE}
[ "$SRCVER" != "$VERSION" ] && mv ${PRGNAM}-${SRCVER} ${PRGNAM}-${VERSION}
cd ${PRGNAM}-${VERSION}
# Starting with openssl-0.9.8h, a typedef was introduced named "X509_EXTENSIONS"
# into openssl/x509.h.  Unfortunately, wine uses a define of the same name
# in include/wincrypt.h. You may need this patch:
#patch -p0 < $SRCDIR/wine_openssl.patch chown -R root:root . chmod -R u+w,go+r-w,a-s .  echo Building ...  [ "${OPENGL}" = "YES" ] && do_opengl="" || do_opengl="out"  LDFLAGS="-L/usr/lib -ldl" \ CFLAGS="$SLKCFLAGS" \ CXXFLAGS="$SLKCFLAGS" \ ./configure \   --prefix=/usr \   --libdir=/usr/lib \   --localstatedir=/var \   --sysconfdir=/etc \   --mandir=/usr/man \   --with-x \   --with-fontconfig \   --with-freetype \   --without-ldap \   --without-nas \   --with${do_opengl}-opengl \   --program-prefix= \   --program-suffix= \   --build=$ARCH-slamd64-linux \   2>&1 | tee $OUTPUT/configure-${PRGNAM}.log
make depend 2>&1 | tee $OUTPUT/make-${PRGNAM}.log
make 2>&1 | tee -a $OUTPUT/make-${PRGNAM}.log
make DESTDIR=$PKG install 2>&1 |tee $OUTPUT/install-${PRGNAM}.log

# Add a desktop menu for the winecfg program:
mkdir -p $PKG/usr/share/pixmaps
cp -a programs/winemenubuilder/wine.xpm $PKG/usr/share/pixmaps/
mkdir -p $PKG/usr/share/applications
cat <<-_EOT_ > $PKG/usr/share/applications/winecfg.desktop
[Desktop Entry]
Exec=winecfg
Icon=/usr/share/pixmaps/wine.xpm
Terminal=false
Name=Wine Configuration
Comment=Configure Wine
Type=Application
Categories=Application;Settings
_EOT_

# Add some documentation to the package:
mkdir -p $PKG/usr/doc/$PRGNAM-$VERSION
cp -a $DOCS $PKG/usr/doc/$PRGNAM-$VERSION || true
cp -a $SRCDIR/$(basename $0) $PKG/usr/doc/$PRGNAM-$VERSION/$PRGNAM.SlackBuild
chown -R root:root $PKG/usr/doc/$PRGNAM-$VERSION
find $PKG/usr/doc -type f -exec chmod 644 {} \;


# Compress the man page(s)
if [ -d $PKG/usr/man ]; then
cd  $PKG/usr/man
find . -type f -name "*.?" -exec gzip -9f {} \;
for i in $(find . -type l -name "*.?") ; do ln -s $( readlink $i ).gz $i.gz ; rm $i ; done
fi

# Strip binaries
find $PKG | xargs file | grep -e "executable" -e "shared object" \
| grep ELF | cut -f1 -d: | xargs strip --strip-unneeded 2>/dev/null

# Add a package description:
mkdir -p $PKG/install
cat $SRCDIR/slack-desc > $PKG/install/slack-desc
if [ -f $SRCDIR/doinst.sh ]; then
cat $SRCDIR/doinst.sh >> $PKG/install/doinst.sh
fi
if [ -f $SRCDIR/slack-required ]; then
cat $SRCDIR/slack-required > $PKG/install/slack-required
fi

# Build the package:
cd $PKG
makepkg --linkadd y --chown n $OUTPUT/${PRGNAM}-${VERSION}-${ARCH}-${BUILD}${TAG}.tgz 2>&1 | tee $OUTPUT/makepkg-${PRGNAM}.log
cd $OUTPUT
md5sum ${PRGNAM}-${VERSION}-${ARCH}-${BUILD}${TAG}.tgz > ${PRGNAM}-${VERSION}-${ARCH}-${BUILD}${TAG}.tgz.md5
cd -
cat $PKG/install/slack-desc | grep "^${PRGNAM}" > $OUTPUT/${PRGNAM}-${VERSION}-${ARCH}-${BUILD}${TAG}.txt
if [ -f $PKG/install/slack-required ]; then
cat $PKG/install/slack-required > $OUTPUT/${PRGNAM}-${VERSION}-${ARCH}-${BUILD}${TAG}.dep
fi

Note that the Wine that I built is a 32-bit Wine version 1.1.33. Therefore, it must link to the 32-bit compatibility package in my Slamd64. I haven't tested whether this configuration will work flawlessly if the 64-bit libraries is linked to the Wine sources in the package build process. Another thing is I installed Fontforge prior to building the Wine package from its source files.

The Fontforge and Wine packages were installed using installpkg utility. Now let's see the default installation screenshot.

As you can see in the image above, the font rendering is bad for long time viewing, because of the "jagged" font rendering. The strain in the eyes just cannot be neglected anymore. Fortunately, someone posted the way to enable "subpixel font rendering" on the web. I was a bit clueless about accessing the registry at first--you need to access the registry to enable the subpixel font rendering. But, I found that you just need to run winefile from your shell (in X of course) and then go to the Windows directory in drive C:\ in your wine installation. The regedit.exe program is there. Now, let's have a look at the whole steps to enable subpixel font rendering:

1. First go to your Windows directory in your wine installation.


2. Run regedit.exe


3. Go to [HKEY_CURRENT_USER\Control Panel\Desktop] and set the following values (create the new entries if they haven't exist yet):
   
   
   
   • "FontSmoothing"="2"
   
   
   • "FontSmoothingType"=dword:00000002
   
   
   • "FontSmoothingGamma"=dword:00000578
   
   
   • "FontSmoothingOrientation"=dword:0000000
   
   
   
   


4. You have to close all applications that use Wine and start them afterwards to see the result

Now, let's see the result. Below are the screenshots of application running in Wine with subpixel font rendering enabled.

I come across this solution after twiddling with various versions of Wine. Some Wine versions older than version 1.1.18 will just crash when I try to change the font in IDA Pro. Even version 1.1.18 will make IDA Pro crash if I linked it with Fontforge. The version which works without crashing is version 1.1.33. But its font is awful without enabling the subpixel font rendering.

Profiling ffmpeg on Linux x86_64

In this blog post I'll elaborate on how to profile ffmpeg v0.5 on Slamd64 12.1 with the GNU profiler (gprof). Well, the profiling is meant to find the bottleneck of ffmpeg (and probably do something with it in the future).
First, rebuild the ffmpeg package with profiling enabled. This is the important excerpt from my build script:

...
./configure \
  --prefix=/usr \
  --libdir=/usr/lib64 \
  --mandir=/usr/man \
  --shlibdir=/usr/lib64 \
  --disable-debug \
...
  --enable-gprof \
  --disable-stripping \
...

#REMEMBER NOT TO STRIP THE FINAL BINARY -- SO COMMENT THE LINES BELOW
#find $PKG | xargs file | grep -e "executable" -e "shared object" | grep ELF \
#  | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true
...

Now, rebuild the ffmpeg package. Install it afterwards.

Second, run ffmpeg to transcode a DVD VOB file into an mpeg4 avi file. This is the example:

ffmpeg -i /mnt/dvd/VIDEO_TS/VTS_01_4.VOB  -f avi -vcodec mpeg4 -b 800k -g 300 -bf 2 -acodec libfaac -ab 128k outlander_01_4.avi

After the transcoding completed. We will have the profiling information in a file named gmon.out in the directory where ffmpeg was executed.

Next up, use gprof to generate a human readable statistics of the previous ffmpeg run.

gprof /usr/bin/ffmpeg gmon.out > stats.txt

At this point we have the profiling information in stats.txt. The following is snippets of the contents of my stats.txt

Flat profile:

Each sample counts as 0.01 seconds.
  %   cumulative   self              self     total          
 time   seconds   seconds    calls  ms/call  ms/call  name   
 45.46      0.20     0.20                             main
 40.92      0.38     0.18    68375     0.00     0.00  output_packet
  6.82      0.41     0.03    38468     0.00     0.00  do_audio_out
  3.41      0.43     0.02    87594     0.00     0.00  write_frame
  2.27      0.44     0.01   107095     0.00     0.00  print_report
  1.14      0.44     0.01        1     5.00     5.00  opt_format
  0.00      0.44     0.00  1982784     0.00     0.00  data_start
  0.00      0.44     0.00        7     0.00     0.00  find_codec_or_die
  0.00      0.44     0.00        7     0.00     0.00  set_context_opts
  0.00      0.44     0.00        4     0.00     0.00  opt_default
  0.00      0.44     0.00        2     0.00     0.00  opt_bitrate
  0.00      0.44     0.00        1     0.00     0.00  av_exit
  0.00      0.44     0.00        1     0.00     0.00  new_audio_stream
  0.00      0.44     0.00        1     0.00     0.00  new_video_stream
  0.00      0.44     0.00        1     0.00     0.00  opt_audio_codec
  0.00      0.44     0.00        1     0.00     0.00  opt_input_file
  0.00      0.44     0.00        1     0.00     0.00  opt_output_file
  0.00      0.44     0.00        1     0.00     0.00  opt_video_codec
  0.00      0.44     0.00        1     0.00     5.00  parse_options
  0.00      0.44     0.00        1     0.00     0.00  print_all_lib_versions
  0.00      0.44     0.00        1     0.00     0.00  show_banner

 %         the percentage of the total running time of the
time       program used by this function.
...
granularity: each sample hit covers 2 byte(s) for 2.27% of 0.44 seconds

index % time    self  children    called     name
                                                 
[1]    100.0    0.20    0.24                 main [1]
                0.18    0.05   68375/68375       output_packet [2]
                0.01    0.00  107095/107095      print_report [5]
                0.00    0.01       1/1           parse_options [7]
                0.00    0.00  715631/1982784     data_start [8]
                0.00    0.00       1/1           show_banner [21]
                0.00    0.00       1/1           av_exit [13]
-----------------------------------------------
                0.18    0.05   68375/68375       main [1]
[2]     51.1    0.18    0.05   68375         output_packet [2]
                0.03    0.01   38468/38468       do_audio_out [3]
                0.01    0.00   29892/87594       write_frame [4]
                0.00    0.00  512643/1982784     data_start [8]
...
Index by function name

  [13] av_exit                [16] opt_audio_codec         [2] output_packet
   [8] data_start             [12] opt_bitrate             [7] parse_options
   [3] do_audio_out           [11] opt_default            [20] print_all_lib_versions
   [9] find_codec_or_die       [6] opt_format              [5] print_report
   [1] main                   [17] opt_input_file         [10] set_context_opts
  [14] new_audio_stream       [18] opt_output_file        [21] show_banner
  [15] new_video_stream       [19] opt_video_codec         [4] write_frame

There is an explanation following every statistics section generated by gprof. Therefore, it shouldn't be hard to analyze the profiling result. That's it for the moment.

OpenMP vs OpenMPI

For beginner "computationalist" like me, it's quite hard to understand the difference between OpenMP and OpenMPI. At first, I thought both of them tackles the same problem in the same way, namely parallel execution. However, after studying them both further, it's clear that OpenMPI uses a distributed-memory architecture while OpenMP uses shared-memory model. Both of the memory architecture can be explained as follows:

• In a distributed-memory architecture, each process doesn't share the same address space as the other process (which very possibly run on different machine). This means each process cannot "see" the other process variable(s). The process must "send a message" to the other process to change variable in the other process. Hence, the "Massage Passing Interface (MPI)". The MPI library such as OpenMPI basically is a sort of "middleware" to facilitate the massage passing between the processes, the process migration, initialization and tear-down.
• In a shared-memory architecture, there is usually one process which contains couple of threads which share the same memory address space, file handles and so on. Hence, the shared memory name. In this architecture, each threads can modify a "precess" global data. Therefore, a semaphore mechanism must be in use. OpenMP simplify the programming for shared memory architeture by providing compiler "extensions" in the form of various standardized "pragma"s. 

Upon reading both of the simplified explanation above, it's obvious that we can combine OpenMPI and OpenMP for paralel execution of code. Say, use OpenMP for "local execution within a machine" and use OpenMPI for inter-machine process communication. 

OpenMPI Slackbuild Script for Slamd64 12.1

I've just got OpenMPI to work on my Slamd64 12.1 system. I really hate cluttering the system. Therefore, I just build the package to ease removing it when I want to upgrade to a newer OpenMPI version later. This is the slackbuild script:

#!/bin/sh

# Slackware build script for Open MPI

# Written by Aleksandar Samardzic 
# Modified for Slamd64 12.1 by Darmawan Salihun  

PRGNAM=openmpi
VERSION=${VERSION:-1.3.3}
ARCH=${ARCH:-x86_64}
BUILD=${BUILD:-1}
TAG=${TAG:-_SBo}

CWD=$(pwd)
TMP=${TMP:-/tmp/SBo}
PKG=$TMP/package-$PRGNAM
OUTPUT=${OUTPUT:-/tmp}

if [ "$ARCH" = "i486" ]; then
  SLKCFLAGS="-O2 -march=i486 -mtune=i686"
elif [ "$ARCH" = "i686" ]; then
  SLKCFLAGS="-O2 -march=i686 -mtune=i686"
elif [ "$ARCH" = "x86_64" ]; then
  SLKCFLAGS="-O2 -fPIC"
fi

set -e

rm -rf $PKG
mkdir -p $TMP $PKG $OUTPUT
cd $TMP
rm -rf $PRGNAM-$VERSION
tar xvf $CWD/$PRGNAM-$VERSION.tar.bz2
cd $PRGNAM-$VERSION
chown -R root:root .
find . \
 \( -perm 777 -o -perm 775 -o -perm 711 -o -perm 555 -o -perm 511 \) \
 -exec chmod 755 {} \; -o \
 \( -perm 666 -o -perm 664 -o -perm 600 -o -perm 444 -o -perm 440 -o -perm 400 \) \
 -exec chmod 644 {} \;


CFLAGS="$SLKCFLAGS" \
CXXFLAGS="$SLKCFLAGS" \
./configure \
  --prefix=/usr \
  --libdir=/usr/lib64 \
  --mandir=/usr/man \
  --sysconfdir=/etc \
  --localstatedir=/var \
  --docdir=/usr/doc/$PRGNAM-$VERSION \
  --enable-static \
  --enable-mpirun-prefix-by-default \
  --build=$ARCH-slamd64-linux

make -j4
make install DESTDIR=$PKG

find $PKG | xargs file | grep -e "executable" -e "shared object" | grep ELF \
  | cut -f 1 -d : | xargs strip --strip-unneeded 2> /dev/null || true

( cd $PKG/usr/man
  find . -type f -exec gzip -9 {} \;
  for i in $(find . -type l) ; do ln -s $(readlink $i).gz $i.gz ; rm $i ; done
)

# Let's not clobber config files
mv $PKG/etc/openmpi-totalview.tcl $PKG/etc/openmpi-totalview.tcl.new
mv $PKG/etc/openmpi-mca-params.conf $PKG/etc/openmpi-mca-params.conf.new
mv $PKG/etc/openmpi-default-hostfile $PKG/etc/openmpi-default-hostfile.new

mkdir -p $PKG/usr/doc/$PRGNAM-$VERSION
cp -a AUTHORS INSTALL LICENSE NEWS README VERSION examples \
  $PKG/usr/doc/$PRGNAM-$VERSION
cat $CWD/$PRGNAM.SlackBuild > $PKG/usr/doc/$PRGNAM-$VERSION/$PRGNAM.SlackBuild

mkdir -p $PKG/install
cat $CWD/slack-desc > $PKG/install/slack-desc

cd $PKG
/sbin/makepkg -l y -c n $OUTPUT/$PRGNAM-$VERSION-$ARCH-$BUILD$TAG.tgz

That's it. Now, I can upgrade the package easily later. Anyway, I used OpenMPI 1.3.3 for my current experimental side projects.

AMI BIOS Reverse Engineering Article

AMI BIOS Reverse Engineering article is up. Check out the details here.

Booting OS Installed on Different Physical Drive in LILO

Booting an OS installed on other physical hard drive could be daunting at times.
However, one LILO keyword could prove very helpful:

master-boot

This keyword means (taken straight from LILO's repertoire):

'master-boot This flag (LILO version 22.5) indicates a DOS/Windows/OS2 or other system which will only boot from BIOS device 0x80, the "C:" drive, or BIOS device 0, the A: drive. When this flag is specified, if this drive is not assigned device code 0x80 or 0 by the BIOS, then the chain loader will dynamically swap the device code actually assigned with device code 0x80 or 0 to make this drive appear to be the first hard or floppy drive, "C:" or "A:".'


In my setup, the default boot drive is sda1 while M$ Windoze XP SP2 is installed on hda1. It seems to be M$ Windoze XP SP2 cannot boot unless its drive appear to be the first hard drive. This is the relevant excerpt from my /etc/lilo.conf:

# LILO configuration file
# generated by 'liloconfig'
#
# Start LILO global section
boot = /dev/sda
#compact        # faster, but won't work on all systems.
lba32
...
# Linux bootable partition config begins
image = /boot/vmlinuz
  root = /dev/sda1
  label = Slamd64-12.1
  read-only  # Partitions should be mounted read-only for checking
...
# Linux bootable partition config ends

# Windows bootable partition config begins
other = /dev/hda1
  label = Windows
  table = /dev/hda
  master-boot
# Windows bootable partition config ends

That's it. You should be able to boot from Windoze or other "insane" OS from other hard drive with LILO.

Traversing "Huge" Source Code Efficiently

It's inevitable that as a software developer, you will encounter such an enormous amount of code and tasked to modify it to siut the goal that you (or the project manager) have set. Of course, you will ask, why don't use Ctags or something like that? Yeah, of course I use Ctags, but sometimes it's just not fine-grained enough or simply refer to the wrong definition of variables or functions and moreover it can't handle assembly source code reliably.

I have made two bash scripts to cope with this problem. The first script helps you in finding a word (usually function names or symbols in assembler source code):

#!/bin/bash
#
# Script to find a word in an subversion checkout
#
#

SEARCH_PATH=""
WORD=""

if [ $# -ne 2  ]; then
 echo "Usage: $(basename $0) path_to_search  word"
 exit 1;
fi 

SEARCH_PATH=$1
WORD=$2

 find ${SEARCH_PATH} -type f -name '*' -exec grep -nH "\<${WORD}\>" \{} \; | awk '$0 !~ /\.svn/ {print}'

The second script is used to find a string:

#!/bin/bash
#
# Script to find a string in an subversion checkout
#
#

SEARCH_PATH=""
WORD=""

if [ $# -ne 2  ]; then
 echo "Usage: $(basename $0) path_to_search  word"
 exit 1;
fi 

SEARCH_PATH=$1
WORD=$2


 find ${SEARCH_PATH} -type f -name '*' -exec grep -nH "${WORD}" \{} \; | awk '$0 !~ /\.svn/ {print}'

What you need to run both of the previous scripts are find, grep, awk and a bash compatible shell.

Another trick is to search where the symbol/function-name in the compiled object file. More on that later.

Pinczakko Official Website moved to Google Sites

The detail is in this post:
http://bioshacking.blogspot.com/2009/09/pinczakko-official-website-relocation.html.

The Dreaded Pure-ftpd

Thanks to Dragkh’s Blog (http://dragkh.wordpress.com/2009/06/21/slackware-12-2-pure-ftpd-v1-0-22-425-sorry-invalid-address-given/) , I finally found the solution to this error message in my embedded ftp client after a few hours of attempts:

ftpget: Sorry, invalid address given

It wasn't the ftp client fault, but it was because the Pure-ftpd (ftp server) misbehaved. The fix is simple, just invoke Pure-ftpd with the -H parameter, like this:

pure-ftpd -B -H

The -H parameter means "Don't resolve host names ("192.0.34.166" will be logged instead of "www.example.com")". It should've been explained in the README file of Pure-ftpd source code, but it wasn't. At least someone poested the fix :-(.

What the h*** is NAT2.5

I've been tracing this bug for nearly two weeks in my free time.
The bug made the Realtek RTL8186 SoC hardware--that I was working with--didn't forward PPPoE packets from its ethernet interface to its WLAN interface even if both of the interfaces had been bridged together by using brctl--the bridge utilities (http://bridge.sourceforge.net/).

After doing some quick code inspection from different subversion revision of the firmware source code--that I'm working with--I found a very suspicious piece of code:

...
/** clues from revision 14. This revision has the eth to WLAN forwarding works  */
        // set nat2.5 disable when client and mac clone is set
        apmib_get(MIB_WLAN_NAT25_MAC_CLONE, (void *)&intVal);
        if ((intVal == 1) && (mode == 1)) {
                pmib->ethBrExtInfo.nat25_disable = 1;
                pmib->ethBrExtInfo.macclone_enable = 1;
        }
        else {
                pmib->ethBrExtInfo.nat25_disable = 0;
                pmib->ethBrExtInfo.macclone_enable = 0;
        }
...

Now, in the code above, macclone is very clear from its name alone. But, what the h*** is Nat2.5? I've been working with IP layer NATs, but I've never heard of this Nat2.5 stuff. Upon searching the internet and cross checking the information with the Application Note I have at hand, I found out that it's some sort of "MAC address translation". It works this way:

Device_1_ethernet <---> RTL8186_ethernet<-->nat2.5<-->RTL8186_WLAN(client_mode)<-->WLAN_Access_Point

Nat2.5 will translate (transform as per the Application Note lingo) the MAC address of ethernet packets from the Device_1_ethernet to RTL8186_WLAN MAC address before forwarding the packets to the WLAN_Access_Point via the radio link.
At first, this sounds counter-intuitive. Why do you need the tranformation? I don't know for sure. Nonetheless, if we go back to the Ethernet vs WLAN/802.11 specification, it's clear that both 802.11 and Ethernet shares the same "Logical Link Control" sublayer (in the upper half of data link layer), but both of them have a different "Medium Access Control (MAC)" sublayer (in the lower half of data link layer). Because MACs from both protocol are different, then there should be "transformation". I think that's the reason.

Anyway, the following is the code in svn revision 115 before the fix

...
       // disable nat2.5 and macclone entirely -- darmawan
       pmib->ethBrExtInfo.nat25_disable = 1;
       pmib->ethBrExtInfo.macclone_enable = 0;
...

and this is the code after the fix.

...
       // Enable nat2.5 when in client mode, 
       // but disable macclone entirely -- darmawan
       apmib_get(MIB_WLAN_MODE, (void *)&mode);
       if (mode == CLIENT_MODE) {
               pmib->ethBrExtInfo.nat25_disable = 0;
       }
       else {
               pmib->ethBrExtInfo.nat25_disable = 1;
       }
       pmib->ethBrExtInfo.macclone_enable = 0;
...

So, that's it. There you have it, the freaking Nat2.5.

Vacuum Months

Haven't been posting anything since July. Well, I am still busy finishing my college. Should be finished by mid september and a lot of updates should be coming up. Particularly moving all of the BIOS articles from Geocities before it's closed by Yahoo.

BIOS Disassembly Ninjutsu Uncovered Status Update

I have received the contract termination document from A-List Publishing for the English Edition of the BIOS Disassembly Ninjutsu Uncovered book that I wrote. Therefore, from now on I plan to publish the book from Indonesia. Read this post for further details.

That's it for the moment.

Please follow The BIOS Blog for further BIOS related information in the future.

Mplayer Build Script for Slamd64 12.1

Most of unofficial package for Slamd64 Linux distribution can be found at http://builds.slamd64.com. However, some of the scripts are not well tested or work out-of-the-box, at least against Slamd64 12.1. One of such a script is the Mplayer build script. The build script for Mplayer lacks the PKGNAM variable which should be set to mplayer like this:

PKGNAM=mplayer

The lack of this variable in Slamd64 12.1 will cause package creation failure. I found this error after reading the pkghelper's scripts in the /etc/pkghelpers.d directory. This is the complete build script with the fix:

#!/bin/sh

# Slackware build script for mplayer

# Copyright 2006-2008  Robby Workman, Northport, AL, USA
# All rights reserved.
#
# Redistribution and use of this script, with or without modification, is
# permitted provided that the following conditions are met:
#
# 1. Redistributions of this script must retain the above copyright
#    notice, this list of conditions and the following disclaimer.
#
# THIS SOFTWARE IS PROVIDED BY THE AUTHOR ''AS IS'' AND ANY EXPRESS OR IMPLIED
# WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO
# EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS;
# OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY,
# WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
# OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF
# ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

# Updated for slamd64, and to PHBuild

PRGNAM=mplayer
PKGNAM=mplayer
VERSION=${VERSION:-svn_20090413}
BUILD=${BUILD:-1}
TAG=${TAG:-_SB64}
TMP=${TMP:-/tmp/SB64}
OUTPUT=${OUTPUT:-/tmp}

. /etc/pkghelpers
pkghelpers_env

CODECSDIR=${CODECSDIR:-/usr/lib64/codecs}
RUNTIME_CPU=${RUNTIME_CPU:-yes}

if [ "$RUNTIME_CPU" = "yes" ]; then
    do_cpu="en"
else
    do_cpu="dis"
fi

# Additional languages + fonts
# Choices are: bg cs de dk el en es fr hu it ja ko mk nb
#              nl pl ro ru sk sv tr uk pt_BR zh_CN zh_TW
OSDFONTS="LiberationSans-Regular.ttf DejaVuSans.ttf Arialuni.ttf arial.ttf Vera.ttf"
LANGUAGES=${LANGUAGES:-"en"}

set -e

rm -rf $PKG
mkdir -p $TMP $PKG $OUTPUT
cd $TMP
rm -rf $PRGNAM-$VERSION
tar xvf $CWD/$PRGNAM-$VERSION.tar.bz2
cd $PRGNAM-$VERSION

# Fix permissions
pkghelpers_permissions

CFLAGS= \
./configure \
  --prefix=/usr \
  --libdir=/usr/lib$LIBSUFFIX \
  --mandir=/usr/man \
  --confdir=/etc/mplayer \
  --enable-gui \
  --${do_cpu}able-runtime-cpudetection \
  --codecsdir="$CODECSDIR" \
  --language="$LANGUAGES" \

make
make install DESTDIR=$PKG

# Now let's build the html docs
( cd DOCS/xml
  for i in $(echo $LANGUAGES | tr , ' ') ; do
    make MAKEFLAGS="-j1" html-single-$i ;
  done
)

mkdir -p $PKG/usr/doc/$PRGNAM-$VERSION
cp -a \
  AUTHORS Changelog Copyright LICENSE README DOCS/tech \
    $PKG/usr/doc/$PRGNAM-$VERSION
# This isn't exactly ideal, but it's better than having the entire
# script fail simply because the docs wouldn't build
( cp -a DOCS/HTML-single $PKG/usr/doc/$PRGNAM-$VERSION/html 2>/dev/null )
find $PKG/usr/doc/$PRGNAM-$VERSION -type d -name '.svn' | xargs rm -rf 
cat $CWD/$PRGNAM.PHBuild > $PKG/usr/doc/$PRGNAM-$VERSION/$PRGNAM.PHBuild


# Don't clobber an existing config file
cat etc/example.conf > $PKG/etc/mplayer/mplayer.conf.new

mkdir -p $PKG/install
cat $CWD/slack-desc > $PKG/install/slack-desc

# We'll link to one of the standard ttf fonts defined above.
# Thanks to Eric Hameleers for this code snippet and permission to use it.
cat << EOF >> $PKG/install/doinst.sh
# Symlink a default TrueType font for OSD:
if [ ! -f usr/share/mplayer/subfont.ttf ]; then
  for font in $OSDFONTS ; do
    if [ -f usr/share/fonts/TTF/\$font ]; then
      ( mkdir -p usr/share/mplayer
        cd usr/share/mplayer 
        ln -fs /usr/share/fonts/TTF/\$font subfont.ttf 
      )
      break
    fi
  done
fi
EOF

cd $PKG
pkghelpers_fixup
pkghelpers_makepkg

That's it. This fixed script has been tested in Slamd64 12.1 installation and works without a hitch.

Introduction to IP and ISDN

I didn't realize until just now that there is a very good introduction to IP+ISDN networks in the office web page :P. Well, not until the DNS relay/caching talks surfaced.
For those interested in the subject, this is the link:
http://www.abilis.net/fileadmin/tutorial/en/index.html

Even though the tutorial mainly talks about the Abilis product, the tutorial contains a lot of primer on IP+ISDN network architecture.

BIOS-related Post Moved to Dedicated Blog

I'm gradually moving all my BIOS-related works, rants and raves to a dedicated blog: http://bioshacking.blogspot.com. The works that were previously in geocities will be moved to that blog as well. The point is, all of those information need to be consolidated and organized for ease of access.

ATI Catalyst for Linux x86-64

Just a short notice about the working configuration for video and audio driver that works in Slamd64 12.1 with the following hardware:
*) AMD Athlon X2 4000+
*) ECS AMD690GM-M2 motherboard
*) 2GB of RAM

The working configuration as follows:

*) Kernel: a.) Linux 2.6.25.20 w/ Intel HD audio enabled + Realtek ALC880 driver module
(needed for ATI Azalia to work in the SB600 chipset).
b.) Linux 2.6.28.9 w/ Intel HD audio enabled + Realtek ALC880 driver module
(needed for ATI Azalia to work in the SB600 chipset).


*) ATI Catalyst for Linux v8.12

Note: Experiment shows that kernel 2.6.28.9 is much more suitable for Catalyst 8.12 in this particular hardware setup

Nelson Mandela

Nelson Mandela was a prominent figure of the 20th century. A lot of lesson can be learned from his amazing life. I haven't read his biography. However, some of his quotes are worth to be taken (at least) as an alternative way of seeing what life means.

“There is no passion to be found playing small - in settling for a life that is less than the one you are capable of living.” - Nelson Mandela

“I learned that courage was not the absence of fear, but the triumph over it. The brave man is not he who does not feel afraid, but he who conquers that fear.” - Nelson Mandela

“It always seems impossible until its done.” - Nelson Mandela

Refactoring "Badly Written" C Source Code for Embedded System

Ok. I'm not an expert on this "Refactoring" subject. I'm tired, sleepless but have to write this because it's very important. I'm gonna lost it on the way to bed if I don't write it now.

So, you have this C code for embedded system written by someone at some hardware company, let's call it X. It's written like someone rushing out to get the next train, and then some other guy come, add some code and some #define here and there. You're the "lucky" guy at another company, let's call the company Y. Y have bought large amount of hardware from X to customize for their specific market. Now, your manager at company Y want you to strip off all unnecessary code from the code that company X releases. In addition, your manager at your company also want you to add some "small" feature not yet exist in the C code produced by X for the specific hardware that your company bought. However, that feature exist in other product from company X which still in the same "family" as the product Y have bought. You found out that the "small" feature is controlled by conditional compilation flag, but with a twist. The flag enables the feature that you need along with some other feature that you don't need. What would you do when the C code cluttered with conditional compilation construct almost everywhere? What would you do to refactor the code so that it can become manageable and you can sleep well at night after that?

Let's see. This is what I would do:

• Take a lesson (or two) from Busybox coding style. Replace long #ifdef with
  function call. The new function enclosed in the same #ifdef and called from previous location and defined as empty function if not used (feature is disabled).
  Example:
  

  
              void func_x()
              {
              ...
              #ifdef ROUTE 
               blah = c+a;
               slez = a +z;
              #endif
              ...
              }
  

  
  transformed to:
  

  
              #ifdef ROUTE
                  void optional_init_blah()
                  {
                  blah = c+a;
                  slez = a +z;
                  }
              #else
                  void optional_init_blah(){}
              #endif // ROUTE
          
              void func_x()
              {
              ...
                  optional_init_blah();
              ...
              }
  

  
  In GCC, the optional_init_blah() function will vanish into thin air if ROUTE is not defined. This way, you can isolate the "optional" function to the header file or just place it in the top of the source file to make the code much more readable.
  


• Now, say there's a ROUTE_SUPPORT definition. However, upon examining the code, you found that you don't it in its entirety. Only its static routing that you need. What would you do? Just break the ROUTE_SUPPORT #define to STATIC_ROUTE_SUPPORT and OTHER_ROUTE_SUPPORT_THAT_YOU_DONT_NEED.
  


• Break every very long function to smaller function. Just small enough to fit in your "mental picture". Let's say, nothing more than 150-200 lines of code.
  


• If you found a "cluster" of #define that define consecutive constants and you are sure they all belong to one logical block, it's better to "lump" the constants into an enum. Otherwise, use const int instead of the #define. This way the compiler will help you check for error(s).
  


• Every time you think you grasp the idea behind a function, add a comment. I suggest using a doxygen compatible comment. That way, you can automate or at least semi-automate the documentation for the code you're working with.
  

That's it for now.
This time greetz go to Ayumi Hamasaki for the endless quiet "background sound" that keep me awake.

Ciao.

The Future of BIOS Disassembly Ninjutsu Uncovered

Now, finally I have latest update from my publisher regarding BIOS Disassembly Ninjutsu Uncovered that I wrote a while ago.

The English version of this book is now out-of-print and will not be printed for a little while until I can find new publisher who wants to acquire the right to re-print it or I might come up with another option(s).

For those who needs this book as fast as possible, please contact http://www.ipgbook.com. They might still have some books left in their inventory.

As soon as I have come up with the solution. I'll post an update in this blog.

If you're asking why it happens, it's because the Russia-based publisher A-list Publishing who publish the book is closing down their US operation due to economic pressure.


As for the Russian version of the book, it will be available as usual from www.bhv.ru.

That's it for the moment.

UPDATE
------------
I have received the contract termination for the English edition of BIOS Disassembly Ninjutsu Uncovered. Please read this post for further details.

A Glimpse of Eastern Indonesia

While many Indonesians hardly appreciate the geographical size of their own country, it's not the case for me because I come from a quite far place in the eastern part and now resides in the western part of the country. There are very small number of country in the world that spans three time zones or more. Indonesia spans three different time zones. Of course this means the country _is_ large. I cannot imagine someone who lives in Russia, which has eleven timezones! Imagine you're living in Moscow and your relative is in the far east Vladivostok.

Now, back to Indonesia. The eastern part is very sparse. A lot of islands and very small inhabitants in some of them. Anyway, these are the pictures of the island where my parents comes from.

picture from the ferry (probably on approach to the Island)

View from the mountain

View upon leaving the island (taken on the ferry)

A traditional house in the coastal region of the island

These are the surprising wind towers (I've never seen them before)

Anyway, I come to hate living in the big city quite a bit because you can hardly concentrate and too much distraction. Maybe I'll move to one of those small "isolated" island soon enough. There will be a lot of work for that to happen. But it will, sooner or later.