Developing C/C++ Software for IBM AIX (or AIX wannabe)

The following are important documentation for C/C++ developers working on IBM AIX platform. Despite IBM insistence to promote Linux in their product line, AIX and System i line of products still command the mindshare of Enterprise decision makers at the moment. So, for those (un)fortunate enough having to deal with AIX and use C/C++ environment, I found these "redbooks" to be indispensable:

1. AIX PDFs, basically everything there is to know about IBM AIX. My favorite section is the Technical Reference section though.
2. Developing and Porting C and C++ Applications on AIX. This one is what it says. DBX is surely explained in there too (for those AIX geeks out there).

Hopefully, this helps those playing around with IBM AIX at the moment. AIX is a little different than Linux or FreeBSD, but it is UNIX after all. Therefore, if your development is gunning for POSIX compliance. You should be fine most of the time.

Compiling and Using Custom Arch Linux Kernel

In my previous post, I have the need to downgrade my Arch Linux kernel version in order to alleviate ACPI bug in Linux kernel 4.X. After a bit of searching through AUR, I stumbled upon a still maintained kernel 3.X, at least until 2017. This kernel is Linux kernel 3.18. Because it resides in AUR, you must compile this kernel version yourself and install it with pacman. In this post, I will explain the process of compiling and installing custom kernel 3.18 from AUR. These are the steps I carried out to do that:

1. Clone the kernel git repository with: git clone https://aur.archlinux.org/linux-lts318.git to directory where you're going to build the kernel.
2. Add Linus Torvalds and Greg KH keys to list of trusted keys in pacman "database" of trusted keys:

   $ sudo pacman-key -r  00411886
   $ sudo pacman-key -r  6092693E
   

3. Import Linus Torvalds and Greg KH public key to your machine if you haven't done that already.

   $ gpg --recv-keys 79BE3E4300411886
   $ gpg --recv-keys 38DBBDC86092693E
   

   Be patient, as it could take a while searching and importing the keys.
4. Now, you can proceed to build the kernel in the git-cloned directory in step 1. cd into the directory and makepkg:

   $ makepkg -s
   

   The -s flag is to make sure that all dependencies are downloaded while building the custom kernel package.
5. Assuming the kernel package build is finished, you can proceed to install the kernel headers and the kernel itself:

   # pacman -U linux-lts318-headers-3.18.20-1-x86_64.pkg.tar.xz
   # pacman -U linux-lts318-3.18.20-1-x86_64.pkg.tar.xz
   

   You are advised to install kernel header first before installing the kernel as mentioned in https://wiki.archlinux.org/index.php/Kernels/Compilation/Arch_Build_System#Installing
6. Once the kernel is installed, the remaining step is to update your machine bootloader. If you are using systemd bootloader (a.k.a gummiboot), all you need to do is modify /boot/loader/entries directory to include entry for the new kernel. For example: Create a new arch-lts.conf file in /boot/loader/entries with the following contents:

   title  Arch Linux 3.18 LTS AUR
   linux  /vmlinuz-linux-lts318
   initrd  /initramfs-linux-lts318.img
   options  root=PARTUUID=[your_root_partition_UUID] rw
   

That's all you need to do to compile and install custom kernel from AUR. If you want to create your own custom kernel, you need to build your own Arch Linux package. The compilation and installation steps should be similar to what's explained above.

NOTE:

• If you have dirmngr problem showing up in your shell, it's probably because you haven't run it yet. You should run it as root:

  # dirmngr < /dev/null

  See: Pacman-key#gpg:_keyserver_receive_failed:_No_dirmngr
• See also GnuPG section in Arch Wiki for more details.
• If you'd rather prevent future kernel upgrade when upgrading packages via pacman -Syu, you can add this line to /etc/pacman.conf:

  IgnorePkg   = linux
  

  The line will prevent pacman from upgrading the linux kernel (linux package).

Makefile.am "option subdir-objects is disabled" Warning and Fix

The "option subdir-objects is disabled" warning is thrown-up by GNU autotools when Makefile.am builds a source file in a subdirectory located beneath the Makefile.am's file. This is an example warning:

.. Makefile.am:4: warning: source file '$(srcdir)/test.c' is in a subdirectory,
.. Makefile.am:4: but option 'subdir-objects' is disabled

This warning is not fatal. What it means roughly: the generated object file will not be placed inside the same directory as the source code. The full explanation is at https://www.gnu.org/software/automake/manual/html_node/List-of-Automake-options.html#List-of-Automake-options (scroll-down to subdir-objects option). This is the important excerpt:

subdir-objects

If this option is specified, then objects are placed into the subdirectory of the build directory corresponding to the subdirectory of the source file. For instance, if the source file is subdir/file.cxx, then the output file would besubdir/file.o.

Fixing this problem is not hard, you just need to add the subdir-objects option to Makefile.am. Below is an example taken from Makefile.am in one of my project. I placed the statement as the first entry in Makefile.am.

AUTOMAKE_OPTIONS = subdir-objects
### ...
### Other statements
### ...

Hopefully, this helps out those experiencing this problem.

Multithreaded Libevent/Libevent2 Server (Code Commentary)

I've been playing with libevent v2.x for a while. In the course of the "experiments", I come across Ron Cemer's  multithreaded libevent implementation (see http://roncemer.com/software-development/multi-threaded-libevent-server-example/ and http://sourceforge.net/projects/libevent-thread/) and also an updated version from Qi Huang (see http://randomindigits.blogspot.co.id/2012/11/libevent-2x-multithreaded-socket-server_4.html and https://github.com/gambellhq/samples/tree/master/libevent2-thread-code).

These multithreaded libevent code are mostly just fine except in one department, i.e. in signal handling. Both versions calls pthread functions from inside the signal handler (albeit not directly). The said practice is unsafe. It should have handle signals in synchronous pattern instead of asynchronous pattern. What I meant by this is the code should have used sigwait() in a dedicated thread to wait for signal(s) and call pthread functions from there instead of calling pthread function inside signal handler which is unsafe.

If you've been "system" programmer, you should already know that signal handler in *NIX runs in a different execution context compared to "normal" threads execution context. Therefore, you shouldn't call anything not prescribed beforehand outside of that "not-normal" execution context.

Now, let's see what exactly I mean (see echoserver_threaded.c and workqueue.c in Qi Huang code):

int runServer(void) {
    ...
    /* Set signal handlers */
    sigset_t sigset;
    sigemptyset(&sigset);
    struct sigaction siginfo = {
        .sa_handler = sighandler,
        .sa_mask = sigset,
        .sa_flags = SA_RESTART,
    };
    sigaction(SIGINT, &siginfo, NULL);
    sigaction(SIGTERM, &siginfo, NULL);
    ...
}
...
static void sighandler(int signal) {
    fprintf(stdout, "Received signal %d: %s.  Shutting down.\n", signal,
            strsignal(signal));
    killServer();
}
....
void killServer(void) {
    fprintf(stdout, "Stopping socket listener event loop.\n");
    if (event_base_loopexit(evbase_accept, NULL)) {
        perror("Error shutting down server");
    }
    fprintf(stdout, "Stopping workers.\n");
    workqueue_shutdown(&workqueue);
}
...
void workqueue_shutdown(workqueue_t *workqueue) {
    worker_t *worker = NULL;
    ...
    /* Remove all workers and jobs from the work queue.
     * wake up all workers so that they will terminate. */
    pthread_mutex_lock(&workqueue->jobs_mutex);
    ...
    pthread_cond_broadcast(&workqueue->jobs_cond);
    pthread_mutex_unlock(&workqueue->jobs_mutex);
}

As you can see in the excerpt above, the sighandler() function calls pthread functions indirectly. This practice is discouraged for sure because as per POSIX standard, pthread calls from signal handler is undefined. Therefore, it's unsafe to do so.

Anyway, feel free to challenge this view :-)