An Endless Xploration..: 2008

Tuesday, December 23, 2008

Setting-up Basic PPPoE Server in Linux (x86_64 Platform)

Setting-up a basic PPPoE server in Linux seems to be
just another server setup task which amounts to just installing
package and then configuring some scripts and you're done.
The fact is, it's probably one of the most mundane server setup task
of your life if you're not careful.

Let's start with the terms itself PPPoE stands for
Point-to-Point Protocol over Ethernet. Which means a point-to-point
connection is made over ethernet. In reality, the physical and logical
connection is not just a simple point-to-point connection but
it usually transform to multipoint-to-point connection or
multipoint-to-multipoint connection. Depending on the "server(s)"
configuration implementation. The PPPoE connection becomes multipoint-to-point
connection because usually the "server" is only one, but the "client(s)" which
connects to it is more than one and it could become multipoint-to-multipoint
when the server itself is load-balanced to more than one machine.
It's important to note that PPPoE actually doesn't recognize the notion
of client and server because PPP is a peer-to-peer protocol.
However, most people and most PPPoE software refers to
the machine dialed-into as the server and the machine
which dial as the client.

In this article I will focus on a simple PPPoE server setup which
will be used to to test very simple point-to-point connection. The point
is the server is needed to test a new pppoe client implementation and
I need to have a well-tested PPPoE server. This article dwells on how I made
the PPPoE server works as intended.

1 System Configuration

In this section, I explain the configuration I used to setup and test the
PPPoE server.

1.1 Hardware Configuration

The following is the connectivity diagram of the systems I used as the testbed.


                            +-------------------+
             +--------------| WLAN Access Point |
             |              +-------------------+
             |
     (ethernet cabling)
             |
 +------+----+------------+
 |      |  eth0           |
 |      +-----------------+
 |                        |
 | PPPoE "server" machine |                            
 |                        |
 |      +-----------------+
 |      |  eth1 (PPPoE)   |
 +------+----+------------+
             |
             |
             |              +------+-----------------+
             |              |      |  wlan0          |
     (ethernet cabling)     |      +-----------------+
             |              |                        |
             |              | PPPoE "client" machine |                            
             |              |                        |
             |              +---------------+        |
             +--------------+  eth0 (PPPoE) |        |
                            +---------------+--------+

In the diagram above, the PPPoE connection starts from the eth0 interface in
the PPPoE client machine and goes through the ethernet cabling and terminated
at the eth1 interface in the PPPoE server machine. The wireless connection
shown in the diagram is a normal wireless TCP/IP connection
I used to connect to the PPPoE server and to configure the PPPoE server.

1.2 Software Configuration

The software configuration for PPPoE is a bit complicated.
Please, follow through the explanation below very carefully.

These are the software that I used:

Operating System: Slamd64 Linux (x86_64 architecture) version 12.1.
with custom compiled linux kernel based on kernel version 2.6.25.17 source code.
These are the kernel compilation parameter that I modify to support kernel mode
PPPoE:


        CONFIG_PPP=m          
        CONFIG_PPP_ASYNC=m   
        CONFIG_PPP_SYNC_TTY=m 
        CONFIG_PPP_DEFLATE=m  
        CONFIG_PPP_BSDCOMP=m 
        CONFIG_PPPOE=m      
        CONFIG_N_HDLC=m    
        CONFIG_UNIX98_PTYS=y

To load the kernel modules automatically at boot, I modify the /etc/rc.d/rc.modules
script to include the following statements:


## Load PPP:
# This module is for PPP support:
/sbin/modprobe ppp_generic
# This PPP plugin supports PPP over serial lines:
/sbin/modprobe ppp_async
# Use this plugin instead for HDLC (used for high-speed leased lines like T1/E1)
/sbin/modprobe ppp_synctty

## This module provides compression for PPP (optional):
/sbin/modprobe ppp_deflate

## This module provides PPPoE for PPP (needed for RP-PPPoE to function):
/sbin/modprobe pppoe

## This module provides N_HDLC for PPP (needed for RP-PPPoE server to function):
/sbin/modprobe n_hdlc

pppd version 2.4.4 from the Slamd64 package in the Slamd64 DVD. I didn't do
any customization to this package. It's installed with installpkg as usual.

rp-pppoe version 3.10. I removed the original rp-pppoe 3.8 that comes by default
with Slamd64 12.1 because I couldn't get it to work. Note that rp-pppoe 3.8 that comes
with Slamd64 12.1 doesn't support kernel mode PPPoE which cause all sorts of troubles
to me. Therefore, I downloaded rp-pppoe 3.10 source code from the net and build a
package out of it with this script (rp-pppoe.PHBuild):
```
#!/bin/sh
PKGNAM=rp-pppoe
VERSION=3.10
BUILD=2

. /etc/pkghelpers
pkghelpers_env

rm -rf $PKG
mkdir -p $PKG
cd $TMP
rm -rf rp-pppoe-$VERSION

tar xzvf $CWD/rp-pppoe-$VERSION.tar.gz
cd rp-pppoe-$VERSION/src
CFLAGS=$SLKCFLAGS \
./configure --prefix=/usr --enable-plugin  
make -j12 || exit 1
make install docdir=/usr/doc/rp-pppoe-$VERSION DESTDIR=$PKG install
( cd $PKG/etc/ppp
  for config in firewall-masq firewall-standalone pppoe-server-options pppoe.conf ; do
    mv $config ${config}.new
  done
)
mkdir -p $PKG/install
zcat $CWD/doinst.sh.gz > $PKG/install/doinst.sh
cat $CWD/slack-desc > $PKG/install/slack-desc

cd $PKG
pkghelpers_fixup
pkghelpers_makepkg
        
```
The script above is a modified script from the source code of the rp-pppoe 3.8 package
you can find in the Slamd64 12.1 DVD (in the source directory).
I made a slight modification to it. I changed the VERSION to 3.10 and
added --enable-plugin as a parameter to the configure
script. This way, rp-pppoe will be built with kernel mode PPPoE support. I have tried to recompile
the rp-pppoe v3.8 that comes with Slamd64 12.1 with the --enable-plugin parameter passed
to the configure script, but I just couldn't make it work. On the other hand,
rp-pppoe v3.10 compilation works flawlessly with this parameter. The result of this build script
is a file named rp-pppoe-3.10-x86_64_slamd64-2.tgz located in /tmp directory.
The build result can be installed directly with installpkg.

The next step after the needed packages installed and the needed kernel customization complete is to
configure rp-pppoe and pppd. This can easily become an error prone process.

In this setup, the authentication used is PAP. Therefore, the following explanation is focused
for PAP-based PPPoE authentication. For CHAP and EAP* authentication please refer to other
relevant articles on the net.

Let's proceed to configure the machines one by one.

For the pppoe server machine, the important configuration files are:
1. /etc/ppp/options.
```
asyncmap 0
crtscts
lock
modem
proxyarp
lcp-echo-interval 30
lcp-echo-failure 4
                
```
  This file stores the default options used
  by pppd when it starts.
2. /etc/ppp/pppoe-server-options.
```
debug
plugin /etc/ppp/plugins/rp-pppoe.so
require-pap
mtu 1492
mru 1492
ktune
proxyarp
lcp-echo-interval 10
lcp-echo-failure 2
nobsdcomp
noccp
novj
noipx
                
```
  This file stores the parameters to be
  passed to pppd when pppoe-server starts.
  
  You can override this pppoe-server configuration file
  by passing the -O parameter to pppoe-server
  when you invoke pppoe-server.
  
  Note the:
```
plugin /etc/ppp/plugins/rp-pppoe.so
```
  This instructs the pppoe server to load the kernel mode pppoe plugin
  when the it runs.
3. /etc/ppp/pap-secrets
```
# Secrets for authentication using PAP
# client        server  secret                  IP addresses
indosatm2       *       prabayar                *
zuruk           *       weiter                  *
                
```
  This file stores the PAP username and password to be used for PPP authentication.
  Note:
  
  The * in the fourth column is mandatory if you want to be able to let
  the client authenticate itself from any IP address.
  It's also important to note that if you fail to give a *
  in the fourth column in all of the lines/entries prior to the valid client entry,
  the authentication will fail.
  For example, once, I forgot to give a * in the last (the fourth) column
  of the indosatm2 client entry and what happens is the client which authenticate itself
  as user zuruk (in the second line) will always fail to authenticate with an
  "LCP Conf-Request timeout" error message even if the secret is right and the allowed IP address in
  the fourth column for this entry has been set to *.
  Another way to overcome the problem is to place the user zuruk entry
  as the first entry (i.e. in the first line) in /etc/ppp/pap-secrets
  and giving a * in the fourth column of this entry.
  Therefore, eliminating the need to give a * in the fourth column of
  other entries in /etc/ppp/pap-secrets. This is the related excerpt from pppd manual page:
```
...
If there are only 3 words on the line, or if the first word is "-", then all
IP  addresses  are  disallowed.  To allow any address, use "*"  
...
                
```

For the pppoe client machine, the important configuration files are:

/etc/ppp/options


asyncmap 0
crtscts
lock
modem
proxyarp
lcp-echo-interval 30
lcp-echo-failure 4

This file stores the default options used
by pppd when it starts.

/etc/ppp/pppoe.conf


ETH=eth1
USER=zuruk
DEMAND=no
DNSTYPE=SERVER
PEERDNS=yes
DNS1=
DNS2=
DEFAULTROUTE=yes
CONNECT_TIMEOUT=30
CONNECT_POLL=2
ACNAME=
SERVICENAME=
PING="."
CF_BASE=`basename $CONFIG`
PIDFILE="/var/run/$CF_BASE-pppoe.pid"
SYNCHRONOUS=no
CLAMPMSS=1412
LCP_INTERVAL=20
LCP_FAILURE=3
PPPOE_TIMEOUT=80
FIREWALL=NONE
LINUX_PLUGIN=/etc/ppp/plugins/rp-pppoe.so
PPPOE_EXTRA=""
PPPD_EXTRA=""

This file stores the default configuration for the pppoe client scripts.
It's parsed by pppoe-start, pppoe-stop and other pppoe client scripts
when they run. Note the:


LINUX_PLUGIN=/etc/ppp/plugins/rp-pppoe.so

This instructs the pppoe client scripts to load the kernel mode pppoe plugin
when the script runs.

/etc/ppp/pap-secrets


# Secrets for authentication using PAP
# client        server  secret                  IP addresses
zuruk           *       weiter                  *

This file stores the PAP username and password to be used for PPP authentication.

It is important to remember to add the option to load the kernel mode pppoe plugin to
both the pppoe-server and client configuration files. Otherwise the kernel mode pppoe plugin
wouldn't be loaded and the PPPoE connection cannot established.

2 Running The PPPoE Server and Client

2.1 Configuring The Ethernet Interfaces

The ethernet should be without any IP address but it should be up.
You can do that with the following command.


root@opusera:~# ifconfig eth1 down 
root@opusera:~# ifconfig eth1 0.0.0.0 
root@opusera:~# ifconfig eth1 up

or in Slamd64 you can comment-out the IP setting in /etc/rc.d/rc.inet1.conf
like so:


# Config information for eth1:
#IPADDR[1]="192.168.2.1"
#NETMASK[1]="255.255.255.0"
#USE_DHCP[1]=""
#DHCP_HOSTNAME[1]=""
# Config information for eth1 (for PPPoE experiments)  :
USE_DHCP[1]=""
DHCP_HOSTNAME[1]=""

The /etc/rc.d/rc.inet1.conf above will make the interface remains down
after boot completed and will prevent the system from assigning an IP address to
the interface. Therefore, if you modify /etc/rc.d/rc.inet1.conf as shown
above, you have to bring the interface up manually before running any pppoe application(s)
with:


root@opusera:~# ifconfig eth1 up

because the interface is still inactive after boot.

2.2 Running The PPPoE Server and Client Application

To execute the pppoe-client application, invoke the pppoe-start script
and pass the ethernet interface name as the first parameter and the username you wish
to use as the second parameter. For example:


root@opusera:~# pppoe-start eth1 zuruk 
.. Connected!

In the pppoe-start invocation above, "eth1" is the ethernet interface to be used for PPPoE and
"zuruk" is the username.

You can run the pppoe-server application with the following command.


root@opunaga:darmawan # pppoe-server -k -I eth0 -F

The -k parameter is mandatory because we want to use the kernel mode pppoe plugin.
The -I eth0 parameter is used to inform pppoe-server to use eth0 as the interface
for PPPoE connection.
The -F parameter instructs pppoe-server to stay in the foreground because in normal
pppoe-server invocation (without -F), it will proceed to run as background process.

3 How to Debug and Fix Connectivity Problems

It's quite hard to get PPPoE connectivity works in the first try in the current state
of PPPoE support in Linux. Therefore, debugging the connectivity problem is important.
There are two application that can help debugging a lot, i.e. tcpdump and
the syslog facility. In Slamd64, the syslog facility can be used with the following command:


root@opunaga:~ # tail -f /var/log/messages

Now, let me start with the scenario of the failed authentication that made me lack of sleep.
In this scenario, the /etc/ppp/pap-secrets file in the machine that I use
as the pppoe-server is wrong. This is the contents of the file:


# Secrets for authentication using PAP
# client        server  secret                  IP addresses
indosatm2       *       prabayar                
zuruk           *       weiter                  *

From section 1.2, it's clear that the user "zuruk" must be placed as the first entry or
the "indosatm2" user should be given a * in the IP addresses column. But, this is not the
case, it will cause an "LCP Conf-Request timeout" as you will see shortly.

Now, let's see what tcpdump shows in the pppoe-server machine.


root@opunaga:darmawan # tcpdump -i eth0  -X                                                                                                                                                                     
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
18:33:08.234708 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:11.235647 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:14.236627 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:17.237613 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:20.238598 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:23.239584 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:33:26.240570 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 015e 111e 0000 0000 0000 0000 0000  ...^............
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

and let's proceed to see what is logged by the syslog.


root@opunaga:darmawan # tail -f /var/log/messages
Dec 23 18:32:59 opunaga pppoe-server[3380]: Session 1 created for client 00:0a:eb:28:62:57 (10.67.15.1) on eth0 using Service-Name ''
Dec 23 18:32:59 opunaga pppd[3380]: Plugin /etc/ppp/plugins/rp-pppoe.so loaded.
Dec 23 18:32:59 opunaga pppd[3380]: RP-PPPoE plugin version 3.10 compiled against pppd 2.4.4
Dec 23 18:32:59 opunaga pppd[3380]: Plugin /etc/ppp/plugins/rp-pppoe.so loaded.
Dec 23 18:32:59 opunaga pppd[3380]: RP-PPPoE plugin version 3.10 compiled against pppd 2.4.4
Dec 23 18:33:07 opunaga kernel: device eth0 entered promiscuous mode
Dec 23 18:33:42 opunaga kernel: device eth0 left promiscuous mode

As you can see, syslog shows nothing wrong with the pppoe-server because the kernel mode pppoe plugin
is loaded just fine, as expected. However, tcpdump shows there are "LCP Conf-Request"s which are not
serviced as they should, causing a timeout in the pppoe-client machine as shown below.


root@opusera:~# pppoe-start eth1 zuruk    
................TIMED OUT
/usr/sbin/pppoe-start: line 193:  2851 Terminated              $CONNECT "$@" >/dev/null 2>&1

Then, let me fix the PAP authentication file and let's see what tcpdump and syslog shows.
The fixed /etc/ppp/pap-secrets file is:


# Secrets for authentication using PAP
# client        server  secret                  IP addresses
zuruk           *       weiter                  *
indosatm2       *       prabayar

In the fixed pap-secrets, user "zuruk" has been moved to the top-most place, i.e.
as the first entry. Now, run the pppoe-server:


root@opunaga:darmawan # pppoe-server -k -I eth0 -F

then, run the pppoe client in the client machine:


root@opusera:~# pppoe-start eth1 zuruk
.. Connected!

meanwhile, also invoke tcpdump:


root@opunaga:darmawan # tcpdump -i eth0  -X                                                                                                                                                                     
tcpdump: WARNING: eth0: no IPv4 address assigned
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 96 bytes
18:53:23.303249 PPPoE PADI [Service-Name] [Host-Uniq 0x7B0C0000]
        0x0000:  1109 0000 000c 0101 0000 0103 0004 7b0c  ..............{.
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:23.303537 PPPoE PADO [AC-Name "opunaga"] [Service-Name] [AC-Cookie 0xECAA58162AC4B0B4F35E25BD7A7C07002B0E0000] [Host-Uniq 0x7B0C0000]
        0x0000:  1107 0000 002f 0102 0007 6f70 756e 6167  ...../....opunag
        0x0010:  6101 0100 0001 0400 14ec aa58 162a c4b0  a..........X.*..
        0x0020:  b4f3 5e25 bd7a 7c07 002b 0e00 0001 0300  ..^%.z|..+......
        0x0030:  047b 0c00 00                             .{...
18:53:23.303618 PPPoE PADR [Service-Name] [Host-Uniq 0x7B0C0000] [AC-Cookie 0xECAA58162AC4B0B4F35E25BD7A7C07002B0E0000]
        0x0000:  1119 0000 0024 0101 0000 0103 0004 7b0c  .....$........{.
        0x0010:  0000 0104 0014 ecaa 5816 2ac4 b0b4 f35e  ........X.*....^
        0x0020:  25bd 7a7c 0700 2b0e 0000 0000 0000       %.z|..+.......
18:53:23.304159 PPPoE PADS [ses 0x1] [Service-Name] [Host-Uniq 0x7B0C0000]
        0x0000:  1165 0001 000c 0101 0000 0103 0004 7b0c  .e............{.
        0x0010:  0000                                     ..
18:53:23.305978 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 a9d3 c102 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:23.316172 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 20
        0x0000:  1100 0001 0014 c021 0101 0012 0104 05d4  .......!........
        0x0010:  0304 c023 0506 ac24 3a51                 ...#...$:Q
18:53:23.316274 PPPoE  [ses 0x1] LCP, Conf-Ack (0x02), id 1, length 20
        0x0000:  1100 0001 0014 c021 0201 0012 0104 05d4  .......!........
        0x0010:  0304 c023 0506 ac24 3a51 0000 0000 0000  ...#...$:Q......
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.307176 PPPoE  [ses 0x1] LCP, Conf-Request (0x01), id 1, length 16
        0x0000:  1100 0001 0010 c021 0101 000e 0104 05d4  .......!........
        0x0010:  0506 a9d3 c102 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.307547 PPPoE  [ses 0x1] LCP, Conf-Ack (0x02), id 1, length 16
        0x0000:  1100 0001 0010 c021 0201 000e 0104 05d4  .......!........
        0x0010:  0506 a9d3 c102                           ......
18:53:26.307649 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 0, length 10
        0x0000:  1100 0001 000a c021 0900 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.307687 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 0, length 10
        0x0000:  1100 0001 000a c021 0900 0008 ac24 3a51  .......!.....$:Q
18:53:26.307714 PPPoE  [ses 0x1] PAP, Auth-Req (0x01), id 1, Peer zuruk, Name weiter
        0x0000:  1100 0001 0013 c023 0101 0011 057a 7572  .......#.....zur
        0x0010:  756b 0677 6569 7465 7200 0000 0000 0000  uk.weiter.......
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.307754 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 0, length 10
        0x0000:  1100 0001 000a c021 0a00 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.307842 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 0, length 10
        0x0000:  1100 0001 000a c021 0a00 0008 ac24 3a51  .......!.....$:Q
18:53:26.308055 PPPoE  [ses 0x1] PAP, Auth-ACK (0x02), id 1, Msg Login ok
        0x0000:  1100 0001 000f c023 0201 000d 084c 6f67  .......#.....Log
        0x0010:  696e 206f 6b                             in.ok
18:53:26.308211 PPPoE  [ses 0x1] IPCP, Conf-Request (0x01), id 1, length 24
        0x0000:  1100 0001 0018 8021 0101 0016 0306 0000  .......!........
        0x0010:  0000 8106 0000 0000 8306 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.308333 PPPoE  [ses 0x1] IPCP, Conf-Request (0x01), id 1, length 12
        0x0000:  1100 0001 000c 8021 0101 000a 0306 0a00  .......!........
        0x0010:  0001                                     ..
18:53:26.308402 PPPoE  [ses 0x1] IPCP, Conf-Ack (0x02), id 1, length 12
        0x0000:  1100 0001 000c 8021 0201 000a 0306 0a00  .......!........
        0x0010:  0001 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.309598 PPPoE  [ses 0x1] IPCP, Conf-Reject (0x04), id 1, length 18
        0x0000:  1100 0001 0012 8021 0401 0010 8106 0000  .......!........
        0x0010:  0000 8306 0000 0000                      ........
18:53:26.309667 PPPoE  [ses 0x1] IPCP, Conf-Request (0x01), id 2, length 12
        0x0000:  1100 0001 000c 8021 0102 000a 0306 0000  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.309766 PPPoE  [ses 0x1] IPCP, Conf-Nack (0x03), id 2, length 12
        0x0000:  1100 0001 000c 8021 0302 000a 0306 0a43  .......!.......C
        0x0010:  0f01                                     ..
18:53:26.309829 PPPoE  [ses 0x1] IPCP, Conf-Request (0x01), id 3, length 12
        0x0000:  1100 0001 000c 8021 0103 000a 0306 0a43  .......!.......C
        0x0010:  0f01 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:26.310369 PPPoE  [ses 0x1] IPCP, Conf-Ack (0x02), id 3, length 12
        0x0000:  1100 0001 000c 8021 0203 000a 0306 0a43  .......!.......C
        0x0010:  0f01                                     ..
18:53:36.308498 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 1, length 10
        0x0000:  1100 0001 000a c021 0901 0008 ac24 3a51  .......!.....$:Q
18:53:36.308594 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 1, length 10
        0x0000:  1100 0001 000a c021 0a01 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:46.307816 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 1, length 10
        0x0000:  1100 0001 000a c021 0901 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:46.308018 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 1, length 10
        0x0000:  1100 0001 000a c021 0a01 0008 ac24 3a51  .......!.....$:Q
18:53:46.309479 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 2, length 10
        0x0000:  1100 0001 000a c021 0902 0008 ac24 3a51  .......!.....$:Q
18:53:46.309553 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 2, length 10
        0x0000:  1100 0001 000a c021 0a02 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
18:53:56.309515 PPPoE  [ses 0x1] LCP, Echo-Request (0x09), id 3, length 10
        0x0000:  1100 0001 000a c021 0903 0008 ac24 3a51  .......!.....$:Q
18:53:56.309598 PPPoE  [ses 0x1] LCP, Echo-Reply (0x0a), id 3, length 10
        0x0000:  1100 0001 000a c021 0a03 0008 a9d3 c102  .......!........
        0x0010:  0000 0000 0000 0000 0000 0000 0000 0000  ................
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

and also invoke syslog utility:


root@opunaga:darmawan # tail -f /var/log/messages                                                                                                 
Dec 23 18:53:23 opunaga pppoe-server[3628]: Session 1 created for client 00:0a:eb:28:62:57 (10.67.15.1) on eth0 using Service-Name ''
Dec 23 18:53:23 opunaga pppd[3628]: Plugin /etc/ppp/plugins/rp-pppoe.so loaded.
Dec 23 18:53:23 opunaga pppd[3628]: RP-PPPoE plugin version 3.10 compiled against pppd 2.4.4
Dec 23 18:53:23 opunaga pppd[3628]: Plugin /etc/ppp/plugins/rp-pppoe.so loaded.
Dec 23 18:53:23 opunaga pppd[3628]: RP-PPPoE plugin version 3.10 compiled against pppd 2.4.4
Dec 23 18:53:23 opunaga pppd[3628]: pppd 2.4.4 started by root, uid 0
Dec 23 18:53:23 opunaga pppd[3628]: Using interface ppp0
Dec 23 18:53:23 opunaga pppd[3628]: Connect: ppp0 <--> eth0
Dec 23 18:53:26 opunaga pppd[3628]: PAP peer authentication succeeded for zuruk
Dec 23 18:53:26 opunaga pppd[3628]: peer from calling number 00:0A:EB:28:62:57 authorized
Dec 23 18:53:26 opunaga pppd[3628]: local  IP address 10.0.0.1
Dec 23 18:53:26 opunaga pppd[3628]: remote IP address 10.67.15.1
Dec 23 18:53:56 opunaga kernel: device eth0 left promiscuous mode

As you can see clearly, the "LCP Conf-Request" packets are replied with the right
"LCP Conf-Ack" packets in the tcpdump output and the PPPoE configuration sequence proceed to IP
setup stage. In the syslog output, you can see the ppp0 interface created
and given the appropriate IP addresses. This means the PPPoE connection has been created
successfully.

Note:

The "LCP Conf-Request" timeout symptoms are not particularly related to wrong
ppp secrets file (either /etc/ppp/pap-secrets or /etc/ppp/chap-secrets).
It can be also related to erratic user mode rp-pppoe plugin as described in numerous
mailing lists, bug reports and forums on the net. The particular case that
I describe in this article happens after I tried the "supposedly" working
kernel mode pppoe plugin. I found that the kernel mode plugin indeed works after
finding out my mistake in the /etc/ppp/pap-secrets configuration.
You have to be aware of these issues in order not to repeat the same mistake.

X. Closing

Greetz go to Armin van Buuren for the great composition which
made the debugging much less painful in the last 48 hours.

Thursday, December 11, 2008

Nokia E61 Bluetooth 3G/GPRS Connection With PPPD in Linux

Using pppd to connect using bluetooth 3G/GPRS in Linux is quite a challenge for
ordinary Linux user. In this post, I explain how I got it working on my system.

This is my system configuration:

OS: Slamd64 12.1. An x86_64 Unofficial Port of Slackware Linux Distribution
with kernel 2.6.25.17 and pppd version 2.4.4

A Turion64 Laptop with 1GB RAM.

Broadcom BCM2045A Bluetooth USB adapter.

Nokia E61 acting as a modem.

As a test case in configuring the system, I will try to connect to the network
for UIM registration purposes.

The relevant configuration files are:

/etc/ppp/chap-secrets

/etc/ppp/pap-secrets

/etc/ppp/options

/etc/ppp/peers/_your_ppp_extra_config_

/etc/ppp/_your_chat_script_

/etc/ppp/ip-up

/etc/ppp/ip-down

The chap-secrets and pap-secrets contains the username and password to
authenticate yourself to the ppp peer/server. It has to be modified to suit your need.
The options file contains the default options for all connections which will
be made by pppd. You should place your connection specific options in a custom file
in /etc/ppp/peers directory and invoke it with:


pppd call _your_ppp_extra_config_

This will initiate the ppp connection to the peer.
You will need the chat script to talk to your modem. The ip-up and ip-down
scripts are optional. I am using it to update the nameserver configuration after the
IP connection through ppp established.

These are the options configured in my /etc/ppp/options file:


asyncmap 0
crtscts
lock
modem
proxyarp
lcp-echo-interval 30
lcp-echo-failure 4

Opening the pppd man pages should help you to understand those options.
Now, my ppp extra configuration file is /etc/ppp/peers/im2.
I'm using the following command to dial into my 3G/GPRS ISP using this file
(dialing should be done as root):


pppd call im2

The contents of /etc/ppp/peers/im2 as follows:


/dev/rfcomm0 crtscts 115200 
connect 'chat -v -f /etc/ppp/chat/im2_chat'
defaultroute name indosatm2 noipdefault usepeerdns idle 0

Actually, this file contains commands and options passed to pppd when I call it
in the shell dump shown above. This is the breakdown:

/dev/rfcomm0 is the modem I'm using in this ppp session.

crtscts means use hardware flowcontrol in this ppp session.

115200 is the baud rate at which I'm connecting.

115200 is the baud rate at which I'm connecting.

The connect 'chat -v -f /etc/ppp/chat/im2_chat'
means invoke the chat program with the /etc/ppp/chat/im2_chat
as the chat file. The chat file is used to initialize the modem and
waiting for connection string from the ISP.

defaultroute means add a default route to the system routing tables
when the ppp connection established.

name indosatm2 directs pppd to find an entry in either /etc/ppp/chap-secrets
or /etc/ppp/pap-secrets which corresponds to indosatm2 as the login/user
name for the connection.

noipdefault enforce the peer/server to supply the IP address during IPCP negotiation when pppd
tries to establish the connection.

usepeerdns asks the peer for upto two DNS server addresses.

idle 0 is an optional parameter which means pppd should disconnect if the ppp link is idle for the
requested amount of time (in seconds).

At this point the file used to dial the ppp server is clear. To check the current status while you are dialling
the ppp server, you can use:


tail -f /var/log/messages

and then checking your network interface when the IP connection has been established with ifconfig.
If ifconfig shows something like:


root@konoha:~ # ifconfig                                                                                                                                                        
ppp0      Link encap:Point-to-Point Protocol  
          inet addr:114.58.67.148  P-t-P:10.6.6.6  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:229 errors:0 dropped:0 overruns:0 frame:0
          TX packets:229 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:271660 (265.2 KiB)  TX bytes:13892 (13.5 KiB)

Then the ppp connection has been established correctly.
Now, let's move to the chat script.
Read the chat man pages for the details. This is my chat script (/etc/ppp/chat/im2_chat).


ECHO ON
ABORT 'NO CARRIER'
ABORT 'NO DIALTONE'
ABORT 'ERROR'
ABORT 'NO ANSWER'
ABORT 'BUSY'
'' ATZ
OK 'ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0'
OK 'AT+IFC=2,2;+CVHU=1'
OK 'ATS7=60+DS=3,0;&K3'
OK 'AT+CGDCONT=1,"IP","indosatm2"'
OK 'ATS0=0'
OK 'ATDT*99#'
~

As you can see, Nokia E61 when used as modem requires quite a lot of initialization strings.
Again, read the chat man pages for the details.

The secrets files are shown below. I provided them here because the login is a universal way
to register the UIM of any user of the Indosat IM2 broom user which pose no security risks to me.
This is the /etc/ppp/chap-secrets file:


# Secrets for authentication using CHAP
# client        server  secret                  IP addresses
indosatm2       *       prabayar

and this is the /etc/ppp/pap-secrets file:


# Secrets for authentication using PAP
# client        server  secret                  IP addresses
indosatm2       *       prabayar

That's it, the secrets file needed to login to register an Indosat IM2 broom 3G/GPRS package.
Note that indosatm2 is the login and prabayar is the password.

I have added new (previously non-existent in default pppd installation) ip-up and ip-down
scripts to handle the name resolution chores once the IP connection through the ppp interface
established. This is the /etc/ppp/ip-up script:


#!/bin/sh
#
# Update /etc/resolv.conf pppd generated file.
# Backup the old /etc/resolv.conf
#

RESOLV_BACKUP="/etc/old_resolv.conf"
RESOLV="/etc/resolv.conf"
PPP_RESOLV="/etc/ppp/resolv.conf"

cp -v ${RESOLV}  ${RESOLV_BACKUP}
cp -vf ${PPP_RESOLV}    ${RESOLV}

unset RESOLV_BACKUP
unset RESOLV
unset PPP_RESOLV

The script above is pretty simple bash script. I think I don't need to explain it.
The /etc/ppp/ip-down script as follows:


#!/bin/sh
#
# Restore /etc/resolv.conf 
#

RESOLV_BACKUP="/etc/old_resolv.conf"
RESOLV="/etc/resolv.conf"

cp -vf ${RESOLV_BACKUP} ${RESOLV}
rm -vf ${RESOLV_BACKUP}

unset RESOLV_BACKUP
unset RESOLV

Note that the variable values in both ip-up and ip-down scripts have to match each other,
otherwise you are deleting the wrong file(s) or non-existent file. It can be dangerous
because these scripts run under root privilege.

Now, let's try to connect to the ISP (Indosat IM2) to register the UIM.


pppd call im2

This is the log in /var/log/messages during the connection establishment:


root@opunaga: # tail -f /var/log/messages
Dec 11 12:09:59 opunaga pppd[6867]: pppd 2.4.4 started by root, uid 0
Dec 11 12:10:00 opunaga hcid[2000]: link_key_request (sba=4E:89:44:0C:46:14, dba=00:12:D1:85:E8:8F)
Dec 11 12:10:01 opunaga chat[6870]: abort on (NO CARRIER)
Dec 11 12:10:01 opunaga chat[6870]: abort on (NO DIALTONE)
Dec 11 12:10:01 opunaga chat[6870]: abort on (ERROR)
Dec 11 12:10:01 opunaga chat[6870]: abort on (NO ANSWER)
Dec 11 12:10:01 opunaga chat[6870]: abort on (BUSY)
Dec 11 12:10:01 opunaga chat[6870]: send (ATZ^M)
Dec 11 12:10:01 opunaga chat[6870]: expect (OK)
Dec 11 12:10:01 opunaga chat[6870]: ATZ^M^M
Dec 11 12:10:01 opunaga chat[6870]: OK
Dec 11 12:10:01 opunaga chat[6870]:  -- got it 
Dec 11 12:10:01 opunaga chat[6870]: send (ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0^M)
Dec 11 12:10:01 opunaga chat[6870]: expect (OK)
Dec 11 12:10:01 opunaga chat[6870]: ^M
Dec 11 12:10:01 opunaga chat[6870]: ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0^M^M
Dec 11 12:10:01 opunaga chat[6870]: OK
Dec 11 12:10:01 opunaga chat[6870]:  -- got it 
Dec 11 12:10:01 opunaga chat[6870]: send (AT+IFC=2,2;+CVHU=1^M)
Dec 11 12:10:01 opunaga chat[6870]: expect (OK)
Dec 11 12:10:01 opunaga chat[6870]: ^M
Dec 11 12:10:01 opunaga chat[6870]: AT+IFC=2,2;+CVHU=1^M^M
Dec 11 12:10:01 opunaga chat[6870]: OK
Dec 11 12:10:01 opunaga chat[6870]:  -- got it 
Dec 11 12:10:01 opunaga chat[6870]: send (ATS7=60+DS=3,0;&K3^M)
Dec 11 12:10:01 opunaga chat[6870]: expect (OK)
Dec 11 12:10:01 opunaga chat[6870]: ^M
Dec 11 12:10:01 opunaga chat[6870]: ATS7=60+DS=3,0;&K3^M^M
Dec 11 12:10:01 opunaga chat[6870]: OK
Dec 11 12:10:01 opunaga chat[6870]:  -- got it 
Dec 11 12:10:01 opunaga chat[6870]: send (AT+CGDCONT=1,"IP","indosatm2"^M)
Dec 11 12:10:02 opunaga chat[6870]: expect (OK)
Dec 11 12:10:02 opunaga chat[6870]: ^M
Dec 11 12:10:02 opunaga chat[6870]: AT+CGDCONT=1,"IP","indosatm2"^M^M
Dec 11 12:10:02 opunaga chat[6870]: OK
Dec 11 12:10:02 opunaga chat[6870]:  -- got it 
Dec 11 12:10:02 opunaga chat[6870]: send (ATS0=0^M)
Dec 11 12:10:02 opunaga chat[6870]: expect (OK)
Dec 11 12:10:02 opunaga chat[6870]: ^M
Dec 11 12:10:02 opunaga chat[6870]: ATS0=0^M^M
Dec 11 12:10:02 opunaga chat[6870]: OK
Dec 11 12:10:02 opunaga chat[6870]:  -- got it 
Dec 11 12:10:02 opunaga chat[6870]: send (ATDT*99#^M)
Dec 11 12:10:02 opunaga chat[6870]: expect (~)
Dec 11 12:10:02 opunaga chat[6870]: ^M
Dec 11 12:10:11 opunaga chat[6870]: ATDT*99#^M^M
Dec 11 12:10:11 opunaga chat[6870]: CONNECT^M
Dec 11 12:10:11 opunaga chat[6870]: ~
Dec 11 12:10:11 opunaga chat[6870]:  -- got it 
Dec 11 12:10:11 opunaga pppd[6867]: Serial connection established.
Dec 11 12:10:11 opunaga pppd[6867]: Using interface ppp0
Dec 11 12:10:11 opunaga pppd[6867]: Connect: ppp0 <--> /dev/rfcomm0
Dec 11 12:10:12 opunaga pppd[6867]: PAP authentication succeeded
Dec 11 12:10:16 opunaga pppd[6867]: local  IP address 192.168.28.189
Dec 11 12:10:16 opunaga pppd[6867]: remote IP address 10.6.6.6
Dec 11 12:10:16 opunaga pppd[6867]: primary   DNS address 202.155.47.130
Dec 11 12:10:16 opunaga pppd[6867]: secondary DNS address 202.155.0.10

and let's see our ppp interface:


root@opunaga:darmawan # ifconfig     
ppp0      Link encap:Point-to-Point Protocol  
          inet addr:192.168.28.189  P-t-P:10.6.6.6  Mask:255.255.255.255
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:24 errors:0 dropped:0 overruns:0 frame:0
          TX packets:29 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:3 
          RX bytes:5902 (5.7 KiB)  TX bytes:2061 (2.0 KiB)

Now, we have everything working as expected ;).

That's it for the moment. I hope you can setup your 3G/GPRS connection with pppd as well.
I've come to not using wvdial for sometime because I prefer using pppd directly which I think more convenient.

Saturday, December 6, 2008

Controlling FSCK in Linux

It's annoying to have filesystem checks every once in a while when you boot your linux machine. I mean in the case everything is just fine, why run fsck? Now, how can you control the amount of mount before an fsck takes place? It's easy, if you are using Ext2 or Ext3 filesystem, just run tune2fs -c . This is the command:


me@machine $ tune2fs -c <number_of_mounts_before_fsck>

That's it. Just replace number_of_mounts_before_fsck with the value of your liking and it should be done.

Tuesday, December 2, 2008

Using find Utility Effectively in *NIX

Sometimes you need to find files with certain criterion as fast as possible. For example in task where you need to strip out files bigger than a predefined size. Say 50KB. This can be done efficiently using the find utility in *NIX with the following command:

 
darmawan@opunaga:AP $ find rootfs/bin -type f -size +50k -exec ls -lah \{} \;

The command above assumes you are searching in rootfs/bin directory. Note the backslashes in the command, they are used to escape the subsequent character(s) after them in order to prevent the shell from interpreting them.

Another rather often used combination is find, grep and awk. The following example shows how to find certain string and ignoring paths with .svn directory in it.

 
darmawan@opunaga:AP $ find rootfs -type f  -exec grep -H -n 'iwcontrol' \{} \; | awk '$0 !~ /\.svn/ {print}'

I think long commands like above should be scripted for ease of use in the long-run.

Friday, November 28, 2008

Misc. Awk Tips

Few quirks about awk that I found:

1. The $0 in expression part of awk invocation such as in


pinczakko@opunaga $ find . -type f | awk '$0 !~ /\<svn\>/  {print}'

doesn't represent any of the field in a record (a line) in awk terminology. $0 represents the whole record (the whole line). Therefore, if you want to match an expression to the entire line, $0 is the way to do that.

2. To match the whole word in awk, use the escaped angle brackets. Escaped angle bracket in awk is \<\> because backslash is the escape character in an awk expression.


pinczakko@opunaga $ find . -type f | awk '$0 !~ /\<svn\>/  {print}'

The invocation above asks awk to print any lines which don't have the word svn in it.

Thursday, November 27, 2008

Removing Duplicated File Automatically

It's annoying to have two or more of the same file laying around in your hard drive. Out of curiosity, I devised a small script to automate the process of finding and deleting duplicated file in two different directories. This is it.


#!/bin/bash
#
# This script checks whether there is a file duplication 
# in two different directrories and deletes the one in 
# ${SRC_DIR} if it found the same file
#

SRC_DIR="/home/darmawan/download"
DST_DIR="/home/sources"
TMP_FILENAMES="__filenames.txt"
CUR_FILE=""
DST_FILE=""
RESULT=""

find ${SRC_DIR} -type f  > ${TMP_FILENAMES} 

while read LINE
do
 unset RESULT
 unset CUR_FILE

 CUR_FILE=$(basename "${LINE}")
 #find ${DST_DIR} -type f -name "${CUR_FILE}" -exec diff -q "${LINE}" '{}' ';'
 RESULT=$(find ${DST_DIR} -type f -name "${CUR_FILE}" -print)  
 
 if [ "${RESULT}" != "" ] ; then 
      echo "File of the same name found at ${LINE} and ${RESULT}" ;
      echo "Diffing.."
      diff -q ${LINE} ${RESULT}

      # If the file differ (diff return value _is_not_0_ ),
      # then print it out, otherwise delete the file in ${SRC_DIR}
      if [ $? -eq 0 ]; then 
       rm -vf ${LINE}
      fi
 fi

done < ${TMP_FILENAMES}

unset LINE
unset FILENAME
unset CUR_FILE
unset DST_FILE
unset RESULT

rm -v ${TMP_FILENAMES}

This is a very rough script. Don't expect robustness out of it.

Thursday, November 20, 2008

Koneksi IM2 Broom di Linux

Hari ini gw nyobain Indosat Broom untuk pertama kali. Gw pikir awalnya bakal sedikit susah buat aktivasi, ternyata cuma sedikit lebih kompleks daripada bikin account email gratisan.

Konfigurasi sistem:
1. Linux Slamd64 12.1
2. Wvdial 1.60
3. AMD Turion64
4. RAM 1GB
5. Nokia E61.

Step garis besarnya:

a. Insert USIM dari starter-pack broom ke handphone/modem yang akan digunakan.

b. Dial ke situs registrasi broom menggunakan username dan password universal buat registrasi. Ini bisa dilakukan langsung. Jadi, pada dasarnya begitu dapat USIM (a.k.a SIM CARD) langsung bisa terhubung ke network registrasi Indosat M2.

c. Isi form registrasi dengan benar, kemudian di submit. Ntar bakal muncul jawaban aktivasi telah selesai.

d. Konfigurasi ulang username dan password sesuai dengan form registrasi.

e. Silahkan browsing, ftp, blogging, rss-feed, etc. sesuai kemauan anda ke semua destinasi.

Ok. Sekarang detailnya.

Step a: Ga perlu diilustrasikan coz sangat gampang.

Step b: Berhubung gw pake wvdial + Linux Slamd64, ini dia wvdial.conf yang gw pake


[Dialer im2_reg]
Init1 = ATZ
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Init3 = AT+IFC=2,2;+CVHU=1
Init4 = ATS7=60+DS=3,0;&K3
Init5 = AT+CGDCONT=1,"IP","indosatm2"
Init6 = ATS0=0
Phone = *99#
Username = indosatm2
Password = prabayar 
Modem Type = USB Modem
Baud = 115200
New PPPD = yes
Dial Command = ATDT
Modem = /dev/rfcomm0
ISDN = 0

Pake wvdial.conf ini gw dial ke situs registrasi im2


root@opunaga # wvdial im2_reg

Step c: Buat registrasi, pake browser gw buka http://www.indosatm2.com/aktivasi. Ini situs registrasinya, trus isi form registrasi. Di form ini ada bagian untuk ngisi username + password yang akan kita pake browsing dan tipe servis Broom yang mau kita pake. Gw milih yg unlimited. Lumayan 100rb sebulan unlimited.

Step d: Tinggal modifikasi username ama password di wvdial.conf yang ada di step c, selesai deh.

Step e: Pake browser kalo pengen browsing, etc.

Sampai di sini. Tinggal dipake aja koneksinya.

Btw, di daerah Mega Kuningan lumayan kenceng koneksinya.

Thursday, November 6, 2008

Combined find, xargs and chown Trick

Sometimes, you want to change the ownership of files (including symlinks) in directories recursively. However, "malformed" directory and file names can become huge stumbling blocks. This is how to do that correctly:


root@copy_cabana# find . -user old_user -group old_group -print0 | xargs -0 chown -v -h new_user:new_group

A few notes:

The -print0 parameter tells find to append the "null" character after each matching filename instead of the usual "newline" character. This is handy to "fight" against malformed filename and directory name that contain white spaces, punctuation, etc.

The -0 parameter tells xargs that the delimiter in its input is not "newline" but a "null" character.

The -h parameter tells chmod to change the ownership of the symlink instead of the file/directory the symlink refers to.

That's it. Now you're prepared for weird filenames.

Tuesday, October 7, 2008

Automated Login on Time-based Public Wireless Access Point

Public wireless access point sometimes employ a time-based logout-login/connect-disconnect cycles to "attract" users into their paid service counterpart in which the time-based logout-login/connect-disconnect cycles removed. Of course that kind of connect-disconnect cycles are annoying. However, with a little bit of bash script. The login-logout chores can be automated. I'll present an example script. You will need a machine that runs *nix and has curl installed. Mind you that in this scheme, it is assumed that the access-point recognize your WLAN card based on it's MAC address. The MAC address will be used as your user name used for login purposes. This is it.


#!/bin/bash
#
# Script to login and renew connection to FreeHotspot@somewhere
#
# Note: This script connects the INTERFACE to the access point and
#       renew the connection automatically. 
#       Press CTRL+C to stop the script
#

INTERFACE=wlan2
ESSID="FreeHotSpot@somewhere"
SESSION_LENGTH=20m
WLAN_KEY=off
LOGOUT_URL="http://192.168.10.254/logout"
LOGIN_URL="http://192.168.10.254/login?username=T-00%3AAA%3CCC%3FFF%3EEE"

if [ $UID -ne 0 ]; then
        echo "This script needs root privilege to run!"
        exit 1;
fi

while [ 1 ];
do
        # It seems that killing dhcpcd is the most reliable way to do now.
        # Otherwise reconnecting is always problematic.
        # Just removing dhcpcd PID file is not enough.
        #
        echo "Killing dhcpcd.."
        killall dhcpcd
        rm -vf /etc/dhcpc/dhcpcd-${INTERFACE}.pid

        echo "Bringing the interface down.."
        ifconfig ${INTERFACE} down

        echo "Configuring ${INTERFACE}.."
        iwconfig ${INTERFACE} essid ${ESSID}
        iwconfig ${INTERFACE} key ${WLAN_KEY}

        echo "Bringing the interface up.."
        ifconfig ${INTERFACE} up

        echo "Activating dhcpcd on ${INTERFACE}.."
        dhcpcd ${INTERFACE}

        if [ -e  /etc/dhcpc/dhcpcd-${INTERFACE}.pid ]; then
                #
                # Always logout first to ensure that there is no "stall"
                # session which will reject your subsequent DNS requests
                #
                echo "IP address obtained. Logging out with curl.."
                curl  ${LOGOUT_URL} > /dev/null
                echo "Logging in with curl.."
                curl  ${LOGIN_URL} > /dev/null
                sleep ${SESSION_LENGTH};
        else
                echo "Unable to obtain dhcp address. Please run the script again. Exiting.. "
                exit 1
        fi
done

The script above must be executed by someone with root privilege. The script above also assumes that to logout from the current session, you have to visit some special URL. The LOGOUT_URL variable contains this URL. You have to modify it to suit your need. You can obtain this URL from your browser by examining the HTML code of the page that shows the logout button. This page usually displayed right after you do a "manual" login in your browser, for example when you are using Firefox or Opera to login and use the access point to connect to the net. Moreover, the script also assumes that you have to visit some special URL to login into a new session. The LOGIN_URL variable contains this URL. You have to modify it to suit your need as well. The IP address in the LOGIN_URL and LOGOUT_URL seems to be owned by the access point. I'm not really sure about it, but in most cases it is. One thing that I want to note that you have to modify the SESSION_LENGTH to suit the logout interval to suit the public access point that you use. Its value must be less than the session expiration time in the public access point that you use. For example, if the session expires in 30 minutes in your particular public access point, then use 29m (or less) as the value of the SESSION_LENGTH variable. Note that 29m corresponds to 29 minutes, because this value is passed to the sleep bash built-in function.

There you have it, automated login script for "smooth" surfing on public access point. As a bonus, let's see how to do automated download as well. In this case, we assume that the download process will resume after some interval. Nonetheless the script is not bug free. You have to stop it downloading things when you think it's finished because it will keep trying downloading even if the download already completed. To stop the script, simply press CTRL+C. Here it is:


#!/bin/bash

# get the file
#
#

INTERVAL=15m
SRC="http://download.mystuff.com/source_code.zip"
DEST="source_code.zip"

# Install a trap (interrupt handler) to handle Ctrl+C
trap 'killall wget; exit 0' 2

while [ 1 ]; do
        # run wget in the background
 (wget -c ${SRC} -O ${DEST} &> /dev/null) &

 # wait for few minutes
 sleep ${INTERVAL}

 # restart the download
 # this is a rather brute force approach to stop current download
 # TODO: fix it!

 killall wget

done

exit 0

Remember to stop the script by using CTRL+C. It's advised to run wget and then terminate wget previous to running this script in order to know the size of the file that you are going to download. With that information you will know when you should stop the script.

That's it for the moment. Ciao

Tuesday, September 30, 2008

Building Wireless Perimeter With WPA2 and RADIUS

Most of the KISS (Keep It Simple Stupid) steps are explained in an article by SmallNetBuilder. However, those guides were not working out-of-the-box for me. I'll talk about it later.

Let's start with my system setup:
1. Slamd64 12.1
2. Freeradius 1.1.7
3. OpenSSL 0.9.8g (comes with Slamd64 12.1)

Now, to the quirks of building Freeradius 1.1.7. I didn't manage to compile Freeradius with the default ./configure and make command in Slamd64. Therefore, I'm only interested to enable EAP-TLS support. Therefore, I use the following commands to build it:


./configure --prefix=/usr  --libdir=/usr/lib64 --sysconfdir=/etc  --mandir=/usr/man \
--enable-strict-dependencies --without-rlm_dbm --without-rlm_krb5  --without-rlm_pam \
--without-rlm_sql_postgresql

make

As you see, Kerberos, pam, and sql support are disabled. In this setup, it's not needed as well. Therefore, it's not a problem.

The next quirk is in the wpa_supplicant configuration file. You have to be very careful especially regarding the entries you put in the certificate related fields. Below is the snippet of the wpa_supplicant configuration file in my laptop.


# WPA2-EAP/CCMP using EAP-TLS
network={
 ssid="your_ap_ssid"
 key_mgmt=WPA-EAP
 proto=RSN
 identity="your_machine_common_name"
 pairwise=CCMP
 group=CCMP
 eap=TLS
 ca_cert="/etc/wireless/cacert.pem"
 private_key="/etc/wireless/linux_laptop.p12"
 private_key_passwd="your_secret_pkcs12_password"
}

In the configuration snippet above, the laptop (a user in RADIUS vocabulary), lump the certificate and the private key together into a PKCS12 file for Windows compatibility reason. This is not needed in Linux. You can split the certificate and the private if you are using Linux by using the following configuration:


# WPA2-EAP/CCMP using EAP-TLS
network={
 ssid="your_ap_ssid"
 key_mgmt=WPA-EAP
 proto=RSN
 identity="your_machine_common_name"
 pairwise=CCMP
 group=CCMP
 eap=TLS
 ca_cert="/etc/wireless/cacert.pem"
 client_cert="/etc/wireless/linux_laptop_cert.pem"
 private_key="/etc/wireless/linux_laptop_key.pem"
 private_key_passwd="your_private_key_password"
}

If you look carefully in both of the configuration above, you will notice that the password for both of the configuration are different. Indeed, the former uses the password that you enter when you make the PKCS12 file while the latter uses the password that you enter when you make the private key. These are different passwords. For RADIUS newbie this can be easily become problem. Therefore, as always, run Freeradius as radiusd -X before you found a working configuration.

Also, take a note on file permissions for Freeradius configuration file because it can be easily becomes a problem. Moreover, if you run the Freeradius daemon as nobody (as explained in the tutorial by SmallNetBuilder), then set the config file group to user nobody, so that it can be read by Freeradius when it runs. Don't forget to disable read-write-execute permission on the configuration file for unwanted users. This way is safer for you.

This Freeradius howto explains a much cleaner approach if you are interested. It's a bit more complicated as well. But, manageable for quite average *NIX administrator and user.

In any case, by protecting your home wireless access point with radius, you have established a strong enough perimeter against malicious wannabes around your access point.

Wednesday, September 24, 2008

MIPS Cross Compiling on Linux x86_64

Cross compiling might be a quite "scary" subject for some programmers. Nevertheless, it's not so difficult as long as you have already obtained the right cross-toolchain.

At first sight, I thought that I have to put lengthy LDFLAGS and CFLAGS arguments in the Makefile of my test program. But, surprisingly, it's not the case. Let's see how it works. My build system is Slamd64 12.1, a Linux x86_64 platform. The target is a MIPS R3000 system. This is the Makefile:


all:    set_power_cck

CC      = mips-uclibc-gcc
STRIP   = mips-uclibc-strip

CFLAGS  = -v 
IFLAGS  =
LDFLAGS =


DEBUG   = -Wall -Os

set_power_cck: Makefile set_power_cck.o
        $(CC) -o $@ $(DEBUG) $(CFLAGS) $(IFLAGS) $(LDFLAGS) set_power_cck.o
        $(STRIP) $@

clean:
        rm -f set_power_cck *.o 

set_power_cck.o: set_power_cck.c
        $(CC) -c -o $@ $(DEBUG) $(CFLAGS) $(IFLAGS) $<

As you see, the only exotic option is in the CC and STRIP program definition which specify a cross-mips toolchain. Actually this is an x86 (not x86_64) cross toolchain, but because my Slamd64 is a multilib linux installation, it's not a problem. In the first run of this makefile, I'm very surprised how the cross-toolchain could find the right include paths and library paths. After tinkering with GCC for sometime, I found that the -v CFLAGS could reveal what exactly happens. Now, let's see how the cross-toolchain found the right library paths and include paths.


darmawan@opunaga:realtek_rtl8186_sdk_v1.4c_svn_co $ make -C AP/set_power_cck/                                                                                                                                   
make: Entering directory `/home/darmawan/_Projects/Antek_SRL/RTL8186_work/realtek_rtl8186_sdk_v1.4c_svn_co/AP/set_power_cck'
mips-uclibc-gcc -c -o set_power_cck.o -Wall -Os -v   set_power_cck.c
Reading specs from /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/specs
Configured with: /root/toolchain/gcc-3.3.x/toolchain_build_mips_nofpu/gcc-3.3.3/configure --prefix=/usr/local/gcc333/lexra-nnop-v5 --build=i386-pc-linux-gnu --host=i386-pc-linux-gnu --target=mips-linux-uclibc --enable-languages=c,c++ --enable-shared --with-gxx-include-dir=/usr/local/gcc333/lexra-nnop-v5/mips-linux-uclibc/include/c++ --disable-__cxa_atexit --enable-target-optspace --with-gnu-ld -with-gnu-as --disable-nls --enable-multilib --without-float --enable-sjlj-exceptions
Thread model: posix
gcc version 3.3.3
 /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/cc1 -quiet -v -D__GNUC__=3 -D__GNUC_MINOR__=3 -D__GNUC_PATCHLEVEL__=3 set_power_cck.c -quiet -dumpbase set_power_cck.c -auxbase-strip set_power_cck.o -Os -Wall -version -msoft-float -o /tmp/ccNOlsiQ.s
GNU C version 3.3.3 (mips-linux-uclibc)
        compiled by GNU C version 2.96 20000731 (Red Hat Linux 7.3 2.96-110).
GGC heuristics: --param ggc-min-expand=94 --param ggc-min-heapsize=120073
ignoring duplicate directory "/usr/local/gcc333/lexra-nnop-v5/mips-linux-uclibc/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/include
 /usr/local/gcc333/lexra-nnop-v5/mips-linux-uclibc/sys-include
End of search list.
set_power_cck.c: In function `set_tx_power_cck':
set_power_cck.c:20: warning: unused variable `tx_power_cck'
set_power_cck.c:22: warning: control reaches end of non-void function
set_power_cck.c: At top level:
set_power_cck.c:19: warning: `set_tx_power_cck' defined but not used
 /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/../../../../mips-linux-uclibc/bin/as -EB -g0 -32 -v -KPIC -o set_power_cck.o /tmp/ccNOlsiQ.s
GNU assembler version 2.14.90.0.7 (mips-linux-uclibc) using BFD version 2.14.90.0.7 20031029
mips-uclibc-gcc -o set_power_cck -Wall -Os -v    set_power_cck.o
Reading specs from /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/specs
Configured with: /root/toolchain/gcc-3.3.x/toolchain_build_mips_nofpu/gcc-3.3.3/configure --prefix=/usr/local/gcc333/lexra-nnop-v5 --build=i386-pc-linux-gnu --host=i386-pc-linux-gnu --target=mips-linux-uclibc --enable-languages=c,c++ --enable-shared --with-gxx-include-dir=/usr/local/gcc333/lexra-nnop-v5/mips-linux-uclibc/include/c++ --disable-__cxa_atexit --enable-target-optspace --with-gnu-ld -with-gnu-as --disable-nls --enable-multilib --without-float --enable-sjlj-exceptions
Thread model: posix
gcc version 3.3.3
 /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/collect2 --eh-frame-hdr -EB -dynamic-linker /lib/ld-uClibc.so.0 -o set_power_cck /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/../../../../mips-linux-uclibc/lib/crt1.o /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/../../../../mips-linux-uclibc/lib/crti.o /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/crtbegin.o -L/usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3 -L/usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/../../../../mips-linux-uclibc/lib set_power_cck.o -lgcc -lgcc_eh -lc -lgcc -lgcc_eh /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/crtend.o /usr/local/gcc333/lexra-nnop-v5/lib/gcc-lib/mips-linux-uclibc/3.3.3/../../../../mips-linux-uclibc/lib/crtn.o
mips-uclibc-strip set_power_cck
make: Leaving directory `/home/darmawan/_Projects/Antek_SRL/RTL8186_work/realtek_rtl8186_sdk_v1.4c_svn_co/AP/set_power_cck'

As you see, the cross-compiler driver (mips-uclibc-gcc) found the right paths for libraries and include files in the spec file definition. Now, it's clear to me what a "compiler driver" really is. This knowledge is very handy when you are working with cross-toolchain. Now, let's see the result using the file utility.


darmawan@opunaga:realtek_rtl8186_sdk_v1.4c_svn_co $ file AP/set_power_cck/set_power_cck                                                                                                                         
AP/set_power_cck/set_power_cck: ELF 32-bit MSB executable, MIPS, MIPS-I version 1 (SYSV), dynamically linked (uses shared libs), stripped

It's very clear that the binary result is indeed MIPS ELF 32-bit executable file. When I run the executable in the target MIPS system, it works just fine as follows:


BusyBox v1.00-pre8 (2008.09.24-13:53+0000) Built-in shell (msh)
Enter 'help' for a list of built-in commands.
# set_power_cck
Usage: $0 (0-255 decimal)
where (0-255 decimal) is the power level gain. Choose a value between 0 and 255 decimal

Cross compiling is not as scary as it sounds, doesn't it? Personally, I think it's scary if you have to build the cross-toolchain yourself. Let's leave it for another time.

Sunday, September 21, 2008

Ndiswrapper-SSB Driver Bug in Slamd64 12.1 (Kernel 2.6.25.9) -- Somehow With A wpa_supplicant Fix

It's quite sad to find out that the current generation of opensource driver for Broadcom BCM43xx WLAN chipset is still unable to get WEP working as expected--after some checks, I found that the driver unable to work with WEP unless you use wpa_supplicant to set the WEP keys. It forces me to go back to use ndiswrapper once more. Nonetheless, using ndiswrapper is not as easy as it seems because the driver has a conflict with the ssb driver module. The workaround is an ugly workaround, but it seems working so far.

First, block all drivers related to WLAN handling and the ssb kernel module in /etc/modprobe.d/blacklist (note that it's very fortunate that I build the ssb driver as a kernel module)


...
# Disable Broadcom b43 driver and all related wifi drivers because 
# we are using ndiswrapper.
# Also, disable ssb and ohci_hcd and then enable them back in /etc/rc.d/rc.local
blacklist b43
blacklist rfkill_input 
blacklist rfkill
blacklist mac80211
blacklist cfg80211
blacklist ohci_hcd 
blacklist ssb 
...

Second, load ndiswrapper driver before ssb loaded using /etc/rc.d/rc.local:


...
# Load  ndiswrapper-based BCM4318 WLAN driver
modprobe ndiswrapper

# Enable ssb and ohci_hcd
modprobe ssb
modprobe ohci_hcd 
...

Now everything worked as expected. Nonetheless, this is an ugly hack just to make the encryption work in this particular WLAN chipset.
Note that I'm using the Windows XP x64 Broadcom BCM43xx driver as the driver which is loaded by ndiswrapper on boot.

----
Update:

After conducting some experiments, I found out that WEP actually works with the opensource Linux BCM43xx driver (b43 driver module along with its dependency modules). The catch is, I have to use wpa_supplicant to set the WEP key. This is the wpa_supplicant configuration file for WEP that I use:


# Static WEP keys

ctrl_interface=/var/run/wpa_supplicant

network={
 ssid="bounsier"
 key_mgmt=NONE
 wep_key0="very_secret_wep_key"
# wep_key1=0102030405
 wep_tx_keyidx=0
}

Further tests, shows that WPA is also working correctly in the current generation of BCM43xx opensource driver with the help of wpa_supplicant as the WPA "back-end".

BIOS Disasembly Ninjutsu Uncovered Russian Edition

The Russian edition of my BIOS Disassembly Ninjutsu Uncovered book actually has been published in August last year. However, I haven't been able to write about it for a while. Here it is:

Cyrillic maybe a problem for most people used to Latin characters. This is what the title said (more or less):

BIOS
Disassemblirovaniye
Modifikatsiya
Programmirovaniye

translated to english:

BIOS
Disassembly
Modification
Programming

Here's the back cover (a.k.a blurb):

For me, it's an honor to have my book translated to other language. But it's quite sad that the book hasn't been translated to Bahasa Indonesia, my native language. Maybe next time when I have enough time to do that. At the moment, the book probably overly technical and doesn't find enough market share to take advantage of the economic of scale to be published in Bahasa Indonesia. Time will tell.

Friday, September 19, 2008

The Diff Ninjutsu (Part 2)

In the previous installment of this series, I haven't show how we can use diff to do the automatic checking of the applied patch. Now. let's look at a sample shell script that do just that.


#!/bin/bash
#
# Script to apply patches to Realtek v1.4 ADK, SDK and CMK in order to 
# upgrade it to version 1.4c (with latest v1.4c patches)
# 
# Author: Darmawan Salihun
# 
# Date: 26-08-2008
#

##### ASSUMPTIONS a.k.a PRECONDITION ####################################
#
# The following directories must have been decompressed prior to the 
# execution of this script!
#
# CMK_1_4b_ROOT_DIR=rtl8186-cmk-1.4b
# CMK_1_4c_ROOT_DIR=rtl8186-cmk-1.4c-release
# ADK_ROOT_DIR=rtl8186-adk-1.4
# SDK_ROOT_DIR=rtl8186-sdk-1.4
# CMK_1_4c_PATCH_DIR=patch/files
#
#########################################################################


# directories definition
CMK_1_4b_ROOT_DIR=rtl8186-cmk-1.4b
CMK_1_4b_ADK_PATCH_DIR=$CMK_1_4b_ROOT_DIR/patch/ADK/AP
CMK_1_4b_SDK_PATCH_DIR=$CMK_1_4b_ROOT_DIR/patch/SDK/linux-2.4.18

CMK_1_4c_ROOT_DIR=rtl8186-cmk-1.4c-release
CMK_1_4c_ADK_PATCH_DIR=$CMK_1_4c_ROOT_DIR/patch/ADK/AP
CMK_1_4c_SDK_PATCH_DIR=$CMK_1_4c_ROOT_DIR/patch/SDK/linux-2.4.18
CMK_1_4c_CONFIG_DIR=$CMK_1_4c_ROOT_DIR/config
CMK_1_4c_LINUX_TOOLS=$CMK_1_4c_ROOT_DIR/tools/LINUX/tools.tar.gz

ADK_ROOT_DIR=rtl8186-adk-1.4
ADK_ASP_DIR=$ADK_ROOT_DIR/rtl8186-ASP-1.4
SDK_ROOT_DIR=rtl8186-sdk-1.4
SDK_LINUX_DIR=$SDK_ROOT_DIR/rtl8186-linux-1.4
SDK_TOOLCHAIN=$SDK_ROOT_DIR/rtl8186-toolchain-1.0.tar.gz
SDK_BOOTCODE=$SDK_ROOT_DIR/rtl8186-btcode-1.4a.tar.gz

CMK_1_4c_PATCH_DIR=patch/files
ANTEK_RTL_SDK_DIR=antek-rtl8186-sdk

# Guard against error during execution
set -e

#
# step 1: prepare target directory (Antek RTL8186 SDK)
# 
echo "Preparing target directory.."
rm -rf $ANTEK_RTL_SDK_DIR
mkdir -v $ANTEK_RTL_SDK_DIR


#
# step 2: prepare original ADK and SDK v1.4
#
echo "Preparing original ADK and SDK.."
rm -rf $ADK_ASP_DIR
tar xzf $ADK_ROOT_DIR/rtl8186-ASP-1.4.tar.gz -C $ADK_ROOT_DIR

rm -rf $SDK_LINUX_DIR
tar xzf $SDK_ROOT_DIR/rtl8186-linux-1.4.tar.gz -C $SDK_ROOT_DIR


#
# step 3: copy original ADK and SDK v1.4 to target directory
#
echo "Copying original ADK and SDK.."
cp -r --remove-destination $ADK_ASP_DIR/AP  $ANTEK_RTL_SDK_DIR
cp -r --remove-destination $SDK_LINUX_DIR/linux-2.4.18  $ANTEK_RTL_SDK_DIR


#
# step 4: upgrade v1.4 to v1.4b in target directory
#
echo "Upgrading ADK and SDK to v1.4b.."
cp -r --remove-destination $CMK_1_4b_ADK_PATCH_DIR/*  $ANTEK_RTL_SDK_DIR/AP
cp -r --remove-destination $CMK_1_4b_SDK_PATCH_DIR/*  $ANTEK_RTL_SDK_DIR/linux-2.4.18 


#
# step 5: check the validity of the applied ADK and SDK v1.4b patch 
#

#
# Check ADK patch validity
#
echo "Checking ADK v1.4b patch validity.."
find $CMK_1_4b_ADK_PATCH_DIR  -type f -name '*' > files.txt 

# Eliminate CMK_1_4b_ADK_PATCH_DIR/ from filenames path
cat /dev/null >  new_files.txt
while read FILENAME 
do
        echo $FILENAME | sed -e "s%$CMK_1_4b_ADK_PATCH_DIR/%%g" >> new_files.txt  # Note: sed uses % as delimiter 
done < files.txt

rm -f files.txt

# Merge list of files into single line, 
# i.e. as the value of TARGETS variable
while read FILENAME 
do
        TARGETS="$TARGETS $FILENAME"
done < new_files.txt

rm -f new_files.txt


for i in $TARGETS; 
do 
        SRC=$CMK_1_4b_ADK_PATCH_DIR/$i
        DEST=$ANTEK_RTL_SDK_DIR/AP/$i
        diff $SRC  $DEST > result.txt 

        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done


# 
# Reset reused variables!!!
#
TARGETS=""
FILENAME=""
SRC=""
DEST=""

#
# Check SDK patch validity
#
echo "Checking SDK 1.4b patch validity.."
find $CMK_1_4b_SDK_PATCH_DIR  -type f -name '*' > files.txt 

# Eliminate CMK_1_4b_SDK_PATCH_DIR/ from filenames path
cat /dev/null >  new_files.txt
while read FILENAME 
do
        echo $FILENAME | sed -e "s%$CMK_1_4b_SDK_PATCH_DIR/%%g" >> new_files.txt  # Note: sed uses % as delimiter 
done < files.txt

rm -f files.txt

# Merge list of files into single line, 
# i.e. as the value of TARGETS variable
while read FILENAME 
do
        TARGETS="$TARGETS $FILENAME"
done < new_files.txt

rm -f new_files.txt

for i in $TARGETS; 
do 
        SRC=$CMK_1_4b_SDK_PATCH_DIR/$i
        DEST=$ANTEK_RTL_SDK_DIR/linux-2.4.18/$i
        diff $SRC  $DEST > result.txt 

        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done

# 
# Reset reused variables!!!
#
TARGETS=""
FILENAME=""
SRC=""
DEST=""


#
# step 6: patch _the ADK and SDK v1.4c patch files_ with latest patches
#
CMK_1_4c_PATCH_TARGETS="patch/SDK/linux-2.4.18/rtl8186/wireless_ag_net.o  patch/ADK/linux-2.4.18/drivers/net/wireless_ag/ieee802_mib.h  patch/SDK/linux-2.4.18/drivers/net/wireless_ag/ieee802_mib.h  patch/SDK/l
inux-2.4.18/rtl8186.tar.gz  patch/ADK/AP/mkimg"


for i in $CMK_1_4c_PATCH_TARGETS;
do 
        DEST=$CMK_1_4c_ROOT_DIR/$i
        SRC=`basename $i`
        SRC=$CMK_1_4c_PATCH_DIR/$SRC
        cp -f $SRC $DEST
done

#
# step 7: check the patch for ADK and SDK v1.4c patch files
#
for i in $CMK_1_4c_PATCH_TARGETS;
do 
        DEST=$CMK_1_4c_ROOT_DIR/$i
        SRC=`basename $i`
        SRC=$CMK_1_4c_PATCH_DIR/$SRC

        diff $SRC  $DEST > result.txt 
        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done


# 
# Reset reused variables!!!
#
TARGETS=""
FILENAME=""
SRC=""
DEST=""


#
# step 8: upgrade ADK and SDK v1.4b to patched ADK and SDK v1.4c in target dir 
#
echo "Upgrading ADK and SDK v1.4b to v1.4c.."
cp -r --remove-destination $CMK_1_4c_ADK_PATCH_DIR/*  $ANTEK_RTL_SDK_DIR/AP
cp -r --remove-destination $CMK_1_4c_SDK_PATCH_DIR/*  $ANTEK_RTL_SDK_DIR/linux-2.4.18 

#
# check ADK v1.4c patch validity
#

echo "Checking ADK v1.4c patch validity.."
find $CMK_1_4c_ADK_PATCH_DIR  -type f -name '*' > files.txt 

# Eliminate CMK_1_4c_ADK_PATCH_DIR/ from filenames path
cat /dev/null >  new_files.txt
while read FILENAME 
do
        echo $FILENAME | sed -e "s%$CMK_1_4c_ADK_PATCH_DIR/%%g" >> new_files.txt  # Note: sed uses % as delimiter 
done < files.txt

rm -f files.txt

# Merge list of files into single line, 
# i.e. as the value of TARGETS variable
while read FILENAME 
do
        TARGETS="$TARGETS $FILENAME"
done < new_files.txt

rm -f new_files.txt


for i in $TARGETS; 
do 
        SRC=$CMK_1_4c_ADK_PATCH_DIR/$i
        DEST=$ANTEK_RTL_SDK_DIR/AP/$i
        diff $SRC  $DEST > result.txt 

        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done


# 
# Reset reused variables!!!
#
TARGETS=""
FILENAME=""
SRC=""
DEST=""

#
# Check SDK v1.4c patch validity
#
echo "Checking SDK v1.4c patch validity.."
find $CMK_1_4c_SDK_PATCH_DIR  -type f -name '*' > files.txt 


# Eliminate CMK_1_4c_SDK_PATCH_DIR/ from filenames path
cat /dev/null >  new_files.txt
while read FILENAME 
do
        echo $FILENAME | sed -e "s%$CMK_1_4c_SDK_PATCH_DIR/%%g" >> new_files.txt  # Note: sed uses % as delimiter 
done < files.txt

rm -f files.txt

# Merge list of files into single line, 
# i.e. as the value of TARGETS variable
while read FILENAME 
do
        TARGETS="$TARGETS $FILENAME"
done < new_files.txt

rm -f new_files.txt

for i in $TARGETS; 
do 
        SRC=$CMK_1_4c_SDK_PATCH_DIR/$i
        DEST=$ANTEK_RTL_SDK_DIR/linux-2.4.18/$i
        diff $SRC  $DEST > result.txt 

        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done



mkdir -v $ANTEK_RTL_SDK_DIR/tool 
cp -rf  $CMK_1_4c_CONFIG_DIR   $ANTEK_RTL_SDK_DIR/tool 
tar xzf $CMK_1_4c_LINUX_TOOLS   -C $ANTEK_RTL_SDK_DIR/tool/ 

cp -vf  $SDK_TOOLCHAIN   $ANTEK_RTL_SDK_DIR/tool 
cp -vf  $SDK_BOOTCODE   $ANTEK_RTL_SDK_DIR/tool 

# inform that we're done
echo "Realtek RTL8186 SDK v1.4c installed."

That's one hell of a script. Nonetheless, the most important part is the automatic patch-checking routine such as:


echo "Checking SDK v1.4c patch validity.."
find $CMK_1_4c_SDK_PATCH_DIR  -type f -name '*' > files.txt 

# Eliminate CMK_1_4c_SDK_PATCH_DIR/ from filenames path
cat /dev/null >  new_files.txt
while read FILENAME 
do
        echo $FILENAME | sed -e "s%$CMK_1_4c_SDK_PATCH_DIR/%%g" >> new_files.txt  # Note: sed uses % as delimiter 
done < files.txt

rm -f files.txt

# Merge list of files into single line, 
# i.e. as the value of TARGETS variable
while read FILENAME 
do
        TARGETS="$TARGETS $FILENAME"
done < new_files.txt

rm -f new_files.txt

for i in $TARGETS; 
do 
        SRC=$CMK_1_4c_SDK_PATCH_DIR/$i
        DEST=$ANTEK_RTL_SDK_DIR/linux-2.4.18/$i
        diff $SRC  $DEST > result.txt 

        if [ -s  result.txt  ]; then 
                echo "Error. $SRC and $DEST differ"
        fi
        rm -f result.txt
done

As you can see, the script first create list of files to be diff-ed and then iterate the diff-ing process on each of those file, creating a diff file (result.txt) in the process. If the size of result.txt is bigger than zero, it means there is at least one difference which means the patch is not correctly applied.

There you have it, an automatic patch-checking script. In the next installment of this series, I will tidy up the code and make some functions because the current version is a quick'n'dirty script.

Saturday, September 13, 2008

The Diff Ninjutsu

Sometimes patches for certain source code which are distributed by programmers, are not in the form of the usual patch like linux kernel patches (either incremental or against the base version) which you can apply by just using the patch utility in *nix. This can cause months of headache for maintainer or someone who come later. This is where utility like diff comes handy. Let's see how to use it effectively.

First, examine the entire directory structure of the different version of the source code


diff -q -r <orig_dir> <new_dir>

Second, examine the differences between different version of certain file of interest.


diff <path_to_orig_file> <path_to_new_file>

Third, create a shell script to do the real patching. For example:


#!/bin/bash
#
# Script to apply patches to Realtek v1.4 ADK, SDK and CMK in order to 
# upgrade it to version 1.4c (with latest v1.4c patches)
# 
# Author: Darmawan Salihun
# 
#

##### ASSUMPTIONS a.k.a PRECONDITION ####################################
#
# The following directories must have been decompressed prior to the 
# execution of this script!
#
# RTL8186_LINUX_DIR="../rtl8186-sdk-1.4/rtl8186-linux-1.4"
# RTL8186_ASP_DIR="../rtl8186-adk-1.4/rtl8186-ASP-1.4"
# PATCH_1_4b_ADK="rtl8186-cmk-1.4b/patch/ADK"
# PATCH_1_4b_SDK="rtl8186-cmk-1.4b/patch/SDK"
# PATCH_1_4c_ADK_DIR="rtl8186-cmk-1.4c-release/patch/ADK"
# PATCH_1_4c_SDK_DIR="rtl8186-cmk-1.4c-release/patch/SDK"
# CMK_1_4c_PATCH_DIR="patch/files"
# CMK_1_4c_CONFIG_DIR="rtl8186-cmk-1.4c-release/config"
#
#########################################################################


# directories definition
PATCH_1_4b_ADK="rtl8186-cmk-1.4b/patch/ADK/*"
PATCH_1_4b_SDK="rtl8186-cmk-1.4b/patch/SDK/*"
PATCH_1_4c_ADK_DIR="rtl8186-cmk-1.4c-release/patch/ADK"
PATCH_1_4c_SDK_DIR="rtl8186-cmk-1.4c-release/patch/SDK"
RTL8186_ASP_DIR="../rtl8186-adk-1.4/rtl8186-ASP-1.4"
RTL8186_LINUX_DIR="../rtl8186-sdk-1.4/rtl8186-linux-1.4"
CMK_1_4c_PATCH_DIR="patch/files"
CMK_1_4c_CONFIG_DIR="rtl8186-cmk-1.4c-release/config"
CMK_1_4c_LINUX_TOOLS="rtl8186-cmk-1.4c-release/tools/LINUX/tools.tar.gz"
SDK_TOOLCHAIN="../rtl8186-sdk-1.4/rtl8186-toolchain-1.0.tar.gz"
SDK_BOOTCODE=" ../rtl8186-sdk-1.4/rtl8186-btcode-1.4a.tar.gz"
ANTEK_RTL_SDK_DIR="antek-rtl8186-sdk"

# Guard against error during execution
set -e

# step 1: upgrade ADK and SDK v1.4 to v1.4b
cp -r -v --remove-destination $PATCH_1_4b_ADK   $RTL8186_ASP_DIR 
cp -r -v --remove-destination $PATCH_1_4b_SDK   $RTL8186_LINUX_DIR 

# step 2: patch _the ADK and SDK v1.4c patch files_ with latest patches
cp -v --remove-destination $CMK_1_4c_PATCH_DIR/wireless_ag_net.o  $PATCH_1_4c_SDK_DIR/linux-2.4.18/rtl8186 
cp -v --remove-destination $CMK_1_4c_PATCH_DIR/ieee802_mib.h  $PATCH_1_4c_ADK_DIR/linux-2.4.18/drivers/net/wireless_ag

cp -v --remove-destination $CMK_1_4c_PATCH_DIR/ieee802_mib.h  $PATCH_1_4c_SDK_DIR/linux-2.4.18/drivers/net/wireless_ag

cp -v --remove-destination $CMK_1_4c_PATCH_DIR/rtl8186.tar.gz  $PATCH_1_4c_SDK_DIR/linux-2.4.18

cp -v --remove-destination $CMK_1_4c_PATCH_DIR/mkimg  $PATCH_1_4c_ADK_DIR/AP

# step 3: upgrade ADK and SDK v1.4b to v1.4c_with_latest_patches
#
#  Note: Watch for the "/*" because the paths are only _directory paths_. 
#        That's why ypu have to append them with an "/*" to ensure you don't break 
#        the directory structure of the ADK/SDK and that you replace the right 
#        files and directories.
#
cp -r -v --remove-destination $PATCH_1_4c_ADK_DIR/*  $RTL8186_ASP_DIR 
cp -r -v --remove-destination $PATCH_1_4c_SDK_DIR/*  $RTL8186_LINUX_DIR

# step 4 create Antek RTL8186 SDK from patched ADK, SDK and CMK
rm -rf $ANTEK_RTL_SDK_DIR
mkdir $ANTEK_RTL_SDK_DIR

cp -rf $RTL8186_ASP_DIR/AP  $ANTEK_RTL_SDK_DIR
cp -rf $RTL8186_LINUX_DIR/linux-2.4.18  $ANTEK_RTL_SDK_DIR

mkdir -v $ANTEK_RTL_SDK_DIR/tool 
cp -rvf  $CMK_1_4c_CONFIG_DIR   $ANTEK_RTL_SDK_DIR/tool 
tar xvzf $CMK_1_4c_LINUX_TOOLS  -C $ANTEK_RTL_SDK_DIR/tool/ 

cp -vf  $SDK_TOOLCHAIN   $ANTEK_RTL_SDK_DIR/tool 
cp -vf  $SDK_BOOTCODE   $ANTEK_RTL_SDK_DIR/tool 

# inform that we're done
echo "Realtek RTL8186 SDK v1.4c installed."

Next, you can do the first step to examine whether you have applied the patch correctly.
Actually, analysis of the patching result can be automated with shell script as well. But, let's leave it for another installment of this diff series of tutorial.

That's it for now. Ciao.

Sunday, August 31, 2008

How to Migrate a Subversion Repository

Versioning system such as subversion is a very important tool for developer. It tracks every changes that you made to the base code. However, at some points an svn checkout and then an svn commit is not enough. For example, when you want to backup the whole repository along with its tracked changes and then move it into a different svn server. With the svnadmin tool, this kind of task is almost a walk in the park. But you have to pay attention to some details.

Now, let's see a simple real world example. Let's say you have downloaded an important piece of code and created a local svn repository in your laptop because somehow your central svn server is not reachable. You have worked with this local repository and have got your version number up to version 20. Later on, you plan to migrate this repository into the central svn server along with the changes recorded in the local repository. The keyword here is migration. This process is explained in the Subversion Book for Subversion 1.2 Chapter 5 Repository Administration in section Repository Maintenance. The tool to migrate the local repository to the central server is svnadmin. These are steps in detail.

1. Using your current version of svnadmin, dump your repositories to dump files. This accomplished with:


$ svnadmin dump path_to_repository > dump_filename

The output of this command would be similar to:


$ svnadmin dump path_to_repository > dump_filename
* Dumped revision 0.
* Dumped revision 1.
* Dumped revision 2.
…
* Dumped revision 19.
* Dumped revision 20.

2. Create a new empty repository in the central server with:


$ svnadmin create

3. Transfer the file to the central svn server.

4. Using svnadmin in the central server, load your dump files into their respective, just-created repositories.


$ svnadmin load path_to_new_repository < dump_filename

The output of this command would be similar to:


$ svnadmin load path_to_new_repository < dump_filename
<<< Started new txn, based on original revision 1
* adding path : A ... done.
* adding path : A/B ... done
...
<<< Started new txn, based on original revision 20
* adding path : A/Z/zeta ... done.
* editing path : A/mu ... done.
------- Committed new rev 20 (loaded from original rev 20) >>>

If you are using a different version of subversion in the central server, you have to pay attention to the following issue.

Be sure to copy any customizations from your old repositories to the new ones, including
DB_CONFIG files and hook scripts. You'll want to pay attention to the release notes for the new release of Subversion to see if any changes since your last upgrade affect those hooks or configuration options.

It's a good idea to test the just migrated svn repository using svn co and then try to compile the source code to ensure everything went ok.

Now, say you are away again with your laptop and have committed some changes to the local repository because you cannot access the central server. Then you want to reflect the changes you have made since the last time you migrate the svn repository. This can be accomplished with:


$ svn dump path_to_repository --revision 20:25 --incremental > dump_filename

In the command above, you inform subversion that you want to backup only the changes from revision 20 to 25. You can then load these changes using svnadmin load command as before.

That's it. Now you can work offline with your laptop and have the local svn repository synchronized with your central server with the help of svnadmin.

Tuesday, August 26, 2008

The Incompatibility of 64-bit GCC With 32-bit Packed Data Structures

There are certain times when we, C programmers take it for granted that when you declare:


unsigned long x;

you expect to have 32-bit unsigned variable. You would expect that you need to declare the variable as:


unsigned long long x;

to have 64-bit unsigned variable.

Well, at some points these are not problematic. But, when you have a packed data structure, say a structure that describes the header of a binary file. You're in a big trouble if the seemingly innocent code:


unsigned long x;

turns out to declare a 64-bit unsigned long instead of the expected 32-bit unsigned long.

Let's look at a real life example. This code:


struct img_header {
 unsigned char signature[SIGNATURE_LEN];
 unsigned long startAddr;
 unsigned long burnAddr;
 unsigned long len;
}__attribute__((packed));

typedef struct img_header IMG_HEADER_T, *IMG_HEADER_Tp;

Would create an 16-bytes IMG_HEADER_T structure with the 32-bit GCC compiler. On the other hand, it would create a 28-bytes IMG_HEADER_T structure with the 64-bit GCC compiler. This is very dangerous when dealing with firmware binary. The workaround on 64-bit GCC compiler is to force the compiler to use the 32-bit "compatibility" mode by using the "-m32" compiler flags. Most 64-bit GCC compiler in 64-bits Linux distributions have this "compatibility" mode. Usually, the compiler flags are placed in the Makefile. You can force this intended behavior there, like this:


...
CC=gcc
CFLAGS = -m32
...

It takes me more than one month to spot this bug :-(. Which is a pity. It's known only after I made a very simple 8-bit checksum utility which spots an excess of 12-bytes in the header file of an intermediate file in the SDK that I worked with. A couple of lessons learned from this incident.

1. Never trust your seemingly innocent Makefile when you're working on a 64-bit system with 64-bit compiler or multilib compiler. Use any "force" options to enforce your intended output from the compiler because in most cases you don't know what the default is unless you do some tests.

2. Always build test stubs to verify intermediate results when something goes awry in the output file of an SDK. This will save your development time.

3. Use common sense to track down the bug. Add debug statements to watch the output of a binary utility in an SDK when using it on 64-bit systems because you won't know whether it will behave as intended or not. Most of todays SDKs has been tested only in 32-bit systems and assumed to be running on the very same architecture.

This is a very hard lesson for me because it wasted time plus a lot of resources to spot the bug. I should've build the "test stub" application which only takes very short time to create in the very beginning. The "test stub":


#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

#define DEBUG
#undef DEBUG


/* 
 * TODO:
 *
 * 1. Add default end offset handling (i.e., EOF equ end offset)
 * 2. Add more robust error handling
 * 3. Add GNU gettext handler for input parameters
 *
 */


static unsigned char sum(char buf[], unsigned long len, unsigned long start)
/*
 * @param buf  pointer to buffer to be 8-bit summed
 * @param start starting offset in buf to calculate the sum
 * @param len  length of the buffer to be summed (in bytes)
 */
{
 unsigned char sum;
 unsigned long i;

 sum = 0;

 for(i=0; i<len; i++ ) {
  sum = sum + buf[start+i]; 
 }

 return sum;
}

static void show_help(char* argv[])
{
 printf("Usage: %s  filename  start_offset_in_file(hex) "
   " end_offset_in_file(hex) \n", argv[0]); // TODO: use GNU's gettext 
 exit(1);
}

int main(int argc, char* argv[])
{
 int stream;
 char* buf;
 struct stat st;
 unsigned long start_offset, end_offset;


 if(argc != 4) {
  show_help(argv);
 }

 stream = open( argv[1], O_RDONLY);

  if (stream == -1 ) {
  printf("Error opening input file!\n");
  exit(1);
 }


 // get file size
 if( fstat(stream, &st) != 0 ) {
  printf("Error, unable to get file size!\n");
  exit(1);
 }

 printf("Input file size = 0x%X bytes\n", st.st_size);

 // allocate buffer for the file size obtained above
 buf = (char*) malloc(st.st_size);
 if( buf == NULL ) {
  printf("Unable to allocate memory for file buffer!\n");
  exit(1);
 }

 // read the opened file to buffer
 read( stream, (void*) buf, st.st_size ); 

 // calculate checksum, passing in the buffer and start offset 
 start_offset = strtol(argv[2], NULL, 16);  
 end_offset = strtol(argv[3], NULL, 16);  

 if ( start_offset >= end_offset  ) {
  printf("Error! Wrong parameter. "
  "end_offset should be bigger than start_offset\n");
  
  free(buf);
  close(stream);
  exit(1);
 }

#ifdef DEBUG // check string conversion routine
 printf("start_offset = 0x%X\n", start_offset );
 printf("end_offset = 0x%X\n", end_offset );
#endif

 printf("File checksum (from offset 0x%X to 0x%X)= 0x%X\n", 
  start_offset, end_offset, 
  sum(buf, end_offset - start_offset, start_offset) );  

 free(buf);
 close(stream);

 return 0;
}

This bug won't bite you if you do development on 64-bit Linux/Unix systems if pay attention to it.

Never take anything for granted when developing on 64-bit or multilib systems

This document is a good source of information on 64-bit/multilib systems portability, particularly for programmers and advanced sysadmins.