Cleaning up the cellmgr_ng and turning it into a MGW/STP

One year ago I was starting the on the BSC NAT application from a Hotel room in Munich, I was flying from Taiwan to Munich and was happy to see the snow from my window. Shortly after this I began to implement something we called cellmgr_ng. The job was to take MTPLevel3 coming out of a library and put the SCCP payload into the IPA protocol and send it to a MSC. As part of MTP Level3 I had to implement the link bring up and such as well (once again without having too much time to read all the specs).

The problems were as most of the times not the ones one would expect. The system behind the E1/T1 link was sending wrong SCCP messages, when a SCCP connection was not closed fast enough it would send out its own release message but was confused about the source local and destination local addresses. As a result I had to add connection tracking to the codebase, I am also rewriting/removing Point Codes to avoid crashes and make the MSC happy.

Last month and this month I had the pleasure to resume the work on this. My knowledge of MTPL3 is at least a bit bigger now, we have done the libosmocore split so I can use more features without copying code around and there was new demand.

The first big thing is to support a LinkSet with multiple Links in it. This allowed me to do a lot of cleanups, some are still to do but now the abstraction is a lot better and I will do testing and force link failures and see how the system is behaving. This will also end with VTY commands to manage links via the telnet interface, more statistics collected and reported, being able to start writing PCAP files for the linksets from the VTY command.

The second big feature is the addition of M2UA to the codebase. The code does not support fail over scenario's yet but this should be relatively easy to add inside the structure now.

The result is that we now have something that could be used as a MediaGateway and convert MTPL2/MTPL3/{SCCP,ISUP} into SCTP/M2UA/MTPL3{SCCP,ISUP} and convert the audio to and from RTP, the other part is a Signalling Transfer Point (STP) with some basic capabilities.

Using systemtap userspace tracing...

At the 27C3 we were running a GSM network and during the preparation I noticed a strange performance problem coming from the database library we are using running. I filled our database with some dummy data and created a file with the queries we normally run and executed time cat queries | sqlite3 file as a mini benchmark. I also hacked this code into our main routine and ran it with time as well. For some reason the code running through the database library was five times slower.

I was a bit puzzled and I decided to use systemtap to explore this to build a hypothesis and to also have the tools to answer the hypothesis. I wanted to find out if if it is slow because our database library is doing some heavy work in the implementation, or because we execute a lot more queries behind the back. I was creating the below probe:


probe process("/usr/lib/libsqlite3.so.0.8.6").function("sqlite3_get_table")
{
a = user_string($zSql);
printf("sqlite3_get_table called '%s'\n", a);
}


This probe will be executed whenever the sqlite3_get_table function of the mentioned library will be called. The $zSql is a variable passed to the sqlite3_get_table function and contains the query to be executed. I am converting the pointer to a local variable and then can print it. Using this simple probe helped me to see which queries were executed by the database library and helped me to do an easy optimisation.

In general it could be very useful to build a set of probes (I think one calls set a tapset) that check for API misusage, e.g. calling functions with certain parameters where something else might be better. E.g. in Glib use truncate instead of assigning "" to the GString, or check for calls to QString::fromUtf16 coming from Qt code itself. On second thought this might be better as a GCC plugin, or both.