Using oeaudit.py

In the last days I have cleaned up my OE Audit tool and it should be usable by everyone now. The tool requires two inputs, one is the list of packages to be build by OE for the given configuration (distro, machine) and the other is the FreeBSD auditfile. This FreeBSD auditfile can be automatically downloaded.

Without much more overhad, here we go.


$ bitbake -s > available
$ export PYTHONPATH=/place/bitbake/lib
$ /OE/contrib/oeaudit/oe_audit.py -f
This will fetch the auditfile from the FreeBSD project for you
$ /OE/contrib/oeaudit/oe_audit.py -a auditfile -p available
Now you will see a list of vulnerabilities in the packages OE is going to use.

Dealing with security issues in the context of OpenEmbedded

One thing that has bothered me while being at Openmoko is the lack of Security Response by the OpenEmbedded Crew. In one way a security issue is just like any other bug and distros don't upgrade each package for each bug fixed upstream but it is getting worse when the security issues exists in the default installation, in a daemon listening to network traffic and such with ready to get exploits on the network.

I think it is really unethically to go around and claim how great OpenEmbedded is and then companies like Openmoko, Palm, etc. ship vulnerable software to their users and it is easy to pass the black pit to companies actually using OpenEmbedded, let me say it is too easy.

There are various things one can do to address these problems. One option is to downgrade and use the classic Buildroot as their maintainers seem to address vulnerabilities in time. I use the word downgrade as these systems provide less functionality, flexibility than OpenEmbedded, e.g. they lack the creating of SDKs, chosing the libc (glibc, uclibc, eglibc) but then again they do their homework and provide people with security updates in time, the other option is to go to a distribution like Debian or Fedora with a proven track record of handling security issues.

But I'm going to talk about the third option that includes improving OpenEmbedded. I had the idea while being at Openmoko but the guy who was assigned to do this was laid off shortly afterwards so it never happened. In general for every package we ship in OpenEmbedded there is an established distribution (e.g. FreeBSD, Debian, Fedora) that is shipping it as well. Or in the seldom cases where OpenEmbedded is the first adopter, the software is kept current and there is not much security research anyway. This means that to provide security upgrades to our users we only need to monitor the big and established guys and that sounds like something that can be partially scripted.

I'm using FreeBSD on my servers and in the FreeBSD world there is an application called portaudit which is looking at your built/installed packages and is comparing the name, package version and patch release to a list of known security issues in the ports tree and then asks you to upgrade, Gentoo has a similiar XML file for each security incident, Debian has a security feed as well.

A long story short, on a flight to iceland I was hacking a python script called oe_audit.py that is using the FreeBSD auditfile and the output of "bitbake -s" (the list of providers and their versions) and then starts comparing these lists. Right now the script is inside the OE tree, it is still a gross hack but I will improve it to be a proper python script. In its first version it has found issues with plenty of packages in OpenEmbedded and thanks to the help of some we are down to only a couple of issues in our tree. In general addressing security issues is not that hard, follow a couple of mailinglists, look at websites, when a CVE is published search for the patch, apply it to our version, be done. Specially given the amount of OE developers we could nominate a security sherif each week that has to do the upgrades... It is not that we see more than three upgrade a week anyway... So this week it would have been Pango, Php, Pulseaudio...

Using setitimer of your favorite posix kernel

As this was requested in a comment of a previous post and knowing your kernel helps to write better performing systems here is a small information of how to use the interval timers provided by your posix kernel.

What is the interval itime?r

The interval timer is managed and provided by your kernel. Everytime the interval of the timer expires the kernel wil send a signal to your application. The kernel is providing three different interval timers for every application. The different timers are for measuring the real time passed on the system, the time your application is actually executed and finally the profiling timer which tmes the time when your application is executed and when the system is executing on behalf of your application. More information can be found in the manpage with the name setitimer.

Why is it useful?

In the QtWebKit Performance Measurement Utilities we are using the interval itimer as the timing implementation for our Benchmark Macros. To be more precise we are using the ITIMER_PROF to measure the time we spend executing in the system and in the application, we are using the smallest possible precision of this timer with one microsecond. The big benefit os using this instead of elapsed real time, e.g. with QTime::elapsed, is that we are not depending so much on system scheduling. This can be really nice as even with a lightly crouded system we can generate stable times, the only thing influecing the timing is the MHZ of the CPU.

How is it implemented?

It is a kernel timer, this means that it is implemented in your kernel. In case of Linux you should be able to find a file called kernel/itimer.c, it defines the syscall setitimer at the bottom of the file. In our case the SIGPROF seems to be generated in kernel/posix-cpu-timers.c in the check_cpu_itimer routine. Of course the timer needs to be accounted by things like kernel/sched.c when scheduling tasks to run...

How to make use of it?

We want to use ITIMER_PROF, according to the manpage this will generate the SIGPROF. This means we need to have a signal handler for that, then we need to have a way to start the timer. So let us start with the SIGPROF handling.


Elapsed time handling
static unsigned int sig_prof = 0;
static void sig_profiling()
{
    ++sig_prof;
}



The signal handler
    struct sigaction sa;
    sa.sa_handler = sig_profiling;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = SA_RESTART;
    if (sigaction(SIGPROF, &sa, 0) != 0) {
        fprintf(stderr, "Failed to register signal handler.\n");
        exit(-1);
    }



Start the timer
tatic void startTimer()
{
    sig_prof = 0;
    struct itimerval tim;
    tim.it_interval.tv_sec = 0;
    tim.it_interval.tv_usec = 1;
    tim.it_value.tv_sec = 0;
    tim.it_value.tv_usec = 1;
    setitimer(ITIMER_PROF, &tim, 0);
}


Discussion of the implementation

What is missing? We are using the sigaction API... we should make use of the siginfo_t passed inside the signal handler.

What if we need a higher precision or need to handle overflows?
There is the POSIX.1b timer API which provides timers in the nanosec region and also providers information about overflows (e.g. when the signal could not be delivered in timer). More information can be found when looking at the timer_create functions.

When is the interval timer not useufl?

Imagine you want to measure time it takes to complete a download and someone wrote code like this:

QTimer::singleShot(this, SLOT(finishDownload())), 300000);


In this case to finish the download a lot of real time will pass and the app might be considered very slow, but it in terms of the itimer only little will be executed as the time we just sleep is not accounted on us. This means the itimer can be the wrong thing to use when you want to measure real time, e.g. latency or time to complete network operations.