Some Blog

Palm released WebCore/WebKit sourcecode for their Palm Pre..

2009-06-19T00:44:00.003+02:00

My friends just told me that you can get the sourcecode from this place. Or a direct link to their WebCore modifications and WebKit modifications.

Taking over memprof

2009-06-08T07:29:00.003+02:00

Where did all my memory go? Who is allocating it, how much is being allocated? From where were theses QImages allocated? valgrind provides an accurate leak checker, but for a running application you might want to know about allocations and browse through them and don't take the performance hit of valgrind (e.g with massif).

There is an easy way to answer these questions, use memprof. memprof used to be a GNOME application, it was unmaintained, the website was gone from the net, but this tool is just way too good to just drop out of the net. After trying to reach the maintainer twice I decided to adopt the orphaned thing.

Check the application out, it is great, it helps me to get an overview of memory allocations for WebKit/GTK+...

Scrolling in GTK+ and WebKit/GTK+

2009-05-10T18:05:00.005+02:00

Model/View GtkAdjustment and GtkScrollbar

The GtkAdjustment is probably best described as a model for scrolling. It has several properties, e.g. the current position (value), the lower and upper possibilities, and the size increments. The GtkScrollbar is operating on top of a GtkAdjustment. It is responsible for taking user events and painting. When scrolling it will look at the lower and upper properties, it will update the value in case of something is happening.

WebCore::Scrollbar in WebCore

The Scrollbar is created by the Scrollbar::createNativeScrollbar "factory". For many platforms the painting/theming and behaviour is entriely done within that class. For GTK+ we will use a GtkScrollbar, this widget happens to not have its own GdkWindow which makes painting a bit more easy, we will just forward the original expose event and let it draw as well (from ScrollbarGtk::paint). WebCore::Scrollbar in WebCore are used in two places. One prominent one is the WebCore::ScrollView which is the base class of the WebCore::FrameView and will be used to enable people to scroll on their content in horizontal and vertical direction, the other big user are scrollable div's (we have a manual test in WebCore/manual-tests/gtk/ to test positioning of these scrollbars).

Scrolling on GtkWidget

The scrolling starts quite innocently with the description of gtk_widget_set_scroll_adjustments. In case of WebKitWebView in WebKit/GTK+ we do want to support scrolling so we need to return TRUE... but how. If you take a look at the class structure of GtkWidget there is a GObject signal identifier you will have to set with the signal you have created. This is done in the WebKitWebView class init and we will remember and use the GtkAdjustment, e.g. set by a GtkScrolledWindow, in the WebCore::ScrollView (base of WebCore::FrameView class). The usage of GtkAdjustment allows to have different means of scrolling, e.g. by fingers on a touchscreen, or by a wheel... the representation of the scroll concept will change, the implementation not...

The problem of mainFrame and subframes

So far we will most of the times only have external GtkAdjustment set on the mainFrame that is embedded in something like a GtkScrolledWindow but there are pages that will create a frameset and you will have subframes that require scrolling (my test case here is Google Images). What we are ending up with is having a WebCore::FrameView with GtkAdjustments set from the WebKitWebView and some WebCore::FrameView without GtkAdjustments set at all. On cases without a GtkAdjustment, there is no one that will place a GtkScrollbar (and resize the WebKitWebView to be next to it), so the WebCore::ScrollView will create a ScrollbarGtk to handle this job. This creates the siutation that one WebCore::FrameView class will or will not have a Scrollbar (and manage its size...).

The current solution

When we have a GtkAdjustment set in the WebCore::ScrollView do not think about scrollbars, the need of them, the positioning of them at all. Simply update the properties including the current value, the visibleWidth and contentWidth. The upside is we have a GtkWidget that is working like a GtkWidget and can be embedded into a GtkScrolledWindow or a MokoFingerScroll (back in the better days). The downside of this include that we have more platform specific code and that we will not send onscroll events... due not going through ScollbarClient::valueChanged....

The new solution

Wrap the GtkAdjustment we get set into a ScrollbarGtk but do not create a GtkScrollbar. This should create a WebCore::Widget with WebCore::Widget::platformWidget() returning zero and this should set the width() and height() of this Scrollbar to zero meaning that the ScrollView calculation (e.g. updating visibleWidth/visibleHeight) is not negatively influenced, we can kill some more #ifdef PLATFORM(GTK) from WebCore::ScrollView and that we properly send onscroll events as all scrolling is going through the WebCore::Scrollbar code path, like with every other port. For having navigation working we must be sure to reset the GtkAdjustment, due the nature of the FrameView this is best done when creating the ScrollBar... The progress of this can be tracked in bug #25646.

FrameView and navigation

2009-05-10T16:54:00.002+02:00

Visiting a site can have certain changes on a FrameView that will not be undone when leaving it. The most prominent example is a fixed background (by CSS) which will disable using blit on scrolling. There might be different things as well. Another one would be the PageCache and navigation to pages in it. The result is that for every page navigation you will create a new FrameView. For most/all ports this is done in FrameLoaderClient::transitionToCommittedForNewPage(). There is even a convience method shared between most ports in the form of Frame::createView.

From an API point of view this means that there is one WebKitWebView (in terms of Gtk+) and this WebKitWebView has one WebCore::Page and this page has exactly one WebCore::Frame the so called mainFrame but throughout the lifetime of WebKitWebView there will be multiple instances of FrameView's setup on the WebCore::Frame mainFrame. This also means that there are times were multiple WebCore::FrameView instances are around but only one should be active at a time.

For WebKit/GTK+ we will need to make sure that all the state of a WebKitWebView is transferred/set to the WebCore::FrameView when it gets created and ever change to a WebKitWebView gets forwarded to it. The most prominet state information are colors and GtkAdjustment (another posting). I think Gustavo is also about to introduce Scrollbar policies soon.

Embedding a GtkWidget into a HTML page

2009-05-08T05:34:00.003+02:00

In the last few days I have suprisingly (actually not, it comes natural when resigning from a job) more time to do stuff that I really want to do. For the last months my webkit involvement has been mostly been reviewing patches created by other people but now I have done something I wanted to add for quite a long time...

I think one of the benefits of having a native web renderer widget in the toolkit is the integration that is possible. You can embed it into other native widgets and you should be able to embed native widgets into the web renderer. I have been working on the second part and the result can be seein in this bug. The attached example is putting the Gtk Scribble example into every plugin.

In the arora web browser this feature is used to implement ClickToFlash and I plan to still this idea too.

Long overdue...

2009-03-25T08:29:00.002+01:00

A long overdue blog post... I'm currently in Taipei... canceled my original flight back to Germany, instead I will go to Hong Kong and then probably back to Taipei. So if you are in this area and would like to talk about WebKit, Linux, KDE, OpenBSC, OpenEmbedded... drop me a mail.

Geo Clue for WebKit/Gtk+

2009-02-03T16:10:00.002+01:00

Apple was adding support for the W3C Geolocation API, I started with a implementation using Gypsy, then after some comments moved on to Geoclue. Then Aurelian Maga was more lucky getting geoclue to emit something useful and fixed my logic errors and then it was about time to put the code into the repository.

The next things will be adding API to control the privacy settings, enabling/disabling support for it and so on..

FOSS.IN and the need for more events

2008-12-18T12:18:00.004+01:00

I have sadly missed this years FOSS.in. The goal of this conference is to turn India into a nation of FOSS contributors. There are plenty of people, awesome food!, there is a huge software industry, companies like Tata Consulting are even on Level 5 five of the CMMI model. This means there is a huge potential! But when I get my daily mail on webkit-dev I recognize that it is still a long way from simply consuming, to try to attempt to think, to contributing. And that many more events like FOSS.in need to occur until there is a noticeable difference.

On the other hand with people like Girish, Prashant, pradeepto, Shreyas I have high hopes that the goal will be reached.

API Reference Manual for WebKit/Gtk+

2008-11-22T03:11:00.003+01:00

As of two days ago the Gtk+ buildbot is finally running the test suite after every build. And today we even have the first regressions that we need to address/understand. The short-, mid- and long term goal is now to get as many tests running as possible and keep them running. There are quite some bugs to be fixed in WebKit/Gtk+, our base libraries, our testing tools so there is plenty of fun left for everyone.

In the webkit IRC channel we sometimes get questions regarding the API of the WebKit/Gtk+ port and if there are examples. On the one hand we are pretty strong. We have the GtkLauncher, some early adopters, when adding API many people made sure that we have API documentation. On the other hand we are weak as there is no introduction, no generated reference manual, no downloadable samples. So I got a bit sidetracked from the regression work and started to integrate gtk-doc and to cut a long story short: You can find an initial version here.

On to the gory details. Integrating gtk-doc requires recursive make due the way the gtk-doc.make rules are written. This file is copied in from gtkdocize. When dealing with documentation you can not have srcdir != builddir. The html result will be put in the source directory and if you try to squeeze in the @VERSION@ you will need srcdir == builddir. Starting from GtkDoc in live.gnome.org and Florian's GPE guide one finds pretty much everything one needs to find. Some things needed some look at the tool sourcecode (perl, eeek!) but are clear now.


WEBKIT_API type
function_name (function*...);

To make gtkdoc-scan parse the above as a function you will need to pass --ignore-decorators="WEBKIT_API" to the tool. Magically plenty of symbols turned up in the -unused.txt file and I was able to copy them into the -sections.txt. Now the tool refused to find any documentation. So remember WebKit/Gtk+ is mostly written in C++ and gtkdoc-mkdb simply does not know about the ".cpp" file extension. So pass --source-suffixes=c,cpp to gtkdoc-mkdb and it will find the sourcecode. Ah functions and their documentation, almost perfect. The last undocumented bit is for -sections.txt. E.g. you have the various _get_type(void) methods and there is little benefit of having them pop up in the API. Inside your "<SECTION>" you can put them into a "<SUBSECTION Private>" and they will be ignored.

The next thing is to get it actually merged into WebKit and integrate having uptodate API reference in the release process.

Link WebKit/Gtk+ faster

2008-11-17T21:07:00.003+01:00

Last year in India Simon showed me an ancient and well hidden qmake feature. With this hidden feature you can easily move some files to a new library.

Why is this good? Well, the new library only contains the files you are currently working on. In most of the cases this will be only a hand full. You will get blazing fast linking and round trip times if you do so.

So today I worked on a issue with SVGFont.cpp and my roundtrip time was way too slow. So I decided to apply this hack to our superb autotools based buildsystem!

So I have a hacky patch which is turning noinst libraries to normal libraries to avoid the running of ar/ranlib. I have created a libWebCoreHack.la, GtkLauncher and DumpRenderTree link to that library now. And you have to move the sources you work on from webcore_sources to webcorehack_sources. With this hack I have almost instant turn around times again.

How to use it: Whenever you work on a area move the usual suspects to the webcorehack_sources, relink once and then you can benefit from the fast link times.

*happy*

Pushing things forward

2008-09-05T14:47:00.002+02:00

There is one thing of Tiny SVG1.2 that I really like. It is the possibility to embed audio and video. For video you can do transformations, filters and the usual stuff of SVG. TinySVG1.2 is popping up in more specs and recently I began to read DIMS again and well and thanks to the support of GMIT I had a go at it.

This shows parts of the TinySVG spec and the video was replaced with Code Rush. It is dog slow and needs some refactoring to SVGSMILElement and parts of the SVG RenderObjects to be merge able. The code should popup in my holger/dims branch soon.

Hacking on the WebKit codebase is so much fun that I seriously wonder if I want more of that...

Acid3, make QtWebKit catch up

2008-08-26T05:03:00.004+02:00

The Mac, Windows and Gtk+ port get 100/100 and pixel perfection for the acid3 test for quite a while but the Qt port was stuck somewhere else. How can this happen? For QtWebKit we decided to use as much of Qt as possible. So instead of relying on ICU we used the Qt Unicode support, instead of relying on libxml2 we used the QXmlStreamReader, instead of doing font matching and font/glyph caching we solely rely on QFont and QTextLayout to do the job. One had to apply some minor fixes to the Qt version of classes such as XMLTokenizer or TextCodec, or integrate SVGFonts into the Qt port as well...

What is next? We will have to get the rendering to be pixel perfect as well and then can merge the changes chunk by chunk. I'm going to sleep now, enjoy the screenshot.

QtWebKit in action

2008-02-18T15:43:00.002+01:00

Following up Simon's QtWebKit in Action and to answer some comments from previous blog posts. Yes with QtWebKit you can embed QWidgets into HTML, you can control their position, size, etc. with JavaScript and apply CSS, invoke slots and such things. Lars, Simon and Girish created a demo for the DevDays it can be seen in action on the site Simon mentioned and you can grab the source from svn://labs.trolltech.com/svn/webkit/demo.

QtWebkit Status update

2008-02-16T21:18:00.005+01:00

Simon and Jasmin went through the API of QtWebKit. Most of the changes where changing names and parameters of methods, signals and slots. And some minor but important behaviour changes. The Plugin API is probably the only remaining feature for Qt4.4 and Lars and Simon have already progressed nicely.

Simon went ahead and integrated QtWebKit into the Assistant on the way finding/marking of text was added to the QWebView and QWebPage API.

Benjamin (icefox) continues adding the basic features to our demo browser. It is an awesome testbed to test QtWebKit, see what APIs is missing and what pages need work.

We (this means mostly Simon) have fixed some rendering bugs of control/form elements with certain styles. Interesting remark. QCommonStyle::drawComplex is drawing PE_FrameLineEdit if you ask it to draw PE_PanelLineEdit, but only if your lineedit has a frame. Most of the styles assume a pristine/virgin QPainter when painting. In QtWebKit this is not true so we have created a StylePainter which will set a Qt::NoBrush and probably other things in the future. We know that painting scrollbars need a tweak when topLeft() is not equal to (0,0) on some styles. Note to myself if you see a drawRect call, the rect will be filled with the brush...

We (see above for the actual meaning of we) have enabled the PageCache. The result is that you can go back and forth without waiting minutes. At least I had to wait minutes as I'm on a GPRS line. For other the effect is not too noticable as they will have broadband and because we are likely to not properly cache the ResourceRequest/ResourceHandle and do a real request on going back. More investigation is needed.

I have been doing some cosmetic changes, I liked the Safari3 feature to resize multiline text edits. I made sure that it is working in QtWebKit as well. I didn't see the sizegrip icon and started to poke WebCore, RenderLayer and wondered why I can't resize. After a while I found a call to WebCore::Settings (each WebCore::Page has a WebCore::Settings and we wrap around it with QWebSettings) in the CSSParser which defines if certain things are resizable, so I set this to true in QWebSettings and still didn't see the icon (after adding a bunch of them to our WebKit qrc). It turned out that the way we used BitmapImage to show PlatformResources wasn't too clever, we stuffed a QPixmap in there but didn't use the size or the content of it.

I have started to look into image usage, both QPixmap and QImage in WebKit and started to look for other areas where we leak, keep a reference too many and such things. Candidates of memory waste are keeping both a QPixmap and QImage of the same image, caching the downloaded data (in the WebCore::DocumentLoader). At this point I would like to find a guide to script gdb, I want to print the last three frames everytime ::ref and ::deref of a DocumentLoader or SharedBuffer gets called. Alternatively I will have to toy with VirtualBox to install Solaris, or get OSX Server and try to legally run it in a VM to get access to DTrace.

Memory usage. I have started to profile QtWebKit. I have used examp console of o-hand.com and massif a valgrind tool. I have to get used to the new reporting of massif. It is not generating a postscript file anymore but you have to use a tool called ms_print to look at the snapshots. I hope some one is writing a cool gui for it soon(tm).

The Joys of debugging WebKit

2008-01-04T18:13:00.000+01:00

When working on WebKit and the platform layer you come to a point where your primary browser is going to be WebKit based (or already is because you use Safari, OmniWeb,...). If you happen to use a platform that is not developed by Apple (Qt, Gtk+, Wx) you are going to be the first to see certain issues. I'm currently in this fantastic situation and have issues to see, understand and resolve.

This is a brief summary of two issues Simon and me were looking into at the wonderful 24C3 this year. We have looked into GMail and Yahoo Mail.

The Yahoo issue started with the attempt to sign into my ancient account, and this time the issue wasn't to remember my password. Our Qt Demo Browser went into a infinite loop of redirections. So what was wrong? The obvious thing to blame is our networking implementation and the brand new QNetworkAccessManager and related classes. And yes some issues were found in our networking implementation. For redirections we have told WebCore more than once that a redirection is happening, we have missed one redirection code. To cut a long story short, this didn't fix the issue. Cookies are always important so we checked them, fall victim of wireshark as it is truncating the decoded HTTP header and it looked like we didn't send the full cookie string. It fooled us for a second or two and then we remembered our HTTP and Cookie implementation must be perfect as Thiago wrote most of it. So I started to find out where this redirection gets triggered, joy it came from a timer, so who started it? And we saw that it was coming from JS, so it was something embedded into the site. We signed in with https://mail.yahoo.com, got redirectred to a US server, then to a German one and then sent back to http://mail.yahoo.com. Note that the last redirection is going from http to https. So once again the WebKit Web Inspector came to rescue and we searched for "http://mail.yahoo.com" in the sourcecode view and two hits where revealed, after not starting the redirection timer when it is going to http://mail.yahoo.com. The second hit was interesting as it pointed to a piece of javascript code that was asking the DOM for cookies. Our CookieJar implementation was a bit selfish as it didn't want to share all relevant cookies with the DOM but only one, after convincing it to return all relevant Cookies the login and sending mails finally worked!

The Gmail issue. You can sign in to Gmail as we pass the google cookie test, you see the full version of Gmail and we were not able to click any links. The JS Code google is sending is highly obfusicated so debugging is no fun at all, we went for the easy solution and updated our User Agent string, magic happened and we were served another version of it and it made our browser terminate in an assert in WebCore::FrameLoader. The next easy thing was to git merge the Safari-3-Branch into QtWebKit, recompiling, login, ASSERT, *joy*. With the usual and fantastic help of the apple folk I got pointed to #15765 and yes we were hitting this bug. I can tell you reducing Gmail to a test case that is actually exposing the bug is no fun, actually it is because it is challenging and something totally new for me. So after understanding and proposing a solution for this common issue, I have to fix the DumpRenderTool/run-webkit-tests relationship and I can then probably land this fix as well. With it you can go to gmail, browser your mail, see your chat logs but we still have some unimportant issues left like being able to send mails...

So QtWebKit is improving day by day, we mostly implement and fix code in the platform layer, sometimes go to other parts of WebCore and do fixes there.I think it is only a matter of time before we start doing fixes and improvements in the HTML, SVG and rendering implementation.

WebKit is highly addictive, try it out :)

PS: I'm going to leave my friends at GMIT at the end of the month, do the math.

WebKit Demos for Qt4.4

2007-12-19T17:04:00.000+01:00

The Qt4.4 Technology Preview 1 is out and webkit is integrated into it. It is the same code as on the public WebKit.git repository, actually the copy in Qt is always a bit behind as we do the development in the public git repository.

After integrating QtWebKit into Qt we needed a small demo application. So we wrote a small browser and the result can be found in demos/browser, an older snapshot can be seen below, and the rsync copy of Qt has a newer and better version. It makes a really good test case for our API and implementation. We see what bits are missing when doing interesting stuff, when trying to login and use webservices we see what is broken. So we started eating our own dog food and most of us enjoy it. This post gets written using this browser and a fully working gmail, flickr is quite near.

I hope you enjoy this small demo application and we will continue to improve this browser, QtWebKit and our Qt4.4 networking stack. So make sure to watch the rsync versions of Qt as it is highly addictive.

Inspecting the Web

2007-12-01T08:41:00.000+01:00

WebKit offers two neat tools for web developers. This is Drosera a JavaScript debugger and the Web Inspector. Some weeks ago I wanted to find out how hard it would be to enable the WebInspector for the WebKit/Qt port and it is really easy. Almost everything of the Web Inspector is portable code, ready to use by the available ports. So everything one has to do is to implement the InspectorClient clas which accounts for a couple of hundred lines of C++ code and you are done. The code can be found in this git branch and this going to be merged to qtwebkit and trunk once we have sorted out #15938.

In the below screenshot you can see the WebKit/Qt and WebKit/Mac Web Inspector next to each other. You can spot some errors on this screenshot, e.g. the WebKit/Qt port doesn't render search menus yet and when using it you can see some other issues in the drawing and networking code. So the Web Inspector makes a perfect test tool as well.