28 January 2011

Building WebkitGTK on Windows

update Mar 2014: The issue regarding mingw-w64 winapi no longer valid as tried with version 3.1.0 and webkitgtk 1.10.2

brief instruction for 1.6.1:
Download source from here file: mingw32-libwebkitgtk-1.6.1*.rpm (opensuse's patch included)
With 1.6.1 we can use msys-perl 5.8.8 instead of strawberry's and none of my patches below except datadir.patch applicable. At maximum configuration (without geolocation) we can use:

configure --enable-mathml --enable-link-prefetch --enable-animation-api --enable-register-protocol-handler --enable-web-timing --enable-datagrid --enable-page-visibility-api --enable-3d-rendering --enable-image-resizer --enable-touch-icon-loading --prefix=/c/GTKDEVEL --with-gtk=2.0 --with-target=win32 --with-unicode-backend=glib


which require additional gstreamer, in my brief test it gain score: 100 on ACID3 and 296 on http://html5test.com. However there are some regressions: annoying text/image flickering and several widget misplaced also plugins still not implemented yet. Best gtk to use with is probably 2.24.8.

  To shrink the runtime dll size to around 40% smaller, force the use of -Os flag (need to manually edit Gnumakefile) or disabling SVG/video will reduce size significantly.



This remaining part is a rough guide to build older WebkitGTK 1.2.x on Windows using MinGW32+MSYS (not cross-compile)

Requirement:
In addition to MinGW+MSYS
important note: It seems the best MinGW to used is the official mingw32 GCC (specifically 4.2, 4.3 or >= 4.5)
Avoid MINGW-W64 (more specifically its win32 api), it will (silently) break JSCore which eventually make many page rendering anomalies.

We need PERL >= 5.8 but since msys only provide 5.6 (there is perl 5.8.8 msys) and msysGit has incomplete Perl 5.8, our choice is Strawberry or ActiveState and as both don't have POSIX feature, autoconf won't run properly (that explain why we patch makefile later). Install perl in non spaced folder like C:\Perl then copy perl.exe and perl[version].dll to MSYS' bin folder

We also need bison, flex and gperf the first two available officially here while gperf (msys version) available here extract them to MSYS' folder.

WebkitGTK (stable) from http://webkitgtk.org/


version 1.2.7 (my last build)

Dependencies:
GTK Bundle 2.22 here
download the followings from here
Gettext (both runtime and tool)
JPEG
Enchant
SQLite
LibXML
GnuTLS
LibSoup (source) here
Intltool here
LibXSLT (source) here

That's the minimal, if we enable video support add GStreamer too
We can use older GTK <2.18 but basically we need to coupling it with at least Glib 2.22.5 rather than supplied Glib 2.20.
Note:
- I only tried GTK < 2.18 since newer are buggy
- except for GTK Bundle, we need both binary and -dev package.


Extract them to C:\GTKDEVEL folder except for Libxslt and LibSoup we need to compile it first

Preparing MSYS
making a profile file:
Sometime we need different configuration for different situation therefor we can make a config file that we can switch between by running it.

example for GTK-related:

export PATH=".:/usr/local/bin:/mingw/bin:/bin:/c/GTKDEVEL/bin:/c/perl/bin:/c/perl/site/bin:/c/WINDOWS/system32:/c/WINDOWS:/c/WINDOWS/System32/Wbem:."
export CFLAGS=""
export CXXFLAGS=""
export LDFLAGS="-L/c/GTKDEVEL/lib -Wl,-s"
export CPPFLAGS="-I/c/GTKDEVEL/include"
export PKG_CONFIG_PATH="c:\\GTKDEVEL\\lib\\pkgconfig"
export PERL5LIB="/c/perl/lib"
export CC=gcc
export CXX=g++
export lt_cv_deplibs_check_method="pass_all"


save above lines as "setgtk" in MSYS' bin folder, now type "source setgtk" in MSYS console everytime we're compiling GTK related stuff.

Compiling LibXSLT
We need to compile this ourselves to use it with WebkitGTK instead the binary as pointed in xmlsoft.org
extract the source somewhere
Run MSYS, cd to source folder and type:
(we may need to disable pthread if we found error)

source setgtk
configure --prefix=/c/GTKDEVEL
make install


Compiling LibSoup
We need to compile this as 1.2.7 require latest version that's not available as binary (thanks to Peter)
cd to source folder and type:

configure --prefix=/c/GTKDEVEL
make install


Compiling webkit
extract the source somewhere
Run MSYS and cd to source folder

Patches:
mingw32.patch (patch from opensuse for 1.2.6,1.2.7)
datadir.patch (from pawel)
My patches: (use at your own risk)
configure.patch (enabling iframe-sandbox by default)
gtk216.patch if build against GTK < 2.17.3
nosvg.patch if svg support disabled

download and copy patches to webkit source folder
applying patches by typing:
patch -p0 -i patchfile

Some perl files that autogenerate C++ sources are using hardcoded GCC path as "/usr/bin/gcc" instead of $CC variable. This is invalid for MSYS and we need to manually change these files:
WebCore/dom/make_names.pl
WebCore/css/make-css-file-arrays.pl
WebCore/bindings/scripts/IDLParser.pm
replace "/usr/bin/gcc" with "gcc" or alternatively apply this patch (probably broken on different version)

now configure with this command: Assuming we don't have gstreamer:
(If we want no pthread, use --disable-fast-malloc too)

configure --prefix=/c/GTKDEVEL --with-target=win32 --with-unicode-backend=glib --enable-web-sockets --enable-3D-transforms --disable-video


before "make" apply makefile patch first!:
patch -p0 -u -i makefile.patch

Since compile can take hours we'd better use parallel "make". For example if we have dualcore type:
make -j2 > logs.txt 

And before you complain "this" and "that" issue please check my bugreport first, and of course please report new bugs :) there are not many Windows zealot there :D
Good Luck!

13 comments:

  1. Thanks for all your articles, now I have webkitgtk+ running.

    I want to create a win application and use that webkitgtk as my GUI.

    Is this possible to make something more than just a browser that only displays html? I need more integration between python <> javascript, how do I make bindings? I want to pass data from python to >> javascript, I need to call javascript function from python. And the other way: need to be able to call python function from javascript and pass some data.

    Is that possible? Or we are able only to display html, and not use it as a gui for our app? Who needs another browser... really?

    ReplyDelete
  2. You mean a desktop webapps? like pyjamas+pywebkitgtk? I'm sure it's possible there some articles already.

    But honestly I haven't tried it myself, so far I just use it to display wikipedia dump (a.k.a as gtkhtml2 replacement).

    ReplyDelete
  3. There are some solutions, still evaluating them:

    1. To use document.title as a virtual pipe to transfer data/commands in both ways. In python there is a func to set that title and also you can listen to changes using "title-changed" event. In javascript you can set it normally and listen to it by implementing a setInterval() and checking when the values changes.

    2. "pyjavascriptcore" project:
    http://bazaar.launchpad.net/~soto255/pyjavascriptcore/trunk/files

    3. "jswebkit" project:
    http://code.google.com/p/gwrite/source/browse?repo=python-jswebkit

    But I don't think I'm gonna implement any of them. I come to conclusions that this webkitgtk is not ready for win32. Have been creating simple pages and some simple scripting and there are some serious problems. Now there is another bug that takes 50% cpu of the python process - randomly started lagging, I did nothing fancy, I only remember doing some resizing of the window.

    ReplyDelete
  4. Could you explain the usecase for 50% CPU usage issue? I'm aware of this one when I opened some unicode page such google.co.jp but I'm not sure

    ReplyDelete
  5. That was random, the page was utf-8 with only a few paragraphs with english text.

    Those that want a stable/fast webkit engine on windows there is really only one solution - use chromium engine which is based on webkit, that one used in Google Chrome. There is a project that makes easy to embed chromium:

    Chromium Embedded Framework (CEF):
    http://code.google.com/p/chromiumembedded/

    It's for C++ (there are bindings for C).
    There binary releases that you can use out of the box.

    There is also port for Java:
    http://code.google.com/p/javachromiumembedded/

    and Delphi:
    http://code.google.com/p/delphichromiumembedded/

    Zakkum: do you know if it is possible to use it from Python, how hard is it to use the api in the dll provided by CEF (I read that there is a COM support in python using pywin32 projet)? I opened the "libcef.dll" in DLL Export viewer and I see about 70 functions available:

    cef_browser_create
    cef_browser_create_sync
    cef_create_url
    cef_currently_on
    cef_do_message_loop_work
    cef_initialize
    cef_parse_url
    ........

    That are creating browser functions, string conversion functions and others.

    ReplyDelete
  6. That COM maybe http://sourceforge.net/projects/comtypes/
    an IE hook too

    about the binding if it's C++ you could try SWIG. (I never made one)

    ReplyDelete
  7. Yeah, webkitgtk still as slow as the first time I use pywebkitgtk. But I kind of like GTK and its sporadic libraries availability. Though development often slow and stalled.

    ReplyDelete
  8. @cezary :
    This link talk about using pywebkit gtk (or mozembed) as the GUI
    * http://www.aclevername.com/articles/python-webgui/

    Then, you could look at the projecto hotot. It's an opensource Python microbligging client (it's mostly used in twitter) this project uses pywebkitgtk for its UI. I think instead of asociating the change of the title they asociate the calling of an alert.
    Here is the link: http://hotot.org/

    ReplyDelete
  9. Could you post your binaries? I'd really like to avoid the whole process.

    ReplyDelete
  10. If you want binaries you could get it from http://ftp5.gwdg.de/pub/opensuse/repositories/windows:/mingw:/win32/SLE_11/noarch

    But they seems configure it in wrong way (using pango instead of freetype), webkitgtk win32 port isn't fully done yet. And why you want binary if you're C developer? If you want pywebkitgtk I have it here http://opensourcepack.blogspot.com/2011/01/conservative-all-in-one-pygtk-installer.html it's version 1.2.7 but have better rendering than 1.6.1 despite lower score of ACID3, you also can extract the installer using 7-zip to get the binaries too.

    ReplyDelete
  11. Hi TumaGonx,

    I jumped on your blog as I'm trying to compile the whole stuff ; great article !

    Unfortunately, the URLs for most of your patchs are broken. Any plan of putting them back online ?

    ReplyDelete
    Replies
    1. Thanks for the head up

      starting from version 1.8.0, WebkitGTK doesn't need patch anymore :) more importantly there is flash support fixes that probably released after 1.8.1 if you wish to wait ;)

      this guide relatively obsolete for version 1.8.0 (need several new dependency library builds)

      Delete
    2. OK thanks for the info, didn't even try the latest version 'cause I prefer to avoid the unknown (and possible random wizard stuff ;-))...

      So I'm going to take at look at 1.8.0 and try my best.

      Delete