crossroad 0.6 released: cross-building GIMP as an example

Hello all!

TL;DR; if you don't care at all about how crossroad works,
and you just want to cross-build GIMP in a few commands,
see at the bottom of the article.

Over the years, I have written a tool to cross-compile software under a GNU/Linux OS for other targets: crossroad. Well to this day, the only supported targets are Windows 32/64-bit. I realized I never advertised this tool much which — even though it has been originally built to help me build and hack a specific project (GIMP) — ended up as a generic cross-compilation system for Linux.

“The crossroads” where Robert Johnson supposedly sold his soul to the Devil, by Joe Mazzola (CC by-sa 2.0)

So today, I will explain crossroad by giving a concrete example:

Cross-compiling GIMP on GNU/Linux for Windows system

The basics

The cool stuff about crossroad is that it allows to cross-build as easily as a native build. For instance f your build system were the autotools, you would run the famous triptych ./configure && make && make install, right? Well with crossroad, you only wrap the configure step but the rest is the same!
cmake is also fully supported, even though my examples below won’t show it (check out the manual for more).

All what crossroad does is preparing you a suitable environment, with the right environment variables, but also wrappers to various tools (for instance, you get the x86_64-w64-mingw32-pkg-config tool inside a crossroad environment, which is no more than a wrapper to detect packages inside the cross-built prefix only).

Enough chit-chat. Let’s install crossroad!

pip3 install crossroad

Running crossroad the first time

I will now go through a full cross-build of GIMP for Windows 64-bit, from a brand new crossroad install. It may be a little long and boring, but I want to show you a full process. So you can just run:

$ crossroad --list-targets
crossroad, version 0.6
Available targets:

Uninstalled targets:
w32 Windows 32-bit
w64 Windows 64-bit

See details about any target with `crossroad --help <TARGET>`.

Ok so at this point, you may have no cross-compilation compiler installed. Let’s see what I need for Windows 64-bit.

$ crossroad --help w64
w64: Setups a cross-compilation environment for Microsoft Windows operating systems (64-bit).

Not available. Some requirements are missing:
- x86_64-w64-mingw32-gcc [package "gcc-mingw-w64-x86-64"] (missing)
- x86_64-w64-mingw32-ld [package "binutils-mingw-w64-x86-64"] (missing)

The package name was actually based off a Mint or a Mageia, I think. If you use Fedora for instance, you will want to install mingw64-gcc and mingw64-binutils.  Just adapt to your case.

Let’s check the output of crossroad again:

$ crossroad --help w64
w64: Setups a cross-compilation environment for Microsoft Windows operating systems (64-bit).

Installed language list:
- C
Uninstalled language list:
- Ada Common package name providing the feature: gnat-mingw-w64-x86-64
- C++ Common package name providing the feature: g++-mingw-w64-x86-64
- OCaml Common package name providing the feature: mingw-ocaml
- Objective C Common package name providing the feature: gobjc++-mingw-w64-x86-64
- fortran Common package name providing the feature: gfortran-mingw-w64-x86-64

We could stop here, but I actually know some programs make use of the C++ compiler, so let’s also install `mingw64-gcc-c++`. And now we are good to go!
I create a Windows 64-bit cross-build environment, that I will call “gimp-build”:

$ crossroad w64 gimp-build
Creating project "gimp-build" for target w64...
You are now at the crossroads...
Your environment has been set to cross-compile the project 'gimp-build' on Windows 64-bit (w64).
Use `crossroad help` to list available commands and `man crossroad` to get a full documentation of crossroad capabilities.
To exit this cross-compilation environment, simply `exit` the current shell session.

Dependencies

babl

Now let’s cross-build babl! Very easy one to start with because it does not have any dependencies:

$ cd /path/to/src/of/babl/
$ crossroad configure && make && make install

Done! You’ll notice that you don’t even need to specify a prefix. crossroad is taking care of it.

GExiv2

$ cd /path/to/src/of/gexiv2
$ crossroad configure

The configure ends up with a pretty clear error:

checking for GLIB... no
configure: error: GLib is required. Please install it.

Well basically you have to install glib. And then you think: the hell is starting! Do I have to build every dependency by hand? Fortunately no! crossroad relies on the OpenSUSE cross platform builds and can automatically download many dependencies for you. The reason I don’t do it for for all dependencies (babl, GEGL, libmypaint and GExiv2 in this tutorial) is when packages are either too old or non available. Install glib with:

$ crossroad install glib2-devel

That’s all! Neat right? Well you also have to install Exiv2 (trying to run the configure again, you’d see another error. I spare you the wasted time). You can run a separate command, or could have installed them both at the same time:

$ crossroad install glib2-devel libexiv2-devel

Try again:

$ crossroad configure

In the configuration summary, you’ll see that introspection and python bindings won’t be built-in. But since they are not needed for GIMP, we don’t care.

$ make && make install

And now GExiv2 has been cross-built too!

gegl

Now for gegl:

$ cd /path/to/src/of/gegl
$ crossroad configure

Once again, you’ll get a few dependency errors. I spare you the try and fail process. Here are all the other mandatory libs to install:

$ crossroad install libjson-glib json-glib-devel libjpeg8-devel libpng-devel

And for a fuller list of optional dependencies:

$ crossroad install cairo-devel libtiff-devel librsvg-2-2 librsvg-devel pango-devel libwebp5 libwebp-devel libjasper-devel gdk-pixbuf-devel

Well there are actually more dependencies you could install or build yourself, but I don’t think they are really needed for GIMP. I’ll just leave it there. Now I build:

$ make

But then… a build error (I’ll admit: I left a dependency error on purpose!)!

In file included from /usr/include/SDL/SDL_main.h:26:0,
from /usr/include/SDL/SDL.h:30,
from /home/jehan/dev/src/gegl/operations/external/sdl-display.c:35:
/usr/include/SDL/SDL_stdinc.h:74:20: fatal error: iconv.h: No such file or directory
compilation terminated.
Makefile:1277: recipe for target 'sdl_display_la-sdl-display.lo' failed
make[4]: *** [sdl_display_la-sdl-display.lo] Error 1
make[4]: Leaving directory '/home/jehan/dev/crossbuild/w64/gegl/operations/external'[…]

Well that’s a good example to debug. Look well at the paths… /usr/include/[…], they are definitely for the native platform! So something is wrong.
Also I don’t even remember having installed SDL through crossroad anyway. Well if you were to check out the config.log, you would understand why the configure script found it:

configure:22069: checking for sdl-config
configure:22087: found /usr/bin/sdl-config
configure:22100: result: /usr/bin/sdl-config

Sometimes, some projects would use their own *-config script instead of relying on pkg-config (SDL seems to also have a pkg-config file, not sure why we’d prefer usind sdl-config, but well…). And while pkg-config allows to really separate the build and target environments, I can’t remove the current $PATH because a lot of native tools are needed in a cross-compilation. All this to say: you may encounter issues with software distributing custom *-config script. Be warned!

Anyway let’s install SDL, don’t forget to re-run configure (otherwise the native lib will still be used), then build again.

$ crossroad install libSDL-devel && crossroad configure && make && make install

Easy said, easy done.

libmypaint

Only a single dependency (json-c), then classical configure, make, make install:

$ crossroad install libjson-c2 libjson-c-devel && crossroad configure && make && make install

GIMP

And now for the main course! Once again, I spare you some of the search of dependencies. Here for mandatory deps:

$ crossroad install atk-devel gtk2-devel libbz2-1 libbz2-devel liblzma-devel liblcms2-2 liblcms2-devel

I’ll skip completely Python dependencies. This is the only major part of GIMP I’m still having issues with in my cross-builds so I will run configure with --disable-python.

As for optional dependencies:

crossroad install libgs8 libgs-devel libmng1 libmng-devel libpoppler-glib8 libpoppler-glib-devel poppler-data xpm-nox-devel headers iso-codes-devel libwmf-devel


Note 1: libwmf-devel has the same issue as SDL, thus the configure script would see it installed if you have it on your main system because of the libwmf-config in your $PATH.

Note 2: you can use crossroad search to discover package names. Even better, sometimes you know a file name that configure is looking for (after a dependency error, check out the configure.ac file or the config.log file for this info). For instance, I can’t find a libghostcript but I see that GIMP‘s configure is looking for the file ghostscript/iapi.h to detect ghostscript:

$ crossroad search --search-files iapi.h
"iapi.h" not found in any package name.
The following packages have files matching the search "iapi.h":
- headers
- libgs-devel

Then using crossroad list-files libgs-devel, I can confirm that this is the right package (crossroad info libgs-devel also confirms this is the package for Ghostscript development files).


Back to building GIMP:

$ crossroad configure --disable-python
$ make &&
make install

Note that I don’t built some options in, like the Ascii Art, the help browser, or OpenEXR because I could not find prebuilt dependencies, but it should not be hard for you to build the dependencies yourself as we did for babl/GEGL/…

Running under Windows!

And now you can for instance make a zip to put on a Windows, like this:

$ crossroad --compress=gimp-build.zip w64 gimp-build

This will just compress the whole target build, where GIMP for Windows is under bin/gimp-2.9.exe. Now this is *not* an installer or whatever finale. The target directory is full of files useful for building, yet completely useless for the finale installation. crossroad is *not* meant for making installers… to this day at least (saying that I would not be against if anyone wanted to implement something of the sort. Why not?! Send me a patch!).
Since I usually run Windows in a Virtualbox VM, rather than zipping a whole archive, I would simply share a directory and symlink the target directory like this:

$ cd /path/to/dir/shared/with/windows/vm/
$ crossroad --symlink w64 gimp

Note: in your Windows environment, you must update the PATH environment variable to add your build’s bin/ directory otherwise Windows won’t find your various built DLLs. You’ll find information on how to do so everywhere in the web.

I use crossroad as a developer, to test and sometimes patch directly GIMP for Windows, without ever getting out of my Linux OS. Well I still test under a Windows VM, which runs in Linux. But I develop and compile in my GNU/Linux shell.
Hopefully this tool can be useful to other people willing to test their program under Windows while hacking on GNU/Linux.

About having a good build system

If you followed good programming practice, and use a full-featured build system such as autotools and cmake, chances are that you can cross-build your program with barely any fix to the code. This is for instance what happened when I ported GExiv2 to autotools after we started using it in GIMP. GExiv2 maintainer would say in the blog post:

This in turn allows gexiv2 to build on Windows, something we’d not targeted at all when we began development but is obviously a hard requirement for GIMP.

Just having a good build system turn their Linux-only library into a multi-platform one. Pretty cool, no? And now libmypaint got the same fate, I autotooled it making it just so easy to cross-compile!

But if you have hand-made makefiles (like GExiv2 did), a scons build (like libmypaint did), or other build systems which don’t have a proper support for cross-building, this won’t be as easy. Think twice when you set up your software. The tool you use for builds is definitely not something to take lightly!

The one-step crossbuild script

This was all for you to see how crossroad is working. If you are only interested into building GIMP specifically, I compiled my whole tutorial below into a single build script that you are welcome to copy paste and run with crossroad like this:

crossroad install glib2-devel libexiv2-devel libjson-glib json-glib-devel \
  libjpeg8-devel libpng-devel cairo-devel libtiff-devel librsvg-2-2 \
  librsvg-devel pango-devel libwebp5 libwebp-devel libjasper-devel \
  gdk-pixbuf-devel libSDL-devel libjson-c2 libjson-c-devel atk-devel \
  gtk2-devel libbz2-1 libbz2-devel liblzma-devel liblcms2-2 \
  liblcms2-devel libgs8 libgs-devel libmng1 libmng-devel \
  libpoppler-glib8 libpoppler-glib-devel poppler-data \
  xpm-nox-devel headers iso-codes-devel libwmf-devel headers
git clone git://git.gnome.org/babl
cd babl && crossroad configure && make && make install && cd ..
git clone git://git.gnome.org/gegl
cd gegl && crossroad configure && make && make install && cd ..
git clone git://git.gnome.org/gexiv2
cd gexiv2 && crossroad configure && make && make install && cd ..
git clone https://github.com/mypaint/libmypaint.git
cd libmypaint && crossroad configure && make && make install && cd ..
git clone git://git.gnome.org/gimp
cd gimp && crossroad configure --disable-python && make && make install

If you copy these into a file, say build-gimp.sh, you can run it at once with:

$ crossroad w64 gimp-build --run build-gimp.sh -n

I hope you enjoyed this tutorial on how to build GIMP with crossroad!

Note: if you like what I do, consider sponsoring our
animation film (CC by-sa), made with GIMP: ZeMarmot!
We fund it with Patreon (USD) and Tipeee (EUR).

22 Replies to “crossroad 0.6 released: cross-building GIMP as an example”

  1. At last an up to date windows build guide for Gimp ! I’ll try this as soon as possible. Hopefully this will encourage more Windows developers to get involved as I think Windows Gimp needs some work at the moment: slow startup times and massive save dialogue delay on some systems.

    1. Cool. Don’t hesitate to give any feedback or if you have any difficulty or bug with crossroad.

      And yes, we are definitely welcoming Windows developers in the GIMP project because that’s seriously lacking. Like: 0… GIMP has no dedicated Windows developer, only other-OS developers (like me) who sometimes fix some Windows bugs here and there when we are bored.

  2. First of all, thanks for this great tool! I am developing an Open Source image editor, called PhotoFlow, and I am now trying to usie crossroad to x-xompile my app for windows.

    I successfully managed to install crossroad and compile part of the dependencies.

    However, after exiting the crossroad shell and then entering it again, I get this cryptic python error:

    photoflow@aferrero-VirtualBox ~/Scratch/build-xroad-w32 $ ~/Scratch/inst/bin/crossroad w32 phf-xroad
    You are now at the crossroads…

    Your environment has been set to cross-compile the project ‘phf-xroad’ on Windows 32-bit (w32).
    Use `crossroad help` to list available commands and `man crossroad` to get a full documentation of crossroad capabilities.
    To exit this cross-compilation environment, simply `exit` the current shell session.
    w32✘phf-xroad photoflow@aferrero-VirtualBox ~/Scratch/build-xroad-w32 $ ~/Scratch/inst/bin/crossroad help
    Fatal Python error: Py_Initialize: Unable to get the locale encoding
    File “/home/photoflow/.local/share/crossroad/roads/w32/phf-xroad/lib/python2.6/encodings/__init__.py”, line 123
    raise CodecRegistryError,\
    ^
    SyntaxError: invalid syntax
    Aborted
    w32✘phf-xroad photoflow@aferrero-VirtualBox ~/Scratch/build-xroad-w32 $ exit

    Do you have any idea what could be wrong?

    Thanks!

    1. Here is a followup of my previous comment: I have apparently solved the Py_Initialize issue by un-setting the PYTHONPATH environment variable… not sure this is the right thing to do, but it seems to work up to now.

      Also, in the cmake Toolchain file I had to add the following line in order to get the proper packages with pkg-config within cmake:

      SET(PKG_CONFIG_EXECUTABLE x86_64-w64-mingw32-pkg-config)

      I hope this helps!

      1. Thanks for all the debugging and the info about the pkg-config variable in the cmake file (most of the packages I crossbuild are autotools, so the cmake crossbuild is much less debugged clearly!). I will add these variables.

        So in the end, did you successfully cross-build PhotoFlow? 🙂

    2. > First of all, thanks for this great tool! I am developing an Open Source image editor, called PhotoFlow, and I am now trying to usie crossroad to x-xompile my app for windows.

      You are welcome, and thanks to you for using it and helping by bringing new use cases. That’s good for making it rock-solid. 🙂

      > File “/home/photoflow/.local/share/crossroad/roads/w32/phf-xroad/lib/python2.6/encodings/__init__.py”, line 123
      > raise CodecRegistryError,\
      > ^
      > SyntaxError: invalid syntax

      It’s trying to load Python 2 code whereas crossroad uses Python 3.
      The way I set the PYTHONPATH would indeed load any python lib and this is wrong. What sucks is that there are not 2 different env variables (PYTHONPATH vs. PYTHON3PATH for instance). But I’m not sure how useful it is to be able to load python libs installed in the crossbuilt environment from the native environment. So I will probably get rid of this for now.

      1. Sorry for the late reply… I have been able to compile PhotoFlow without too much troubles, and the x-compiled windows version runs fine under wine.

        One issue I had is that I am still on an Linux system with gcc 4.8, while the OpenSUSE packages are compiled with gcc 5.x. Due to an ABI change in the standard C++ library, I had to compile from sources all packages that involve C++ code (glibmm, gtkmm, libsigc++, etc…).

        The only library I could not compile successfully with crossroad is OpenEXR, actually due to ilmbase. If anyone has been able to get those two build successfully, I’d be glad to get some help 😉

        I’ll be advertising your tool as much as I can during the next weeks, as I find it really helpful!

        1. > Sorry for the late reply…

          No prob! 🙂

          > I have been able to compile PhotoFlow without too much troubles, and the x-compiled windows version runs fine under wine.

          Nice.

          > Due to an ABI change in the standard C++ library, I had to compile from sources all packages that involve C++ code

          Oh right. That’s the kind of things which may happen obviously. Let’s say these OpenSUSE pre-compiled packages are a useful facility, but not completely foolproof.

          > OpenEXR, actually due to ilmbase. If anyone has been able to get those two build successfully, I’d be glad to get some help

          I can’t remember if I ever tried to cross-build them by the past. I usually only make an effort to cross-build a difficult dependency if I need it to debug something. And I don’t think I ever needed to debug the OpenEXR plugins for Windows in GIMP.

          Some software are clearly a hell to cross-build. My worse experience is probably Python, which even had (last I tried, that was probably more than a year ago) code in their configure step specifically to forbid cross-compilation with MinGW-w64 tools. Obviously just removing this code was not enough. The build system was deeply broken (which is probably why they chose to forbid attempts, which is not the right choice IMO). Anyway in the end, I never managed to cross-compile Python. :-/

  3. Thanks for your work on this, it really helps take away a lot of pain…
    While trying to set up a fresh Mint 18 VM for the purpose of cross compiling GIMP, I noticed a few things:
    – By default, several Python related packages which appear to be required by crossroad are not installed. I had to do a bit of trial and error to get everything before I could install it. If we could have a list of requirements instead, it would help!
    – After installing the packages for a W64 environment, and following your steps above to start a “gimp-build” project, I don’t get the full prompt I was expecting, as you mention…

    I only got:

    $ crossroad w64 gimp-build
    You are now at the corssroads…

    $

    Is that an indication that something is not set up?

    Further down, I tried this:
    $ crossroad install glib2-devel
    “install” is not a platform known by ‘crossroad’. Do not hesitate to contribute:

    So it feels that I’m missing something, but it doesn’t give me much to go on here… 🙂
    Could you please help?

    1. > By default, several Python related packages which appear to be required by crossroad are not installed. I had to do a bit of trial and error to get everything before I could install it. If we could have a list of requirements instead, it would help!

      Since Python does not really have a “configuration” step, I am trying as much as possible to test the dependencies at runtime and have the appropriate warning outputs when they are missing, rather than just crashing with ugly code errors. But I should probably document all the dependencies better in the README as well. You are right.
      You remember which dependencies in particular you had problems with, by the way?

      > I don’t get the full prompt I was expecting, as you mention…

      I have only support for bash and zsh only, so it is possible that your prompt does not change if you were using another shell. That would not mean crossroad does not work, just you would not have a feedback through the prompt.
      This said, last I tested, Mint uses bash by default (as do most distributions), so unless you customized your shell in your fresh VM, there should be a prompt update.
      Also the next error:

      > $ crossroad install glib2-devel
      > “install” is not a platform known by ‘crossroad’. Do not hesitate to contribute:

      … indicates that your crossbuild environment was not set up. Indeed `install` is a command available only inside the w32/w64 environments. Here it seems to consider itself still out of the environment, hence tries to create an environment for a platform nicknamed “install” (which does not exist, obviously).

      Just to be sure, you ran this second command in the same shell session, just after the previous one? And not like in a different tab or terminal?

      I will make a test in a Mint VM as well. Which flavour of Mint did you use? Cinnamon?

      > So it feels that I’m missing something, but it doesn’t give me much to go on here…

      It indeed does not give much. That’s strange there is no error nor anything to diagnose. Hopefully I will manage to reproduce!

      1. I’ll re-do the process and write down what I needed to install on a clean VM, unfortunately I didn’t do so the first time. 🙂

        Mint uses bash, so I don’t think it’s a problem. I also had similar results with Kubuntu later on, on which I also had to manually install dependencies before I could install crossroad. Perhaps something is missing (should have been a dependency but it didn’t show up?), I’m not sure yet.

        I tested this on a freshly installed Mint 18 Cinnamon and a Kubuntu 16.04.1 VM. It was done according to the steps you described, so no new tab or terminal. 🙂

        Many months ago I had it running and it was working fine (I used it to compile a GIMP plugin for Windows), but since then I lost that VM and I was trying to set it up from scratch. I therefore assume that it has to be related to the latest versions of those distros somehow.

      2. I have just re-done all the steps on a new Mint 18 installation. I’ve kept a log of all the steps after installing the distro and fully updating it to the latest packages.

        That includes the extra dependencies I had to install, and also the steps I took to get to the problem I mentioned above.

        I could just paste it here, but it would clutter up the comments section somewhat. Would you like me to send it somewhere or put it online somehow instead?

          1. Hi!
            Have you been able to resolve the problem with >> “install” is not a platform known by ‘crossroad’ ? I am facing the same issue on my Ubuntu 16.04 installation.

  4. Hi,

    For everyone who had the *“install” is not a platform known by ‘crossroad’* issue, I discovered there is actually a problem with pip which fucks up some files in crossroad.
    The solution for you is to install crossroad from the source:

    For instance to install locally, this should suffice:
    > ./setup install –prefix=$HOME/.local

    I will look into how to fixing the pip install.

Leave a Reply

Your email address will not be published. Required fields are marked *

Comment ID: +GnOyw