Colorisation intelligente dans GIMP

En tant que membre de l’équipe Image du laboratoire GREYC (CRNS, ENSICAEN, Université de Caen), j’ai implémenté un algorithme de “remplissage de dessin au trait” dans GIMP, aussi appelé “colorisation intelligente“. Vous avez peut-être entendu parler du même algorithme dans G’Mic (développé par la même équipe), donc quand on m’a proposé l’emploi, cet algorithme m’a rapidement intéressé. Ce devint ma première mission!

Le problème

Conceptuellement le remplissage de dessin au trait est simple: vous dessinez une forme au stylo noir, disons un cercle approximé, et vous souhaitez le remplir d’une couleur de votre choix. Vous pouviez déjà le faire, plus ou moins, avec l’outil de remplissage, en remplissant les couleurs similaires… à 2 problèmes près:

  1. Si le trait n’est pas bien fermé (il y a des “trous”), la couleur fuite. Les trous peuvent être le fait d’erreur de dessin, cependant on ne les trouve pas forcément aisément (cela peut être un trou d’un pixel au pire des cas), et perdre du temps à les trouver n’est pas très marrant. En outre, cela peut être un choix conscient voire artistique.
  2. Le remplissage laisse en général des pixels non coloriés proche des bordures du traits, à cause de l’interpolation, l’anti-aliasing ou pour d’autres raisons (à moins de ne dessiner qu’avec des pixels pleins, style “pixel art”), ce qui n’est pas un résultat acceptable.
2 principaux problèmes du remplissage des couleurs similaires

En conséquence, probablement aucun coloriste numérique n’utilise l’outil de remplissage directement. Diverses autres méthodes nécessitent par exemple l’outil de sélection contiguë (ou d’autres outils de sélection), l’agrandissement ou réduction de la sélection, puis seulement à la fin le remplissage. Parfois peindre directement avec une brosse est la méthode la plus adaptée. Assister à un atelier d’Aryeom sur le sujet de la colorisation est d’ailleurs absolument fascinant. Elle peut enseigner une dizaine de méthodes différentes utilisées par les coloristes. Elle-même n’utilise pas toujours la même procédure (cela dépend de la situation). Pour le project ZeMarmot, j’ai aussi écrit des scripts Python d’aide à la colorisation, qu’Aryeom utilise maintenant depuis des années, et qui fait un boulot très raisonnable d’accélération de cette tâche ingrate (mais la tâche reste ingrate malgré tout).

L’algorithme

Le papier de recherche s’intitule “Un algorithme semi-guidé performant de colorisation en aplats pour le dessin au trait” (on notera qu’il existe une version anglaise, mais le papier français est plus détaillé: “A Fast and Efficient Semi-guided Algorithm for Flat Coloring Line-arts“). J’ai travaillé sur la base d’un code C++ de Sébastien Fourey, avec les avis de ce dernier ainsi que de David Tschumperlé, tous deux co-auteurs du papier.

Pour nos besoins, je me suis intéressé à ces deux étapes de l’algorithme:

  1. La fermeture des traits, par la caractérisation de “points d’intérêt”, lesquels sont les bords de traits avec courbures extrêmes (on peut alors estimer que ce sont probablement des extrémités de lignes), puis en joignant ces points d’intérêts en définissant des facteurs de qualités à base des angles de normales, ou de distance maximum. Les lignes peuvent être fermées avec soit des splines (c’est à dire des courbes) soit des segments droits.
  2. La colorisation proprement dite, en “mangeant” un peu sous les pixels de traits, ainsi de s’assurer de l’absence de pixels non-coloriés près des bords.

Comme on peut le voir, l’algorithme prend donc en compte les 2 problématiques (que j’ai numérotées dans le même ordre, comme par hasard 😉)! Néanmoins je n’ai implémenté que la première étape de l’algorithme et ai adapté la seconde avec une solution propre (quoique basée sur des concepts similaires) à cause de problématiques d’utilisabilité.

Et voici donc le remplissage par détection de traits dans GIMP:

Remplissage par détection de trait

Je ne vais pas réexpliquer l’algorithme en détail. Si c’est ce qui vous intéresse, je suggère plutôt de lire le papier de recherche (10 pages), lequel est très clair, et a même des images très explicites. Si vous préférez lire du code, comme moi, plutôt que des équations, vous pouvez aussi regarder directement l’implémentation dans GIMP, principalement contenue dans le fichier gimplineart.c, en commençant en particulier avec la fonction gimp_line_art_close().

Ci-dessous, je vais plutôt me focaliser sur les améliorations que j’ai faites à l’algorithme, et que vous ne trouverez pas dans les papiers de recherche. Je rappelle aussi que nous avons travaillé avec l’animatrice/peintre Aryeom Han (réalisatrice de ZeMarmot) comme artiste-conseil, pour rendre l’implémentation utile pour de vrai, non juste théoriquement.

Note: cet article se concentre sur l’aspect technique de la fonctionnalité. Si vous souhaitez seulement savoir comment l’utiliser, je conseille d’attendre quelques jours ou semaines. Nous ferons une courte (néanmoins aussi exhaustive que possible) vidéo pour montrer comment l’utiliser ainsi que le sens de chaque option.

Étape 1: fermeture des traits

Pour donner un aperçu rapide de la logique algorithmique, voici un exemple simple à partir d’une ébauche de dessin par Aryeom (bon exemple puisqu’une telle esquisse est pleines de “trous”!). À gauche, vous pouvez voir l’esquisse, au milieu comment elle est traitée par l’algorithme (cette version de l’image n’est jamais visible par le peintre), et à droite mon essai de colorisation en aplats (avec l’outil de remplissage seulement, pas de brosse, rien!) en moins d’une minute (chronométrée)!

De lignes à colorisation, avec représentation interne au centre

Note: bien sûr, il ne faut pas voir cela comme un exemple de “travail final”. Le travail de colorisation est en général en plusieurs étapes, la première consistant en l’application d’aplats de couleur. Cet outil ne participe qu’à cette première étape et l’image peut nécessiter du travail supplémentaire (d’autant plus avec cet exemple qui se base sur une esquisse).

Estimer une épaisseur de trait globale locale (algo amélioré)

Un aspect de l’algorithme est rapidement apparu comme suboptimal. Comme dit plus haut, nous détectons les points clés grâce à la courbure des lignes. Cela pose problème avec les très grosses brosses (soit parce que vous peignez dans un style “lignes épaisses” ou bien juste parce que vous peignez en très haute résolution, donc avec des lignes “fines de dizaines de pixels”, etc.). L’extrémité de telles lignes peut alors présenter une courbure basse et donc ne pas être détectées. Dans le papier originel, la solution proposée au problème est:

Afin de rendre la méthode de fermeture indépendante de la résolution de l’image, une étape préliminaire permet de réduire si besoin l’épaisseur des tracés à quelques pixels, en utilisant une érosion morphologique. Le rayon utilisé pour cette érosion est déterminé automatiquement par estimation de la largeur des traits présents dans le dessin.

Section ‘2.1. Pré-traitement de l’image anti-aliasée’

Malheureusement calculer une estimation de largeur de traits unique pour un dessin entier présente des limites, puisque cela part du principe que le trait a une épaisseur constante. Demandez donc à un calligraphe ce qu’il en pense pour rigoler! 😉

En outre bien que cela fonctionnait globalement, des effets secondaires pouvaient apparaître. Par exemple des trous inexistants au préalable pouvaient apparaître (en érodant une ligne plus fine que la moyenne). Le papier était conscient de ce problème mais l’écartait en supposant que ce trou serait de toutes façons refermé dans l’étape de fermeture qui suivait nécessairement:

Il est à noter que d’éventuelles déconnexions provoquées par l’érosion appliquée, qui restent rares, seront de toute façon compensées par l’étape suivante de fermeture des traits.

Section ‘2.1. Pré-traitement de l’image anti-aliasée’

Pourtant dès les tests initiaux qu’Aryeom a effectuées avec la première version implémentée de l’outil, elle a rencontré des cas similaires. Une fois même, nous avions une zone parfaitement fermée à la base qui laissait fuiter la couleur, une fois l’algorithme appliqué ⇒ nous obtenions donc l’effet inverse du but de l’algoritme! Paradoxal! Pire, alors que trouver des micros trous dans des traits pour les combler est compliqué, trouver des micros trous invisibles (car existants seulement dans une représentation interne du dessin) tient de la gageure.

Pour couronner le tout, cette érosion ne semblait même pas vraiment bien accomplir son but, puisqu’on arrivait aisément à créer des dessins avec de grosses brosses où aucun point clé n’était détecté malgré l’étape d’érosion. Au final donc, cette étape d’estimation d’épaisseur de trait globale+érosion apportait plus de problèmes qu’elle n’en réglait.

Conclusion: pas glop!

Après de longues discussions avec David Tschumperlé et Sébastien Fourey, nous en sommes arrivés à une évolution de l’algorithme, en calculant des épaisseurs de ligne locales pour chaque pixel de trait (plutôt qu’une unique épaisseur de trait pour le dessin entier), simplement basé sur une fonction distance du dessin. Nous pouvons alors décider si une courbure est symptomatique d’un point clé relativement à l’épaisseur locale (plutôt qu’un test absolu sur un dessin érodé par une épaisseur moyenne).

Non seulement cela fonctionnait mieux, cela ne créait pas de nouveaux trous invisibles dans des zones fermées, détectait des points clés sur de très gros traits en supposant la variabilité des traits (que ce soit un choix stylistique ou parce que la perfection n’est pas de ce monde!), mais en plus c’était même plus rapide!

En exemple, la version originelle de l’algorithme n’aurait pas réussi à détecter les points d’intérêt pour fermer cette zone avec de si gros traits. Le nouvel algorithme n’a aucun problème:

» Pour ceux intéressés, voir le code du changement «

Parallélisation pour traitement rapide

La fermeture de traits est clairement l’étape la plus longue du traitement. Bien que cela reste raisonnable sur des images FullHD voire même 4K, sur mon ordinateur, cela pouvait tout de même prendre une demi-seconde de traitement. Pour un outil intéractif, une demi seconde, c’est un siècle! Sans compter si on se met à traiter des images énormes (pas impossible de nos jours), cela peut alors prendre plusieurs secondes.

J’effectue donc ce calcul en parallèle afin qu’il soit exécuté au plus tôt (dès que l’outil de remplissage est sélectionné). Puisque les gens ne sont pas des robots, cela rend l’intéraction bien plus rapide en apparence, voire dans de nombreux cas, on peut ne même pas se rendre compte qu’il y a eu temps de traitement.

Available line art “Source” in the tool options

Partiellement pour la même raison, vous pourrez remarquer une option “Source” qui propose plus qu’à l’habitude dans d’autres outils (“Échantilloner sur tous les calques” ou sur le calque actif uniquement). Pour cet outil, vous pouvez aussi choisir le calque au dessus ou dessous du calque actif. C’est le résultat à la fois d’une décision logique (la couleur appliquée n’est pas du trait par définition) et pour des raisons de performance (il n’est pas nécessaire de recalculer la fermeture à chaque ajout de couleur).

Étape 2: remplissage

Rendre l’algorithme interactif et résistant aux erreurs

Le papier propose de remplir toutes les zones d’un coup à l’aide d’un algorithme de watershed.

J’ai fait le choix de ne pas honorer cette étape de l’algorithme, principalement pour raison d’utilisabilité. Lorsque j’ai vu les images de démonstration de cet algorithme sur G’Mic pour la première fois, le résultat semblait très prometteur; puis j’ai vu l’interface graphique, et cela semblait peu utilisable. Mais comme je ne suis pas le peintre de l’équipe, je l’ai montré à Aryeom. Ses premiers mots après la démo furent en substance: “je n’utiliserai pas“. Notez bien qu’on ne parle pas du résultat final (qui n’est pas mal du tout), mais bien de l’intéraction humain-machine. La colorisation est fastidieuse, mais si la colorisation intelligente l’est encore plus, pourquoi utiliser?

Qu’est-ce qui est donc fastidieux? G’Mic propose plusieurs variantes pour colorier une image: vous pouvez par exemple laisser l’algorithme colorier aléatoirement les zones, ce qui permet ensuite de sélectionner chaque aplat indépendamment pour recolorisation; vous pouvez aussi guider l’algorithme avec des touches de couleurs, en espérant qu’il fonctionnera suffisamment bien pour inonder les bonnes zones avec les bonnes couleurs. Je propose aussi de regarder cet article sympa de David Revoy, qui avait contribué à la version de base de l’algorithme.

filtre « Colorize lineart [smart coloring] »
La colorisation intelligente dans G’Mic est un peu complexe…

Ce sont des méthodes intéressantes et très sûrement utilisables, voire efficaces, dans certains cas, mais ce n’est pas une méthode générique que vous voudrez utiliser dans tous les cas.

Déjà cela implique beaucoup d’étapes pour colorier un seul dessin. Pour l’animation (par exemple le projet ZeMarmot), c’est encore pire, car nous devons coloriser des dizaines ou centaines de calques.
En outre, cela implique que l’algorithme ne peut se tromper. Or nous savons bien qu’une telle hypothèse est absurde! Des résultats non voulus peuvent se produire, ce qui n’est pas obligatoirement un problème! Ce qu’on demande à un tel algorithme est de fonctionner la plupart du temps, du moment que l’on peut toujours revenir à des méthodes plus traditionnelles pour les rares cas où cela ne fonctionne pas. Si vous devez défaire la colorisation globale (car faite en étape unique), puis essayer de comprendre l’algorithme pour refaire vos touches de couleurs en espérant que la prochaine fois, cela marche mieux afin d’essayer encore, un peu à l’aveuglette, alors l’utilisation d’un algorithme “intelligent” ne peut être que perte de temps.

À la place, nous avions besoin d’un procédé de colorisation intéractif et progressif, de sorte que les erreurs de l’algorithme puissent être simplement contournées en revenant temporairement à des techniques traditionnelles (juste pour quelques zones). C’est pourquoi j’ai basé l’intéraction sur l’outil de remplissage qui existait déjà, de sorte que la colorisation (par détection de traits) fonctionne comme cela a toujours fonctionné: on clique et on voit la zone cliquée être remplie devant ses yeux… une zone à la fois!

C’est simple et surtout résistant aux erreur. En effet si l’algorithme ne détecte pas proprement la zone que vous espériez, vous pouvez simplement annuler pour corriger seulement cette zone.
En outre je voulais éviter de créer un nouvel outil si possible (il y en a déjà tellement!). Après tout, il s’agit du même cas d’usage dont s’est toujours occupé l’outil de remplissage, n’est-ce pas? Il s’agit simplement d’un algorithme différent pour décider comment se fait le remplissage. Il est donc tout à fait logique que ce ne soit qu’une option dans le même outil.

En conclusion, je remplace le watershedding sur l’image totale en utilisant encore une carte de distance. Nous avions déjà vu que cela sert comme estimation décente d’épaisseur (ou de demi-épaisseur pour être précis) locale des lignes. Donc quand on remplit avec une couleur, on utilise cette information pour inonder sous les pixels de lignes (jusqu’au centre de la ligne approximativement). Cela permet ainsi de s’assurer qu’aucun espace non colorié ne soit visible entre le remplissage et les traits. Simple, rapide et efficace.
C’est une sorte de watershedding local, en plus simple et rapide, et cela m’a aussi permis d’ajouter un paramètre “Max flooding” pour garder l’inondation de couleur sous contrôle.

Colorisation intelligente sans la partie “intelligente”!

Un usage possible, et bien cool, de ce nouvel algorithme est de se passer de la première étape, c’est-à-dire ne pas calculer la fermeture des traits! Cela peut être très utile si vous utilisez un style de trait sans rupture (design simple avec lignes solides par exemple) et n’avez donc pas besoin d’aide algorithmique de fermeture. Ainsi vous pouvez remplir des zones d’un seul clic, sans vous préoccuper des la sursegmentation ou de la durée du traitement.

Pour cela, mettez le paramère “Maximum gap length” à 0. Voici un exemple de design très simple (à gauche) rempli avec l’algorithme historique par couleurs similaires (au centre) et par détection de traits (à droite), en un clic:

Gauche: AstroGNU par Aryeom – Centre: remplissage par couleurs similaires – Droite: remplissage par détection de traits

Vous voyez le “halo blanc” entre la couleur rouge et les lignes noires sur l’image du milieu? La différence de qualité avec le résultat à droite est assez frappant et explique pourquoi l’algorithme de remplissage historique par “couleurs similaires” n’est pas utilisable (directement) pour du travail qualitatif de colorisation, alors que le nouvel algorithme par détection de traits l’est.

Outil de remplissage amélioré pour tous!

En bonus, j’ai dû améliorer l’intéraction de l’outil de remplissage de manière générique, pour tout algorithme. Cela reste similaire, mais avec ces détails qui font toute la différence:

Clic et glisse

J’ai mentionné plus haut un problème de “sursegmentation”. En effet on peut appeler un algorithme “intelligent” pour le rendre attrayant, cela ne le rend pas pour autant “humainement intelligent”. En particulier, nous créons des lignes de fermetures artificielles basées sur des règles géométriques, pas de reconnaissance réelle de forme ni en fonction du sens du dessin, et encore moins en lisant dans les pensées de l’artiste! Donc souvent, cela segmentera trop, en d’autres termes, l’algorithme créera plus de zones artificielles qu’idéalement souhaitées (si en plus, vous peignez avec une brosse un peu crénelée, cela peut être très mauvais). Regardons à nouveau l’image précédente:

L’image est “sur-segmentée”

Le problème est clair. on voudra sûrement coloriser le chien avec un aplat de couleur unique, par exemple. Pourtant il a été divisé en une vingtaine de zones! Avec l’ancienne intéraction de l’outil de Remplissage, vous devriez alors cliquer 20 fois (au lieu du clic idéal à 1), ce qui est contre-productif. J’ai donc mis-à-jour l’outil pour autoriser le clic glissé, tel un outil de peinture avec brosse: cliquez, ne relâchez pas et glissez sur les zones à colorier. Cela est désormais possible avec le remplissage par détection de traits ainsi que sur couleurs similaires (pour le remplissage sur sélection, c’est par contre non pertinent, bien sûr). Cela rend le remplissage de dessin sursegmenté bien moins problématique puisque cela peut être fait d’un seul tracé. Ce n’est pas aussi rapide que le clic unique idéal, néanmoins cela reste tolérable.

Prélèvement de couleurs

Une autre amélioration notable est le prélèvement aisé de couleurs avec ctrl-clic (sans avoir besoin de sélectionner la pipette à couleurs). Tous les outils de peinture avaient déjà cette fonctionnalité, mais pas encore l’outil de remplissage. Pourtant on travaille là aussi clairement avec la couleur. Par conséquent pouvoir très aisément changer de couleur par prélèvement sur les pixels alentour (ce qui est un usage très commun des peintres numériques) rend l’outil extrêmement productif.

Avec ces quelques changements, l’outil de remplissage est désormais un citoyen de première classe (même si vous n’avez pas besoin de la colorisation intelligente).

Limitations et travail futur possible

Un algorithme pour peintres numériques

La Colorisation intelligente est faite pour travailler sur des dessins au trait. Cela n’est pas fait pour fonctionner sur des images hors dessin, en particulier des photographies. Bien sûr, on découvre toujours des usages non prévus que des artistes aventureux inventent. Peut-être cela se produira ici aussi. Mais pour le moment, pour autant que je puisse voir, c’est vraiment réservé aux peintres numériques.

Traiter des cas autres que lignes noires sur fond blanc?

Les lignes sont détectés de la manière la plus basique possible, avec un seuil soit sur le canal Alpha (si vous travaillez sur des calques transparent) ou sur le niveau de gris (particulièrement utile si vous travaillez avec des scans de dessins).

Ainsi la version actuelle de l’algorithme peut avoir quelques difficultés pour détecter des traits, si par exemple vous scannez un dessin sur papier non blanc. Ou tout simplement, vous pourriez avoir envie de dessiner en blanc sur fond noir (chacun est libre!). Je ne suis cependant pas certain si ces cas d’usage sont suffisamment courants pour valoir d’ajouter toujours plus de paramètres à l’outil. On verra.

Plus d’optimisation

Je trouve que l’algorithme reste relativement rapide sur des images de taille raisonnable, mais je le trouve encore un peu lent sur de grosses images (ce qui n’est pas un cas rare non plus, notamment dans l’industrie de l’impression), malgré le code multi-thread. Je ne serais donc absolument pas surpris si dans certains cas, vous ne préfériez pas simplement revenir à vos anciennes techniques de colorisation.

J’espère pouvoir revenir tranquillement sur mon code bientôt pour l’optimiser davantage.

Des bordures de couleur peu esthétiques

Le bord du remplissage n’est clairement pas aussi “joli” qu’il pourrait l’être par exemple en coloriant avec une brosse (où le bord montrerait un peu de “texture”. Par exemple, jetons à nouveau un œil à notre exemple d’origine.

L’endroit où le bord du remplissage est visible aura probablement besoin d’une retouche. J’ai ajouté un paramère d'”antialiasing” mais clairement ce n’est pas une vraie solution dans la plupart des cas. Ça ne remplace pas une édition manuelle avec une brosse.

Le pire cas est lorsque vous planifiez de retirer les traits après colorisation. Aryeom fait parfois ce type de dessins où les lignes ne servent qu’en support de travail avant d’être retirées pour le rendu final (en fait une scène entière dans ZeMarmot, un peu “rêveuse”, est faite ainsi). Dans ce cas, vous avez besoin d’un contrôle parfait de la qualité des bords des zones de couleurs. Voici un exemple où le dessin final est fait uniquement de couleurs, sans traits externes:

Image de production du film ZeMarmot, par Aryeom

Pas encore d’interface (API) pour les plug-ins

Je n’ai pas encore ajouté de fonctions pour que les scripts et plug-ins puissent faire du remplissage par détection de trait. Cela est fait exprès car je préfère attendre un peu après la première sortie, notamment pour m’assurer que nous n’avons pas besoin de plus ou meilleurs paramètres, puisque une API se doit de rester stable et de ne plus changer une fois créée, au contraire d’une interface graphique qui peut se permettre plus facilement des changements.

En fait vous vous êtes peut-être rendus compte que toutes les options disponibles dans G’Mic ne sont pas disponibles dans les options d’outils de GIMP (même si elles sont toutes implémentées). C’est parce que j’essaie de rendre l’outil moins confus, étant donné que nombres de ces options nécessitent de comprendre la logique intime de l’algorithme. Plutôt que de passer son temps à trifouiller des curseurs au hasard, j’essaie de trouver une interface qui nécessite peu d’options.

Outil de sélection contiguë

La détection de traits n’est pour l’instant implémentée que pour l’outil de remplissage, mais cela serait aussi tout à fait adapté comme méthode de sélection alternative pour l’outil de sélection contiguë. À suivre…

Segmenter moins

Avec des lignes propres et nets, et des formes simples, l’algorithme marche vraiment bien. Dès que vous complexifiez votre dessin, et surtout utilisez des traits avec un peu de caractère (par exemple la série de brosses “Acrylic”, fournie dans GIMP par défaut), un peu trop de points clés faux-positifs sont détectés, et par conséquent le dessin sur-segmente. Nous sommes tombés sur de tels cas, donc j’ai essayé diverses choses, mais pour l’instant je ne trouve aucune solution miracle. Par exemple récemment j’ai essayé d’appliquer un flou médian sur le trait avant la détection de points clés. L’idée est de lisser les imperfections du trait crénelé. Sur mon exemple de base, cela a bien marché:

Centre: sur-segmentation avec l’algorithme actuel
Droite: toujours trop segmenté, mais bien moins, après avoir appliqué d’abord un flou médian

Malheureusement cela a rendu le résultat sur d’autres images très mauvais, notamment en créant des trous (un problème dont nous nous étions débarrassé en ne faisant plus d’étape d’érosion!).

Centre: résultat très acceptable avec l’algorithme actuel
Droite: mauvais résultat qui ferait fuiter la couleur et qui perd de nombreux détails

Donc je cherche toujours. Je ne sais pas si on trouvera une vraie solution. Ce n’est clairement pas un sujet facile. On verra!

En règle générale, la sursegmentation (faux positifs) est un problème, mais il reste moindre que ne pas réussir à fermer des trous (faux négatifs), notamment grâce à la nouvelle intéraction clic et glisse. J’ai déjà amélioré quelques problèmes à ce sujet, tels les micro-zones que le papier de recherche appelle des “régions non significatives” (or elles sont vraiment significatives pour les peintres numériques, car rien n’est plus embêtant à remplir que des petits trous de quelques pixels); et récemment j’ai aussi réglé des problèmes liés à l’approximation rapide de la surface des régions, laquelle peut être fausse dans le cas de régions ouvertes.

Conclusion

Ce projet fut vraiment intéressant car il a confronté des algorithmes de recherche à la réalité du travail de dessin au quotidien. C’était d’autant plus intéressant avec la rencontre de 3 mondes: la recherche (un algorithme vraiment cool pensé par des esprits brillants du CNRS/ENSICAEN), le développement (moi!) et l’artiste (Aryeom/ZeMarmot). Par ailleurs, pour bien donner le crédit à qui de droit, beaucoup des améliorations d’interface furent des idées et propositions d’Aryeom, laquelle a testé cet algorithme sur le terrain, c’est à dire sur de vrais projets. Cette coopération CNRS/ZeMarmot s’est même tellement bien passée qu’Aryeom a été invitée fin janvier pour présenter son travail et ses problématiques de dessin/animation lors d’un seminaire à l’université ENSICAEN.

Bien sûr, je considère encore ce projet comme un “travail en cours”. Comme noté plus haut, divers aspects doivent encore être améliorés. Ce n’est plus mon projet principal mais je reviendrai clairement régulièrement pour améliorer ce qui doit l’être. C’est néanmoins déjà dans un état tout à fait utilisable. J’espère donc que de nombreuses personnes l’utiliseront et apprécieront. Dites nous ce que vous en pensez!

Un dernier commentaire est que les idées derrière les meilleurs algorithmes ne semblent pas nécessairement les plus incroyables techniquement. Cet algorithme de colorisation intelligente est basé sur des transformations très simples. Cela ne l’empêche pas de fonctionner très bien et d’être relativement rapide (avec quelques limites bien sûr), de ne pas prendre toute votre mémoire vive en bloquant l’interface du logiciel pendant 10 minutes… Pour moi, cela est bien plus impressionnant que certains algorithmes certes brillant, et pourtant inutilisables sur les ordinateurs de bureau. C’est de ce type d’algorithme dont on a besoin pour les logiciels de graphisme pour le bureau. C’est donc très cool et je suis d’autant plus heureux de travailler avec cette équipe talentueuse de G’Mic/CNRS. 🙂

Amusez vous bien à colorier vos dessins! 🙂

Smart Colorization in GIMP

As part of the Image team at GREYC lab (CRNS, ENSICAEN, University of Caen), I implemented the “fill by line art” algorithm in GIMP, also known as “Smart Colorization“. You may know this algorithm in G’Mic (developed by the same team), so when they proposed me to work with them, I wanted to implement this algorithm in GIMP core. Thus it became my first assignment.

The Problem

The concept of filling by line art is simple conceptually: you draw a shape with a black pen, say an approximate circle, and you want to fill the inside with a color of your choice. You could already do this, more or less, with the bucket fill, when filling by color similarities. Unfortunately it has 2 major issues:

  1. If ever the line art is not properly closed (“holes” in the lines), the color will leak outside. This might be a small painting mistake sometimes, yet if you don’t see the holes (it could be just 1 pixel in worst case), wasting time finding it is not funny. Othertimes it may even be an artistic choice (“rougher” line style for instance, or any of the billion reasons why you’d want non-closed lines).
  2. Usually you get very ugly non-colored pixels next to line borders because of interpolation, aliasing or whatever (unless you draw with very hard lines) and this is not acceptable result.
2 main problems of Bucket Fill by color similarities

As a consequence, probably no digital colorists ever use the bucket fill directly. There are various other methods, often with the fuzzy select tool (or other selection tools), growing/shrinking the selection then bucket-filling it. Just even painting directly is sometimes the best. Attending one of Aryeom’s workshop on the matter of colorization is absolutely amazing and enlightening as she can teach you a dozen of different methods, and she herself does not always use the same method (she would say it depends on the situation). On ZeMarmot project, I also made some custom quick-colorization Python scripts which Aryeom have used for years now and which do a pretty decent job for optimizing this tedious job (though it’s still tedious!).

The algorithm

The research paper is called “A Fast and Efficient Semi-guided Algorithm for Flat Coloring Line-arts“. I have worked based on C++ code by Sébastien Fourey, with both his and David Tschumperlé’s input, being both co-authors of the paper.

For our needs, it basically has 2 main steps:

  1. Closing the line arts, which is done by finding “key-points” which are line edges with extreme local curvatures (we estimate them likely to be the end for unfinished lines), then closing the lines by joining the keypoints based on some “quality” criteria such as close opposite angles or maximum distances. Lines can be closed either with splines (i.e. curves) or straight segments.
  2. Actually colorizing the created closed regions, “eating” a bit under the line art pixels to ensure absence of uncolorized pixels near borders.

You can see that the algorithm actually deals with both issues previously raised (which I conveniently numbered in the same order 😉)! Nevertheless I only implemented the first step of the algorithm and went with my own solution for the second step (still based on similar concepts though), because of usability issues.

Here is what bucket-filling by line art looks like in GIMP:

Bucket Fill by line art detection

I am not going to re-explain the algorithm in full. So if interested by technical details, I suggest that you just read the research paper which is quite clear with nice self-explanatory images too. If you are more “fluent” with code, such as myself, than with math equations, you may also look at my implementation in GIMP, which is mostly self-contained in gimplineart.c, and in particular start with gimp_line_art_close().

Below I will focus more on improvements we made to the algorithm, and which you won’t even find in the paper. I remind that we also worked with the animator/painter Aryeom Han (the director of ZeMarmot project) as artist advisor so that the implementation is actually useful for real work.

Note: this article is mostly about the more technical side of things. If you are just interested into how to use the tool efficiently, wait a few days or weeks. We will make a short yet (hopefully) exhaustive video showing the usage of every option.

Step 1: line art closure

To get a basic idea of what’s going on under the hood, here is an example (using an unfinished sketch by Aryeom, which are good examples for the purpose of the algorithm as sketches are full of holes!). The leftest image is the scribble as-is, the middle is how it is transformed by the algorithm (you never see this version of the image!), the rightest one is a quick attempt to flat-colorize it by myself (using the bucket fill tool only, no brush, nothing!) in less than 1 minute (no joke, I timed!).

From line arts to colorized sketch, with internal closure representation in the middle

Note: of course do not look at this result from a “finished work” point of view. Colorization work is often done in several steps, the first step being flat colors. This tool is only working on this first step and the image may still need additional tweaks (even more with this example as I ran it off a very rough sketch image).

Estimate a global local line width (improved from paper)

I very quickly noticed that a part of the algorithm seemed a bit suboptimal. As said above, we need to detect key-points based on line curvatures. This was a problem with big brushes (either because you paint very fat lines on a small image or because you just paint very high resolution artworks hence even your thin lines may be dozens of pixels). The end of such lines may have very low curvatures and go undetected. In the original paper, the proposed solution was:

For the sake of independence with respect to the image resolution, a second step allows to reduce the width of the strokes to a few pixels, if necessary, using a morphological erosion. The radius to be used for this erosion is set automatically by estimating the width of the strokes found in the drawing.

Section ‘3. Pre-Processing of the anti-aliased image

Unfortunately computing a single estimation of the line width for the whole drawing assumes line arts in a same drawing all have the same thickness! Just ask a calligrapher how ridiculous it sounds! 😉

As first consequence, even though it still worked many times, it could add previously non-existing holes (if eroding a line thiner than the average line width). The research paper acknowledged this issue but discarded it, assuming the closure step (which necessary follows) would anyway close the new hole again:

One should note that disconnections that may result from the morphological erosion step applied here, which remain rare, will be corrected anyway by the closing step to be applied thereafter.

Section ‘3. Pre-Processing of the anti-aliased image

Yet in very early tests which Aryeom did with the first version of my implementation, she quite quickly encountered cases where the closing step did not close the created holes. In even at least one instance, we even had a perfectly closed line art which leaked the color fill ⇒ we got the reverse of why the algorithm was created! Quite paradoxical! To add to the worse, while finding micro holes in not well closed lines can be hard, finding non-existing holes (because only created in an internal, non-visible, representation) is close to impossible.

And the ironical part is that the erosion did not even performed its goal that well as it was very easy to create fat lines where no key points got found despite the erosion. In the end, the global width estimation+erosion idea added more problems than it solved.

Conclusion: this was bad.

After much discussion with David Tschumperlé and Sébastien Fourey, we came up to an evolution of the algorithm, computing approximate local line width for every line art pixel (instead of one single line width for the whole drawing), simply based on a distance map of the line art. Then we made the test deciding whether an edgel curvature was keypoint material relatively to the local line width (instead of eroding and assuming a theoretical global line width).

Not only did it perform a lot better, did not create unfillable holes in perfectly closed areas, did detect better end points on very huge strokes, allowed for variable strokes in a same drawing (whether as a stylistic choice or simply because perfection is hard), but it even ended up faster!

As an example, the original version of the algorithm would have failed to detect end-points for, and close this shape with such fat lines. The updated algorithm has no such problem:

» For these interested, see the commit «

Parallelization for fast processing

Line art closure was clearly the most time-expensive step. Though it does perform quite well on FullHD or even 4K images, it can still spend half a second on my laptop. And half a second for an interactive tool is huge! And if we run this on very big images (which are not uncommon at all nowadays), it may even take several seconds.

So I parallelized this whole process in order to run it as early as possible (as soon as the Bucket Fill tool is selected). As people are not robots, this makes the interaction quite seamless and you may not even notice the processing.

Available line art “Source” in the tool options

Partially for the same reason, you may notice that the “Source” option proposes you much more than all other tools (“Sample merged” versus active layer). Here you can also choose the layer above or below the active one. There was both a logical reason (you do not necessarily want to count colors as line arts) and a performance one (the software does not have to continuously recompute the closure at every edit).

Step 2: color filling

Making the algorithm interactive and error-safe

The original algorithm was based on filling all zones at once by using a watershed algorithm to fill the whole image.

I did the choice to just drop this part of the algorithm. It was mostly a usability choice. Basically the first time I saw the images demonstrating the algorithm on G’Mic, I found the result cool, but the implementation GUI seemed completely impractical. Still I am not the painter in the team, so I showed it to Aryeom. Her first words after seeing how it worked were “I will never use this” (or something along the line!). Note well that we are not talking about the end result, but about the computer-human interaction. As we said, colorizing is tedious, but if the smart colorization is even more tedious, then why use it?

So what’s tedious? There were several variant interactions in G’Mic: you can for instance let the algorithm color every zone randomly, then you could select each plain color zones for recolorization; there is also the possibility to guide the algorithm with color spots, hoping it performs as you want. I also suggest this interesting article by David Revoy, who contributed to the original version of the algorithm.

filtre « Colorize lineart [smart coloring] »
Smart colorization in G’Mic, a bit overwhelming…

Maybe they were interesting workflows sometimes, yet this is not always something you want to do in your drawings.

For once, it takes a lot of steps to color a single drawing. For an animation (i.e. ZeMarmot project), this is even worse, as we do it for dozens or hundreds of layers.
A second reason is that such workflows assume that the algorithm is never wrong. And we know that it is a very bold assumption! Undesired results may happen. This is not necessarily a problem; what you ask to such algorithm is to work well most of the time, as long as you can always go back to more down-to-earth methods in the rare edge cases. But if you have to undo the colorization for the whole image, change the color spots trying to guess what went wrong and try again, then using “smart” algorithm is only a waste of time.

Instead we needed a colorization process which is interactive and progressive, so that errors of the algorithms are easily bypassable by temporarily reverting to traditional techniques (just for some zones). This is why I based my interaction on the existing bucket fill tool. Colorization (by line arts) should work as it always used to: you click in a zone and see it being filled in front of your eyes… one zone at a time!

It is straightforward and very error-safe. If the algorithm doesn’t detect well the zone you are working on right now, just undo and fix this zone only.
Moreover I really wanted to not have to create a new tool if possible (there are so many already). This is basically the same use case as the Bucket Fill tool, right? It’s just a different algorithm to decide how to fill, that’s all. So it makes perfect sense to be an option for this same tool.

Therefore my solution to remplace watershedding the whole image at once is to use again a distance-transform map of the lines (both original line art and invisible closure lines). As I remind, this is a decent estimation of the local line thickness. Then when the bucket fill occurs, I flood the color a few more pixels (until the half width of the line arts), thus ensuring we don’t leave ugly non-colorized looking holes near the borders. Simple, fast and efficient.
This is somehow a local watershedding, but simpler and much faster, and also allows to add a “Max flooding” option to keep the color under control.

Using smart colorization without the “smart” part!

One very cool usage one can have of this new algorithm is to bypass the first step, i.e. not compute the line art closure! This can be very useful when you are using line style without any holes (simpler design with strong lines) so don’t need any algorithmic help for line closure. This way, you can fill colors in a single click without caring about over-segmentation or processing time.

To do so, just set the option “Maximum gap length” to 0. Here an example of a very simple design (left), filled by similar colors (middle) vs by line art detection (right), in a single click:

Left: AstroGNU by Aryeom – Middle: Fill similar colors – Right: Fill by line art detection

See this kind of “white halo” between the red color and the black lines in the middle image? Quality difference with right one is striking, right? While the historical “Fill similar colors” is absolutely non usable (by itself) for quality colorization work, the new line art detection mode is perfectly usable.

Bucket Fill enhanced for all cases!

As a bonus, I had to improve the current interaction of the Bucket Fill tool for all algorithms as well. It is mostly similar, but a bit improved. What is?

Click and hold

A main issue was to take care of the problem of sur-segmentation. You may call an algorithm “smart” or “intelligent”, this won’t make it “human-smart”. In particular here it creates artificial lines based on geometry, not on actual recognition of shapes or meaning, and even less by reading the painter’s thoughts! So often it will over-segment, i.e. creates more artificial zones than desired (if you paint with a crenelated brush, then it can be really bad). Let’s take again previous image as an example:

Closure is “over-segmented”

You can clearly see the issue. The dog for instance is most likely in a single color, but it ends up as about 20 zones! In such case, with former bucket fill interaction, you would then end up clicking 20 times (instead of ideally 1). Counter-productive, right? So I updated the Bucket Fill tool to work more like a brush. You can click, hold and move the cursor over various regions. This change works both with the line art algorithm and when colorizing similar colors (only for selection fill, it is meaningless). This makes filling over-segmented area much less a problem as you can still do it in a single stroke. This is not as fast as a single click, yet quite quick.

Color picking

Another nice change was to allow easy color picking with ctrl-click (no need to select the Color Picker tool). All paint tools could already do this, but not the Bucket Fill, even though it also works with colors. Being able to quickly change color (by picking nearby pixel, which is a very common usage by professional digital painters) makes the bucket fill tool very productive.

With these few changes, the Bucket Fill is now a first class citizen (even if you don’t use the smart colorization).

Limits and possible future works

Algorithm targeted at digital painters

Smart colorization works on “line arts”. This algorithm won’t perform well on random “non line art” images, and in particular is not made to work on photography. Though I’m sure some people may find interesting ways to use it elsewhere, as far as I can see, this is designed for digital painters only.

More than black line on white background use case?

Lines are detected the most basic possible way, with a threshold either on the alpha channel (if you work with transparent layers) or on the grayscale value (particularly useful when working on white paper scans for instance).

Therefore current version of the algorithm may have difficulties detecting the line arts, for instance if you scan a drawing done on not completely white paper. Or say you simply want to draw with light lines on dark backgrounds (everyone is free!). Not sure how often this occurs, and whether we should really pile up the tool with color options. We’ll see.

More optimization

Though I said it is very usable on many images of reasonable size, I consider this new algorithm still a bit slow when working on very big images (which is not so uncommon in the graphics printing industry where you often need higher resolutions than screen industry), even despite all the multi-thread code. So I would not be surprised than in some cases, you may come back to use your old-style colorization techniques.

I am hoping to be able to soon come back on my code and optimize it more.

Not so nice color borders

Color on borders does not have the nice “texture” you have when colorizing with brushes. For instance let’s look closer at our original example here.

The part where the border of the color shows will likely need edit. I added an “antialiasing” option, but this is clearly not the real solution in most cases either, and manual edit with a brush after color filling may be necessary.

Worse case is when you want to remove the lines afterwards. Aryeom does sometimes such drawing style where the lines are only for the first steps, then removed for the final render (actually a whole dreamy scene of ZeMarmot is done like this), then you’d want perfect control of the colorization border quality. This is one such example where the end painting is made only of colors, no border lines:

An image cut of ZeMarmot movie in production, drawn by Aryeom

No API yet

We are still missing API functions to be able to run line art detection in scripts. This is on purpose, as I am waiting a bit more to make sure I don’t miss any important usage, since an API has to be stable (unlike the GUI which can be updated more easily since our new release politics).

Actually you may have noticed that even in the graphical interface, I haven’t made visible all the options you can find in G’Mic (even though they all are implemented). This is because I am trying to not make this tool over-confusing as many of these options are based on the intimate understanding of the algorithm. So that none has to play constantly with sliders at random and without understanding, I am testing the best way to not over-expose options.

Fuzzy selection tool

This algorithm is currently only implemented for the bucket fill tool, but it would be perfectly suited as an alternative method in the Fuzzy Select tool as well!

Improve segmentation issue

With very clean lines and simple shapes, the algorithm is very neat. But as soon as you draw complicated drawing, and in particular use very crenellated brushes (for instance the default “Acrylic” series of GIMP brushes), it starts to detect a bit too many false-positive end points, hence over-segmenting. We encountered such issues so I tried various improvements, but so far none is working on all cases. One such improvement was to apply a median blur on the line arts before the detection of end points, as it would definitely smooth dents in the lines. And it did work quite nicely on one such example:

Middle: over-segmentation with current algorithm
Right: still segmented, yet much better, when median-blurring first

Unfortunately it works very badly on very thin lines as it would create holes (a problem we got rid of when we replaced the erosion step with local line widths!).

Middle: very acceptable result with current algorithm
Right: bad result as color would leak and we lost many details

So I’m still working on this. Ideally that’s an issue for which we can find a solution, though it is not an easy topic. We’ll see!

As a general rule, over-segmenting is a problem (false positives), but it is better than failing to close holes for such tool (false negatives), especially thanks to the new click-and-hold interaction. I did improve already several issues on this matter (for instance micro-zones which the paper calls “not significant regions” yet which are very significant for digital painters as they are annoying to fill; and recently an issue with the chosen algorithm to approximate very quickly region areas, and which may be wrong on open areas).

Conclusion

I will conclude that this project is already a very interesting one to confront research algorithms to reality with real-life day-to-day work. This was even more interesting as we even confronted 3 worlds: research (an algorithm thought by brilliant minds at CNRS/ENSICAEN), engineering (myself) and artists (Aryeom/ZeMarmot). By the way, many of my GUI improvements were ideas and propositions from Aryeom who tested our work on real projects on the field. This joint CNRS/ZeMarmot cooperation went so well that Aryeom was invited to present her work and all her drawing/animation problematics in a seminar at ENSICAEN university.

Now it is still a work-in-progress in my opinion. As you could see, several aspects deserve to be improved. It is not on my main track anymore, but I will definitely improve it as I will see fit. Yet it is already definitely in a releasable state, and I do hope that many people will use and appreciate it. Tell us what you think if you try!

A last comment is that ideas behind the best algorithms are not necessarily the most incredible technically. This Smart Colorization algorithm is based on many simple transformations yet it performs well, is quite fast (within some limits; as I said above), does not take all your memory nor make GIMP hang for 10 minutes… To me, this is much more impressive than maybe much more brilliant algorithms, yet unusuable on everyone’s desktop. This is what you need in a graphics desktop software when you actually want some work done. And that’s very cool. 🙂

Have fun everyone!

Notre vision pour LILA et ZeMarmot

Imaginez un studio de film, avec des artistes et techniciens qualifiés, qui travaillent sur des films ou des séries intéressantes… et qui les partagent sous une Licence Libre, pour être visibles par tous, partout (télé, cinéma, web…), partageables et réutilisables.

Imaginez maintenant que ce studio utilise essentiellement du Logiciel Libre (et de l’Open Hardware si disponible), qu’ils le corrigent, voire modifient et l’améliorent au besoin, aussi bien pour des logiciels finaux (tels que GIMP, Blender, Inkscape…), de bureau (tel GNOME), voire jusqu’au système d’exploitation (GNU/Linux) et tout le reste!

Voici donc mon rêve pour l’association à but non lucratif LILA, et pour le projet ZeMarmot (notre premier film, d’animation). C’est ce que je vise depuis le début, mais je me dis que ce n’était peut-être pas suffisamment clair.

Si vous aimez ce rêve, je vous encourage à nous aider en donnant par Patreon, Tipeee, Liberapay, ou tout autre moyen (donation directe bancaire, Paypal, etc.). Car comme toute association à but non lucratif, LILA et ZeMarmot vit par et pour vous!

Si vous voulez lire plus, je rajoute des détails ci-dessous!

Mon emploi actuel

Au niveau personnel, j’ai récemment été engagé pour un an par le CNRS pour développer du code touchant à GIMP et G’Mic.

J’ai peu de salaire stable depuis quelques années, et je suis donc content que ce soit pour travailler sur GIMP (ce que j’ai fait bénévolement ou peu payé pendant 6 ans!)!
Pour ajouter un peu de contexte, l’équipe de G’Mic m’avait d’abord proposé de travailler sur un plug-in Photoshop, ce que j’ai poliment refusé. Je n’ai rien contre Photoshop, mais ce n’est sûrement pas le boulot de mes rêves. Le projet fut alors retravaillé pour que je puisse continuer à travailler avec GIMP. Deux projets principaux ont été identifiés:

  1. La gestion d’extension dans GIMP, ce dont j’avais déjà parlé (sans savoir à l’époque que je serais engagé pour cela), puisque cela aidera beaucoup G’Mic à être installé. J’en profiterai aussi pour améliorer les extensions de manière générale (ce que je prévoyais d’ailleurs depuis le début).
  2. L’implémentation de leur algorithme de colorisation intelligente dans GIMP. Ce projet vint de ma propre initiative quand ils m’ont proposé de travailler ensemble, car cela rentrait très bien dans mes propres plans, et rendrait enfin leur super algo “utile” (l’interaction dans G’Mic est des plus douloureuses!). J’en reparlerai dans un article dédié. Voici pour donner une idée:

Et ZeMarmot dans tout ça?

ZeMarmot est mon projet adoré (de même que celui d’Aryeom). Je le chéris et c’est là que je vois un futur (pas forcément ZeMarmot en soi, mais là où ça va mener). Ainsi même si j’ai une autre source de revenu temporaire, je voudrais réitérer que si vous aimez ce que j’ai fait jusque là, alors c’est à ZeMarmot qu’il faut donner. Financer ZeMarmot est le seul moyen de me permettre de continuer à améliorer GIMP sur le long terme.

Je vois cette année avec le CNRS comme une opportunité de permettre à ZeMarmot de s’épanouir. Car soyons clair, ce n’est pas encore le cas. Chaque année, nous répétons la routine de demander votre aide, et d’ailleurs on est très peu doué sur ce point (quand je vois notamment que les autres assos et fondations ont commencé leurs campagnes de dons depuis un mois!). On est des techniciens essentiellement (développeurs, animateurs…), et on est vraiment nul en marketing. Donc voici notre demande à la dernière seconde!

À ce jour, nous sommes financés à peine au dessus des 1000 € par mois, ce qui ne permet même pas de payer un salaire à temps plein au minimum légal en France. Ainsi en 2018, LILA a été capable d’engager Aryeom (réalisation/animation) et moi-même (développement) en moyenne 6 jours par mois. C’est peu! Pourtant c’est ce qui nous a fait vivre.
On a estimé qu’il faut au moins 2100€ par mois pour une personne, et qu’en vrai nous avons besoin de 5000€ par personne pour un salaire raisonnable et des conditions de travail acceptables (nous avons vu d’ailleurs que la fondation Blender fait aussi cette même estimation), même si cela reste sous les prix du marché d’ailleurs. Notre financement actuel est donc 4 fois trop petit pour un minimum déraisonnable et 10 fois sous le minimum raisonnable. Sans même parler de la lointaine possibilité d’engager plus de gens. Triste, hein?

LILA en 2 mot

LILA est enregistrée officiellement en France comme une association à but non lucratif, loi 1901. Son numéro d’activité est celui d’une production de film, lui donnant un statut vraiment unique lui autorisant d’engager des gens pour la production de films libres, ce qui est fait depuis 3 ans.

Le but de cette production n’est pas l’enrichissement d’actionnaires quelconques (il n’y en a pas). Nous voulons créer nos œuvres, les faire connaître et passer au projet suivant. Car nous aimons ce que nous faisons. C’est pour cela que ZeMarmot est sous licence Creative Commons by-sa, permettant à chacun de télécharger le film, le partager avec amis et famille, et même de le vendre ou de le modifier! Sans blague! Nous proposerons même chaque image source avec les calques!

En outre LILA paye de vrai salaire pour chaque participant. En effet, nous ne considérons pas que “Art Libre” signifie “œuvre au rabais” ou même “amateurisme”. C’est un projet sympa? Oui. Mais c’est aussi professionnel.

Ainsi si LILA était soudainement financé au dessus de toutes nos espérances, cela ne ferait pas de salaires indécents. Simplement LILA pourra embaucher plus de personnes pour faire des superbes films et logiciels plus rapidement et ainsi rendre le monde de l’Art plus agréable. C’est ça être une association à but non lucratif!

Et le logiciel Libre alors?

Là c’est l’autre aspect du studio: nous utilisons uniquement des logiciels libres! Non seulement cela, mais aussi pour en développons! Je ne parle pas de libérer un script interne, mal fait et utilisé par 3 personnes dans le monde tous les 36 du mois. En particulier, nous faisons partie de l’équipe de développement de GIMP! Ces dernières années, un quart des commits de GIMP sont les notres (ce qui peut être aisément vérifié, en particulier les commits à mon nom “Jehan”, de même que ceux d’Aryeom et Lionel N.) Je suis aussi à l’origine de la relaxe de notre politique de sortie pour que nous sortions davantage de versions de GIMP avec de nouvelles fonctionnalités (cela fait des années que je demandais cela et fut finalement acté depuis GIMP 2.10.0!). Il me semble évident que LILA a eu une contribution positive et importante pour GIMP.

Bien sûr GIMP est donc notre projet logiciel principal, ce qui n’a pas empêché divers patches ici ou là dans d’autres logiciels, parfois majeurs! Sans compter nos rapports de bugs très réguliers quand nous n’avons pas le temps de corriger nous-même… nous sommes aussi des utilisateurs importants de tablettes graphiques, avons des contacts avec des dévs de Wacom et Red Hat (on est d’ailleurs désolés, on sait qu’on peut être un peu chiant parfois avec nos bugs! 😛). Et ainsi de suite. Ainsi la seule chose nous empêchant d’en faire plus est le temps. On a besoin de plus de mains, ce qui ne peut être amélioré que si notre financement nous permet enfin d’engager de nouveaux développeurs!

Aussi soyons clairs: ce n’est pas un truc temporaire. Nous croyons tout simplement aux Logiciel Libre. Nous pensons que c’est la chose à faire, que chacun doit avoir accès aux meilleurs logiciels et aussi que c’est ainsi que peuvent être faits les meilleurs logiciels. Je le disais d’ailleurs: je fais environ 1/4 du code de GIMP. Cela signifie que 3/4 ne sont pas faits par moi! Et c’est sans parler de GEGL (le moteur graphique de GIMP). En d’autres termes, je ne pourrais pas en faire autant seul. J’adore travailler avec certains autres des développeurs les plus brillants que j’ai eu l’occasion de rencontrer. Non seulement cela, mais les autres développeurs de GIMP sont également sympas et agréables. Que demander de plus? C’est ça le logiciel libre.
C’est ainsi que l’utilisation et la contribution au Logiciel Libre est dans les statuts de notre studio, notre “contrat en tant que studio à but non lucratif” et cela ne disparaîtra donc pas.

2018 en revue

Une revue rapide des choses que j’ai faites en 2018:

  • 633 commits, soit près de 2 commits par jour en moyenne, dans la branche master de GIMP (sans compter ce qui est dans les branches de fonctionnalité, mon travail en cours) + mes patches dans divers projets que nous utilisons (GEGL, glib, GTK+, libwebp, Appstream…)
  • Aider le projet MyPaint pour préparer une nouvelle sortie de libmypaint v1 (espérons début 2019), et la création du paquet de données mypaint-brushes (maintenant un paquet officiel de MyPaint!).
  • La création et maintenance continue du flatpak de GIMP sur flathub (d’après ce qu’on nous a dit, le logiciel le plus téléchargé de flathub!)
  • La sauvegarde automatique des images en cas de plantage de GIMP
  • Outils de debug (traces d’exécution, infos de plateforme…)
  • Prise en charge basique de HiDPI sur GIMP 2.10 (et plus à venir pour le futur GIMP 3)
  • Travail en cours pour la gestion d’extension dans GIMP
  • Maintenance de données divers (icônes, brosses, appdata, etc.) de GIMP
  • Travail sur les tablettes et périphériques d’entrée
  • Mentor pour un stagiaire FSF (amélioration de la prise en charge de JPEG 2000)
  • Correction de la plupart des cas d’enfer du DLL des plug-ins sous Windows (problème majeur il y a encore peu!)
  • Revue et amélioration de nombreuses fonctionnalités (redressement d’image dans l’outil de Mesure, libheif, libwebp, plug-in de capture d’écran, texte vertical dans l’outil Texte, et bien plus!)
  • L’option de colorisation intelligence dans l’outil de remplissage

Et probablement plein de choses que j’oublie! J’aide aussi à la maintenance du site et à l’écriture d’article sur gimp.org (63 commits cette année). Et tout cela sans compter les patches sans rapport avec GIMP que je fais aussi (par exemple pour les méthodes d’entrée en Coréen) ou les nombreux rapports de bugs que nous écrivons ou aidons à corriger (notamment en étant les premiers à installer Linux sur une Wacom MobileStudio, ou du moins les premiers à en parler, des bugs furent corrigés jusque dans le noyau, et Wayland).

Quant à Aryeom, en 2018, elle a beaucoup travaillé sur ZeMarmot bien sûr (l’animation requiert énormément de boulot; un jour, on devra peut-être donner plus d’information sur le sujet), nous rapprochant davantage de la sortie du pilote (à ce sujet, nous avons récemment créé un compte Instagram où Aryeom poste régulièrement des images et vidéos courtes de son travail en cours, pour qui est intéressé!). Et aussi elle a participé à des projets tiers (nous rappelons que ZeMarmot ne peut financer que quelques jours par mois pour l’instant!), tel qu’un jeu de société interne pour l’association “Petits Frères des Pauvres“, une vidéo marketing pour le logiciel libre Peertube, des designs de pin’s pour la Free Software Foundation. Elle a aussi donné quelques cours de peinture numérique et retouche avec GIMP à l’université.

Bien sûr elle préférerait passer tout son temps sur ZeMarmot uniquement. Mais encore une fois… on a besoin de vous pour permettre cela!

Le Futur

Comment je vois notre futur: dans quelques années, on peut payer plusieurs artistes (réalisatrice, animateurs, artistes peintres, musiciens…). LILA sera enfin un studio certes petit mais productif.

Et bien sûr cela signifie aussi plusieurs développeurs, donc plus de contrôle sur nos outils de production libres. J’ai tellement de rêves! Enfin un éditeur vidéo stable et puissant sans être tordu (en contribuant au Blender VSE, à Kdenlive ou d’autres projets)? Et quand aurons nous des outils de compositing professionnels maintenus (2018 fut un peu triste)? Sans parler de communication entre outils pour éditer des XCF dans GIMP, voir les changements live dans Blender, etc.? Tant d’espoirs! Tant de rêves à réaliser si on avait le financement!

Les rêves peuvent se réaliser si vous aidez!

Que pouvez-vous y faire? Vous pouvez aider ce studio à devenir viable. Me faire engager par le CNRS est cool personnellement mais fut un peu triste pour ce que cela signifie pour le projet. Cela signifie un manque de dépendance notamment. Sans compter que c’est une situation encore très précaire et temporaire. Ce n’est pas une situation stable, de long terme.

Si nous atteignions 5000€ par mois en 2019, cela serait un premier pas énorme pour le projet et la preuve de viabilité du rêve de studio libre.

Nous aiderez-vous à créer un Studio d’Animation Libre? Le graphisme professionnel 2D Libre est à notre porte. Il lui faut juste un peu d’aide pour lui permettre de passer le pas de porte! 🙂

» Financez dans Patreon (USD $) «
» Financez dans Tipeee (EUR €) «
» Financez dans Liberapay «
» Autres méthodes de donation (notamment virement ou Paypal) «

Passez de bonnes fêtes de fin d’année! Et joyeuse nouvelle année!

The dream of LILA and ZeMarmot

Imagine a movie studio, with many good artists and technicians working on cool movies or series… and releasing them under Libre Licenses for anyone to see, share and reuse. These movies could be watched freely on the web, or screened in cinemas or on TV, wherever.

Imagine now that this studio fully uses Free Software (and Open Hardware, when available!), and while it produces movies, they debug and fix upstream software they use, and also improve them as needed (new features, better interaction and design…). This would include end software (such as GIMP, Blender, Inkscape…), as well as the desktop (GNOME currently), or the operating system (GNU/Linux) and more.

Now you know my dream with the non-profit association LILA, and ZeMarmot project (our first movie project, an animation film). This is what I am aiming for. Actually this is what I have been hoping for since the start, but maybe it was not clear enough, so I decided to spell it out.

If you like this dream, I would like to encourage you to help us make it real by donating either through Patreon, Tipeee, Liberapay, or other means (such as direct donation by wire transfer, Paypal, etc.).

If you want to read more first, I am going to add details below!

My current job

I hinted that there were some cool stuff going on lately on a personal level, and now here it is: I have been hired by CNRS for a year to develop things in relationship to GIMP and G’Mic.

This is the first time in years I have a sustainable income, and this is to continue working on GIMP (something I have done for 6 years, before this job!). How cool is that?
For the full story, I was first approached by the G’Mic team to work on a Photoshop plug-in, which I politely declined. I have nothing against Photoshop, but this is not my dream job. Then the project got reworked for me to continue working on GIMP instead. In particular we identified 2 main projects:

  1. The extension management in GIMP I already talked about (back then, I had no idea I would be hired for it), since it will help G’Mic spread a lot. I will also use the opportunity to improve plug-in support in GIMP.
  2. Implementing their smart colorization algorithm within GIMP.  This was actually my own idea when they proposed to work with me, as this fits very well with my own plans, and would finally make their algorithm “useful” for real work (the interaction within G’Mic is a bit too painful!). I will talk a bit more about this soon in a dedicated post, but here is some teaser:

Where does ZeMarmot project stand?

ZeMarmot is my pet project (together with Aryeom’s), I love it, I cherish it, and as I said, this is where I see a future (not necessarily just ZeMarmot, but what it will lead to). Even though I now have another temporary source of income, I really want to stress that if you like what I do in GIMP, you should really fund ZeMarmot to ensure that I can continue once this contract ends.

This year with CNRS is an opportunity to give the project the chance to bloom. Because let’s be honest, it has not bloomed yet! Every year is the same story where we are asking for your help. And when I see all other foundations and non-profits having started to ask for help a month before, I know we are very bad at it. We are technical people here (developers, animators…), who suck at marketing and are asking at the last second.

Right now, we are crowdfunded barely above 1000 € a month, which can’t even afford to pay someone full time at legal minimum wage in the country we live in. Therefore in 2018, LILA has been able to hire Aryeom (direction/animation work) and myself (development) 6 days a month each, on average. It’s not much, right? Yet this is what we have been living off.
We need much more funding. To be clear, the minimum wage (full time) in France requires about 2100€, and we estimate that we’d need 5000€ per person to fund a real salary for such jobs (same estimation as the Blender Foundation does), though it is still below average market value . So LILA is 4 times away to be able to afford 2 salaries at minimum wage, and 10 times away to be able to pay reasonable salaries (hence also even hire more people). How sad is that?

What is LILA exactly?

LILA is officially registered in France as a non-profit association. It also has an activity number classifying it as a movie production, which makes it a very rare non-profit organization, allowed to hire people for producing movies, which it has now done for nearly 3 years now.

The goal of this production is not to enrich any shareholders (there are no such things here). We want to create our art, and spread it, then go to the next project, because we love this. This is why ZeMarmot is to be released under a Creative Commons by-sa license, which will allow you to download the movie, share it with your friends and family, even sell or modify it. No kidding! We will even upload every source image with layers and all!

Still LILA intends to pay appropriate salary to every person working. Because we don’t believe that Libre Art means “crappy work” or “amateur”. It’s for fun? Yes. But it’s also professional

So if it had crazy funding, LILA would not give us decadent salaries, but would hire more people to help us making the art/entertainment world a better place! That’s also what it means being a non-profit.

And Free Software in all that?

There is this other aspect of our studio: we use Free Software! Not only that, we also develop Free Software! When I say we develop Free Software, I don’t even mean we release once in a while some weird internal script used by 2 people in the world. Mostly we are part of the GIMP team. In the last few years, we are about a fourth of the commits of GIMP (you can just check the commits in particular by myself “Jehan”, as well as ones by Aryeom and Lionel N.). I have also pushed for years to improve the release policy to be able to get new features more often (which finally happened since GIMP 2.10.0!). I believe we are providing positive and meaningful contributions.

GIMP is our main project these days, but over the years, I have had a few patches in many important software! And we regularly report bugs when we don’t have time to fix them ourselves… we are early graphics tablet adopters so we are in contact with some developers from Wacom or Red Hat (and we are sorry to them, because we know sometimes I can be annoying with some bugs! 😛). And so on. Only thing preventing me from doing more is time. I know I just need more hands, which would only happen if we had enough funding to start hiring other Free Software developers.

And let it be known, this is not a temporary thing because we don’t want to pay some proprietary license or whatever. No, we just believe in Free Software. We believe this is the right thing to do, because everyone should have access to the best software. But also, we are making much better software this way. I said we do about 1/4 of GIMP commits. This still means that 3/4 are made by other people, and this is even forgetting GEGL (GIMP graphics engine), which is awesome too. Basically we would not be able to do that well just by ourselves. We really enjoy working with some of the sharpest minds I had the chance to work with in software. Not only this, other GIMP developers are really cool and agreeable as well. What better to ask for? This is what Free Software is.
So yeah: using and contributing Free Software is actually in our non-profit studio bylaws, our “contract as a non-profit” and we won’t drop this

2018 in review

Just a very quick review of things I brought in GIMP in 2018:

  • 633 commits, hence nearly 2 commits a day in average, in master branch of GIMP (and more on feature branches in-progress) + patches on various projects we use (GEGL, glib, GTK+, libwebp, Appstream…)
  • Helping MyPaint so that they can soon release a new libmypaint v1 (hopefully early 2019), and creating the data package mypaint-brushes (now an upstream MyPaint package!).
  • Creating and maintaining GIMP flatpak on flathub (according to what we were told, the most downloaded software on flathub!)
  • Automatic image backup on a crash of GIMP
  • Debug tools for automatic gathering of debug data (stack traces, platform info…)
  • HiDPI basic support on GIMP 2.10 (and more work on HiDPI on future GIMP 3)
  • Work-in-progress for extension management in GIMP
  • Maintenance of some data (icons, brushes, appdata, etc.) in GIMP
  • Tablet and input debugging
  • Mentored a FSF intern (improved JPEG 2000 support)
  • Fixed most cases of DLL hell of plug-ins in Windows (used to be the cause for a huge number of bug reports!)
  • Reviewed and improved many features (auto-straighten in Measure tool, libheif and libwebp support, screenshot plug-in, vertical text in text tool, and so much more that I can’t list them all!)
  • Smart colorization option in bucket fill

And everything I probably forget about. I also help a lot on maintaining the website and writing news on gimp.org (63 commits this year). And this all doesn’t count non-GIMP related patches I do sometimes (for instance on Korean input) or the many reports we write and help to fix (notably since we were probably the first to install Linux on a Wacom MobileStudio, or at least talk about it, several bugs were fixed because we reported them and helped debug, even down to the kernel, or Wayland).

And then there is what Aryeom did in 2018, which is a lot of work for ZeMarmot of course (working on animation is a loooot of work; maybe we could do some blog post about it sometime), so we are getting closer to the pilote release (by the way, we recently created an Instagram account where Aryeom posts some images and short videos of her work in progress!). And also some side projects to be able to sustain (I remind that LILA could pay her only an average of 6 days a month officially!), such as an internal board game for a big French non-profit (“Petits Frères des Pauvres“) helping penniless people, a marketing video for the Peertube free software, and pin designs for the Free Software Foundation. She also gave some courses of digital painting and retouching with GIMP at university.

Note that she would also prefer to work only on ZeMarmot full time, but once again… we need your help for this!

The Future

This is how I see our hopeful future: in a few years, we have enough to pay several artists (director, animators, background artists, music…) and developers. LILA will therefore be a small but finally a bit more productive studio

And of course it also means more developers too, hence more control over our Free Software pipeline. I have so many dreams: finally a better video non-linear editor (be it contributing to Blender VSE, Kdenlive or any other which we will decided to use), stable, powerful yet not convoluted? Finally some dedicated Free Software compositing and effects tool for professional (2018 was a bit sad there, right)? Finally more communication between all the tools so that we can just edit our XCF in GIMP and see some changes live in Blender? So many hopes! So many things I wish we had the funds to do!

Help the dream come true

So what can you do? Well you can help the studio increase its funds to first finally be able to just survive. The fact I got hired by CNRS is very cool, but in the same time so sad, because it means that our project was not self-dependent enough. Somehow I had to accept the CNRS contract to save our project. Let it not be the end!

What if we reached 5000€ a month in 2019? This would be a huge milestone for us and the proof our dream is viable.

Will you help us create a non-profit Libre Animation Studio? Professional 2D graphics with Free Software is just there, at our door. It only takes a little help from everyone interested to help it through the entrance! 🙂

» Fund in Patreon (USD $) «
» Fund in Tipeee (EUR €) «
» Fund in Liberapay «
» Other donation methods (including wire transfer or Paypal) «

And have fun end-of-year holidays everyone, and a happy new year!

Crowdfunding for extension management in GIMP (and other improvements)

One of my long-brooded project for GIMP was an extension management system. I had it for years (older message from me I found about this was back in 2013, just a year after I started contributing).
I finally started working on it a few weeks ago!

I actually have a looot of projects regarding extending GIMP: better plug-in API, bidirectional communication with GIMP core, even GUI hacking through plug-ins, why not! But the first issue was: installing plug-ins and resources suck right now!
Let me explain the plan of things to come further below, but first a bit of a reminder:

You can crowdfund ZeMarmot/GIMP!

Everything done so far and to be done only happen thanks to ZeMarmot project. This project is paying for GIMP development, since this is all done for us to improve our production pipeline.

Right now, we don’t even have enough monthly funding to pay a single person at minimum wage. Yet we managed to survive at 2 for years. We had times of slight depressions and burnouts. Actually we kind of always have these lately.

We hope to at least crowdfund enough to pay 2 salaries at minimum wages! That would require about 4 times our current monthly funding. This is why for once I decided to detail more of my development plans in advance (rather than in the end when it is done, as I remind we are major GIMP developers) and I start by the crowdfunding call rather than the technicals, because we really need you!

If you like how GIMP has improved lately, and want it to continue this way, can you help us? If yes, our project is being funded on the 2 following platforms:

» Patreon crowdfunding (USD) «

» Tipeee crowdfunding (EUR) «

What is an extension?

What I call an extension is anything which can be installed in GIMP. In my current development code, an extension can already pack:

  • plug-ins
  • brushes (core and MyPaint ones)
  • dynamics
  • patterns
  • gradients
  • palettes
  • tool presets

In the future, it could also be:

  • scripts
  • templates
  • themes
  • icons
  • … and whatever else data I may forget!

What is management?

What I call “management” is basically being able to:

  • Search for new extensions (made available in remote repositories);
  • install new extensions;
  • uninstall extensions;
  • disable extensions (make them inactive without full removal);
  • update extensions: when the extension creator makes a new version available, GIMP should tell you and give you the possibility to update in  a single click.

You probably all have an idea how it would work. For instance your web browser (i.e. Firefox, Chrome, etc.) probably has extensions as well and you may have searched/installed some. Well basically I want the same thing in GIMP.
Until now, “installing” an extension in GIMP implied making a web search, finding some compressed archive (sometimes on shaddy websites) with weird instructions asking you to put and unzip files into some hidden folder on your disk. Sometimes it was working, sometimes not and usually many barely understood what one were asked to do. Well it sucked.

Extension creator point of view

If you create resources for GIMP, be them brushes, splash images or plug-ins, here is what you’ll get: we are going to set up a website where you can upload your extensions. We don’t know yet if we will recycle the old registry.gimp.org domain (now down) or set up a new one (extensions.gimp.org?). We’ll see.
The code for the website will be on gitlab.gnome.org, though right now the repository is empty (working on GIMP core side first) as I just requested for its creation a few days ago.

Technically an extension is mostly about just adding metadata to your data: an extension name, a description, even screenshots if relevant. Your website URL is also welcome, as well as your bug tracker URL, so that we can redirect people to you when your extension has problems (making your development more efficient). In my current implementation, I chose the AppStream format for the metadata, which is well known by Free Software developers, quite featureful and easy to write (we still have some details to figure out for Windows and probably non-Linux support in general, but I am confident this will go well). Metadata is the main component to be able to search and manage extensions correctly!

We are extending it a bit with a few tags so that you can describe what kind of data your extension provides. For instance, here is the skeleton for metadata of an extension which ships a set of brushes:

<?xml version="1.0" encoding="UTF-8"?>
<component type="addon">
 <id>info.libreart.brushset</id>
 <extends>org.gimp.GIMP</extends>
 <name>My cool brush set</name>
 <summary>A collection of brushes for GIMP</summary>
 <url type="homepage">https://libreart.info</url>
 <metadata_license>CC0-1.0</metadata_license>
 <project_license>GPL-3.0+</project_license>
 <releases>
  <release version="0.1" date="2018-06-07" />
 </releases>
 <requires>
  <id version="2.10" compare="ge">org.gimp.GIMP</id>
 </requires>
 <metadata>
  <value key="GIMP::brush-path">brushes</value>
 </metadata>
</component>

That’s it. You put the brushes inside a brushes/ subdirectory (this is what is described in the “GIMP::brush-path” key) and you are done.

GIMP point of view

One of the cool thing which could happen on GIMP side is that we may transform some of our core data and plug-ins into core extensions as well. This would have 2 direct consequences:

  1. It will be possible to easily disable features or data you don’t care about. I read some people even went as far as deleting files to make their GIMP menus clearer by removing features one never use. Well now it should be unneeded.
  2. Updates could happen out-of-releases: a release of GIMP is a lot of work and preparation. You noticed how now we do them faster now? Well this won’t stop. Nevertheless if we can also update a core extension in the extension repository, this would make our job easier, we could do less core GIMP releases, yet you would get even faster updates for all the small features out there.

What about security?

Well that’s the big question! Let’s be clear: currently security of plug-ins in GIMP sucks.

So the first thing is that our upload website should make basic file type checks and compare them with the metadata listing. If your metadata announces you ship brushes, and we find executables in there, we would block it.

Also all executables (i.e. plug-ins or scripts) would be held for manual review. That also means we’ll need to find people in the community to do the review. I predict that it will require some time for things to set up smoothly and the road may be bumpy at first.

Finally we won’t accept built-files immediately. If code is being compiled, we would need to compile it ourselves on our servers. This is obviously a whole new layer of complexity (even more because GIMP can run on Linux, Windows, macOS, BSDs…). So at first, we will probably not allow C and C++ extensions on our repository. But WAIT! I know that some very famous and well-maintained extensions exist and are compiled. We all think of G’Mic of course! We may make exceptions for trustworthy plug-in creators (with a well-known track record), to allow them to upload their compiled plug-ins as extensions. But these will be really exceptional.

Obviously this will be a difficult path. We all know how security is a big deal, and GIMP is not so good here. At some point, we should even run every extension in a sandbox for instance. Well some say: the trip is long, but the way is clear.

Current code

As said, I am already working on it. Current code can already load extensions, read the metadata, enable and disable the extensions. These are the bases. I still have a lot to do on the bases (for instance the plug-ins need some special-casing for enabling/disabling).

Then I will need to work on getting the extension list from repositories, and work on the web side (Patrick David should help me there, since you can recall he also made our new website and Pixls.us, the trendy community site for photographers using FLOSS).

I cannot say when this will end and I do other things in the same time (lately I have worked on improving GIMP code for HiDPI, and of course I need to work more on our animation plug-in; and much much more!). But I hope a releasable state should be done by end of the year. 🙂

Thanks all!
And don’t forget we are really in need for more funding on Tipeee or Patreon (see also ZeMarmot generic donation page) to help us going forward!

ZeMarmot, main contributor of GIMP 2.10.0-RC1!

Two weeks ago, we released GIMP 2.10.0-RC1! This is our first release candidate before the stable release GIMP 2.10.0. Yes, you heard it well, the release you have been waiting for, for 6 years, is just around the corner!

What has ZeMarmot done exactly?

This was a very exciting release, and for the first time, I was even the major contributor, with 270 commits out of 784 (in-between 2.9.8 and 2.10-RC1, so counting the last 4 months). This is more than the third of all changes between GIMP 2.9.8 and 2.10.0 RC1. As for other participants in ZeMarmot project, Aryeom herself designed 2 missing icons, when she wanting to have a change of mind from animating marmots, frame after frame, and Lionel (one of the board member of the LILA association heading the project) worked on improving ruler subdivision for inches.

So yeah, at every release, ZeMarmot is more and more a major actor of GIMP. We are quite proud of it. Usually, after every release, I would list here a more detailed list on what I worked on, but there is so much in there, that this time, I won’t do the full listing. Let’s just make a quick summary of the most important works:

  • Improve debugging with a dialog gathering data and backtraces. We already discussed about it.
  • Auto-save of unsaved images just before a crash! Yep you read it well! So I know that GIMP is very stable (we had a bunch of remarks about this after we announced the feature). Yet this is software, and in software, shit happens. I am a developer of GIMP and one of the first to tell it: yes GIMP is not perfect; it is very stable, but still, it can crash. Even more: it will crash, some day, when you will expect it the least (it’s always when it strikes! 😜). So now we are prepared. 😉
    To be accurate, it is not a 100% safeguard. By nature, a crash is a state where a process is very unstable. Hence we try what we can, and sometimes I’m sure the auto-save will fail. But in my tests, it succeeded most of the time. And that’s what matters when it will save your work!
  • Color-managing the color picker on macOS.
  • Screenshot with the generic Freedesktop API has been implemented. It is meant to replace all Linux desktop environment’s specific APIs eventually but needs to get reasonable features first. Therefore currently GNOME/KDE and X11 implementations still have priority.
  • New preferences settings for metadata export, and making so that all export plug-ins respect these settings (among other things, thanks to the creation of new API for plug-in developers: gimp_export_exif()gimp_export_xmp() and gimp_export_iptc()). This is a topic many may not care about, though it can also be a major feature to others since metadata can contain sensitive information and we know there are some people who prefer for their image editor to just never export metadata.
  • Various new APIs for plug-in developers, like:
    • gimp_get_pdb_status() to return the status of the last PDB call. This is needed for plug-ins which depend on other plug-ins’
      procedures. If for instance, a second-level plug-in is interrupted interactively, we don’t want to process this as an error but as a cancellation.
    • gimp_stack_trace_available(), gimp_stack_trace_print() and gimp_stack_trace_query() for debugging plug-ins as well.
    • gimp_context_get_distance_metric() and gimp_context_set_distance_metric() for distance metric used in gimp_edit_blend() (and future usage).
  • Improving gimp_edit_blend() API to use “gegl:distance-transform” operation, making it much faster.
  • Improve splash image down-scaling so that it appears at reasonable size on a wide range of screens.
  • Some vulnerabilities were reported in end of 2017, so I fixed CVE-2017-17784, CVE-2017-17786, CVE-2017-17787 and CVE-2017-17789.

In the middle of this all, I already announced the new package mypaint-brushes.
And then there are all the little micro-features, but even more all the bugs we fixed! Actually I don’t think I could have fixed that many hard-to-reproduced bugs before, as easily as these last few months thank to our new debugging system. So I am really so happy I implemented this. Even though some may think it is not that important (not an actual feature for painting or editing images or whatnot), which is something I read on some forum, I personally feel like this is one of my major contributions to this software because I know it will improve its stability and robustness even more (it already started). 😀

I told you, earlier, that bug fixing and stability was my personal goal until 2.10. As you can see, I really meant it. 🙂

Mentoring a student to code in GIMP

In other cool news, a student, Darshan Kadu, approached us with a FSF internship, and continued one of the existing port, for JPEG2000 format support (finishing the port of our implementation from Jasper to OpenJPEG, since the former is getting deprecated everywhere), and I am the internship mentor. This is not the first time I worked with university students since I gave a few courses in a university in Paris last year, and I must say I like the interaction (whether with a single intern or with a whole class).

Now I had a few exchanges about possible future projects for interns in GIMP, possibly in GSoC (for memory, GIMP project used to do a few GSoC but stopped a few years ago; I believe the last GSoC internship happened just when I came in the project myself so I never had the chance to mentor someone in GSoC). So as I saw people proposing for interns to do all sort of major implementations in the core of GIMP, I’d like to comment on this.

Personally I am mostly interested in mentoring because I like accompanying students when they discover real development world. If we can get a few cool patches in the process, this is cool. But this event should not be considered as a cheap way to get code. Any project which relies mostly on this to improve will mostly get hardly maintainable code, which took a lot of your time as a mentor but may never end up merged in your master tree.

Whatever your job is, picture yourself when you were student. Now imagine that you suddenly dropped your student self into your current position which you are in after years of hard work and learning the job though actual real-life projects, successes and failures. Do you really believe that your student self would have just revolutionized your current job in just a few months? I hope you don’t say yes, because that would be a bit sad if you don’t believe you evolved since you were students! ;p

So yeah, GSoC or any internship is very cool because we can work with some bright minds of the future of software. But these are not and should never be a magical way to get cheap, fast and good code. If anyone really believe this, something is wrong!

Release Candidate, uh? Stable release soon then?

Yep we are definitely working as hard as we can to get the stable release as soon as possible. We even just got under the 2-digit of blocker bugs the other day (though we are now back at 10 blockers)!

Actually we had to discuss a bit before naming it a RC. Indeed we believe that this is not really Release Candidate material. Well… not if we wanted to be really thorough. But in the same time, we are a bit tired of dragging the developement of GIMP 2.10 (for 6 long years now!). And making it a RC was our way of pushing a bit the development and ourselves, like a step in the right direction. Now you should really expect the stable release soon, unless we discover something really bad (knock on wood!) in the meantime.

Have fun testing GIMP 2.10 RC1 everyone! 😀

 

Reminder: my Free Software coding can be funded on:
Liberapay, Patreon or Tipeee through ZeMarmot project.

Automatic bug report and stack traces for GIMP

While I was working on yet-another-crash without a backtrace, I realized that we could just generate automatic backtraces upon crashes and tell people about it. This is how I ended up writing a debug tool for GIMP, popping-up a dialog with a nice text encouraging to report bugs. You’ll notice that the main text is non-technical. The goal is not to display non-understandable error messages which nobody will understand. All the technical part is in the below section and is just to be copied by a single button click and reported to us verbatim. 🙂

This technical part contains: GIMP version (and commit information if available), compiler, main dependency version, and finally the errors and backtraces of these errors.

Note: this doesn’t “report” the bug on your behalf. Anyone still has to make the conscious action and go on our bug tracker. But we make things easier and just a few buttons and a copy-paste away.

Someone asked me if I could make a blog post about it, so here it is.

How does this work?

Used to be based on glib…

We already had some backtracing capability in GIMP, mostly using glib API g_on_error_stack_trace(). The main problems of this API:

  • that this function outputs to stdout (which means that you needed to run GIMP from terminal to get the trace, and until now this was only used with specific command line options or on unstable builds);
  • sometimes it was not working for weird reasons;
  • it works only in Unix-like operating systems (in particular not in Windows);
  • it is based on gdb only (as I soon discovered)

So I ended up looking what this function was doing. As I said, the basics is that it simply uses gdb if it is installed on the machine. I am still unsure why, but it was doing so using the interactive mode, therefore entering commands through the standard input with a pipe. Why is it weird? Because gdb has a batch mode especially done for such non-interactive calls. I suspect actually that some of the times g_on_error_stack_trace() failed to work correctly was maybe because it was stuck (but I am not sure, I have not tried to dig much more, so maybe I say shit). But the worse issue was that it was simply printing to stdout. So if I wanted to get the output inside a string in order to use it in the graphical interface (we should not expect people to run GIMP from a terminal!), I had to do more piping of the output. Well at some point, that was just ridiculous to stack processes one after another after another after…

… then based directly on GDB…

This is how I started to reimplement the feature. I simply run gdb in batch mode, and I keep the result in a string for later display in a dialog. This was actually very straightforward. See commit bb88a2d52f.

This also allowed me to get a slightly better stacktrace since I could customize the command. So I request “backtrace full“, getting us local variable contents.

… and LLDB…

Then I remembered that some bug reporter on macOS was using lldb, the debugger from the LLVM project. Since LLVM is default on macOS, I assumed that LLDB is much more common there than gdb too. So I added support for it. This was quite easy too, I just had to search for command equivalency. See commit 4ca31b0571.

… and finally the GNU libc!

Finally I was told of the backtrace() and backtrace_symbols() API. This seems to be a GNU-only API (man says these are GNU extensions). Anyway this should make these always present on common Linux distributions, which is very good news. It means that we will always get “something” on Linux (also the result is much quicker than calling gdb or lldb). Unfortunately the output of  backtrace() is not that exhaustive: basically you get function names, and in particular neither file name nor line number even if you built with debug info, nor variable and parameters contents. So it’s a bit less useful. Yet it’s better than nothing! See commit 4fd1c6c97c.

So in the end, my tool tries gdb, then if absent, lldb, and finally fallbacks to backtrace() if available. This should hopefully gives us traces of crashes and errors in most cases!

The difficulties

Issue 1: do not rely on memory allocation after a crash

There were still a few issues. One of them is that you may notice that I use this dialog for 2 kind of errors: fatal errors (crashes) and non-fatal errors (WARNING, CRITICAL, etc.). l use the same code, but while testing, I realized that I often could not create the dialog from the main process when GIMP crashed. In Linux at least, once the program crashed, I was able to catch the terminating signal enough to do last minute actions, but it seems allocating more memory was not amongst the possible actions (that was my assumption based on tests, I may be wrong, don’t take this for manual talk). Well I guess that makes sense to forbid more memory management, especially if the crash is related to memory bugs. This means that even just creating a new dialog is not possible (requiring allocation of a new GTK+ widget).

This is why when crashing, I run the dialog as a separate process, whereas I run it from within the main process for non-fatal bugs.

Issue 2: backtrace() needs to be run by the main process

When running as a separate process, should the back trace be generated by this other process or from the main process? At first it made sense to have it generated through the new process, but then this has 2 inconveniences:

  1. I am duplicating the back-trace generation code (since I sometimes need to run it from within GIMP, sometimes from outside) and code duplication is never good (even maintenance-wise, you end up with different version. This sucks). You can make common core code as exception, but it’s just not ideal (it makes the build rules complicated).
  2. From the outside process, I can attach to the main process with gdb or lldb but I cannot use backtrace()anymore. That would mean that a lot of people would not get the auto-generated traces (not everyone installs a debugger!).

This is why I decided that the backtrace is always generated by the main process and in case of a crash, it is passed along through a file, instead of a parameter. I could have piped it which would have been just as easy, but Dr. Mingw (see below the Win32 section) was already using a file. So I chose to do the same to be as consistent across platforms as possible (also a file has some advantages: in the extreme case where the dialog breaks too, we may ask a bug reporter to look if a file has still been generated with the info).

Also since — as I said in issue 1 — memory allocations are more likely to fail during crash handling, you need to use backtrace_symbols_fd() instead of backtrace_symbols().
The _fd() variant is guaranteed to run without memory allocation (this is written in the man). And now we have traces on most systems, still with GNUlibc fallback!

Issue 3: error avalanches

Another issue is that, in case of non-fatal errors, you may often have a few of them one after another. Sometimes they may be generated as dominos (you get the second as a consequence of the first error), sometimes it’s because of long-running operations which would just reproduce the same errors many times.

Worse case scenario: a long-time contributor, Massimo, directed me to a bug which would output dozens of thousands of errors in a few seconds. Actually that depends on the size of a selection, and in some of my tests, I had hundreds of thousands of errors!
Obviously you don’t want to create a dialog each time (this example was not even a bug which crashes GIMP, but creating hundreds of thousands of dialogs may do the killing job!). So you have to just update the current dialog with additional errors. But even doing so is very time consuming. Updating a dialog hundreds of thousands of times in a few seconds is at least likely to freeze the whole GUI for a dozen of minutes (I know, I tried!).

So I decided to limit the backtracing, but even the error handling. In a single dialog, I add up to 3 backtraces and 10 errors at most. Any more errors would just be redirected to stderr.

Issue 4: debugging preferences

Moreover do we want the dialog to appear for every kind of errors? In particular, we have WARNING, CRITICAL then all fatal errors. CRITICAL are usually really bad, so we definitely want debug info here. But what about WARNING? I mean, they  are bad too, and they are signs of a bug somewhere. But these are more minor bugs, sometimes also bugs on external data which we warn about (and have no control on). Also we often output warnings when we encounter bugs in other software (for instance, one of the recent bugs where my dialog worked was on a bug in KDE’s API for color picking, and there is not much we can do about it in GIMP but report upstream). So I added finer-grained settings, because you certainly don’t want to make creating with GIMP painful if it pop-ups errors every few hours!

Actually it is even possible to disable all debugging through GIMP preferences, even during crashes, if someone is really not interested at all in reporting bugs, hence contributing to GIMP improvement.

Note: on Windows, the debugging preferences page doesn’t exist at all because the backend we use is not customizable anyway. See dedicated section below.

Issue 5: multi-threading

As explained, we don’t only handle crashes, but also runtime errors. Since GEGL is so close to the GIMP project, it made sense to handle its errors as well (actually long-term, it would make sense to handle errors from any dependency, but let’s do it step by step). So I also catch GEGL’s WARNINGs and CRITICALs. But then I realized that since GEGL uses a lot of multi-threading, getting a backtrace from the main thread when the error happened in another was completely useless.

This combined to the fact GTK+ code must be run in the main thread, therefore to create or update my debugging dialog, I need to pass the information from the thread where the bug occurred to the main GTK+ thread. This can be done with gdk_threads_add_idle_full(). This call obviously adds a delay so you’d end up getting traces from the wrong code, and after an unknown delay. This is double useless.

As a consequence, to handle multi-threaded debugging, I needed to make sure that the stack trace was generated from the thread the error happened, without any delay, and only then it could be sent to the main thread with an idle function.

Issue 6: the tweaking

Then you have all these little details to make the experience not too terrible (at least I am not saying we should make it a good experience, a bug is never a good experience! ;P).

For instance handling a crash, I add a button “Restart”, allowing — as the name implies — to at least restart GIMP immediately.

When non-fatal bugs are reported, we should advise people to save their images and restart GIMP (of course, for crashes, they won’t have the possibility to save themselves, so don’t make them sadder by reminding them).

Also I have to be extra careful to not generate new WARNING or CRITICAL from within this code because then you could create cyclic calls. You don’t want to end up crashing the software because of the debugger which initially fired up only for a minor bug.

Well you get the idea! These are the kind of tweaking you just discover as you implement such a system and you have to take care of them as you go on.

Future work

Something we have been discussing would be to save the opened images in backup files upon crashing. Of course with some kind of crashes, it may not be possible, but that is worth trying at least!

I’ve actually started working on it (with commit d916fedf92  from yesterday). As expected, it’s working most of the time, but while testing various crash conditions, I had some cases where last-second backup failed. I have not dived into the code yet to understand why and what, and if there is a solution to these.

GIMP is quite stable now (at least on GNU/Linux), and quite rarely crashes (well I say this but we had some instability these last few days because of core changes in selection and channels so the auto-debug dialog was very useful). But for this one time when it happens, handling it the most gracefully possible implies saving the current state of work. Then obviously next step will be to propose recovery on next GIMP start.

More on this later as I will continue working on it…

What about Windows?

Now the last remaining issue is Win32! Having GDB or LLDB there might be possible (I have not checked) but probably not the best path. It turns out a contributor, Mukund Sivaraman, did already add support for backtrace generation on Windows upon a crash, back in 2015. This is using the ExcHndl library from the Dr. Mingw project. Basically this is extremely easy to use since there are only 2 functions in the API: one to init the library, one to choose a file where the backtrace will be outputted.

void ExcHndlInit(void);
bool ExcHndlSetLogFileNameA(const char *szLogFileName);

So yes, since 2015, backtraces were simply outputted into a file somewhere, and people just never knew where and how to find it. What I did was simply to piggy-back on this feature, grab the backtrace from the generated file, and display it in our GUI. And that’s it!

Since I needed my own code to run after Dr. Mingw, I had a look how this tool actually made its job. In its code, I saw it was using  SetUnhandledExceptionFilter()to run its action just before the crash. What I did was adding another exception handler with the same function, but registering my handler first beforeinit() Dr. Mingw. This way Dr. Mingw call my handler immediately after its own because it keeps track of any handler previously set and call it after itself.
See commits ae3cd00fbd and 4e5a5dbb87.

Now this has a few limitations: the backtrace generated by Dr. Mingw is not that complete compared to a good gdb backtrace. Also sometimes, I had some crashes which this tool would not catch. I am no Win32 expert and did not spend much time on it, so I don’t know why.
Finally this works only on crashes, in particular I cannot generate backtraces on a whim as I can do on other platforms, which allows to generate backtraces even on WARNINGs or CRITICALs messages for easier debugging, even without a crash.

Well in the end, Win32 always ends up less featured and most annoying to debug. I guess there is nothing to be done since I remind we are still looking for Win32 developers on GIMP. We have had very few contributions of Windows developers for all the years I’ve been around, quite sadly! If you are interested to contribute on this cool piece of software, be very welcome!

We got our first reports with automatic traces!

Even though the tool is still only present in the development version, some people build GIMP from master, and we already got a few bug reports with traces included directly! This is very cool.
Actually even Aryeom got such dialogs, which resulted in some bug fixes already (and more to come)! 🙂

So yeah when I fixed my first bugs thanks to these automatically generated back traces, that made me happy because I felt this new tool will make life a lot simpler and I knew my time was well spent. 😉

You’d think a developer of GIMP would not be happy to get a back trace. And yeah, I’d prefer that GIMP was perfectly bug-free. But there is no such things, and as long as we get bugs, we may as well get well-illustrated reports to easily fix them. This is why I am happy! We are constantly on our way to a much more stable GIMP.

Yeah!

Reminder: my Free Software coding can be funded on:
Liberapay, Patreon or Tipeee through ZeMarmot project.

Ibus-Hangul and Compose key: the incredible journey of a simple patch

Today I decided to tell how I reported a bug (then ended up fixing it) on a non-GIMP related project. Well I do regularly this kind of stuff, and this could have just been one more of these silent commits to a random project as I did many times in my life. But since I decided recently to post more articles, well… I may as well tell a story as one-time contributor (as opposed to “regular contributor”) for once!

Also I think the whole process of reporting a bug on projects you don’t know at all — worse! A whole stack of software you don’t know much! — is quite interesting for people wondering how they should report bugs happening to them.

Finally another reason is to advertise a bit my work, since I remind I am trying to get my Free Software coding crowdfunded through ZeMarmot project.  I am hopeful these kinds of side-stories highlight how our project is useful for Free Software as a whole, not just GIMP. Because yes, that seems unrelated, but in the same time, if I can be funded to hack Free Software full-time, it means more such unrelated bugs can be fixed! So you can consider that this kind of patch is also funded when you crowdfund ZeMarmot. 😉

The context

My main input is the ibus-hangul (Korean) input engine,  and the main languages I write are English (ibus-hangul is basically the same as “English (US)” layout when in LATIN mode, which is my default mode) and French! Sometimes when needed, I can easily switch to writing Korean while keeping the same layout.

To write French, I simply mapped a Compose key on the CapsLock key (which I don’t need). Why not switching to a layout with French characters? Well mostly because I don’t like the default French layout and am used to the US one (I have used it for a few years now), and I am also used to using Compose (I can write quickly enough all French characters with Compose, most people would not know the difference in term of typing speed).
Moreover I don’t mind changing layouts but this can be a bit confusing at times, especially if you need to change your layouts every few minutes. It’s better to just keep the same if possible.

Finally this is also the input which Aryeom uses (her main language being Korean) and she writes both in Korean and in French on daily use.

The route from bug to patch…

Or: how to report a bug … when you know very little about a software stack of dozens of software (really device input goes through so many layers that it’s making me crazy!)

Step 0: the bug!

We were using successfully our setup up until Fedora 25: Korean’s ibus-hangul IME with a Compose key. This Compose key settings was done through GNOME Tweaks Tool.

 

Then some fateful day, we updated to Fedora 26. And paf! The Compose key stopped working. Oh fateful day!

We hoped that it would be fixed by Fedora 27 (since we updated late to Fedora 26, Fedora 27 release was just a month away), but it didn’t.

Step 1: I click this button and it doesn’t work => GUI problem?

Back then though, I didn’t link the bug to ibus or ibus-hangul yet.

I first tried to track this issue in GNOME Tweaks tool (hoping that might have just been just a GUI bug; by the way, now the bug is fixed and the issue was not in Tweaks, but it seems I cannot close the report in gitlab and nobody is answering! If someone from the project reads me: just close the report, please) since that’s where I was setting the feature.

Of course, my secret hope was to just see other developers take care of it and not having to dive into unknown code. Don’t get me wrong, I’d love to fix every bug I encounter myself, but I also have a limited life span! ;P
Also I am lazy. I hear that’s a good trait by the way, nothing to be ashamed of. So really hoping someone else does the job at your place is my best expectation.

Step 2: going deeper into layers

But then after waiting for more than a month with a broken compose key, I couldn’t stand anymore to write my French texts without accents. So I investigated a bit more by myself and whistled in the wind (as you can see, I still haven’t had a single answer in 3 months on this first bug report. Bit sad 😞), looking for which dconf key Tweaks Tool was changing, and seeing that it worked at least here: the dconf key was properly updated so Tweaks was likely not at fault.

So was GNOME the culprit not doing proper action with this dconf value? To decide this, after I understood that this was just a X11 options (the bug also happened on Wayland though), I tried various combinations of trying to set “compose:caps” directly with setxkbmap, or even updating XKBOPTIONS directly inside X11 config files (something I hoped I would never have to do ever again, and for years it seemed like it would hold) and restarting X.

It didn’t work. So I became more certain the problem may be deeper and something was broken in X and Wayland. I asked someone who knows better than I on this whole graphics desktop stack (Thanks Carlos for always helping me out! I know I can be annoying sometimes! 😛).  He advised me to open a bug report at libxkbcommon, which was used both for X11 and Wayland. So I did.

Step 3: …and deeper…

Continuing my own investigations, I realized that the problem was not happening on every input method, but even stranger, the input order changed the behavior! 😥

This is when I was told the order indeed matters and that’s expected, which I find a bit weird since it makes for very particularly unique bugs. But anyway I was not here to argue about these kind of design decision. So I followed libxkbcommon‘s developer’s recommendation and opened another report at xkeyboard-config instead.

Step 4: and back up!

Then I left time pass again for about a month, but once again failed to get an answer, therefore tried to investigate more myself, once again. I reread and thought back about my previous investigations and realized that the broken inputs were all Ibus ones (it turns out I only use Ibus inputs nowadays, mainly the Korean or sometimes the Japanese one). For people wondering what is Ibus, this is the input framework for writing languages complicated enough for which just a layout change would not work (for instance Japanese, Korean, Chinese, etc.). This is when I first started to be on the right path.

But is the issue in Ibus or specifically Ibus-Hangul?
To answer this question, I thought to be clever (I wasn’t necessarily!) by checking Fedora repository contents. There I realized that Ibus-Hangul package didn’t change its version (1.5.0) from Fedora 25 to 27. Since my issue appeared in Fedora 26, I went with the assumption that the broken part was most likely Ibus (which indeed upgraded from Ibus 1.5.14 to 1.5.16). So I opened a bug report for ibus.

This time, I had a very quick response (thanks Takao!), then a few exchanges not only telling me that the issue was indeed triggered by a change in Ibus, yet that was not really a bug in Ibus (well it’s a bit sad that Ibus does not guarantee compatibility, but I can also understand why). Ibus-Hangul had to be updated to follow Ibus changes. Better, the developer of Ibus gave me the necessary hints to update Ibus-Hangul.

Step 5: the patch

And then a few hours later, I was able to clone, read, understand and finally simply patch myself Ibus-Hangul. I immediately installed it for myself and for Aryeom, then I went and did a pull-request on upstream project (⚠ important: never forget to contribute your patches upstream! I know, it means a bit more work; sometimes making clean code which will be accepted upstream may even take longer than making the quick-and-dirty fix for just yourself. Yet that’s how the whole software ecosystem will improve! Imagine how bugged Free Software would be if others were not contributing their patches!).

Hopefully my pull request will be accepted soon even though ibus-hangul activity seems a bit slow (yet existing! There were commits 2 months ago). But anyone in our case reading this, you can already build and install with my patch if you need.

This is also a good example to work from if ever you use another Ibus input method with the same problem. That should be quite easy to update all Ibus engines hopefully.

Step 6: it’s never finished!

The issue is mostly fixed, at least on Ibus side apparently. There is still some remnant of issue, which is that my remapped CapsLock was now working both as a CapsLock and a Compose key which is obviously not the expected result! So if I want to write ‘œ’, it would output ‘Œ’ and locks capitalization. That’s not good…

Getting again a bit of help from Takao Fujiware from Ibus project (really thank you a lot!), it seems that the bug is in GNOME this time, since I can work around it by running directly a command line command:

setxkbmap -layout us -option compose:caps

Also I realize that the issue happens even with non-Ibus inputs this time. So somehow it would seem that GNOME may not set properly the option? Let’s wait and see for answers on this report. At least this time, I have a workaround (I even have 2 different workarounds, but let’s not complicate things!)! 🙂

Conclusion

On our beloved GNU/Linux operating systems, the text input situation is mostly ok, though sometimes I must admit I wonder how many people use slightly more exotic input settings since I regularly encounter funny bugs. Well I have lived in Japan for 2 years and in Korea for 1 year, so I also know that Free Software communities are unfortunately less common over there. And among these communities, people also needing a Compose key must be even rarer.

As you may know, I love languages, so I really hope that things will improve regarding any language support. 🙂

As for the whole route from a bug to a patch, it is interesting to see how it took 5 bug reports to finally get to the right project, and a little more than 3 months for a patch written in a few minutes once I knew what this was about. Somehow this is part of the “unseen”, slow, lengthy and boring work done by every contributors out there (even more one-time contributors since they start without the whole package). And no matter how good you may think you are, you are always a newbie somewhere. I personally even consider to be a constant beginner on most topics, even the ones I know best. This makes things easier because I don’t set impossible expectations on myself but also others.

Thanks for reading my story!

Reminder: my Free Software coding can be funded on:
Liberapay, Patreon or Tipeee through ZeMarmot project.

New “mypaint-brushes” package

Since January 1st, GIMP depends on the mypaint-brushes” repository which I am maintaining until MyPaint project finally takes it alongside its other repositories.

I am hoping that I won’t have to maintain this for long and am looking forward for the MyPaint developers to take care of it (and last I heard of it, in the bug report, they wanted to). So this blog post is also to say that I am not trying to fork MyPaint or anything. 😛 I am just taking a little advance because we cannot wait much longer unfortunately since GIMP now uses libmypaint and we are really looking into releasing GIMP 2.10 as soon as we can. Therefore we need to have MyPaint brushes as their own separate package, which will allow:

  • Not having MyPaint as a de-facto dependency of GIMP since brushes are currently part of MyPaint itself (not separate) and our new MyPaint brush tool in GIMP is basically useless without the brushes.
  • We can now check the existence and path of the brushes at build time (through pkg-config), instead of guessing, making sure GIMP will have brushes to work with.
  • The libmypaint library has been recently versionned because of changes in its API (current libmypaint repository’s master branch is the future version 2). Since libmypaint 2 has no release yet, GIMP uses libmypaint 1 (and likely still will when we will release).
    Similarly the mypaint-brushes package I created is also versionned because the brush format has evolved with the API. It has new settings and new inputs, which were crashing older libmypaint. New settings crashes have been fixed, but the new input crashes still exist to this day. Obviously this is not good and we cannot afford GIMP to crash when using newer brushes. So we need versionned brushes.
  • Even if the crashes were all fixed, some brush settings changed their behavior/meaning (for instance the computation of the speed). That means basically that the brush format changed in a non-compatible way. Another reason to version brushes.

So that’s a new package out there, but an important one and any distribution out there which wishes to package GIMP will also have to package it.

Also if other projects use libmypaint, I would suggest you also depend on mypaint-brushes (as should MyPaint itself actually). 🙂

P.S.: of course, this does not fix the custom brushes that someone would import from MyPaint to GIMP, but at least we’ll have good default brushes.

P.P.S.: if you build yourself the development version of GIMP, check out the INSTALL file carefully. In particular, you don’t want to install the master branch (as I said above, master is version 2, we use version 1) and when installing in a non-standard prefix, make sure your PKG_CONFIG_PATH environment variable is properly set.

Reminder: my Free Software coding can be funded on:
Liberapay, Patreon or Tipeee through ZeMarmot project.