Aryeom is quite an advanced computer worker, yet there are times when she is lost as everyone else. This is when I come and help her. That’s part of my job!
She used to set custom icons in her Projects/ folders to recognize various status (not started, in-progress, done projects…) as well as unique icons for particular projects (ZeMarmot!). In GNOME Files/Nautilus, changing the default icon for any file is very easily done by a right click > Properties, then choosing an image to make it an icon. This is a very cool process when done progressively (changing the icon of a project folder when it is added, in-progress or finished, for instance, which doesn’t happen every minute!).
Aryeom’s Projects/ folder with icons representing project status
Unfortunately Aryeom had the case when she relocated her icon folder to a better location. And then, disaster! The icons did not follow (they could have if they had been replicated as data somewhere, similarly to how thumbnails are saved in ~/.cache/thumbnails; here that could have been under ~/.config since you want to keep these as settings… it turns out they weren’t) and she suddenly got a Projects/ folder full of white squares instead of nice custom icons. Aryeom was pissed because she didn’t want to move back her custom icons folder to the old location, neither did she want to change manually each and every past project icon (she has a lot). So she just stopped marking project status with custom icons because while until now, that was an organization system she was really proud of, from this day on, she felt that became a waste of time.
Side note: that is one of these times when the software is actually working against the human being, instead of helping. This feature is cool, and some people want to use it. But then we are basically asking people to always keep the icon images to the same place, never reorganize, never rename, never move. If you ever do one of these actions, you lose your carefully organized icons (maybe hundreds through the years) in an instant with no easy way to salvage them.
This is just a note for people who will try to design a similar feature in the future: unless you don’t want people to use it, or only trained engineers who understand the inner-working of files and metadata, just think twice for the right implementation.
I didn’t do a thing about this problem for a few weeks, but today, I had a closer look. So it turns out that these icons are stored as GIO metadata. Not sure where this metadata is actually stored (well I didn’t search either!), but I found out how to query then change it.
Query the metadata: gio info /a/file/
Change it: gio set /a/file metadata::custom-icon "file://URI"
You’ll note that this path is set as a file URI, not as a shell path. And so I quickly set up a bash script (not bothering about any kind of portability, that was a quick and dirty script for single use) to quickly change all the folder icons with a single ./fixicons.sh ~/Desktop/Projects/* call. Here it is:
#!/bin/bash
for f in "$@"
do
icon=`gio info --attributes=metadata::custom-icon "$f"|grep metadata::custom-icon`
if [ "$?" -eq 0 ]; then
icon=`echo $icon |sed 's/^ *metadata::custom-icon: //'`
icon=`echo $icon |sed 's$old/path/to/change/$new/path/$'`
gio set "$f" metadata::custom-icon "$icon"
fi
done
I wrote it down here in case that helps any of you! Of course, up to everyone to customize it appropriately (in particular the regular expression of the second sed, for the piece of the URI you want to transform).
I do quite regularly similar helping scripts, for myself, for Aryeom, sometimes just out of curiosity, other times to improve my daily digital work. Over the years, I have hundreds of such scripts, a pretty well set up environment (in some previous job, I was even doing some cool workshops for coworkers about customizing one’s working environment to automatize all the boring repetitive activities or the things you don’t want to remember).
Most of these scripts are not in a state to be “released” since they are just quick and dirty hardcoded stuff (rarely they would get so useful that I feel I should release them; that’s how crossroad initially started as just my script to cross-compile GIMP until it became a fully generic cross-compilation tool for any software, to crossbuild for Win32, Android, etc.).
So I figured that maybe I should try and give more of my scripts and tricks to the world other ways, at least as small posts through this website, if not as proper generic tools. 🙂
I am hoping that I won’t have to maintain this for long and am looking forward for the MyPaint developers to take care of it (and last I heard of it, in the bug report, they wanted to). So this blog post is also to say that I am not trying to fork MyPaint or anything. 😛 I am just taking a little advance because we cannot wait much longer unfortunately since GIMP now uses libmypaint and we are really looking into releasing GIMP 2.10 as soon as we can. Therefore we need to have MyPaint brushes as their own separate package, which will allow:
Not having MyPaint as a de-facto dependency of GIMP since brushes are currently part of MyPaint itself (not separate) and our new MyPaint brush tool in GIMP is basically useless without the brushes.
We can now check the existence and path of the brushes at build time (through pkg-config), instead of guessing, making sure GIMP will have brushes to work with.
The libmypaint library has been recently versionned because of changes in its API (current libmypaint repository’s master branch is the future version 2). Since libmypaint 2 has no release yet, GIMP uses libmypaint 1 (and likely still will when we will release).
Similarly the mypaint-brushes package I created is also versionned because the brush format has evolved with the API. It has new settings and new inputs, which were crashing older libmypaint. New settings crashes have been fixed, but the new input crashes still exist to this day. Obviously this is not good and we cannot afford GIMP to crash when using newer brushes. So we need versionned brushes.
Even if the crashes were all fixed, some brush settings changed their behavior/meaning (for instance the computation of the speed). That means basically that the brush format changed in a non-compatible way. Another reason to version brushes.
So that’s a new package out there, but an important one and any distribution out there which wishes to package GIMP will also have to package it.
Also if other projects use libmypaint, I would suggest you also depend on mypaint-brushes (as should MyPaint itself actually). 🙂
P.S.: of course, this does not fix the custom brushes that someone would import from MyPaint to GIMP, but at least we’ll have good default brushes.
P.P.S.: if you build yourself the development version of GIMP, check out the INSTALL file carefully. In particular, you don’t want to install the master branch (as I said above, master is version 2, we use version 1) and when installing in a non-standard prefix, make sure your PKG_CONFIG_PATH environment variable is properly set.
Here it is, GIMP 2.9.8 has been released some days ago now, the latest development version of GIMP! As it is customary now, let’s list our involvement in this version so that our supporters on crowdfunding platforms know what they funded. 🙂
Since it also happens to be the end of the year, I complete this post with our end-of-year report, as we also did in 2016.
What we did for GIMP 2.9.8!
During this release span, I focused most of my efforts on bug fixing. I finished a few features here and there but actually even restrained myself from coding too many new stuff! Why? Because I believe we have enough and at some point, we should just release GIMP 2.10. Of course, GIMP 2.10 could be even twice as awesome if we push it by a few more months, and 5 times more awesome with even more time. But then in the end, if you never see it, what’s the point, right? Actually I even plan on just doing this (bug fix and finishing what was started) until we get 2.10 out. Let’s stop feature craziness!
Apart from a lot of bug fixes, I did a lot of bug triaging these last months (looks like I participated in 122 bug reports between 2.9.6 and 2.9.8, i.e. 3 and a half months). And this month, I also reorganized our bug tracker still for the same reason (pushing GIMP 2.10 release forward) by reviewing the 50+ bugs we had in the GIMP 2.10 milestone to set as blockers only the ones we should really look into. Right now that’s down to 25 such bugs!
I also put some efforts in our stable flatpak release, which is how since October 16, GIMP has officially had its flatpak package on flathub! It is of course visible on the GNU/Linux section of GIMP’s download page with a nice “Install GIMP flatpak” orange button (notice also the cool drawing on this page? That’s Aryeom’s!).
Right now, you can only install the stable release, i.e. GIMP 2.8.x (flathub only accepts stable builds) but if you get it there, when GIMP 2.10 will be out, you will automatically get an official update!
In any case, this flatpak thing (in particular keeping our development flatpak manifest up-to-date with git code and testing the builds) is taking a lot of maintenance time!
As a whole that was 122 commits authored by me in GIMP repository between GIMP 2.9.6 and 2.9.8, out of 474 commits (so ~25%), and I pushed a few more commit from third-party when I reviewed them…
We also had again 2 guest commits by Lionel N., board member of LILA association, the non-profit managing ZeMarmot project.
Of course, though I said I focused on bug fixes, there are still a bunch of cool features I participed to during this release:
Support of password-protected PDF for import (the 2-commit feature implemented by Lionel from LILA!) and new procedure `file-pdf-load2()` API for plug-ins and scripts to open password-protected PDF files, but also multi-page PDFs (loading a multi-page PDF was already possible through the GUI but not by scripts and plug-ins).
Help system improvements: upon detection of locally installed manuals in several languages, GIMP will now allow selection of the preferred manual language in the Preferences dialog (Interface > Help System). I felt this was an important feature because we regularly had people not understanding why the manual they installed was not seen by GIMP. And they were right, especially since we don’t have as many manual languages as GUI languages. For example, we have 3 Chinese translations (zh_CN|TW|HK) but only a zh_CN manual. I could definitely imagine someone with a zh_HK GUI to go for the zh_CN manual as a fallback.
Verbose version (command line: gimp -v) now displays C compiler information (useful for debugging).
Canvas rotation and flipping information are now visible in the status bar, and this information is interactive (clicking the flip icons will unflip the canvas; clicking the rotation angle opens the “Select Rotation Angle” dialog). Some people were indeed noting that with the ability to flip/rotate the canvas, in some cases, you may end up “lost” on whether it is currently rotated, flipped or whatnot. After all, the status bar already has zoom information and flip/rotation is quite a similar feature. 🙂
Review HGT support patches and improve a bunch of stuff with auto-detection of the format variants (SRTM-1 and SRTM-3), and also a `file-hgt-load()` API for scripts and plug-ins.
But really, as I said, I think my bug fixes and maintenance of previous code was actually much more important than this above list, even though it is so less fancy (and I am a bit sad I cannot list bug fixes in a non-boring way!). And I will just focus more and more on fixes and stability to get GIMP 2.10 out as soon as possible.
ZeMarmot in 2017: our report!
You know it, ZeMarmot is not only about GIMP, even though this software is a huge part of it! ZeMarmot is about making an animation film in 2D drawing, traditional animation yet with digital means (i.e. drawing on computer, not paper). We draw with GIMP. Well Aryeom Han, animator and animation film director does so (not me). And we crowdfund this project.
Financial status
This year was a bit tough mentally and we really started to wonder if this project was a good idea for our lives. Project finances increased continuously yet very slowly, and were still extra low all throughout 2017 (under 400 € a month).
In October, I finally shouted a cry for help after my computer broke, and we are so happy that many people heard it! The funding increased by about twice.
Now let’s be clear! Our current funding is more or less 1000€ since October. This is a lot better than what we had before and it gave us a lot of hope. Yet it still does not pay full time salaries for 2 people (faaar from it, actually it cannot even pay a single full time salary obviously). So we still hope you will not forget us and if you appreciate our project and what we do, both on GIMP development and/or on ZeMarmot movie, please we will be very thankful if you can donate to the project.
We were conscious that the lack of news on the animation side was not the best. On the other hand, animation just takes time. That’s the way it is.
Depending on the complexity and details of the animation (as chosen by the team), a minute of animation can take a month of work or more (just search the web, all links say the same).
Of course, it depends on your artistic choices. If you do vector animation or Limited Animation (Simpsons or the likes), you can animate a lot faster. Basically you don’t take the same time to animate South Park or a Disney movie (which is not a problem, it’s a choice; I appreciate The Simpsons or South Park too). For ZeMarmot, as you know, we chose a detailed style with full traditional animation. At times we regretted this choice a bit but that’s the way things are.
That’s how Aryeom decided to live stream herself working! She took a few days to search software and found the Free Software OBS, understand how things work (well she also managed to break Fedora once by reinstalling NVidia drivers while following tutorials! :p), made many tests throughout December and since December 25, public livestreams started.
The work is regularly live-streamed at this address:
Unfortunately we have not found the right organization yet to plan and give a schedule of future livestreams. So for the time being, the best is either to follow us on Twitter, subscribe to the Youtube channel, or just try to have a regular look at above link.
Previous livestreams are recorded and listed automatically on the channel once the streaming ends, so you can also have a look later on older (not live anymore) streaming. Be aware though: this is real live of someone working really. That means it is real time, not accelerated 20 times (as all these speed painting you can see everywhere), errors may happen and are not edited out afterwards. There is no sound and Aryeom doesn’t interact with people. She is focused and doesn’t look at what is streamed (we will sometimes look at the chat though and may answer questions but don’t consider it as a granted “feature”; this is a peek at an animator work, not a service). The artist sometimes goes for a rest, and so on. It can even be boring at times. Also the longer recorded streaming in the list is more than 7 hours straight! Fair warning. 😉
Still I think that’s an awesome experiment and we already had some very cool comments, like people thanking us (some in English, many in French) because that it is a bit like being allowed to sneak into an animation studio to observe the animators working.
Art+Code symbiosis
We regularly have the question: “why don’t you have separate crowdfundings for development and animation sides?”
Answer: because this project is a whole. It is symbiotic: I do Free Software because I use it; if I didn’t have ZeMarmot (or another project where we use GIMP), then I would likely not contribute to GIMP. It is that simple. Aryeom as well would likely not use GIMP if she didn’t have a developer by her side.
We remind that is how I started my first patches: because we had crashes and many issues with GIMP and this was not enough for us as a professional tool at the time. We are very happy to tell you that now, it is. Not only because of us, far from it, let’s be clear! We are so happy to hack together on GIMP with several very talented developers (among them, Mitch, GIMP maintainer who is still here after 20 years!). But we were allowed to do our part and this is the reason we stuck around.
To illustrate, just a few hours ago, thanks to Aryeom’s streaming of her work, we were able to have an unexpected live demo of how well we work together. During her live, GIMP crashed! Ouch! In a few minutes, she was able to find reproduction steps during the live streaming. Less than two hours later, I fixed the crash then improved my fix in the master repository of GIMP (it actually took even just a few minutes to reproduce and fix the crash, but well I also had other priorities which I could not drop immediately!).
That’s how well we work together and what you pay for when you donate to ZeMarmot project, and finally that’s the reason why it is a 2-people project, not 2 separate projects. 🙂
So if you ever hold your donation because you only want to pay for a movie, or at the opposite only want to pay for GIMP development, I hope you will review your judgment and see why you get to your goal even better by paying for both!
GIMP Motion
GIMP Motion is our plug-in for animation in GIMP (we talked about it earlier for simple then complex animations). You can also see it in action in Aryeom’s live streamings by the way, nearly daily now.
Unfortunately I have kind of neglected it lately, and that’s mostly for the reason I told earlier: because I am really focusing on getting core GIMP 2.10 out. So I hope this won’t just drag forever (seriously I want to get done with GIMP 2 and go forward with GIMP 3!)
That means that GIMP Motion will likely not be a part of GIMP 2.10. Yet it will be a part of a further GIMP 2.10.x release since we decided earlier that we would relax the no-feature policy on minor releases, which is how I decided that GIMP Motion was not ready to be part of a stable release. That’s exactly why I pushed for this no-feature policy relax for years (ever since 2014, cf. the section “GIMP Meeting(s)” on our LGM 2014 report!): so that we don’t have to rush half-done features nor push important releases forever.
Well we still use it internally, but that’s still very very rough and has many bugs. Be warned if you try it!
Documentating the process of animation
This year, we have been a bit light on documenting the process. Well we had a post on animatics, key-framing, etc. and one on background design. We also had a few talks during the NUMOK festival in Paris, as well as in the JM2L meeting in South of France where we could give some interesting details on the process as well which are not written here yet (but should be soon).
I am the first to admit that’s not enough since Aryeom and I really want to document the techniques behind giving life to still images. But as I said, we had been a bit down, overworked and penniless this year so this fell a little behind. Hopefully we’ll do better next year.
ZeMarmot in the news
This was also cool since, we got on local TVs twice this year thanks to JM2L! In all the videos, GIMP is clearly mentioned and shown on screen as well as Aryeom hacking animations on GIMP in GNOME. 🙂
We have been featured on France 3 (also in writing):
Then on PleinSud TV (at 2:32):
And a mention in a newspaper, Nice Matin:
New material
Thanks to the increased funding at end of year, we were able to renew a bit our material. In particular we bought a Wacom MobileStudio Pro (basically a laptop-tablet from Wacom) on which the first thing we did was to erase Windows and install a GNU/Linux (Fedora 27) and GIMP. And that worked well. We still opened more than a dozen of bug reports here and there, so don’t expect things to be perfect yet. But we are working on it!
We documented a bit our process on a Twitter moment and unfortunately had to take a pause because of hardware issues. We indeed had to send the tablet back to after-service, which gave it back after more than 3 weeks (2 days ago)!
Now we will have to start it all again. Be prepared, we might publish a very complete guide soon on how to get a very cool Wacom laptop with Linux and GIMP. 🙂
Conclusion
This year was hard but eventful, and the end of year gave us more hope after funding increased and we were able to upgrade our material.
The streaming of Aryeom GIMPing was also a very cool idea and we are happy to see that people seem to like it. To this day we only got positive feedbacks. We should have started this sooner!
We do hope that things will continue to improve. We love what we do and our project, and we really wish we will soon be able to say proudly that we are able to make a living by hacking Free Software and Libre Art.
When this day will come, this will just be a very very happy day. 🙂
Happy New Year 2018 everyone!
Lately a recurrent contributor to the GIMP project (Massimo Valentini) contributed a patch to support HGT files. From this initial commit, since I found this data quite cool, I improved the support a bit (auto-detection of the variants and special-casing in particular, as well as making an API for scripts).
So what is HGT? That’s topography data basically just containing elevation in meters of various landscape (HGT stands for “height“), gathered by the Shuttle Radar Topography Mission (SRTM) run by various space agencies (NASA, National Geospatial-Intelligence Agency, German and Italian space agencies…). To know more, you can read here and there.
HGT download source: https://dds.cr.usgs.gov/srtm/version2_1/ (go inside SRTM1/ and SRTM3/ directories for respectively 1 arc-second and 3 arc-seconds sampled data)
You probably won’t find other links of interest since not everyone can do such data (not everyone has satellites!).
Here is what it can look like after processing: left is an image obtained from NASA PDF, and right is the same data imported in GIMP followed by a gradient mapping.
So the support is not perfect yet because to get a nice looking result, you need to do it in several steps and that involves likely a bunch of tweaking. My output above is not that good (colors look a bit radioactive compared to the NASA one!) but that’s mostly because I didn’t take the time to tweak more.
And so that’s why I am writing this blog post. Someone trying to import HGT files in GIMP may be a bit disconcerted at first (so I’m hoping you’ll find this blog post to go further). At first you’d likely get a nearly uniform-looking grey image and may think that HGT import is broken. It is not.
What’s happening? Why is the imported HGT all uniform grey?
GIMP by default will convert the HGT data into greyscale. That is not a problem by itself since we can have very well contrasted greys. But that doesn’t happen for HGT import. Why?
HGT contains elevation data as signed 16-bit integers representing meters. In other words, it represents elevation from -32767 m to 32767 m (with an exception for -32768 which means “void”, i.e. invalid data; since that’s raw data with minimum processing, it can indeed contain errors). Therefore once mapped to [0; 1] range, color 0 (pure black) is invalid, ]0; 0.5] is anything under water level and [0.5; 1] is above water elevation.
Considering that on earth, the highest point is Mount Everest at 8848m, when mapped to our [0; 1] range, we see it has value 0.635. So you can see the problem: most things on earth will be represented with greys really close to 0.5 and that’s why there is no contrast.
How to get nice colors and contrast?
There are several solutions, but the one proposed by the contributor was to use the “Gradient Map” plug-in. That’s a good idea. Basically you remap your greys from 0 to 1 into color gradients.
Now you can try to create a gradient by setting random stops through the GUI, but that will most likely be quite a challenge. A better idea is to do it a bit more “scientifically” (i.e. to use numbers, which you can also do through the GUI by using the new blend tool, though not as accurately as I’d like with only 2 decimal places). This is what did Massimo here by creating a gradient file which would map “magenta for invalid data, blue below zero, green to 1000 m, yellow to 2000m, and gray to white above“. From this base, I added a bit of random tweaking because I was trying to get an output similar to the NASA document (just for the sake of it), so you can get a look at how my own gradient file looks like. But if you are looking to, say, create a relief map with accurate elevation/color mapping, you’d prefer to stick by the number-only approach.
Then once you get your gradient “code”, copy it in a file with the extension .ggr inside the gradients/ folder of your GIMP config, and just use it when running “Gradient Map” filter.
Just to explain a bit the format: for each line, you get the startpoint, midpoint and endpoint coordinates (in the [0; 1] range), followed by 4 values for RGBA (also in [0; 1] range) for the startpoint then again 4 values for RGBA endpoint color. Then you get an integer for the blending mode (you likely want to keep it linear, i.e. 0, for a relief map), then the coloring value (leave it to 0 as well, which is RGB). Finally the last 2 integers are whether the startpoint and endpoint must be fixed colors, or correspond to foreground, background, etc. You will likely want to keep them as fixed colors (0).
means: gradient from 0 meter (0.5) to 1000 m ((0.515267 – 0.5) × 216 ≈ 1000) is a linear gradient from RGBA 0-1-0-1 (green) to RGBA 0-0.5-0-1. That is:
start mid end Rs Gs Bs As Re Ge Be Ae 0 0 0 0
where start is the start elevation and end the end elevation in [0; 1] range; and RsGsBsAs and ReGeBeAe are respectively the start and end gradient colors.
That’s how you can easily map the elevation into colors! I hope that’s clear! 🙂
Can’t we have nicer support with a GUI?
Yes of course. This was fun and cool to review then improve this feature, and we should not let quality patches rot in our bugtracker, but that’s not my priority (as you know) so I stopped improving the feature (if I don’t stop myself from all these funny stuff out there, when would I work on ZeMarmot?!).
I gladfully accept new patches to improve the support and have left myself 2 bug reports to leave ideas about how to improve the current tools:
Improve “Gradient Map” filter to provide on-canvas preview and editing, similarly to the blend tool, because I realize this filter is powerful but that is a bit of a pain to use right now (iterations of edit gradient, run the filter for test, cancel, again and again).
Map gradients directly from HGT import with preview and [0; 1] range remapped to elevation in meters in the dialog so that we don’t have to constantly recompute values back and forth and edit .ggr files by hand.
In the meantime, I leave this blog post so that the format is at least understandable and HGT import usable to moderately technical people. 🙂
That’s it! Hopefully this post will be useful to someone needing to process HGT files with GIMP and willing to understand how this works, until we get more intuitive support.
