Faster packages building in Gentoo with tmpfs and binhostson Monday, March 27, 2023
I am not going to talk about methods that are already widely known and documented like distcc and ccache. They are already documented in the official Gentoo documentation and there is no reason to repeat the same here.
Also, from my point of view, the process to activate those features is not very simple and the result does not make it worth it.
But there is another unofficial and not so documented way to speed up builds: using tmpfs for the build directory (
PORTAGE_TMPDIR). tmpfs uses RAM as a file system, which allows a much faster access. Since disk access is one of the things that has the biggest impact on compile times, changing these RAM accesses achieves a significant improvement in compile times in a quite simple way.
The necessary steps are quite simple:
- Add a new entry in
/etc/fstabthat mounts a new
tmpfsfilesystem that we will use to compile packages:
The value we put in
tmpfs /var/tmp/portage-tmpfs tmpfs size=4G,mode=775,gid=portage,uid=portage
sizeis the size of that new filesystem, which will limit the size of packages that can be compiled on that filesystem (more on that later).
- Mount the new filesystem:
- Change the value of
/etc/portage/make.confso it uses the new filesystem:
That’s all. From now on, packages compiled with Portage will use
tmpfs, bringing a noticeable speed increase.
However, as I mentioned in the first step, the size of the packages we can compile in
tmpfs will be limited by the size we provide in the
size parameter, which may not be enough for larger packages like
To fix this, we can use
package.env to add an exception for certain packages so they are still compiled in the hard drive instead of
- Create a new
/etc/portage/env/notmpfsfile that resets
PORTAGE_TMPDIRto its original value in the hard drive instead of the new directory in
- In another
/etc/portage/package.env/notmpfsfile, we list the packages that we want to use the environment variables of the
notmpfsfile from the previous step (i.e., the packages that will compile on disk):
app-office/libreoffice notmpfs app-emulation/virtualbox notmpfs dev-lang/mono notmpfs dev-lang/rust notmpfs dev-lang/spidermonkey notmpfs dev-qt/qtwebengine notmpfs www-client/firefox notmpfs
That’s all. Those packages will use a different directory to compile.
Another thing we can do to speed up the installation of packages is simply not to compile them.
Although one of the main characteristics of Gentoo is that all packages are compiled and no official precompiled packages are offered, it still offers the possibility of generating binary packages from the packages that are compiled in a way that they can be reused.
This is useful for example if you use Gentoo on several machines that are similar or use the same architecture, so packages would be built only on one of the machines and reused on the others. However, packages may not be reusable if the USE flags or dependencies change between machines.
For more information, see the official documentation on binary packages in Gentoo.
Also, you can totally avoid the compilation of packages if you use a binhost or binary package repository, but we have to be sure we are using a trusted repository where the compiled packages have not been modified with malicious code. Also, some packages might be compiled with options that are incompatible with your system such as a newer version of glibc or a different CHOST.
List of binary repositories
Some repositories available for AMD64 architecture:
- Although it’s still being tested, Gentoo does have a repository of binary packages.
- Redcore Linux
- Calculate Linux
- I also have my own binhost
- f1r.eu. I don’t really know who is behind this repository and some of them are compiled with a newer version of
~amd64, so use them with care
For ARM64, I don’t know any other repository than my own which I build when I compile Pinetoo for the PinePhone.