Let's get this straight: it is entirely normal for a #OpenSource project to accumulate bug reports over time. They're not a thing to be ashamed of.

On the contrary, if you see a nontrivial project with a very small number of bug reports, it usually means one of the following:

a. you've hit a malicious fake,
b. the project is very young and it doesn't have many users (so it's likely buggy),
c. the project is actively shoving issues under the carpet.

None of that is a good sign. You don't want to use that (except for b., if you're ready to be the beta tester).

#FreeSoftware #Gentoo #Python

How to absolutely *not* do #OpenSource: require people to commit to work on other issues with your project in order to file bugs. So, sorry, #Typer, I won't be filing bugs. You figure out how you messed up your release yourself.

https://github.com/fastapi/typer/discussions/new?category=questions

#FastAPI #Python #FreeSoftware #Gentoo

#SigStore / #PyPI attestations: #PGP is hard! We must invent a new signing scheme that's so much easier on users.

The tools, after I've spent hours *integrating* them into #Gentoo, and getting them working for everything before:
* Verifying google_auth-2.46.0.tar.gz ...
Provenance signed by a Google Cloud account, but no service account provided; use '--gcp-service-account'

Yeah, I'm sure that's *so much simpler* than PGP.

#security

Yeah, why not neglect all the good recommendations in the #Python ecosystem, and instead fork your own C extension package, force people to build it with #ZigLang (it's still C), add unconditional dependency on that, and on top of that, refuse to publish wheels, "allowing for optimised compilation according to your machine's specific architecture and capabilities, instead of some (low performance) common denominator."

Fortunately, looks like #Gentoo can just ignore all the fancy crap and compile it with GCC.

https://pypi.org/project/ruamel.yaml.clibz/

@itsfoss
Migrating from #Debian to #Gentoo.

#Linux #LinuxWomen

Going to #39c3 this weekend, am excited!

Will any #gentoo people be there? Would love to meet up.

I'm building webkit-gtk right now. It's one of these messy packages where a few source files need a lot of memory to compile, and ninja can randomly order jobs so that all of them suddenly start compiling simultaneously. So to keep things going smoothly without OOM-ing, I've been dynamically adjusting the available job count via steve the #jobserver.

While doing that, I've noticed that ninja isn't taking new jobs immediately after I increased the job count. So I've started debugging steve, and couldn't find out anything wrong with it. Finally, I've looked into ninja and realized how lazy their code is.

So, there are two main approaches to acquiring job tokens. Either you do blocking reads, and therefore wait for a token to become available, or you use polling to get noticed when it becomes available. Ninja instead does non-blocking reads, and if there are no more tokens available… it waits till one of its own jobs finish.

This roughly means that as other processes release tokens, ninja won't take them until one of its own jobs finish. And if ninja didn't manage to acquire any job tokens to begin with, it is just running a single process via implicit slot, and that process finishing provides it with the only chance to acquire additional tokens. So realistically speaking, as long as there are other build jobs running in parallel, ninja is going to need to be incredibly lucky to ever get a job token, since all other processes will grab the available tokens immediately.

This isn't something that steve can fix.

#Gentoo #NinjaBuild

Started for the adventure, stayed for the stability. 🐧

#linux #gentoo #debian

Alt...

Me adventuring with Gentoo in 2015: A picture of a man who has hope in his eyes and color in his life. Me using Debian to save time in 2025: A picture of a man who is dead inside looking for the smallest bit of hope. There is no color in his black-and-white life.

Why not switch your crappy software from crappy #SCons build system to even more crappy #Bazel build system over a patch release, call it "fully backwards compatible", and then effectively leave all the distros stuck on old versions with half a dozen vulnerabilities?

Yeah, just a random reminder that #MongoDB is total crap, and you shouldn't use it. Or expect people to package it for you.

https://jira.mongodb.org/browse/SERVER-103927

#Gentoo

#Sphinx joined the list of packages dropping #Python 3.11 (and therefore #PyPy) support. Of course, we could just go through the effort of dropping it from respective packages in #Gentoo, given it's not technically that common… but honestly, at this point I have zero motivation to put the extra effort for this, just to learn that next month some core package starts requiring Python 3.12.

So, would anyone really mind if I removed Python 3.11 and PyPy support completely from Gentoo packages?

0 days since it turned out that GNU #make does not respect the #jobserver protocol it designed itself in yet another way. Or to put it otherwise, I've wasted my whole morning implemented something that cannot work because it didn't occur to me that GNU make people only set rules without caring to actually follow them.

https://savannah.gnu.org/bugs/index.php?67822

#Gentoo #Linux

Inevitable moment:

f4e4ddaf69337 dev-build/steve: is no longer simple

#Gentoo

#Gentoo #jobserver revealed another problem with steve in particular, and (I believe) the jobserver protocol in general: blocking clients are prioritized over polling clients.

The problem is simple: when handling blocking reads, steve can issue a job token immediately. When handing a poll, it merely indicates that a token is available, and the client must issue another read request to get it. So if tokens are scarce and there are both blocking and polling clients running, the former are likely to be taking all the incoming tokens.

My idea of working around this is to implement temporary reservations. If a client polls for a token, we reserve one for it. The reserved token can afterwards be only read by the same client. This way, both blocking and polling clients get a token — the former get it immediately, the latter get it reserved for them. And if there are no tokens available, both get into a single FIFO queue, for a poor man's round-robin (steve also throttles all reads to one token at a time).

However, polls technically don't guarantee that the client will eventually read the token, so we need to handle reservation expirations as well.

Worked on some more #Gentoo global #jobserver goodies today.

Firstly, Portage jobserver support patch: https://github.com/gentoo/portage/pull/1528. It's not yet perfect, but it seems to work. If enabled via FEATURES=jobserver-token, it causes emerge to acquire a job token for every job it starts, therefore:

1. emerge won't start more jobs than the jobserver specifies. For example, `steve -j12` will cap emerge at 12 jobs, even if you specify --jobs=16. However, lower cap for emerge will be respected, so you can leave a bunch of job tokens free for other processes to use.

2. emerge will share the job pool with other jobserver-enabled software such as GNU make or Ninja. When emerge is using up all 12 job tokens, all builders inside it will run serial. When it's running 6 jobs, all builders will be able to start up to 6 extra jobs, etc. Ideally, no more than 12 active jobs will be running at a time over the system.

3. Jobserver-enabled software will now correctly respect jobserver job count. When running with externally-provided jobserver, these tools assumed that the jobserver already acquired one token for them, and therefore ran one extra job without a token; say, 13 jobs for a single emerge process. Now the token is actually acquired, so the accounts will match.

Secondly, I've packaged dev-python/pytest-jobserver with a bunch of fixes to make it behave correctly and support Gentoo jobserver. We're not integrating it with the eclasses yet, but you can install it and force-enabled via EPYTEST_FLAGS="-p jobserver". With it, pytest-xdist will acquire job tokens while running tests, and therefore parallel #PyTest jobs will also be counted towards total job count.

Again, it's not a perfect solution, but it works reasonably. The plugin still starts -n jobs as specified by the arguments, but it acquired job tokens prior to executing every test, therefore delaying actual testing until tokens are available. It doesn't seem to cause noticeable overhead either.

BREAKING: #CPython 3.13.10 and 3.14.1 changed the multiprocessing message format in patch release. As a result, programs using multiprocessing may break randomly if they are running while #Python is upgraded (i.e. need restarting).

But apparently it's not a big deal, since all the cool kids are running Python in containers, and nobody is using Python for system tools anymore. Everything has been RIIR-ed and Python is only omnipresent in some backwaters like #Gentoo.

https://github.com/python/cpython/issues/142206

New on my #Gentoo blog: One #jobserver to rule them all

"""
A common problem with running Gentoo builds is concurrency. Many packages include extensive build steps that are either fully serial, or cannot fully utilize the available CPU threads throughout. This problem becomes less pronounced when running building multiple packages in parallel, but then we are risking overscheduling for packages that do take advantage of parallel builds.

Fortunately, there are a few tools at our disposal that can improve the situation. Most recently, they were joined by two experimental system-wide jobservers: #guildmaster and #steve. In this post, I’d like to provide the background on them, and discuss the problems they are facing.
"""

https://blogs.gentoo.org/mgorny/2025/11/30/one-jobserver-to-rule-them-all/

#Steve the #Jobserver has undergone a major rewrite over the last week. It's now implemented using CUSE, the #FUSE API for character devices. It is using pidfd to track processes acquiring job tokens, and automatically reclaims them if processes die without returning them, preventing dead processes from effectively locking the system jobserver.

The code's still a bit ugly — it's a C-changed-midway-to-C++, with libevent for event loops and (still) FUSE's ugly argument parsing.

If someone wants to play with it, the live ebuild is available in #Gentoo as dev-build/steve.

https://gitweb.gentoo.org/proj/steve.git/

Według https://freepg.org/, 5 dystrybucji używa teraz #FreePG: #ArchLinux, #Debian, #Fedora, #NixOS i #Ubuntu. #Gentoo dołącza do tej listy, ale miast cichaczem wprowadzać istotne zmiany w GnuPG, dostarczamy odrębną paczkę (app-crypt/freepg), i oznaczamy ją odpowiednio:

$ gpg --version
gpg (GnuPG) 2.5.13-freepg

Ah, my two nemeses: OOM and ENOSPC.

#Gentoo

If you think #Gentoo was boring recently, I've been doing some stuff to make it more interesting. No need to thank me.

#FlexiBLAS: now default in order to break more ~arch systems
#FreePG: available as an alternative on ~arch, but dependencies need to be updated still to allow it more
#ZlibNG: started experimenting with it locally, flag still masked

Używanie #CVS to przestępstwo, za które karą jest konieczność opieki nad pakietem CVS w #Gentoo. Tak, patrzę właśnie na 18 łatek na tym pakiecie, włączając w nie poprawki bezpieczeństwa.

Powinna obowiązywać zasada, że jeżeli dana paczka ma wiele systemów budowania do wyboru, a dana dystrybucja buduje ją przy pomocy #CMake, to powinna usuwać wszystkie zainstalowane pliki CMake, żeby oprogramowanie tworzone na tej dystrybucji było przenośne.

#Debian #Fedora #Gentoo

Another post on #Quansight PBC blog: "BLAS/LAPACK #packaging"

https://labs.quansight.org/blog/blas-lapack-packaging

"""
#BLAS and #LAPACK are the standard libraries for linear algebra. The original implementation, often called Netlib LAPACK, developed since the 1980s, nowadays serves primarily as the origin of the standard interface, the reference implementation and a conformance test suite. The end users usually use optimized implementations of the same interfaces. The choice ranges from generically tuned libraries such as OpenBLAS and BLIS, through libraries focused on specific hardware such as Intel® oneMKL, Arm Performance Libraries or the Accelerate framework on macOS, to ATLAS that aims to automatically optimize for a specific system.

The diversity of available libraries, developed in parallel with the standard interfaces, along with vendor-specific extensions and further downstream changes, adds quite a bit of complexity around using these libraries in software, and distributing such software afterwards. This problem entangles implementation authors, consumer software authors, build system maintainers and distribution maintainers. Software authors generally wish to distribute their packages built against a generically optimized BLAS/LAPACK implementation. Advanced users often wish to be able to use a different implementation, more suited to their particular needs. Distributions wish to be able to consistently build software against their system libraries, and ideally provide users the ability to switch between different implementations. Then, build systems need to provide the scaffolding for all of that.

I have recently taken up the work to provide such a scaffolding for the Meson build system; to add support for BLAS and LAPACK dependencies to Meson. While working on it, I had to learn a lot about BLAS/LAPACK packaging: not only how the different implementations differ from one another, but also what is changed by their respective downstream packaging. In this blog post, I would like to organize and share what I have learned.
"""

#CondaForge #Debian #Fedora #Gentoo

Remember the package that recently had some trailing junk in the .tar.gz that broke GNU tar, and replied to my bug report with a comprehensive #LLM analysis and a slightly sloppy release checking workflow?

They've made a new release and this time the source distribution is completely broken gzip stream.

Honestly, bumping #Python packages for #Gentoo all these years, I don't recall ever seeing a problem with gzip streams. And then, #autobahn starts using #ClaudeCode heavily, and two bad releases in a row. I can't help but consider the project compromised at this point.

https://github.com/crossbario/autobahn-python/issues/1716
https://github.com/crossbario/autobahn-python/issues/1735

#NoAI #AI

Pamiętacie, jak wspominałem o paczce, która miała trochę śmieci w pliku .tar.gz, przez co GNU tar rzucał błędami, a autor odpowiadział na moje zgłoszenie rozbudowaną analizą z #LLM-a i zaimplementował nieco zabełkocony kod do sprawdzania poprawności archiwum?

Wyszła nowa wersja, i tym razem archiwum to całkiem zepsuty gzip.

Szczerze mówiąc, przez te wszystkie lata opieki nad paczkami Pythona w #Gentoo, chyba nigdy nie spotkałem się z problemem z gzipami. A teraz #autibahn zaczyna intensywnie korzystać z #ClaudeCode i mamy dwa wydania z rzędu z dziwnymi problemami. Cóż, dla mnie ten projekt jest już spalony.

https://github.com/crossbario/autobahn-python/issues/1716
https://github.com/crossbario/autobahn-python/issues/1735

#Python #AI

So #Gentoo is now shipping "extremist" software, apparently.

https://bugs.gentoo.org/964663

To teraz #Gentoo jakoby zawiera "ekstremistyczne" oprogramowanie.

https://bugs.gentoo.org/964663

O, fajno. Wygląda na to, że do wsparcia Pythona 3.14 w #Django w #Gentoo brakowało tylko jednej łatki. I działa!

#Python

I have the—perhaps incorrect—impression that the Gentoo project depends on a lot of custom (an sometimes legacy) software, and is suffering from decisions made decades ago...
Well, that and the fact that some components of it are incredibly complex (sob Portage <3 sob).

But maybe I'm missing the point here... -c-

New on blog: "How we incidentally uncovered a 7-year old bug in gentoo-ci"

"""
“Gentoo CI” is the service providing periodic linting for the Gentoo repository. It is a part of the Repository mirror and CI project that I’ve started in 2015. Of course, it all started as a temporary third-party solution, but it persisted, was integrated into Gentoo Infrastructure and grew organically into quite a monstrosity.

It’s imperfect in many ways. In particular, it has only some degree of error recovery and when things go wrong beyond that, it requires a manual fix. Often the “fix” is to stop mirroring a problematic repository. Over time, I’ve started having serious doubts about the project, and proposed sunsetting most of it.

Lately, things have been getting worse. What started as a minor change in behavior of Git triggered a whole cascade of failures, leading to me finally announcing the deadline for sunsetting the mirroring of third-party repositories, and starting ripping non-critical bits out of it. Interesting enough, this whole process led me to finally discover the root cause of most of these failures — a bug that has existed since the very early version of the code, but happened to be hidden by the hacky error recovery code. Here’s the story of it.
"""

https://blogs.gentoo.org/mgorny/2025/10/12/how-we-incidentally-uncovered-a-7-year-old-bug-in-gentoo-ci/

#Gentoo