The author seems to have completely missed the point that they can just publish their own repositories to allow users on both of these platforms to update. It would also mean they could avoid the problems they’re having building source packages.
Their complaints about supporting instalations with older libc versions are also addressable by just building against the oldest libc they wish to support.
I don’t see the problem. For years I’ve used the public Open Build Service instance at openSUSE to package and publish software for openSUSE and Fedora (never cared for Debian/Ubuntu because I packaged the software only for my own use but it’s possible).
You can put the repositories on pretty much any static hosting. I haven’t tried it, but I’m pretty sure that, with a bit of fiddling, you could even put them on a github pages site.
Both distros have good tooling for maintaining repositories, and I’d say it’s probably easier than getting your package into the official repos and keeping it updated there, especially on distros like Debian which favour stability over having the latest versions of packages.
No distribution system is going to be entirely friction free for the author, but I’ve not encountered any significant issues hosting deb or rpm repos.
Sort of, but each of them uses a different way of making the repository, so really it’s more a case of different repositories in different directories, but they can all be on the same website.
Not really. The user adds the repository to the list of repositories their package manager is using, and the packages in that repository appear as normal packages for them to select. When the author publishes a new version of their package it’ll appear as an update in all users package managers.
The user adds the repository to the list of repositories their package manager is using,
Fun fact: when we were doing devops as config management, we distributed repository configs AS packages … which we updated as required.
and the packages in that repository appear as normal packages for them to select. When the author publishes a new version of their package it’ll appear as an update in all users package managers.
So easy.
These days, with CI, ya berge your patch and the fix is built and sent to testing immediately, ready to be assessed for promotion, which then signs and pushes the artifact into the pipeline for synchronization and application everywhere.
Aside from the actual smarts to build a good package, the tooling makes every other part of is zero-friction.
I suspect you’re confusing the package manager with the supply chain. This should always be separate, lest we get some systemd-like spongiform blob with no ability to swap out and improve parts without discarding the whole.
I know right? This is literally what flat pack was meant to address. Just package has push it and let any system use it. So it is a little more complicated and another step on top of everything.
To be fair, he does address why flatpak isn’t really suitable for his package in particular, as it needs to get out of the sandbix flatpak puts it in, which is hacky. None-the-less, it does feel like he’s making a mountain out of a mole hill.
Ah that’s fair yeah. It’s getting better. But without some sort of system where a flat pack can request the permissions that it needs at install or runtime. It’s far from where it should be. Trivial to do with an app like flat seal. But that’s not typically installed out of the box. It’s all command line by default and very obtuse to the average user.
This is literally what flat pack was meant to address. Just package has push it and let any system use it. So it is a little more complicated and another step on top of everything.
Flatpak is absolutely terrible. As a second repository, it breaks source of truth, doesn’t share dependency or SBOM data with the system and at worst begs for dependency hell.
How does it lead to dependency hell? I thought the point was that it didn’t share dependencies with the host, so different apps could have different versions of the same package and everything would still work.
It does sound a bit daunting to begin with, but the process is actually fairly straightforward, and once you understand it, publishing your package to your repositories just becomes another step in your build pipeline.
Just give the source away. Don’t package it at all (or at most, do so for your own distro). If your software is good enough, others will do the packaging for you. You’ll have to field a few inquiries, maybe, until someone steps up, but if you’ve got a canned response already written those don’t take long to handle.
This dev’s problems are twofold: first, he’s in too much of a hurry to get this thing out everywhere, instead of waiting for others to take the burden off him. Second, he wrote his code in Rust, which is difficult to package correctly in a self-contained manner because it tries to force you into using its own language-specific dependency system. He picked up a footgun and shot himself with it, in other words, and is now hopping around cursing.
Just give the source away. Don’t package it at all (or at most, do so for your own distro). If your software is good enough, others will do the packaging for you.
As someone who’s developed and shipped software you may know, who’s worked in rel/eng, OS security, escalations (we research the fix for the sploit and build the artifact for the update pipeline; everyone calls it something else) before being just an admin to manage hordes of machines; with the knowledge gained in every post I’ve ever held, let me say:
THIS. ABSOLUTELY THIS.
If you are a great coder but suck at packaging or hate it, then don’t. You’d produce something sub-par anyway.
Let the knowledgeable enthusiastic obsessively-detailed people do it for you and you’ll be happier and so will they.
There’s nothing wrong with the language that I’m aware of, but some aspects of the ecosystem seem to have not been thought out with Linux packaging in mind, let’s say.
some aspects of the ecosystem seem to have not been thought out
Absolutely. But this is a natural effect of ditching mentors and documentors more than 20 years ago – coders now have no idea in general how to do supply-chains right; nor why.
That would make it worse. Then it would only be usable on systems based off of canonical. Not even debian. Snap does have a few small technical benefits over something like flat pack currently. But if you want to just package it and run it anywhere. Flat pack is the go-to
I once asked on LQ why on earth we somehow still can’t just settle on one fucking packaging format.
That was literally the point of AppImage and then Canonical/Red Hat decided they needed to do Snaps and Flatpak. It makes no fucking sense to have all 3 of these things when the whole point was that there would be ONE fucking format that could be run on any distro. That was the whole point.
They all have advantages and disadvantages in certain conditions. While others are around for two decades already and got a new wrapper around the existing one all 10 years or so.
I couldn’t find nix2rpm or nix2pacman, but there exist tools to convert between formats (alien, debtap, rpmtap, and one more who’s name I can’t remember but I remeber as being the most versatile).
What I’m trying to say, is that when people said “just use nix”, they probably really mean to use nix as a platform to build other packages withouth doing extra work.
Now they would still have to build for macos and windows, but they are already doing that anyways.
As a sidenote, there is also this: github.com/pacur/pacur , which is an aur like repo that buids debs, rpm’s, and pacman packages. So there’s semi-automatic updates, via a publuc repo you can out stuff on.
The real elite solution, imo, is to host forgejo, or use codeberg, which insanely has a package registry for every possible format of packages. So you can directly just push there, after building however you want.
But if developer’s were good at packaging, I wouldn’t be so mad when they try to do it.
Because this:
The next version will include a new built-in self-updating mechanism
Downloading unsigned, unverified binaries directly from the latest versioned github release?
Makes it so that all that’s needed for getting malware on the system is pwning the developers account via some supply chain malware, that hooks into there browser and pushes a release.
And every additional developer who can release, or every github actions that is potentially vulnerable but can be made to release, or claude (since the author is letting it commit, which requires it to run without sandboxing afaik) becomes more attack surface.
There are ways to fix this. Conventional distros use multi party signing of commits and releases, where developers continously verify eachother and look over changes.
More newfangled flows involve using github actions to build immutable releases, directly from tagged versions of the code.
But random developer #3989 isn’t doing this. They are distributing their software in a way that malware distributors will be ery happy to see after pwning their account.
I want devs to use nix, because then I can build or run their program directly from the source code. It sidesteps so many issues with visibility of the supply chain, or being unable to inspect what I am running.
I like nix becuase I can make developers like the above satisfied by giving them a way to easily build static binaries, or other formats.
Yeah. Honestly it might be a good idea to start a community project that’s a “nix2everyDistro”, and then go around and open PRs on FOSS projects to wire their project up to build in nix + output “every distro”
What I’m trying to say, is that when people said “just use nix”, they probably really mean to use nix as a platform to build other packages withouth doing extra work.
What you should be saying is something that recognizes that not all systems adhere to FHS. It needs to account for the non-compliant ones, and that includes Lennart’s delightful deviations when he obviously just knew better than the Industry.
So what’s wrong with just releasing a monolithic static binary for each architecture you want to run it on?
I read the post, and it sounds like what the author did in the end. I just don’t get why it was the last thing he tried. I don’t get why Linux software packaging has to be so complex.
So what’s wrong with just releasing a monolithic static binary for each architecture you want to run it on?
Release engineering works on something that has only recently been codified into a SLSA scale. At the top end is nixos and their very firm validation of code all the way along the supply chain.
At the bottom of the scale, a SLSA score of 0, is “here’s a monolithic binary I found on the website of someone on the interwebs. Even though we don’t know how it was built, nor can we reproduce it or confirm it’s what he built or look at any of the parts to see whether the statically-linked libraries are now at-risk, let’s just run that.”
You … do see how irresponsible that is on every level, right?
Like, 20 years ago I dropped a patch in for Apache, and these days I manage reams of machines using apache, sendmain, winbind, nginx, postfix, etc; to manage those without a proper infrastructure is absolute madness.
I seem to recall many years ago having to build some rpm packages. It was really not that difficult iirc. Cant imagine .deb or whatever is all that much harder.
When I packaged software, I did RPM for Fedora and openSUSE (because that’s what I used at that time) but out of curiosity I looked at .deb as well and boy are there weird things there.
There are many similarities between RPM and pkgbuild for Arch, so I could easily copy almost entire pkgbuilds from AUR and merely adapt a few variables but whatever the inventors of .deb concocted is crazy. Archives within archives and other stuff I actively repressed memories of.
I looked at .deb as well and boy are there weird things there.
There are many similarities between RPM and pkgbuild for Arch, so I could easily copy almost entire pkgbuilds from AUR and merely adapt a few variables but whatever the inventors of .deb concocted is crazy.
That build method dates back super-far, and probably mirrors unix packaging of the day. This is not a good thing, in that it did miss out on some validation features that RPM and NIX have, and that’s a problem that hasn’t been fixed (in unix or debian) even now (25+ years after we noticed it in the enterprise OS company I worked at, and which invalidated debian even then as a product we wanted to support alongside unix).
I do release a Flatpak but I’m actually working against it. It’s designed for self-contained sandboxed desktop GUI applications and I’m releasing a terminal-based TUI that allows you to rampage around your machine and network. I’m passing some horrible “bad practice” flags to get it to work. Flatpak sandboxing is not a good fit.
Okay sure but is this really such an issue? Installing the app normally is equivalent to passing those flags anyway right?
It is equivalent until Flatpak adds a new (probably completely justified) limitation to its sandbox that breaks your (needs unrestricted system-wide permissions) application, sure. Nevermind if you decide to add more features that go beyond what the sandbox allows, and then you need to figure out why they’re broken in your flatpak and what new flags you need to unbreak them. The problem is you’re constantly swimming upstream against a very strong current. It’s not a very sustainable mode of development. Like OP said, flatpak is designed for sandboxed desktop GUIs. It may be convenient at first to shoehorn your TUI application into it, but you’re working against its design and intent, and that is a genuine problem, for many reasons.
npm -g is another TUI install method I loathe. pipx/uv shares most of the same issues. Like the author, I’ve gone through almost all of the same options they have, and come to basically the same conclusions. There really is no good option for packaging and managing userspace TUI utilities. And it’s brutal. I hate that so many terminal utilities have standardized to use completely unrestricted, opaque and dangerous curl/sh copy-pastes, leaving anyone who’s not both fluent in shell script and willing to invest the time to inspect what the scripts are doing out in the cold, bravely running commands that are potentially malicious with no safety net. I understand it, it’s viable, it works, I do it myself for my own projects, but we have to acknowledge that it’s a symptom of the lack of any better solution for TUI packaging, and it’s okay to admit that it sucks, and we could do something so much better.
The author seems to have completely missed the point that they can just publish their own repositories to allow users on both of these platforms to update. It would also mean they could avoid the problems they’re having building source packages.
Their complaints about supporting instalations with older libc versions are also addressable by just building against the oldest libc they wish to support.
i guess the problem with that is having to maintain the repository setup. they’re basically web servers with signed files.
That does solve one problem at the cost of a bunch of other potential problems.
I don’t see the problem. For years I’ve used the public Open Build Service instance at openSUSE to package and publish software for openSUSE and Fedora (never cared for Debian/Ubuntu because I packaged the software only for my own use but it’s possible).
You can put the repositories on pretty much any static hosting. I haven’t tried it, but I’m pretty sure that, with a bit of fiddling, you could even put them on a github pages site.
Both distros have good tooling for maintaining repositories, and I’d say it’s probably easier than getting your package into the official repos and keeping it updated there, especially on distros like Debian which favour stability over having the latest versions of packages.
No distribution system is going to be entirely friction free for the author, but I’ve not encountered any significant issues hosting deb or rpm repos.
Harbottle does this. It’s very well-done.
Can’t you just put all your packages in the same repository but different directories? Example: debian/.deb, fedora/.rpm, appimage/appimage…
Sort of, but each of them uses a different way of making the repository, so really it’s more a case of different repositories in different directories, but they can all be on the same website.
that defeats the purpose of a package manager
Not really. The user adds the repository to the list of repositories their package manager is using, and the packages in that repository appear as normal packages for them to select. When the author publishes a new version of their package it’ll appear as an update in all users package managers.
Fun fact: when we were doing devops as config management, we distributed repository configs AS packages … which we updated as required.
So easy.
These days, with CI, ya berge your patch and the fix is built and sent to testing immediately, ready to be assessed for promotion, which then signs and pushes the artifact into the pipeline for synchronization and application everywhere.
Aside from the actual smarts to build a good package, the tooling makes every other part of is zero-friction.
I suspect you’re confusing the package manager with the supply chain. This should always be separate, lest we get some systemd-like spongiform blob with no ability to swap out and improve parts without discarding the whole.
Unless you’re Lennart, it’s a bad idea.
yes i was. didn’t get great sleep, my bad.
I know right? This is literally what flat pack was meant to address. Just package has push it and let any system use it. So it is a little more complicated and another step on top of everything.
To be fair, he does address why flatpak isn’t really suitable for his package in particular, as it needs to get out of the sandbix flatpak puts it in, which is hacky. None-the-less, it does feel like he’s making a mountain out of a mole hill.
Ah that’s fair yeah. It’s getting better. But without some sort of system where a flat pack can request the permissions that it needs at install or runtime. It’s far from where it should be. Trivial to do with an app like flat seal. But that’s not typically installed out of the box. It’s all command line by default and very obtuse to the average user.
Flatpak is absolutely terrible. As a second repository, it breaks source of truth, doesn’t share dependency or SBOM data with the system and at worst begs for dependency hell.
At best it’s XKCD 927.
How does it lead to dependency hell? I thought the point was that it didn’t share dependencies with the host, so different apps could have different versions of the same package and everything would still work.
All of that sounds unintuituve and complicated, which I think is the main complaint overall.
It does sound a bit daunting to begin with, but the process is actually fairly straightforward, and once you understand it, publishing your package to your repositories just becomes another step in your build pipeline.
Just give the source away. Don’t package it at all (or at most, do so for your own distro). If your software is good enough, others will do the packaging for you. You’ll have to field a few inquiries, maybe, until someone steps up, but if you’ve got a canned response already written those don’t take long to handle.
This dev’s problems are twofold: first, he’s in too much of a hurry to get this thing out everywhere, instead of waiting for others to take the burden off him. Second, he wrote his code in Rust, which is difficult to package correctly in a self-contained manner because it tries to force you into using its own language-specific dependency system. He picked up a footgun and shot himself with it, in other words, and is now hopping around cursing.
As someone who’s developed and shipped software you may know, who’s worked in rel/eng, OS security, escalations (we research the fix for the sploit and build the artifact for the update pipeline; everyone calls it something else) before being just an admin to manage hordes of machines; with the knowledge gained in every post I’ve ever held, let me say:
THIS. ABSOLUTELY THIS.
If you are a great coder but suck at packaging or hate it, then don’t. You’d produce something sub-par anyway.
Let the knowledgeable enthusiastic obsessively-detailed people do it for you and you’ll be happier and so will they.
Well, most arent. But create a software they want or need.
But I thought rust was 100% safe and the best? /s
There’s nothing wrong with the language that I’m aware of, but some aspects of the ecosystem seem to have not been thought out with Linux packaging in mind, let’s say.
Absolutely. But this is a natural effect of ditching mentors and documentors more than 20 years ago – coders now have no idea in general how to do supply-chains right; nor why.
I’m just making fun of the people who treat rust like the next best thing after sliced bread and pretend it will make it so you have 0 bugs.
Well, that sounds really immature.
Heckling introduces stress, providing incentive for improvement if the misbehaving member has that potential; otherwise shunning.
Their behaviour or me making fun of them?
Yes
I love how he literally packages for any system EXCEPT snap
That would make it worse. Then it would only be usable on systems based off of canonical. Not even debian. Snap does have a few small technical benefits over something like flat pack currently. But if you want to just package it and run it anywhere. Flat pack is the go-to
I think you can install snaps on Debian
They already support many repos. Adding Snap does not make it worse for the others.
I once asked on LQ why on earth we somehow still can’t just settle on one fucking packaging format.
That was literally the point of AppImage and then Canonical/Red Hat decided they needed to do Snaps and Flatpak. It makes no fucking sense to have all 3 of these things when the whole point was that there would be ONE fucking format that could be run on any distro. That was the whole point.
Couldn’t even do that right. FFS
Well great news, the world has mostly settled on Flatpak now.
They all have advantages and disadvantages in certain conditions. While others are around for two decades already and got a new wrapper around the existing one all 10 years or so.
Who benefits of the disagreement? Did they contribute?
Aur is back btw. I can push my packages again as a maintainer towards aur
Do nix and then use nix2appimage, or nix bundle to package the app compressed as an arx archive without the startup times they complain about.
You can also use github.com/DavHau/nix-portable to bundle it a bit better, without needing nix on the host.
And then, based off a quick search:
github.com/neobrain/nix2flatpak
github.com/barstoolbluz/nix2deb
I couldn’t find nix2rpm or nix2pacman, but there exist tools to convert between formats (alien, debtap, rpmtap, and one more who’s name I can’t remember but I remeber as being the most versatile).
What I’m trying to say, is that when people said “just use nix”, they probably really mean to use nix as a platform to build other packages withouth doing extra work.
On the other hand,
You can also use one of the newfangled appimage like formats: docs.pkgforge.dev/formats/packages
This one is linked in there and creates a static executable from any binary: github.com/VHSgunzo/sharun
Now they would still have to build for macos and windows, but they are already doing that anyways.
As a sidenote, there is also this: github.com/pacur/pacur , which is an aur like repo that buids debs, rpm’s, and pacman packages. So there’s semi-automatic updates, via a publuc repo you can out stuff on.
The real elite solution, imo, is to host forgejo, or use codeberg, which insanely has a package registry for every possible format of packages. So you can directly just push there, after building however you want.
But if developer’s were good at packaging, I wouldn’t be so mad when they try to do it.
Because this:
Downloading unsigned, unverified binaries directly from the latest versioned github release?
Makes it so that all that’s needed for getting malware on the system is pwning the developers account via some supply chain malware, that hooks into there browser and pushes a release.
And every additional developer who can release, or every github actions that is potentially vulnerable but can be made to release, or claude (since the author is letting it commit, which requires it to run without sandboxing afaik) becomes more attack surface.
There are ways to fix this. Conventional distros use multi party signing of commits and releases, where developers continously verify eachother and look over changes.
More newfangled flows involve using github actions to build immutable releases, directly from tagged versions of the code.
But random developer #3989 isn’t doing this. They are distributing their software in a way that malware distributors will be ery happy to see after pwning their account.
I want devs to use nix, because then I can build or run their program directly from the source code. It sidesteps so many issues with visibility of the supply chain, or being unable to inspect what I am running.
I like nix becuase I can make developers like the above satisfied by giving them a way to easily build static binaries, or other formats.
Yeah. Honestly it might be a good idea to start a community project that’s a “nix2everyDistro”, and then go around and open PRs on FOSS projects to wire their project up to build in nix + output “every distro”
What you should be saying is something that recognizes that not all systems adhere to FHS. It needs to account for the non-compliant ones, and that includes Lennart’s delightful deviations when he obviously just knew better than the Industry.
So what’s wrong with just releasing a monolithic static binary for each architecture you want to run it on?
I read the post, and it sounds like what the author did in the end. I just don’t get why it was the last thing he tried. I don’t get why Linux software packaging has to be so complex.
Release engineering works on something that has only recently been codified into a SLSA scale. At the top end is nixos and their very firm validation of code all the way along the supply chain.
At the bottom of the scale, a SLSA score of 0, is “here’s a monolithic binary I found on the website of someone on the interwebs. Even though we don’t know how it was built, nor can we reproduce it or confirm it’s what he built or look at any of the parts to see whether the statically-linked libraries are now at-risk, let’s just run that.”
You … do see how irresponsible that is on every level, right?
Like, 20 years ago I dropped a patch in for Apache, and these days I manage reams of machines using apache, sendmain, winbind, nginx, postfix, etc; to manage those without a proper infrastructure is absolute madness.
I don’t care for fresh (whatever it is) either, yet here we are.
I seem to recall many years ago having to build some rpm packages. It was really not that difficult iirc. Cant imagine .deb or whatever is all that much harder.
When I packaged software, I did RPM for Fedora and openSUSE (because that’s what I used at that time) but out of curiosity I looked at .deb as well and boy are there weird things there.
There are many similarities between RPM and pkgbuild for Arch, so I could easily copy almost entire pkgbuilds from AUR and merely adapt a few variables but whatever the inventors of .deb concocted is crazy. Archives within archives and other stuff I actively repressed memories of.
That build method dates back super-far, and probably mirrors unix packaging of the day. This is not a good thing, in that it did miss out on some validation features that RPM and NIX have, and that’s a problem that hasn’t been fixed (in unix or debian) even now (25+ years after we noticed it in the enterprise OS company I worked at, and which invalidated debian even then as a product we wanted to support alongside unix).
Okay sure but is this really such an issue? Installing the app normally is equivalent to passing those flags anyway right?
It is equivalent until Flatpak adds a new (probably completely justified) limitation to its sandbox that breaks your (needs unrestricted system-wide permissions) application, sure. Nevermind if you decide to add more features that go beyond what the sandbox allows, and then you need to figure out why they’re broken in your flatpak and what new flags you need to unbreak them. The problem is you’re constantly swimming upstream against a very strong current. It’s not a very sustainable mode of development. Like OP said, flatpak is designed for sandboxed desktop GUIs. It may be convenient at first to shoehorn your TUI application into it, but you’re working against its design and intent, and that is a genuine problem, for many reasons.
npm -gis another TUI install method I loathe. pipx/uv shares most of the same issues. Like the author, I’ve gone through almost all of the same options they have, and come to basically the same conclusions. There really is no good option for packaging and managing userspace TUI utilities. And it’s brutal. I hate that so many terminal utilities have standardized to use completely unrestricted, opaque and dangerous curl/sh copy-pastes, leaving anyone who’s not both fluent in shell script and willing to invest the time to inspect what the scripts are doing out in the cold, bravely running commands that are potentially malicious with no safety net. I understand it, it’s viable, it works, I do it myself for my own projects, but we have to acknowledge that it’s a symptom of the lack of any better solution for TUI packaging, and it’s okay to admit that it sucks, and we could do something so much better.I’ve installed one TUI app through brew (spicetify) and it seemed perfectly cromulent.