Not a review of CentOS Stream 8
I have so many notes. I started running CentOS Stream 8 five months ago, and I wanted to document every win, loss, setback and solution as I tried to make what isn’t really a desktop distribution — Red Hat Enterprise Linux (and its clones) — into my laptop operating system.
I’m not going to look at my notes. This non-review won’t write itself, and the notes aren’t going to do it for me.
In 2019, I was running Debian Buster and eventually Bullseye on my 2017 HP Envy laptop equipped with a 250 GB HP NVMe M.2 SSD.
By early 2021, I wanted a bigger SSD and bought a 1 TB Samsung EVO NVMe M.2 SSD. I opened the laptop and swapped in the bigger drive.
Since I had a new drive, it was a good time to try a new (to me) operating system. I had been following the controversy over IBM/Red Hat ending its CentOS Linux 8 distribution very early and “replacing” it with the rolling CentOS Stream 8.
I am a big fan of Fedora, and CentOS/RHEL has always struck me as a kind of, sort of Fedora LTS.
RHEL and its downstream clones are cut from Fedora releases. And they are supported for a much longer period of time — with five years promised for CentOS Stream users and the full 10 years for RHEL and its new downstreams, AlmaLinux and Rocky Linux. (Nobody is suggesting you stick to the same desktop system for 10 years. I don’t remember ever using the same Linux distribution for more than two years on a system I use every day. I only offer these support periods as a point of reference.)
While various Red Hat employees have tried to disabuse me of thinking that CentOS (be it the dying Linux or the living Stream) is a Fedora LTS, at some level I wanted a Fedora-like system that didn’t require a fairly major upgrade every six months to a year. And RHEL/CentOS, along with new clones Alma and Rocky, are certainly that.
And working with an OS that is “closer” to the Red Hat mothership had a certain appeal.
The CentOS team — meaning the Red Hat people who work on CentOS — have justified the replacement of CentOS Linux by CentOS Stream with two things that make it different:
- CentOS Stream is supposed to be slightly upstream of Red Hat Enterprise Linux (though in practice it’s more complicated)
- CentOS Stream will have a real community of not just users but also contributors (My wish: Make the CentOS community as much like the Fedora community as possible.)
These two points intrigued me, and with assurances that Stream is not anywhere near as far upstream as Fedora and includes packages that undergo testing in the bowels of Red Hat before they are pushed to possibly unwitting Stream users, I took the leap.
I decided to download the 700 MB Boot ISO and not the 10 GB DVD ISO of CentOS Stream 8. (Point of order: Even though nobody is using DVD-R discs any more, why continue calling it a DVD ISO if it’s too big to fit on a DVD?)
At the time, the Boot ISO had a bug that prevented it from offering users a working mirror to do the installation. I figured out how to manually enter a mirror and complete the installation, and one of my first participatory acts as a CentOS Stream user was filing a bug on the installer.
Filing that bug was more than a little complicated. CentOS systems that aren’t Stream have their own repos. But Stream’s development happens inside Red Hat. You file bugs through the Red Hat Bugzilla. I already had a Fedora account, and it works in the RH Bugzilla, so I was off to the races.
The bug was fixed. It was the first inkling that CentOS Stream might be like Fedora in that broken things do get fixed in a reasonable time frame.
But something bothered me. I saw messages in forums here and there about this bug, but there weren’t very many people running into it. From what I could tell, that meant very few were using the 700 MB installer — and I don’t think it was because the 10 GB installer was so overwhelmingly popular.
Still, I can’t think of any other Linux distribution that would release an ISO without testing it. How could even a single tester have missed this?
My contention — then and now — is that CentOS Stream 8 is not being used by very many people, and definitely not actively in an “I’m excited about this distribution and want you to know it” way.
You don’t see a lot of tweets from users. The CentOS subreddit, which actively insults Stream in its description, is nearly dead. The CentOS Forums aren’t terribly busy overall, and there are very few posts about Stream.
Red Hat/CentOS people say theirs is a mailing-list culture.
So I joined the CentOS mailing lists. Not a lot is happening there either. Maybe things will get more busy.
What CentOS Stream needs is some good word of mouth. Today this happened:
https://twitter.com/carlwgeorge/status/1445598349789306880
(Note: I think the above tweet can only be seen by approved followers, of which I am not one)
If CERN is OK using CentOS Stream on thousands of mission-critical computers, it’s probably OK for you and me.
A half-dozen more of these, and Stream will have some momentum.
I’m not here to talk about whether CentOS Stream 8 is drawing a large user base. I wish it was. Overall I’ve been pretty happy with Stream, and I don’t think it’s necessary for almost all users to opt for “real” RHEL or “true” RHEL clones like the new AlmaLinux and Rocky Linux.
One thing Alma and Rocky are doing is making the community the center of what they do. Just as CentOS — the project — should take its cues from the Fedora Project, it should also see what Alma and Rocky are doing and do more of that. Much more.
I worry that Red Hat and IBM’s idea of “community” for Stream is largely centered around the giant corporations like Facebook who are eager to get their patches into RHEL and don’t at all mind dogfooding Stream as part of that process. What about the individual user and contributor? I hope the CentOS team doesn’t ignore that part of the Linux community in favor of the large enterprises who are very keen to contribute directly to RHEL.
Have I been a complainer in the CentOS community? Yeah, probably. A lot of that has to do with a lack of understanding on my part — and lack of explanation on the project’s part — of how CentOS Stream 8 is developed.
I like the idea of Stream being the upstream to RHEL. I am very happy to get updates a few days or a week earlier than RHEL.
But the kernel — the most important part of any and every Linux system — is more of a mystery. I started seeing kernel updates for RHEL, and five, 10, 15 days would go by with nothing for Stream 8. Did all those CVEs get fixed a week or two earlier than that?
I have been looking at CentOS Stream kernel source (yeah, this has driven me to that) and comparing the CVEs cited there with Red Hat update messages published on LWN.net. So far it looks like at least some (but by no means all) are being fixed a couple of weeks earlier in the Stream kernel. What would be better? Letting users know it’s happening. Emails or web sites that detail security and bug fixes? CentOS needs them. I’ve heard that the Red Hat connection makes this difficult. They need to find a way.
And what isn’t getting fixed in Stream in a timely manner? It’s hard to tell.
Hint to Red Hat managers and CentOS developers: CentOS Stream should absolutely be on the LWN.net Security page.
Still, I’ve gotten to the point where I trust Red Hat to deliver — or at least hope they will deliver — a decent kernel to Stream. Red Hat people have gone out of their way to say that this and all other problems in CentOS Stream’s workflows should be ironed out in the Stream 9 cycle. EVERYTHING, they say.
Maybe it’s just me, but I’ve noticed in recent months that Red Hatters have backed off of their campaign to get everyone to switch to Stream 8.
Since the announcement was made about CentOS Linux 8’s EOL, both Alma and Rocky have built up their infrastructures and communities. They are pretty much ready to give whoever still wants a downstream RHEL clone exactly that.
I have no evidence supporting this whatsoever, but I get this feeling: Now that Alma and Rocky are here, Red Hatters are very much OK with CentOS Linux users going directly to the new downstream distributions and skipping Stream altogether.
There’s a path to contribute to RHEL and its downstreams, and that is CentOS Stream. They have that. Even if Alma or Rocky is your system of choice, you would still file a bug or submit a patch for Stream.
Maybe Red Hat doesn’t care if very few CentOS Linux users switch over to CentOS Stream. Maybe they’re OK with it being a piece of the RHEL development pipeline that for the first time allows “outsiders” to contribute directly to the company’s flagship product.
CentOS, Alma and Rocky are all working on developing SIGs — special interest groups — to help fill in the gaps in RHEL and its clones. While I’ve heard rumblings, I haven’t yet seen a Desktop SIG. EPEL fills in many gaps, and Flathub fills in more.
But there are still missing pieces that the community can fill in. I hope it happens and that Red Hat Enterprise Linux, CentOS Stream and all of the RHEL downstreams become better distributions for desktop use as the result of that work.
The beauty of an effort to create a comprehensive desktop repository for these distributions is that it — or they — can exist independently of RHEL and Stream and be added to all of these systems just like any other repo.
As for Red Hat and Stream, there is always time for them to get it right. Or at least get it more right. I hope it happens, and I’m willing to help.
And a few more positive reviews from CERN-sized users wouldn’t hurt, either.
Discuss this post on Twitter.