Rolling Back Linux Kernel Updates

About a month ago, after running some updates on my computer, I ran into a problem: I could boot to a user screen, enter my info, and click login, but past that the computer was unresponsive to any clicks on the mouse or keystrokes from the keyboard. The desktop was also effectively blank, as if it had not fully loaded.

I have been running Fedora 41 exclusively on my Framework 13 laptop since I got the laptop back in November. Fedora has been excellent and overall very smooth to use and this was the first problem I had. A few Google searches made me think that I had run a kernel update that had some hiccups.

To solve this, I needed to roll my system back to an earlier Linux kernel and take a few other actions to keep everything running smoothly. Here are some quick notes for how I rolled back to an earlier kernel and made that the default, and then how I began accepting updates again after some time had passed.

Rolling the Kernel Back and Setting a New Default

First, I learned that three kernels are kept in Fedora, in part to guard against this sort of problem. You can view your kernels and move between them by pressing the ctrl key immediately after the Framework logo appears. This takes you to the GRUB screen. It took me a few tries to get there, but once there you can easily choose a kernel to boot into.

I used the kernel immediately preceding the latest version and found it booted fine. Great! But how do I change the default kernel so that I don’t have to keep using the GRUB screen just to boot my computer?

Enter grubby, a command line utility for managing your boot configuration.

You can see your default kernel using the --default-kernel flag. Right now, my system shows the following:

dtburrows@fedora:~$ sudo grubby --default-kernel
/boot/vmlinuz-6.12.13-200.fc41.x86_64

That’s the kernel I’ve been using since I ran into this trouble with a kernel update.

As you might expect, you can update your default kernel with the same tool.

There are two ways to set your default kernel: by kernel path and by index. The indexes are relative and could shift depending on updates. I wanted to make sure I always used this particular version, so I used the kernel path.

Use grubby to list all available kernels:

grubby --info=ALL | grep ^kernel

Copy the kernel path, then use --set-default flag to update your default kernel:

grubby --set-default="/boot/vmlinuz-6.12.13-200.fc41.x86_64"

Locking Kernels

Although this solved my boot issue, I was worried about keeping my default kernel from getting overwritten by later updates. A few inquiries led me to the versionlock command, part of the DNF5 package manager. The versionlock command allows you to protect packages from later updates.

I used sudo dnf versionlock add and my particular kernel package to lock that kernel from later updates.

It doesn’t mean later updates don’t downloaded/installed. If I look at my installed kernels, I currently see this:

dtburrows@fedora:~$ sudo grubby --info=ALL | grep ^kernel
kernel="/boot/vmlinuz-6.13.6-200.fc41.x86_64"
kernel="/boot/vmlinuz-6.13.5-200.fc41.x86_64"
kernel="/boot/vmlinuz-6.12.13-200.fc41.x86_64"
kernel="/boot/vmlinuz-0-rescue-a6a9c747370443a1a3126f833db9694e"

Using versionlock just means that the kernel I’ve locked doesn’t get overwritten through the normal process of updates. It’s one of the three kernels that are, by default, saved.

Testing and Restoring the Default Kernel to Latest Version

I didn’t want to stay with the old kernel forever, and this morning some system updates reminded me it was probably time to try out the latest kernel.

I have found getting to the GRUB screen via the keyboard upon startup to be unreliable, so I changed my settings to show the GRUB screen upon every startup automatically. I did this using sudo grub2-editenv - unset menu_auto_hide, per instructions found here.

I booted into the latest kernel and test drove it for a few minutes to make sure the basics seemed functional. They did!

To change my boot settings to boot into the most recent kernel, I used the index path option in grubby, setting to 0 (which appears to always be the latest path):

sudo grubby --set-default-index=0

I haven’t removed the versionlock on my “safe” kernel yet, just in case. I’ll probably let it go in a few days.

Takeaways

Although this boot problem gave me a bit of a panic at the start - no one likes finding their computer unusable - I learned a number of things from the process:

• Three kernels are kept for later use, by default.
• The GRUB menu lets you move between them, as well as any designated recovery kernels.
  • It’s not always easy to get in manually, so useful to set the GRUB menu to appear on every boot.
• Use grubby on the command line for managing those kernels.
• Use versionlock to lock packages, making sure they aren’t subject to updates

I’d rather not have to mess with these things too often, but all are useful to know and have a bit of experience with.