Patching and Building the Fedora Kernel¶
It’s not easy to get up to date information on how to create a custom kernel on Fedora. There’s plenty of information out there, but most of it is outdated and only halfway true.
Here’s what I was able to find out by combining non-outdated information into a working procedure. It goes as follows.
Why?¶
Tracking down a bug on my new USB-C/Thunderbolt-only laptop where an external HDMI monitor is plugged via an adapter.
Plug external monitor
Unplug external monitor
Wait a few seconds
Watch kernel go up in flames
Feb 26 13:58:08 zen kernel: BUG: kernel NULL pointer dereference, address: 0000000000000080
Feb 26 13:58:08 zen kernel: #PF: supervisor read access in kernel mode
Feb 26 13:58:08 zen kernel: #PF: error_code(0x0000) - not-present page
Feb 26 13:58:08 zen kernel: PGD 0 P4D 0
Feb 26 13:58:08 zen kernel: Oops: 0000 [#1] SMP PTI
Feb 26 13:58:08 zen kernel: CPU: 0 PID: 281 Comm: kworker/0:2 Not tainted 5.4.13-201.fc31.x86_64 #1
Feb 26 13:58:08 zen kernel: Hardware name: ASUSTeK COMPUTER INC. ZenBook S UX391UA/UX391UA, BIOS UX391UA.204 05/18/2018
Feb 26 13:58:08 zen kernel: Workqueue: events ucsi_connector_change [typec_ucsi]
Feb 26 13:58:08 zen kernel: RIP: 0010:ucsi_displayport_remove_partner+0xa/0x20 [typec_ucsi]
Feb 26 13:58:08 zen kernel: Code: 38 00 c7 43 28 00 00 00 00 48 83 c7 10 5b e9 6d f3 3d fb 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 48 85 ff 74 0f <48> 8b 47 78 48 c7 00 00 00 00 00 c6 40 3d 00 c3 66 0f 1f 44 00 00
Feb 26 13:58:08 zen kernel: RSP: 0018:ffff9daa002d3df8 EFLAGS: 00010202
Feb 26 13:58:08 zen kernel: RAX: 0000000000000008 RBX: ffff901f4ce35710 RCX: 0000000000003c7d
Feb 26 13:58:08 zen kernel: RDX: 0000000000003c7c RSI: 0000000000000001 RDI: 0000000000000008
Feb 26 13:58:08 zen kernel: RBP: 0000000000000000 R08: ffffffffbc7dbe60 R09: ffff9daa002d3cf0
Feb 26 13:58:08 zen kernel: R10: ffff901f54951a18 R11: ffff901f56a68a78 R12: ffff901f4ce35710
Feb 26 13:58:08 zen kernel: R13: 0000000000000001 R14: ffff901f4ce35860 R15: ffff901f4ce355d8
Feb 26 13:58:08 zen kernel: FS: 0000000000000000(0000) GS:ffff901f56a00000(0000) knlGS:0000000000000000
Feb 26 13:58:08 zen kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 26 13:58:08 zen kernel: CR2: 0000000000000080 CR3: 000000028fbbc003 CR4: 00000000003606f0
Feb 26 13:58:08 zen kernel: Call Trace:
Feb 26 13:58:08 zen kernel: ucsi_unregister_altmodes+0x77/0x90 [typec_ucsi]
Feb 26 13:58:08 zen kernel: ucsi_unregister_partner.part.0+0x13/0x30 [typec_ucsi]
Feb 26 13:58:08 zen kernel: ucsi_connector_change+0x247/0x340 [typec_ucsi]
Feb 26 13:58:08 zen kernel: process_one_work+0x1b5/0x360
Feb 26 13:58:08 zen kernel: worker_thread+0x50/0x3c0
Feb 26 13:58:08 zen kernel: kthread+0xf9/0x130
Feb 26 13:58:08 zen kernel: ? process_one_work+0x360/0x360
Feb 26 13:58:08 zen kernel: ? kthread_park+0x90/0x90
Feb 26 13:58:08 zen kernel: ret_from_fork+0x35/0x40
Feb 26 13:58:08 zen kernel: Modules linked in: cdc_ether usbnet r8152 mii rc_cec typec_displayport thunderbolt uinput rfcomm ccm xt_CHECKSUM xt_MASQUERADE nf_nat_tftp nf_conntrack_tftp tun bridge stp llc nf_conntrack_netbios_ns nf_conntrack_broadcast xt_CT ip6t_REJECT nf_reject_ipv6 ip6t_rpfilter ipt_REJECT nf_reject_ipv4 xt_conntrack ebtable_nat ebtable_broute ip6table_nat ip6table_mangle ip6table_raw ip6table_security iptable_nat nf_nat iptable_mangle iptable_raw iptable_security nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c ip_set nfnetlink ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter cmac bnep sunrpc vfat fat snd_hda_codec_hdmi snd_soc_skl snd_soc_sst_ipc snd_soc_sst_dsp snd_hda_ext_core x86_pkg_temp_thermal intel_powerclamp snd_soc_acpi_intel_match coretemp snd_soc_acpi kvm_intel snd_soc_core snd_hda_codec_realtek snd_hda_codec_generic snd_compress ledtrig_audio ac97_bus kvm snd_pcm_dmaengine iwlmvm snd_hda_intel snd_intel_dspcfg irqbypass snd_hda_codec uvcvideo mac80211
Feb 26 13:58:08 zen kernel: btusb iTCO_wdt btrtl crct10dif_pclmul videobuf2_vmalloc iTCO_vendor_support videobuf2_memops mei_hdcp btbcm btintel crc32_pclmul intel_rapl_msr snd_hda_core videobuf2_v4l2 libarc4 ghash_clmulni_intel videobuf2_common snd_hwdep intel_cstate bluetooth snd_seq asus_nb_wmi iwlwifi videodev snd_seq_device intel_uncore asus_wmi snd_pcm intel_rapl_perf mc cdc_acm sparse_keymap wmi_bmof ecdh_generic cfg80211 intel_wmi_thunderbolt ecc pcspkr snd_timer snd i2c_i801 soundcore joydev idma64 intel_xhci_usb_role_switch mei_me ucsi_acpi processor_thermal_device typec_ucsi intel_rapl_common mei roles intel_soc_dts_iosf intel_pch_thermal typec int3403_thermal int340x_thermal_zone int3400_thermal acpi_thermal_rel acpi_pad binfmt_misc ip_tables rfkill i915 i2c_algo_bit drm_kms_helper hid_multitouch drm nvme crc32c_intel nvme_core serio_raw wmi i2c_hid video fuse
Feb 26 13:58:08 zen kernel: CR2: 0000000000000080
Feb 26 13:58:08 zen kernel: ---[ end trace 76d248e576fee192 ]---
Feb 26 13:58:08 zen kernel: RIP: 0010:ucsi_displayport_remove_partner+0xa/0x20 [typec_ucsi]
Feb 26 13:58:08 zen kernel: Code: 38 00 c7 43 28 00 00 00 00 48 83 c7 10 5b e9 6d f3 3d fb 66 66 2e 0f 1f 84 00 00 00 00 00 66 90 0f 1f 44 00 00 48 85 ff 74 0f <48> 8b 47 78 48 c7 00 00 00 00 00 c6 40 3d 00 c3 66 0f 1f 44 00 00
Feb 26 13:58:08 zen kernel: RSP: 0018:ffff9daa002d3df8 EFLAGS: 00010202
Feb 26 13:58:08 zen kernel: RAX: 0000000000000008 RBX: ffff901f4ce35710 RCX: 0000000000003c7d
Feb 26 13:58:08 zen kernel: RDX: 0000000000003c7c RSI: 0000000000000001 RDI: 0000000000000008
Feb 26 13:58:08 zen kernel: RBP: 0000000000000000 R08: ffffffffbc7dbe60 R09: ffff9daa002d3cf0
Feb 26 13:58:08 zen kernel: R10: ffff901f54951a18 R11: ffff901f56a68a78 R12: ffff901f4ce35710
Feb 26 13:58:08 zen kernel: R13: 0000000000000001 R14: ffff901f4ce35860 R15: ffff901f4ce355d8
Feb 26 13:58:08 zen kernel: FS: 0000000000000000(0000) GS:ffff901f56a00000(0000) knlGS:0000000000000000
Feb 26 13:58:08 zen kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Feb 26 13:58:08 zen kernel: CR2: 0000000000000080 CR3: 000000028fbbc003 CR4: 00000000003606f0
Looking at the cause, we see that the crash is a NULL pointer that is
deferenced in ucsi_displayport_remove_partner(),
Feb 26 13:58:08 zen kernel: BUG: kernel NULL pointer dereference, address: 0000000000000080
... (blah) ...
Feb 26 13:58:08 zen kernel: RIP: 0010:ucsi_displayport_remove_partner+0xa/0x20 [typec_ucsi]
A look at the offending source code shows that the fix must be
easy. The only pointer that is being accessed is dp, and that
pointer is not checked for NULL-ness.
master)¶ 1void ucsi_displayport_remove_partner(struct typec_altmode *alt)
2{
3 struct ucsi_dp *dp;
4
5 if (!alt)
6 return;
7
8 dp = typec_altmode_get_drvdata(alt);
9 dp->data.conf = 0;
10 dp->data.status = 0;
11 dp->initialized = false;
12}
The fix is really easy,
--- /home/jfasch/My-Projects/jfasch-home/blog/2020/02/displayport.c.orig
+++ /home/jfasch/My-Projects/jfasch-home/blog/2020/02/displayport.c
@@ -271,6 +271,8 @@
return;
dp = typec_altmode_get_drvdata(alt);
+ if (!dp)
+ return;
dp->data.conf = 0;
dp->data.status = 0;
dp->initialized = false;
The bug is tracked as Red Hat Bugzilla – Bug 1785972, and upstream is already involved. John Stebbins and I provided testing, but roundtrips are rather long. What makes things worse is that there appear to be multiple issues, not just one.
I’ll focus on mine: patch, build, report, and help with testing. First, see how kernels are built in Fedora.
Find Kernel Source (Git), Fix It, and Create Patch¶
Fedora has a Git repository at kernel.org where they apply their
own patches on top of the vanilla kernel. Clone that, and create a
development branch. (I am on Fedora 31, so I’m basing the branch on
that.)
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/jwboyer/fedora.git
$ git checkout -b jfasch-fix remotes/origin/f31
Fix drivers/usb/typec/ucsi/displayport.c as sketched above, and
commit.
$ git commit -am 'fix RIP:ucsi_displayport_remove_partner()'
The remainder of the procedure will build the kernel RPM. A kernel RPM build works by applying a set of patches on top of the base vanilla kernel [1], so we create a patch for later use.
I made only one commit for which I want to create a patch. Find out
the revision that this patch is based upon; it is one revision before
the HEAD.
$ git show --quiet HEAD~1
commit 4382f76bc8ef9fce5e7e96d4cdb0c073564ad249 (tag: kernel-5.5.6-201.fc31, origin/f31)
Author: Josh Boyer <jwboyer@fedoraproject.org>
Date: Mon Feb 24 23:09:20 2020 +0000
kernel-5.5.6-201.fc31 configs
Create the patch which we will pick up later,
$ git format-patch -o /tmp 4382f76bc8ef9fce5e7e96d4cdb0c073564ad249
/tmp/0001-fix-RIP-ucsi_displayport_remove_partner.patch
Prepare the RPM Build¶
Install Prerequisites¶
$ sudo dnf install rpmdevtools koji
Setup rpmbuild Directory¶
Setup an empty RPM tree. This will simply create a directory
rpmbuild skeleton tree in the home directory.
$ rpmdev-setuptree
$ tree ~/rpmbuild/
/home/jfasch/rpmbuild/
├── BUILD
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS
Download and Install Source RPM For Crashing Kernel¶
Find out the version of the crashing kernel (the one that is currently running),
$ uname -r
5.4.13-201.local.fc31.x86_64
Download the corresponding source RPM from their build engine,
$ koji download-build --arch=src kernel-5.4.13-201.fc31
$ ls -l *.rpm
kernel-5.4.13-201.fc31.src.rpm
Install the RPM. This will fill the ~/rpmbuild/ skeleton with the
kenrel build instructions.
$ rpm -ivh kernel-5.4.13-201.fc31.src.rpm
$ tree ~/rpmbuild/
/home/jfasch/rpmbuild/
├── BUILD
├── RPMS
├── SOURCES
│ ├── 0001-crypto-ccp-Release-all-allocated-memory-if-sha-type-.patch
│ ├── 0001-Drop-that-for-now.patch
... (blah) ...
├── SPECS
│ └── kernel.spec
└── SRPMS
Apply the Patch¶
Copy the patch from above into the build tree, where the other patches are,
$ cp /tmp/0001-fix-RIP-ucsi_displayport_remove_partner.patch \
~/rpmbuild/SOURCES/RIP-ucsi_displayport_remove_partner.patch
Edit the build specification, ~/rpmbuild/SPECS/kernel.spec, to
contain a descriptive version
apply the patch
--- /home/jfasch/My-Projects/jfasch-home/blog/2020/02/kernel.spec.orig
+++ /home/jfasch/My-Projects/jfasch-home/blog/2020/02/kernel.spec
@@ -53,7 +53,7 @@
%global zipsed -e 's/\.ko$/\.ko.xz/'
%endif
-# define buildid .local
+%define buildid .jfasch
%if 0%{?fedora}
%define primary_target fedora
@@ -825,6 +825,7 @@
# ALSA code from v5.6 (Intel ASoC Sound Open Firmware driver support)
Patch601: alsa-5.6.patch
+Patch999: RIP-ucsi_displayport_remove_partner.patch
# END OF PATCH DEFINITIONS
Build the Kernel RPMs¶
$ cd ~/rpmbuild/SPECS/
$ rpmbuild -bb --target=x86_64 kernel.spec
... roedel ...
Before doing this, make sure the following is available:
Enough RAM
Enough disk
Patience, coffee, or something else to do
Install Kernel¶
$ sudo rpm -ivh --oldpackage \
~/rpmbuild/RPMS/x86_64/kernel-core-5.4.13-201.jfasch.fc31.x86_64.rpm \
~/rpmbuild/RPMS/x86_64/kernel-modules-5.4.13-201.jfasch.fc31.x86_64.rpm
Note
The kernel is not signed, obviously, so you might have to disable Secure Boot in your UEFI.
Footnotes