Terry : kpatch

This page last changed on Aug 23, 2014 by terry.

kpatch

What is kpatch?

kpatch is a Linux dynamic kernel patching infrastructure which allows you to patch a running kernel without rebooting or restarting any processes. It enables sysadmins to apply critical security patches to the kernel immediately, without having to wait for long-running tasks to complete, for users to log off, or for scheduled reboot windows. It gives more control over uptime without sacrificing security or stability.

kpatch is currently in active development. For now, it should not be used in production environments.

kpatch project on GitHub https://github.com/dynup/kpatch

Installation

NOTE: You need about 15GB of free disk space for the kpatch-build cache in $HOME/.kpatch and for ccache.

Ubuntu 14.04

Install the dependencies for compiling kpatch

apt-get install make gcc libelf-dev

Install the dependencies for the kpatch-build command

# Install development tools to build packages
apt-get install dpkg-dev

# Install packages required to compile linux kernel
apt-get build-dep linux

# optional, but highly recommended
apt-get install ccache
ccache --max-size=5G

Install kernel debug symbols

# Add ddebs repository
codename=$(lsb_release -sc)
sudo tee /etc/apt/sources.list.d/ddebs.list << EOF
deb http://ddebs.ubuntu.com/ ${codename} main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-security main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-updates main restricted universe multiverse
deb http://ddebs.ubuntu.com/ ${codename}-proposed main restricted universe multiverse
EOF

# add APT key
wget -Nq http://ddebs.ubuntu.com/dbgsym-release-key.asc -O- | sudo apt-key add -
apt-get update && apt-get install linux-image-$(uname -r)-dbgsym

CentOS 7

Install the dependencies for compiling kpatch

sudo yum install gcc kernel-devel elfutils elfutils-devel

Install the dependencies for the kpatch-build command

sudo yum install rpmdevtools pesign yum-utils zlib-devel \
  binutils-devel newt-devel python-devel perl-ExtUtils-Embed \
  audit-libs audit-libs-devel numactl-devel pciutils-devel bison

# enable CentOS 7 debug repo
sudo yum-config-manager --enable debug

# supporting packages to compile kernel
sudo yum-builddep kernel

# install kernel debuginfo packages
sudo debuginfo-install kernel

# optional, but highly recommended - enable EPEL 7
sudo yum install ccache
ccache --max-size=5G

Oracle Linux 7

Install the dependencies for compiling kpatch

 sudo yum install gcc kernel-devel elfutils elfutils-devel

Install the dependencies for the kpatch-build command

sudo yum install rpmdevtools pesign yum-utils zlib-devel \
  binutils-devel newt-devel python-devel perl-ExtUtils-Embed \
  audit-libs numactl-devel pciutils-devel bison

# enable ol7_optional_latest repo
sudo yum-config-manager --enable ol7_optional_latest

# install supporting packages to compile kernel
sudo yum-builddep kernel

# manually install kernel debuginfo packages
# because debuginfo for OL7 is NOT a YUM repo yet -_-z
rpm -ivh https://oss.oracle.com/ol7/debuginfo/kernel-debuginfo-$(uname -r).rpm
rpm -ivh https://oss.oracle.com/ol7/debuginfo/kernel-debuginfo-common-x86_64-$(uname -r).rpm

# optional, but highly recommended - enable EPEL 7
sudo yum install ccache
ccache --max-size=5G

NOTE: debuginfo-install kernel will NOT work as the ol7/debuginfo is NOT a YUM repo yet, no repodata directory structure. The administrator needs to run createrepo against the directory.

Build kpatch from source

Build kpatch from source code

# checkout the source code from GitHub
git clone https://github.com/dynup/kpatch.git

cd kpatch

# make use multi-core or multi-processor
make -j$(nproc)

# installs to /usr/local
make install

NOTE: The binaries are installed to /usr/local

Quick Start

NOTE: While kpatch is designed to work with any recent Linux kernel on any distribution, the "kpatch-build" command has ONLY been tested and confirmed to work on Fedora 20, RHEL 7, CentOS 7, Oracle Linux 7 (RHCK), Ubuntu 14.04 and Debian 7.

First, make a source code patch against the kernel tree using diff, git, or quilt.

For Ubuntu, get the source code by

apt-get install linux-source

Example: Change the output of /proc/version

Copy (rsync) the linux kernel source tree to current working directory

Original Linux kernel source tree
NOTE: source tree should come from

  • Ubuntu / Debian -> linux-source
    Location -> /usr/src/linux-source-3.14.tar.xz
  • RHEL / CentOS / Oracle Linux -> kernel-debuginfo-common-x86_64
    Location->  /usr/src/debug/kernel-$(uname -r)/linux-$(uname -r)

Source -> $HOME/linux.orig

Target -> $HOME/linux.kpatch

Modify $HOME/linux.kpatch/fs/proc/version.c

Generate a patch

diff -u patch $HOME/linux.orig/fs/proc/version.c $HOME/linux.kpatch/fs/proc/version.c > version.patch

# or use colordiff

colordiff -u patch $HOME/linux.orig/fs/proc/version.c $HOME/linux.kpatch/fs/proc/version.c > version.patch

The patch looks like

version.patch
--- linux.orig/fs/proc/version.c    2014-07-17 04:25:31.000000000 +1000
+++ linux.kpatch/fs/proc/version.c    2014-08-18 18:01:16.624716235 +1000
@@ -8,6 +8,7 @@
 static int version_proc_show(struct seq_file *m, void *v)
 {
     seq_printf(m, linux_proc_banner,
+        "kpatch @ Ubuntu",
         utsname()->sysname,
         utsname()->release,
         utsname()->version);

Build the dynamic kernel patch module

kpatch-build -d version.patch

NOTE: For Ubuntu and Debian, modify kpatch-build script to choose a closer mirror site. For example vim $(which kpatch-build).

NOTE: the -t vmlinux option is used to tell kpatch-build to only look for changes in the vmlinux base kernel image, which is much faster than also compiling all the kernel modules. If your patch affects a kernel module, you can either omit this option to build everything, and have kpatch-build detect which modules changed, or you can specify the affected kernel build targets with multiple -t options.

That outputs a patch module named kpatch-version.ko in the current directory.

Now apply it to the running kernel

root@kpatch:/root# kpatch load kpatch-version.ko 
loading core module: /usr/local/lib/kpatch/3.13.0-32-generic/kpatch.ko
loading patch module: kpatch-version.ko
root@kpatch:/root# kpatch list
Loaded patch modules:
kpatch_version

Installed patch modules:
root@kpatch:/root# lsmod | grep kpatch
kpatch_version         13805  1 
kpatch                 18442  1 kpatch_version
root@kpatch:/root# cat /proc/version
kpatch @ Ubuntu version Linux (buildd@kissel) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) 3.13.0-32-generic
root@kpatch:/root# kpatch unload kpatch_version
disabling patch module: kpatch_version
unloading patch module: kpatch_version
root@kpatch:/root# kpatch list
Loaded patch modules:

Installed patch modules:
root@kpatch:/root# cat /proc/version
Linux version 3.13.0-32-generic (buildd@kissel) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014

Attachments

  File Modified
PDF File
kpatch-linuxcon_3.pdf
Aug 23, 2014 by Terry Wang

 

Attachments:

kpatch-linuxcon_3.pdf (application/pdf)