Ok, I've started a new course on the Linux kernel. It's mostly labs and assignments with lectures.

The course Operating System 2 or Linux Kernel Labs is focused on the Linux kernel, on how it works and how to interact with it. It explores major concepts such as processes, filesystems, I/O, and more. Most of the labs and assignments involve building kernel drivers, giving us real hands-on practice by loading them into a running kernel.
You can go on the website to get more informations about it:

Linux Kernel version: 5.10.14
Labs: To discover concepts
Assignments
Machine configuration/Setup: https://gitlab.cs.pub.ro/so2

In this serie I'll talk mostly about the labs and assignments

Lectures

The lectures cover system calls, processes, and interrupts.

It covers how system calls are bound to a number, processes with their virtual memory and the task_struct that represents a process in the Linux kernel, and other topics.

Let's move on to the labs (they were more interesting and less boring, sorry 😅). And if you took a course on operating systems like the one offered by MIT, there is some common knowledge and patterns you'll quickly notice.
You'll feel much less overwhelmed by all the concepts of the Linux kernel.

LABS

Kernel Modules

Link

The tasks in this lab are about creating, loading, and unloading kernel modules.

Kernel modules (.ko files) are compiled source code that can be loaded into a running kernel, so there is no need to stop or restart the OS. They run at the kernel level. Inside kernel modules, we can use the kernel API to do many things like getting information about processes, changing their behavior, allocating memory, and much more.
You write them like C code (with some conventions), write a Kbuild file, and that's it.

What's Kbuild ?

Kbuild is the primary build system used by the Linux Kernel . It compiles both the core kernel and loadable modules, handling complex dependencies, compiler flags, and architecture-specific configurations across various hardware platforms.

Google.

Here is an example of a kernel module

Kbuild

obj-m = hello_mod.o

hello_module.c

#include <linux/module.h>// always required
// export functions like module_init/module_exit
 
#include <linux/init.h>
 
#include <linux/kernel.h> // export basic kernel macros 
// like pr_info/printk, pr_debug

MODULE_DESCRIPTION("Simple module");
MODULE_AUTHOR("Kernel Hacker");
MODULE_LICENSE("GPL"); // Module license and code that can be used. For example: "GPL" unlocks GPL-only kernel symbols.

static int my_hello_init(void)
{
	pr_debug("Hello!\n");
	return 0;
}

static void hello_exit(void)
{
	pr_debug("Goodbye!\n");
}

module_init(my_hello_init); // Will invoke my_hello_init after loading the module

module_exit(hello_exit); // Will invoke hello_exit after unloading

And with a command like the following:

make -C /lib/modules/$(uname -r)/build M=$(pwd) modules

where

-C /lib/modules/$(uname -r)/build: kernel build system directory. Points to headers needed to build modules against the running kernel.
M=$(pwd): your module directory (where your Kbuild sits)
modules: target (we're building a module)

So after that we'll have a hello_mod.ko

Then:

load it into the kernel: insmod hello_mod.ko (need root privileges)
unload it from the kernel: rmmod hello_mod (.ko is not required when unloading)
to list loaded modules: lsmod

Something to remember, debugging in the kernel cannot be done like in our usual C code with a printf, these functions are not available at the kernel level. But we can read the debug info from the kernel log by using dmesg (display message).
You can read more about it on the course, internet and play with it.

Kernel API

Link

The tasks here are about some Kernel API to interact with...the kernel:

Memory allocation: kmalloc
Locking with mutex like mutex_lock , spinlock like spin_lock or read_lock (for locking when reading) and many more
Atomic methods like atomic_read to read atomic_t variables safely with no race condition
List: doubly linked list (yes the kernel has a dedicated list data structure)

It's the list api that I really like. Why ? The way to use it: a little ambiguous but meaningful.

List API

The list data structure of the kernel is a doubly linked list defined in include/linux/types.h and its methods are in include/linux/list.h .

What's interesting is that it's an independent structure that we can add as an attribute to our own structure to have all the capabilities it offers.
Let's assume you have a structure like

my_data

struct my_data {
    u8 access;
    char name[25];
    int num;
};

// and use it like
struct my_data data1;
struct my_data data2;
...

And you want to have a list of my_data, you just need to:

add the struct list_head as an attribute to my_data
Set a head that will be used to access your list

struct my_data {
    u8 access;
    char name[25];
    int num;

    struct list_head my_list; // (1)
};

LIST_HEAD(my_head); // (2)

Easy.

So as list_head has prev and next as attributes which are respectively pointers to the previous and next data (in our case data1, data2), we can easily access them through my_list.

But did you notice that there is no link between my_list and head 😅 ? and also the way we define the head ? 😅 . A little funny and confusing.

Yes at this point, adding list_head to my_data is like

we'll have a list of my_data and my_list attribute will be used as the corresponding node of the target data in the list

And by doing LIST_HEAD(my_head), LIST_HEAD is a macro that takes a name (here my_head) and internally sets it as a list head, so it's like

my_head will be the head of a list (of type struct list_head of course) and set its prev to itself

So at this point, no link between the declarations.

To now connect both or start using your list, you need to use the provided methods that work on the list: we'll see just 3 (there are plenty of them). Some of them are macros, others are regular functions.

Add element: list_add

To add an element to the list we use

list_add(struct list_head *new, struct list_head *head)

so something like

list_add(&data1.my_list, &head);

Access element: list_entry, list_for_each

we can use the provided macro

list_for_each(pos, head);

that iterates over the list where:

pos: is a pointer to a struct list_head used as a cursor to allow stepping from one element of the node to another
head: is a pointer to the head

Internally it's the usual loop we write to iterate over a list, here is the code of this macro in include/linux/list.h

/**
 * list_for_each	-	iterate over a list
 * @pos:	the &struct list_head to use as a loop cursor.
 * @head:	the head for your list.
 */

#define list_for_each(pos, head) \
	for (pos = (head)->next; pos != (head); pos = pos->next)

Then inside this macro we'll use

list_entry(ptr, type, member); // return a pointer to struct my_data

That returns the entry at each iteration.
where

ptr: is the cursor defined for list_for_each
type: literally the type of struct (struct my_data)
member: the list_head attribute added to the struct

So it's used like this

struct list_head *cursor;
struct my_data *temp;

list_for_each(cursor, &head) {
	temp = list_entry(cursor, struct my_data, my_list);
	pr_info("(%d, %s) ", temp->access, ti->name);
}

It can be a little confusing but by doing the various labs you can start to feel it.

There are more macros to interact with the list that you can find in include/linux/list.h

Assignments 0: Kernel API

Link

Github Link to my solution.

The assignment is about the use of the Kernel API. Here's a short version of the tasks:

Create a list.ko kernel module that uses the Linux kernel list API to store strings.
Expose a /proc/list/management write-only interface to process addf, adde, delf, and dela commands like echo "addf name" > /proc/list/management).
Allow reading the list contents line-by-line through the read-only /proc/list/preview file through cat /proc/list/preview
A skeleton is already provided for the setup of preview and management proc files. This part of the code is worth reading.
A test check/_checker is also provided so we can check if our implementation respects the constraints.

The work is to parse the write commands and implement for each of the four of them a function to handle the operation. In my case I implemented:

list_proc_addf for addf: Github Link
list_proc_delf for delf: Github Link
and do the same for the remaining 2 commands
Then the function to allocate memory for the string and set its value: Github Link

If you do the lab on kernel api, it'll be easy.

Here is

Few things we need to keep in mind or understand from this assignment:

Don't forget to purge (free) the memory you allocate in the module before the exit. In my case I implemented list_proc_purge that I called inside the list_exit which is used as the module exit entry point module_exit(when we remove the module with rmmod). Otherwise there will be a memory leak in kernel memory.
Don't forget to use copy_from_user and copy_to_user that respectively copy data (buffer) from user space to kernel space and copy data from kernel space to user space (for security reasons...)
We can use seq_* methods like seq_puts, seq_printf and many more to output data in a /proc file. In our case by doing cat /proc/list/preview. These functions are better for interacting with /proc than printk which outputs data in the kernel log (dmesg)
Look at include/linux/proc_fs.h and fs/proc/generic.c to have a list of functions to interact with /proc, like creating a directory, sub files...

Next weeks

The next weeks are about character device drivers, which are kernel modules that allow access to hardware devices, device files, I/O access, interrupts.

By the way, here's my GitHub repo for the assignments: kernel-labs-OS-2

Figure 1: Kernel API assignment tests passed.

Figure 2: addf and adde commands and /proc/list/preview showcase

Figure 3: delf and dela showcase

Linux Kernel Labs: Kernel Module and Kernel API

Comments

Linux Kernel Labs - 5.10.14

More from this blog

OS Design and Implementation: Recap

OS Design and Implementation: (Week 8-9) - Scheduling; File system

OS Design and Implementation: (Week 6 - 7) - Interrupts & Device Driver; Locks

OS Design and Implementation: (Week 5) - Page fault

Lectures

LABS

Kernel Modules

Kernel API

List API

Add element: list_add

Access element: list_entry, list_for_each

Assignments 0: Kernel API

Next weeks

Command Palette

Comments

Linux Kernel Labs - 5.10.14

More from this blog

Lectures

LABS

Kernel Modules

Kernel API

List API

Add element: list_add

Access element: list_entry, list_for_each

Assignments 0: Kernel API

Next weeks