Chapter 6 and 7 are about Interrupts, device driver and locks.

Interrupts and Device Driver

A driver is the code in an operating system that manages a particular device (network card, mouse, screen...): it configures the device hardware, tells the device to perform operations, handles the resulting interrupts, and interacts with processes using the device

An interrupt is a type of trap that allows a device to communicate with the operating system.

Here are some details about interrupts and device drivers:

• The CPU does not directly control I/O devices; instead, it communicates with a microcontroller installed on the devices via a control bus.

• Memory-mapped I/O is a communication technique where each device is allocated a specific memory space in RAM, allowing the CPU to read or write data.

• Polling, or programmed I/O, is another technique where the OS periodically checks if there is data to read from the device.

• Interrupt-based I/O is akin to a trap, as discussed in OS Design and Implementation (Week 4) - Trap.

Locks

Sure! Everyone is familiar with locks: they enable the OS to safeguard shared resources during use.In this chapter, we explore the internals of locks. Although I don't grasp everything, I gain new insights about them.

Labs

There are three labs where we were tasked with building a network driver and adding a custom lock mechanism to xv6. I completed the first one:

• [Device Drivers] Network Interface Card (e1000) driver: Implement e1000_transmit and e1000_recv in kernel/e1000.c to manage packet flow by manipulating the hardware's transmit and receive descriptor rings via DMA and control registers. You must successfully pass the "txone" and "rxone" tests by correctly handling buffer allocation, hardware status checks, and synchronization. Completed

• [Device Drivers] UDP Receive: Implement ip_rx, sys_recv, and sys_bind in kernel/net.c to enable UDP packet reception by filtering incoming IP traffic, queuing up to 16 packets per bound port, and delivering them to user processes. You must handle the transition between network and host byte orders using ntohs/ntohl and ensure the system correctly manages packet buffers to pass all nettest verification suites. I implemented ip_rx, sys_bind, i didn't complete the sys_recv.

• [Locks] Read-Write Lock: You must implement a read-write spinlock in xv6 that allows multiple concurrent readers but ensures exclusive access for a single writer. To prevent writer starvation, your lock must implement a priority scheme where any pending writer blocks new readers from acquiring the lock until the writer has finished.

Next week

Next weeks is about Scheduling.

By the way, here's my GitHub repo for the lab course: https://github.com/TawalMc/MIT-6.1810Fall2025-xv6

I did it last year: Dev Retro 2024, so I’ll do it again.

Let's thank God for keeping us healthy and alive this year.

Year of Decisions

This year was mainly about making decisions. What should I study? What should I do next? What am I interested in? What can I do, and what do I want?
If you've read my Year In Review 2024 (called Dev Retro 2024), you'll know that I set some goals based on what I knew at that time. Well, I changed some of them this year.

So, I'll talk about my daily job, what I did in my free time, and the decisions I made.

At work

Compared to 2024, not much has changed. Just take a look at the section At work from my Year in Review of 2024.

I'm still a backend developer working on some services at Uptimise, a startup I joined in 2022. We officially launched the product this year, and it's been great working with everyone. This year, I didn't do any major refactoring or big projects. We focused on making the two versions, Uptimise Benin and Uptimise Ivory Coast, more stable. There's still work to do, but we're making progress.

Now, let's talk about my personal projects and what I did in my free time..

Networking: Cisco Networking Basics

I'm not good in networking, and everyone knows it. So this year, I decided to change that and learn the basics of networking.
From June to July, I took the Cisco Networking Basics course. I worked on it during nights and weekends, and it went well. I really enjoyed it. I learned a lot and now feel less clueless when talking about networking at work, configuring my router, or handling network tasks on my computer..

I wrote a series about it here: Networking Basics

Operating System Engineering

It's no secret that I'm interested in system engineering and have a background in embedded systems. Currently, I'm a web (backend) developer, but I really want to switch to this part of the industry. Will I do it? I don't know. The only thing I know is that I want it, and I'll try to make it happen. Failure is part of the game, just like success, right?

After taking the Computer Architecture and Organization course (From Nand to Tetris) in 2024, I decided to move to the next step: Operating System Design and Implementation. The goal is to understand OS concepts and contribute to the Linux Kernel.

So, in November, I started the MIT Course on OS Fall 2025 (https://pdos.csail.mit.edu/6.1810/2025/schedule.html). It's 15 weeks of lectures with labs, books, and papers to complete. I'm currently in week 3. Yes, the lessons and labs are not easy to complete, at least for me, but I'm doing my best 😅.

Blogging

I restarted blogging in the middle of this year. My goal was to write regularly, aiming for 1 or 2 blog posts per month about my work. It didn't go exactly as planned, but it wasn't bad at all. This year, I wrote 9 articles on Networking and OS. These articles cover what I learned in those courses.

The good thing is that I wrote at my own pace: 1 or 2 articles per month, or more if I could. I wrote a little bit whenever I had time, like writing 10 lines at night or spending an hour on an article on Saturday. The idea is to make it a part of my daily routine..

Social Media post: LinkedIn and Twitter

Some of my friends, Trinité (my Lead), Fréjus (our Product Designer), Alade, and Cissé, advised me to start posting about my progress on LinkedIn. So, that's what I did. It helps me connect with people who share my interests. Mostly, I share summaries of the articles I wrote on these two networks.
My LinkedIn: Tawaliou ALAO
My Twitter: @TawalMc

Rejection for Master Application

In the section titled Rejection for Master Application in my Year In Review 2024, I explained that I was rejected for a Master's in Embedded Systems through Campus France and for Electrical and Computer Engineering (ECE) at CMU Rwanda. I decided to reapply for ECE this year. So, yes, I reapplied, and guess what? I got rejected again. Why? My application wasn't outstanding, so I guess that's the truth. Maybe I didn't do something right, or my background doesn't align well with the courses, or my path isn't impressive. Anyway, I don't care anymore 😂. I applied to a few other universities, like King Abdullah University of Science and Technology, and got rejected too 😂..

And some problems arise: Should I apply for Campus France (Licence 3) and do everything to leave the country 😂? Or should I stay and pursue a master's degree here? Should I even pursue a master's degree at all?
You know, when you wake up with these kinds of thoughts and questions in your mind, it's hard to stay focused. The combination of the unknown, fear of bad choices, and failures increases indecision.
So I did something I often do in situations like this: I asked for advice from elders. It's the same thing: don't fear failure, don't fear making bad choices, fear not making a choice at all..

So I made two decisions:

• Stay. No more applications, at least for a master's abroad.

• Go all-in on system programming. Put all my effort into this field. That's why, in the sections above, I mentioned that I started the OS Engineering course by MIT..

Is this a good choice? Will it pay off? I don’t know.
Right now, I mostly care about the process rather than the results.
There are also many reasons behind my decision to focus on System Programming. In a future article, I’ll discuss some of these reasons that motivate me to choose this path..

To give you a hint, here are two of them::

• I currently need time, I need to have full control over my free time and the freedom to study system programming.

• Try hard and take risks to achieve it. This article by Aaron Francis: What if you tried hard explained it

I wonder what would happen if you gave it all you had and even risked embarrassing yourself if it didn't work out.

Aaron Francis (https://aaronfrancis.com/2024/what-if-you-tried-hard-dac139a5)

What for the next (2026, 2027, …) ?

To be honest, as usual, I don't have clear, well-defined plans that I begin at the start of the year. No, nothing like that. I just have some important goals. I usually start working in this direction from October (3 months before the new year). This helps me adjust based on how things unfold.

So for the next few years:

• Be consistent in blogging, aiming for at least 1 or 2 articles per month

• Get involved in open source projects (System Programming)

• Linux Kernel

• Read at least 6 papers on Operating System available on:

  • Operating System Design and Implementation (OSDI) by USENIX

Last Words

A few things to keep in mind:

• Things don’t always go as planned, so it's good to be flexible.

• Try hard at whatever interests you. At least give it a shot.

• Be patient, seriously.

• Avoid comparisons, as they only create frustration and prevent you from focusing on yourself.

• Make choices, even if they might seem silly. Make decisions based on what you know, want, care about, and fear. Don’t hold back when you need to decide.

• Sometimes, or often, life isn't easy, and that's just how it is.

• And have hope.

There are many people I want to think about this year, from my parents, my closest friends, and my girlfriend to my colleagues and elders. But there will be a dedicated blog post for that in the future (I hope).

Share your Year in Review with me. It inspires me..

This chapter covers the organization of the operating system:

• There is a separation between the kernel and user space: user space code does not directly call kernel space code.

• Processes are isolated, with each process running as if it has its own dedicated CPU and virtual memory, which is its own memory space (it cannot access memory outside this space without requesting more).

• The underlying architecture (CPU) provides a system call mechanism (RISC-V uses the ecall instruction) to switch from user code to kernel code.

• The kernel performs checks when a system call is triggered to ensure the operation is allowed and the correct arguments are passed.

• Every system call, like read for example, is assigned a number (the read system call number in xv6 is 5) that the kernel uses to identify which system call the user code triggered.

• And more... I'll explain "System Call" in more detail in my next blog post.

And the labs? I spent most of my time trying to understand the assignments and figuring out where to start. Here are the labs:

• Implement a system call called “interpose” that limits the use of system calls based on a bitmask passed as an argument to interpose.

• Exploit a vulnerability in xv6 that lets a process read data from another process's virtual memory. This lab helps illustrate why kernel code must be well-written.

Currently, I'm getting hands-on experience with GDB for more effective debugging of the C and kernel code.

Next week, I'll start Week 3: Page Tables.

So… here’s an update on my learning path.

2023 – 2024: Building a Modern Computer from First Principles (aka Nand2Tetris). I studied the fundamentals of computer architecture; from logic gates and the CPU to the operating system and compiler.

Mid 2025: Introduction to Networking Basics.

Late 2025 – :

I want to learn more about OS internals, especially the Linux kernel and similar topics. So, some days ago, I started the MIT 6.1810 course on Design and Implementation of Operating Systems.

The course is based on xv6, a simple Unix-like teaching operating system. The goal is to study OS concepts by extending xv6.

What’s good about this course:

• It’s well structured and has a clear schedule. (This year’s session started in September 2025 and is expected to end in December 2025.)

• It combines theory with practical labs, so you don’t just read; you build as well.

I’ll spend some nights and weekends working on it, enjoy the process, and share updates about my progress here.

I’m currently working on the Week 1 assignment (Lab: Utilities), where we’re implementing Unix-like utilities using the xv6 API.

By the way, MIT offers really well-structured courses on various topics in Computer Science.

MIT 61.810 Fall 2025 (Operating System Engineering) link:[https://pdos.csail.mit.edu/6.1810/2025/overview.html]

Update [07 November 2025]

Lectures

I just finished week 1 of the course. This week, I read Chapter 1: Operating System Interface of the book on xv6. This chapter introduces:

• The xv6 system calls, such as open, which "opens" a file by returning a file descriptor, and fork, which creates a process.

• How processes behave, including how to create a child process and execute a command.

• Importantly, what a file descriptor is (an entry in a table for each process that lists files opened by the process), which allows a process to read or write.

• A method for processes to communicate (Inter-Process Communication): Pipes, which use file descriptors.

I also read the chapters on Pointers (Arrays, Characters, Structs) from The C Programming Language (Second Edition) by Kernighan and Ritchie (K&R) to refresh key concepts about manipulating pointers.

Then there are the assignments or labs.

Lab Assignments: Unix utilities

The labs focused on writing Unix-like utilities. Here are a few, along with brief descriptions::

• sixfive (not so easy): For each input file, sixfive must print all the numbers in the file that are multiples of 5 or 6. Number are a sequence of decimal digits separated by characters in the string " -\r\t\n./,".

• memdump (not so easy): Implement the function memdump(char fmt, char data). memdump()'s purpose is to print the contents of the memory pointed to by data in the format described by the fmt argument.

• find (not easy): Write a simple version of the UNIX find program for xv6: find all the files in a directory tree with a specific name.

• pingpong (easy and from Fall 2020): Write a program that uses UNIX system calls to ping-pong a byte between two processes over a pair of pipes, one for each direction

These labs were not too difficult and had clear instructions, but I often felt stuck and unsure of what to do. However, after reading chapter 1 of the xv6 book multiple times and reviewing the source code of some existing utilities, I completed them. Here are a few things to remember (and I'm not ready to forget them)::

1. When creating a child with the fork system call, don't forget to use wait in the parent. This ensures that the parent waits for the child to finish executing. This way, both the parent and child code will run properly.

•   if (fork() == 0) // Child
    {
        write(1, "hello ", 6);
        exit(0);
    }
    else // Parent
    {
        wait(0); // Don't forget to wait here so child can execute
        write(1, "world\n", 6);
    }
  

  Forgetting this, make me stuck on the pingpong task.

2. When we define an array of int and another of char, like int tab_1[5] and char tab_2[5], we know that tab_1 and tab_2 are pointers to the first element ([0]) of the array. Also, tab_1[1] is equivalent to *(tab_1 + 1), and the same applies to tab_2[1], which is equivalent to *(tab_2 + 1)..

   • But if we assume that the memory address of tab_1 is 4000 and the address of tab_2 is 5000, what will be the address of tab_1[1] and tab_2[2]? The address of tab_1[1] will be 4000 + 4 bytes = 4004, not 4001, since sizeof(int) = 4 bytes on most modern 32-bit/64-bit computers..

     Why? Because tab_1 is a series of contiguous memory addresses that store integers, each with a size of 4 bytes (sizeof(int)). To move from one address or index to the next in the array, you need to step by 4 bytes. So, you move 4 bytes at a time. tab_1[0] occupies the memory blocks from 4000 to 4003, and tab_1[1] starts at 4004.

     The same rule applies to tab_2, but here sizeof(char) \= 1, so the address of tab_2[1] will be 5000 + 1 byte = 5001.

     The rule is simple: New Address = Start Address + (N * sizeof(Type))

   • But here's another question: when we use *(tab_1 + 1) or tab_1[1], how does the compiler know to add N * sizeof(Type) to the initial address? Well, this is done at compile time when the compiler generates the assembly code. It uses a table called the Symbol Table, which holds information about variables, functions, and other elements in our code. This allows it to check the data type and perform the appropriate calculations..

From Gemini

When the compiler reads your source code, it builds an internal data structure called the Symbol Table. This table tracks every identifier (variable, function, type, etc.) in your program and records all its key attributes, including its data type.

For a pointer variable declared as int *p; the Symbol Table records that p is a pointer to an int.

Done for now, see you next time!

It's good to write the utility and get back into C to remember these concepts before diving into OS.

I'm starting Week 2 of the course.

By the way, here's my GitHub repo for the lab course: https://github.com/TawalMc/MIT-6.1810Fall2025-xv6

So, I'll show you the C code first, followed by the Python code.

showip.c

#define _POSIX_C_SOURCE 200112L

#include <stdio.h>
#include <string.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <sys/socket.h>

int main(int argc, char const *argv[])
{
    int status;
    struct addrinfo hints, *res, *p;
    char ipstr[INET6_ADDRSTRLEN];

    if (argc != 2)
    {
        fprintf(stderr, "usage: showip hostname\n");
        return 1;
    }

    memset(&hints, 0, sizeof hints);
    hints.ai_family = AF_UNSPEC;
    hints.ai_socktype = SOCK_STREAM;

    if ((status = getaddrinfo(argv[1], "80", &hints, &res)) != 0)
    {
        fprintf(stderr, "gai error: %s\n", gai_strerror(status));
        return 2;
    }

    printf("IP addressess for %s: \n\n", argv[1]);

    for (p = res; p != NULL; p = p->ai_next)
    {
        void *addr;
        char *ipver;
        struct sockaddr_in *ipv4;
        struct sockaddr_in6 *ipv6;

        if (p->ai_family == AF_INET)
        {
            ipv4 = (struct sockaddr_in *)p->ai_addr;
            addr = &(ipv4->sin_addr);
            ipver = "IPv4";
        }
        else
        {
            ipv6 = (struct sockaddr_in6 *)p->ai_addr;
            addr = &(ipv6->sin6_addr);
            ipver = "IPv6";
        }

        inet_ntop(p->ai_family, addr, ipstr, sizeof ipstr);
        printf(" %s: %s\n", ipver, ipstr);
    }

    freeaddrinfo(res);
    return 0;
}

And now the showip.py

import socket
import sys
from socket import AddressFamily

FAMILY_TYPE = {"AF_INET": "IPv4", "AF_INET6": "IPv6"}
PORT = '80'

def main():
    host = sys.argv[1]
    try:
        host_info = socket.getaddrinfo(host,  # hostname
                                       PORT,  # hostname port
                                       socket.AF_UNSPEC,  # IPv4/IPv6
                                       socket.SOCK_STREAM # TCP,
                                       flags=socket.AI_CANONNAME
                                       )
        for info in host_info:
            family = AddressFamily(info[0]).name
            text = f"{FAMILY_TYPE[family]}: {info[4][0]} --> {info[3]}"
            print(text)
    except socket.gaierror:
        print(f"Hostname {host} could not be resolved.", file=sys.stderr)
        return

if __name__ == "__main__":
    if len(sys.argv) != 2:
        raise ValueError("usage: showip hostname")
    main()

When we run the Python code, we get this:

# [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, 'hashnode.com', ('66.33.60.194', 8081)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('76.76.21.22', 8081))]
$ IPv4: 66.33.60.194 -->  hashnode.com
  IPv4: 76.76.21.22 -->

We can see that even though the Python code is shorter, it uses a similar API to the C code. This is because Python simply wraps the socket API provided in C, which helps reduce some of the developer's workload. The key part is using getaddrinfo (get address info), which is defined like this::

int getaddrinfo(const char *node,
                const char *service,
                const struct addrinfo *hints,
                struct addrinfo **res);

socket.getaddrinfo(host, port, family=0, type=0, proto=0, flags=0)

# and returned value
# list if of tuples (family, type, proto, canonname, sockaddr)

You can already see the similarity.

In the C code, the result is stored in a linked list, and the head's address is passed to getaddrinfo. This is why we iterate over res and res->ai_next. In Python, the library takes care of this iteration, so we just go through the list of tuples until the end.

Why a list of IPs? Because a hostname can have multiple IPs (both IPv4 and IPv6) associated with it.

Good, right? Yeah, but I wondered how Python's getaddrinfo is implemented behind the scenes, so I asked ChatGPT, and it gave me this: https://chatgpt.com/s/t_68eba17bdb4481919241ce01a7890c5b

You can read it to get some ideas (I didn't understand everything).

Done for now, see you next time!

It's great to see the work Python has already done for developers and how things operate behind the scenes with C. I'm currently learning about network programming, so there might be a future article on sockets or similar topics.

I used to subscribe to a monthly unlimited internet package from a popular Mobile Network Operator in my country, which is also an ISP (Internet Service Provider). I won't mention the company's name, so I'll call it OPERATOR. In my opinion, it was expensive and slow. It cost 15,000 XOF, about $27 per month, and once you used 25GB, your internet speed dropped from 2MB to 1MB. After using the first 25GB, the rest of the month was a struggle: the speed was slow, and you could barely browse websites other than YouTube. Under these conditions, taking online courses was tough. So, after some recommendations from friends, I decided to switch to another Mobile Network Operator that offers fiber optic for faster internet.

TL;DR: I'm really satisfied right now. I hope it stays this way.

Now, here's the problem: the location where the fiber optic and main router were installed doesn't let my phone connect to the Wi-Fi when I'm in my room. The solution is to extend the network. The good news is that we have a second router at home. So, I need to connect the second router to the first one and place it near my room.

I did it, and it was easy. I just followed a few steps and asked chat.gpt if I had any doubts. I used the second router as a switch, so now I have one local network.

Let's name the router that comes with the optic fiber (the one provided by the ISP) MAIN_ROUTER, and the second one SECOND_ROUTER. Here are the steps and reasons:

• Get the MAIN_ROUTER's configuration info:

  • Access the admin page: The admin ID, password, and IP are on the back of the router.

  • Write down information like:

    • Gateway (router address): This will be used by a host as the Destination IP Address for transmission outside of the LAN.

    • DNS server: It's important to save the correct address; otherwise, even if you have an internet connection, you can't reach websites.

    • IP range, subnet mask.

• Connect the SECOND_ROUTER to the first one through the LAN port (usually orange or yellow).

• Configure the SECOND_ROUTER by:

  • Change its mode to Bridge or Access Point, or select an option other than acting as a Router if available.

  • Entering the same configuration info listed above on its admin page.

  • Disabling its DHCP Server to ensure that the MAIN_ROUTER is the only one assigning IP addresses to hosts.

  • Giving it a static address outside the IP range of the MAIN_ROUTER but in the same subnet. For example, if the IP range of MAIN_ROUTER is 192.168.1.100 - 192.168.1.254 (and the subnet is 192.168.1.1/24), you can set the SECOND_ROUTER IP address to 192.168.1.2.68.1.2.

It's not really that complicated, but I like it.

I started the course not only because I lacked knowledge in networking but also because I believe networking is an important part of the IT field. Now that I have completed the course, I am more comfortable with terms like subnet, broadcast domain, DNS resolver, and others when reading articles or documentation for tools I use daily.

Here are some things I appreciated about the course:

• Well-written chapters with clear use cases and illustrations

• Short videos with good explanations and visuals, each lasting a maximum of 8 minutes (I prefer text over video when studying)

• Practice chapters using CISCO Packet Tracer

• Short and long exams after each chapter, featuring multiple-choice questions to assess learners' knowledge

I really appreciate the course's structure and highly recommend it.

What's next? I'm continuing with network programming because there are still things I need and want to learn in this area. I started with Beej's networking guide.

To illustrate this, let's use this network schema as an example.

Transmission inside a LAN

Let’s say Host 1 wants to send data or packets to Host 2. Both hosts are in the same LAN. There are two possible scenarios: either Host 1 already knows Host 2's MAC address, or Host 1 only knows Host 2's IP address but not the MAC address.

MAC Addr is known

In this case, Host 1's ARP table already contains the IP and MAC addresses of Host 2, making the process straightforward.

In fact:

1. Host 1 will put its IP and MAC addresses in the source address sections, then use Host 2's IP and MAC addresses as the destination and send the packet to switch S1.

2. At switch S1, after it de-encapsulates the frame, it will check its MAC Address Table to see if there is a port associated with the destination MAC. If yes, it will forward the frame to this port. If not, it will broadcast the message to all ports except the sending port. In this last case, the frame will be processed only by the host with the corresponding MAC address; others will discard it.

You can get more information about this case in this article I wrote on Ethernet and Switch.

MAC Addr is not known

Host 1's ARP table doesn’t contain the IP and MAC addresses of Host 2. Therefore, before sending the packets, Host 1 will try to discover the MAC address of Host 2 with an ARP request. ARP stands for Address Resolution Protocol. So, what happens is an intermediate step where:

1. Host 1 will send a Broadcast MAC addr frame with the IP address of Host 2.

2. Switch S1 will forward it to all ports (except the sending one), and each of the hosts connected to the switch will compare the IP addr in the frame to their own IP.

3. The corresponding host (Host 2) will be the only one to reply and will send back its MAC addr.

4. Host 1 will then store the MAC addr in its ARP Table and will set the destination MAC addr of the frame with this new value.

5. Then the packets will be sent following the steps in the case of MAC Addr is known (above).

Note: Routers do not forward broadcast domain packets, so Host A and B will not receive these packets.

Here is the Ethernet Frame that will be sent (this is a simplified version)

Following text summarise what’s said in this paragraph.

The method that a host uses to send messages to a destination on a remote network differs from the way a host sends messages on the same local network. When a host needs to send a message to another host located on the same network, it will forward the message directly. A host will use ARP to discover the MAC address of the destination host. The IPv4 packet contains the destination IPv4 address and encapsulates the packet into a frame containing the MAC address of the destination and forwards it out.

from cisco

Transmission outside a LAN

The transmission of a packet outside a LAN is done at Layer 2 of the TCP/IP model; the Internet Layer. So we need the IP of the destination host. At this layer, we necessarily need a Router to route the traffic (transmit the packet) outside the LAN. So what happens?

Let's suppose that Host 1 wants to send a packet to Host B.

• The Destination MAC addr will be the Default Gateway MAC addr in the LAN: XXXX, then

• The Router will:

  • get the frame,

  • de-encapsulate it,

  • evaluate the best path by which to send the packet,

  • replace the DEST MAC Addr with the next Router MAC Addr,

  • replace the SRC MAC Addr with its MAC Addr in the new LAN: QQQQ,

  • send the packet.

  • The next router does the same thing, but if it’s the last router and the packet is already in the DEST LAN, it puts its MAC Addr in the LAN as SRC MAC addr.

As the last router and the destination host are in the same LAN, the steps in “Transmission inside a LAN” will be executed to either get and set the destination MAC addr of the packet or send the packet if the destination MAC addr is already known.

The following text summarizes what's said in this paragraph.

When a host needs to send a message to a remote network, it must use the router. The host includes the IP address of the destination host within the packet just like before. However, when it encapsulates the packet into a frame, it uses the MAC address of the router as the destination for the frame. In this way, the router will receive and accept the frame based on the MAC address.

from cisco

The picture below shows changes in the packet sent from a LAN to another.

Magic, right? I really like how networking is designed, and I wonder how the field developed like that.

DHCP

DHCP stands for Dynamic Host Configuration Protocol and allows hosts to be dynamically assigned IP addresses (IPv4/IPv6). So, you’re not required to know the range of addresses available for hosts in the network, and even if the host restarts, it will automatically get an IP. To have DHCP configured on the host, you need a DHCP Server from which the host will get its IP dynamically. The following are the steps that occur when DHCP is configured (in four steps):

1. Host (which is a DHCP Client) sends a broadcast (255.255.255.255) message «DHCP DISCOVER» to the DHCP server asking for an IP.

2. The DHCP server responds back with a unicast or broadcast message, a «DHCP OFFER» that contains an IP + Subnet mask (like 192.168.11.2 + 255.255.255.0) for the new host.

3. The host responds with a broadcast message, a «DHCP REQUEST» asking for confirmation to use the IP sent.

4. DHCP confirms with an ACKNOWLEDGMENT, a unicast/broadcast message «DHCP ACK» sent back that confirms to the host that it can use the IP sent above.

Next, here's a diagram that shows the flow when a DHCP Client request an IP

Two additional interesting facts:

• Your Router (modem) can act like a DHCP Server and provide IPs dynamically to hosts that are connected to it.

• Normally, you can manually configure your devices (phone, computer) to request IPs dynamically, but usually these devices are set by default to use DHCP to automatically request an IP.

Now, if you want to implement the DHCP Protocol in C/Linux yourself, you can follow this post, Understanding DHCP with Socket Functions by David Zhu (LinkedIn), who publishes good and in-depth content on Networking.

So briefly, this is what happens when our phone/computer connects to Wi-Fi and automatically gets an IP address without us needing to do any configurations.

Now, let’s talk a little bit about routers.

Router

Let’s list some properties of a Router:

• A router allows communication between different LANs or with a remote network. So a host in the LAN 192.168.2.5/24 can send a message to a host in the LAN 192.168.3.4/24.

• A router can act as a DHCP Server for hosts in a LAN and be a DHCP Client within an Internet Service Provider (ISP) to get an IP for internet communication.

• The Default Gateway is the router’s IP in a LAN. So each LAN has its Default Gateway within the router.

• Has an ARP (Address Resolution Protocol) Table: {MAC Addr, IPv4 Addr}

• Has a Routing table: {Port, Network IPv4 Addr, Type of connection} and best paths to reach a network

Here is an example of a router with its address.

That is it.

There is a lot to say about Router and DHCP, but I think now we can understand a little bit about them with this information. I am open to any suggestions regarding improvements to what is said in this article.

And in the next article, I’ll write about: How a host sends data to a host in the same LAN and in a remote LAN (yes, we’ll talk about Router again).

Ethernet switch

Ethernet Frame

The Ethernet switch, or simply switch, lets devices connect within a LAN. Its job is to forward or transmit packets (also known as Ethernet Frames) from a source device to a destination device using the MAC address of the Network Interface Card (NIC) in each device. A packet is just a sequence of bits (0s and 1s).

In the image below, you can see an example of a packet.

So, we might ask, in a LAN, how does the switch know the device's MAC address to determine where to forward a packet? Let's look at this picture of a simple LAN.

My initial thought was that when a device connects to the LAN, it sends its NIC's MAC address to the switch, like a registration. This way, the switch would have the MAC addresses of all devices connected to the LAN, and when it receives a packet, it could use the destination MAC address in the packet to send it to the correct device.

But that's not exactly how it works: In reality, the NIC in devices doesn't know if it's connected to a switch, so it doesn't need to send its MAC address. Also, the protocol must be designed to work independently of specific devices like the switch..

MAC Address Table

The switch has a MAC address table where it stores device MAC addresses along with the port each device is connected to.

When a packet is sent:

1. The switch first checks if the Source Address is already in its MAC address table.

2. If it is, and the Destination Address is in the table, it forwards the packet to the port where the destination device is connected. If the Destination Address is not in the table, it broadcasts the packet to all ports except the source port.

3. If the Source Address is not in the table, the switch adds it with the corresponding port, then proceeds as in step 2, either broadcasting or directly forwarding the packet.

4. When a device connected to the switch receives the packet, it processes the packet if the Destination Address matches its MAC address; otherwise, it discards the packet.

Here is a short illustration of this process: that.

As we can see, it's a "simple" method to send packets and save MAC addresses simultaneously, using fewer resources. The table updates each time a new packet is sent.

Let me share another thing I learned about IPv4..

IPv4 (Network + Host)

In IPv4, there are Network and Host parts. The network part, also known as the Subnet, indicates which network the device is part of, while the host part identifies the specific device. Refer to the following image.

Here are a few things to understand:

• When we have an IPv4 address 192.168.5.123/24, the 24 indicates that the first 24 bits (192.168.5.0) represent the network address.

• The subnet mask, 255.255.255.0, serves the same purpose as the 24: it identifies the network. So, if we have these two pieces of information: 192.168.5.123 and 255.255.255.0, it means that 192.168.5.0 is the network address.

To conclude, there can be a broadcast address like 192.168.5.255, which acts as a sub-group for devices with the same address. If a packet is sent to this address 192.168.5.255, all devices with this broadcast address will receive the packet.

Now, on Linux, when we run ifconfig -a and see a display like the following, we understand what the second line means.

Conclusion

There are many things I read, use, and see, but I don’t really understand their meanings. It's good to have a little bit of knowledge about them, like the whole world of networking.

Why ?

When I was working at YoupiLab as embedded software developer, my CEO told me multiple times that I need to learn networking basics that its a great part in IT. Now that I’m working as backend developer at Uptimise, my Lead told me that if I want to advance in the field I need to be comfortable with Networking.

In fact, I don’t know too much about networking besides the fact that I can access my React application locally on localhost (127.0.0.1) and that generally, I need the IP address + Port to access an application, like 127.0.0.1:3000 for a React app 😅 (shame).

As I was interested in system programming, I thought, why not learn networking programming (system) at the same time? So I started this month with the Networking Basics by Cisco Academy.

First Steps

The course seems good, with around 17 chapters, and the first 4 are about generalities (I’m at chapter 5). It provides good information to conceptualize what networking is, with the support of Cisco Packet Tracer to design networks.

These insights made me realize some things I did in the past but didn’t really understand because I searched “How to” on Google. It’s not bad, but now, I’m gradually understanding why some things are done in certain ways.

For example:

• On a router (even a wireless router), there is a built-in switch that allows devices to connect to the local network (LAN).

• The ports usually colored in yellow or orange on a router are for LAN, and the usually blue port (one) is for the internet or Wide Area Network (WAN). That’s why when we want a device to be wired to the router, we connect it to the yellow ports through these (you know) cables with multiple colors.

Basics right ? Yes, that’s why the course is named so.

There is an analogy that I get at this point that helps me remember all of these terms (IP address, LAN, WLAN, Gateway, Subnet Mask, Router, ISP, …):

The IP addresses for the end devices can range from 192.168.0.2 - 192.168.0.254. Each device will get a unique IP address in the same network.

The subnet mask is used to differentiate the host and the network ID portion of the IP address. You can relate the IP address to your street address. The subnet mask defines the length of the street name. The network part of the address is your street, 192.168.0. The house number is the host port of the IP address. For the IP address 192.168.0.2, the house number is 2 and the street is 192.168.0. If there is more than one house on the same street, for example, house number 3, will have an address 192.168.0.3. The maximum number of houses on this street is 253, ranging from 2 to 254.

The default gateway is analogous to the street intersection. The traffic from the 192.168.0 street has to exit through the intersection to another street. Another street is another network. In this network, default gateway is the wireless router that directs the traffic from the local network to the cable modem, and the traffic is then sent to the ISP.

From Networking Basics by CISCO Academy.

I understand that the default gateway that we read when doing Router config is the Router IP address, simple !

Conclusion

The first 4 chapters may seem boring because the general info is already known, but from chapter 5, it starts to get really technical. As Linux networking is built on top of this knowledge, I hope at the end of the course to be able to understand more about Linux networking.

February 2025, I built a Linux utility in Go, learnt about how container are built (in Linux environment) and planning to learn about Linux Driver (Kernel Module). Here are some details.

Sys Prog, Coding Challenges, Go

I try Build your Own wc tool by The Coding challenges in Go. The challenge is to rewrite in the langage of your choice from scratch the word counter utility found in Linux environment. And you have the freedom to expand the features. wc prints the number of lines, words, characters, bytes of file(s). You can read more details in the links I paste above.

In my case, I built it, and add some sugar features:

• handle multiple files: I use goroutines so the process of multiple files with the same size will took approximatively the amount of time of on file of the same size (good !)

• nice output: in my case the output is like

 l: 111, w:100, m: 200, c: 200 text.txt

• I’ll add(I forget this feature) to get argument from standard input and not only from files.

While doing this project, I learnt more about

• unicodes, characters sets and how Go handle them here: Go characters are known as Rune(😏)

• newline vs number of lines

Container: under the hood

I try to understand how containers like Docker are made and try to build a container from scratch. And even if I didn’t go to too far, I have learnt many things. And Yes: Linux is awesome, incredible as OS.

A container in Linux env, is based on: namespaces and cgroups. Namespaces help to have process in isolation and cgroups to put limit on ressources (memory, cpu, storage...) used by process

Here are the namespaces under linux:

• Mount (mnt): for mount points and provide a complete filesystems to the process which is now different of the host filesystems. That’s why you’ve a complete filesystems (root, dev, home with access) when you’re inside a container.

• Process ID (pid): a parent process with an ID can see other process but its children only see process from the same parent. That’s why when you’re inside a container you don’t get PID from applications/process outside the container.

• Network (net): this namespace virtualize the network stack. You remember Docker Network ? Yeah this is the namespace that provide this capabilities: having your own network isolation inside on the host

• User ID (user): This namespace provide you everything related to user and their management: access/privilege, groups. You remember when creating a Docker image you can specify the user with USER command and create group ?

• Inter-process Communication (System V IPC)

• UTS (Unix Time-Sharing, hostname): with this you have can hostname, DNS for specific process

Here are some details about Linux Namespaces and Linux cgroups.

Liz Rice give many talks on how to build a simple and lightweight container system from scratch in Go

I don’t know but, I think it’s the main reason why Docker is not avalable on Windows without WSL.

Right, hope you learn something from this articles, I’ll be back for sure.

Sys Prog

I was lazy to read more about process and how executable are running in Unix environment, let’s hope that until the end of the month I do it

Go

I decided to learn Go this year. And I found a good website to learn it: Learn Go with tests. It teaches it by using a Test Driven Development (TDD) approach and I like it. I could read the syntax, start quickly with some projects on CodingChallenges but I choose to learn with this technique and after I’ll do the codingchallenges projects. So now I did:

• Hello world, an introduction to TDD, Go module, how we could structure a Go projects and integers chapters.

• The loop chapter. Go has just one loop statement: the for-loop that can be used in different way

• Array and slices: array a static list with the size encoded in the type and slice a dynamic version of array

DSA-LeetCode

• Some exercises on LeetCode based on patterns: Sliding Window

Django Auth + Permissions

Here a start a toy project to deep dive into Django auth and permissions like:

• I’m using: Django+Django REST Framework+Django Allauth+Django Rest Auth

• having two apps with two user’s models for the authentication: I finally use two models for each kind of user and they inherited from a main user with few attributes which is used in the authentication flow.

• I’ll write some custom permissions and tests some stuffs

• I’m also interested to try an IAM like Authelia(no keycloak is huge we’re using it in my work)

System Design

I started to read about sys design: lazy to work on syst design. I think I like read more about syst prog and kernel (lol)

I’ll try in this article to talk a little bit about what happens to me inthis year (2024), my failures, my success, what I learn, what I don’t still understand. Why Dev Retro ? In 2023 I competed for Hashnode DevRetro 2022 and it was the name of the competition, and I like it, so

Introduction

Ok, so first of all, I’m trying to restart with my blog. I started blogging in 2020-2021 but I was not regular so I’m planning to restart and write a monthly update about my work (mostly). I think this will help me to track my progress so I’ll write more for me and on my studies and works monthly.

2024 ? I was (and I’m still) still looking for answers to many questions ! It was a nice year, many struggles, a lot of works, a lot of reflexions. Mostly around my work as backend developper at Uptimsie (even if I was doing fullstack tasks 😅) and my personal work: online courses.

At work

So at work, it went well (I think). We’ve started to port and update Uptimise Benin codebase to Ivory-Coast. I suppose that it’s what we call “scale” right ? Anyway, we need to change some parts of our services to handle in the way Ivory-Coast manage human ressources. We are talking about: operations, default setup, flow and so on. So we need everything related to the technique to work well: API, services, CI/CD. The CI/CD part you know and I know that it’s not me (😅). Our Lead did it. Like always !

And guess what ? I was appointed to lead the development of Uptimise Ivory-Coast. I was not hyped, it’s good, but at the beginning I was not so interested: you’ll be sometimes the first that the CEO will call if there is problem. Can you imagine ? A lot of responsabilities ! And this is what our Lead did so far.

But we know that it’s a normal thing, learn to lead some colleagues, be the one that a bug in production is assigned to, to resolve it asap. It’s normal for a career evolution ! And guess what again, I finally liked it 😂. You know why: I could assign an issue to our interns or delegate some tasks that are important but could boring me to them 😂. I know it’s malicious but I love it.

Fun Fact: Our Lead gave me access to some environments/servers and the first time I run our CI/CD after some updates, I broke things in production. I was like: “Damn, why this happen the first time I got the access, am I so bad !”

After May I think, I started to get tired, there was a lot of work to do for a small team like ours. And as I was responsible of some microservices when we building the Benin’s version, I’ve asked our Lead to do some refactoring in thes services for Ivory-Coast version. Don’t ask me why 😂. Sometimes I make a “git blame” and was shocked that I wrote some parts of the running code. I was like: “How could someone wrote this “shit” code" and the lead will look at you like (I’m sure he thinks something like): “are serious, you don’t know that your shit code is every where”. Let’s praise the Clean Code Shit Code.

I finally did the refactoring for a part of one service, I love it. And it’s one of the reason (personally) I dont like freelancing or projects that you deliver to a client after few months and forget about it. I like to see the evolution of my own code, learn to improve it. If I cannot understand my own code after some months I suppose others could not too.

And now what I’ve learned from my work:

• Good comunication is the key: inside the teams, with those who are outside (CEO, product designer)

• I don’t like to talk with the clients, be in discussion with them, maybe it’ll change but (you’ll learn from the next section) I prefer to stay in background do my job, get paid and it’s all. Everything related to “meeting”, “talk”, “long discussion” just annoyed me.

• It’s good to have some elders that advise you on how to do some stuffs.

• And there is so much to learn as a backend developer that sometimes I doubt that I could handle it.

• Learn as I much you could, as much you and leave the rest in the hand of…God !

• Progress at your own pace. You are only competing with yourself and your past versions.

Personal

I love personal plans, personal works, why ? Because it’s the only time that you’ve for yourself to do dumb, cool, shit, things ! But if you don’t find a good balance with your work, you could loose everywhere.

Backend dev is great but…

I’m interested by learning low level stuffs too. Not “Embedded System” particularly because what I love is not to design the electrical circuit or build a hardware. But I’m more interested with the code that run on these hardware. It’s technically called: “Embedded software” or “System Programming”. You know the topics like: process and memory management, file system, devices drivers. I finally understood it after studying the Nand To Tetris course.

I was able to study from “October 2023” to “July 2024” (10 months) a course about these thing even if I’m not sure that one day I’ll need it.

So now let’s talk in depth about my personal work in 2024

Online Low Level Courses

Online Low Level Courses (it’s a fake study) that I’ve imagined for my self to study system programming.

It contains:

From Nand To Tetris: Building a Modern Computer From First Principles

From Nand to Tetris: Building a Modern Computer From First Principles is a populare online course available on Coursera that I studied from October 2023 to July 2024. It’s in two parts:

• Hardware: Logic gate, CPU, RAM, ALU and Registers implementations in HDL, Assembly Language,

• Software: Jack programming language, Virtual Machine, Compiler, OS and their respective implementations

I enjoy study this course during my weekends and the nights. I was able to complete it (without the OS part) even if I was tired from daily tasks at work.

After this course I’ve two ooptions: OS concepts and compiler. I choose OS because I think it could help me to understand more backend development and become one day may be “Infrastructure Engineer” or “Platform Engineer”.

If you want to know more about my experience while studying Nand2tetris, I wrote a more detailed thread about it on Twitter

Advanced Embedded Linux Development Specialization by CU Boulder

I’ve started to study the course Linux System Programming and Introduction to Buildroot which is the first part of Advanced Embedded Linux Development Specialization. This course introduce me to the linux kernel source code and the linux system programming: fork, execv, thread, kernel cross-platform build, …
They provide short and well explained videos with assignment to complete. In addition they are two books as ressources to read: “The Linux System Programming” and “Mastering Embedded Linux Programming”.

Rejection for master application

I applied for two masters and was rejected by both (I was shocked):

• embedded system in five universities in France.

• electrical and computer engineering at CMU.

I re-apply again for the last one and now I don’t know even if i’m accepted I’ll go to do it.

Why ? Now that I understand that I could learn system programming by myself and that it’s what I like in embedded system field and that backend dev is cool too, I was less attracted by the master.

Next Year(s)

You kow backend development is paying my bills and until I was able to land a job in system programming I’ll try to improve my skills in backend development. There’s no doubt about it. I started to love it, what do you want, it’s what help me to buy my courses, so 😂. But for the next year it’s simple:

• Backend development, I’ll continue to learn about : System Design, Django or a new backend framework, microservice, complex stuffs, data base

• Data Structure and Algorithm: I need to up my skill in this area

• System Programming: Learn Go and container from scratch, Linux driver developmment, maybe networking and why not to try to contribute to the Linux Kernel

• Coding Challenges Projects: Complete some projects on the website and work mainly around CLI projects, server/HTPP/Networking and containers projects

Conclusion

During this year 2024, I learn more about myself, to accept the life situations and change/adapt to what’ll happen. And some lessons for myself:

• Be yourself

• Learn what you can

• Do your best

• Take rest

• Learn/Study what make you happy

• You’re not in a running agains other but you’re competing whith your version of yesterday

• Time fly, you need to make some big decisions

• Be humble

And keep this in mind

Unit testing with tRPC + Prisma?

Unit testing in software development is the process to check if a small (unit) part of the code executes like you want. You write code to test if some code behaves like you want without logging something. And I try to do it with tRPC + Prisma while using T3 Stack and it was funny. Here is the procedure I want to test

getByProjectIdAndPos: protectedProcedure
    .input(
      z.object({
        projectId: z.string().nullish(),
        pos: z.number().nonnegative()
      })
    )
        .query(async ({ ctx, input }) => {
            // 1. should return undefined if projectId is not provided
      if (!input.projectId) return undefined

      const countExercises = await ctx.prisma.exercise.count({
        where: { projectId: input.projectId }
      })

            // 2. should throw an error if the provided pos is superior to number of exercises
      if (input.pos >= countExercises) {
        throw new TRPCError({
          code: 'NOT_FOUND',
          message: `No exercise at position '${input.pos + 1}'`
        })
      }

            // 3. should return one exercise if position < countExercises
            // 4. should return undefined if no exercise at this position
      return await ctx.prisma.exercise.findFirst({
        select: defaultExerciseSelect,
        where: {
          projectId: input.projectId
        },
        orderBy: {
          order: 'asc'
        },
        skip: input.pos,
        take: 1
      })
    })

Based on the position of the exercise provided, this procedure returns an exercise or throws an error. You can read the comments in the code to know what we want to test. But we have a few problems:

Some difficulties to test this procedure

1. How to test a protected procedure?

2. How to test the body of this procedure where we're using Prisma client?

One answer for all of them: Mocking

So we need to mock the object that made this procedure to be protected and after we can mock Prisma client methods too.

What makes a tRPC procedure protected in T3 Stack?

A protected procedure is a tRPC procedure whose access is done when the request is authenticated so the request needs a valid session. Check the session on NextAuth to know more about it. So we must mock a session if we want to test this procedure. And since we're using T3 Stack and Prisma as ORM, the prisma client also is set when creating the tRPC context. Look at the file src/server/api/trpc.ts file you will get this

// I keep only useful code for the explanations
import type {CreateNextContextOptions} from "@trpc/server/adapters/next";
import type {Session} from "next-auth";

import {getServerAuthSession} from "../auth";
import {prisma} from "../db";

import {initTRPC, TRPCError} from "@trpc/server";

type CreateContextOptions = {
  session: Session | null;
};

/**
 * This helper generates the "internals" for a tRPC context. If you need to use
 * it, you can export it from here
 *
 * Examples of things you may need it for:
 * - testing, so we dont have to mock Next.js' req/res
 * - trpc's `createSSGHelpers` where we don't have req/res
 * @see https://create.t3.gg/en/usage/trpc#-servertrpccontextts
 */
export const createInnerTRPCContext = (opts: CreateContextOptions) => {
  return {
    session: opts.session,
    prisma,
  };
};

/**
 * This is the actual context you'll use in your router. It will be used to
 * process every request that goes through your tRPC endpoint
 * @link https://trpc.io/docs/context
 */
export const createTRPCContext = async (opts: CreateNextContextOptions) => {
  const { req, res } = opts;

  // Get the session from the server using the unstable_getServerSession wrapper function
  const session = await getServerAuthSession({ req, res });

  return createInnerTRPCContext({
    session,
  });
};

/**
 * 3. ROUTER & PROCEDURE (THE IMPORTANT BIT)
 *
 * These are the pieces you use to build your tRPC API. You should import these
 * a lot in the /src/server/api/routers folder
 */

/**
 * This is how you create new routers and subrouters in your tRPC API
 * @see https://trpc.io/docs/router
 */
export const createTRPCRouter = t.router;

/**
 * Public (unauthed) procedure
 *
 * This is the base piece you use to build new queries and mutations on your
 * tRPC API. It does not guarantee that a user querying is authorized, but you
 * can still access user session data if they are logged in
 */
export const publicProcedure = t.procedure;

/**
 * Reusable middleware that enforces users are logged in before running the
 * procedure
 */
const enforceUserIsAuthed = t.middleware(({ ctx, next }) => {
  if (!ctx.session || !ctx.session.user) {
    throw new TRPCError({ code: "UNAUTHORIZED" });
  }
  return next({
    ctx: {
      // infers the `session` as non-nullable
      session: { ...ctx.session, user: ctx.session.user },
    },
  });
});

/**
 * Protected (authed) procedure
 *
 * If you want a query or mutation to ONLY be accessible to logged in users, use
 * this. It verifies the session is valid and guarantees ctx.session.user is not
 * null
 *
 * @see https://trpc.io/docs/procedures
 */
export const protectedProcedure = t.procedure.use(enforceUserIsAuthed);

You can see that the createInnerTRPCContext takes a session as parameters and returns this session with an instance of the prisma client: it's what we'll use... Inside the section tRPC on T3 Stack documentation, you can find the solution

test("protected example router", async () => {
  const ctx = await createInnerTRPCContext({
    session: {
      user: { id: "123", name: "John Doe" },
      expires: "1",
    },
  });
  const caller = appRouter.createCaller(ctx);

  // ...
});

or simply

test("protected example router", async () => {
  const session = {
      user: { id: "123", name: "John Doe" },
      expires: "1",
    },

  // The two lines below are the most important
  const ctx = await createInnerTRPCContext({session});
  const caller = appRouter.createCaller(ctx);
});

In our case we'll add the prisma client instance

// I'll explain this line after: we create a mock version of the prisma client to avoid to have a direct access to the db
import prismaMock from '@/server/__mocks__/db'

test("protected example router", async () => {
  const session = {
      user: { id: "123", name: "John Doe" },
      expires: "1",
    },

  const ctx = await createInnerTRPCContext({session});
  const caller = appRouter.createCaller({...ctx, prisma: prismaMock});

   // And then
   caller.exercise.getByProjectIdAndPos({ projectId: '1', pos: 1 })    
});

we'll after explain the first line import prismaMock from '@/server/__mocks__/db.

Setup our environment

Add vitest and mock package

I'm using vitest because it's blazing fast 🤣 and seems to be the new choice for tests in JS area.

yarn add vitest vitest-mock-extended -D

File for test

Let's create the file inside src/server/api/routers/tests/exo where we'll put our tests related to exo.

import { beforeEach, describe, expect, it, vi } from 'vitest'

vi.mock('../../../db') // 1

describe('exo procedures testing', () => {
  // 2  
  beforeEach(() => {
    vi.restoreAllMocks()
  })
)

1. Vitest will mock the module found from the given path

2. Will reset all mocked functions/methods

NB: In the version of T3 Stack that I'm using (7.1.0, yes I need to upgrade), I add a default export of prisma inside src/server/db.ts

Mock the prisma client

Inside /server/__mocks__/db, we have a deep-mocked (mock Prisma's method like findFirst, create, etc) version of the Prisma client which is reset before we run each test to avoid having direct access to the DB (we'll install vitest soon).

import type { PrismaClient } from "@prisma/client"
import {beforeEach} from "vitest"
import {mockDeep, mockReset} from "vitest-mock-extended";

beforeEach(() => {
    mockReset(db)
})

const db = mockDeep<PrismaClient>()
export default db

Mock the session object

Not something hard, we need just to create a Session object

src/server/api/routers/tests/exo

import type { Session } from 'next-auth'
// code ...

describe("exo's procedures testing", () => {
    const session: Session = {
    expires: '1',
    user: {
      id: 'clgb17vnp000008jjere5g15i',
      username: ''
    }
  }
// code ..
}

So now we can imitate the protected behavior of our procedures after mocking:

• our session object

• Prisma client

Configure tRPC for tests

What we're doing is like to create a little environment (by mocking some external libs) for our tests so we'll do the same with tRPC by creating a context that'll be used in our tests only. So let's add after the session object the following code to our test file

src/server/api/routers/tests/exo

  // 1
  const ctx = createInnerTRPCContext({ session })
  // 2  
  const caller = appRouter.createCaller({ ...ctx, prisma: prismaMock }

1. We pass to createInnerTRPCContext function provided by tRPC the session object we created (the mocked version of the session). So it's like we have an authenticated user represented by this session.

2. caller will allow us to have access to the procedure we defined in our tRPC routers so it takes the mocked version of prisma and the context we created earlier

With that we can do something like this

await caller.targetTable.targetProcedure(/* input here*/)

Final Setup

If we put everything together we get this

import { beforeEach, describe, expect, it, vi } from 'vitest'
import type { Session } from 'next-auth'

// 1- mock prisma module
vi.mock('../../../db') 

// 2- our tests 
describe('exo procedures testing', () => {
// 3- Reset everything  
  beforeEach(() => {
    vi.restoreAllMocks()
  })

  // 4- session mocked  
  const session: Session = {
    expires: '1',
    user: {
      id: 'clgb17vnp000008jjere5g15i',
      username: ''
    }
  }

  // 5- init tRPC for test
  const ctx = createInnerTRPCContext({ session })
  const caller = appRouter.createCaller({ ...ctx, prisma: prismaMock }   

  describe("procedure 1 - tests", () => {
  // put tests here  
  })  
)

So

In this article we set up our environment to write tests. In the next part of this series we'll write some tests to test 🤣 our setup. You can check these useful links:

• The Ultimate Guide to Testing with Prisma

• Sample integration test - tRPC and T3 Stack

• Example of tests with tRPC - tRPC Github

Reminder

• NextAuth needs/uses for authentication a model User with some pre-configured columns like name, image, email, etc.

• With T3 Stack we use Prisma as ORM

• We have added some columns to the User model like the username

NextAuth and Module augmentation

NextAuth is used for authentication in Next.js applications and is packed in T3 Stack. When using NextAuth, the hook useSession returns session data that can be checked to verify if someone is signed like this:

const index = () => {
    const { data: session } = useSession()
    if (session) {
        return <div>You're signed as {session.user.email}</div>    
    }
    return <div>Not signed, we'll redirect you</div>
}

But for security reasons, the session data returned (by default) contained just

export interface DefaultSession {
    user?: {
        name?: string | null;
        email?: string | null;
        image?: string | null;
    };
    expires: ISODateString;
}

So what can we do if we want to add a new property like the username which is a column of our User model that we define with Prisma?

T3 dedicated's section to NextAuth and NextAuth itself already documented this part. To do it, we'll use module augmentation offered by TypeScript which allows a user to override classes, modules,... types that he/she doesn't access (maybe from a lib). To do it,

1. We can override the Session interface in the next-auth module declaration inside the file types/next-auth.d.ts.

    import { DefaultSession } from "next-auth";
   
    declare module "next-auth" {
      interface Session {
        user?: {
          id: string;
          username: string | null  
        } & DefaultSession["user"];
      }
    }
   

   We import next-auth module and override the Session interface by adding not only a username but also merging the type of user in DefaultSession so we still have the default properties of the user provided by NextAuth.

2. Then in the /pages/api/auth/[...nextauth].ts file we can override session callback by adding username like this

    callbacks: {
        session({ session, user }) {
          if (session.user) {
            session.user.id = user.id;
            session.user.username = user.username;
          }
          return session;
        },
      },
   

   But if you leave it like this, you'll get this warning from TS: Property 'username' does not exist on type 'User | AdapterUser'. Why?

   Because, if you go to the TS definition of User (used by Nextauth) you'll see that it extends DefaultUser which has the following definition (the minimum)

    export interface DefaultUser {
        id: string;
        name?: string | null;
        email?: string | null;
        image?: string | null;
    }
   

   So to fix this error, I think you guess the solution, yes: override the declaration of User as we did it with Session little above.

3. So back to the types/next-auth.d.ts, we can override User after the Session interface (inside next-auth declaration) like this

      interface User extends DefaultUser{
        username: string | null
      }
   

   We've just override User interface by firstly extending DefaultUser (provided by NextAuth) and secondly adding a username.

Let's move to another tip.

Models types deductions (with tRPC)

With Prisma, we define our tables as models inside schema.prisma file but we don't have a direct access to their types in TypeScript because it generates types definitions in node_modules/.prisma/client/index.d.ts by default (read this link if you want to change this output file location). But there are some ways to get these types:

1. Use the return type of Prisma client methods like this

    const getById = async () => await prisma.post.findUnique({
    //....
    })
   
    type Post = ReturnType<typeof getById>
   

2. Or you can use the GetPayload version of the model type where we can specify all relations between this model and others

    import { PrismaClient, Prisma } from "@prisma/client";
    type User = Prisma.UserGetPayload<{
        // here you can specify all relations like: select, include, ...
    }>
   

   You can read more here and on this link.

I don't recommend you these solutions because we have tRPC (don't blame me).

In our case, we use Prisma client through tRPC. And you'll be happy to know that tRPC has a utility to infer routers' input/return types. Let's suppose that we have a router named user and we define inside this router a procedure retrieve. To get the return type in plain tRPC of this procedure we can do:

import type { inferRouterInputs } from "@trpc/server"

type User = inferRouterInputs<AppRouter>['user']['retrieve']

Nice! But how will you be if I say that in T3 Stack there is a little helper to make this operation easier:

import {RouterOutputs} from "@utils/api";
type User = RouterOutputs['project']['getById']

And yes, if you go to look inside the @utils/api you'll see that

export type RouterOutputs = inferRouterOutputs<AppRouter>;

Anyway, it's soft to use.

Bonus:

With this approach, TS can warm you that User type could be null, so to avoid it you can use the NonNullable utility:

import {RouterOutputs} from "@utils/api";
type User = NonNullable<RouterOutputs['project']['getById']>

Right!

Conclusion

I love the end-to-end typesafe that we get from tRPC and it's very cool to work with T3 Stack where they put together some great libs and frameworks to make full-stack development with JS much easier.

We are at the end of this article. I hope you enjoy it. If I get another tip while building my SaaS, I'll share them. Feedback or anything that could help me to improve this post is welcome.

The project

As the title said, I'm building a SaaS as a side project (maybe I can call it SaaSaSP 🤣). The idea is to allow people to build exercises with questions and responses on the platform and share this with other people who can practice. This platform will be used on one hand by students during revision periods and on another hand, by teachers who can share homework with their students.

I only work evenings and Saturdays on the project (because I have a full-time job). And currently, I can say that I am at 20% of the first version of the product.

The Design

I don't have a design so I went to websites with which I like the design and I try to reproduce them for the pages I need. For example:

• Sign in: I'm stunned by the login page of Adobe so I tend to reproduce it

  

  

• Dashboard: I like vercel and sanity dashboards

  

  

I don't finish to build the dashboard so I haven't concrete pictures.

The Stack and why this?

Yes, the big part. To make my choice here are the criteria I followed:

1. I want to use tools with which I have some prior experiences to code the application quickly and which are popular and stable so that if I encounter a bug I can find resources to resolve it. In my case, I use only: (TS)React/Next.js. So the frontend lib/framework must be one of these two.

2. I don't want to deal too much with CSS and I hope the UI lib will have a lot of basic components with a good styling approach. Here I was thinking about 3 UI lib based on React (in order of consideration given):

   • React Spectrum & React Aria: A collection of lib and tools from Adobe developers. They implement Adobe design systems which I find nice. It has many components including a date range picker, drags and drops, and useful resources. What I like about this lib are the efforts the maintainers put into accessibility and WAI-ARIA. Also, they have many hooks like useDateRangePicker that you can use to build a custom component, here is a date range picker 😛.

   • Mantine: A relatively new lib with a bunch of pre-built advanced components like a search bar with a filter (more featured than what I built here). Its strength lies in the ease with which you can customize its components with a better developer experience than with Material UI.

   • Next UI: A new lib too, with a beautiful design around gradients and based on...React Aria ( 😎 Yes, the same as I mentioned above)

   • Chakra UI: One of my favourite UI components libs. Simple, easy to use with a very well-designed approach and built by a Nigerian (😎 a border country to Benin, my country). For my side projects, I use Chakra UI. It's very well documented and even if It doesn't have a lot of pre-built components like a date range picker, I was comfortable using it.

     So my choice?: Chakra, why?

     I found NextUI css API a little complicated (it supports styled-components), with React Spectrum the button's onClick prop becomes onPress (can you imagine ? I don't know the reason) and I don't like Mantine basic style (😄 someone who is not good in CSS criticizes a UI components libs). But Chakra, everything seems easy, simple and intuitive: _hover is a prop that you can provide with style when the mouse hover your component (really intuitive, isn't it ?)

     NB: I am not a fan of Material UI, customization is an obstacle course where you need to check the css class name of children in chrome inspector to overrides their styles

3. I want a real full-stack React/Next framework that'll allow me to write JS code in both the server and front end. If it can include a config for database and authentication it'll be very good. And no, I don't have the time and energy to go with API first architecture so I cannot have the Node server separate from the frontend. So I have 3 choices:

   • Redwood.js: It's a good candidate and I like the name and the full battery included: Routing, GraphQL, TypeScript, Prisma, and Authentication. It's good but I need to jump to GraphQL and they have their routing system (It'll take me some time to be ready, the time I don't have 😄). But I want to test it one day

   • Blitz.js: Good too, built on top of...Next.js. It comes with authentication, is database agnostic and has a CLI tool: blitz. Two things I like about it: Route manifest which allows you to refer to a page by name through an object instead of hand-coded string (useful with dynamic route) and their use of Zod to validate input in form (it's the first time I hear about Zod). But they were talking about pivoting to a framework agnostic and they were relatively new, so I didn't pick them

   • T3 Stack: On the home page you can read: The best way to start a full-stack, typesafe Next.js app. So I choose it 😄. Ok I'll explain. T3 Stack doesn't bring anything new but it puts together well know/established tools and libs to let you focus on your business logic. Firstly the core of this stack is also TypeScrpt/Next.js. Secondly, for authentication, they use NextAuth.js, for the API they use tRPC which allows you to share your types between client and server code (input validations, return types) by writing typed procedures on the server side. tRPC has a plugin for the well-known React Query that T3 Stack includes for you. For the database, this stack uses the popular ORM Prisma so you can connect your application to all databases (PostgreSQL in my case) supported by Prisma. When configuring it asks to install tailwindcss for styling. Something great is that: you can install what you need only.

     You can understand that I chose T3 Stack. At the end of the project, I'll share my feelings after using this stack.

There is another choice I need to do: the deployment platform. As I haven't finished or have a beta version yet, I haven't made my choice yet. But I'm thinking between: fly.io, render and vercel.

We are athe end of this article, in my next article I'll share some tricks that I'm using with this stack.