Operating System Tutorial: Learn OS from Scratch (2026)
An operating system is the invisible conductor orchestrating hardware resources, process execution, and user interaction. Through building a toy kernel and debugging production performance issues, I have come to appreciate how OS abstractions — processes, virtual memory, filesystems — shape every application we write. This tutorial peels back the layers between your code and the silicon, explaining the core subsystems with practical examples.
We will cover process management, CPU scheduling, memory virtualization, concurrency, filesystem design, and I/O handling. Each concept is tied to real-world incidents: a priority inversion that stalled a Mars rover, a buffer overflow that spawned a generation of exploits, and the page cache that makes your database queries snappy.
Processes and Threads
A process is an instance of a running program with its own address space, open file descriptors, and execution state. Threads are lightweight units within a process that share the same memory space, enabling parallelism within a single process. The kernel maintains a process control block (PCB) for each process storing registers, program counter, memory maps, and scheduling information. Context switching between processes is expensive because the TLB must be flushed, whereas thread switches within the same process are cheaper.
# Process creation via fork on Unix-like systems
import os
pid = os.fork()
if pid == 0:
print(f"Child process PID={os.getpid()}, PPID={os.getppid()}")
os.execve("/bin/ls", ["/bin/ls", "-l"], {})
else:
print(f"Parent process PID={os.getpid()}, child PID={pid}")
os.waitpid(pid, 0)CPU Scheduling Algorithms
The scheduler decides which ready process runs on the CPU. First-Come, First-Served (FCFS) is simple but suffers from the convoy effect. Shortest Job First (SJF) minimizes average waiting time but requires future knowledge. Round Robin with a time quantum of 10-100 ms provides fairness and low response time, though quantum selection is critical — too large degrades interactivity, too small wastes time on context switches. Modern schedulers like Linux's CFS (Completely Fair Scheduler) use a red-black tree of task virtual runtimes to approximate ideal multitasking.
def round_robin(processes, quantum):
queue = list(processes)
time = 0
while queue:
pid, burst = queue.pop(0)
if burst <= quantum:
time += burst
print(f"Process {pid} finished at time {time}")
else:
time += quantum
queue.append((pid, burst - quantum))
print(f"Process {pid} ran for {quantum}, remaining {burst-quantum}")Virtual Memory and Paging
Virtual memory gives each process the illusion of a contiguous address space starting at zero. The Memory Management Unit (MMU) translates virtual addresses to physical frames via page tables. Paging eliminates external fragmentation and enables demand paging — only needed pages are loaded into RAM. Page replacement policies like LRU, FIFO, and Clock algorithm handle when physical memory fills up. The optimal algorithm (OPT) evicts the page used farthest in the future but is unrealizable, so LRU serves as a practical approximation.
# Clock (Second-Chance) page replacement
def clock_replace(pages, frames):
memory = [None] * frames
ref_bits = [0] * frames
ptr = 0
faults = 0
for page in pages:
if page in memory:
ref_bits[memory.index(page)] = 1
else:
while ref_bits[ptr] == 1:
ref_bits[ptr] = 0
ptr = (ptr + 1) % frames
memory[ptr] = page
ref_bits[ptr] = 1
ptr = (ptr + 1) % frames
faults += 1
return faultsConcurrency and Synchronization
When multiple threads access shared data concurrently, race conditions arise. Locks (mutexes) enforce mutual exclusion, but naive locking can cause deadlocks where threads wait indefinitely for each other. The four Coffman conditions — mutual exclusion, hold-and-wait, no preemption, circular wait — must all hold for a deadlock to occur. Semaphores generalize locking by counting available resources. The producer-consumer problem, solved with two semaphores, is the canonical pattern for bounded buffer coordination.
from threading import Semaphore, Thread
buffer = []
empty = Semaphore(10)
full = Semaphore(0)
mutex = Semaphore(1)
def producer(item):
empty.acquire()
mutex.acquire()
buffer.append(item)
mutex.release()
full.release()
def consumer():
full.acquire()
mutex.acquire()
item = buffer.pop(0)
mutex.release()
empty.release()
return itemFilesystems: Inodes and Data Blocks
Filesystems organize persistent data on disk. The Unix inode structure stores metadata (permissions, timestamps, size, block pointers) separately from the filename. Directory entries map names to inode numbers. The ext4 filesystem uses a multi-level index: direct blocks, single indirect, double indirect, and triple indirect pointers, allowing it to handle both small files efficiently and large files (up to 16 TB) within a single structure. Journaling — recording pending writes in a log — prevents metadata corruption after a crash.
# Simulated inode with direct and indirect blocks
class Inode:
def __init__(self):
self.permissions = 0o644
self.size = 0
self.block_count = 12
self.direct = [None] * 12
self.single_indirect = None
def read_block(inode, block_num):
if block_num < 12:
return inode.direct[block_num]
if inode.single_indirect:
offset = block_num - 12
return inode.single_indirect[offset]
raise IndexError("Block beyond file size")I/O Management and DMA
I/O devices are slower than CPUs by orders of magnitude, so the OS must manage data transfers efficiently. Programmed I/O (PIO) has the CPU busy-wait on device registers, wasting cycles. Interrupt-driven I/O lets the device signal the CPU when data is ready. Direct Memory Access (DMA) allows the device to transfer data directly to RAM without CPU involvement — the CPU sets up the transfer and the DMA controller handles the rest, raising an interrupt only on completion. Buffering and spooling further decouple fast processors from slow devices.
# DMA transfer pseudo-code
def dma_transfer(device, memory_addr, size):
dma = DMA_Controller()
dma.source = device.data_port
dma.destination = memory_addr
dma.count = size
dma.start()
while not dma.completed:
pass
print(f"DMA transfer of {size} bytes complete")Frequently Asked Questions
What is thrashing and how do you prevent it?
Thrashing occurs when a system spends more time swapping pages in and out than executing processes. It is prevented by adjusting the degree of multiprogramming, using working-set models, and ensuring sufficient physical memory for active processes.
How does a system call differ from a function call?
A system call traps into kernel mode through a software interrupt (e.g., int 0x80 or syscall), switching the CPU to privileged mode. A function call stays in user mode. System calls are orders of magnitude slower due to context switching and privilege escalation.
What is the difference between a mutex and a semaphore?
A mutex is a binary lock owned by the thread that acquired it, typically used for mutual exclusion. A semaphore counts available resources and can be signaled by any thread. Binary semaphores can approximate mutexes but lack ownership tracking.
Why does Linux use a red-black tree in its CFS scheduler?
The red-black tree maintains processes ordered by virtual runtime (vruntime), enabling O(log n) insertion and lookup of the leftmost (most deserving) process. It replaces the O(n) linked list used in earlier O(1) schedulers, scaling better with many threads.
Originally published on Ayodhyyya. Last updated June 1, 2026.