Computer Architecture Tutorial: Learn CPU Design from Scratch (2026)
Computer architecture is the science and art of designing processors that execute programs as fast as possible within power and cost constraints. After working on processor design teams at a major semiconductor company, I have learned that understanding architecture is essential for writing high-performance software. This tutorial covers the CPU pipeline from fetch through retirement, the memory hierarchy from registers to disk, and advanced techniques like SIMD, out-of-order execution, and speculation.
We will build a pipeline simulator in Python, analyze cache performance, and explore how modern CPUs break the sequential programming model to achieve instruction-level parallelism.
The Five-Stage RISC Pipeline
The classic five-stage pipeline (Fetch, Decode, Execute, Memory, Writeback) allows multiple instructions to be in flight simultaneously. In the ideal case, throughput approaches one instruction per cycle. However, hazards prevent this: structural hazards (resource conflicts), data hazards (RAW, WAR, WAW), and control hazards (branches). Data forwarding paths bypass the writeback stage to reduce stalls.
class FiveStagePipeline:
def __init__(self):
self.regs=[0]*32; self.mem=[0]*4096; self.pc=0
self.s={'IF':None,'ID':None,'EX':None,'MEM':None,'WB':None}
def fetch(self):
if self.pc>26,'rs':(i>>21)&0x1F,'rt':(i>>16)&0x1F,'rd':(i>>11)&0x1F}
self.s['IF']=None
def execute(self):
if self.s['ID']:
self.s['EX']={'res':self.rd(self.s['ID']['rs'])+self.rd(self.s['ID']['rt']),'rd':self.s['ID']['rd']}
self.s['ID']=None
def mem_acc(self):
if self.s['EX']: self.s['MEM']=self.s['EX']; self.s['EX']=None
def wb(self):
if self.s['MEM']:
if self.s['MEM']['rd']!=0: self.regs[self.s['MEM']['rd']]=self.s['MEM']['res']
self.s['MEM']=None
def rd(self,r):
if r==0: return 0
if self.s['EX'] and self.s['EX']['rd']==r: return self.s['EX']['res']
if self.s['MEM'] and self.s['MEM']['rd']==r: return self.s['MEM']['res']
return self.regs[r] Cache Hierarchy and Memory Wall
The memory wall is the growing gap between processor and DRAM speed. Caches exploit temporal locality (recently accessed data is likely reused) and spatial locality (nearby data is likely accessed soon). A typical hierarchy has L1 (32KB, 1-2 cycles), L2 (256KB, 5-10 cycles), L3 (8-32MB, 20-50 cycles), and main memory (80-200 cycles).
class Cache:
def __init__(self,size,assoc,line=64):
self.ls=line; self.nl=size//line; self.assoc=assoc; self.ns=self.nl//assoc
self.l=[{'v':False,'tag':0,'lru':0} for _ in range(self.nl)]; self.h=0; self.m=0
def acc(self,addr):
ba=addr//self.ls; tag=ba//self.ns; si=ba%self.ns; st=si*self.assoc
for i,way in enumerate(self.l[st:st+self.assoc]):
if way['v'] and way['tag']==tag: self.h+=1; return True
self.m+=1
ev=st+min(range(self.assoc),key=lambda i:self.l[st+i]['lru'])
self.l[ev]={'v':True,'tag':tag,'lru':0}
return FalseSIMD and Vector Processing
SIMD allows a single instruction to operate on multiple data elements simultaneously. x86 SSE/AVX extend registers to 128/256/512 bits. ARM NEON provides similar capabilities. Auto-vectorization by compilers can exploit SIMD from scalar code, but manual intrinsics often yield better results for multimedia, scientific computing, and machine learning.
#include
float dot(const float* a, const float* b, int n) {
__m256 sum = _mm256_setzero_ps(); int i;
for (i=0; i<=n-8; i+=8) {
__m256 va = _mm256_loadu_ps(&a[i]); __m256 vb = _mm256_loadu_ps(&b[i]);
sum = _mm256_fmadd_ps(va, vb, sum);
}
__m128 hi = _mm256_extractf128_ps(sum,1); __m128 lo = _mm256_castps256_ps128(sum);
__m128 s = _mm_add_ps(lo,hi); s=_mm_hadd_ps(s,s); s=_mm_hadd_ps(s,s);
float r; _mm_store_ss(&r,s);
for (; i Out-of-Order Execution and Tomasulos Algorithm
Out-of-order execution allows the processor to execute instructions as operands become ready. Tomasulo's algorithm uses reservation stations, a reorder buffer (ROB), and register renaming to eliminate WAR and WAW hazards. The ROB commits results in program order, ensuring precise exceptions.
class ResStation:
def __init__(self): self.op=None; self.vj=None; self.vk=None; self.busy=False
class Tomasulo:
def __init__(self):
self.pregs=[0]*64; self.rename={}; self.free=list(range(16,64))
self.rs=[ResStation() for _ in range(4)]
def issue(self, inst):
rs = next((r for r in self.rs if not r.busy), None)
if not rs: return False
rs.busy=True; rs.op=inst.op
if inst.rd!=0: rs.addr=self.free.pop(); self.rename[inst.rd]=rs.addr
return TrueBranch Prediction and Speculative Execution
Branch mispredictions cost 10-20 cycles as the pipeline must be flushed. Predictors use 2-bit saturating counters, global history shift registers, and tournament predictors. Speculative execution allows the CPU to execute instructions past a predicted branch. If mispredicted, results are discarded — but side effects in the cache can persist (Spectre).
class BranchPredictor:
def __init__(self, gb=12, pt=4096):
self.ghr=0; self.gb=gb; self.pht=[2]*pt; self.m=0; self.t=0
def pred(self, pc):
self.t+=1; idx=(pc&(len(self.pht)-1))^self.ghr
return self.pht[idx]>=2
def update(self, pc, taken):
idx=(pc&(len(self.pht)-1))^self.ghr
if taken: self.pht[idx]=min(3,self.pht[idx]+1)
else: self.pht[idx]=max(0,self.pht[idx]-1)
self.ghr=((self.ghr<<1)|taken)&((1<Memory Consistency Models: x86-TSO vs ARM
The memory consistency model defines the order in which memory operations from one thread become visible to others. x86-TSO provides Total Store Order with store forwarding. ARM and RISC-V use relaxed models requiring explicit fence instructions. The C++ memory model with acquire/release semantics abstracts across architectures.
#include
#include
std::atomic x{0}, y{0}; int data{0};
void t1() { data=42; x.store(1, std::memory_order_release); }
void t2() { while(!y.load(std::memory_order_acquire)); assert(data==42); }
void t3() { while(!x.load(std::memory_order_acquire)); data=100; y.store(1, std::memory_order_release); } Frequently Asked Questions
What is the difference between RISC and CISC?
RISC uses simple, fixed-length instructions that execute in one cycle. CISC has variable-length, powerful instructions. Modern x86 CPUs decode CISC to RISC-like micro-ops internally.
What is Amdahl's law in architecture?
Even if we make the execution unit 100x faster, overall speedup is bounded if the front-end (fetch/decode) remains unchanged. The speedup is limited by the non-improved fraction.
What is the difference between hyper-threading and multi-core?
Multi-core duplicates entire CPU cores. Hyper-threading (SMT) shares a core's execution resources between two logical threads, adding 15-30% throughput at minimal area cost.
Why did the industry move from frequency scaling to multi-core?
The power wall: dynamic power ~ V^2 * f. Above 3-4 GHz, power density becomes unmanageable (dark silicon). Multi-core achieves higher throughput per watt.
Originally published on Ayodhyyya. Last updated June 1, 2026.