Pipelining and Hazards in Processors With easy examples.

Pipelining is a technique that overlaps the execution of multiple instructions, like an assembly line overlaps the building of multiple cars. A classic processor pipeline splits every instruction into 5 stages — fetch, decode, execute, memory access, write-back — so that while one instruction is being executed, the next is being decoded and the one after is being fetched.

Pipelining matters because it multiplies throughput — instructions completed per second — without requiring faster transistors. It is the reason a 3 GHz processor can retire several billion instructions per second, and understanding it explains why branches and data dependencies can slow programs down.

Key Concepts

• Pipeline stages: IF (instruction fetch), ID (decode), EX (execute), MEM (memory access), WB (write back).
• Throughput vs latency: pipelining improves throughput (instructions per second) but a single instruction still takes 5 cycles of latency.
• Data hazard: an instruction needs a result still being computed — solved by forwarding or stalling.
• Control hazard: a branch's outcome is unknown while later instructions are already fetched — handled by branch prediction or flushing.
• Structural hazard: two stages need the same hardware resource at once — avoided by duplicating hardware.

Formulas

• Without pipelining: n instructions x k stages = n x k cycles.
• With pipelining: k + (n - 1) cycles — k cycles to fill the pipe, then one instruction completes per cycle.
• Speedup: (n x k) / (k + n - 1). As n grows large, speedup approaches k, the number of stages.
• CPI (cycles per instruction): ideal pipelined CPI = 1; stalls from hazards push it above 1.

Worked Example

Run 100 instructions through a 5-stage pipeline with no hazards:

1. Without pipelining: 100 x 5 = 500 cycles.
2. With pipelining: 5 + (100 - 1) = 5 + 99 = 104 cycles.
3. Speedup = 500 / 104, about 4.8x — close to the theoretical maximum of 5.
4. Now add a data hazard causing 20 stall cycles: total = 124 cycles, CPI = 124/100 = 1.24, and speedup drops to 500/124, about 4.0x.

The math shows both the promise and the fragility: an ideal pipeline nearly quintuples throughput, but every stall cycle is subtracted directly from that gain — which is why forwarding logic and branch predictors exist.

Common Misconceptions

• Pipelining makes each instruction faster. Latency stays at 5 cycles; what improves is how many instructions finish per second.
• Deeper pipelines are always better. More stages mean bigger penalties when branches mispredict — a 20-stage pipeline flush wastes 20 cycles of work.
• Hazards make pipelining useless. Forwarding, stalling, and prediction handle hazards in hardware; real pipelines sustain near-ideal throughput.

Real-World Applications

• Modern CPUs: use 10-20+ stage pipelines with sophisticated branch predictors.
• Superscalar processors: run multiple pipelines in parallel, issuing several instructions per cycle.
• GPUs: pipeline thousands of threads through deep execution stages.
• Compiler scheduling: instruction schedulers reorder code to keep pipelines full.

Pipelining turns sequential instruction execution into an assembly line: five stages, one instruction finishing per cycle, nearly five times the throughput. Hazards — data, control, and structural — are the friction in that machine, and the last forty years of processor design have largely been the story of hiding them.