Skip to content

Chapter 6: On-Chip Interconnect

A modern system-on-chip is a city of dozens of blocks: CPU clusters, DMA engines, memory controllers, accelerators and a swarm of peripherals, all of which must reach memory and each other. The interconnect is the road network that lets them share access without gridlock. This chapter builds up from the golden rule of on-chip protocols, the VALID/READY handshake, through the AMBA family (APB, AHB and AXI), TileLink, Wishbone and CHI, to full networks-on-chip, and ends with two small models that show where the performance of a bus really comes from.

You need the idea of a memory-mapped address from Chapters 1 and 4 and the notion of coherence from Chapter 5 for the remarks on TileLink and CHI.

6.1 Why a chip needs roads

A single core wired directly to a single memory needs no interconnect. A real chip has many blocks that start transactions and many that answer them, and wiring each directly to every other would need about \(n^2/2\) connections for \(n\) blocks, which would cover the die. Instead the blocks attach to a shared fabric that routes, arbitrates and decodes on their behalf. A city does not build a private road from every house to every shop; it builds a road network with junctions and traffic lights.

Who asks, and who answers

A manager starts transactions: "read address A", "write this to B". CPU cores, GPUs and DMA engines (hardware that copies blocks of data by itself) are managers. Older documents say master. A subordinate responds: a memory, a block of control registers, a FIFO, a serial port. Older documents say slave. One block can be both: a DMA engine moves data as a manager, but its settings are written by the CPU, so to the CPU it is a subordinate.

Every subordinate owns an address range

0x0000_0000 - 0x0000_FFFF   boot ROM
0x1000_0000 - 0x1000_0FFF   interrupt controller
0x1001_0000 - 0x1001_0FFF   UART0
0x1001_1000 - 0x1001_1FFF   SPI0
0x2000_0000 - 0x2FFF_FFFF   PCIe / accelerator
0x8000_0000 - 0xFFFF_FFFF   DRAM (via the memory controller)

This table is the chip's memory map. A store to 0x1001_0000 does not touch RAM at all: it lands in a register of the serial port, UART0. (The underscore only groups hex digits for reading, like the commas in 1,000,000.)

The interconnect's three jobs

  1. Address decoding. Look at each request's address and send it to the one subordinate that owns it: in hardware, just a comparator for each address range.
  2. Arbitration. When two managers want the same subordinate in the same cycle, pick a winner (in turn, by priority, or weighted by quality of service) and make the other wait. Fairness and the avoidance of starvation live here.
  3. Bridging. Join parts that differ: a 32-bit slow device hanging off a 128-bit fast backbone needs data-width conversion, a clock-domain crossing, and often protocol translation.

6.2 Bus, crossbar, network

There are three ways to lay the roads, from cheap and simple to big and scalable.

  • A shared bus is one set of wires that everyone time-shares. It is cheap and simple, but only one transfer can occur at a time, and contention rises fast. (Classic AHB and shared Wishbone.)
  • A crossbar is a full switch: any manager can reach any subordinate, and non-conflicting pairs proceed in parallel. It gives great bandwidth, but its area and wiring grow as managers × subordinates. Most AXI interconnects are full or partial crossbars.
  • A network-on-chip (NoC) sends packets through routers, like a tiny internet on the die. It scales to hundreds of endpoints and is the standard choice for many-core CPUs and large SoCs.
A shared bus (one transfer at a time), a crossbar (non-clashing transfers in parallel) and a network on chip (routers with a block at each).
Figure 6.1. A shared bus (one transfer at a time), a crossbar (non-clashing transfers in parallel) and a network on chip (routers with a block at each).
Shared bus Crossbar Network-on-chip
Scales to A few managers About 16 × 16 Hundreds of nodes
Wiring cost Low Grows as managers × subordinates Grows about linearly
Delay Low when quiet; bad when busy Low and steady A few cycles per hop, predictable
At once 1 transfer Many, if there is no clash Many flows
Typical use Slow peripherals The main backbone of most chips Many-core CPUs, GPUs

Real chips mix them: a fast crossbar or network in the middle connects the CPU clusters, memory and high-bandwidth accelerators, and slow devices hang off a cheap bus behind a single bridge. Spend wires where the traffic is.

6.3 The VALID/READY handshake

Nearly every modern on-chip protocol is built on one tiny rule.

  • The sender drives VALID: "there is something on the wires this cycle".
  • The receiver drives READY: "I can take it this cycle".
  • A transfer happens on a rising clock edge if and only if both are high on that edge.
  • Once VALID is raised, the sender must hold it, and keep the data steady, until the transfer happens.

It is like handing over a parcel. The courier holds it out (VALID); you hold out your hands (READY). It changes hands only when both are out, and the courier must not snatch it back while waiting.

A VALID/READY handshake: VALID is high for cycles 1 to 4 and READY for cycles 3 and 4, so the two transfers happen at the clock edges that end cycles 3 and 4.
Figure 6.2. A VALID/READY handshake: VALID is high for cycles 1 to 4 and READY for cycles 3 and 4, so the two transfers happen at the clock edges that end cycles 3 and 4.

Example 6.1: counting transfers

VALID is high in cycles 1 to 4; READY is high only in cycles 3 and 4. Which cycles transfer data?

In cycles 1 and 2 VALID is high but READY is low: no transfer, and the sender holds D0 steady. In cycle 3 both are high: D0 is transferred at the edge that ends cycle 3. In cycle 4 both are still high: D1 is transferred at the edge that ends cycle 4. After that VALID falls. There are two transfers.

Two gifts come from this one rule.

Back-pressure. A busy receiver simply keeps READY low. The sender waits with its data; nothing is lost and nothing overflows. Flow control comes free, with no explicit credit accounting on the wire. Symmetrically, a sender with nothing to send holds VALID low and the receiver idles.

Easy composition. Because each handshake is self-contained and stateless per beat, a register (a skid buffer) can be dropped into any channel to break a long timing path, and channels can be forked and joined in a crossbar, and everything still works, just a cycle later. This is latency-insensitive design: pipeline stages can be inserted freely without breaking function.

6.4 The AMBA family: APB, AHB and AXI

Arm's AMBA (Advanced Microcontroller Bus Architecture) standards are free to use and are the most widely used on-chip protocols in the world, far beyond Arm cores. They are three protocols for three tiers of the SoC.

  • APB, the Advanced Peripheral Bus. Simple, low-power and unpipelined: one access at a time in two phases (setup, then access), no bursts, no outstanding transactions. It suits slow registers: timers, GPIO and the configuration of serial ports.
  • AHB, the Advanced High-performance Bus. Pipelined: the address phase of the next transfer overlaps the data phase of the current one. One shared bus with a central arbiter; supports bursts. It was the classic higher-bandwidth backbone before AXI.
  • AXI, the Advanced eXtensible Interface. Today's high-performance choice: five independent channels, separate reads and writes, bursts, and many outstanding, out-of-order transactions through IDs.

AXI comes in three sizes. AXI4 is the full burst protocol (up to 256 beats per burst). AXI4-Lite is a stripped-down subset with single beats, no bursts and no IDs, ideal for simple register interfaces on accelerators. AXI4-Stream drops addressing entirely: a pure point-to-point flow of data (TVALID, TREADY, TDATA, TLAST) for video, audio and DSP pipelines.

The five AXI channels

AXI's power comes from splitting reads and writes into fully independent channels, each its own VALID/READY handshake. Writes use three channels, reads two, and they run concurrently and out of lock-step.

Channel Short Driven by Carries
Write address AW manager AWADDR, AWID, AWLEN (burst length), AWSIZE, AWBURST
Write data W manager WDATA, WSTRB (byte-lane enables), WLAST
Write response B subordinate BRESP (OKAY, EXOKAY, SLVERR, DECERR), BID
Read address AR manager ARADDR, ARID, ARLEN, ARSIZE, ARBURST
Read data R subordinate RDATA, RRESP, RID, RLAST

Bursts: one address, many beats

In a burst, one address beat describes a whole run of data beats. AWLEN (or ARLEN) gives the number of beats minus one, AWSIZE gives the bytes per beat as a power of two, and AWBURST gives the mode.

Mode Address on each beat Used for
INCR goes up Copying a contiguous block of memory
WRAP goes up, then wraps inside an aligned block Cache-line fills, "the word you are waiting for first" (critical word first)
FIXED stays the same Pouring data into one FIFO register
AXI write burst: 4 beats of 4 bytes to 0x8000_1000
  AW:  AWADDR=0x80001000  AWLEN=3 (4 beats)  AWSIZE=2 (4 bytes)  AWBURST=INCR  AWID=5
  W :  WDATA=D0  WSTRB=1111  WLAST=0
  W :  WDATA=D1  WSTRB=1111  WLAST=0
  W :  WDATA=D2  WSTRB=1111  WLAST=0
  W :  WDATA=D3  WSTRB=1111  WLAST=1      <- the last beat
  B :  BRESP=OKAY  BID=5                  <- one reply for the whole burst
A four-beat AXI write burst on three independent channels: one address beat on AW, four data beats on W with WLAST on the last, and one response on B.
Figure 6.3. A four-beat AXI write burst on three independent channels: one address beat on AW, four data beats on W with WLAST on the last, and one response on B.

Outstanding transactions and IDs

A manager may send a new address before earlier requests have been answered. Requests still waiting for their answer are outstanding transactions. Each carries an ID (AxID). Answers with different IDs may come back in any order, so a fast on-chip memory can overtake a slow DRAM read; answers with the same ID stay in order. The fabric matches each R or B beat to its request by ID.

6.5 Two models of performance

Interconnect performance rests on two levers: amortising the per-transaction overhead with bursts, and hiding latency with outstanding transactions.

Example 6.2: why bursts pay

Suppose issuing an address costs a fixed 5 cycles of overhead and each 8-byte beat costs 1 cycle. Move 64 bytes.

  • As eight single beats: \(8 \times (5 + 1) = 48\) cycles, a rate of \(64/48 = 1.33\) bytes per cycle.
  • As one 8-beat burst: \(5 + 8 = 13\) cycles, a rate of \(64/13 = 4.9\) bytes per cycle.

The same data moves at \(48/13 \approx 3.7\) times the throughput, because the address overhead is paid once. This is exactly why cache-line fills and DMA copies use long INCR or WRAP bursts.

Example 6.3: hiding a 20-cycle delay

A memory takes 20 cycles to answer but accepts a new request every cycle. One request at a time, you get one answer per 20 cycles. Keep 20 requests in flight (pipeline the AR channel, with distinct IDs) and, once the pipe is full, you get one answer every cycle: 20 times the throughput. Little's law says it in general:

\[\text{throughput} = \frac{\text{requests in flight}}{\text{delay}}\]

With 4 requests in flight the throughput is \(4/20 = 0.2\) answers per cycle; to reach 1 per cycle you need \(1 \times 20 = 20\) in flight.

AXI is not faster than APB because of its clock

Both can run at the same frequency. AXI wins through concurrency: separate read and write channels, bursts that amortise addressing, and many outstanding, reorderable transactions that hide memory latency. APB moves one register at a time with per-access overhead and no overlap. A "faster" bus that allows only one outstanding transaction can still be slower than a modest one that keeps 16 in flight.

TileLink is an open interconnect standard from SiFive, native to the Rocket and BOOM cores and to the rocket-chip and Chipyard generators (Chapter 8). One protocol scales from a trivial register bus to full cache coherence, through three conformance levels.

Level Name Capability Channels used Rough AXI analogue
TL-UL Uncached lightweight Single-beat reads and writes: simple memory-mapped registers A, D AXI4-Lite
TL-UH Uncached heavyweight Adds bursts, atomics and hints such as prefetch, but no caching A, D AXI4 (full)
TL-C Cached (coherent) Full cache coherence: permission acquire and release, probes A, B, C, D, E Arm's coherent buses

The five channels A to E have a strict priority order that provably prevents deadlock.

Channel Direction Purpose
A manager to subordinate Request: read (Get), write (Put), atomic, or acquire-permission
B subordinate to manager Probe: ask a caching manager to give up or downgrade a block (TL-C only)
C manager to subordinate Release or probe response: write back dirty data, acknowledge a probe (TL-C only)
D subordinate to manager Response or grant: read data, write acknowledgement, permission grant
E manager to subordinate Grant acknowledgement: the final handshake that closes a coherence transaction (TL-C only)

Both TileLink and AXI use VALID/READY, support bursts and outstanding transactions. The key differences are that TileLink builds coherence into the protocol itself (TL-C) instead of needing a separate coherent variant, that it enforces the channel priority to prove deadlock freedom formally, and that it is open and RISC-V-native. AXI is more widespread in commercial IP and has the vast AMBA ecosystem behind it. Chipyard SoCs typically use TileLink inside and bridge to AXI at the memory-controller and PCIe edges.

Wishbone is the other end of the scale: a public-domain, deliberately minimal bus from OpenCores, common in open-source IP and in the LiteX ecosystem. Classic Wishbone is a simple synchronous handshake (STB, ACK and CYC) supporting single and (in its pipelined B4 form) burst cycles. Its charm is simplicity, at the cost of deep outstanding transactions, reordering and coherence.

CHI, Arm's Coherent Hub Interface, is the top of the AMBA stack, aimed at large, many-core, cache-coherent systems. Unlike AXI's channel-and-address model, CHI is packet-based with credit flow control, designed to scale across a coherent mesh, carrying snoops and coherence messages between many caches and home nodes. It is overkill for a microcontroller and essential for a 64-core server chip.

6.7 Inside a network-on-chip

Once there are too many endpoints for a crossbar, a packet-switched NoC takes over, and its vocabulary mirrors computer networking, shrunk onto silicon.

  • Each block attaches to a router; routers connect to their neighbours by point-to-point links.
  • A message is chopped into packets, and each packet into fixed-size flits (flow-control units): a head flit carries the routing information, body flits carry the payload and a tail flit closes the packet. Flits stream through the routers, often in wormhole fashion, so that one packet spans several routers at once.
  • Topologies. A ring is cheap (two links per node) but its latency grows linearly with size. A mesh is a two-dimensional grid with good bisection bandwidth and simple routing, and is the many-core favourite. A torus is a mesh with wrap-around links, halving the worst-case hop count at the cost of longer wires.
  • Routing picks the path. XY routing, which goes along X first and then along Y, is simple and cannot deadlock on a mesh. Adaptive routing dodges congestion but must avoid routing deadlock, for example with virtual channels.
  • Flow control decides when a flit may advance. Credit-based schemes track the free buffer slots downstream. Virtual channels share one physical link between several queues, so that one blocked packet does not hold up all the others (head-of-line blocking).
XY routing on a four-by-four mesh: a packet from router (0,0) to router (3,2) travels along X first, then along Y.
Figure 6.4. XY routing on a four-by-four mesh: a packet from router (0,0) to router (3,2) travels along X first, then along Y.

6.8 Summary

  • Managers start transactions, subordinates answer. The interconnect decodes addresses, arbitrates and bridges.
  • Shared bus, crossbar and network-on-chip run from cheap and serial to parallel and scalable; big chips use all three.
  • VALID/READY: a transfer happens only when both are high. Back-pressure and easy composition come free.
  • AMBA: APB for slow registers, AHB the older backbone, AXI with five channels, bursts and outstanding, reorderable transactions.
  • TileLink scales from registers to coherence for RISC-V; Wishbone is the simple open bus; CHI is the large coherent one.
  • Performance comes from concurrency, not clock rate: bursts amortise addressing, and outstanding transactions hide latency (Little's law).

6.9 Exercises

Exercise 6.1. VALID is high in cycles 1 to 4 and READY is high only in cycles 3 and 4. How many transfers happen, and in which cycles?

Answer

Two: at the clock edges that end cycles 3 and 4. In cycles 1 and 2 the sender holds its data and waits.

Exercise 6.2. VALID is high in cycles 1 to 6, and READY is 0, 1, 0, 1, 1, 0 in cycles 1 to 6. In which cycles do transfers occur?

Answer

A transfer needs both high: cycles 2, 4 and 5, so three transfers. In cycles 1, 3 and 6 the sender holds its current data.

Exercise 6.3. "AXI is faster than APB because it runs at a higher clock." Is that true?

Answer

Not really. Both can run at the same clock. AXI wins through concurrency: separate read and write channels, bursts that pay the address cost once, and many requests in flight.

Exercise 6.4. Why must an AXI burst of 16 beats of 8 bytes be described by AWLEN = 15 and AWSIZE = 3?

Answer

AWLEN holds the number of beats minus one: \(16 - 1 = 15\). AWSIZE holds the base-2 logarithm of the bytes per beat: \(2^3 = 8\) bytes. The burst moves \(16 \times 8 = 128\) bytes from one address beat.

Exercise 6.5. Address overhead is 4 cycles and each beat costs 1 cycle. Compare the cycles needed to move 16 beats as 16 single transfers and as one burst, and the ratio of throughputs.

Answer

One at a time: \(16 \times (4 + 1) = 80\) cycles. One burst: \(4 + 16 = 20\) cycles. The burst has \(80/20 = 4\) times the throughput.

Exercise 6.6. A memory answers 24 cycles after a request and accepts a new request every cycle. A manager can have 6 requests outstanding. What is its throughput, and how many must be outstanding to reach one answer per cycle?

Answer

By Little's law, throughput \(= 6/24 = 0.25\) answers per cycle. To reach 1 per cycle it needs \(1 \times 24 = 24\) requests in flight.

Exercise 6.7. Using the memory map in Section 6.1, which subordinate receives a store to 0x1000_0800? To 0x1001_1008? To 0x8000_1000? What should the fabric do with an access to 0x1002_0000?

Answer

0x1000_0800 lies in 0x1000_0000 to 0x1000_0FFF: the interrupt controller. 0x1001_1008 lies in 0x1001_1000 to 0x1001_1FFF: SPI0. 0x8000_1000 lies in the DRAM range. 0x1002_0000 lies in no range, so no subordinate claims it; the fabric should answer with an error response (in AXI, a decode error, DECERR) rather than hang.

Exercise 6.8. In a 4 × 4 mesh with XY routing, list the routers a packet visits from router (1,3) to router (3,0), writing routers as (x,y).

Answer

Along X first: (1,3), (2,3), (3,3). Then along Y: (3,2), (3,1), (3,0). That is five hops between six routers.

Exercise 6.9. Give one reason a large SoC uses an APB bus for its UART and SPI controllers while using a crossbar or NoC for CPU, memory and accelerators.

Answer

Peripheral registers are accessed rarely and slowly, so the cheap, simple, low-power APB with its two-phase access suffices, and a single bridge connects it to the backbone. The CPU, memory and accelerators exchange large volumes of traffic that needs the parallelism of a crossbar or NoC, and spending wires and gates there is where the bandwidth is.