Two questions that need different answers
When someone asks whether an FPGA design is fast, I want to know what “fast” means for the application. Does one result need to arrive quickly, or must the system process a large stream without falling behind?
Latency is the time between a defined input event and its corresponding output event. Throughput is the rate of completed work. A design can deliver high throughput while taking many cycles to produce the result for one particular input.
For a streaming interface, I usually define latency from accepted input to valid output, or to accepted output if downstream waiting is included. Stating the measurement points avoids confusion about queues and backpressure.
A 20-cycle pipeline can accept one sample per clock
Consider a pipeline with a defined input-to-output latency of 20 clock cycles and an initiation interval of one cycle. The initiation interval, often called II, is the spacing between new operations that the pipeline can start.
Assume the source supplies an input every cycle, the destination accepts every result, and there are no stalls. Sample A is accepted at clock edge 0 and its result becomes valid at edge 20. B is accepted at edge 1 and completes at edge 21. C enters at edge 2 and completes at edge 22.
The samples overlap inside the pipeline. Different stages work on different samples at the same time. Once the pipeline is full, it produces one result every cycle even though each result took 20 cycles to travel through it.
Put both measures into real units
At 100 MHz, one clock period is 10 ns. The example therefore has 200 ns latency and a maximum steady-state throughput of 100 million samples per second, assuming one sample per transaction.
The simple relationships are:
- Latency in seconds = latency in cycles × clock period.
- Steady-state throughput = clock frequency ÷ initiation interval, for one sample per operation.
If II is four cycles at the same clock, the maximum rate is 25 million samples per second. That does not tell me the latency; it is a separate property of the implementation.
For a finite uninterrupted batch of N samples with latency L and II of one, the last output occurs L + N − 1 cycles after the first input under this edge convention. Startup and drain matter when measuring short bursts.
What pipelining improves, and what it may cost
Registers can split a long combinational path and make a higher clock frequency achievable. They also allow independent operations to overlap. The cost includes extra registers, control alignment, and often more latency measured in cycles.
More cycles do not automatically mean more time. A hypothetical 10-cycle design at 50 MHz and a 20-cycle design at 100 MHz both have 200 ns latency. If both achieve II of one, the second has twice the steady-state throughput. These are teaching examples, not measured project results.
In a low-latency system, throughput alone is not enough. I examine the input-to-output delay in nanoseconds and the latency added by queues, arbitration, buffering, and external interfaces. In a video stream, sustained throughput may be the first requirement, while a control loop can be more sensitive to response delay.
Valid gaps and backpressure change the observed result
The ideal rate assumes a new accepted input every cycle. If valid is low, there is no new transaction. If ready is low in a ready/valid interface, the transfer has not been accepted. I count accepted work rather than clock edges alone.
A pipeline may insert bubbles, stall all stages, or hold completed results in an output queue. Its behavior must be defined. Under backpressure, input-to-accepted-output latency can increase, and sustained throughput is limited by the slower source, processing stage, or destination.
A data dependency can also prevent II of one. A shared multiplier, memory-port limit, or feedback recurrence may require several cycles before another operation can start. Twenty stages alone do not prove one-sample-per-cycle performance.
How I verify latency and throughput
I tag or track accepted inputs, compare each output with the matching expected result, and record the relevant clock edges. For a fixed-latency contract, I check that valid and data arrive after the specified delay. For variable latency, I use a scoreboard and the interface’s completion rules.
I test continuous traffic, isolated samples, bursts, valid gaps, stalls, and reset. I measure both first-result latency and sustained output spacing, and check that no transaction is lost, duplicated, or reordered unexpectedly.
A useful performance statement might be: “20-cycle input-to-valid-output latency, II = 1, at 100 MHz, without stalls.” It communicates much more than calling the design fast. AMD’s pipeline documentation describes the same distinction between latency and initiation interval.