PCI Latency Timer Howto

By Eric Seppanen
Monday, May 7, 2001

The question was recently asked on the linuxbios mailing list,

> are there recommended algorithms for setting the timers? just pick some
> nice number? I've never been sure what to do with those timers.

And since I knew a bit about them but they weren't terribly well documented, I wrote this response. In general this is meant for people writing PCI configuration software. It probably won't help you fix your broken PC.

PCI latency timers are a mechanism for PCI bus-mastering devices to share the PCI bus fairly. "Fair" in this case means that devices won't use such a large portion of the available PCI bus bandwidth that other devices aren't able to get needed work done.

How this works is that each PCI device that can operate in bus-master mode is required to implement a timer, called the Latency Timer, that limits the time that device can hold the PCI bus. The timer starts when the device gains bus ownership, and counts down at the rate of the PCI clock. When the counter reaches zero, the device is required to release the bus. If no other devices are waiting for bus ownership, it may simply grab the bus again and transfer more data.

If the latency timer is set too low, PCI devices will interrupt their transfers unnecessarily often, hurting performance. If it's set too high, devices that require frequent bus access may overflow their buffers, losing data.

So in theory there's a compromise that can be reached somewhere in between, where all devices can get good performance plus a reasonable guarantee that they will get bus access on a timely basis.

Here's my best impression of the various ways you can handle the timers:

The Bad Way: Don't set them at all (in all cases I've looked at this results in a setting of zero).

The Better Way: just pick a number (I use 48 in my code) and set all devices and bridges to that.

The Best Way, in theory, though I doubt anybody does it:

For each card, read the MIN_GNT and MAX_LAT values, and calculate the following:

Then, calculate the desired latency timer values for each device such that:

  1. each card's latency timer should be greater than that card's min_grant_clocks.
  2. each card's latency timer should be set as high as possible. This is because the latency timer will truncate pci busmaster cycles when it expires, so a too-low setting will hurt performance.
  3. all latency timers should be set low enough that if boards A, B, and C all start doing maximum length transfers, board D will still get the bus before max_latency_clocks has elapsed. So,
    A.latency_timer + B.latency_timer + C.latency_timer < D.max_latency_clocks
    Of course, the device manufacturer might have included themselves in the calculation, and in that case the requirement would be:
    A.latency_timer + B.latency_timer + C.latency_timer + D.latency_timer < D.max_latency_clocks
    which is easier to calculate anyway, since A+B+C+D could simply be called system_max_latency. If it's greater than any card's max_latency_clocks, you might have trouble.

Whew.

Needless to say, factory BIOSes are all over the board on this, and I suspect that most simply choose the "better" option (fixed values).

In truth, rule number 1 seems to get violated all the time without any ill effects. Since everybody else ignores rule #1 and I doubt anybody even tries to satisfy rule #3, the "better" option seems like an acceptable compromise.

Eric


Various Bits of Evidence:
If you want to learn for yourself how latency timers are configured, here are some starting points:
The PCI Local Bus Specification, Revision 2.2, has this to say:

(in section 6.2.4, Miscellaneous Registers)

Latency Timer

This register specifies, in units of PCI bus clocks, the value of the Latency Timer for this PCI bus master (refer to section 3.5.). This register must be implemented as writable by any master that can burst for more than two data phases. This register may be implemented as read-only for devices that burst two or fewer data phases, but the hardwired value must be limited to 16 or less. A typical implementation would be to build the five high-order bits (leaving the bottom three as read-only), resulting in a timer granularity of eight clocks. At RST#, the register must be initialized to 0 (if programmable).

...

MIN_GNT and MAX_LAT

These read-only byte registers are used to specify the device's desired settings for Latency Timer values. For both registers, the value specifies a period of time in units of 1/4 microsecond. Values of 0 indicate that the device has no major requirements for the settings of Latency Timers.

MIN_GNT is used for specifying how long a burst period the device needs assuming a clock rate of 33MHz. MAX_LAT is used for specifying how often the device needs to gain access to the PCI bus.

Devices should specify values that will allow them to most effectively use the PCI bus as well as their internal resources. Values should be chosen assuming that the target does not insert any wait states.

Implementation Example: Choosing MIN_GNT and MAX_LAT

A fast Ethernet controller (100Mbps) has a 64-byte buffer for each transfer direction. Optimal usage of these internal resources is achieved when the device treats each buffer as two 32-byte ping-pong buffers. Each 32-byte buffer has eight DWORDS of data to be transferred, resulting in eight data phases on the PCI bus. These eight data phases translate to 1/4 microsecond at 33MHz, so the MIN_GNT value for this device is "1". When moving data, the device will need to empty or fill a 32-byte buffer every 3.2 µs (assuming a throughput of 10 MB/s). This would correspond to a MAX_LAT value of 12.


The Linux kernel, whenever pci_set_master() is called for a device, checks if the latency timer is less than 16, and if so sets it to 64. The code comments:
/*
 *  If we set up a device for bus mastering, we need to check the latency
 *  timer as certain crappy BIOSes forget to set it properly.
 */

Update added Jan 9, 2002

I recently picked up a book that I'd recommend for anyone that has to deal with these sorts of issues on a regular basis. It's PCI System Architecture by Tom Shanley and Don Anderson, and published by Mindshare, Inc.

The book does the best job I've seen of covering PCI latency in a precise and very readable way. It also pointed out some interesting things:

You can see that these issues aren't exactly getting clarified with time. Unfortunately, this information only reinforces my gut feeling that the "correct" approach is so unclear that the only sane way to set the latency timer registers is to pick a safe, commonly used value (32-64), and just use that for everything.

Eric


..