next up previous
Next: FreeBSD's MSI Implementation Up: PCI Interrupts for x86 Previous: FreeBSD's INTx Implementation

PCI Message Signaled Interrupts

Legacy PCI INTx interrupts work, but they have several limitations. First, each PCI function is only allowed a single interrupt. Second, PCI INTx interrupts use a separate signal from the address and data lines used for PCI data transactions.

The single interrupt restriction can be a bottleneck for high-performance devices. For example, on some Ethernet adapters, the transmit and receive units run in parallel, but a single interrupt forces the driver to process events from the two units serially. Another case where a device can benefit from multiple handlers is a device that generates interrupts for specific performance-critical events very often while also generating interrupts for other events less often. An Ethernet device can be an example of this as well. A busy Ethernet device will generate several receive and transmit completion interrupts while handling traffic, and it can also generate other interrupts for events like link status changes. If the device were able to split out the receive and transmit completion events into dedicated interrupts, those interrupt handlers could be smaller and faster allowing for less overhead for those events.

Using a separate signal from the normal address and data lines for PCI INTx interrupts raises several issues. First, on many x86 systems this requires separate physical traces on the motherboard to connect the signals to interrupt controller input pins. Second, the platform and operating system have to work together to route the interrupts. The largest issue, however, is that by using separate signals, the interrupt may be raised on the CPU before all of the effects of the event that triggered the interrupt are visible to the CPU. As a result, all PCI device driver interrupt handlers must begin with a read from a register on the PCI device. This read will not be completed until any pending transactions in between the CPU and the PCI device complete, and thus guarantees that all the effects of the event that triggered the interrupt will be visible to the CPU. This adds extra latency and work to the interrupt handler even if the handler doesn't need to read a PCI register. For example, if an Ethernet device had dedicated interrupt handlers for receive and transmit completion events, those handlers only need to walk the descriptor rings in RAM in the common case. Forcing them to start with a dummy read would just add overhead and latency.

Starting with PCI 2.2, a new interrupt mechanism known as Message Signaled Interrupts (MSI) was introduced to address these concerns. With MSI, each PCI function can have one or more interrupt messages. Each message has associated address and data registers whose values are assigned by the operating system. When a PCI function asserts an interrupt using MSI, it performs a PCI write operation that writes the value of the data register to the address specified in the address register. The platform must ensure that something is listening for writes to the addresses used by MSI messages and translate them into interrupt requests to one or more CPUs.

The format of the message address and data fields is platform-specific. For x86 platforms, the message data contains the IDT vector to trigger when the interrupt occurs. Thus, MSI interrupts are able to bypass the entire interrupt routing maze, and the operating system can directly link MSI interrupts to IDT vectors.


next up previous
Next: FreeBSD's MSI Implementation Up: PCI Interrupts for x86 Previous: FreeBSD's INTx Implementation