From e9e852af347ae3ccee4e7abb01f9ef91387980f9 Mon Sep 17 00:00:00 2001 From: Jonathan Bell Date: Wed, 6 Nov 2024 11:07:55 +0000 Subject: [PATCH] drivers: usb: xhci: prevent a theoretical race on non-coherent platforms For platforms that have xHCI controllers attached over PCIe, and non-coherent routes to main memory, a theoretical race exists between posting new TRBs to a ring, and writing to the doorbell register. In a contended system, write traffic from the CPU may be stalled before the memory controller, whereas the CPU to Endpoint route is separate and not likely to be contended. Similarly, the DMA route from the endpoint to main memory may be separate and uncontended. Therefore the xHCI can receive a doorbell write and find a stale view of a transfer ring. In cases where only a single TRB is ping-ponged at a time, this can cause the endpoint to not get polled at all. Adding a readl() before the write forces a round-trip transaction across PCIe, definitively serialising the CPU along the PCI producer-consumer ordering rules. Signed-off-by: Jonathan Bell --- drivers/usb/host/xhci-ring.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) --- a/drivers/usb/host/xhci-ring.c +++ b/drivers/usb/host/xhci-ring.c @@ -505,6 +505,19 @@ void xhci_ring_ep_doorbell(struct xhci_h trace_xhci_ring_ep_doorbell(slot_id, DB_VALUE(ep_index, stream_id)); + /* + * For non-coherent systems with PCIe DMA (such as Pi 4, Pi 5) there + * is a theoretical race between the TRB write and barrier, which + * is reported complete as soon as the write leaves the CPU domain, + * the doorbell write, which may be reported as complete by the RC + * at some arbitrary point, and the visibility of new TRBs in system + * RAM by the endpoint DMA engine. + * + * This read before the write positively serialises the CPU state + * by incurring a round-trip across the link. + */ + readl(db_addr); + writel(DB_VALUE(ep_index, stream_id), db_addr); /* flush the write */ readl(db_addr);