From patchwork Tue Oct 23 18:17:07 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 149463 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp1032278ljp; Tue, 23 Oct 2018 11:19:40 -0700 (PDT) X-Google-Smtp-Source: ACcGV638dQHxOEp10/Z44ejj6aeiIj4QVDXs0XIJkTWII/RZs9JoOF+J0yG4i0FFfzOcTKH9JwfQ X-Received: by 2002:a25:21d7:: with SMTP id h206-v6mr37258890ybh.16.1540318780169; Tue, 23 Oct 2018 11:19:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540318780; cv=none; d=google.com; s=arc-20160816; b=D4LKTNPc0iN6J5BCJp0GN/GWKgTSpqbmyfkJJQUaDFjFQKI7LWaVoPtcnG8yUI6W06 +4SC201AyQoHJFh6wYI8pdRA5/ApAz1daWVDptCYylqtRfCIo1MhN9FZJdHEhzCUcsjs PpR/GAqBmE0v3KxfSlVHBrncttm4YDF/IHGsKgs92kHeUwhtQLTiFMCXZiA0/HHRR5FP BbQ00sTsgOl+hABc8tTfjKGifrWTLEk1aEHEcDPmL+SqJnF6yk9Hbk+Zi8hZWZsnxa+N gzx3SVZYULBDc7GGuow2q+8IKYVMHrKjYGElTmquzMZXgIcB13ZqkHTvG04nQlkqxCEO hP1A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from; bh=w+A9x2To7WLLw4oFD75Pu6RJiryrp5G1ckkIivDw+dM=; b=zd6E2rA9LsWoFrkSZNE68ALNIxRvHLADBBnkhANDMHE9/hSY+v605rMyqgkqz2eLTP StDVH+hZRWcAnjh5qyVchAOCicRcLHkp1UsDT/6ow/PhkSdTRtmHflrsZ0o/YXuZJBxm c3avqMhYnbtn8Etcl2PqFdQlQXD61efL8HCsbt2EIyKmdRNATIDjAX1nPe1sB8rmBFBS 4p36Or/s4SXdzBEoFO4n+o+KwztZ5p9KmBlfqsuDiYv7wFT2azyg5YKtRilS1CL6hgUa gK63cyO5RQB6QOYnThaVlPPPmeyBN4qh1FHhzRH7Szyh07x2dCguSUxO4HTHqIG1j4BF C1bQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id k186-v6si1384190ywb.357.2018.10.23.11.19.39 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 23 Oct 2018 11:19:40 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1gF1F4-0006P1-Ee; Tue, 23 Oct 2018 18:17:22 +0000 Received: from us1-rack-dfw2.inumbo.com ([104.130.134.6]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1gF1F3-0006Og-Us for xen-devel@lists.xen.org; Tue, 23 Oct 2018 18:17:21 +0000 X-Inumbo-ID: 78e08c14-d6ef-11e8-a8a5-bc764e045a96 Received: from foss.arm.com (unknown [217.140.101.70]) by us1-rack-dfw2.inumbo.com (Halon) with ESMTP id 78e08c14-d6ef-11e8-a8a5-bc764e045a96; Tue, 23 Oct 2018 20:14:25 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9969B15AB; Tue, 23 Oct 2018 11:17:19 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.196.50]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id AE1D53F5D3; Tue, 23 Oct 2018 11:17:18 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xen.org Date: Tue, 23 Oct 2018 19:17:07 +0100 Message-Id: <20181023181709.11883-3-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20181023181709.11883-1-julien.grall@arm.com> References: <20181023181709.11883-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH 2/4] xen/arm: gic: Ensure ordering between read of INTACK and shared data X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andre.przywara@arm.com, Julien Grall , sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" When an IPI is generated by a CPU, the pattern looks roughly like: dsb(sy); On the receiving CPU we rely on the fact that, once we've taken the interrupt, then the freshly written shared data must be visible to us. Put another way, the CPU isn't going to speculate taking an interrupt. Unfortunately, this assumption turns out to be broken. Consider that CPUx wants to send an IPI to CPUy, which will cause CPUy to read some shared_data. Before CPUx has done anything, a random peripheral raises an IRQ to the GIC and the IRQ line on CPUy is raised. CPUy then takes the IRQ and starts executing the entry code, heading towards gic_handle_irq. Furthermore, let's assume that a bunch of the previous interrupts handled by CPUy were SGIs, so the branch predictor kicks in and speculates that irqnr will be <16 and we're likely to head into handle_IPI. The prefetcher then grabs a speculative copy of shared_data which contains a stale value. Meanwhile, CPUx gets round to updating shared_data and asking the GIC to send an SGI to CPUy. Internally, the GIC decides that the SGI is more important than the peripheral interrupt (which hasn't yet been ACKed) but doesn't need to do anything to CPUy, because the IRQ line is already raised. CPUy then reads the ACK register on the GIC, sees the SGI value which confirms the branch prediction and we end up with a stale shared_data value. This patch fixes the problem by adding an smp_rmb() to the IPI entry code in do_SGI. At the same time document the write barrier. Based on Linux commit f86c4fbd930ff6fecf3d8a1c313182bd0f49f496 "irqchip/gic: Ensure ordering between read of INTACK and shared data". Signed-off-by: Julien Grall Reviewed-by: Andrii Anisov Reviewed-by: Stefano Stabellini --- This patch is candidate for backporting up to Xen 4.9. --- xen/arch/arm/gic.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/xen/arch/arm/gic.c b/xen/arch/arm/gic.c index 305fbd66dd..30c0fba0d7 100644 --- a/xen/arch/arm/gic.c +++ b/xen/arch/arm/gic.c @@ -300,6 +300,11 @@ void send_SGI_mask(const cpumask_t *cpumask, enum gic_sgi sgi) { ASSERT(sgi < 16); /* There are only 16 SGIs */ + /* + * Ensure that stores to Normal memory are visible to the other CPUs + * before issuing the IPI. + * Matches the read barrier in do_sgi. + */ dsb(sy); gic_hw_ops->send_SGI(sgi, SGI_TARGET_LIST, cpumask); } @@ -313,6 +318,11 @@ void send_SGI_self(enum gic_sgi sgi) { ASSERT(sgi < 16); /* There are only 16 SGIs */ + /* + * Ensure that stores to Normal memory are visible to the other CPUs + * before issuing the IPI. + * Matches the read barrier in do_sgi. + */ dsb(sy); gic_hw_ops->send_SGI(sgi, SGI_TARGET_SELF, NULL); } @@ -321,6 +331,11 @@ void send_SGI_allbutself(enum gic_sgi sgi) { ASSERT(sgi < 16); /* There are only 16 SGIs */ + /* + * Ensure that stores to Normal memory are visible to the other CPUs + * before issuing the IPI. + * Matches the read barrier in do_sgi. + */ dsb(sy); gic_hw_ops->send_SGI(sgi, SGI_TARGET_OTHERS, NULL); } @@ -356,6 +371,13 @@ static void do_sgi(struct cpu_user_regs *regs, enum gic_sgi sgi) /* Lower the priority */ gic_hw_ops->eoi_irq(desc); + /* + * Ensure any shared data written by the CPU sending + * the IPI is read after we've read the ACK register on the GIC. + * Matches the write barrier in send_SGI_* helpers. + */ + smp_rmb(); + switch (sgi) { case GIC_SGI_EVENT_CHECK: