From patchwork Thu May 10 13:06:32 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 135413 Delivered-To: patch@linaro.org Received: by 10.46.151.6 with SMTP id r6csp990428lji; Thu, 10 May 2018 06:06:50 -0700 (PDT) X-Google-Smtp-Source: AB8JxZoGK8srXq1JDFM08KADeXvTqR0vO6ey6FYExIJ/DhcNIC4Z3MFG1dseycSYWYwJD/kr5JjW X-Received: by 2002:a62:2f44:: with SMTP id v65-v6mr1337995pfv.83.1525957609839; Thu, 10 May 2018 06:06:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525957609; cv=none; d=google.com; s=arc-20160816; b=JbGPnoKRuqNwIdhgPO4iBHhG8ry+u1nsy5GywL7tq5NTf49C4Mfl3LiquF7gBGf18X Pj5Jv89rHcLy+brC/lUgCwLlMguHKlRGOUt58aHV/W7ED5MuF+kLyjTsUokg7Ff7U3vz 9FjXNJPm9dSc9pvw0+yMkxOuNtDYA0l8izq07eg6leIKGgKYJbfVQSnbq9e/mPAEOC3X 4P2M4/V7ZUbSRyy9a34k/60mUpmbOKSvq+FeNLwh287x/NhdMiDuKuCWSp1qk32j71W6 x63LbrBOkNrBZqe36C0Q/MSHJvrB8soBJ0p1Sk2DIHj7JrRimrFcD+0EGMGFZU3Ms8gi 0QhA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :arc-authentication-results; bh=BRvOicauNAScp8u9/rZn+iuSYPDsNoB1h9tgtkfHPAs=; b=oMES0SdCakufKRk/X0RtYICMDV408C5sVt9nevxbOQBDqFSa+hWStBXSnJOPM3VKbi iupQm121Goa0fAMlQ/QSnKGs6Bc2iBJQD0JV4h4rnOunMk5SDLGhnsREPADYz2Isi85Y 3Ar9LpAGI47DjuvpbP8MWeX6zymAoObkBIcCU6IWkkkib7yvvka1xVexkHRqbx+rh/c6 P4QI8DGDUpML2SFFuB4M4egQ/4WV+RkBpJY9658GVCmrRZxgE6iTi5ehfKh9FExe6hK6 LGpJgIlSqmS/0KW469q73k8WXk/5a3rqyWnbHsZnjhdAg+YlqIHyGJ+wxsEXMMG7K1t9 PGUA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j3-v6si766673pld.300.2018.05.10.06.06.49; Thu, 10 May 2018 06:06:49 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757124AbeEJNGr (ORCPT + 29 others); Thu, 10 May 2018 09:06:47 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:55676 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756808AbeEJNGq (ORCPT ); Thu, 10 May 2018 09:06:46 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id A54A280D; Thu, 10 May 2018 06:06:45 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9495D3F592; Thu, 10 May 2018 06:06:44 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: Mark Rutland , Ingo Molnar , Peter Zijlstra , Will Deacon Subject: [PATCH] perf/ring_buffer: ensure atomicity and order of updates Date: Thu, 10 May 2018 14:06:32 +0100 Message-Id: <20180510130632.34497-1-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Userspace can read/write the user page at any point in time, and in perf_output_put_handle() we're very careful to use memory barriers to ensure ordering between updates to data and the user page. We don't use barriers when updating aux_head, where similar ordering constraints apply. This could result in userspace seeing stale data, or data being overwritten while userspace was still consuming it. Further, we update data_head and aux_head with plain assignments, which the compiler can tear, potentially resulting in userspace seeing erroneous values. We can solve both of these problems by using smp_store_release to update data_head and aux_head, so let's do so. Signed-off-by: Mark Rutland Cc: Ingo Molnar Cc: Peter Zijlstra Cc: Will Deacon --- kernel/events/ring_buffer.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-) -- 2.11.0 diff --git a/kernel/events/ring_buffer.c b/kernel/events/ring_buffer.c index 6c6b3c48db71..839b207e4c77 100644 --- a/kernel/events/ring_buffer.c +++ b/kernel/events/ring_buffer.c @@ -63,10 +63,10 @@ static void perf_output_put_handle(struct perf_output_handle *handle) * kernel user * * if (LOAD ->data_tail) { LOAD ->data_head - * (A) smp_rmb() (C) + * (A) smp_rmb() (C) * STORE $data LOAD $data - * smp_wmb() (B) smp_mb() (D) - * STORE ->data_head STORE ->data_tail + * smp_mb() (D) + * RELEASE ->data_head (B) STORE ->data_tail * } * * Where A pairs with D, and B pairs with C. @@ -83,8 +83,7 @@ static void perf_output_put_handle(struct perf_output_handle *handle) * * See perf_output_begin(). */ - smp_wmb(); /* B, matches C */ - rb->user_page->data_head = head; + smp_store_release(&rb->user_page->data_head, head); /* B, matches C */ /* * Now check if we missed an update -- rely on previous implied @@ -464,7 +463,7 @@ void perf_aux_output_end(struct perf_output_handle *handle, unsigned long size) handle->aux_flags); } - rb->user_page->aux_head = rb->aux_head; + smp_store_release(&rb->user_page->aux_head, rb->aux_head); if (rb_need_aux_wakeup(rb)) wakeup = true; @@ -496,7 +495,7 @@ int perf_aux_output_skip(struct perf_output_handle *handle, unsigned long size) rb->aux_head += size; - rb->user_page->aux_head = rb->aux_head; + smp_store_release(&rb->user_page->aux_head, rb->aux_head); if (rb_need_aux_wakeup(rb)) { perf_output_wakeup(handle); handle->wakeup = rb->aux_wakeup + rb->aux_watermark;