From patchwork Fri May 4 13:55:33 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 134978 Delivered-To: patch@linaro.org Received: by 10.46.151.6 with SMTP id r6csp81800lji; Fri, 4 May 2018 06:55:50 -0700 (PDT) X-Google-Smtp-Source: AB8JxZpsQ2rLNFszLeVwysqMJdo8VUcuxVj+pfAmBu/I49/lIKsOUUnEvvyQFGzVqWUqm8NG2XgG X-Received: by 2002:a65:44cc:: with SMTP id g12-v6mr22367748pgs.110.1525442150720; Fri, 04 May 2018 06:55:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525442150; cv=none; d=google.com; s=arc-20160816; b=xi2g1ydoL9fKpvM5gxSjcZxZC8pT9nXnGjwVjIIbVhsTJ+Q0iBq/TeAn3Sb0aE4VRH jYRRlCqaRJpb2dEthKOmBs+T0wjUjoaT8a7CEfgFa4vK99KS1LvAFP1Oxukbr7u0MFil 1438I6OSaTwZE/HiwapDTBK4MCb4QghjXu8u8wAnbSORBeiGjSDmCfHOwqCaB2bzdKGH ZeHmDUiAyWxRCtRvmYHRgt8FtNKiwcZnuBFJSTiTPiLzXPex2JM4akxHbHyFvH/LFrJI tOQ6sZw334V+nn22YC6AcDzE07QcPooqWYYTMZQtLeOYanKH2GIOjLII+Zr9ZMjMcONd l7CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=vJCkCLeozTTSpN1PdLfGExEdxJfOA2HLJfltzQIBJ1o=; b=Ydf70vgdQc41x2lNT7CvkI3kNjqQcmLMLAW0ExhFc6SXH5XInZfEeXdd/QQf45RIZA 9644OoFlaJIOI4r8KsE22NYCr2D1DHtU3w64ga5CWu2ZWBcjuqRCL06poSE79RVWAYiH OgFzkk5vuHCDurn/w0YMzSAPpi19bevYaeD1h1vfiCy0cl+X7dWMj3Q2S3j2n3NsIJjn sp+p6mArYI6O17HDEEZkFOqgTGyqDSql+pY8YFk8Ce+rrhYWY7wGy9JBmcN6fGIvgti/ 3b1WAB7id8WtwGgXeU9NXH5eu1H1ZqxhhMaDd1ehIiC0V2asfGsGo03No5+zqfpIgugk 0U1Q== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c2si16578325pfh.215.2018.05.04.06.55.50; Fri, 04 May 2018 06:55:50 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752034AbeEDNzq (ORCPT + 29 others); Fri, 4 May 2018 09:55:46 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:53656 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751573AbeEDNzn (ORCPT ); Fri, 4 May 2018 09:55:43 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5F2111596; Fri, 4 May 2018 06:55:43 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0A67A3F487; Fri, 4 May 2018 06:55:41 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, aryabinin@virtuozzo.com, dvyukov@google.com, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org Subject: [PATCH 1/3] kcov: ensure irq code sees a valid area Date: Fri, 4 May 2018 14:55:33 +0100 Message-Id: <20180504135535.53744-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180504135535.53744-1-mark.rutland@arm.com> References: <20180504135535.53744-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org For kernels built with CONFIG_PREEMPT, some C code may execute before or after the interrupt handler, while the hardirq count is zero. In these cases, in_task() can return true. A task can be interrupted in the middle of a KCOV_DISABLE ioctl while it resets the task's kcov data via kcov_task_init(). Instrumented code executed during this period will call __sanitizer_cov_trace_pc(), and as in_task() returns true, will inspect t->kcov_mode before trying to write to t->kcov_area. In kcov_init_task() Since we update t->kcov_{mode,area,size} with plain stores, which may be re-ordered, torn, etc. Thus __sanitizer_cov_trace_pc() may see bogus values for any of these fields, and may attempt to write to memory which is not mapped. Let's avoid this by using WRITE_ONCE() to set t->kcov_mode, with a barrier() to ensure this is ordered before we clear t->kov_{area,size}. This ensures that any code execute while kcov_init_task() is preempted will either see valid values for t->kcov_{area,size}, or will see that t->kcov_mode is KCOV_MODE_DISABLED, and bail out without touching t->kcov_area. Signed-off-by: Mark Rutland Cc: Andrew Morton Cc: Andrey Ryabinin Cc: Dmitry Vyukov --- kernel/kcov.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.11.0 diff --git a/kernel/kcov.c b/kernel/kcov.c index 2c16f1ab5e10..5be9a60a959f 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -241,7 +241,8 @@ static void kcov_put(struct kcov *kcov) void kcov_task_init(struct task_struct *t) { - t->kcov_mode = KCOV_MODE_DISABLED; + WRITE_ONCE(t->kcov_mode, KCOV_MODE_DISABLED); + barrier(); t->kcov_size = 0; t->kcov_area = NULL; t->kcov = NULL; From patchwork Fri May 4 13:55:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 134980 Delivered-To: patch@linaro.org Received: by 10.46.151.6 with SMTP id r6csp82099lji; Fri, 4 May 2018 06:56:08 -0700 (PDT) X-Google-Smtp-Source: AB8JxZppOpr8lLEx92WlNcuXpT+5YCmqZDH9999jpcy34FcuSEMYC4mD7++xcIFp+B4K5cKARwRx X-Received: by 10.98.65.132 with SMTP id g4mr27188351pfd.51.1525442167867; Fri, 04 May 2018 06:56:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525442167; cv=none; d=google.com; s=arc-20160816; b=oOOOx+7f6ELYFChtqG6jJyfS036euVTOyhKMe9e252zHb95jo/wQlEk9Kvm2mYiEgH vt9/5JT3SRQrSuyf13a3i4Yp/jojVrz12kihYcagea0ExpIKAOOIk8Zw9GmEuDEkGG6x uJiocXkjXDuUkES+c1YRYLlzhq0HltDkefHsMxCj741CRkT6AOwVFhBBMXEtA1Fdd7nc NhUKPWXEDUNzin/jMnDjD7MZyZtbcZIyfs5NoYve0o0V9uIu8OsKXJzYEmqWNPfRPYui FKYjGr+nna3e6+Uj7KT3YOlFI8cSJAdgRq6TQWPLQRvd0xcQhAHJtRIQVKc7cMHxxAo0 yqoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=cTRUh9O/DbaSwfkMjNxEw3Fi6B/IMK0RM1SN8oys07w=; b=aTkxpn43zYdV6Zy2Yxs+DbejkSGMoPuaxhDC/0jsgiUmTcYdOzreMLOJXCmd4Gmy/z 1U3LQfLx5WtcgCwiiJ8UajxD8JXMOmIecgoeQ/8spj1W6mB4py1aBB+ZK8egOEyVBao2 B9H94v4GBIFsWUhrL0yi3etlNl39in9cphslHn3eWNRmIc0SAhMOFzAxxdKJIwYgYANh DpIUg4Gsha7eOUpiBMyoT7PjmXjw4x8W7uyPEdTAXkgvslpqG2O+/HHAKzeB7IIalPzm wRsZLKzaj1JkPofL9O9d62MhCwObWXheH1lSYSO1Atpk7DQrElIOqIZrb5A41X1xy9c9 3txQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b2-v6si4766492pgc.569.2018.05.04.06.56.07; Fri, 04 May 2018 06:56:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752054AbeEDN4F (ORCPT + 29 others); Fri, 4 May 2018 09:56:05 -0400 Received: from foss.arm.com ([217.140.101.70]:53662 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752026AbeEDNzp (ORCPT ); Fri, 4 May 2018 09:55:45 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6A570164F; Fri, 4 May 2018 06:55:45 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 161B23F487; Fri, 4 May 2018 06:55:43 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, aryabinin@virtuozzo.com, dvyukov@google.com, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org Subject: [PATCH 2/3] kcov: prefault the kcov_area Date: Fri, 4 May 2018 14:55:34 +0100 Message-Id: <20180504135535.53744-3-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180504135535.53744-1-mark.rutland@arm.com> References: <20180504135535.53744-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On many architectures the vmalloc area is lazily faulted in upon first access. This is problematic for KCOV, as __sanitizer_cov_trace_pc accesses the (vmalloc'd) kcov_area, and fault handling code may be instrumented. If an access to kcov_area faults, this will result in mutual recursion through the fault handling code and __sanitizer_cov_trace_pc(), eventually leading to stack corruption and/or overflow. We can avoid this by faulting in the kcov_area before __sanitizer_cov_trace_pc() is permitted to access it. Once it has been faulted in, it will remain present in the process page tables, and will not fault again. Signed-off-by: Mark Rutland Cc: Andrew Morton Cc: Andrey Ryabinin Cc: Dmitry Vyukov --- kernel/kcov.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) -- 2.11.0 diff --git a/kernel/kcov.c b/kernel/kcov.c index 5be9a60a959f..3b82f8e258da 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -324,6 +324,17 @@ static int kcov_close(struct inode *inode, struct file *filep) return 0; } +static void kcov_fault_in_area(struct kcov *kcov) +{ + unsigned long stride = PAGE_SIZE / sizeof(unsigned long); + unsigned long *area = kcov->area; + unsigned long offset; + + for (offset = 0; offset < kcov->size; offset += stride) { + READ_ONCE(area[offset]); + } +} + static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd, unsigned long arg) { @@ -372,6 +383,7 @@ static int kcov_ioctl_locked(struct kcov *kcov, unsigned int cmd, #endif else return -EINVAL; + kcov_fault_in_area(kcov); /* Cache in task struct for performance. */ t->kcov_size = kcov->size; t->kcov_area = kcov->area; From patchwork Fri May 4 13:55:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 134979 Delivered-To: patch@linaro.org Received: by 10.46.151.6 with SMTP id r6csp81867lji; Fri, 4 May 2018 06:55:54 -0700 (PDT) X-Google-Smtp-Source: AB8JxZq723//wra/+EsIt/Rknsk8h49Re1wqV4oXAN4IfamoC3veUKxl/yMuPQZZHuGz1CgLzleB X-Received: by 2002:a63:6703:: with SMTP id b3-v6mr8009167pgc.176.1525442154052; Fri, 04 May 2018 06:55:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525442154; cv=none; d=google.com; s=arc-20160816; b=olOpuXWVd4+M7gJa/eMlP2m7QVf7oTqaVYhBLemYMCPAeNMQlx5I409uoQcbGjJk7i GFjaOKnxFNO4czAxrVtDZtJVo3ocIGZAOaPSwL6wqPp3v731EbiZj7ZXMyvPbNVlmKpE 4ver8mjCrHlcT5TNTsbUqHdOqw8K/FE/m2yN37NajT/6m5z/pmVFHjZU2CLGMpAT0SKS LXEYYzX9RJoiE82uDukaoAnmsNRF77fcgkx2gx4XUMKo3ividP6ySplangswjMPVMQ2w yr/CJn/vXlut5NYLw3YIx5i6xDkFd4BsVisAHBI2rEezj+URXDnOMdNVEZlP3Zgasc3U gX3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=RVOLOa834jkWMcgNJrRuz9V5mIwOxVMPFpN7ZfeB6SU=; b=upGqHsyH2ZcZfHMoI6LMV+1Guzj+leu+ZiDd2Jn+73+8IcQrps/Ps7h1USWp24jmIe YZrr5BDHCKiSCpNVwFguCTbyXtc7BlLluK2BY7Byv4XPUKcb9Ps4DDxkJ7l/Aahsen7s PsPaNPvHlNgJ71Wqs5vvCq5/h8UDNf2t6awwg+rf6DsH/nt+cs1v4XvVgWqOZR2KCumA uLUIhv2yhpDG1XG6Pbsuz7qCbEcZqiStAKPsIH86ngZ1Z9k1WohsBn+CXO7fHUZgcz15 1RDzMJkqIcHmrhP0YwVavK1uMthaLQfyoCiMsYPbXa0AutKx4WioZTnIZ8zjaVTp5G/P zM4A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b2-v6si4766492pgc.569.2018.05.04.06.55.53; Fri, 04 May 2018 06:55:54 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752049AbeEDNzu (ORCPT + 29 others); Fri, 4 May 2018 09:55:50 -0400 Received: from usa-sjc-mx-foss1.foss.arm.com ([217.140.101.70]:53680 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751573AbeEDNzr (ORCPT ); Fri, 4 May 2018 09:55:47 -0400 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5A7FA1682; Fri, 4 May 2018 06:55:47 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 0667F3F487; Fri, 4 May 2018 06:55:45 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: akpm@linux-foundation.org, aryabinin@virtuozzo.com, dvyukov@google.com, mark.rutland@arm.com, mingo@redhat.com, peterz@infradead.org Subject: [PATCH 3/3] sched/core / kcov: avoid kcov_area during task switch Date: Fri, 4 May 2018 14:55:35 +0100 Message-Id: <20180504135535.53744-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180504135535.53744-1-mark.rutland@arm.com> References: <20180504135535.53744-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org During a context switch, we first switch_mm() to the next task's mm, then switch_to() that new task. This means that vmalloc'd regions which had previously been faulted in can transiently disappear in the context of the prev task. Functions instrumented by KCOV may try to access a vmalloc'd kcov_area during this window, and as the fault handling code is instrumented, this results in a recursive fault. We must avoid accessing any kcov_area during this window. We can do so with a new flag in kcov_mode, set prior to switching the mm, and cleared once the new task is live. Since task_struct::kcov_mode isn't always a specific enum kcov_mode value, this is made an unsigned int. The manipulation is hidden behind kcov_{prepare,finish}_switch() helpers, which are empty for !CONFIG_KCOV kernels. Signed-off-by: Mark Rutland Cc: Andrew Morton Cc: Andrey Ryabinin Cc: Dmitry Vyukov Cc: Ingo Molnar Cc: Peter Zijlstra --- include/linux/kcov.h | 14 ++++++++++++++ include/linux/sched.h | 2 +- kernel/kcov.c | 2 +- kernel/sched/core.c | 4 ++++ 4 files changed, 20 insertions(+), 2 deletions(-) -- 2.11.0 diff --git a/include/linux/kcov.h b/include/linux/kcov.h index 3ecf6f5e3a5f..b76a1807028d 100644 --- a/include/linux/kcov.h +++ b/include/linux/kcov.h @@ -22,13 +22,27 @@ enum kcov_mode { KCOV_MODE_TRACE_CMP = 3, }; +#define KCOV_IN_CTXSW (1 << 30) + void kcov_task_init(struct task_struct *t); void kcov_task_exit(struct task_struct *t); +#define kcov_prepare_switch(t) \ +do { \ + (t)->kcov_mode |= KCOV_IN_CTXSW; \ +} while (0) + +#define kcov_finish_switch(t) \ +do { \ + (t)->kcov_mode &= ~KCOV_IN_CTXSW; \ +} while (0) + #else static inline void kcov_task_init(struct task_struct *t) {} static inline void kcov_task_exit(struct task_struct *t) {} +static inline void kcov_prepare_switch(struct task_struct *t) {} +static inline void kcov_finish_switch(struct task_struct *t) {} #endif /* CONFIG_KCOV */ #endif /* _LINUX_KCOV_H */ diff --git a/include/linux/sched.h b/include/linux/sched.h index b3d697f3b573..3d9edcf57e21 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1078,7 +1078,7 @@ struct task_struct { #ifdef CONFIG_KCOV /* Coverage collection mode enabled for this task (0 if disabled): */ - enum kcov_mode kcov_mode; + unsigned int kcov_mode; /* Size of the kcov_area: */ unsigned int kcov_size; diff --git a/kernel/kcov.c b/kernel/kcov.c index 3b82f8e258da..f73ab8dc3ba7 100644 --- a/kernel/kcov.c +++ b/kernel/kcov.c @@ -58,7 +58,7 @@ struct kcov { static bool check_kcov_mode(enum kcov_mode needed_mode, struct task_struct *t) { - enum kcov_mode mode; + unsigned int mode; /* * We are interested in code coverage as a function of a syscall inputs, diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 5e10aaeebfcc..52078516803a 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -7,6 +7,8 @@ */ #include "sched.h" +#include + #include #include @@ -2632,6 +2634,7 @@ static inline void prepare_task_switch(struct rq *rq, struct task_struct *prev, struct task_struct *next) { + kcov_prepare_switch(prev); sched_info_switch(rq, prev, next); perf_event_task_sched_out(prev, next); fire_sched_out_preempt_notifiers(prev, next); @@ -2700,6 +2703,7 @@ static struct rq *finish_task_switch(struct task_struct *prev) finish_task(prev); finish_lock_switch(rq); finish_arch_post_lock_switch(); + kcov_finish_switch(current); fire_sched_in_preempt_notifiers(current); /*