From patchwork Fri Jan 5 14:57:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 123530 Delivered-To: patch@linaro.org Received: by 10.140.22.227 with SMTP id 90csp911195qgn; Fri, 5 Jan 2018 06:58:32 -0800 (PST) X-Google-Smtp-Source: ACJfBosW8PAgU+GY6tdhFkOWPDMzcsm5PO6pvewSWB3caamVUiTTUmj6Kjux/1QHVsy8caxNEBLR X-Received: by 10.159.241.140 with SMTP id s12mr3355844plr.297.1515164312511; Fri, 05 Jan 2018 06:58:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1515164312; cv=none; d=google.com; s=arc-20160816; b=gKtrodMSBf/jNiiYrjneooqjZ9aLa7DmjQbFfEh55WhjeVxGH27Zejl1z6y4tVVEGR KdkcROLvz3Px1KYQD7//sOt4vkgafdBRlUV+nFlG/Y7cGzBMFFCeJNiyBtzGeKykFM5u z2U5O7hOr2TfwYYUDfZp9ctQZCRoD8kDeCOKab3ypwrTpSmaqQE1yB4R1sdEcnsppqmJ KZK3UDsyME8adM4vNmLkveDwOdrLZPaxpA5grSUmiditMr7rTKxisGB5uGb738toRnN0 Ljsp9NYfEzk8o+zGBjM0mhAxqD0IoeYoAc2X7MirjxRP4xyW0G6eNKI+xqRI5RWXg4hg nIiQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:arc-authentication-results; bh=acDhIT+STMBo9is2Ni5mTehLq9lIFU4oTA/Bek7nWaU=; b=T0bA6wbacdMZdy67nTX8DJTzv1ce7ZLWY+G6EUr6UGUYYozGTkuTbqIglXLXv+mlu8 pOE0OVSVC6WOKwYMZT5QnBjM4UzGbfbSRZaTPJC7oac+LOhvAWp5Lccn3/aZJjL6pUZC yhCuPv7dw35L3UcQTyftv4ny3PgB/KRRaOrm+ooMtl/AUZX57rWeuiIUJN7zJcEIaICE GxloD0nUJCcPa0Dtw4cc8tH/ElFpjC5E+ikMu73Orre+Cjq7UhqGJVS6VnZ1bkBDOJt+ gqEB9Sc/v3d1OKd6LcSAIQ6D3qWMKcf/cqG2DhjckxJdYbtDKDjbuNisGNKKG/wHdG/I VjHQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n17si3624402pgc.536.2018.01.05.06.58.32; Fri, 05 Jan 2018 06:58:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752277AbeAEO6a (ORCPT + 26 others); Fri, 5 Jan 2018 09:58:30 -0500 Received: from foss.arm.com ([217.140.101.70]:46092 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752214AbeAEO6M (ORCPT ); Fri, 5 Jan 2018 09:58:12 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 82CEA1610; Fri, 5 Jan 2018 06:58:11 -0800 (PST) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 9366A3F581; Fri, 5 Jan 2018 06:58:09 -0800 (PST) From: Mark Rutland To: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org Cc: dan.j.williams@intel.com, elena.reshetova@intel.com, corbet@lwn.net, alan@linux.intel.com, peterz@infradead.org, will.deacon@arm.com, gregkh@linuxfoundation.org, tglx@linutronix.de, Mark Rutland Subject: [RFCv2 4/4] bpf: inhibit speculated out-of-bounds pointers Date: Fri, 5 Jan 2018 14:57:50 +0000 Message-Id: <20180105145750.53294-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180105145750.53294-1-mark.rutland@arm.com> References: <20180105145750.53294-1-mark.rutland@arm.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Note: this patch is an *example* use of the nospec API. It is understood that this is incomplete, etc. Under speculation, CPUs may mis-predict branches in bounds checks. Thus, memory accesses under a bounds check may be speculated even if the bounds check fails, providing a primitive for building a side channel. The EBPF map code has a number of such bounds-checks accesses in map_lookup_elem implementations. This patch modifies these to use the nospec helpers to inhibit such side channels. The JITted lookup_elem implementations remain potentially vulnerable, and are disabled (with JITted code falling back to the C implementations). Signed-off-by: Mark Rutland Signed-off-by: Will Deacon Cc: Dan Williams Cc: Peter Zijlstra --- kernel/bpf/arraymap.c | 20 +++++++++++++------- kernel/bpf/cpumap.c | 5 ++--- kernel/bpf/devmap.c | 3 ++- kernel/bpf/sockmap.c | 3 ++- 4 files changed, 19 insertions(+), 12 deletions(-) -- 2.11.0 diff --git a/kernel/bpf/arraymap.c b/kernel/bpf/arraymap.c index 7c25426d3cf5..deaad334a100 100644 --- a/kernel/bpf/arraymap.c +++ b/kernel/bpf/arraymap.c @@ -117,15 +117,20 @@ static void *array_map_lookup_elem(struct bpf_map *map, void *key) { struct bpf_array *array = container_of(map, struct bpf_array, map); u32 index = *(u32 *)key; + void *ptr, *high; if (unlikely(index >= array->map.max_entries)) return NULL; - return array->value + array->elem_size * index; + ptr = array->value + array->elem_size * index; + high = array->value + array->elem_size * array->map.max_entries; + + return nospec_ptr(ptr, array->value, high); } /* emit BPF instructions equivalent to C code of array_map_lookup_elem() */ -static u32 array_map_gen_lookup(struct bpf_map *map, struct bpf_insn *insn_buf) +static u32 __maybe_unused array_map_gen_lookup(struct bpf_map *map, + struct bpf_insn *insn_buf) { struct bpf_insn *insn = insn_buf; u32 elem_size = round_up(map->value_size, 8); @@ -153,11 +158,14 @@ static void *percpu_array_map_lookup_elem(struct bpf_map *map, void *key) { struct bpf_array *array = container_of(map, struct bpf_array, map); u32 index = *(u32 *)key; + void __percpu *pptr; if (unlikely(index >= array->map.max_entries)) return NULL; - return this_cpu_ptr(array->pptrs[index]); + pptr = nospec_array_ptr(array->pptrs, index, array->map.max_entries); + + return this_cpu_ptr(pptr); } int bpf_percpu_array_copy(struct bpf_map *map, void *key, void *value) @@ -302,7 +310,6 @@ const struct bpf_map_ops array_map_ops = { .map_lookup_elem = array_map_lookup_elem, .map_update_elem = array_map_update_elem, .map_delete_elem = array_map_delete_elem, - .map_gen_lookup = array_map_gen_lookup, }; const struct bpf_map_ops percpu_array_map_ops = { @@ -610,8 +617,8 @@ static void *array_of_map_lookup_elem(struct bpf_map *map, void *key) return READ_ONCE(*inner_map); } -static u32 array_of_map_gen_lookup(struct bpf_map *map, - struct bpf_insn *insn_buf) +static u32 __maybe_unused array_of_map_gen_lookup(struct bpf_map *map, + struct bpf_insn *insn_buf) { u32 elem_size = round_up(map->value_size, 8); struct bpf_insn *insn = insn_buf; @@ -644,5 +651,4 @@ const struct bpf_map_ops array_of_maps_map_ops = { .map_fd_get_ptr = bpf_map_fd_get_ptr, .map_fd_put_ptr = bpf_map_fd_put_ptr, .map_fd_sys_lookup_elem = bpf_map_fd_sys_lookup_elem, - .map_gen_lookup = array_of_map_gen_lookup, }; diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index ce5b669003b2..6769a0e30c8c 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -551,13 +551,12 @@ void cpu_map_free(struct bpf_map *map) struct bpf_cpu_map_entry *__cpu_map_lookup_elem(struct bpf_map *map, u32 key) { struct bpf_cpu_map *cmap = container_of(map, struct bpf_cpu_map, map); - struct bpf_cpu_map_entry *rcpu; if (key >= map->max_entries) return NULL; - rcpu = READ_ONCE(cmap->cpu_map[key]); - return rcpu; + return READ_ONCE(*nospec_array_ptr(cmap->cpu_map, key, + map->max_entries)); } static void *cpu_map_lookup_elem(struct bpf_map *map, void *key) diff --git a/kernel/bpf/devmap.c b/kernel/bpf/devmap.c index ebdef54bf7df..5a1050d270a0 100644 --- a/kernel/bpf/devmap.c +++ b/kernel/bpf/devmap.c @@ -254,7 +254,8 @@ struct net_device *__dev_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - dev = READ_ONCE(dtab->netdev_map[key]); + dev = READ_ONCE(*nospec_array_ptr(dtab->netdev_map, key, + map->max_entries)); return dev ? dev->dev : NULL; } diff --git a/kernel/bpf/sockmap.c b/kernel/bpf/sockmap.c index 5ee2e41893d9..e912de3cd4ce 100644 --- a/kernel/bpf/sockmap.c +++ b/kernel/bpf/sockmap.c @@ -630,7 +630,8 @@ struct sock *__sock_map_lookup_elem(struct bpf_map *map, u32 key) if (key >= map->max_entries) return NULL; - return READ_ONCE(stab->sock_map[key]); + return READ_ONCE(*nospec_array_ptr(stab->sock_map, key, + map->max_entries)); } static int sock_map_delete_elem(struct bpf_map *map, void *key)