From patchwork Fri Nov 23 22:18:03 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 151922 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp2699838ljp; Fri, 23 Nov 2018 14:18:30 -0800 (PST) X-Google-Smtp-Source: AFSGD/UFrQEKZ1ruEsEOoyMNyXcFezrPsLSw7GLn0nKzPRMWbWpDdwpU01LmX2HdUnGz+MYIjerT X-Received: by 2002:a63:5f95:: with SMTP id t143mr15928798pgb.395.1543011510675; Fri, 23 Nov 2018 14:18:30 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543011510; cv=none; d=google.com; s=arc-20160816; b=0XJN8Jp/RZqMd0MVeyTRHRCJ+KbU2TW5PBtO/H+CNw3aXfNlpEIj3x2K+UETve5ixL nvatUETmlNj51ye/1XHvyjx16HthUBue12EPNP8+RO/06Zw5GiuEqJz/dPgLv/YeNc+4 aPdhI4ebNedVbtI4wY2hfJlRc2vPsR6jT7cRMJyjm9iR1C0K7iH48n6181BUJ9zsaC9d g4KFRmGlXhdalomnBxSjPpuug3fa7a782rJSt9a7mv/ftunGIXVXsnIShrNkCyVyeons p2AbdYzuQz0bYrNbhLmRYhGh0LyNZR8LlQH9gEtx7lX2nNUYZvgWGWcBMtXJEm7KaEq0 fNtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=2t2DKZwLo4uiuBiBMlCY8fawnGcKKtVedWhh8SZ5pW8=; b=kEAMFumPSG9EpqLqAci0YuMNKUDY+jPWcQP+vNHBggllaxgWfcivtXXtALo/9Frjdc qPtF/E+ZKEqNr74kjIDmhzZHMlIJPVVPbhZEO58odfDcEuBKQxm144SOcmXASkfzW7kT PYTFeWtCBSpGc+Yo6rsXdwlm4POPvNyo+1W7TY85PRgQnhJwdZFrPz3rVs1kl+fdooIb HsBUBsNWs812/xFZdK1r2ysNxEH7uOELjUOL57DUlf3/95ktdqFD6IB5WdMCTxA7j1Ht Jfkd2eZfdoli4bGNvd2CjHfzMQDnPoJpF7Fc4u7kCRjtZCjKG2PXnwTkVH4xXaIPzS2o aRQg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="fEXGYNj/"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a11si51459722pga.198.2018.11.23.14.18.30; Fri, 23 Nov 2018 14:18:30 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="fEXGYNj/"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727394AbeKXJEa (ORCPT + 32 others); Sat, 24 Nov 2018 04:04:30 -0500 Received: from mail-wr1-f66.google.com ([209.85.221.66]:42210 "EHLO mail-wr1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727302AbeKXJE3 (ORCPT ); Sat, 24 Nov 2018 04:04:29 -0500 Received: by mail-wr1-f66.google.com with SMTP id q18so13558311wrx.9 for ; Fri, 23 Nov 2018 14:18:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=2t2DKZwLo4uiuBiBMlCY8fawnGcKKtVedWhh8SZ5pW8=; b=fEXGYNj/zjrne2auIEZkw4UbItpB9cnQsXYQXrVwYhXuatn1twG7bEQKehvRb5hzEk 4PhZp3v8/zDBoWML41sVaMoIFQ8v9OJnM91SQ/zPdClh1RwbG4tD5YtqYIFIww1mxqbo ks7GhN+jEpgtuDsOAY/q5P/1DWS5iDHK5ja/Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2t2DKZwLo4uiuBiBMlCY8fawnGcKKtVedWhh8SZ5pW8=; b=g6jmNSIEkJfviULK4ieBSE7tzjq28hKeYOpv6wZ5bXMRF5CxQNrad/2/2g98Jf8+EG xm7EPtmhyBRdSmfMU16zmdMgTMShHtOhMxreVOHsz0g3zqpi1vqjvBhEk+xKC2SVViop ohhnj1BcgXNozS2CWk/QtQKQfXCyDAHT+xBRx4V9cZO85SpguUQ/lreSSBbriGxzo6sL Gcijbk1/azjHVTHojEcJUpp96f9nAegrBTYePRi+C1h92X6Q6R1STVK+itZwZT8DMojr 1qGS5/fv2Fqsir4Q+Op9mgdNhSf1e6GWIiu2pFFuhQk09TVmhQzuzuUSQypBp73ZLYTx yNBw== X-Gm-Message-State: AA+aEWYkt8XuyJNctcKuY6qFDaiEvYjY7ecovb3XfxCKXaIEW+Iiycoa MoleTDBGThbH8cbu5W5fach5IDNc32w= X-Received: by 2002:adf:a452:: with SMTP id e18mr3846291wra.190.1543011504211; Fri, 23 Nov 2018 14:18:24 -0800 (PST) Received: from harold.home ([2a01:cb1d:112:6f00:6913:f64b:5e59:5ba5]) by smtp.gmail.com with ESMTPSA id y13sm12578267wrw.85.2018.11.23.14.18.22 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Nov 2018 14:18:23 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , "David S. Miller" , linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org Subject: [PATCH v4 1/2] bpf: add __weak hook for allocating executable memory Date: Fri, 23 Nov 2018 23:18:03 +0100 Message-Id: <20181123221804.440-2-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181123221804.440-1-ard.biesheuvel@linaro.org> References: <20181123221804.440-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org By default, BPF uses module_alloc() to allocate executable memory, but this is not necessary on all arches and potentially undesirable on some of them. So break out the module_alloc() and module_memfree() calls into __weak functions to allow them to be overridden in arch code. Signed-off-by: Ard Biesheuvel --- kernel/bpf/core.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) -- 2.19.1 diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 1a796e0799ec..78e9b76201b3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -609,6 +609,16 @@ static void bpf_jit_uncharge_modmem(u32 pages) atomic_long_sub(pages, &bpf_jit_current); } +void *__weak bpf_jit_alloc_exec(unsigned long size) +{ + return module_alloc(size); +} + +void __weak bpf_jit_free_exec(void *addr) +{ + module_memfree(addr); +} + struct bpf_binary_header * bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, unsigned int alignment, @@ -626,7 +636,7 @@ bpf_jit_binary_alloc(unsigned int proglen, u8 **image_ptr, if (bpf_jit_charge_modmem(pages)) return NULL; - hdr = module_alloc(size); + hdr = bpf_jit_alloc_exec(size); if (!hdr) { bpf_jit_uncharge_modmem(pages); return NULL; @@ -650,7 +660,7 @@ void bpf_jit_binary_free(struct bpf_binary_header *hdr) { u32 pages = hdr->pages; - module_memfree(hdr); + bpf_jit_free_exec(hdr); bpf_jit_uncharge_modmem(pages); } From patchwork Fri Nov 23 22:18:04 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 151923 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp2699847ljp; Fri, 23 Nov 2018 14:18:31 -0800 (PST) X-Google-Smtp-Source: AFSGD/XtSh0hLJwRyhHqze/S+TwPoLu6crOgJew+NGYoCkJl9twlRWk2m6lJDH+8AHXP6quUEFya X-Received: by 2002:a17:902:7791:: with SMTP id o17mr17328158pll.60.1543011511157; Fri, 23 Nov 2018 14:18:31 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1543011511; cv=none; d=google.com; s=arc-20160816; b=qGvfBrhkMs1h+djFoVi8ct5Ke4DnI++ZCdUmcUBHIV0w8DVJ7/UaQLN1io4xn0+AQi e6Q4uMQ/6Yjj5z9uqR5Wy1ko3zjb/ir6mkfV+7hMbX3E4eSv+2a9n9qTSLfPwEe4RvqL PNlvVmmNxoYoXDFG+0sBK2LFSnJ7RR6RvDCa7gQynyZQ6B+nFSraqiD+znWt3H4D1taj /fmBcsZT7ZTs9biwXYQrll0gYs92pzjlrCwDujPeXoz25HhnTYXgOST8jhbMK5NIOmRQ qd46jLN/AxOM+bLJOrkWsa3JWuKS3DZz8A5z8FtF6Muson3zDtpx2GfFhrWYknG25Tx2 UF6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=+ZFGLcFsJGBJa7lwONQ9ze1SyW/x9MifBvOkbZkDguY=; b=b6RTRs34BecNPmHvEggklVaNYCuYPmTLRSok9bIMZ11btM2Jbgnd+LmRr7lpT9mJyi uwaiklGC1OeEm3ko25u0RTvZcsxdNn281K65Ut1SiR4Z2xmiRzqN9DAR9vOzzyQOzT4Q YweyLNyTOOk/gfMXD+oqZglzQt0otTNMQS/oMM60SDACBj3g0T8UPNtS4XBfS29EeAAT EcLJhSFm+Vj/twIHxRT6J0P5Ffdc5GjW/uuaOaVwLOQfXGORpiyDLIR7703eRB5IHGjg JHegQJ+/ob10mYCol64KYQFa9qw9sAfjBV7JgfuszB2IJf1g3w/WMGKDwrcU5bki/9fn YjdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Emne+hOT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a11si51459722pga.198.2018.11.23.14.18.30; Fri, 23 Nov 2018 14:18:31 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Emne+hOT; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727432AbeKXJEb (ORCPT + 32 others); Sat, 24 Nov 2018 04:04:31 -0500 Received: from mail-wm1-f68.google.com ([209.85.128.68]:33978 "EHLO mail-wm1-f68.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727338AbeKXJEb (ORCPT ); Sat, 24 Nov 2018 04:04:31 -0500 Received: by mail-wm1-f68.google.com with SMTP id y185so8402304wmd.1 for ; Fri, 23 Nov 2018 14:18:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+ZFGLcFsJGBJa7lwONQ9ze1SyW/x9MifBvOkbZkDguY=; b=Emne+hOT+VIshM/P0PVECpzJvCjCTEAHU4CgmMgHaVVjc/CQIdqc/nhh+BSX0p41t8 9zNZI9dpIytkxwi9W+rED1uFCrlRDR7ofv6T67YKYbF4UAF5BTvwdv8hzdRUDKfqBBlh xe07d88+VV0aODCzewN9dJkVy3UaruscOzP+I= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+ZFGLcFsJGBJa7lwONQ9ze1SyW/x9MifBvOkbZkDguY=; b=ChQzvrOCOaJTYvPL9G/qBzoAygdrdLFSuY/caKTDQ+vLsTtcglhwU+h9Gx5yqCHh6v ttQXCAO9v/o9wRb/RxH9sXnXO6goRhX0AGTqhH8mWbvEUZgiJoVrdNAIXTJLJujTC6fl 4awwU/YRnLio7fvoevllZ0B9kGqLTOxrPYBj1cw3Wxse1MIrLNdp6srhLv0cOs5Blk0b l+azVpu2uhNntPZfZHSUq0cz/eNUIIi3qRZ5uN2EmEDDV0VqRFbgh1RrO2CSh8YIgu5r H6pFM0/YdNuqOEGbIu4eDFrFhRsKfTZeDD3t2CrvIHn9j/1el88b/YULXHHqEm69XvTs UeFg== X-Gm-Message-State: AGRZ1gIewVWhlVcrE6/iKFuj5YeeuJD8nw+S3hwErynniK/E1+6da5QI mmR8rZGDuoze9rPQcc06iPFRgeJpewY= X-Received: by 2002:a1c:b54b:: with SMTP id e72mr15403683wmf.73.1543011505644; Fri, 23 Nov 2018 14:18:25 -0800 (PST) Received: from harold.home ([2a01:cb1d:112:6f00:6913:f64b:5e59:5ba5]) by smtp.gmail.com with ESMTPSA id y13sm12578267wrw.85.2018.11.23.14.18.24 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 23 Nov 2018 14:18:24 -0800 (PST) From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Daniel Borkmann , Alexei Starovoitov , Rick Edgecombe , Eric Dumazet , Jann Horn , Kees Cook , Jessica Yu , Arnd Bergmann , Catalin Marinas , Will Deacon , Mark Rutland , "David S. Miller" , linux-arm-kernel@lists.infradead.org, netdev@vger.kernel.org Subject: [PATCH v4 2/2] arm64/bpf: don't allocate BPF JIT programs in module memory Date: Fri, 23 Nov 2018 23:18:04 +0100 Message-Id: <20181123221804.440-3-ard.biesheuvel@linaro.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20181123221804.440-1-ard.biesheuvel@linaro.org> References: <20181123221804.440-1-ard.biesheuvel@linaro.org> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The arm64 module region is a 128 MB region that is kept close to the core kernel, in order to ensure that relative branches are always in range. So using the same region for programs that do not have this restriction is wasteful, and preferably avoided. Now that the core BPF JIT code permits the alloc/free routines to be overridden, implement them by vmalloc()/vfree() calls from a dedicated 128 MB region set aside for BPF programs. This ensures that BPF programs are still in branching range of each other, which is something the JIT currently depends upon (and is not guaranteed when using module_alloc() on KASLR kernels like we do currently). It also ensures that placement of BPF programs does not correlate with the placement of the core kernel or modules, making it less likely that leaking the former will reveal the latter. This also solves an issue under KASAN, where shadow memory is needlessly allocated for all BPF programs (which don't require KASAN shadow pages since they are not KASAN instrumented) Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/memory.h | 5 ++++- arch/arm64/net/bpf_jit_comp.c | 13 +++++++++++++ 2 files changed, 17 insertions(+), 1 deletion(-) -- 2.19.1 Acked-by: Will Deacon diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index b96442960aea..ee20fc63899c 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -62,8 +62,11 @@ #define PAGE_OFFSET (UL(0xffffffffffffffff) - \ (UL(1) << (VA_BITS - 1)) + 1) #define KIMAGE_VADDR (MODULES_END) +#define BPF_JIT_REGION_START (VA_START + KASAN_SHADOW_SIZE) +#define BPF_JIT_REGION_SIZE (SZ_128M) +#define BPF_JIT_REGION_END (BPF_JIT_REGION_START + BPF_JIT_REGION_SIZE) #define MODULES_END (MODULES_VADDR + MODULES_VSIZE) -#define MODULES_VADDR (VA_START + KASAN_SHADOW_SIZE) +#define MODULES_VADDR (BPF_JIT_REGION_END) #define MODULES_VSIZE (SZ_128M) #define VMEMMAP_START (PAGE_OFFSET - VMEMMAP_SIZE) #define PCI_IO_END (VMEMMAP_START - SZ_2M) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index a6fdaea07c63..76c2ab40c02d 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -940,3 +940,16 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_prog *prog) tmp : orig_prog); return prog; } + +void *bpf_jit_alloc_exec(unsigned long size) +{ + return __vmalloc_node_range(size, PAGE_SIZE, BPF_JIT_REGION_START, + BPF_JIT_REGION_END, GFP_KERNEL, + PAGE_KERNEL_EXEC, 0, NUMA_NO_NODE, + __builtin_return_address(0)); +} + +void bpf_jit_free_exec(void *addr) +{ + return vfree(addr); +}