[11/23] io_uring: enable mmap'ing additional CQs

Message ID	0b54c7e684a2a9c3cb1a3cddfe48790e79bfbb89.1621424513.git.asml.silence@gmail.com
State	New
Headers	show Return-Path: <netdev-owner@kernel.org> From: Pavel Begunkov <asml.silence@gmail.com> To: io-uring@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Jens Axboe <axboe@kernel.dk>, Alexei Starovoitov <ast@kernel.org>, Daniel Borkmann <daniel@iogearbox.net>, Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau <kafai@fb.com>, Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>, John Fastabend <john.fastabend@gmail.com>, KP Singh <kpsingh@kernel.org>, Horst Schirmeier <horst.schirmeier@tu-dortmund.de>, "Franz-B . Tuneke" <franz-bernhard.tuneke@tu-dortmund.de>, Christian Dietrich <stettberger@dokucode.de> Subject: [PATCH 11/23] io_uring: enable mmap'ing additional CQs Date: Wed, 19 May 2021 15:13:22 +0100 Message-Id: <0b54c7e684a2a9c3cb1a3cddfe48790e79bfbb89.1621424513.git.asml.silence@gmail.com> In-Reply-To: <cover.1621424513.git.asml.silence@gmail.com> References: <cover.1621424513.git.asml.silence@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk
Series	io_uring BPF requests \| expand [RFC,v2,00/23] io_uring BPF requests [01/23] io_uring: shuffle rarely used ctx fields [02/23] io_uring: localise fixed resources fields [03/23] io_uring: remove dependency on ring->sq/cq_entries [04/23] io_uring: deduce cq_mask from cq_entries [05/23] io_uring: kill cached_cq_overflow [06/23] io_uring: rename io_get_cqring [07/23] io_uring: extract struct for CQ [08/23] io_uring: internally pass CQ indexes [09/23] io_uring: extract cq size helper [10/23] io_uring: add support for multiple CQs [11/23] io_uring: enable mmap'ing additional CQs [12/23] bpf: add IOURING program type [13/23] io_uring: implement bpf prog registration [14/23] io_uring: add support for bpf requests [15/23] io_uring: enable BPF to submit SQEs [16/23] io_uring: enable bpf to submit CQEs [17/23] io_uring: enable bpf to reap CQEs [18/23] libbpf: support io_uring [19/23] io_uring: pass user_data to bpf executor [20/23] bpf: Add bpf_copy_to_user() helper [21/23] io_uring: wire bpf copy to user [22/23] io_uring: don't wait on CQ exclusively [23/23] io_uring: enable bpf reqs to wait for CQs

Message ID

0b54c7e684a2a9c3cb1a3cddfe48790e79bfbb89.1621424513.git.asml.silence@gmail.com

State

New

Headers

From: Pavel Begunkov <asml.silence@gmail.com>
To: io-uring@vger.kernel.org, netdev@vger.kernel.org,
	bpf@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: Jens Axboe <axboe@kernel.dk>, Alexei Starovoitov <ast@kernel.org>,
	Daniel Borkmann <daniel@iogearbox.net>,
	Andrii Nakryiko <andrii@kernel.org>, Martin KaFai Lau <kafai@fb.com>,
	Song Liu <songliubraving@fb.com>, Yonghong Song <yhs@fb.com>,
	John Fastabend <john.fastabend@gmail.com>, KP Singh <kpsingh@kernel.org>,
	Horst Schirmeier <horst.schirmeier@tu-dortmund.de>,
	"Franz-B . Tuneke" <franz-bernhard.tuneke@tu-dortmund.de>,
	Christian Dietrich <stettberger@dokucode.de>
Subject: [PATCH 11/23] io_uring: enable mmap'ing additional CQs
Date: Wed, 19 May 2021 15:13:22 +0100
Message-Id: <0b54c7e684a2a9c3cb1a3cddfe48790e79bfbb89.1621424513.git.asml.silence@gmail.com>
In-Reply-To: <cover.1621424513.git.asml.silence@gmail.com>
References: <cover.1621424513.git.asml.silence@gmail.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

Series

io_uring BPF requests | expand

Commit Message

Pavel Begunkov May 19, 2021, 2:13 p.m. UTC

TODO: get rid of extra offset

Signed-off-by: Pavel Begunkov <asml.silence@gmail.com>
---
 fs/io_uring.c                 | 13 ++++++++++++-
 include/uapi/linux/io_uring.h |  2 ++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index 067cfb3a6e4a..1a4c9e513ac9 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -9207,6 +9207,7 @@  static void *io_uring_validate_mmap_request(struct file *file,
 	struct io_ring_ctx *ctx = file->private_data;
 	loff_t offset = pgoff << PAGE_SHIFT;
 	struct page *page;
+	unsigned long cq_idx;
 	void *ptr;
 
 	switch (offset) {
@@ -9218,7 +9219,15 @@  static void *io_uring_validate_mmap_request(struct file *file,
 		ptr = ctx->sq_sqes;
 		break;
 	default:
-		return ERR_PTR(-EINVAL);
+		if (offset < IORING_OFF_CQ_RING_EXTRA)
+			return ERR_PTR(-EINVAL);
+		offset -= IORING_OFF_CQ_RING_EXTRA;
+		if (offset % IORING_STRIDE_CQ_RING)
+			return ERR_PTR(-EINVAL);
+		cq_idx = offset / IORING_STRIDE_CQ_RING;
+		if (cq_idx >= ctx->cq_nr)
+			return ERR_PTR(-EINVAL);
+		ptr = ctx->cqs[cq_idx].rings;
 	}
 
 	page = virt_to_head_page(ptr);
@@ -9615,6 +9624,8 @@  static int io_allocate_scq_urings(struct io_ring_ctx *ctx,
 
 	return 0;
 err:
+	while (ctx->cq_nr > 1)
+		io_mem_free(ctx->cqs[--ctx->cq_nr].rings);
 	io_mem_free(ctx->rings);
 	ctx->rings = NULL;
 	return ret;
diff --git a/include/uapi/linux/io_uring.h b/include/uapi/linux/io_uring.h
index 92b61ca09ea5..67a97c793de7 100644
--- a/include/uapi/linux/io_uring.h
+++ b/include/uapi/linux/io_uring.h
@@ -203,6 +203,8 @@  enum {
 #define IORING_OFF_SQ_RING		0ULL
 #define IORING_OFF_CQ_RING		0x8000000ULL
 #define IORING_OFF_SQES			0x10000000ULL
+#define IORING_OFF_CQ_RING_EXTRA	0x1200000ULL
+#define IORING_STRIDE_CQ_RING		0x0100000ULL
 
 /*
  * Filled with the offset for mmap(2)

[11/23] io_uring: enable mmap'ing additional CQs

Commit Message

Patch