From patchwork Fri Apr 11 20:32:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= X-Patchwork-Id: 880652 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C764280CCD; Fri, 11 Apr 2025 20:32:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403550; cv=none; b=APW/qGeicDZ7fdYMDjj/oI2RpM9Un2TDbXHX6ohpvSwca4K9mCnnBAiyDt7Cc/uA/+QCLRVTb06lVCteRuuYRsruMyUbWzj8ejbyuvdgo0lNTYqa5nj27ETS7DIjRLEjLuytd+9q01Ws66yPZMgp0tcL1RFKiSEFhP8AIwAavuA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403550; c=relaxed/simple; bh=ejORcpvGfrfykxujY30gBt0iabsPSkXI6NglRqseVT4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=osZbZVRnerPABwwaVe1LqAWCZ5Fn8uCvtiD/hP79OvfTEerBXSl7Sc+zyn9qPa8KgruV2bVi1e0SN+QOiZC+pR2HVOohDwEkMfSv2TG0TjNC1EIdg046ofJys2yLUX81cXSMILu8RP2CtqoQFEgCSAbT/cBpH/U4hc/hlubH8dc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=VpzGMXts; arc=none smtp.client-ip=217.70.183.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="VpzGMXts" Received: by mail.gandi.net (Postfix) with ESMTPSA id 285C4439D4; Fri, 11 Apr 2025 20:32:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1744403540; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x0Lf7W9bQWXXAkTM40dnJ+uKraq0dPHGV+hXAWhTfnE=; b=VpzGMXts8vgeVU0Ddi28PFbkrr/1n86d1hEvDDcle5wS5M0lcVaWSGqTxCtPLuPev9r/Tx E2ORWC9W6nXSL8Y18Wo/dJEmtwBjAQT0DAPcPgEC6ihScWncCoiKeKZ1H+y7Jr9SiVrTAY LhaDvIx9k7Fku99th6Vtjs4eoHgyeZleFXOszZbgeSoDGiHw52wkOusp3dz5Oo3lR+OpEC RUvZgfKfzXSwDwmgolZgjTWZtlIOvpsBZv8yzn9ma4AxcVQaiLEsbgi2imv8inV7yEU55Q HQX9Pzt0vzhuue/BtlfvqOoZumlBd3DBNUQfA4S7DnePLIztKZtYJrTopLvZHQ== From: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= Date: Fri, 11 Apr 2025 22:32:10 +0200 Subject: [PATCH RFC bpf-next 1/4] bpf: add struct largest member size in func model Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250411-many_args_arm64-v1-1-0a32fe72339e@bootlin.com> References: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> In-Reply-To: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Mykola Lysenko , Shuah Khan , Maxime Coquelin , Alexandre Torgue , Florent Revest Cc: Bastien Curutchet , ebpf@linuxfoundation.org, Thomas Petazzoni , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, =?utf-8?q?Alexis_Lothor=C3=A9_?= =?utf-8?q?=28eBPF_Foundation=29?= X-Mailer: b4 0.14.2 X-GND-State: clean X-GND-Score: -100 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvuddvjeekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthekredtredtjeenucfhrhhomheptehlvgigihhsucfnohhthhhorhorucdlvgeurffhucfhohhunhgurghtihhonhdmuceorghlvgigihhsrdhlohhthhhorhgvsegsohhothhlihhnrdgtohhmqeenucggtffrrghtthgvrhhnpeeljeektefffeevleegkeelhfethffgieegudevffejheelieeffeejtddujeegueenucfkphepvdgrtddvmeekgedvkeemfhelgegtmegvtddtmeemfhekheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpedvrgdtvdemkeegvdekmehfleegtgemvgdttdemmehfkeehpdhhvghloheplgduledvrdduieekrddurdduleejngdpmhgrihhlfhhrohhmpegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdpnhgspghrtghpthhtohepfedtpdhrtghpthhtohepshgufhesfhhomhhitghhvghvrdhmvgdprhgtphhtthhopehmrghrthhinhdrlhgruheslhhinhhugidruggvvhdprhgtphhtthhopeiguhhkuhhohhgriheshhhurgifvghitghlohhuugdrtghomhdprhgtphhtthhopehjohhlshgrsehkvghrnhgvl hdrohhrghdprhgtphhtthhopegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdprhgtphhtthhopehmhihkohhlrghlsehfsgdrtghomhdprhgtphhtthhopeifihhllheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepmhgtohhquhgvlhhinhdrshhtmhefvdesghhmrghilhdrtghomh X-GND-Sasl: alexis.lothore@bootlin.com In order to properly JIT the trampolines needed to attach BPF programs to functions, some architectures like ARM64 need to know about the alignment needed for the function arguments. Such alignment can generally be deduced from the argument size, but that's not completely true for composite types. In the specific case of ARM64, the AAPCS64 ABI defines that a composite type which needs to be passed through stack must be aligned on the maximum between 8 and the largest alignment constraint of its first-level members. So the JIT compiler needs more information about the arguments to make sure to generate code that respects those alignment constraints. For struct arguments, add information about the size of the largest first-level member in the struct btf_func_model to allow the JIT compiler to guess the needed alignment. The information is quite specific, but it allows to keep arch-specific concerns (ie: guessing the final needed alignment for an argument) isolated in each JIT compiler. Signed-off-by: Alexis Lothoré (eBPF Foundation) --- include/linux/bpf.h | 1 + kernel/bpf/btf.c | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index 3f0cc89c0622cb1a097999afb78c17102593b6bb..8b34dcf60a0ce09228ff761b962ab67b6a3e2263 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -1106,6 +1106,7 @@ struct btf_func_model { u8 nr_args; u8 arg_size[MAX_BPF_FUNC_ARGS]; u8 arg_flags[MAX_BPF_FUNC_ARGS]; + u8 arg_largest_member_size[MAX_BPF_FUNC_ARGS]; }; /* Restore arguments before returning from trampoline to let original function diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 16ba36f34dfab7531babf5753cab9f368cddefa3..5d40911ec90210086a6175d569abb6e52d75ad17 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -7318,6 +7318,29 @@ static int __get_type_size(struct btf *btf, u32 btf_id, return -EINVAL; } +static u8 __get_largest_member_size(struct btf *btf, const struct btf_type *t) +{ + const struct btf_member *member; + const struct btf_type *mtype; + u8 largest_member_size = 0; + int i; + + if (!__btf_type_is_struct(t)) + return largest_member_size; + + for_each_member(i, t, member) { + mtype = btf_type_by_id(btf, member->type); + while (mtype && btf_type_is_modifier(mtype)) + mtype = btf_type_by_id(btf, mtype->type); + if (!mtype) + return -EINVAL; + if (mtype->size > largest_member_size) + largest_member_size = mtype->size; + } + + return largest_member_size; +} + static u8 __get_type_fmodel_flags(const struct btf_type *t) { u8 flags = 0; @@ -7396,6 +7419,8 @@ int btf_distill_func_proto(struct bpf_verifier_log *log, } m->arg_size[i] = ret; m->arg_flags[i] = __get_type_fmodel_flags(t); + m->arg_largest_member_size[i] = + __get_largest_member_size(btf, t); } m->nr_args = nargs; return 0; From patchwork Fri Apr 11 20:32:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= X-Patchwork-Id: 880385 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8B2E283C8A; Fri, 11 Apr 2025 20:32:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403552; cv=none; b=cSXwS3ePB5PbtehNJ5ovidiEMvbGl3ze4Knqc4YcANqIV/lL0vlKu8MYiqtVkVJ6rhlNhwyiJkoORFO+F+WYKBBp9azDNWDRBmP8WahysAfxVlmNgAf2zS/Y3YHES89Lw46O6zKVEG+xKldAYXLR5/IVJOoyGr1J5fe1z6Die/Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403552; c=relaxed/simple; bh=wxnQ0ttI+gxkqntD0fF99Mhv0sfuyDUXwOM1qxLQaXM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=MSNtWKWqZXr0J68Suc+ytoQpK9+OO3eWk4wsufbDde94qwlBgF4R3hagG5zBxZAELLNe0rHmc7C1j+mq9Elwy8JRVfXDnsHuE+GzeADBRzcimQEih7b77TdfyzB/UJaX5A2KI9tJqbu6wDNo0Kt7M9CT8JZSbG/zVM977oXPCVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=K3NVb9Cq; arc=none smtp.client-ip=217.70.183.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="K3NVb9Cq" Received: by mail.gandi.net (Postfix) with ESMTPSA id AA9D5439EE; Fri, 11 Apr 2025 20:32:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1744403542; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ec28mwfnSBXDui4uhcD+4fTqV49TTiQGTbLP7ywekj0=; b=K3NVb9CqxwIVa5GBFmmnfNIejUKa7TPH67HXLOQbeIMGv/nyA8hAPhYvtX4u788aqOprmI 7IPQ53KmzL4KEO35oQ7Pw1EinQfPoZmXsm/7NVRf0yzEMHh/zcXBUTeSAGbbjnH5ftKt9C lyTgu52YADl265PVzGvkqAMmRyg76Ojabkw4dDxFo0xu0E976brSf5mBbk2z6zZQd+v6BW WnrxPjlxcdh9c8naQTUrw+DCFAwQdeRpRFiA6LoVF4etSovHNUabLmLeQv98gtFdnmzA6N W/45Ybb4Md3QnP+ykRalkxm9n+vrin5L01HRT3aCzRYKRxH3T4UaAyvyhzyLNQ== From: =?utf-8?q?Alexis_Lothor=C3=A9?= Date: Fri, 11 Apr 2025 22:32:11 +0200 Subject: [PATCH RFC bpf-next 2/4] bpf, arm64: Support up to 12 function arguments Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250411-many_args_arm64-v1-2-0a32fe72339e@bootlin.com> References: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> In-Reply-To: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Mykola Lysenko , Shuah Khan , Maxime Coquelin , Alexandre Torgue , Florent Revest Cc: Bastien Curutchet , ebpf@linuxfoundation.org, Thomas Petazzoni , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, =?utf-8?q?Alexis_Lothor=C3=A9_?= =?utf-8?q?=28eBPF_Foundation=29?= , Xu Kuohai X-Mailer: b4 0.14.2 X-GND-State: clean X-GND-Score: -100 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvuddvjeekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthekredtredtjeenucfhrhhomheptehlvgigihhsucfnohhthhhorhoruceorghlvgigihhsrdhlohhthhhorhgvsegsohhothhlihhnrdgtohhmqeenucggtffrrghtthgvrhhnpeegveeghfetffeuhfehkeekleffffdvuefggfevjefftddvffduheettdeiveetteenucfkphepvdgrtddvmeekgedvkeemfhelgegtmegvtddtmeemfhekheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpedvrgdtvdemkeegvdekmehfleegtgemvgdttdemmehfkeehpdhhvghloheplgduledvrdduieekrddurdduleejngdpmhgrihhlfhhrohhmpegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdpnhgspghrtghpthhtohepfedupdhrtghpthhtohepshgufhesfhhomhhitghhvghvrdhmvgdprhgtphhtthhopehmrghrthhinhdrlhgruheslhhinhhugidruggvvhdprhgtphhtthhopeiguhhkuhhohhgriheshhhurgifvghitghlohhuugdrtghomhdprhgtphhtthhopehjohhlshgrsehkvghrnhgvlhdrohhrghdprhgtphhtthhopegrlhgvgihis hdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdprhgtphhtthhopehmhihkohhlrghlsehfsgdrtghomhdprhgtphhtthhopeifihhllheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepmhgtohhquhgvlhhinhdrshhtmhefvdesghhmrghilhdrtghomh X-GND-Sasl: alexis.lothore@bootlin.com From: Xu Kuohai Currently ARM64 bpf trampoline supports up to 8 function arguments. According to the statistics from commit 473e3150e30a ("bpf, x86: allow function arguments up to 12 for TRACING"), there are about 200 functions accept 9 to 12 arguments, so adding support for up to 12 function arguments. Due to bpf only supporting function arguments up to 16 bytes, according to AAPCS64, starting from the first argument, each argument is first attempted to be loaded to 1 or 2 smallest registers from x0-x7, if there are no enough registers to hold the entire argument, then all remaining arguments starting from this one are pushed to the stack for passing. Signed-off-by: Xu Kuohai Co-developed-by: Alexis Lothoré Signed-off-by: Alexis Lothoré --- arch/arm64/net/bpf_jit_comp.c | 235 ++++++++++++++++++++++++++++++++---------- 1 file changed, 182 insertions(+), 53 deletions(-) diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c index 70d7c89d3ac907798e86e0051e7b472c252c1412..3618f42b1a0019590d47db0ec0fce1f9f51b01af 100644 --- a/arch/arm64/net/bpf_jit_comp.c +++ b/arch/arm64/net/bpf_jit_comp.c @@ -2064,7 +2064,7 @@ bool bpf_jit_supports_subprog_tailcalls(void) } static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_link *l, - int args_off, int retval_off, int run_ctx_off, + int bargs_off, int retval_off, int run_ctx_off, bool save_ret) { __le32 *branch; @@ -2106,7 +2106,7 @@ static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_link *l, branch = ctx->image + ctx->idx; emit(A64_NOP, ctx); - emit(A64_ADD_I(1, A64_R(0), A64_SP, args_off), ctx); + emit(A64_ADD_I(1, A64_R(0), A64_SP, bargs_off), ctx); if (!p->jited) emit_addr_mov_i64(A64_R(1), (const u64)p->insnsi, ctx); @@ -2131,7 +2131,7 @@ static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_link *l, } static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_links *tl, - int args_off, int retval_off, int run_ctx_off, + int bargs_off, int retval_off, int run_ctx_off, __le32 **branches) { int i; @@ -2141,7 +2141,7 @@ static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_links *tl, */ emit(A64_STR64I(A64_ZR, A64_SP, retval_off), ctx); for (i = 0; i < tl->nr_links; i++) { - invoke_bpf_prog(ctx, tl->links[i], args_off, retval_off, + invoke_bpf_prog(ctx, tl->links[i], bargs_off, retval_off, run_ctx_off, true); /* if (*(u64 *)(sp + retval_off) != 0) * goto do_fexit; @@ -2155,23 +2155,142 @@ static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_links *tl, } } -static void save_args(struct jit_ctx *ctx, int args_off, int nregs) +struct arg_aux { + /* how many args are passed through registers, the rest of the args are + * passed through stack + */ + int args_in_regs; + /* how many registers are used to pass arguments */ + int regs_for_args; + /* how much stack is used for additional args passed to bpf program + * that did not fit in original function registers + **/ + int bstack_for_args; + /* home much stack is used for additional args passed to the + * original function when called from trampoline (this one needs + * arguments to be properly aligned) + */ + int ostack_for_args; + u8 arg_align[MAX_BPF_FUNC_ARGS]; +}; + +static u8 calc_arg_align(const struct btf_func_model *m, int i) +{ + if (m->arg_flags[i] & BTF_FMODEL_STRUCT_ARG) + return max(8, m->arg_largest_member_size[i]); + return max(8, m->arg_size[i]); +} + +static void calc_arg_aux(const struct btf_func_model *m, + struct arg_aux *a) { int i; + int nregs; + int slots; + int stack_slots; + + for (i = 0; i < m->nr_args; i++) + a->arg_align[i] = calc_arg_align(m, i); + + /* verifier ensures m->nr_args <= MAX_BPF_FUNC_ARGS */ + for (i = 0, nregs = 0; i < m->nr_args; i++) { + slots = (m->arg_size[i] + 7) / 8; + if (nregs + slots <= 8) /* passed through register ? */ + nregs += slots; + else + break; + } - for (i = 0; i < nregs; i++) { - emit(A64_STR64I(i, A64_SP, args_off), ctx); - args_off += 8; + a->args_in_regs = i; + a->regs_for_args = nregs; + a->ostack_for_args = 0; + + /* the rest arguments are passed through stack */ + for (a->ostack_for_args = 0, a->bstack_for_args = 0; + i < m->nr_args; i++) { + stack_slots = (m->arg_size[i] + 7) / 8; + /* AAPCS 64 C.14: arguments passed on stack must be aligned to + * max(8, arg_natural_alignment) + */ + a->bstack_for_args += stack_slots * 8; + a->ostack_for_args = round_up(a->ostack_for_args + + stack_slots * 8, a->arg_align[i]); + } +} + +static void clear_garbage(struct jit_ctx *ctx, int reg, int effective_bytes) +{ + if (effective_bytes) { + int garbage_bits = 64 - 8 * effective_bytes; +#ifdef CONFIG_CPU_BIG_ENDIAN + /* garbage bits are at the right end */ + emit(A64_LSR(1, reg, reg, garbage_bits), ctx); + emit(A64_LSL(1, reg, reg, garbage_bits), ctx); +#else + /* garbage bits are at the left end */ + emit(A64_LSL(1, reg, reg, garbage_bits), ctx); + emit(A64_LSR(1, reg, reg, garbage_bits), ctx); +#endif } } -static void restore_args(struct jit_ctx *ctx, int args_off, int nregs) +static void save_args(struct jit_ctx *ctx, int bargs_off, int oargs_off, + const struct btf_func_model *m, + const struct arg_aux *a, + bool for_call_origin) { int i; + int reg; + int doff; + int soff; + int slots; + u8 tmp = bpf2a64[TMP_REG_1]; + + /* store arguments to the stack for the bpf program, or restore + * arguments from stack for the original function + */ + for (reg = 0; reg < a->regs_for_args; reg++) { + emit(for_call_origin ? + A64_LDR64I(reg, A64_SP, bargs_off) : + A64_STR64I(reg, A64_SP, bargs_off), + ctx); + bargs_off += 8; + } + + soff = 32; /* on stack arguments start from FP + 32 */ + doff = (for_call_origin ? oargs_off : bargs_off); + + /* save on stack arguments */ + for (i = a->args_in_regs; i < m->nr_args; i++) { + slots = (m->arg_size[i] + 7) / 8; + /* AAPCS C.14: additional arguments on stack must be + * aligned on max(8, arg_natural_alignment) + */ + soff = round_up(soff, a->arg_align[i]); + if (for_call_origin) + doff = round_up(doff, a->arg_align[i]); + /* verifier ensures arg_size <= 16, so slots equals 1 or 2 */ + while (slots-- > 0) { + emit(A64_LDR64I(tmp, A64_FP, soff), ctx); + /* if there is unused space in the last slot, clear + * the garbage contained in the space. + */ + if (slots == 0 && !for_call_origin) + clear_garbage(ctx, tmp, m->arg_size[i] % 8); + emit(A64_STR64I(tmp, A64_SP, doff), ctx); + soff += 8; + doff += 8; + } + } +} - for (i = 0; i < nregs; i++) { - emit(A64_LDR64I(i, A64_SP, args_off), ctx); - args_off += 8; +static void restore_args(struct jit_ctx *ctx, int bargs_off, int nregs) +{ + int reg; + + for (reg = 0; reg < nregs; reg++) { + emit(A64_LDR64I(reg, A64_SP, bargs_off), ctx); + bargs_off += 8; } } @@ -2194,17 +2313,21 @@ static bool is_struct_ops_tramp(const struct bpf_tramp_links *fentry_links) */ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, struct bpf_tramp_links *tlinks, void *func_addr, - int nregs, u32 flags) + const struct btf_func_model *m, + const struct arg_aux *a, + u32 flags) { int i; int stack_size; int retaddr_off; int regs_off; int retval_off; - int args_off; - int nregs_off; + int bargs_off; + int nfuncargs_off; int ip_off; int run_ctx_off; + int oargs_off; + int nfuncargs; struct bpf_tramp_links *fentry = &tlinks[BPF_TRAMP_FENTRY]; struct bpf_tramp_links *fexit = &tlinks[BPF_TRAMP_FEXIT]; struct bpf_tramp_links *fmod_ret = &tlinks[BPF_TRAMP_MODIFY_RETURN]; @@ -2213,31 +2336,38 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, bool is_struct_ops = is_struct_ops_tramp(fentry); /* trampoline stack layout: - * [ parent ip ] - * [ FP ] - * SP + retaddr_off [ self ip ] - * [ FP ] + * [ parent ip ] + * [ FP ] + * SP + retaddr_off [ self ip ] + * [ FP ] * - * [ padding ] align SP to multiples of 16 + * [ padding ] align SP to multiples of 16 * - * [ x20 ] callee saved reg x20 - * SP + regs_off [ x19 ] callee saved reg x19 + * [ x20 ] callee saved reg x20 + * SP + regs_off [ x19 ] callee saved reg x19 * - * SP + retval_off [ return value ] BPF_TRAMP_F_CALL_ORIG or - * BPF_TRAMP_F_RET_FENTRY_RET + * SP + retval_off [ return value ] BPF_TRAMP_F_CALL_ORIG or + * BPF_TRAMP_F_RET_FENTRY_RET + * [ arg reg N ] + * [ ... ] + * SP + bargs_off [ arg reg 1 ] for bpf * - * [ arg reg N ] - * [ ... ] - * SP + args_off [ arg reg 1 ] + * SP + nfuncargs_off [ arg regs count ] * - * SP + nregs_off [ arg regs count ] + * SP + ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag * - * SP + ip_off [ traced function ] BPF_TRAMP_F_IP_ARG flag + * SP + run_ctx_off [ bpf_tramp_run_ctx ] * - * SP + run_ctx_off [ bpf_tramp_run_ctx ] + * [ stack arg N ] + * [ ... ] + * SP + oargs_off [ stack arg 1 ] for original func */ stack_size = 0; + oargs_off = stack_size; + if (flags & BPF_TRAMP_F_CALL_ORIG) + stack_size += a->ostack_for_args; + run_ctx_off = stack_size; /* room for bpf_tramp_run_ctx */ stack_size += round_up(sizeof(struct bpf_tramp_run_ctx), 8); @@ -2247,13 +2377,14 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, if (flags & BPF_TRAMP_F_IP_ARG) stack_size += 8; - nregs_off = stack_size; + nfuncargs_off = stack_size; /* room for args count */ stack_size += 8; - args_off = stack_size; + bargs_off = stack_size; /* room for args */ - stack_size += nregs * 8; + nfuncargs = a->regs_for_args + a->bstack_for_args / 8; + stack_size += 8 * nfuncargs; /* room for return value */ retval_off = stack_size; @@ -2300,11 +2431,11 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, } /* save arg regs count*/ - emit(A64_MOVZ(1, A64_R(10), nregs, 0), ctx); - emit(A64_STR64I(A64_R(10), A64_SP, nregs_off), ctx); + emit(A64_MOVZ(1, A64_R(10), nfuncargs, 0), ctx); + emit(A64_STR64I(A64_R(10), A64_SP, nfuncargs_off), ctx); - /* save arg regs */ - save_args(ctx, args_off, nregs); + /* save args for bpf */ + save_args(ctx, bargs_off, oargs_off, m, a, false); /* save callee saved registers */ emit(A64_STR64I(A64_R(19), A64_SP, regs_off), ctx); @@ -2320,7 +2451,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, } for (i = 0; i < fentry->nr_links; i++) - invoke_bpf_prog(ctx, fentry->links[i], args_off, + invoke_bpf_prog(ctx, fentry->links[i], bargs_off, retval_off, run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET); @@ -2330,12 +2461,13 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, if (!branches) return -ENOMEM; - invoke_bpf_mod_ret(ctx, fmod_ret, args_off, retval_off, + invoke_bpf_mod_ret(ctx, fmod_ret, bargs_off, retval_off, run_ctx_off, branches); } if (flags & BPF_TRAMP_F_CALL_ORIG) { - restore_args(ctx, args_off, nregs); + /* save args for original func */ + save_args(ctx, bargs_off, oargs_off, m, a, true); /* call original func */ emit(A64_LDR64I(A64_R(10), A64_SP, retaddr_off), ctx); emit(A64_ADR(A64_LR, AARCH64_INSN_SIZE * 2), ctx); @@ -2354,7 +2486,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, } for (i = 0; i < fexit->nr_links; i++) - invoke_bpf_prog(ctx, fexit->links[i], args_off, retval_off, + invoke_bpf_prog(ctx, fexit->links[i], bargs_off, retval_off, run_ctx_off, false); if (flags & BPF_TRAMP_F_CALL_ORIG) { @@ -2368,7 +2500,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im, } if (flags & BPF_TRAMP_F_RESTORE_REGS) - restore_args(ctx, args_off, nregs); + restore_args(ctx, bargs_off, a->regs_for_args); /* restore callee saved register x19 and x20 */ emit(A64_LDR64I(A64_R(19), A64_SP, regs_off), ctx); @@ -2428,14 +2560,13 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags, .idx = 0, }; struct bpf_tramp_image im; + struct arg_aux aaux; int nregs, ret; nregs = btf_func_model_nregs(m); - /* the first 8 registers are used for arguments */ - if (nregs > 8) - return -ENOTSUPP; - ret = prepare_trampoline(&ctx, &im, tlinks, func_addr, nregs, flags); + calc_arg_aux(m, &aaux); + ret = prepare_trampoline(&ctx, &im, tlinks, func_addr, m, &aaux, flags); if (ret < 0) return ret; @@ -2462,9 +2593,10 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *ro_image, u32 flags, struct bpf_tramp_links *tlinks, void *func_addr) { - int ret, nregs; - void *image, *tmp; u32 size = ro_image_end - ro_image; + struct arg_aux aaux; + void *image, *tmp; + int ret; /* image doesn't need to be in module memory range, so we can * use kvmalloc. @@ -2480,13 +2612,10 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *ro_image, .write = true, }; - nregs = btf_func_model_nregs(m); - /* the first 8 registers are used for arguments */ - if (nregs > 8) - return -ENOTSUPP; jit_fill_hole(image, (unsigned int)(ro_image_end - ro_image)); - ret = prepare_trampoline(&ctx, im, tlinks, func_addr, nregs, flags); + calc_arg_aux(m, &aaux); + ret = prepare_trampoline(&ctx, im, tlinks, func_addr, m, &aaux, flags); if (ret > 0 && validate_code(&ctx) < 0) { ret = -EINVAL; From patchwork Fri Apr 11 20:32:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= X-Patchwork-Id: 880651 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82A9C28F945; Fri, 11 Apr 2025 20:32:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403554; cv=none; b=qofVbXIQgdUTnG79xxJwZB0F0+ftpSBGK5ycUb+oxFjnn0ZABvmx64OewBTjEEFznyqBO2lNoRVytyaDJCnrN6cWSBNaba2giApWmFuUSo+5RwSGK2g9yy6Fi/Lvhn5qbOvXgB/rZ7HXEztUVeaYDtp1T4mVwS75vdfQKg2Atto= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403554; c=relaxed/simple; bh=HloyfiKjkX/PA8jWhKINWnOcKdN5lYwdXphgi4gbqks=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=l5Pl1UEG7h8p68lDL5c7Z6jZcDEj3WUdN/Kfd2za6ih3FqUs0S/iSGqjzWqjyFw69egnBV8YzZt6tm5YK/O1lW/WbiyAYcKpoCJsVyxZP9rOn2H+0RtER8YMq2YtKfEnW7mnZAkfa3P3ZGmOxUaYstf4UJo63svZQvtbhmzk/Gc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=epNuqUzg; arc=none smtp.client-ip=217.70.183.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="epNuqUzg" Received: by mail.gandi.net (Postfix) with ESMTPSA id 68733439F5; Fri, 11 Apr 2025 20:32:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1744403543; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E1LEEOj+dRfAxqML0b7ZiCS95DdAAf0qejLXDfRihnk=; b=epNuqUzgNVnwVYaVaVvzgEHjHohpnkYYZdTN5Cs1TMjSCVY5TnZv/cwwLKcoByehiP9mQv Slcom2nwmqxj/wDz0krvJG7A5sbXZDaCvPfpP+mo3X5PPZ4XyevvfxmvhNgX2nUwI39M/t 6PeRwfRO0fp9OGqWj3Q+NyRjB1s0414UbuV2X58QYWd3JpDDlSlxI+4OhbbOWQOuniFtfZ Z6dUQuBpF74Iv/gSZFntRnLUx2WOkzzFoT76wH1/jpnd71MJrdQs+qIuk22cocqtbFVQgj 27e6YPQySVwhGPcTmXduwVhxOV++HOyrWkpA3NSgL2PhUrhiZa3Hz114eBazag== From: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= Date: Fri, 11 Apr 2025 22:32:12 +0200 Subject: [PATCH RFC bpf-next 3/4] bpf/selftests: add tests to validate proper arguments alignment on ARM64 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250411-many_args_arm64-v1-3-0a32fe72339e@bootlin.com> References: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> In-Reply-To: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Mykola Lysenko , Shuah Khan , Maxime Coquelin , Alexandre Torgue , Florent Revest Cc: Bastien Curutchet , ebpf@linuxfoundation.org, Thomas Petazzoni , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, =?utf-8?q?Alexis_Lothor=C3=A9_?= =?utf-8?q?=28eBPF_Foundation=29?= X-Mailer: b4 0.14.2 X-GND-State: clean X-GND-Score: -100 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvuddvjeekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthekredtredtjeenucfhrhhomheptehlvgigihhsucfnohhthhhorhorucdlvgeurffhucfhohhunhgurghtihhonhdmuceorghlvgigihhsrdhlohhthhhorhgvsegsohhothhlihhnrdgtohhmqeenucggtffrrghtthgvrhhnpeeljeektefffeevleegkeelhfethffgieegudevffejheelieeffeejtddujeegueenucfkphepvdgrtddvmeekgedvkeemfhelgegtmegvtddtmeemfhekheenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepihhnvghtpedvrgdtvdemkeegvdekmehfleegtgemvgdttdemmehfkeehpdhhvghloheplgduledvrdduieekrddurdduleejngdpmhgrihhlfhhrohhmpegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdpnhgspghrtghpthhtohepfedtpdhrtghpthhtohepshgufhesfhhomhhitghhvghvrdhmvgdprhgtphhtthhopehmrghrthhinhdrlhgruheslhhinhhugidruggvvhdprhgtphhtthhopeiguhhkuhhohhgriheshhhurgifvghitghlohhuugdrtghomhdprhgtphhtthhopehjohhlshgrsehkvghrnhgvl hdrohhrghdprhgtphhtthhopegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdprhgtphhtthhopehmhihkohhlrghlsehfsgdrtghomhdprhgtphhtthhopeifihhllheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepmhgtohhquhgvlhhinhdrshhtmhefvdesghhmrghilhdrtghomh X-GND-Sasl: alexis.lothore@bootlin.com When dealing with large types (>8 bytes), ARM64 trampolines need to take extra care about the arguments alignment to respect the calling convention set by AAPCS64. Add two tests ensuring that the BPF trampoline arranges arguments with the relevant layout. The two new tests involve almost the same arguments, except that the second one requires a more specific alignment to be set by the trampoline when preparing arguments before calling the the target function. Signed-off-by: Alexis Lothoré (eBPF Foundation) --- .../selftests/bpf/prog_tests/tracing_struct.c | 23 ++++++++ tools/testing/selftests/bpf/progs/tracing_struct.c | 10 +++- .../selftests/bpf/progs/tracing_struct_many_args.c | 67 ++++++++++++++++++++++ .../testing/selftests/bpf/test_kmods/bpf_testmod.c | 50 ++++++++++++++++ 4 files changed, 149 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/bpf/prog_tests/tracing_struct.c b/tools/testing/selftests/bpf/prog_tests/tracing_struct.c index 19e68d4b353278bf8e2917e62f62c89d14d7fe80..ecb5d38539f8b2fc275f93713ce2a7aad908b929 100644 --- a/tools/testing/selftests/bpf/prog_tests/tracing_struct.c +++ b/tools/testing/selftests/bpf/prog_tests/tracing_struct.c @@ -108,6 +108,29 @@ static void test_struct_many_args(void) ASSERT_EQ(skel->bss->t9_i, 27, "t9:i"); ASSERT_EQ(skel->bss->t9_ret, 258, "t9 ret"); + ASSERT_EQ(skel->bss->t10_a_a, 27, "t10:a.a"); + ASSERT_EQ(skel->bss->t10_a_b, 28, "t10:a.b"); + ASSERT_EQ(skel->bss->t10_b_a, 29, "t10:b.a"); + ASSERT_EQ(skel->bss->t10_b_b, 30, "t10:b.b"); + ASSERT_EQ(skel->bss->t10_c_a, 31, "t10:c.a"); + ASSERT_EQ(skel->bss->t10_c_b, 32, "t10:c.b"); + ASSERT_EQ(skel->bss->t10_d_a, 33, "t10:d.a"); + ASSERT_EQ(skel->bss->t10_d_b, 34, "t10:d.b"); + ASSERT_EQ(skel->bss->t10_e, 35, "t10:e"); + ASSERT_EQ(skel->bss->t10_f_a, 36, "t10:f.a"); + ASSERT_EQ(skel->bss->t10_f_b, 37, "t10:f.b"); + + ASSERT_EQ(skel->bss->t10_ret, 352, "t10 ret"); + + ASSERT_EQ(skel->bss->t11_a, 38, "t11:a"); + ASSERT_EQ(skel->bss->t11_b, 39, "t11:b"); + ASSERT_EQ(skel->bss->t11_c, 40, "t11:c"); + ASSERT_EQ(skel->bss->t11_d, 41, "t11:d"); + ASSERT_EQ(skel->bss->t11_e, 42, "t11:e"); + ASSERT_EQ(skel->bss->t11_f, 43, "t11:f"); + + ASSERT_EQ(skel->bss->t11_ret, 243, "t11 ret"); + destroy_skel: tracing_struct_many_args__destroy(skel); } diff --git a/tools/testing/selftests/bpf/progs/tracing_struct.c b/tools/testing/selftests/bpf/progs/tracing_struct.c index c435a3a8328ab1580c63967a7f0ab779aa7ca4d4..feabf618a2e011b0d08eeaa6cc09fba1922ecb3f 100644 --- a/tools/testing/selftests/bpf/progs/tracing_struct.c +++ b/tools/testing/selftests/bpf/progs/tracing_struct.c @@ -18,6 +18,15 @@ struct bpf_testmod_struct_arg_3 { int b[]; }; +struct bpf_testmod_struct_arg_4 { + __u64 a; + __u64 b; +}; + +struct bpf_testmod_struct_arg_5 { + __int128 a; +}; + long t1_a_a, t1_a_b, t1_b, t1_c, t1_ret, t1_nregs; __u64 t1_reg0, t1_reg1, t1_reg2, t1_reg3; long t2_a, t2_b_a, t2_b_b, t2_c, t2_ret; @@ -25,7 +34,6 @@ long t3_a, t3_b, t3_c_a, t3_c_b, t3_ret; long t4_a_a, t4_b, t4_c, t4_d, t4_e_a, t4_e_b, t4_ret; long t5_ret; int t6; - SEC("fentry/bpf_testmod_test_struct_arg_1") int BPF_PROG2(test_struct_arg_1, struct bpf_testmod_struct_arg_2, a, int, b, int, c) { diff --git a/tools/testing/selftests/bpf/progs/tracing_struct_many_args.c b/tools/testing/selftests/bpf/progs/tracing_struct_many_args.c index 4742012ace06af949d7f15a21131aaef7ab006e4..6f086b3d32d5f89e426aa4b79daa24bdb42861db 100644 --- a/tools/testing/selftests/bpf/progs/tracing_struct_many_args.c +++ b/tools/testing/selftests/bpf/progs/tracing_struct_many_args.c @@ -18,6 +18,14 @@ struct bpf_testmod_struct_arg_5 { long t7_a, t7_b, t7_c, t7_d, t7_e, t7_f_a, t7_f_b, t7_ret; long t8_a, t8_b, t8_c, t8_d, t8_e, t8_f_a, t8_f_b, t8_g, t8_ret; long t9_a, t9_b, t9_c, t9_d, t9_e, t9_f, t9_g, t9_h_a, t9_h_b, t9_h_c, t9_h_d, t9_i, t9_ret; +__u64 t10_a_a, t10_a_b, t10_b_a, t10_b_b, t10_c_a, t10_c_b, t10_d_a, t10_d_b, + t10_f_a, t10_f_b; +short t10_e; +int t10_ret; +__int128 t11_a, t11_b, t11_c, t11_d, t11_f; +short t11_e; +int t11_ret; + SEC("fentry/bpf_testmod_test_struct_arg_7") int BPF_PROG2(test_struct_many_args_1, __u64, a, void *, b, short, c, int, d, @@ -92,4 +100,63 @@ int BPF_PROG2(test_struct_many_args_6, __u64, a, void *, b, short, c, int, d, vo return 0; } +SEC("fentry/bpf_testmod_test_struct_arg_10") +int BPF_PROG2(test_struct_many_args_7, struct bpf_testmod_struct_arg_4, a, + struct bpf_testmod_struct_arg_4, b, + struct bpf_testmod_struct_arg_4, c, + struct bpf_testmod_struct_arg_4, d, short, e, + struct bpf_testmod_struct_arg_4, f) +{ + t10_a_a = a.a; + t10_a_b = a.b; + t10_b_a = b.a; + t10_b_b = b.b; + t10_c_a = c.a; + t10_c_b = c.b; + t10_d_a = d.a; + t10_d_b = d.b; + t10_e = e; + t10_f_a = f.a; + t10_f_b = f.b; + return 0; +} + +SEC("fexit/bpf_testmod_test_struct_arg_10") +int BPF_PROG2(test_struct_many_args_8, struct bpf_testmod_struct_arg_4, a, + struct bpf_testmod_struct_arg_4, b, + struct bpf_testmod_struct_arg_4, c, + struct bpf_testmod_struct_arg_4, d, short, e, + struct bpf_testmod_struct_arg_4, f, int, ret) +{ + t10_ret = ret; + return 0; +} + +SEC("fentry/bpf_testmod_test_struct_arg_11") +int BPF_PROG2(test_struct_many_args_9, struct bpf_testmod_struct_arg_5, a, + struct bpf_testmod_struct_arg_5, b, + struct bpf_testmod_struct_arg_5, c, + struct bpf_testmod_struct_arg_5, d, int, e, + struct bpf_testmod_struct_arg_5, f) +{ + t11_a = a.a; + t11_b = b.a; + t11_c = c.a; + t11_d = d.a; + t11_e = e; + t11_f = f.a; + return 0; +} + +SEC("fexit/bpf_testmod_test_struct_arg_11") +int BPF_PROG2(test_struct_many_args_10, struct bpf_testmod_struct_arg_5, a, + struct bpf_testmod_struct_arg_5, b, + struct bpf_testmod_struct_arg_5, c, + struct bpf_testmod_struct_arg_5, d, int, e, + struct bpf_testmod_struct_arg_5, f, int, ret) +{ + t11_ret = ret; + return 0; +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c index f38eaf0d35efa712ec288f06983c86b02c0d3e0e..96c80da725c978f2e48df8602dd63155971a7bf6 100644 --- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c +++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c @@ -62,6 +62,15 @@ struct bpf_testmod_struct_arg_5 { long d; }; +struct bpf_testmod_struct_arg_6 { + u64 a; + u64 b; +}; + +struct bpf_testmod_struct_arg_7 { + __int128 a; +}; + __bpf_hook_start(); noinline int @@ -128,6 +137,29 @@ bpf_testmod_test_struct_arg_9(u64 a, void *b, short c, int d, void *e, char f, return bpf_testmod_test_struct_arg_result; } +noinline int bpf_testmod_test_struct_arg_10(struct bpf_testmod_struct_arg_6 a, + struct bpf_testmod_struct_arg_6 b, + struct bpf_testmod_struct_arg_6 c, + struct bpf_testmod_struct_arg_6 d, + short e, + struct bpf_testmod_struct_arg_6 f) +{ + bpf_testmod_test_struct_arg_result = + a.a + a.b + b.a + b.b + c.a + c.b + d.a + d.b + e + f.a + f.b; + return bpf_testmod_test_struct_arg_result; +} + +noinline int bpf_testmod_test_struct_arg_11(struct bpf_testmod_struct_arg_7 a, + struct bpf_testmod_struct_arg_7 b, + struct bpf_testmod_struct_arg_7 c, + struct bpf_testmod_struct_arg_7 d, + short e, + struct bpf_testmod_struct_arg_7 f) +{ + bpf_testmod_test_struct_arg_result = a.a + b.a + c.a + d.a + e + f.a; + return bpf_testmod_test_struct_arg_result; +} + noinline int bpf_testmod_test_arg_ptr_to_struct(struct bpf_testmod_struct_arg_1 *a) { bpf_testmod_test_struct_arg_result = a->a; @@ -394,6 +426,16 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, struct bpf_testmod_struct_arg_3 *struct_arg3; struct bpf_testmod_struct_arg_4 struct_arg4 = {21, 22}; struct bpf_testmod_struct_arg_5 struct_arg5 = {23, 24, 25, 26}; + struct bpf_testmod_struct_arg_6 struct_arg6_a = {27, 28}; + struct bpf_testmod_struct_arg_6 struct_arg6_b = {29, 30}; + struct bpf_testmod_struct_arg_6 struct_arg6_c = {31, 32}; + struct bpf_testmod_struct_arg_6 struct_arg6_d = {33, 34}; + struct bpf_testmod_struct_arg_6 struct_arg6_f = {36, 37}; + struct bpf_testmod_struct_arg_7 struct_arg7_a = {38}; + struct bpf_testmod_struct_arg_7 struct_arg7_b = {39}; + struct bpf_testmod_struct_arg_7 struct_arg7_c = {40}; + struct bpf_testmod_struct_arg_7 struct_arg7_d = {41}; + struct bpf_testmod_struct_arg_7 struct_arg7_f = {43}; int i = 1; while (bpf_testmod_return_ptr(i)) @@ -411,6 +453,14 @@ bpf_testmod_test_read(struct file *file, struct kobject *kobj, (void)bpf_testmod_test_struct_arg_9(16, (void *)17, 18, 19, (void *)20, 21, 22, struct_arg5, 27); + (void)bpf_testmod_test_struct_arg_10(struct_arg6_a, struct_arg6_b, + struct_arg6_c, struct_arg6_d, 35, + struct_arg6_f); + + (void)bpf_testmod_test_struct_arg_11(struct_arg7_a, struct_arg7_b, + struct_arg7_c, struct_arg7_d, 42, + struct_arg7_f); + (void)bpf_testmod_test_arg_ptr_to_struct(&struct_arg1_2); (void)trace_bpf_testmod_test_raw_tp_null(NULL); From patchwork Fri Apr 11 20:32:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= X-Patchwork-Id: 880386 Received: from relay9-d.mail.gandi.net (relay9-d.mail.gandi.net [217.70.183.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C25678F45; Fri, 11 Apr 2025 20:32:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.70.183.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403550; cv=none; b=uPWVVQnBbma+um1Teb7FxqWW1DDVAdXOhp6e8PGIjNbXi0KEsjgXOt61u4is+pc0ptoU9FDRFr1DkSXeGn1jI/GZVB0mJ8xGdulrw+AdhwhE6JtcNS++6qZ6dnS7tNSmoGpsggx4tBT2shzHtRzJaGmmvn1382ZmSqZFVndfL84= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403550; c=relaxed/simple; bh=ZeVDLiQI9RmmI4E45qBwDAYffUDqetaFi3++XzpfqNU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=FB0C1zHjMiqMb0XY2b/0YJMwZC6AGf9bN3srCkH2vTCqfmttIfCYkcEXm/tWu5mUwFVPyxT6jNgwprdMg6Rdlnx3YgqDoqaRhv8EVwbLU+Gj8rGwi4u8JAG1MNB5lTcgmOxSwU0oAtKRGO73NMdQhzy2H8nuGbqH+TJkgy8WoFw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com; spf=pass smtp.mailfrom=bootlin.com; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b=bUWKuQ0E; arc=none smtp.client-ip=217.70.183.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=bootlin.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=bootlin.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=bootlin.com header.i=@bootlin.com header.b="bUWKuQ0E" Received: by mail.gandi.net (Postfix) with ESMTPSA id 0E33A439F7; Fri, 11 Apr 2025 20:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bootlin.com; s=gm1; t=1744403545; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ubuKm8CEtpcOaMisxU9dLcEBpCyqH9xv+s9t40knzhQ=; b=bUWKuQ0Ec7Enbe5JweK3jfBowxndLsuHmjklLcqXu/lZEHUjGcp7upJ6wZH1C1BqVyxaJA 1VPtBdBvfNwDNGMbavInD66RnojP+d3tiYiz4rQ/rfU7+VjZCw7t0z4SUf0A2dl28VsMlW bKj/3cl9+WRGD/o2iarX2ut3XCzdcFnNdWIrZ7cNtjwotm+sgGc4nMAXVrcPbFQxXRc+wq 3/JAjqYxwxUE538Z89Y7ex+mSgfcx6hT0KxhkGBmLX4q+0rUNkmyzzrUWRkwJj21wBet18 ubu/5mUcIQ4t8Azcc6pWFR+X6dBQMewHzeMmXAsJ0vTxWwv33GBClEG/GXJQRg== From: =?utf-8?q?Alexis_Lothor=C3=A9_=28eBPF_Foundation=29?= Date: Fri, 11 Apr 2025 22:32:13 +0200 Subject: [PATCH RFC bpf-next 4/4] bpf/selftests: enable tracing tests for ARM64 Precedence: bulk X-Mailing-List: linux-kselftest@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20250411-many_args_arm64-v1-4-0a32fe72339e@bootlin.com> References: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> In-Reply-To: <20250411-many_args_arm64-v1-0-0a32fe72339e@bootlin.com> To: Alexei Starovoitov , Daniel Borkmann , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Eduard Zingerman , Song Liu , Yonghong Song , KP Singh , Stanislav Fomichev , Hao Luo , Jiri Olsa , Puranjay Mohan , Xu Kuohai , Catalin Marinas , Will Deacon , Mykola Lysenko , Shuah Khan , Maxime Coquelin , Alexandre Torgue , Florent Revest Cc: Bastien Curutchet , ebpf@linuxfoundation.org, Thomas Petazzoni , bpf@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-stm32@st-md-mailman.stormreply.com, =?utf-8?q?Alexis_Lothor=C3=A9_?= =?utf-8?q?=28eBPF_Foundation=29?= X-Mailer: b4 0.14.2 X-GND-State: clean X-GND-Score: -100 X-GND-Cause: gggruggvucftvghtrhhoucdtuddrgeefvddrtddtgddvuddvjeekucetufdoteggodetrfdotffvucfrrhhofhhilhgvmecuifetpfffkfdpucggtfgfnhhsuhgsshgtrhhisggvnecuuegrihhlohhuthemuceftddunecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenucfjughrpefhfffugggtgffkfhgjvfevofesthekredtredtjeenucfhrhhomheptehlvgigihhsucfnohhthhhorhorucdlvgeurffhucfhohhunhgurghtihhonhdmuceorghlvgigihhsrdhlohhthhhorhgvsegsohhothhlihhnrdgtohhmqeenucggtffrrghtthgvrhhnpeeljeektefffeevleegkeelhfethffgieegudevffejheelieeffeejtddujeegueenucfkphepvdgrtddvmeekgedvkeemfhelgegtmegvtddtmeemfhekheenucevlhhushhtvghrufhiiigvpedvnecurfgrrhgrmhepihhnvghtpedvrgdtvdemkeegvdekmehfleegtgemvgdttdemmehfkeehpdhhvghloheplgduledvrdduieekrddurdduleejngdpmhgrihhlfhhrohhmpegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdpnhgspghrtghpthhtohepfedtpdhrtghpthhtohepshgufhesfhhomhhitghhvghvrdhmvgdprhgtphhtthhopehmrghrthhinhdrlhgruheslhhinhhugidruggvvhdprhgtphhtthhopeiguhhkuhhohhgriheshhhurgifvghitghlohhuugdrtghomhdprhgtphhtthhopehjohhlshgrsehkvghrnhgvl hdrohhrghdprhgtphhtthhopegrlhgvgihishdrlhhothhhohhrvgessghoohhtlhhinhdrtghomhdprhgtphhtthhopehmhihkohhlrghlsehfsgdrtghomhdprhgtphhtthhopeifihhllheskhgvrhhnvghlrdhorhhgpdhrtghpthhtohepmhgtohhquhgvlhhinhdrshhtmhefvdesghhmrghilhdrtghomh X-GND-Sasl: alexis.lothore@bootlin.com The fentry_many_args, fexit_many_args and struct_many_args tests were disabled on ARM64 due to the lack of many args support. With the previous commits bringing in this missing support, drop the last denied tests. Signed-off-by: Alexis Lothoré (eBPF Foundation) --- tools/testing/selftests/bpf/DENYLIST.aarch64 | 3 --- 1 file changed, 3 deletions(-) diff --git a/tools/testing/selftests/bpf/DENYLIST.aarch64 b/tools/testing/selftests/bpf/DENYLIST.aarch64 deleted file mode 100644 index 6d8feda27ce9de07d77d6e384666082923e3dc76..0000000000000000000000000000000000000000 --- a/tools/testing/selftests/bpf/DENYLIST.aarch64 +++ /dev/null @@ -1,3 +0,0 @@ -fentry_test/fentry_many_args # fentry_many_args:FAIL:fentry_many_args_attach unexpected error: -524 -fexit_test/fexit_many_args # fexit_many_args:FAIL:fexit_many_args_attach unexpected error: -524 -tracing_struct/struct_many_args # struct_many_args:FAIL:tracing_struct_many_args__attach unexpected error: -524