From patchwork Mon Aug 30 17:34:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 504561 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BFA99C4320A for ; Mon, 30 Aug 2021 17:34:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A4AEF60F6C for ; Mon, 30 Aug 2021 17:34:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238233AbhH3Rfk (ORCPT ); Mon, 30 Aug 2021 13:35:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238217AbhH3Rf2 (ORCPT ); Mon, 30 Aug 2021 13:35:28 -0400 Received: from mail-pl1-x642.google.com (mail-pl1-x642.google.com [IPv6:2607:f8b0:4864:20::642]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A04CC06175F; Mon, 30 Aug 2021 10:34:34 -0700 (PDT) Received: by mail-pl1-x642.google.com with SMTP id u1so5160158plq.5; Mon, 30 Aug 2021 10:34:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=K5cq3ITD1cfF+ztNNUFbtlqiEu6d9XvUJN2jwcM08J4=; b=To9RAbtuLqECfVSOYEkumdLxk5WGgl6Sjb5CcBHkg5N189Dmk9SCqjpu8Iqv4G21Ge 0KOcnF7aYZG7Ev2ogowCz3UKeJcYsDV47ozkmwiO1CxI/jHwb954u9Y4QEq2wimGyMam nul9l41kCjfWzNYYOR0FWzQHzSAUu10KiQx2igLtoIxniUYRY+NOWjm7bOXq7tq9S2xS siNMbTBfddV4t9aU4XJYyiX0s18XZxwjEXOHDXsrTPlLq5VlWSkZ/r7V9qMVdaPSKU5n PG3zGNB/0ds+iJ3BF87AUHNsgU8FA1Zg/zVHuw5Xs7QlcI8Q4GbIf5Uel7hvRBvCqoum MPzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=K5cq3ITD1cfF+ztNNUFbtlqiEu6d9XvUJN2jwcM08J4=; b=CFX9o/3kbkBaqA2KUsJlBiz0pwDMfQKMG76QMjs3cIX0FMfnQJsRjehNzt0xtOdpcT exSz9SMlfKy5iPf0sEssDtsmXgEIvtoufZcEoBEtHE3LEdJGomRxsZUv1Ow39uW7uvph 7HaFOTPspp3AKw9M813xbpl/v5TYyNf+DVXymb38FlyiDQppfoT0viyVXbooPuJ06oCV t4GlbntVO5HquOfdU5gyEnjtjn+ZxJmPCIdTgSlTgdv8Z3iwJVB06Utx3sHvi4KyNihv TiO9XWxYKTkhS9xhhmlJiLZ6UCDDh7JI8PVYU/F58jmz2cIsvQN2vxzRR3plNcIEnyAT GQMw== X-Gm-Message-State: AOAM530mXxveB1SN+h9w5oJv7cVfHFg4zqRHmlII1CuSz7lSxCfWStV4 DSC1SQqM5FGBD/BFEjdRqlXbyk/8jKFOSg== X-Google-Smtp-Source: ABdhPJwtgkcWKIw7zPEE2iZtUtKG+Q9LbhajhTE/azKkMgGIJKW8CDfUo15sp8u2fBdHAwKwU+QUpQ== X-Received: by 2002:a17:902:7484:b0:135:8c25:d33a with SMTP id h4-20020a170902748400b001358c25d33amr664020pll.83.1630344873791; Mon, 30 Aug 2021 10:34:33 -0700 (PDT) Received: from localhost ([2405:201:6014:d820:9cc6:d37f:c2fd:dc6]) by smtp.gmail.com with ESMTPSA id k3sm14540717pfc.16.2021.08.30.10.34.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 10:34:33 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?b?w7hyZ2Vuc2Vu?= , netdev@vger.kernel.org Subject: [PATCH bpf-next RFC v1 1/8] bpf: Introduce BPF support for kernel module function calls Date: Mon, 30 Aug 2021 23:04:17 +0530 Message-Id: <20210830173424.1385796-2-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210830173424.1385796-1-memxor@gmail.com> References: <20210830173424.1385796-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=15018; h=from:subject; bh=j2skxd1DWGnmyG1yn9IaiA+KLdmbZHDLfDIgAt6hYv8=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhLRX9O2tGNtGap/T0Fu0mmE9JIUBocNBfVfVGn0TC hvx0+UKJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYS0V/QAKCRBM4MiGSL8RyixUD/ 91WSUE0SLEILreVthPZMXlgTtuQp3ImmazoznoQAijiLxv/oSzY7KKQURnK+5UdZ9CaZYWCblWofBL K7cRaKgrl4qEA2MInojuPP6WqEOJTZLeROfM2oaHZ/Et+xC9WGT+yAyAzpxUwCSEeNp7Mcx0V+3MRY kJGC2QDyybL0TZZOoZja36ZWCKL8CY73PFfGQNHOTLECx0YstjpTeXAA8iHbOSrGurW/jYC3qrDSKF 2xs2ch+KWYD9wiozphl92mVyMXBISgru0h4E2j/aywUB61D5in1uQ4RC6ipv7H4ofi5PQsgpcvICgu bd6PoFQtDhvVKZZSXiea5y3AiLsVMj3lpeJStq2cZeqTPiKaXqqGm/D4mJrYzJ9iwIoAYYKsrKO6JW PWbiuoFpSGMfvdQVan7q37mKUEpQdNwXd0YQu0/XhtHJ4gDTZvBFS/SxOPINFA1utZnWsScOCq1cnH brQqKxcAOWisHjthSF185/euzV2i5yqOTiKRqXqk0aFqLvKM65OZv1AgNLlObNQ7hi+gjl0C+MOCLq tJ6My4H+A2UWICZJZ0mQHz8DniNn3jxFn+pZu9VbMlJqj5bRc2tSJZbtYiBnBF1tODPxNGqZq5+UJ+ TwTegMXS8IMhA01zX0IINcsIiaRJ59RRZhHdVA/IZpwBT7c6CQk4WL/C6lBw== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This change adds support on the kernel side to allow for BPF programs to call kernel module functions. Userspace will prepare an array of module BTF fds that is passed in during BPF_PROG_LOAD. In the kernel, the module BTF array is placed in the auxilliary struct for bpf_prog. The verifier then uses insn->off to index into this table. insn->off is used by subtracting one from it, as userspace has to set the index of array in insn->off incremented by 1. This lets us denote vmlinux btf by insn->off == 0, and the prog->aux->kfunc_btf_tab[insn->off - 1] for module BTFs. Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/bpf.h | 1 + include/linux/filter.h | 9 ++++ include/uapi/linux/bpf.h | 3 +- kernel/bpf/core.c | 14 ++++++ kernel/bpf/syscall.c | 55 +++++++++++++++++++++- kernel/bpf/verifier.c | 85 ++++++++++++++++++++++++++-------- tools/include/uapi/linux/bpf.h | 3 +- 7 files changed, 147 insertions(+), 23 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index f4c16f19f83e..39f59e5f3a26 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -874,6 +874,7 @@ struct bpf_prog_aux { void *jit_data; /* JIT specific data. arch dependent */ struct bpf_jit_poke_descriptor *poke_tab; struct bpf_kfunc_desc_tab *kfunc_tab; + struct bpf_kfunc_btf_tab *kfunc_btf_tab; u32 size_poke_tab; struct bpf_ksym ksym; const struct bpf_prog_ops *ops; diff --git a/include/linux/filter.h b/include/linux/filter.h index 7d248941ecea..46451891633d 100644 --- a/include/linux/filter.h +++ b/include/linux/filter.h @@ -592,6 +592,15 @@ struct bpf_prog { struct bpf_insn insnsi[]; }; +#define MAX_KFUNC_DESCS 256 +/* There can only be at most MAX_KFUNC_DESCS module BTFs for kernel module + * function calls. + */ +struct bpf_kfunc_btf_tab { + u32 nr_btfs; + struct btf_mod_pair btfs[]; +}; + struct sk_filter { refcount_t refcnt; struct rcu_head rcu; diff --git a/include/uapi/linux/bpf.h b/include/uapi/linux/bpf.h index 791f31dd0abe..4cbb2082a553 100644 --- a/include/uapi/linux/bpf.h +++ b/include/uapi/linux/bpf.h @@ -1334,8 +1334,9 @@ union bpf_attr { /* or valid module BTF object fd or 0 to attach to vmlinux */ __u32 attach_btf_obj_fd; }; - __u32 :32; /* pad */ + __u32 kfunc_btf_fds_cnt; /* reuse hole for count of BTF fds below */ __aligned_u64 fd_array; /* array of FDs */ + __aligned_u64 kfunc_btf_fds; /* array of BTF FDs for module kfunc support */ }; struct { /* anonymous struct used by BPF_OBJ_* commands */ diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c index 9f4636d021b1..73ba6d862df3 100644 --- a/kernel/bpf/core.c +++ b/kernel/bpf/core.c @@ -2249,12 +2249,26 @@ static void bpf_free_used_btfs(struct bpf_prog_aux *aux) kfree(aux->used_btfs); } +static void bpf_free_kfunc_btf_tab(struct bpf_prog_aux *aux) +{ + struct bpf_kfunc_btf_tab *tab = aux->kfunc_btf_tab; + + if (tab) { + while (tab->nr_btfs--) { + module_put(tab->btfs[tab->nr_btfs].module); + btf_put(tab->btfs[tab->nr_btfs].btf); + } + kfree(tab); + } +} + static void bpf_prog_free_deferred(struct work_struct *work) { struct bpf_prog_aux *aux; int i; aux = container_of(work, struct bpf_prog_aux, work); + bpf_free_kfunc_btf_tab(aux); bpf_free_used_maps(aux); bpf_free_used_btfs(aux); if (bpf_prog_is_dev_bound(aux)) diff --git a/kernel/bpf/syscall.c b/kernel/bpf/syscall.c index 4e50c0bfdb7d..bbbd664b2872 100644 --- a/kernel/bpf/syscall.c +++ b/kernel/bpf/syscall.c @@ -2156,16 +2156,16 @@ static bool is_perfmon_prog_type(enum bpf_prog_type prog_type) } /* last field in 'union bpf_attr' used by this command */ -#define BPF_PROG_LOAD_LAST_FIELD fd_array +#define BPF_PROG_LOAD_LAST_FIELD kfunc_btf_fds static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) { enum bpf_prog_type type = attr->prog_type; struct bpf_prog *prog, *dst_prog = NULL; struct btf *attach_btf = NULL; - int err; char license[128]; bool is_gpl; + int err; if (CHECK_ATTR(BPF_PROG_LOAD)) return -EINVAL; @@ -2204,6 +2204,8 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) return -EPERM; if (is_perfmon_prog_type(type) && !perfmon_capable()) return -EPERM; + if (attr->kfunc_btf_fds_cnt > MAX_KFUNC_DESCS) + return -E2BIG; /* attach_prog_fd/attach_btf_obj_fd can specify fd of either bpf_prog * or btf, we need to check which one it is @@ -2254,6 +2256,55 @@ static int bpf_prog_load(union bpf_attr *attr, bpfptr_t uattr) return -ENOMEM; } + if (attr->kfunc_btf_fds_cnt) { + struct bpf_kfunc_btf_tab *tab; + int fds[MAX_KFUNC_DESCS], i; + bpfptr_t kfunc_btf_fds; + u32 kfunc_btf_size, n; + + kfunc_btf_size = min_t(u32, MAX_KFUNC_DESCS, attr->kfunc_btf_fds_cnt); + kfunc_btf_fds = make_bpfptr(attr->kfunc_btf_fds, uattr.is_kernel); + + err = -EFAULT; + if (copy_from_bpfptr(fds, kfunc_btf_fds, kfunc_btf_size * sizeof(int))) + goto free_prog; + + err = -ENOMEM; + + n = kfunc_btf_size; + kfunc_btf_size *= sizeof(prog->aux->kfunc_btf_tab->btfs[0]); + kfunc_btf_size += sizeof(*prog->aux->kfunc_btf_tab); + prog->aux->kfunc_btf_tab = kzalloc(kfunc_btf_size, GFP_KERNEL); + if (!prog->aux->kfunc_btf_tab) + goto free_prog; + + tab = prog->aux->kfunc_btf_tab; + for (i = 0; i < n; i++) { + struct btf_mod_pair *p; + struct btf *mod_btf; + + mod_btf = btf_get_by_fd(fds[i]); + if (IS_ERR(mod_btf)) { + err = PTR_ERR(mod_btf); + goto free_prog; + } + if (!btf_is_module(mod_btf)) { + err = -EINVAL; + btf_put(mod_btf); + goto free_prog; + } + + p = &tab->btfs[tab->nr_btfs]; + p->module = btf_try_get_module(mod_btf); + if (!p->module) { + btf_put(mod_btf); + goto free_prog; + } + p->btf = mod_btf; + tab->nr_btfs++; + } + } + prog->expected_attach_type = attr->expected_attach_type; prog->aux->attach_btf = attach_btf; prog->aux->attach_btf_id = attr->attach_btf_id; diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 206c221453cf..de0670a8b1df 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -1632,7 +1632,6 @@ struct bpf_kfunc_desc { s32 imm; }; -#define MAX_KFUNC_DESCS 256 struct bpf_kfunc_desc_tab { struct bpf_kfunc_desc descs[MAX_KFUNC_DESCS]; u32 nr_descs; @@ -1660,13 +1659,45 @@ find_kfunc_desc(const struct bpf_prog *prog, u32 func_id) sizeof(tab->descs[0]), kfunc_desc_cmp_by_id); } -static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) +static struct btf *find_kfunc_desc_btf(struct bpf_verifier_env *env, + u32 func_id, s16 offset) +{ + struct bpf_kfunc_btf_tab *btf_tab; + + btf_tab = env->prog->aux->kfunc_btf_tab; + /* offset can be MAX_KFUNC_DESCS, since index into the array is offset - 1, + * as we reserve offset == 0 for btf_vmlinux + */ + if (offset < 0 || offset > MAX_KFUNC_DESCS) { + verbose(env, "offset %d is incorrect for kernel function call\n", (int)offset); + return ERR_PTR(-EINVAL); + } + + if (offset) { + if (!btf_tab) { + verbose(env, + "offset %d for kfunc call but no kernel module BTFs passed\n", + (int)offset); + return ERR_PTR(-EINVAL); + } else if (offset > btf_tab->nr_btfs) { + verbose(env, + "offset %d incorrect for module BTF array with %u descriptors\n", + (int)offset, btf_tab->nr_btfs); + return ERR_PTR(-EINVAL); + } + return btf_tab->btfs[offset - 1].btf; + } + return btf_vmlinux ?: ERR_PTR(-ENOENT); +} + +static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id, s16 offset) { const struct btf_type *func, *func_proto; struct bpf_kfunc_desc_tab *tab; struct bpf_prog_aux *prog_aux; struct bpf_kfunc_desc *desc; const char *func_name; + struct btf *desc_btf; unsigned long addr; int err; @@ -1699,6 +1730,12 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) prog_aux->kfunc_tab = tab; } + desc_btf = find_kfunc_desc_btf(env, func_id, offset); + if (IS_ERR(desc_btf)) { + verbose(env, "failed to find BTF for kernel function\n"); + return PTR_ERR(desc_btf); + } + if (find_kfunc_desc(env->prog, func_id)) return 0; @@ -1707,20 +1744,20 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) return -E2BIG; } - func = btf_type_by_id(btf_vmlinux, func_id); + func = btf_type_by_id(desc_btf, func_id); if (!func || !btf_type_is_func(func)) { verbose(env, "kernel btf_id %u is not a function\n", func_id); return -EINVAL; } - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func_proto = btf_type_by_id(desc_btf, func->type); if (!func_proto || !btf_type_is_func_proto(func_proto)) { verbose(env, "kernel function btf_id %u does not have a valid func_proto\n", func_id); return -EINVAL; } - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); + func_name = btf_name_by_offset(desc_btf, func->name_off); addr = kallsyms_lookup_name(func_name); if (!addr) { verbose(env, "cannot find address for kernel function %s\n", @@ -1731,7 +1768,7 @@ static int add_kfunc_call(struct bpf_verifier_env *env, u32 func_id) desc = &tab->descs[tab->nr_descs++]; desc->func_id = func_id; desc->imm = BPF_CAST_CALL(addr) - __bpf_call_base; - err = btf_distill_func_proto(&env->log, btf_vmlinux, + err = btf_distill_func_proto(&env->log, desc_btf, func_proto, func_name, &desc->func_model); if (!err) @@ -1815,7 +1852,7 @@ static int add_subprog_and_kfunc(struct bpf_verifier_env *env) } else if (bpf_pseudo_call(insn)) { ret = add_subprog(env, i + insn->imm + 1); } else { - ret = add_kfunc_call(env, insn->imm); + ret = add_kfunc_call(env, insn->imm, insn->off); } if (ret < 0) @@ -2152,12 +2189,17 @@ static int get_prev_insn_idx(struct bpf_verifier_state *st, int i, static const char *disasm_kfunc_name(void *data, const struct bpf_insn *insn) { const struct btf_type *func; + struct btf *desc_btf; if (insn->src_reg != BPF_PSEUDO_KFUNC_CALL) return NULL; - func = btf_type_by_id(btf_vmlinux, insn->imm); - return btf_name_by_offset(btf_vmlinux, func->name_off); + desc_btf = find_kfunc_desc_btf(data, insn->imm, insn->off); + if (IS_ERR(desc_btf)) + return ""; + + func = btf_type_by_id(desc_btf, insn->imm); + return btf_name_by_offset(desc_btf, func->name_off); } /* For given verifier state backtrack_insn() is called from the last insn to @@ -6482,12 +6524,17 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) const char *func_name, *ptr_type_name; u32 i, nargs, func_id, ptr_type_id; const struct btf_param *args; + struct btf *desc_btf; int err; + desc_btf = find_kfunc_desc_btf(env, insn->imm, insn->off); + if (IS_ERR(desc_btf)) + return PTR_ERR(desc_btf); + func_id = insn->imm; - func = btf_type_by_id(btf_vmlinux, func_id); - func_name = btf_name_by_offset(btf_vmlinux, func->name_off); - func_proto = btf_type_by_id(btf_vmlinux, func->type); + func = btf_type_by_id(desc_btf, func_id); + func_name = btf_name_by_offset(desc_btf, func->name_off); + func_proto = btf_type_by_id(desc_btf, func->type); if (!env->ops->check_kfunc_call || !env->ops->check_kfunc_call(func_id)) { @@ -6497,7 +6544,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) } /* Check the arguments */ - err = btf_check_kfunc_arg_match(env, btf_vmlinux, func_id, regs); + err = btf_check_kfunc_arg_match(env, desc_btf, func_id, regs); if (err) return err; @@ -6505,15 +6552,15 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) mark_reg_not_init(env, regs, caller_saved[i]); /* Check return type */ - t = btf_type_skip_modifiers(btf_vmlinux, func_proto->type, NULL); + t = btf_type_skip_modifiers(desc_btf, func_proto->type, NULL); if (btf_type_is_scalar(t)) { mark_reg_unknown(env, regs, BPF_REG_0); mark_btf_func_reg_size(env, BPF_REG_0, t->size); } else if (btf_type_is_ptr(t)) { - ptr_type = btf_type_skip_modifiers(btf_vmlinux, t->type, + ptr_type = btf_type_skip_modifiers(desc_btf, t->type, &ptr_type_id); if (!btf_type_is_struct(ptr_type)) { - ptr_type_name = btf_name_by_offset(btf_vmlinux, + ptr_type_name = btf_name_by_offset(desc_btf, ptr_type->name_off); verbose(env, "kernel function %s returns pointer type %s %s is not supported\n", func_name, btf_type_str(ptr_type), @@ -6521,7 +6568,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) return -EINVAL; } mark_reg_known_zero(env, regs, BPF_REG_0); - regs[BPF_REG_0].btf = btf_vmlinux; + regs[BPF_REG_0].btf = desc_btf; regs[BPF_REG_0].type = PTR_TO_BTF_ID; regs[BPF_REG_0].btf_id = ptr_type_id; mark_btf_func_reg_size(env, BPF_REG_0, sizeof(void *)); @@ -6532,7 +6579,7 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn) for (i = 0; i < nargs; i++) { u32 regno = i + 1; - t = btf_type_skip_modifiers(btf_vmlinux, args[i].type, NULL); + t = btf_type_skip_modifiers(desc_btf, args[i].type, NULL); if (btf_type_is_ptr(t)) mark_btf_func_reg_size(env, regno, sizeof(void *)); else @@ -11070,7 +11117,6 @@ static int do_check(struct bpf_verifier_env *env) env->jmps_processed++; if (opcode == BPF_CALL) { if (BPF_SRC(insn->code) != BPF_K || - insn->off != 0 || (insn->src_reg != BPF_REG_0 && insn->src_reg != BPF_PSEUDO_CALL && insn->src_reg != BPF_PSEUDO_KFUNC_CALL) || @@ -12425,6 +12471,7 @@ static int jit_subprogs(struct bpf_verifier_env *env) func[i]->aux->stack_depth = env->subprog_info[i].stack_depth; func[i]->jit_requested = 1; func[i]->aux->kfunc_tab = prog->aux->kfunc_tab; + func[i]->aux->kfunc_btf_tab = prog->aux->kfunc_btf_tab; func[i]->aux->linfo = prog->aux->linfo; func[i]->aux->nr_linfo = prog->aux->nr_linfo; func[i]->aux->jited_linfo = prog->aux->jited_linfo; diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 791f31dd0abe..4cbb2082a553 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -1334,8 +1334,9 @@ union bpf_attr { /* or valid module BTF object fd or 0 to attach to vmlinux */ __u32 attach_btf_obj_fd; }; - __u32 :32; /* pad */ + __u32 kfunc_btf_fds_cnt; /* reuse hole for count of BTF fds below */ __aligned_u64 fd_array; /* array of FDs */ + __aligned_u64 kfunc_btf_fds; /* array of BTF FDs for module kfunc support */ }; struct { /* anonymous struct used by BPF_OBJ_* commands */ From patchwork Mon Aug 30 17:34:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 504560 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7DEAEC432BE for ; Mon, 30 Aug 2021 17:34:59 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 65FC060F56 for ; Mon, 30 Aug 2021 17:34:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238275AbhH3Rfw (ORCPT ); Mon, 30 Aug 2021 13:35:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36186 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234054AbhH3Rfe (ORCPT ); Mon, 30 Aug 2021 13:35:34 -0400 Received: from mail-pg1-x543.google.com (mail-pg1-x543.google.com [IPv6:2607:f8b0:4864:20::543]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4669FC061575; Mon, 30 Aug 2021 10:34:40 -0700 (PDT) Received: by mail-pg1-x543.google.com with SMTP id e7so14139843pgk.2; Mon, 30 Aug 2021 10:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=GE2vKVy8HgkCUTYMNvHrEbPsiAi4yZkY5Ysm66dS+LY=; b=awrRi8FO6CibGOLs2sFhOeMm03ZpDiRvRBEgNi5PfEGU7QYHRc/njdT6g8YU6Z7UT5 rBmU1OGU61MJD8ZJj8sMEbQLwOHwPd3RRAmQF7fjnxgvZyonWXg5uj3pR6FTi0Kt3ddf BcsqqIoyA+43NsrcqL48YebgLzwT4ZaHNtoQT32ybkdlTzf4mGJr5kKKRRN5j7HE4WJh VFBjUR36flEngNfNEwAxnE/FacuuC+bLVPAbYdYWjYvbWsZv82RoGh8QXDZXWg5TMueN tP5dSFvHgO3peiBSPP1ZPZd1GIHfrfZhRZ7zT41T8gViOh3mLRsgDPb+0xQleqxGLsYq 6Y7g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=GE2vKVy8HgkCUTYMNvHrEbPsiAi4yZkY5Ysm66dS+LY=; b=fd1j0oOXMO7BXV+OV/L7+FIf03xoLR01byv0qqDdU2tm8iFVtW43RRVOB3UgLULx74 RPWlqvWlvMko0T5GUrDLcFtGE9FKLT0FeFNpo/zpLnq69KXqvH/z/+MrypWOYFcTYplw HoCAEUOyGhRRvZe0gJeMxV6TEBV3itGGn0Q7MU8hgxR9gC4Ep4JPlveInILpRHvtTexC d4AKGmdv1074xenPROZGdWCv5V/yH4UMdszUCHJiFA0oBZ9txD3pCdNBH/wN2ZzZ7NdG 3ON+4IXsrGUO75dPgCsD9KGWirFIP7KeNt3kv+cqY1mTOSCKmVkoG9vyX2WJU56UAI53 lhLw== X-Gm-Message-State: AOAM532+aSOlv+/et6VCFz8ApkCg3+Qrl6/SUJS+afwPSdF/IPJ+Osex v1pMPCAuiOLUKf2LGEe+MGhvBO4wa3yjuw== X-Google-Smtp-Source: ABdhPJwsR8GQewTlzJtrnL6A0TFJgWHpWvPirNKwSSoMkbaJ/Mtles7JPcFDkQKq42IKxkbM/+cC4A== X-Received: by 2002:a63:111d:: with SMTP id g29mr22947935pgl.222.1630344879548; Mon, 30 Aug 2021 10:34:39 -0700 (PDT) Received: from localhost ([2405:201:6014:d820:9cc6:d37f:c2fd:dc6]) by smtp.gmail.com with ESMTPSA id s83sm10637340pfc.204.2021.08.30.10.34.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 10:34:39 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?b?w7hyZ2Vuc2Vu?= , netdev@vger.kernel.org Subject: [PATCH bpf-next RFC v1 3/8] libbpf: Support kernel module function calls Date: Mon, 30 Aug 2021 23:04:19 +0530 Message-Id: <20210830173424.1385796-4-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210830173424.1385796-1-memxor@gmail.com> References: <20210830173424.1385796-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=5110; h=from:subject; bh=f9uEtT2cd1efCyrBHSxXPofXfnm19SbwIJqwSdAJ+UI=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhLRX9FQ7WEk5vF5q019TXwYaMliAXntH+shsv7FxZ AzErmVaJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYS0V/QAKCRBM4MiGSL8RysFfEA C1I+EAmxsguspj3Y/RqmXgZ+ZyZmPCWHAYY9zAtR5uX/taPDPRG64HMzRPjM1VFXFEKSmx09AiAna1 kVX+A9+EFYeWdyuutOaqObE5+wa3sVX9c/BV/RyzO12QcgssR6/ujuWvegfDYhkHqTVgd72EltvSyD f5czsqR1+SPu7vChpdCnLhfZjZXV80tiQpPkGq5Q2nboIFUNLg9Y7bvCFIv3+oR6ZfcTktDA3FVX5w FU9jzoFdiCI+oP2SOAgjOAlTVv5lrYjN+uGIIH8yakAke47Rl1POyNhak9VJNtDmmnCXV1OGK1Sxlp xlI+j6KfMawmL82cO2WyYKadbnkQyFAvrHhomuIl70LnQMRSXeWfGoxFm5LNIPA7PMYWP42Tx8msOi +89GGthbPs7lHNrmjFC0XHrofbVOK8ZUFiG0f+qV6C2wzjOUEOO8YvqUZ6HlRX8rkMumZ77o9lITkk Ds6kEkk+6e8FsqNmHX/gl9ghPaZ67LLnNmvBzluNAkTxAvcl92s66wH3YYunxW5Xo/s2lh5RQ7omJl WRr5t29o2phDewCSQeZUpg524SdYsyqUUpK6WPs9x0xKaqEz7SxXGb5HvugH8DgQCRF3ElRaFbMWyL fhUd05Q0w9FBJrt0Rpd2zPi2VOntntYIx5C3/CDUDfawDB+T+kjXi63hM+qQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Signed-off-by: Kumar Kartikeya Dwivedi --- tools/lib/bpf/bpf.c | 3 ++ tools/lib/bpf/libbpf.c | 71 +++++++++++++++++++++++++++++++-- tools/lib/bpf/libbpf_internal.h | 2 + 3 files changed, 73 insertions(+), 3 deletions(-) diff --git a/tools/lib/bpf/bpf.c b/tools/lib/bpf/bpf.c index 2401fad090c5..df2d1ceba146 100644 --- a/tools/lib/bpf/bpf.c +++ b/tools/lib/bpf/bpf.c @@ -265,6 +265,9 @@ int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr) attr.line_info_cnt = load_attr->line_info_cnt; attr.line_info = ptr_to_u64(load_attr->line_info); + attr.kfunc_btf_fds = ptr_to_u64(load_attr->kfunc_btf_fds); + attr.kfunc_btf_fds_cnt = load_attr->kfunc_btf_fds_cnt; + if (load_attr->name) memcpy(attr.prog_name, load_attr->name, min(strlen(load_attr->name), (size_t)BPF_OBJ_NAME_LEN - 1)); diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c index 88d8825fc6f6..c4677ef97caa 100644 --- a/tools/lib/bpf/libbpf.c +++ b/tools/lib/bpf/libbpf.c @@ -419,6 +419,12 @@ struct extern_desc { /* local btf_id of the ksym extern's type. */ __u32 type_id; + /* offset to be patched in for insn->off, + * this is 0 for btf_vmlinux, and index + 1 + * for module BTF, where index is BTF index in + * obj->kfunc_btf_fds.fds array + */ + __u32 offset; } ksym; }; }; @@ -515,6 +521,13 @@ struct bpf_object { void *priv; bpf_object_clear_priv_t clear_priv; + struct { + struct hashmap *map; + int *fds; + size_t cap_cnt; + __u32 n_fds; + } kfunc_btf_fds; + char path[]; }; #define obj_elf_valid(o) ((o)->efile.elf) @@ -5327,6 +5340,7 @@ bpf_object__relocate_data(struct bpf_object *obj, struct bpf_program *prog) ext = &obj->externs[relo->sym_off]; insn[0].src_reg = BPF_PSEUDO_KFUNC_CALL; insn[0].imm = ext->ksym.kernel_btf_id; + insn[0].off = ext->ksym.offset; break; case RELO_SUBPROG_ADDR: if (insn[0].src_reg != BPF_PSEUDO_FUNC) { @@ -6122,6 +6136,11 @@ load_program(struct bpf_program *prog, struct bpf_insn *insns, int insns_cnt, load_attr.log_level = prog->log_level; load_attr.prog_flags = prog->prog_flags; + if (prog->obj->kfunc_btf_fds.n_fds) { + load_attr.kfunc_btf_fds = prog->obj->kfunc_btf_fds.fds; + load_attr.kfunc_btf_fds_cnt = prog->obj->kfunc_btf_fds.n_fds; + } + if (prog->obj->gen_loader) { bpf_gen__prog_load(prog->obj->gen_loader, &load_attr, prog - prog->obj->programs); @@ -6723,9 +6742,49 @@ static int bpf_object__resolve_ksym_func_btf_id(struct bpf_object *obj, } if (kern_btf != obj->btf_vmlinux) { - pr_warn("extern (func ksym) '%s': function in kernel module is not supported\n", - ext->name); - return -ENOTSUP; + size_t index; + void *value; + + /* Lazy initialize btf->fd index map */ + if (!obj->kfunc_btf_fds.map) { + obj->kfunc_btf_fds.map = hashmap__new(bpf_core_hash_fn, bpf_core_equal_fn, + NULL); + if (!obj->kfunc_btf_fds.map) + return -ENOMEM; + + obj->kfunc_btf_fds.fds = calloc(8, sizeof(*obj->kfunc_btf_fds.fds)); + if (!obj->kfunc_btf_fds.fds) { + hashmap__free(obj->kfunc_btf_fds.map); + return -ENOMEM; + } + obj->kfunc_btf_fds.cap_cnt = 8; + } + + if (!hashmap__find(obj->kfunc_btf_fds.map, kern_btf, &value)) { + size_t *cap_cnt = &obj->kfunc_btf_fds.cap_cnt; + /* Not found, insert BTF fd into slot, and grab next + * index from the fd array. + */ + ret = libbpf_ensure_mem((void **)&obj->kfunc_btf_fds.fds, + cap_cnt, sizeof(int), obj->kfunc_btf_fds.n_fds + 1); + if (ret) + return ret; + index = obj->kfunc_btf_fds.n_fds++; + obj->kfunc_btf_fds.fds[index] = kern_btf_fd; + value = (void *)index; + ret = hashmap__add(obj->kfunc_btf_fds.map, kern_btf, &value); + if (ret) + return ret; + + } else { + index = (size_t)value; + } + /* index starts from 0, so shift offset by 1 as offset == 0 is reserved + * for btf_vmlinux in the kernel + */ + ext->ksym.offset = index + 1; + } else { + ext->ksym.offset = 0; } kern_func = btf__type_by_id(kern_btf, kfunc_id); @@ -6901,6 +6960,12 @@ int bpf_object__load_xattr(struct bpf_object_load_attr *attr) err = bpf_gen__finish(obj->gen_loader); } + /* clean up kfunc_btf */ + hashmap__free(obj->kfunc_btf_fds.map); + obj->kfunc_btf_fds.map = NULL; + zfree(&obj->kfunc_btf_fds.fds); + obj->kfunc_btf_fds.cap_cnt = obj->kfunc_btf_fds.n_fds = 0; + /* clean up module BTFs */ for (i = 0; i < obj->btf_module_cnt; i++) { close(obj->btf_modules[i].fd); diff --git a/tools/lib/bpf/libbpf_internal.h b/tools/lib/bpf/libbpf_internal.h index 533b0211f40a..701719d9caaf 100644 --- a/tools/lib/bpf/libbpf_internal.h +++ b/tools/lib/bpf/libbpf_internal.h @@ -276,6 +276,8 @@ struct bpf_prog_load_params { __u32 log_level; char *log_buf; size_t log_buf_sz; + int *kfunc_btf_fds; + __u32 kfunc_btf_fds_cnt; }; int libbpf__bpf_prog_load(const struct bpf_prog_load_params *load_attr); From patchwork Mon Aug 30 17:34:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 504559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45E78C432BE for ; Mon, 30 Aug 2021 17:35:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2AD8360E77 for ; Mon, 30 Aug 2021 17:35:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238160AbhH3Rfz (ORCPT ); Mon, 30 Aug 2021 13:35:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238147AbhH3Rfj (ORCPT ); Mon, 30 Aug 2021 13:35:39 -0400 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F2A57C061764; Mon, 30 Aug 2021 10:34:45 -0700 (PDT) Received: by mail-pf1-x441.google.com with SMTP id 7so12748294pfl.10; Mon, 30 Aug 2021 10:34:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=jvLCEjluWiKv4zpLIy1fBtQofZjH2laDX82OKUaTGAdLmCDRPoKNaieeyjWQaFJXiU +2Rki+ft3m+hw9hlATBeHeirRi6MK//1hvbMUdUb1KKAzdIyltxdO2DbTN4yhW9DC2ke AQ91Cj9oFAS03W+4xuJoSntdHWdCS+pbOioSj5NGbc5kIAD2iJEf+tkplvWljyPouhcU tmn+xaASv4pBqRIiAlBHvdPwByjg2B3dCXbXtRediqWO1+D9U7NOLQV16ZlNp29qL/gc uOzhecsg+x9qAvxD0o7/2Sy7xNTvAIb1ybQxRcHOdESPG0G77R4UM0Cs7fhUaRLAEJJK H/aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=yU4+xx6iAwgsGxgz5Ht4AFD9WjITvbQsNdCqZ6ULtK4=; b=c39cJH/LtYdntDTjyQ7S2v/DzVd120TdFd4rLMfifAE7djOyB9t8+q9ugMKD5T8wc0 qSL+DfiLfprjDq3CWZIK0YZKddPILEB14NzxQVPv976VIGQAqUQFn8r1e6N00tIbZJs8 8lgkkMrjrr3jFs+UDjTrxb3IgPuqww6MJCPqG15vongwtW5gbzMMaQrb3tGY3FpQCXld xYL161EoCIahpXrEKj1EiVMp8+2jgoNIYiMRTmD64XOEabyzFj2e2k/lZIG8QoRJSqI1 QCL17XD+nHRMXNpnOykFP2z5tfWEg4rkBBRKIVwHlN9hvf3qcnVrzNWfL0khKvnTIuxR c1cA== X-Gm-Message-State: AOAM531Jgxs9rYoeL+TU6557ioCag/FOPhAHmwJZPBqnilKmgz7HC+0o G68eWF8w5br9Tjd8NehW8DVbfX1VjXdx8Q== X-Google-Smtp-Source: ABdhPJynjRPrD2xFf/w7J1BHLShTqTaKUB6DMCuljgvWg+RkaCIn1n5+I+WOxi1UKEXjpWJFWhltzg== X-Received: by 2002:a62:1816:0:b0:3fc:c349:af7c with SMTP id 22-20020a621816000000b003fcc349af7cmr10036437pfy.31.1630344885395; Mon, 30 Aug 2021 10:34:45 -0700 (PDT) Received: from localhost ([2405:201:6014:d820:9cc6:d37f:c2fd:dc6]) by smtp.gmail.com with ESMTPSA id m64sm18223072pga.55.2021.08.30.10.34.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 10:34:45 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?b?w7hyZ2Vuc2Vu?= , netdev@vger.kernel.org Subject: [PATCH bpf-next RFC v1 5/8] tools: Allow specifying base BTF file in resolve_btfids Date: Mon, 30 Aug 2021 23:04:21 +0530 Message-Id: <20210830173424.1385796-6-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210830173424.1385796-1-memxor@gmail.com> References: <20210830173424.1385796-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2098; h=from:subject; bh=kZhRM1Uo6vUje4gN/WsUjSsDNLJ5u0WuItuVyu5D9Zk=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhLRX9DhwehFEbRic2pkL2wCQ10r1eCg0KKIKoWVSG lKYtwOmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYS0V/QAKCRBM4MiGSL8RygKPD/ 9tQw7XRzgDl94ByIZTRqMn5kChomef301BN3pM457RFSBeomwD+oe6V9DNDACdf0GqL/bwyMiu3f5Y gy6/GkAuhpTVtMVcpyx3Tjqx8KiOs1/96yjSr170AFVBCwtzS9csDgpi532ip5h/b3IUHTJWKSBb5b QTsy8RCeSkWBibloygFNUpXaG+bqYiZ8V54JcHtVET/2tsBayqUekFN+a4DPbw9ImA+l/D/I+VZXpA N/6UEB0+/5IBFeMjvRdA6xAjm3sXEhe3aeO6Umxn1pnH2+kdwelkI20meCwc03jhqnus3VEntp9wMh vlu9ZVhdhMqla7DFBJyhENvK7nrMPLqNgQIDr9HDOVOx+QEO6bhh9qUUED5YGaoEWzxNDWLGalcdwJ nT9Ou6NXpdBOXrqT9USmuM0cq1CS2XZLAuj2viYGKwjExU6cZt5dhpY+xwn+5Fx5ZqpGbn9eE/86GY 6CGcLXmEG9F0YvM547eOxyY3dM08t2hEGJrzIQH6N6PH6btmst+mfvlI3qWjF1VY6WQg4h7ZwV7HtN lo5RLGSbdzxuVDm3KOHU8Mso17uhLfBhXJM3voliluh0z+vM2Mwflza3E8EQ2g5wAcuTEOsG33LQmQ kK+jvElZXp2yVp+3cnTN2l/lSVkdoIZ4dHUR1mL8/hTsEySSqxPR0VwJ5Y1g== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commits allows specifying the base BTF for resolving btf id lists/sets during link time in the resolve_btfids tool. The base BTF is set to NULL if no path is passed. This allows resolving BTF ids for module kernel objects. Signed-off-by: Kumar Kartikeya Dwivedi --- tools/bpf/resolve_btfids/main.c | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/tools/bpf/resolve_btfids/main.c b/tools/bpf/resolve_btfids/main.c index de6365b53c9c..206e1120082f 100644 --- a/tools/bpf/resolve_btfids/main.c +++ b/tools/bpf/resolve_btfids/main.c @@ -89,6 +89,7 @@ struct btf_id { struct object { const char *path; const char *btf; + const char *base_btf_path; struct { int fd; @@ -477,16 +478,27 @@ static int symbols_resolve(struct object *obj) int nr_structs = obj->nr_structs; int nr_unions = obj->nr_unions; int nr_funcs = obj->nr_funcs; + struct btf *base_btf = NULL; int err, type_id; struct btf *btf; __u32 nr_types; - btf = btf__parse(obj->btf ?: obj->path, NULL); + if (obj->base_btf_path) { + base_btf = btf__parse(obj->base_btf_path, NULL); + err = libbpf_get_error(base_btf); + if (err) { + pr_err("FAILED: load base BTF from %s: %s\n", + obj->base_btf_path, strerror(-err)); + return -1; + } + } + + btf = btf__parse_split(obj->btf ?: obj->path, base_btf); err = libbpf_get_error(btf); if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); - return -1; + goto out; } err = -1; @@ -545,6 +557,7 @@ static int symbols_resolve(struct object *obj) err = 0; out: + btf__free(base_btf); btf__free(btf); return err; } @@ -697,6 +710,8 @@ int main(int argc, const char **argv) "BTF data"), OPT_BOOLEAN(0, "no-fail", &no_fail, "do not fail if " BTF_IDS_SECTION " section is not found"), + OPT_STRING('s', "base-btf", &obj.base_btf_path, "file", + "path of file providing base BTF data"), OPT_END() }; int err = -1; From patchwork Mon Aug 30 17:34:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kumar Kartikeya Dwivedi X-Patchwork-Id: 504558 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-15.7 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 15FD5C43216 for ; Mon, 30 Aug 2021 17:35:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E6BE260E77 for ; Mon, 30 Aug 2021 17:35:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238271AbhH3RgA (ORCPT ); Mon, 30 Aug 2021 13:36:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36244 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238246AbhH3Rfp (ORCPT ); Mon, 30 Aug 2021 13:35:45 -0400 Received: from mail-pj1-x1042.google.com (mail-pj1-x1042.google.com [IPv6:2607:f8b0:4864:20::1042]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CABE6C06175F; Mon, 30 Aug 2021 10:34:51 -0700 (PDT) Received: by mail-pj1-x1042.google.com with SMTP id j1so9990655pjv.3; Mon, 30 Aug 2021 10:34:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bJX0AyGbd9hRRY146ZMmI0dinbkaM5zuyV0rfjoDisU=; b=Qq+CnHyv5m4HjssH1RgPO4TujobZBA1MM1tvwHto6xYnnJMncTeF7B5N6q5oxyqHNg 19n0gQAbAlaZauXZsRS6Jib6MbIv/EV5qjoNJ/GRC9kgA2bACVtnG2EVAN1T6YcFF8Wc Z3IQK7nVdWg5uUhOcC6S388gqpPFGIVv/3vcWR+PyAgPsaQJVcL6yAvSuSyoS7rdlNxA H4j12VeBUjrLeuJ1+u0FAEslnbhesE5jODhNXxUaKfMtnbjGhonUmiY9b6iFkQsFb6XD tXRMzG817PUezbRQByoGrmpXcsqHlYkgr8+hiswlOTAj475KcCqZ4okRhtEgPTRjIppV TwXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bJX0AyGbd9hRRY146ZMmI0dinbkaM5zuyV0rfjoDisU=; b=MpuMUoHhynj9RVJXqPYNxPMFtZpuRR+513q1sUfVFp/AIwsguGJ474Q6Lw2gUqF+SI kAN1REa1i2BxHvFcJOiozFb/K/3CzQ5Qn4ay7naZMtI9sgy7t6J84yUl7u5r33cRIyf7 IfTk3XtNCy+SFXiTSu/+K7115uUBn31n3aciL6nJjnh67UnXOc9mfh3hCJI1K7IFDcWU 6pb7fYngGrTj9zPaSAnOM2/r/nqnvTAXpCVL7FjTOQ/KHEayD44gyJgCZNXi1WDzZlkj ++1Zdo4y8PUiZuX+F/a2E+CHDlRnWCeL1/tCEr2uf2VJGoIqMKXgFz8gjEkAX6Qe3FaO N4IQ== X-Gm-Message-State: AOAM533nAedHtJSy8SbvtsL0+As28cPQyafp6io2/hKnuPbzdNJsAsR+ I/N12EazQ53bshsyDkMMFp2ejdn5k9vk7Q== X-Google-Smtp-Source: ABdhPJz/HTeG8P9fK1cv0kRx0EaFE8jdd16Xk43wqj3v72r26cg1mLqqGjFCG6dfzzwk8ypJTc4IeQ== X-Received: by 2002:a17:90a:bd81:: with SMTP id z1mr176824pjr.207.1630344891137; Mon, 30 Aug 2021 10:34:51 -0700 (PDT) Received: from localhost ([2405:201:6014:d820:9cc6:d37f:c2fd:dc6]) by smtp.gmail.com with ESMTPSA id fh2sm106158pjb.12.2021.08.30.10.34.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Aug 2021 10:34:50 -0700 (PDT) From: Kumar Kartikeya Dwivedi To: bpf@vger.kernel.org Cc: Kumar Kartikeya Dwivedi , Alexei Starovoitov , Daniel Borkmann , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , Jesper Dangaard Brouer , =?utf-8?q?Toke_H=C3=B8iland-J?= =?utf-8?b?w7hyZ2Vuc2Vu?= , netdev@vger.kernel.org Subject: [PATCH bpf-next RFC v1 7/8] bpf: enable TCP congestion control kfunc from modules Date: Mon, 30 Aug 2021 23:04:23 +0530 Message-Id: <20210830173424.1385796-8-memxor@gmail.com> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210830173424.1385796-1-memxor@gmail.com> References: <20210830173424.1385796-1-memxor@gmail.com> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=8888; h=from:subject; bh=+wtSmbpHsLJhBkA9+XpBdzDEaE4kFWY7r1NXIZIdqpA=; b=owEBbQKS/ZANAwAIAUzgyIZIvxHKAcsmYgBhLRX9ZywhGY9TnA5oV1T2mVNV2NjEuhC+tXxzaki0 NE2MmwmJAjMEAAEIAB0WIQRLvip+Buz51YI8YRFM4MiGSL8RygUCYS0V/QAKCRBM4MiGSL8Rym8IEA CbKSm7lCT4wFhp1hkv/DIm8aHSzaj7l9r/KH5pcSbulg5NakptpCwOM9Z3j6H8EWfwjKV/NwYRqhaJ UXL4U/VD7f8b0MRokY82d0FWGu9iDy21kc3+c7ssIs4vd7yNQYDXWpnoq1A1ltxVvshsy1FRRSsM4G 84q8YpvXJHEqOUJkH2swbsRQs6mU65iS2U8OaA1HjnXAZojTEjIB3+IYZ2wRYKSD/eCBBtGUxSSDtu jsWY1tJaGSpFcV7PVmSiFNNdXYOkzWhlApg+8QvT1GJ6Jp/9isZhR+8badRvmjgTY3DS55lMqE+/c2 8iSeLMxRQxfnfqjBEzi+1z3yvf4AteH827uRXBNy1jGF4kjQspOnmlBSl5jN7gvWlTsFEernI692C2 F0x2GIY6VFfY7YxfiJmbIamvqRUhrISe63a2oE3eNY3lJIt7uqQ/rdprLlRyqSCVrh4w30t3CzEyQi ln8DejTFHDD9aeccglaBN4iX8wCKWVzscTG3937kVbpGFRlG2z3lhDzVH988a1WTuIQ+atYgEuV5vc 3TM3Qk0uf/nEeT9k7afitbwWqGIOppElZgDX0A/Gx3/sfeWGMiJvGsyeMyX0NAPZUTYfEHczPuAIf2 M0rFGckyl94WT9mConWlndDjA9ntwOO8Xx9p7S69K8UL4L9j4Am7TDSbZNuQ== X-Developer-Key: i=memxor@gmail.com; a=openpgp; fpr=4BBE2A7E06ECF9D5823C61114CE0C88648BF11CA Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This commit moves BTF ID lookup into the newly added registration helper, in a way that the bbr, cubic, and dctcp implementation set up their sets in the bpf_tcp_ca kfunc_btf_set list, while the ones not dependent on modules are looked up from the wrapper function. This lifts the restriction for them to be compiled as built in objects, and can be loaded as modules if required. Also modify link-vmlinux.sh to resolve_btfids in TCP congestion control modules if the config option is set, using the base BTF support added in the previous commit. See following commits for background on use of: CONFIG_X86 ifdef: 569c484f9995 (bpf: Limit static tcp-cc functions in the .BTF_ids list to x86) CONFIG_DYNAMIC_FTRACE ifdef: 7aae231ac93b (bpf: tcp: Limit calling some tcp cc functions to CONFIG_DYNAMIC_FTRACE) Signed-off-by: Kumar Kartikeya Dwivedi --- include/linux/btf.h | 2 ++ kernel/bpf/btf.c | 2 ++ net/ipv4/bpf_tcp_ca.c | 34 +++------------------------------- net/ipv4/tcp_bbr.c | 28 +++++++++++++++++++++++++++- net/ipv4/tcp_cubic.c | 26 +++++++++++++++++++++++++- net/ipv4/tcp_dctcp.c | 26 +++++++++++++++++++++++++- scripts/Makefile.modfinal | 1 + 7 files changed, 85 insertions(+), 34 deletions(-) diff --git a/include/linux/btf.h b/include/linux/btf.h index d024b0eb43f9..8c0f29ed2af9 100644 --- a/include/linux/btf.h +++ b/include/linux/btf.h @@ -253,4 +253,6 @@ struct kfunc_btf_set { #define DEFINE_KFUNC_BTF_SET(set, name) \ struct kfunc_btf_set name = { LIST_HEAD_INIT(name.list), (set) } +DECLARE_KFUNC_BTF_SET_REG(bpf_tcp_ca); + #endif diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c index 35873495761d..cc12470a55f9 100644 --- a/kernel/bpf/btf.c +++ b/kernel/bpf/btf.c @@ -6249,3 +6249,5 @@ BTF_ID_LIST_GLOBAL_SINGLE(btf_task_struct_ids, struct, task_struct) mutex_unlock(&type##_kfunc_btf_set_mutex); \ } \ EXPORT_SYMBOL_GPL(unregister_##type##_kfunc_btf_set) + +DEFINE_KFUNC_BTF_SET_REG(bpf_tcp_ca); diff --git a/net/ipv4/bpf_tcp_ca.c b/net/ipv4/bpf_tcp_ca.c index 0dcee9df1326..804f2f912fe9 100644 --- a/net/ipv4/bpf_tcp_ca.c +++ b/net/ipv4/bpf_tcp_ca.c @@ -223,41 +223,13 @@ BTF_ID(func, tcp_reno_cong_avoid) BTF_ID(func, tcp_reno_undo_cwnd) BTF_ID(func, tcp_slow_start) BTF_ID(func, tcp_cong_avoid_ai) -#ifdef CONFIG_X86 -#ifdef CONFIG_DYNAMIC_FTRACE -#if IS_BUILTIN(CONFIG_TCP_CONG_CUBIC) -BTF_ID(func, cubictcp_init) -BTF_ID(func, cubictcp_recalc_ssthresh) -BTF_ID(func, cubictcp_cong_avoid) -BTF_ID(func, cubictcp_state) -BTF_ID(func, cubictcp_cwnd_event) -BTF_ID(func, cubictcp_acked) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_DCTCP) -BTF_ID(func, dctcp_init) -BTF_ID(func, dctcp_update_alpha) -BTF_ID(func, dctcp_cwnd_event) -BTF_ID(func, dctcp_ssthresh) -BTF_ID(func, dctcp_cwnd_undo) -BTF_ID(func, dctcp_state) -#endif -#if IS_BUILTIN(CONFIG_TCP_CONG_BBR) -BTF_ID(func, bbr_init) -BTF_ID(func, bbr_main) -BTF_ID(func, bbr_sndbuf_expand) -BTF_ID(func, bbr_undo_cwnd) -BTF_ID(func, bbr_cwnd_event) -BTF_ID(func, bbr_ssthresh) -BTF_ID(func, bbr_min_tso_segs) -BTF_ID(func, bbr_set_state) -#endif -#endif /* CONFIG_DYNAMIC_FTRACE */ -#endif /* CONFIG_X86 */ BTF_SET_END(bpf_tcp_ca_kfunc_ids) static bool bpf_tcp_ca_check_kfunc_call(u32 kfunc_btf_id) { - return btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id); + if (btf_id_set_contains(&bpf_tcp_ca_kfunc_ids, kfunc_btf_id)) + return true; + return __bpf_check_bpf_tcp_ca_kfunc_call(kfunc_btf_id); } static const struct bpf_verifier_ops bpf_tcp_ca_verifier_ops = { diff --git a/net/ipv4/tcp_bbr.c b/net/ipv4/tcp_bbr.c index 6274462b86b4..1fea15dd0e05 100644 --- a/net/ipv4/tcp_bbr.c +++ b/net/ipv4/tcp_bbr.c @@ -56,6 +56,8 @@ * otherwise TCP stack falls back to an internal pacing using one high * resolution timer per TCP socket and may use more resources. */ +#include +#include #include #include #include @@ -1152,14 +1154,38 @@ static struct tcp_congestion_ops tcp_bbr_cong_ops __read_mostly = { .set_state = bbr_set_state, }; +BTF_SET_START(tcp_bbr_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, bbr_init) +BTF_ID(func, bbr_main) +BTF_ID(func, bbr_sndbuf_expand) +BTF_ID(func, bbr_undo_cwnd) +BTF_ID(func, bbr_cwnd_event) +BTF_ID(func, bbr_ssthresh) +BTF_ID(func, bbr_min_tso_segs) +BTF_ID(func, bbr_set_state) +#endif +#endif +BTF_SET_END(tcp_bbr_kfunc_ids) + +static DEFINE_KFUNC_BTF_SET(&tcp_bbr_kfunc_ids, tcp_bbr_kfunc_btf_set); + static int __init bbr_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bbr) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&tcp_bbr_cong_ops); + ret = tcp_register_congestion_control(&tcp_bbr_cong_ops); + if (ret) + return ret; + register_bpf_tcp_ca_kfunc_btf_set(&tcp_bbr_kfunc_btf_set); + return 0; } static void __exit bbr_unregister(void) { + unregister_bpf_tcp_ca_kfunc_btf_set(&tcp_bbr_kfunc_btf_set); tcp_unregister_congestion_control(&tcp_bbr_cong_ops); } diff --git a/net/ipv4/tcp_cubic.c b/net/ipv4/tcp_cubic.c index 4a30deaa9a37..5b36b9442797 100644 --- a/net/ipv4/tcp_cubic.c +++ b/net/ipv4/tcp_cubic.c @@ -25,6 +25,8 @@ */ #include +#include +#include #include #include #include @@ -482,8 +484,25 @@ static struct tcp_congestion_ops cubictcp __read_mostly = { .name = "cubic", }; +BTF_SET_START(tcp_cubic_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, cubictcp_init) +BTF_ID(func, cubictcp_recalc_ssthresh) +BTF_ID(func, cubictcp_cong_avoid) +BTF_ID(func, cubictcp_state) +BTF_ID(func, cubictcp_cwnd_event) +BTF_ID(func, cubictcp_acked) +#endif +#endif +BTF_SET_END(tcp_cubic_kfunc_ids) + +static DEFINE_KFUNC_BTF_SET(&tcp_cubic_kfunc_ids, tcp_cubic_kfunc_btf_set); + static int __init cubictcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct bictcp) > ICSK_CA_PRIV_SIZE); /* Precompute a bunch of the scaling factors that are used per-packet @@ -514,11 +533,16 @@ static int __init cubictcp_register(void) /* divide by bic_scale and by constant Srtt (100ms) */ do_div(cube_factor, bic_scale * 10); - return tcp_register_congestion_control(&cubictcp); + ret = tcp_register_congestion_control(&cubictcp); + if (ret) + return ret; + register_bpf_tcp_ca_kfunc_btf_set(&tcp_cubic_kfunc_btf_set); + return 0; } static void __exit cubictcp_unregister(void) { + unregister_bpf_tcp_ca_kfunc_btf_set(&tcp_cubic_kfunc_btf_set); tcp_unregister_congestion_control(&cubictcp); } diff --git a/net/ipv4/tcp_dctcp.c b/net/ipv4/tcp_dctcp.c index 79f705450c16..efc47b4c7a11 100644 --- a/net/ipv4/tcp_dctcp.c +++ b/net/ipv4/tcp_dctcp.c @@ -36,6 +36,8 @@ * Glenn Judd */ +#include +#include #include #include #include @@ -236,14 +238,36 @@ static struct tcp_congestion_ops dctcp_reno __read_mostly = { .name = "dctcp-reno", }; +BTF_SET_START(tcp_dctcp_kfunc_ids) +#ifdef CONFIG_X86 +#ifdef CONFIG_DYNAMIC_FTRACE +BTF_ID(func, dctcp_init) +BTF_ID(func, dctcp_update_alpha) +BTF_ID(func, dctcp_cwnd_event) +BTF_ID(func, dctcp_ssthresh) +BTF_ID(func, dctcp_cwnd_undo) +BTF_ID(func, dctcp_state) +#endif +#endif +BTF_SET_END(tcp_dctcp_kfunc_ids) + +static DEFINE_KFUNC_BTF_SET(&tcp_dctcp_kfunc_ids, tcp_dctcp_kfunc_btf_set); + static int __init dctcp_register(void) { + int ret; + BUILD_BUG_ON(sizeof(struct dctcp) > ICSK_CA_PRIV_SIZE); - return tcp_register_congestion_control(&dctcp); + ret = tcp_register_congestion_control(&dctcp); + if (ret) + return ret; + register_bpf_tcp_ca_kfunc_btf_set(&tcp_dctcp_kfunc_btf_set); + return 0; } static void __exit dctcp_unregister(void) { + unregister_bpf_tcp_ca_kfunc_btf_set(&tcp_dctcp_kfunc_btf_set); tcp_unregister_congestion_control(&dctcp); } diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 5e9b8057fb24..0755d4b8b74a 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -58,6 +58,7 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ -f vmlinux ]; then \ LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J --btf_base vmlinux $@; \ + $(RESOLVE_BTFIDS) --no-fail -s vmlinux $@; \ else \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ fi;