From patchwork Fri Mar 19 20:59:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 405270 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1EDEBC433E3 for ; Fri, 19 Mar 2021 20:59:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E90F86197D for ; Fri, 19 Mar 2021 20:59:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230240AbhCSU7X convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:23 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:3796 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230235AbhCSU7T (ORCPT ); Fri, 19 Mar 2021 16:59:19 -0400 Received: from pps.filterd (m0148461.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKwTPO004087 for ; Fri, 19 Mar 2021 13:59:19 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 37bs1ew1rr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:19 -0700 Received: from intmgw001.37.frc1.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:18 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 1639A2ED268B; Fri, 19 Mar 2021 13:59:13 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , Subject: [PATCH bpf-next 1/3] bpftool: improve skeleton generation for objects without BTF Date: Fri, 19 Mar 2021 13:59:07 -0700 Message-ID: <20210319205909.1748642-2-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 priorityscore=1501 mlxscore=0 adultscore=0 mlxlogscore=999 malwarescore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190144 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org If BPF object file is using global variables, but is compiled without BTF or ends up having only some of DATASEC types due to static linking, generated skeleton won't compile, as some parts of skeleton would assume memory-mapped struct definitions for each special data section. Fix this by generating empty struct definition for such data sections. The benefit of that is that they still will be memory-mapped by skeleton handling code in libbpf, and user-space parts of user application would be able to access data in them (though pointer casting), assuming they know the memory layout they need. Fixes: 985ead416df3 ("bpftool: Add skeleton codegen command") Signed-off-by: Andrii Nakryiko --- tools/bpf/bpftool/gen.c | 81 ++++++++++++++++++++++++++++++++--------- 1 file changed, 64 insertions(+), 17 deletions(-) diff --git a/tools/bpf/bpftool/gen.c b/tools/bpf/bpftool/gen.c index 31ade77f5ef8..6ec7c2e90a01 100644 --- a/tools/bpf/bpftool/gen.c +++ b/tools/bpf/bpftool/gen.c @@ -85,6 +85,18 @@ static const char *get_map_ident(const struct bpf_map *map) else return NULL; } +static const char *datasec_ident(const char *sec_name) +{ + if (strcmp(sec_name, ".data") == 0) + return "data"; + if (strcmp(sec_name, ".bss") == 0) + return "bss"; + if (strcmp(sec_name, ".rodata") == 0) + return "rodata"; + if (strcmp(sec_name, ".kconfig") == 0) + return "kconfig"; + return NULL; +} static void codegen_btf_dump_printf(void *ctx, const char *fmt, va_list args) { @@ -104,18 +116,12 @@ static int codegen_datasec_def(struct bpf_object *obj, char var_ident[256]; bool strip_mods = false; - if (strcmp(sec_name, ".data") == 0) { - sec_ident = "data"; - } else if (strcmp(sec_name, ".bss") == 0) { - sec_ident = "bss"; - } else if (strcmp(sec_name, ".rodata") == 0) { - sec_ident = "rodata"; - strip_mods = true; - } else if (strcmp(sec_name, ".kconfig") == 0) { - sec_ident = "kconfig"; - } else { + sec_ident = datasec_ident(sec_name); + if (!sec_ident) return 0; - } + + if (strcmp(sec_name, ".rodata") == 0) + strip_mods = true; printf(" struct %s__%s {\n", obj_name, sec_ident); for (i = 0; i < vlen; i++, sec_var++) { @@ -188,22 +194,63 @@ static int codegen_datasecs(struct bpf_object *obj, const char *obj_name) struct btf *btf = bpf_object__btf(obj); int n = btf__get_nr_types(btf); struct btf_dump *d; + struct bpf_map *map; + const struct btf_type *sec; + const char *sec_ident, *map_ident; int i, err = 0; d = btf_dump__new(btf, NULL, NULL, codegen_btf_dump_printf); if (IS_ERR(d)) return PTR_ERR(d); - for (i = 1; i <= n; i++) { - const struct btf_type *t = btf__type_by_id(btf, i); + bpf_object__for_each_map(map, obj) { + /* only generate definitions for memory-mapped internal maps */ + if (!bpf_map__is_internal(map)) + continue; + if (!(bpf_map__def(map)->map_flags & BPF_F_MMAPABLE)) + continue; - if (!btf_is_datasec(t)) + map_ident = get_map_ident(map); + if (!map_ident) continue; - err = codegen_datasec_def(obj, btf, d, t, obj_name); - if (err) - goto out; + sec = NULL; + for (i = 1; i <= n; i++) { + const struct btf_type *t = btf__type_by_id(btf, i); + const char *name; + + if (!btf_is_datasec(t)) + continue; + + name = btf__str_by_offset(btf, t->name_off); + sec_ident = datasec_ident(name); + if (!sec_ident) + continue; + + if (strcmp(sec_ident, map_ident) == 0) { + sec = t; + break; + } + } + + /* In rare cases when BPF object file is using global + * variables, but is compiled without BTF, we will have + * special internal map, but no corresponding DATASEC BTF + * type. In such case, generate empty structs for each such + * map. It will still be memory-mapped as a convenience for + * applications that know exact memory layout to expect. + */ + if (!sec) { + printf(" struct %s__%s {\n", obj_name, map_ident); + printf(" } *%s;\n", map_ident); + } else { + err = codegen_datasec_def(obj, btf, d, sec, obj_name); + if (err) + goto out; + } } + + out: btf_dump__free(d); return err; From patchwork Fri Mar 19 20:59:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 405815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7CF2C433DB for ; Fri, 19 Mar 2021 21:00:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8AED36197D for ; Fri, 19 Mar 2021 21:00:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230370AbhCSU7y convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:54 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:22420 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230218AbhCSU7W (ORCPT ); Fri, 19 Mar 2021 16:59:22 -0400 Received: from pps.filterd (m0109333.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKuRiM005100 for ; Fri, 19 Mar 2021 13:59:22 -0700 Received: from mail.thefacebook.com ([163.114.132.120]) by mx0a-00082601.pphosted.com with ESMTP id 37bs1wd1v1-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:22 -0700 Received: from intmgw001.06.ash9.facebook.com (2620:10d:c085:108::4) by mail.thefacebook.com (2620:10d:c085:21d::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:21 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 41FA52ED268B; Fri, 19 Mar 2021 13:59:15 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , , Jiri Olsa Subject: [PATCH bpf-next 2/3] libbpf: skip BTF fixup if object file has no BTF Date: Fri, 19 Mar 2021 13:59:08 -0700 Message-ID: <20210319205909.1748642-3-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 lowpriorityscore=0 phishscore=0 spamscore=0 priorityscore=1501 adultscore=0 clxscore=1015 malwarescore=0 impostorscore=0 mlxscore=0 suspectscore=0 mlxlogscore=999 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190143 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Skip BTF fixup step when input object file is missing BTF altogether. Reported-by: Jiri Olsa Fixes: 8fd27bf69b86 ("libbpf: Add BPF static linker BTF and BTF.ext support") Signed-off-by: Andrii Nakryiko --- tools/lib/bpf/linker.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tools/lib/bpf/linker.c b/tools/lib/bpf/linker.c index b4fff912dce2..5e0aa2f2c0ca 100644 --- a/tools/lib/bpf/linker.c +++ b/tools/lib/bpf/linker.c @@ -1313,6 +1313,9 @@ static int linker_fixup_btf(struct src_obj *obj) struct src_sec *sec; int i, j, n, m; + if (!obj->btf) + return 0; + n = btf__get_nr_types(obj->btf); for (i = 1; i <= n; i++) { struct btf_var_secinfo *vi; From patchwork Fri Mar 19 20:59:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrii Nakryiko X-Patchwork-Id: 405269 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04E8AC433E3 for ; Fri, 19 Mar 2021 21:00:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D657761989 for ; Fri, 19 Mar 2021 21:00:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230411AbhCSU74 convert rfc822-to-8bit (ORCPT ); Fri, 19 Mar 2021 16:59:56 -0400 Received: from mx0b-00082601.pphosted.com ([67.231.153.30]:26088 "EHLO mx0b-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230264AbhCSU7W (ORCPT ); Fri, 19 Mar 2021 16:59:22 -0400 Received: from pps.filterd (m0109332.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 12JKx9dL000562 for ; Fri, 19 Mar 2021 13:59:21 -0700 Received: from maileast.thefacebook.com ([163.114.130.16]) by mx0a-00082601.pphosted.com with ESMTP id 37cx90hydv-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NOT) for ; Fri, 19 Mar 2021 13:59:21 -0700 Received: from intmgw006.03.ash8.facebook.com (2620:10d:c0a8:1b::d) by mail.thefacebook.com (2620:10d:c0a8:83::4) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2176.2; Fri, 19 Mar 2021 13:59:20 -0700 Received: by devbig012.ftw2.facebook.com (Postfix, from userid 137359) id 6C6AA2ED268B; Fri, 19 Mar 2021 13:59:17 -0700 (PDT) From: Andrii Nakryiko To: , , , CC: , Subject: [PATCH bpf-next 3/3] selftests/bpf: allow compiling BPF objects without BTF Date: Fri, 19 Mar 2021 13:59:09 -0700 Message-ID: <20210319205909.1748642-4-andrii@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210319205909.1748642-1-andrii@kernel.org> References: <20210319205909.1748642-1-andrii@kernel.org> MIME-Version: 1.0 X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.369, 18.0.761 definitions=2021-03-19_12:2021-03-19,2021-03-19 signatures=0 X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 phishscore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 suspectscore=0 mlxscore=0 bulkscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2009150000 definitions=main-2103190144 X-FB-Internal: deliver Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add ability to skip BTF generation for some BPF object files. This is done through using a convention of .nobtf.c file name suffix. Also add third statically linked file to static_linked selftest. This file has no BTF, causing resulting object file to have only some of DATASEC BTF types. It also is using (from BPF code) global variables. This tests both libbpf's static linking logic and bpftool's skeleton generation logic. Signed-off-by: Andrii Nakryiko --- tools/testing/selftests/bpf/Makefile | 21 +++++++---- .../selftests/bpf/prog_tests/static_linked.c | 6 +++- .../bpf/progs/test_static_linked3.nobtf.c | 36 +++++++++++++++++++ 3 files changed, 56 insertions(+), 7 deletions(-) create mode 100644 tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c diff --git a/tools/testing/selftests/bpf/Makefile b/tools/testing/selftests/bpf/Makefile index 6448c626498f..0a481a75a416 100644 --- a/tools/testing/selftests/bpf/Makefile +++ b/tools/testing/selftests/bpf/Makefile @@ -270,7 +270,7 @@ IS_LITTLE_ENDIAN = $(shell $(CC) -dM -E - bss->var1, 1 * 2 + 2 + 3, "var1"); ASSERT_EQ(skel->bss->var2, 4 * 3 + 5 + 6, "var2"); + err = bpf_map_lookup_elem(bpf_map__fd(skel->maps.legacy_map), &key, &value); + ASSERT_OK(err, "legacy_map_lookup"); + ASSERT_EQ(value, 1 * 3 + 3, "legacy_map_value"); + cleanup: test_static_linked__destroy(skel); } diff --git a/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c b/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c new file mode 100644 index 000000000000..e5fbde21381c --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_static_linked3.nobtf.c @@ -0,0 +1,36 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2021 Facebook */ + +#include +#include + +/* global variables don't need BTF to be used, but are extremely unconvenient + * to be consumed from user-space without BPF skeleton, that uses BTF + */ + +static volatile int mul3 = 3; +static volatile int add3 = 3; + +/* same "subprog" name in all files */ +static __noinline int subprog(int x) +{ + /* but different formula */ + return x * mul3 + add3; +} + +struct bpf_map_def SEC("maps") legacy_map = { + .type = BPF_MAP_TYPE_ARRAY, + .key_size = sizeof(int), + .value_size = sizeof(int), + .max_entries = 1, +}; + +SEC("raw_tp/sys_enter") +int handler3(const void *ctx) +{ + int key = 0, value = subprog(1); + + bpf_map_update_elem(&legacy_map, &key, &value, BPF_ANY); + + return 0; +}