From patchwork Wed Jun 16 22:55:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 461856 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A963DC48BE6 for ; Wed, 16 Jun 2021 22:55:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 95770613CD for ; Wed, 16 Jun 2021 22:55:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230352AbhFPW5l (ORCPT ); Wed, 16 Jun 2021 18:57:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53674 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229602AbhFPW5k (ORCPT ); Wed, 16 Jun 2021 18:57:40 -0400 Received: from mail-io1-xd2f.google.com (mail-io1-xd2f.google.com [IPv6:2607:f8b0:4864:20::d2f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4B296C061574 for ; Wed, 16 Jun 2021 15:55:34 -0700 (PDT) Received: by mail-io1-xd2f.google.com with SMTP id s26so962604ioe.9 for ; Wed, 16 Jun 2021 15:55:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=KgcDupNr6WzckA/mD3zDMlYk7N37gqCIpQ3Xwu0Qvtc=; b=DCHWGrfhDE7QG2ybVwvdLsjflOHljDqYk/q3y9A6pUkHQ4wdy9inFN3ugjG8Fxt6zU 4RNzWEhUfxBE4c5u9XxUiwbSyQobBgSZksF7RkyIcm8DOil0zL0SoWcV5N5uEFpCuoJx 8F3K3NIJk9kwAva+s9IHnuqWfcMlheM1gGibrEt9r/vKf73uR/26V8iU1aiUNLMWa7MH k8yixSUHg8TqYr5ALQUiQxQD+gfJA0QuOPNxhn6NXvco45Bc6qp3qEog2HKTPE10ZdZk OXTSmpviqwAG8C/ddvXzk5CDjPLWu+C1PtVx97MQxeU/uUF8kXhraObCzqMayjxmW0U9 YE/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=KgcDupNr6WzckA/mD3zDMlYk7N37gqCIpQ3Xwu0Qvtc=; b=ShsbwudCBPRWaBxjPSjPryqhKFbQJukbFcBZL+nqWMS9C8+tEG/INdKq6I1BuXmJ7U 8su/loHUlNPG/QQXh7njUKhI3JhlBknKaJtub/Xp8EvIOF3d2fhh+psXL2HlvF1e7/MM 4OwRzlOVgOs0Z3x8kxDM3PMRka3CNxp0GZ7Ubdt4nSl755mqR+P2LCxiEkb4CFKjs17A RrSrZsQC0LW8IdG9zTFbXF2zzAra2Ey+BNf7J/kpfec7lRA3sRjWu4GSStUg5FNVEN5P nhmuXbdyHPUFtHkqqnxmrVgaAtWTIRp8iOyaSxKJO27yxcMYgfpkw36nXB0qxMDuqDWy j0Gg== X-Gm-Message-State: AOAM533cSnsGRs7N8Z2noLj9I/CkRZYMRrxFOnivb7aTh2lrnoR/NUEH GuyxFJmnfTu8bL61KWSfGZw= X-Google-Smtp-Source: ABdhPJxNazkXRdv4QQyBCHnj3N4TKx0Dj8NSwwNKV5JSbe4zxk5T2tvld3bmsGo0tRcfiHAhc9nVlw== X-Received: by 2002:a6b:e40a:: with SMTP id u10mr1336978iog.200.1623884133700; Wed, 16 Jun 2021 15:55:33 -0700 (PDT) Received: from [127.0.1.1] ([172.243.157.240]) by smtp.gmail.com with ESMTPSA id z14sm1827639ilb.48.2021.06.16.15.55.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Jun 2021 15:55:33 -0700 (PDT) Subject: [PATCH bpf v2 2/4] bpf: map_poke_descriptor is being called with an unstable poke_tab[] From: John Fastabend To: maciej.fijalkowski@intel.com, ast@kernel.org, daniel@iogearbox.net, andriin@fb.com Cc: john.fastabend@gmail.com, netdev@vger.kernel.org, netdev@vger.kernel.org Date: Wed, 16 Jun 2021 15:55:19 -0700 Message-ID: <162388411986.151936.3914295553899556046.stgit@john-XPS-13-9370> In-Reply-To: <162388400488.151936.1658153981415911010.stgit@john-XPS-13-9370> References: <162388400488.151936.1658153981415911010.stgit@john-XPS-13-9370> User-Agent: StGit/0.23-85-g6af9 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org When populating poke_tab[] of a subprog we call map_poke_track() after doing bpf_jit_add_poke_descriptor(). But, bpf_jit_add_poke_descriptor() may, likely will, realloc the poke_tab[] structure and free the old one. So that prog->aux->poke_tab is not stable. However, the aux pointer is referenced from bpf_array_aux and poke_tab[] is used to 'track' prog<->map link. This way when progs are released the entry in the map is dropped and vice versa when the map is released we don't drop it too soon if a prog is in the process of calling it. I wasn't able to trigger any errors here, for example having map_poke_run run with a poke_tab[] pointer that was free'd from bpf_jit_add_poke_descriptor(), but it looks possible and at very least is very fragile. This patch moves poke_track call out of loop that is calling add_poke so that we only ever add stable aux->poke_tab pointers to the map's bpf_array_aux struct. Further, we need this in the next patch to fix a real bug where progs are not 'untracked'. Signed-off-by: John Fastabend --- kernel/bpf/verifier.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 6e2ebcb0d66f..066fac9b5460 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -12126,8 +12126,12 @@ static int jit_subprogs(struct bpf_verifier_env *env) } func[i]->insnsi[insn_idx - subprog_start].imm = ret + 1; + } - map_ptr = func[i]->aux->poke_tab[ret].tail_call.map; + for (j = 0; j < func[i]->aux->size_poke_tab; j++) { + int ret; + + map_ptr = func[i]->aux->poke_tab[j].tail_call.map; ret = map_ptr->ops->map_poke_track(map_ptr, func[i]->aux); if (ret < 0) { verbose(env, "tracking tail call prog failed\n"); From patchwork Wed Jun 16 22:55:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Fastabend X-Patchwork-Id: 461855 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87770C48BE5 for ; Wed, 16 Jun 2021 22:56:16 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 60B99613CB for ; Wed, 16 Jun 2021 22:56:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231249AbhFPW6V (ORCPT ); Wed, 16 Jun 2021 18:58:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53828 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231419AbhFPW6U (ORCPT ); Wed, 16 Jun 2021 18:58:20 -0400 Received: from mail-io1-xd2b.google.com (mail-io1-xd2b.google.com [IPv6:2607:f8b0:4864:20::d2b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67697C061574 for ; Wed, 16 Jun 2021 15:56:11 -0700 (PDT) Received: by mail-io1-xd2b.google.com with SMTP id 5so997172ioe.1 for ; Wed, 16 Jun 2021 15:56:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:from:to:cc:date:message-id:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=3DLASFQ15Z81rloOq6n84/zQvsm6YmqeDbgPpYmXx0A=; b=p1wQF/DPc15yWjfECOkMCxK5ue6n1s4xTNEUohTbIgU7wHjjfk0rnvsCAOTsNa33tg +0/KFTiwv7HeYU9FGPUyzNgZcDJV8nnp4IIxmv7uloSIbdNlb/GsENI7mA4IUS2JrkvW kiu+5zaTJoFM/qYk2Bwua3yvFYp6zHewknSGo8Adf0kaCxPp+5X9HYUNgNYSlVqYgPKg wOO+lZLOzPExHWxubDDJgp2dmKPIu7iU5gpk5HaVK7SuauTKtuKmNMjCBci/W8kjitns s2u3Y77cC30WBk5n4ZpNOL8YUhlktqBlh85NMeFBOVVccbAtlUd4loSxCspSmXqIK1q/ SaSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:from:to:cc:date:message-id:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=3DLASFQ15Z81rloOq6n84/zQvsm6YmqeDbgPpYmXx0A=; b=F7X/S5oletRuqRXB7HaE3Q4BnpJ6ve9f7euGIiNJtKhSrP1McsQxR/0fRO59W9XsVg qCjsw/ZWIZmzSn1fwmhxlKaXTLm05r9V0kjdMSlpuKVP7r+V7hqM+gMQaxU0enVjfZH+ UbnBYdpg34D8AbutMalln2bsKvHvVF8VZCQZUF4XmGWy2NHlkQ3SUO2Ik2CPq3B037dW Q0RfQ9NwIHoOyi91qLfdN08K2grcNz+Lr4X7KVEuTeEhDXyFCAGqu3aTsXfwj8Nt8ilm 4FIXEVXhdH6YQj0jYBEbqwUJnRWme65FPZO/r0klC+d/WPQ8YqfoO0MUb8+8WmpL8cqd uaFg== X-Gm-Message-State: AOAM531nMq3z+ZZkuhGzz7RAivYC8MOrFlibZK/uD99h5d9/2JJNUAyK XmyBZo4fv9OfryWN/t3UcuM= X-Google-Smtp-Source: ABdhPJyCVBdgBagf/DYUJEryH3KtpkoowCnB39rkVTChGwAaMpZuF5JE40FUYwysgY37xhoLFjbv/A== X-Received: by 2002:a05:6602:2a43:: with SMTP id k3mr1314309iov.47.1623884170683; Wed, 16 Jun 2021 15:56:10 -0700 (PDT) Received: from [127.0.1.1] ([172.243.157.240]) by smtp.gmail.com with ESMTPSA id n2sm1908745iod.54.2021.06.16.15.56.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Jun 2021 15:56:10 -0700 (PDT) Subject: [PATCH bpf v2 4/4] bpf: selftest to verify mixing bpf2bpf calls and tailcalls with insn patch From: John Fastabend To: maciej.fijalkowski@intel.com, ast@kernel.org, daniel@iogearbox.net, andriin@fb.com Cc: john.fastabend@gmail.com, netdev@vger.kernel.org, netdev@vger.kernel.org Date: Wed, 16 Jun 2021 15:55:57 -0700 Message-ID: <162388415754.151936.11542697725599301296.stgit@john-XPS-13-9370> In-Reply-To: <162388400488.151936.1658153981415911010.stgit@john-XPS-13-9370> References: <162388400488.151936.1658153981415911010.stgit@john-XPS-13-9370> User-Agent: StGit/0.23-85-g6af9 MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org This adds some extra noise to the tailcall_bpf2bpf4 tests that will cause verify to patch insns. This then moves around subprog start/end insn index and poke descriptor insn index to ensure that verify and JIT will continue to track these correctly. If done correctly verifier should pass this program same as before and JIT should emit tail call logic. Signed-off-by: John Fastabend --- tools/testing/selftests/bpf/prog_tests/tailcalls.c | 36 ++++++++++++++------ .../selftests/bpf/progs/tailcall_bpf2bpf4.c | 20 +++++++++++ 2 files changed, 45 insertions(+), 11 deletions(-) diff --git a/tools/testing/selftests/bpf/prog_tests/tailcalls.c b/tools/testing/selftests/bpf/prog_tests/tailcalls.c index ee27d68d2a1c..b5940e6ca67c 100644 --- a/tools/testing/selftests/bpf/prog_tests/tailcalls.c +++ b/tools/testing/selftests/bpf/prog_tests/tailcalls.c @@ -715,6 +715,8 @@ static void test_tailcall_bpf2bpf_3(void) bpf_object__close(obj); } +#include "tailcall_bpf2bpf4.skel.h" + /* test_tailcall_bpf2bpf_4 checks that tailcall counter is correctly preserved * across tailcalls combined with bpf2bpf calls. for making sure that tailcall * counter behaves correctly, bpf program will go through following flow: @@ -727,10 +729,15 @@ static void test_tailcall_bpf2bpf_3(void) * the loop begins. At the end of the test make sure that the global counter is * equal to 31, because tailcall counter includes the first two tailcalls * whereas global counter is incremented only on loop presented on flow above. + * + * The noise parameter is used to insert bpf_map_update calls into the logic + * to force verifier to patch instructions. This allows us to ensure jump + * logic remains correct with instruction movement. */ -static void test_tailcall_bpf2bpf_4(void) +static void test_tailcall_bpf2bpf_4(bool noise) { - int err, map_fd, prog_fd, main_fd, data_fd, i, val; + int err, map_fd, prog_fd, main_fd, data_fd, i; + struct tailcall_bpf2bpf4__bss val; struct bpf_map *prog_array, *data_map; struct bpf_program *prog; struct bpf_object *obj; @@ -774,11 +781,6 @@ static void test_tailcall_bpf2bpf_4(void) goto out; } - err = bpf_prog_test_run(main_fd, 1, &pkt_v4, sizeof(pkt_v4), 0, - &duration, &retval, NULL); - CHECK(err || retval != sizeof(pkt_v4) * 3, "tailcall", "err %d errno %d retval %d\n", - err, errno, retval); - data_map = bpf_object__find_map_by_name(obj, "tailcall.bss"); if (CHECK_FAIL(!data_map || !bpf_map__is_internal(data_map))) return; @@ -787,10 +789,22 @@ static void test_tailcall_bpf2bpf_4(void) if (CHECK_FAIL(map_fd < 0)) return; + i = 0; + val.noise = noise; + val.count = 0; + err = bpf_map_update_elem(data_fd, &i, &val, BPF_ANY); + if (CHECK_FAIL(err)) + goto out; + + err = bpf_prog_test_run(main_fd, 1, &pkt_v4, sizeof(pkt_v4), 0, + &duration, &retval, NULL); + CHECK(err || retval != sizeof(pkt_v4) * 3, "tailcall", "err %d errno %d retval %d\n", + err, errno, retval); + i = 0; err = bpf_map_lookup_elem(data_fd, &i, &val); - CHECK(err || val != 31, "tailcall count", "err %d errno %d count %d\n", - err, errno, val); + CHECK(err || val.count != 31, "tailcall count", "err %d errno %d count %d\n", + err, errno, val.count); out: bpf_object__close(obj); @@ -815,5 +829,7 @@ void test_tailcalls(void) if (test__start_subtest("tailcall_bpf2bpf_3")) test_tailcall_bpf2bpf_3(); if (test__start_subtest("tailcall_bpf2bpf_4")) - test_tailcall_bpf2bpf_4(); + test_tailcall_bpf2bpf_4(false); + if (test__start_subtest("tailcall_bpf2bpf_5")) + test_tailcall_bpf2bpf_4(true); } diff --git a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf4.c b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf4.c index 9a1b166b7fbe..e89368a50b97 100644 --- a/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf4.c +++ b/tools/testing/selftests/bpf/progs/tailcall_bpf2bpf4.c @@ -2,6 +2,13 @@ #include #include +struct { + __uint(type, BPF_MAP_TYPE_ARRAY); + __uint(max_entries, 1); + __uint(key_size, sizeof(__u32)); + __uint(value_size, sizeof(__u32)); +} nop_table SEC(".maps"); + struct { __uint(type, BPF_MAP_TYPE_PROG_ARRAY); __uint(max_entries, 3); @@ -9,11 +16,22 @@ struct { __uint(value_size, sizeof(__u32)); } jmp_table SEC(".maps"); -static volatile int count; +int count = 0; +int noise = 0; + +__always_inline int subprog_noise(void) +{ + __u32 key = 0; + + bpf_map_lookup_elem(&nop_table, &key); + return 0; +} __noinline int subprog_tail_2(struct __sk_buff *skb) { + if (noise) + subprog_noise(); bpf_tail_call_static(skb, &jmp_table, 2); return skb->len * 3; }