From patchwork Fri Jun 19 22:57:21 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Bianconi X-Patchwork-Id: 217498 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46BDAC433E1 for ; Fri, 19 Jun 2020 22:58:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 271EE22403 for ; Fri, 19 Jun 2020 22:58:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592607501; bh=cvu87u4U1bfkyE8xlkGmVTo6h6j7+k8okDg/ORG4hwA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=KkLwbNjArYY696OW2HMYlpOnvIXl5LVtMkAFJF1F+lsb+ShLPuOHuukPxAAnwem2S dVUsxaZd1gRkue/hM7ZO/qeqv0RnM72gest8qIaVPYoQIs9MZv3ukOdp1OzVFAN7M1 30MqjNzElsiOrPl6tOcojNQsSzpvxQKjoI3L3+fI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730301AbgFSW6L (ORCPT ); Fri, 19 Jun 2020 18:58:11 -0400 Received: from mail.kernel.org ([198.145.29.99]:53380 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730245AbgFSW6F (ORCPT ); Fri, 19 Jun 2020 18:58:05 -0400 Received: from localhost.localdomain.com (unknown [151.48.138.186]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 09BF82246B; Fri, 19 Jun 2020 22:58:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1592607484; bh=cvu87u4U1bfkyE8xlkGmVTo6h6j7+k8okDg/ORG4hwA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Y1TPvKpqDhDuwAVBwwxL+SfG83gTC42O1g+k9LfkiuGTgGpY9uLQIXZAANMuTxu+k uxqEShuikuoWqnLgdq0akoQx1i3wzyZE9OFR9O7Cek3CB0CByt5KofesVAYjq2gx4T +cdMZSL8FXmX/KTDhK1hxs86M6cWxHxhgdz3OtV8= From: Lorenzo Bianconi To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: davem@davemloft.net, ast@kernel.org, brouer@redhat.com, daniel@iogearbox.net, toke@redhat.com, lorenzo.bianconi@redhat.com, dsahern@kernel.org Subject: [PATCH v2 bpf-next 5/8] bpf: cpumap: implement XDP_REDIRECT for eBPF programs attached to map entries Date: Sat, 20 Jun 2020 00:57:21 +0200 Message-Id: <3b5801edede448a3d2868f63957dcb88ff7ff57a.1592606391.git.lorenzo@kernel.org> X-Mailer: git-send-email 2.26.2 In-Reply-To: References: MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Introduce XDP_REDIRECT support for eBPF programs attached to cpumap entries. This patch has been tested on Marvell ESPRESSObin using a modified version of xdp_redirect_cpu sample in order to attach a XDP program to CPUMAP entries to perform a redirect on the mvneta interface. In particular the following scenario has been tested: rq (cpu0) --> mvneta - XDP_REDIRECT (cpu0) --> CPUMAP - XDP_REDIRECT (cpu1) --> mvneta $./xdp_redirect_cpu -p xdp_cpu_map0 -d eth0 -c 1 -e xdp_redirect \ -f xdp_redirect_kern.o -m tx_port -r eth0 tx: 285.2 Kpps rx: 285.2 Kpps Attacching a simple XDP program on eth0 to perform XDP_TX gives comparable results: tx: 288.4 Kpps rx: 288.4 Kpps Co-developed-by: Jesper Dangaard Brouer Signed-off-by: Jesper Dangaard Brouer Signed-off-by: Lorenzo Bianconi --- include/net/xdp.h | 1 + include/trace/events/xdp.h | 6 ++++-- kernel/bpf/cpumap.c | 17 +++++++++++++++-- 3 files changed, 20 insertions(+), 4 deletions(-) diff --git a/include/net/xdp.h b/include/net/xdp.h index 441716a0c0a4..71a4e30003e5 100644 --- a/include/net/xdp.h +++ b/include/net/xdp.h @@ -99,6 +99,7 @@ struct xdp_frame { }; struct xdp_cpumap_stats { + unsigned int redirect; unsigned int pass; unsigned int drop; }; diff --git a/include/trace/events/xdp.h b/include/trace/events/xdp.h index e2c99f5bee39..cd24e8a59529 100644 --- a/include/trace/events/xdp.h +++ b/include/trace/events/xdp.h @@ -190,6 +190,7 @@ TRACE_EVENT(xdp_cpumap_kthread, __field(int, sched) __field(unsigned int, xdp_pass) __field(unsigned int, xdp_drop) + __field(unsigned int, xdp_redirect) ), TP_fast_assign( @@ -201,18 +202,19 @@ TRACE_EVENT(xdp_cpumap_kthread, __entry->sched = sched; __entry->xdp_pass = xdp_stats->pass; __entry->xdp_drop = xdp_stats->drop; + __entry->xdp_redirect = xdp_stats->redirect; ), TP_printk("kthread" " cpu=%d map_id=%d action=%s" " processed=%u drops=%u" " sched=%d" - " xdp_pass=%u xdp_drop=%u", + " xdp_pass=%u xdp_drop=%u xdp_redirect=%u", __entry->cpu, __entry->map_id, __print_symbolic(__entry->act, __XDP_ACT_SYM_TAB), __entry->processed, __entry->drops, __entry->sched, - __entry->xdp_pass, __entry->xdp_drop) + __entry->xdp_pass, __entry->xdp_drop, __entry->xdp_redirect) ); TRACE_EVENT(xdp_cpumap_enqueue, diff --git a/kernel/bpf/cpumap.c b/kernel/bpf/cpumap.c index dedf33d8c8d0..e0160e24be81 100644 --- a/kernel/bpf/cpumap.c +++ b/kernel/bpf/cpumap.c @@ -240,7 +240,7 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, xdp_set_return_frame_no_direct(); xdp.rxq = &rxq; - rcu_read_lock(); + rcu_read_lock_bh(); prog = READ_ONCE(rcpu->prog); for (i = 0; i < n; i++) { @@ -266,6 +266,16 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, stats->pass++; } break; + case XDP_REDIRECT: + err = xdp_do_redirect(xdpf->dev_rx, &xdp, + prog); + if (unlikely(err)) { + xdp_return_frame(xdpf); + stats->drop++; + } else { + stats->redirect++; + } + break; default: bpf_warn_invalid_xdp_action(act); /* fallthrough */ @@ -276,7 +286,10 @@ static int cpu_map_bpf_prog_run_xdp(struct bpf_cpu_map_entry *rcpu, } } - rcu_read_unlock(); + if (stats->redirect) + xdp_do_flush_map(); + + rcu_read_unlock_bh(); /* resched point, may call do_softirq() */ xdp_clear_return_frame_no_direct(); return nframes;