From patchwork Mon May 11 18:52:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219446 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B971FC54E4B for ; Mon, 11 May 2020 18:53:12 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9605620720 for ; Mon, 11 May 2020 18:53:12 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="iGRDEaI0" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731176AbgEKSw1 (ORCPT ); Mon, 11 May 2020 14:52:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1729215AbgEKSw0 (ORCPT ); Mon, 11 May 2020 14:52:26 -0400 Received: from mail-wr1-x444.google.com (mail-wr1-x444.google.com [IPv6:2a00:1450:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 78CC0C05BD0A for ; Mon, 11 May 2020 11:52:24 -0700 (PDT) Received: by mail-wr1-x444.google.com with SMTP id v12so12270314wrp.12 for ; Mon, 11 May 2020 11:52:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nSivWb3QDofiBZMqyO7kOgrM+Q+I6G7D3bjxIkCgD9M=; b=iGRDEaI02V4EZxUWsAedwQTbA+LE7wyG5CfS9CJIa0S7AM/HWQBgO1+uqUbDAo8uZz jkUvjsyGvzHcGaBzn2ZwGX441PyfvEeyr83VC2ewDcMGPXiLetzrxEa1Jq2h+fAWfFHQ q4jng9c+AK6aq7N5pATqJJkkVSz0Bh7VXxWZQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nSivWb3QDofiBZMqyO7kOgrM+Q+I6G7D3bjxIkCgD9M=; b=EXPPahUqBRpa7CzqQRbLS53e8vIjDUn/ke3w3d4eHxBsXxnbZfVMcYT07Dd/Y/mmGe /1/1ReKomyPGVw9afOnUc3cZREKz8rOt20llp8dykX4vO4p8uI14nFWkqgZTM9Pc3xkd XSGr8uyteRAlC6dvEPnwsDQ7oLNgUY5GG9ApIm+NwHm3uUm8NLXJwG6q3/L3RUG62VrD xQHjsl1jyWmXMJITpxcNMrEc7JMpgVW1KvppVm8+eizHzxPhkAiTDay17B5G91qjFAvg FMGVisKbr0ero0wjzqC/oDidCLJFT9NNrAeLU/8JmHx503RHW1IgpAOe53vjRh7vCfG5 c4tQ== X-Gm-Message-State: AGi0Pub/0HacLmt+0LQj4WBYxVybHppWzjk3KULEzlGR/2f2gb46KOS4 GY/rsM2p+RFlrpoSWt27RSo19eMWGK8= X-Google-Smtp-Source: APiQypJIc+6evEBthviUr24GRcnrqU3j4tR0VGFuZlUSDqQbIAJqIdUQGkv7jP4H+hWIhKkJ7xp79A== X-Received: by 2002:adf:f146:: with SMTP id y6mr21628372wro.132.1589223142787; Mon, 11 May 2020 11:52:22 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id d1sm9933382wrc.26.2020.05.11.11.52.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:22 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau , Lorenz Bauer Subject: [PATCH bpf-next v2 01/17] flow_dissector: Extract attach/detach/query helpers Date: Mon, 11 May 2020 20:52:02 +0200 Message-Id: <20200511185218.1422406-2-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Move generic parts of callbacks for querying, attaching, and detaching a single BPF program for reuse by other BPF program types. Subsequent patch makes use of the extracted routines. Reviewed-by: Lorenz Bauer Signed-off-by: Jakub Sitnicki --- include/linux/bpf.h | 8 +++++ net/core/filter.c | 68 +++++++++++++++++++++++++++++++++++++++ net/core/flow_dissector.c | 61 +++++++---------------------------- 3 files changed, 88 insertions(+), 49 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index cf4b6e44f2bc..1cf4fae7987d 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -32,6 +32,7 @@ struct btf; struct btf_type; struct exception_table_entry; struct seq_operations; +struct mutex; extern struct idr btf_idr; extern spinlock_t btf_idr_lock; @@ -1696,4 +1697,11 @@ enum bpf_text_poke_type { int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type t, void *addr1, void *addr2); +int bpf_prog_query_one(struct bpf_prog __rcu **pprog, + const union bpf_attr *attr, + union bpf_attr __user *uattr); +int bpf_prog_attach_one(struct bpf_prog __rcu **pprog, struct mutex *lock, + struct bpf_prog *prog, u32 flags); +int bpf_prog_detach_one(struct bpf_prog __rcu **pprog, struct mutex *lock); + #endif /* _LINUX_BPF_H */ diff --git a/net/core/filter.c b/net/core/filter.c index da0634979f53..48ed970f4ae1 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -8738,6 +8738,74 @@ int sk_get_filter(struct sock *sk, struct sock_filter __user *ubuf, return ret; } +int bpf_prog_query_one(struct bpf_prog __rcu **pprog, + const union bpf_attr *attr, + union bpf_attr __user *uattr) +{ + __u32 __user *prog_ids = u64_to_user_ptr(attr->query.prog_ids); + u32 prog_id, prog_cnt = 0, flags = 0; + struct bpf_prog *attached; + + if (attr->query.query_flags) + return -EINVAL; + + rcu_read_lock(); + attached = rcu_dereference(*pprog); + if (attached) { + prog_cnt = 1; + prog_id = attached->aux->id; + } + rcu_read_unlock(); + + if (copy_to_user(&uattr->query.attach_flags, &flags, sizeof(flags))) + return -EFAULT; + if (copy_to_user(&uattr->query.prog_cnt, &prog_cnt, sizeof(prog_cnt))) + return -EFAULT; + + if (!attr->query.prog_cnt || !prog_ids || !prog_cnt) + return 0; + + if (copy_to_user(prog_ids, &prog_id, sizeof(u32))) + return -EFAULT; + + return 0; +} + +int bpf_prog_attach_one(struct bpf_prog __rcu **pprog, struct mutex *lock, + struct bpf_prog *prog, u32 flags) +{ + struct bpf_prog *attached; + + if (flags) + return -EINVAL; + + attached = rcu_dereference_protected(*pprog, + lockdep_is_held(lock)); + if (attached == prog) { + /* The same program cannot be attached twice */ + return -EINVAL; + } + rcu_assign_pointer(*pprog, prog); + if (attached) + bpf_prog_put(attached); + + return 0; +} + +int bpf_prog_detach_one(struct bpf_prog __rcu **pprog, struct mutex *lock) +{ + struct bpf_prog *attached; + + attached = rcu_dereference_protected(*pprog, + lockdep_is_held(lock)); + if (!attached) + return -ENOENT; + RCU_INIT_POINTER(*pprog, NULL); + bpf_prog_put(attached); + + return 0; +} + #ifdef CONFIG_INET static void bpf_init_reuseport_kern(struct sk_reuseport_kern *reuse_kern, struct sock_reuseport *reuse, diff --git a/net/core/flow_dissector.c b/net/core/flow_dissector.c index 3eff84824c8b..5ff99ed175bd 100644 --- a/net/core/flow_dissector.c +++ b/net/core/flow_dissector.c @@ -73,46 +73,22 @@ EXPORT_SYMBOL(skb_flow_dissector_init); int skb_flow_dissector_prog_query(const union bpf_attr *attr, union bpf_attr __user *uattr) { - __u32 __user *prog_ids = u64_to_user_ptr(attr->query.prog_ids); - u32 prog_id, prog_cnt = 0, flags = 0; - struct bpf_prog *attached; struct net *net; - - if (attr->query.query_flags) - return -EINVAL; + int ret; net = get_net_ns_by_fd(attr->query.target_fd); if (IS_ERR(net)) return PTR_ERR(net); - rcu_read_lock(); - attached = rcu_dereference(net->flow_dissector_prog); - if (attached) { - prog_cnt = 1; - prog_id = attached->aux->id; - } - rcu_read_unlock(); + ret = bpf_prog_query_one(&net->flow_dissector_prog, attr, uattr); put_net(net); - - if (copy_to_user(&uattr->query.attach_flags, &flags, sizeof(flags))) - return -EFAULT; - if (copy_to_user(&uattr->query.prog_cnt, &prog_cnt, sizeof(prog_cnt))) - return -EFAULT; - - if (!attr->query.prog_cnt || !prog_ids || !prog_cnt) - return 0; - - if (copy_to_user(prog_ids, &prog_id, sizeof(u32))) - return -EFAULT; - - return 0; + return ret; } int skb_flow_dissector_bpf_prog_attach(const union bpf_attr *attr, struct bpf_prog *prog) { - struct bpf_prog *attached; struct net *net; int ret = 0; @@ -145,16 +121,9 @@ int skb_flow_dissector_bpf_prog_attach(const union bpf_attr *attr, } } - attached = rcu_dereference_protected(net->flow_dissector_prog, - lockdep_is_held(&flow_dissector_mutex)); - if (attached == prog) { - /* The same program cannot be attached twice */ - ret = -EINVAL; - goto out; - } - rcu_assign_pointer(net->flow_dissector_prog, prog); - if (attached) - bpf_prog_put(attached); + ret = bpf_prog_attach_one(&net->flow_dissector_prog, + &flow_dissector_mutex, prog, + attr->attach_flags); out: mutex_unlock(&flow_dissector_mutex); return ret; @@ -162,21 +131,15 @@ int skb_flow_dissector_bpf_prog_attach(const union bpf_attr *attr, int skb_flow_dissector_bpf_prog_detach(const union bpf_attr *attr) { - struct bpf_prog *attached; - struct net *net; + struct net *net = current->nsproxy->net_ns; + int ret; - net = current->nsproxy->net_ns; mutex_lock(&flow_dissector_mutex); - attached = rcu_dereference_protected(net->flow_dissector_prog, - lockdep_is_held(&flow_dissector_mutex)); - if (!attached) { - mutex_unlock(&flow_dissector_mutex); - return -ENOENT; - } - RCU_INIT_POINTER(net->flow_dissector_prog, NULL); - bpf_prog_put(attached); + ret = bpf_prog_detach_one(&net->flow_dissector_prog, + &flow_dissector_mutex); mutex_unlock(&flow_dissector_mutex); - return 0; + + return ret; } /** From patchwork Mon May 11 18:52:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219454 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40D86C47255 for ; Mon, 11 May 2020 18:52:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1710F20752 for ; Mon, 11 May 2020 18:52:29 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="lN6akM13" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731193AbgEKSw2 (ORCPT ); Mon, 11 May 2020 14:52:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731118AbgEKSw1 (ORCPT ); Mon, 11 May 2020 14:52:27 -0400 Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5F756C061A0C for ; Mon, 11 May 2020 11:52:27 -0700 (PDT) Received: by mail-wr1-x442.google.com with SMTP id x17so12329792wrt.5 for ; Mon, 11 May 2020 11:52:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FvXoRuhTO+OxNF0cbAiPa+Gek9SWWsXi4MoNMSh2OYs=; b=lN6akM13XSE2xICX2oJeLUaQpxpRfD86G5Fg8THYpbFg2B9oM6eP/Qd3dayzo24jbB 4K66X/3QSxRSboA7ONrOl7m05HVl3UcogusI0fZa+CVwYlg8GmCQ+Ur9lx4noBN24Kr+ euPFVc1ZMy6yKWbHEzK4AYM1tbHH7IOkHVgrE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FvXoRuhTO+OxNF0cbAiPa+Gek9SWWsXi4MoNMSh2OYs=; b=STLt6ZsLd2DSrhcQTjOYtYcZz46ueFvrZbCJJb2O69xaB//021Ldpwa4V4tPFlM16c nj+9o8cpWKJqodI3/h7JNyQ9/SZSP1Kr2cs2EGV3ULtVFHW3HfJGCy8XAihBRY9dOdf4 xcSTsUOd4KIe1wrPSNy6Zka1A04YfqkkGvIaZsDfICm64pK/f+lD4NndJ0KCUuHaj+TK uacZXV1q/HNrIQIzzSRCgNGxidpJEecSosOx9Y/Lakif35HKreiPt/PCJmsEXUPh7KG2 KsSErXfAK9kSD1fBNM0MiqMLWkHqEEhQ5SUkoNefllvtLBBL+gPrbVXkMuTIboY83Qx6 yFrA== X-Gm-Message-State: AGi0Pub6M2gRTya/OZhelrZBLwEDnSWrQKT22b8r89qdYHZYA2S7bkAn vKGOJw561ZPeCw2b9Ok36JqlUMfS0Yk= X-Google-Smtp-Source: APiQypJxh99m7yN54Y1joe7NaE8hRQ595h3QhvRwfDoyeebRy9cDyEUuNOjoJTrrz38t1z99ugb66g== X-Received: by 2002:adf:97d9:: with SMTP id t25mr11823884wrb.176.1589223145799; Mon, 11 May 2020 11:52:25 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id p190sm27952831wmp.38.2020.05.11.11.52.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:25 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau , Lorenz Bauer Subject: [PATCH bpf-next v2 03/17] inet: Store layer 4 protocol in inet_hashinfo Date: Mon, 11 May 2020 20:52:04 +0200 Message-Id: <20200511185218.1422406-4-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Make it possible to identify the protocol of sockets stored in hashinfo without looking up a socket. Subsequent patches make use the new field at the socket lookup time to ensure that BPF program selects only sockets with matching protocol. Reviewed-by: Lorenz Bauer Signed-off-by: Jakub Sitnicki --- include/net/inet_hashtables.h | 3 +++ net/dccp/proto.c | 2 +- net/ipv4/tcp_ipv4.c | 2 +- 3 files changed, 5 insertions(+), 2 deletions(-) diff --git a/include/net/inet_hashtables.h b/include/net/inet_hashtables.h index ad64ba6a057f..6072dfbd1078 100644 --- a/include/net/inet_hashtables.h +++ b/include/net/inet_hashtables.h @@ -144,6 +144,9 @@ struct inet_hashinfo { unsigned int lhash2_mask; struct inet_listen_hashbucket *lhash2; + /* Layer 4 protocol of the stored sockets */ + int protocol; + /* All the above members are written once at bootup and * never written again _or_ are predominantly read-access. * diff --git a/net/dccp/proto.c b/net/dccp/proto.c index 4af8a98fe784..c826419e68e6 100644 --- a/net/dccp/proto.c +++ b/net/dccp/proto.c @@ -45,7 +45,7 @@ EXPORT_SYMBOL_GPL(dccp_statistics); struct percpu_counter dccp_orphan_count; EXPORT_SYMBOL_GPL(dccp_orphan_count); -struct inet_hashinfo dccp_hashinfo; +struct inet_hashinfo dccp_hashinfo = { .protocol = IPPROTO_DCCP }; EXPORT_SYMBOL_GPL(dccp_hashinfo); /* the maximum queue length for tx in packets. 0 is no limit */ diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index 6c05f1ceb538..77e4f4e4c73c 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -87,7 +87,7 @@ static int tcp_v4_md5_hash_hdr(char *md5_hash, const struct tcp_md5sig_key *key, __be32 daddr, __be32 saddr, const struct tcphdr *th); #endif -struct inet_hashinfo tcp_hashinfo; +struct inet_hashinfo tcp_hashinfo = { .protocol = IPPROTO_TCP }; EXPORT_SYMBOL(tcp_hashinfo); static u32 tcp_v4_init_seq(const struct sk_buff *skb) From patchwork Mon May 11 18:52:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB510C47255 for ; Mon, 11 May 2020 18:53:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AD5C0206DB for ; Mon, 11 May 2020 18:53:06 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="vGtK6+5I" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731300AbgEKSxG (ORCPT ); Mon, 11 May 2020 14:53:06 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52344 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731214AbgEKSwb (ORCPT ); Mon, 11 May 2020 14:52:31 -0400 Received: from mail-wm1-x343.google.com (mail-wm1-x343.google.com [IPv6:2a00:1450:4864:20::343]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9EA52C05BD09 for ; Mon, 11 May 2020 11:52:31 -0700 (PDT) Received: by mail-wm1-x343.google.com with SMTP id m12so14247313wmc.0 for ; Mon, 11 May 2020 11:52:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=M0vCBt1D4iC0eNJSSMfqS7W0wA2E5Kan2xeXfsA3ILU=; b=vGtK6+5IMQq2lXuqJCle4VpwFMNrCzuQ45bkU70sjzrX0aB/IzvNSrCP/uyR0uEDH4 J++9ehUEcJy1GI6K4InFE9cWncot0Gmu9gs81y6G98x5NkCnN6Yw1PcL/9tofskOFj5k k0JdFguYx9bWxDmHE9oUwr4bzh1R3moJ0A6Bg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=M0vCBt1D4iC0eNJSSMfqS7W0wA2E5Kan2xeXfsA3ILU=; b=atB2ZQigzGOvASxr2QHWYKB6ENcjmecmOOZzyx+QXjdxgv9m+iu4mneHMhrNbuWz8+ IsWkoBNO81hOdMtpXnVohg1wqB6dT2K3J0LeMm2Py2bu75zQUewLBJuKdds/VBobRIEz Z7e+o8YX7tzUdgcBsQnVURyZtQJ4bWfLjFm/qrfJ0uACPi+uM3HiTZPzRqVLuPmOmksl u+KLgOAhEvZs/FKGNbucLcyGorhaMwFRf4pugIkHlTiyMZvFOwOco1ZhBKUDlM5P/nSn Mxh3DNwIGgvQF9bzOkEHNuli0bhK7/r9wIt2KPuP1drCQk8fE3zvHPD3D/v6Vwu8mNDG swnw== X-Gm-Message-State: AGi0PuaR2FB039/Vauvd6RzbM16/sj95/FI80TpgLvJDnzrK4150xNTa QBm5JMO8wvJyHpxg4d4ffp7Z7KEG0dE= X-Google-Smtp-Source: APiQypKw+cunp3PJFKZ9BE3HaSOt/lJ2FAuAJGIRZ2whrxILB22+1PKRkJk8LWco5pqyuzI75b1iEA== X-Received: by 2002:a1c:6042:: with SMTP id u63mr18978565wmb.65.1589223150101; Mon, 11 May 2020 11:52:30 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id b2sm15346343wrm.30.2020.05.11.11.52.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:29 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau Subject: [PATCH bpf-next v2 06/17] inet6: Extract helper for selecting socket from reuseport group Date: Mon, 11 May 2020 20:52:07 +0200 Message-Id: <20200511185218.1422406-7-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Prepare for calling into reuseport from inet6_lookup_listener as well. Signed-off-by: Jakub Sitnicki --- net/ipv6/inet6_hashtables.c | 31 ++++++++++++++++++++++--------- 1 file changed, 22 insertions(+), 9 deletions(-) diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c index fbe9d4295eac..03942eef8ab6 100644 --- a/net/ipv6/inet6_hashtables.c +++ b/net/ipv6/inet6_hashtables.c @@ -111,6 +111,23 @@ static inline int compute_score(struct sock *sk, struct net *net, return score; } +static inline struct sock *lookup_reuseport(struct net *net, struct sock *sk, + struct sk_buff *skb, int doff, + const struct in6_addr *saddr, + __be16 sport, + const struct in6_addr *daddr, + unsigned short hnum) +{ + struct sock *reuse_sk = NULL; + u32 phash; + + if (sk->sk_reuseport) { + phash = inet6_ehashfn(net, daddr, hnum, saddr, sport); + reuse_sk = reuseport_select_sock(sk, phash, skb, doff); + } + return reuse_sk; +} + /* called with rcu_read_lock() */ static struct sock *inet6_lhash2_lookup(struct net *net, struct inet_listen_hashbucket *ilb2, @@ -123,21 +140,17 @@ static struct sock *inet6_lhash2_lookup(struct net *net, struct inet_connection_sock *icsk; struct sock *sk, *result = NULL; int score, hiscore = 0; - u32 phash = 0; inet_lhash2_for_each_icsk_rcu(icsk, &ilb2->head) { sk = (struct sock *)icsk; score = compute_score(sk, net, hnum, daddr, dif, sdif, exact_dif); if (score > hiscore) { - if (sk->sk_reuseport) { - phash = inet6_ehashfn(net, daddr, hnum, - saddr, sport); - result = reuseport_select_sock(sk, phash, - skb, doff); - if (result) - return result; - } + result = lookup_reuseport(net, sk, skb, doff, + saddr, sport, daddr, hnum); + if (result) + return result; + result = sk; hiscore = score; } From patchwork Mon May 11 18:52:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3FA8FC54E4B for ; Mon, 11 May 2020 18:52:37 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2392D20720 for ; Mon, 11 May 2020 18:52:37 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="G3uudaq2" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731236AbgEKSwf (ORCPT ); Mon, 11 May 2020 14:52:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731228AbgEKSwd (ORCPT ); Mon, 11 May 2020 14:52:33 -0400 Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1DA7AC05BD0B for ; Mon, 11 May 2020 11:52:33 -0700 (PDT) Received: by mail-wm1-x344.google.com with SMTP id n5so5639512wmd.0 for ; Mon, 11 May 2020 11:52:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lXY32cPktzhafRRWtWGOpRuM3Po7H5+m/zDtuRoVn8o=; b=G3uudaq2H+LHMIuy2V8uCDg/wetvQgEYQYys19LPCNs7x0UaZBnNFJVzHRDkPfEAnk PkhuWRV400r55AKOYpUdln06fXPs1gzCKaThph47657LpUN1X+S2Fvlm7W1yNeVL1kzx Pf0To8XIPcAGPxFGbyLybGWCfndXoWUF5Fm8c= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lXY32cPktzhafRRWtWGOpRuM3Po7H5+m/zDtuRoVn8o=; b=M7CnYE7UEJdjoAy73GYzhsaiwV/YgJen6H2afuNaTPTp7Bn0G6FreuWY80pG5lyVZa GqKeEw8Ns3l+L2ROSwSXhiPn11ojmi+IsL8PVfB4Pv8ESnju4Upxc/dQZmsDk9acpwK0 9J4OPF//DFbLT2VSatwCSHGPx7h+nfRsY/joV08tQr3BuYevEbpm6DSWY+sTNSNxoQs4 C7TalQNZX2NiosusFf0Mcbt9Ki+0Ob5F4MP5bkuP10IineYd6ryWSKeSHS56ajdHBN1N ql1H2IZyrgZalTNcL6kbllRKDtFZK/HM0TupGmdHt9/xSZJd6YSDrDcJfqL1R9WKZCCe aJhQ== X-Gm-Message-State: AGi0PuaNIq2Zgki+4o/R1KKGAmPviyrCvDkbz/rsk30Kqp8T9yxHbxKR 0nlcgA4VrcDFV7iHq9rgdNj1qwOZPUk= X-Google-Smtp-Source: APiQypIb/6QR1xwc8HMNRkKb5PiaCQAJx5viEmcQac4oH43IjJL4Zjtq18xZr/EjQmvQtDmTYWXxyA== X-Received: by 2002:a1c:e906:: with SMTP id q6mr13593932wmc.62.1589223151527; Mon, 11 May 2020 11:52:31 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id c17sm19072157wrn.59.2020.05.11.11.52.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:31 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau , Marek Majkowski , Lorenz Bauer Subject: [PATCH bpf-next v2 07/17] inet6: Run SK_LOOKUP BPF program on socket lookup Date: Mon, 11 May 2020 20:52:08 +0200 Message-Id: <20200511185218.1422406-8-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Following ipv4 stack changes, run a BPF program attached to netns before looking up a listening socket. Program can return a listening socket to use as result of socket lookup, fail the lookup, or take no action. Suggested-by: Marek Majkowski Reviewed-by: Lorenz Bauer Signed-off-by: Jakub Sitnicki --- include/net/inet6_hashtables.h | 20 ++++++++++++++++++++ net/ipv6/inet6_hashtables.c | 15 ++++++++++++++- 2 files changed, 34 insertions(+), 1 deletion(-) diff --git a/include/net/inet6_hashtables.h b/include/net/inet6_hashtables.h index 81b965953036..8b8c0cb92ea8 100644 --- a/include/net/inet6_hashtables.h +++ b/include/net/inet6_hashtables.h @@ -21,6 +21,7 @@ #include #include +#include struct inet_hashinfo; @@ -103,6 +104,25 @@ struct sock *inet6_lookup(struct net *net, struct inet_hashinfo *hashinfo, const int dif); int inet6_hash(struct sock *sk); + +static inline struct sock *inet6_lookup_run_bpf(struct net *net, u8 protocol, + const struct in6_addr *saddr, + __be16 sport, + const struct in6_addr *daddr, + u16 dport) +{ + struct bpf_sk_lookup_kern ctx = { + .family = AF_INET6, + .protocol = protocol, + .v6.saddr = *saddr, + .v6.daddr = *daddr, + .sport = sport, + .dport = dport, + }; + + return bpf_sk_lookup_run(net, &ctx); +} + #endif /* IS_ENABLED(CONFIG_IPV6) */ #define INET6_MATCH(__sk, __net, __saddr, __daddr, __ports, __dif, __sdif) \ diff --git a/net/ipv6/inet6_hashtables.c b/net/ipv6/inet6_hashtables.c index 03942eef8ab6..6d91de89fd2b 100644 --- a/net/ipv6/inet6_hashtables.c +++ b/net/ipv6/inet6_hashtables.c @@ -167,9 +167,22 @@ struct sock *inet6_lookup_listener(struct net *net, const unsigned short hnum, const int dif, const int sdif) { struct inet_listen_hashbucket *ilb2; - struct sock *result = NULL; + struct sock *result, *reuse_sk; unsigned int hash2; + /* Lookup redirect from BPF */ + result = inet6_lookup_run_bpf(net, hashinfo->protocol, + saddr, sport, daddr, hnum); + if (IS_ERR(result)) + return NULL; + if (result) { + reuse_sk = lookup_reuseport(net, result, skb, doff, + saddr, sport, daddr, hnum); + if (reuse_sk) + result = reuse_sk; + goto done; + } + hash2 = ipv6_portaddr_hash(net, daddr, hnum); ilb2 = inet_lhash2_bucket(hashinfo, hash2); From patchwork Mon May 11 18:52:09 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CD183C54E4B for ; Mon, 11 May 2020 18:52:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id A8122206DB for ; Mon, 11 May 2020 18:52:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="qNGUFYTd" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731253AbgEKSwl (ORCPT ); Mon, 11 May 2020 14:52:41 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731238AbgEKSwg (ORCPT ); Mon, 11 May 2020 14:52:36 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F0381C061A0C for ; Mon, 11 May 2020 11:52:34 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id h17so3422462wrc.8 for ; Mon, 11 May 2020 11:52:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PJC838rCblZYwK9d/soNBjN4ZElrFcTAEZBfSJoBG5Y=; b=qNGUFYTdql4aolrkJIC9ZsIglFJNx/4qqNWFgIWYu+sov/g1abKfyDJbOGXrhxRXYU Kfa4U6m4jBJC792ESqPOO6fAZzq1E46WhQ2uMzvWWFs3VGo6jYGgKQf9EHo4kt0RvzSn jir58MAzbFI9bs+AjGBQJYlCYumbErMQ886ao= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PJC838rCblZYwK9d/soNBjN4ZElrFcTAEZBfSJoBG5Y=; b=YVnR/M8LW9nm2g80U/0KPLOZKKiZP06/vyfrwRauRiW9x9ydDaEZqU3FckeJB84qSv VQ00WA6TVzRiRJkFil64297C/FOQJ+wEJHZBsUnxfF8Dv3/CdSVRdbEF+vIy+Qf8R2Tn SVKYwBDoL7CtU+LU4Yr7Ku0+8cwp5VQQT0BfM1QfimOGf7m93G/dYgg29RAQWO3psOKj 78dT68Fx7euA7cJCRj/OEhLKX1n2gDQsnygsKVKdbTwGjy4hQgz5n7BJYs5xjaEYBrt5 uZ0pYNw4orj1zZiglxhJWENLEzZiymDkLqGoyAFa5OXU84MfhlfXOaqxCZbC0tz6IeZU MpNA== X-Gm-Message-State: AGi0PuZZd4sDt3yEYLs/281l6HRmVk4mK74ODf3l9syHEd1jd3ttNyRf pNcZwpW5IMMAauB4bIoWUcBAU1z54WU= X-Google-Smtp-Source: APiQypJfxDp9v0xWUehGXJK+z2SnYMFWN7t/kXjw8HfT1v5wQgpE6fGjb6T3pXbOlpV+w6MmUQot0Q== X-Received: by 2002:a5d:65ce:: with SMTP id e14mr20300601wrw.314.1589223153053; Mon, 11 May 2020 11:52:33 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id m1sm19653478wrx.44.2020.05.11.11.52.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:32 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau , Lorenz Bauer Subject: [PATCH bpf-next v2 08/17] udp: Store layer 4 protocol in udp_table Date: Mon, 11 May 2020 20:52:09 +0200 Message-Id: <20200511185218.1422406-9-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Because UDP and UDP-Lite share code, we pass the L4 protocol identifier alongside the UDP socket table to functions which need to distinguishing between the two protocol. Put the protocol identifier in the UDP table itself, so that the protocol is known to any function in the call chain that operates on socket table. Subsequent patches make use the new udp_table field at the socket lookup time to ensure that BPF program selects only sockets with matching protocol. Reviewed-by: Lorenz Bauer Signed-off-by: Jakub Sitnicki --- include/net/udp.h | 10 ++++++---- net/ipv4/udp.c | 15 +++++++-------- net/ipv4/udp_impl.h | 2 +- net/ipv4/udplite.c | 4 ++-- net/ipv6/udp.c | 12 ++++++------ net/ipv6/udp_impl.h | 2 +- net/ipv6/udplite.c | 2 +- 7 files changed, 24 insertions(+), 23 deletions(-) diff --git a/include/net/udp.h b/include/net/udp.h index a8fa6c0c6ded..f81c46c71fee 100644 --- a/include/net/udp.h +++ b/include/net/udp.h @@ -63,16 +63,18 @@ struct udp_hslot { /** * struct udp_table - UDP table * - * @hash: hash table, sockets are hashed on (local port) - * @hash2: hash table, sockets are hashed on (local port, local address) - * @mask: number of slots in hash tables, minus 1 - * @log: log2(number of slots in hash table) + * @hash: hash table, sockets are hashed on (local port) + * @hash2: hash table, sockets are hashed on local (port, address) + * @mask: number of slots in hash tables, minus 1 + * @log: log2(number of slots in hash table) + * @protocol: layer 4 protocol of the stored sockets */ struct udp_table { struct udp_hslot *hash; struct udp_hslot *hash2; unsigned int mask; unsigned int log; + int protocol; }; extern struct udp_table udp_table; void udp_table_init(struct udp_table *, const char *); diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index 32564b350823..ce96b1746ddf 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -113,7 +113,7 @@ #include #include -struct udp_table udp_table __read_mostly; +struct udp_table udp_table __read_mostly = { .protocol = IPPROTO_UDP }; EXPORT_SYMBOL(udp_table); long sysctl_udp_mem[3] __read_mostly; @@ -2145,8 +2145,7 @@ EXPORT_SYMBOL(udp_sk_rx_dst_set); static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb, struct udphdr *uh, __be32 saddr, __be32 daddr, - struct udp_table *udptable, - int proto) + struct udp_table *udptable) { struct sock *sk, *first = NULL; unsigned short hnum = ntohs(uh->dest); @@ -2202,7 +2201,7 @@ static int __udp4_lib_mcast_deliver(struct net *net, struct sk_buff *skb, } else { kfree_skb(skb); __UDP_INC_STATS(net, UDP_MIB_IGNOREDMULTI, - proto == IPPROTO_UDPLITE); + udptable->protocol == IPPROTO_UDPLITE); } return 0; } @@ -2279,8 +2278,7 @@ static int udp_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb, * All we need to do is get the socket, and then do a checksum. */ -int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, - int proto) +int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable) { struct sock *sk; struct udphdr *uh; @@ -2288,6 +2286,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, struct rtable *rt = skb_rtable(skb); __be32 saddr, daddr; struct net *net = dev_net(skb->dev); + int proto = udptable->protocol; bool refcounted; /* @@ -2330,7 +2329,7 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, if (rt->rt_flags & (RTCF_BROADCAST|RTCF_MULTICAST)) return __udp4_lib_mcast_deliver(net, skb, uh, - saddr, daddr, udptable, proto); + saddr, daddr, udptable); sk = __udp4_lib_lookup_skb(skb, uh->source, uh->dest, udptable); if (sk) @@ -2504,7 +2503,7 @@ int udp_v4_early_demux(struct sk_buff *skb) int udp_rcv(struct sk_buff *skb) { - return __udp4_lib_rcv(skb, &udp_table, IPPROTO_UDP); + return __udp4_lib_rcv(skb, &udp_table); } void udp_destroy_sock(struct sock *sk) diff --git a/net/ipv4/udp_impl.h b/net/ipv4/udp_impl.h index 6b2fa77eeb1c..7013535f9084 100644 --- a/net/ipv4/udp_impl.h +++ b/net/ipv4/udp_impl.h @@ -6,7 +6,7 @@ #include #include -int __udp4_lib_rcv(struct sk_buff *, struct udp_table *, int); +int __udp4_lib_rcv(struct sk_buff *, struct udp_table *); int __udp4_lib_err(struct sk_buff *, u32, struct udp_table *); int udp_v4_get_port(struct sock *sk, unsigned short snum); diff --git a/net/ipv4/udplite.c b/net/ipv4/udplite.c index 5936d66d1ce2..4e4e85de95b2 100644 --- a/net/ipv4/udplite.c +++ b/net/ipv4/udplite.c @@ -14,12 +14,12 @@ #include #include "udp_impl.h" -struct udp_table udplite_table __read_mostly; +struct udp_table udplite_table __read_mostly = { .protocol = IPPROTO_UDPLITE }; EXPORT_SYMBOL(udplite_table); static int udplite_rcv(struct sk_buff *skb) { - return __udp4_lib_rcv(skb, &udplite_table, IPPROTO_UDPLITE); + return __udp4_lib_rcv(skb, &udplite_table); } static int udplite_err(struct sk_buff *skb, u32 info) diff --git a/net/ipv6/udp.c b/net/ipv6/udp.c index 7d4151747340..f7866fded418 100644 --- a/net/ipv6/udp.c +++ b/net/ipv6/udp.c @@ -741,7 +741,7 @@ static void udp6_csum_zero_error(struct sk_buff *skb) */ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb, const struct in6_addr *saddr, const struct in6_addr *daddr, - struct udp_table *udptable, int proto) + struct udp_table *udptable) { struct sock *sk, *first = NULL; const struct udphdr *uh = udp_hdr(skb); @@ -803,7 +803,7 @@ static int __udp6_lib_mcast_deliver(struct net *net, struct sk_buff *skb, } else { kfree_skb(skb); __UDP6_INC_STATS(net, UDP_MIB_IGNOREDMULTI, - proto == IPPROTO_UDPLITE); + udptable->protocol == IPPROTO_UDPLITE); } return 0; } @@ -836,11 +836,11 @@ static int udp6_unicast_rcv_skb(struct sock *sk, struct sk_buff *skb, return 0; } -int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, - int proto) +int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable) { const struct in6_addr *saddr, *daddr; struct net *net = dev_net(skb->dev); + int proto = udptable->protocol; struct udphdr *uh; struct sock *sk; bool refcounted; @@ -905,7 +905,7 @@ int __udp6_lib_rcv(struct sk_buff *skb, struct udp_table *udptable, */ if (ipv6_addr_is_multicast(daddr)) return __udp6_lib_mcast_deliver(net, skb, - saddr, daddr, udptable, proto); + saddr, daddr, udptable); /* Unicast */ sk = __udp6_lib_lookup_skb(skb, uh->source, uh->dest, udptable); @@ -1014,7 +1014,7 @@ INDIRECT_CALLABLE_SCOPE void udp_v6_early_demux(struct sk_buff *skb) INDIRECT_CALLABLE_SCOPE int udpv6_rcv(struct sk_buff *skb) { - return __udp6_lib_rcv(skb, &udp_table, IPPROTO_UDP); + return __udp6_lib_rcv(skb, &udp_table); } /* diff --git a/net/ipv6/udp_impl.h b/net/ipv6/udp_impl.h index 20e324b6f358..acd5a942c633 100644 --- a/net/ipv6/udp_impl.h +++ b/net/ipv6/udp_impl.h @@ -8,7 +8,7 @@ #include #include -int __udp6_lib_rcv(struct sk_buff *, struct udp_table *, int); +int __udp6_lib_rcv(struct sk_buff *, struct udp_table *); int __udp6_lib_err(struct sk_buff *, struct inet6_skb_parm *, u8, u8, int, __be32, struct udp_table *); diff --git a/net/ipv6/udplite.c b/net/ipv6/udplite.c index bf7a7acd39b1..f442ed595e6f 100644 --- a/net/ipv6/udplite.c +++ b/net/ipv6/udplite.c @@ -14,7 +14,7 @@ static int udplitev6_rcv(struct sk_buff *skb) { - return __udp6_lib_rcv(skb, &udplite_table, IPPROTO_UDPLITE); + return __udp6_lib_rcv(skb, &udplite_table); } static int udplitev6_err(struct sk_buff *skb, From patchwork Mon May 11 18:52:10 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219452 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80814C54E4B for ; Mon, 11 May 2020 18:52:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 638C4206DB for ; Mon, 11 May 2020 18:52:40 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="Pum5gsnv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731245AbgEKSwj (ORCPT ); Mon, 11 May 2020 14:52:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52370 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731240AbgEKSwg (ORCPT ); Mon, 11 May 2020 14:52:36 -0400 Received: from mail-wr1-x442.google.com (mail-wr1-x442.google.com [IPv6:2a00:1450:4864:20::442]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6C8C8C05BD0C for ; Mon, 11 May 2020 11:52:36 -0700 (PDT) Received: by mail-wr1-x442.google.com with SMTP id l11so6399271wru.0 for ; Mon, 11 May 2020 11:52:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fqZSv5iRURJG8P51i+cIZ/1WA6SqrjwO2OnTuk3Zhrk=; b=Pum5gsnvccdNS1zTSXxzuZDTRth+mciNw2wr7ab+clOGUr6iqIq8+sFuHIBheS118W vqhlml+Z4u2lsT/BWExtWn0OZFdkR/q86ksb94R1OVqCziQoaC7jnFLNsYqM8Uww1RGj qcEJEcdlmzH4lMYe/srGAyq3SwLQmXckRybco= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fqZSv5iRURJG8P51i+cIZ/1WA6SqrjwO2OnTuk3Zhrk=; b=i7jm+DckuahqoxcPsfbiRLbd9ckH8nm4GM1TzVG371oXmshp/wTDRuGsrc2nbsZAmW dUxh5ud4dflBL4mz5YPT0X+eKm9S18wOj+VGfwNBo9pzZgS/zfGlzQHsti708ieP89De qK+ZCfTmxN7dyYy5RDWyZqKWO217iWFqIZM5/x0YEcb41+2TrkFd+xAZH9Y4dsJzhwXw w+aANcgKO18TihdvvHOFgMohOWPVcfDI2drabHoSa2txyeX8ReL+94A7AajWIxxJZ4hu kSDDQddmfAPZnafnQlxM34UsYcsb/8+5b9W6rq2DBvDZLUPtdDWl1VuzLkP6XLBF3I8+ kJ6w== X-Gm-Message-State: AGi0PuaRhWt8+uHZPD7K5xlp0nzQIgkqcoxW3Sek63eIsxhKtLyMOOJ4 uQNYPYjIZj7gyFqQc83BskAaUajGSZA= X-Google-Smtp-Source: APiQypKn+F+frbKABB4Fsjd2F19jXvMI8GXONGm3qWXkgo+gz3IJu5XwRyWEItdP1i4KXg4uDjfUgw== X-Received: by 2002:adf:80ee:: with SMTP id 101mr21084470wrl.156.1589223154745; Mon, 11 May 2020 11:52:34 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id q14sm19682582wrc.66.2020.05.11.11.52.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:34 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau Subject: [PATCH bpf-next v2 09/17] udp: Extract helper for selecting socket from reuseport group Date: Mon, 11 May 2020 20:52:10 +0200 Message-Id: <20200511185218.1422406-10-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Prepare for calling into reuseport from __udp4_lib_lookup as well. Signed-off-by: Jakub Sitnicki --- net/ipv4/udp.c | 34 ++++++++++++++++++++++++---------- 1 file changed, 24 insertions(+), 10 deletions(-) diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c index ce96b1746ddf..d4842f29294a 100644 --- a/net/ipv4/udp.c +++ b/net/ipv4/udp.c @@ -405,6 +405,25 @@ static u32 udp_ehashfn(const struct net *net, const __be32 laddr, udp_ehash_secret + net_hash_mix(net)); } +static inline struct sock *lookup_reuseport(struct net *net, struct sock *sk, + struct sk_buff *skb, + __be32 saddr, __be16 sport, + __be32 daddr, unsigned short hnum) +{ + struct sock *reuse_sk = NULL; + u32 hash; + + if (sk->sk_reuseport && sk->sk_state != TCP_ESTABLISHED) { + hash = udp_ehashfn(net, daddr, hnum, saddr, sport); + reuse_sk = reuseport_select_sock(sk, hash, skb, + sizeof(struct udphdr)); + /* Fall back to scoring if group has connections */ + if (reuseport_has_conns(sk, false)) + return NULL; + } + return reuse_sk; +} + /* called with rcu_read_lock() */ static struct sock *udp4_lib_lookup2(struct net *net, __be32 saddr, __be16 sport, @@ -415,7 +434,6 @@ static struct sock *udp4_lib_lookup2(struct net *net, { struct sock *sk, *result; int score, badness; - u32 hash = 0; result = NULL; badness = 0; @@ -423,15 +441,11 @@ static struct sock *udp4_lib_lookup2(struct net *net, score = compute_score(sk, net, saddr, sport, daddr, hnum, dif, sdif); if (score > badness) { - if (sk->sk_reuseport && - sk->sk_state != TCP_ESTABLISHED) { - hash = udp_ehashfn(net, daddr, hnum, - saddr, sport); - result = reuseport_select_sock(sk, hash, skb, - sizeof(struct udphdr)); - if (result && !reuseport_has_conns(sk, false)) - return result; - } + result = lookup_reuseport(net, sk, skb, + saddr, sport, daddr, hnum); + if (result) + return result; + badness = score; result = sk; } From patchwork Mon May 11 18:52:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219450 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBCCEC54E8D for ; Mon, 11 May 2020 18:52:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9B6FA20720 for ; Mon, 11 May 2020 18:52:48 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="rn9QDejR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731254AbgEKSwq (ORCPT ); Mon, 11 May 2020 14:52:46 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52366 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731261AbgEKSwm (ORCPT ); Mon, 11 May 2020 14:52:42 -0400 Received: from mail-wr1-x443.google.com (mail-wr1-x443.google.com [IPv6:2a00:1450:4864:20::443]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5ADCFC05BD10 for ; Mon, 11 May 2020 11:52:42 -0700 (PDT) Received: by mail-wr1-x443.google.com with SMTP id h17so3422827wrc.8 for ; Mon, 11 May 2020 11:52:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iEWx9NWx+iVvT9c6guWm8v93q5pJBOFsw45aCQnsS2o=; b=rn9QDejRxWjnube88IJ1I3sbEoNxPlyuucXwVnCkOZgg9RkXLuvr7TGmSgEHVKuANK 6Y7JHBUdhy0+RH/UGWesc5+Xrram+wNJ317SJpSKLoPWDyntoGw1QIwSMDaZis35Koyw AO+/jt8kRwv4jYC/msXrrh7EZipfhyBnubOLQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iEWx9NWx+iVvT9c6guWm8v93q5pJBOFsw45aCQnsS2o=; b=mCx2lEwl4rJ/jwStW6ZWS+f6zspHe6AbWO3s0KPYxrhW+H+8JLkUxlriHuy01MQGgs 1lAv4CqAI49JBxPh269PfxYZoC2njIfvwshsxXnv4GUzYtDvGdDxanfda+Uh2hl3cHY8 yHZ+xvm1pkmw8zKfp0/UeQr1CrH/mTC+6kMIqR2fsrjBZ0aUGltp5hoD2D7SxtuFzayO KpiwghsHfbNC4ndoUjs0FY4tssmQZ4VORgv1mfw0AQ06Go5yXqkXKgeHOtpBntT+roVA siVCLjP1q1g9j+9YHhnhrRfttrlOc6V6X8Fnzgtv4TZdFLS1v7M+B6y4KEjr+CS2nHEe mhXA== X-Gm-Message-State: AGi0PubT7qzd5DgdX6rhPW3slf+3dfm8MHbPZYOamps6xh8IEBZ20bbk 5c2tgxZICJYLbcQfgeHIIJZP4hPhisE= X-Google-Smtp-Source: APiQypJ73bJM6PxpUfbG6vXTcl6G+xrXXy2TvCHqmviDyjTl9bjH8b6e9lMIZDGJyzVjqqHwrXyYpg== X-Received: by 2002:a5d:42c9:: with SMTP id t9mr21231809wrr.246.1589223160772; Mon, 11 May 2020 11:52:40 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id 88sm19716885wrq.77.2020.05.11.11.52.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:40 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau , Lorenz Bauer Subject: [PATCH bpf-next v2 13/17] bpf: Sync linux/bpf.h to tools/ Date: Mon, 11 May 2020 20:52:14 +0200 Message-Id: <20200511185218.1422406-14-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Newly added program, context type and helper is used by tests in a subsequent patch. Synchronize the header file. Reviewed-by: Lorenz Bauer Signed-off-by: Jakub Sitnicki --- Notes: v2: - Update after changes to bpf.h in earlier patch. tools/include/uapi/linux/bpf.h | 52 ++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) diff --git a/tools/include/uapi/linux/bpf.h b/tools/include/uapi/linux/bpf.h index 9d1932e23cec..03edf4ec7b7e 100644 --- a/tools/include/uapi/linux/bpf.h +++ b/tools/include/uapi/linux/bpf.h @@ -188,6 +188,7 @@ enum bpf_prog_type { BPF_PROG_TYPE_STRUCT_OPS, BPF_PROG_TYPE_EXT, BPF_PROG_TYPE_LSM, + BPF_PROG_TYPE_SK_LOOKUP, }; enum bpf_attach_type { @@ -220,6 +221,7 @@ enum bpf_attach_type { BPF_MODIFY_RETURN, BPF_LSM_MAC, BPF_TRACE_ITER, + BPF_SK_LOOKUP, __MAX_BPF_ATTACH_TYPE }; @@ -3050,6 +3052,10 @@ union bpf_attr { * * int bpf_sk_assign(struct sk_buff *skb, struct bpf_sock *sk, u64 flags) * Description + * Helper is overloaded depending on BPF program type. This + * description applies to **BPF_PROG_TYPE_SCHED_CLS** and + * **BPF_PROG_TYPE_SCHED_ACT** programs. + * * Assign the *sk* to the *skb*. When combined with appropriate * routing configuration to receive the packet towards the socket, * will cause *skb* to be delivered to the specified socket. @@ -3070,6 +3076,38 @@ union bpf_attr { * call from outside of TC ingress. * * **-ESOCKTNOSUPPORT** Socket type not supported (reuseport). * + * int bpf_sk_assign(struct bpf_sk_lookup *ctx, struct bpf_sock *sk, u64 flags) + * Description + * Helper is overloaded depending on BPF program type. This + * description applies to **BPF_PROG_TYPE_SK_LOOKUP** programs. + * + * Select the *sk* as a result of a socket lookup. + * + * For the operation to succeed passed socket must be compatible + * with the packet description provided by the *ctx* object. + * + * L4 protocol (*IPPROTO_TCP* or *IPPROTO_UDP*) must be an exact + * match. While IP family (*AF_INET* or *AF_INET6*) must be + * compatible, that is IPv6 sockets that are not v6-only can be + * selected for IPv4 packets. + * + * Only TCP listeners and UDP sockets, that is sockets which have + * *SOCK_RCU_FREE* flag set, can be selected. + * + * The *flags* argument must be zero. + * Return + * 0 on success, or a negative errno in case of failure. + * + * **-EAFNOSUPPORT** is socket family (*sk->family*) is not + * compatible with packet family (*ctx->family*). + * + * **-EINVAL** if unsupported flags were specified. + * + * **-EPROTOTYPE** if socket L4 protocol (*sk->protocol*) doesn't + * match packet protocol (*ctx->protocol*). + * + * **-ESOCKTNOSUPPORT** if socket does not use RCU freeing. + * * u64 bpf_ktime_get_boot_ns(void) * Description * Return the time elapsed since system boot, in nanoseconds. @@ -4058,4 +4096,18 @@ struct bpf_pidns_info { __u32 pid; __u32 tgid; }; + +/* User accessible data for SK_LOOKUP programs. Add new fields at the end. */ +struct bpf_sk_lookup { + __u32 family; /* Protocol family (AF_INET, AF_INET6) */ + __u32 protocol; /* IP protocol (IPPROTO_TCP, IPPROTO_UDP) */ + /* IP addresses allow 1,2,4-byte read and are in network byte order. */ + __u32 remote_ip4; + __u32 remote_ip6[4]; + __u32 remote_port; /* network byte order */ + __u32 local_ip4; + __u32 local_ip6[4]; + __u32 local_port; /* host byte order */ +}; + #endif /* _UAPI__LINUX_BPF_H__ */ From patchwork Mon May 11 18:52:16 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.2 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI,SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS, UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ED1AC47255 for ; Mon, 11 May 2020 18:52:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 3CCE2206DB for ; Mon, 11 May 2020 18:52:51 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="JJ7RBd3Y" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731286AbgEKSwt (ORCPT ); Mon, 11 May 2020 14:52:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52352 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731273AbgEKSwq (ORCPT ); Mon, 11 May 2020 14:52:46 -0400 Received: from mail-wr1-x441.google.com (mail-wr1-x441.google.com [IPv6:2a00:1450:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ACFA6C061A0C for ; Mon, 11 May 2020 11:52:45 -0700 (PDT) Received: by mail-wr1-x441.google.com with SMTP id k1so12327509wrx.4 for ; Mon, 11 May 2020 11:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DMqKMKWuzCcR6FxP9hi6qzxvVY3wjBoadkJ8o2Mh5D0=; b=JJ7RBd3YRIwSXmyf5NPu//CnAnU+v4tsq2Btmu/NTeC81ikDA8I9wPfKjWSeUe6jm8 aqxRm1wU3VDslngOuNnoizn6g8CnmaBQmqyq8O5RpodbpwaEuRInIWINHDWNkbhf/mT/ lLtxPlZfJsLpJh6jeM4C49jrOJ46iSLLjnm2g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DMqKMKWuzCcR6FxP9hi6qzxvVY3wjBoadkJ8o2Mh5D0=; b=aG8L+qKby4yMISoT1oT29Uc3toqckSFD89qZ8xsQ6B0Voe0HQ63Q4umZvdBRFCYHGZ z1Yh48RbLBzC9XeD6f7JZ8Y7c+L//NIBYiVNcntg/1w1QPGMIGrP5SsvOmHZVy9z2iko 358jDnx1KgKi11qUS8jaQJi1+LndwgfI9ccmR3DNgEsnhpfvyplKslV1IYqRQQJ/kf6L NejBTJ+H2bbdr5iRHAz7T4Q/ReMJhdrqv3xES56CtlXf5kwv7W+3zeLrjuvgKhHMDe07 Whfakgdu1NPFdT4fDILzYY+4dw7jYImjC/uozoFFmh5wSGKHCK679rupJ/r/gOY6XdGd MMYQ== X-Gm-Message-State: AGi0PubI5P8RAUFjwYs7miWC9mvDqpZ1cb5sjROuh0/OQzaJdtcy4IBT UbHZDidlhaNDOdmJSyTLQd9yEpye9Zg= X-Google-Smtp-Source: APiQypLCY6N3PM2wsnBC3way/PtTOY12KkcMxt0ZEytY46n8Gass3dyNRQCQPGuQZvfwdqopEEB8wg== X-Received: by 2002:a5d:4e0a:: with SMTP id p10mr20116949wrt.215.1589223163698; Mon, 11 May 2020 11:52:43 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id y10sm15892107wrd.95.2020.05.11.11.52.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:43 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau Subject: [PATCH bpf-next v2 15/17] selftests/bpf: Add verifier tests for bpf_sk_lookup context access Date: Mon, 11 May 2020 20:52:16 +0200 Message-Id: <20200511185218.1422406-16-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Exercise verifier access checks for bpf_sk_lookup context fields. Signed-off-by: Jakub Sitnicki --- Notes: v2: - Adjust for fields renames in struct bpf_sk_lookup. .../selftests/bpf/verifier/ctx_sk_lookup.c | 694 ++++++++++++++++++ 1 file changed, 694 insertions(+) create mode 100644 tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c diff --git a/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c b/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c new file mode 100644 index 000000000000..223163172fa9 --- /dev/null +++ b/tools/testing/selftests/bpf/verifier/ctx_sk_lookup.c @@ -0,0 +1,694 @@ +{ + "valid 1,2,4-byte read bpf_sk_lookup remote_ip4", + .insns = { + /* 4-byte read */ + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4)), + /* 2-byte read */ + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4) + 2), + /* 1-byte read */ + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4) + 3), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup remote_ip4", + .insns = { + /* 8-byte read */ + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup remote_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + /* 4-byte write */ + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup remote_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + /* 4-byte write */ + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup remote_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + /* 2-byte write */ + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup remote_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + /* 1-byte write */ + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 1,2,4-byte read bpf_sk_lookup local_ip4", + .insns = { + /* 4-byte read */ + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4)), + /* 2-byte read */ + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4) + 2), + /* 1-byte read */ + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4) + 3), + BPF_MOV64_IMM(BPF_REG_0, 0), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup local_ip4", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup local_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup local_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup local_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup local_ip4", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x7f000001U), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip4)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 1,2,4-byte read bpf_sk_lookup remote_ip6", + .insns = { + /* 4-byte read */ + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[3])), + /* 2-byte read */ + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[3]) + 2), + /* 1-byte read */ + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[3]) + 3), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup remote_ip6", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup remote_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup remote_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup remote_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup remote_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 1,2,4-byte read bpf_sk_lookup local_ip6", + .insns = { + /* 4-byte read */ + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[3])), + /* 2-byte read */ + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[3]) + 2), + /* 1-byte read */ + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[3]) + 3), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup local_ip6", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup local_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup local_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup local_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup local_ip6", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 0x00000001U), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_ip6[0])), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 4-byte read bpf_sk_lookup remote_port", + .insns = { + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup remote_port", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte read bpf_sk_lookup remote_port", + .insns = { + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte read bpf_sk_lookup remote_port", + .insns = { + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup remote_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup remote_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup remote_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup remote_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, remote_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 4-byte read bpf_sk_lookup local_port", + .insns = { + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup local_port", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte read bpf_sk_lookup local_port", + .insns = { + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte read bpf_sk_lookup local_port", + .insns = { + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup local_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup local_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup local_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup local_port", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, local_port)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 4-byte read bpf_sk_lookup family", + .insns = { + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup family", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte read bpf_sk_lookup family", + .insns = { + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte read bpf_sk_lookup family", + .insns = { + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup family", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup family", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup family", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup family", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, family)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "valid 4-byte read bpf_sk_lookup protocol", + .insns = { + BPF_LDX_MEM(BPF_W, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .result = ACCEPT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte read bpf_sk_lookup protocol", + .insns = { + BPF_LDX_MEM(BPF_DW, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte read bpf_sk_lookup protocol", + .insns = { + BPF_LDX_MEM(BPF_H, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte read bpf_sk_lookup protocol", + .insns = { + BPF_LDX_MEM(BPF_B, BPF_REG_0, BPF_REG_1, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 8-byte write bpf_sk_lookup protocol", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_DW, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 4-byte write bpf_sk_lookup protocol", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_W, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 2-byte write bpf_sk_lookup protocol", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_H, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, +{ + "invalid 1-byte write bpf_sk_lookup protocol", + .insns = { + BPF_MOV64_IMM(BPF_REG_0, 1234), + BPF_STX_MEM(BPF_B, BPF_REG_1, BPF_REG_0, + offsetof(struct bpf_sk_lookup, protocol)), + BPF_EXIT_INSN(), + }, + .errstr = "invalid bpf_context access", + .result = REJECT, + .prog_type = BPF_PROG_TYPE_SK_LOOKUP, +}, From patchwork Mon May 11 18:52:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jakub Sitnicki X-Patchwork-Id: 219448 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.0 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 405BFC54E4B for ; Mon, 11 May 2020 18:52:57 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 10FA4206DB for ; Mon, 11 May 2020 18:52:57 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=cloudflare.com header.i=@cloudflare.com header.b="sYU4PWUj" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1731284AbgEKSww (ORCPT ); Mon, 11 May 2020 14:52:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52422 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1731273AbgEKSwu (ORCPT ); Mon, 11 May 2020 14:52:50 -0400 Received: from mail-wm1-x344.google.com (mail-wm1-x344.google.com [IPv6:2a00:1450:4864:20::344]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2F6C7C05BD0C for ; Mon, 11 May 2020 11:52:49 -0700 (PDT) Received: by mail-wm1-x344.google.com with SMTP id u16so20530208wmc.5 for ; Mon, 11 May 2020 11:52:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cloudflare.com; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ze/h/CNZ/91B28Ipr0bBfUZchvfb9nsJ7KusesHA9+8=; b=sYU4PWUj/xVA9pUSitEbqXibw2TD+mMqSKc1kfkitZAnyYlZ9GChb7zJ1okvNzTrNZ RUT6AK20LeyGZBqQNy4qUBbvGncNp7dCXLySELi1zY7U5TYfrr+Z0XiZqwWJKGX0NbNv IPb/OqOGnvV3IXFzyz8bArdbTZaFdDFG7s3eM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ze/h/CNZ/91B28Ipr0bBfUZchvfb9nsJ7KusesHA9+8=; b=VoNR+f5YkTkDQ+GL2J4Es7pfr4hSHJU0kZGBtur/Al39mgm/lECm7wNwK3u9bx8a/P rgeX4okhi6UjFNE2kLfZPJgzXoFQC1rEgx0uzEIieVRDxSF094pXQ0WxVHwmVAGOI9Lg WhvS0blVHhwdRQzoGPpQn9nxl7VJyVNK8eDfwUGqt1U5cK2iZkLHUCct2GquKDJlSepT OPZdvQft4R94hrBMeWZcfqgHvIPHJ1blVB1FNLzQrygekxqq6gvr6qukyxY6f/MizsSH qEphlNWo805IhbtaW7M6Xmki1yvsDZgh9QIaMIIxs6JLpo+j9Vxk/a3wDmaJLwTiVCId UV0w== X-Gm-Message-State: AGi0PuaygRtsv3Y4ovR8jFfZl2Yw45DTkOS+ovQGiCvwDDHWqzSqNsH1 RrHccSv4gA3A1PpipGCHdRILo/HCtf8= X-Google-Smtp-Source: APiQypI9RoFMZOoV3cnZFuQqxWCCY8o/0MmyVLJgIyn+Wz5pmYK2zSUXVKmpjsFVNq51VcWjQuoSeg== X-Received: by 2002:a1c:7415:: with SMTP id p21mr32405160wmc.93.1589223166721; Mon, 11 May 2020 11:52:46 -0700 (PDT) Received: from cloudflare.com ([2a02:a310:c262:aa00:b35e:8938:2c2a:ba8b]) by smtp.gmail.com with ESMTPSA id n6sm16988638wrt.58.2020.05.11.11.52.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 May 2020 11:52:46 -0700 (PDT) From: Jakub Sitnicki To: netdev@vger.kernel.org, bpf@vger.kernel.org Cc: dccp@vger.kernel.org, kernel-team@cloudflare.com, Alexei Starovoitov , Daniel Borkmann , "David S. Miller" , Eric Dumazet , Gerrit Renker , Jakub Kicinski , Andrii Nakryiko , Martin KaFai Lau Subject: [PATCH bpf-next v2 17/17] selftests/bpf: Tests for BPF_SK_LOOKUP attach point Date: Mon, 11 May 2020 20:52:18 +0200 Message-Id: <20200511185218.1422406-18-jakub@cloudflare.com> X-Mailer: git-send-email 2.25.3 In-Reply-To: <20200511185218.1422406-1-jakub@cloudflare.com> References: <20200511185218.1422406-1-jakub@cloudflare.com> MIME-Version: 1.0 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Add tests to test_progs that exercise: - attaching/detaching/querying sk_lookup program, - overriding socket lookup result for TCP/UDP with BPF sk_lookup by a) selecting a socket fetched from a SOCKMAP, or b) failing the lookup with no match. Tests cover two special cases: - selecting an IPv6 socket (non v6-only) to receive an IPv4 packet, - using BPF sk_lookup together with BPF sk_reuseport program. Signed-off-by: Jakub Sitnicki --- Notes: v2: - Adjust for fields renames in struct bpf_sk_lookup. .../selftests/bpf/prog_tests/sk_lookup.c | 999 ++++++++++++++++++ .../selftests/bpf/progs/test_sk_lookup_kern.c | 162 +++ 2 files changed, 1161 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/sk_lookup.c create mode 100644 tools/testing/selftests/bpf/progs/test_sk_lookup_kern.c diff --git a/tools/testing/selftests/bpf/prog_tests/sk_lookup.c b/tools/testing/selftests/bpf/prog_tests/sk_lookup.c new file mode 100644 index 000000000000..96765b156f6f --- /dev/null +++ b/tools/testing/selftests/bpf/prog_tests/sk_lookup.c @@ -0,0 +1,999 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +// Copyright (c) 2020 Cloudflare +/* + * Test BPF attach point for INET socket lookup (BPF_SK_LOOKUP). + * + * Tests exercise: + * + * 1. attaching/detaching/querying BPF sk_lookup program, + * 2. overriding socket lookup result by: + * a) selecting a listening (TCP) or receiving (UDP) socket, + * b) failing the lookup with no match. + * + * Special cases covered are: + * - selecting an IPv6 socket (non v6-only) to receive an IPv4 packet, + * - using BPF sk_lookup together with BPF sk_reuseport program. + * + * Tests run in a dedicated network namespace. + */ + +#define _GNU_SOURCE +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#include "bpf_rlimit.h" +#include "bpf_util.h" +#include "cgroup_helpers.h" +#include "test_sk_lookup_kern.skel.h" +#include "test_progs.h" + +/* External (address, port) pairs the client sends packets to. */ +#define EXT_IP4 "127.0.0.1" +#define EXT_IP6 "fd00::1" +#define EXT_PORT 7007 + +/* Internal (address, port) pairs the server listens/receives at. */ +#define INT_IP4 "127.0.0.2" +#define INT_IP4_V6 "::ffff:127.0.0.2" +#define INT_IP6 "fd00::2" +#define INT_PORT 8008 + +#define IO_TIMEOUT_SEC 3 + +enum { + SERVER_A = 0, + SERVER_B = 1, + MAX_SERVERS, +}; + +struct inet_addr { + const char *ip; + unsigned short port; +}; + +struct test { + const char *desc; + struct bpf_program *lookup_prog; + struct bpf_program *reuseport_prog; + struct bpf_map *sock_map; + int sotype; + struct inet_addr send_to; + struct inet_addr recv_at; +}; + +static bool is_ipv6(const char *ip) +{ + return !!strchr(ip, ':'); +} + +static int make_addr(const char *ip, int port, struct sockaddr_storage *addr) +{ + struct sockaddr_in6 *addr6 = (void *)addr; + struct sockaddr_in *addr4 = (void *)addr; + int ret; + + errno = 0; + if (is_ipv6(ip)) { + ret = inet_pton(AF_INET6, ip, &addr6->sin6_addr); + if (CHECK_FAIL(ret <= 0)) { + log_err("failed to convert IPv6 address '%s'", ip); + return -1; + } + addr6->sin6_family = AF_INET6; + addr6->sin6_port = htons(port); + } else { + ret = inet_pton(AF_INET, ip, &addr4->sin_addr); + if (CHECK_FAIL(ret <= 0)) { + log_err("failed to convert IPv4 address '%s'", ip); + return -1; + } + addr4->sin_family = AF_INET; + addr4->sin_port = htons(port); + } + return 0; +} + +static int setup_reuseport_prog(int sock_fd, struct bpf_program *reuseport_prog) +{ + int err, prog_fd; + + prog_fd = bpf_program__fd(reuseport_prog); + if (prog_fd < 0) { + errno = -prog_fd; + log_err("failed to get fd for program '%s'", + bpf_program__name(reuseport_prog)); + return -1; + } + + err = setsockopt(sock_fd, SOL_SOCKET, SO_ATTACH_REUSEPORT_EBPF, + &prog_fd, sizeof(prog_fd)); + if (CHECK_FAIL(err)) { + log_err("failed to ATTACH_REUSEPORT_EBPF"); + return -1; + } + + return 0; +} + +static socklen_t inetaddr_len(const struct sockaddr_storage *addr) +{ + return (addr->ss_family == AF_INET ? sizeof(struct sockaddr_in) : + addr->ss_family == AF_INET6 ? sizeof(struct sockaddr_in6) : 0); +} + +static int make_socket_with_addr(int sotype, const char *ip, int port, + struct sockaddr_storage *addr) +{ + struct timeval timeo = { .tv_sec = IO_TIMEOUT_SEC }; + int err, fd; + + err = make_addr(ip, port, addr); + if (err) + return -1; + + fd = socket(addr->ss_family, sotype, 0); + if (CHECK_FAIL(fd < 0)) { + log_err("failed to create listen socket"); + return -1; + } + + err = setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &timeo, sizeof(timeo)); + if (CHECK_FAIL(err)) { + log_err("failed to set SO_SNDTIMEO"); + return -1; + } + + err = setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeo, sizeof(timeo)); + if (CHECK_FAIL(err)) { + log_err("failed to set SO_RCVTIMEO"); + return -1; + } + + return fd; +} + +static int make_server(int sotype, const char *ip, int port, + struct bpf_program *reuseport_prog) +{ + struct sockaddr_storage addr = {0}; + const int one = 1; + int err, fd = -1; + + fd = make_socket_with_addr(sotype, ip, port, &addr); + if (fd < 0) + return -1; + + /* Enabled for UDPv6 sockets for IPv4-mapped IPv6 to work. */ + if (sotype == SOCK_DGRAM) { + err = setsockopt(fd, SOL_IP, IP_RECVORIGDSTADDR, &one, + sizeof(one)); + if (CHECK_FAIL(err)) { + log_err("failed to enable IP_RECVORIGDSTADDR"); + goto fail; + } + } + + if (sotype == SOCK_DGRAM && addr.ss_family == AF_INET6) { + err = setsockopt(fd, SOL_IPV6, IPV6_RECVORIGDSTADDR, &one, + sizeof(one)); + if (CHECK_FAIL(err)) { + log_err("failed to enable IPV6_RECVORIGDSTADDR"); + goto fail; + } + } + + if (sotype == SOCK_STREAM) { + err = setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &one, + sizeof(one)); + if (CHECK_FAIL(err)) { + log_err("failed to enable SO_REUSEADDR"); + goto fail; + } + } + + if (reuseport_prog) { + err = setsockopt(fd, SOL_SOCKET, SO_REUSEPORT, &one, + sizeof(one)); + if (CHECK_FAIL(err)) { + log_err("failed to enable SO_REUSEPORT"); + goto fail; + } + } + + err = bind(fd, (void *)&addr, inetaddr_len(&addr)); + if (CHECK_FAIL(err)) { + log_err("failed to bind listen socket"); + goto fail; + } + + if (sotype == SOCK_STREAM) { + err = listen(fd, SOMAXCONN); + if (CHECK_FAIL(err)) { + log_err("failed to listen on port %d", port); + goto fail; + } + } + + /* Late attach reuseport prog so we can have one init path */ + if (reuseport_prog) { + err = setup_reuseport_prog(fd, reuseport_prog); + if (err) + goto fail; + } + + return fd; +fail: + close(fd); + return -1; +} + +static int make_client(int sotype, const char *ip, int port) +{ + struct sockaddr_storage addr = {0}; + int err, fd; + + fd = make_socket_with_addr(sotype, ip, port, &addr); + if (fd < 0) + return -1; + + err = connect(fd, (void *)&addr, inetaddr_len(&addr)); + if (CHECK_FAIL(err)) { + log_err("failed to connect client socket"); + goto fail; + } + + return fd; +fail: + close(fd); + return -1; +} + +static int send_byte(int fd) +{ + ssize_t n; + + errno = 0; + n = send(fd, "a", 1, 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed/partial send"); + return -1; + } + return 0; +} + +static int recv_byte(int fd) +{ + char buf[1]; + ssize_t n; + + n = recv(fd, buf, sizeof(buf), 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed/partial recv"); + return -1; + } + return 0; +} + +static int tcp_recv_send(int server_fd) +{ + char buf[1]; + int ret, fd; + ssize_t n; + + fd = accept(server_fd, NULL, NULL); + if (CHECK_FAIL(fd < 0)) { + log_err("failed to accept"); + return -1; + } + + n = recv(fd, buf, sizeof(buf), 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed/partial recv"); + ret = -1; + goto close; + } + + n = send(fd, buf, n, 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed/partial send"); + ret = -1; + goto close; + } + + ret = 0; +close: + close(fd); + return ret; +} + +static void v4_to_v6(struct sockaddr_storage *ss) +{ + struct sockaddr_in6 *v6 = (struct sockaddr_in6 *)ss; + struct sockaddr_in v4 = *(struct sockaddr_in *)ss; + + v6->sin6_family = AF_INET6; + v6->sin6_port = v4.sin_port; + v6->sin6_addr.s6_addr[10] = 0xff; + v6->sin6_addr.s6_addr[11] = 0xff; + memcpy(&v6->sin6_addr.s6_addr[12], &v4.sin_addr.s_addr, 4); +} + +static int udp_recv_send(int server_fd) +{ + char cmsg_buf[CMSG_SPACE(sizeof(struct sockaddr_storage))]; + struct sockaddr_storage _src_addr = { 0 }; + struct sockaddr_storage *src_addr = &_src_addr; + struct sockaddr_storage *dst_addr = NULL; + struct msghdr msg = { 0 }; + struct iovec iov = { 0 }; + struct cmsghdr *cm; + char buf[1]; + int ret, fd; + ssize_t n; + + iov.iov_base = buf; + iov.iov_len = sizeof(buf); + + msg.msg_name = src_addr; + msg.msg_namelen = sizeof(*src_addr); + msg.msg_iov = &iov; + msg.msg_iovlen = 1; + msg.msg_control = cmsg_buf; + msg.msg_controllen = sizeof(cmsg_buf); + + errno = 0; + n = recvmsg(server_fd, &msg, 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed to receive"); + return -1; + } + if (CHECK_FAIL(msg.msg_flags & MSG_CTRUNC)) { + log_err("truncated cmsg"); + return -1; + } + + for (cm = CMSG_FIRSTHDR(&msg); cm; cm = CMSG_NXTHDR(&msg, cm)) { + if ((cm->cmsg_level == SOL_IP && + cm->cmsg_type == IP_ORIGDSTADDR) || + (cm->cmsg_level == SOL_IPV6 && + cm->cmsg_type == IPV6_ORIGDSTADDR)) { + dst_addr = (struct sockaddr_storage *)CMSG_DATA(cm); + break; + } + log_err("warning: ignored cmsg at level %d type %d", + cm->cmsg_level, cm->cmsg_type); + } + if (CHECK_FAIL(!dst_addr)) { + log_err("failed to get destination address"); + return -1; + } + + /* Server socket bound to IPv4-mapped IPv6 address */ + if (src_addr->ss_family == AF_INET6 && + dst_addr->ss_family == AF_INET) { + v4_to_v6(dst_addr); + } + + /* Reply from original destination address. */ + fd = socket(dst_addr->ss_family, SOCK_DGRAM, 0); + if (CHECK_FAIL(fd < 0)) { + log_err("failed to create tx socket"); + return -1; + } + + ret = bind(fd, (struct sockaddr *)dst_addr, sizeof(*dst_addr)); + if (CHECK_FAIL(ret)) { + log_err("failed to bind tx socket"); + goto out; + } + + msg.msg_control = NULL; + msg.msg_controllen = 0; + n = sendmsg(fd, &msg, 0); + if (CHECK_FAIL(n <= 0)) { + log_err("failed to send echo reply"); + ret = -1; + goto out; + } + + ret = 0; +out: + close(fd); + return ret; +} + +static int tcp_echo_test(int client_fd, int server_fd) +{ + int err; + + err = send_byte(client_fd); + if (err) + return -1; + err = tcp_recv_send(server_fd); + if (err) + return -1; + err = recv_byte(client_fd); + if (err) + return -1; + + return 0; +} + +static int udp_echo_test(int client_fd, int server_fd) +{ + int err; + + err = send_byte(client_fd); + if (err) + return -1; + err = udp_recv_send(server_fd); + if (err) + return -1; + err = recv_byte(client_fd); + if (err) + return -1; + + return 0; +} + +static int attach_lookup_prog(struct bpf_program *prog) +{ + const char *prog_name = bpf_program__name(prog); + enum bpf_attach_type attach_type; + int err, prog_fd; + + prog_fd = bpf_program__fd(prog); + if (CHECK_FAIL(prog_fd < 0)) { + errno = -prog_fd; + log_err("failed to get fd for program '%s'", prog_name); + return -1; + } + + attach_type = bpf_program__get_expected_attach_type(prog); + err = bpf_prog_attach(prog_fd, -1 /* target fd */, attach_type, 0); + if (CHECK_FAIL(err)) { + log_err("failed to attach program '%s'", prog_name); + return -1; + } + + return 0; +} + +static int detach_lookup_prog(struct bpf_program *prog) +{ + const char *prog_name = bpf_program__name(prog); + enum bpf_attach_type attach_type; + int err, prog_fd; + + prog_fd = bpf_program__fd(prog); + if (CHECK_FAIL(prog_fd < 0)) { + errno = -prog_fd; + log_err("failed to get fd for program '%s'", prog_name); + return -1; + } + + attach_type = bpf_program__get_expected_attach_type(prog); + err = bpf_prog_detach2(prog_fd, -1 /* attachable fd */, attach_type); + if (CHECK_FAIL(err)) { + log_err("failed to detach program '%s'", prog_name); + return -1; + } + + return 0; +} + +static int update_lookup_map(struct bpf_map *map, int index, int sock_fd) +{ + int err, map_fd; + uint64_t value; + + map_fd = bpf_map__fd(map); + if (CHECK_FAIL(map_fd < 0)) { + errno = -map_fd; + log_err("failed to get map FD"); + return -1; + } + + value = (uint64_t)sock_fd; + err = bpf_map_update_elem(map_fd, &index, &value, BPF_NOEXIST); + if (CHECK_FAIL(err)) { + log_err("failed to update redir_map @ %d", index); + return -1; + } + + return 0; +} + +static void query_lookup_prog(struct test_sk_lookup_kern *skel) +{ + struct bpf_program *lookup_prog = skel->progs.lookup_pass; + enum bpf_attach_type attach_type; + __u32 attach_flags = 0; + __u32 prog_ids[1] = { 0 }; + __u32 prog_cnt = 1; + int net_fd = -1; + int err; + + net_fd = open("/proc/self/ns/net", O_RDONLY); + if (CHECK_FAIL(net_fd < 0)) { + log_err("failed to open /proc/self/ns/net"); + return; + } + + err = attach_lookup_prog(lookup_prog); + if (err) + goto close; + + attach_type = bpf_program__get_expected_attach_type(lookup_prog); + err = bpf_prog_query(net_fd, attach_type, 0 /* query flags */, + &attach_flags, prog_ids, &prog_cnt); + if (CHECK_FAIL(err)) { + log_err("failed to query lookup prog"); + goto detach; + } + + errno = 0; + if (CHECK_FAIL(attach_flags != 0)) { + log_err("wrong attach_flags on query: %u", attach_flags); + goto detach; + } + if (CHECK_FAIL(prog_cnt != 1)) { + log_err("wrong program count on query: %u", prog_cnt); + goto detach; + } + if (CHECK_FAIL(prog_ids[0] == 0)) { + log_err("invalid program id on query: %u", prog_ids[0]); + goto detach; + } + +detach: + detach_lookup_prog(lookup_prog); +close: + close(net_fd); +} + +static void run_lookup_prog(const struct test *t) +{ + int client_fd, server_fds[MAX_SERVERS] = { -1 }; + int i, err, server_idx; + + err = attach_lookup_prog(t->lookup_prog); + if (err) + return; + + for (i = 0; i < ARRAY_SIZE(server_fds); i++) { + server_fds[i] = make_server(t->sotype, t->recv_at.ip, + t->recv_at.port, t->reuseport_prog); + if (server_fds[i] < 0) + goto close; + + err = update_lookup_map(t->sock_map, i, server_fds[i]); + if (err) + goto detach; + + /* want just one server for non-reuseport test */ + if (!t->reuseport_prog) + break; + } + + client_fd = make_client(t->sotype, t->send_to.ip, t->send_to.port); + if (client_fd < 0) + goto close; + + /* reuseport prog always selects server B */ + server_idx = t->reuseport_prog ? SERVER_B : SERVER_A; + + if (t->sotype == SOCK_STREAM) + tcp_echo_test(client_fd, server_fds[server_idx]); + else + udp_echo_test(client_fd, server_fds[server_idx]); + + close(client_fd); +close: + for (i = 0; i < ARRAY_SIZE(server_fds); i++) + close(server_fds[i]); +detach: + detach_lookup_prog(t->lookup_prog); +} + +static void test_override_lookup(struct test_sk_lookup_kern *skel) +{ + const struct test tests[] = { + { + .desc = "TCP IPv4 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { EXT_IP4, INT_PORT }, + }, + { + .desc = "TCP IPv4 redir addr", + .lookup_prog = skel->progs.redir_ip4, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, EXT_PORT }, + }, + { + .desc = "TCP IPv4 redir and reuseport", + .lookup_prog = skel->progs.select_sock_a, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, INT_PORT }, + .reuseport_prog = skel->progs.select_sock_b, + }, + { + .desc = "TCP IPv6 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { EXT_IP6, INT_PORT }, + }, + { + .desc = "TCP IPv6 redir addr", + .lookup_prog = skel->progs.redir_ip6, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, EXT_PORT }, + }, + { + .desc = "TCP IPv4->IPv6 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .recv_at = { INT_IP4_V6, INT_PORT }, + .send_to = { EXT_IP4, EXT_PORT }, + }, + { + .desc = "TCP IPv6 redir and reuseport", + .lookup_prog = skel->progs.select_sock_a, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, INT_PORT }, + .reuseport_prog = skel->progs.select_sock_b, + }, + { + .desc = "UDP IPv4 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { EXT_IP4, INT_PORT }, + }, + { + .desc = "UDP IPv4 redir addr", + .lookup_prog = skel->progs.redir_ip4, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, EXT_PORT }, + }, + { + .desc = "UDP IPv4 redir and reuseport", + .lookup_prog = skel->progs.select_sock_a, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, INT_PORT }, + .reuseport_prog = skel->progs.select_sock_b, + }, + { + .desc = "UDP IPv6 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { EXT_IP6, INT_PORT }, + }, + { + .desc = "UDP IPv6 redir addr", + .lookup_prog = skel->progs.redir_ip6, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, EXT_PORT }, + }, + { + .desc = "UDP IPv4->IPv6 redir port", + .lookup_prog = skel->progs.redir_port, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .recv_at = { INT_IP4_V6, INT_PORT }, + .send_to = { EXT_IP4, EXT_PORT }, + }, + { + .desc = "UDP IPv6 redir and reuseport", + .lookup_prog = skel->progs.select_sock_a, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, INT_PORT }, + .reuseport_prog = skel->progs.select_sock_b, + }, + }; + const struct test *t; + + for (t = tests; t < tests + ARRAY_SIZE(tests); t++) { + if (test__start_subtest(t->desc)) + run_lookup_prog(t); + } +} + +static void drop_on_lookup(const struct test *t) +{ + struct sockaddr_storage dst = { 0 }; + int client_fd, server_fd, err; + ssize_t n; + + if (attach_lookup_prog(t->lookup_prog)) + return; + + server_fd = make_server(t->sotype, t->recv_at.ip, t->recv_at.port, + t->reuseport_prog); + if (server_fd < 0) + goto detach; + + client_fd = make_socket_with_addr(t->sotype, t->send_to.ip, + t->send_to.port, &dst); + if (client_fd < 0) + goto close_srv; + + err = connect(client_fd, (void *)&dst, inetaddr_len(&dst)); + if (t->sotype == SOCK_DGRAM) { + err = send_byte(client_fd); + if (err) + goto close_all; + + /* Read out asynchronous error */ + n = recv(client_fd, NULL, 0, 0); + err = n == -1; + } + if (CHECK_FAIL(!err || errno != ECONNREFUSED)) + log_err("expected ECONNREFUSED on connect"); + +close_all: + close(client_fd); +close_srv: + close(server_fd); +detach: + detach_lookup_prog(t->lookup_prog); +} + +static void test_drop_on_lookup(struct test_sk_lookup_kern *skel) +{ + const struct test tests[] = { + { + .desc = "TCP IPv4 drop on lookup", + .lookup_prog = skel->progs.lookup_drop, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { EXT_IP4, EXT_PORT }, + }, + { + .desc = "TCP IPv6 drop on lookup", + .lookup_prog = skel->progs.lookup_drop, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { EXT_IP6, EXT_PORT }, + }, + { + .desc = "UDP IPv4 drop on lookup", + .lookup_prog = skel->progs.lookup_drop, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { EXT_IP4, EXT_PORT }, + }, + { + .desc = "UDP IPv6 drop on lookup", + .lookup_prog = skel->progs.lookup_drop, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { EXT_IP6, INT_PORT }, + }, + }; + const struct test *t; + + for (t = tests; t < tests + ARRAY_SIZE(tests); t++) { + if (test__start_subtest(t->desc)) + drop_on_lookup(t); + } +} + +static void drop_on_reuseport(const struct test *t) +{ + struct sockaddr_storage dst = { 0 }; + int client, server1, server2, err; + ssize_t n; + + if (attach_lookup_prog(t->lookup_prog)) + return; + + server1 = make_server(t->sotype, t->recv_at.ip, t->recv_at.port, + t->reuseport_prog); + if (server1 < 0) + goto detach; + + err = update_lookup_map(t->sock_map, SERVER_A, server1); + if (err) + goto detach; + + /* second server on destination address we should never reach */ + server2 = make_server(t->sotype, t->send_to.ip, t->send_to.port, + NULL /* reuseport prog */); + if (server2 < 0) + goto close_srv1; + + client = make_socket_with_addr(t->sotype, t->send_to.ip, + t->send_to.port, &dst); + if (client < 0) + goto close_srv2; + + err = connect(client, (void *)&dst, inetaddr_len(&dst)); + if (t->sotype == SOCK_DGRAM) { + err = send_byte(client); + if (err) + goto close_all; + + /* Read out asynchronous error */ + n = recv(client, NULL, 0, 0); + err = n == -1; + } + if (CHECK_FAIL(!err || errno != ECONNREFUSED)) + log_err("expected ECONNREFUSED on connect"); + +close_all: + close(client); +close_srv2: + close(server2); +close_srv1: + close(server1); +detach: + detach_lookup_prog(t->lookup_prog); +} + +static void test_drop_on_reuseport(struct test_sk_lookup_kern *skel) +{ + const struct test tests[] = { + { + .desc = "TCP IPv4 drop on reuseport", + .lookup_prog = skel->progs.select_sock_a, + .reuseport_prog = skel->progs.reuseport_drop, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, INT_PORT }, + }, + { + .desc = "TCP IPv6 drop on reuseport", + .lookup_prog = skel->progs.select_sock_a, + .reuseport_prog = skel->progs.reuseport_drop, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, INT_PORT }, + }, + { + .desc = "UDP IPv4 drop on reuseport", + .lookup_prog = skel->progs.select_sock_a, + .reuseport_prog = skel->progs.reuseport_drop, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_DGRAM, + .send_to = { EXT_IP4, EXT_PORT }, + .recv_at = { INT_IP4, INT_PORT }, + }, + { + .desc = "TCP IPv6 drop on reuseport", + .lookup_prog = skel->progs.select_sock_a, + .reuseport_prog = skel->progs.reuseport_drop, + .sock_map = skel->maps.redir_map, + .sotype = SOCK_STREAM, + .send_to = { EXT_IP6, EXT_PORT }, + .recv_at = { INT_IP6, INT_PORT }, + }, + + }; + const struct test *t; + + for (t = tests; t < tests + ARRAY_SIZE(tests); t++) { + if (test__start_subtest(t->desc)) + drop_on_reuseport(t); + } +} + +static void run_tests(struct test_sk_lookup_kern *skel) +{ + if (test__start_subtest("query lookup prog")) + query_lookup_prog(skel); + test_override_lookup(skel); + test_drop_on_lookup(skel); + test_drop_on_reuseport(skel); +} + +static int switch_netns(int *saved_net) +{ + static const char * const setup_script[] = { + "ip -6 addr add dev lo " EXT_IP6 "/128 nodad", + "ip -6 addr add dev lo " INT_IP6 "/128 nodad", + "ip link set dev lo up", + NULL, + }; + const char * const *cmd; + int net_fd, err; + + net_fd = open("/proc/self/ns/net", O_RDONLY); + if (CHECK_FAIL(net_fd < 0)) { + log_err("open(/proc/self/ns/net)"); + return -1; + } + + err = unshare(CLONE_NEWNET); + if (CHECK_FAIL(err)) { + log_err("unshare(CLONE_NEWNET)"); + goto close; + } + + for (cmd = setup_script; *cmd; cmd++) { + err = system(*cmd); + if (CHECK_FAIL(err)) { + log_err("system(%s)", *cmd); + goto close; + } + } + + *saved_net = net_fd; + return 0; + +close: + close(net_fd); + return -1; +} + +static void restore_netns(int saved_net) +{ + int err; + + err = setns(saved_net, CLONE_NEWNET); + if (CHECK_FAIL(err)) + log_err("setns(CLONE_NEWNET)"); + + close(saved_net); +} + +void test_sk_lookup(void) +{ + struct test_sk_lookup_kern *skel; + int err, saved_net; + + err = switch_netns(&saved_net); + if (err) + return; + + skel = test_sk_lookup_kern__open_and_load(); + if (CHECK_FAIL(!skel)) { + errno = 0; + log_err("failed to open and load BPF skeleton"); + goto restore_netns; + } + + run_tests(skel); + + test_sk_lookup_kern__destroy(skel); +restore_netns: + restore_netns(saved_net); +} diff --git a/tools/testing/selftests/bpf/progs/test_sk_lookup_kern.c b/tools/testing/selftests/bpf/progs/test_sk_lookup_kern.c new file mode 100644 index 000000000000..fc3ad9a69484 --- /dev/null +++ b/tools/testing/selftests/bpf/progs/test_sk_lookup_kern.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0 OR BSD-3-Clause +// Copyright (c) 2020 Cloudflare + +#include +#include + +#include +#include + +#define IP4(a, b, c, d) \ + bpf_htonl((((__u32)(a) & 0xffU) << 24) | \ + (((__u32)(b) & 0xffU) << 16) | \ + (((__u32)(c) & 0xffU) << 8) | \ + (((__u32)(d) & 0xffU) << 0)) +#define IP6(aaaa, bbbb, cccc, dddd) \ + { bpf_htonl(aaaa), bpf_htonl(bbbb), bpf_htonl(cccc), bpf_htonl(dddd) } + +#define MAX_SOCKS 32 + +struct { + __uint(type, BPF_MAP_TYPE_SOCKMAP); + __uint(max_entries, MAX_SOCKS); + __type(key, __u32); + __type(value, __u64); +} redir_map SEC(".maps"); + +enum { + SERVER_A = 0, + SERVER_B = 1, +}; + +enum { + NO_FLAGS = 0, +}; + +static const __u32 DST_PORT = 7007; +static const __u32 DST_IP4 = IP4(127, 0, 0, 1); +static const __u32 DST_IP6[] = IP6(0xfd000000, 0x0, 0x0, 0x00000001); + +SEC("sk_lookup/lookup_pass") +int lookup_pass(struct bpf_sk_lookup *ctx) +{ + return BPF_OK; +} + +SEC("sk_lookup/lookup_drop") +int lookup_drop(struct bpf_sk_lookup *ctx) +{ + return BPF_DROP; +} + +SEC("sk_reuseport/reuse_pass") +int reuseport_pass(struct sk_reuseport_md *ctx) +{ + return SK_PASS; +} + +SEC("sk_reuseport/reuse_drop") +int reuseport_drop(struct sk_reuseport_md *ctx) +{ + return SK_DROP; +} + +/* Redirect packets destined for port DST_PORT to socket at redir_map[0]. */ +SEC("sk_lookup/redir_port") +int redir_port(struct bpf_sk_lookup *ctx) +{ + __u32 key = SERVER_A; + struct bpf_sock *sk; + int err; + + if (ctx->local_port != DST_PORT) + return BPF_OK; + + sk = bpf_map_lookup_elem(&redir_map, &key); + if (!sk) + return BPF_OK; + + err = bpf_sk_assign(ctx, sk, NO_FLAGS); + bpf_sk_release(sk); + return err ? BPF_DROP : BPF_REDIRECT; +} + +/* Redirect packets destined for DST_IP4 address to socket at redir_map[0]. */ +SEC("sk_lookup/redir_ip4") +int redir_ip4(struct bpf_sk_lookup *ctx) +{ + __u32 key = SERVER_A; + struct bpf_sock *sk; + int err; + + if (ctx->family != AF_INET) + return BPF_OK; + if (ctx->local_port != DST_PORT) + return BPF_OK; + if (ctx->local_ip4 != DST_IP4) + return BPF_OK; + + sk = bpf_map_lookup_elem(&redir_map, &key); + if (!sk) + return BPF_OK; + + err = bpf_sk_assign(ctx, sk, NO_FLAGS); + bpf_sk_release(sk); + return err ? BPF_DROP : BPF_REDIRECT; +} + +/* Redirect packets destined for DST_IP6 address to socket at redir_map[0]. */ +SEC("sk_lookup/redir_ip6") +int redir_ip6(struct bpf_sk_lookup *ctx) +{ + __u32 key = SERVER_A; + struct bpf_sock *sk; + int err; + + if (ctx->family != AF_INET6) + return BPF_OK; + if (ctx->local_port != DST_PORT) + return BPF_OK; + if (ctx->local_ip6[0] != DST_IP6[0] || + ctx->local_ip6[1] != DST_IP6[1] || + ctx->local_ip6[2] != DST_IP6[2] || + ctx->local_ip6[3] != DST_IP6[3]) + return BPF_OK; + + sk = bpf_map_lookup_elem(&redir_map, &key); + if (!sk) + return BPF_OK; + + err = bpf_sk_assign(ctx, sk, NO_FLAGS); + bpf_sk_release(sk); + return err ? BPF_DROP : BPF_REDIRECT; +} + +SEC("sk_lookup/select_sock_a") +int select_sock_a(struct bpf_sk_lookup *ctx) +{ + __u32 key = SERVER_A; + struct bpf_sock *sk; + int err; + + sk = bpf_map_lookup_elem(&redir_map, &key); + if (!sk) + return BPF_OK; + + err = bpf_sk_assign(ctx, sk, NO_FLAGS); + bpf_sk_release(sk); + return err ? BPF_DROP : BPF_REDIRECT; +} + +SEC("sk_reuseport/select_sock_b") +int select_sock_b(struct sk_reuseport_md *ctx) +{ + __u32 key = SERVER_B; + int err; + + err = bpf_sk_select_reuseport(ctx, &redir_map, &key, NO_FLAGS); + return err ? SK_DROP : SK_PASS; +} + +char _license[] SEC("license") = "Dual BSD/GPL"; +__u32 _version SEC("version") = 1;