From patchwork Thu Jun 17 21:27:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462823 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72470C49361 for ; Thu, 17 Jun 2021 21:28:01 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4EB8160D07 for ; Thu, 17 Jun 2021 21:28:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233014AbhFQVaI (ORCPT ); Thu, 17 Jun 2021 17:30:08 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:57445 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231319AbhFQVaH (ORCPT ); Thu, 17 Jun 2021 17:30:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=EUNzvsFw1gUIvG2m24sl/ooKd0gr/8H4zQ7xV3X+3nUQMQ3pemtMyGQYnUwe0UpCDk+MAb hrTFNdCiHlBjvTPSE5Yec49g3fVC1LcqQzabvfS12SIH/1RxIONYQMhGgHNX6e7r39KGVC 5h9UwmpcYgvw5XrRG41kQdwJ/sc4C4E= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-448-RAlx-oEhOrCWzs-0P0Ozbw-1; Thu, 17 Jun 2021 17:27:57 -0400 X-MC-Unique: RAlx-oEhOrCWzs-0P0Ozbw-1 Received: by mail-ed1-f69.google.com with SMTP id cb4-20020a0564020b64b02903947455afa5so577617edb.9 for ; Thu, 17 Jun 2021 14:27:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ESlzoBrCtV1p5Bz5L61JWXgmG5edOzrWTpf13f6lwsQ=; b=Wdm4UeVfxkAS3u2CiQ8Rj4TOV/RpDPQ1CxyRnsda2JhSnZxGj8O+eTp3JW2/6A/ufX 7hZybsG3ZFDeNvEVRYJ7yZBO2TRRR1CpMV4EnY9alQEJKupvlzvsOQKXLltCpqv/WAXC Mb99h6hoQbKcsK2Rd/LmskWtmt9t6GuisOObSb+Bn5BePLgPm2KdIDsxj5TZd19bBLub NzmpuveqkKmn+94nhP3idV6qCmB2cZt05xiSVgEvf8S/HSxC7nlFfnB3fBI6dIwqzT88 VTOt3GZ1FHSelLqTkjlCYkNATztOCk4WdIO5+Ylt+byg2a1ZnSxkT41LQOJweTgtlSHS 3OAg== X-Gm-Message-State: AOAM530+o2lIDdTve3PPHn8Su/y23pz1Bl9xVT54iZsDU5zwFNC5Jv0I THY0I6Zzu4l8uKPKJ4s+bVtw/8S4Qc/5OBOQCikIdILNPeoAStuYKXHHZ4TKm03/St5Ge4GC2yu u6nDD+usZ4rJCOxbm X-Received: by 2002:a17:906:b03:: with SMTP id u3mr7263305ejg.95.1623965275935; Thu, 17 Jun 2021 14:27:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxVq90bXdFT/ATvZR+KbPRvFhmdGeDC2xS3RJdzmjp7zYHS+Yy5ZefFxwZeuZ2Lr3aFcVh7Nw== X-Received: by 2002:a17:906:b03:: with SMTP id u3mr7263295ejg.95.1623965275782; Thu, 17 Jun 2021 14:27:55 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id h9sm5028627edt.18.2021.06.17.14.27.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:27:55 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 775961802A5; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= Subject: [PATCH bpf-next v3 01/16] rcu: Create an unrcu_pointer() to remove __rcu from a pointer Date: Thu, 17 Jun 2021 23:27:33 +0200 Message-Id: <20210617212748.32456-2-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org From: "Paul E. McKenney" The xchg() and cmpxchg() functions are sometimes used to carry out RCU updates. Unfortunately, this can result in sparse warnings for both the old-value and new-value arguments, as well as for the return value. The arguments can be dealt with using RCU_INITIALIZER(): old_p = xchg(&p, RCU_INITIALIZER(new_p)); But a sparse warning still remains due to assigning the __rcu pointer returned from xchg to the (most likely) non-__rcu pointer old_p. This commit therefore provides an unrcu_pointer() macro that strips the __rcu. This macro can be used as follows: old_p = unrcu_pointer(xchg(&p, RCU_INITIALIZER(new_p))); Reported-by: Toke Høiland-Jørgensen Signed-off-by: Paul E. McKenney Signed-off-by: Toke Høiland-Jørgensen --- include/linux/rcupdate.h | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/include/linux/rcupdate.h b/include/linux/rcupdate.h index 9455476c5ba2..d7895b81264e 100644 --- a/include/linux/rcupdate.h +++ b/include/linux/rcupdate.h @@ -363,6 +363,20 @@ static inline void rcu_preempt_sleep_check(void) { } #define rcu_check_sparse(p, space) #endif /* #else #ifdef __CHECKER__ */ +/** + * unrcu_pointer - mark a pointer as not being RCU protected + * @p: pointer needing to lose its __rcu property + * + * Converts @p from an __rcu pointer to a __kernel pointer. + * This allows an __rcu pointer to be used with xchg() and friends. + */ +#define unrcu_pointer(p) \ +({ \ + typeof(*p) *_________p1 = (typeof(*p) *__force)(p); \ + rcu_check_sparse(p, __rcu); \ + ((typeof(*p) __force __kernel *)(_________p1)); \ +}) + #define __rcu_access_pointer(p, space) \ ({ \ typeof(*p) *_________p1 = (typeof(*p) *__force)READ_ONCE(p); \ From patchwork Thu Jun 17 21:27:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462822 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNWANTED_LANGUAGE_BODY,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A038AC49EA5 for ; Thu, 17 Jun 2021 21:28:03 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7E21761375 for ; Thu, 17 Jun 2021 21:28:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233057AbhFQVaK (ORCPT ); Thu, 17 Jun 2021 17:30:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46090 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232361AbhFQVaH (ORCPT ); Thu, 17 Jun 2021 17:30:07 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965278; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=x+C1vvBih2DDvMDumUaYzwO9FAV6D2CItQN3YHn0G7w=; b=UyVLYk+R40TPN2gjxxJ6AnCrY6BqKrtvb/0N0vPT7aiuiIw/e9n0o+5Jhq00VvueZDnc5v ANOLoNNYA3sOIM+A0ZDWza8+EMrc8CWERzq9bOFS9Vw2Lt2nZxhSYw4Q910PAv0E9Fc2J4 /bZRNSO5Pd3OMB3LK6crZ5N4dmaDIm0= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-245-iO4mQaxmNCWliZExLPJ2JQ-1; Thu, 17 Jun 2021 17:27:57 -0400 X-MC-Unique: iO4mQaxmNCWliZExLPJ2JQ-1 Received: by mail-ed1-f71.google.com with SMTP id i22-20020a05640242d6b0290392e051b029so2389052edc.11 for ; Thu, 17 Jun 2021 14:27:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=x+C1vvBih2DDvMDumUaYzwO9FAV6D2CItQN3YHn0G7w=; b=mzqpfMZWJt4rf6n4SwvfZW+VUbb1nXzTkGv6cez9Y5Ih+Km+qvideACGqZmi1kj7qY BrfkvvNUKtpQtb8kJ7Tk4Zy++4FFYJzg4//ec2hAucSkhbhAxaCARWdxOxGmcvsLGIop +mkmseOVgC9P/ASuDXF1ntRm6wFjnymWmwID6J0QMRLwXH8pYKgQR9p1J2FynqDnwL6U aNJxZv91eSCDcDP28xTxmus3ynUwRN7mXJHLsjgq7IwbLH3DHhQ+nhK6R5YwsvrwVMVQ NzgvhnnW3BlKS8JoyEaPTFmfNvbkNdGZcMW5OxQbNtfGcI+K/p5JMsBiO94qX7p4zkHA NnLQ== X-Gm-Message-State: AOAM5306xf4sNrQvE5HaYjzPhqU2r2Hq3v0KuTUejnSqPatBFWWo1ZdG nq5zuM0/6BteMxCjNN1oCKfZjcX4kt4KhVn/+L209UwUQEW08/qzZkKkZLd3oYpxfPnzK5eo5lC uIfTBHECcwnPWJfSQ X-Received: by 2002:a05:6402:51d1:: with SMTP id r17mr451089edd.91.1623965276166; Thu, 17 Jun 2021 14:27:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzuJMoUy1hXydwhfMQBzqzr6BrEg7lbvkkFSNeTBiotlbBoHDl0qp+bpeu9UXkB60eR8jNOsQ== X-Received: by 2002:a05:6402:51d1:: with SMTP id r17mr451071edd.91.1623965276019; Thu, 17 Jun 2021 14:27:56 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id t18sm5186222edw.47.2021.06.17.14.27.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:27:55 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 7E6BC18071E; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= Subject: [PATCH bpf-next v3 02/16] bpf: allow RCU-protected lookups to happen from bh context Date: Thu, 17 Jun 2021 23:27:34 +0200 Message-Id: <20210617212748.32456-3-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org XDP programs are called from a NAPI poll context, which means the RCU reference liveness is ensured by local_bh_disable(). Add rcu_read_lock_bh_held() as a condition to the RCU checks for map lookups so lockdep understands that the dereferences are safe from inside *either* an rcu_read_lock() section *or* a local_bh_disable() section. While both bh_disabled and rcu_read_lock() provide RCU protection, they are semantically distinct, so we need both conditions to prevent lockdep complaints. This change is done in preparation for removing the redundant rcu_read_lock()s from drivers. Acked-by: Martin KaFai Lau Signed-off-by: Toke Høiland-Jørgensen --- kernel/bpf/hashtab.c | 21 ++++++++++++++------- kernel/bpf/helpers.c | 6 +++--- kernel/bpf/lpm_trie.c | 6 ++++-- 3 files changed, 21 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/hashtab.c b/kernel/bpf/hashtab.c index 6f6681b07364..72c58cc516a3 100644 --- a/kernel/bpf/hashtab.c +++ b/kernel/bpf/hashtab.c @@ -596,7 +596,8 @@ static void *__htab_map_lookup_elem(struct bpf_map *map, void *key) struct htab_elem *l; u32 hash, key_size; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -989,7 +990,8 @@ static int htab_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1082,7 +1084,8 @@ static int htab_lru_map_update_elem(struct bpf_map *map, void *key, void *value, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1148,7 +1151,8 @@ static int __htab_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1202,7 +1206,8 @@ static int __htab_lru_percpu_map_update_elem(struct bpf_map *map, void *key, /* unknown flags */ return -EINVAL; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1276,7 +1281,8 @@ static int htab_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; @@ -1311,7 +1317,8 @@ static int htab_lru_map_delete_elem(struct bpf_map *map, void *key) u32 hash, key_size; int ret; - WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_trace_held() && + !rcu_read_lock_bh_held()); key_size = map->key_size; diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 544773970dbc..e880f6bb6f28 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -28,7 +28,7 @@ */ BPF_CALL_2(bpf_map_lookup_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return (unsigned long) map->ops->map_lookup_elem(map, key); } @@ -44,7 +44,7 @@ const struct bpf_func_proto bpf_map_lookup_elem_proto = { BPF_CALL_4(bpf_map_update_elem, struct bpf_map *, map, void *, key, void *, value, u64, flags) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_update_elem(map, key, value, flags); } @@ -61,7 +61,7 @@ const struct bpf_func_proto bpf_map_update_elem_proto = { BPF_CALL_2(bpf_map_delete_elem, struct bpf_map *, map, void *, key) { - WARN_ON_ONCE(!rcu_read_lock_held()); + WARN_ON_ONCE(!rcu_read_lock_held() && !rcu_read_lock_bh_held()); return map->ops->map_delete_elem(map, key); } diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c index 1b7b8a6f34ee..423549d2c52e 100644 --- a/kernel/bpf/lpm_trie.c +++ b/kernel/bpf/lpm_trie.c @@ -232,7 +232,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) /* Start walking the trie from the root node ... */ - for (node = rcu_dereference(trie->root); node;) { + for (node = rcu_dereference_check(trie->root, rcu_read_lock_bh_held()); + node;) { unsigned int next_bit; size_t matchlen; @@ -264,7 +265,8 @@ static void *trie_lookup_elem(struct bpf_map *map, void *_key) * traverse down. */ next_bit = extract_bit(key->data, node->prefixlen); - node = rcu_dereference(node->child[next_bit]); + node = rcu_dereference_check(node->child[next_bit], + rcu_read_lock_bh_held()); } if (!found) From patchwork Thu Jun 17 21:27:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462815 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1B632C48BE5 for ; Thu, 17 Jun 2021 21:37:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 054C66128B for ; Thu, 17 Jun 2021 21:37:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231656AbhFQVjk (ORCPT ); Thu, 17 Jun 2021 17:39:40 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23971 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231379AbhFQVjh (ORCPT ); Thu, 17 Jun 2021 17:39:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cJz59YB8g7oLdL+mJzpIs/zDJlYd7kwgcbU7x/ViItE=; b=OLrvLcJPRWOqKsrReXwUvbhvPVmJY+iNlWFuzI7TTPyvIQhfaA+ZF1YO6EzX+H9bbAKLhl 1yK/DyhW89yKI2rF3Wk4B42PzReCX3N+TqM50kOq+Z03IZftjVB4BHRqb3aIJaxm8QHNtZ lIjr2YfHRf9J0tqoS/SGVT28h+cchEs= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-251-COK0ROIlOceJH0zP0X2lZw-1; Thu, 17 Jun 2021 17:37:26 -0400 X-MC-Unique: COK0ROIlOceJH0zP0X2lZw-1 Received: by mail-ed1-f72.google.com with SMTP id y16-20020a0564024410b0290394293f6816so2383778eda.20 for ; Thu, 17 Jun 2021 14:37:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cJz59YB8g7oLdL+mJzpIs/zDJlYd7kwgcbU7x/ViItE=; b=aR/kdJtUUb/Omtp86Ekt/euR7ds6Rn46QUkykC3unekW1wPLrdavTHcHLe2eKPEsle I20cTqr6naP2vFzuKyl6Gfiqjim6sX2xmcYC9lMyPFRCJCOMPwmaY6tEiqCabc0u1IYG eWOM3UtEBD0Uwdx9DUxJFzWj8Hqy2SPSjJjOOxu2ZU6xP8FvjjQ2BwP17NANq2cfilka qdBLrbkCeR/NzDFE642WznkKo4elmc/7u/TuDv/RARdrUmJa++ejsU6vBJef+GKdfASl MEQTwTHOUeiNu55Y/lfmkay/lgbGY3L0MA0eT6BX16QI51Uz1CRYQuVpNqOncVUY9OM/ 1ZhA== X-Gm-Message-State: AOAM532G17W5PShaEhtqeMzqVB+UyjD4x9RZu9xgLRptBc3h+J0qooPN juH4dsUaqTEpW79nS17BDKYmMB/rjqdvvcCGPoFWR519rycsMRG9YZJRiKRYy4Kk0+WmzHPR+cm tc6TvHkrmO5RIZjC8 X-Received: by 2002:a17:906:6bd8:: with SMTP id t24mr7529621ejs.501.1623965845819; Thu, 17 Jun 2021 14:37:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7MiYndSZPPnikrer9lLYqdDjZqLkcm6ZdagJQugiaU6quK0vkJPkwzDn0Y2udQcNQ/YaluA== X-Received: by 2002:a17:906:6bd8:: with SMTP id t24mr7529608ejs.501.1623965845672; Thu, 17 Jun 2021 14:37:25 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id f4sm125915ejj.49.2021.06.17.14.37.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:37:22 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id A1B4A180730; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Madalin Bucur , Ioana Ciornei , Ioana Radulescu , Camelia Groza Subject: [PATCH bpf-next v3 07/16] freescale: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:39 +0200 Message-Id: <20210617212748.32456-8-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The dpaa and dpaa2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Madalin Bucur Cc: Ioana Ciornei Cc: Ioana Radulescu Reviewed-by: Camelia Groza Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/freescale/dpaa/dpaa_eth.c | 11 ++++------- drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c | 6 +++--- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c index 177c020bf34a..98fdcbde687a 100644 --- a/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c +++ b/drivers/net/ethernet/freescale/dpaa/dpaa_eth.c @@ -2558,13 +2558,9 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, u32 xdp_act; int err; - rcu_read_lock(); - xdp_prog = READ_ONCE(priv->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) return XDP_PASS; - } xdp_init_buff(&xdp, DPAA_BP_RAW_SIZE - DPAA_TX_PRIV_DATA_SIZE, &dpaa_fq->xdp_rxq); @@ -2585,6 +2581,9 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, } #endif + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp); /* Update the length and the offset of the FD */ @@ -2638,8 +2637,6 @@ static u32 dpaa_run_xdp(struct dpaa_priv *priv, struct qm_fd *fd, void *vaddr, break; } - rcu_read_unlock(); - return xdp_act; } diff --git a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c index 8433aa730c42..964d85c9e37d 100644 --- a/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c +++ b/drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c @@ -352,8 +352,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, u32 xdp_act = XDP_PASS; int err, offset; - rcu_read_lock(); - xdp_prog = READ_ONCE(ch->xdp.prog); if (!xdp_prog) goto out; @@ -363,6 +361,9 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, xdp_prepare_buff(&xdp, vaddr + offset, XDP_PACKET_HEADROOM, dpaa2_fd_get_len(fd), false); + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ xdp_act = bpf_prog_run_xdp(xdp_prog, &xdp); /* xdp.data pointer may have changed */ @@ -414,7 +415,6 @@ static u32 dpaa2_eth_run_xdp(struct dpaa2_eth_priv *priv, ch->xdp.res |= xdp_act; out: - rcu_read_unlock(); return xdp_act; } From patchwork Thu Jun 17 21:27:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462821 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A47FDC49EA4 for ; Thu, 17 Jun 2021 21:28:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 889F8613DB for ; Thu, 17 Jun 2021 21:28:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233086AbhFQVaM (ORCPT ); Thu, 17 Jun 2021 17:30:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:41539 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233044AbhFQVaK (ORCPT ); Thu, 17 Jun 2021 17:30:10 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BrDykLhiGSf6xXkjN+JUrFWMpiV7dzYyZbBTLShnH1w=; b=fcC6mJKVUS8MeJKuL3dtGKeJO53YzPBwteu9zeVkoub0oKz8RlXTf0WfmrnCYuoe575aJA nYNuiZ6J3YlIwPuj+scZnxmVtgY1QfNmgmU6XGTmv5qCLrXt0lfkWZbU7o3UO40I4xxvV3 7nWTupPAq8HY4VNdQAIQQqgPrL+S68w= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-361-5qb6aApYPouiYc0cK4GtOQ-1; Thu, 17 Jun 2021 17:28:00 -0400 X-MC-Unique: 5qb6aApYPouiYc0cK4GtOQ-1 Received: by mail-ej1-f70.google.com with SMTP id l6-20020a1709062a86b029046ec0ceaf5cso584280eje.8 for ; Thu, 17 Jun 2021 14:28:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BrDykLhiGSf6xXkjN+JUrFWMpiV7dzYyZbBTLShnH1w=; b=cB1/0QPkYKy6DzdzydaHUBp2uP+V7rE3HnG1LVqYttbfK2SgwLhFuwlxZDQKn5oZyK aXdjRKw4CwHI/MQu7KRaO89yShjtyXt8P59f0F3ZSNFqrcNAShasnzzY07F7vs93bMgA xcarPb8RwTzxt1M2kblPhvnd2iNeZ/vVyZETVDPyhgAwIDGAEUR6SgfxhSvQmrBHD8Eg 7GIQRtHP5I3rRU0o1NAChvPb7ssRo/U3mLKwn+b2KXBj2NkWxab5SikjzReckgxDCLBg i0b7eQJCkj77Xe604yMiGSut0wWKHyCqaOd2DOJt4tCjBY+fbYnDkIR6Aeg90/5OiLWy 3ncg== X-Gm-Message-State: AOAM533cDl8fBU08PGEeq2yhtth9hHu8AaqZaBXRpOklRR03lQol48We zenr8jyHFCHf0fneUsN9cNVQsP20Y1oum2DHnF9uP63NcXP17xfzDxDwWhU85R2Vpso+WxY4QNT LaP1vASz0JY9QgwPD X-Received: by 2002:a17:906:29c8:: with SMTP id y8mr7433863eje.312.1623965279724; Thu, 17 Jun 2021 14:27:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw2cjGWsNm7HTPrQRwMQa1xBLKp0M4CHdCL2Y/WsZaVaFHpBIn5mKLkfJDTlSnxEUOYcRWEHg== X-Received: by 2002:a17:906:29c8:: with SMTP id y8mr7433846eje.312.1623965279559; Thu, 17 Jun 2021 14:27:59 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id v26sm111414ejk.70.2021.06.17.14.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:27:58 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id A8776180731; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Jesse Brandeburg , Tony Nguyen , intel-wired-lan@lists.osuosl.org Subject: [PATCH bpf-next v3 08/16] net: intel: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:40 +0200 Message-Id: <20210617212748.32456-9-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The Intel drivers all have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Jesse Brandeburg Cc: Tony Nguyen Cc: intel-wired-lan@lists.osuosl.org Tested-by: Jesper Dangaard Brouer # i40e Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/intel/i40e/i40e_txrx.c | 5 +++-- drivers/net/ethernet/intel/i40e/i40e_xsk.c | 11 +++++------ drivers/net/ethernet/intel/ice/ice_txrx.c | 6 +----- drivers/net/ethernet/intel/ice/ice_xsk.c | 6 +----- drivers/net/ethernet/intel/igb/igb_main.c | 5 +++-- drivers/net/ethernet/intel/igc/igc_main.c | 10 +++++----- drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 5 +++-- drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 9 ++++----- drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c | 5 +++-- 9 files changed, 28 insertions(+), 34 deletions(-) diff --git a/drivers/net/ethernet/intel/i40e/i40e_txrx.c b/drivers/net/ethernet/intel/i40e/i40e_txrx.c index de70c16ef619..7fc5bdb5cd9e 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_txrx.c +++ b/drivers/net/ethernet/intel/i40e/i40e_txrx.c @@ -2298,7 +2298,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2306,6 +2305,9 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) prefetchw(xdp->data_hard_start); /* xdp_frame write */ + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ act = bpf_prog_run_xdp(xdp_prog, xdp); switch (act) { case XDP_PASS: @@ -2329,7 +2331,6 @@ static int i40e_run_xdp(struct i40e_ring *rx_ring, struct xdp_buff *xdp) break; } xdp_out: - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/i40e/i40e_xsk.c b/drivers/net/ethernet/intel/i40e/i40e_xsk.c index 46d884417c63..a5982cd112be 100644 --- a/drivers/net/ethernet/intel/i40e/i40e_xsk.c +++ b/drivers/net/ethernet/intel/i40e/i40e_xsk.c @@ -153,8 +153,10 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); - /* NB! xdp_prog will always be !NULL, due to the fact that + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + * + * NB! xdp_prog will always be !NULL, due to the fact that * this path is enabled by setting an XDP program. */ xdp_prog = READ_ONCE(rx_ring->xdp_prog); @@ -162,9 +164,7 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? I40E_XDP_REDIR : I40E_XDP_CONSUMED; } switch (act) { @@ -184,7 +184,6 @@ static int i40e_run_xdp_zc(struct i40e_ring *rx_ring, struct xdp_buff *xdp) result = I40E_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ice/ice_txrx.c b/drivers/net/ethernet/intel/ice/ice_txrx.c index e2b4b29ea207..1a311e91fb6d 100644 --- a/drivers/net/ethernet/intel/ice/ice_txrx.c +++ b/drivers/net/ethernet/intel/ice/ice_txrx.c @@ -1129,15 +1129,11 @@ int ice_clean_rx_irq(struct ice_ring *rx_ring, int budget) xdp.frame_sz = ice_rx_frame_truesize(rx_ring, size); #endif - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); - if (!xdp_prog) { - rcu_read_unlock(); + if (!xdp_prog) goto construct_skb; - } xdp_res = ice_run_xdp(rx_ring, &xdp, xdp_prog); - rcu_read_unlock(); if (!xdp_res) goto construct_skb; if (xdp_res & (ICE_XDP_TX | ICE_XDP_REDIR)) { diff --git a/drivers/net/ethernet/intel/ice/ice_xsk.c b/drivers/net/ethernet/intel/ice/ice_xsk.c index faa7b8d96adb..d6da377f5ac3 100644 --- a/drivers/net/ethernet/intel/ice/ice_xsk.c +++ b/drivers/net/ethernet/intel/ice/ice_xsk.c @@ -463,7 +463,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) struct ice_ring *xdp_ring; u32 act; - rcu_read_lock(); /* ZC patch is enabled only when XDP program is set, * so here it can not be NULL */ @@ -473,9 +472,7 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? ICE_XDP_REDIR : ICE_XDP_CONSUMED; } switch (act) { @@ -496,7 +493,6 @@ ice_run_xdp_zc(struct ice_ring *rx_ring, struct xdp_buff *xdp) break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c index 038a9fd1af44..0b68d589218a 100644 --- a/drivers/net/ethernet/intel/igb/igb_main.c +++ b/drivers/net/ethernet/intel/igb/igb_main.c @@ -8387,7 +8387,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -8395,6 +8394,9 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, prefetchw(xdp->data_hard_start); /* xdp_frame write */ + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ act = bpf_prog_run_xdp(xdp_prog, xdp); switch (act) { case XDP_PASS: @@ -8420,7 +8422,6 @@ static struct sk_buff *igb_run_xdp(struct igb_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index ea998d2defa4..333057ce60c7 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -2175,18 +2175,18 @@ static struct sk_buff *igc_xdp_run_prog(struct igc_adapter *adapter, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(adapter->xdp_prog); if (!prog) { res = IGC_XDP_PASS; - goto unlock; + goto out; } + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ res = __igc_xdp_run_prog(adapter, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c index c5ec17d19c59..9cebe7894111 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_main.c @@ -2199,7 +2199,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) @@ -2207,6 +2206,9 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, prefetchw(xdp->data_hard_start); /* xdp_frame write */ + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ act = bpf_prog_run_xdp(xdp_prog, xdp); switch (act) { case XDP_PASS: @@ -2237,7 +2239,6 @@ static struct sk_buff *ixgbe_run_xdp(struct ixgbe_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } diff --git a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c index 91ad5b902673..4a075e667082 100644 --- a/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c +++ b/drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c @@ -100,15 +100,15 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, struct xdp_frame *xdpf; u32 act; - rcu_read_lock(); + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ xdp_prog = READ_ONCE(rx_ring->xdp_prog); act = bpf_prog_run_xdp(xdp_prog, xdp); if (likely(act == XDP_REDIRECT)) { err = xdp_do_redirect(rx_ring->netdev, xdp, xdp_prog); - result = !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; - rcu_read_unlock(); - return result; + return !err ? IXGBE_XDP_REDIR : IXGBE_XDP_CONSUMED; } switch (act) { @@ -132,7 +132,6 @@ static int ixgbe_run_xdp_zc(struct ixgbe_adapter *adapter, result = IXGBE_XDP_CONSUMED; break; } - rcu_read_unlock(); return result; } diff --git a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c index ba2ed8a43d2d..ab05ebc3d350 100644 --- a/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c +++ b/drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c @@ -1054,12 +1054,14 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, struct bpf_prog *xdp_prog; u32 act; - rcu_read_lock(); xdp_prog = READ_ONCE(rx_ring->xdp_prog); if (!xdp_prog) goto xdp_out; + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ act = bpf_prog_run_xdp(xdp_prog, xdp); switch (act) { case XDP_PASS: @@ -1079,7 +1081,6 @@ static struct sk_buff *ixgbevf_run_xdp(struct ixgbevf_adapter *adapter, break; } xdp_out: - rcu_read_unlock(); return ERR_PTR(-result); } From patchwork Thu Jun 17 21:27:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462816 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AF77C2B9F4 for ; Thu, 17 Jun 2021 21:37:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2EBC261241 for ; Thu, 17 Jun 2021 21:37:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231467AbhFQVjh (ORCPT ); Thu, 17 Jun 2021 17:39:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:43546 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231226AbhFQVjf (ORCPT ); Thu, 17 Jun 2021 17:39:35 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=b/T3H4vaV8F+Q+x5m1dW8YU/RP90nVjw+/NyR+B0cto=; b=RwTWpJxetiam5oVIzIWEXQj8BpTAo/DJPK3GC2CVlO/TNGpHrrJOpk3q9z6DDuRKf0WlCN kKf1cy7PlGzVvqnDIZGxgXLViga0IE47kFpyqOu2Eebvva5aE/MqyKmQHH3BAhr5YU8Zsy ul6BfmvcRzQ39ndPuGV02O20b48FIMs= Received: from mail-ej1-f70.google.com (mail-ej1-f70.google.com [209.85.218.70]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-192-aTse4hqMMhaXo48DaLLnKA-1; Thu, 17 Jun 2021 17:37:25 -0400 X-MC-Unique: aTse4hqMMhaXo48DaLLnKA-1 Received: by mail-ej1-f70.google.com with SMTP id m16-20020a1709066d10b029047b77529dc3so2155604ejr.16 for ; Thu, 17 Jun 2021 14:37:25 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=b/T3H4vaV8F+Q+x5m1dW8YU/RP90nVjw+/NyR+B0cto=; b=stJ89fK39yqLyMt6TzGK1uMSk4mpnyV3qlFmWEB0ZHjfg3hB9z3YusUtlqxNGvntj4 XQd13pN6A/VvWQGP7SyhwP1gRvCOQVq9pMiTmVdezmJg1e7yy/8T3T8/hFv3ekn6FPrm F6gIoc3qgc6VpXWdZvY+kzzdXijUbR8DsTobI850MQEP8MCPBZuMtPAGWbBxO36P/pui k7og1m+JNtBJe++m2+m0vIPKy76ogKM4OBjxkV3UxIEhsWQHMJxB0K0d4nf/d//Wnf/g PodRneHeZ2e4hWZLYjbvFXH878cY45nol2ni4zcy9Wxpbw6WoU+7H5jk2CGFyyo7LBZS u1dQ== X-Gm-Message-State: AOAM531IIryJgTOWx3+iPJ6gypNXpq8lDgXN6NNyLds0MlpfJEKMg2bj vYFp3y1ds6LIfuvCNWRLuP/r5Sxq5Ywj1J6Juuxtj+cMvbm7oD3IoTuRjbahD4+Je7lsu4HrRKf GmUB3lf7nPfKPxcd8 X-Received: by 2002:a05:6402:b5a:: with SMTP id bx26mr493960edb.81.1623965844135; Thu, 17 Jun 2021 14:37:24 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzRYMJ1okfOU42zzGSz1z3QdJrJk3hRHhSFfun2L1fNVGPvJLVoSQn8Ec15Jt6SRMeYoFq9xQ== X-Received: by 2002:a05:6402:b5a:: with SMTP id bx26mr493935edb.81.1623965843903; Thu, 17 Jun 2021 14:37:23 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id i10sm123544eja.3.2021.06.17.14.37.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:37:22 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id AF9FF180732; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Thomas Petazzoni , Marcin Wojtas , Russell King Subject: [PATCH bpf-next v3 09/16] marvell: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:41 +0200 Message-Id: <20210617212748.32456-10-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The mvneta and mvpp2 drivers have rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Thomas Petazzoni Cc: Marcin Wojtas Cc: Russell King Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/marvell/mvneta.c | 6 ++++-- drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c | 8 ++++---- 2 files changed, 8 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/marvell/mvneta.c b/drivers/net/ethernet/marvell/mvneta.c index 7d5cd9bc6c99..c2e9cbebc8d1 100644 --- a/drivers/net/ethernet/marvell/mvneta.c +++ b/drivers/net/ethernet/marvell/mvneta.c @@ -2370,7 +2370,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, /* Get number of received packets */ rx_todo = mvneta_rxq_busy_desc_num_get(pp, rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(pp->xdp_prog); /* Fairness NAPI loop */ @@ -2421,6 +2420,10 @@ static int mvneta_rx_swbm(struct napi_struct *napi, goto next; } + /* This code is invoked within a single NAPI poll cycle and thus + * under local_bh_disable(), which provides the needed RCU + * protection. + */ if (xdp_prog && mvneta_run_xdp(pp, rxq, xdp_prog, &xdp_buf, frame_sz, &ps)) goto next; @@ -2448,7 +2451,6 @@ static int mvneta_rx_swbm(struct napi_struct *napi, xdp_buf.data_hard_start = NULL; sinfo.nr_frags = 0; } - rcu_read_unlock(); if (xdp_buf.data_hard_start) mvneta_xdp_put_buff(pp, rxq, &xdp_buf, &sinfo, -1); diff --git a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c index b2259bf1d299..658db1720826 100644 --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c @@ -3852,8 +3852,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, int rx_done = 0; u32 xdp_ret = 0; - rcu_read_lock(); - xdp_prog = READ_ONCE(port->xdp_prog); /* Get number of received packets and clamp the to-do */ @@ -3925,6 +3923,10 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, MVPP2_MH_SIZE + MVPP2_SKB_HEADROOM, rx_bytes, false); + /* This code is invoked within a single NAPI poll cycle + * and thus under local_bh_disable(), which provides the + * needed RCU protection. + */ ret = mvpp2_run_xdp(port, xdp_prog, &xdp, pp, &ps); if (ret) { @@ -3988,8 +3990,6 @@ static int mvpp2_rx(struct mvpp2_port *port, struct napi_struct *napi, mvpp2_bm_pool_put(port, pool, dma_addr, phys_addr); } - rcu_read_unlock(); - if (xdp_ret & MVPP2_XDP_REDIR) xdp_do_flush_map(); From patchwork Thu Jun 17 21:27:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462817 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0410BC2B9F4 for ; Thu, 17 Jun 2021 21:37:27 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C24D061241 for ; Thu, 17 Jun 2021 21:37:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231225AbhFQVje (ORCPT ); Thu, 17 Jun 2021 17:39:34 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:57291 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229683AbhFQVje (ORCPT ); Thu, 17 Jun 2021 17:39:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965845; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=c8O4Yimc+TSjNewaWd3nzsqZvk1ap2sDHxPur5LQuO8=; b=E83l7R/Gy7JYnTYle3W79jnwRTZbT8gZhKv8vdc3nyBA3R5nyP797qTZPRUCvuNlBjSuJb 3sNF/r2LBDAC/ZNN0OPIO86fFDDC8ZuxCWcFNTHHxIat8Zqg4HQkolFSv3YYM4CDzJg7zc vsOdD0275yJELZFG4thQv0Zv3dfLG1k= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-187-pyiRdLR3OiGp8WzFXKPy2g-1; Thu, 17 Jun 2021 17:37:24 -0400 X-MC-Unique: pyiRdLR3OiGp8WzFXKPy2g-1 Received: by mail-ed1-f69.google.com with SMTP id g13-20020a056402090db02903935a4cb74fso2419558edz.1 for ; Thu, 17 Jun 2021 14:37:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=c8O4Yimc+TSjNewaWd3nzsqZvk1ap2sDHxPur5LQuO8=; b=sIMQiGq6vq0baHwZTOY57rSkfGG4hLilFZoyXXL9oMnLWBa8LYcFhuKOLH4n6M5Tdx 2po0JfPzt2GLPil+PSPyk7zGSJlty76IkgN+23znD4lyrwbD7Lu0XPEt4A1cOrS+zyov e+IVbjXiQ7wKVPuN9mKw+uOMb9XQa9NXGgsxJFhU7iCpo0clEjw/hRrpe/rTvJP7xjcG nBg79mY6+4rnQUWzBO98brVRRKy5Q3crIOjbUNgoR+rX+RR6LGeLDW148QVsWrg+z+9V T1MDNcOR+mnBFUa2PT6M51wN6xkXlNCyqNEgKplWV26ubQvy0cIfnceCJeTfLgJMIk54 b51g== X-Gm-Message-State: AOAM530Po+D5qLrEcWWsmkWgzC8rwhLMxia5zB+ZlZ7+4+rR7l9ED6xa iVaqzfaG56FiSgrsjMScgYJv20Y4Uh7kSo/PifvZoL5hWMRzNtlDarjb5GmnXn2XFhTiaP/ijot 2HAPEZTAcb2cWG3QU X-Received: by 2002:a17:906:b857:: with SMTP id ga23mr7467085ejb.296.1623965843566; Thu, 17 Jun 2021 14:37:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyncqixvPI2i/r8fEKf9J5GXPyMqsF90pHqtqhIlR+8XIzCseASzejpmol2cBYRKDrxwm28Mg== X-Received: by 2002:a17:906:b857:: with SMTP id ga23mr7467067ejb.296.1623965843412; Thu, 17 Jun 2021 14:37:23 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id cf26sm121229ejb.38.2021.06.17.14.37.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:37:22 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id B62E9180733; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Tariq Toukan Subject: [PATCH bpf-next v3 10/16] mlx4: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:42 +0200 Message-Id: <20210617212748.32456-11-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The mlx4 driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Also switch the RCU dereferences in the driver loop itself to the _bh variants. Cc: Tariq Toukan Reviewed-by: Tariq Toukan Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/mellanox/mlx4/en_rx.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx4/en_rx.c b/drivers/net/ethernet/mellanox/mlx4/en_rx.c index e35e4d7ef4d1..3f08c14d0441 100644 --- a/drivers/net/ethernet/mellanox/mlx4/en_rx.c +++ b/drivers/net/ethernet/mellanox/mlx4/en_rx.c @@ -679,9 +679,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud ring = priv->rx_ring[cq_ring]; - /* Protect accesses to: ring->xdp_prog, priv->mac_hash list */ - rcu_read_lock(); - xdp_prog = rcu_dereference(ring->xdp_prog); + xdp_prog = rcu_dereference_bh(ring->xdp_prog); xdp_init_buff(&xdp, priv->frag_info[0].frag_stride, &ring->xdp_rxq); doorbell_pending = false; @@ -744,7 +742,7 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud /* Drop the packet, since HW loopback-ed it */ mac_hash = ethh->h_source[MLX4_EN_MAC_HASH_IDX]; bucket = &priv->mac_hash[mac_hash]; - hlist_for_each_entry_rcu(entry, bucket, hlist) { + hlist_for_each_entry_rcu_bh(entry, bucket, hlist) { if (ether_addr_equal_64bits(entry->mac, ethh->h_source)) goto next; @@ -899,8 +897,6 @@ int mlx4_en_process_rx_cq(struct net_device *dev, struct mlx4_en_cq *cq, int bud break; } - rcu_read_unlock(); - if (likely(polled)) { if (doorbell_pending) { priv->tx_cq[TX_XDP][cq_ring]->xdp_busy = true; From patchwork Thu Jun 17 21:27:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462818 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 89933C49361 for ; Thu, 17 Jun 2021 21:28:24 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 78244613CE for ; Thu, 17 Jun 2021 21:28:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233045AbhFQVab (ORCPT ); Thu, 17 Jun 2021 17:30:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:35938 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233121AbhFQVaO (ORCPT ); Thu, 17 Jun 2021 17:30:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965285; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dAdah8VGuQn72TJlVoq7ahYZ+lMNJR+3mmSVKWZMutE=; b=jT3oPPxpdUesieYuMOXbqqnft7SsufHlevpPhfFazFqeDEMQcv66Reg+ULQ0rC1jcxAtxv lx1HTwlp+h9l0XmVrRyPlsEPEG7EvPnDQMWK1HSKbTjvOmDt5wbdVIUSb2aJ3Xl22DbxRj u3qnsfdA3g5WOcv5+p784A/yGHljj2Q= Received: from mail-ej1-f71.google.com (mail-ej1-f71.google.com [209.85.218.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-127-LpDbrwB5NVWVS4YfrZwlbA-1; Thu, 17 Jun 2021 17:28:03 -0400 X-MC-Unique: LpDbrwB5NVWVS4YfrZwlbA-1 Received: by mail-ej1-f71.google.com with SMTP id p20-20020a1709064994b02903cd421d7803so3009151eju.22 for ; Thu, 17 Jun 2021 14:28:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=dAdah8VGuQn72TJlVoq7ahYZ+lMNJR+3mmSVKWZMutE=; b=nRl8GfqEzk529Reg2ThihDPexCzr3VVinxb+vI0F9UgmiEwtsGSTRxS1q+AMg2hfAD Lzff/tBpEYvH7Vu9E+9oELna14vPme9KfzaoKlG/99xXKehJhkRyxlTqTZJrzyohh++p a+X92v6d9FemlaOMjJDS3bGeK54RN5HgPXtGXHKVFqUElGplZuYD9/7070jlxUd3t8rm fWd+4zVvoPSq4/wwC2qHDKYFYdmXX75Uyd3h2X48Eqja+ogTbxUiEhu8XlgaMl1e201c a/dOt8Na9M+HlHPNoPB1WOZnRpuH1HHItNF1aFY9dW1d5EvFrp/V4f4sVArQMFeYg85r DTlw== X-Gm-Message-State: AOAM531ye0hwZi/CcrsXoywsqrRobQ2FxRIN+4+Yx9/v4Qv7+8bzUO/H 0E6b1X5g2td7QwbYzG/5vtVanZbawIJj50sUP+Y87ApmrhGL9G3eTysVNJE6/3aobVv9V3xbqPZ +XQ9vFpVQ14fXKVKy X-Received: by 2002:a50:ff0a:: with SMTP id a10mr436465edu.273.1623965282740; Thu, 17 Jun 2021 14:28:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzAR7MBrW4DjSCrWUTnO3yJRhsZzoQfEPwCrPjITjLEgp/KVyK6TzDvfdi+b7OXK36PkMlVfA== X-Received: by 2002:a50:ff0a:: with SMTP id a10mr436435edu.273.1623965282380; Thu, 17 Jun 2021 14:28:02 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id mm25sm100204ejb.125.2021.06.17.14.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:27:59 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id BD795180734; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Simon Horman , oss-drivers@netronome.com Subject: [PATCH bpf-next v3 11/16] nfp: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:43 +0200 Message-Id: <20210617212748.32456-12-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The nfp driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. While this is not actually an issue for the nfp driver because it doesn't support XDP_REDIRECT (and thus doesn't call xdp_do_flush()), the rcu_read_lock() is still unneeded. And With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Simon Horman Cc: oss-drivers@netronome.com Reviewed-by: Simon Horman Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/netronome/nfp/nfp_net_common.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c index eeb30680b4dc..a3d59abed6ae 100644 --- a/drivers/net/ethernet/netronome/nfp/nfp_net_common.c +++ b/drivers/net/ethernet/netronome/nfp/nfp_net_common.c @@ -1819,7 +1819,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) struct xdp_buff xdp; int idx; - rcu_read_lock(); xdp_prog = READ_ONCE(dp->xdp_prog); true_bufsz = xdp_prog ? PAGE_SIZE : dp->fl_bufsz; xdp_init_buff(&xdp, PAGE_SIZE - NFP_NET_RX_BUF_HEADROOM, @@ -1919,6 +1918,10 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) pkt_off - NFP_NET_RX_BUF_HEADROOM, pkt_len, true); + /* This code is invoked within a single NAPI poll cycle + * and thus under local_bh_disable(), which provides the + * needed RCU protection. + */ act = bpf_prog_run_xdp(xdp_prog, &xdp); pkt_len = xdp.data_end - xdp.data; @@ -2036,7 +2039,6 @@ static int nfp_net_rx(struct nfp_net_rx_ring *rx_ring, int budget) if (!nfp_net_xdp_complete(tx_ring)) pkts_polled = budget; } - rcu_read_unlock(); return pkts_polled; } From patchwork Thu Jun 17 21:27:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462819 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id ACD2CC2B9F4 for ; Thu, 17 Jun 2021 21:28:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9944560D07 for ; Thu, 17 Jun 2021 21:28:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233226AbhFQVaV (ORCPT ); Thu, 17 Jun 2021 17:30:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:54038 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233089AbhFQVaN (ORCPT ); Thu, 17 Jun 2021 17:30:13 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965284; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=2HHxZ70enPSVJrHvmI2lNUfsNsQ5hyZDG+eWNYytmQM=; b=PMT4XWzUBZDWG2f+2uMkyfZd5fEyla64cz85KHk9bNYJ45s1rCJVt6ZVhd2SLNDSGOAQyC +JNoHEPL8QCsnMpVOcUuE8fH8HZ/cOF780EmEPhNQsKN8XhXS6Cq3bqMbw1oD6x8gOzxVj YabyHRcShouDE6Gk9l/nZR9sWvE7I3Y= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-290-pgaYTxFtPmmyzo4euNWy8g-1; Thu, 17 Jun 2021 17:28:03 -0400 X-MC-Unique: pgaYTxFtPmmyzo4euNWy8g-1 Received: by mail-ed1-f71.google.com with SMTP id x8-20020aa7d3880000b029038fe468f5f4so2372187edq.10 for ; Thu, 17 Jun 2021 14:28:02 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=2HHxZ70enPSVJrHvmI2lNUfsNsQ5hyZDG+eWNYytmQM=; b=MoiKTGnv+XuZhqf675jyDauJ2Q6xKnb4D9I32D5LKov1jhUToAM8fxccdCD3G8HfLB kvRNCVaWJ+fzMp8uhS6MomzSvvkvickhrbzh4W4+fGFFETc0WB0/LQUOalNTU6rUUGZY WVExzQ4WMNVp00NgihmQFoKZ2ouGDP+sV/iRYpO556mXb0NaAyBCNn7ORc3IwaC3++QW KHXsduwwQJYNw4NPUjiKWHrE3PrTp2fUZMubeVk82OimPWtRe0aeO1dT67Uh0mUEDFKx NlD7u8o22WODv5A5Zud9y824BScpBwzVxsBKV02p7a37fp1NpEodVwrYKgpihMAGuG+i h2Og== X-Gm-Message-State: AOAM532cDEKy53L5uf7OconRv4e9e8Yu8me9wJOtsd1RoA5rK+p9EyTs nuNsiIjlJXDm8Hv/+yqAo7qTf234szYaqJFYvInSaQZDPgWNojbGY8vQnN7fjBBICTKwlBXdlWo Iu+P8f4cJEbn42rjZ X-Received: by 2002:a05:6402:27d0:: with SMTP id c16mr475911ede.60.1623965282046; Thu, 17 Jun 2021 14:28:02 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzo+0no19/AfrC5WVLy9yZXt8Bo09UqWLQ/taswl5RGBhcCnNAeCwZslfDXtE9wkSElMgquYg== X-Received: by 2002:a05:6402:27d0:: with SMTP id c16mr475898ede.60.1623965281944; Thu, 17 Jun 2021 14:28:01 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([45.145.92.2]) by smtp.gmail.com with ESMTPSA id q15sm5324872edr.84.2021.06.17.14.27.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:28:01 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id D5B79180737; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Jassi Brar , Ilias Apalodimas Subject: [PATCH bpf-next v3 14/16] netsec: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:46 +0200 Message-Id: <20210617212748.32456-15-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The netsec driver has a rcu_read_lock()/rcu_read_unlock() pair around the full RX loop, covering everything up to and including xdp_do_flush(). This is actually the correct behaviour, but because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), it is also technically redundant. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep the rcu_read_lock() around anymore, so let's just remove it. Cc: Jassi Brar Cc: Ilias Apalodimas Acked-by: Ilias Apalodimas Signed-off-by: Toke Høiland-Jørgensen --- drivers/net/ethernet/socionext/netsec.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index dfc85cc68173..e07b7c870046 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -958,7 +958,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) xdp_init_buff(&xdp, PAGE_SIZE, &dring->xdp_rxq); - rcu_read_lock(); xdp_prog = READ_ONCE(priv->xdp_prog); dma_dir = page_pool_get_dma_dir(dring->page_pool); @@ -1019,6 +1018,10 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) pkt_len, false); if (xdp_prog) { + /* This code is invoked within a single NAPI poll cycle + * and thus under local_bh_disable(), which provides the + * needed RCU protection. + */ xdp_result = netsec_run_xdp(priv, xdp_prog, &xdp); if (xdp_result != NETSEC_XDP_PASS) { xdp_act |= xdp_result; @@ -1069,8 +1072,6 @@ static int netsec_process_rx(struct netsec_priv *priv, int budget) } netsec_finalize_xdp_rx(priv, xdp_act, xdp_xmit); - rcu_read_unlock(); - return done; } From patchwork Thu Jun 17 21:27:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= X-Patchwork-Id: 462820 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER, INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BE0FC49EA3 for ; Thu, 17 Jun 2021 21:28:09 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88405611BE for ; Thu, 17 Jun 2021 21:28:09 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233161AbhFQVaQ (ORCPT ); Thu, 17 Jun 2021 17:30:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:24174 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233045AbhFQVaL (ORCPT ); Thu, 17 Jun 2021 17:30:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1623965283; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=h2Vi9XceSPVZfLpkffqV3pCVibBmY0ByVpnQRv2lfdE=; b=YS+1cSJxvroIY0LLcif7PJxPOT5xDNu+a1+KYWHTjA4lbIf9RJlCEXH6Ik5rzBa8aB4xL6 xtDtwo1a8kuT5kguHB3gAC6nzRs8xLREqI02OkwjHGLQ3sGnczXLHCVcIIgk6m/IvS7MLU 6ByvBVcrlQUiau+Ng7snyZ+hbTKsTP8= Received: from mail-ej1-f72.google.com (mail-ej1-f72.google.com [209.85.218.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-465-Wndln36GNJ25wPqYgM2_cA-1; Thu, 17 Jun 2021 17:28:02 -0400 X-MC-Unique: Wndln36GNJ25wPqYgM2_cA-1 Received: by mail-ej1-f72.google.com with SMTP id u4-20020a1709061244b02904648b302151so3021096eja.17 for ; Thu, 17 Jun 2021 14:28:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=h2Vi9XceSPVZfLpkffqV3pCVibBmY0ByVpnQRv2lfdE=; b=sp6uiEzwGjlZsHxNtPobbQRtrl+3njOnBhOxPuVDhuRBNUvuOGt7j1UR6ymuITXcjN 7+z+u4U67fK8t3je0QW42VkTrjm6WqOXvchcgee9RJI+3VeNs9fuPl5jKSFTAz4niopf zCXTccSKtwZCkkMrvKJPu3bjLykoSljDMjTsXQkT/OjNex0Gut2I5wI9PO/1FbRCtXJa URbNtmUB2eZrAjvE0ILQ+PjTATd5a0nvqRsB+1HMBthEfcnnXh0TSY8DRFr7qCldQKBt IJLpywTN7UGcq/9EU2nkPKGOHyAvVdkRdEs8W2Zw6oiWzb/ZGFhdt+OiAnWgTC6YLi78 iuSA== X-Gm-Message-State: AOAM533eZnv7ENNXY4Q/mCjocYzdc3DaH0f2bRLGHSVpu0BeeTDCCLib u4zqY3jGQKlKJkFnhEGyHlOa249XK1WkNWLZbutw6eT6m6vi9VNJ0LPHD098jp6nHrNq7AGHXI2 CCdkpw9KKbW7QtmCG X-Received: by 2002:a05:6402:42cc:: with SMTP id i12mr418827edc.345.1623965280725; Thu, 17 Jun 2021 14:28:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx7GdrC2OmFDGjTuzCjTi7XENVhneXkF+uyTA4qSW3+SnXwzyHh4ZUMgJUipj8lkd9TnOv6ZA== X-Received: by 2002:a05:6402:42cc:: with SMTP id i12mr418777edc.345.1623965280307; Thu, 17 Jun 2021 14:28:00 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a0c:4d80:42:443::2]) by smtp.gmail.com with ESMTPSA id hg25sm108948ejc.51.2021.06.17.14.27.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 17 Jun 2021 14:27:59 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id DC3F4180738; Thu, 17 Jun 2021 23:27:54 +0200 (CEST) From: =?utf-8?q?Toke_H=C3=B8iland-J=C3=B8rgensen?= To: bpf@vger.kernel.org, netdev@vger.kernel.org Cc: Martin KaFai Lau , Hangbin Liu , Jesper Dangaard Brouer , Magnus Karlsson , "Paul E . McKenney" , Jakub Kicinski , =?utf-8?q?Toke_H=C3=B8iland-?= =?utf-8?q?J=C3=B8rgensen?= , Giuseppe Cavallaro , Alexandre Torgue , Jose Abreu Subject: [PATCH bpf-next v3 15/16] stmmac: remove rcu_read_lock() around XDP program invocation Date: Thu, 17 Jun 2021 23:27:47 +0200 Message-Id: <20210617212748.32456-16-toke@redhat.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20210617212748.32456-1-toke@redhat.com> References: <20210617212748.32456-1-toke@redhat.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org The stmmac driver has rcu_read_lock()/rcu_read_unlock() pairs around XDP program invocations. However, the actual lifetime of the objects referred by the XDP program invocation is longer, all the way through to the call to xdp_do_flush(), making the scope of the rcu_read_lock() too small. This turns out to be harmless because it all happens in a single NAPI poll cycle (and thus under local_bh_disable()), but it makes the rcu_read_lock() misleading. Rather than extend the scope of the rcu_read_lock(), just get rid of it entirely. With the addition of RCU annotations to the XDP_REDIRECT map types that take bh execution into account, lockdep even understands this to be safe, so there's really no reason to keep it around. Cc: Giuseppe Cavallaro Cc: Alexandre Torgue Cc: Jose Abreu Signed-off-by: Toke Høiland-Jørgensen Acked-by: Wong Vee Khee Tested-by: Song, Yoong Siang --- drivers/net/ethernet/stmicro/stmmac/stmmac_main.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index bf9fe25fed69..5dcc8a42abf9 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -4654,7 +4654,6 @@ static int stmmac_xdp_xmit_back(struct stmmac_priv *priv, return res; } -/* This function assumes rcu_read_lock() is held by the caller. */ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog, struct xdp_buff *xdp) @@ -4662,6 +4661,9 @@ static int __stmmac_xdp_run_prog(struct stmmac_priv *priv, u32 act; int res; + /* This code is invoked within a single NAPI poll cycle and thus under + * local_bh_disable(), which provides the needed RCU protection. + */ act = bpf_prog_run_xdp(prog, xdp); switch (act) { case XDP_PASS: @@ -4696,17 +4698,14 @@ static struct sk_buff *stmmac_xdp_run_prog(struct stmmac_priv *priv, struct bpf_prog *prog; int res; - rcu_read_lock(); - prog = READ_ONCE(priv->xdp_prog); if (!prog) { res = STMMAC_XDP_PASS; - goto unlock; + goto out; } res = __stmmac_xdp_run_prog(priv, prog, xdp); -unlock: - rcu_read_unlock(); +out: return ERR_PTR(-res); } @@ -4976,10 +4975,8 @@ static int stmmac_rx_zc(struct stmmac_priv *priv, int limit, u32 queue) buf->xdp->data_end = buf->xdp->data + buf1_len; xsk_buff_dma_sync_for_cpu(buf->xdp, rx_q->xsk_pool); - rcu_read_lock(); prog = READ_ONCE(priv->xdp_prog); res = __stmmac_xdp_run_prog(priv, prog, buf->xdp); - rcu_read_unlock(); switch (res) { case STMMAC_XDP_PASS: