| Message ID | 20210717204057.67495-4-martin.blumenstingl@googlemail.com |
|---|---|
| State | New |
| Headers | show |
| Series | rtw88: prepare locking for SDIO support | expand |
On Sat, 2021-07-17 at 22:40 +0200, Martin Blumenstingl wrote: > > --- a/drivers/net/wireless/realtek/rtw88/mac80211.c > +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c > @@ -721,7 +721,7 @@ static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, > br_data.rtwdev = rtwdev; > br_data.vif = vif; > br_data.mask = mask; > - rtw_iterate_stas_atomic(rtwdev, rtw_ra_mask_info_update_iter, &br_data); > + rtw_iterate_stas(rtwdev, rtw_ra_mask_info_update_iter, &br_data); And then you pretty much immediately break that invariant here, namely that you're calling this within the set_bitrate_mask() method called by mac80211. That's not actually fundamentally broken today, but it does *severely* restrict what we can do in mac80211 wrt. locking, and I really don't want to keep the dozen or so locks forever, this needs simplification because clearly we don't even know what should be under what lock. So like I said on the other patch, I don't have a fundamental objection to taking such a patch, but the locking mess that this gets us into is something I'd rather not have. Maybe just don't support set_bitrate_mask for SDIO drivers for now? The other cases look OK, it's being called from outside contexts (wowlan, etc.) johannes
Hi Johannes, Hi Ping-Ke, On Mon, Jul 19, 2021 at 8:36 AM Johannes Berg <johannes@sipsolutions.net> wrote: > > On Sat, 2021-07-17 at 22:40 +0200, Martin Blumenstingl wrote: > > > > --- a/drivers/net/wireless/realtek/rtw88/mac80211.c > > +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c > > @@ -721,7 +721,7 @@ static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, > > br_data.rtwdev = rtwdev; > > br_data.vif = vif; > > br_data.mask = mask; > > - rtw_iterate_stas_atomic(rtwdev, rtw_ra_mask_info_update_iter, &br_data); > > + rtw_iterate_stas(rtwdev, rtw_ra_mask_info_update_iter, &br_data); > > And then you pretty much immediately break that invariant here, namely > that you're calling this within the set_bitrate_mask() method called by > mac80211. you are right, I was not aware of this > That's not actually fundamentally broken today, but it does *severely* > restrict what we can do in mac80211 wrt. locking, and I really don't > want to keep the dozen or so locks forever, this needs simplification > because clearly we don't even know what should be under what lock. To me it's also not clear what the goal of the whole locking is. The lock in ieee80211_iterate_stations_atomic is obviously for the mac80211-internal state-machine But I *believe* that there's a second purpose (rtw88 specific) - here's my understanding of that part: - rtw_sta_info contains a "mac_id" which is an identifier for a specific station used by the rtw88 driver and is shared with the firmware - rtw_ops_sta_{add,remove} uses rtwdev->mutex to protect the rtw88 side of this "mac_id" identifier - (for some reason rtw_update_sta_info doesn't use rtwdev->mutex) So now I am wondering if the ieee80211_iterate_stations_atomic lock is also used to protect any modifications to rtw_sta_info. Ping-Ke, I am wondering if the attached patch (untested - to better demonstrate what I want to say) would: - allow us to move the register write outside of ieee80211_iterate_stations_atomic - mean we can keep ieee80211_iterate_stations_atomic (instead of the non-atomic variant) - protect the code managing the "mac_id" with rtwdev->mutex consistently > The other cases look OK, it's being called from outside contexts > (wowlan, etc.) Thanks for reviewing this Johannes! Best regards, Martin diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c index 7650a1ca0e9e..be39c6d0ee31 100644 --- a/drivers/net/wireless/realtek/rtw88/mac80211.c +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c @@ -689,6 +689,8 @@ struct rtw_iter_bitrate_mask_data { struct rtw_dev *rtwdev; struct ieee80211_vif *vif; const struct cfg80211_bitrate_mask *mask; + unsigned int num_si; + struct rtw_sta_info *si[RTW_MAX_MAC_ID_NUM]; }; static void rtw_ra_mask_info_update_iter(void *data, struct ieee80211_sta *sta) @@ -709,7 +711,8 @@ static void rtw_ra_mask_info_update_iter(void *data, struct ieee80211_sta *sta) } si->use_cfg_mask = true; - rtw_update_sta_info(br_data->rtwdev, si); + + br_data->si[br_data->num_si++] = si; } static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, @@ -717,11 +720,20 @@ static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, const struct cfg80211_bitrate_mask *mask) { struct rtw_iter_bitrate_mask_data br_data; + unsigned int i; + + mutex_lock(&rtwdev->mutex); br_data.rtwdev = rtwdev; br_data.vif = vif; br_data.mask = mask; - rtw_iterate_stas(rtwdev, rtw_ra_mask_info_update_iter, &br_data); + br_data.num_si = 0; + rtw_iterate_stas_atomic(rtwdev, rtw_ra_mask_info_update_iter, &br_data); + + for (i = 0; i < br_data.num_si; i++) + rtw_update_sta_info(rtwdev, br_data.si[i]); + + mutex_unlock(&rtwdev->mutex); } static int rtw_ops_set_bitrate_mask(struct ieee80211_hw *hw,
> -----Original Message----- > From: Martin Blumenstingl [mailto:martin.blumenstingl@googlemail.com] > Sent: Monday, July 26, 2021 5:51 AM > To: Johannes Berg; Pkshih > Cc: linux-wireless@vger.kernel.org; tony0620emma@gmail.com; kvalo@codeaurora.org; > netdev@vger.kernel.org; linux-kernel@vger.kernel.org; Neo Jou; Jernej Skrabec > Subject: Re: [PATCH RFC v1 3/7] rtw88: Use rtw_iterate_stas where the iterator reads or writes registers > > Hi Johannes, Hi Ping-Ke, > > On Mon, Jul 19, 2021 at 8:36 AM Johannes Berg <johannes@sipsolutions.net> wrote: > > > > On Sat, 2021-07-17 at 22:40 +0200, Martin Blumenstingl wrote: > > > > > > --- a/drivers/net/wireless/realtek/rtw88/mac80211.c > > > +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c > > > @@ -721,7 +721,7 @@ static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, > > > br_data.rtwdev = rtwdev; > > > br_data.vif = vif; > > > br_data.mask = mask; > > > - rtw_iterate_stas_atomic(rtwdev, rtw_ra_mask_info_update_iter, &br_data); > > > + rtw_iterate_stas(rtwdev, rtw_ra_mask_info_update_iter, &br_data); > > > > And then you pretty much immediately break that invariant here, namely > > that you're calling this within the set_bitrate_mask() method called by > > mac80211. > you are right, I was not aware of this > > > That's not actually fundamentally broken today, but it does *severely* > > restrict what we can do in mac80211 wrt. locking, and I really don't > > want to keep the dozen or so locks forever, this needs simplification > > because clearly we don't even know what should be under what lock. > To me it's also not clear what the goal of the whole locking is. > The lock in ieee80211_iterate_stations_atomic is obviously for the > mac80211-internal state-machine > But I *believe* that there's a second purpose (rtw88 specific) - > here's my understanding of that part: > - rtw_sta_info contains a "mac_id" which is an identifier for a > specific station used by the rtw88 driver and is shared with the > firmware > - rtw_ops_sta_{add,remove} uses rtwdev->mutex to protect the rtw88 > side of this "mac_id" identifier > - (for some reason rtw_update_sta_info doesn't use rtwdev->mutex) I am thinking rtw88 needs to maintain sta and vif lists itself, and these lists are also protected by rtwdev->mutex. When rtw88 wants to iterate all sta/vif, it holds rtwdev->mutex to do list_for_each_entry. No need to hold mac80211 locks. > > So now I am wondering if the ieee80211_iterate_stations_atomic lock is > also used to protect any modifications to rtw_sta_info. > Ping-Ke, I am wondering if the attached patch (untested - to better > demonstrate what I want to say) would: > - allow us to move the register write outside of > ieee80211_iterate_stations_atomic > - mean we can keep ieee80211_iterate_stations_atomic (instead of the > non-atomic variant) > - protect the code managing the "mac_id" with rtwdev->mutex consistently I think your attached patch can work well. > > > The other cases look OK, it's being called from outside contexts > > (wowlan, etc.) > Thanks for reviewing this Johannes! > -- Ping-Ke
> > I am thinking rtw88 needs to maintain sta and vif lists itself, I would tend to prefer drivers do not maintain separate lists - that's just duplicated book-keeping and prone to state mismatch errors? But OTOH the locking does make things complex. johannes
diff --git a/drivers/net/wireless/realtek/rtw88/mac80211.c b/drivers/net/wireless/realtek/rtw88/mac80211.c index 6f5629852416..7650a1ca0e9e 100644 --- a/drivers/net/wireless/realtek/rtw88/mac80211.c +++ b/drivers/net/wireless/realtek/rtw88/mac80211.c @@ -721,7 +721,7 @@ static void rtw_ra_mask_info_update(struct rtw_dev *rtwdev, br_data.rtwdev = rtwdev; br_data.vif = vif; br_data.mask = mask; - rtw_iterate_stas_atomic(rtwdev, rtw_ra_mask_info_update_iter, &br_data); + rtw_iterate_stas(rtwdev, rtw_ra_mask_info_update_iter, &br_data); } static int rtw_ops_set_bitrate_mask(struct ieee80211_hw *hw, diff --git a/drivers/net/wireless/realtek/rtw88/main.c b/drivers/net/wireless/realtek/rtw88/main.c index 207161a8f5bd..6e0d25f0afe3 100644 --- a/drivers/net/wireless/realtek/rtw88/main.c +++ b/drivers/net/wireless/realtek/rtw88/main.c @@ -577,7 +577,7 @@ static void __fw_recovery_work(struct rtw_dev *rtwdev) rcu_read_lock(); rtw_iterate_keys_rcu(rtwdev, NULL, rtw_reset_key_iter, rtwdev); rcu_read_unlock(); - rtw_iterate_stas_atomic(rtwdev, rtw_reset_sta_iter, rtwdev); + rtw_iterate_stas(rtwdev, rtw_reset_sta_iter, rtwdev); rtw_iterate_vifs(rtwdev, rtw_reset_vif_iter, rtwdev); rtw_enter_ips(rtwdev); } diff --git a/drivers/net/wireless/realtek/rtw88/phy.c b/drivers/net/wireless/realtek/rtw88/phy.c index 569dd3cfde35..8f2827ecb514 100644 --- a/drivers/net/wireless/realtek/rtw88/phy.c +++ b/drivers/net/wireless/realtek/rtw88/phy.c @@ -240,7 +240,7 @@ static void rtw_phy_stat_rssi(struct rtw_dev *rtwdev) data.rtwdev = rtwdev; data.min_rssi = U8_MAX; - rtw_iterate_stas_atomic(rtwdev, rtw_phy_stat_rssi_iter, &data); + rtw_iterate_stas(rtwdev, rtw_phy_stat_rssi_iter, &data); dm_info->pre_min_rssi = dm_info->min_rssi; dm_info->min_rssi = data.min_rssi; @@ -484,7 +484,7 @@ static void rtw_phy_ra_info_update(struct rtw_dev *rtwdev) if (rtwdev->watch_dog_cnt & 0x3) return; - rtw_iterate_stas_atomic(rtwdev, rtw_phy_ra_info_update_iter, rtwdev); + rtw_iterate_stas(rtwdev, rtw_phy_ra_info_update_iter, rtwdev); } static u32 rtw_phy_get_rrsr_mask(struct rtw_dev *rtwdev, u8 rate_idx) diff --git a/drivers/net/wireless/realtek/rtw88/util.h b/drivers/net/wireless/realtek/rtw88/util.h index 0c23b5069be0..b0dfadf8b82a 100644 --- a/drivers/net/wireless/realtek/rtw88/util.h +++ b/drivers/net/wireless/realtek/rtw88/util.h @@ -13,6 +13,8 @@ struct rtw_dev; #define rtw_iterate_vifs_atomic(rtwdev, iterator, data) \ ieee80211_iterate_active_interfaces_atomic(rtwdev->hw, \ IEEE80211_IFACE_ITER_NORMAL, iterator, data) +#define rtw_iterate_stas(rtwdev, iterator, data) \ + ieee80211_iterate_stations(rtwdev->hw, iterator, data) #define rtw_iterate_stas_atomic(rtwdev, iterator, data) \ ieee80211_iterate_stations_atomic(rtwdev->hw, iterator, data) #define rtw_iterate_keys(rtwdev, vif, iterator, data) \ diff --git a/drivers/net/wireless/realtek/rtw88/wow.c b/drivers/net/wireless/realtek/rtw88/wow.c index fc9544f4e5e4..9c4050d4c6e2 100644 --- a/drivers/net/wireless/realtek/rtw88/wow.c +++ b/drivers/net/wireless/realtek/rtw88/wow.c @@ -429,7 +429,7 @@ static void rtw_wow_fw_media_status(struct rtw_dev *rtwdev, bool connect) data.rtwdev = rtwdev; data.connect = connect; - rtw_iterate_stas_atomic(rtwdev, rtw_wow_fw_media_status_iter, &data); + rtw_iterate_stas(rtwdev, rtw_wow_fw_media_status_iter, &data); } static void rtw_wow_config_pno_rsvd_page(struct rtw_dev *rtwdev,
Upcoming SDIO support may sleep in the read/write handlers. Switch all users of rtw_iterate_stas_atomic() which are either reading or writing a register to rtw_iterate_stas(). Signed-off-by: Martin Blumenstingl <martin.blumenstingl@googlemail.com> --- drivers/net/wireless/realtek/rtw88/mac80211.c | 2 +- drivers/net/wireless/realtek/rtw88/main.c | 2 +- drivers/net/wireless/realtek/rtw88/phy.c | 4 ++-- drivers/net/wireless/realtek/rtw88/util.h | 2 ++ drivers/net/wireless/realtek/rtw88/wow.c | 2 +- 5 files changed, 7 insertions(+), 5 deletions(-)