| Message ID | 20210125232023.78649-1-ljp@linux.ibm.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [net,v2] ibmvnic: Ensure that CRQ entry read are correctly ordered | expand |
On Mon, 25 Jan 2021 17:20:23 -0600 Lijun Pan wrote: > Ensure that received Command-Response Queue (CRQ) entries are > properly read in order by the driver. dma_rmb barrier has > been added before accessing the CRQ descriptor to ensure > the entire descriptor is read before processing. > > Fixes: 032c5e82847a ("Driver for IBM System i/p VNIC protocol") > Signed-off-by: Lijun Pan <ljp@linux.ibm.com> > --- > v2: drop dma_wmb according to Jakub's opinion > > drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++ > 1 file changed, 8 insertions(+) > > diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c > index 9778c83150f1..d84369bd5fc9 100644 > --- a/drivers/net/ethernet/ibm/ibmvnic.c > +++ b/drivers/net/ethernet/ibm/ibmvnic.c > @@ -5084,6 +5084,14 @@ static void ibmvnic_tasklet(struct tasklet_struct *t) > while (!done) { > /* Pull all the valid messages off the CRQ */ > while ((crq = ibmvnic_next_crq(adapter)) != NULL) { > + /* Ensure that the entire CRQ descriptor queue->msgs > + * has been loaded before reading its contents. I still find this sentence confusing, maybe you mean to say stored instead of loaded? > + * This barrier makes sure ibmvnic_next_crq()'s > + * crq->generic.first & IBMVNIC_CRQ_CMD_RSP is loaded > + * before ibmvnic_handle_crq()'s > + * switch(gen_crq->first) and switch(gen_crq->cmd). Yup, that makes perfect sense. It's about ordering of the loads. > + */ > + dma_rmb(); > ibmvnic_handle_crq(crq, adapter); > crq->generic.first = 0; > }
On Wed, Jan 27, 2021 at 7:06 PM Jakub Kicinski <kuba@kernel.org> wrote: > > On Mon, 25 Jan 2021 17:20:23 -0600 Lijun Pan wrote: > > Ensure that received Command-Response Queue (CRQ) entries are > > properly read in order by the driver. dma_rmb barrier has > > been added before accessing the CRQ descriptor to ensure > > the entire descriptor is read before processing. > > > > Fixes: 032c5e82847a ("Driver for IBM System i/p VNIC protocol") > > Signed-off-by: Lijun Pan <ljp@linux.ibm.com> > > --- > > v2: drop dma_wmb according to Jakub's opinion > > > > drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++ > > 1 file changed, 8 insertions(+) > > > > diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c > > index 9778c83150f1..d84369bd5fc9 100644 > > --- a/drivers/net/ethernet/ibm/ibmvnic.c > > +++ b/drivers/net/ethernet/ibm/ibmvnic.c > > @@ -5084,6 +5084,14 @@ static void ibmvnic_tasklet(struct tasklet_struct *t) > > while (!done) { > > /* Pull all the valid messages off the CRQ */ > > while ((crq = ibmvnic_next_crq(adapter)) != NULL) { > > + /* Ensure that the entire CRQ descriptor queue->msgs > > + * has been loaded before reading its contents. > > I still find this sentence confusing, maybe you mean to say stored > instead of loaded? The above 2 lines are the general description. The below 4 lines are detailed explanations. If it is still confusing, we can delete the above 2 lines of comments. > > > + * This barrier makes sure ibmvnic_next_crq()'s > > + * crq->generic.first & IBMVNIC_CRQ_CMD_RSP is loaded > > + * before ibmvnic_handle_crq()'s > > + * switch(gen_crq->first) and switch(gen_crq->cmd). > > Yup, that makes perfect sense. It's about ordering of the loads. > > > + */ > > + dma_rmb(); > > ibmvnic_handle_crq(crq, adapter); > > crq->generic.first = 0; > > } >
On Wed, 27 Jan 2021 19:22:25 -0600 Lijun Pan wrote: > On Wed, Jan 27, 2021 at 7:06 PM Jakub Kicinski <kuba@kernel.org> wrote: > > > > On Mon, 25 Jan 2021 17:20:23 -0600 Lijun Pan wrote: > > > Ensure that received Command-Response Queue (CRQ) entries are > > > properly read in order by the driver. dma_rmb barrier has > > > been added before accessing the CRQ descriptor to ensure > > > the entire descriptor is read before processing. > > > > > > Fixes: 032c5e82847a ("Driver for IBM System i/p VNIC protocol") > > > Signed-off-by: Lijun Pan <ljp@linux.ibm.com> > > > --- > > > v2: drop dma_wmb according to Jakub's opinion > > > > > > drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++ > > > 1 file changed, 8 insertions(+) > > > > > > diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c > > > index 9778c83150f1..d84369bd5fc9 100644 > > > --- a/drivers/net/ethernet/ibm/ibmvnic.c > > > +++ b/drivers/net/ethernet/ibm/ibmvnic.c > > > @@ -5084,6 +5084,14 @@ static void ibmvnic_tasklet(struct tasklet_struct *t) > > > while (!done) { > > > /* Pull all the valid messages off the CRQ */ > > > while ((crq = ibmvnic_next_crq(adapter)) != NULL) { > > > + /* Ensure that the entire CRQ descriptor queue->msgs > > > + * has been loaded before reading its contents. > > > > I still find this sentence confusing, maybe you mean to say stored > > instead of loaded? > > The above 2 lines are the general description. The below 4 lines are > detailed explanations. If it is still confusing, we can delete the above > 2 lines of comments. Yes, I'd find the comment clearer without them, thanks.
On Wed, Jan 27, 2021 at 7:31 PM Jakub Kicinski <kuba@kernel.org> wrote: > > On Wed, 27 Jan 2021 19:22:25 -0600 Lijun Pan wrote: > > On Wed, Jan 27, 2021 at 7:06 PM Jakub Kicinski <kuba@kernel.org> wrote: > > > > > > On Mon, 25 Jan 2021 17:20:23 -0600 Lijun Pan wrote: > > > > Ensure that received Command-Response Queue (CRQ) entries are > > > > properly read in order by the driver. dma_rmb barrier has > > > > been added before accessing the CRQ descriptor to ensure > > > > the entire descriptor is read before processing. > > > > > > > > Fixes: 032c5e82847a ("Driver for IBM System i/p VNIC protocol") > > > > Signed-off-by: Lijun Pan <ljp@linux.ibm.com> > > > > --- > > > > v2: drop dma_wmb according to Jakub's opinion > > > > > > > > drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++ > > > > 1 file changed, 8 insertions(+) > > > > > > > > diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c > > > > index 9778c83150f1..d84369bd5fc9 100644 > > > > --- a/drivers/net/ethernet/ibm/ibmvnic.c > > > > +++ b/drivers/net/ethernet/ibm/ibmvnic.c > > > > @@ -5084,6 +5084,14 @@ static void ibmvnic_tasklet(struct tasklet_struct *t) > > > > while (!done) { > > > > /* Pull all the valid messages off the CRQ */ > > > > while ((crq = ibmvnic_next_crq(adapter)) != NULL) { > > > > + /* Ensure that the entire CRQ descriptor queue->msgs > > > > + * has been loaded before reading its contents. > > > > > > I still find this sentence confusing, maybe you mean to say stored > > > instead of loaded? > > > > The above 2 lines are the general description. The below 4 lines are > > detailed explanations. If it is still confusing, we can delete the above > > 2 lines of comments. > > Yes, I'd find the comment clearer without them, thanks. v3 has been sent out: https://patchwork.kernel.org/project/netdevbpf/patch/20210128013442.88319-1-ljp@linux.ibm.com/ Thanks, Lijun
diff --git a/drivers/net/ethernet/ibm/ibmvnic.c b/drivers/net/ethernet/ibm/ibmvnic.c index 9778c83150f1..d84369bd5fc9 100644 --- a/drivers/net/ethernet/ibm/ibmvnic.c +++ b/drivers/net/ethernet/ibm/ibmvnic.c @@ -5084,6 +5084,14 @@ static void ibmvnic_tasklet(struct tasklet_struct *t) while (!done) { /* Pull all the valid messages off the CRQ */ while ((crq = ibmvnic_next_crq(adapter)) != NULL) { + /* Ensure that the entire CRQ descriptor queue->msgs + * has been loaded before reading its contents. + * This barrier makes sure ibmvnic_next_crq()'s + * crq->generic.first & IBMVNIC_CRQ_CMD_RSP is loaded + * before ibmvnic_handle_crq()'s + * switch(gen_crq->first) and switch(gen_crq->cmd). + */ + dma_rmb(); ibmvnic_handle_crq(crq, adapter); crq->generic.first = 0; }
Ensure that received Command-Response Queue (CRQ) entries are properly read in order by the driver. dma_rmb barrier has been added before accessing the CRQ descriptor to ensure the entire descriptor is read before processing. Fixes: 032c5e82847a ("Driver for IBM System i/p VNIC protocol") Signed-off-by: Lijun Pan <ljp@linux.ibm.com> --- v2: drop dma_wmb according to Jakub's opinion drivers/net/ethernet/ibm/ibmvnic.c | 8 ++++++++ 1 file changed, 8 insertions(+)