| Message ID | 20240203023645.31105-36-quic_wcheng@quicinc.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Introduce QC USB SND audio offloading support | expand |
On Sat, 03 Feb 2024 03:36:27 +0100, Wesley Cheng wrote: > > With USB audio offloading, an audio session is started from the ASoC > platform sound card and PCM devices. Likewise, the USB SND path is still > readily available for use, in case the non-offload path is desired. In > order to prevent the two entities from attempting to use the USB bus, > introduce a flag that determines when either paths are in use. > > If a PCM device is already in use, the check will return an error to > userspace notifying that the stream is currently busy. This ensures that > only one path is using the USB substream. > > Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> Hm, I'm not sure whether it's safe to hold chip->mutex there for the long code path. It even kicks off the auto-resume, which may call various functions at resuming, and some of them may re-hold chip->mutex. If it's only about the open flag, protect only the flag access with the mutex, not covering the all open function. At least the re-entry can be avoided by that. thanks, Takashi > --- > sound/usb/card.h | 1 + > sound/usb/pcm.c | 19 +++++++++++++++++-- > 2 files changed, 18 insertions(+), 2 deletions(-) > > diff --git a/sound/usb/card.h b/sound/usb/card.h > index ed4a664e24e5..6d59995440c3 100644 > --- a/sound/usb/card.h > +++ b/sound/usb/card.h > @@ -165,6 +165,7 @@ struct snd_usb_substream { > unsigned int pkt_offset_adj; /* Bytes to drop from beginning of packets (for non-compliant devices) */ > unsigned int stream_offset_adj; /* Bytes to drop from beginning of stream (for non-compliant devices) */ > > + unsigned int opened:1; /* pcm device opened */ > unsigned int running: 1; /* running status */ > unsigned int period_elapsed_pending; /* delay period handling */ > > diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c > index 3adb09ce1702..c2cb52cd5d23 100644 > --- a/sound/usb/pcm.c > +++ b/sound/usb/pcm.c > @@ -1241,8 +1241,15 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) > struct snd_usb_stream *as = snd_pcm_substream_chip(substream); > struct snd_pcm_runtime *runtime = substream->runtime; > struct snd_usb_substream *subs = &as->substream[direction]; > + struct snd_usb_audio *chip = subs->stream->chip; > int ret; > > + mutex_lock(&chip->mutex); > + if (subs->opened) { > + mutex_unlock(&chip->mutex); > + return -EBUSY; > + } > + > runtime->hw = snd_usb_hardware; > /* need an explicit sync to catch applptr update in low-latency mode */ > if (direction == SNDRV_PCM_STREAM_PLAYBACK && > @@ -1259,13 +1266,17 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) > > ret = setup_hw_info(runtime, subs); > if (ret < 0) > - return ret; > + goto out; > ret = snd_usb_autoresume(subs->stream->chip); > if (ret < 0) > - return ret; > + goto out; > ret = snd_media_stream_init(subs, as->pcm, direction); > if (ret < 0) > snd_usb_autosuspend(subs->stream->chip); > + subs->opened = 1; > +out: > + mutex_unlock(&chip->mutex); > + > return ret; > } > > @@ -1274,6 +1285,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) > int direction = substream->stream; > struct snd_usb_stream *as = snd_pcm_substream_chip(substream); > struct snd_usb_substream *subs = &as->substream[direction]; > + struct snd_usb_audio *chip = subs->stream->chip; > int ret; > > snd_media_stop_pipeline(subs); > @@ -1287,6 +1299,9 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) > > subs->pcm_substream = NULL; > snd_usb_autosuspend(subs->stream->chip); > + mutex_lock(&chip->mutex); > + subs->opened = 0; > + mutex_unlock(&chip->mutex); > > return 0; > }
Hi Takashi, On 2/6/2024 5:07 AM, Takashi Iwai wrote: > On Sat, 03 Feb 2024 03:36:27 +0100, > Wesley Cheng wrote: >> >> With USB audio offloading, an audio session is started from the ASoC >> platform sound card and PCM devices. Likewise, the USB SND path is still >> readily available for use, in case the non-offload path is desired. In >> order to prevent the two entities from attempting to use the USB bus, >> introduce a flag that determines when either paths are in use. >> >> If a PCM device is already in use, the check will return an error to >> userspace notifying that the stream is currently busy. This ensures that >> only one path is using the USB substream. >> >> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> > > Hm, I'm not sure whether it's safe to hold chip->mutex there for the > long code path. It even kicks off the auto-resume, which may call > various functions at resuming, and some of them may re-hold > chip->mutex. > That's a good point. > If it's only about the open flag, protect only the flag access with > the mutex, not covering the all open function. At least the re-entry > can be avoided by that. > Sure, let me re-order the check/assignment and the mutex locking. Since this is now checked here in USB PCM and the QC offload driver, we want to make sure that if there was some application attempting to open both at the same time, we prevent any possible races. I think the best way to address this would be something like: static int snd_usb_pcm_open(struct snd_pcm_substream *substream) { ... mutex_lock(&chip->mutex); if (subs->opened) { mutex_unlock(&chip->mutex); return -EBUSY; } subs->opened = 1; mutex_unlock(&chip->mutex); //Execute bulk of PCM open routine ... return 0; // If any errors are seen, unwind err_resume: snd_usb_autosuspend(subs->stream->chip); err_open: mutex_lock(&chip->mutex); subs->opened = 0; mutex_unlock(&chip->mutex); return ret; } Set the opened flag first, so that if QC offload checks it, it can exit early and vice versa. Otherwise, if we set the opened flag at the same position as the previous patch, we may be calling the other routines in parallel to the QC offload enable stream routine. The only thing with this patch is that we'd need some error handling unwinding. Thanks Wesley Cheng
On Wed, 07 Feb 2024 01:08:00 +0100, Wesley Cheng wrote: > > Hi Takashi, > > On 2/6/2024 5:07 AM, Takashi Iwai wrote: > > On Sat, 03 Feb 2024 03:36:27 +0100, > > Wesley Cheng wrote: > >> > >> With USB audio offloading, an audio session is started from the ASoC > >> platform sound card and PCM devices. Likewise, the USB SND path is still > >> readily available for use, in case the non-offload path is desired. In > >> order to prevent the two entities from attempting to use the USB bus, > >> introduce a flag that determines when either paths are in use. > >> > >> If a PCM device is already in use, the check will return an error to > >> userspace notifying that the stream is currently busy. This ensures that > >> only one path is using the USB substream. > >> > >> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> > > > > Hm, I'm not sure whether it's safe to hold chip->mutex there for the > > long code path. It even kicks off the auto-resume, which may call > > various functions at resuming, and some of them may re-hold > > chip->mutex. > > > > That's a good point. > > > If it's only about the open flag, protect only the flag access with > > the mutex, not covering the all open function. At least the re-entry > > can be avoided by that. > > > > Sure, let me re-order the check/assignment and the mutex locking. > Since this is now checked here in USB PCM and the QC offload driver, > we want to make sure that if there was some application attempting to > open both at the same time, we prevent any possible races. > > I think the best way to address this would be something like: > > static int snd_usb_pcm_open(struct snd_pcm_substream *substream) > { > ... > mutex_lock(&chip->mutex); > if (subs->opened) { > mutex_unlock(&chip->mutex); > return -EBUSY; > } > subs->opened = 1; > mutex_unlock(&chip->mutex); > > //Execute bulk of PCM open routine > ... > return 0; > > // If any errors are seen, unwind > err_resume: > snd_usb_autosuspend(subs->stream->chip); > err_open: > mutex_lock(&chip->mutex); > subs->opened = 0; > mutex_unlock(&chip->mutex); > > return ret; > } > > Set the opened flag first, so that if QC offload checks it, it can > exit early and vice versa. Otherwise, if we set the opened flag at > the same position as the previous patch, we may be calling the other > routines in parallel to the QC offload enable stream routine. The > only thing with this patch is that we'd need some error handling > unwinding. The above is what I had in mind. But, thinking on this again, you might be able to get the same result by using the ALSA PCM core substream open_mutex and hw_opened flag. This is already held and set at snd_pcm_core() (the hw_opened flag is set after open callback, though). The offload driver can use those instead of the own lock and flag, too, although it's not really well-mannered behavior (hence you need proper comments). thanks, Takashi
Hi Takashi, On 2/6/2024 11:05 PM, Takashi Iwai wrote: > On Wed, 07 Feb 2024 01:08:00 +0100, > Wesley Cheng wrote: >> >> Hi Takashi, >> >> On 2/6/2024 5:07 AM, Takashi Iwai wrote: >>> On Sat, 03 Feb 2024 03:36:27 +0100, >>> Wesley Cheng wrote: >>>> >>>> With USB audio offloading, an audio session is started from the ASoC >>>> platform sound card and PCM devices. Likewise, the USB SND path is still >>>> readily available for use, in case the non-offload path is desired. In >>>> order to prevent the two entities from attempting to use the USB bus, >>>> introduce a flag that determines when either paths are in use. >>>> >>>> If a PCM device is already in use, the check will return an error to >>>> userspace notifying that the stream is currently busy. This ensures that >>>> only one path is using the USB substream. >>>> >>>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> >>> >>> Hm, I'm not sure whether it's safe to hold chip->mutex there for the >>> long code path. It even kicks off the auto-resume, which may call >>> various functions at resuming, and some of them may re-hold >>> chip->mutex. >>> >> >> That's a good point. >> >>> If it's only about the open flag, protect only the flag access with >>> the mutex, not covering the all open function. At least the re-entry >>> can be avoided by that. >>> >> >> Sure, let me re-order the check/assignment and the mutex locking. >> Since this is now checked here in USB PCM and the QC offload driver, >> we want to make sure that if there was some application attempting to >> open both at the same time, we prevent any possible races. >> >> I think the best way to address this would be something like: >> >> static int snd_usb_pcm_open(struct snd_pcm_substream *substream) >> { >> ... >> mutex_lock(&chip->mutex); >> if (subs->opened) { >> mutex_unlock(&chip->mutex); >> return -EBUSY; >> } >> subs->opened = 1; >> mutex_unlock(&chip->mutex); >> >> //Execute bulk of PCM open routine >> ... >> return 0; >> >> // If any errors are seen, unwind >> err_resume: >> snd_usb_autosuspend(subs->stream->chip); >> err_open: >> mutex_lock(&chip->mutex); >> subs->opened = 0; >> mutex_unlock(&chip->mutex); >> >> return ret; >> } >> >> Set the opened flag first, so that if QC offload checks it, it can >> exit early and vice versa. Otherwise, if we set the opened flag at >> the same position as the previous patch, we may be calling the other >> routines in parallel to the QC offload enable stream routine. The >> only thing with this patch is that we'd need some error handling >> unwinding. > > The above is what I had in mind. > > But, thinking on this again, you might be able to get the same result > by using the ALSA PCM core substream open_mutex and hw_opened flag. > This is already held and set at snd_pcm_core() (the hw_opened flag is > set after open callback, though). The offload driver can use those > instead of the own lock and flag, too, although it's not really > well-mannered behavior (hence you need proper comments). > I think I had looked into this as well previously, and it was difficult to achieve, because from the USB offloading perspective, we don't ever call: snd_usb_pcm_open() This is actually where we populate the pcm_substream parameter within struct snd_usb_substream based on when userspace opens the USB SND PCM device (which is not the case for offloading). So the offload driver doesn't have a way to fetch the struct snd_pcm that is allocated to the PCM device created by the USB SND card. Thanks Wesley Cheng
Hi Takashi, On 2/7/2024 4:02 PM, Wesley Cheng wrote: > Hi Takashi, > > On 2/6/2024 11:05 PM, Takashi Iwai wrote: >> On Wed, 07 Feb 2024 01:08:00 +0100, >> Wesley Cheng wrote: >>> >>> Hi Takashi, >>> >>> On 2/6/2024 5:07 AM, Takashi Iwai wrote: >>>> On Sat, 03 Feb 2024 03:36:27 +0100, >>>> Wesley Cheng wrote: >>>>> >>>>> With USB audio offloading, an audio session is started from the ASoC >>>>> platform sound card and PCM devices. Likewise, the USB SND path is >>>>> still >>>>> readily available for use, in case the non-offload path is >>>>> desired. In >>>>> order to prevent the two entities from attempting to use the USB bus, >>>>> introduce a flag that determines when either paths are in use. >>>>> >>>>> If a PCM device is already in use, the check will return an error to >>>>> userspace notifying that the stream is currently busy. This >>>>> ensures that >>>>> only one path is using the USB substream. >>>>> >>>>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> >>>> >>>> Hm, I'm not sure whether it's safe to hold chip->mutex there for the >>>> long code path. It even kicks off the auto-resume, which may call >>>> various functions at resuming, and some of them may re-hold >>>> chip->mutex. >>>> >>> >>> That's a good point. >>> >>>> If it's only about the open flag, protect only the flag access with >>>> the mutex, not covering the all open function. At least the re-entry >>>> can be avoided by that. >>>> >>> >>> Sure, let me re-order the check/assignment and the mutex locking. >>> Since this is now checked here in USB PCM and the QC offload driver, >>> we want to make sure that if there was some application attempting to >>> open both at the same time, we prevent any possible races. >>> >>> I think the best way to address this would be something like: >>> >>> static int snd_usb_pcm_open(struct snd_pcm_substream *substream) >>> { >>> ... >>> mutex_lock(&chip->mutex); >>> if (subs->opened) { >>> mutex_unlock(&chip->mutex); >>> return -EBUSY; >>> } >>> subs->opened = 1; >>> mutex_unlock(&chip->mutex); >>> >>> //Execute bulk of PCM open routine >>> ... >>> return 0; >>> >>> // If any errors are seen, unwind >>> err_resume: >>> snd_usb_autosuspend(subs->stream->chip); >>> err_open: >>> mutex_lock(&chip->mutex); >>> subs->opened = 0; >>> mutex_unlock(&chip->mutex); >>> >>> return ret; >>> } >>> >>> Set the opened flag first, so that if QC offload checks it, it can >>> exit early and vice versa. Otherwise, if we set the opened flag at >>> the same position as the previous patch, we may be calling the other >>> routines in parallel to the QC offload enable stream routine. The >>> only thing with this patch is that we'd need some error handling >>> unwinding. >> >> The above is what I had in mind. >> >> But, thinking on this again, you might be able to get the same result >> by using the ALSA PCM core substream open_mutex and hw_opened flag. >> This is already held and set at snd_pcm_core() (the hw_opened flag is >> set after open callback, though). The offload driver can use those >> instead of the own lock and flag, too, although it's not really >> well-mannered behavior (hence you need proper comments). >> > > I think I had looked into this as well previously, and it was difficult > to achieve, because from the USB offloading perspective, we don't ever > call: snd_usb_pcm_open() > > This is actually where we populate the pcm_substream parameter within > struct snd_usb_substream based on when userspace opens the USB SND PCM > device (which is not the case for offloading). So the offload driver > doesn't have a way to fetch the struct snd_pcm that is allocated to the > PCM device created by the USB SND card. > Sorry, took a look at it again, and found a way. Although not pretty, we can access it using: subs->stream->pcm->streams[direction].substream->hw_opened Thanks Wesley Cheng
Hi Takashi, On 2/8/2024 12:33 AM, Takashi Iwai wrote: > On Thu, 08 Feb 2024 02:12:00 +0100, > Wesley Cheng wrote: >> >> Hi Takashi, >> >> On 2/7/2024 4:02 PM, Wesley Cheng wrote: >>> Hi Takashi, >>> >>> On 2/6/2024 11:05 PM, Takashi Iwai wrote: >>>> On Wed, 07 Feb 2024 01:08:00 +0100, >>>> Wesley Cheng wrote: >>>>> >>>>> Hi Takashi, >>>>> >>>>> On 2/6/2024 5:07 AM, Takashi Iwai wrote: >>>>>> On Sat, 03 Feb 2024 03:36:27 +0100, >>>>>> Wesley Cheng wrote: >>>>>>> >>>>>>> With USB audio offloading, an audio session is started from the ASoC >>>>>>> platform sound card and PCM devices. Likewise, the USB SND path >>>>>>> is still >>>>>>> readily available for use, in case the non-offload path is >>>>>>> desired. In >>>>>>> order to prevent the two entities from attempting to use the USB bus, >>>>>>> introduce a flag that determines when either paths are in use. >>>>>>> >>>>>>> If a PCM device is already in use, the check will return an error to >>>>>>> userspace notifying that the stream is currently busy. This >>>>>>> ensures that >>>>>>> only one path is using the USB substream. >>>>>>> >>>>>>> Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> >>>>>> >>>>>> Hm, I'm not sure whether it's safe to hold chip->mutex there for the >>>>>> long code path. It even kicks off the auto-resume, which may call >>>>>> various functions at resuming, and some of them may re-hold >>>>>> chip->mutex. >>>>>> >>>>> >>>>> That's a good point. >>>>> >>>>>> If it's only about the open flag, protect only the flag access with >>>>>> the mutex, not covering the all open function. At least the re-entry >>>>>> can be avoided by that. >>>>>> >>>>> >>>>> Sure, let me re-order the check/assignment and the mutex locking. >>>>> Since this is now checked here in USB PCM and the QC offload driver, >>>>> we want to make sure that if there was some application attempting to >>>>> open both at the same time, we prevent any possible races. >>>>> >>>>> I think the best way to address this would be something like: >>>>> >>>>> static int snd_usb_pcm_open(struct snd_pcm_substream *substream) >>>>> { >>>>> ... >>>>> mutex_lock(&chip->mutex); >>>>> if (subs->opened) { >>>>> mutex_unlock(&chip->mutex); >>>>> return -EBUSY; >>>>> } >>>>> subs->opened = 1; >>>>> mutex_unlock(&chip->mutex); >>>>> >>>>> //Execute bulk of PCM open routine >>>>> ... >>>>> return 0; >>>>> >>>>> // If any errors are seen, unwind >>>>> err_resume: >>>>> snd_usb_autosuspend(subs->stream->chip); >>>>> err_open: >>>>> mutex_lock(&chip->mutex); >>>>> subs->opened = 0; >>>>> mutex_unlock(&chip->mutex); >>>>> >>>>> return ret; >>>>> } >>>>> >>>>> Set the opened flag first, so that if QC offload checks it, it can >>>>> exit early and vice versa. Otherwise, if we set the opened flag at >>>>> the same position as the previous patch, we may be calling the other >>>>> routines in parallel to the QC offload enable stream routine. The >>>>> only thing with this patch is that we'd need some error handling >>>>> unwinding. >>>> >>>> The above is what I had in mind. >>>> >>>> But, thinking on this again, you might be able to get the same result >>>> by using the ALSA PCM core substream open_mutex and hw_opened flag. >>>> This is already held and set at snd_pcm_core() (the hw_opened flag is >>>> set after open callback, though). The offload driver can use those >>>> instead of the own lock and flag, too, although it's not really >>>> well-mannered behavior (hence you need proper comments). >>>> >>> >>> I think I had looked into this as well previously, and it was >>> difficult to achieve, because from the USB offloading perspective, >>> we don't ever call: snd_usb_pcm_open() >>> >>> This is actually where we populate the pcm_substream parameter >>> within struct snd_usb_substream based on when userspace opens the >>> USB SND PCM device (which is not the case for offloading). So the >>> offload driver doesn't have a way to fetch the struct snd_pcm that >>> is allocated to the PCM device created by the USB SND card. >>> >> >> Sorry, took a look at it again, and found a way. Although not pretty, >> we can access it using: >> subs->stream->pcm->streams[direction].substream->hw_opened > > Yes, it's not easy to follow it. So if we want to this path, worth > for a detailed comment. That said, I don't mind to introduce the new > local mutex and flag as you did if the above became too messy in the > end. > If you don't mind, I prefer to keep it the way it was with the local mutex and flag. It makes it a lot easier to follow, and for other users to adopt as well compared to the long equation above :). Thanks Wesley Cheng
diff --git a/sound/usb/card.h b/sound/usb/card.h index ed4a664e24e5..6d59995440c3 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -165,6 +165,7 @@ struct snd_usb_substream { unsigned int pkt_offset_adj; /* Bytes to drop from beginning of packets (for non-compliant devices) */ unsigned int stream_offset_adj; /* Bytes to drop from beginning of stream (for non-compliant devices) */ + unsigned int opened:1; /* pcm device opened */ unsigned int running: 1; /* running status */ unsigned int period_elapsed_pending; /* delay period handling */ diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c index 3adb09ce1702..c2cb52cd5d23 100644 --- a/sound/usb/pcm.c +++ b/sound/usb/pcm.c @@ -1241,8 +1241,15 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) struct snd_usb_stream *as = snd_pcm_substream_chip(substream); struct snd_pcm_runtime *runtime = substream->runtime; struct snd_usb_substream *subs = &as->substream[direction]; + struct snd_usb_audio *chip = subs->stream->chip; int ret; + mutex_lock(&chip->mutex); + if (subs->opened) { + mutex_unlock(&chip->mutex); + return -EBUSY; + } + runtime->hw = snd_usb_hardware; /* need an explicit sync to catch applptr update in low-latency mode */ if (direction == SNDRV_PCM_STREAM_PLAYBACK && @@ -1259,13 +1266,17 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) ret = setup_hw_info(runtime, subs); if (ret < 0) - return ret; + goto out; ret = snd_usb_autoresume(subs->stream->chip); if (ret < 0) - return ret; + goto out; ret = snd_media_stream_init(subs, as->pcm, direction); if (ret < 0) snd_usb_autosuspend(subs->stream->chip); + subs->opened = 1; +out: + mutex_unlock(&chip->mutex); + return ret; } @@ -1274,6 +1285,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) int direction = substream->stream; struct snd_usb_stream *as = snd_pcm_substream_chip(substream); struct snd_usb_substream *subs = &as->substream[direction]; + struct snd_usb_audio *chip = subs->stream->chip; int ret; snd_media_stop_pipeline(subs); @@ -1287,6 +1299,9 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) subs->pcm_substream = NULL; snd_usb_autosuspend(subs->stream->chip); + mutex_lock(&chip->mutex); + subs->opened = 0; + mutex_unlock(&chip->mutex); return 0; }
With USB audio offloading, an audio session is started from the ASoC platform sound card and PCM devices. Likewise, the USB SND path is still readily available for use, in case the non-offload path is desired. In order to prevent the two entities from attempting to use the USB bus, introduce a flag that determines when either paths are in use. If a PCM device is already in use, the check will return an error to userspace notifying that the stream is currently busy. This ensures that only one path is using the USB substream. Signed-off-by: Wesley Cheng <quic_wcheng@quicinc.com> --- sound/usb/card.h | 1 + sound/usb/pcm.c | 19 +++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-)