diff mbox series

[v13,09/16] media: uapi: Define audio sample format fourcc type

Message ID 1708936109-11587-10-git-send-email-shengjiu.wang@nxp.com
State Superseded
Headers show
Series Add audio support in v4l2 framework | expand

Commit Message

Shengjiu Wang Feb. 26, 2024, 8:28 a.m. UTC
The audio sample format definition is from alsa,
the header file is include/uapi/sound/asound.h, but
don't include this header file directly, because in
user space, there is another copy in alsa-lib.
There will be conflict in userspace for include
videodev2.h & asound.h and asoundlib.h

Here still use the fourcc format.

Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
---
 .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
 .../userspace-api/media/v4l/pixfmt.rst        |  1 +
 drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
 include/uapi/linux/videodev2.h                | 23 +++++
 4 files changed, 124 insertions(+)
 create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst

Comments

Shengjiu Wang Feb. 27, 2024, 2:24 a.m. UTC | #1
On Mon, Feb 26, 2024 at 9:55 PM Nicolas Dufresne <nicolas@ndufresne.ca>
wrote:
>
> Le lundi 26 février 2024 à 16:28 +0800, Shengjiu Wang a écrit :
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
>
> I'd like to join Mauro's voice that duplicating the audio formats is a
bad idea.
> We have the same issues with video formats when you look at V4L2 vs DRM.
You're
> rationale is that videodev2.h will be ambiguous if it includes asound.h,
but
> looking at this change, there is no reason why you need to include
asound.h in
> videodev2.h at all. The format type can be abstracted out with a uint32
in the
> API, and then it would be up to the users to include and use the
appropriate
> formats IDs.
>

Thanks.

There is another reason mentioned by Hans:
"
















*The problem is that within V4L2 we use fourcc consistently to describe
aformat, including in VIDIOC_ENUM_FMT. And the expectation is that the
fourcccan be printed to a human readable string (there is even a printk
format forthat these days).But the pcm values are all small integers (and
can even be 0!), andprinting the fourcc will give garbage. It doesn't work
well at allwith the V4L2 API. But by having a straightforward conversion
between thepcm identifier and a fourcc it was really easy to deal with
this.There might even be applications today that call VIDIOC_ENUM_FMT to
seewhat is supported and fail if it is not a proper fourcc is returned.It
will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c).One of
the early versions of this patch series did precisely what you request,but
it just doesn't work well within the V4L2 uAPI.*
*"*

Best regards
Shengjiu Wang

> Nicolas
>
> >
> > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> > ---
> >  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
> >  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
> >  include/uapi/linux/videodev2.h                | 23 +++++
> >  4 files changed, 124 insertions(+)
> >  create mode 100644
Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index 000000000000..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*************
> > +Audio Formats
> > +*************
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +    :widths:       3 1 4
> > +
> > +    * - Identifier
> > +      - Code
> > +      - Details
> > +    * .. _V4L2-AUDIO-FMT-S8:
> > +
> > +      - ``V4L2_AUDIO_FMT_S8``
> > +      - 'S8'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S16_LE``
> > +      - 'S16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U16_LE``
> > +      - 'U16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_LE``
> > +      - 'S24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_LE``
> > +      - 'U24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S32_LE``
> > +      - 'S32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U32_LE``
> > +      - 'U32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> > +      - 'FLOAT_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> > +      - 'IEC958_SUBFRAME_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_3LE``
> > +      - 'S24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_3LE``
> > +      - 'U24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S20_3LE``
> > +      - 'S20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U20_3LE``
> > +      - 'U20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst
b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > index 11dab4a90630..2eb6fdd3b43d 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >      colorspaces
> >      colorspaces-defs
> >      colorspaces-details
> > +    pixfmt-audio
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c
b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 961abcdf7290..be229c69e991 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc
*fmt)
> >       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed";
break;
> >       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed";
break;
> >       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed";
break;
> > +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
> > +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned
LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit
Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit
Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed
LE"; break;
> > +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned
LE"; break;
> > +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float
LE"; break;
> > +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958
LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit
Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit
Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit
Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit
Unsigned LE"; break;
> >
> >       default:
> >               /* Compressed formats */
> > diff --git a/include/uapi/linux/videodev2.h
b/include/uapi/linux/videodev2.h
> > index 2c03d2dfadbe..673a6235a029 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -843,6 +843,29 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P')
/* Rockchip ISP1 3A Parameters */
> >  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K',
'1', 'S') /* Rockchip ISP1 3A Statistics */
> >
> > +/*
> > + * Audio-data formats
> > + * All these audio formats use a fourcc starting with 'AU'
> > + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
> > + */
> > +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U',
'0', '0')
> > +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A',
'U', '0', '2')
> > +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A',
'U', '0', '4')
> > +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A',
'U', '0', '6')
> > +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A',
'U', '0', '8')
> > +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A',
'U', '1', '0')
> > +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A',
'U', '1', '2')
> > +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A',
'U', '1', '4')
> > +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U',
'1', '8')
> > +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A',
'U', '3', '2')
> > +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A',
'U', '3', '4')
> > +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A',
'U', '3', '6')
> > +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A',
'U', '3', '8')
> > +
> > +#define v4l2_fourcc_to_audfmt(fourcc)        \
> > +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10
 \
> > +                                + ((((fourcc) >> 24) & 0xff) - '0'))
> > +
> >  /* priv field value to indicates that subsequent fields are valid. */
> >  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
> >
>
Shengjiu Wang Feb. 27, 2024, 3:44 a.m. UTC | #2
On Mon, Feb 26, 2024 at 9:55 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>
> Le lundi 26 février 2024 à 16:28 +0800, Shengjiu Wang a écrit :
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
>
> I'd like to join Mauro's voice that duplicating the audio formats is a bad idea.
> We have the same issues with video formats when you look at V4L2 vs DRM. You're
> rationale is that videodev2.h will be ambiguous if it includes asound.h, but
> looking at this change, there is no reason why you need to include asound.h in
> videodev2.h at all. The format type can be abstracted out with a uint32 in the
> API, and then it would be up to the users to include and use the appropriate
> formats IDs.
>

Resend for the plain text issue

Thanks.

There is another reason mentioned by Hans:

"
The problem is that within V4L2 we use fourcc consistently to describe a
format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc
can be printed to a human readable string (there is even a printk format for
that these days).

But the pcm values are all small integers (and can even be 0!), and
printing the fourcc will give garbage. It doesn't work well at all
with the V4L2 API. But by having a straightforward conversion between the
pcm identifier and a fourcc it was really easy to deal with this.

There might even be applications today that call VIDIOC_ENUM_FMT to see
what is supported and fail if it is not a proper fourcc is returned.

It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c).

One of the early versions of this patch series did precisely what you request,
but it just doesn't work well within the V4L2 uAPI.
"

Best regards
Shengjiu Wang
> Nicolas
>
> >
> > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> > ---
> >  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
> >  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
> >  include/uapi/linux/videodev2.h                | 23 +++++
> >  4 files changed, 124 insertions(+)
> >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index 000000000000..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*************
> > +Audio Formats
> > +*************
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +    :widths:       3 1 4
> > +
> > +    * - Identifier
> > +      - Code
> > +      - Details
> > +    * .. _V4L2-AUDIO-FMT-S8:
> > +
> > +      - ``V4L2_AUDIO_FMT_S8``
> > +      - 'S8'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S16_LE``
> > +      - 'S16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U16_LE``
> > +      - 'U16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_LE``
> > +      - 'S24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_LE``
> > +      - 'U24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S32_LE``
> > +      - 'S32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U32_LE``
> > +      - 'U32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> > +      - 'FLOAT_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> > +      - 'IEC958_SUBFRAME_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_3LE``
> > +      - 'S24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_3LE``
> > +      - 'U24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S20_3LE``
> > +      - 'S20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U20_3LE``
> > +      - 'U20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > index 11dab4a90630..2eb6fdd3b43d 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >      colorspaces
> >      colorspaces-defs
> >      colorspaces-details
> > +    pixfmt-audio
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 961abcdf7290..be229c69e991 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
> >       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
> >       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
> > +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
> > +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
> > +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
> >
> >       default:
> >               /* Compressed formats */
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 2c03d2dfadbe..673a6235a029 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -843,6 +843,29 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> >  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> >
> > +/*
> > + * Audio-data formats
> > + * All these audio formats use a fourcc starting with 'AU'
> > + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
> > + */
> > +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
> > +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
> > +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
> > +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
> > +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
> > +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
> > +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
> > +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
> > +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
> > +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
> > +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
> > +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
> > +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
> > +
> > +#define v4l2_fourcc_to_audfmt(fourcc)        \
> > +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
> > +                                + ((((fourcc) >> 24) & 0xff) - '0'))
> > +
> >  /* priv field value to indicates that subsequent fields are valid. */
> >  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
> >
>
Hans Verkuil Feb. 27, 2024, 8:34 a.m. UTC | #3
On 27/02/2024 04:44, Shengjiu Wang wrote:
> On Mon, Feb 26, 2024 at 9:55 PM Nicolas Dufresne <nicolas@ndufresne.ca> wrote:
>>
>> Le lundi 26 février 2024 à 16:28 +0800, Shengjiu Wang a écrit :
>>> The audio sample format definition is from alsa,
>>> the header file is include/uapi/sound/asound.h, but
>>> don't include this header file directly, because in
>>> user space, there is another copy in alsa-lib.
>>> There will be conflict in userspace for include
>>> videodev2.h & asound.h and asoundlib.h
>>>
>>> Here still use the fourcc format.
>>
>> I'd like to join Mauro's voice that duplicating the audio formats is a bad idea.
>> We have the same issues with video formats when you look at V4L2 vs DRM. You're
>> rationale is that videodev2.h will be ambiguous if it includes asound.h, but
>> looking at this change, there is no reason why you need to include asound.h in
>> videodev2.h at all. The format type can be abstracted out with a uint32 in the
>> API, and then it would be up to the users to include and use the appropriate
>> formats IDs.
>>
> 
> Resend for the plain text issue
> 
> Thanks.
> 
> There is another reason mentioned by Hans:
> 
> "
> The problem is that within V4L2 we use fourcc consistently to describe a
> format, including in VIDIOC_ENUM_FMT. And the expectation is that the fourcc
> can be printed to a human readable string (there is even a printk format for
> that these days).
> 
> But the pcm values are all small integers (and can even be 0!), and
> printing the fourcc will give garbage. It doesn't work well at all
> with the V4L2 API. But by having a straightforward conversion between the
> pcm identifier and a fourcc it was really easy to deal with this.
> 
> There might even be applications today that call VIDIOC_ENUM_FMT to see
> what is supported and fail if it is not a proper fourcc is returned.
> 
> It will certainly report nonsense in v4l_print_fmtdesc() (v4l2-ioctl.c).
> 
> One of the early versions of this patch series did precisely what you request,
> but it just doesn't work well within the V4L2 uAPI.
> "

Right, and remember that the fourcc codes are really meant to be shown as
characters. v4l2-ctl and friends all use this, and we all talk about them
as such (NV12, YUYV, etc.). If I would just stick in the standard PCM IDs,
then they would all show up as '....'.

<snip>

>>> +/*
>>> + * Audio-data formats
>>> + * All these audio formats use a fourcc starting with 'AU'
>>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
>>> + */
>>> +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
>>> +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
>>> +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
>>> +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
>>> +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
>>> +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
>>> +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
>>> +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
>>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
>>> +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
>>> +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
>>> +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
>>> +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
>>> +
>>> +#define v4l2_fourcc_to_audfmt(fourcc)        \
>>> +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
>>> +                                + ((((fourcc) >> 24) & 0xff) - '0'))
>>> +

We did add a #define to go from a fourcc to an audio format.

Would it help to add a corresponding v4l2_audfmt_to_fourcc define as well?

#define v4l2_audfmt_to_fourcc(audfmt) \
	v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)

I would have no problem with that.

The mapping between the audio format and the fourcc is very simple, it is just
converted to something that can be safely shown as ASCII characters.

Regards,

	Hans

>>>  /* priv field value to indicates that subsequent fields are valid. */
>>>  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
>>>
>>
Hans Verkuil March 8, 2024, 7:34 a.m. UTC | #4
Hi Shengjiu,

After thinking it over I think this patch can be improved:

On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> The audio sample format definition is from alsa,
> the header file is include/uapi/sound/asound.h, but
> don't include this header file directly, because in
> user space, there is another copy in alsa-lib.
> There will be conflict in userspace for include
> videodev2.h & asound.h and asoundlib.h
> 
> Here still use the fourcc format.
> 
> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> ---
>  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
>  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
>  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
>  include/uapi/linux/videodev2.h                | 23 +++++
>  4 files changed, 124 insertions(+)
>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> 
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> new file mode 100644
> index 000000000000..04b4a7fbd8f4
> --- /dev/null
> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> @@ -0,0 +1,87 @@
> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> +
> +.. _pixfmt-audio:
> +
> +*************
> +Audio Formats
> +*************
> +
> +These formats are used for :ref:`audiomem2mem` interface only.

Here you should also document that all these fourccs start with 'AU' and are
reserved for mappings of the snd_pcm_format_t type.

Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
define (see also below).

> +
> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> +
> +.. cssclass:: longtable
> +
> +.. flat-table:: Audio Format
> +    :header-rows:  1
> +    :stub-columns: 0
> +    :widths:       3 1 4
> +
> +    * - Identifier
> +      - Code
> +      - Details
> +    * .. _V4L2-AUDIO-FMT-S8:
> +
> +      - ``V4L2_AUDIO_FMT_S8``
> +      - 'S8'
> +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> +    * .. _V4L2-AUDIO-FMT-S16-LE:
> +
> +      - ``V4L2_AUDIO_FMT_S16_LE``
> +      - 'S16_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-U16-LE:
> +
> +      - ``V4L2_AUDIO_FMT_U16_LE``
> +      - 'U16_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-S24-LE:
> +
> +      - ``V4L2_AUDIO_FMT_S24_LE``
> +      - 'S24_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-U24-LE:
> +
> +      - ``V4L2_AUDIO_FMT_U24_LE``
> +      - 'U24_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-S32-LE:
> +
> +      - ``V4L2_AUDIO_FMT_S32_LE``
> +      - 'S32_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-U32-LE:
> +
> +      - ``V4L2_AUDIO_FMT_U32_LE``
> +      - 'U32_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> +
> +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> +      - 'FLOAT_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> +
> +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> +      - 'IEC958_SUBFRAME_LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> +
> +      - ``V4L2_AUDIO_FMT_S24_3LE``
> +      - 'S24_3LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> +
> +      - ``V4L2_AUDIO_FMT_U24_3LE``
> +      - 'U24_3LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> +
> +      - ``V4L2_AUDIO_FMT_S20_3LE``
> +      - 'S20_3LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> +
> +      - ``V4L2_AUDIO_FMT_U20_3LE``
> +      - 'U20_3LE'
> +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> index 11dab4a90630..2eb6fdd3b43d 100644
> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
>      colorspaces
>      colorspaces-defs
>      colorspaces-details
> +    pixfmt-audio
> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> index 961abcdf7290..be229c69e991 100644
> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>  	case V4L2_PIX_FMT_Y210:		descr = "10-bit YUYV Packed"; break;
>  	case V4L2_PIX_FMT_Y212:		descr = "12-bit YUYV Packed"; break;
>  	case V4L2_PIX_FMT_Y216:		descr = "16-bit YUYV Packed"; break;
> +	case V4L2_AUDIO_FMT_S8:		descr = "8-bit Signed"; break;
> +	case V4L2_AUDIO_FMT_S16_LE:	descr = "16-bit Signed LE"; break;
> +	case V4L2_AUDIO_FMT_U16_LE:		descr = "16-bit Unsigned LE"; break;
> +	case V4L2_AUDIO_FMT_S24_LE:		descr = "24(32)-bit Signed LE"; break;
> +	case V4L2_AUDIO_FMT_U24_LE:		descr = "24(32)-bit Unsigned LE"; break;
> +	case V4L2_AUDIO_FMT_S32_LE:		descr = "32-bit Signed LE"; break;
> +	case V4L2_AUDIO_FMT_U32_LE:		descr = "32-bit Unsigned LE"; break;
> +	case V4L2_AUDIO_FMT_FLOAT_LE:		descr = "32-bit Float LE"; break;
> +	case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE:	descr = "32-bit IEC958 LE"; break;
> +	case V4L2_AUDIO_FMT_S24_3LE:		descr = "24(24)-bit Signed LE"; break;
> +	case V4L2_AUDIO_FMT_U24_3LE:		descr = "24(24)-bit Unsigned LE"; break;
> +	case V4L2_AUDIO_FMT_S20_3LE:		descr = "20(24)-bit Signed LE"; break;
> +	case V4L2_AUDIO_FMT_U20_3LE:		descr = "20(24)-bit Unsigned LE"; break;
>  
>  	default:
>  		/* Compressed formats */
> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> index 2c03d2dfadbe..673a6235a029 100644
> --- a/include/uapi/linux/videodev2.h
> +++ b/include/uapi/linux/videodev2.h
> @@ -843,6 +843,29 @@ struct v4l2_pix_format {
>  #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>  #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>  
> +/*
> + * Audio-data formats
> + * All these audio formats use a fourcc starting with 'AU'
> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.

Also document here that fourccs starting with 'AU' are reserved for
the snd_pcm_format_t to fourcc mappings.

That to avoid that someone adds a 'AUXX' fourcc later.

> + */
> +#define V4L2_AUDIO_FMT_S8			v4l2_fourcc('A', 'U', '0', '0')
> +#define V4L2_AUDIO_FMT_S16_LE			v4l2_fourcc('A', 'U', '0', '2')
> +#define V4L2_AUDIO_FMT_U16_LE			v4l2_fourcc('A', 'U', '0', '4')
> +#define V4L2_AUDIO_FMT_S24_LE			v4l2_fourcc('A', 'U', '0', '6')
> +#define V4L2_AUDIO_FMT_U24_LE			v4l2_fourcc('A', 'U', '0', '8')
> +#define V4L2_AUDIO_FMT_S32_LE			v4l2_fourcc('A', 'U', '1', '0')
> +#define V4L2_AUDIO_FMT_U32_LE			v4l2_fourcc('A', 'U', '1', '2')
> +#define V4L2_AUDIO_FMT_FLOAT_LE			v4l2_fourcc('A', 'U', '1', '4')
> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE	v4l2_fourcc('A', 'U', '1', '8')
> +#define V4L2_AUDIO_FMT_S24_3LE			v4l2_fourcc('A', 'U', '3', '2')
> +#define V4L2_AUDIO_FMT_U24_3LE			v4l2_fourcc('A', 'U', '3', '4')
> +#define V4L2_AUDIO_FMT_S20_3LE			v4l2_fourcc('A', 'U', '3', '6')
> +#define V4L2_AUDIO_FMT_U20_3LE			v4l2_fourcc('A', 'U', '3', '8')
> +
> +#define v4l2_fourcc_to_audfmt(fourcc)	\
> +	(__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
> +				   + ((((fourcc) >> 24) & 0xff) - '0'))
> +

As I suggested in an earlier reply, add this:

#define v4l2_audfmt_to_fourcc(audfmt) \
 	v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)

Even though it is not used in the drivers, since this is a public header used
by drivers and applications, it makes sense to provide the reverse mapping as well.

Please test it in actual code to make sure there are no compilation warnings.

Regards,

	Hans

>  /* priv field value to indicates that subsequent fields are valid. */
>  #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe
>
Shengjiu Wang March 8, 2024, 11:52 a.m. UTC | #5
On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> Hi Shengjiu,
>
> After thinking it over I think this patch can be improved:
>
> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> > The audio sample format definition is from alsa,
> > the header file is include/uapi/sound/asound.h, but
> > don't include this header file directly, because in
> > user space, there is another copy in alsa-lib.
> > There will be conflict in userspace for include
> > videodev2.h & asound.h and asoundlib.h
> >
> > Here still use the fourcc format.
> >
> > Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> > ---
> >  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
> >  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
> >  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
> >  include/uapi/linux/videodev2.h                | 23 +++++
> >  4 files changed, 124 insertions(+)
> >  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > new file mode 100644
> > index 000000000000..04b4a7fbd8f4
> > --- /dev/null
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> > @@ -0,0 +1,87 @@
> > +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> > +
> > +.. _pixfmt-audio:
> > +
> > +*************
> > +Audio Formats
> > +*************
> > +
> > +These formats are used for :ref:`audiomem2mem` interface only.
>
> Here you should also document that all these fourccs start with 'AU' and are
> reserved for mappings of the snd_pcm_format_t type.
>
> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
> define (see also below).

How about below description?
'
All these fourccs starting with 'AU' are reserved for mappings
of the snd_pcm_format_t type.

The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
type to fourcc. The first character is 'A', the second character
is 'U', the third character is ten's digit of snd_pcm_format_t,
the fourth character is unit's digit of snd_pcm_format_t.

The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
snd_pcm_format_t type.
'
Best regards
Shengjiu Wang
>
> > +
> > +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> > +
> > +.. cssclass:: longtable
> > +
> > +.. flat-table:: Audio Format
> > +    :header-rows:  1
> > +    :stub-columns: 0
> > +    :widths:       3 1 4
> > +
> > +    * - Identifier
> > +      - Code
> > +      - Details
> > +    * .. _V4L2-AUDIO-FMT-S8:
> > +
> > +      - ``V4L2_AUDIO_FMT_S8``
> > +      - 'S8'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S16_LE``
> > +      - 'S16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U16-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U16_LE``
> > +      - 'U16_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_LE``
> > +      - 'S24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_LE``
> > +      - 'U24_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S32_LE``
> > +      - 'S32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U32-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U32_LE``
> > +      - 'U32_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> > +      - 'FLOAT_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> > +      - 'IEC958_SUBFRAME_LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S24_3LE``
> > +      - 'S24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U24_3LE``
> > +      - 'U24_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_S20_3LE``
> > +      - 'S20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> > +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> > +
> > +      - ``V4L2_AUDIO_FMT_U20_3LE``
> > +      - 'U20_3LE'
> > +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> > diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > index 11dab4a90630..2eb6fdd3b43d 100644
> > --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> > +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> > @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >      colorspaces
> >      colorspaces-defs
> >      colorspaces-details
> > +    pixfmt-audio
> > diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> > index 961abcdf7290..be229c69e991 100644
> > --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> > +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> > @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
> >       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
> >       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
> > +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
> > +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
> > +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
> > +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
> > +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
> > +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
> >
> >       default:
> >               /* Compressed formats */
> > diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> > index 2c03d2dfadbe..673a6235a029 100644
> > --- a/include/uapi/linux/videodev2.h
> > +++ b/include/uapi/linux/videodev2.h
> > @@ -843,6 +843,29 @@ struct v4l2_pix_format {
> >  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> >  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> >
> > +/*
> > + * Audio-data formats
> > + * All these audio formats use a fourcc starting with 'AU'
> > + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
>
> Also document here that fourccs starting with 'AU' are reserved for
> the snd_pcm_format_t to fourcc mappings.
>
> That to avoid that someone adds a 'AUXX' fourcc later.
>
> > + */
> > +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
> > +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
> > +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
> > +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
> > +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
> > +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
> > +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
> > +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
> > +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
> > +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
> > +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
> > +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
> > +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
> > +
> > +#define v4l2_fourcc_to_audfmt(fourcc)        \
> > +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
> > +                                + ((((fourcc) >> 24) & 0xff) - '0'))
> > +
>
> As I suggested in an earlier reply, add this:
>
> #define v4l2_audfmt_to_fourcc(audfmt) \
>         v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)
>
> Even though it is not used in the drivers, since this is a public header used
> by drivers and applications, it makes sense to provide the reverse mapping as well.
>
> Please test it in actual code to make sure there are no compilation warnings.
>
> Regards,
>
>         Hans
>
> >  /* priv field value to indicates that subsequent fields are valid. */
> >  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
> >
>
Hans Verkuil March 8, 2024, 12:06 p.m. UTC | #6
On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
> On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> Hi Shengjiu,
>>
>> After thinking it over I think this patch can be improved:
>>
>> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
>>> The audio sample format definition is from alsa,
>>> the header file is include/uapi/sound/asound.h, but
>>> don't include this header file directly, because in
>>> user space, there is another copy in alsa-lib.
>>> There will be conflict in userspace for include
>>> videodev2.h & asound.h and asoundlib.h
>>>
>>> Here still use the fourcc format.
>>>
>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
>>> ---
>>>  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
>>>  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
>>>  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
>>>  include/uapi/linux/videodev2.h                | 23 +++++
>>>  4 files changed, 124 insertions(+)
>>>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>>
>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>> new file mode 100644
>>> index 000000000000..04b4a7fbd8f4
>>> --- /dev/null
>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>> @@ -0,0 +1,87 @@
>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>> +
>>> +.. _pixfmt-audio:
>>> +
>>> +*************
>>> +Audio Formats
>>> +*************
>>> +
>>> +These formats are used for :ref:`audiomem2mem` interface only.
>>
>> Here you should also document that all these fourccs start with 'AU' and are
>> reserved for mappings of the snd_pcm_format_t type.
>>
>> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
>> define (see also below).
> 
> How about below description?
> '
> All these fourccs starting with 'AU' are reserved for mappings

All these fourccs -> All FourCCs

> of the snd_pcm_format_t type.
> 
> The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t

convert -> convert the

> type to fourcc. The first character is 'A', the second character

fourcc -> a FourCC

> is 'U', the third character is ten's digit of snd_pcm_format_t,
> the fourth character is unit's digit of snd_pcm_format_t.

"is 'U', and the remaining two characters is the snd_pcm_format_t
value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
to 'AU32'."

> 
> The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to

fourccs -> FourCCs

> snd_pcm_format_t type.

BTW, given the way snd_pcm_format_t is defined I am fairly certain
some casts are needed for the v4l2_audfmt_to_fourcc define.

Regards,

	Hans

> '
> Best regards
> Shengjiu Wang
>>
>>> +
>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
>>> +
>>> +.. cssclass:: longtable
>>> +
>>> +.. flat-table:: Audio Format
>>> +    :header-rows:  1
>>> +    :stub-columns: 0
>>> +    :widths:       3 1 4
>>> +
>>> +    * - Identifier
>>> +      - Code
>>> +      - Details
>>> +    * .. _V4L2-AUDIO-FMT-S8:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S8``
>>> +      - 'S8'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-S16-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S16_LE``
>>> +      - 'S16_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-U16-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_U16_LE``
>>> +      - 'U16_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-S24-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S24_LE``
>>> +      - 'S24_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-U24-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_U24_LE``
>>> +      - 'U24_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-S32-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S32_LE``
>>> +      - 'S32_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-U32-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_U32_LE``
>>> +      - 'U32_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
>>> +      - 'FLOAT_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
>>> +      - 'IEC958_SUBFRAME_LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-S24-3LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S24_3LE``
>>> +      - 'S24_3LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-U24-3LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_U24_3LE``
>>> +      - 'U24_3LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-S20-3LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_S20_3LE``
>>> +      - 'S20_3LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
>>> +    * .. _V4L2-AUDIO-FMT-U20-3LE:
>>> +
>>> +      - ``V4L2_AUDIO_FMT_U20_3LE``
>>> +      - 'U20_3LE'
>>> +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
>>> index 11dab4a90630..2eb6fdd3b43d 100644
>>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
>>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
>>>      colorspaces
>>>      colorspaces-defs
>>>      colorspaces-details
>>> +    pixfmt-audio
>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> index 961abcdf7290..be229c69e991 100644
>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
>>>       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
>>>       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
>>> +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
>>> +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
>>> +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
>>> +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
>>> +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
>>> +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
>>> +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
>>> +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
>>> +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
>>> +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
>>> +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
>>> +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
>>> +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
>>>
>>>       default:
>>>               /* Compressed formats */
>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>> index 2c03d2dfadbe..673a6235a029 100644
>>> --- a/include/uapi/linux/videodev2.h
>>> +++ b/include/uapi/linux/videodev2.h
>>> @@ -843,6 +843,29 @@ struct v4l2_pix_format {
>>>  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>>>  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>>>
>>> +/*
>>> + * Audio-data formats
>>> + * All these audio formats use a fourcc starting with 'AU'
>>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
>>
>> Also document here that fourccs starting with 'AU' are reserved for
>> the snd_pcm_format_t to fourcc mappings.
>>
>> That to avoid that someone adds a 'AUXX' fourcc later.
>>
>>> + */
>>> +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
>>> +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
>>> +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
>>> +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
>>> +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
>>> +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
>>> +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
>>> +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
>>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
>>> +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
>>> +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
>>> +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
>>> +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
>>> +
>>> +#define v4l2_fourcc_to_audfmt(fourcc)        \
>>> +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
>>> +                                + ((((fourcc) >> 24) & 0xff) - '0'))
>>> +
>>
>> As I suggested in an earlier reply, add this:
>>
>> #define v4l2_audfmt_to_fourcc(audfmt) \
>>         v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)
>>
>> Even though it is not used in the drivers, since this is a public header used
>> by drivers and applications, it makes sense to provide the reverse mapping as well.
>>
>> Please test it in actual code to make sure there are no compilation warnings.
>>
>> Regards,
>>
>>         Hans
>>
>>>  /* priv field value to indicates that subsequent fields are valid. */
>>>  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
>>>
>>
Shengjiu Wang March 8, 2024, 1:52 p.m. UTC | #7
On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
> > On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >>
> >> Hi Shengjiu,
> >>
> >> After thinking it over I think this patch can be improved:
> >>
> >> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> >>> The audio sample format definition is from alsa,
> >>> the header file is include/uapi/sound/asound.h, but
> >>> don't include this header file directly, because in
> >>> user space, there is another copy in alsa-lib.
> >>> There will be conflict in userspace for include
> >>> videodev2.h & asound.h and asoundlib.h
> >>>
> >>> Here still use the fourcc format.
> >>>
> >>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> >>> ---
> >>>  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
> >>>  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
> >>>  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
> >>>  include/uapi/linux/videodev2.h                | 23 +++++
> >>>  4 files changed, 124 insertions(+)
> >>>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>
> >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>> new file mode 100644
> >>> index 000000000000..04b4a7fbd8f4
> >>> --- /dev/null
> >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>> @@ -0,0 +1,87 @@
> >>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>> +
> >>> +.. _pixfmt-audio:
> >>> +
> >>> +*************
> >>> +Audio Formats
> >>> +*************
> >>> +
> >>> +These formats are used for :ref:`audiomem2mem` interface only.
> >>
> >> Here you should also document that all these fourccs start with 'AU' and are
> >> reserved for mappings of the snd_pcm_format_t type.
> >>
> >> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
> >> define (see also below).
> >
> > How about below description?
> > '
> > All these fourccs starting with 'AU' are reserved for mappings
>
> All these fourccs -> All FourCCs
>
> > of the snd_pcm_format_t type.
> >
> > The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
>
> convert -> convert the
>
> > type to fourcc. The first character is 'A', the second character
>
> fourcc -> a FourCC
>
> > is 'U', the third character is ten's digit of snd_pcm_format_t,
> > the fourth character is unit's digit of snd_pcm_format_t.
>
> "is 'U', and the remaining two characters is the snd_pcm_format_t
> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
> to 'AU32'."
>
> >
> > The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
>
> fourccs -> FourCCs
>
> > snd_pcm_format_t type.
>
> BTW, given the way snd_pcm_format_t is defined I am fairly certain
> some casts are needed for the v4l2_audfmt_to_fourcc define.
>
> Regards,
>
>         Hans
>
> > '
> > Best regards
> > Shengjiu Wang
> >>
> >>> +
> >>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> >>> +
> >>> +.. cssclass:: longtable
> >>> +
> >>> +.. flat-table:: Audio Format
> >>> +    :header-rows:  1
> >>> +    :stub-columns: 0
> >>> +    :widths:       3 1 4
> >>> +
> >>> +    * - Identifier
> >>> +      - Code
> >>> +      - Details
> >>> +    * .. _V4L2-AUDIO-FMT-S8:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S8``
> >>> +      - 'S8'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-S16-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S16_LE``
> >>> +      - 'S16_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-U16-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_U16_LE``
> >>> +      - 'U16_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-S24-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S24_LE``
> >>> +      - 'S24_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-U24-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_U24_LE``
> >>> +      - 'U24_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-S32-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S32_LE``
> >>> +      - 'S32_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-U32-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_U32_LE``
> >>> +      - 'U32_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> >>> +      - 'FLOAT_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> >>> +      - 'IEC958_SUBFRAME_LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S24_3LE``
> >>> +      - 'S24_3LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_U24_3LE``
> >>> +      - 'U24_3LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_S20_3LE``
> >>> +      - 'S20_3LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> >>> +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> >>> +
> >>> +      - ``V4L2_AUDIO_FMT_U20_3LE``
> >>> +      - 'U20_3LE'
> >>> +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> >>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>> index 11dab4a90630..2eb6fdd3b43d 100644
> >>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >>>      colorspaces
> >>>      colorspaces-defs
> >>>      colorspaces-details
> >>> +    pixfmt-audio
> >>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> index 961abcdf7290..be229c69e991 100644
> >>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>>       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
> >>>       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
> >>>       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
> >>> +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
> >>> +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
> >>> +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
> >>> +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
> >>> +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
> >>> +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
> >>> +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
> >>> +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
> >>> +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
> >>> +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
> >>> +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
> >>> +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
> >>> +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
> >>>
> >>>       default:
> >>>               /* Compressed formats */
> >>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >>> index 2c03d2dfadbe..673a6235a029 100644
> >>> --- a/include/uapi/linux/videodev2.h
> >>> +++ b/include/uapi/linux/videodev2.h
> >>> @@ -843,6 +843,29 @@ struct v4l2_pix_format {
> >>>  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> >>>  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> >>>
> >>> +/*
> >>> + * Audio-data formats
> >>> + * All these audio formats use a fourcc starting with 'AU'
> >>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
> >>
> >> Also document here that fourccs starting with 'AU' are reserved for
> >> the snd_pcm_format_t to fourcc mappings.
> >>
> >> That to avoid that someone adds a 'AUXX' fourcc later.
> >>
> >>> + */
> >>> +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
> >>> +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
> >>> +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
> >>> +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
> >>> +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
> >>> +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
> >>> +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
> >>> +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
> >>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
> >>> +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
> >>> +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
> >>> +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
> >>> +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
> >>> +
> >>> +#define v4l2_fourcc_to_audfmt(fourcc)        \
> >>> +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
> >>> +                                + ((((fourcc) >> 24) & 0xff) - '0'))
> >>> +
> >>
> >> As I suggested in an earlier reply, add this:
> >>
> >> #define v4l2_audfmt_to_fourcc(audfmt) \
> >>         v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)
> >>
> >> Even though it is not used in the drivers, since this is a public header used
> >> by drivers and applications, it makes sense to provide the reverse mapping as well.
> >>
> >> Please test it in actual code to make sure there are no compilation warnings.

I test this definition, the compiler doesn't report warning.

best regards
Shengjiu Wang

> >>
> >> Regards,
> >>
> >>         Hans
> >>
> >>>  /* priv field value to indicates that subsequent fields are valid. */
> >>>  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
> >>>
> >>
>
Hans Verkuil March 8, 2024, 2:04 p.m. UTC | #8
On 08/03/2024 2:52 pm, Shengjiu Wang wrote:
> On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>
>> On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
>>> On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>>>>
>>>> Hi Shengjiu,
>>>>
>>>> After thinking it over I think this patch can be improved:
>>>>
>>>> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
>>>>> The audio sample format definition is from alsa,
>>>>> the header file is include/uapi/sound/asound.h, but
>>>>> don't include this header file directly, because in
>>>>> user space, there is another copy in alsa-lib.
>>>>> There will be conflict in userspace for include
>>>>> videodev2.h & asound.h and asoundlib.h
>>>>>
>>>>> Here still use the fourcc format.
>>>>>
>>>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
>>>>> ---
>>>>>  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
>>>>>  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
>>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
>>>>>  include/uapi/linux/videodev2.h                | 23 +++++
>>>>>  4 files changed, 124 insertions(+)
>>>>>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>>>>
>>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>>>> new file mode 100644
>>>>> index 000000000000..04b4a7fbd8f4
>>>>> --- /dev/null
>>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
>>>>> @@ -0,0 +1,87 @@
>>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
>>>>> +
>>>>> +.. _pixfmt-audio:
>>>>> +
>>>>> +*************
>>>>> +Audio Formats
>>>>> +*************
>>>>> +
>>>>> +These formats are used for :ref:`audiomem2mem` interface only.
>>>>
>>>> Here you should also document that all these fourccs start with 'AU' and are
>>>> reserved for mappings of the snd_pcm_format_t type.
>>>>
>>>> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
>>>> define (see also below).
>>>
>>> How about below description?
>>> '
>>> All these fourccs starting with 'AU' are reserved for mappings
>>
>> All these fourccs -> All FourCCs
>>
>>> of the snd_pcm_format_t type.
>>>
>>> The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
>>
>> convert -> convert the
>>
>>> type to fourcc. The first character is 'A', the second character
>>
>> fourcc -> a FourCC
>>
>>> is 'U', the third character is ten's digit of snd_pcm_format_t,
>>> the fourth character is unit's digit of snd_pcm_format_t.
>>
>> "is 'U', and the remaining two characters is the snd_pcm_format_t
>> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
>> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
>> to 'AU32'."
>>
>>>
>>> The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
>>
>> fourccs -> FourCCs
>>
>>> snd_pcm_format_t type.
>>
>> BTW, given the way snd_pcm_format_t is defined I am fairly certain
>> some casts are needed for the v4l2_audfmt_to_fourcc define.
>>
>> Regards,
>>
>>         Hans
>>
>>> '
>>> Best regards
>>> Shengjiu Wang
>>>>
>>>>> +
>>>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
>>>>> +
>>>>> +.. cssclass:: longtable
>>>>> +
>>>>> +.. flat-table:: Audio Format
>>>>> +    :header-rows:  1
>>>>> +    :stub-columns: 0
>>>>> +    :widths:       3 1 4
>>>>> +
>>>>> +    * - Identifier
>>>>> +      - Code
>>>>> +      - Details
>>>>> +    * .. _V4L2-AUDIO-FMT-S8:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S8``
>>>>> +      - 'S8'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-S16-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S16_LE``
>>>>> +      - 'S16_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-U16-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_U16_LE``
>>>>> +      - 'U16_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-S24-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S24_LE``
>>>>> +      - 'S24_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-U24-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_U24_LE``
>>>>> +      - 'U24_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-S32-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S32_LE``
>>>>> +      - 'S32_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-U32-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_U32_LE``
>>>>> +      - 'U32_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
>>>>> +      - 'FLOAT_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
>>>>> +      - 'IEC958_SUBFRAME_LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-S24-3LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S24_3LE``
>>>>> +      - 'S24_3LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-U24-3LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_U24_3LE``
>>>>> +      - 'U24_3LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-S20-3LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_S20_3LE``
>>>>> +      - 'S20_3LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
>>>>> +    * .. _V4L2-AUDIO-FMT-U20-3LE:
>>>>> +
>>>>> +      - ``V4L2_AUDIO_FMT_U20_3LE``
>>>>> +      - 'U20_3LE'
>>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
>>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
>>>>> index 11dab4a90630..2eb6fdd3b43d 100644
>>>>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
>>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
>>>>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
>>>>>      colorspaces
>>>>>      colorspaces-defs
>>>>>      colorspaces-details
>>>>> +    pixfmt-audio
>>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> index 961abcdf7290..be229c69e991 100644
>>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
>>>>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
>>>>>       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
>>>>>       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
>>>>>       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
>>>>> +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
>>>>> +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
>>>>> +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
>>>>>
>>>>>       default:
>>>>>               /* Compressed formats */
>>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
>>>>> index 2c03d2dfadbe..673a6235a029 100644
>>>>> --- a/include/uapi/linux/videodev2.h
>>>>> +++ b/include/uapi/linux/videodev2.h
>>>>> @@ -843,6 +843,29 @@ struct v4l2_pix_format {
>>>>>  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
>>>>>  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
>>>>>
>>>>> +/*
>>>>> + * Audio-data formats
>>>>> + * All these audio formats use a fourcc starting with 'AU'
>>>>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
>>>>
>>>> Also document here that fourccs starting with 'AU' are reserved for
>>>> the snd_pcm_format_t to fourcc mappings.
>>>>
>>>> That to avoid that someone adds a 'AUXX' fourcc later.
>>>>
>>>>> + */
>>>>> +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
>>>>> +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
>>>>> +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
>>>>> +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
>>>>> +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
>>>>> +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
>>>>> +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
>>>>> +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
>>>>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
>>>>> +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
>>>>> +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
>>>>> +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
>>>>> +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
>>>>> +
>>>>> +#define v4l2_fourcc_to_audfmt(fourcc)        \
>>>>> +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
>>>>> +                                + ((((fourcc) >> 24) & 0xff) - '0'))
>>>>> +
>>>>
>>>> As I suggested in an earlier reply, add this:
>>>>
>>>> #define v4l2_audfmt_to_fourcc(audfmt) \
>>>>         v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)
>>>>
>>>> Even though it is not used in the drivers, since this is a public header used
>>>> by drivers and applications, it makes sense to provide the reverse mapping as well.
>>>>
>>>> Please test it in actual code to make sure there are no compilation warnings.
> 
> I test this definition, the compiler doesn't report warning.

typedef int __bitwise snd_pcm_format_t;

And __bitwise is apparently a sparse static analyzer attribute, so I suspect that the
v4l2_audfmt_to_fourcc definition will cause a sparse warning. So you need to check
with sparse.

Regards,

	Hans

> 
> best regards
> Shengjiu Wang
> 
>>>>
>>>> Regards,
>>>>
>>>>         Hans
>>>>
>>>>>  /* priv field value to indicates that subsequent fields are valid. */
>>>>>  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
>>>>>
>>>>
>>
Shengjiu Wang March 11, 2024, 2:07 a.m. UTC | #9
On Fri, Mar 8, 2024 at 10:04 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
>
> On 08/03/2024 2:52 pm, Shengjiu Wang wrote:
> > On Fri, Mar 8, 2024 at 8:06 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >>
> >> On 08/03/2024 12:52 pm, Shengjiu Wang wrote:
> >>> On Fri, Mar 8, 2024 at 3:34 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
> >>>>
> >>>> Hi Shengjiu,
> >>>>
> >>>> After thinking it over I think this patch can be improved:
> >>>>
> >>>> On 26/02/2024 9:28 am, Shengjiu Wang wrote:
> >>>>> The audio sample format definition is from alsa,
> >>>>> the header file is include/uapi/sound/asound.h, but
> >>>>> don't include this header file directly, because in
> >>>>> user space, there is another copy in alsa-lib.
> >>>>> There will be conflict in userspace for include
> >>>>> videodev2.h & asound.h and asoundlib.h
> >>>>>
> >>>>> Here still use the fourcc format.
> >>>>>
> >>>>> Signed-off-by: Shengjiu Wang <shengjiu.wang@nxp.com>
> >>>>> ---
> >>>>>  .../userspace-api/media/v4l/pixfmt-audio.rst  | 87 +++++++++++++++++++
> >>>>>  .../userspace-api/media/v4l/pixfmt.rst        |  1 +
> >>>>>  drivers/media/v4l2-core/v4l2-ioctl.c          | 13 +++
> >>>>>  include/uapi/linux/videodev2.h                | 23 +++++
> >>>>>  4 files changed, 124 insertions(+)
> >>>>>  create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>>
> >>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>> new file mode 100644
> >>>>> index 000000000000..04b4a7fbd8f4
> >>>>> --- /dev/null
> >>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
> >>>>> @@ -0,0 +1,87 @@
> >>>>> +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
> >>>>> +
> >>>>> +.. _pixfmt-audio:
> >>>>> +
> >>>>> +*************
> >>>>> +Audio Formats
> >>>>> +*************
> >>>>> +
> >>>>> +These formats are used for :ref:`audiomem2mem` interface only.
> >>>>
> >>>> Here you should also document that all these fourccs start with 'AU' and are
> >>>> reserved for mappings of the snd_pcm_format_t type.
> >>>>
> >>>> Also document the v4l2_fourcc_to_audfmt define and the v4l2_audfmt_to_fourcc
> >>>> define (see also below).
> >>>
> >>> How about below description?
> >>> '
> >>> All these fourccs starting with 'AU' are reserved for mappings
> >>
> >> All these fourccs -> All FourCCs
> >>
> >>> of the snd_pcm_format_t type.
> >>>
> >>> The v4l2_audfmt_to_fourcc() is defined to convert snd_pcm_format_t
> >>
> >> convert -> convert the
> >>
> >>> type to fourcc. The first character is 'A', the second character
> >>
> >> fourcc -> a FourCC
> >>
> >>> is 'U', the third character is ten's digit of snd_pcm_format_t,
> >>> the fourth character is unit's digit of snd_pcm_format_t.
> >>
> >> "is 'U', and the remaining two characters is the snd_pcm_format_t
> >> value in ASCII. Example: SNDRV_PCM_FORMAT_S16_LE (with value 2)
> >> maps to 'AU02' and SNDRV_PCM_FORMAT_S24_3LE (with value 32) maps
> >> to 'AU32'."
> >>
> >>>
> >>> The v4l2_fourcc_to_audfmt() is defined to convert these fourccs to
> >>
> >> fourccs -> FourCCs
> >>
> >>> snd_pcm_format_t type.
> >>
> >> BTW, given the way snd_pcm_format_t is defined I am fairly certain
> >> some casts are needed for the v4l2_audfmt_to_fourcc define.
> >>
> >> Regards,
> >>
> >>         Hans
> >>
> >>> '
> >>> Best regards
> >>> Shengjiu Wang
> >>>>
> >>>>> +
> >>>>> +.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
> >>>>> +
> >>>>> +.. cssclass:: longtable
> >>>>> +
> >>>>> +.. flat-table:: Audio Format
> >>>>> +    :header-rows:  1
> >>>>> +    :stub-columns: 0
> >>>>> +    :widths:       3 1 4
> >>>>> +
> >>>>> +    * - Identifier
> >>>>> +      - Code
> >>>>> +      - Details
> >>>>> +    * .. _V4L2-AUDIO-FMT-S8:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S8``
> >>>>> +      - 'S8'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-S16-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S16_LE``
> >>>>> +      - 'S16_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-U16-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_U16_LE``
> >>>>> +      - 'U16_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-S24-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S24_LE``
> >>>>> +      - 'S24_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-U24-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_U24_LE``
> >>>>> +      - 'U24_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-S32-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S32_LE``
> >>>>> +      - 'S32_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-U32-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_U32_LE``
> >>>>> +      - 'U32_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_FLOAT_LE``
> >>>>> +      - 'FLOAT_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
> >>>>> +      - 'IEC958_SUBFRAME_LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-S24-3LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S24_3LE``
> >>>>> +      - 'S24_3LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-U24-3LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_U24_3LE``
> >>>>> +      - 'U24_3LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-S20-3LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_S20_3LE``
> >>>>> +      - 'S20_3LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
> >>>>> +    * .. _V4L2-AUDIO-FMT-U20-3LE:
> >>>>> +
> >>>>> +      - ``V4L2_AUDIO_FMT_U20_3LE``
> >>>>> +      - 'U20_3LE'
> >>>>> +      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
> >>>>> diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>>>> index 11dab4a90630..2eb6fdd3b43d 100644
> >>>>> --- a/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>>>> +++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
> >>>>> @@ -36,3 +36,4 @@ see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
> >>>>>      colorspaces
> >>>>>      colorspaces-defs
> >>>>>      colorspaces-details
> >>>>> +    pixfmt-audio
> >>>>> diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>> index 961abcdf7290..be229c69e991 100644
> >>>>> --- a/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>> +++ b/drivers/media/v4l2-core/v4l2-ioctl.c
> >>>>> @@ -1471,6 +1471,19 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
> >>>>>       case V4L2_PIX_FMT_Y210:         descr = "10-bit YUYV Packed"; break;
> >>>>>       case V4L2_PIX_FMT_Y212:         descr = "12-bit YUYV Packed"; break;
> >>>>>       case V4L2_PIX_FMT_Y216:         descr = "16-bit YUYV Packed"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S8:         descr = "8-bit Signed"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S16_LE:     descr = "16-bit Signed LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_U16_LE:             descr = "16-bit Unsigned LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S24_LE:             descr = "24(32)-bit Signed LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_U24_LE:             descr = "24(32)-bit Unsigned LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S32_LE:             descr = "32-bit Signed LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_U32_LE:             descr = "32-bit Unsigned LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_FLOAT_LE:           descr = "32-bit Float LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE: descr = "32-bit IEC958 LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S24_3LE:            descr = "24(24)-bit Signed LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_U24_3LE:            descr = "24(24)-bit Unsigned LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_S20_3LE:            descr = "20(24)-bit Signed LE"; break;
> >>>>> +     case V4L2_AUDIO_FMT_U20_3LE:            descr = "20(24)-bit Unsigned LE"; break;
> >>>>>
> >>>>>       default:
> >>>>>               /* Compressed formats */
> >>>>> diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
> >>>>> index 2c03d2dfadbe..673a6235a029 100644
> >>>>> --- a/include/uapi/linux/videodev2.h
> >>>>> +++ b/include/uapi/linux/videodev2.h
> >>>>> @@ -843,6 +843,29 @@ struct v4l2_pix_format {
> >>>>>  #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
> >>>>>  #define V4L2_META_FMT_RK_ISP1_STAT_3A        v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
> >>>>>
> >>>>> +/*
> >>>>> + * Audio-data formats
> >>>>> + * All these audio formats use a fourcc starting with 'AU'
> >>>>> + * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
> >>>>
> >>>> Also document here that fourccs starting with 'AU' are reserved for
> >>>> the snd_pcm_format_t to fourcc mappings.
> >>>>
> >>>> That to avoid that someone adds a 'AUXX' fourcc later.
> >>>>
> >>>>> + */
> >>>>> +#define V4L2_AUDIO_FMT_S8                    v4l2_fourcc('A', 'U', '0', '0')
> >>>>> +#define V4L2_AUDIO_FMT_S16_LE                        v4l2_fourcc('A', 'U', '0', '2')
> >>>>> +#define V4L2_AUDIO_FMT_U16_LE                        v4l2_fourcc('A', 'U', '0', '4')
> >>>>> +#define V4L2_AUDIO_FMT_S24_LE                        v4l2_fourcc('A', 'U', '0', '6')
> >>>>> +#define V4L2_AUDIO_FMT_U24_LE                        v4l2_fourcc('A', 'U', '0', '8')
> >>>>> +#define V4L2_AUDIO_FMT_S32_LE                        v4l2_fourcc('A', 'U', '1', '0')
> >>>>> +#define V4L2_AUDIO_FMT_U32_LE                        v4l2_fourcc('A', 'U', '1', '2')
> >>>>> +#define V4L2_AUDIO_FMT_FLOAT_LE                      v4l2_fourcc('A', 'U', '1', '4')
> >>>>> +#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE    v4l2_fourcc('A', 'U', '1', '8')
> >>>>> +#define V4L2_AUDIO_FMT_S24_3LE                       v4l2_fourcc('A', 'U', '3', '2')
> >>>>> +#define V4L2_AUDIO_FMT_U24_3LE                       v4l2_fourcc('A', 'U', '3', '4')
> >>>>> +#define V4L2_AUDIO_FMT_S20_3LE                       v4l2_fourcc('A', 'U', '3', '6')
> >>>>> +#define V4L2_AUDIO_FMT_U20_3LE                       v4l2_fourcc('A', 'U', '3', '8')
> >>>>> +
> >>>>> +#define v4l2_fourcc_to_audfmt(fourcc)        \
> >>>>> +     (__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
> >>>>> +                                + ((((fourcc) >> 24) & 0xff) - '0'))
> >>>>> +
> >>>>
> >>>> As I suggested in an earlier reply, add this:
> >>>>
> >>>> #define v4l2_audfmt_to_fourcc(audfmt) \
> >>>>         v4l2_fourcc('A', 'U', '0' + (audfmt) / 10, '0' + (audfmt) % 10)
> >>>>
> >>>> Even though it is not used in the drivers, since this is a public header used
> >>>> by drivers and applications, it makes sense to provide the reverse mapping as well.
> >>>>
> >>>> Please test it in actual code to make sure there are no compilation warnings.
> >
> > I test this definition, the compiler doesn't report warning.
>
> typedef int __bitwise snd_pcm_format_t;
>
> And __bitwise is apparently a sparse static analyzer attribute, so I suspect that the
> v4l2_audfmt_to_fourcc definition will cause a sparse warning. So you need to check
> with sparse.
>

Thanks,  with sparse there is warning, I will fix it.

best regards
Shengjiu Wang

> Regards,
>
>         Hans
>
> >
> > best regards
> > Shengjiu Wang
> >
> >>>>
> >>>> Regards,
> >>>>
> >>>>         Hans
> >>>>
> >>>>>  /* priv field value to indicates that subsequent fields are valid. */
> >>>>>  #define V4L2_PIX_FMT_PRIV_MAGIC              0xfeedcafe
> >>>>>
> >>>>
> >>
>
diff mbox series

Patch

diff --git a/Documentation/userspace-api/media/v4l/pixfmt-audio.rst b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
new file mode 100644
index 000000000000..04b4a7fbd8f4
--- /dev/null
+++ b/Documentation/userspace-api/media/v4l/pixfmt-audio.rst
@@ -0,0 +1,87 @@ 
+.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later
+
+.. _pixfmt-audio:
+
+*************
+Audio Formats
+*************
+
+These formats are used for :ref:`audiomem2mem` interface only.
+
+.. tabularcolumns:: |p{5.8cm}|p{1.2cm}|p{10.3cm}|
+
+.. cssclass:: longtable
+
+.. flat-table:: Audio Format
+    :header-rows:  1
+    :stub-columns: 0
+    :widths:       3 1 4
+
+    * - Identifier
+      - Code
+      - Details
+    * .. _V4L2-AUDIO-FMT-S8:
+
+      - ``V4L2_AUDIO_FMT_S8``
+      - 'S8'
+      - Corresponds to SNDRV_PCM_FORMAT_S8 in ALSA
+    * .. _V4L2-AUDIO-FMT-S16-LE:
+
+      - ``V4L2_AUDIO_FMT_S16_LE``
+      - 'S16_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_S16_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-U16-LE:
+
+      - ``V4L2_AUDIO_FMT_U16_LE``
+      - 'U16_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_U16_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-S24-LE:
+
+      - ``V4L2_AUDIO_FMT_S24_LE``
+      - 'S24_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_S24_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-U24-LE:
+
+      - ``V4L2_AUDIO_FMT_U24_LE``
+      - 'U24_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_U24_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-S32-LE:
+
+      - ``V4L2_AUDIO_FMT_S32_LE``
+      - 'S32_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_S32_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-U32-LE:
+
+      - ``V4L2_AUDIO_FMT_U32_LE``
+      - 'U32_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_U32_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-FLOAT-LE:
+
+      - ``V4L2_AUDIO_FMT_FLOAT_LE``
+      - 'FLOAT_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_FLOAT_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-IEC958-SUBFRAME-LE:
+
+      - ``V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE``
+      - 'IEC958_SUBFRAME_LE'
+      - Corresponds to SNDRV_PCM_FORMAT_IEC958_SUBFRAME_LE in ALSA
+    * .. _V4L2-AUDIO-FMT-S24-3LE:
+
+      - ``V4L2_AUDIO_FMT_S24_3LE``
+      - 'S24_3LE'
+      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+    * .. _V4L2-AUDIO-FMT-U24-3LE:
+
+      - ``V4L2_AUDIO_FMT_U24_3LE``
+      - 'U24_3LE'
+      - Corresponds to SNDRV_PCM_FORMAT_U24_3LE in ALSA
+    * .. _V4L2-AUDIO-FMT-S20-3LE:
+
+      - ``V4L2_AUDIO_FMT_S20_3LE``
+      - 'S20_3LE'
+      - Corresponds to SNDRV_PCM_FORMAT_S24_3LE in ALSA
+    * .. _V4L2-AUDIO-FMT-U20-3LE:
+
+      - ``V4L2_AUDIO_FMT_U20_3LE``
+      - 'U20_3LE'
+      - Corresponds to SNDRV_PCM_FORMAT_U20_3LE in ALSA
diff --git a/Documentation/userspace-api/media/v4l/pixfmt.rst b/Documentation/userspace-api/media/v4l/pixfmt.rst
index 11dab4a90630..2eb6fdd3b43d 100644
--- a/Documentation/userspace-api/media/v4l/pixfmt.rst
+++ b/Documentation/userspace-api/media/v4l/pixfmt.rst
@@ -36,3 +36,4 @@  see also :ref:`VIDIOC_G_FBUF <VIDIOC_G_FBUF>`.)
     colorspaces
     colorspaces-defs
     colorspaces-details
+    pixfmt-audio
diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c
index 961abcdf7290..be229c69e991 100644
--- a/drivers/media/v4l2-core/v4l2-ioctl.c
+++ b/drivers/media/v4l2-core/v4l2-ioctl.c
@@ -1471,6 +1471,19 @@  static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 	case V4L2_PIX_FMT_Y210:		descr = "10-bit YUYV Packed"; break;
 	case V4L2_PIX_FMT_Y212:		descr = "12-bit YUYV Packed"; break;
 	case V4L2_PIX_FMT_Y216:		descr = "16-bit YUYV Packed"; break;
+	case V4L2_AUDIO_FMT_S8:		descr = "8-bit Signed"; break;
+	case V4L2_AUDIO_FMT_S16_LE:	descr = "16-bit Signed LE"; break;
+	case V4L2_AUDIO_FMT_U16_LE:		descr = "16-bit Unsigned LE"; break;
+	case V4L2_AUDIO_FMT_S24_LE:		descr = "24(32)-bit Signed LE"; break;
+	case V4L2_AUDIO_FMT_U24_LE:		descr = "24(32)-bit Unsigned LE"; break;
+	case V4L2_AUDIO_FMT_S32_LE:		descr = "32-bit Signed LE"; break;
+	case V4L2_AUDIO_FMT_U32_LE:		descr = "32-bit Unsigned LE"; break;
+	case V4L2_AUDIO_FMT_FLOAT_LE:		descr = "32-bit Float LE"; break;
+	case V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE:	descr = "32-bit IEC958 LE"; break;
+	case V4L2_AUDIO_FMT_S24_3LE:		descr = "24(24)-bit Signed LE"; break;
+	case V4L2_AUDIO_FMT_U24_3LE:		descr = "24(24)-bit Unsigned LE"; break;
+	case V4L2_AUDIO_FMT_S20_3LE:		descr = "20(24)-bit Signed LE"; break;
+	case V4L2_AUDIO_FMT_U20_3LE:		descr = "20(24)-bit Unsigned LE"; break;
 
 	default:
 		/* Compressed formats */
diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h
index 2c03d2dfadbe..673a6235a029 100644
--- a/include/uapi/linux/videodev2.h
+++ b/include/uapi/linux/videodev2.h
@@ -843,6 +843,29 @@  struct v4l2_pix_format {
 #define V4L2_META_FMT_RK_ISP1_PARAMS	v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */
 #define V4L2_META_FMT_RK_ISP1_STAT_3A	v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */
 
+/*
+ * Audio-data formats
+ * All these audio formats use a fourcc starting with 'AU'
+ * followed by the SNDRV_PCM_FORMAT_ value from asound.h.
+ */
+#define V4L2_AUDIO_FMT_S8			v4l2_fourcc('A', 'U', '0', '0')
+#define V4L2_AUDIO_FMT_S16_LE			v4l2_fourcc('A', 'U', '0', '2')
+#define V4L2_AUDIO_FMT_U16_LE			v4l2_fourcc('A', 'U', '0', '4')
+#define V4L2_AUDIO_FMT_S24_LE			v4l2_fourcc('A', 'U', '0', '6')
+#define V4L2_AUDIO_FMT_U24_LE			v4l2_fourcc('A', 'U', '0', '8')
+#define V4L2_AUDIO_FMT_S32_LE			v4l2_fourcc('A', 'U', '1', '0')
+#define V4L2_AUDIO_FMT_U32_LE			v4l2_fourcc('A', 'U', '1', '2')
+#define V4L2_AUDIO_FMT_FLOAT_LE			v4l2_fourcc('A', 'U', '1', '4')
+#define V4L2_AUDIO_FMT_IEC958_SUBFRAME_LE	v4l2_fourcc('A', 'U', '1', '8')
+#define V4L2_AUDIO_FMT_S24_3LE			v4l2_fourcc('A', 'U', '3', '2')
+#define V4L2_AUDIO_FMT_U24_3LE			v4l2_fourcc('A', 'U', '3', '4')
+#define V4L2_AUDIO_FMT_S20_3LE			v4l2_fourcc('A', 'U', '3', '6')
+#define V4L2_AUDIO_FMT_U20_3LE			v4l2_fourcc('A', 'U', '3', '8')
+
+#define v4l2_fourcc_to_audfmt(fourcc)	\
+	(__force snd_pcm_format_t)(((((fourcc) >> 16) & 0xff) - '0') * 10  \
+				   + ((((fourcc) >> 24) & 0xff) - '0'))
+
 /* priv field value to indicates that subsequent fields are valid. */
 #define V4L2_PIX_FMT_PRIV_MAGIC		0xfeedcafe