mbox series

[v5,00/14] media: mtk-vcodec: support for MT8183 decoder

Message ID 20210519143011.1175546-1-acourbot@chromium.org
Headers show
Series media: mtk-vcodec: support for MT8183 decoder | expand

Message

Alexandre Courbot May 19, 2021, 2:29 p.m. UTC
This series adds support for the stateless API into mtk-vcodec, by first
separating the stateful ops into their own source file, and introducing
a new set of ops suitable for stateless decoding. As such, support for
stateful decoders should remain completely unaffected.

This series has been tested with both MT8183 and MT8173. Decoding was
working for both chips, and in the case of MT8173 no regression has been
spotted.

Patches 1-5 fix a few compliance issues with the decoder and encoder, most
notably by adding support for the START and STOP command for the latter. These
patches were last in the previous series but have been moved to the beginning so
they can be applied sooner.

Patches 6-9 separates the "stateful" part of the driver into its own file and
add support for the new firmware and pixel format used by MT8183.

Patches 10-14 add support for H.264 stateless decoding and MT8183.

Changes since v4:
* Moved compliance fix patches to the head of the series.
* Select MEDIA_CONTROLLER_REQUEST_API.
* Properly capitalize MM21's format description string.
* Reorganize stateless code as suggested by Hans.
* Fix compilation errors when DEBUG is defined.
* Merge double-free fixup patch into the patch that introduced the issue (was
  a separate patch coming right after the one introducing the issue).

Changes since v3:
* Stop checking that controls are set for every request.
* Add V4L2_CID_STATELESS_H264_START_CODE control.
* Stop mapping OUTPUT buffers and getting the NAL type from them, use the
  nal_ref_idc field instead.
* Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only.
* Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook.

Changes since v2:
* Add follow-up patches fixing support for START/STOP commands for the
  encoder, and stateful decoder.

Alexandre Courbot (8):
  media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD
  media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits
  media: mtk-vcodec: make flush buffer reusable by encoder
  media: mtk-vcodec: venc: support START and STOP commands
  media: mtk-vcodec: vdec: handle firmware version field
  media: mtk-vcodec: support version 2 of decoder firmware ABI
  media: add Mediatek's MM21 format
  dt-bindings: media: document mediatek,mt8183-vcodec-dec

Hirokazu Honda (1):
  media: mtk-vcodec: vdec: Support H264 profile control

Yunfei Dong (5):
  media: mtk-vcodec: vdec: move stateful ops into their own file
  media: mtk-vcodec: vdec: support stateless API
  media: mtk-vcodec: vdec: support stateless H.264 decoding
  media: mtk-vcodec: vdec: add media device if using stateless api
  media: mtk-vcodec: enable MT8183 decoder

 .../bindings/media/mediatek-vcodec.txt        |   1 +
 .../media/v4l/pixfmt-reserved.rst             |   7 +
 drivers/media/platform/Kconfig                |   3 +
 drivers/media/platform/mtk-vcodec/Makefile    |   3 +
 .../platform/mtk-vcodec/mtk_vcodec_dec.c      | 817 +++---------------
 .../platform/mtk-vcodec/mtk_vcodec_dec.h      |  27 +-
 .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  66 +-
 .../mtk-vcodec/mtk_vcodec_dec_stateful.c      | 668 ++++++++++++++
 .../mtk-vcodec/mtk_vcodec_dec_stateless.c     | 366 ++++++++
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  58 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 135 ++-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |   4 +
 .../mtk-vcodec/vdec/vdec_h264_req_if.c        | 780 +++++++++++++++++
 .../media/platform/mtk-vcodec/vdec_drv_if.c   |   3 +
 .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +
 .../media/platform/mtk-vcodec/vdec_ipi_msg.h  |  23 +-
 .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  43 +-
 .../media/platform/mtk-vcodec/vdec_vpu_if.h   |   5 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/videodev2.h                |   1 +
 20 files changed, 2290 insertions(+), 722 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c
 create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c

--
2.31.1.751.gd2f1c929bd-goog

Comments

Hans Verkuil May 27, 2021, 8:08 a.m. UTC | #1
Hi Alexandre,

On 19/05/2021 16:29, Alexandre Courbot wrote:
> This series adds support for the stateless API into mtk-vcodec, by first

> separating the stateful ops into their own source file, and introducing

> a new set of ops suitable for stateless decoding. As such, support for

> stateful decoders should remain completely unaffected.

> 

> This series has been tested with both MT8183 and MT8173. Decoding was

> working for both chips, and in the case of MT8173 no regression has been

> spotted.

> 

> Patches 1-5 fix a few compliance issues with the decoder and encoder, most

> notably by adding support for the START and STOP command for the latter. These

> patches were last in the previous series but have been moved to the beginning so

> they can be applied sooner.

> 

> Patches 6-9 separates the "stateful" part of the driver into its own file and

> add support for the new firmware and pixel format used by MT8183.

> 

> Patches 10-14 add support for H.264 stateless decoding and MT8183.

> 

> Changes since v4:

> * Moved compliance fix patches to the head of the series.

> * Select MEDIA_CONTROLLER_REQUEST_API.

> * Properly capitalize MM21's format description string.

> * Reorganize stateless code as suggested by Hans.

> * Fix compilation errors when DEBUG is defined.

> * Merge double-free fixup patch into the patch that introduced the issue (was

>   a separate patch coming right after the one introducing the issue).

> 

> Changes since v3:

> * Stop checking that controls are set for every request.

> * Add V4L2_CID_STATELESS_H264_START_CODE control.

> * Stop mapping OUTPUT buffers and getting the NAL type from them, use the

>   nal_ref_idc field instead.

> * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only.

> * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook.

> 

> Changes since v2:

> * Add follow-up patches fixing support for START/STOP commands for the

>   encoder, and stateful decoder.

> 

> Alexandre Courbot (8):

>   media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD

>   media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits

>   media: mtk-vcodec: make flush buffer reusable by encoder

>   media: mtk-vcodec: venc: support START and STOP commands

>   media: mtk-vcodec: vdec: handle firmware version field

>   media: mtk-vcodec: support version 2 of decoder firmware ABI

>   media: add Mediatek's MM21 format

>   dt-bindings: media: document mediatek,mt8183-vcodec-dec

> 

> Hirokazu Honda (1):

>   media: mtk-vcodec: vdec: Support H264 profile control

> 

> Yunfei Dong (5):

>   media: mtk-vcodec: vdec: move stateful ops into their own file

>   media: mtk-vcodec: vdec: support stateless API

>   media: mtk-vcodec: vdec: support stateless H.264 decoding

>   media: mtk-vcodec: vdec: add media device if using stateless api

>   media: mtk-vcodec: enable MT8183 decoder


Running scripts/checkpatch.pl --strict over this patch series gives
a lot of warnings and checks. A lot of these look like they are easy
to fix and reasonable.

Regards,

	Hans

> 

>  .../bindings/media/mediatek-vcodec.txt        |   1 +

>  .../media/v4l/pixfmt-reserved.rst             |   7 +

>  drivers/media/platform/Kconfig                |   3 +

>  drivers/media/platform/mtk-vcodec/Makefile    |   3 +

>  .../platform/mtk-vcodec/mtk_vcodec_dec.c      | 817 +++---------------

>  .../platform/mtk-vcodec/mtk_vcodec_dec.h      |  27 +-

>  .../platform/mtk-vcodec/mtk_vcodec_dec_drv.c  |  66 +-

>  .../mtk-vcodec/mtk_vcodec_dec_stateful.c      | 668 ++++++++++++++

>  .../mtk-vcodec/mtk_vcodec_dec_stateless.c     | 366 ++++++++

>  .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  58 +-

>  .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 135 ++-

>  .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |   4 +

>  .../mtk-vcodec/vdec/vdec_h264_req_if.c        | 780 +++++++++++++++++

>  .../media/platform/mtk-vcodec/vdec_drv_if.c   |   3 +

>  .../media/platform/mtk-vcodec/vdec_drv_if.h   |   1 +

>  .../media/platform/mtk-vcodec/vdec_ipi_msg.h  |  23 +-

>  .../media/platform/mtk-vcodec/vdec_vpu_if.c   |  43 +-

>  .../media/platform/mtk-vcodec/vdec_vpu_if.h   |   5 +

>  drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +

>  include/uapi/linux/videodev2.h                |   1 +

>  20 files changed, 2290 insertions(+), 722 deletions(-)

>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateful.c

>  create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_dec_stateless.c

>  create mode 100644 drivers/media/platform/mtk-vcodec/vdec/vdec_h264_req_if.c

> 

> --

> 2.31.1.751.gd2f1c929bd-goog

>
Alexandre Courbot May 27, 2021, 10:10 a.m. UTC | #2
On Thu, May 27, 2021 at 5:08 PM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:
>

> Hi Alexandre,

>

> On 19/05/2021 16:29, Alexandre Courbot wrote:

> > This series adds support for the stateless API into mtk-vcodec, by first

> > separating the stateful ops into their own source file, and introducing

> > a new set of ops suitable for stateless decoding. As such, support for

> > stateful decoders should remain completely unaffected.

> >

> > This series has been tested with both MT8183 and MT8173. Decoding was

> > working for both chips, and in the case of MT8173 no regression has been

> > spotted.

> >

> > Patches 1-5 fix a few compliance issues with the decoder and encoder, most

> > notably by adding support for the START and STOP command for the latter. These

> > patches were last in the previous series but have been moved to the beginning so

> > they can be applied sooner.

> >

> > Patches 6-9 separates the "stateful" part of the driver into its own file and

> > add support for the new firmware and pixel format used by MT8183.

> >

> > Patches 10-14 add support for H.264 stateless decoding and MT8183.

> >

> > Changes since v4:

> > * Moved compliance fix patches to the head of the series.

> > * Select MEDIA_CONTROLLER_REQUEST_API.

> > * Properly capitalize MM21's format description string.

> > * Reorganize stateless code as suggested by Hans.

> > * Fix compilation errors when DEBUG is defined.

> > * Merge double-free fixup patch into the patch that introduced the issue (was

> >   a separate patch coming right after the one introducing the issue).

> >

> > Changes since v3:

> > * Stop checking that controls are set for every request.

> > * Add V4L2_CID_STATELESS_H264_START_CODE control.

> > * Stop mapping OUTPUT buffers and getting the NAL type from them, use the

> >   nal_ref_idc field instead.

> > * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only.

> > * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook.

> >

> > Changes since v2:

> > * Add follow-up patches fixing support for START/STOP commands for the

> >   encoder, and stateful decoder.

> >

> > Alexandre Courbot (8):

> >   media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD

> >   media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits

> >   media: mtk-vcodec: make flush buffer reusable by encoder

> >   media: mtk-vcodec: venc: support START and STOP commands

> >   media: mtk-vcodec: vdec: handle firmware version field

> >   media: mtk-vcodec: support version 2 of decoder firmware ABI

> >   media: add Mediatek's MM21 format

> >   dt-bindings: media: document mediatek,mt8183-vcodec-dec

> >

> > Hirokazu Honda (1):

> >   media: mtk-vcodec: vdec: Support H264 profile control

> >

> > Yunfei Dong (5):

> >   media: mtk-vcodec: vdec: move stateful ops into their own file

> >   media: mtk-vcodec: vdec: support stateless API

> >   media: mtk-vcodec: vdec: support stateless H.264 decoding

> >   media: mtk-vcodec: vdec: add media device if using stateless api

> >   media: mtk-vcodec: enable MT8183 decoder

>

> Running scripts/checkpatch.pl --strict over this patch series gives

> a lot of warnings and checks. A lot of these look like they are easy

> to fix and reasonable.


Apologies, I forgot to use --strict. It's not pretty indeed. I've
fixed most of the problems reported ; a few are more tricky or would
require extra cleanup patches like converting e.g. uint32_t to u32
when adding a member to a struct, which would make sense if we convert
all members of the struct (or better, the whole driver) separately.
Hopefully these can be overlooked for the time being.
Hans Verkuil May 27, 2021, 10:18 a.m. UTC | #3
On 27/05/2021 12:10, Alexandre Courbot wrote:
> On Thu, May 27, 2021 at 5:08 PM Hans Verkuil <hverkuil-cisco@xs4all.nl> wrote:

>>

>> Hi Alexandre,

>>

>> On 19/05/2021 16:29, Alexandre Courbot wrote:

>>> This series adds support for the stateless API into mtk-vcodec, by first

>>> separating the stateful ops into their own source file, and introducing

>>> a new set of ops suitable for stateless decoding. As such, support for

>>> stateful decoders should remain completely unaffected.

>>>

>>> This series has been tested with both MT8183 and MT8173. Decoding was

>>> working for both chips, and in the case of MT8173 no regression has been

>>> spotted.

>>>

>>> Patches 1-5 fix a few compliance issues with the decoder and encoder, most

>>> notably by adding support for the START and STOP command for the latter. These

>>> patches were last in the previous series but have been moved to the beginning so

>>> they can be applied sooner.

>>>

>>> Patches 6-9 separates the "stateful" part of the driver into its own file and

>>> add support for the new firmware and pixel format used by MT8183.

>>>

>>> Patches 10-14 add support for H.264 stateless decoding and MT8183.

>>>

>>> Changes since v4:

>>> * Moved compliance fix patches to the head of the series.

>>> * Select MEDIA_CONTROLLER_REQUEST_API.

>>> * Properly capitalize MM21's format description string.

>>> * Reorganize stateless code as suggested by Hans.

>>> * Fix compilation errors when DEBUG is defined.

>>> * Merge double-free fixup patch into the patch that introduced the issue (was

>>>   a separate patch coming right after the one introducing the issue).

>>>

>>> Changes since v3:

>>> * Stop checking that controls are set for every request.

>>> * Add V4L2_CID_STATELESS_H264_START_CODE control.

>>> * Stop mapping OUTPUT buffers and getting the NAL type from them, use the

>>>   nal_ref_idc field instead.

>>> * Make V4L2_CID_MIN_BUFFERS_FOR_CAPTURE control stateful-only.

>>> * Set vb2_buffer's field to V4L2_FIELD_NONE in buffer validation hook.

>>>

>>> Changes since v2:

>>> * Add follow-up patches fixing support for START/STOP commands for the

>>>   encoder, and stateful decoder.

>>>

>>> Alexandre Courbot (8):

>>>   media: mtk-vcodec: vdec: use helpers in VIDIOC_(TRY_)DECODER_CMD

>>>   media: mtk-vcodec: vdec: clamp OUTPUT resolution to hardware limits

>>>   media: mtk-vcodec: make flush buffer reusable by encoder

>>>   media: mtk-vcodec: venc: support START and STOP commands

>>>   media: mtk-vcodec: vdec: handle firmware version field

>>>   media: mtk-vcodec: support version 2 of decoder firmware ABI

>>>   media: add Mediatek's MM21 format

>>>   dt-bindings: media: document mediatek,mt8183-vcodec-dec

>>>

>>> Hirokazu Honda (1):

>>>   media: mtk-vcodec: vdec: Support H264 profile control

>>>

>>> Yunfei Dong (5):

>>>   media: mtk-vcodec: vdec: move stateful ops into their own file

>>>   media: mtk-vcodec: vdec: support stateless API

>>>   media: mtk-vcodec: vdec: support stateless H.264 decoding

>>>   media: mtk-vcodec: vdec: add media device if using stateless api

>>>   media: mtk-vcodec: enable MT8183 decoder

>>

>> Running scripts/checkpatch.pl --strict over this patch series gives

>> a lot of warnings and checks. A lot of these look like they are easy

>> to fix and reasonable.

> 

> Apologies, I forgot to use --strict. It's not pretty indeed. I've

> fixed most of the problems reported ; a few are more tricky or would

> require extra cleanup patches like converting e.g. uint32_t to u32

> when adding a member to a struct, which would make sense if we convert

> all members of the struct (or better, the whole driver) separately.

> Hopefully these can be overlooked for the time being.


Fair enough, just mention that in the cover letter.

Regards,

	Hans

>