mbox series

[v2,0/8] Add support for Wave6 video codec driver

Message ID 20250422093119.595-1-nas.chung@chipsnmedia.com
Headers show
Series Add support for Wave6 video codec driver | expand

Message

Nas Chung April 22, 2025, 9:31 a.m. UTC 
This patch series introduces support for the Chips&Media Wave6 video
codec IP, a completely different hardware architecture compared to Wave5.

The wave6 driver is a M2M stateful encoder/decoder driver.
It supports various video formats, including H.264 and H.265,
for both encoding and decoding.
While other versions of the Wave6 IP may support VP9 decoding and
AV1 decoding and encoding those formats are not implemented or validated
in this driver at this time.

On NXP i.MX SoCs, the Wave6 IP functionality is split between two regions:
VPU Control region, Manages shared resources such as firmware memory.
VPU Core region, Provides encoding and decoding capabilities.
The VPU core cannot operate independently without the VPU control region.

This driver has been tested with GStreamer on:
- NXP i.MX95 board
- pre-silicon FPGA environment

Test results for decoder fluster:
- JVT-AVC_V1, Ran 77/135 tests successfully              in 35.519 secs
- JVT-FR-EXT, Ran 25/69 tests successfully               in 17.725 secs
- JCT-VC-HEVC_V1, Ran 132/147 tests successfully         in 81.549 secs
- All failures are due to unsupported hardware features:
-- 10bit, Resolutions higher than 4K, FMO, MBAFF
-- Extended profile, Field encoding and High422 sreams.

Test results for v4l2-compliance:
v4l2-compliance 1.29.0-5359, 64 bits, 64-bit time_t
v4l2-compliance SHA: 2a91a869eb8a 2025-04-12 11:35:53

Compliance test for wave6-dec device /dev/video0:
                fail: ../utils/v4l2-compliance/v4l2-test-controls.cpp(1180): !have_source_change || !have_eos
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-dec device /dev/video0: 48, Succeeded: 47, Failed: 1, Warnings: 0

Compliance test for wave6-enc device /dev/video1:
                fail: ../utils/v4l2-compliance/v4l2-test-controls.cpp(1169): node->codec_mask & STATEFUL_ENCODER
        test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: FAIL
Total for wave6-enc device /dev/video1: 48, Succeeded: 47, Failed: 1, Warnings: 0

Note: the failures are all related with the eos event.

Changelog:

v2:
- Refined DT bindings to better represent the hardware
- Reworked driver to align with the parent(VPU) and child(CTRL, CORE)
- Fixed build issues reported by CI tools (Smatch, Sparse, TRACE)
- Improved commit messages with clearer descriptions
- Added kernel-doc for exported functions
- Removed redundant print statements and unused code
- Reordered patches to prevent build failures

Nas Chung (8):
  media: v4l2-common: Add YUV24 format info
  dt-bindings: media: nxp: Add Wave6 video codec device
  media: chips-media: wave6: Add Wave6 driver
  media: chips-media: wave6: Add Wave6 control driver
  media: chips-media: wave6: Add Wave6 VPU interface
  media: chips-media: wave6: Improve debugging capabilities
  media: chips-media: wave6: Add v4l2 m2m driver support
  media: chips-media: wave6: Add Wave6 core driver

 .../bindings/media/cnm,wave633c.yaml          |  165 +
 MAINTAINERS                                   |    8 +
 drivers/media/platform/chips-media/Kconfig    |    1 +
 drivers/media/platform/chips-media/Makefile   |    1 +
 .../media/platform/chips-media/wave6/Kconfig  |   35 +
 .../media/platform/chips-media/wave6/Makefile |   20 +
 .../platform/chips-media/wave6/wave6-hw.c     | 3108 +++++++++++++++++
 .../platform/chips-media/wave6/wave6-hw.h     |   69 +
 .../chips-media/wave6/wave6-regdefine.h       |  675 ++++
 .../platform/chips-media/wave6/wave6-trace.h  |  336 ++
 .../platform/chips-media/wave6/wave6-vdi.c    |   49 +
 .../platform/chips-media/wave6/wave6-vdi.h    |   45 +
 .../chips-media/wave6/wave6-vpu-core.c        |  461 +++
 .../chips-media/wave6/wave6-vpu-core.h        |  105 +
 .../chips-media/wave6/wave6-vpu-ctrl.c        |  860 +++++
 .../chips-media/wave6/wave6-vpu-dbg.c         |  230 ++
 .../chips-media/wave6/wave6-vpu-dbg.h         |   14 +
 .../chips-media/wave6/wave6-vpu-dec.c         | 1854 ++++++++++
 .../chips-media/wave6/wave6-vpu-enc.c         | 2699 ++++++++++++++
 .../chips-media/wave6/wave6-vpu-v4l2.c        |  387 ++
 .../platform/chips-media/wave6/wave6-vpu.c    |  469 +++
 .../platform/chips-media/wave6/wave6-vpu.h    |   85 +
 .../platform/chips-media/wave6/wave6-vpuapi.c |  998 ++++++
 .../platform/chips-media/wave6/wave6-vpuapi.h |  987 ++++++
 .../chips-media/wave6/wave6-vpuconfig.h       |   84 +
 .../chips-media/wave6/wave6-vpuerror.h        |  262 ++
 drivers/media/v4l2-core/v4l2-common.c         |    1 +
 27 files changed, 14008 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/cnm,wave633c.yaml
 create mode 100644 drivers/media/platform/chips-media/wave6/Kconfig
 create mode 100644 drivers/media/platform/chips-media/wave6/Makefile
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-hw.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-regdefine.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-trace.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vdi.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-core.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-ctrl.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dbg.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-dec.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-enc.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu-v4l2.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpu.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.c
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuapi.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuconfig.h
 create mode 100644 drivers/media/platform/chips-media/wave6/wave6-vpuerror.h