From patchwork Wed May 14 15:41:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Michael Riesch via B4 Relay X-Patchwork-Id: 890072 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D76C71CB518; Wed, 14 May 2025 15:53:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747237983; cv=none; b=AUhAlOonlnxWuKAzP2AzybHq5Yg0gyIXKAKZZkX3miDzi70IrS6QKGcl1wnPiuGFwtm39op9buF9xO2QEkA7PSOkYUD5cIFNJ/xoXlwHew3w7KrwaTR88+/325TJcR4HE5eohK5+0OEQPm2Np6eLoXP0J5nAhSJvhqraQxKvKV8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1747237983; c=relaxed/simple; bh=DOpSdkG/S5EqqBi/SrOg3DjxqUug30rZ7ybUcIwe4ms=; h=From:Subject:Date:Message-Id:MIME-Version:Content-Type:To:Cc; b=b0m6CarePKwMk2pqidUsDczr6okmp5kWdZm5JLkjj+6CjGfhL/oDIj0mGucK7hlAQ3LN+svSq2BVKoSFB/d5E3U/HNBhu8rLSTUuIaG4JSN3GmqNd9uQhKA7N1eqcPNQ3Nqt/E/OA3VK26DNZayGnEFL0k17d2FNYb6582MV314= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=toKuLSgl; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="toKuLSgl" Received: by smtp.kernel.org (Postfix) with ESMTPS id 9980EC4CEE3; Wed, 14 May 2025 15:53:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1747237982; bh=DOpSdkG/S5EqqBi/SrOg3DjxqUug30rZ7ybUcIwe4ms=; h=From:Subject:Date:To:Cc:Reply-To:From; b=toKuLSglANWwZ8qrrOyaBWQuRW1FQBzWeIQhqKSZm+rQZnakw9zFaVIIEqX76ja0/ +XYHkmvV60tVEq/ZyCf0wARlzxaZqNDNymVdM8iZX7hYG0xOCw1eSLCQ98DOqWSaQJ +GsVsqCItJjq9SQt9N825KSoZm1N+B1+JwTtZMQt3T13gPM+nveTOKU80ZrOHCeVAw 1OHH4R1HroTm6h7u8dGZeJDwmz42frnQ4UJqhiLtG9xHS2pM7F5IZzGNjAhEui8GCp yL55p7vmTbm+k+2wUXF5Pqst14pUDkXSUKVAeTVaG4RcOrBXhqnunwDxqsblYJlwnG CWc8iRkuAD3lw== Received: from aws-us-west-2-korg-lkml-1.web.codeaurora.org (localhost.localdomain [127.0.0.1]) by smtp.lore.kernel.org (Postfix) with ESMTP id 848E4C3ABD8; Wed, 14 May 2025 15:53:02 +0000 (UTC) From: Michael Riesch via B4 Relay Subject: [PATCH v7 00/14] media: rockchip: add a driver for the rockchip camera interface Date: Wed, 14 May 2025 17:41:01 +0200 Message-Id: <20240220-rk3568-vicap-v7-0-7581fd96a33a@collabora.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-B4-Tracking: v=1; b=H4sIAI25JGgC/33Ry2rDMBAF0F8JWldh9IzVVf6jZKHHqBZNLCMbt SXk3yuntGnA6fJKzLkDcyYTloQTed6cScGappSHFnZPG+J7O7wiTaFlwoFL4BxoeRNKd7Qmb0f qjBPRGK6ilKSNODshdcUOvl+GTtiHRCsTy99YMKaPa9XLoeU+TXMun9fmypbXW0nVtKNzHpO/7 6uMAuW6A8ui1RLc/j0f4/fW2wFnssCV/2KMs91jjDfMsSC7LnoPQaxi4gdTwEE/xkTDtAksMoP KS1jF5B+MmceYbBga0BrAonDrmLph4r/NVMNi61NcCRmFX8X0/QHuBd2EwKOKDpEpxfY+H4/W5 WK3Pp/I4XK5fAGYiItKSQIAAA== To: Mehdi Djait , Maxime Chevallier , =?utf-8?q?Th=C3=A9o_Leb?= =?utf-8?q?run?= , Thomas Petazzoni , Gerald Loacker , Bryan O'Donoghue , Markus Elfring , Laurent Pinchart , Mauro Carvalho Chehab , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Heiko Stuebner , Kever Yang , Nicolas Dufresne , Sebastian Reichel , Collabora Kernel Team , Paul Kocialkowski , Alexander Shiyan , Val Packett , Rob Herring , Philipp Zabel , Sakari Ailus Cc: linux-media@vger.kernel.org, devicetree@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-rockchip@lists.infradead.org, Michael Riesch , Michael Riesch , Mehdi Djait , Krzysztof Kozlowski X-Mailer: b4 0.12.0 X-Developer-Signature: v=1; a=ed25519-sha256; t=1747237265; l=9970; i=michael.riesch@collabora.com; s=20250410; h=from:subject:message-id; bh=DOpSdkG/S5EqqBi/SrOg3DjxqUug30rZ7ybUcIwe4ms=; b=OJxOjjOuE5EpIxnxd1FkziZY6SzedCn6/7oF/ePCPWsuR8XHq5S+5YVNZjWBgpOiaD9EApWHC V7UOjR92lavCuu+wTptoUgbl6uWubWccM+sf+17QKcg1dVXRh+jSaWx X-Developer-Key: i=michael.riesch@collabora.com; a=ed25519; pk=+MWX1fffLFZtTPG/I6XdYm/+OSvpRE8D9evQaWbiN04= X-Endpoint-Received: by B4 Relay for michael.riesch@collabora.com/20250410 with auth_id=371 X-Original-From: Michael Riesch Reply-To: michael.riesch@collabora.com Habidere, This series introduces support for the Rockchip Camera Interface (CIF), which is featured in many Rockchip SoCs in different variations. For example, the PX30 Video Input Processor (VIP) is able to receive video data via the Digital Video Port (DVP, a parallel data interface) and transfer it into system memory using a double-buffering mechanism called ping-pong mode. The RK3568 Video Capture (VICAP) unit, on the other hand, features a DVP and a MIPI CSI-2 receiver that can receive video data independently (both using the ping-pong scheme). The different variants may have additional features, such as scaling and/or cropping. Finally, the RK3588 VICAP unit constitutes an essential piece of the camera interface with one DVP, six MIPI CSI-2 receivers, scale/crop units, and a data path multiplexer (to scaler units, to ISP, ...). The v7 of the series adds a media controller centric V4L2 device driver for the Rockchip CIF with - support for the PX30 VIP (not tested, though, due to the lack of HW) - support for the RK3568 VICAP, including - capturing frames from the DVP - capturing frames from the MIPI CSI-2 receiver - abstraction for the ping-pong scheme to allow for future extensions - abstraction for the INTERFACE and CROP parts to allow for future extensions - initial support for different virtual channels (not tested, though, due to the lack of HW) and a V4L2 subdevice driver for the Rockchip MIPI CSI-2 Receiver. The patches are functional and have been tested successfully on a custom RK3568 board including the ITE Tech. IT6801 HDMI receiver and the Sony IMX415 image sensor as subdevices attached to the DVP and the MIPI CSI-2 receiver, respectively. The IT6801 driver still needs some loving care but shall be submitted as well at some point. However, several features are not yet addressed, such as - support for the RK3588 variant (-> next item on my TODO) - support for the MUX/SCALE/TOISP block in the RK3588 VICAP (which provides the base for image processing on the RK3588) - support for the scaling unit in the PX30 (-> cannot do due to the lack of HW) - support for the interface to the Rockchip ISP in the RK3568 (-> cannot do, latest information from Rockchip points out there IS NO HW CONNECTION BETWEEN VICAP AND ISP ON RK3568) Looking forward to your comments! To: Mehdi Djait To: Maxime Chevallier To: Théo Lebrun To: Thomas Petazzoni To: Gerald Loacker To: Bryan O'Donoghue To: Markus Elfring To: Sakari Ailus To: Laurent Pinchart To: Mauro Carvalho Chehab To: Rob Herring To: Krzysztof Kozlowski To: Conor Dooley To: Heiko Stuebner To: Kever Yang To: Nicolas Dufresne To: Sebastian Reichel To: Collabora Kernel Team To: Paul Kocialkowski To: Alexander Shiyan To: Val Packett To: Rob Herring To: Philipp Zabel Cc: linux-media@vger.kernel.org Cc: devicetree@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: linux-arm-kernel@lists.infradead.org Cc: linux-rockchip@lists.infradead.org Signed-off-by: Michael Riesch Signed-off-by: Michael Riesch Changes in v7: - moved MIPI CSI-2 receiver driver into separate directory (Laurent) - rkcsi: fixed return values (Bryan) - rkcsi: fixed v4l2_get_link_freq to use pad instead of control handler (Sakari) - rkcsi: added data-lanes property (Mehdi) - rkcif: fixed formatting (Bryan) - fixed "int" -> "unsigned int" in all for loops (Sakari) - rkcif-stream: fixed minimum number of required buffers (Mehdi) - rkcif-stream: used guards for the spinlock (Markus and Mehdi) - rkcif-stream: made driver less noisy with dev_dbg (Mehdi) - rkcif-stream: fixed issues detected by v4l2-compliance (Mehdi) - rkcif-dvp-capture: fixed return value propagation in _register() (Mehdi) - removed stray "phy-names" from required properties (Rob's bot) - Link to v6: https://lore.kernel.org/r/20240220-rk3568-vicap-v6-0-d2f5fbee1551@collabora.com Changes in v6: - rebased onto v6.15-rc1 - renamed "MIPI CSI HOST" -> "MIPI CSI RECEIVER" (Laurent) - s/@wolfvision.net/@collabora.com where appropriate - renamed DVP delay property and moved it to the endpoint (Sakari) - implemented DT review comments (Krzysztof and Sakari) - implemented driver review comments (Sakari) - fixed issues raised by media-ci (yet again) - added documentation including a RK3568 topology (new patch 1) (Sakari) - added patch that enables rkcif in the defconfig (new patch 9) - Link to v5: https://lore.kernel.org/r/20250306-v6-8-topic-rk3568-vicap-v5-0-f02152534f3c@wolfvision.net Changes in v5: - fixed issues raised by media-ci - fixed dt bindings (comments by Rob and Sakari) - fixed probe on systems with no DVP in DT (comment by Alexander) - fixed error path in register offset calculation - split off MIPI CSI host driver into separate module (comment by Mehdi) - added MODULE_DEVICE_TABLE() for both drivers (comment by Mehdi) - Link to v4: https://lore.kernel.org/r/20250219-v6-8-topic-rk3568-vicap-v4-0-e906600ae3b0@wolfvision.net Changes in v4: - added support for the MIPI CSI-2 receiver (new patches 4, 6, 7, 10) - fixed asserts on stream stop - fixed register address lookup - fixed link validiation callback - fixed issues raised by Rob's bot, kernel test robot, and media-ci - Link to v3: https://lore.kernel.org/r/20250206-v6-8-topic-rk3568-vicap-v3-0-69d1f19e5c40@wolfvision.net Changes in v3: - renamed the driver "cif" -> "rkcif" - rebased onto v6.14-rc1 - abstracted the generic INTERFACE+CROP part - addressed comments by Rob and Sakari - added V4L2 MPLANE formats to DVP - added patch that enables the RK3568 VICAP DVP on PF5 IO Expander - fixed formatting issues raised by media-ci bot - Link to v2: https://lore.kernel.org/r/20241217-v6-8-topic-rk3568-vicap-v2-0-b1d488fcc0d3@wolfvision.net Changes in v2: - merged with Mehdi's v13 - refactored the complete driver towards a media controller centric driver - abstracted the generic ping-pong stream (can be used for DVP as well as for CSI-2) - switched to MPLANE API - added support for notifications - Link to v1: https://lore.kernel.org/r/20240220-v6-8-topic-rk3568-vicap-v1-0-2680a1fa640b@wolfvision.net Reviewed-by: Gerald Loacker Tested-by: Gerald Loacker Tested-by: Gerald Loacker --- Mehdi Djait (2): media: dt-bindings: add rockchip px30 vip arm64: dts: rockchip: add the vip node to px30 Michael Riesch (12): Documentation: admin-guide: media: add rockchip camera interface media: dt-bindings: video-interfaces: add defines for sampling modes media: dt-bindings: add rockchip rk3568 vicap media: dt-bindings: add rockchip rk3568 mipi csi-2 receiver media: rockchip: add a driver for the rockchip camera interface media: rockchip: add driver for mipi csi-2 receiver media: rockchip: rkcif: add support for mipi csi-2 capture arm64: defconfig: enable rockchip camera interface arm64: defconfig: enable rockchip mipi csi-2 receiver arm64: dts: rockchip: add vicap node to rk356x arm64: dts: rockchip: add mipi csi-2 receiver node to rk356x arm64: dts: rockchip: enable vicap dvp on wolfvision pf5 io expander .../admin-guide/media/rkcif-rk3568-vicap.dot | 21 + Documentation/admin-guide/media/rkcif.rst | 83 ++ Documentation/admin-guide/media/v4l-drivers.rst | 1 + .../bindings/media/rockchip,px30-vip.yaml | 122 +++ .../bindings/media/rockchip,rk3568-mipi-csi.yaml | 117 +++ .../bindings/media/rockchip,rk3568-vicap.yaml | 170 ++++ MAINTAINERS | 17 + arch/arm64/boot/dts/rockchip/px30.dtsi | 12 + .../rk3568-wolfvision-pf5-io-expander.dtso | 20 + arch/arm64/boot/dts/rockchip/rk356x-base.dtsi | 75 ++ arch/arm64/configs/defconfig | 2 + drivers/media/platform/rockchip/Kconfig | 2 + drivers/media/platform/rockchip/Makefile | 2 + drivers/media/platform/rockchip/rkcif/Kconfig | 18 + drivers/media/platform/rockchip/rkcif/Makefile | 7 + .../platform/rockchip/rkcif/rkcif-capture-dvp.c | 864 +++++++++++++++++++++ .../platform/rockchip/rkcif/rkcif-capture-dvp.h | 24 + .../platform/rockchip/rkcif/rkcif-capture-mipi.c | 722 +++++++++++++++++ .../platform/rockchip/rkcif/rkcif-capture-mipi.h | 22 + .../media/platform/rockchip/rkcif/rkcif-common.h | 236 ++++++ drivers/media/platform/rockchip/rkcif/rkcif-dev.c | 302 +++++++ .../platform/rockchip/rkcif/rkcif-interface.c | 425 ++++++++++ .../platform/rockchip/rkcif/rkcif-interface.h | 30 + drivers/media/platform/rockchip/rkcif/rkcif-regs.h | 154 ++++ .../media/platform/rockchip/rkcif/rkcif-stream.c | 634 +++++++++++++++ .../media/platform/rockchip/rkcif/rkcif-stream.h | 31 + drivers/media/platform/rockchip/rkcsi/Kconfig | 15 + drivers/media/platform/rockchip/rkcsi/Makefile | 3 + drivers/media/platform/rockchip/rkcsi/rkcsi.c | 735 ++++++++++++++++++ include/dt-bindings/media/video-interfaces.h | 4 + 30 files changed, 4870 insertions(+) --- base-commit: 82f2b0b97b36ee3fcddf0f0780a9a0825d52fec3 change-id: 20240220-rk3568-vicap-b9b3f9925f44 Best regards,