| Message ID | 20220208155027.891055-1-jeanmichel.hautbois@ideasonboard.com |
|---|---|
| Headers | show |
| Series | Add support for BCM2835 camera interface (unicam) | expand |
Hi Jean-Michel, Am 08.02.22 um 16:50 schrieb Jean-Michel Hautbois: > Add both MIPI CSI-2 nodes in the core bcm2711 tree. Use the 3-cells > interrupt declaration, corresponding clocks and default as disabled. > > Signed-off-by: Jean-Michel Hautbois <jeanmichel.hautbois@ideasonboard.com> > Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com> > --- > arch/arm/boot/dts/bcm2711-rpi.dtsi | 15 +++++++++++++++ > arch/arm/boot/dts/bcm2711.dtsi | 22 ++++++++++++++++++++++ > 2 files changed, 37 insertions(+) > > diff --git a/arch/arm/boot/dts/bcm2711-rpi.dtsi b/arch/arm/boot/dts/bcm2711-rpi.dtsi > index ca266c5d9f9b..97ee494891af 100644 > --- a/arch/arm/boot/dts/bcm2711-rpi.dtsi > +++ b/arch/arm/boot/dts/bcm2711-rpi.dtsi > @@ -1,6 +1,7 @@ > // SPDX-License-Identifier: GPL-2.0 > #include "bcm2835-rpi.dtsi" > > +#include <dt-bindings/power/raspberrypi-power.h> > #include <dt-bindings/reset/raspberrypi,firmware-reset.h> > > / { > @@ -18,6 +19,20 @@ aliases { > }; > }; > > +&csi0 { > + clocks = <&clocks BCM2835_CLOCK_CAM0>, > + <&firmware_clocks 4>; > + clock-names = "lp", "vpu"; > + power-domains = <&power RPI_POWER_DOMAIN_UNICAM0>; > +}; > + > +&csi1 { > + clocks = <&clocks BCM2835_CLOCK_CAM1>, > + <&firmware_clocks 4>; > + clock-names = "lp", "vpu"; > + power-domains = <&power RPI_POWER_DOMAIN_UNICAM1>; > +}; see comment below > + > &firmware { > firmware_clocks: clocks { > compatible = "raspberrypi,firmware-clocks"; > diff --git a/arch/arm/boot/dts/bcm2711.dtsi b/arch/arm/boot/dts/bcm2711.dtsi > index dff18fc9a906..21eb10122e50 100644 > --- a/arch/arm/boot/dts/bcm2711.dtsi > +++ b/arch/arm/boot/dts/bcm2711.dtsi > @@ -293,6 +293,28 @@ hvs: hvs@7e400000 { > interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>; > }; > > + csi0: csi@7e800000 { > + compatible = "brcm,bcm2835-unicam"; > + reg = <0x7e800000 0x800>, > + <0x7e802000 0x4>; > + reg-names = "unicam", "cmi"; > + interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>; > + status = "disabled"; > + port { > + }; > + }; > + > + csi1: csi@7e801000 { > + compatible = "brcm,bcm2835-unicam"; > + reg = <0x7e801000 0x800>, > + <0x7e802004 0x4>; > + reg-names = "unicam", "cmi"; > + interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>; > + status = "disabled"; > + port { > + }; > + }; > + i didn't noticed before that everything except of the interrupts are identical to the bcm283x boards [1]. So please define both nodes in bcm283x.dtsi (with bcm283x interrupts) and just override the interrupts for bcm2711 in this dtsi file. As a result the clocks and power domains from above can go to bcm2835-rpi.dtsi. > pixelvalve3: pixelvalve@7ec12000 { > compatible = "brcm,bcm2711-pixelvalve3"; > reg = <0x7ec12000 0x100>;
Am 13.02.22 um 11:35 schrieb Stefan Wahren: > Hi Jean-Michel, > > Am 08.02.22 um 16:50 schrieb Jean-Michel Hautbois: >> + >> &firmware { >> firmware_clocks: clocks { >> compatible = "raspberrypi,firmware-clocks"; >> diff --git a/arch/arm/boot/dts/bcm2711.dtsi b/arch/arm/boot/dts/bcm2711.dtsi >> index dff18fc9a906..21eb10122e50 100644 >> --- a/arch/arm/boot/dts/bcm2711.dtsi >> +++ b/arch/arm/boot/dts/bcm2711.dtsi >> @@ -293,6 +293,28 @@ hvs: hvs@7e400000 { >> interrupts = <GIC_SPI 97 IRQ_TYPE_LEVEL_HIGH>; >> }; >> >> + csi0: csi@7e800000 { >> + compatible = "brcm,bcm2835-unicam"; >> + reg = <0x7e800000 0x800>, >> + <0x7e802000 0x4>; >> + reg-names = "unicam", "cmi"; >> + interrupts = <GIC_SPI 102 IRQ_TYPE_LEVEL_HIGH>; >> + status = "disabled"; >> + port { >> + }; >> + }; >> + >> + csi1: csi@7e801000 { >> + compatible = "brcm,bcm2835-unicam"; >> + reg = <0x7e801000 0x800>, >> + <0x7e802004 0x4>; >> + reg-names = "unicam", "cmi"; >> + interrupts = <GIC_SPI 103 IRQ_TYPE_LEVEL_HIGH>; >> + status = "disabled"; >> + port { >> + }; >> + }; >> + > i didn't noticed before that everything except of the interrupts are > identical to the bcm283x boards [1]. So please define both nodes in > bcm283x.dtsi (with bcm283x interrupts) and just override the interrupts > for bcm2711 in this dtsi file. > > As a result the clocks and power domains from above can go to > bcm2835-rpi.dtsi. sorry, i forgot the downstream link: [1] - https://github.com/raspberrypi/linux/blob/rpi-5.15.y/arch/arm/boot/dts/bcm270x.dtsi#L88 > >> pixelvalve3: pixelvalve@7ec12000 { >> compatible = "brcm,bcm2711-pixelvalve3"; >> reg = <0x7ec12000 0x100>;
Hi Jean-Michel, Am 20.02.22 um 15:30 schrieb Jean-Michel Hautbois: > Hi Stefan, > > On 16/02/2022 21:57, Stefan Wahren wrote: >> Hi Jean-Michel, >> >> Am 08.02.22 um 16:50 schrieb Jean-Michel Hautbois: >>> Hello ! >>> >>> ... >>> >>> In order to properly configure the media pipeline, it is needed to call >>> the usual ioctls, and configure routing in order to send the embedded >>> data from the sensor to the "unicam-embedded" device node : >>> >>> ``` >>> media=0 >>> media-ctl -d${media} -l "'imx219 2-0010':0->'unicam-subdev':0 [1]" >>> media-ctl -d${media} -l "'unicam-subdev':1->'unicam-image':0 [1]" >>> media-ctl -d${media} -v -R "'unicam-subdev' [0/0->1/0[1],0/1->2/0[1]]" >>> media-ctl -d${media} -V "'imx219 2-0010':0/0 >>> [fmt:SRGGB10_1X10/3280x2464 field:none]" >>> v4l2-ctl -d0 --set-fmt-video >>> width=3280,height=2464,pixelformat='pRAA',field=none >>> media-ctl -d${media} -v -V "'imx219 2-0010':0/1 >>> [fmt:METADATA_8/16384x1 field:none]" >>> media-ctl -d${media} -p >>> ``` >> >> i tried to test the unicam driver on a Raspberry Pi 4 with the imx219 >> camera (based on 5.17-rc4). The unicam & imx219 driver probes and >> /dev/video0 is created. >> >> If a execute the first media-ctl command, it complains with invalid >> argument 22. Is there a more fool-proof variant to configure this (a >> script or something else)? I never used the unicam driver before. >> >> Here is the output of >> >> $ mediactl -d0 -p >> > > Based on your output, I suppose the issue is the naming of the imx219 > media entity ('imx219 2-0010' vs 'imx219 5-0010'). > You could add a '-v' in the line to help you I suppose. > > A more bullet-proof version of the commands is certainly doable, not > sure how though as I would not like to rewrite a libcamera-like > command ;-). Today i had a little bit time and manage to get the commands working. After running the yavta command, the kernel put the following: [ 405.908674] cma: cma_alloc: linux,cma: alloc failed, req-size: 2484 pages, ret: -12 [ 405.908695] unicam fe801000.csi: dma alloc of size 10174464 failed As per default the CMA size with mainline DTB is 64 MB. After that i increase the value to 256 MB and the issue disappeared. Maybe this is something which could be integrated into the overlay. But at least a note about this in the cover letter of the next version of this series would be nice. Best regards
Hello ! This patch series adds the BCM2835 CCP2/CSI2 camera interface named unicam. This driver is already used in the out-of-tree linux-rpi repository [1], and this work aims to support it in mainline. The series is based on top of: - Rebased on top of 5.17-rc1 Tomi Valkeinen's multiplexed stream work, on his multistream/work branch [2] which has been submitted as v10 at the time of this writing. The objective is to demonstrate the use of multiplexed streams on a real world widely used example as well as supporting unicam mainline. - The "Add 12bit and 14bit luma-only formats" series [3] is partly applied (the Y10P format bug) the new formats are now part of this series. The series is made of 11 patches: - 1/11 and 2/11 introduce the new formats needed for the unicam driver - 3/11 introduces dt-bindings documentation - 4/11 adds the driver support in media/platform - 5/11 adds the MAINTAINERS entry - 6/11 introduces the csi nodes in the bcm2711 file, in a disabled state - 7/11 to 11/11 modifies imx219 driver to make it use the multiplexed streams API All those patches are in my tree [4]. Patch 5/12 comes from the linux-rpi work [1] with substantial modifications: - Removed the non-mc API which is deprecated, and not needed upstream - Moved from one video node with two source pads (image and embedded) to two video nodes - Added a subdev between the sensor and the video nodes to properly route the streams - Added support for multiplexed streams API In order to test it, one will need a RPi board and the camv2 (imx219) sensor. An updated v4l-utils is also needed [5] to have support for multiplexed streams. v4l2-compliance passes on both video devices, without streaming testing though with one exception: as the colorspace support is removed in v3, we now have: test VIDIOC_G_FMT: OK fail: v4l2-test-formats.cpp(363): colorspace >= 0xff fail: v4l2-test-formats.cpp(465): testColorspace(!node->is_io_mc, pix.pixelformat, pix.colorspace, pix.ycbcr_enc, pix.quantization) test VIDIOC_TRY_FMT: FAIL fail: v4l2-test-formats.cpp(363): colorspace >= 0xff fail: v4l2-test-formats.cpp(465): testColorspace(!node->is_io_mc, pix.pixelformat, pix.colorspace, pix.ycbcr_enc, pix.quantization) test VIDIOC_S_FMT: FAIL This series since its v2 does integrate the device tree nodes into the upstream BCM2835 or BCM2711 device tree support. In order to properly configure the media pipeline, it is needed to call the usual ioctls, and configure routing in order to send the embedded data from the sensor to the "unicam-embedded" device node : ``` media=0 media-ctl -d${media} -l "'imx219 2-0010':0->'unicam-subdev':0 [1]" media-ctl -d${media} -l "'unicam-subdev':1->'unicam-image':0 [1]" media-ctl -d${media} -v -R "'unicam-subdev' [0/0->1/0[1],0/1->2/0[1]]" media-ctl -d${media} -V "'imx219 2-0010':0/0 [fmt:SRGGB10_1X10/3280x2464 field:none]" v4l2-ctl -d0 --set-fmt-video width=3280,height=2464,pixelformat='pRAA',field=none media-ctl -d${media} -v -V "'imx219 2-0010':0/1 [fmt:METADATA_8/16384x1 field:none]" media-ctl -d${media} -p ``` Be sure to configure the routes before setting the format, as s_routing resets the default format. The media-ctl topology is: ``` pi@raspberrypi:~ $ media-ctl -d${media} -p Media controller API version 5.16.0 Media device information ------------------------ driver unicam model unicam serial bus info platform:fe801000.csi hw revision 0x0 driver version 5.16.0 Device topology - entity 1: unicam-subdev (3 pads, 3 links, 2 routes) type V4L2 subdev subtype Unknown flags 0 device node name /dev/v4l-subdev0 routes: 0/0 -> 1/0 [ACTIVE] 0/1 -> 2/0 [ACTIVE] pad0: Sink [stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw] [stream:1 fmt:METADATA_8/16384x1 field:none] <- "imx219 2-0010":0 [ENABLED,IMMUTABLE] pad1: Source [stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw] -> "unicam-image":0 [ENABLED,IMMUTABLE] pad2: Source [stream:0 fmt:METADATA_8/16384x1 field:none] -> "unicam-embedded":0 [ENABLED,IMMUTABLE] - entity 5: imx219 2-0010 (1 pad, 1 link, 2 routes) type V4L2 subdev subtype Sensor flags 0 device node name /dev/v4l-subdev1 routes: 0/0 -> 0/0 [ACTIVE, IMMUTABLE, SOURCE] 0/0 -> 0/1 [ACTIVE, SOURCE] pad0: Source [stream:0 fmt:SRGGB10_1X10/3280x2464 field:none colorspace:raw crop.bounds:(8,8)/3280x2464 crop:(8,8)/3280x2464] [stream:1 fmt:METADATA_8/16384x1 field:none crop.bounds:(8,8)/3280x2464 crop:(8,8)/3280x2464] -> "unicam-subdev":0 [ENABLED,IMMUTABLE] - entity 9: unicam-image (1 pad, 1 link, 0 route) type Node subtype V4L flags 1 device node name /dev/video0 pad0: Sink <- "unicam-subdev":1 [ENABLED,IMMUTABLE] - entity 15: unicam-embedded (1 pad, 1 link, 0 route) type Node subtype V4L flags 0 device node name /dev/video1 pad0: Sink <- "unicam-subdev":2 [ENABLED,IMMUTABLE] ``` Then a frame can be capture with yavta: `yavta --capture=1 /dev/video0 -F/tmp/test-#.bin` In v3, capture on the metadata node (/dev/video1 in my case) can't be started if capture on image node (/dev/video0) is not already started. This can be tested using yavta, letting it capture frames indefinitely and start another yavta instance on the /dev/video1 node. [1]: https://github.com/raspberrypi/linux/tree/rpi-5.15.y [2]: https://git.kernel.org/pub/scm/linux/kernel/git/tomba/linux.git/log/?h=multistream/work [3]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7099 [4]: https://github.com/jhautbois/linux-rpi/tree/jmh/unicam-multiplexed-streams [5]: https://github.com/jhautbois/v4l-utils/tree/jmh/tomi-multiplexed-streams Jean-Michel Hautbois (11): media: v4l: Add V4L2-PIX-FMT-Y12P format media: v4l: Add V4L2-PIX-FMT-Y14P format dt-bindings: media: Add bindings for bcm2835-unicam media: bcm2835-unicam: Add support for CCP2/CSI2 camera interface media: MAINTAINERS: add bcm2835 unicam driver ARM: dts: bcm2711: Add unicam CSI nodes media: imx219: Rename mbus codes array media: imx219: Switch from open to init_cfg media: imx219: Introduce the set_routing operation media: imx219: use a local v4l2_subdev to simplify reading media: imx219: Add support for the V4L2 subdev active state .../bindings/media/brcm,bcm2835-unicam.yaml | 117 + .../media/v4l/pixfmt-yuv-luma.rst | 44 + MAINTAINERS | 8 + arch/arm/boot/dts/bcm2711-rpi.dtsi | 15 + arch/arm/boot/dts/bcm2711.dtsi | 22 + drivers/media/i2c/imx219.c | 324 ++- drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile | 2 + drivers/media/platform/bcm2835/Kconfig | 21 + drivers/media/platform/bcm2835/Makefile | 3 + .../platform/bcm2835/bcm2835-unicam-regs.h | 253 ++ .../media/platform/bcm2835/bcm2835-unicam.c | 2570 +++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 2 + include/uapi/linux/videodev2.h | 2 + 14 files changed, 3256 insertions(+), 128 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/brcm,bcm2835-unicam.yaml create mode 100644 drivers/media/platform/bcm2835/Kconfig create mode 100644 drivers/media/platform/bcm2835/Makefile create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam-regs.h create mode 100644 drivers/media/platform/bcm2835/bcm2835-unicam.c