mbox series

[v2,0/2] media: Add driver for ST VD56G3 camera sensor

Message ID 20240521162950.6987-1-sylvain.petinot@foss.st.com
Headers show
Series media: Add driver for ST VD56G3 camera sensor | expand

Message

Sylvain Petinot May 21, 2024, 4:29 p.m. UTC
Hello,

This serie adds support for STMicroelectronics VD56G3 camera sensor.
This is a 1.5M pixel global shutter camera available in both Mono (VD56G3) and
colour (VD66GY) variants.

The following features are supported:
- Auto exposure with expo bias or
- Manual exposure with analog / digital gain
- H/V flip
- vblank/hblank/link freq
- Test pattern
- Supported resolutions in both raw8/raw10 :
   - 1124x1364
   - 1120x1360
   - 1024x1280
   - 1024x768
   - 768x1024
   - 720x1280
   - 640x480
   - 480x640
   - 320x240

This driver supports coldstart parameters for internal AE feature.
To make it work, the driver save gain/expo values in ctrl's cur.val during
poweroff phase. This implementation transgress V4L2 rules... Any advice to make
it proper would be greatly appreciated.

Driver tested on RB5 and RPI (with and without libcamera) for V1. V2 mainly
tested on RPI.

---

v1 -> v2:
- driver: Drop VD56G3_NUM_SUPPLIES
- driver: Rename 'ext_clock' to 'xclk_freq'
- driver: Make use of 'container_of_const' instead of 'container_of'
- driver: Drop usage of WARN()
- driver: Move a few variables to unsigned int
- driver: Add defines for the different Cut revisions
- driver: Replace dev_warn() by dev_err() in situation we're returning errors
- driver: Ensure sensor has dedicated 3.5ms to boot after reset
- driver: Take into account return value of __v4l2_ctrl_modify_range() and
  __v4l2_ctrl_s_ctrl() functions
- driver: Merge vd56g3_power_on() and vd56g3_boot()
- dt-bindings: Lowercase power supply names
- dt-bindings: Drop clock-lanes property
- dt-bindings: Drop unecessary 'items' contraint for lane-polarities
- dt-bindings: Drop unused labels


Sylvain Petinot (2):
  media: dt-bindings: Add ST VD56G3 camera sensor binding
  media: i2c: Add driver for ST VD56G3 camera sensor

 .../bindings/media/i2c/st,st-vd56g3.yaml      |  132 ++
 MAINTAINERS                                   |    9 +
 drivers/media/i2c/Kconfig                     |   11 +
 drivers/media/i2c/Makefile                    |    1 +
 drivers/media/i2c/st-vd56g3.c                 | 1608 +++++++++++++++++
 5 files changed, 1761 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
 create mode 100644 drivers/media/i2c/st-vd56g3.c

Comments

Krzysztof Kozlowski May 21, 2024, 5:37 p.m. UTC | #1
On 21/05/2024 18:29, Sylvain Petinot wrote:
> Add devicetree bindings Documentation for ST VD56G3 & ST VD66GY camera
> sensors. Update MAINTAINERS file.
> 

A nit, subject: drop second/last, redundant "binding". The "dt-bindings"
prefix is already stating that these are bindings.
See also:
https://elixir.bootlin.com/linux/v6.7-rc8/source/Documentation/devicetree/bindings/submitting-patches.rst#L18


> Signed-off-by: Sylvain Petinot <sylvain.petinot@foss.st.com>
> ---
>  .../bindings/media/i2c/st,st-vd56g3.yaml      | 132 ++++++++++++++++++
>  MAINTAINERS                                   |   9 ++
>  2 files changed, 141 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
> 
> diff --git a/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
> new file mode 100644
> index 000000000000..22cb2557e311
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml

Why duplicated 'st'?

> @@ -0,0 +1,132 @@
> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
> +# Copyright (c) 2024 STMicroelectronics SA.
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/media/i2c/st,st-vd56g3.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: STMicroelectronics VD56G3 Global Shutter Image Sensor
> +
> +maintainers:
> +  - Benjamin Mugnier <benjamin.mugnier@foss.st.com>
> +  - Sylvain Petinot <sylvain.petinot@foss.st.com>
> +
> +description: |-
> +  The STMicroelectronics VD56G3 is a 1.5 M pixel global shutter image sensor

This claims device is VD56G3, not ST-VD56G3.

> +  with an active array size of 1124 x 1364 (portrait orientation). It is
> +  programmable through I2C, the address is fixed to 0x10. The sensor output is
> +  available via CSI-2, which is configured as either 1 or 2 data lanes. The
> +  sensor provides 8 GPIOS that can be used for external LED signal
> +  (synchronized with sensor integration periods)
> +
> +properties:
> +  compatible:
> +    enum:
> +      - st,st-vd56g3
> +      - st,st-vd66gy
> +    description:
> +      Two variants are availables; VD56G3 is a monochrome sensor while VD66GY
> +      is a colour variant.
> +
> +  reg:
> +    maxItems: 1
> +
> +  clocks:
> +    maxItems: 1
> +
> +  vcore-supply:
> +    description: Digital core power supply (1.15V)
> +
> +  vddio-supply:
> +    description: Digital IO power supply (1.8V)
> +
> +  vana-supply:
> +    description: Analog power supply (2.8V)
> +
> +  reset-gpios:
> +    description: Sensor reset active low GPIO (XSHUTDOWN)
> +    maxItems: 1
> +
> +  st,leds:
> +    description:
> +      Sensor's GPIOs used for external LED control. Signal being the enveloppe
> +      of the integration time.

More information is needed. GPIOs coming from LED or SoC? What's the
meaning of values?

> +    $ref: /schemas/types.yaml#/definitions/uint32-array
> +    minItems: 1
> +    maxItems: 8
> +    items:
> +      minimum: 0
> +      maximum: 7
> +
> +  port:
> +    $ref: /schemas/graph.yaml#/$defs/port-base

missing additionalProperties: false

> +
> +    properties:
> +      endpoint:
> +        $ref: /schemas/media/video-interfaces.yaml#
> +        unevaluatedProperties: false
> +
> +        properties:
> +          data-lanes:
> +            minItems: 1
> +            maxItems: 2
> +            items:
> +              enum: [1, 2]


> +
> +          link-frequencies:
> +            minItems: 1

maxItems is enough

> +            maxItems: 1
> +            items:
> +              enum: [402000000, 750000000]
> +
> +          lane-polarities:
> +            minItems: 1
> +            maxItems: 3
> +            description: Any lane can be inverted or not.
> +
> +        required:
> +          - data-lanes
> +          - link-frequencies
> +
> +required:
> +  - compatible
> +  - reg
> +  - clocks
> +  - vcore-supply
> +  - vddio-supply
> +  - vana-supply
> +  - reset-gpios
> +  - port
> +


Not a video-interface-device.yaml type of device?

Best regards,
Krzysztof
Sylvain Petinot May 27, 2024, 1:14 p.m. UTC | #2
Hi Krzysztof,

Thanks for the review.

On 5/21/2024 7:37 PM, Krzysztof Kozlowski wrote:
> On 21/05/2024 18:29, Sylvain Petinot wrote:
>> Add devicetree bindings Documentation for ST VD56G3 & ST VD66GY camera
>> sensors. Update MAINTAINERS file.
>>
> 
> A nit, subject: drop second/last, redundant "binding". The "dt-bindings"
> prefix is already stating that these are bindings.
> See also:
> https://elixir.bootlin.com/linux/v6.7-rc8/source/Documentation/devicetree/bindings/submitting-patches.rst#L18
> 

Ok, fixed in V3.

> 
>> Signed-off-by: Sylvain Petinot <sylvain.petinot@foss.st.com>
>> ---
>>  .../bindings/media/i2c/st,st-vd56g3.yaml      | 132 ++++++++++++++++++
>>  MAINTAINERS                                   |   9 ++
>>  2 files changed, 141 insertions(+)
>>  create mode 100644 Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
>>
>> diff --git a/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
>> new file mode 100644
>> index 000000000000..22cb2557e311
>> --- /dev/null
>> +++ b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
> 
> Why duplicated 'st'?

Legacy : our first st-mipid02 driver was upstream this way few years back.

We have 3 options :

1- keep this unpleasant naming to keep consistency with st-mipid02 [1]
and st-vgxy61 [2]
2- rename this driver properly ('vd56g3') and keep the two others the
old way (I personally don't like this option)
3- rename this driver properly ('vd56g3') and in a second patch rename
the two others drivers.

I would be interested to get Sakari's opinion on this subject.

[1]:
https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-mipid02.c

[2]:
https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-vgxy61.c

> 
>> @@ -0,0 +1,132 @@
>> +# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
>> +# Copyright (c) 2024 STMicroelectronics SA.
>> +%YAML 1.2
>> +---
>> +$id: http://devicetree.org/schemas/media/i2c/st,st-vd56g3.yaml#
>> +$schema: http://devicetree.org/meta-schemas/core.yaml#
>> +
>> +title: STMicroelectronics VD56G3 Global Shutter Image Sensor
>> +
>> +maintainers:
>> +  - Benjamin Mugnier <benjamin.mugnier@foss.st.com>
>> +  - Sylvain Petinot <sylvain.petinot@foss.st.com>
>> +
>> +description: |-
>> +  The STMicroelectronics VD56G3 is a 1.5 M pixel global shutter image sensor
> 
> This claims device is VD56G3, not ST-VD56G3.

Sure, linked with previous point.

> 
>> +  with an active array size of 1124 x 1364 (portrait orientation). It is
>> +  programmable through I2C, the address is fixed to 0x10. The sensor output is
>> +  available via CSI-2, which is configured as either 1 or 2 data lanes. The
>> +  sensor provides 8 GPIOS that can be used for external LED signal
>> +  (synchronized with sensor integration periods)
>> +
>> +properties:
>> +  compatible:
>> +    enum:
>> +      - st,st-vd56g3
>> +      - st,st-vd66gy
>> +    description:
>> +      Two variants are availables; VD56G3 is a monochrome sensor while VD66GY
>> +      is a colour variant.
>> +
>> +  reg:
>> +    maxItems: 1
>> +
>> +  clocks:
>> +    maxItems: 1
>> +
>> +  vcore-supply:
>> +    description: Digital core power supply (1.15V)
>> +
>> +  vddio-supply:
>> +    description: Digital IO power supply (1.8V)
>> +
>> +  vana-supply:
>> +    description: Analog power supply (2.8V)
>> +
>> +  reset-gpios:
>> +    description: Sensor reset active low GPIO (XSHUTDOWN)
>> +    maxItems: 1
>> +
>> +  st,leds:
>> +    description:
>> +      Sensor's GPIOs used for external LED control. Signal being the enveloppe
>> +      of the integration time.
> 
> More information is needed. GPIOs coming from LED or SoC? What's the
> meaning of values?

The vd56g3 image sensor provides 8 GPIOS that can be used for different
use cases (external led controls, synchronization between master/slave
sensors, external sensor trigger, etc.). This submission supports only
the first use case: the control of one(or multiple) external LED.

The vd56g3 sensor family are optimized for visible and near infrared
scenes. In NIR, external IR leds are generally used for illumination.

With such use case, a led (or a led driver) can be connected directly to
one of the 8 GPIOs of the sensor. On the driver side, when a led is
configured in the dt, the driver will configure the sensor accordingly.
It will also offer an optional "V4L2_FLASH_LED_MODE_FLASH" control to
start/stop the external control.

Different signal modes are supported by the HW, but the default
(implemented) one is a "strobe" mode where signal is the envelope of the
integration time (IR led is on while image sensor is integrating).

> 
>> +    $ref: /schemas/types.yaml#/definitions/uint32-array
>> +    minItems: 1
>> +    maxItems: 8
>> +    items:
>> +      minimum: 0
>> +      maximum: 7
>> +
>> +  port:
>> +    $ref: /schemas/graph.yaml#/$defs/port-base
> 
> missing additionalProperties: false

Ok, fixed in V3.

> 
>> +
>> +    properties:
>> +      endpoint:
>> +        $ref: /schemas/media/video-interfaces.yaml#
>> +        unevaluatedProperties: false
>> +
>> +        properties:
>> +          data-lanes:
>> +            minItems: 1
>> +            maxItems: 2
>> +            items:
>> +              enum: [1, 2]
> 
> 
>> +
>> +          link-frequencies:
>> +            minItems: 1
> 
> maxItems is enough

Ok, fixed in V3.

> 
>> +            maxItems: 1
>> +            items:
>> +              enum: [402000000, 750000000]
>> +
>> +          lane-polarities:
>> +            minItems: 1
>> +            maxItems: 3
>> +            description: Any lane can be inverted or not.
>> +
>> +        required:
>> +          - data-lanes
>> +          - link-frequencies
>> +
>> +required:
>> +  - compatible
>> +  - reg
>> +  - clocks
>> +  - vcore-supply
>> +  - vddio-supply
>> +  - vana-supply
>> +  - reset-gpios
>> +  - port
>> +
> 
> 
> Not a video-interface-device.yaml type of device?

Good point, something I'll consider in V3

> 
> Best regards,
> Krzysztof
> 

--
Sylvain
Krzysztof Kozlowski May 27, 2024, 7 p.m. UTC | #3
On 27/05/2024 15:14, Sylvain Petinot wrote:
>>
>>> Signed-off-by: Sylvain Petinot <sylvain.petinot@foss.st.com>
>>> ---
>>>  .../bindings/media/i2c/st,st-vd56g3.yaml      | 132 ++++++++++++++++++
>>>  MAINTAINERS                                   |   9 ++
>>>  2 files changed, 141 insertions(+)
>>>  create mode 100644 Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
>>>
>>> diff --git a/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
>>> new file mode 100644
>>> index 000000000000..22cb2557e311
>>> --- /dev/null
>>> +++ b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
>>
>> Why duplicated 'st'?
> 
> Legacy : our first st-mipid02 driver was upstream this way few years back.
> 
> We have 3 options :
> 
> 1- keep this unpleasant naming to keep consistency with st-mipid02 [1]
> and st-vgxy61 [2]

? Unpleasant?
Please follow generic rules. Filename must match compatible and
compatible must follow vendor,device format.

> 2- rename this driver properly ('vd56g3') and keep the two others the
> old way (I personally don't like this option)

We do not talk about driver here. Does not matter.

> 3- rename this driver properly ('vd56g3') and in a second patch rename
> the two others drivers.
> 
> I would be interested to get Sakari's opinion on this subject.

About what? Renaming drivers?

> 
> [1]:
> https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-mipid02.c
> 
> [2]:
> https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-vgxy61.c

Howe are these drivers anyhow related to the *binding*?


...

>>> +
>>> +  st,leds:
>>> +    description:
>>> +      Sensor's GPIOs used for external LED control. Signal being the enveloppe
>>> +      of the integration time.
>>
>> More information is needed. GPIOs coming from LED or SoC? What's the
>> meaning of values?
> 
> The vd56g3 image sensor provides 8 GPIOS that can be used for different
> use cases (external led controls, synchronization between master/slave
> sensors, external sensor trigger, etc.). This submission supports only
> the first use case: the control of one(or multiple) external LED.

What your driver supports is not really relevant. Describe hardware.

> 
> The vd56g3 sensor family are optimized for visible and near infrared
> scenes. In NIR, external IR leds are generally used for illumination.
> 
> With such use case, a led (or a led driver) can be connected directly to
> one of the 8 GPIOs of the sensor. On the driver side, when a led is
> configured in the dt, the driver will configure the sensor accordingly.
> It will also offer an optional "V4L2_FLASH_LED_MODE_FLASH" control to
> start/stop the external control.
> 
> Different signal modes are supported by the HW, but the default
> (implemented) one is a "strobe" mode where signal is the envelope of the
> integration time (IR led is on while image sensor is integrating).

You did not explain the meaning of the property. Please describe the
hardware and the meaning of values used in this property.



Best regards,
Krzysztof
Sakari Ailus May 27, 2024, 8:08 p.m. UTC | #4
Hi Sylvain,

On Mon, May 27, 2024 at 03:14:35PM +0200, Sylvain Petinot wrote:
> >> diff --git a/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
> >> new file mode 100644
> >> index 000000000000..22cb2557e311
> >> --- /dev/null
> >> +++ b/Documentation/devicetree/bindings/media/i2c/st,st-vd56g3.yaml
> > 
> > Why duplicated 'st'?
> 
> Legacy : our first st-mipid02 driver was upstream this way few years back.
> 
> We have 3 options :
> 
> 1- keep this unpleasant naming to keep consistency with st-mipid02 [1]
> and st-vgxy61 [2]
> 2- rename this driver properly ('vd56g3') and keep the two others the
> old way (I personally don't like this option)
> 3- rename this driver properly ('vd56g3') and in a second patch rename
> the two others drivers.
> 
> I would be interested to get Sakari's opinion on this subject.
> 
> [1]:
> https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-mipid02.c
> 
> [2]:
> https://elixir.bootlin.com/linux/v6.9.1/source/drivers/media/i2c/st-vgxy61.c

The driver could be renamed to align with a large majority that use the
same name as the bindings without the vendor prefix. You could add
MODULE_ALIAS() to help user space to cope with the change.

The DT compatible string indeed should reflect the name of the device, the
driver is indeed another matter.