@@ -553,6 +553,27 @@ A stream at a specific point in the media pipeline is identified by the
sub-device and a (pad, stream) pair. For sub-devices that do not support
multiplexed streams the 'stream' field is always 0.
+.. _v4l2-subdev-internal-source-pads:
+
+Internal source pads and routing
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Cases where a single sub-device source pad is traversed by multiple streams, one
+or more of which originate from within the sub-device itself, are special as
+there is no external sink pad for such routes. In those cases, the sources of
+the internally generated streams are represented by internal source pads, which
+are sink pads that have the :ref:`MEDIA_PAD_FL_INTERNAL <MEDIA-PAD-FL-INTERNAL>`
+pad flag set.
+
+Internal pads have all the properties of an external pad, including formats and
+selections. The format in this case is the source format of the stream. An
+internal pad always has a single stream only (0).
+
+Routes from an internal source pad to an external source pad are typically not
+modifiable but they can be activated and deactivated using the
+:ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE <v4l2-subdev-routing-flags>` flag, depending
+on driver capabilities.
+
Interaction between routes, streams, formats and selections
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -668,3 +689,127 @@ To configure this pipeline, the userspace must take the following steps:
the configurations along the stream towards the receiver, using
:ref:`VIDIOC_SUBDEV_S_FMT <VIDIOC_SUBDEV_G_FMT>` ioctls to configure each
stream endpoint in each sub-device.
+
+Internal pads setup example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+A simple example of a multiplexed stream setup might be as follows:
+
+- An IMX219 camera sensor source sub-device, with one sink pad (0), one source pad
+ (1), an internal sink pad (2) that represents the source of embedded
+ data. There are two routes, one from the sink pad to the source, and another
+ from the internal sink pad to the source pad. Both streams are always active,
+ i.e. there is no need to separately enable the embedded data stream. The
+ sensor uses the CSI-2 bus.
+
+- A CSI-2 receiver in the SoC (Receiver). The receiver has a single sink pad
+ (pad 0), connected to the bridge, and two source pads (pads 1-2), going to the
+ DMA engine. The receiver demultiplexes the incoming streams to the source
+ pads.
+
+- DMA Engines in the SoC (DMA Engine), one for each stream. Each DMA engine is
+ connected to a single source pad in the receiver.
+
+The sensor, the bridge and the receiver are modeled as V4L2 sub-devices,
+exposed to userspace via /dev/v4l-subdevX device nodes. The DMA engines are
+modeled as V4L2 devices, exposed to userspace via /dev/videoX nodes.
+
+To configure this pipeline, the userspace must take the following steps:
+
+1) Set up media links between entities: connect the sensors to the bridge,
+ bridge to the receiver, and the receiver to the DMA engines. This step does
+ not differ from normal non-multiplexed media controller setup.
+
+2) Configure routing
+
+.. flat-table:: Camera sensor. There are no configurable routes.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from the sink pad
+ * - 2/0
+ - 1/1
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from the internal sink pad
+
+.. flat-table:: Receiver routing table. Typically both routes need to be
+ explicitly set.
+ :header-rows: 1
+
+ * - Sink Pad/Stream
+ - Source Pad/Stream
+ - Routing Flags
+ - Comments
+ * - 0/0
+ - 1/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Pixel data stream from camera sensor
+ * - 0/1
+ - 2/0
+ - V4L2_SUBDEV_ROUTE_FL_ACTIVE
+ - Metadata stream from camera sensor
+
+The options available in sensor's routing configuration are dictated by hardware
+capabilities: typically camera sensors always produce an image data stream while
+it may be possible to enable and disable the embedded data stream.
+
+3) Configure formats and selections
+
+ This example assumes that the formats are propagated from sink pad to the
+ source pad as-is. The tables contain fields of both struct v4l2_subdev_format
+ and struct v4l2_mbus_framefmt.
+
+.. flat-table:: Formats set on the sub-devices. Bold values are set, others are
+ static or propagated. The order is aligned with configured
+ routes.
+ :header-rows: 1
+ :fill-cells:
+
+ * - Sub-device
+ - Pad/Stream
+ - Width
+ - Height
+ - Code
+ * - :rspan:`3` Camera sensor sub-device (IMX219)
+ - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_SRGGB10
+ * - 0/0
+ - **3296**
+ - **2480**
+ - **MEDIA_BUS_FMT_SRGGB10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_IMX219_EMBEDDED
+ * - 1/1
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+ * - :rspan:`3` Receiver
+ - 0/0
+ - **3296**
+ - **2480**
+ - **MEDIA_BUS_FMT_SRGGB10**
+ * - 1/0
+ - 3296
+ - 2480
+ - MEDIA_BUS_FMT_SRGGB10
+ * - 0/1
+ - **3296**
+ - **2**
+ - **MEDIA_BUS_FMT_META_10**
+ * - 2/0
+ - 3296
+ - 2
+ - MEDIA_BUS_FMT_META_10
+
+The embedded data format does not need to be configured as the format is
+dictated by the pixel data format in this case.
Document internal source pads, pads that have both SINK and INTERNAL flags set. Use the IMX219 camera sensor as an example. Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> --- .../userspace-api/media/v4l/dev-subdev.rst | 145 ++++++++++++++++++ 1 file changed, 145 insertions(+)