From patchwork Tue Mar 5 16:08:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 779053 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D5F212AACB for ; Tue, 5 Mar 2024 16:09:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654966; cv=none; b=Uxz2bCor2GAVOs9LFo1zgG8usvrgGmcI+HJu278CdX0gl34Om1wfo6kzqLjfW7UPLaKz7I4w9UVyZTAJ93Cl/kJff2XQ0dldB6Rno618H2N/ptDL4vxwriErWffVVeuJCx1rbtgIHxuUP4DZ+Bn+gN5zKUxY+8U7YTKb89Rr7tI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654966; c=relaxed/simple; bh=Y3NTG7AWW9v7rgd6I35hnNTm9dye0y1q6+B5wmezxXw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qtyNzDB4H+hO4L/FLRMJcwdkjANdgFqOuQt5Gcc4F22OsNbW/g/BG6kx4Wj7Jldzsp5EKojZcCHrxZivcX2F3msjNzzhKn+KDPao5T8PCkNeLZtobxJuHbs8an0KQQ4kybi9bFLtICt51CdbfbEQYPar4MGZLEIc7p9Rij7zi94= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=Er3CYNY0; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="Er3CYNY0" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id C6357EF2; Tue, 5 Mar 2024 17:08:56 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1709654937; bh=Y3NTG7AWW9v7rgd6I35hnNTm9dye0y1q6+B5wmezxXw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Er3CYNY0QxpO2rjMpB/aMTTAbUNopTi2oiRgKI19kdCJahXgzR/5QhLdeo99nb8hT sdxeLXvmirGWwdB8fAvo3vwT0OeB6h6iMYeCYBrvd+Sh9JXlCg/HL2NOZHBFJbfds7 i0Wzy/HwOb3NNvs11T1GUeu/XBA4FnpBXIFbbsoQ= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v3 2/9] media: uapi: Add MIPI packed 14bit luma Date: Tue, 5 Mar 2024 17:08:45 +0100 Message-ID: <20240305160855.147483-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> References: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Dave Stevenson Add the Y14P format to describe monochrome 14 bit image format packed according to the RAW14 format as defined by the MIPI CSI-2 specification. Signed-off-by: Dave Stevenson Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart --- .../media/v4l/pixfmt-yuv-luma.rst | 37 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 3 files changed, 39 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst index 585d736f36e5..3af6e3cb70c4 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst @@ -36,6 +36,8 @@ are often referred to as greyscale formats. - Byte 2 - Byte 3 - Byte 4 + - Byte 5 + - Byte 6 * .. _V4L2-PIX-FMT-GREY: @@ -47,6 +49,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-IPU3-Y10: @@ -58,6 +62,8 @@ are often referred to as greyscale formats. - Y'\ :sub:`2`\ [3:0] Y'\ :sub:`1`\ [9:6] - Y'\ :sub:`3`\ [1:0] Y'\ :sub:`2`\ [9:4] - Y'\ :sub:`3`\ [9:2] + - ... + - ... * .. _V4L2-PIX-FMT-Y10: @@ -69,6 +75,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-Y10BPACK: @@ -80,6 +88,8 @@ are often referred to as greyscale formats. - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`2`\ [9:6] - Y'\ :sub:`2`\ [5:0] Y'\ :sub:`3`\ [9:8] - Y'\ :sub:`3`\ [7:0] + - ... + - ... * .. _V4L2-PIX-FMT-Y10P: @@ -91,6 +101,8 @@ are often referred to as greyscale formats. - Y'\ :sub:`2`\ [9:2] - Y'\ :sub:`3`\ [9:2] - Y'\ :sub:`3`\ [1:0] Y'\ :sub:`2`\ [1:0] Y'\ :sub:`1`\ [1:0] Y'\ :sub:`0`\ [1:0] + - ... + - ... * .. _V4L2-PIX-FMT-Y12: @@ -102,6 +114,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-Y012: @@ -113,6 +127,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-Y12P: @@ -124,6 +140,8 @@ are often referred to as greyscale formats. - Y'\ :sub:`1`\ [3:0] Y'\ :sub:`0`\ [3:0] - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-Y14: @@ -135,6 +153,21 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... + + * .. _V4L2-PIX-FMT-Y14P: + + - ``V4L2_PIX_FMT_Y14P`` + - 'Y14P' + + - Y'\ :sub:`0`\ [13:6] + - Y'\ :sub:`1`\ [13:6] + - Y'\ :sub:`2`\ [13:6] + - Y'\ :sub:`3`\ [13:6] + - Y'\ :sub:`1`\ [1:0] Y'\ :sub:`0`\ [5:0] + - Y'\ :sub:`2`\ [3:0] Y'\ :sub:`1`\ [5:2] + - Y'\ :sub:`3`\ [5:0] Y'\ :sub:`2`\ [5:4] * .. _V4L2-PIX-FMT-Y16: @@ -146,6 +179,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... * .. _V4L2-PIX-FMT-Y16-BE: @@ -157,6 +192,8 @@ are often referred to as greyscale formats. - ... - ... - ... + - ... + - ... .. raw:: latex diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 9cb09334161b..d0724240a446 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1312,6 +1312,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y10BPACK: descr = "10-bit Greyscale (Packed)"; break; case V4L2_PIX_FMT_Y10P: descr = "10-bit Greyscale (MIPI Packed)"; break; case V4L2_PIX_FMT_Y12P: descr = "12-bit Greyscale (MIPI Packed)"; break; + case V4L2_PIX_FMT_Y14P: descr = "14-bit Greyscale (MIPI Packed)"; break; case V4L2_PIX_FMT_IPU3_Y10: descr = "10-bit greyscale (IPU3 Packed)"; break; case V4L2_PIX_FMT_Y8I: descr = "Interleaved 8-bit Greyscale"; break; case V4L2_PIX_FMT_Y12I: descr = "Interleaved 12-bit Greyscale"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 11ebf9b22ccb..911f00ed28fd 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -599,6 +599,7 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_Y10BPACK v4l2_fourcc('Y', '1', '0', 'B') /* 10 Greyscale bit-packed */ #define V4L2_PIX_FMT_Y10P v4l2_fourcc('Y', '1', '0', 'P') /* 10 Greyscale, MIPI RAW10 packed */ #define V4L2_PIX_FMT_Y12P v4l2_fourcc('Y', '1', '2', 'P') /* 12 Greyscale, MIPI RAW12 packed */ +#define V4L2_PIX_FMT_Y14P v4l2_fourcc('Y', '1', '4', 'P') /* 14 Greyscale, MIPI RAW14 packed */ #define V4L2_PIX_FMT_IPU3_Y10 v4l2_fourcc('i', 'p', '3', 'y') /* IPU3 packed 10-bit greyscale */ /* Palette formats */ From patchwork Tue Mar 5 16:08:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 779051 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C30E12BEB8 for ; Tue, 5 Mar 2024 16:09:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654972; cv=none; b=n852Yai/x8xM/GxENrO7bRLcFxbHx/yRX7T3VjcmdQYZlq3m7dPjhdnxrxWY0dJssqMiBwaDmdjwAr2aUXwBnLyK0scjP3sRR2hDiCbE6pVCymA72cZLbwchYocfSL7Ud4w8t5UGufD7QI3P2kiOsmqWkl1nkdFA6EKW9/RfSLE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654972; c=relaxed/simple; bh=JNOOEqP9k9Z7oeJjNf0NEvCljjoBqsRhxOsm1P91wF8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lFxDVotJbcm/bcYZBomd9xdE5CosuV75pH7eZO3czUUQZOeJ/DRc3H3XwvqO+Jzmf+lSErt5n3IDxTavhtqNcIqltG4XLSN1cVvtIWcQFT/lb9JafaO8xFde/SvpcOHpLY1gzPBzflyJrDY9mkKikji7vixFvUIx1M6kqsYnmAY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=lB3YW+xc; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="lB3YW+xc" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id E289D15B5; Tue, 5 Mar 2024 17:08:59 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1709654940; bh=JNOOEqP9k9Z7oeJjNf0NEvCljjoBqsRhxOsm1P91wF8=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lB3YW+xcm8nrqBwMF8t7D0a/J40xX8gJ0DoJfGjLJAl1iekprwr6buVOgun+mz11M n8dzZ7HiXd/sgymegyVS9rUqPfs9NSDkhkVlOPn/eqswfG1vMnjXP2ehdy+RJpB/XG cEHdvPO/5W2p2Xm6WtXrNMa47YPHKveCxly/UPtU= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v3 6/9] media: uapi: Add PiSP Compressed RAW Bayer formats Date: Tue, 5 Mar 2024 17:08:49 +0100 Message-ID: <20240305160855.147483-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> References: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add Raspberry Pi compressed RAW Bayer formats. The compression algorithm description is provided by Nick Hollinghurst from Raspberry Pi. Signed-off-by: Jacopo Mondi --- .../userspace-api/media/v4l/pixfmt-bayer.rst | 1 + .../media/v4l/pixfmt-pisp-comp-rggb.rst | 70 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 10 +++ include/uapi/linux/videodev2.h | 12 ++++ 4 files changed, 93 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst diff --git a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst index 2500413e5f43..81b69ea4084a 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst @@ -30,3 +30,4 @@ orders. See also `the Wikipedia article on Bayer filter pixfmt-srggb14 pixfmt-srggb14p pixfmt-srggb16 + pixfmt-pisp-comp-rggb diff --git a/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst b/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst new file mode 100644 index 000000000000..7be1b25b2241 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-pisp-comp-rggb.rst @@ -0,0 +1,70 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _v4l2-pix-fmt-pisp-comp1-rggb: +.. _v4l2-pix-fmt-pisp-comp1-grbg: +.. _v4l2-pix-fmt-pisp-comp1-gbrg: +.. _v4l2-pix-fmt-pisp-comp1-bggr: +.. _v4l2-pix-fmt-pisp-comp1-mono: +.. _v4l2-pix-fmt-pisp-comp2-rggb: +.. _v4l2-pix-fmt-pisp-comp2-grbg: +.. _v4l2-pix-fmt-pisp-comp2-gbrg: +.. _v4l2-pix-fmt-pisp-comp2-bggr: +.. _v4l2-pix-fmt-pisp-comp2-mono: + +================================================ +Raspberry Pi PiSP compressed 8-bit Bayer formats +================================================ + +Description +=========== + +Raspberry Pi ISP uses a family of three fixed-rate compressed Bayer formats. +A black-level offset may be subtracted to improve compression efficiency; +the nominal black level and amount of offset must be signalled out of band. +Each scanline is padded to a multiple of 8 pixels wide, and each block of 8 +horizontally-contiguous pixels is coded using 8 bytes. + +Mode 1 uses a quantization and delta-based coding scheme which preserves up to +12 significant bits. Mode 2 is a simple sqrt-like companding scheme with 6 PWL +chords, preserving up to 12 significant bits. Mode 3 combines both companding +(with 4 chords) and the delta scheme, preserving up to 14 significant bits. + +The remainder of this description applies to Modes 1 and 3. + +Each block of 8 pixels is separated into even and odd phases of 4 pixels, +coded independently by 32-bit words at successive locations in memory. +The two LS bits of each 32-bit word give its "quantization mode". + +In quantization mode 0, the lowest 321 quantization levels are multiples of +FSD/4096 and the remaining levels are successive multiples of FSD/2048. +Quantization modes 1 and 2 use linear quantization with step sizes of +FSD/1024 and FSD/512 respectively. Each of the four pixels is quantized +independently, with rounding to the nearest level. +In quantization mode 2 where the middle two samples have quantized values +(q1,q2) both in the range [384..511], they are coded using 9 bits for q1 +followed by 7 bits for (q2 & 127). Otherwise, for quantization modes +0, 1 and 2: a 9-bit field encodes MIN(q1,q2) which must be in the range +[0..511] and a 7-bit field encodes (q2-q1+64) which must be in [0..127]. + +Each of the outer samples (q0,q3) is encoded using a 7-bit field based +on its inner neighbour q1 or q2. In quantization mode 2 where the inner +sample has a quantized value in the range [448..511], the field value is +(q0-384). Otherwise for quantization modes 0, 1 and 2: The outer sample +is encoded as (q0-MAX(0,q1-64)). q3 is likewise coded based on q2. +Each of these values must be in the range [0..127]. All these fields +of 2, 9, 7, 7, 7 bits respectively are packed in little-endian order +to give a 32-bit word with LE byte order. + +Quantization mode 3 has a "7.5-bit" escape, used when none of the above +encodings will fit. Each pixel value is quantized to the nearest of 176 +levels, where the lowest 48 levels are multiples of FSD/256 and the +remaining levels are multiples of FSD/512 (level 175 represents values +very close to FSD and may require saturating arithmetic to decode). + +Each pair of quantized pixels (q0,q1) or (q2,q3) is jointly coded +by a 15-bit field: 2816*(q0>>4) + 16*q1 + (q0&15). +Three fields of 2, 15, 15 bits are packed in LE order {15,15,2}. + +An implementation of a software decoder of compressed formats is available +in `Raspberry Pi camera applications code base +_`. diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 18e9aa1297e1..fc7af7337ab4 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1516,6 +1516,16 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile Mode"; break; case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit Raster Mode"; break; case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed Format"; break; + case V4L2_PIX_FMT_PISP_COMP1_RGGB: descr = "PiSP 8b RGRG/GBGB mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GRBG: descr = "PiSP 8b GRGR/BGBG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GBRG: descr = "PiSP 8b GBGB/RGRG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_BGGR: descr = "PiSP 8b BGBG/GRGR mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_MONO: descr = "PiSP 8b monochrome mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_RGGB: descr = "PiSP 8b RGRG/GBGB mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GRBG: descr = "PiSP 8b GRGR/BGBG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GBRG: descr = "PiSP 8b GBGB/RGRG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_BGGR: descr = "PiSP 8b BGBG/GRGR mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_MONO: descr = "PiSP 8b monochrome mode2 compr"; break; default: if (fmt->description[0]) return; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index c3daf61f43d5..feab8a153d2e 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -818,6 +818,18 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_IPU3_SGRBG10 v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */ #define V4L2_PIX_FMT_IPU3_SRGGB10 v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */ +/* Raspberry Pi PiSP compressed formats. */ +#define V4L2_PIX_FMT_PISP_COMP1_RGGB v4l2_fourcc('P', 'C', '1', 'R') /* PiSP 8-bit mode 1 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GRBG v4l2_fourcc('P', 'C', '1', 'G') /* PiSP 8-bit mode 1 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GBRG v4l2_fourcc('P', 'C', '1', 'g') /* PiSP 8-bit mode 1 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_BGGR v4l2_fourcc('P', 'C', '1', 'B') /* PiSP 8-bit mode 1 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_MONO v4l2_fourcc('P', 'C', '1', 'M') /* PiSP 8-bit mode 1 compressed monochrome */ +#define V4L2_PIX_FMT_PISP_COMP2_RGGB v4l2_fourcc('P', 'C', '2', 'R') /* PiSP 8-bit mode 2 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GRBG v4l2_fourcc('P', 'C', '2', 'G') /* PiSP 8-bit mode 2 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GBRG v4l2_fourcc('P', 'C', '2', 'g') /* PiSP 8-bit mode 2 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_BGGR v4l2_fourcc('P', 'C', '2', 'B') /* PiSP 8-bit mode 2 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_MONO v4l2_fourcc('P', 'C', '2', 'M') /* PiSP 8-bit mode 2 compressed monochrome */ + /* SDR formats - used only for Software Defined Radio devices */ #define V4L2_SDR_FMT_CU8 v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */ #define V4L2_SDR_FMT_CU16LE v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */ From patchwork Tue Mar 5 16:08:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 779050 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BD05B12CD87 for ; Tue, 5 Mar 2024 16:09:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654976; cv=none; b=O11fw/jriR9vEqG9YxKxkQEgXxpzk7IYERKC0Fe9vuKUpgU+4itMSM5ZwtZrs8F6moEapULYvY4nwmsyEQodR7PVyp8MYYkTW2fe1VzM1aV209nc0ZtfoH1sySsKTlWUtlgHwVI0RcU+D8TpKPjAdApbh7l9Dv+I37uxFcZ+bwY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709654976; c=relaxed/simple; bh=IQWQXWa2/TXbjRQ3o3c+T81LEpn5xoFC0Vk7B2uUak0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=he+cJMbKsTuNlnVdxfnYQyUCuCaWyB5Y0vHYctJCDMEXxqEXC/LyKOIYLqxZN/7igLyrICHID5NcAaJI/k4/4/pK2nY/moJmD41sZ6HcIMabbSrrK/UpLcEfUCTr5OvQmgCnV9lpzaJ1mKmsIDD3VHyPIFJ9AHIOr/LHh6nRZZo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=JMvZP1DH; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="JMvZP1DH" Received: from localhost.localdomain (unknown [IPv6:2001:b07:5d2e:52c9:cc1e:e404:491f:e6ea]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id CC31A3211; Tue, 5 Mar 2024 17:09:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1709654943; bh=IQWQXWa2/TXbjRQ3o3c+T81LEpn5xoFC0Vk7B2uUak0=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=JMvZP1DHDRspdduEr6T5olgK6dZFO43wz1T2z0AaQcMoy4su73LiNbcuuOTcKQkox aZ3vYihKZBcL6S+tqjy/3wYOa4fPM59OAHGPEsSNfyRpywuyKb1GLjpcjpuP2d/H18 DKyq52PQbjjC725fjM8BQHtxmzxKmFdiUwhuJdks= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v3 9/9] media: admin-guide: Document the Raspberry Pi PiSP BE Date: Tue, 5 Mar 2024 17:08:52 +0100 Message-ID: <20240305160855.147483-10-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.43.2 In-Reply-To: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> References: <20240305160855.147483-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add documentation for the PiSP Back End memory-to-memory ISP. Signed-off-by: Jacopo Mondi --- .../admin-guide/media/raspberrypi-pisp-be.dot | 20 ++++ .../admin-guide/media/raspberrypi-pisp-be.rst | 112 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 133 insertions(+) create mode 100644 Documentation/admin-guide/media/raspberrypi-pisp-be.dot create mode 100644 Documentation/admin-guide/media/raspberrypi-pisp-be.rst diff --git a/Documentation/admin-guide/media/raspberrypi-pisp-be.dot b/Documentation/admin-guide/media/raspberrypi-pisp-be.dot new file mode 100644 index 000000000000..55671dc1d443 --- /dev/null +++ b/Documentation/admin-guide/media/raspberrypi-pisp-be.dot @@ -0,0 +1,20 @@ +digraph board { + rankdir=TB + n00000001 [label="{{ 0 | 1 | 2 | 7} | pispbe\n | { 3 | 4 | 5 | 6}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port3 -> n0000001c [style=bold] + n00000001:port4 -> n00000022 [style=bold] + n00000001:port5 -> n00000028 [style=bold] + n00000001:port6 -> n0000002e [style=bold] + n0000000a [label="pispbe-input\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n0000000a -> n00000001:port0 [style=bold] + n00000010 [label="pispbe-tdn_input\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000010 -> n00000001:port1 [style=bold] + n00000016 [label="pispbe-stitch_input\n/dev/video2", shape=box, style=filled, fillcolor=yellow] + n00000016 -> n00000001:port2 [style=bold] + n0000001c [label="pispbe-output0\n/dev/video3", shape=box, style=filled, fillcolor=yellow] + n00000022 [label="pispbe-output1\n/dev/video4", shape=box, style=filled, fillcolor=yellow] + n00000028 [label="pispbe-tdn_output\n/dev/video5", shape=box, style=filled, fillcolor=yellow] + n0000002e [label="pispbe-stitch_output\n/dev/video6", shape=box, style=filled, fillcolor=yellow] + n00000034 [label="pispbe-config\n/dev/video7", shape=box, style=filled, fillcolor=yellow] + n00000034 -> n00000001:port7 [style=bold] +} diff --git a/Documentation/admin-guide/media/raspberrypi-pisp-be.rst b/Documentation/admin-guide/media/raspberrypi-pisp-be.rst new file mode 100644 index 000000000000..60fb4a882d31 --- /dev/null +++ b/Documentation/admin-guide/media/raspberrypi-pisp-be.rst @@ -0,0 +1,112 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================================= +Raspberry Pi PiSP Back End Memory-to-Memory ISP (pisp-be) +========================================================= + +The PiSP Back End +================= + +The PiSP Back End is a memory-to-memory Image Signal Processor (ISP) which reads +image data from DRAM memory and performs image processing as specified by the +application through the parameters in a configuration buffer, before writing +pixel data back to memory through two distinct output channels. + +The ISP registers and programming model are documented in the `Raspberry Pi +Image Signal Processor (ISP) Specification document +`_ + +The PiSP Back End ISP processes images in tiles to allow concurrent processing +of different images. The handling of image tessellation and the computation of +low-level configuration parameters is realized by a free software library +called `libpisp `_. + +The full image processing pipeline, which involves capturing RAW Bayer data from +an image sensor through a MIPI CSI-2 compatible capture interface, storing them +in DRAM memory and processing them in the PiSP Back End to obtain images usable +by an application is implemented in `libcamera `_ as part of +the Raspberry Pi platform support. + +The pisp-be driver +================== + +The Raspberry Pi PiSP Back End (pisp-be) driver is located under +drivers/media/platform/raspberrypi/pisp-be. It uses the `V4L2 API` to register +a number of video capture and output devices, the `V4L2 subdev API` to register +a subdevice for the ISP that connects the video devices in a single media graph +realized using the `Media Controller (MC) API`. + +As the PiSP Back End device processes images in tiles, it can be multiplexed +between several different execution contexts. The driver registers two +`contexts` represented as two distinct media graph instances but could +potentially register more of them. + +The media topology registered by the `pisp-be` driver is represented below: + +.. _pips-be-topology: + +.. kernel-figure:: raspberrypi-pisp-be.dot + :alt: Diagram of the default media pipeline topology + :align: center + + +Each media graph instance registers the following video device nodes: + +- pispbe-input: output device for images to be submitted to the ISP for + processing. +- pispbe-tdn_input: output device for temporal denoise. +- pispbe-stitch_input: output device for image stitching (HDR). +- pispbe-output0: first capture device for processed images. +- pispbe-output1: second capture device for processed images. +- pispbe-tdn_output: capture device for temporal denoise. +- pispbe-stitch_output: capture device for image stitching (HDR). +- pispbe-config: output device for ISP configuration parameters. + +pispbe-input +------------ + +Images to be processed by the ISP are queued to the `pispbe-input` output device +node. For a list of image formats supported as input to the ISP refer to the +PiSP specification document. + +pispbe-tdn_input, pispbe-tdn_output +----------------------------------- + +The `pispbe-tdn_input` output video device receives images to be processed by +the temporal denoise block which are captured from the `pispbe-tdn_output` +capture video device. Userspace is responsible for maintaining queues on both +devices, and ensuring that buffers completed on the output are queued to the +input. + +pispbe-stitch_input, pispbe-stitch_output +----------------------------------------- + +To realize HDR (high dynamic range) image processing the image stitching and +tonemapping blocks are used. The `pispbe-stitch_output` writes images to memory +and the `pispbe-stitch_input` receives the previously written frame to process +it along with the current input image. Userspace is responsible for maintaining +queues on both devices, and ensuring that buffers completed on the output are +queued to the input. + +pispbe-output0, pispbe-output1 +------------------------------ + +The two capture devices write to memory the pixel data as processed by the ISP. + +pispbe-config +------------- + +The `pispbe-config` output video devices receives a buffer of configuration +parameters that define the desired image processing to be performed by the ISP. + +The format of the ISP configuration parameter is defined by +:c:type:`pisp_be_tiles_config` C structure and the meaning of each parameter is +described in the `PiSP specification` document. + +ISP configuration +================= + +The ISP configuration is described solely by the content of the parameters +buffer. The only parameter that userspace needs to configure using the V4L2 API +is the image format on the output and capture video devices for validation of +the content of the parameters buffer. diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index f4bb2605f07e..c15a6bb952d9 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -22,6 +22,7 @@ Video4Linux (V4L) driver-specific documentation omap4_camera philips qcom_camss + raspberrypi-pisp-be rcar-fdp1 rkisp1 saa7134