From patchwork Wed Jun 26 18:14:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 807671 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 358F7190465 for ; Wed, 26 Jun 2024 18:14:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425700; cv=none; b=CG9L8NoKS++i7ZzG/agwR8v7UWGKRsc04DDfrTP2DR+b4sW738BoBet0sSIful+JeSrMLd8yYNOa+pWNl9oBYLwixDSw3hql/0IUR4ITyVh0mTOr/hdX6Xzvz3P/FH27yvnstWNX7RAzlGMbVNada9gZc3j8OvcAs5BiTqrrFOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425700; c=relaxed/simple; bh=9oBBE4eCRFGMnPWILMFEzK0l9cq+AskckdZDQXB0s68=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PCXqIFcKh7tpDpT9nIl15pIwVNykyhu7DlE8krfJt2rr+EMeUQ9ilI0RjGoBiFw5p7KpYtdTLQu3BbLarprw+6jBI434zWfekL198Kgvtcx4Cdlh3J5gfT44o65/2hUE34C+rCayAEOdUxqgeMri1haR50gLChCBQIpV6evy6RE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=rFudMnRL; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="rFudMnRL" Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4DD6B73E; Wed, 26 Jun 2024 20:14:33 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1719425673; bh=9oBBE4eCRFGMnPWILMFEzK0l9cq+AskckdZDQXB0s68=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rFudMnRLa+G/QU8OG0VSYoz6kTquR0SP9SFiQpBhOjUr5QqIcuWJzNKyoaNRX5XKo +Hf0euqyftHmahDRK9iGCsVxpFGUiALkP4SWwSVyw9BTQYBCBGSyKgP7RBOCiyR34B Tf4tBTwXBgF/1+5xKjhficC2IFrcVLUn3ii4EYOM= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v11 1/8] media: uapi: pixfmt-luma: Document MIPI CSI-2 packing Date: Wed, 26 Jun 2024 20:14:31 +0200 Message-ID: <20240626181440.195137-2-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> References: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The Y10P, Y12P and Y14P format variants are packed according to the RAW10, RAW12 and RAW14 formats as defined by the MIPI CSI-2 specification. Document it. Signed-off-by: Jacopo Mondi Reviewed-by: Laurent Pinchart Reviewed-by: Naushir Patuck --- Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst index b3c5779521d8..f02e6cf3516a 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-luma.rst @@ -209,3 +209,7 @@ are often referred to as greyscale formats. For Y012 and Y12 formats, Y012 places its data in the 12 high bits, with padding zeros in the 4 low bits, in contrast to the Y12 format, which has its padding located in the most significant bits of the 16 bit word. + + The 'P' variations of the Y10, Y12 and Y14 formats are packed according to + the RAW10, RAW12 and RAW14 packing scheme as defined by the MIPI CSI-2 + specification. From patchwork Wed Jun 26 18:14:33 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 807670 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 58AD2190497 for ; Wed, 26 Jun 2024 18:15:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425705; cv=none; b=MtywH8vsPupN36tM9r5oc7qMld+gtewCT8ZCHHX6qxtBBfRbKPeNQVw+ytYDH6E5obrScqDivzGV5XVfanAlKEeLPuOLjGFI+3a4H6qcJ1jxBqRHBXg5VbQNojXxztcsCNkjZ2jbtIx8/vOeVh5IRw6BeMTFiy2LUzJUngWrgsc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425705; c=relaxed/simple; bh=mU+14fbfm6f+gue3GlVxwAAStGwWml5EwZ0ZnmKS2d4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KFjL7lhWF4dR5NnD1Is7g5lZrZzNxkFP3N4kOoCId2Cvaf+TazVU4eENN+/SO8/4/W/DFmBSpOwS2ANjwEZR+4VLhg0NiwZN/ffhm+RiA0fgonMzt2cE2jF/UMryiTHwg9dQscn7+qjOA/RFheOrWr5e0JYVP0oqiuhpczH5U1c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=MXNN+Uzv; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="MXNN+Uzv" Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id DFBDC1890; Wed, 26 Jun 2024 20:14:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1719425675; bh=mU+14fbfm6f+gue3GlVxwAAStGwWml5EwZ0ZnmKS2d4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=MXNN+UzvnpFbPatESc6PpY1CBvSyoqsuTV59KdKd75KJzQl6zmFcBxJpvxN1BpbJ/ oErPM8TIsoFfzQzisfhyawn7xL6SQwlm+oTaNd7SxxUGPmu3u6eW3B/Npeb9MvbKtN o/5Dab3mfnFqa4V3ZXbIpDhkV0jYDC0LIXIQsRV4= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v11 3/8] media: uapi: Add Raspberry Pi PiSP Back End uAPI Date: Wed, 26 Jun 2024 20:14:33 +0200 Message-ID: <20240626181440.195137-4-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> References: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add the Raspberry Pi PiSP Back End uAPI header. The header defines the data type used to configure the PiSP Back End ISP. The detailed description of the types and of the ISP configuration procedure is available at https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf Signed-off-by: Jacopo Mondi --- MAINTAINERS | 7 + .../linux/media/raspberrypi/pisp_be_config.h | 927 ++++++++++++++++++ .../linux/media/raspberrypi/pisp_common.h | 199 ++++ 3 files changed, 1133 insertions(+) create mode 100644 include/uapi/linux/media/raspberrypi/pisp_be_config.h create mode 100644 include/uapi/linux/media/raspberrypi/pisp_common.h diff --git a/MAINTAINERS b/MAINTAINERS index ad1ecb2c4aed..861ee24cc1f7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -18805,6 +18805,13 @@ L: linux-edac@vger.kernel.org S: Maintained F: drivers/ras/amd/fmpm.c +RASPBERRY PI PISP BACK END +M: Jacopo Mondi +L: Raspberry Pi Kernel Maintenance +L: linux-media@vger.kernel.org +S: Maintained +F: include/uapi/linux/media/raspberrypi/ + RC-CORE / LIRC FRAMEWORK M: Sean Young L: linux-media@vger.kernel.org diff --git a/include/uapi/linux/media/raspberrypi/pisp_be_config.h b/include/uapi/linux/media/raspberrypi/pisp_be_config.h new file mode 100644 index 000000000000..1684ae068d4f --- /dev/null +++ b/include/uapi/linux/media/raspberrypi/pisp_be_config.h @@ -0,0 +1,927 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * PiSP Back End configuration definitions. + * + * Copyright (C) 2021 - Raspberry Pi Ltd + * + */ +#ifndef _UAPI_PISP_BE_CONFIG_H_ +#define _UAPI_PISP_BE_CONFIG_H_ + +#include + +#include "pisp_common.h" + +/* byte alignment for inputs */ +#define PISP_BACK_END_INPUT_ALIGN 4u +/* alignment for compressed inputs */ +#define PISP_BACK_END_COMPRESSED_ALIGN 8u +/* minimum required byte alignment for outputs */ +#define PISP_BACK_END_OUTPUT_MIN_ALIGN 16u +/* preferred byte alignment for outputs */ +#define PISP_BACK_END_OUTPUT_MAX_ALIGN 64u + +/* minimum allowed tile width anywhere in the pipeline */ +#define PISP_BACK_END_MIN_TILE_WIDTH 16u +/* minimum allowed tile width anywhere in the pipeline */ +#define PISP_BACK_END_MIN_TILE_HEIGHT 16u + +#define PISP_BACK_END_NUM_OUTPUTS 2 +#define PISP_BACK_END_HOG_OUTPUT 1 + +#define PISP_BACK_END_NUM_TILES 64 + +enum pisp_be_bayer_enable { + PISP_BE_BAYER_ENABLE_INPUT = 0x000001, + PISP_BE_BAYER_ENABLE_DECOMPRESS = 0x000002, + PISP_BE_BAYER_ENABLE_DPC = 0x000004, + PISP_BE_BAYER_ENABLE_GEQ = 0x000008, + PISP_BE_BAYER_ENABLE_TDN_INPUT = 0x000010, + PISP_BE_BAYER_ENABLE_TDN_DECOMPRESS = 0x000020, + PISP_BE_BAYER_ENABLE_TDN = 0x000040, + PISP_BE_BAYER_ENABLE_TDN_COMPRESS = 0x000080, + PISP_BE_BAYER_ENABLE_TDN_OUTPUT = 0x000100, + PISP_BE_BAYER_ENABLE_SDN = 0x000200, + PISP_BE_BAYER_ENABLE_BLC = 0x000400, + PISP_BE_BAYER_ENABLE_STITCH_INPUT = 0x000800, + PISP_BE_BAYER_ENABLE_STITCH_DECOMPRESS = 0x001000, + PISP_BE_BAYER_ENABLE_STITCH = 0x002000, + PISP_BE_BAYER_ENABLE_STITCH_COMPRESS = 0x004000, + PISP_BE_BAYER_ENABLE_STITCH_OUTPUT = 0x008000, + PISP_BE_BAYER_ENABLE_WBG = 0x010000, + PISP_BE_BAYER_ENABLE_CDN = 0x020000, + PISP_BE_BAYER_ENABLE_LSC = 0x040000, + PISP_BE_BAYER_ENABLE_TONEMAP = 0x080000, + PISP_BE_BAYER_ENABLE_CAC = 0x100000, + PISP_BE_BAYER_ENABLE_DEBIN = 0x200000, + PISP_BE_BAYER_ENABLE_DEMOSAIC = 0x400000, +}; + +enum pisp_be_rgb_enable { + PISP_BE_RGB_ENABLE_INPUT = 0x000001, + PISP_BE_RGB_ENABLE_CCM = 0x000002, + PISP_BE_RGB_ENABLE_SAT_CONTROL = 0x000004, + PISP_BE_RGB_ENABLE_YCBCR = 0x000008, + PISP_BE_RGB_ENABLE_FALSE_COLOUR = 0x000010, + PISP_BE_RGB_ENABLE_SHARPEN = 0x000020, + /* Preferred colours would occupy 0x000040 */ + PISP_BE_RGB_ENABLE_YCBCR_INVERSE = 0x000080, + PISP_BE_RGB_ENABLE_GAMMA = 0x000100, + PISP_BE_RGB_ENABLE_CSC0 = 0x000200, + PISP_BE_RGB_ENABLE_CSC1 = 0x000400, + PISP_BE_RGB_ENABLE_DOWNSCALE0 = 0x001000, + PISP_BE_RGB_ENABLE_DOWNSCALE1 = 0x002000, + PISP_BE_RGB_ENABLE_RESAMPLE0 = 0x008000, + PISP_BE_RGB_ENABLE_RESAMPLE1 = 0x010000, + PISP_BE_RGB_ENABLE_OUTPUT0 = 0x040000, + PISP_BE_RGB_ENABLE_OUTPUT1 = 0x080000, + PISP_BE_RGB_ENABLE_HOG = 0x200000 +}; + +#define PISP_BE_RGB_ENABLE_CSC(i) (PISP_BE_RGB_ENABLE_CSC0 << (i)) +#define PISP_BE_RGB_ENABLE_DOWNSCALE(i) (PISP_BE_RGB_ENABLE_DOWNSCALE0 << (i)) +#define PISP_BE_RGB_ENABLE_RESAMPLE(i) (PISP_BE_RGB_ENABLE_RESAMPLE0 << (i)) +#define PISP_BE_RGB_ENABLE_OUTPUT(i) (PISP_BE_RGB_ENABLE_OUTPUT0 << (i)) + +/* + * We use the enable flags to show when blocks are "dirty", but we need some + * extra ones too. + */ +enum pisp_be_dirty { + PISP_BE_DIRTY_GLOBAL = 0x0001, + PISP_BE_DIRTY_SH_FC_COMBINE = 0x0002, + PISP_BE_DIRTY_CROP = 0x0004 +}; + +/** + * struct pisp_be_global_config - PiSP global enable bitmaps + * @bayer_enables: Bayer input enable flags + * @rgb_enables: RGB output enable flags + * @bayer_order: Bayer input format ordering + * @pad: Padding bytes + */ +struct pisp_be_global_config { + __u32 bayer_enables; + __u32 rgb_enables; + __u8 bayer_order; + __u8 pad[3]; +} __attribute__((packed)); + +/** + * struct pisp_be_input_buffer_config - PiSP Back End input buffer + * @addr: Input buffer address + */ +struct pisp_be_input_buffer_config { + /* low 32 bits followed by high 32 bits (for each of up to 3 planes) */ + __u32 addr[3][2]; +} __attribute__((packed)); + +/** + * struct pisp_be_dpc_config - PiSP Back End DPC config + * + * Defective Pixel Correction configuration + * + * @coeff_level: Coefficient for the darkest neighbouring pixel value + * @coeff_range: Coefficient for the range of pixels for this Bayer channel + * @pad: Padding byte + * @flags: DPC configuration flags + */ +struct pisp_be_dpc_config { + __u8 coeff_level; + __u8 coeff_range; + __u8 pad; +#define PISP_BE_DPC_FLAG_FOLDBACK 1 + __u8 flags; +} __attribute__((packed)); + +/** + * struct pisp_be_geq_config - PiSP Back End GEQ config + * + * Green Equalisation configuration + * + * @offset: Offset value for threshold calculation + * @slope_sharper: Slope/Sharper configuration + * @min: Minimum value the threshold may have + * @max: Maximum value the threshold may have + */ +struct pisp_be_geq_config { + __u16 offset; +#define PISP_BE_GEQ_SHARPER BIT(15) +#define PISP_BE_GEQ_SLOPE ((1 << 10) - 1) + /* top bit is the "sharper" flag, slope value is bottom 10 bits */ + __u16 slope_sharper; + __u16 min; + __u16 max; +} __attribute__((packed)); + +/** + * struct pisp_be_tdn_input_buffer_config - PiSP Back End TDN input buffer + * @addr: TDN input buffer address + */ +struct pisp_be_tdn_input_buffer_config { + /* low 32 bits followed by high 32 bits */ + __u32 addr[2]; +} __attribute__((packed)); + +/** + * struct pisp_be_tdn_config - PiSP Back End TDN config + * + * Temporal Denoise configuration + * + * @black_level: Black level value subtracted from pixels + * @ratio: Multiplier for the LTA input frame + * @noise_constant: Constant offset value used in noise estimation + * @noise_slope: Noise estimation multiplier + * @threshold: Threshold for TDN operations + * @reset: Disable TDN operations + * @pad: Padding byte + */ +struct pisp_be_tdn_config { + __u16 black_level; + __u16 ratio; + __u16 noise_constant; + __u16 noise_slope; + __u16 threshold; + __u8 reset; + __u8 pad; +} __attribute__((packed)); + +/** + * struct pisp_be_tdn_output_buffer_config - PiSP Back End TDN output buffer + * @addr: TDN output buffer address + */ +struct pisp_be_tdn_output_buffer_config { + /* low 32 bits followed by high 32 bits */ + __u32 addr[2]; +} __attribute__((packed)); + +/** + * struct pisp_be_sdn_config - PiSP Back End SDN config + * + * Spatial Denoise configuration + * + * @black_level: Black level subtracted from pixel for noise estimation + * @leakage: Proportion of the original undenoised value to mix in + * denoised output + * @pad: Padding byte + * @noise_constant: Noise constant used for noise estimation + * @noise_slope: Noise slope value used for noise estimation + * @noise_constant2: Second noise constant used for noise estimation + * @noise_slope2: Second slope value used for noise estimation + */ +struct pisp_be_sdn_config { + __u16 black_level; + __u8 leakage; + __u8 pad; + __u16 noise_constant; + __u16 noise_slope; + __u16 noise_constant2; + __u16 noise_slope2; +} __attribute__((packed)); + +/** + * struct pisp_be_stitch_input_buffer_config - PiSP Back End Stitch input + * @addr: Stitch input buffer address + */ +struct pisp_be_stitch_input_buffer_config { + /* low 32 bits followed by high 32 bits */ + __u32 addr[2]; +} __attribute__((packed)); + +#define PISP_BE_STITCH_STREAMING_LONG 0x8000 +#define PISP_BE_STITCH_EXPOSURE_RATIO_MASK 0x7fff + +/** + * struct pisp_be_stitch_config - PiSP Back End Stitch config + * + * Stitch block configuration + * + * @threshold_lo: Low threshold value + * @threshold_diff_power: Low and high threshold difference + * @pad: Padding bytes + * @exposure_ratio: Multiplier to convert long exposure pixels into + * short exposure pixels + * @motion_threshold_256: Motion threshold above which short exposure + * pixels are used + * @motion_threshold_recip: Reciprocal of motion_threshold_256 value + */ +struct pisp_be_stitch_config { + __u16 threshold_lo; + __u8 threshold_diff_power; + __u8 pad; + + /* top bit indicates whether streaming input is the long exposure */ + __u16 exposure_ratio; + + __u8 motion_threshold_256; + __u8 motion_threshold_recip; +} __attribute__((packed)); + +/** + * struct pisp_be_stitch_output_buffer_config - PiSP Back End Stitch output + * @addr: Stitch input buffer address + */ +struct pisp_be_stitch_output_buffer_config { + /* low 32 bits followed by high 32 bits */ + __u32 addr[2]; +} __attribute__((packed)); + +/** + * struct pisp_be_cdn_config - PiSP Back End CDN config + * + * Colour Denoise configuration + * + * @thresh: Constant for noise estimation + * @iir_strength: Relative strength of the IIR part of the filter + * @g_adjust: Proportion of the change assigned to the G channel + */ +struct pisp_be_cdn_config { + __u16 thresh; + __u8 iir_strength; + __u8 g_adjust; +} __attribute__((packed)); + +#define PISP_BE_LSC_LOG_GRID_SIZE 5 +#define PISP_BE_LSC_GRID_SIZE (1 << PISP_BE_LSC_LOG_GRID_SIZE) +#define PISP_BE_LSC_STEP_PRECISION 18 + +/** + * struct pisp_be_lsc_config - PiSP Back End LSC config + * + * Lens Shading Correction configuration + * + * @grid_step_x: Reciprocal of cell size width + * @grid_step_y: Reciprocal of cell size height + * @lut_packed: Jointly-coded RGB gains for each LSC grid + */ +struct pisp_be_lsc_config { + /* (1<<18) / grid_cell_width */ + __u16 grid_step_x; + /* (1<<18) / grid_cell_height */ + __u16 grid_step_y; + /* RGB gains jointly encoded in 32 bits */ +#define PISP_BE_LSC_LUT_SIZE (PISP_BE_LSC_GRID_SIZE + 1) + __u32 lut_packed[PISP_BE_LSC_LUT_SIZE][PISP_BE_LSC_LUT_SIZE]; +} __attribute__((packed)); + +/** + * struct pisp_be_lsc_extra - PiSP Back End LSC Extra config + * @offset_x: Horizontal offset into the LSC table of this tile + * @offset_y: Vertical offset into the LSC table of this tile + */ +struct pisp_be_lsc_extra { + __u16 offset_x; + __u16 offset_y; +} __attribute__((packed)); + +#define PISP_BE_CAC_LOG_GRID_SIZE 3 +#define PISP_BE_CAC_GRID_SIZE (1 << PISP_BE_CAC_LOG_GRID_SIZE) +#define PISP_BE_CAC_STEP_PRECISION 20 + +/** + * struct pisp_be_cac_config - PiSP Back End CAC config + * + * Chromatic Aberration Correction config + * + * @grid_step_x: Reciprocal of cell size width + * @grid_step_y: Reciprocal of cell size height + * @lut: Pixel shift for the CAC grid + */ +struct pisp_be_cac_config { + /* (1<<20) / grid_cell_width */ + __u16 grid_step_x; + /* (1<<20) / grid_cell_height */ + __u16 grid_step_y; + /* [gridy][gridx][rb][xy] */ +#define PISP_BE_CAC_LUT_SIZE (PISP_BE_CAC_GRID_SIZE + 1) + __s8 lut[PISP_BE_CAC_LUT_SIZE][PISP_BE_CAC_LUT_SIZE][2][2]; +} __attribute__((packed)); + +/** + * struct pisp_be_cac_extra - PiSP Back End CAC extra config + * @offset_x: Horizontal offset into the CAC table of this tile + * @offset_y: Horizontal offset into the CAC table of this tile + */ +struct pisp_be_cac_extra { + __u16 offset_x; + __u16 offset_y; +} __attribute__((packed)); + +#define PISP_BE_DEBIN_NUM_COEFFS 4 + +/** + * struct pisp_be_debin_config - PiSP Back End Debin config + * + * Debinning configuration + * + * @coeffs: Filter coefficients for debinning + * @h_enable: Horizontal debinning enable + * @v_enable: Vertical debinning enable + * @pad: Padding bytes + */ +struct pisp_be_debin_config { + __s8 coeffs[PISP_BE_DEBIN_NUM_COEFFS]; + __s8 h_enable; + __s8 v_enable; + __s8 pad[2]; +} __attribute__((packed)); + +#define PISP_BE_TONEMAP_LUT_SIZE 64 + +/** + * struct pisp_be_tonemap_config - PiSP Back End Tonemap config + * + * Tonemapping configuration + * + * @detail_constant: Constant value for threshold calculation + * @detail_slope: Slope value for threshold calculation + * @iir_strength: Relative strength of the IIR fiter + * @strength: Strength factor + * @lut: Look-up table for tonemap curve + */ +struct pisp_be_tonemap_config { + __u16 detail_constant; + __u16 detail_slope; + __u16 iir_strength; + __u16 strength; + __u32 lut[PISP_BE_TONEMAP_LUT_SIZE]; +} __attribute__((packed)); + +/** + * struct pisp_be_demosaic_config - PiSP Back End Demosaic config + * + * Demosaic configuration + * + * @sharper: Use other Bayer channels to increase sharpness + * @fc_mode: Built-in false colour suppression mode + * @pad: Padding bytes + */ +struct pisp_be_demosaic_config { + __u8 sharper; + __u8 fc_mode; + __u8 pad[2]; +} __attribute__((packed)); + +/** + * struct pisp_be_ccm_config - PiSP Back End CCM config + * + * Colour Correction Matrix configuration + * + * @coeffs: Matrix coefficients + * @pad: Padding bytes + * @offsets: Offsets triplet + */ +struct pisp_be_ccm_config { + __s16 coeffs[9]; + __u8 pad[2]; + __s32 offsets[3]; +} __attribute__((packed)); + +/** + * struct pisp_be_sat_control_config - PiSP Back End SAT config + * + * Saturation Control configuration + * + * @shift_r: Left shift for Red colour channel + * @shift_g: Left shift for Green colour channel + * @shift_b: Left shift for Blue colour channel + * @pad: Padding byte + */ +struct pisp_be_sat_control_config { + __u8 shift_r; + __u8 shift_g; + __u8 shift_b; + __u8 pad; +} __attribute__((packed)); + +/** + * struct pisp_be_false_colour_config - PiSP Back End False Colour config + * + * False Colour configuration + * + * @distance: Distance of neighbouring pixels, either 1 or 2 + * @pad: Padding bytes + */ +struct pisp_be_false_colour_config { + __u8 distance; + __u8 pad[3]; +} __attribute__((packed)); + +#define PISP_BE_SHARPEN_SIZE 5 +#define PISP_BE_SHARPEN_FUNC_NUM_POINTS 9 + +/** + * struct pisp_be_sharpen_config - PiSP Back End Sharpening config + * + * Sharpening configuration + * + * @kernel0: Coefficient for filter 0 + * @pad0: Padding byte + * @kernel1: Coefficient for filter 1 + * @pad1: Padding byte + * @kernel2: Coefficient for filter 2 + * @pad2: Padding byte + * @kernel3: Coefficient for filter 3 + * @pad3: Padding byte + * @kernel4: Coefficient for filter 4 + * @pad4: Padding byte + * @threshold_offset0: Offset for filter 0 response calculation + * @threshold_slope0: Slope multiplier for the filter 0 response calculation + * @scale0: Scale factor for filter 0 response calculation + * @pad5: Padding byte + * @threshold_offset1: Offset for filter 0 response calculation + * @threshold_slope1: Slope multiplier for the filter 0 response calculation + * @scale1: Scale factor for filter 0 response calculation + * @pad6: Padding byte + * @threshold_offset2: Offset for filter 0 response calculation + * @threshold_slope2: Slope multiplier for the filter 0 response calculation + * @scale2: Scale factor for filter 0 response calculation + * @pad7: Padding byte + * @threshold_offset3: Offset for filter 0 response calculation + * @threshold_slope3: Slope multiplier for the filter 0 response calculation + * @scale3: Scale factor for filter 0 response calculation + * @pad8: Padding byte + * @threshold_offset4: Offset for filter 0 response calculation + * @threshold_slope4: Slope multiplier for the filter 0 response calculation + * @scale4: Scale factor for filter 0 response calculation + * @pad9: Padding byte + * @positive_strength: Factor to scale the positive sharpening strength + * @positive_pre_limit: Maximum allowed possible positive sharpening value + * @positive_func: Gain factor applied to positive sharpening response + * @positive_limit: Final gain factor applied to positive sharpening + * @negative_strength: Factor to scale the negative sharpening strength + * @negative_pre_limit: Maximum allowed possible negative sharpening value + * @negative_func: Gain factor applied to negative sharpening response + * @negative_limit: Final gain factor applied to negative sharpening + * @enables: Filter enable mask + * @white: White output pixel filter mask + * @black: Black output pixel filter mask + * @grey: Grey output pixel filter mask + */ +struct pisp_be_sharpen_config { + __s8 kernel0[PISP_BE_SHARPEN_SIZE * PISP_BE_SHARPEN_SIZE]; + __s8 pad0[3]; + __s8 kernel1[PISP_BE_SHARPEN_SIZE * PISP_BE_SHARPEN_SIZE]; + __s8 pad1[3]; + __s8 kernel2[PISP_BE_SHARPEN_SIZE * PISP_BE_SHARPEN_SIZE]; + __s8 pad2[3]; + __s8 kernel3[PISP_BE_SHARPEN_SIZE * PISP_BE_SHARPEN_SIZE]; + __s8 pad3[3]; + __s8 kernel4[PISP_BE_SHARPEN_SIZE * PISP_BE_SHARPEN_SIZE]; + __s8 pad4[3]; + __u16 threshold_offset0; + __u16 threshold_slope0; + __u16 scale0; + __u16 pad5; + __u16 threshold_offset1; + __u16 threshold_slope1; + __u16 scale1; + __u16 pad6; + __u16 threshold_offset2; + __u16 threshold_slope2; + __u16 scale2; + __u16 pad7; + __u16 threshold_offset3; + __u16 threshold_slope3; + __u16 scale3; + __u16 pad8; + __u16 threshold_offset4; + __u16 threshold_slope4; + __u16 scale4; + __u16 pad9; + __u16 positive_strength; + __u16 positive_pre_limit; + __u16 positive_func[PISP_BE_SHARPEN_FUNC_NUM_POINTS]; + __u16 positive_limit; + __u16 negative_strength; + __u16 negative_pre_limit; + __u16 negative_func[PISP_BE_SHARPEN_FUNC_NUM_POINTS]; + __u16 negative_limit; + __u8 enables; + __u8 white; + __u8 black; + __u8 grey; +} __attribute__((packed)); + +/** + * struct pisp_be_sh_fc_combine_config - PiSP Back End Sharpening and + * False Colour config + * + * Sharpening and False Colour configuration + * + * @y_factor: Control amount of desaturation of pixels being darkened + * @c1_factor: Control amount of brightening of a pixel for the Cb + * channel + * @c2_factor: Control amount of brightening of a pixel for the Cr + * channel + * @pad: Padding byte + */ +struct pisp_be_sh_fc_combine_config { + __u8 y_factor; + __u8 c1_factor; + __u8 c2_factor; + __u8 pad; +} __attribute__((packed)); + +#define PISP_BE_GAMMA_LUT_SIZE 64 + +/** + * struct pisp_be_gamma_config - PiSP Back End Gamma configuration + * @lut: Gamma curve look-up table + */ +struct pisp_be_gamma_config { + __u32 lut[PISP_BE_GAMMA_LUT_SIZE]; +} __attribute__((packed)); + +/** + * struct pisp_be_crop_config - PiSP Back End Crop config + * + * Crop configuration + * + * @offset_x: Number of pixels cropped from the left of the tile + * @offset_y: Number of pixels cropped from the top of the tile + * @width: Width of the cropped tile output + * @height: Height of the cropped tile output + */ +struct pisp_be_crop_config { + __u16 offset_x, offset_y; + __u16 width, height; +} __attribute__((packed)); + +#define PISP_BE_RESAMPLE_FILTER_SIZE 96 + +/** + * struct pisp_be_resample_config - PiSP Back End Resampling config + * + * Resample configuration + * + * @scale_factor_h: Horizontal scale factor + * @scale_factor_v: Vertical scale factor + * @coef: Resample coefficients + */ +struct pisp_be_resample_config { + __u16 scale_factor_h, scale_factor_v; + __s16 coef[PISP_BE_RESAMPLE_FILTER_SIZE]; +} __attribute__((packed)); + +/** + * struct pisp_be_resample_extra - PiSP Back End Resample config + * + * Resample configuration + * + * @scaled_width: Width in pixels of the scaled output + * @scaled_height: Height in pixels of the scaled output + * @initial_phase_h: Initial horizontal phase + * @initial_phase_v: Initial vertical phase + */ +struct pisp_be_resample_extra { + __u16 scaled_width; + __u16 scaled_height; + __s16 initial_phase_h[3]; + __s16 initial_phase_v[3]; +} __attribute__((packed)); + +/** + * struct pisp_be_downscale_config - PiSP Back End Downscale config + * + * Downscale configuration + * + * @scale_factor_h: Horizontal scale factor + * @scale_factor_v: Vertical scale factor + * @scale_recip_h: Horizontal reciprocal factor + * @scale_recip_v: Vertical reciprocal factor + */ +struct pisp_be_downscale_config { + __u16 scale_factor_h; + __u16 scale_factor_v; + __u16 scale_recip_h; + __u16 scale_recip_v; +} __attribute__((packed)); + +/** + * struct pisp_be_downscale_extra - PiSP Back End Downscale Extra config + * @scaled_width: Scaled image width + * @scaled_height: Scaled image height + */ +struct pisp_be_downscale_extra { + __u16 scaled_width; + __u16 scaled_height; +} __attribute__((packed)); + +/** + * struct pisp_be_hog_config - PiSP Back End HOG config + * + * Histogram of Oriented Gradients configuration + * + * @compute_signed: Set 0 for unsigned gradients, 1 for signed + * @channel_mix: Channels proportions to use + * @stride: Stride in bytes between blocks directly below + */ +struct pisp_be_hog_config { + __u8 compute_signed; + __u8 channel_mix[3]; + __u32 stride; +} __attribute__((packed)); + +struct pisp_be_axi_config { + __u8 r_qos; /* Read QoS */ + __u8 r_cache_prot; /* Read { prot[2:0], cache[3:0] } */ + __u8 w_qos; /* Write QoS */ + __u8 w_cache_prot; /* Write { prot[2:0], cache[3:0] } */ +} __attribute__((packed)); + +/** + * enum pisp_be_transform - PiSP Back End Transform flags + * @PISP_BE_TRANSFORM_NONE: No transform + * @PISP_BE_TRANSFORM_HFLIP: Horizontal flip + * @PISP_BE_TRANSFORM_VFLIP: Vertical flip + * @PISP_BE_TRANSFORM_ROT180: 180 degress rotation + */ +enum pisp_be_transform { + PISP_BE_TRANSFORM_NONE = 0x0, + PISP_BE_TRANSFORM_HFLIP = 0x1, + PISP_BE_TRANSFORM_VFLIP = 0x2, + PISP_BE_TRANSFORM_ROT180 = + (PISP_BE_TRANSFORM_HFLIP | PISP_BE_TRANSFORM_VFLIP) +}; + +struct pisp_be_output_format_config { + struct pisp_image_format_config image; + __u8 transform; + __u8 pad[3]; + __u16 lo; + __u16 hi; + __u16 lo2; + __u16 hi2; +} __attribute__((packed)); + +/** + * struct pisp_be_output_buffer_config - PiSP Back End Output buffer + * @addr: Output buffer address + */ +struct pisp_be_output_buffer_config { + /* low 32 bits followed by high 32 bits (for each of 3 planes) */ + __u32 addr[3][2]; +} __attribute__((packed)); + +/** + * struct pisp_be_hog_buffer_config - PiSP Back End HOG buffer + * @addr: HOG buffer address + */ +struct pisp_be_hog_buffer_config { + /* low 32 bits followed by high 32 bits */ + __u32 addr[2]; +} __attribute__((packed)); + +/** + * struct pisp_be_config - RaspberryPi PiSP Back End Processing configuration + * + * @global: Global PiSP configuration + * @input_format: Input image format + * @decompress: Decompress configuration + * @dpc: Defective Pixel Correction configuration + * @geq: Green Equalisation configuration + * @tdn_input_format: Temporal Denoise input format + * @tdn_decompress: Temporal Denoise decompress configuration + * @tdn: Temporal Denoise configuration + * @tdn_compress: Temporal Denoise compress configuration + * @tdn_output_format: Temporal Denoise output format + * @sdn: Spatial Denoise configuration + * @blc: Black Level Correction configuration + * @stitch_compress: Stitch compress configuration + * @stitch_output_format: Stitch output format + * @stitch_input_format: Stitch input format + * @stitch_decompress: Stitch decompress configuration + * @stitch: Stitch configuration + * @lsc: Lens Shading Correction configuration + * @wbg: White Balance Gain configuration + * @cdn: Colour Denoise configuration + * @cac: Colour Aberration Correction configuration + * @debin: Debinning configuration + * @tonemap: Tonemapping configuration + * @demosaic: Demosaicing configuration + * @ccm: Colour Correction Matrix configuration + * @sat_control: Saturation Control configuration + * @ycbcr: YCbCr colour correction configuration + * @sharpen: Sharpening configuration + * @false_colour: False colour correction + * @sh_fc_combine: Sharpening and False Colour correction + * @ycbcr_inverse: Inverse YCbCr colour correction + * @gamma: Gamma curve configuration + * @csc: Color Space Conversion configuration + * @downscale: Downscale configuration + * @resample: Resampling configuration + * @output_format: Output format configuration + * @hog: HOG configuration + */ +struct pisp_be_config { + struct pisp_be_global_config global; + struct pisp_image_format_config input_format; + struct pisp_decompress_config decompress; + struct pisp_be_dpc_config dpc; + struct pisp_be_geq_config geq; + struct pisp_image_format_config tdn_input_format; + struct pisp_decompress_config tdn_decompress; + struct pisp_be_tdn_config tdn; + struct pisp_compress_config tdn_compress; + struct pisp_image_format_config tdn_output_format; + struct pisp_be_sdn_config sdn; + struct pisp_bla_config blc; + struct pisp_compress_config stitch_compress; + struct pisp_image_format_config stitch_output_format; + struct pisp_image_format_config stitch_input_format; + struct pisp_decompress_config stitch_decompress; + struct pisp_be_stitch_config stitch; + struct pisp_be_lsc_config lsc; + struct pisp_wbg_config wbg; + struct pisp_be_cdn_config cdn; + struct pisp_be_cac_config cac; + struct pisp_be_debin_config debin; + struct pisp_be_tonemap_config tonemap; + struct pisp_be_demosaic_config demosaic; + struct pisp_be_ccm_config ccm; + struct pisp_be_sat_control_config sat_control; + struct pisp_be_ccm_config ycbcr; + struct pisp_be_sharpen_config sharpen; + struct pisp_be_false_colour_config false_colour; + struct pisp_be_sh_fc_combine_config sh_fc_combine; + struct pisp_be_ccm_config ycbcr_inverse; + struct pisp_be_gamma_config gamma; + struct pisp_be_ccm_config csc[PISP_BACK_END_NUM_OUTPUTS]; + struct pisp_be_downscale_config downscale[PISP_BACK_END_NUM_OUTPUTS]; + struct pisp_be_resample_config resample[PISP_BACK_END_NUM_OUTPUTS]; + struct pisp_be_output_format_config + output_format[PISP_BACK_END_NUM_OUTPUTS]; + struct pisp_be_hog_config hog; +} __attribute__((packed)); + +/** + * enum pisp_tile_edge - PiSP Back End Tile position + * @PISP_LEFT_EDGE: Left edge tile + * @PISP_RIGHT_EDGE: Right edge tile + * @PISP_TOP_EDGE: Top edge tile + * @PISP_BOTTOM_EDGE: Bottom edge tile + */ +enum pisp_tile_edge { + PISP_LEFT_EDGE = (1 << 0), + PISP_RIGHT_EDGE = (1 << 1), + PISP_TOP_EDGE = (1 << 2), + PISP_BOTTOM_EDGE = (1 << 3) +}; + +/** + * struct pisp_tile - Raspberry Pi PiSP Back End tile configuration + * + * Tile parameters: each set of tile parameters is a 160-bytes block of data + * which contains the tile processing parameters. + * + * @edge: Edge tile flag + * @pad0: Padding bytes + * @input_addr_offset: Top-left pixel offset, in bytes + * @input_addr_offset2: Top-left pixel offset, in bytes for the second/ + * third image planes + * @input_offset_x: Horizontal offset in pixels of this tile in the + * input image + * @input_offset_y: Vertical offset in pixels of this tile in the + * input image + * @input_width: Width in pixels of this tile + * @input_height: Height in pixels of the this tile + * @tdn_input_addr_offset: TDN input image offset, in bytes + * @tdn_output_addr_offset: TDN output image offset, in bytes + * @stitch_input_addr_offset: Stitch input image offset, in bytes + * @stitch_output_addr_offset: Stitch output image offset, in bytes + * @lsc_grid_offset_x: Horizontal offset in the LSC table for this tile + * @lsc_grid_offset_y: Vertical offset in the LSC table for this tile + * @cac_grid_offset_x: Horizontal offset in the CAC table for this tile + * @cac_grid_offset_y: Horizontal offset in the CAC table for this tile + * @crop_x_start: Number of pixels cropped from the left of the + * tile + * @crop_x_end: Number of pixels cropped from the right of the + * tile + * @crop_y_start: Number of pixels cropped from the top of the + * tile + * @crop_y_end: Number of pixels cropped from the bottom of the + * tile + * @downscale_phase_x: Initial horizontal phase in pixels + * @downscale_phase_y: Initial vertical phase in pixels + * @resample_in_width: Width in pixels of the tile entering the + * Resample block + * @resample_in_height: Height in pixels of the tile entering the + * Resample block + * @resample_phase_x: Initial horizontal phase for the Resample block + * @resample_phase_y: Initial vertical phase for the Resample block + * @output_offset_x: Horizontal offset in pixels where the tile will + * be written into the output image + * @output_offset_y: Vertical offset in pixels where the tile will be + * written into the output image + * @output_width: Width in pixels in the output image of this tile + * @output_height: Height in pixels in the output image of this tile + * @output_addr_offset: Offset in bytes into the output buffer + * @output_addr_offset2: Offset in bytes into the output buffer for the + * second and third plane + * @output_hog_addr_offset: Offset in bytes into the HOG buffer where + * results of this tile are to be written + */ +struct pisp_tile { + __u8 edge; /* enum pisp_tile_edge */ + __u8 pad0[3]; + /* 4 bytes */ + __u32 input_addr_offset; + __u32 input_addr_offset2; + __u16 input_offset_x; + __u16 input_offset_y; + __u16 input_width; + __u16 input_height; + /* 20 bytes */ + __u32 tdn_input_addr_offset; + __u32 tdn_output_addr_offset; + __u32 stitch_input_addr_offset; + __u32 stitch_output_addr_offset; + /* 36 bytes */ + __u32 lsc_grid_offset_x; + __u32 lsc_grid_offset_y; + /* 44 bytes */ + __u32 cac_grid_offset_x; + __u32 cac_grid_offset_y; + /* 52 bytes */ + __u16 crop_x_start[PISP_BACK_END_NUM_OUTPUTS]; + __u16 crop_x_end[PISP_BACK_END_NUM_OUTPUTS]; + __u16 crop_y_start[PISP_BACK_END_NUM_OUTPUTS]; + __u16 crop_y_end[PISP_BACK_END_NUM_OUTPUTS]; + /* 68 bytes */ + /* Ordering is planes then branches */ + __u16 downscale_phase_x[3 * PISP_BACK_END_NUM_OUTPUTS]; + __u16 downscale_phase_y[3 * PISP_BACK_END_NUM_OUTPUTS]; + /* 92 bytes */ + __u16 resample_in_width[PISP_BACK_END_NUM_OUTPUTS]; + __u16 resample_in_height[PISP_BACK_END_NUM_OUTPUTS]; + /* 100 bytes */ + /* Ordering is planes then branches */ + __u16 resample_phase_x[3 * PISP_BACK_END_NUM_OUTPUTS]; + __u16 resample_phase_y[3 * PISP_BACK_END_NUM_OUTPUTS]; + /* 124 bytes */ + __u16 output_offset_x[PISP_BACK_END_NUM_OUTPUTS]; + __u16 output_offset_y[PISP_BACK_END_NUM_OUTPUTS]; + __u16 output_width[PISP_BACK_END_NUM_OUTPUTS]; + __u16 output_height[PISP_BACK_END_NUM_OUTPUTS]; + /* 140 bytes */ + __u32 output_addr_offset[PISP_BACK_END_NUM_OUTPUTS]; + __u32 output_addr_offset2[PISP_BACK_END_NUM_OUTPUTS]; + /* 156 bytes */ + __u32 output_hog_addr_offset; + /* 160 bytes */ +} __attribute__((packed)); + +/** + * struct pisp_be_tiles_config - Raspberry Pi PiSP Back End configuration + * @tiles: Tile descriptors + * @num_tiles: Number of tiles + * @config: PiSP Back End configuration + */ +struct pisp_be_tiles_config { + struct pisp_tile tiles[PISP_BACK_END_NUM_TILES]; + __u32 num_tiles; + struct pisp_be_config config; +} __attribute__((packed)); + +#endif /* _UAPI_PISP_BE_CONFIG_H_ */ diff --git a/include/uapi/linux/media/raspberrypi/pisp_common.h b/include/uapi/linux/media/raspberrypi/pisp_common.h new file mode 100644 index 000000000000..b2522e29c976 --- /dev/null +++ b/include/uapi/linux/media/raspberrypi/pisp_common.h @@ -0,0 +1,199 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * RP1 PiSP common definitions. + * + * Copyright (C) 2021 - Raspberry Pi Ltd. + * + */ +#ifndef _UAPI_PISP_COMMON_H_ +#define _UAPI_PISP_COMMON_H_ + +#include + +struct pisp_image_format_config { + /* size in pixels */ + __u16 width; + __u16 height; + /* must match struct pisp_image_format below */ + __u32 format; + __s32 stride; + /* some planar image formats will need a second stride */ + __s32 stride2; +} __attribute__((packed)); + +enum pisp_bayer_order { + /* + * Note how bayer_order&1 tells you if G is on the even pixels of the + * checkerboard or not, and bayer_order&2 tells you if R is on the even + * rows or is swapped with B. Note that if the top (of the 8) bits is + * set, this denotes a monochrome or greyscale image, and the lower bits + * should all be ignored. + */ + PISP_BAYER_ORDER_RGGB = 0, + PISP_BAYER_ORDER_GBRG = 1, + PISP_BAYER_ORDER_BGGR = 2, + PISP_BAYER_ORDER_GRBG = 3, + PISP_BAYER_ORDER_GREYSCALE = 128 +}; + +enum pisp_image_format { + /* + * Precise values are mostly tbd. Generally these will be portmanteau + * values comprising bit fields and flags. This format must be shared + * throughout the PiSP. + */ + PISP_IMAGE_FORMAT_BPS_8 = 0x00000000, + PISP_IMAGE_FORMAT_BPS_10 = 0x00000001, + PISP_IMAGE_FORMAT_BPS_12 = 0x00000002, + PISP_IMAGE_FORMAT_BPS_16 = 0x00000003, + PISP_IMAGE_FORMAT_BPS_MASK = 0x00000003, + + PISP_IMAGE_FORMAT_PLANARITY_INTERLEAVED = 0x00000000, + PISP_IMAGE_FORMAT_PLANARITY_SEMI_PLANAR = 0x00000010, + PISP_IMAGE_FORMAT_PLANARITY_PLANAR = 0x00000020, + PISP_IMAGE_FORMAT_PLANARITY_MASK = 0x00000030, + + PISP_IMAGE_FORMAT_SAMPLING_444 = 0x00000000, + PISP_IMAGE_FORMAT_SAMPLING_422 = 0x00000100, + PISP_IMAGE_FORMAT_SAMPLING_420 = 0x00000200, + PISP_IMAGE_FORMAT_SAMPLING_MASK = 0x00000300, + + PISP_IMAGE_FORMAT_ORDER_NORMAL = 0x00000000, + PISP_IMAGE_FORMAT_ORDER_SWAPPED = 0x00001000, + + PISP_IMAGE_FORMAT_SHIFT_0 = 0x00000000, + PISP_IMAGE_FORMAT_SHIFT_1 = 0x00010000, + PISP_IMAGE_FORMAT_SHIFT_2 = 0x00020000, + PISP_IMAGE_FORMAT_SHIFT_3 = 0x00030000, + PISP_IMAGE_FORMAT_SHIFT_4 = 0x00040000, + PISP_IMAGE_FORMAT_SHIFT_5 = 0x00050000, + PISP_IMAGE_FORMAT_SHIFT_6 = 0x00060000, + PISP_IMAGE_FORMAT_SHIFT_7 = 0x00070000, + PISP_IMAGE_FORMAT_SHIFT_8 = 0x00080000, + PISP_IMAGE_FORMAT_SHIFT_MASK = 0x000f0000, + + PISP_IMAGE_FORMAT_UNCOMPRESSED = 0x00000000, + PISP_IMAGE_FORMAT_COMPRESSION_MODE_1 = 0x01000000, + PISP_IMAGE_FORMAT_COMPRESSION_MODE_2 = 0x02000000, + PISP_IMAGE_FORMAT_COMPRESSION_MODE_3 = 0x03000000, + PISP_IMAGE_FORMAT_COMPRESSION_MASK = 0x03000000, + + PISP_IMAGE_FORMAT_HOG_SIGNED = 0x04000000, + PISP_IMAGE_FORMAT_HOG_UNSIGNED = 0x08000000, + PISP_IMAGE_FORMAT_INTEGRAL_IMAGE = 0x10000000, + PISP_IMAGE_FORMAT_WALLPAPER_ROLL = 0x20000000, + PISP_IMAGE_FORMAT_THREE_CHANNEL = 0x40000000, + + /* Lastly a few specific instantiations of the above. */ + PISP_IMAGE_FORMAT_SINGLE_16 = PISP_IMAGE_FORMAT_BPS_16, + PISP_IMAGE_FORMAT_THREE_16 = PISP_IMAGE_FORMAT_BPS_16 | + PISP_IMAGE_FORMAT_THREE_CHANNEL +}; + +#define PISP_IMAGE_FORMAT_bps_8(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) == PISP_IMAGE_FORMAT_BPS_8) +#define PISP_IMAGE_FORMAT_bps_10(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) == PISP_IMAGE_FORMAT_BPS_10) +#define PISP_IMAGE_FORMAT_bps_12(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) == PISP_IMAGE_FORMAT_BPS_12) +#define PISP_IMAGE_FORMAT_bps_16(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) == PISP_IMAGE_FORMAT_BPS_16) +#define PISP_IMAGE_FORMAT_bps(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) ? \ + 8 + (2 << (((fmt) & PISP_IMAGE_FORMAT_BPS_MASK) - 1)) : 8) +#define PISP_IMAGE_FORMAT_shift(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_SHIFT_MASK) / PISP_IMAGE_FORMAT_SHIFT_1) +#define PISP_IMAGE_FORMAT_three_channel(fmt) \ + ((fmt) & PISP_IMAGE_FORMAT_THREE_CHANNEL) +#define PISP_IMAGE_FORMAT_single_channel(fmt) \ + (!((fmt) & PISP_IMAGE_FORMAT_THREE_CHANNEL)) +#define PISP_IMAGE_FORMAT_compressed(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_COMPRESSION_MASK) != \ + PISP_IMAGE_FORMAT_UNCOMPRESSED) +#define PISP_IMAGE_FORMAT_sampling_444(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_SAMPLING_MASK) == \ + PISP_IMAGE_FORMAT_SAMPLING_444) +#define PISP_IMAGE_FORMAT_sampling_422(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_SAMPLING_MASK) == \ + PISP_IMAGE_FORMAT_SAMPLING_422) +#define PISP_IMAGE_FORMAT_sampling_420(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_SAMPLING_MASK) == \ + PISP_IMAGE_FORMAT_SAMPLING_420) +#define PISP_IMAGE_FORMAT_order_normal(fmt) \ + (!((fmt) & PISP_IMAGE_FORMAT_ORDER_SWAPPED)) +#define PISP_IMAGE_FORMAT_order_swapped(fmt) \ + ((fmt) & PISP_IMAGE_FORMAT_ORDER_SWAPPED) +#define PISP_IMAGE_FORMAT_interleaved(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_PLANARITY_MASK) == \ + PISP_IMAGE_FORMAT_PLANARITY_INTERLEAVED) +#define PISP_IMAGE_FORMAT_semiplanar(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_PLANARITY_MASK) == \ + PISP_IMAGE_FORMAT_PLANARITY_SEMI_PLANAR) +#define PISP_IMAGE_FORMAT_planar(fmt) \ + (((fmt) & PISP_IMAGE_FORMAT_PLANARITY_MASK) == \ + PISP_IMAGE_FORMAT_PLANARITY_PLANAR) +#define PISP_IMAGE_FORMAT_wallpaper(fmt) \ + ((fmt) & PISP_IMAGE_FORMAT_WALLPAPER_ROLL) +#define PISP_IMAGE_FORMAT_HOG(fmt) \ + ((fmt) & \ + (PISP_IMAGE_FORMAT_HOG_SIGNED | PISP_IMAGE_FORMAT_HOG_UNSIGNED)) + +#define PISP_WALLPAPER_WIDTH 128 /* in bytes */ + +struct pisp_bla_config { + __u16 black_level_r; + __u16 black_level_gr; + __u16 black_level_gb; + __u16 black_level_b; + __u16 output_black_level; + __u8 pad[2]; +} __attribute__((packed)); + +struct pisp_wbg_config { + __u16 gain_r; + __u16 gain_g; + __u16 gain_b; + __u8 pad[2]; +} __attribute__((packed)); + +struct pisp_compress_config { + /* value subtracted from incoming data */ + __u16 offset; + __u8 pad; + /* 1 => Companding; 2 => Delta (recommended); 3 => Combined (for HDR) */ + __u8 mode; +} __attribute__((packed)); + +struct pisp_decompress_config { + /* value added to reconstructed data */ + __u16 offset; + __u8 pad; + /* 1 => Companding; 2 => Delta (recommended); 3 => Combined (for HDR) */ + __u8 mode; +} __attribute__((packed)); + +enum pisp_axi_flags { + /* + * round down bursts to end at a 32-byte boundary, to align following + * bursts + */ + PISP_AXI_FLAG_ALIGN = 128, + /* for FE writer: force WSTRB high, to pad output to 16-byte boundary */ + PISP_AXI_FLAG_PAD = 64, + /* for FE writer: Use Output FIFO level to trigger "panic" */ + PISP_AXI_FLAG_PANIC = 32, +}; + +struct pisp_axi_config { + /* + * burst length minus one, which must be in the range 0:15; OR'd with + * flags + */ + __u8 maxlen_flags; + /* { prot[2:0], cache[3:0] } fields, echoed on AXI bus */ + __u8 cache_prot; + /* QoS field(s) (4x4 bits for FE writer; 4 bits for other masters) */ + __u16 qos; +} __attribute__((packed)); + +#endif /* _UAPI_PISP_COMMON_H_ */ From patchwork Wed Jun 26 18:14:35 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 807669 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2A41190678 for ; Wed, 26 Jun 2024 18:15:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425708; cv=none; b=Uvs+UNgDHpnspBzYDnVpgVZIW0hgkYOdmMEzs+HSYPHrBe5FfJWkGG7c/or4wfyyYxgh7O6WvVhsDutAAlb38qyTys6QnVc9fCUf9XGcSqdztnaG51zHlqpY/kD2qFCBN8m4MrPlygmPYAZo8tXtjkIHJRNfkE3Vu2eh57Q81iA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425708; c=relaxed/simple; bh=w36VwzSV7/Xz4Ojtvo7WQ5jNXSDe/muKSCHHYrnEy0o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ij03Iz+fQ+aznNZe6kWDVq0SpOO8kpI7+fbiWJgFyaUQGNcny64b8yIzFKIbDzTgk7TIWSVtf6dHpph+ZTg6/SMPjU6/HkSxCo782tu90nmPifP6Q9cDu7eAhgyrbGbQoJOYv9FIVKszAmSAn6iWboXM1KGqA+CnLZvY6WgwMaE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=DM/Yu8nE; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="DM/Yu8nE" Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 8C6791AEB; Wed, 26 Jun 2024 20:14:36 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1719425677; bh=w36VwzSV7/Xz4Ojtvo7WQ5jNXSDe/muKSCHHYrnEy0o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DM/Yu8nEm7CstVrHZfjUiBo3o740YJndcyHuM/jtyuu03feolUYePDX5StNOlO++V dz+N2m70lOufvj8Z+ajXzak32ihySdV+SGFB3Ub8+p3M846pUhNTxperuHGAIpHNtl fQrI9jTxPU3MZ0ZnAj2qYCEG+SciCeoPj0u9BWNI= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v11 5/8] media: uapi: Add PiSP Compressed RAW Bayer formats Date: Wed, 26 Jun 2024 20:14:35 +0200 Message-ID: <20240626181440.195137-6-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> References: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add Raspberry Pi compressed RAW Bayer formats. The compression algorithm description is provided by Nick Hollinghurst from Raspberry Pi. Signed-off-by: Jacopo Mondi Reviewed-by: Naushir Patuck --- .../userspace-api/media/v4l/pixfmt-bayer.rst | 1 + .../media/v4l/pixfmt-srggb8-pisp-comp.rst | 74 +++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 10 +++ include/uapi/linux/videodev2.h | 12 +++ 4 files changed, 97 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-srggb8-pisp-comp.rst diff --git a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst index 2500413e5f43..ed3eb432967d 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-bayer.rst @@ -20,6 +20,7 @@ orders. See also `the Wikipedia article on Bayer filter :maxdepth: 1 pixfmt-srggb8 + pixfmt-srggb8-pisp-comp pixfmt-srggb10 pixfmt-srggb10p pixfmt-srggb10alaw8 diff --git a/Documentation/userspace-api/media/v4l/pixfmt-srggb8-pisp-comp.rst b/Documentation/userspace-api/media/v4l/pixfmt-srggb8-pisp-comp.rst new file mode 100644 index 000000000000..5a82a15559d6 --- /dev/null +++ b/Documentation/userspace-api/media/v4l/pixfmt-srggb8-pisp-comp.rst @@ -0,0 +1,74 @@ +.. SPDX-License-Identifier: GFDL-1.1-no-invariants-or-later + +.. _v4l2-pix-fmt-pisp-comp1-rggb: +.. _v4l2-pix-fmt-pisp-comp1-grbg: +.. _v4l2-pix-fmt-pisp-comp1-gbrg: +.. _v4l2-pix-fmt-pisp-comp1-bggr: +.. _v4l2-pix-fmt-pisp-comp1-mono: +.. _v4l2-pix-fmt-pisp-comp2-rggb: +.. _v4l2-pix-fmt-pisp-comp2-grbg: +.. _v4l2-pix-fmt-pisp-comp2-gbrg: +.. _v4l2-pix-fmt-pisp-comp2-bggr: +.. _v4l2-pix-fmt-pisp-comp2-mono: + +************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_PIX_FMT_PISP_COMP1_RGGB ('PC1R'), V4L2_PIX_FMT_PISP_COMP1_GRBG ('PC1G'), V4L2_PIX_FMT_PISP_COMP1_GBRG ('PC1g'), V4L2_PIX_FMT_PISP_COMP1_BGGR ('PC1B), V4L2_PIX_FMT_PISP_COMP1_MONO ('PC1M'), V4L2_PIX_FMT_PISP_COMP2_RGGB ('PC2R'), V4L2_PIX_FMT_PISP_COMP2_GRBG ('PC2G'), V4L2_PIX_FMT_PISP_COMP2_GBRG ('PC2g'), V4L2_PIX_FMT_PISP_COMP2_BGGR ('PC2B), V4L2_PIX_FMT_PISP_COMP2_MONO ('PC2M') +************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************************** + +================================================ +Raspberry Pi PiSP compressed 8-bit Bayer formats +================================================ + +Description +=========== + +The Raspberry Pi ISP (PiSP) uses a family of three fixed-rate compressed Bayer +formats. A black-level offset may be subtracted to improve compression +efficiency; the nominal black level and amount of offset must be signalled out +of band. Each scanline is padded to a multiple of 8 pixels wide, and each block +of 8 horizontally-contiguous pixels is coded using 8 bytes. + +Mode 1 uses a quantization and delta-based coding scheme which preserves up to +12 significant bits. Mode 2 is a simple sqrt-like companding scheme with 6 PWL +chords, preserving up to 12 significant bits. Mode 3 combines both companding +(with 4 chords) and the delta scheme, preserving up to 14 significant bits. + +The remainder of this description applies to Modes 1 and 3. + +Each block of 8 pixels is separated into even and odd phases of 4 pixels, +coded independently by 32-bit words at successive locations in memory. +The two LS bits of each 32-bit word give its "quantization mode". + +In quantization mode 0, the lowest 321 quantization levels are multiples of +FSD/4096 and the remaining levels are successive multiples of FSD/2048. +Quantization modes 1 and 2 use linear quantization with step sizes of +FSD/1024 and FSD/512 respectively. Each of the four pixels is quantized +independently, with rounding to the nearest level. +In quantization mode 2 where the middle two samples have quantized values +(q1,q2) both in the range [384..511], they are coded using 9 bits for q1 +followed by 7 bits for (q2 & 127). Otherwise, for quantization modes +0, 1 and 2: a 9-bit field encodes MIN(q1,q2) which must be in the range +[0..511] and a 7-bit field encodes (q2-q1+64) which must be in [0..127]. + +Each of the outer samples (q0,q3) is encoded using a 7-bit field based +on its inner neighbour q1 or q2. In quantization mode 2 where the inner +sample has a quantized value in the range [448..511], the field value is +(q0-384). Otherwise for quantization modes 0, 1 and 2: The outer sample +is encoded as (q0-MAX(0,q1-64)). q3 is likewise coded based on q2. +Each of these values must be in the range [0..127]. All these fields +of 2, 9, 7, 7, 7 bits respectively are packed in little-endian order +to give a 32-bit word with LE byte order. + +Quantization mode 3 has a "7.5-bit" escape, used when none of the above +encodings will fit. Each pixel value is quantized to the nearest of 176 +levels, where the lowest 95 levels are multiples of FSD/256 and the +remaining levels are multiples of FSD/128 (level 175 represents values +very close to FSD and may require saturating arithmetic to decode). + +Each pair of quantized pixels (q0,q1) or (q2,q3) is jointly coded +by a 15-bit field: 2816*(q0>>4) + 16*q1 + (q0&15). +Three fields of 2, 15, 15 bits are packed in LE order {15,15,2}. + +An implementation of a software decoder of compressed formats is available +in `Raspberry Pi camera applications code base +`_. diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 7c684788997c..5eb4d797d259 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1532,6 +1532,16 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_MT2110T: descr = "Mediatek 10bit Tile Mode"; break; case V4L2_PIX_FMT_MT2110R: descr = "Mediatek 10bit Raster Mode"; break; case V4L2_PIX_FMT_HEXTILE: descr = "Hextile Compressed Format"; break; + case V4L2_PIX_FMT_PISP_COMP1_RGGB: descr = "PiSP 8b RGRG/GBGB mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GRBG: descr = "PiSP 8b GRGR/BGBG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_GBRG: descr = "PiSP 8b GBGB/RGRG mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_BGGR: descr = "PiSP 8b BGBG/GRGR mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP1_MONO: descr = "PiSP 8b monochrome mode1 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_RGGB: descr = "PiSP 8b RGRG/GBGB mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GRBG: descr = "PiSP 8b GRGR/BGBG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_GBRG: descr = "PiSP 8b GBGB/RGRG mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_BGGR: descr = "PiSP 8b BGBG/GRGR mode2 compr"; break; + case V4L2_PIX_FMT_PISP_COMP2_MONO: descr = "PiSP 8b monochrome mode2 compr"; break; default: if (fmt->description[0]) return; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 96fc0456081e..4e91362da6da 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -816,6 +816,18 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_IPU3_SGRBG10 v4l2_fourcc('i', 'p', '3', 'G') /* IPU3 packed 10-bit GRBG bayer */ #define V4L2_PIX_FMT_IPU3_SRGGB10 v4l2_fourcc('i', 'p', '3', 'r') /* IPU3 packed 10-bit RGGB bayer */ +/* Raspberry Pi PiSP compressed formats. */ +#define V4L2_PIX_FMT_PISP_COMP1_RGGB v4l2_fourcc('P', 'C', '1', 'R') /* PiSP 8-bit mode 1 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GRBG v4l2_fourcc('P', 'C', '1', 'G') /* PiSP 8-bit mode 1 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_GBRG v4l2_fourcc('P', 'C', '1', 'g') /* PiSP 8-bit mode 1 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_BGGR v4l2_fourcc('P', 'C', '1', 'B') /* PiSP 8-bit mode 1 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP1_MONO v4l2_fourcc('P', 'C', '1', 'M') /* PiSP 8-bit mode 1 compressed monochrome */ +#define V4L2_PIX_FMT_PISP_COMP2_RGGB v4l2_fourcc('P', 'C', '2', 'R') /* PiSP 8-bit mode 2 compressed RGGB bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GRBG v4l2_fourcc('P', 'C', '2', 'G') /* PiSP 8-bit mode 2 compressed GRBG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_GBRG v4l2_fourcc('P', 'C', '2', 'g') /* PiSP 8-bit mode 2 compressed GBRG bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_BGGR v4l2_fourcc('P', 'C', '2', 'B') /* PiSP 8-bit mode 2 compressed BGGR bayer */ +#define V4L2_PIX_FMT_PISP_COMP2_MONO v4l2_fourcc('P', 'C', '2', 'M') /* PiSP 8-bit mode 2 compressed monochrome */ + /* SDR formats - used only for Software Defined Radio devices */ #define V4L2_SDR_FMT_CU8 v4l2_fourcc('C', 'U', '0', '8') /* IQ u8 */ #define V4L2_SDR_FMT_CU16LE v4l2_fourcc('C', 'U', '1', '6') /* IQ u16le */ From patchwork Wed Jun 26 18:14:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 807668 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8044B190480 for ; Wed, 26 Jun 2024 18:15:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425712; cv=none; b=g77/fUgDOAD/Apy3X6tQwAirzJ99bnmBJOhL9a1BAMJtWVV3UCL8xwKpAS3MjIiy1FFbtvWAvMCqKc4UfNi2fXjUuIC2CBA1WfOJwZQDTQnd/C02cMZBBsohAIk8r/1381z3K4PWgQHgsNmifUS4LBv+KrStJRkBFBZAQUavKcc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1719425712; c=relaxed/simple; bh=oyuCgxTv4hl2JY8XNMK4A6DznDCW7nfLR3RghWmXy8Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X93bYp5Zvew6ySxHV6ansaEIcgLs0qfgm9sVmclrEclc/SMpqtCZ5MkumGTmEYUX8WSf+LhHqmG6nweF132pZp5Mr4rmdAYNBhBPxQCxNhovu2MH7RG/fdX819zQeatKhcW60otq+nZA1JaCAP/EBMcRT+R+1UZGpHlKUwkqWeQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=k769Gkg1; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="k769Gkg1" Received: from localhost.localdomain (93-61-96-190.ip145.fastwebnet.it [93.61.96.190]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 4F8E91BA0; Wed, 26 Jun 2024 20:14:39 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1719425679; bh=oyuCgxTv4hl2JY8XNMK4A6DznDCW7nfLR3RghWmXy8Q=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=k769Gkg1v9U3iY0F/GZNr1M7mDgYoB3c7/Ulv0YEIzr/B6gSN1snOGmTCAFGR1DQB HUACxCRMm33iFFbuSaTVdOZbIa/j6uhTewy5+7hLN79wuFJM9EGiz965m6SQ0NdEEm Yumdx5MA/3zYAN9308ONGXjPApa5cuwZnkBtsuis= From: Jacopo Mondi To: Linux Media Mailing List Cc: Jacopo Mondi , David Plowman , Naushir Patuck , Nick Hollinghurst , Dave Stevenson , Tomi Valkeinen , Laurent Pinchart , Kieran Bingham , Sakari Ailus , Hans Verkuil , Mauro Carvalho Chehab Subject: [PATCH v11 8/8] media: admin-guide: Document the Raspberry Pi PiSP BE Date: Wed, 26 Jun 2024 20:14:38 +0200 Message-ID: <20240626181440.195137-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> References: <20240626181440.195137-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add documentation for the PiSP Back End memory-to-memory ISP. Signed-off-by: Jacopo Mondi --- .../admin-guide/media/raspberrypi-pisp-be.dot | 20 ++++ .../admin-guide/media/raspberrypi-pisp-be.rst | 109 ++++++++++++++++++ .../admin-guide/media/v4l-drivers.rst | 1 + 3 files changed, 130 insertions(+) create mode 100644 Documentation/admin-guide/media/raspberrypi-pisp-be.dot create mode 100644 Documentation/admin-guide/media/raspberrypi-pisp-be.rst diff --git a/Documentation/admin-guide/media/raspberrypi-pisp-be.dot b/Documentation/admin-guide/media/raspberrypi-pisp-be.dot new file mode 100644 index 000000000000..55671dc1d443 --- /dev/null +++ b/Documentation/admin-guide/media/raspberrypi-pisp-be.dot @@ -0,0 +1,20 @@ +digraph board { + rankdir=TB + n00000001 [label="{{ 0 | 1 | 2 | 7} | pispbe\n | { 3 | 4 | 5 | 6}}", shape=Mrecord, style=filled, fillcolor=green] + n00000001:port3 -> n0000001c [style=bold] + n00000001:port4 -> n00000022 [style=bold] + n00000001:port5 -> n00000028 [style=bold] + n00000001:port6 -> n0000002e [style=bold] + n0000000a [label="pispbe-input\n/dev/video0", shape=box, style=filled, fillcolor=yellow] + n0000000a -> n00000001:port0 [style=bold] + n00000010 [label="pispbe-tdn_input\n/dev/video1", shape=box, style=filled, fillcolor=yellow] + n00000010 -> n00000001:port1 [style=bold] + n00000016 [label="pispbe-stitch_input\n/dev/video2", shape=box, style=filled, fillcolor=yellow] + n00000016 -> n00000001:port2 [style=bold] + n0000001c [label="pispbe-output0\n/dev/video3", shape=box, style=filled, fillcolor=yellow] + n00000022 [label="pispbe-output1\n/dev/video4", shape=box, style=filled, fillcolor=yellow] + n00000028 [label="pispbe-tdn_output\n/dev/video5", shape=box, style=filled, fillcolor=yellow] + n0000002e [label="pispbe-stitch_output\n/dev/video6", shape=box, style=filled, fillcolor=yellow] + n00000034 [label="pispbe-config\n/dev/video7", shape=box, style=filled, fillcolor=yellow] + n00000034 -> n00000001:port7 [style=bold] +} diff --git a/Documentation/admin-guide/media/raspberrypi-pisp-be.rst b/Documentation/admin-guide/media/raspberrypi-pisp-be.rst new file mode 100644 index 000000000000..0fcf46f26276 --- /dev/null +++ b/Documentation/admin-guide/media/raspberrypi-pisp-be.rst @@ -0,0 +1,109 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================================= +Raspberry Pi PiSP Back End Memory-to-Memory ISP (pisp-be) +========================================================= + +The PiSP Back End +================= + +The PiSP Back End is a memory-to-memory Image Signal Processor (ISP) which reads +image data from DRAM memory and performs image processing as specified by the +application through the parameters in a configuration buffer, before writing +pixel data back to memory through two distinct output channels. + +The ISP registers and programming model are documented in the `Raspberry Pi +Image Signal Processor (PiSP) Specification document`_ + +The PiSP Back End ISP processes images in tiles. The handling of image +tessellation and the computation of low-level configuration parameters is +realized by a free software library called `libpisp +`_. + +The full image processing pipeline, which involves capturing RAW Bayer data from +an image sensor through a MIPI CSI-2 compatible capture interface, storing them +in DRAM memory and processing them in the PiSP Back End to obtain images usable +by an application is implemented in `libcamera `_ as +part of the Raspberry Pi platform support. + +The pisp-be driver +================== + +The Raspberry Pi PiSP Back End (pisp-be) driver is located under +drivers/media/platform/raspberrypi/pisp-be. It uses the `V4L2 API` to register +a number of video capture and output devices, the `V4L2 subdev API` to register +a subdevice for the ISP that connects the video devices in a single media graph +realized using the `Media Controller (MC) API`. + +The media topology registered by the `pisp-be` driver is represented below: + +.. _pips-be-topology: + +.. kernel-figure:: raspberrypi-pisp-be.dot + :alt: Diagram of the default media pipeline topology + :align: center + + +The media graph registers the following video device nodes: + +- pispbe-input: output device for images to be submitted to the ISP for + processing. +- pispbe-tdn_input: output device for temporal denoise. +- pispbe-stitch_input: output device for image stitching (HDR). +- pispbe-output0: first capture device for processed images. +- pispbe-output1: second capture device for processed images. +- pispbe-tdn_output: capture device for temporal denoise. +- pispbe-stitch_output: capture device for image stitching (HDR). +- pispbe-config: output device for ISP configuration parameters. + +pispbe-input +------------ + +Images to be processed by the ISP are queued to the `pispbe-input` output device +node. For a list of image formats supported as input to the ISP refer to the +`Raspberry Pi Image Signal Processor (PiSP) Specification document`_. + +pispbe-tdn_input, pispbe-tdn_output +----------------------------------- + +The `pispbe-tdn_input` output video device receives images to be processed by +the temporal denoise block which are captured from the `pispbe-tdn_output` +capture video device. Userspace is responsible for maintaining queues on both +devices, and ensuring that buffers completed on the output are queued to the +input. + +pispbe-stitch_input, pispbe-stitch_output +----------------------------------------- + +To realize HDR (high dynamic range) image processing the image stitching and +tonemapping blocks are used. The `pispbe-stitch_output` writes images to memory +and the `pispbe-stitch_input` receives the previously written frame to process +it along with the current input image. Userspace is responsible for maintaining +queues on both devices, and ensuring that buffers completed on the output are +queued to the input. + +pispbe-output0, pispbe-output1 +------------------------------ + +The two capture devices write to memory the pixel data as processed by the ISP. + +pispbe-config +------------- + +The `pispbe-config` output video devices receives a buffer of configuration +parameters that define the desired image processing to be performed by the ISP. + +The format of the ISP configuration parameter is defined by +:c:type:`pisp_be_tiles_config` C structure and the meaning of each parameter is +described in the `Raspberry Pi Image Signal Processor (PiSP) Specification +document`_. + +ISP configuration +================= + +The ISP configuration is described solely by the content of the parameters +buffer. The only parameter that userspace needs to configure using the V4L2 API +is the image format on the output and capture video devices for validation of +the content of the parameters buffer. + +.. _Raspberry Pi Image Signal Processor (PiSP) Specification document: https://datasheets.raspberrypi.com/camera/raspberry-pi-image-signal-processor-specification.pdf diff --git a/Documentation/admin-guide/media/v4l-drivers.rst b/Documentation/admin-guide/media/v4l-drivers.rst index 4120eded9a13..b6af448b9fe9 100644 --- a/Documentation/admin-guide/media/v4l-drivers.rst +++ b/Documentation/admin-guide/media/v4l-drivers.rst @@ -23,6 +23,7 @@ Video4Linux (V4L) driver-specific documentation omap4_camera philips qcom_camss + raspberrypi-pisp-be rcar-fdp1 rkisp1 saa7134