mbox series

[v5,00/10] media: rkvdec: Add H.264 High 10 and 4:2:2 profile support

Message ID 20240618194647.742037-1-jonas@kwiboo.se
Headers show
Series media: rkvdec: Add H.264 High 10 and 4:2:2 profile support | expand

Message

Jonas Karlman June 18, 2024, 7:46 p.m. UTC
This is a re-revival of an old series [1] now that NV15/NV20/NV30 support
for display driver have landed in linux v6.7.

This series adds H.264 High 10 and 4:2:2 profile support to the Rockchip
Video Decoder driver.

Patch 1 adds helpers for calculating plane bytesperline and sizeimage.
Patch 2 adds two new pixelformats for semi-planer 10-bit 4:2:0/4:2:2 YUV.

Patch 3 change to use bytesperline and buffer height to configure strides.
Patch 4 change to use values from SPS/PPS control to configure the HW.

Patch 5-9 refactor code to support filtering of CAPUTRE formats based
on the image format returned from a get_image_fmt ops.

Patch 10 adds final bits to support H.264 High 10 and 4:2:2 profiles.

Tested on a ROCK Pi 4 (RK3399) and Rock64 (RK3328):

  v4l2-compliance 1.26.1, 64 bits, 64-bit time_t
  ...
  Total for rkvdec device /dev/video3: 47, Succeeded: 47, Failed: 0, Warnings: 0

  Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
  ...
  Ran 65/69 tests successfully

  Running test suite JVT-AVC_V1 with decoder FFmpeg-H.264-V4L2-request
  ...
  Ran 129/135 tests successfully

Before this series:

  Running test suite JVT-FR-EXT with decoder FFmpeg-H.264-V4L2-request
  ...
  Ran 44/69 tests successfully

Changes in v5:
- Drop Remove SPS validation at streaming start patch
- Move buffer align from rkvdec_fill_decoded_pixfmt to min/step_width
- Use correct profiles for V4L2_CID_MPEG_VIDEO_H264_PROFILE
- Collect r-b and t-b tags
- Fluster test score same as v4, see [4] and [5]
Link to v4: https://lore.kernel.org/linux-media/20231105165521.3592037-1-jonas@kwiboo.se/

Changes in v4:
- Fix failed v4l2-compliance tests related to CAPTURE queue
- Rework CAPTURE format filter anv validate to use an image format
- Run fluster test suite JVT-FR-EXT [4] and JVT-AVC_V1 [5]
Link to v3: https://lore.kernel.org/linux-media/20231029183427.1781554-1-jonas@kwiboo.se/

Changes in v3:
- Drop merged patches
- Use bpp and bpp_div instead of prior misuse of block_w/block_h
- New patch to use values from SPS/PPS control to configure the HW
- New patch to remove an unnecessary call to validate sps at streaming start
- Reworked pixel format validation
Link to v2: https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/

Changes in v2:
- Collect r-b tags
- SPS pic width and height in mbs validation moved to rkvdec_try_ctrl
- New patch to not override output buffer sizeimage
- Reworked pixel format validation
- Only align decoded buffer instead of changing frmsize step_width
Link to v1: https://lore.kernel.org/linux-media/20200701215616.30874-1-jonas@kwiboo.se/

To fully runtime test this series you may need ffmpeg patches from [2],
this series is also available at [3].

[1] https://lore.kernel.org/linux-media/20200706215430.22859-1-jonas@kwiboo.se/
[2] https://github.com/Kwiboo/FFmpeg/commits/v4l2-request-n6.1-dev/
[3] https://github.com/Kwiboo/linux-rockchip/commits/linuxtv-rkvdec-high-10-v5/
[4] https://gist.github.com/Kwiboo/f4ac15576b2c72887ae2bc5d58b5c865
[5] https://gist.github.com/Kwiboo/459a1c8f1dcb56e45dc7a7a29cc28adf

Regards,
Jonas

Alex Bee (1):
  media: rkvdec: h264: Don't hardcode SPS/PPS parameters

Jonas Karlman (9):
  media: v4l2-common: Add helpers to calculate bytesperline and
    sizeimage
  media: v4l2: Add NV15 and NV20 pixel formats
  media: rkvdec: h264: Use bytesperline and buffer height as virstride
  media: rkvdec: Extract rkvdec_fill_decoded_pixfmt into helper
  media: rkvdec: Move rkvdec_reset_decoded_fmt helper
  media: rkvdec: Extract decoded format enumeration into helper
  media: rkvdec: Add image format concept
  media: rkvdec: Add get_image_fmt ops
  media: rkvdec: h264: Support High 10 and 4:2:2 profiles

 .../media/v4l/pixfmt-yuv-planar.rst           | 128 ++++++++++
 drivers/media/v4l2-core/v4l2-common.c         |  80 ++++---
 drivers/media/v4l2-core/v4l2-ioctl.c          |   2 +
 drivers/staging/media/rkvdec/rkvdec-h264.c    |  64 +++--
 drivers/staging/media/rkvdec/rkvdec.c         | 226 +++++++++++++-----
 drivers/staging/media/rkvdec/rkvdec.h         |  18 +-
 include/uapi/linux/videodev2.h                |   2 +
 7 files changed, 399 insertions(+), 121 deletions(-)

Comments

Dan Carpenter June 19, 2024, 7:21 a.m. UTC | #1
On Tue, Jun 18, 2024 at 07:46:32PM +0000, Jonas Karlman wrote:
> +static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
> +				   enum rkvdec_image_fmt image_fmt)
>  {
>  	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
> +	unsigned int i, j;
>  
>  	if (WARN_ON(!desc))
>  		return 0;
>  
> -	if (index >= desc->num_decoded_fmts)
> -		return 0;
> +	for (i = 0, j = 0; i < desc->num_decoded_fmts; i++) {
> +		if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
> +					   image_fmt) &&
> +		    index == j++)

I really don't care for this j++ which is tucked inside a condition.

> +			return desc->decoded_fmts[i].fourcc;
> +	}

	int fmt_idx = -1;
	unsigned int i;

	for (i = 0, i < desc->num_decoded_fmts; i++) {
		if (!rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
					    image_fmt))
			continue;
		fmt_idx++;
		if (index == fmt_idx)
			return desc->decoded_fmts[i].fourcc;
	}

	return 0;

>  
> -	return desc->decoded_fmts[index];
> +	return 0;
>  }

Don't resend if this is the only issue, but if you do resend could you
get rid of the j++?

regards,
dan carpenter
Jonas Karlman June 27, 2024, 9:39 a.m. UTC | #2
Hi Dan,

On 2024-06-19 09:21, Dan Carpenter wrote:
> On Tue, Jun 18, 2024 at 07:46:32PM +0000, Jonas Karlman wrote:
>> +static u32 rkvdec_enum_decoded_fmt(struct rkvdec_ctx *ctx, int index,
>> +				   enum rkvdec_image_fmt image_fmt)
>>  {
>>  	const struct rkvdec_coded_fmt_desc *desc = ctx->coded_fmt_desc;
>> +	unsigned int i, j;
>>  
>>  	if (WARN_ON(!desc))
>>  		return 0;
>>  
>> -	if (index >= desc->num_decoded_fmts)
>> -		return 0;
>> +	for (i = 0, j = 0; i < desc->num_decoded_fmts; i++) {
>> +		if (rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
>> +					   image_fmt) &&
>> +		    index == j++)
> 
> I really don't care for this j++ which is tucked inside a condition.
> 
>> +			return desc->decoded_fmts[i].fourcc;
>> +	}
> 
> 	int fmt_idx = -1;
> 	unsigned int i;
> 
> 	for (i = 0, i < desc->num_decoded_fmts; i++) {
> 		if (!rkvdec_image_fmt_match(desc->decoded_fmts[i].image_fmt,
> 					    image_fmt))
> 			continue;
> 		fmt_idx++;
> 		if (index == fmt_idx)
> 			return desc->decoded_fmts[i].fourcc;
> 	}
> 
> 	return 0;
> 
>>  
>> -	return desc->decoded_fmts[index];
>> +	return 0;
>>  }
> 
> Don't resend if this is the only issue, but if you do resend could you
> get rid of the j++?

Thanks, I will send a v6 later today/tomorrow and will include your
proposed change.

Regards,
Jonas

> 
> regards,
> dan carpenter