From patchwork Thu Jan 12 12:56:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641782 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C50B3C67871 for ; Thu, 12 Jan 2023 12:56:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231960AbjALM44 (ORCPT ); Thu, 12 Jan 2023 07:56:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43150 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232416AbjALM4h (ORCPT ); Thu, 12 Jan 2023 07:56:37 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF9B75F46; Thu, 12 Jan 2023 04:56:33 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id 2D0E86602DB9; Thu, 12 Jan 2023 12:56:32 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528192; bh=w39uQHzOIPIO90WhRE4IjTj/4vTR7fwhKBVkNmGEQWM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=NBGxuiJVU2ZpQxkVaZeH5CCCS70AGYjTI2usSFgDYvqxG8S1aMWZyj8I5xBnf+XQa A9xd20BDkGD/y+dweWOYWDuG/iqx0M4ZSAXqhbiY4PCZp2ss/gdd6Cfi/J+h4/z0mV 2v1arb4yPppKzNsKyLOPG3o05bVXUVQcqv1Nv8By09CsEXEW/82QCkhHyC9Qjii4UP 9k1B/ZiCjsBeXmpBEtrf/CDIH6uVBePNk2JZ2P1sc1NheN7/vGI+tEkhYTAIWkht+T G1O14WBe3J1zzQFR6kwGybsbOWqEmRYVBqU4PCivCWju4bfEF1fAjb/HVCPDSdYg89 bVDiRKaYnQPaw== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:16 +0100 Subject: [PATCH v2 01/12] media: v4l2: Add NV15 pixel format MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-1-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=6437; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=uy2g9GPcP+lRwPYfgkcuiU1U/soD1j3DQtWijfz/vHI=; b=/pxqbPJRJoRG0ZRyvRot/+yIY4H7+UpAZVlvL+UZ7AKc8JtwF17tqhJfcI1anbbRdRkx4HIuWV2Q xDA5Hg6pCybcCwPBMeJzHeOj63Bav5GFBGgjC+fTds/GMfujTawu X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jonas Karlman Add the NV15 pixel format used by the Rockchip Video Decoder for 10-bit buffers. NV15 is a packed 10-bit 4:2:0 Y/CbCr format similar to P010 and P210 but has no padding between components. Instead, luminance and chrominance samples are grouped into 4s so that each group is packed into an integer number of bytes: YYYY = UVUV = 4 * 10 bits = 40 bits = 5 bytes The '15' suffix refers to the optimum effective bits per pixel which is achieved when the total number of luminance samples is a multiple of 8 for NV15. Signed-off-by: Jonas Karlman Signed-off-by: Sebastian Fricke --- .../userspace-api/media/v4l/pixfmt-yuv-planar.rst | 75 ++++++++++++++++++++++ drivers/media/v4l2-core/v4l2-common.c | 2 + drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 4 files changed, 79 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst index f1d5bb7b806d..7d8d228f8063 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-yuv-planar.rst @@ -79,6 +79,13 @@ All components are stored with the same number of bits per component. - Cr, Cb - Yes - Linear + * - V4L2_PIX_FMT_NV15 + - 'NV15' + - 15 + - 4:2:0 + - Cb, Cr + - Yes + - Linear * - V4L2_PIX_FMT_NV12M - 'NM12' - 8 @@ -183,6 +190,7 @@ horizontally. .. _V4L2-PIX-FMT-NV12: .. _V4L2-PIX-FMT-NV21: +.. _V4L2-PIX-FMT-NV15: .. _V4L2-PIX-FMT-NV12M: .. _V4L2-PIX-FMT-NV21M: .. _V4L2-PIX-FMT-P010: @@ -586,6 +594,73 @@ Data in the 10 high bits, zeros in the 6 low bits, arranged in little endian ord - Cb\ :sub:`11` - Cr\ :sub:`11` +.. _V4L2_PIX_FMT_NV15: + +NV15 +---- + +Similar to P010, a semi-planar 10-bit Y/CbCr format, but all components are +packed without any padding between each other. As a side-effect, each group of +4 components are stored over 5 bytes (YYYY or UVUV = 4 * 10 bits = 40 bits = 5 +bytes). + +.. flat-table:: Sample 4x4 NV15 Image + :header-rows: 0 + :stub-columns: 0 + + * - start + 0: + - Y'\ :sub:`00` + - Y'\ :sub:`01` + - Y'\ :sub:`02` + - Y'\ :sub:`03` + * - start + 8: + - Y'\ :sub:`04` + - Y'\ :sub:`10` + - Y'\ :sub:`11` + - Y'\ :sub:`12` + * - start + 16: + - Y'\ :sub:`13` + - Y'\ :sub:`14` + - Y'\ :sub:`20` + - Y'\ :sub:`21` + * - start + 24: + - Y'\ :sub:`22` + - Y'\ :sub:`23` + - Y'\ :sub:`24` + - Y'\ :sub:`30` + * - start + 32: + - Y'\ :sub:`31` + - Y'\ :sub:`32` + - Y'\ :sub:`33` + - Y'\ :sub:`34` + + * - start + 0: + - Cb\ :sub:`00` + - Cr\ :sub:`00` + - Cb\ :sub:`01` + - Cr\ :sub:`01` + * - start + 8: + - Cb\ :sub:`02` + - Cr\ :sub:`02` + - Cb\ :sub:`03` + - Cr\ :sub:`03` + * - start + 16: + - Cb\ :sub:`04` + - Cr\ :sub:`04` + - Cb\ :sub:`10` + - Cr\ :sub:`10` + * - start + 24: + - Cb\ :sub:`11` + - Cr\ :sub:`11` + - Cb\ :sub:`12` + - Cr\ :sub:`12` + * - start + 32: + - Cb\ :sub:`13` + - Cr\ :sub:`13` + - Cb\ :sub:`14` + - Cr\ :sub:`14` + +.. raw:: latex Fully Planar YUV Formats ======================== diff --git a/drivers/media/v4l2-core/v4l2-common.c b/drivers/media/v4l2-core/v4l2-common.c index 40f56e044640..be23e319fb3a 100644 --- a/drivers/media/v4l2-core/v4l2-common.c +++ b/drivers/media/v4l2-core/v4l2-common.c @@ -262,6 +262,8 @@ const struct v4l2_format_info *v4l2_format_info(u32 format) /* YUV planar formats */ { .format = V4L2_PIX_FMT_NV12, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 2 }, { .format = V4L2_PIX_FMT_NV21, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 2 }, + { .format = V4L2_PIX_FMT_NV15, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 5, 5, 0, 0 }, .hdiv = 2, .vdiv = 2, + .block_w = { 4, 2, 0, 0 }, .block_h = { 1, 1, 0, 0 } }, { .format = V4L2_PIX_FMT_NV16, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 }, { .format = V4L2_PIX_FMT_NV61, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 2, .vdiv = 1 }, { .format = V4L2_PIX_FMT_NV24, .pixel_enc = V4L2_PIXEL_ENC_YUV, .mem_planes = 1, .comp_planes = 2, .bpp = { 1, 2, 0, 0 }, .hdiv = 1, .vdiv = 1 }, diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 8e0a0ff62a70..1c80ad78ef00 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1343,6 +1343,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_M420: descr = "YUV 4:2:0 (M420)"; break; case V4L2_PIX_FMT_NV12: descr = "Y/UV 4:2:0"; break; case V4L2_PIX_FMT_NV21: descr = "Y/VU 4:2:0"; break; + case V4L2_PIX_FMT_NV15: descr = "10-bit Y/UV 4:2:0 (Packed)"; break; case V4L2_PIX_FMT_NV16: descr = "Y/UV 4:2:2"; break; case V4L2_PIX_FMT_NV61: descr = "Y/VU 4:2:2"; break; case V4L2_PIX_FMT_NV24: descr = "Y/UV 4:4:4"; break; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 1befd181a4cc..e9731286dc77 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -621,6 +621,7 @@ struct v4l2_pix_format { /* two planes -- one Y, one Cr + Cb interleaved */ #define V4L2_PIX_FMT_NV12 v4l2_fourcc('N', 'V', '1', '2') /* 12 Y/CbCr 4:2:0 */ #define V4L2_PIX_FMT_NV21 v4l2_fourcc('N', 'V', '2', '1') /* 12 Y/CrCb 4:2:0 */ +#define V4L2_PIX_FMT_NV15 v4l2_fourcc('N', 'V', '1', '5') /* 15 Y/CbCr 4:2:0 10-bit packed */ #define V4L2_PIX_FMT_NV16 v4l2_fourcc('N', 'V', '1', '6') /* 16 Y/CbCr 4:2:2 */ #define V4L2_PIX_FMT_NV61 v4l2_fourcc('N', 'V', '6', '1') /* 16 Y/CrCb 4:2:2 */ #define V4L2_PIX_FMT_NV24 v4l2_fourcc('N', 'V', '2', '4') /* 24 Y/CbCr 4:4:4 */ From patchwork Thu Jan 12 12:56:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641781 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 47A82C54EBD for ; Thu, 12 Jan 2023 12:57:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231743AbjALM5A (ORCPT ); Thu, 12 Jan 2023 07:57:00 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43164 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231651AbjALM4k (ORCPT ); Thu, 12 Jan 2023 07:56:40 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E901B641D; Thu, 12 Jan 2023 04:56:39 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id 9E63B6602DBA; Thu, 12 Jan 2023 12:56:38 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528198; bh=8FFms7ZFvtbGRTeN4REh/YGhyU+oeFdqYWO/CK2KT8c=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hXDggCyHhkjfeIswJAT/y/OZ3Uv10V8Cte6Wd/pyHY8rOY6dr9GSn0wpvJqTPLhAn huVgJgPug9zwf6cZosazKC86t1j54jjlPgFuIsy+QRuqZT1+iPX9piPWAJMlNjqjuG OScE+wYlc6cVvt/TBZahO964vjW3VE37MbhQ94bfU0QB+iyniOOy6KJEcIgZxSjqQK fotcKNvAIDnTYGKuQZXOYWdvsbeIDsUTfJ2+wAx0S2d8E1/NX8hW0NhtfMSaHK0Ur2 Dao5hfPydn5QXY2ImcTCrAm24Q/K0DZKTPJECRS5W5DOD0MUySUoXpHHjb1QhUGGu/ qieM9+NzZLXQg== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:18 +0100 Subject: [PATCH v2 03/12] staging: media: rkvdec: Helper for buffer queue busy check MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-3-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=2269; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=8FFms7ZFvtbGRTeN4REh/YGhyU+oeFdqYWO/CK2KT8c=; b=ub/wDdTpC9jeMsoCSRmmqPKTVKH8soQUtOakduE6Kmcgt/JvOwM+tMckRDcgsVcmdLW9umjznn05 a7TSEWbUC2GDJJri/M43YYI86BB6vTZEyOuxzEaRrXkA+V76F2vz X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Implement a helper function, wrapping around getting a vb2 queue and checking whether it is busy, to reduce the amount of code duplication in the driver. Signed-off-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 7bab7586918c..c849f6c20279 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -27,6 +27,17 @@ #include "rkvdec.h" #include "rkvdec-regs.h" +static int rkvdec_queue_busy(struct rkvdec_ctx *ctx, enum v4l2_buf_type buf_type) +{ + struct vb2_queue *vq; + + vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, buf_type); + if (vb2_is_busy(vq)) + return -EBUSY; + else + return 0; +} + static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); @@ -311,13 +322,10 @@ static int rkvdec_s_capture_fmt(struct file *file, void *priv, struct v4l2_format *f) { struct rkvdec_ctx *ctx = fh_to_rkvdec_ctx(priv); - struct vb2_queue *vq; int ret; /* Change not allowed if queue is busy */ - vq = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, - V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); - if (vb2_is_busy(vq)) + if (rkvdec_queue_busy(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) != 0) return -EBUSY; ret = rkvdec_try_capture_fmt(file, priv, f); @@ -335,7 +343,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, struct v4l2_m2m_ctx *m2m_ctx = ctx->fh.m2m_ctx; const struct rkvdec_coded_fmt_desc *desc; struct v4l2_format *cap_fmt; - struct vb2_queue *peer_vq, *vq; + struct vb2_queue *vq; int ret; /* @@ -354,8 +362,7 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, * queue, we can't allow doing so when the CAPTURE queue has buffers * allocated. */ - peer_vq = v4l2_m2m_get_vq(m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); - if (vb2_is_busy(peer_vq)) + if (rkvdec_queue_busy(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) != 0) return -EBUSY; ret = rkvdec_try_output_fmt(file, priv, f); From patchwork Thu Jan 12 12:56:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641779 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49C0FC54EBD for ; Thu, 12 Jan 2023 12:57:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231948AbjALM5K (ORCPT ); Thu, 12 Jan 2023 07:57:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43276 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231728AbjALM4n (ORCPT ); Thu, 12 Jan 2023 07:56:43 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C51F24914A; Thu, 12 Jan 2023 04:56:42 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id 830346602DB1; Thu, 12 Jan 2023 12:56:41 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528201; bh=AVEg4DSMK0gqYca27eD54R157QB5YV2sWChYYaBFpmI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=LM1vgt+pOaabBAJ85FdbLo9wq5mUT0w0x5nAszJ37VGijYHKcIXfdOuoVfBt0gFKM 6V0TsNYP9Ceg5NVgCMb5s0IZWo5rZ1dbo1r5fvwCxpo0VEyHldVhngCZ0XpZuXi7cB aNfDgYu++punO6h4oxFkcZ7KjeWXaYxHFhfFxOuq9Le8mNU8nv04VPbmGJwPlU5+L9 H+xIH29P1Jx2f8ei32MK42TPAd34QOcwHrFPJGwMZzJFHYJ2kzjYxAbPEAb3fIUCjd 6xgX3fHo389Rsu+JdAl860N6hgqDWhXpo04SbF0lJbmDCfInbfMID2Zwd3C7+fJY9z H5eTXOOfjX8wQ== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:19 +0100 Subject: [PATCH v2 04/12] staging: media: rkvdec: Block start streaming until both queues run MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-4-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=1025; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=AVEg4DSMK0gqYca27eD54R157QB5YV2sWChYYaBFpmI=; b=4fInMsBgcT6jf30p/7TvMJW1s4sjmIUMu2MIX6uCwRuxM7iBhvTpC/00Nu4xtEZo2TrulpGwWaVw pfVDB17lDTsDfzQFYB1UAn0aetGnBIbCWNnkmQVCdoIqa/IR7mT0 X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Ensure that both the CAPTURE and the OUTPUT queue are running (e.g. busy -> have buffers allocated) before starting the actual streaming process. Signed-off-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c849f6c20279..e0e95d06e216 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -562,6 +562,13 @@ static int rkvdec_start_streaming(struct vb2_queue *q, unsigned int count) if (V4L2_TYPE_IS_CAPTURE(q->type)) return 0; + /* + * Make sure that both the output and the capture queue are running + */ + if (rkvdec_queue_busy(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) == 0 || + rkvdec_queue_busy(ctx, V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) == 0) + return -EAGAIN; + desc = ctx->coded_fmt_desc; if (WARN_ON(!desc)) return -EINVAL; From patchwork Thu Jan 12 12:56:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641780 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4CD68C54EBD for ; Thu, 12 Jan 2023 12:57:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231660AbjALM5G (ORCPT ); Thu, 12 Jan 2023 07:57:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43318 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231679AbjALM4v (ORCPT ); Thu, 12 Jan 2023 07:56:51 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e5ab]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 10D2939F81; Thu, 12 Jan 2023 04:56:49 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id BA61F6602DB1; Thu, 12 Jan 2023 12:56:47 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528207; bh=Da6QtFYOuId0q8spuTSgR9gP0GfvHSM7GfQl76avmIU=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=dHzAuxLpAj79iY9LVET5QkBL9VPhVA1ioJQwPn2iFXHU4fDdFn9jzLrSF87VecJ2y 1g4/d6x53E0SC3CNDg90z3NYtWNIMSmbDKQL9hU958IXnDuafam/WNGpmJbUwSYfrB 7XAEnIX34wDmWKijisW8hShchRspOnId3nGqxD0wNFptvHvdLBIZ2hP5HxLP0mbtsB 8vfs4fP10dIJERtoHYCSiY6fm95Y2n60qeW6NRfZAfazo04rhOlRNAw44VAk+2TSC5 Y/yrD0dScreg28gKD8HaSfR1/oASNvZ/pY/7vp1xqbDHuvtoJq1cHwigZI9Im2CVlN e0Za9curbjdng== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:21 +0100 Subject: [PATCH v2 06/12] staging: media: rkvdec: Add a valid pixel format check as callback MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-6-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=3923; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=Da6QtFYOuId0q8spuTSgR9gP0GfvHSM7GfQl76avmIU=; b=Ei9OhrN5lMhBxVdjk4LBAl2cBQ2833Mt1irFmo2edGvikdXUqvhqqNyW0wxQ84YZ9drS3FRc9gZs 0Ef1O9TjAyFyFdBeAO/IeTnNjUYTD+y6bGpxE1du+FdS0vTDEOur X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Provide a callback for codec variant drivers to indicate the correct output pixel-format. It will either utilize the SPS structure stored via the S_CTRL IOCTL or return the default pixel-format. Signed-off-by: Jonas Karlman Signed-off-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 45 +++++++++++++++++++++++++++++------ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 39 insertions(+), 7 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e0e95d06e216..a46f918926a2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -38,6 +38,20 @@ static int rkvdec_queue_busy(struct rkvdec_ctx *ctx, enum v4l2_buf_type buf_type return 0; } +/* + * Use the current SPS, set by the user via the VIDIOC_S_CTRL IOCTL, + * to determine the optimal pixel-format. Each codec is responsible + * for choosing the appropriate pixel-format for a given parameter set. + */ +static int rkvdec_get_valid_fmt(struct rkvdec_ctx *ctx) +{ + const struct rkvdec_coded_fmt_desc *coded_desc = ctx->coded_fmt_desc; + + if (coded_desc->ops->valid_fmt) + return coded_desc->ops->valid_fmt(ctx); + return ctx->coded_fmt_desc->decoded_fmts[0]; +} + static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); @@ -200,8 +214,9 @@ static void rkvdec_reset_coded_fmt(struct rkvdec_ctx *ctx) static void rkvdec_reset_decoded_fmt(struct rkvdec_ctx *ctx) { struct v4l2_format *f = &ctx->decoded_fmt; + u32 valid_fmt = rkvdec_get_valid_fmt(ctx); - rkvdec_reset_fmt(ctx, f, ctx->coded_fmt_desc->decoded_fmts[0]); + rkvdec_reset_fmt(ctx, f, valid_fmt); f->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; v4l2_fill_pixfmt_mp(&f->fmt.pix_mp, ctx->coded_fmt_desc->decoded_fmts[0], @@ -260,13 +275,17 @@ static int rkvdec_try_capture_fmt(struct file *file, void *priv, if (WARN_ON(!coded_desc)) return -EINVAL; - for (i = 0; i < coded_desc->num_decoded_fmts; i++) { - if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat) - break; - } + if (ctx->sps) { + pix_mp->pixelformat = rkvdec_get_valid_fmt(ctx); + } else { + for (i = 0; i < coded_desc->num_decoded_fmts; i++) { + if (coded_desc->decoded_fmts[i] == pix_mp->pixelformat) + break; + } - if (i == coded_desc->num_decoded_fmts) - pix_mp->pixelformat = coded_desc->decoded_fmts[0]; + if (i == coded_desc->num_decoded_fmts) + pix_mp->pixelformat = coded_desc->decoded_fmts[0]; + } /* Always apply the frmsize constraint of the coded end. */ pix_mp->width = max(pix_mp->width, ctx->coded_fmt.fmt.pix_mp.width); @@ -435,6 +454,18 @@ static int rkvdec_enum_capture_fmt(struct file *file, void *priv, if (WARN_ON(!ctx->coded_fmt_desc)) return -EINVAL; + /* + * According to the specification the driver can only return formats + * that are supported by both the current encoded format and the + * provided controls + */ + if (ctx->sps) { + if (f->index) + return -EINVAL; + f->pixelformat = rkvdec_get_valid_fmt(ctx); + return 0; + } + if (f->index >= ctx->coded_fmt_desc->num_decoded_fmts) return -EINVAL; diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 332126e7b812..e353a4403e5b 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -66,6 +66,7 @@ vb2_to_rkvdec_decoded_buf(struct vb2_buffer *buf) struct rkvdec_coded_fmt_ops { int (*adjust_fmt)(struct rkvdec_ctx *ctx, struct v4l2_format *f); + u32 (*valid_fmt)(struct rkvdec_ctx *ctx); int (*start)(struct rkvdec_ctx *ctx); void (*stop)(struct rkvdec_ctx *ctx); int (*run)(struct rkvdec_ctx *ctx); From patchwork Thu Jan 12 12:56:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 941CFC54EBC for ; Thu, 12 Jan 2023 12:57:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232241AbjALM5Q (ORCPT ); Thu, 12 Jan 2023 07:57:16 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42866 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231946AbjALM44 (ORCPT ); Thu, 12 Jan 2023 07:56:56 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B5C097661; Thu, 12 Jan 2023 04:56:55 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id 71D096602DB1; Thu, 12 Jan 2023 12:56:54 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528214; bh=NR4NgmAwqSKt432W3wHLkNsbDlb7Ku6yU6hoKm5PFhw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=iFWWVGd0PM+MpyqfisZJ8ZATeRpN9ueRYPloz+zab6riogtw9/H/jMvGclFgaXL27 Iecy/xV1AUO4NhXd2HknaBrbA0KNS+e4fS8zCri1Gh8xW2Nod13mZ+LxeV5n8F5yjc +cURbe6rY59CdaVeW9cYcNKzo9bRGgp7M0wXHO+aM/vRP9rOQbnS+mBCiM8vilslz/ G93/nO79P39GLfJEMTbEWrRV42SsXRH753uhUx1YgS5OC67wi014XyGjHPU2+rin1H 7X5xq9KghhFbMBukRC+/7069InJwB7ymRkR6URNI299eTDzRWoAjRSU5t8Ca41MoFF wfWWZlcj7oERg== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:23 +0100 Subject: [PATCH v2 08/12] staging: media: rkvdec: Add a valid SPS check as a callback MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-8-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=3157; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=NR4NgmAwqSKt432W3wHLkNsbDlb7Ku6yU6hoKm5PFhw=; b=IQDkaNXJm1EiGyDmWoVKY4f+RD7CQ6A9pSeRltErtvX0pqcjW0RWXs2BZkhZonGnlhJ66v4IbQHv T3KBCKDWAP+0YbahHXPuYtABeVBdfI70Rp3wIpLpoKQRRGG5TI/Z X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a callback for each codec variant, that ensures that a given format is compatible with the SPS that is stored in the context of the instance. This is used for two scenarios: - New output queue format is set, which is incompatible with the currently stored SPS - Start streaming is called but the SPS doesn't match the current format anymore Signed-off-by: Sebastian Fricke --- drivers/staging/media/rkvdec/rkvdec.c | 35 +++++++++++++++++++++++++++++++++++ drivers/staging/media/rkvdec/rkvdec.h | 1 + 2 files changed, 36 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index e8c750a7343a..8d948bcc4e46 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -52,6 +52,26 @@ static int rkvdec_get_valid_fmt(struct rkvdec_ctx *ctx) return ctx->coded_fmt_desc->decoded_fmts[0]; } +static int rkvdec_sps_check(struct rkvdec_ctx *ctx, struct v4l2_pix_format_mplane *pix_mp) +{ + const struct rkvdec_coded_fmt_desc *coded_desc = ctx->coded_fmt_desc; + + /* + * When a codec doesn't implement the SPS check, handle it as if no + * SPS exists for the codec. + */ + if (coded_desc->ops->sps_check) { + if (!ctx->sps) + return -EINVAL; + + if (!pix_mp) + pix_mp = &ctx->decoded_fmt.fmt.pix_mp; + return coded_desc->ops->sps_check(ctx, ctx->sps, pix_mp); + } + + return 0; +} + static int rkvdec_get_sps_attributes(struct rkvdec_ctx *ctx, void *sps, struct sps_attributes *attributes) { @@ -375,6 +395,13 @@ static int rkvdec_s_output_fmt(struct file *file, void *priv, struct vb2_queue *vq; int ret; + /* + * When the new output format doesn't match the existing SPS stored in + * the context, then the SPS needs to be reset by user-space. + */ + if (rkvdec_sps_check(ctx, &f->fmt.pix_mp) < 0) + ctx->sps = NULL; + /* * In order to support dynamic resolution change, the decoder admits * a resolution change, as long as the pixelformat remains. Can't be @@ -603,6 +630,14 @@ static int rkvdec_start_streaming(struct vb2_queue *q, unsigned int count) if (V4L2_TYPE_IS_CAPTURE(q->type)) return 0; + /* + * An incompatible SPS at this point is invalid, abort the process. + */ + if (rkvdec_sps_check(ctx, NULL) < 0) { + ctx->sps = NULL; + return -EINVAL; + } + /* * Make sure that both the output and the capture queue are running */ diff --git a/drivers/staging/media/rkvdec/rkvdec.h b/drivers/staging/media/rkvdec/rkvdec.h index 7b6702c360fd..d74821f9fd0b 100644 --- a/drivers/staging/media/rkvdec/rkvdec.h +++ b/drivers/staging/media/rkvdec/rkvdec.h @@ -75,6 +75,7 @@ struct rkvdec_coded_fmt_ops { int (*adjust_fmt)(struct rkvdec_ctx *ctx, struct v4l2_format *f); u32 (*valid_fmt)(struct rkvdec_ctx *ctx); + int (*sps_check)(struct rkvdec_ctx *ctx, void *sps, struct v4l2_pix_format_mplane *pix_mp); int (*get_sps_attributes)(struct rkvdec_ctx *ctx, void *sps, struct sps_attributes *attributes); int (*start)(struct rkvdec_ctx *ctx); From patchwork Thu Jan 12 12:56:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sebastian Fricke X-Patchwork-Id: 641777 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02E41C54EBD for ; Thu, 12 Jan 2023 12:57:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232658AbjALM5n (ORCPT ); Thu, 12 Jan 2023 07:57:43 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44106 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232322AbjALM5d (ORCPT ); Thu, 12 Jan 2023 07:57:33 -0500 Received: from madras.collabora.co.uk (madras.collabora.co.uk [46.235.227.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0501A5130D; Thu, 12 Jan 2023 04:57:05 -0800 (PST) Received: from localhost (89-26-75-29.goll.dyn.salzburg-online.at [89.26.75.29]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: sebastianfricke) by madras.collabora.co.uk (Postfix) with ESMTPSA id B56856602DB1; Thu, 12 Jan 2023 12:57:03 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=collabora.com; s=mail; t=1673528223; bh=KAJmQ51TusAs/OBMlUcE6FAB/zXgKsXdACD+yS0cJJs=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=abuLMAtI+0Ssmb0Qet0J4hPnVAdraA31EB3fvZ/zDjZvc2UyLQpbuGpOiFbxXhOqc S9NUy9xnqLwAqNNtyQFTUiP7+lJpwbtRYRwTevw/9OIF62BM43kv9v/Y8O5Dan8zJD zy3+0jfREptqFrzuMIuITsGVocKGn8VB9qrbRTUXlbka8E6UkheBghOkMgwWlfjhJ6 bvU2BcEkzGokpvaFOODMzAKJe3+Xa3+EXgkqaaYK3u3a9zVb6+KOCsN5F9v9WFMQIW dAqSQiVNpyzqaypDHC12Df8AFUhQ4HVKlMv25c6RbKfbpJOAj4W627U9Xoa2Xkd1Up n9oHDOyuMFo1Q== From: Sebastian Fricke Date: Thu, 12 Jan 2023 13:56:26 +0100 Subject: [PATCH v2 11/12] staging: media: rkvdec: Enable S_CTRL IOCTL MIME-Version: 1.0 Message-Id: <20230101-patch-series-v2-6-2-rc1-v2-11-fa1897efac14@collabora.com> References: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> In-Reply-To: <20230101-patch-series-v2-6-2-rc1-v2-0-fa1897efac14@collabora.com> To: Mauro Carvalho Chehab , Ezequiel Garcia , Greg Kroah-Hartman Cc: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rockchip@lists.infradead.org, linux-staging@lists.linux.dev, Jonas Karlman , Alex Bee , Nicolas Dufresne , Collabora Kernel-domain , Robert Beckett , Laurent Pinchart , Andrzej Pietrasiewicz , Benjamin Gaignard , Sebastian Fricke X-Mailer: b4 0.11.1 X-Developer-Signature: v=1; a=ed25519-sha256; t=1673528186; l=3704; i=sebastian.fricke@collabora.com; s=linux-media; h=from:subject:message-id; bh=KAJmQ51TusAs/OBMlUcE6FAB/zXgKsXdACD+yS0cJJs=; b=3Q8dYQ7JJV91gBKcZI7SdzmsUpSj8tSF8FmAoOpA4+8doicKKzpHZXOKqfCS3w0RSMTTwbQ545S3 /EmOOuLQCwWk/liG7p+9gfjw2YEBe+DwYIkP0cAdOuppKUob+jYn X-Developer-Key: i=sebastian.fricke@collabora.com; a=ed25519; pk=pYXedPwrTtErcj7ERYeo/IpTrpe4QbJuEzSB52fslBg= Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Enable user-space to set the SPS of the current byte-stream on the decoder. This action will enable the decoder to pick the optimal pixel-format for the capture queue, whenever it is required. Signed-off-by: Sebastian Fricke Signed-off-by: Jonas Karlman --- drivers/staging/media/rkvdec/rkvdec.c | 81 +++++++++++++++++++++++++++++++++++ 1 file changed, 81 insertions(+) diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index b303c6e0286d..3d413c5ad1d2 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -93,6 +93,79 @@ static int rkvdec_get_sps_attributes(struct rkvdec_ctx *ctx, void *sps, return 0; } +static int rkvdec_set_sps(struct rkvdec_ctx *ctx, struct v4l2_ctrl *ctrl) +{ + struct v4l2_pix_format_mplane *pix_mp; + struct sps_attributes attributes = {0}; + void *new_sps = NULL; + + /* + * SPS structures are not filled until the control handler is set up + */ + if (!ctx->fh.ctrl_handler) + return 0; + + switch (ctrl->id) { + case V4L2_CID_STATELESS_H264_SPS: + new_sps = (void *)ctrl->p_new.p_h264_sps; + break; + case V4L2_CID_STATELESS_HEVC_SPS: + new_sps = (void *)ctrl->p_new.p_hevc_sps; + break; + default: + dev_err(ctx->dev->dev, "Unsupported stateless control ID: %x\n", ctrl->id); + return -EINVAL; + }; + rkvdec_get_sps_attributes(ctx, new_sps, &attributes); + + /* + * Providing an empty SPS is valid but we do not store it. + */ + if (attributes.width == 0 && attributes.height == 0) + return 0; + + pix_mp = &ctx->decoded_fmt.fmt.pix_mp; + + /* + * SPS must match the provided format dimension, if it doesn't userspace has to + * first reset the output format + */ + if ((attributes.width > pix_mp->width) || (attributes.height > pix_mp->height)) { + dev_err(ctx->dev->dev, + "Dimension mismatch. [%s SPS] W: %d, H: %d, [Format] W: %d, H: %d)\n", + ctrl->id == V4L2_CID_STATELESS_HEVC_SPS ? "HEVC" : "H264", + attributes.width, attributes.height, pix_mp->width, pix_mp->height); + return -EINVAL; + } + + if (ctx->sps && pix_mp->pixelformat == rkvdec_get_valid_fmt(ctx)) { + /* + * Userspace is allowed to change the SPS at any point, if the + * pixel format doesn't differ from the format in the context, + * just accept the change even if buffers are queued + */ + ctx->sps = new_sps; + } else { + /* + * Do not accept changing the SPS, while buffers are queued, + * when the new SPS would cause switching the CAPTURE pixel format + */ + if (pix_mp->pixelformat != rkvdec_get_valid_fmt(ctx)) { + if (rkvdec_queue_busy(ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE)) + return -EBUSY; + } + ctx->sps = new_sps; + /* + * For the initial SPS setting and when the pixel format is + * changed adjust the pixel format stored in the context + */ + pix_mp->pixelformat = rkvdec_get_valid_fmt(ctx); + rkvdec_fill_decoded_pixfmt(ctx, pix_mp); + } + + return 0; +} + static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) { struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); @@ -104,8 +177,16 @@ static int rkvdec_try_ctrl(struct v4l2_ctrl *ctrl) return 0; } +static int rkvdec_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct rkvdec_ctx *ctx = container_of(ctrl->handler, struct rkvdec_ctx, ctrl_hdl); + + return rkvdec_set_sps(ctx, ctrl); +} + static const struct v4l2_ctrl_ops rkvdec_ctrl_ops = { .try_ctrl = rkvdec_try_ctrl, + .s_ctrl = rkvdec_s_ctrl, }; static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = {