From patchwork Tue Aug 25 03:52:27 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7EFDC433E3 for ; Tue, 25 Aug 2020 03:53:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8C399208E4 for ; Tue, 25 Aug 2020 03:53:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728603AbgHYDxF (ORCPT ); Mon, 24 Aug 2020 23:53:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46288 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbgHYDxE (ORCPT ); Mon, 24 Aug 2020 23:53:04 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 476A0C061574; Mon, 24 Aug 2020 20:53:04 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 64AF6298CD5 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 01/19] media: uapi: h264: Update reference lists Date: Tue, 25 Aug 2020 00:52:27 -0300 Message-Id: <20200825035245.594982-2-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jernej Skrabec When dealing with interlaced frames, reference lists must tell if each particular reference is meant for top or bottom field. This info is currently not provided at all in the H264 related controls. Change reference lists to hold a structure, which specifies an index into the DPB array and the field/frame specification for the picture. Currently the only user of these lists is Cedrus which is just compile fixed here. Actual usage of will come in a following commit. Signed-off-by: Jernej Skrabec Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- v4: * As suggested by Hans, reduce the footprint and solve the holes by converting the enum to a simple char. v3: * Rename to avoid mentioning the DPB. v2: * As pointed out by Jonas, enum v4l2_h264_dpb_reference here. --- .../media/v4l/ext-ctrls-codec.rst | 44 ++++++++++++++++++- .../staging/media/sunxi/cedrus/cedrus_h264.c | 6 +-- include/media/h264-ctrls.h | 21 ++++++--- 3 files changed, 60 insertions(+), 11 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index d0d506a444b1..df1c4fc5cb48 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1843,10 +1843,10 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - __u32 - ``slice_group_change_cycle`` - - * - __u8 + * - struct :c:type:`v4l2_h264_reference` - ``ref_pic_list0[32]`` - Reference picture list after applying the per-slice modifications - * - __u8 + * - struct :c:type:`v4l2_h264_reference` - ``ref_pic_list1[32]`` - Reference picture list after applying the per-slice modifications * - __u32 @@ -1926,6 +1926,46 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - ``chroma_offset[32][2]`` - +``Picture Reference`` + +.. c:type:: v4l2_h264_reference + +.. cssclass:: longtable + +.. flat-table:: struct v4l2_h264_reference + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - __u8 + - ``fields`` + - Specifies how the picture is referenced. See :ref:`Reference Fields ` + * - __u8 + - ``index`` + - Index into the :c:type:`v4l2_ctrl_h264_decode_params`.dpb array. + +.. _h264_ref_fields: + +``Reference Fields`` + +.. cssclass:: longtable + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 1 1 2 + + * - ``V4L2_H264_TOP_FIELD_REF`` + - 0x1 + - The top field in field pair is used for short-term reference. + * - ``V4L2_H264_BOTTOM_FIELD_REF`` + - 0x2 + - The bottom field in field pair is used for short-term reference. + * - ``V4L2_H264_FRAME_REF`` + - 0x3 + - The frame (or the top/bottom fields, if it's a field pair) + is used for short-term reference. + ``V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (struct)`` Specifies the decode parameters (as extracted from the bitstream) for the associated H264 slice data. This includes the necessary diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index 54ee2aa423e2..cce527bbdf86 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -166,8 +166,8 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, struct cedrus_run *run, - const u8 *ref_list, u8 num_ref, - enum cedrus_h264_sram_off sram) + const struct v4l2_h264_reference *ref_list, + u8 num_ref, enum cedrus_h264_sram_off sram) { const struct v4l2_ctrl_h264_decode_params *decode = run->h264.decode_params; struct vb2_queue *cap_q; @@ -188,7 +188,7 @@ static void _cedrus_write_ref_list(struct cedrus_ctx *ctx, int buf_idx; u8 dpb_idx; - dpb_idx = ref_list[i]; + dpb_idx = ref_list[i].index; dpb = &decode->dpb[dpb_idx]; if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 080fd1293c42..3c613b84e5ae 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -19,6 +19,8 @@ */ #define V4L2_H264_NUM_DPB_ENTRIES 16 +#define V4L2_H264_REF_LIST_LEN (2 * V4L2_H264_NUM_DPB_ENTRIES) + /* Our pixel format isn't stable at the moment */ #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ @@ -140,6 +142,17 @@ struct v4l2_h264_pred_weight_table { #define V4L2_H264_SLICE_FLAG_DIRECT_SPATIAL_MV_PRED 0x04 #define V4L2_H264_SLICE_FLAG_SP_FOR_SWITCH 0x08 +#define V4L2_H264_TOP_FIELD_REF 0x1 +#define V4L2_H264_BOTTOM_FIELD_REF 0x2 +#define V4L2_H264_FRAME_REF 0x3 + +struct v4l2_h264_reference { + __u8 fields; + + /* Index into v4l2_ctrl_h264_decode_params.dpb[] */ + __u8 index; +}; + struct v4l2_ctrl_h264_slice_params { /* Size in bytes, including header */ __u32 size; @@ -178,12 +191,8 @@ struct v4l2_ctrl_h264_slice_params { __u8 num_ref_idx_l1_active_minus1; __u32 slice_group_change_cycle; - /* - * Entries on each list are indices into - * v4l2_ctrl_h264_decode_params.dpb[]. - */ - __u8 ref_pic_list0[32]; - __u8 ref_pic_list1[32]; + struct v4l2_h264_reference ref_pic_list0[V4L2_H264_REF_LIST_LEN]; + struct v4l2_h264_reference ref_pic_list1[V4L2_H264_REF_LIST_LEN]; __u32 flags; }; From patchwork Tue Aug 25 03:52:29 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 00B1BC433DF for ; Tue, 25 Aug 2020 03:53:21 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF9B820866 for ; Tue, 25 Aug 2020 03:53:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728657AbgHYDxQ (ORCPT ); Mon, 24 Aug 2020 23:53:16 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbgHYDxQ (ORCPT ); Mon, 24 Aug 2020 23:53:16 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 86255C061574; Mon, 24 Aug 2020 20:53:15 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 5EBC7298CD5 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 03/19] media: uapi: h264: Split prediction weight parameters Date: Tue, 25 Aug 2020 00:52:29 -0300 Message-Id: <20200825035245.594982-4-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The prediction weight parameters are only required under certain conditions, which depend on slice header parameters. As specified in section 7.3.3 Slice header syntax, the prediction weight table is present if: ((weighted_pred_flag && (slice_type == P || slice_type == SP)) || \ (weighted_bipred_idc == 1 && slice_type == B)) Given its size, it makes sense to move this table to its control, so applications can avoid passing it if the slice doesn't specify it. Before this change struct v4l2_ctrl_h264_slice_params was 960 bytes. With this change, it's 188 bytes and struct v4l2_ctrl_h264_pred_weight is 772 bytes. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- v3: * As suggested by Jonas, add a macro for the prediction weight table condition and make the control optional in cedrus. v2: * Fix missing Cedrus changes and mssing control declaration, as noted by Hans and Jernej. --- .../media/v4l/ext-ctrls-codec.rst | 19 ++++++++++++------- drivers/media/v4l2-core/v4l2-ctrls.c | 8 ++++++++ drivers/staging/media/sunxi/cedrus/cedrus.c | 7 +++++++ drivers/staging/media/sunxi/cedrus/cedrus.h | 1 + .../staging/media/sunxi/cedrus/cedrus_dec.c | 2 ++ .../staging/media/sunxi/cedrus/cedrus_h264.c | 12 +++--------- include/media/h264-ctrls.h | 12 ++++++++++-- include/media/v4l2-ctrls.h | 2 ++ 8 files changed, 45 insertions(+), 18 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 2cce412f5be1..9e4421a7c6a6 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1879,18 +1879,23 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - - 0x00000008 - -``Prediction Weight Table`` +``V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS (struct)`` + Prediction weight table defined according to :ref:`h264`, + section 7.4.3.2 "Prediction Weight Table Semantics". + The prediction weight table must be passed by applications + under the conditions explained in section 7.3.3 "Slice header + syntax". - The bitstream parameters are defined according to :ref:`h264`, - section 7.4.3.2 "Prediction Weight Table Semantics". For further - documentation, refer to the above specification, unless there is - an explicit comment stating otherwise. + .. note:: + + This compound control is not yet part of the public kernel API and + it is expected to change. -.. c:type:: v4l2_h264_pred_weight_table +.. c:type:: v4l2_ctrl_h264_pred_weights .. cssclass:: longtable -.. flat-table:: struct v4l2_h264_pred_weight_table +.. flat-table:: struct v4l2_ctrl_h264_pred_weights :header-rows: 0 :stub-columns: 0 :widths: 1 1 2 diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index 3f3fbcd60cc6..76c8dc8fb31c 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -897,6 +897,7 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: return "H264 Decode Parameters"; case V4L2_CID_MPEG_VIDEO_H264_DECODE_MODE: return "H264 Decode Mode"; case V4L2_CID_MPEG_VIDEO_H264_START_CODE: return "H264 Start Code"; + case V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS: return "H264 Prediction Weight Table"; case V4L2_CID_MPEG_VIDEO_MPEG2_LEVEL: return "MPEG2 Level"; case V4L2_CID_MPEG_VIDEO_MPEG2_PROFILE: return "MPEG2 Profile"; case V4L2_CID_MPEG_VIDEO_MPEG4_I_FRAME_QP: return "MPEG4 I-Frame QP Value"; @@ -1412,6 +1413,9 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS: *type = V4L2_CTRL_TYPE_H264_DECODE_PARAMS; break; + case V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS: + *type = V4L2_CTRL_TYPE_H264_PRED_WEIGHTS; + break; case V4L2_CID_MPEG_VIDEO_VP8_FRAME_HEADER: *type = V4L2_CTRL_TYPE_VP8_FRAME_HEADER; break; @@ -1790,6 +1794,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, case V4L2_CTRL_TYPE_H264_SPS: case V4L2_CTRL_TYPE_H264_PPS: case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: + case V4L2_CTRL_TYPE_H264_PRED_WEIGHTS: case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: break; @@ -2553,6 +2558,9 @@ static struct v4l2_ctrl *v4l2_ctrl_new(struct v4l2_ctrl_handler *hdl, case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: elem_size = sizeof(struct v4l2_ctrl_h264_decode_params); break; + case V4L2_CTRL_TYPE_H264_PRED_WEIGHTS: + elem_size = sizeof(struct v4l2_ctrl_h264_pred_weights); + break; case V4L2_CTRL_TYPE_VP8_FRAME_HEADER: elem_size = sizeof(struct v4l2_ctrl_vp8_frame_header); break; diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index bc27f9430eeb..826324faad7e 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -78,6 +78,13 @@ static const struct cedrus_control cedrus_controls[] = { .codec = CEDRUS_CODEC_H264, .required = true, }, + { + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS, + }, + .codec = CEDRUS_CODEC_H264, + .required = false, + }, { .cfg = { .id = V4L2_CID_MPEG_VIDEO_H264_DECODE_MODE, diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.h b/drivers/staging/media/sunxi/cedrus/cedrus.h index 96765555ab8a..93c843ae14bb 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.h +++ b/drivers/staging/media/sunxi/cedrus/cedrus.h @@ -62,6 +62,7 @@ struct cedrus_h264_run { const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; const struct v4l2_ctrl_h264_slice_params *slice_params; const struct v4l2_ctrl_h264_sps *sps; + const struct v4l2_ctrl_h264_pred_weights *pred_weights; }; struct cedrus_mpeg2_run { diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c index 58c48e4fdfe9..6385026d1b6b 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_dec.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_dec.c @@ -57,6 +57,8 @@ void cedrus_device_run(void *priv) V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); run.h264.sps = cedrus_find_control_data(ctx, V4L2_CID_MPEG_VIDEO_H264_SPS); + run.h264.pred_weights = cedrus_find_control_data(ctx, + V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS); break; case V4L2_PIX_FMT_HEVC_SLICE: diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index cce527bbdf86..d5636dbbb622 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -256,10 +256,8 @@ static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx, static void cedrus_write_pred_weight_table(struct cedrus_ctx *ctx, struct cedrus_run *run) { - const struct v4l2_ctrl_h264_slice_params *slice = - run->h264.slice_params; - const struct v4l2_h264_pred_weight_table *pred_weight = - &slice->pred_weight_table; + const struct v4l2_ctrl_h264_pred_weights *pred_weight = + run->h264.pred_weights; struct cedrus_dev *dev = ctx->dev; int i, j, k; @@ -367,11 +365,7 @@ static void cedrus_set_params(struct cedrus_ctx *ctx, cedrus_skip_bits(dev, slice->header_bit_size); - if (((pps->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) && - (slice->slice_type == V4L2_H264_SLICE_TYPE_P || - slice->slice_type == V4L2_H264_SLICE_TYPE_SP)) || - (pps->weighted_bipred_idc == 1 && - slice->slice_type == V4L2_H264_SLICE_TYPE_B)) + if (V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, slice)) cedrus_write_pred_weight_table(ctx, run); if ((slice->slice_type == V4L2_H264_SLICE_TYPE_P) || diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 3c613b84e5ae..31c6f4c5963b 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -36,6 +36,7 @@ #define V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS (V4L2_CID_MPEG_BASE+1004) #define V4L2_CID_MPEG_VIDEO_H264_DECODE_MODE (V4L2_CID_MPEG_BASE+1005) #define V4L2_CID_MPEG_VIDEO_H264_START_CODE (V4L2_CID_MPEG_BASE+1006) +#define V4L2_CID_MPEG_VIDEO_H264_PRED_WEIGHTS (V4L2_CID_MPEG_BASE+1007) /* enum v4l2_ctrl_type type values */ #define V4L2_CTRL_TYPE_H264_SPS 0x0110 @@ -43,6 +44,7 @@ #define V4L2_CTRL_TYPE_H264_SCALING_MATRIX 0x0112 #define V4L2_CTRL_TYPE_H264_SLICE_PARAMS 0x0113 #define V4L2_CTRL_TYPE_H264_DECODE_PARAMS 0x0114 +#define V4L2_CTRL_TYPE_H264_PRED_WEIGHTS 0x0115 enum v4l2_mpeg_video_h264_decode_mode { V4L2_MPEG_VIDEO_H264_DECODE_MODE_SLICE_BASED, @@ -125,7 +127,14 @@ struct v4l2_h264_weight_factors { __s16 chroma_offset[32][2]; }; -struct v4l2_h264_pred_weight_table { +#define V4L2_H264_CTRL_PRED_WEIGHTS_REQUIRED(pps, slice) \ + ((((pps)->flags & V4L2_H264_PPS_FLAG_WEIGHTED_PRED) && \ + ((slice)->slice_type == V4L2_H264_SLICE_TYPE_P || \ + (slice)->slice_type == V4L2_H264_SLICE_TYPE_SP)) || \ + ((pps)->weighted_bipred_idc == 1 && \ + (slice)->slice_type == V4L2_H264_SLICE_TYPE_B)) + +struct v4l2_ctrl_h264_pred_weights { __u16 luma_log2_weight_denom; __u16 chroma_log2_weight_denom; struct v4l2_h264_weight_factors weight_factors[2]; @@ -175,7 +184,6 @@ struct v4l2_ctrl_h264_slice_params { __s32 delta_pic_order_cnt0; __s32 delta_pic_order_cnt1; - struct v4l2_h264_pred_weight_table pred_weight_table; /* Size in bits of dec_ref_pic_marking() syntax element. */ __u32 dec_ref_pic_marking_bit_size; /* Size in bits of pic order count syntax. */ diff --git a/include/media/v4l2-ctrls.h b/include/media/v4l2-ctrls.h index f40e2cbb21d3..cb25f345e9ad 100644 --- a/include/media/v4l2-ctrls.h +++ b/include/media/v4l2-ctrls.h @@ -51,6 +51,7 @@ struct video_device; * @p_h264_scaling_matrix: Pointer to a struct v4l2_ctrl_h264_scaling_matrix. * @p_h264_slice_params: Pointer to a struct v4l2_ctrl_h264_slice_params. * @p_h264_decode_params: Pointer to a struct v4l2_ctrl_h264_decode_params. + * @p_h264_pred_weights: Pointer to a struct v4l2_ctrl_h264_pred_weights. * @p_vp8_frame_header: Pointer to a VP8 frame header structure. * @p_hevc_sps: Pointer to an HEVC sequence parameter set structure. * @p_hevc_pps: Pointer to an HEVC picture parameter set structure. @@ -74,6 +75,7 @@ union v4l2_ctrl_ptr { struct v4l2_ctrl_h264_scaling_matrix *p_h264_scaling_matrix; struct v4l2_ctrl_h264_slice_params *p_h264_slice_params; struct v4l2_ctrl_h264_decode_params *p_h264_decode_params; + struct v4l2_ctrl_h264_pred_weights *p_h264_pred_weights; struct v4l2_ctrl_vp8_frame_header *p_vp8_frame_header; struct v4l2_ctrl_hevc_sps *p_hevc_sps; struct v4l2_ctrl_hevc_pps *p_hevc_pps; From patchwork Tue Aug 25 03:52:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0188DC433E3 for ; Tue, 25 Aug 2020 03:53:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DF6E620866 for ; Tue, 25 Aug 2020 03:53:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728688AbgHYDx2 (ORCPT ); Mon, 24 Aug 2020 23:53:28 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:52948 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbgHYDx1 (ORCPT ); Mon, 24 Aug 2020 23:53:27 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id D2952298CD5 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 05/19] media: uapi: h264: Increase size of 'first_mb_in_slice' field Date: Tue, 25 Aug 2020 00:52:31 -0300 Message-Id: <20200825035245.594982-6-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Slice header syntax element 'first_mb_in_slice' can point to the last macroblock, currently the field can only reference 65536 macroblocks which is insufficient for 8K videos. Although unlikely, a 8192x4320 video (where macroblocks are 16x16), would contain 138240 macroblocks on a frame. As per the H264 specification, 'first_mb_in_slice' can be up to PicSizeInMbs - 1, so increase the size of the field to 32-bits. Note that v4l2_ctrl_h264_slice_params struct will be modified in a follow-up commit, and so we defer its 64-bit padding. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 2 +- include/media/h264-ctrls.h | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 591d3d35c429..a417a1ae50d5 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1774,7 +1774,7 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - __u32 - ``header_bit_size`` - - * - __u16 + * - __u32 - ``first_mb_in_slice`` - * - __u8 diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 31c6f4c5963b..9a6722edf004 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -172,7 +172,8 @@ struct v4l2_ctrl_h264_slice_params { /* Offset in bits to slice_data() from the beginning of this slice. */ __u32 header_bit_size; - __u16 first_mb_in_slice; + __u32 first_mb_in_slice; + __u8 slice_type; __u8 pic_parameter_set_id; __u8 colour_plane_id; From patchwork Tue Aug 25 03:52:33 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256023 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E082FC433E3 for ; Tue, 25 Aug 2020 03:53:43 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C58D120882 for ; Tue, 25 Aug 2020 03:53:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728708AbgHYDxj (ORCPT ); Mon, 24 Aug 2020 23:53:39 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:52988 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728068AbgHYDxi (ORCPT ); Mon, 24 Aug 2020 23:53:38 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id F2E64298D57 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 07/19] media: uapi: h264: Increase size of DPB entry pic_num Date: Tue, 25 Aug 2020 00:52:33 -0300 Message-Id: <20200825035245.594982-8-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org DPB entry PicNum maximum value is 2*MaxFrameNum for interlaced content (field_pic_flag=1). As specified, MaxFrameNum is 2^(log2_max_frame_num_minus4 + 4) and log2_max_frame_num_minus4 is in the range of 0 to 12, which means pic_num should be a 32-bit field. The v4l2_h264_dpb_entry struct needs to be padded to avoid a hole, which might be also useful to allow future uAPI extensions. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- .../userspace-api/media/v4l/ext-ctrls-codec.rst | 9 ++++++--- drivers/media/v4l2-core/v4l2-ctrls.c | 13 +++++++++++++ include/media/h264-ctrls.h | 3 ++- include/media/v4l2-h264.h | 2 +- 4 files changed, 22 insertions(+), 5 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 49febae8fd0f..3808c46718ad 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -2051,15 +2051,18 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - ``timestamp`` field in struct :c:type:`v4l2_buffer`. Use the :c:func:`v4l2_timeval_to_ns()` function to convert the struct :c:type:`timeval` in struct :c:type:`v4l2_buffer` to a __u64. - * - __u16 - - ``frame_num`` + * - __u32 + - ``pic_num`` - * - __u16 - - ``pic_num`` + - ``frame_num`` - * - __u8 - ``fields`` - Specifies how the DPB entry is referenced. See :ref:`Reference Fields ` + * - __u8 + - ``reserved[5]`` + - Applications and drivers must set this to zero. * - __s32 - ``top_field_order_cnt`` - diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index 76c8dc8fb31c..b9457789fa55 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -1725,6 +1725,8 @@ static void std_log(const struct v4l2_ctrl *ctrl) #define zero_padding(s) \ memset(&(s).padding, 0, sizeof((s).padding)) +#define zero_reserved(s) \ + memset(&(s).reserved, 0, sizeof((s).reserved)) /* * Compound controls validation requires setting unused fields/flags to zero @@ -1735,6 +1737,7 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, { struct v4l2_ctrl_mpeg2_slice_params *p_mpeg2_slice_params; struct v4l2_ctrl_vp8_frame_header *p_vp8_frame_header; + struct v4l2_ctrl_h264_decode_params *p_h264_dec_params; struct v4l2_ctrl_hevc_sps *p_hevc_sps; struct v4l2_ctrl_hevc_pps *p_hevc_pps; struct v4l2_ctrl_hevc_slice_params *p_hevc_slice_params; @@ -1796,7 +1799,17 @@ static int std_validate_compound(const struct v4l2_ctrl *ctrl, u32 idx, case V4L2_CTRL_TYPE_H264_SCALING_MATRIX: case V4L2_CTRL_TYPE_H264_PRED_WEIGHTS: case V4L2_CTRL_TYPE_H264_SLICE_PARAMS: + break; + case V4L2_CTRL_TYPE_H264_DECODE_PARAMS: + p_h264_dec_params = p; + + for (i = 0; i < V4L2_H264_NUM_DPB_ENTRIES; i++) { + struct v4l2_h264_dpb_entry *dpb_entry = + &p_h264_dec_params->dpb[i]; + + zero_reserved(*dpb_entry); + } break; case V4L2_CTRL_TYPE_VP8_FRAME_HEADER: diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 0529e75cce5f..91f6f0d43e11 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -213,9 +213,10 @@ struct v4l2_ctrl_h264_slice_params { struct v4l2_h264_dpb_entry { __u64 reference_ts; + __u32 pic_num; __u16 frame_num; - __u16 pic_num; __u8 fields; + __u8 reserved[5]; /* Note that field is indicated by v4l2_buffer.field */ __s32 top_field_order_cnt; __s32 bottom_field_order_cnt; diff --git a/include/media/v4l2-h264.h b/include/media/v4l2-h264.h index bc9ebb560ccf..1a5f26fc2a9a 100644 --- a/include/media/v4l2-h264.h +++ b/include/media/v4l2-h264.h @@ -33,7 +33,7 @@ struct v4l2_h264_reflist_builder { struct { s32 pic_order_count; int frame_num; - u16 pic_num; + u32 pic_num; u16 longterm : 1; } refs[V4L2_H264_NUM_DPB_ENTRIES]; s32 cur_pic_order_count; From patchwork Tue Aug 25 03:52:35 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256022 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49653C433DF for ; Tue, 25 Aug 2020 03:53:55 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 25D1E207FB for ; Tue, 25 Aug 2020 03:53:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728731AbgHYDxv (ORCPT ); Mon, 24 Aug 2020 23:53:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46408 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728730AbgHYDxu (ORCPT ); Mon, 24 Aug 2020 23:53:50 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7CB85C061574; Mon, 24 Aug 2020 20:53:49 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 5DDFA298CBD From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 09/19] media: uapi: h264: Clarify SLICE_BASED mode Date: Tue, 25 Aug 2020 00:52:35 -0300 Message-Id: <20200825035245.594982-10-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Currently, the SLICE_BASED and FRAME_BASED modes documentation is misleading and not matching the intended use-cases. Drop non-required fields SLICE_PARAMS 'start_byte_offset' and DECODE_PARAMS 'num_slices' and clarify the decoding modes in the documentation. On SLICE_BASED mode, a single slice is expected per OUTPUT buffer, and therefore 'start_byte_offset' is not needed (since the offset to the slice is the start of the buffer). This mode requires the use of CAPTURE buffer holding, and so the number of slices shall not be required. On FRAME_BASED mode, the devices are expected to take care of slice parsing. Neither SLICE_PARAMS are required (and shouldn't be exposed by frame-based drivers), nor the number of slices. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- .../media/v4l/ext-ctrls-codec.rst | 39 +++++-------------- include/media/h264-ctrls.h | 4 -- 2 files changed, 10 insertions(+), 33 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index e74a2531ab67..304b6012b358 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1748,9 +1748,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - This compound control is not yet part of the public kernel API and it is expected to change. - This structure is expected to be passed as an array, with one - entry for each slice included in the bitstream buffer. - .. c:type:: v4l2_ctrl_h264_slice_params .. cssclass:: longtable @@ -1760,17 +1757,9 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - :stub-columns: 0 :widths: 1 1 2 - * - __u32 - - ``start_byte_offset`` - Offset (in bytes) from the beginning of the OUTPUT buffer to the start - of the slice. If the slice starts with a start code, then this is the - offset to such start code. When operating in slice-based decoding mode - (see :c:type:`v4l2_mpeg_video_h264_decode_mode`), this field should - be set to 0. When operating in frame-based decoding mode, this field - should be 0 for the first slice. * - __u32 - ``header_bit_size`` - - + - Offset in bits to slice_data() from the beginning of this slice. * - __u32 - ``first_mb_in_slice`` - @@ -1998,12 +1987,6 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - struct :c:type:`v4l2_h264_dpb_entry` - ``dpb[16]`` - - * - __u16 - - ``num_slices`` - - Number of slices needed to decode the current frame/field. When - operating in slice-based decoding mode (see - :c:type:`v4l2_mpeg_video_h264_decode_mode`), this field - should always be set to one. * - __u16 - ``nal_ref_idc`` - NAL reference ID value coming from the NAL Unit header @@ -2121,22 +2104,20 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - ``V4L2_MPEG_VIDEO_H264_DECODE_MODE_SLICE_BASED`` - 0 - Decoding is done at the slice granularity. - In this mode, ``num_slices`` field in struct - :c:type:`v4l2_ctrl_h264_decode_params` should be set to 1, - and ``start_byte_offset`` in struct - :c:type:`v4l2_ctrl_h264_slice_params` should be set to 0. The OUTPUT buffer must contain a single slice. + When this mode is selected, the ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` + control shall be set. When multiple slices compose a frame, + use of ``V4L2_BUF_CAP_SUPPORTS_M2M_HOLD_CAPTURE_BUF`` flag + is required. * - ``V4L2_MPEG_VIDEO_H264_DECODE_MODE_FRAME_BASED`` - 1 - - Decoding is done at the frame granularity. - In this mode, ``num_slices`` field in struct - :c:type:`v4l2_ctrl_h264_decode_params` should be set to the number - of slices in the frame, and ``start_byte_offset`` in struct - :c:type:`v4l2_ctrl_h264_slice_params` should be set accordingly - for each slice. For the first slice, ``start_byte_offset`` should - be zero. + - Decoding is done at the frame granularity, The OUTPUT buffer must contain all slices needed to decode the frame. The OUTPUT buffer must also contain both fields. + This mode will be supported by devices that + parse the slice(s) header(s) in hardware. When this mode is + selected, the ``V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS`` + control shall not be set. ``V4L2_CID_MPEG_VIDEO_H264_START_CODE (enum)`` Specifies the H264 slice start code expected for each slice. diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 77d0ec51ae43..e4cae02a765f 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -163,9 +163,6 @@ struct v4l2_h264_reference { }; struct v4l2_ctrl_h264_slice_params { - /* Offset in bytes to the start of slice in the OUTPUT buffer. */ - __u32 start_byte_offset; - /* Offset in bits to slice_data() from the beginning of this slice. */ __u32 header_bit_size; @@ -224,7 +221,6 @@ struct v4l2_h264_dpb_entry { struct v4l2_ctrl_h264_decode_params { struct v4l2_h264_dpb_entry dpb[V4L2_H264_NUM_DPB_ENTRIES]; - __u16 num_slices; __u16 nal_ref_idc; __s32 top_field_order_cnt; __s32 bottom_field_order_cnt; From patchwork Tue Aug 25 03:52:37 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256021 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C320C433E1 for ; Tue, 25 Aug 2020 03:54:07 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E766C20882 for ; Tue, 25 Aug 2020 03:54:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728758AbgHYDyC (ORCPT ); Mon, 24 Aug 2020 23:54:02 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46436 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728577AbgHYDyB (ORCPT ); Mon, 24 Aug 2020 23:54:01 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E189AC061574; Mon, 24 Aug 2020 20:54:00 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id B837C298CBD From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 11/19] media: uapi: h264: Rename and clarify PPS_FLAG_SCALING_MATRIX_PRESENT Date: Tue, 25 Aug 2020 00:52:37 -0300 Message-Id: <20200825035245.594982-12-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Applications are expected to fill V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX if a non-flat scaling matrix applies to the picture. This is the case if SPS scaling_matrix_present_flag or PPS pic_scaling_matrix_present_flag are set, and should be handled by applications. On one hand, the PPS bitstream syntax element signals the presence of a Picture scaling matrix modifying the Sequence (SPS) scaling matrix. On the other hand, our flag should indicate if the scaling matrix V4L2 control is applicable to this request. Rename the flag from PPS_FLAG_PIC_SCALING_MATRIX_PRESENT to PPS_FLAG_SCALING_MATRIX_PRESENT, to avoid mixing this flag with bitstream syntax element pic_scaling_matrix_present_flag, and clarify the meaning of our flag. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst | 5 +++-- include/media/h264-ctrls.h | 2 +- 2 files changed, 4 insertions(+), 3 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst index 4cd9fd9c5a89..e2b94b1d0ab0 100644 --- a/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst +++ b/Documentation/userspace-api/media/v4l/ext-ctrls-codec.rst @@ -1695,9 +1695,10 @@ enum v4l2_mpeg_video_h264_hierarchical_coding_type - * - ``V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE`` - 0x00000040 - - * - ``V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT`` + * - ``V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT`` - 0x00000080 - - + - Indicates that ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX`` + must be used for this picture. ``V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX (struct)`` Specifies the scaling matrix (as extracted from the bitstream) for diff --git a/include/media/h264-ctrls.h b/include/media/h264-ctrls.h index 0c15ff938873..ec4799154438 100644 --- a/include/media/h264-ctrls.h +++ b/include/media/h264-ctrls.h @@ -99,7 +99,7 @@ struct v4l2_ctrl_h264_sps { #define V4L2_H264_PPS_FLAG_CONSTRAINED_INTRA_PRED 0x0010 #define V4L2_H264_PPS_FLAG_REDUNDANT_PIC_CNT_PRESENT 0x0020 #define V4L2_H264_PPS_FLAG_TRANSFORM_8X8_MODE 0x0040 -#define V4L2_H264_PPS_FLAG_PIC_SCALING_MATRIX_PRESENT 0x0080 +#define V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT 0x0080 struct v4l2_ctrl_h264_pps { __u8 pic_parameter_set_id; From patchwork Tue Aug 25 03:52:39 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256020 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B02ADC433DF for ; Tue, 25 Aug 2020 03:54:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 940F9208E4 for ; Tue, 25 Aug 2020 03:54:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728788AbgHYDyN (ORCPT ); Mon, 24 Aug 2020 23:54:13 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:53080 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728577AbgHYDyM (ORCPT ); Mon, 24 Aug 2020 23:54:12 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 04892298D7E From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 13/19] media: rkvdec: Don't require unneeded H264_SLICE_PARAMS Date: Tue, 25 Aug 2020 00:52:39 -0300 Message-Id: <20200825035245.594982-14-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Now that slice invariant parameters have been moved, the driver no longer needs this control, so drop it. Signed-off-by: Ezequiel Garcia Reviewed-by: Jonas Karlman Tested-by: Jonas Karlman --- v2: * Fix wrongly removed SPS. --- drivers/staging/media/rkvdec/rkvdec-h264.c | 4 ---- drivers/staging/media/rkvdec/rkvdec.c | 5 ----- 2 files changed, 9 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index 90f211839be2..c45cd1617b3b 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -109,7 +109,6 @@ struct rkvdec_h264_reflists { struct rkvdec_h264_run { struct rkvdec_run base; const struct v4l2_ctrl_h264_decode_params *decode_params; - const struct v4l2_ctrl_h264_slice_params *slices_params; const struct v4l2_ctrl_h264_sps *sps; const struct v4l2_ctrl_h264_pps *pps; const struct v4l2_ctrl_h264_scaling_matrix *scaling_matrix; @@ -1066,9 +1065,6 @@ static void rkvdec_h264_run_preamble(struct rkvdec_ctx *ctx, ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS); run->decode_params = ctrl ? ctrl->p_cur.p : NULL; - ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, - V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS); - run->slices_params = ctrl ? ctrl->p_cur.p : NULL; ctrl = v4l2_ctrl_find(&ctx->ctrl_hdl, V4L2_CID_MPEG_VIDEO_H264_SPS); run->sps = ctrl ? ctrl->p_cur.p : NULL; diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index c8151328fb70..7c5129593921 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -59,11 +59,6 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = { .mandatory = true, .cfg.id = V4L2_CID_MPEG_VIDEO_H264_DECODE_PARAMS, }, - { - .per_request = true, - .mandatory = true, - .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SLICE_PARAMS, - }, { .per_request = true, .mandatory = true, From patchwork Tue Aug 25 03:52:41 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256019 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 30A47C433E1 for ; Tue, 25 Aug 2020 03:54:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1A882207FB for ; Tue, 25 Aug 2020 03:54:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728804AbgHYDyY (ORCPT ); Mon, 24 Aug 2020 23:54:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728197AbgHYDyX (ORCPT ); Mon, 24 Aug 2020 23:54:23 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5630BC061574; Mon, 24 Aug 2020 20:54:23 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id BBCD9298D7E From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec Subject: [PATCH v4 15/19] media: cedrus: h264: Fix frame list construction Date: Tue, 25 Aug 2020 00:52:41 -0300 Message-Id: <20200825035245.594982-16-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Jernej Skrabec Current frame list construction algorithm assumes that decoded image will be output into its own buffer. That is true for progressive content but not for interlaced where each field is decoded separately into same buffer. Fix that by checking if capture buffer is listed in DPB. If it is, reuse it. Signed-off-by: Jernej Skrabec --- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index 1e89a8438f36..fe041b444385 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -101,7 +101,7 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, struct cedrus_dev *dev = ctx->dev; unsigned long used_dpbs = 0; unsigned int position; - unsigned int output = 0; + int output = -1; unsigned int i; cap_q = v4l2_m2m_get_vq(ctx->fh.m2m_ctx, V4L2_BUF_TYPE_VIDEO_CAPTURE); @@ -124,6 +124,11 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, position = cedrus_buf->codec.h264.position; used_dpbs |= BIT(position); + if (run->dst->vb2_buf.timestamp == dpb->reference_ts) { + output = position; + continue; + } + if (!(dpb->flags & V4L2_H264_DPB_ENTRY_FLAG_ACTIVE)) continue; @@ -131,13 +136,11 @@ static void cedrus_write_frame_list(struct cedrus_ctx *ctx, dpb->top_field_order_cnt, dpb->bottom_field_order_cnt, &pic_list[position]); - - output = max(position, output); } - position = find_next_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM, - output); - if (position >= CEDRUS_H264_FRAME_NUM) + if (output >= 0) + position = output; + else position = find_first_zero_bit(&used_dpbs, CEDRUS_H264_FRAME_NUM); output_buf = vb2_to_cedrus_buffer(&run->dst->vb2_buf); From patchwork Tue Aug 25 03:52:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256018 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1514C433DF for ; Tue, 25 Aug 2020 03:54:41 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C307A207FB for ; Tue, 25 Aug 2020 03:54:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728822AbgHYDyg (ORCPT ); Mon, 24 Aug 2020 23:54:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46530 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728197AbgHYDyf (ORCPT ); Mon, 24 Aug 2020 23:54:35 -0400 Received: from bhuna.collabora.co.uk (bhuna.collabora.co.uk [IPv6:2a00:1098:0:82:1000:25:2eeb:e3e3]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A4C5C061574; Mon, 24 Aug 2020 20:54:34 -0700 (PDT) Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id 20E6A298CD5 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 17/19] media: rkvdec: Use H264_SCALING_MATRIX only when required Date: Tue, 25 Aug 2020 00:52:43 -0300 Message-Id: <20200825035245.594982-18-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Baseline, Main and Extended profiles are specified to not support a scaling matrix. Also, High profiles can optionally specify a scaling matrix, using SPS and PPS NAL units. To meet this expectation, applications are required to set the V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX control and set the V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT flag only when a scaling matrix is specified for a picture. Implement this on rkvdec, which has hardware support for this case. Signed-off-by: Ezequiel Garcia Tested-by: Jonas Karlman --- drivers/staging/media/rkvdec/rkvdec-h264.c | 10 +++++++--- drivers/staging/media/rkvdec/rkvdec.c | 1 - 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/staging/media/rkvdec/rkvdec-h264.c b/drivers/staging/media/rkvdec/rkvdec-h264.c index c45cd1617b3b..7cc3b478a5f4 100644 --- a/drivers/staging/media/rkvdec/rkvdec-h264.c +++ b/drivers/staging/media/rkvdec/rkvdec-h264.c @@ -708,9 +708,9 @@ static void assemble_hw_pps(struct rkvdec_ctx *ctx, WRITE_PPS(pps->second_chroma_qp_index_offset, SECOND_CHROMA_QP_INDEX_OFFSET); - /* always use the matrix sent from userspace */ - WRITE_PPS(1, SCALING_LIST_ENABLE_FLAG); - + WRITE_PPS(!!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT), + SCALING_LIST_ENABLE_FLAG); + /* To be on the safe side, program the scaling matrix address */ scaling_distance = offsetof(struct rkvdec_h264_priv_tbl, scaling_list); scaling_list_address = h264_ctx->priv_tbl.dma + scaling_distance; WRITE_PPS(scaling_list_address, SCALING_LIST_ADDRESS); @@ -792,9 +792,13 @@ static void assemble_hw_scaling_list(struct rkvdec_ctx *ctx, struct rkvdec_h264_run *run) { const struct v4l2_ctrl_h264_scaling_matrix *scaling = run->scaling_matrix; + const struct v4l2_ctrl_h264_pps *pps = run->pps; struct rkvdec_h264_ctx *h264_ctx = ctx->priv; struct rkvdec_h264_priv_tbl *tbl = h264_ctx->priv_tbl.cpu; + if (!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT)) + return; + BUILD_BUG_ON(sizeof(tbl->scaling_list.scaling_list_4x4) != sizeof(scaling->scaling_list_4x4)); BUILD_BUG_ON(sizeof(tbl->scaling_list.scaling_list_8x8) != diff --git a/drivers/staging/media/rkvdec/rkvdec.c b/drivers/staging/media/rkvdec/rkvdec.c index 9f59dfb62d3f..d25c4a37e2af 100644 --- a/drivers/staging/media/rkvdec/rkvdec.c +++ b/drivers/staging/media/rkvdec/rkvdec.c @@ -68,7 +68,6 @@ static const struct rkvdec_ctrl_desc rkvdec_h264_ctrl_descs[] = { .cfg.id = V4L2_CID_MPEG_VIDEO_H264_PPS, }, { - .mandatory = true, .cfg.id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, }, { From patchwork Tue Aug 25 03:52:45 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ezequiel Garcia X-Patchwork-Id: 256017 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A229C433DF for ; Tue, 25 Aug 2020 03:54:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 366E32087D for ; Tue, 25 Aug 2020 03:54:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728836AbgHYDyt (ORCPT ); Mon, 24 Aug 2020 23:54:49 -0400 Received: from bhuna.collabora.co.uk ([46.235.227.227]:53186 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727935AbgHYDys (ORCPT ); Mon, 24 Aug 2020 23:54:48 -0400 Received: from [127.0.0.1] (localhost [127.0.0.1]) (Authenticated sender: ezequiel) with ESMTPSA id C560F298D44 From: Ezequiel Garcia To: linux-media@vger.kernel.org, linux-kernel@vger.kernel.org Cc: Tomasz Figa , kernel@collabora.com, Jonas Karlman , Hans Verkuil , Alexandre Courbot , Jeffrey Kardatzke , Nicolas Dufresne , Philipp Zabel , Maxime Ripard , Paul Kocialkowski , Jernej Skrabec , Ezequiel Garcia Subject: [PATCH v4 19/19] media: cedrus: Use H264_SCALING_MATRIX only when required Date: Tue, 25 Aug 2020 00:52:45 -0300 Message-Id: <20200825035245.594982-20-ezequiel@collabora.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20200825035245.594982-1-ezequiel@collabora.com> References: <20200825035245.594982-1-ezequiel@collabora.com> MIME-Version: 1.0 Sender: linux-media-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Baseline, Main and Extended profiles are specified to not support a scaling matrix. Also, High profiles can optionally specify a scaling matrix, using SPS and PPS NAL units. To meet this expectation, applications are required to set the V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX control and set the V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT flag only when a scaling matrix is specified for a picture. Implement this on cedrus, which has hardware support for this case. Signed-off-by: Ezequiel Garcia --- drivers/staging/media/sunxi/cedrus/cedrus.c | 2 +- drivers/staging/media/sunxi/cedrus/cedrus_h264.c | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/staging/media/sunxi/cedrus/cedrus.c b/drivers/staging/media/sunxi/cedrus/cedrus.c index 826324faad7e..6ebb39e0c0ce 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus.c @@ -76,7 +76,7 @@ static const struct cedrus_control cedrus_controls[] = { .id = V4L2_CID_MPEG_VIDEO_H264_SCALING_MATRIX, }, .codec = CEDRUS_CODEC_H264, - .required = true, + .required = false, }, { .cfg = { diff --git a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c index fe041b444385..28319351e909 100644 --- a/drivers/staging/media/sunxi/cedrus/cedrus_h264.c +++ b/drivers/staging/media/sunxi/cedrus/cedrus_h264.c @@ -238,8 +238,12 @@ static void cedrus_write_scaling_lists(struct cedrus_ctx *ctx, { const struct v4l2_ctrl_h264_scaling_matrix *scaling = run->h264.scaling_matrix; + const struct v4l2_ctrl_h264_pps *pps = run->h264.pps; struct cedrus_dev *dev = ctx->dev; + if (!(pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT)) + return; + cedrus_h264_write_sram(dev, CEDRUS_SRAM_H264_SCALING_LIST_8x8_0, scaling->scaling_list_8x8[0], sizeof(scaling->scaling_list_8x8[0])); @@ -442,6 +446,8 @@ static void cedrus_set_params(struct cedrus_ctx *ctx, reg |= (pps->second_chroma_qp_index_offset & 0x3f) << 16; reg |= (pps->chroma_qp_index_offset & 0x3f) << 8; reg |= (pps->pic_init_qp_minus26 + 26 + slice->slice_qp_delta) & 0x3f; + if (pps->flags & V4L2_H264_PPS_FLAG_SCALING_MATRIX_PRESENT) + reg |= VE_H264_SHS_QP_SCALING_MATRIX_DEFAULT; cedrus_write(dev, VE_H264_SHS_QP, reg); // clear status flags