| Message ID | 20250507-video-iris-hevc-vp9-v4-0-58db3660ac61@quicinc.com |
|---|---|
| Headers | show |
| Series | Add support for HEVC and VP9 codecs in decoder | expand |
On 07/05/2025 09:39, Dikshita Agarwal wrote: > Hi All, > > This patch series adds initial support for the HEVC(H.265) and VP9 > codecs in iris decoder. The objective of this work is to extend the > decoder's capabilities to handle HEVC and VP9 codec streams, > including necessary format handling and buffer management. > In addition, the series also includes a set of fixes to address issues > identified during testing of these additional codecs. > > These patches also address the comments and feedback received from the > RFC patches previously sent. I have made the necessary improvements > based on the community's suggestions. > > Changes in v4: > - Splitted patch patch 06/23 in two patches (Bryan) > - Simplified the conditional logic in patch 13/23 (Bryan) > - Improved commit description for patch patch 13/23 (Nicolas) > - Fix the value of H265_NUM_TILE_ROW macro (Neil) > - Link to v3: https://lore.kernel.org/r/20250502-qcom-iris-hevc-vp9-v3-0-552158a10a7d@quicinc.com > > Changes in v3: > - Introduced two wrappers with explicit names to handle destroy internal > buffers (Nicolas) > - Used sub state check instead of introducing new boolean (Vikash) > - Addressed other comments (Vikash) > - Reorderd patches to have all fixes patches first (Dmitry) > - Link to v2: > https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com > > Changes in v2: > - Added Changes to make sure all buffers are released in session close > (bryna) > - Added tracking for flush responses to fix a timing issue. > - Added a handling to fix timing issue in reconfig > - Splitted patch 06/20 in two patches (Bryan) > - Added missing fixes tag (bryan) > - Updated fluster report (Nicolas) > - Link to v1: > https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com > > Changes sinces RFC: > - Added additional fixes to address issues identified during further > testing. > - Moved typo fix to a seperate patch [Neil] > - Reordered the patches for better logical flow and clarity [Neil, > Dmitry] > - Added fixes tag wherever applicable [Neil, Dmitry] > - Removed the default case in the switch statement for codecs [Bryan] > - Replaced if-else statements with switch-case [Bryan] > - Added comments for mbpf [Bryan] > - RFC: > https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/ > > This patch series depends on [1] & [2] > [1] > https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/ > [2] > https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/ > > These patches are tested on SM8250 and SM8550 with v4l2-ctl and > Gstreamer for HEVC and VP9 decoders, at the same time ensured that > the existing H264 decoder functionality remains uneffected. > > Note: 1 of the fluster compliance test is fixed with firmware [3] > [3]: > https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u > > The result of fluster test on SM8550: > 131/147 testcases passed while testing JCT-VC-HEVC_V1 with > GStreamer-H.265-V4L2-Gst1.0. > The failing test case: > - 10 testcases failed due to unsupported 10 bit format. > - DBLK_A_MAIN10_VIXS_4 > - INITQP_B_Main10_Sony_1 > - TSUNEQBD_A_MAIN10_Technicolor_2 > - WP_A_MAIN10_Toshiba_3 > - WP_MAIN10_B_Toshiba_3 > - WPP_A_ericsson_MAIN10_2 > - WPP_B_ericsson_MAIN10_2 > - WPP_C_ericsson_MAIN10_2 > - WPP_E_ericsson_MAIN10_2 > - WPP_F_ericsson_MAIN10_2 > - 4 testcase failed due to unsupported resolution > - PICSIZE_A_Bossen_1 > - PICSIZE_B_Bossen_1 > - WPP_D_ericsson_MAIN10_2 > - WPP_D_ericsson_MAIN_2 > - 2 testcase failed due to CRC mismatch > - RAP_A_docomo_6 > - RAP_B_Bossen_2 > - BUG reported: > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 > Analysis - First few frames in this discarded by firmware and are > sent to driver with 0 filled length. Driver send such buffers to > client with timestamp 0 and payload set to 0 and > make buf state to VB2_BUF_STATE_ERROR. Such buffers should be > dropped by GST. But instead, the first frame displayed as green > frame and when a valid buffer is sent to client later with same 0 > timestamp, its dropped, leading to CRC mismatch for first frame. > > 235/305 testcases passed while testing VP9-TEST-VECTORS with > GStreamer-VP9-V4L2-Gst1.0. > The failing test case: > - 64 testcases failed due to unsupported resolution > - vp90-2-02-size-08x08.webm > - vp90-2-02-size-08x10.webm > - vp90-2-02-size-08x16.webm > - vp90-2-02-size-08x18.webm > - vp90-2-02-size-08x32.webm > - vp90-2-02-size-08x34.webm > - vp90-2-02-size-08x64.webm > - vp90-2-02-size-08x66.webm > - vp90-2-02-size-10x08.webm > - vp90-2-02-size-10x10.webm > - vp90-2-02-size-10x16.webm > - vp90-2-02-size-10x18.webm > - vp90-2-02-size-10x32.webm > - vp90-2-02-size-10x34.webm > - vp90-2-02-size-10x64.webm > - vp90-2-02-size-10x66.webm > - vp90-2-02-size-16x08.webm > - vp90-2-02-size-16x10.webm > - vp90-2-02-size-16x16.webm > - vp90-2-02-size-16x18.webm > - vp90-2-02-size-16x32.webm > - vp90-2-02-size-16x34.webm > - vp90-2-02-size-16x64.webm > - vp90-2-02-size-16x66.webm > - vp90-2-02-size-18x08.webm > - vp90-2-02-size-18x10.webm > - vp90-2-02-size-18x16.webm > - vp90-2-02-size-18x18.webm > - vp90-2-02-size-18x32.webm > - vp90-2-02-size-18x34.webm > - vp90-2-02-size-18x64.webm > - vp90-2-02-size-18x66.webm > - vp90-2-02-size-32x08.webm > - vp90-2-02-size-32x10.webm > - vp90-2-02-size-32x16.webm > - vp90-2-02-size-32x18.webm > - vp90-2-02-size-32x32.webm > - vp90-2-02-size-32x34.webm > - vp90-2-02-size-32x64.webm > - vp90-2-02-size-32x66.webm > - vp90-2-02-size-34x08.webm > - vp90-2-02-size-34x10.webm > - vp90-2-02-size-34x16.webm > - vp90-2-02-size-34x18.webm > - vp90-2-02-size-34x32.webm > - vp90-2-02-size-34x34.webm > - vp90-2-02-size-34x64.webm > - vp90-2-02-size-34x66.webm > - vp90-2-02-size-64x08.webm > - vp90-2-02-size-64x10.webm > - vp90-2-02-size-64x16.webm > - vp90-2-02-size-64x18.webm > - vp90-2-02-size-64x32.webm > - vp90-2-02-size-64x34.webm > - vp90-2-02-size-64x64.webm > - vp90-2-02-size-64x66.webm > - vp90-2-02-size-66x08.webm > - vp90-2-02-size-66x10.webm > - vp90-2-02-size-66x16.webm > - vp90-2-02-size-66x18.webm > - vp90-2-02-size-66x32.webm > - vp90-2-02-size-66x34.webm > - vp90-2-02-size-66x64.webm > - vp90-2-02-size-66x66.webm > - 2 testcases failed due to unsupported format > - vp91-2-04-yuv422.webm > - vp91-2-04-yuv444.webm > - 1 testcase failed with CRC mismatch > - vp90-2-22-svc_1280x720_3.ivf > - Bug reported: > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 > - 2 testcase failed due to unsupported resolution after sequence change > - vp90-2-21-resize_inter_320x180_5_1-2.webm > - vp90-2-21-resize_inter_320x180_7_1-2.webm > - 1 testcase failed due to unsupported stream > - vp90-2-16-intra-only.webm > > The result of fluster test on SM8250: > 133/147 testcases passed while testing JCT-VC-HEVC_V1 with > GStreamer-H.265-V4L2-Gst1.0. > The failing test case: > - 10 testcases failed due to unsupported 10 bit format. > - DBLK_A_MAIN10_VIXS_4 > - INITQP_B_Main10_Sony_1 > - TSUNEQBD_A_MAIN10_Technicolor_2 > - WP_A_MAIN10_Toshiba_3 > - WP_MAIN10_B_Toshiba_3 > - WPP_A_ericsson_MAIN10_2 > - WPP_B_ericsson_MAIN10_2 > - WPP_C_ericsson_MAIN10_2 > - WPP_E_ericsson_MAIN10_2 > - WPP_F_ericsson_MAIN10_2 > - 4 testcase failed due to unsupported resolution > - PICSIZE_A_Bossen_1 > - PICSIZE_B_Bossen_1 > - WPP_D_ericsson_MAIN10_2 > - WPP_D_ericsson_MAIN_2 > > 232/305 testcases passed while testing VP9-TEST-VECTORS with > GStreamer-VP9-V4L2-Gst1.0. > The failing test case: > - 64 testcases failed due to unsupported resolution > - vp90-2-02-size-08x08.webm > - vp90-2-02-size-08x10.webm > - vp90-2-02-size-08x16.webm > - vp90-2-02-size-08x18.webm > - vp90-2-02-size-08x32.webm > - vp90-2-02-size-08x34.webm > - vp90-2-02-size-08x64.webm > - vp90-2-02-size-08x66.webm > - vp90-2-02-size-10x08.webm > - vp90-2-02-size-10x10.webm > - vp90-2-02-size-10x16.webm > - vp90-2-02-size-10x18.webm > - vp90-2-02-size-10x32.webm > - vp90-2-02-size-10x34.webm > - vp90-2-02-size-10x64.webm > - vp90-2-02-size-10x66.webm > - vp90-2-02-size-16x08.webm > - vp90-2-02-size-16x10.webm > - vp90-2-02-size-16x16.webm > - vp90-2-02-size-16x18.webm > - vp90-2-02-size-16x32.webm > - vp90-2-02-size-16x34.webm > - vp90-2-02-size-16x64.webm > - vp90-2-02-size-16x66.webm > - vp90-2-02-size-18x08.webm > - vp90-2-02-size-18x10.webm > - vp90-2-02-size-18x16.webm > - vp90-2-02-size-18x18.webm > - vp90-2-02-size-18x32.webm > - vp90-2-02-size-18x34.webm > - vp90-2-02-size-18x64.webm > - vp90-2-02-size-18x66.webm > - vp90-2-02-size-32x08.webm > - vp90-2-02-size-32x10.webm > - vp90-2-02-size-32x16.webm > - vp90-2-02-size-32x18.webm > - vp90-2-02-size-32x32.webm > - vp90-2-02-size-32x34.webm > - vp90-2-02-size-32x64.webm > - vp90-2-02-size-32x66.webm > - vp90-2-02-size-34x08.webm > - vp90-2-02-size-34x10.webm > - vp90-2-02-size-34x16.webm > - vp90-2-02-size-34x18.webm > - vp90-2-02-size-34x32.webm > - vp90-2-02-size-34x34.webm > - vp90-2-02-size-34x64.webm > - vp90-2-02-size-34x66.webm > - vp90-2-02-size-64x08.webm > - vp90-2-02-size-64x10.webm > - vp90-2-02-size-64x16.webm > - vp90-2-02-size-64x18.webm > - vp90-2-02-size-64x32.webm > - vp90-2-02-size-64x34.webm > - vp90-2-02-size-64x64.webm > - vp90-2-02-size-64x66.webm > - vp90-2-02-size-66x08.webm > - vp90-2-02-size-66x10.webm > - vp90-2-02-size-66x16.webm > - vp90-2-02-size-66x18.webm > - vp90-2-02-size-66x32.webm > - vp90-2-02-size-66x34.webm > - vp90-2-02-size-66x64.webm > - vp90-2-02-size-66x66.webm > - 2 testcases failed due to unsupported format > - vp91-2-04-yuv422.webm > - vp91-2-04-yuv444.webm > - 1 testcase failed with CRC mismatch > - vp90-2-22-svc_1280x720_3.ivf > - Bug raised: > https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 > - 5 testcase failed due to unsupported resolution after sequence change > - vp90-2-21-resize_inter_320x180_5_1-2.webm > - vp90-2-21-resize_inter_320x180_7_1-2.webm > - vp90-2-21-resize_inter_320x240_5_1-2.webm > - vp90-2-21-resize_inter_320x240_7_1-2.webm > - vp90-2-18-resize.ivf > - 1 testcase failed with CRC mismatch > - vp90-2-16-intra-only.webm > Analysis: First few frames are marked by firmware as NO_SHOW frame. > Driver make buf state to VB2_BUF_STATE_ERROR for such frames. > Such buffers should be dropped by GST. But instead, the first frame > is being displayed and when a valid buffer is sent to client later > with same timestamp, its dropped, leading to CRC mismatch for first > frame. > > To: Vikash Garodia <quic_vgarodia@quicinc.com> > To: Abhinav Kumar <quic_abhinavk@quicinc.com> > To: Bryan O'Donoghue <bryan.odonoghue@linaro.org> > To: Mauro Carvalho Chehab <mchehab@kernel.org> > To: Hans Verkuil <hverkuil@xs4all.nl> > To: Stefan Schmidt <stefan.schmidt@linaro.org> > Cc: linux-media@vger.kernel.org > Cc: linux-arm-msm@vger.kernel.org > Cc: linux-kernel@vger.kernel.org > Cc: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> > Cc: Neil Armstrong <neil.armstrong@linaro.org> > Cc: Nicolas Dufresne <nicolas.dufresne@collabora.com> > Cc: Dan Carpenter <dan.carpenter@linaro.org> > > Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com> > --- > Dikshita Agarwal (25): > media: iris: Skip destroying internal buffer if not dequeued > media: iris: Update CAPTURE format info based on OUTPUT format > media: iris: Avoid updating frame size to firmware during reconfig > media: iris: Drop port check for session property response > media: iris: Prevent HFI queue writes when core is in deinit state > media: iris: Remove error check for non-zero v4l2 controls > media: iris: Remove deprecated property setting to firmware > media: iris: Fix missing function pointer initialization > media: iris: Fix NULL pointer dereference > media: iris: Fix typo in depth variable > media: iris: Track flush responses to prevent premature completion > media: iris: Fix buffer preparation failure during resolution change > media: iris: Send V4L2_BUF_FLAG_ERROR for capture buffers with 0 filled length > media: iris: Skip flush on first sequence change > media: iris: Remove unnecessary re-initialization of flush completion > media: iris: Add handling for corrupt and drop frames > media: iris: Add handling for no show frames > media: iris: Improve last flag handling > media: iris: Remove redundant buffer count check in stream off > media: iris: Add a comment to explain usage of MBPS > media: iris: Add HEVC and VP9 formats for decoder > media: iris: Add platform capabilities for HEVC and VP9 decoders > media: iris: Set mandatory properties for HEVC and VP9 decoders. > media: iris: Add internal buffer calculation for HEVC and VP9 decoders > media: iris: Add codec specific check for VP9 decoder drain handling > > drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- > drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- > drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- > drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + > .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- > .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- > .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- > .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- > .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + > .../platform/qcom/iris/iris_hfi_gen2_response.c | 56 ++- > drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- > drivers/media/platform/qcom/iris/iris_instance.h | 6 + > .../platform/qcom/iris/iris_platform_common.h | 28 +- > .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- > .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- > .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- > drivers/media/platform/qcom/iris/iris_state.c | 2 +- > drivers/media/platform/qcom/iris/iris_state.h | 1 + > drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- > drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- > drivers/media/platform/qcom/iris/iris_vdec.h | 11 + > drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- > drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- > drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- > 24 files changed, 1159 insertions(+), 211 deletions(-) > --- > base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa > change-id: 20250507-video-iris-hevc-vp9-59096b189050 > prerequisite-message-id: <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org> > prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 > prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 > prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e > prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add > prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 > prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 > prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e > prerequisite-message-id: <20250501-qcs8300_iris-v7-0-b229d5347990@quicinc.com> > prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea > prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 > prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 > prerequisite-patch-id: 500bc3b8391940d3ebca222d2098b737414b2af4 > prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f > > Best regards, Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-QRD Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8550-HDK Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-QRD Tested-by: Neil Armstrong <neil.armstrong@linaro.org> # on SM8650-HDK Thanks, Neil
On 07/05/2025 08:39, Dikshita Agarwal wrote: > During reconfig, the firmware sends the resolution aligned to 8 bytes. > If the driver sends the same resolution back to the firmware the resolution > will be aligned to 16 bytes not 8. > > The alignment mismatch would then subsequently cause the firmware to > send another redundant sequence change event. > > Fix this by not setting the resolution property during reconfig. > > Cc: stable@vger.kernel.org > Fixes: 3a19d7b9e08b ("media: iris: implement set properties to firmware during streamon") > Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com> Reviewed-by: Bryan O'Donoghue <bryan.odonoghue@linaro.org>
On 5/7/2025 8:37 PM, Bryan O'Donoghue wrote: > On 07/05/2025 08:39, Dikshita Agarwal wrote: >> Firmware might hold the DPB buffers for reference in case of sequence >> change, so skip destroying buffers for which QUEUED flag is not removed. >> Also, make sure that all buffers are released during streamoff. >> >> Cc: stable@vger.kernel.org >> Fixes: 73702f45db81 ("media: iris: allocate, initialize and queue >> internal buffers") >> Reviewed-by: Vikash Garodia <quic_vgarodia@quicinc.com> >> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com> > > I'll take your word for it on firmware respecting the software contract on > close() wrt to DMA - however I think you should split this patch into two > separate patches along the lines of the "also" in your commit log. > > 1. Skip destroying buffers for QUEUED flag > 2. Make sure all buffers are released during stream off > > These are two separate fixes IMO. Sure will split. Please let me know if you have any other comments if no comments, will send v5 with addressing above. Thanks, Dikshita > > --- > bod
Hi All, This patch series adds initial support for the HEVC(H.265) and VP9 codecs in iris decoder. The objective of this work is to extend the decoder's capabilities to handle HEVC and VP9 codec streams, including necessary format handling and buffer management. In addition, the series also includes a set of fixes to address issues identified during testing of these additional codecs. These patches also address the comments and feedback received from the RFC patches previously sent. I have made the necessary improvements based on the community's suggestions. Changes in v4: - Splitted patch patch 06/23 in two patches (Bryan) - Simplified the conditional logic in patch 13/23 (Bryan) - Improved commit description for patch patch 13/23 (Nicolas) - Fix the value of H265_NUM_TILE_ROW macro (Neil) - Link to v3: https://lore.kernel.org/r/20250502-qcom-iris-hevc-vp9-v3-0-552158a10a7d@quicinc.com Changes in v3: - Introduced two wrappers with explicit names to handle destroy internal buffers (Nicolas) - Used sub state check instead of introducing new boolean (Vikash) - Addressed other comments (Vikash) - Reorderd patches to have all fixes patches first (Dmitry) - Link to v2: https://lore.kernel.org/r/20250428-qcom-iris-hevc-vp9-v2-0-3a6013ecb8a5@quicinc.com Changes in v2: - Added Changes to make sure all buffers are released in session close (bryna) - Added tracking for flush responses to fix a timing issue. - Added a handling to fix timing issue in reconfig - Splitted patch 06/20 in two patches (Bryan) - Added missing fixes tag (bryan) - Updated fluster report (Nicolas) - Link to v1: https://lore.kernel.org/r/20250408-iris-dec-hevc-vp9-v1-0-acd258778bd6@quicinc.com Changes sinces RFC: - Added additional fixes to address issues identified during further testing. - Moved typo fix to a seperate patch [Neil] - Reordered the patches for better logical flow and clarity [Neil, Dmitry] - Added fixes tag wherever applicable [Neil, Dmitry] - Removed the default case in the switch statement for codecs [Bryan] - Replaced if-else statements with switch-case [Bryan] - Added comments for mbpf [Bryan] - RFC: https://lore.kernel.org/linux-media/20250305104335.3629945-1-quic_dikshita@quicinc.com/ This patch series depends on [1] & [2] [1] https://lore.kernel.org/linux-media/20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org/ [2] https://lore.kernel.org/linux-media/20250424-qcs8300_iris-v5-0-f118f505c300@quicinc.com/ These patches are tested on SM8250 and SM8550 with v4l2-ctl and Gstreamer for HEVC and VP9 decoders, at the same time ensured that the existing H264 decoder functionality remains uneffected. Note: 1 of the fluster compliance test is fixed with firmware [3] [3]: https://lore.kernel.org/linux-firmware/1a511921-446d-cdc4-0203-084c88a5dc1e@quicinc.com/T/#u The result of fluster test on SM8550: 131/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 - 2 testcase failed due to CRC mismatch - RAP_A_docomo_6 - RAP_B_Bossen_2 - BUG reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4392 Analysis - First few frames in this discarded by firmware and are sent to driver with 0 filled length. Driver send such buffers to client with timestamp 0 and payload set to 0 and make buf state to VB2_BUF_STATE_ERROR. Such buffers should be dropped by GST. But instead, the first frame displayed as green frame and when a valid buffer is sent to client later with same 0 timestamp, its dropped, leading to CRC mismatch for first frame. 235/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug reported: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 2 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - 1 testcase failed due to unsupported stream - vp90-2-16-intra-only.webm The result of fluster test on SM8250: 133/147 testcases passed while testing JCT-VC-HEVC_V1 with GStreamer-H.265-V4L2-Gst1.0. The failing test case: - 10 testcases failed due to unsupported 10 bit format. - DBLK_A_MAIN10_VIXS_4 - INITQP_B_Main10_Sony_1 - TSUNEQBD_A_MAIN10_Technicolor_2 - WP_A_MAIN10_Toshiba_3 - WP_MAIN10_B_Toshiba_3 - WPP_A_ericsson_MAIN10_2 - WPP_B_ericsson_MAIN10_2 - WPP_C_ericsson_MAIN10_2 - WPP_E_ericsson_MAIN10_2 - WPP_F_ericsson_MAIN10_2 - 4 testcase failed due to unsupported resolution - PICSIZE_A_Bossen_1 - PICSIZE_B_Bossen_1 - WPP_D_ericsson_MAIN10_2 - WPP_D_ericsson_MAIN_2 232/305 testcases passed while testing VP9-TEST-VECTORS with GStreamer-VP9-V4L2-Gst1.0. The failing test case: - 64 testcases failed due to unsupported resolution - vp90-2-02-size-08x08.webm - vp90-2-02-size-08x10.webm - vp90-2-02-size-08x16.webm - vp90-2-02-size-08x18.webm - vp90-2-02-size-08x32.webm - vp90-2-02-size-08x34.webm - vp90-2-02-size-08x64.webm - vp90-2-02-size-08x66.webm - vp90-2-02-size-10x08.webm - vp90-2-02-size-10x10.webm - vp90-2-02-size-10x16.webm - vp90-2-02-size-10x18.webm - vp90-2-02-size-10x32.webm - vp90-2-02-size-10x34.webm - vp90-2-02-size-10x64.webm - vp90-2-02-size-10x66.webm - vp90-2-02-size-16x08.webm - vp90-2-02-size-16x10.webm - vp90-2-02-size-16x16.webm - vp90-2-02-size-16x18.webm - vp90-2-02-size-16x32.webm - vp90-2-02-size-16x34.webm - vp90-2-02-size-16x64.webm - vp90-2-02-size-16x66.webm - vp90-2-02-size-18x08.webm - vp90-2-02-size-18x10.webm - vp90-2-02-size-18x16.webm - vp90-2-02-size-18x18.webm - vp90-2-02-size-18x32.webm - vp90-2-02-size-18x34.webm - vp90-2-02-size-18x64.webm - vp90-2-02-size-18x66.webm - vp90-2-02-size-32x08.webm - vp90-2-02-size-32x10.webm - vp90-2-02-size-32x16.webm - vp90-2-02-size-32x18.webm - vp90-2-02-size-32x32.webm - vp90-2-02-size-32x34.webm - vp90-2-02-size-32x64.webm - vp90-2-02-size-32x66.webm - vp90-2-02-size-34x08.webm - vp90-2-02-size-34x10.webm - vp90-2-02-size-34x16.webm - vp90-2-02-size-34x18.webm - vp90-2-02-size-34x32.webm - vp90-2-02-size-34x34.webm - vp90-2-02-size-34x64.webm - vp90-2-02-size-34x66.webm - vp90-2-02-size-64x08.webm - vp90-2-02-size-64x10.webm - vp90-2-02-size-64x16.webm - vp90-2-02-size-64x18.webm - vp90-2-02-size-64x32.webm - vp90-2-02-size-64x34.webm - vp90-2-02-size-64x64.webm - vp90-2-02-size-64x66.webm - vp90-2-02-size-66x08.webm - vp90-2-02-size-66x10.webm - vp90-2-02-size-66x16.webm - vp90-2-02-size-66x18.webm - vp90-2-02-size-66x32.webm - vp90-2-02-size-66x34.webm - vp90-2-02-size-66x64.webm - vp90-2-02-size-66x66.webm - 2 testcases failed due to unsupported format - vp91-2-04-yuv422.webm - vp91-2-04-yuv444.webm - 1 testcase failed with CRC mismatch - vp90-2-22-svc_1280x720_3.ivf - Bug raised: https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/4371 - 5 testcase failed due to unsupported resolution after sequence change - vp90-2-21-resize_inter_320x180_5_1-2.webm - vp90-2-21-resize_inter_320x180_7_1-2.webm - vp90-2-21-resize_inter_320x240_5_1-2.webm - vp90-2-21-resize_inter_320x240_7_1-2.webm - vp90-2-18-resize.ivf - 1 testcase failed with CRC mismatch - vp90-2-16-intra-only.webm Analysis: First few frames are marked by firmware as NO_SHOW frame. Driver make buf state to VB2_BUF_STATE_ERROR for such frames. Such buffers should be dropped by GST. But instead, the first frame is being displayed and when a valid buffer is sent to client later with same timestamp, its dropped, leading to CRC mismatch for first frame. To: Vikash Garodia <quic_vgarodia@quicinc.com> To: Abhinav Kumar <quic_abhinavk@quicinc.com> To: Bryan O'Donoghue <bryan.odonoghue@linaro.org> To: Mauro Carvalho Chehab <mchehab@kernel.org> To: Hans Verkuil <hverkuil@xs4all.nl> To: Stefan Schmidt <stefan.schmidt@linaro.org> Cc: linux-media@vger.kernel.org Cc: linux-arm-msm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Cc: Dmitry Baryshkov <dmitry.baryshkov@oss.qualcomm.com> Cc: Neil Armstrong <neil.armstrong@linaro.org> Cc: Nicolas Dufresne <nicolas.dufresne@collabora.com> Cc: Dan Carpenter <dan.carpenter@linaro.org> Signed-off-by: Dikshita Agarwal <quic_dikshita@quicinc.com> --- Dikshita Agarwal (25): media: iris: Skip destroying internal buffer if not dequeued media: iris: Update CAPTURE format info based on OUTPUT format media: iris: Avoid updating frame size to firmware during reconfig media: iris: Drop port check for session property response media: iris: Prevent HFI queue writes when core is in deinit state media: iris: Remove error check for non-zero v4l2 controls media: iris: Remove deprecated property setting to firmware media: iris: Fix missing function pointer initialization media: iris: Fix NULL pointer dereference media: iris: Fix typo in depth variable media: iris: Track flush responses to prevent premature completion media: iris: Fix buffer preparation failure during resolution change media: iris: Send V4L2_BUF_FLAG_ERROR for capture buffers with 0 filled length media: iris: Skip flush on first sequence change media: iris: Remove unnecessary re-initialization of flush completion media: iris: Add handling for corrupt and drop frames media: iris: Add handling for no show frames media: iris: Improve last flag handling media: iris: Remove redundant buffer count check in stream off media: iris: Add a comment to explain usage of MBPS media: iris: Add HEVC and VP9 formats for decoder media: iris: Add platform capabilities for HEVC and VP9 decoders media: iris: Set mandatory properties for HEVC and VP9 decoders. media: iris: Add internal buffer calculation for HEVC and VP9 decoders media: iris: Add codec specific check for VP9 decoder drain handling drivers/media/platform/qcom/iris/iris_buffer.c | 35 +- drivers/media/platform/qcom/iris/iris_buffer.h | 3 +- drivers/media/platform/qcom/iris/iris_ctrls.c | 35 +- drivers/media/platform/qcom/iris/iris_hfi_common.h | 1 + .../platform/qcom/iris/iris_hfi_gen1_command.c | 48 ++- .../platform/qcom/iris/iris_hfi_gen1_defines.h | 5 +- .../platform/qcom/iris/iris_hfi_gen1_response.c | 37 +- .../platform/qcom/iris/iris_hfi_gen2_command.c | 143 +++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h | 5 + .../platform/qcom/iris/iris_hfi_gen2_response.c | 56 ++- drivers/media/platform/qcom/iris/iris_hfi_queue.c | 2 +- drivers/media/platform/qcom/iris/iris_instance.h | 6 + .../platform/qcom/iris/iris_platform_common.h | 28 +- .../media/platform/qcom/iris/iris_platform_gen2.c | 198 ++++++++-- .../platform/qcom/iris/iris_platform_qcs8300.h | 126 +++++-- .../platform/qcom/iris/iris_platform_sm8250.c | 15 +- drivers/media/platform/qcom/iris/iris_state.c | 2 +- drivers/media/platform/qcom/iris/iris_state.h | 1 + drivers/media/platform/qcom/iris/iris_vb2.c | 18 +- drivers/media/platform/qcom/iris/iris_vdec.c | 116 +++--- drivers/media/platform/qcom/iris/iris_vdec.h | 11 + drivers/media/platform/qcom/iris/iris_vidc.c | 36 +- drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 397 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 46 ++- 24 files changed, 1159 insertions(+), 211 deletions(-) --- base-commit: 398a1b33f1479af35ca915c5efc9b00d6204f8fa change-id: 20250507-video-iris-hevc-vp9-59096b189050 prerequisite-message-id: <20250417-topic-sm8x50-iris-v10-v7-0-f020cb1d0e98@linaro.org> prerequisite-patch-id: afffe7096c8e110a8da08c987983bc4441d39578 prerequisite-patch-id: b93c37dc7e09d1631b75387dc1ca90e3066dce17 prerequisite-patch-id: b7b50aa1657be59fd51c3e53d73382a1ee75a08e prerequisite-patch-id: 30960743105a36f20b3ec4a9ff19e7bca04d6add prerequisite-patch-id: 2bba98151ca103aa62a513a0fbd0df7ae64d9868 prerequisite-patch-id: 0e43a6d758b5fa5ab921c6aa3c19859e312b47d0 prerequisite-patch-id: 35f8dae1416977e88c2db7c767800c01822e266e prerequisite-message-id: <20250501-qcs8300_iris-v7-0-b229d5347990@quicinc.com> prerequisite-patch-id: e35b05c527217206ae871aef0d7b0261af0319ea prerequisite-patch-id: 07ba0745c7d72796567e0a57f5c8e5355a8d2046 prerequisite-patch-id: 3398937a7fabb45934bb98a530eef73252231132 prerequisite-patch-id: 500bc3b8391940d3ebca222d2098b737414b2af4 prerequisite-patch-id: 2e72fe4d11d264db3d42fa450427d30171303c6f Best regards,