From patchwork Fri Jun 24 08:23:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 585037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF0A2CCA481 for ; Fri, 24 Jun 2022 08:23:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231558AbiFXIXs (ORCPT ); Fri, 24 Jun 2022 04:23:48 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229571AbiFXIXr (ORCPT ); Fri, 24 Jun 2022 04:23:47 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05C726E7B8; Fri, 24 Jun 2022 01:23:45 -0700 (PDT) X-UUID: 3deed1638bc74cbb96bce916dcf56ee6-20220624 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.6, REQID:e746a608-63d2-483f-914a-767cf5e5ffff, OB:0, LO B:0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:0,FILE:0,RULE:Release_Ham,ACT ION:release,TS:0 X-CID-META: VersionHash:b14ad71, CLOUDID:c770f22d-1756-4fa3-be7f-474a6e4be921, C OID:IGNORED,Recheck:0,SF:nil,TC:nil,Content:0,EDM:-3,IP:nil,URL:1,File:nil ,QS:nil,BEC:nil,COL:0 X-UUID: 3deed1638bc74cbb96bce916dcf56ee6-20220624 Received: from mtkmbs11n1.mediatek.inc [(172.21.101.185)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1135623711; Fri, 24 Jun 2022 16:23:43 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Fri, 24 Jun 2022 16:23:41 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 24 Jun 2022 16:23:40 +0800 From: Irui Wang To: Hans Verkuil , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Irui Wang , , , , , , , Subject: [PATCH v4, 3/8] media: mediatek: vcodec: Refactor venc power manage function Date: Fri, 24 Jun 2022 16:23:30 +0800 Message-ID: <20220624082335.10165-4-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220624082335.10165-1-irui.wang@mediatek.com> References: <20220624082335.10165-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk functions because of sub-devices, sub-devices will init their own "pm/clk" instead, so refactor the pm function with args "platform_device *" and "mtk_vcodec_pm*". Signed-off-by: Irui Wang --- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 2 +- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c | 8 ++++++++ .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c | 9 +++------ .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h | 3 ++- 4 files changed, 14 insertions(+), 8 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c index 65a8251a5a68..3eea70c96cdd 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c @@ -257,7 +257,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) return PTR_ERR(dev->fw_handler); dev->venc_pdata = of_device_get_match_data(&pdev->dev); - ret = mtk_vcodec_init_enc_clk(dev); + ret = mtk_vcodec_init_enc_clk(dev->plat_dev, &dev->pm); if (ret < 0) { dev_err(&pdev->dev, "Failed to get mtk vcodec clock source!"); goto err_enc_pm; diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c index 02582cce4863..12f7a1545c5a 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c @@ -13,6 +13,7 @@ #include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_hw.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" static const struct of_device_id mtk_venc_hw_ids[] = { @@ -118,6 +119,13 @@ static int mtk_venc_hw_probe(struct platform_device *pdev) "Invalid hardware id %d\n", sub_core->hw_id); + ret = mtk_vcodec_init_enc_clk(sub_core->plat_dev, &sub_core->pm); + if (ret < 0) + return dev_err_probe(dev, ret, + "Failed to get venc core clock source!"); + + pm_runtime_enable(&pdev->dev); + main_dev->enc_hw_dev[sub_core->hw_id] = sub_core; sub_core->main_dev = main_dev; diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c index 7055954eb2af..75de5031d292 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c @@ -12,17 +12,13 @@ #include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" -int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev) +int mtk_vcodec_init_enc_clk(struct platform_device *pdev, + struct mtk_vcodec_pm *pm) { - struct platform_device *pdev; - struct mtk_vcodec_pm *pm; struct mtk_vcodec_clk *enc_clk; struct mtk_vcodec_clk_info *clk_info; int ret, i; - pdev = mtkdev->plat_dev; - pm = &mtkdev->pm; - memset(pm, 0, sizeof(struct mtk_vcodec_pm)); pm->dev = &pdev->dev; enc_clk = &pm->venc_clk; @@ -58,6 +54,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev) return 0; } +EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk); void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) { diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h index bc455cefc0cd..361dec5be47f 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h @@ -9,7 +9,8 @@ #include "mtk_vcodec_drv.h" -int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *dev); +int mtk_vcodec_init_enc_clk(struct platform_device *pdev, + struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); From patchwork Fri Jun 24 08:23:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 585036 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06AA9C43334 for ; Fri, 24 Jun 2022 08:23:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231659AbiFXIX4 (ORCPT ); Fri, 24 Jun 2022 04:23:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44810 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231643AbiFXIXw (ORCPT ); Fri, 24 Jun 2022 04:23:52 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4AF456E7AE; Fri, 24 Jun 2022 01:23:50 -0700 (PDT) X-UUID: be7e712c58d34f2fadfa3a64cfa84173-20220624 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.6, REQID:4f38cd66-a03a-4e56-bc82-16c1c49980b3, OB:0, LO B:0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:100,FILE:0,RULE:Release_Ham,A CTION:release,TS:100 X-CID-INFO: VERSION:1.1.6, REQID:4f38cd66-a03a-4e56-bc82-16c1c49980b3, OB:0, LOB: 0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:100,FILE:0,RULE:Spam_GS981B3D,A CTION:quarantine,TS:100 X-CID-META: VersionHash:b14ad71, CLOUDID:1b71f22d-1756-4fa3-be7f-474a6e4be921, C OID:f3eacff926b3,Recheck:0,SF:28|17|19|48,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,QS:nil,BEC:nil,COL:0 X-UUID: be7e712c58d34f2fadfa3a64cfa84173-20220624 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1648332409; Fri, 24 Jun 2022 16:23:44 +0800 Received: from mtkmbs07n1.mediatek.inc (172.21.101.16) by mtkmbs11n2.mediatek.inc (172.21.101.187) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Fri, 24 Jun 2022 16:23:43 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Fri, 24 Jun 2022 16:23:42 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 24 Jun 2022 16:23:41 +0800 From: Irui Wang To: Hans Verkuil , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Irui Wang , , , , , , , Subject: [PATCH v4, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding Date: Fri, 24 Jun 2022 16:23:31 +0800 Message-ID: <20220624082335.10165-5-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220624082335.10165-1-irui.wang@mediatek.com> References: <20220624082335.10165-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a bit for indicating support multi-core encoding, because multi-core encoding need more working buffers for encoder hardware. The working buffers are allocated from kernel side then pass to scp firmware side through shared memory, the struct definition must be kept align between kernel and scp firmware side. New another shared memory struct for multi-core encoding. Signed-off-by: Irui Wang --- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 2 + .../mediatek/vcodec/mtk_vcodec_util.c | 19 ++ .../mediatek/vcodec/mtk_vcodec_util.h | 2 + .../mediatek/vcodec/venc/venc_h264_if.c | 171 +++++++++++++++--- 4 files changed, 170 insertions(+), 24 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h index 8919bdf2eef5..3a291cac6b5e 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h @@ -29,6 +29,8 @@ #define WAIT_INTR_TIMEOUT_MS 1000 #define IS_VDEC_LAT_ARCH(hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE) #define IS_VDEC_INNER_RACING(capability) ((capability) & MTK_VCODEC_INNER_RACING) +#define MTK_VENC_MULTICORE_ENABLE BIT(1) +#define IS_VENC_MULTICORE(capability) ((capability) & MTK_VENC_MULTICORE_ENABLE) /* * enum mtk_hw_reg_idx - MTK hw register base index diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c index ace78c4b5b9e..a723243626c0 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c @@ -11,6 +11,7 @@ #include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_hw.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, @@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, } EXPORT_SYMBOL(mtk_vcodec_get_reg_addr); +void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *ctx, + unsigned int hw_id) +{ + struct mtk_venc_hw_dev *sub_core; + + if (hw_id >= MTK_VENC_HW_MAX) { + mtk_v4l2_err("Invalid hw_id = %d", hw_id); + return NULL; + } + + sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[hw_id]; + if (!sub_core) + return NULL; + + return sub_core->reg_base; +} +EXPORT_SYMBOL(mtk_venc_get_core_reg_addr); + int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data, struct mtk_vcodec_mem *mem) { diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h index 71956627a0e2..a74c98aa355b 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h @@ -50,6 +50,8 @@ struct mtk_vcodec_dev; void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, unsigned int reg_idx); +void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *data, + unsigned int hw_id); int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data, struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data, diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c index 4d9b8798dffe..888cee39d324 100644 --- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c @@ -50,6 +50,24 @@ enum venc_h264_vpu_work_buf { VENC_H264_VPU_WORK_BUF_MAX, }; +/* + * enum venc_multi_core_work_buf - h264 multi core encoder buffer index + */ +enum venc_multi_core_work_buf { + VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE0, + VENC_MULTI_CORE_WORK_BUF_RC_CODE, + VENC_MULTI_CORE_WORK_BUF_REC_LUMA, + VENC_MULTI_CORE_WORK_BUF_REC_CHROMA, + VENC_MULTI_CORE_WORK_BUF_REF_LUMA, + VENC_MULTI_CORE_WORK_BUF_REF_CHROMA, + VENC_MULTI_CORE_WORK_BUF_MV_INFO_1, + VENC_MULTI_CORE_WORK_BUF_MV_INFO_2, + VENC_MULTI_CORE_WORK_BUF_SKIP_FRAME, + VENC_MULTI_CORE_WORK_BUF_RC_INFO_CORE1, + VENC_MULTI_CORE_WORK_BUF_FR_RC_INFO, + VENC_MULTI_CORE_WORK_BUF_MAX, +}; + /* * enum venc_h264_bs_mode - for bs_mode argument in h264_enc_vpu_encode */ @@ -94,6 +112,24 @@ struct venc_h264_vpu_config { u32 wfd; }; +struct venc_multi_core_config { + u32 input_fourcc; + u32 bitrate; + u32 pic_w; + u32 pic_h; + u32 buf_w; + u32 buf_h; + u32 gop_size; + u32 intra_period; + u32 framerate; + u32 profile; + u32 level; + u32 wfd; + u32 max_qp; + u32 min_qp; + u32 reserved[8]; +}; + /* * struct venc_h264_vpu_buf - Structure for buffer information * AP-W/R : AP is writer/reader on this item @@ -127,6 +163,11 @@ struct venc_h264_vsi { struct venc_h264_vpu_buf work_bufs[VENC_H264_VPU_WORK_BUF_MAX]; }; +struct venc_multi_core_vsi { + struct venc_multi_core_config config; + struct venc_h264_vpu_buf work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX]; +}; + /* * struct venc_h264_inst - h264 encoder AP driver instance * @hw_base: h264 encoder hardware register base @@ -143,8 +184,8 @@ struct venc_h264_vsi { * @ctx: context for v4l2 layer integration */ struct venc_h264_inst { - void __iomem *hw_base; - struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX]; + void __iomem *hw_base[MTK_VENC_HW_MAX]; + struct mtk_vcodec_mem work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX]; struct mtk_vcodec_mem pps_buf; bool work_buf_allocated; unsigned int frm_cnt; @@ -152,12 +193,13 @@ struct venc_h264_inst { unsigned int prepend_hdr; struct venc_vpu_inst vpu_inst; struct venc_h264_vsi *vsi; + struct venc_multi_core_vsi *core_vsi; struct mtk_vcodec_ctx *ctx; }; static inline u32 h264_read_reg(struct venc_h264_inst *inst, u32 addr) { - return readl(inst->hw_base + addr); + return readl(inst->hw_base[MTK_VENC_CORE_0] + addr); } static unsigned int h264_get_profile(struct venc_h264_inst *inst, @@ -228,13 +270,20 @@ static unsigned int h264_get_level(struct venc_h264_inst *inst, static void h264_enc_free_work_buf(struct venc_h264_inst *inst) { int i; + struct mtk_vcodec_ctx *ctx = inst->ctx; + int max_work_buf; mtk_vcodec_debug_enter(inst); + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) + max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX; + else + max_work_buf = VENC_H264_VPU_WORK_BUF_MAX; + /* Except the SKIP_FRAME buffers, * other buffers need to be freed by AP. */ - for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) { + for (i = 0; i < max_work_buf; i++) { if (i != VENC_H264_VPU_WORK_BUF_SKIP_FRAME) mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]); } @@ -248,11 +297,21 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst) { int i; int ret = 0; - struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs; + struct mtk_vcodec_ctx *ctx = inst->ctx; + struct venc_h264_vpu_buf *wb; + int max_work_buf; mtk_vcodec_debug_enter(inst); - for (i = 0; i < VENC_H264_VPU_WORK_BUF_MAX; i++) { + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) { + wb = inst->core_vsi->work_bufs; + max_work_buf = VENC_MULTI_CORE_WORK_BUF_MAX; + } else { + wb = inst->vsi->work_bufs; + max_work_buf = VENC_H264_VPU_WORK_BUF_MAX; + } + + for (i = 0; i < max_work_buf; i++) { /* * This 'wb' structure is set by VPU side and shared to AP for * buffer allocation and IO virtual addr mapping. For most of @@ -358,6 +417,26 @@ static int h264_frame_type(struct venc_h264_inst *inst) return VENC_H264_P_FRM; /* Note: B frames are not supported */ } } + +static int h264_core_frame_type(struct venc_h264_inst *inst) +{ + struct venc_multi_core_vsi *vsi = inst->core_vsi; + + if ((vsi->config.gop_size != 0 && + (inst->frm_cnt % vsi->config.gop_size) == 0) || + (inst->frm_cnt == 0 && vsi->config.gop_size == 0)) { + /* IDR frame */ + return VENC_H264_IDR_FRM; + } else if ((vsi->config.intra_period != 0 && + (inst->frm_cnt % vsi->config.intra_period) == 0) || + (inst->frm_cnt == 0 && vsi->config.intra_period == 0)) { + /* I frame */ + return VENC_H264_I_FRM; + } else { + return VENC_H264_P_FRM; /* Note: B frames are not supported */ + } +} + static int h264_encode_sps(struct venc_h264_inst *inst, struct mtk_vcodec_mem *bs_buf, unsigned int *bs_size) @@ -440,12 +519,16 @@ static int h264_encode_frame(struct venc_h264_inst *inst, int ret = 0; unsigned int irq_status; struct venc_frame_info frame_info; + struct mtk_vcodec_ctx *ctx = inst->ctx; mtk_vcodec_debug_enter(inst); mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt); frame_info.frm_count = inst->frm_cnt; frame_info.skip_frm_count = inst->skip_frm_cnt; - frame_info.frm_type = h264_frame_type(inst); + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) + frame_info.frm_type = h264_core_frame_type(inst); + else + frame_info.frm_type = h264_frame_type(inst); mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n", frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); @@ -501,7 +584,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf, static int h264_enc_init(struct mtk_vcodec_ctx *ctx) { const bool is_ext = MTK_ENC_CTX_IS_EXT(ctx); - int ret = 0; + int ret, i; struct venc_h264_inst *inst; inst = kzalloc(sizeof(*inst), GFP_KERNEL); @@ -511,13 +594,23 @@ static int h264_enc_init(struct mtk_vcodec_ctx *ctx) inst->ctx = ctx; inst->vpu_inst.ctx = ctx; inst->vpu_inst.id = is_ext ? SCP_IPI_VENC_H264 : IPI_VENC_H264; - inst->hw_base = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS); mtk_vcodec_debug_enter(inst); ret = vpu_enc_init(&inst->vpu_inst); - inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi; + inst->hw_base[0] = mtk_vcodec_get_reg_addr(inst->ctx, VENC_SYS); + + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) { + inst->core_vsi = + (struct venc_multi_core_vsi *)inst->vpu_inst.vsi; + + for (i = 1; i < MTK_VENC_HW_MAX; i++) + inst->hw_base[i] = + mtk_venc_get_core_reg_addr(inst->ctx, i); + } else { + inst->vsi = (struct venc_h264_vsi *)inst->vpu_inst.vsi; + } mtk_vcodec_debug_leave(inst); @@ -624,31 +717,61 @@ static int h264_enc_encode(void *handle, return ret; } +static void h264_enc_set_configs(struct venc_h264_inst *inst, + struct venc_enc_param *enc_prm) +{ + inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt; + inst->vsi->config.bitrate = enc_prm->bitrate; + inst->vsi->config.pic_w = enc_prm->width; + inst->vsi->config.pic_h = enc_prm->height; + inst->vsi->config.buf_w = enc_prm->buf_width; + inst->vsi->config.buf_h = enc_prm->buf_height; + inst->vsi->config.gop_size = enc_prm->gop_size; + inst->vsi->config.framerate = enc_prm->frm_rate; + inst->vsi->config.intra_period = enc_prm->intra_period; + inst->vsi->config.profile = + h264_get_profile(inst, enc_prm->h264_profile); + inst->vsi->config.level = + h264_get_level(inst, enc_prm->h264_level); + inst->vsi->config.wfd = 0; +} + +static void h264_enc_set_core_configs(struct venc_h264_inst *inst, + struct venc_enc_param *enc_prm) +{ + inst->core_vsi->config.input_fourcc = enc_prm->input_yuv_fmt; + inst->core_vsi->config.bitrate = enc_prm->bitrate; + inst->core_vsi->config.pic_w = enc_prm->width; + inst->core_vsi->config.pic_h = enc_prm->height; + inst->core_vsi->config.buf_w = enc_prm->buf_width; + inst->core_vsi->config.buf_h = enc_prm->buf_height; + inst->core_vsi->config.gop_size = enc_prm->gop_size; + inst->core_vsi->config.framerate = enc_prm->frm_rate; + inst->core_vsi->config.intra_period = enc_prm->intra_period; + inst->core_vsi->config.profile = + h264_get_profile(inst, enc_prm->h264_profile); + inst->core_vsi->config.level = + h264_get_level(inst, enc_prm->h264_level); + inst->core_vsi->config.wfd = 0; +} + static int h264_enc_set_param(void *handle, enum venc_set_param_type type, struct venc_enc_param *enc_prm) { int ret = 0; struct venc_h264_inst *inst = (struct venc_h264_inst *)handle; + struct mtk_vcodec_ctx *ctx = inst->ctx; mtk_vcodec_debug(inst, "->type=%d", type); switch (type) { case VENC_SET_PARAM_ENC: - inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt; - inst->vsi->config.bitrate = enc_prm->bitrate; - inst->vsi->config.pic_w = enc_prm->width; - inst->vsi->config.pic_h = enc_prm->height; - inst->vsi->config.buf_w = enc_prm->buf_width; - inst->vsi->config.buf_h = enc_prm->buf_height; - inst->vsi->config.gop_size = enc_prm->gop_size; - inst->vsi->config.framerate = enc_prm->frm_rate; - inst->vsi->config.intra_period = enc_prm->intra_period; - inst->vsi->config.profile = - h264_get_profile(inst, enc_prm->h264_profile); - inst->vsi->config.level = - h264_get_level(inst, enc_prm->h264_level); - inst->vsi->config.wfd = 0; + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) + h264_enc_set_core_configs(inst, enc_prm); + else + h264_enc_set_configs(inst, enc_prm); + ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm); if (ret) break; From patchwork Fri Jun 24 08:23:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 585035 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 178A8C43334 for ; Fri, 24 Jun 2022 08:24:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231709AbiFXIX6 (ORCPT ); Fri, 24 Jun 2022 04:23:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44856 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231676AbiFXIXz (ORCPT ); Fri, 24 Jun 2022 04:23:55 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4A4FD6F7A0; Fri, 24 Jun 2022 01:23:53 -0700 (PDT) X-UUID: 70bd9afa1e974cabab752daa1b1ac74a-20220624 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.6, REQID:7dab53ef-0011-4396-90f3-35d641f1be6e, OB:30, L OB:0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:100,FILE:0,RULE:Release_Ham, ACTION:release,TS:100 X-CID-INFO: VERSION:1.1.6, REQID:7dab53ef-0011-4396-90f3-35d641f1be6e, OB:30, LOB :0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:100,FILE:0,RULE:Spam_GS981B3D, ACTION:quarantine,TS:100 X-CID-META: VersionHash:b14ad71, CLOUDID:6571f22d-1756-4fa3-be7f-474a6e4be921, C OID:b617216b67d0,Recheck:0,SF:28|17|19|48,TC:nil,Content:0,EDM:-3,IP:nil,U RL:1,File:nil,QS:nil,BEC:nil,COL:0 X-UUID: 70bd9afa1e974cabab752daa1b1ac74a-20220624 Received: from mtkcas11.mediatek.inc [(172.21.101.40)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 619834160; Fri, 24 Jun 2022 16:23:46 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Fri, 24 Jun 2022 16:23:45 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 24 Jun 2022 16:23:44 +0800 From: Irui Wang To: Hans Verkuil , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Irui Wang , , , , , , , Subject: [PATCH v4, 6/8] media: mediatek: vcodec: Refactor encoder clock on/off function Date: Fri, 24 Jun 2022 16:23:33 +0800 Message-ID: <20220624082335.10165-7-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220624082335.10165-1-irui.wang@mediatek.com> References: <20220624082335.10165-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org when enable multi-core encoding, encoder cores use their own clock, refactor clock management functions with used encoder hardware id. Signed-off-by: Irui Wang --- .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 95 ++++++++++++++++--- .../mediatek/vcodec/mtk_vcodec_enc_pm.h | 4 +- .../platform/mediatek/vcodec/venc_drv_if.c | 4 +- 3 files changed, 84 insertions(+), 19 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c index a6f30b3a6bc5..e9b4e95ca012 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c @@ -60,7 +60,9 @@ EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk); static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx) { struct mtk_venc_hw_dev *sub_core; + struct mtk_vcodec_clk *clk; int ret, i; + int j = 0; /* multi-core encoding need power on all available cores */ for (i = 0; i < MTK_VENC_HW_MAX; i++) { @@ -73,12 +75,27 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx) mtk_v4l2_err("power on sub_core[%d] fail %d", i, ret); goto pm_on_fail; } + + clk = &sub_core->pm.venc_clk; + for (j = 0; j < clk->clk_num; j++) { + ret = clk_prepare(clk->clk_info[j].vcodec_clk); + if (ret) { + mtk_v4l2_err("prepare clk [%s] fail %d", + clk->clk_info[j].clk_name, ret); + goto pm_on_fail; + } + } } return ret; pm_on_fail: for (i -= 1; i >= 0; i--) { sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i]; + + clk = &sub_core->pm.venc_clk; + for (j -= 1; j >= 0; j--) + clk_unprepare(clk->clk_info[j].vcodec_clk); + pm_runtime_put_sync(&sub_core->plat_dev->dev); } return ret; @@ -87,7 +104,9 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx) int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) { struct mtk_vcodec_pm *pm = &ctx->dev->pm; + struct mtk_vcodec_clk *clk; int ret; + int i = 0; ret = pm_runtime_resume_and_get(pm->dev); if (ret) { @@ -95,6 +114,16 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) return ret; } + clk = &pm->venc_clk; + for (i = 0; i < clk->clk_num; i++) { + ret = clk_prepare(clk->clk_info[i].vcodec_clk); + if (ret) { + mtk_v4l2_err("prepare clk [%s] fail %d", + clk->clk_info[i].clk_name, ret); + goto clk_error; + } + } + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) { ret = mtk_enc_core_power_on(ctx); if (ret) { @@ -104,6 +133,9 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) } return ret; +clk_error: + for (i -= 1; i >= 0; i--) + clk_unprepare(clk->clk_info[i].vcodec_clk); core_error: pm_runtime_put_sync(pm->dev); return ret; @@ -112,7 +144,8 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx) { struct mtk_venc_hw_dev *sub_core; - int ret, i; + struct mtk_vcodec_clk *clk; + int i, ret, j; /* multi-core encoding need power off all available cores */ for (i = 0; i < MTK_VENC_HW_MAX; i++) { @@ -120,6 +153,10 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx) if (!sub_core) continue; + clk = &sub_core->pm.venc_clk; + for (j = clk->clk_num - 1; j >= 0; j--) + clk_unprepare(clk->clk_info[j].vcodec_clk); + ret = pm_runtime_put_sync(&sub_core->plat_dev->dev); if (ret) mtk_v4l2_err("power off sub_core[%d] fail %d", i, ret); @@ -129,27 +166,44 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx) void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx) { struct mtk_vcodec_pm *pm = &ctx->dev->pm; - int ret; - - ret = pm_runtime_put_sync(pm->dev); - if (ret) - mtk_v4l2_err("pm_runtime_put_sync fail %d", ret); + struct mtk_vcodec_clk *clk; + int ret, i; if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) { mtk_enc_core_power_off(ctx); } + + clk = &pm->venc_clk; + for (i = clk->clk_num - 1; i >= 0; i--) + clk_unprepare(clk->clk_info[i].vcodec_clk); + + ret = pm_runtime_put_sync(pm->dev); + if (ret) + mtk_v4l2_err("pm_runtime_put_sync fail %d", ret); } -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int hw_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; + struct mtk_venc_hw_dev *sub_core; + struct mtk_vcodec_clk *enc_clk; + int ret, i = 0; + if (hw_id == MTK_VENC_CORE_0) { + enc_clk = &dev->pm.venc_clk; + } else if (hw_id == MTK_VENC_CORE_1) { + sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id]; + enc_clk = &sub_core->pm.venc_clk; + } else { + mtk_v4l2_err("invalid hw id : %d", hw_id); + return; + } + for (i = 0; i < enc_clk->clk_num; i++) { - ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk); + ret = clk_enable(enc_clk->clk_info[i].vcodec_clk); if (ret) { - mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i, - enc_clk->clk_info[i].clk_name, ret); + mtk_v4l2_err("venc clk_enable %d %s fail %d", i, + enc_clk->clk_info[i].clk_name, ret); goto clkerr; } } @@ -158,14 +212,25 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) clkerr: for (i -= 1; i >= 0; i--) - clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + clk_disable(enc_clk->clk_info[i].vcodec_clk); } -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int hw_id) { - struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; + struct mtk_venc_hw_dev *sub_core; + struct mtk_vcodec_clk *enc_clk; int i = 0; + if (hw_id == MTK_VENC_CORE_0) { + enc_clk = &dev->pm.venc_clk; + } else if (hw_id == MTK_VENC_CORE_1) { + sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id]; + enc_clk = &sub_core->pm.venc_clk; + } else { + mtk_v4l2_err("invalid hw id : %d", hw_id); + return; + } + for (i = enc_clk->clk_num - 1; i >= 0; i--) - clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); + clk_disable(enc_clk->clk_info[i].vcodec_clk); } diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h index 9065dec4ed4f..b5a5a7990639 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h @@ -14,7 +14,7 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev, int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx); void mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx); -void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int hw_id); +void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int hw_id); #endif /* _MTK_VCODEC_ENC_PM_H_ */ diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c index 65a27e39ef5b..6cbdb7e30bb3 100644 --- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.c +++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.c @@ -64,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx; spin_unlock_irqrestore(&ctx->dev->irqlock, flags); - mtk_vcodec_enc_clock_on(&ctx->dev->pm); + mtk_vcodec_enc_clock_on(ctx->dev, 0); ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result); - mtk_vcodec_enc_clock_off(&ctx->dev->pm); + mtk_vcodec_enc_clock_off(ctx->dev, 0); spin_lock_irqsave(&ctx->dev->irqlock, flags); ctx->dev->curr_ctx = NULL; From patchwork Fri Jun 24 08:23:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 585034 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0CF0C433EF for ; Fri, 24 Jun 2022 08:24:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231656AbiFXIYE (ORCPT ); Fri, 24 Jun 2022 04:24:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:44914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231643AbiFXIYA (ORCPT ); Fri, 24 Jun 2022 04:24:00 -0400 Received: from mailgw01.mediatek.com (unknown [60.244.123.138]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 150066E7B8; Fri, 24 Jun 2022 01:23:57 -0700 (PDT) X-UUID: 5fef8495a7b74750a4e6f0856508281a-20220624 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.6, REQID:f270c385-2188-4ede-8866-fd7f9a0c21e3, OB:0, LO B:0,IP:0,URL:5,TC:0,Content:-5,EDM:0,RT:0,SF:0,FILE:0,RULE:Release_Ham,ACT ION:release,TS:0 X-CID-META: VersionHash:b14ad71, CLOUDID:664378d8-850a-491d-a127-60d9309b2b3e, C OID:IGNORED,Recheck:0,SF:nil,TC:nil,Content:0,EDM:-3,IP:nil,URL:1,File:nil ,QS:nil,BEC:nil,COL:0 X-UUID: 5fef8495a7b74750a4e6f0856508281a-20220624 Received: from mtkmbs10n2.mediatek.inc [(172.21.101.183)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 943728301; Fri, 24 Jun 2022 16:23:50 +0800 Received: from mtkcas10.mediatek.inc (172.21.101.39) by mtkmbs10n1.mediatek.inc (172.21.101.34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.15; Fri, 24 Jun 2022 16:23:48 +0800 Received: from localhost.localdomain (10.17.3.154) by mtkcas10.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Fri, 24 Jun 2022 16:23:46 +0800 From: Irui Wang To: Hans Verkuil , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , , CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Irui Wang , , , , , , , Subject: [PATCH v4, 8/8] media: mediatek: vcodec: Return encoding result in asynchronous mode Date: Fri, 24 Jun 2022 16:23:35 +0800 Message-ID: <20220624082335.10165-9-irui.wang@mediatek.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220624082335.10165-1-irui.wang@mediatek.com> References: <20220624082335.10165-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org when enable multi-core encoding, the wait IRQ done synchronous function should not be called, so the encoding result can't return to client in device_run. Move the buffer done function in IRQ handler. Signed-off-by: Irui Wang --- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 73 +++++++++++++++++-- .../platform/mediatek/vcodec/mtk_vcodec_enc.h | 7 +- .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 28 ++++++- .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 13 +++- .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 1 + .../mediatek/vcodec/mtk_vcodec_util.h | 1 + .../mediatek/vcodec/venc/venc_h264_if.c | 20 +++-- .../platform/mediatek/vcodec/venc_drv_if.h | 2 + 9 files changed, 138 insertions(+), 13 deletions(-) diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h index d54f12026189..6b05ee201fd3 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h @@ -302,6 +302,9 @@ struct vdec_pic_info { * @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. * @encoded_frame_cnt: number of encoded frames + * @pfrm_buf: used to store current ctx's frame buffer + * @pbs_buf: used to store current ctx's bitstream buffer + * @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx { enum mtk_instance_type type; @@ -352,6 +355,9 @@ struct mtk_vcodec_ctx { struct vdec_msg_queue msg_queue; int encoded_frame_cnt; + struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX]; + struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX]; + unsigned int hdr_size; }; /* diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c index e0168f9cb2f5..8196407b50b6 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c @@ -929,6 +929,8 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q) mtk_v4l2_debug(2, "[%d]-> type=%d", ctx->id, q->type); + mtk_venc_lock_all(ctx); + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { while ((dst_buf = v4l2_m2m_dst_buf_remove(ctx->m2m_ctx))) { vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); @@ -1164,6 +1166,7 @@ static void mtk_venc_worker(struct work_struct *work) * is dequeued. */ if (src_buf == &ctx->empty_flush_buf.vb) { + mtk_venc_lock_all(ctx); vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0); dst_buf->flags |= V4L2_BUF_FLAG_LAST; v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); @@ -1178,9 +1181,12 @@ static void mtk_venc_worker(struct work_struct *work) frm_buf.fb_addr[i].size = (size_t)src_buf->vb2_buf.planes[i].length; } + frm_buf.frm_addr = src_buf; + bs_buf.va = vb2_plane_vaddr(&dst_buf->vb2_buf, 0); bs_buf.dma_addr = vb2_dma_contig_plane_dma_addr(&dst_buf->vb2_buf, 0); bs_buf.size = (size_t)dst_buf->vb2_buf.planes[0].length; + bs_buf.buf = dst_buf; mtk_v4l2_debug(2, "Framebuf PA=%llx Size=0x%zx;PA=0x%llx Size=0x%zx;PA=0x%llx Size=%zu", @@ -1206,11 +1212,14 @@ static void mtk_venc_worker(struct work_struct *work) v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR); mtk_v4l2_err("venc_if_encode failed=%d", ret); } else { - v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); - vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size); - v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); - mtk_v4l2_debug(2, "venc_if_encode bs size=%d", - enc_result.bs_size); + if (!IS_VENC_MULTICORE(ctx->dev->enc_capability)) { + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); + vb2_set_plane_payload(&dst_buf->vb2_buf, 0, + enc_result.bs_size); + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); + mtk_v4l2_debug(2, "venc_if_encode bs size=%d", + enc_result.bs_size); + } } ctx->encoded_frame_cnt++; @@ -1420,6 +1429,34 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, return vb2_queue_init(dst_vq); } +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id, + unsigned int bs_size, bool time_out, bool key_frame) +{ + struct vb2_v4l2_buffer *src_vb2_v4l2 = NULL; + struct vb2_v4l2_buffer *dst_vb2_v4l2 = NULL; + + /* + * the frm_buf(src_buf) and bs_buf(dst_buf) can be obtained from ctx, + * then put them to done list, user can get them by dqbuf call + */ + src_vb2_v4l2 = ctx->pfrm_buf[hw_id]; + dst_vb2_v4l2 = ctx->pbs_buf[hw_id]; + + if (src_vb2_v4l2 && dst_vb2_v4l2) { + dst_vb2_v4l2->vb2_buf.timestamp = + src_vb2_v4l2->vb2_buf.timestamp; + dst_vb2_v4l2->timecode = src_vb2_v4l2->timecode; + + if (key_frame) + dst_vb2_v4l2->flags |= V4L2_BUF_FLAG_KEYFRAME; + + v4l2_m2m_buf_done(src_vb2_v4l2, VB2_BUF_STATE_DONE); + vb2_set_plane_payload(&dst_vb2_v4l2->vb2_buf, 0, bs_size); + v4l2_m2m_buf_done(dst_vb2_v4l2, VB2_BUF_STATE_DONE); + } +} +EXPORT_SYMBOL_GPL(mtk_venc_buf_done); + int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id) { struct mtk_vcodec_dev *dev = ctx->dev; @@ -1427,6 +1464,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int hw_id) mutex_unlock(&dev->enc_mutex[hw_id]); return 0; } +EXPORT_SYMBOL_GPL(mtk_venc_unlock); int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id) { @@ -1435,6 +1473,31 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int hw_id) mutex_lock(&dev->enc_mutex[hw_id]); return 0; } +EXPORT_SYMBOL_GPL(mtk_venc_lock); + +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx) +{ + unsigned int i; + struct mtk_vcodec_dev *dev = ctx->dev; + + /* + * For multi-core mode encoding, there are may be bufs being encoded + * when get the empty flush buffer or stop streaming, for example, the + * buffer with LAST flag will return to client before the encoding + * buffers, which will cause frame lost. + + * The encoder device mutex will be locked during encoding process, + * when encode done, the mutex unlocked. So if all encoder device mutex + * can be locked, which means there are no bufs being encoded at this + * time, then the buffer with LAST flag can return to client properly. + */ + + for (i = 0; i < MTK_VENC_HW_MAX; i++) { + mutex_lock(&dev->enc_mutex[i]); + mutex_unlock(&dev->enc_mutex[i]); + } +} +EXPORT_SYMBOL_GPL(mtk_venc_lock_all); void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx) { diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h index 29f5c8d1b59f..5ab17381c7ba 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h @@ -20,6 +20,9 @@ #define MTK_VENC_IRQ_STATUS_OFFSET 0x05C #define MTK_VENC_IRQ_ACK_OFFSET 0x060 +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2 /** * struct mtk_video_enc_buf - Private data related to each VB2 buffer. @@ -46,5 +49,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx); int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx); void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx); - +void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int hw_id, + unsigned int bs_size, bool time_out, bool key_frame); +void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx); #endif /* _MTK_VCODEC_ENC_H_ */ diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c index 006f3ce9c0d8..3210c24fefa5 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c @@ -89,6 +89,9 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv) struct mtk_vcodec_ctx *ctx; unsigned long flags; void __iomem *addr; + unsigned int bs_size; + unsigned int frm_type; + bool is_key_frame = 0; spin_lock_irqsave(&dev->irqlock, flags); ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0]; @@ -101,8 +104,32 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv) ctx->irq_status = readl(dev->reg_base[dev->venc_pdata->core_id] + (MTK_VENC_IRQ_STATUS_OFFSET)); + bs_size = readl(dev->reg_base[dev->venc_pdata->core_id] + + (VENC_PIC_BITSTREAM_BYTE_CNT)); + frm_type = readl(dev->reg_base[dev->venc_pdata->core_id] + + (VENC_PIC_FRM_TYPE)); + clean_irq_status(ctx->irq_status, addr); + if (IS_VENC_MULTICORE(dev->enc_capability)) { + if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) { + if (ctx->hdr_size != 0) { + bs_size += ctx->hdr_size; + ctx->hdr_size = 0; + } + + if (frm_type & VENC_PIC_KEY_FRM) + is_key_frame = 1; + + mtk_venc_buf_done(ctx, 0, bs_size, 0, is_key_frame); + mtk_vcodec_enc_clock_off(dev, 0); + mtk_venc_unlock(ctx, 0); + } else { + wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); + } + return IRQ_HANDLED; + } + wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); return IRQ_HANDLED; } @@ -290,7 +317,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } dev->enc_irq = platform_get_irq(pdev, 0); - irq_set_status_flags(dev->enc_irq, IRQ_NOAUTOEN); ret = devm_request_irq(&pdev->dev, dev->enc_irq, mtk_vcodec_enc_irq_handler, 0, pdev->name, dev); diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c index c151686f36a0..565acb3837d3 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c @@ -52,6 +52,9 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv) struct mtk_vcodec_ctx *ctx; unsigned long flags; void __iomem *addr; + unsigned int bs_size; + unsigned int frm_type; + bool is_key_frame = 0; spin_lock_irqsave(&main_dev->irqlock, flags); ctx = main_dev->curr_enc_ctx[dev->hw_id]; @@ -63,9 +66,17 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv) addr = dev->reg_base + MTK_VENC_IRQ_ACK_OFFSET; ctx->irq_status = readl(dev->reg_base + MTK_VENC_IRQ_STATUS_OFFSET); + bs_size = readl(dev->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT); + frm_type = readl(dev->reg_base + VENC_PIC_FRM_TYPE); clean_hw_irq_status(ctx->irq_status, addr); - wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0); + if (frm_type & VENC_PIC_KEY_FRM) + is_key_frame = 1; + + mtk_venc_buf_done(ctx, dev->hw_id, bs_size, 0, is_key_frame); + mtk_vcodec_enc_clock_off(main_dev, dev->hw_id); + mtk_venc_unlock(ctx, dev->hw_id); + return IRQ_HANDLED; } diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c index e9b4e95ca012..e7914da9dfa5 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c @@ -234,3 +234,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int hw_id) for (i = enc_clk->clk_num - 1; i >= 0; i--) clk_disable(enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off); diff --git a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h index a74c98aa355b..a4b19f4ef759 100644 --- a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +++ b/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h @@ -15,6 +15,7 @@ struct mtk_vcodec_mem { size_t size; void *va; dma_addr_t dma_addr; + void *buf; }; struct mtk_vcodec_fb { diff --git a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c index 1abdcc324b0e..0b28d075ac49 100644 --- a/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +++ b/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c @@ -22,7 +22,6 @@ static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc}; #define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker) -#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 /* * enum venc_h264_frame_type - h264 encoder output bitstream frame type @@ -554,6 +553,11 @@ static int h264_encode_frame(struct venc_h264_inst *inst, return ret; } + if (IS_VENC_MULTICORE(ctx->dev->enc_capability)) { + ++inst->frm_cnt; + return ret; + } + irq_status = h264_enc_wait_venc_done(inst); if (irq_status != MTK_VENC_IRQ_STATUS_FRM) { mtk_vcodec_err(inst, "irq_status=%d failed", irq_status); @@ -638,8 +642,6 @@ static int h264_enc_encode(void *handle, mtk_vcodec_debug(inst, "opt %d ->", opt); - enable_irq(ctx->dev->enc_irq); - switch (opt) { case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: { unsigned int bs_size_hdr; @@ -662,6 +664,13 @@ static int h264_enc_encode(void *handle, unsigned int bs_size_hdr; unsigned int bs_size_frm; + /* + * the frm_buf and bs_buf need to recorded into current ctx, + * when encoding done, the target buffers can be got from ctx. + */ + ctx->pfrm_buf[ctx->hw_id] = frm_buf->frm_addr; + ctx->pbs_buf[ctx->hw_id] = bs_buf->buf; + if (!inst->prepend_hdr) { ret = h264_encode_frame(inst, frm_buf, bs_buf, &result->bs_size, ctx->hw_id); @@ -696,7 +705,9 @@ static int h264_enc_encode(void *handle, if (ret) goto encode_err; - result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; + + result->bs_size = ctx->hdr_size + bs_size_frm; mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d", hdr_sz, filler_sz, bs_size_frm, @@ -715,7 +726,6 @@ static int h264_enc_encode(void *handle, encode_err: - disable_irq(ctx->dev->enc_irq); mtk_vcodec_debug(inst, "opt %d <-", opt); return ret; diff --git a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h index e676ccf6bd25..7e24b7f573d7 100644 --- a/drivers/media/platform/mediatek/vcodec/venc_drv_if.h +++ b/drivers/media/platform/mediatek/vcodec/venc_drv_if.h @@ -108,9 +108,11 @@ struct venc_frame_info { /* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses + * @frm_addr: current v4l2 buffer address */ struct venc_frm_buf { struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES]; + void *frm_addr; }; /*