From patchwork Mon Aug 16 10:59:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9BE3EC432BE for ; Mon, 16 Aug 2021 11:00:35 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8A55B61354 for ; Mon, 16 Aug 2021 11:00:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235964AbhHPLA5 (ORCPT ); Mon, 16 Aug 2021 07:00:57 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:41376 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S235941AbhHPLA4 (ORCPT ); Mon, 16 Aug 2021 07:00:56 -0400 X-UUID: 78217859e08049f8b6a909ef524d1f1a-20210816 X-UUID: 78217859e08049f8b6a909ef524d1f1a-20210816 Received: from mtkcas10.mediatek.inc [(172.21.101.39)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 1652081063; Mon, 16 Aug 2021 19:00:19 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkexhb01.mediatek.inc (172.21.101.102) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:18 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:17 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 1/9] dt-bindings: media: mtk-vcodec: Add binding for MT8195 two venc cores Date: Mon, 16 Aug 2021 18:59:26 +0800 Message-ID: <20210816105934.28265-2-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Enable MT8195 two H.264 venc cores, updates vcodec binding document. Signed-off-by: Irui Wang --- Documentation/devicetree/bindings/media/mediatek-vcodec.txt | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt index de961699ba0a..eb2e24c32426 100644 --- a/Documentation/devicetree/bindings/media/mediatek-vcodec.txt +++ b/Documentation/devicetree/bindings/media/mediatek-vcodec.txt @@ -11,6 +11,8 @@ Required properties: "mediatek,mt8173-vcodec-dec" for MT8173 decoder. "mediatek,mt8192-vcodec-enc" for MT8192 encoder. "mediatek,mt8195-vcodec-enc" for MT8195 encoder. + "mediatek,mtk-venc-core0" for MT8195 avc core0 device. + "mediatek,mtk-venc-core1" for MT8195 avc core1 device. - reg : Physical base address of the video codec registers and length of memory mapped region. - interrupts : interrupt number to the cpu. From patchwork Mon Aug 16 10:59:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 913B7C432BE for ; Mon, 16 Aug 2021 11:00:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 7C14E61BE1 for ; Mon, 16 Aug 2021 11:00:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236096AbhHPLBM (ORCPT ); Mon, 16 Aug 2021 07:01:12 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:41646 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S235938AbhHPLBH (ORCPT ); Mon, 16 Aug 2021 07:01:07 -0400 X-UUID: cbd0c93bfe41465dbc6768a4c3dcd255-20210816 X-UUID: cbd0c93bfe41465dbc6768a4c3dcd255-20210816 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 969808951; Mon, 16 Aug 2021 19:00:31 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs05n2.mediatek.inc (172.21.101.140) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:30 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:29 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 4/9] media: mtk-vcodec: Add venc power on/off interface Date: Mon, 16 Aug 2021 18:59:29 +0800 Message-ID: <20210816105934.28265-5-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Frame-racing mode need power on/off all venc available power, pm_runtime_xx helper is not appropriate called directly, add new power on/off interface for it. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 18 +--- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 9 +- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 82 +++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 + 4 files changed, 96 insertions(+), 17 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index 416f356af363..d22f9c8fd063 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -16,6 +16,7 @@ #include "mtk_vcodec_intr.h" #include "mtk_vcodec_util.h" #include "venc_drv_if.h" +#include "mtk_vcodec_enc_pm.h" #define MTK_VENC_MIN_W 160U #define MTK_VENC_MIN_H 128U @@ -801,7 +802,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) */ if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE)) { ret = -EIO; - goto err_start_stream; + goto err_set_param; } /* Do the initialization when both start_streaming have been called */ @@ -813,12 +814,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) return 0; } - ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev); - if (ret < 0) { - mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret); - goto err_start_stream; - } - mtk_venc_set_param(ctx, ¶m); ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, ¶m); if (ret) { @@ -845,11 +840,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) return 0; err_set_param: - ret = pm_runtime_put(&ctx->dev->plat_dev->dev); - if (ret < 0) - mtk_v4l2_err("pm_runtime_put fail %d", ret); - -err_start_stream: for (i = 0; i < q->num_buffers; ++i) { struct vb2_buffer *buf = vb2_get_buffer(q, i); @@ -903,10 +893,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q) if (ret) mtk_v4l2_err("venc_if_deinit failed=%d", ret); - ret = pm_runtime_put(&ctx->dev->plat_dev->dev); - if (ret < 0) - mtk_v4l2_err("pm_runtime_put fail %d", ret); - ctx->state = MTK_STATE_FREE; } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index 9c2224b199d0..b78435e8223a 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -253,6 +253,12 @@ static int fops_vcodec_open(struct file *file) mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability); } + ret = mtk_vcodec_enc_power_on(ctx); + if (ret < 0) { + mtk_v4l2_err("encoder power on failed %d", ret); + goto err_load_fw; + } + mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ", ctx->id, ctx, ctx->m2m_ctx); @@ -285,11 +291,12 @@ static int fops_vcodec_release(struct file *file) mtk_v4l2_debug(1, "[%d] encoder", ctx->id); mutex_lock(&dev->dev_mutex); + v4l2_m2m_ctx_release(ctx->m2m_ctx); mtk_vcodec_enc_release(ctx); v4l2_fh_del(&ctx->fh); v4l2_fh_exit(&ctx->fh); v4l2_ctrl_handler_free(&ctx->ctrl_hdl); - v4l2_m2m_ctx_release(ctx->m2m_ctx); + mtk_vcodec_enc_power_off(ctx); list_del_init(&ctx->list); kfree(ctx); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c index 5edeb93fca84..d9df0e3701f1 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c @@ -12,6 +12,7 @@ #include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" +#include "mtk_vcodec_enc_hw.h" int mtk_vcodec_init_enc_pm(struct platform_device *pdev, struct mtk_vcodec_pm *pm) @@ -120,3 +121,84 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm) for (i = enc_clk->clk_num - 1; i >= 0; i--) clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk); } + +int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev) +{ + int i, ret; + struct mtk_venc_comp_dev *venc_comp; + + /* + * frame_racing mode needs power on all available component devices. + */ + for (i = 0; i < MTK_VENC_HW_MAX; i++) { + venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i]; + if (!venc_comp) + return 0; + + ret = mtk_smi_larb_get(venc_comp->pm.larbvenc); + if (ret < 0) { + mtk_v4l2_err("power on core[%d] fail %d", i, ret); + goto pw_err; + } + } + return 0; + +pw_err: + for (i -= 1; i >= 0; i--) { + venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i]; + + mtk_smi_larb_put(venc_comp->pm.larbvenc); + } + + return ret; +} + +int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev) +{ + int i; + struct mtk_venc_comp_dev *venc_comp; + + /*power off all available component device*/ + for (i = 0; i < MTK_VENC_HW_MAX; i++) { + venc_comp = (struct mtk_venc_comp_dev *)dev->enc_comp_dev[i]; + if (!venc_comp) + return 0; + + mtk_smi_larb_put(venc_comp->pm.larbvenc); + } + + return 0; +} + +int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) +{ + int ret; + struct mtk_vcodec_dev *dev = ctx->dev; + + if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + ret = mtk_venc_enable_comp_hw(dev); + if (ret < 0) { + mtk_v4l2_err("enable venc comp hw fail :%d", ret); + return ret; + } + } else { + ret = mtk_smi_larb_get(dev->pm.larbvenc); + if (ret < 0) { + mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret); + return ret; + } + } + return 0; +} + +int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx) +{ + struct mtk_vcodec_dev *dev = ctx->dev; + + if (dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) + mtk_venc_disable_comp_hw(dev); + else + mtk_smi_larb_put(dev->pm.larbvenc); + + return 0; +} diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h index f4a4aa441185..2e76b858fed1 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h @@ -16,4 +16,8 @@ void mtk_vcodec_release_enc_pm(struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm); void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm); +int mtk_venc_enable_comp_hw(struct mtk_vcodec_dev *dev); +int mtk_venc_disable_comp_hw(struct mtk_vcodec_dev *dev); +int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx); +int mtk_vcodec_enc_power_off(struct mtk_vcodec_ctx *ctx); #endif /* _MTK_VCODEC_ENC_PM_H_ */ From patchwork Mon Aug 16 10:59:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497803 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1CD8AC432BE for ; Mon, 16 Aug 2021 11:00:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 0815661BE1 for ; Mon, 16 Aug 2021 11:00:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236222AbhHPLBU (ORCPT ); Mon, 16 Aug 2021 07:01:20 -0400 Received: from mailgw01.mediatek.com ([60.244.123.138]:53432 "EHLO mailgw01.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236188AbhHPLBN (ORCPT ); Mon, 16 Aug 2021 07:01:13 -0400 X-UUID: eac94a34b5254d8185808eeca63fb589-20210816 X-UUID: eac94a34b5254d8185808eeca63fb589-20210816 Received: from mtkexhb02.mediatek.inc [(172.21.101.103)] by mailgw01.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-SHA384 256/256) with ESMTP id 391395510; Mon, 16 Aug 2021 19:00:38 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs07n1.mediatek.inc (172.21.101.16) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:36 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:35 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , Tiffany Lin , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 6/9] media: mtk-vcodec: Add new venc drv interface for frame_racing mode Date: Mon, 16 Aug 2021 18:59:31 +0800 Message-ID: <20210816105934.28265-7-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Frame-racing mode encoding need more venc working buffers, it will break the compatibility if we just add venc_vsi in AP-Kernel but not in firmware, so add a new venc driver interface to distinguish the sigle_core_mode and frame_racing mode. The new driver interface can be used for different codecs in the future. Signed-off-by: Irui Wang --- drivers/media/platform/mtk-vcodec/Makefile | 1 + .../platform/mtk-vcodec/mtk_vcodec_drv.h | 1 + .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 + .../platform/mtk-vcodec/mtk_vcodec_util.h | 4 + .../platform/mtk-vcodec/venc/venc_common_if.c | 629 ++++++++++++++++++ .../media/platform/mtk-vcodec/venc_drv_if.c | 5 +- .../media/platform/mtk-vcodec/venc_drv_if.h | 1 + 7 files changed, 659 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/mtk-vcodec/venc/venc_common_if.c diff --git a/drivers/media/platform/mtk-vcodec/Makefile b/drivers/media/platform/mtk-vcodec/Makefile index 661d4afeb628..a42ce704bf04 100644 --- a/drivers/media/platform/mtk-vcodec/Makefile +++ b/drivers/media/platform/mtk-vcodec/Makefile @@ -15,6 +15,7 @@ mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc/venc_h264_if.o \ + venc/venc_common_if.o \ mtk_vcodec_enc.o \ mtk_vcodec_enc_drv.o \ mtk_vcodec_enc_pm.o \ diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index c160b9e505eb..1c6c1eca649f 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -352,6 +352,7 @@ struct mtk_vcodec_enc_pdata { }; #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext) +#define MTK_ENC_HW_MODE(ctx) ((ctx)->dev->venc_pdata->hw_mode) /** * struct mtk_vcodec_dev - driver data diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c index ac5973b6735f..5057e2cba627 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c @@ -9,6 +9,7 @@ #include "mtk_vcodec_drv.h" #include "mtk_vcodec_util.h" +#include "mtk_vcodec_enc_hw.h" /* For encoder, this will enable logs in venc/*/ bool mtk_vcodec_dbg; @@ -33,6 +34,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, } EXPORT_SYMBOL(mtk_vcodec_get_reg_addr); +void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data, + unsigned int hw_id) +{ + struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)data; + struct mtk_venc_comp_dev *venc; + + if (!data || hw_id >= MTK_VENC_HW_MAX) { + mtk_v4l2_err("Invalid arguments, hw_id=%d", hw_id); + return NULL; + } + venc = (struct mtk_venc_comp_dev *)ctx->dev->enc_comp_dev[hw_id]; + if (!venc) + return NULL; + + return venc->reg_base; +} +EXPORT_SYMBOL(mtk_get_venc_comp_reg_addr); + int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data, struct mtk_vcodec_mem *mem) { diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h index b999d7b84ed1..0027747c0a27 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h @@ -75,6 +75,10 @@ extern bool mtk_vcodec_dbg; void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, unsigned int reg_idx); + +void __iomem *mtk_get_venc_comp_reg_addr(struct mtk_vcodec_ctx *data, + unsigned int hw_id); + int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data, struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data, diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c new file mode 100644 index 000000000000..01b7d93241af --- /dev/null +++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c @@ -0,0 +1,629 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2021 MediaTek Inc. + */ + +#include "../venc_vpu_if.h" +#include "../venc_drv_base.h" +#include "../mtk_vcodec_intr.h" +#include "../mtk_vcodec_enc.h" +#include "../mtk_vcodec_drv.h" +#include "../mtk_vcodec_util.h" +#include "../mtk_vcodec_enc_hw.h" + +static const char h264_filler_marker[] = {0x0, 0x0, 0x0, 0x1, 0xc}; + +#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker) +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 + +enum venc_vpu_work_buf { + VENC_VPU_WORK_BUF_RC_INFO_CORE0, + VENC_VPU_WORK_BUF_RC_INFO_CORE1, + VENC_VPU_WORK_BUF_FR_RC_INFO, + VENC_VPU_WORK_BUF_RC_CODE, + VENC_VPU_WORK_BUF_REC_LUMA, + VENC_VPU_WORK_BUF_REC_CHROMA, + VENC_VPU_WORK_BUF_REF_LUMA, + VENC_VPU_WORK_BUF_REF_CHROMA, + VENC_VPU_WORK_BUF_MV_INFO_1, + VENC_VPU_WORK_BUF_MV_INFO_2, + VENC_VPU_WORK_BUF_SKIP_FRAME, + VENC_VPU_WORK_BUF_MAX, +}; + +enum venc_frame_type { + VENC_IDR_FRM, + VENC_I_FRM, + VENC_P_FRM, + VENC_B_FRM, +}; + +enum venc_bs_mode { + VENC_BS_MODE_SPS, + VENC_BS_MODE_PPS, + VENC_BS_MODE_FRAME, +}; + +struct venc_vpu_buf { + u32 iova; + u32 vpua; + u32 size; +}; + +struct venc_vpu_config { + u32 input_fourcc; + u32 bitrate; + u32 pic_w; + u32 pic_h; + u32 buf_w; + u32 buf_h; + u32 gop_size; + u32 intra_period; + u32 framerate; + u32 profile; + u32 level; + u32 wfd; + u32 max_qp; + u32 min_qp; + u32 reserved[8]; +}; + +struct venc_vsi { + struct venc_vpu_config config; + struct venc_vpu_buf work_bufs[VENC_VPU_WORK_BUF_MAX]; +}; + +struct venc_common_inst { + void __iomem *hw_base[MTK_VENC_HW_MAX]; + struct mtk_vcodec_mem work_bufs[VENC_VPU_WORK_BUF_MAX]; + struct mtk_vcodec_mem pps_buf; + bool work_buf_allocated; + unsigned int frm_cnt; + unsigned int skip_frm_cnt; + unsigned int prepend_hdr; + struct venc_vpu_inst vpu_inst; + struct venc_vsi *vsi; + struct mtk_vcodec_ctx *ctx; +}; + +static inline u32 venc_read_reg(struct venc_common_inst *inst, + u32 addr, int hw_id) +{ + return readl(inst->hw_base[hw_id] + addr); +} + +static unsigned int h264_get_profile(struct venc_common_inst *inst, + unsigned int profile) +{ + switch (profile) { + case V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE: + return 66; + case V4L2_MPEG_VIDEO_H264_PROFILE_MAIN: + return 77; + case V4L2_MPEG_VIDEO_H264_PROFILE_HIGH: + return 100; + case V4L2_MPEG_VIDEO_H264_PROFILE_CONSTRAINED_BASELINE: + mtk_vcodec_err(inst, "unsupported CONSTRAINED_BASELINE"); + return 0; + case V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED: + mtk_vcodec_err(inst, "unsupported EXTENDED"); + return 0; + default: + mtk_vcodec_debug(inst, "unsupported profile %d", profile); + return 100; + } +} + +static unsigned int h264_get_level(struct venc_common_inst *inst, + unsigned int level) +{ + switch (level) { + case V4L2_MPEG_VIDEO_H264_LEVEL_1B: + mtk_vcodec_err(inst, "unsupported 1B"); + return 0; + case V4L2_MPEG_VIDEO_H264_LEVEL_1_0: + return 10; + case V4L2_MPEG_VIDEO_H264_LEVEL_1_1: + return 11; + case V4L2_MPEG_VIDEO_H264_LEVEL_1_2: + return 12; + case V4L2_MPEG_VIDEO_H264_LEVEL_1_3: + return 13; + case V4L2_MPEG_VIDEO_H264_LEVEL_2_0: + return 20; + case V4L2_MPEG_VIDEO_H264_LEVEL_2_1: + return 21; + case V4L2_MPEG_VIDEO_H264_LEVEL_2_2: + return 22; + case V4L2_MPEG_VIDEO_H264_LEVEL_3_0: + return 30; + case V4L2_MPEG_VIDEO_H264_LEVEL_3_1: + return 31; + case V4L2_MPEG_VIDEO_H264_LEVEL_3_2: + return 32; + case V4L2_MPEG_VIDEO_H264_LEVEL_4_0: + return 40; + case V4L2_MPEG_VIDEO_H264_LEVEL_4_1: + return 41; + case V4L2_MPEG_VIDEO_H264_LEVEL_4_2: + return 42; + case V4L2_MPEG_VIDEO_H264_LEVEL_5_0: + return 50; + case V4L2_MPEG_VIDEO_H264_LEVEL_5_1: + return 51; + default: + mtk_vcodec_debug(inst, "unsupported level %d", level); + return 31; + } +} + +static void venc_free_work_buf(struct venc_common_inst *inst) +{ + int i; + + mtk_vcodec_debug_enter(inst); + + for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) { + if (i != VENC_VPU_WORK_BUF_SKIP_FRAME) + mtk_vcodec_mem_free(inst->ctx, &inst->work_bufs[i]); + } + + mtk_vcodec_mem_free(inst->ctx, &inst->pps_buf); + + mtk_vcodec_debug_leave(inst); +} + +static int venc_alloc_work_buf(struct venc_common_inst *inst) +{ + int i; + int ret = 0; + struct venc_vpu_buf *wb = inst->vsi->work_bufs; + + mtk_vcodec_debug_enter(inst); + + for (i = 0; i < VENC_VPU_WORK_BUF_MAX; i++) { + /* + * This 'wb' structure is set by VPU side and shared to AP for + * buffer allocation and IO virtual addr mapping. For most of + * the buffers, AP will allocate the buffer according to 'size' + * field and store the IO virtual addr in 'iova' field. There + * are two exceptions: + * (1) RC_CODE buffer, it's pre-allocated in the VPU side, and + * save the VPU addr in the 'vpua' field. The AP will translate + * the VPU addr to the corresponding IO virtual addr and store + * in 'iova' field for reg setting in VPU side. + * (2) SKIP_FRAME buffer, it's pre-allocated in the VPU side, + * and save the VPU addr in the 'vpua' field. The AP will + * translate the VPU addr to the corresponding AP side virtual + * address and do some memcpy access to move to bitstream buffer + * assigned by v4l2 layer. + */ + inst->work_bufs[i].size = wb[i].size; + if (i == VENC_VPU_WORK_BUF_SKIP_FRAME) { + struct mtk_vcodec_fw *handler; + + handler = inst->vpu_inst.ctx->dev->fw_handler; + inst->work_bufs[i].va = + mtk_vcodec_fw_map_dm_addr(handler, wb[i].vpua); + inst->work_bufs[i].dma_addr = 0; + } else { + ret = mtk_vcodec_mem_alloc(inst->ctx, + &inst->work_bufs[i]); + if (ret) { + mtk_vcodec_err(inst, + "cannot allocate buf %d", i); + goto err_alloc; + } + /* + * This RC_CODE is pre-allocated by VPU and saved in VPU + * addr. So we need use memcpy to copy RC_CODE from VPU + * addr into IO virtual addr in 'iova' field for reg + * setting in VPU side. + */ + if (i == VENC_VPU_WORK_BUF_RC_CODE) { + struct mtk_vcodec_fw *handler; + void *tmp_va; + + handler = inst->vpu_inst.ctx->dev->fw_handler; + tmp_va = mtk_vcodec_fw_map_dm_addr(handler, + wb[i].vpua); + memcpy(inst->work_bufs[i].va, tmp_va, + wb[i].size); + } + } + wb[i].iova = inst->work_bufs[i].dma_addr; + + mtk_vcodec_debug(inst, + "work_buf[%d] va=0x%p iova=%pad size=%zu", + i, inst->work_bufs[i].va, + &inst->work_bufs[i].dma_addr, + inst->work_bufs[i].size); + } + + /* the pps_buf is used by AP side only */ + inst->pps_buf.size = 128; + ret = mtk_vcodec_mem_alloc(inst->ctx, &inst->pps_buf); + if (ret) { + mtk_vcodec_err(inst, "cannot allocate pps_buf"); + goto err_alloc; + } + + mtk_vcodec_debug_leave(inst); + + return ret; + +err_alloc: + venc_free_work_buf(inst); + + return ret; +} + +static unsigned int venc_wait_comp_done(struct venc_common_inst *inst, + unsigned int hw_id) +{ + unsigned int irq_status = 0; + struct mtk_vcodec_ctx *ctx = (struct mtk_vcodec_ctx *)inst->ctx; + + if (!mtk_vcodec_wait_for_done_ctx(ctx, MTK_INST_IRQ_RECEIVED, + WAIT_INTR_TIMEOUT_MS)) { + irq_status = ctx->irq_status; + mtk_vcodec_debug(inst, "irq_status %x <-", irq_status); + } + return irq_status; +} + +static int venc_frame_type(struct venc_common_inst *inst) +{ + if ((inst->vsi->config.gop_size != 0 && + (inst->frm_cnt % inst->vsi->config.gop_size) == 0) || + (inst->frm_cnt == 0 && inst->vsi->config.gop_size == 0)) { + /* IDR frame */ + return VENC_IDR_FRM; + } else if ((inst->vsi->config.intra_period != 0 && + (inst->frm_cnt % inst->vsi->config.intra_period) == 0) || + (inst->frm_cnt == 0 && + inst->vsi->config.intra_period == 0)) { + /* I frame */ + return VENC_I_FRM; + } else { + return VENC_P_FRM; /* Note: B frames are not supported */ + } +} + +static int venc_encode_sps(struct venc_common_inst *inst, + struct mtk_vcodec_mem *bs_buf, + unsigned int *bs_size) +{ + int ret = 0; + unsigned int irq_status; + + mtk_vcodec_debug_enter(inst); + + ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL, + bs_buf, bs_size, NULL); + if (ret) + return ret; + + irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0); + if (irq_status != MTK_VENC_IRQ_STATUS_SPS) { + mtk_vcodec_err(inst, "expect irq status %d", + MTK_VENC_IRQ_STATUS_SPS); + return -EINVAL; + } + + *bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT, + MTK_VENC_CORE0); + mtk_vcodec_debug(inst, "bs size %d <-", *bs_size); + + return ret; +} + +static int venc_encode_pps(struct venc_common_inst *inst, + struct mtk_vcodec_mem *bs_buf, + unsigned int *bs_size) +{ + int ret = 0; + unsigned int irq_status; + + mtk_vcodec_debug_enter(inst); + + ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL, + bs_buf, bs_size, NULL); + if (ret) + return ret; + + irq_status = venc_wait_comp_done(inst, MTK_VENC_CORE0); + if (irq_status != MTK_VENC_IRQ_STATUS_PPS) { + mtk_vcodec_err(inst, "expect irq status %d", + MTK_VENC_IRQ_STATUS_PPS); + return -EINVAL; + } + + *bs_size = venc_read_reg(inst, VENC_PIC_BITSTREAM_BYTE_CNT, + MTK_VENC_CORE0); + mtk_vcodec_debug(inst, "bs size %d <-", *bs_size); + + return ret; +} + +static int venc_encode_header(struct venc_common_inst *inst, + struct mtk_vcodec_mem *bs_buf, + unsigned int *bs_size) +{ + int ret = 0; + unsigned int bs_size_sps; + unsigned int bs_size_pps; + + ret = venc_encode_sps(inst, bs_buf, &bs_size_sps); + if (ret) + return ret; + + ret = venc_encode_pps(inst, &inst->pps_buf, &bs_size_pps); + if (ret) + return ret; + + memcpy(bs_buf->va + bs_size_sps, inst->pps_buf.va, bs_size_pps); + *bs_size = bs_size_sps + bs_size_pps; + + return ret; +} + +static int venc_encode_frame(struct venc_common_inst *inst, + struct venc_frm_buf *frm_buf, + struct mtk_vcodec_mem *bs_buf, + unsigned int *bs_size, + int hw_id) +{ + int ret = 0; + struct venc_frame_info frame_info; + + mtk_vcodec_debug_enter(inst); + mtk_vcodec_debug(inst, "frm_cnt = %d\n ", inst->frm_cnt); + frame_info.frm_count = inst->frm_cnt; + frame_info.skip_frm_count = inst->skip_frm_cnt; + frame_info.frm_type = venc_frame_type(inst); + mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n", + frame_info.frm_count, frame_info.skip_frm_count, + frame_info.frm_type); + ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf, + bs_buf, bs_size, &frame_info); + if (ret) + return ret; + + ++inst->frm_cnt; + mtk_vcodec_debug(inst, "frm %d bs_size %d key_frm %d <-", + inst->frm_cnt, *bs_size, inst->vpu_inst.is_key_frm); + + return ret; +} + +static void h264_encode_filler(struct venc_common_inst *inst, void *buf, + int size) +{ + unsigned char *p = buf; + + if (size < H264_FILLER_MARKER_SIZE) { + mtk_vcodec_err(inst, "filler size too small %d", size); + return; + } + + memcpy(p, h264_filler_marker, ARRAY_SIZE(h264_filler_marker)); + size -= H264_FILLER_MARKER_SIZE; + p += H264_FILLER_MARKER_SIZE; + memset(p, 0xff, size); +} + +static int venc_init(struct mtk_vcodec_ctx *ctx) +{ + int i; + int ret = 0; + struct venc_common_inst *inst; + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + inst->ctx = ctx; + inst->vpu_inst.ctx = ctx; + inst->vpu_inst.id = SCP_IPI_VENC_H264; + + mtk_vcodec_debug_enter(inst); + + ret = vpu_enc_init(&inst->vpu_inst); + + inst->vsi = (struct venc_vsi *)inst->vpu_inst.vsi; + + for (i = 0; i < MTK_VENC_HW_MAX; i++) + inst->hw_base[i] = mtk_get_venc_comp_reg_addr(ctx, i); + + mtk_vcodec_debug_leave(inst); + + if (ret) + kfree(inst); + else + ctx->drv_handle = inst; + + return ret; +} + +static int venc_encode(void *handle, + enum venc_start_opt opt, + struct venc_frm_buf *frm_buf, + struct mtk_vcodec_mem *bs_buf, + struct venc_done_result *result) +{ + int ret; + struct venc_common_inst *inst = (struct venc_common_inst *)handle; + struct mtk_vcodec_ctx *ctx = inst->ctx; + + switch (opt) { + case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: { + unsigned int bs_size_hdr; + + ret = venc_encode_header(inst, bs_buf, &bs_size_hdr); + if (ret) { + mtk_vcodec_err(inst, "encode header failed: %d", ret); + return ret; + } + + result->bs_size = bs_size_hdr; + result->is_key_frm = false; + + break; + } + + case VENC_START_OPT_ENCODE_FRAME: { + int hdr_sz; + int hdr_sz_ext; + int filler_sz = 0; + const int bs_align = 128; + struct mtk_vcodec_mem tmp_bs_buf; + unsigned int bs_size_hdr; + unsigned int bs_size_frm; + + if (!inst->prepend_hdr) { + ret = venc_encode_frame(inst, frm_buf, bs_buf, + &result->bs_size, ctx->hw_id); + if (ret) { + mtk_vcodec_err(inst, "encode frame failed: %d", + ret); + return ret; + } + + result->is_key_frm = inst->vpu_inst.is_key_frm; + break; + } + + mtk_vcodec_debug(inst, "venc_encode_frame prepend SPS/PPS"); + + ret = venc_encode_header(inst, bs_buf, &bs_size_hdr); + if (ret) { + mtk_vcodec_err(inst, "encode prepend hdr failed: %d", + ret); + return ret; + } + + hdr_sz = bs_size_hdr; + hdr_sz_ext = (hdr_sz & (bs_align - 1)); + if (hdr_sz_ext) { + filler_sz = bs_align - hdr_sz_ext; + if (hdr_sz_ext + H264_FILLER_MARKER_SIZE > bs_align) + filler_sz += bs_align; + h264_encode_filler(inst, bs_buf->va + hdr_sz, + filler_sz); + } + + tmp_bs_buf.va = bs_buf->va + hdr_sz + filler_sz; + tmp_bs_buf.dma_addr = bs_buf->dma_addr + hdr_sz + filler_sz; + tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz); + + ret = venc_encode_frame(inst, frm_buf, &tmp_bs_buf, + &bs_size_frm, ctx->hw_id); + + if (ret) { + mtk_vcodec_err(inst, "encode hdr frame failed: %d", + ret); + return ret; + } + + result->bs_size = hdr_sz + filler_sz + bs_size_frm; + + mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d", + hdr_sz, filler_sz, bs_size_frm, + result->bs_size); + + inst->prepend_hdr = 0; + result->is_key_frm = inst->vpu_inst.is_key_frm; + break; + } + + default: + mtk_vcodec_err(inst, "venc_start_opt %d not supported", opt); + return -EINVAL; + } + + return ret; +} + +static int venc_set_param(void *handle, + enum venc_set_param_type type, + struct venc_enc_param *enc_prm) +{ + int ret = 0; + struct venc_common_inst *inst = (struct venc_common_inst *)handle; + + mtk_vcodec_debug(inst, "->type=%d", type); + + switch (type) { + case VENC_SET_PARAM_ENC: + inst->vsi->config.input_fourcc = enc_prm->input_yuv_fmt; + inst->vsi->config.bitrate = enc_prm->bitrate; + inst->vsi->config.pic_w = enc_prm->width; + inst->vsi->config.pic_h = enc_prm->height; + inst->vsi->config.buf_w = enc_prm->buf_width; + inst->vsi->config.buf_h = enc_prm->buf_height; + inst->vsi->config.gop_size = enc_prm->gop_size; + inst->vsi->config.framerate = enc_prm->frm_rate; + inst->vsi->config.intra_period = enc_prm->intra_period; + inst->vsi->config.profile = + h264_get_profile(inst, enc_prm->h264_profile); + inst->vsi->config.level = + h264_get_level(inst, enc_prm->h264_level); + inst->vsi->config.wfd = 0; + + ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm); + if (ret) + break; + if (inst->work_buf_allocated) { + venc_free_work_buf(inst); + inst->work_buf_allocated = false; + } + ret = venc_alloc_work_buf(inst); + if (ret) + break; + inst->work_buf_allocated = true; + break; + + case VENC_SET_PARAM_PREPEND_HEADER: + inst->prepend_hdr = 1; + mtk_vcodec_debug(inst, "set prepend header mode"); + break; + case VENC_SET_PARAM_FORCE_INTRA: + case VENC_SET_PARAM_GOP_SIZE: + case VENC_SET_PARAM_INTRA_PERIOD: + inst->frm_cnt = 0; + inst->skip_frm_cnt = 0; + fallthrough; + default: + ret = vpu_enc_set_param(&inst->vpu_inst, type, enc_prm); + break; + } + + mtk_vcodec_debug_leave(inst); + return ret; +} + +static int venc_deinit(void *handle) +{ + int ret = 0; + struct venc_common_inst *inst = (struct venc_common_inst *)handle; + + mtk_vcodec_debug_enter(inst); + + ret = vpu_enc_deinit(&inst->vpu_inst); + + if (inst->work_buf_allocated) + venc_free_work_buf(inst); + + mtk_vcodec_debug_leave(inst); + kfree(inst); + + return ret; +} + +const struct venc_common_if venc_if = { + .init = venc_init, + .encode = venc_encode, + .set_param = venc_set_param, + .deinit = venc_deinit, +}; diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c index 6cbdb7e30bb3..93936d002bed 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c @@ -25,7 +25,10 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) ctx->enc_if = &venc_vp8_if; break; case V4L2_PIX_FMT_H264: - ctx->enc_if = &venc_h264_if; + if (MTK_ENC_HW_MODE(ctx) == VENC_FRAME_RACING_MODE) + ctx->enc_if = &venc_if; + else + ctx->enc_if = &venc_h264_if; break; default: return -EINVAL; diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h index 0b04a1020873..8f384ba24ddf 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h @@ -125,6 +125,7 @@ struct venc_done_result { extern const struct venc_common_if venc_h264_if; extern const struct venc_common_if venc_vp8_if; +extern const struct venc_common_if venc_if; /* * venc_if_init - Create the driver handle From patchwork Mon Aug 16 10:59:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Irui Wang X-Patchwork-Id: 497802 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, UNPARSEABLE_RELAY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11DC6C4338F for ; Mon, 16 Aug 2021 11:01:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id ED8C460EFF for ; Mon, 16 Aug 2021 11:00:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236170AbhHPLB3 (ORCPT ); Mon, 16 Aug 2021 07:01:29 -0400 Received: from mailgw02.mediatek.com ([210.61.82.184]:41878 "EHLO mailgw02.mediatek.com" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S236156AbhHPLBQ (ORCPT ); Mon, 16 Aug 2021 07:01:16 -0400 X-UUID: 2a05ca71d8954faa89a1a0b2865355f6-20210816 X-UUID: 2a05ca71d8954faa89a1a0b2865355f6-20210816 Received: from mtkmbs10n1.mediatek.inc [(172.21.101.34)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 904561087; Mon, 16 Aug 2021 19:00:41 +0800 Received: from mtkmbs10n2.mediatek.inc (172.21.101.183) by mtkmbs02n1.mediatek.inc (172.21.101.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 16 Aug 2021 19:00:40 +0800 Received: from MTKCAS06.mediatek.inc (172.21.101.30) by mtkmbs10n2.mediatek.inc (172.21.101.183) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.2.792.3; Mon, 16 Aug 2021 19:00:39 +0800 Received: from localhost.localdomain (10.17.3.153) by MTKCAS06.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.0.1497.2 via Frontend Transport; Mon, 16 Aug 2021 19:00:38 +0800 From: Irui Wang To: Hans Verkuil , Tzung-Bi Shih , Alexandre Courbot , "Tiffany Lin" , Andrew-CT Chen , Mauro Carvalho Chehab , Rob Herring , Matthias Brugger , Tomasz Figa , Yong Wu CC: Hsin-Yi Wang , Maoguang Meng , Longfei Wang , Yunfei Dong , Fritz Koenig , Irui Wang , , , , , , , Subject: [PATCH 7/9] media: mtk-vcodec: Add frame racing mode encode process Date: Mon, 16 Aug 2021 18:59:32 +0800 Message-ID: <20210816105934.28265-8-irui.wang@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20210816105934.28265-1-irui.wang@mediatek.com> References: <20210816105934.28265-1-irui.wang@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The frame_racing mode encoding is try to use the two venc cores: frame#0 use core#0, frame#1 use core#1, frame#2 use core#0..., Lock the device and enabe the clock by used core, for sequence header encoding, it always used core#0. Signed-off-by: Irui Wang --- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 5 +- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 35 ++++---- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 6 +- .../platform/mtk-vcodec/venc/venc_common_if.c | 6 +- .../platform/mtk-vcodec/venc/venc_h264_if.c | 6 +- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 2 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 85 ++++++++++++++----- .../media/platform/mtk-vcodec/venc_drv_if.h | 4 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 11 ++- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 11 files changed, 118 insertions(+), 49 deletions(-) diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h index 1c6c1eca649f..7a9c2dd52b91 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h @@ -273,6 +273,7 @@ struct vdec_pic_info { * @decoded_frame_cnt: number of decoded frames * @lock: protect variables accessed by V4L2 threads and worker thread such as * mtk_video_dec_buf. + * @enc_idx: used to record encoded frame count */ struct mtk_vcodec_ctx { enum mtk_instance_type type; @@ -313,6 +314,8 @@ struct mtk_vcodec_ctx { int decoded_frame_cnt; struct mutex lock; + int hw_id; + int enc_idx; }; enum mtk_chip { @@ -422,7 +425,7 @@ struct mtk_vcodec_dev { int enc_irq; struct mutex dec_mutex; - struct mutex enc_mutex; + struct mutex enc_mutex[MTK_VENC_HW_MAX]; struct mtk_vcodec_pm pm; unsigned int dec_capability; diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c index d22f9c8fd063..6b37bd092fdf 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c @@ -1090,21 +1090,26 @@ static void mtk_venc_worker(struct work_struct *work) if (enc_result.is_key_frm) dst_buf->flags |= V4L2_BUF_FLAG_KEYFRAME; - if (ret) { - v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR); - dst_buf->vb2_buf.planes[0].bytesused = 0; - v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR); - mtk_v4l2_err("venc_if_encode failed=%d", ret); - } else { - v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); - dst_buf->vb2_buf.planes[0].bytesused = enc_result.bs_size; - v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); - mtk_v4l2_debug(2, "venc_if_encode bs size=%d", - enc_result.bs_size); + if (ctx->dev->venc_pdata->hw_mode == VENC_SINGLE_CORE_MODE) { + if (ret) { + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_ERROR); + dst_buf->vb2_buf.planes[0].bytesused = 0; + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR); + mtk_v4l2_err("venc_if_encode failed=%d", ret); + } else { + v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE); + dst_buf->vb2_buf.planes[0].bytesused = + enc_result.bs_size; + v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE); + mtk_v4l2_debug(2, "venc_if_encode bs size=%d", + enc_result.bs_size); + } } v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx); + ctx->enc_idx++; + mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>", src_buf->vb2_buf.index, dst_buf->vb2_buf.index, ret, enc_result.bs_size); @@ -1305,19 +1310,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, return vb2_queue_init(dst_vq); } -int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx) +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; - mutex_unlock(&dev->enc_mutex); + mutex_unlock(&dev->enc_mutex[core_id]); return 0; } -int mtk_venc_lock(struct mtk_vcodec_ctx *ctx) +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id) { struct mtk_vcodec_dev *dev = ctx->dev; - mutex_lock(&dev->enc_mutex); + mutex_lock(&dev->enc_mutex[core_id]); return 0; } diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h index 30b6d7f22ed7..8d486f532025 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h @@ -41,8 +41,8 @@ struct mtk_video_enc_buf { extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops; extern const struct v4l2_m2m_ops mtk_venc_m2m_ops; -int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx); -int mtk_venc_lock(struct mtk_vcodec_ctx *ctx); +int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id); +int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id); int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, struct vb2_queue *dst_vq); void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx); diff --git a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c index b78435e8223a..ff1f05a5ca28 100644 --- a/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +++ b/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c @@ -320,7 +320,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) struct resource *res; phandle rproc_phandle; enum mtk_vcodec_fw_type fw_type; - int ret; + int ret, i; struct component_match *match = NULL; dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL); @@ -384,7 +384,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } } - mutex_init(&dev->enc_mutex); + for (i = 0; i < MTK_VENC_HW_MAX; i++) + mutex_init(&dev->enc_mutex[i]); + mutex_init(&dev->dev_mutex); spin_lock_init(&dev->irqlock); diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c index 01b7d93241af..1580bd1beaac 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_common_if.c @@ -300,7 +300,7 @@ static int venc_encode_sps(struct venc_common_inst *inst, mtk_vcodec_debug_enter(inst); ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_SPS, NULL, - bs_buf, bs_size, NULL); + bs_buf, bs_size, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -328,7 +328,7 @@ static int venc_encode_pps(struct venc_common_inst *inst, mtk_vcodec_debug_enter(inst); ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_PPS, NULL, - bs_buf, bs_size, NULL); + bs_buf, bs_size, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -386,7 +386,7 @@ static int venc_encode_frame(struct venc_common_inst *inst, frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); ret = vpu_enc_encode(&inst->vpu_inst, VENC_BS_MODE_FRAME, frm_buf, - bs_buf, bs_size, &frame_info); + bs_buf, bs_size, &frame_info, hw_id); if (ret) return ret; diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c index b6a4f2074fa5..a90b06a3fe5a 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c @@ -368,7 +368,7 @@ static int h264_encode_sps(struct venc_h264_inst *inst, mtk_vcodec_debug_enter(inst); ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, - bs_buf, bs_size, NULL); + bs_buf, bs_size, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -395,7 +395,7 @@ static int h264_encode_pps(struct venc_h264_inst *inst, mtk_vcodec_debug_enter(inst); ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, - bs_buf, bs_size, NULL); + bs_buf, bs_size, NULL, MTK_VENC_CORE0); if (ret) return ret; @@ -452,7 +452,7 @@ static int h264_encode_frame(struct venc_h264_inst *inst, frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, - bs_buf, bs_size, &frame_info); + bs_buf, bs_size, &frame_info, MTK_VENC_CORE0); if (ret) return ret; diff --git a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c index 8267a9c4fd25..a68bf53d7d81 100644 --- a/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c +++ b/drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c @@ -303,7 +303,7 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst, mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt); ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, bs_size, - NULL); + NULL, MTK_VENC_CORE0); if (ret) return ret; diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.c b/drivers/media/platform/mtk-vcodec/venc_drv_if.c index 93936d002bed..09fcec05a655 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.c @@ -15,6 +15,7 @@ #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_pm.h" +#include "mtk_vcodec_enc_hw.h" int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) { @@ -34,9 +35,9 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) return -EINVAL; } - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, 0); ret = ctx->enc_if->init(ctx); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, 0); return ret; } @@ -46,9 +47,9 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx, { int ret = 0; - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, 0); ret = ctx->enc_if->set_param(ctx->drv_handle, type, in); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, 0); return ret; } @@ -59,24 +60,12 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, struct venc_done_result *result) { int ret = 0; - unsigned long flags; - - mtk_venc_lock(ctx); - spin_lock_irqsave(&ctx->dev->irqlock, flags); - ctx->dev->curr_ctx = ctx; - spin_unlock_irqrestore(&ctx->dev->irqlock, flags); - - mtk_vcodec_enc_clock_on(ctx->dev, 0); + venc_encode_prepare(ctx, opt); ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf, bs_buf, result); - mtk_vcodec_enc_clock_off(ctx->dev, 0); - - spin_lock_irqsave(&ctx->dev->irqlock, flags); - ctx->dev->curr_ctx = NULL; - spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + venc_encode_unprepare(ctx, opt); - mtk_venc_unlock(ctx); return ret; } @@ -87,11 +76,67 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx) if (!ctx->drv_handle) return 0; - mtk_venc_lock(ctx); + mtk_venc_lock(ctx, 0); ret = ctx->enc_if->deinit(ctx->drv_handle); - mtk_venc_unlock(ctx); + mtk_venc_unlock(ctx, 0); ctx->drv_handle = NULL; return ret; } + +void venc_encode_prepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt) +{ + unsigned long flags; + struct mtk_venc_comp_dev *venc; + + if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + if (ctx->enc_idx & 0x01) + ctx->hw_id = MTK_VENC_CORE1; + else + ctx->hw_id = MTK_VENC_CORE0; + } else { + ctx->hw_id = MTK_VENC_CORE0; + } + mtk_venc_lock(ctx, ctx->hw_id); + + spin_lock_irqsave(&ctx->dev->irqlock, flags); + + if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + venc = ctx->dev->enc_comp_dev[ctx->hw_id]; + + venc->curr_ctx = ctx; + } else { + ctx->dev->curr_ctx = ctx; + } + + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + + mtk_vcodec_enc_clock_on(ctx->dev, ctx->hw_id); +} + +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt) +{ + unsigned long flags; + struct mtk_venc_comp_dev *venc; + + /*clock off and unlock after irq done*/ + if (ctx->dev->venc_pdata->hw_mode == VENC_FRAME_RACING_MODE) { + if (opt == VENC_START_OPT_ENCODE_SEQUENCE_HEADER) { + mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id); + spin_lock_irqsave(&ctx->dev->irqlock, flags); + venc = ctx->dev->enc_comp_dev[ctx->hw_id]; + venc->curr_ctx = NULL; + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + mtk_venc_unlock(ctx, ctx->hw_id); + } + } else { + mtk_vcodec_enc_clock_off(ctx->dev, ctx->hw_id); + spin_lock_irqsave(&ctx->dev->irqlock, flags); + ctx->dev->curr_ctx = NULL; + spin_unlock_irqrestore(&ctx->dev->irqlock, flags); + mtk_venc_unlock(ctx, ctx->hw_id); + } +} diff --git a/drivers/media/platform/mtk-vcodec/venc_drv_if.h b/drivers/media/platform/mtk-vcodec/venc_drv_if.h index 8f384ba24ddf..7bd1f437d8e3 100644 --- a/drivers/media/platform/mtk-vcodec/venc_drv_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_drv_if.h @@ -168,4 +168,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, struct mtk_vcodec_mem *bs_buf, struct venc_done_result *result); +void venc_encode_prepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt); +void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx, + enum venc_start_opt opt); #endif /* _VENC_DRV_IF_H_ */ diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c index be6d8790a41e..5bee5f1828f1 100644 --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.c +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.c @@ -226,9 +226,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, struct venc_frm_buf *frm_buf, struct mtk_vcodec_mem *bs_buf, unsigned int *bs_size, - struct venc_frame_info *frame_info) + struct venc_frame_info *frame_info, + int hw_id) { const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx); + const int hw_mode = MTK_ENC_HW_MODE(vpu->ctx); + size_t msg_size = is_ext ? sizeof(struct venc_ap_ipi_msg_enc_ext) : sizeof(struct venc_ap_ipi_msg_enc); @@ -262,6 +265,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, out.data[1] = frame_info->skip_frm_count; out.data[2] = frame_info->frm_type; } + + if (hw_mode == VENC_FRAME_RACING_MODE) { + out.data_item = 4; + out.data[3] = hw_id; + } + if (vpu_enc_send_msg(vpu, &out, msg_size)) { mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail", bs_mode); diff --git a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h index f9be9cab7ff7..e55656781f6e 100644 --- a/drivers/media/platform/mtk-vcodec/venc_vpu_if.h +++ b/drivers/media/platform/mtk-vcodec/venc_vpu_if.h @@ -46,7 +46,8 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, struct venc_frm_buf *frm_buf, struct mtk_vcodec_mem *bs_buf, unsigned int *bs_size, - struct venc_frame_info *frame_info); + struct venc_frame_info *frame_info, + int hw_id); int vpu_enc_deinit(struct venc_vpu_inst *vpu); #endif