From patchwork Thu Apr 29 13:28:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 429255 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp1580373jao; Thu, 29 Apr 2021 06:28:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxAGWYnQyYDK33DO8nAzIxSE5pzNordst1v3WoO4rfjOWJDjLeTCrtjwp42AuIgMJ/Rd8+i X-Received: by 2002:a63:b515:: with SMTP id y21mr32745276pge.253.1619702933575; Thu, 29 Apr 2021 06:28:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619702933; cv=none; d=google.com; s=arc-20160816; b=stnToSDcTrQDDWjC3bUvJr9jtX8GH35Dg3LsBDPePHgrG362XZm0U5xMdK3U2VkIAz cfbIghn6V0vabjki9ru6+5QlkHmVzbD84GMQmIjQy2SUMThDDsF0prrCzVulLcRq2NLA E4U2PDrnxVuKMgIg9hu2e5jPRYc31xfgXJ7+vltHjgRbLKSVBfhDrJ6w/+mQlmjOjqxy pRDSZIlSeOMvrXZXOE+Y0nFAYRtLcOBoo0TDGBpdh0mE7ptb2qH0J3teOfFS9KTTS74H 5t0sWxvq7U+9n1fB/ApVeW9et+642fBHoIgwRSJIRFIagXTSvaGFojIcauqivItXQRrH ghLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=ryhh0/XlJ3AbDBsHZICjqkSID/+spC7zQgSt9QUY5NE=; b=rTvtfxKiaXdHurXh+peAG06Z5wNKemiJaZKgYdvEcFmva2aOwab4wOjXNGFC5P46Ai MArQ7NUlETrLNV7HUADE0gyo+edqCa6cVzXTjJdaz6q3unLhCs8qSyKgO+OOvbtji7lH Gs5Srap6y2FBhXvIwWrFqSwl65QWUQkI5U+vTR6WrjOSCJnA+ckWu2vyBIJImbEAaECh M4RuNzj8ozbGRKyUdCWmjRQv5KmaTAqIzRPlmmxjbdY5mm0rd+35NsL6ZdyA6b0hoCsa bgkXA04IWJlBZrJLEuUxWme3jSMQ/9M6X6/SndoPtO0CSgLgiHsIvj0UbRvuH56IhzbI mWkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xX0MmYQ9; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a10si275495pgh.377.2021.04.29.06.28.53; Thu, 29 Apr 2021 06:28:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xX0MmYQ9; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237302AbhD2N3e (ORCPT + 4 others); Thu, 29 Apr 2021 09:29:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236635AbhD2N3d (ORCPT ); Thu, 29 Apr 2021 09:29:33 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B4FE6C06138B for ; Thu, 29 Apr 2021 06:28:46 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id z6so5676510wrm.4 for ; Thu, 29 Apr 2021 06:28:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ryhh0/XlJ3AbDBsHZICjqkSID/+spC7zQgSt9QUY5NE=; b=xX0MmYQ9zNKEXCV1CVjGtNmmzO7Yycr/LRdCnQecFgH3MSu50bUKiwZH1EReMzHQWw JCcy8B3GiSFLEADrZiE5F2UUi/EqWvkXrxu3vTBUq4c3iGp2IFo6N6eAshhUaHfa63T0 kj3aZR3S9OPCXtvHDsuNb6lUJUmExfy/Mx8e4Uuy1V2sAjNeysUCzNLbZysmCuEGxy67 H/sCdZxhGZ9caUzR9tRo2pYx13/DIXnnTYR+EYB0sR7WjF/Xj461R6U+pw7yFHAxzHTl AUxxRZSLW9iIimgO2/xgjlN7wz2kd5VZR2cBuZJBdK8c23czS1rtXdO9jHbVxA227c+W Xocg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ryhh0/XlJ3AbDBsHZICjqkSID/+spC7zQgSt9QUY5NE=; b=bC2to0zIUa7qMyZyoidu6QXJp9HOtsp8mo3udkwFlmR5n8bPiH/tq65l/mtBSYRoGQ b/VncvLkkufWpieOrHvL9Px/yEgfZKl/wJlJVbPF5vnQ3S0irqfhH/du1u4d+aeGOZY2 eqvnVGkG+Ay8mfC2ii3Kva3UCydXmsrwhIYDBFe3orXgW8tFBo0xVR9WldM3JGWi/tFg DQpwlPKVZdw80r6aukrx1C+OwfLTuvcs8zYVRuxd+ylz83ZLUeMRHOvFn5OmTh4Huns+ 7CTnAfaclhrKeorQfg7MfJvhqXeRdWHKlFpzQrGs+FpYVuYv7U4skmUwPGq8yRd7TXpz 3dMw== X-Gm-Message-State: AOAM531NLYl0imIOhkF5JSCllEkbZOBiHeiJ6Eaunxz+8PBhrezE3bKL hp55NrGgozQnDP5+94Oq1eiAEfAxvZxQ4L3p X-Received: by 2002:a5d:6787:: with SMTP id v7mr34870624wru.386.1619702925323; Thu, 29 Apr 2021 06:28:45 -0700 (PDT) Received: from localhost.localdomain (hst-221-38.medicom.bg. [84.238.221.38]) by smtp.gmail.com with ESMTPSA id n10sm4811462wrw.37.2021.04.29.06.28.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 06:28:45 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org Cc: Hans Verkuil , Stanimir Varbanov Subject: [RFC/WIP 1/4] media: Add HEIC compressed pixel format Date: Thu, 29 Apr 2021 16:28:30 +0300 Message-Id: <20210429132833.2802390-2-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> References: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add HEIC (High-Efficiency Image Container) pixel format. This an image container which use HEVC codec to encoded images. Signed-off-by: Stanimir Varbanov --- .../userspace-api/media/v4l/pixfmt-compressed.rst | 12 ++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 1 + include/uapi/linux/videodev2.h | 1 + 3 files changed, 14 insertions(+) -- 2.25.1 diff --git a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst index ba6c0c961204..246bff90dcac 100644 --- a/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst +++ b/Documentation/userspace-api/media/v4l/pixfmt-compressed.rst @@ -186,6 +186,18 @@ Compressed Formats If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM`` then the decoder has no requirements since it can parse all the information from the raw bytestream. + * .. _V4L2-PIX-FMT-HEIC: + + - ``V4L2_PIX_FMT_HEIC`` + - 'HEIC' + - High Efficiency Image Container is an image container file format which + uses HEVC encoding and it is a variant of HEIF (High Efficiency Image File) + format. + The decoder expects one Access Unit per buffer. + The encoder generates one Access Unit per buffer. + If :ref:`VIDIOC_ENUM_FMT` reports ``V4L2_FMT_FLAG_CONTINUOUS_BYTESTREAM`` + then the decoder has no requirements since it can parse all the + information from the raw bytestream. * .. _V4L2-PIX-FMT-HEVC-SLICE: - ``V4L2_PIX_FMT_HEVC_SLICE`` diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 31d1342e61e8..3a1b4c3a76c8 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1454,6 +1454,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_S5C_UYVY_JPG: descr = "S5C73MX interleaved UYVY/JPEG"; break; case V4L2_PIX_FMT_MT21C: descr = "Mediatek Compressed Format"; break; case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break; + case V4L2_PIX_FMT_HEIC: descr = "HEIC Image Format"; break; default: if (fmt->description[0]) return; diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 79dbde3bcf8d..2153b5c31d46 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -699,6 +699,7 @@ struct v4l2_pix_format { #define V4L2_PIX_FMT_FWHT v4l2_fourcc('F', 'W', 'H', 'T') /* Fast Walsh Hadamard Transform (vicodec) */ #define V4L2_PIX_FMT_FWHT_STATELESS v4l2_fourcc('S', 'F', 'W', 'H') /* Stateless FWHT (vicodec) */ #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */ +#define V4L2_PIX_FMT_HEIC v4l2_fourcc('H', 'E', 'I', 'C') /* HEIC HEVC image format */ /* Vendor-specific formats */ #define V4L2_PIX_FMT_CPIA1 v4l2_fourcc('C', 'P', 'I', 'A') /* cpia1 YUV */ From patchwork Thu Apr 29 13:28:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 429256 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp1580380jao; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwjQ2qkn9AnlH/WbUxHjlTHCAyVla1KWMMvl5bpQo+uJMBGIv4sL20uS4Jjhh1wyEVpyz1n X-Received: by 2002:aa7:838d:0:b029:221:cd7d:90d8 with SMTP id u13-20020aa7838d0000b0290221cd7d90d8mr32640119pfm.61.1619702933878; Thu, 29 Apr 2021 06:28:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619702933; cv=none; d=google.com; s=arc-20160816; b=aNmERw49cuLkCB6h1DoD5dngKZPwbSS0F/ZTlL0OCqqiQ/c7IMQXTRrNRL1nOEGij8 en9Q/6UKF7Q7e8J08BRai0OOv/A/OlXJG4psGrD/k+lWHpGHcgLXP4A30cXl0MjHBbBw Ec1zC+83n31SrGLJccU5ZmAN66JUa+dHx4E7p9+L3jmTJNEYJKBuBT2ifvX6gwVKDI/z Atdf7eoBwXhaWAHWdcXwj9IBnBDi5rtou83tkLDu1R6zq8reC+r3/hrz5jD1vIauEzqV X89mNoPU2i0Z+WDrwUBs6I93e4ph9GBlL/yfsmrGK1ujT4QxY8tkKFs+LBdol5oRkvVV jzzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qNUeNwA8nY60aEA3aVaI9L7S03k8N0gBbtn34g0NOKs=; b=PWstAUALrOT7nbFTl3MqaAwl0F36IgIbi9+1mFolNen1oITkMp5HsV4Rc/Fa7wgqJN jqftfCS/v6ILilD6okTn6ijlTbvPRovUd5V8pCX0BIS1QDown0sRMqTUUSHlEMSkmOIi oSSJTTOn038dv6dazcQmGWuFYKE+njWXQslC9HeE54pX66R7auoFFR00gq+xim8KOeCY CKUsy4irlbqzSVSPjfhoxZUss3r2o65h+6+9OshxmQ2TR54RrMs6xMNkcOhLLgCgj59v zEjkzBneXhJe8WEzjnqtvz8aRWW/RTc24f1Wn8KWn6jOLTTix2jEQxeJFPzi4wcC1CWb 7pGQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bFqRGON3; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a10si275495pgh.377.2021.04.29.06.28.53; Thu, 29 Apr 2021 06:28:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bFqRGON3; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237335AbhD2N3f (ORCPT + 4 others); Thu, 29 Apr 2021 09:29:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55040 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236635AbhD2N3f (ORCPT ); Thu, 29 Apr 2021 09:29:35 -0400 Received: from mail-wr1-x434.google.com (mail-wr1-x434.google.com [IPv6:2a00:1450:4864:20::434]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5BF73C06138B for ; Thu, 29 Apr 2021 06:28:47 -0700 (PDT) Received: by mail-wr1-x434.google.com with SMTP id m9so54220825wrx.3 for ; Thu, 29 Apr 2021 06:28:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=qNUeNwA8nY60aEA3aVaI9L7S03k8N0gBbtn34g0NOKs=; b=bFqRGON3t7Z00rpCFWYJtBn7zFtU/v29Pd6a69EusoEdkP42dwWrQd06jvDrWSayJK e8eEmr1AFEOhXkqiXJo7A/Wucs/ylMbSOtVehr8Y0tEdInZlwjhtfpY/rFukAc8swRpn xMMi5eSnec/w+wLfQzXQeG395ZHBhhpW01hpVLITi2zjTD5fAWum5wG3dXUl+s4UIH4t ectIo38QY3EmWDTVlKxAuQQOqJlz/PRlFo3ZhCa/j0u/oPIO4hwRRbDenquGoFOltj9f pRwsVuaCmqaIEjNKOcNFdIsYZHRAtsNHLTzF9znMkaYgNMsPQQMflpzHMY/KwCmaePO1 QG0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=qNUeNwA8nY60aEA3aVaI9L7S03k8N0gBbtn34g0NOKs=; b=F9kyASbBNkPSA/tfunubBTA1EyTV/RzN1cdrv6irVONvB6p5lFPNlr5SnWFeDNJ3Hx cG2hNyPNpZe1W8nmWhVuTXkWGl325eJGp1Oq7GVdzDlrMopzKaqGAy/yDWVX6/Ew05ba EAaLxGIFgesZn24Qp0M10RYDbUSTkM8KI8wLTLUj6y6tXJfJDLe8GDa+632y0fS84om0 gwrvYm7XxHzs98mQVMGy0mldFvmLlEmZf+jk5BUrKLqKBfVyob3eHivNQ1cxFaiqhvgE Qrvninr7G/rmxZAKWb76PqaqkT1J+MkkT6KdQOPkGPKCYLumxyMdw9QrDLNxBvhl0a3H v6Jg== X-Gm-Message-State: AOAM530ksbLrT+OuYCYLgAPZEKf28WgxSYc5FkmXUL2iKny4d4gQrIQz BOROXpKacLbujaUi/e6aGD6TbmN5dnwNb+qt X-Received: by 2002:adf:dd52:: with SMTP id u18mr10780042wrm.32.1619702925979; Thu, 29 Apr 2021 06:28:45 -0700 (PDT) Received: from localhost.localdomain (hst-221-38.medicom.bg. [84.238.221.38]) by smtp.gmail.com with ESMTPSA id n10sm4811462wrw.37.2021.04.29.06.28.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 06:28:45 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org Cc: Hans Verkuil , Stanimir Varbanov Subject: [RFC/WIP 2/4] v4l2-ctrls: Add HEIC grid size control Date: Thu, 29 Apr 2021 16:28:31 +0300 Message-Id: <20210429132833.2802390-3-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> References: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add a control to set HEIC image grid size. This sets a grid (partition) size for image encoding. It will be used to instruct image encoder to produce CAPTURE buffers for any grid of the input image, and it is applicable when the input YUV buffer resolution is bigger then the hardware can support. Signed-off-by: Stanimir Varbanov --- drivers/media/v4l2-core/v4l2-ctrls.c | 3 +++ include/uapi/linux/v4l2-controls.h | 1 + 2 files changed, 4 insertions(+) -- 2.25.1 diff --git a/drivers/media/v4l2-core/v4l2-ctrls.c b/drivers/media/v4l2-core/v4l2-ctrls.c index ca50e21e2838..6b443f946906 100644 --- a/drivers/media/v4l2-core/v4l2-ctrls.c +++ b/drivers/media/v4l2-core/v4l2-ctrls.c @@ -1034,6 +1034,8 @@ const char *v4l2_ctrl_get_name(u32 id) case V4L2_CID_MPEG_VIDEO_HEVC_DECODE_MODE: return "HEVC Decode Mode"; case V4L2_CID_MPEG_VIDEO_HEVC_START_CODE: return "HEVC Start Code"; + case V4L2_CID_MPEG_VIDEO_HEIC_GRID_SIZE: return "HEIC Grid Size"; + /* CAMERA controls */ /* Keep the order of the 'case's the same as in v4l2-controls.h! */ case V4L2_CID_CAMERA_CLASS: return "Camera Controls"; @@ -1282,6 +1284,7 @@ void v4l2_ctrl_fill(u32 id, const char **name, enum v4l2_ctrl_type *type, case V4L2_CID_MPEG_VIDEO_MV_H_SEARCH_RANGE: case V4L2_CID_MPEG_VIDEO_MV_V_SEARCH_RANGE: case V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY: + case V4L2_CID_MPEG_VIDEO_HEIC_GRID_SIZE: *type = V4L2_CTRL_TYPE_INTEGER; break; case V4L2_CID_MPEG_VIDEO_FORCE_KEY_FRAME: diff --git a/include/uapi/linux/v4l2-controls.h b/include/uapi/linux/v4l2-controls.h index 6f8c08507bf4..6ae6cf3c815c 100644 --- a/include/uapi/linux/v4l2-controls.h +++ b/include/uapi/linux/v4l2-controls.h @@ -800,6 +800,7 @@ enum v4l2_mpeg_video_frame_skip_mode { #define V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY (V4L2_CID_CODEC_BASE + 653) #define V4L2_CID_MPEG_VIDEO_DEC_DISPLAY_DELAY_ENABLE (V4L2_CID_CODEC_BASE + 654) +#define V4L2_CID_MPEG_VIDEO_HEIC_GRID_SIZE (V4L2_CID_CODEC_BASE + 655) /* MPEG-class control IDs specific to the CX2341x driver as defined by V4L2 */ #define V4L2_CID_CODEC_CX2341X_BASE (V4L2_CTRL_CLASS_CODEC | 0x1000) From patchwork Thu Apr 29 13:28:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 429257 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp1580387jao; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz4lg0EghCYrAFcLK5YAPZ23/soY0KCmGjzwFMdvMwFQVBDpVtBjCO3nmp4lQn3KgiSuXyb X-Received: by 2002:a17:90a:fb57:: with SMTP id iq23mr9271688pjb.150.1619702934236; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619702934; cv=none; d=google.com; s=arc-20160816; b=PRqGtrndxa/O8iQySQMEU9G+p93Z+oi7kHyGTpnGhQnK/f6IdrVMacHQyjojlIf7jl Gr5BhI+Ci5ATHxuyc9UoxTiNkCZbo4phl7CC2QQM+7LUtIjfzZYJCPaj34Cfx91uUsCQ b4RipmPobmwHS5mQIseuto+T9MVj3EL1HDwJQLyhhvOuKCIXwIUzCvT6qy/0/oTdSYcf Vk2SvX0SmwXa0tkcoHMuKBdVR34t25Ojb7ijfMsGM5xzynVc7N12Y/c1lFC4LaSDDbFs niBBZiNOPxo4EzghvffJTh/wwgoffHdJ1UiABZ6AybFw1h3yLHWgHk4ALEp2/Gc7Shr6 aGIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=/DN5H6mB286RrlpmsoTsbcLl9bW682ZJpzMoaSdcrUc=; b=Zeo3+0w4O9s8cruMWkcYEFqIuHeENWVP6Non82sD8Td8V8RkPgXHsPwbpg8Ds4Fwj/ 4CjAl9noRQlDq77ZxO4xDayPa/ZCS8mJgjLvxWqbXEdfYQJJ7sXhO7M4lOZyrX1GY1iL 6JVfDc1uosrvPygzphHIqSOqkBAAlxiDyFV9/kQ2AxaKNkbF2ap5Sq7vGu5RWYhmBbsn boc5VqdV/F51w58sxJVZ/fcmuL1tu7O9wVySjUFHMw93XhkIgiuDvo/0DtW4dj/kYYSP nrqmg4COaRr2whPbBBlQsuaXNJlVVvqf12lwF4/2fE22l/fyOsaA5Hk+TduMEH0IItxu 5ygw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GRF2czyY; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a10si275495pgh.377.2021.04.29.06.28.54; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GRF2czyY; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237362AbhD2N3g (ORCPT + 4 others); Thu, 29 Apr 2021 09:29:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55042 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236635AbhD2N3f (ORCPT ); Thu, 29 Apr 2021 09:29:35 -0400 Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37B1DC06138C for ; Thu, 29 Apr 2021 06:28:48 -0700 (PDT) Received: by mail-wr1-x42b.google.com with SMTP id l2so14646597wrm.9 for ; Thu, 29 Apr 2021 06:28:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/DN5H6mB286RrlpmsoTsbcLl9bW682ZJpzMoaSdcrUc=; b=GRF2czyYK7IRjfQzIepe9WtvcoJ+QRO9Wb52H9yRGZVh94OZ+JkKcYLjbIM6q9belu sc5zMe8sPkGnlB7DB9gCpCygTZKIH0rDtwWQBXKPAV5SYeEffpkV3KInXD6LWluKYJkp dBmQfdj5mKc1E9BxpjGe6f/lu031jYy75BmCD+GwSCmx8fUCB2ZUwFeQz2FpOnI/uM3Z kGf754LiMx8PkJY3ThRhd4mrYS/QkJGA7F2raI36yWQHO5qQo5IFC+XmTAWPjXgUSbmU FVvM/ruinikEn7Lqj7xl3lJT4FTTnr9VZHg6weOxLD/J1fKMzXztkAKwtCCU6IMCj0K/ LLeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/DN5H6mB286RrlpmsoTsbcLl9bW682ZJpzMoaSdcrUc=; b=WV5DVEKN2MYb/cAc7GxlGebtwhJAEKIaCV/yUQFSh/DdEaj3duA17DUBMMN27jVglB 2McZfa9CuQL8Esm8gKUb+8BcEUN3vU30nuJTo2SZGSdnEM1qLLGpxRUthBuJkujCyueF pvARc7FHieozevxCqJmqayYXkVpHDcM9N401TEZgw0YggoDALnWt8aVOtc2C//5ymKA3 sVXX0CpQlbvteteHaIB5RBlzkLPs0o3xPLNt+sUsZ+e+jj6Q3vHT5JsBSBogBDIDQ0gB 4by9rZZx6EYZ6dqz8ddhHufDWca/jhJMWhzaWMFioPsbyAU5OmqYyPmtn57nO0U4gVfs ZyOw== X-Gm-Message-State: AOAM533s2PNZDUu885sEZc119dX5oq8ot1VsBenl0/vgzSZuOyyeNjoQ SYehkUqr0RYiZIRXfSY0ZvyupM+v+yTIBOTO X-Received: by 2002:adf:e491:: with SMTP id i17mr42641390wrm.314.1619702926671; Thu, 29 Apr 2021 06:28:46 -0700 (PDT) Received: from localhost.localdomain (hst-221-38.medicom.bg. [84.238.221.38]) by smtp.gmail.com with ESMTPSA id n10sm4811462wrw.37.2021.04.29.06.28.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 06:28:46 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org Cc: Hans Verkuil , Stanimir Varbanov Subject: [RFC/WIP 3/4] venus: helpers: Add a new helper for buffer processing Date: Thu, 29 Apr 2021 16:28:32 +0300 Message-Id: <20210429132833.2802390-4-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> References: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org The new helper will be used from encoder and decoder drivers to enqueue buffers for processing by firmware. Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/helpers.c | 20 ++++++++++++++++++++ drivers/media/platform/qcom/venus/helpers.h | 1 + 2 files changed, 21 insertions(+) -- 2.25.1 diff --git a/drivers/media/platform/qcom/venus/helpers.c b/drivers/media/platform/qcom/venus/helpers.c index dc8ef13d0c95..8e7f2ec0cd01 100644 --- a/drivers/media/platform/qcom/venus/helpers.c +++ b/drivers/media/platform/qcom/venus/helpers.c @@ -1450,6 +1450,26 @@ void venus_helper_vb2_buf_queue(struct vb2_buffer *vb) } EXPORT_SYMBOL_GPL(venus_helper_vb2_buf_queue); +void venus_helper_process_buf(struct vb2_buffer *vb) +{ + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue); + int ret; + + cache_payload(inst, vb); + + if (vb2_start_streaming_called(vb->vb2_queue)) { + ret = is_buf_refed(inst, vbuf); + if (ret) + return; + + ret = session_process_buf(inst, vbuf); + if (ret) + return_buf_error(inst, vbuf); + } +} +EXPORT_SYMBOL_GPL(venus_helper_process_buf); + void venus_helper_buffers_done(struct venus_inst *inst, unsigned int type, enum vb2_buffer_state state) { diff --git a/drivers/media/platform/qcom/venus/helpers.h b/drivers/media/platform/qcom/venus/helpers.h index e6269b4be3af..c7cea33e2803 100644 --- a/drivers/media/platform/qcom/venus/helpers.h +++ b/drivers/media/platform/qcom/venus/helpers.h @@ -19,6 +19,7 @@ void venus_helper_buffers_done(struct venus_inst *inst, unsigned int type, int venus_helper_vb2_buf_init(struct vb2_buffer *vb); int venus_helper_vb2_buf_prepare(struct vb2_buffer *vb); void venus_helper_vb2_buf_queue(struct vb2_buffer *vb); +void venus_helper_process_buf(struct vb2_buffer *vb); void venus_helper_vb2_stop_streaming(struct vb2_queue *q); int venus_helper_vb2_start_streaming(struct venus_inst *inst); void venus_helper_m2m_device_run(void *priv); From patchwork Thu Apr 29 13:28:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stanimir Varbanov X-Patchwork-Id: 429258 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp1580395jao; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJykdA0U/yF8PZkxnWyvVw3e8DSwlwYo+EyhBqoPDufTQ2GivBoenYWg5L2/Cj9yyALqGek+ X-Received: by 2002:a17:902:ea0c:b029:eb:7b6:13ba with SMTP id s12-20020a170902ea0cb02900eb07b613bamr34874506plg.25.1619702934587; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619702934; cv=none; d=google.com; s=arc-20160816; b=OjTJPxkrntzVAl+ktyW7Mdz/kiYH1XU/D8ga77K5eL+vzhkIiaNA/1zrHmDGDZakUM bfBff5CqIqm2ij7B+ug92kasgLX1hKxC6chXHn1s96wwfoty5q2ApVKBYpRd/dFBnaTS zAUgXFDlxpEeKprHKJw47gaGcYp+aUrjbOCE/7G9x4XJB5wR/V75wTkx7vNoJcL4aX4m oaztSQFdqU+mafmuYZPXGZQ82BXjmUkIaNL7hc/TJY2Aq1T3MSOorROAQl3tOqxqFgKq yBgL9x46o72DRHf4LShtJXp6p+a+tGnjQf3CGNdeuOKMPdg2zaeNEKnvHd83Vh3S8BCa KHCA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=Oy1qNwnuHeibPjIkP+hgeYDcQYApY+KrxCJjPlibiZU=; b=VOeETlWfMV3KTUeBPJ1Wf2dxq84Vi5oPbvyzhYM9L8W+AcYLTYkHQo5RaxlTs5fqNh +WAO3LmMrXuYeMCL906C2ooHG+3g/Cpn2YTp6nIGghOymwRkdvK/OgdBCy9kR96uzRfA 6U956Zg/oIdyUe4ArOU2OtN9BFsGoYsiIRdYPdA1wJRLXuO/G6cfbf4uWC0HBhykVWN6 NDB83iSG41VC+iuJL8FsXmDqqDuw8/dFXicNs6AB3hmp31atZtyqs++2sqXi4Nweo8JV T7rdfa5zOPlDFzwqhVco4L46K97lAlCtPrPci6z2PxJpBBbyDQzRFGDu3j5I4wNVmRXY Ey+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ndIOLOxZ; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id a10si275495pgh.377.2021.04.29.06.28.54; Thu, 29 Apr 2021 06:28:54 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ndIOLOxZ; spf=pass (google.com: domain of linux-media-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-media-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237580AbhD2N3i (ORCPT + 4 others); Thu, 29 Apr 2021 09:29:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55052 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236635AbhD2N3h (ORCPT ); Thu, 29 Apr 2021 09:29:37 -0400 Received: from mail-wr1-x433.google.com (mail-wr1-x433.google.com [IPv6:2a00:1450:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9F4F2C06138B for ; Thu, 29 Apr 2021 06:28:49 -0700 (PDT) Received: by mail-wr1-x433.google.com with SMTP id h15so14707780wre.11 for ; Thu, 29 Apr 2021 06:28:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Oy1qNwnuHeibPjIkP+hgeYDcQYApY+KrxCJjPlibiZU=; b=ndIOLOxZamBY6rnBB/+RDD3eu5FoeiL7lB7R5jvMjZARt/ZRvjs9SOqX6B6vVN95gk /UbxwLgZzCArrTg4Ws0yTOnRGnHTZ/ZeXM69PSg5WpIbczZXZwHwJ04zDffVyBEGe+KP k8AkBTpsAxvJ6KOAWym2JLD19Z4g1BGU/eHLVIpHGbZS2qAL2V7hQGvlpfkUMbmyVZVE TRswFCB1duxaU8jIDjFEC9PpA6uhc1Mdy6zhjgALlFexGnrX5Of9zU9Nj6wKCX1qQVZh UPP3wzFVUTPyH1qzfNPbJkZcBNnbnWdXH1XzcCQpzJo42JPBAEFenKNbfSppGBO/d+ah 8YZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Oy1qNwnuHeibPjIkP+hgeYDcQYApY+KrxCJjPlibiZU=; b=GX8iIYh2gd9kqYWxqeg0ZtHqDDOKXGpjVDDvgYFSsTiAHciO9GKV/u+90YkmhKanRq IfXblDlSegzaNRuw30mkDhmPhQml01P7v115oo4GgpLP37DHbfkLb0mght4UJrd6OvPL Cc4OO+4iAQuYAwtcFbc8yf4lo+oOOetqorbHcwEg84UpRyCUHLMMyvHtE/xT5UO0/oLj Tj0PU7oVcnQTLuFFED/e8W4ZCv/OyacHsUwW+y9lnjNOiakcv38IYFQT24Zsn+cThRdn ctWbBhMFGtHjmZnKFh9+/Ii04wyTayuKcWtQxXm9M7dvUXBeoMQRgi4lK1pHlNrWJzP0 cRcA== X-Gm-Message-State: AOAM533TBuiomCPKBvXgptTdCINaVKqUr4VxTmKxezPS0QrDpI5hD8VJ GAfajH/e+98iHxu0qgqmCW3pprrFlhxU3p59 X-Received: by 2002:adf:a316:: with SMTP id c22mr2268642wrb.202.1619702927459; Thu, 29 Apr 2021 06:28:47 -0700 (PDT) Received: from localhost.localdomain (hst-221-38.medicom.bg. [84.238.221.38]) by smtp.gmail.com with ESMTPSA id n10sm4811462wrw.37.2021.04.29.06.28.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 29 Apr 2021 06:28:47 -0700 (PDT) From: Stanimir Varbanov To: linux-media@vger.kernel.org Cc: Hans Verkuil , Stanimir Varbanov Subject: [RFC/WIP 4/4] venus: Add HEIC image encoder Date: Thu, 29 Apr 2021 16:28:33 +0300 Message-Id: <20210429132833.2802390-5-stanimir.varbanov@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> References: <20210429132833.2802390-1-stanimir.varbanov@linaro.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org This adds support for HEIC image encoder to Venus driver as a separate device video node. Signed-off-by: Stanimir Varbanov --- drivers/media/platform/qcom/venus/Makefile | 2 + drivers/media/platform/qcom/venus/core.h | 10 + drivers/media/platform/qcom/venus/hfi_cmds.c | 10 +- .../media/platform/qcom/venus/hfi_helper.h | 5 + drivers/media/platform/qcom/venus/ienc.c | 1348 +++++++++++++++++ drivers/media/platform/qcom/venus/ienc.h | 14 + .../media/platform/qcom/venus/ienc_ctrls.c | 83 + 7 files changed, 1471 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/qcom/venus/ienc.c create mode 100644 drivers/media/platform/qcom/venus/ienc.h create mode 100644 drivers/media/platform/qcom/venus/ienc_ctrls.c -- 2.25.1 diff --git a/drivers/media/platform/qcom/venus/Makefile b/drivers/media/platform/qcom/venus/Makefile index 91ee6be10292..1277e08b73f4 100644 --- a/drivers/media/platform/qcom/venus/Makefile +++ b/drivers/media/platform/qcom/venus/Makefile @@ -9,7 +9,9 @@ venus-core-objs += core.o helpers.o firmware.o \ venus-dec-objs += vdec.o vdec_ctrls.o venus-enc-objs += venc.o venc_ctrls.o +venus-ienc-objs += ienc.o ienc_ctrls.o obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-core.o obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-dec.o obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-enc.o +obj-$(CONFIG_VIDEO_QCOM_VENUS) += venus-ienc.o diff --git a/drivers/media/platform/qcom/venus/core.h b/drivers/media/platform/qcom/venus/core.h index 6de596b5a9d1..c5919724f18e 100644 --- a/drivers/media/platform/qcom/venus/core.h +++ b/drivers/media/platform/qcom/venus/core.h @@ -150,11 +150,13 @@ struct venus_core { struct reset_control *resets[VIDC_RESETS_NUM_MAX]; struct video_device *vdev_dec; struct video_device *vdev_enc; + struct video_device *vdev_ienc; struct v4l2_device v4l2_dev; const struct venus_resources *res; struct device *dev; struct device *dev_dec; struct device *dev_enc; + struct device *dev_ienc; unsigned int use_tz; struct video_firmware { struct device *dev; @@ -261,6 +263,13 @@ struct venc_controls { u32 base_priority_id; }; +struct ienc_controls { + u32 profile; + u32 level; + u32 image_quality; + u32 grid_size; +}; + struct venus_buffer { struct vb2_v4l2_buffer vb; struct list_head list; @@ -370,6 +379,7 @@ struct venus_inst { union { struct vdec_controls dec; struct venc_controls enc; + struct ienc_controls ienc; } controls; struct v4l2_fh fh; unsigned int streamon_cap, streamon_out; diff --git a/drivers/media/platform/qcom/venus/hfi_cmds.c b/drivers/media/platform/qcom/venus/hfi_cmds.c index fb64046d1e35..48232db4e491 100644 --- a/drivers/media/platform/qcom/venus/hfi_cmds.c +++ b/drivers/media/platform/qcom/venus/hfi_cmds.c @@ -1268,7 +1268,15 @@ pkt_session_set_property_6xx(struct hfi_session_set_property_pkt *pkt, cq->frame_quality = in->frame_quality; pkt->shdr.hdr.size += sizeof(u32) + sizeof(*cq); break; - } default: + } + case HFI_PROPERTY_CONFIG_HEIC_GRID_ENABLE: { + struct hfi_heic_grid_enable *in = pdata, *grid = prop_data; + + grid->grid_enable = in->grid_enable; + pkt->shdr.hdr.size += sizeof(u32) + sizeof(*grid); + break; + } + default: return pkt_session_set_property_4xx(pkt, cookie, ptype, pdata); } diff --git a/drivers/media/platform/qcom/venus/hfi_helper.h b/drivers/media/platform/qcom/venus/hfi_helper.h index f367f43c9fb7..3836dedd61b0 100644 --- a/drivers/media/platform/qcom/venus/hfi_helper.h +++ b/drivers/media/platform/qcom/venus/hfi_helper.h @@ -534,6 +534,7 @@ #define HFI_PROPERTY_CONFIG_VENC_LTRPERIOD 0x200600c #define HFI_PROPERTY_CONFIG_VENC_PERF_MODE 0x200600e #define HFI_PROPERTY_CONFIG_HEIC_FRAME_QUALITY 0x2006014 +#define HFI_PROPERTY_CONFIG_HEIC_GRID_ENABLE 0x2006015 /* * HFI_PROPERTY_PARAM_VPE_COMMON_START @@ -772,6 +773,10 @@ struct hfi_heic_frame_quality { u32 reserved[3]; }; +struct hfi_heic_grid_enable { + u32 grid_enable; +}; + struct hfi_quantization { u32 qp_i; u32 qp_p; diff --git a/drivers/media/platform/qcom/venus/ienc.c b/drivers/media/platform/qcom/venus/ienc.c new file mode 100644 index 000000000000..a54a3d666233 --- /dev/null +++ b/drivers/media/platform/qcom/venus/ienc.c @@ -0,0 +1,1348 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved. + * Copyright (C) 2017 Linaro Ltd. + */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "hfi_venus_io.h" +#include "hfi_parser.h" +#include "core.h" +#include "helpers.h" +#include "ienc.h" +#include "pm_helpers.h" + +/* + * Three resons to keep MPLANE formats (despite that the number of planes + * currently is one): + * - the MPLANE formats allow only one plane to be used + * - the downstream driver use MPLANE formats too + * - future firmware versions could add support for >1 planes + */ +static const struct venus_format ienc_formats[] = { + { + .pixfmt = V4L2_PIX_FMT_NV12, + .num_planes = 1, + .type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE, + }, { + .pixfmt = V4L2_PIX_FMT_HEIC, + .num_planes = 1, + .type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE, + }, +}; + +static const struct venus_format * +find_format(struct venus_inst *inst, u32 pixfmt, u32 type) +{ + const struct venus_format *fmt = ienc_formats; + unsigned int size = ARRAY_SIZE(ienc_formats); + unsigned int i; + u32 check_pixfmt; + + for (i = 0; i < size; i++) { + if (fmt[i].pixfmt == pixfmt) + break; + } + + if (i == size || fmt[i].type != type) + return NULL; + + check_pixfmt = fmt[i].pixfmt; + + if (check_pixfmt == V4L2_PIX_FMT_HEIC) + check_pixfmt = V4L2_PIX_FMT_HEVC; + + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + !venus_helper_check_codec(inst, check_pixfmt)) + return NULL; + + return &fmt[i]; +} + +static const struct venus_format * +find_format_by_index(struct venus_inst *inst, unsigned int index, u32 type) +{ + const struct venus_format *fmt = ienc_formats; + unsigned int size = ARRAY_SIZE(ienc_formats); + unsigned int i, k = 0; + u32 check_pixfmt; + + if (index > size) + return NULL; + + for (i = 0; i < size; i++) { + bool valid; + + if (fmt[i].type != type) + continue; + + check_pixfmt = fmt[i].pixfmt; + if (check_pixfmt == V4L2_PIX_FMT_HEIC) + check_pixfmt = V4L2_PIX_FMT_HEVC; + + valid = type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE || + venus_helper_check_codec(inst, check_pixfmt); + if (k == index && valid) + break; + if (valid) + k++; + } + + if (i == size) + return NULL; + + return &fmt[i]; +} + +static int +ienc_querycap(struct file *file, void *fh, struct v4l2_capability *cap) +{ + strscpy(cap->driver, "qcom-venus", sizeof(cap->driver)); + strscpy(cap->card, "Qualcomm Venus image encoder", sizeof(cap->card)); + strscpy(cap->bus_info, "platform:qcom-venus", sizeof(cap->bus_info)); + + return 0; +} + +static int ienc_enum_fmt(struct file *file, void *fh, struct v4l2_fmtdesc *f) +{ + struct venus_inst *inst = to_inst(file); + const struct venus_format *fmt; + + fmt = find_format_by_index(inst, f->index, f->type); + + memset(f->reserved, 0, sizeof(f->reserved)); + + if (!fmt) + return -EINVAL; + + f->pixelformat = fmt->pixfmt; + + return 0; +} + +static const struct venus_format * +ienc_try_fmt_common(struct venus_inst *inst, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct v4l2_plane_pix_format *pfmt = pixmp->plane_fmt; + const struct venus_format *fmt; + u32 sizeimage; + + memset(pfmt[0].reserved, 0, sizeof(pfmt[0].reserved)); + memset(pixmp->reserved, 0, sizeof(pixmp->reserved)); + + fmt = find_format(inst, pixmp->pixelformat, f->type); + if (!fmt) { + if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) + pixmp->pixelformat = V4L2_PIX_FMT_HEIC; + else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + pixmp->pixelformat = V4L2_PIX_FMT_NV12; + else + return NULL; + fmt = find_format(inst, pixmp->pixelformat, f->type); + } + + pixmp->width = clamp(pixmp->width, frame_width_min(inst), + frame_width_max(inst)); + pixmp->height = clamp(pixmp->height, frame_height_min(inst), + frame_height_max(inst)); + + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + pixmp->height = ALIGN(pixmp->height, 32); + + pixmp->width = ALIGN(pixmp->width, 2); + pixmp->height = ALIGN(pixmp->height, 2); + + if (pixmp->field == V4L2_FIELD_ANY) + pixmp->field = V4L2_FIELD_NONE; + pixmp->num_planes = fmt->num_planes; + pixmp->flags = 0; + + sizeimage = venus_helper_get_framesz(pixmp->pixelformat, + pixmp->width, + pixmp->height); + pfmt[0].sizeimage = max(ALIGN(pfmt[0].sizeimage, SZ_4K), sizeimage); + + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + pfmt[0].bytesperline = ALIGN(pixmp->width, 128); + else + pfmt[0].bytesperline = 0; + + return fmt; +} + +static int ienc_try_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct venus_inst *inst = to_inst(file); + + ienc_try_fmt_common(inst, f); + + return 0; +} + +static int ienc_s_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct venus_inst *inst = to_inst(file); + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct v4l2_pix_format_mplane orig_pixmp; + const struct venus_format *fmt; + struct v4l2_format format; + u32 pixfmt_out = 0, pixfmt_cap = 0; + struct vb2_queue *q; + + q = v4l2_m2m_get_vq(inst->m2m_ctx, f->type); + if (!q) + return -EINVAL; + + if (vb2_is_busy(q)) + return -EBUSY; + + orig_pixmp = *pixmp; + + fmt = ienc_try_fmt_common(inst, f); + if (!fmt) + return -EINVAL; + + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + pixfmt_out = pixmp->pixelformat; + pixfmt_cap = inst->fmt_cap->pixfmt; + } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + pixfmt_cap = pixmp->pixelformat; + pixfmt_out = inst->fmt_out->pixfmt; + } + + memset(&format, 0, sizeof(format)); + + format.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + format.fmt.pix_mp.pixelformat = pixfmt_out; + format.fmt.pix_mp.width = orig_pixmp.width; + format.fmt.pix_mp.height = orig_pixmp.height; + ienc_try_fmt_common(inst, &format); + + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + inst->out_width = format.fmt.pix_mp.width; + inst->out_height = format.fmt.pix_mp.height; + inst->colorspace = pixmp->colorspace; + inst->ycbcr_enc = pixmp->ycbcr_enc; + inst->quantization = pixmp->quantization; + inst->xfer_func = pixmp->xfer_func; + } + + memset(&format, 0, sizeof(format)); + + format.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + format.fmt.pix_mp.pixelformat = pixfmt_cap; + format.fmt.pix_mp.width = orig_pixmp.width; + format.fmt.pix_mp.height = orig_pixmp.height; + ienc_try_fmt_common(inst, &format); + + inst->width = format.fmt.pix_mp.width; + inst->height = format.fmt.pix_mp.height; + + if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + inst->fmt_out = fmt; + } else if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + inst->fmt_cap = fmt; + inst->output_buf_size = pixmp->plane_fmt[0].sizeimage; + } + + return 0; +} + +static int ienc_g_fmt(struct file *file, void *fh, struct v4l2_format *f) +{ + struct v4l2_pix_format_mplane *pixmp = &f->fmt.pix_mp; + struct venus_inst *inst = to_inst(file); + const struct venus_format *fmt; + + if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) + fmt = inst->fmt_cap; + else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + fmt = inst->fmt_out; + else + return -EINVAL; + + pixmp->pixelformat = fmt->pixfmt; + + if (f->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + pixmp->width = inst->width; + pixmp->height = inst->height; + pixmp->colorspace = inst->colorspace; + pixmp->ycbcr_enc = inst->ycbcr_enc; + pixmp->quantization = inst->quantization; + pixmp->xfer_func = inst->xfer_func; + } else if (f->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) { + pixmp->width = inst->out_width; + pixmp->height = inst->out_height; + } + + ienc_try_fmt_common(inst, f); + + return 0; +} + +static int +ienc_g_selection(struct file *file, void *fh, struct v4l2_selection *s) +{ + struct venus_inst *inst = to_inst(file); + + if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT) + return -EINVAL; + + switch (s->target) { + case V4L2_SEL_TGT_CROP_DEFAULT: + case V4L2_SEL_TGT_CROP_BOUNDS: + s->r.width = inst->out_width; + s->r.height = inst->out_height; + break; + case V4L2_SEL_TGT_CROP: + s->r.width = inst->width; + s->r.height = inst->height; + break; + default: + return -EINVAL; + } + + s->r.top = 0; + s->r.left = 0; + + return 0; +} + +static int +ienc_s_selection(struct file *file, void *fh, struct v4l2_selection *s) +{ + struct venus_inst *inst = to_inst(file); + + if (s->type != V4L2_BUF_TYPE_VIDEO_OUTPUT) + return -EINVAL; + + if (s->r.width > inst->out_width || + s->r.height > inst->out_height) + return -EINVAL; + + s->r.width = ALIGN(s->r.width, 2); + s->r.height = ALIGN(s->r.height, 2); + + switch (s->target) { + case V4L2_SEL_TGT_CROP: + s->r.top = 0; + s->r.left = 0; + inst->width = s->r.width; + inst->height = s->r.height; + break; + default: + return -EINVAL; + } + + return 0; +} + +static int ienc_s_parm(struct file *file, void *fh, struct v4l2_streamparm *a) +{ + struct venus_inst *inst = to_inst(file); + struct v4l2_outputparm *out = &a->parm.output; + struct v4l2_fract *timeperframe = &out->timeperframe; + u64 us_per_frame, fps; + + if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + return -EINVAL; + + memset(out->reserved, 0, sizeof(out->reserved)); + + if (!timeperframe->denominator) + timeperframe->denominator = inst->timeperframe.denominator; + if (!timeperframe->numerator) + timeperframe->numerator = inst->timeperframe.numerator; + + out->capability = V4L2_CAP_TIMEPERFRAME; + + us_per_frame = timeperframe->numerator * (u64)USEC_PER_SEC; + do_div(us_per_frame, timeperframe->denominator); + + if (!us_per_frame) + return -EINVAL; + + fps = (u64)USEC_PER_SEC; + do_div(fps, us_per_frame); + + inst->timeperframe = *timeperframe; + inst->fps = fps; + + return 0; +} + +static int ienc_g_parm(struct file *file, void *fh, struct v4l2_streamparm *a) +{ + struct venus_inst *inst = to_inst(file); + + if (a->type != V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + a->type != V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE) + return -EINVAL; + + a->parm.output.capability |= V4L2_CAP_TIMEPERFRAME; + a->parm.output.timeperframe = inst->timeperframe; + + return 0; +} + +static int ienc_enum_framesizes(struct file *file, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + struct venus_inst *inst = to_inst(file); + const struct venus_format *fmt; + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + + fmt = find_format(inst, fsize->pixel_format, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (!fmt) { + fmt = find_format(inst, fsize->pixel_format, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (!fmt) + return -EINVAL; + } + + if (fsize->index) + return -EINVAL; + + fsize->stepwise.min_width = frame_width_min(inst); + fsize->stepwise.max_width = frame_width_max(inst); + fsize->stepwise.step_width = frame_width_step(inst); + fsize->stepwise.min_height = frame_height_min(inst); + fsize->stepwise.max_height = frame_height_max(inst); + fsize->stepwise.step_height = frame_height_step(inst); + + return 0; +} + +static int ienc_enum_frameintervals(struct file *file, void *fh, + struct v4l2_frmivalenum *fival) +{ + struct venus_inst *inst = to_inst(file); + const struct venus_format *fmt; + unsigned int framerate_factor = 1; + + fival->type = V4L2_FRMIVAL_TYPE_STEPWISE; + + fmt = find_format(inst, fival->pixel_format, + V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE); + if (!fmt) { + fmt = find_format(inst, fival->pixel_format, + V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE); + if (!fmt) + return -EINVAL; + } + + if (fival->index) + return -EINVAL; + + if (!fival->width || !fival->height) + return -EINVAL; + + if (fival->width > frame_width_max(inst) || + fival->width < frame_width_min(inst) || + fival->height > frame_height_max(inst) || + fival->height < frame_height_min(inst)) + return -EINVAL; + + if (IS_V1(inst->core)) { + /* framerate is reported in 1/65535 fps unit */ + framerate_factor = (1 << 16); + } + + fival->stepwise.min.numerator = 1; + fival->stepwise.min.denominator = frate_max(inst) / framerate_factor; + fival->stepwise.max.numerator = 1; + fival->stepwise.max.denominator = frate_min(inst) / framerate_factor; + fival->stepwise.step.numerator = 1; + fival->stepwise.step.denominator = frate_max(inst) / framerate_factor; + + return 0; +} + +static int ienc_encoder_cmd(struct file *file, void *fh, + struct v4l2_encoder_cmd *ec) +{ + struct venus_inst *inst = to_inst(file); + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + struct hfi_frame_data fdata = {0}; + int ret = 0; + + ret = v4l2_m2m_ioctl_try_encoder_cmd(file, fh, ec); + if (ret < 0) + return ret; + + mutex_lock(&inst->lock); + + if (!vb2_is_streaming(&m2m_ctx->cap_q_ctx.q) || + !vb2_is_streaming(&m2m_ctx->out_q_ctx.q)) + goto unlock; + + if (m2m_ctx->is_draining) { + ret = -EBUSY; + goto unlock; + } + + if (ec->cmd == V4L2_ENC_CMD_STOP) { + if (v4l2_m2m_has_stopped(m2m_ctx)) { + ret = 0; + goto unlock; + } + + m2m_ctx->is_draining = true; + + fdata.buffer_type = HFI_BUFFER_INPUT; + fdata.flags |= HFI_BUFFERFLAG_EOS; + fdata.device_addr = 0; + fdata.clnt_data = (u32)-1; + + ret = hfi_session_process_buf(inst, &fdata); + if (ret) + goto unlock; + } + + if (ec->cmd == V4L2_ENC_CMD_START && v4l2_m2m_has_stopped(m2m_ctx)) { + vb2_clear_last_buffer_dequeued(&m2m_ctx->cap_q_ctx.q); + inst->m2m_ctx->has_stopped = false; + venus_helper_process_initial_out_bufs(inst); + venus_helper_process_initial_cap_bufs(inst); + } + +unlock: + mutex_unlock(&inst->lock); + return ret; +} + +static const struct v4l2_ioctl_ops ienc_ioctl_ops = { + .vidioc_querycap = ienc_querycap, + .vidioc_enum_fmt_vid_cap = ienc_enum_fmt, + .vidioc_enum_fmt_vid_out = ienc_enum_fmt, + .vidioc_s_fmt_vid_cap_mplane = ienc_s_fmt, + .vidioc_s_fmt_vid_out_mplane = ienc_s_fmt, + .vidioc_g_fmt_vid_cap_mplane = ienc_g_fmt, + .vidioc_g_fmt_vid_out_mplane = ienc_g_fmt, + .vidioc_try_fmt_vid_cap_mplane = ienc_try_fmt, + .vidioc_try_fmt_vid_out_mplane = ienc_try_fmt, + .vidioc_g_selection = ienc_g_selection, + .vidioc_s_selection = ienc_s_selection, + .vidioc_reqbufs = v4l2_m2m_ioctl_reqbufs, + .vidioc_querybuf = v4l2_m2m_ioctl_querybuf, + .vidioc_create_bufs = v4l2_m2m_ioctl_create_bufs, + .vidioc_prepare_buf = v4l2_m2m_ioctl_prepare_buf, + .vidioc_qbuf = v4l2_m2m_ioctl_qbuf, + .vidioc_expbuf = v4l2_m2m_ioctl_expbuf, + .vidioc_dqbuf = v4l2_m2m_ioctl_dqbuf, + .vidioc_streamon = v4l2_m2m_ioctl_streamon, + .vidioc_streamoff = v4l2_m2m_ioctl_streamoff, + .vidioc_s_parm = ienc_s_parm, + .vidioc_g_parm = ienc_g_parm, + .vidioc_enum_framesizes = ienc_enum_framesizes, + .vidioc_enum_frameintervals = ienc_enum_frameintervals, + .vidioc_subscribe_event = v4l2_ctrl_subscribe_event, + .vidioc_unsubscribe_event = v4l2_event_unsubscribe, + .vidioc_try_encoder_cmd = v4l2_m2m_ioctl_try_encoder_cmd, + .vidioc_encoder_cmd = ienc_encoder_cmd, +}; + +static int ienc_set_properties(struct venus_inst *inst) +{ + struct ienc_controls *ctr = &inst->controls.ienc; + struct hfi_multi_slice_control mslice; + struct hfi_intra_period intra_period; + struct hfi_ltr_mode ltr_mode; + struct hfi_framerate frate; + struct hfi_intra_refresh ir; + struct hfi_enable en; + u32 ptype, rate_control; + u32 profile, level, hier_p; + int ret; + + ret = venus_helper_set_work_mode(inst); + if (ret) + return ret; + + ptype = HFI_PROPERTY_CONFIG_FRAME_RATE; + frate.buffer_type = HFI_BUFFER_OUTPUT; + frate.framerate = inst->fps * (1 << 16); + + ret = hfi_session_set_property(inst, ptype, &frate); + if (ret) + return ret; + + ptype = HFI_PROPERTY_PARAM_VENC_RATE_CONTROL; + rate_control = HFI_RATE_CONTROL_CQ; + + ret = hfi_session_set_property(inst, ptype, &rate_control); + if (ret) + return ret; + + if (rate_control == HFI_RATE_CONTROL_CQ && ctr->image_quality) { + struct hfi_heic_frame_quality quality = {0}; + + ptype = HFI_PROPERTY_CONFIG_HEIC_FRAME_QUALITY; + quality.frame_quality = ctr->image_quality; + + ret = hfi_session_set_property(inst, ptype, &quality); + if (ret) + return ret; + } + + if (ctr->grid_size) { + struct hfi_heic_grid_enable grid = {}; + + ptype = HFI_PROPERTY_CONFIG_HEIC_GRID_ENABLE; + grid.grid_enable = 1; + + ret = hfi_session_set_property(inst, ptype, &grid); + if (ret) + return ret; + } + + /* disable multi slice mode aka set single mode */ + ptype = HFI_PROPERTY_PARAM_VENC_MULTI_SLICE_CONTROL; + mslice.multi_slice = HFI_MULTI_SLICE_OFF; + mslice.slice_size = 0; + + ret = hfi_session_set_property(inst, ptype, &mslice); + if (ret) + return ret; + + /* disable LTR */ + ptype = HFI_PROPERTY_PARAM_VENC_LTRMODE; + ltr_mode.ltr_mode = HFI_LTR_MODE_DISABLE; + ltr_mode.ltr_count = 0; + ltr_mode.trust_mode = 0; + + ret = hfi_session_set_property(inst, ptype, <r_mode); + if (ret) + return ret; + + /* disable layer encoding */ + ptype = HFI_PROPERTY_CONFIG_VENC_HIER_P_ENH_LAYER; + hier_p = 0; + + ret = hfi_session_set_property(inst, ptype, &hier_p); + if (ret) + return ret; + + /* disable IR */ + ptype = HFI_PROPERTY_PARAM_VENC_INTRA_REFRESH; + ir.mode = HFI_INTRA_REFRESH_NONE; + ir.cir_mbs = 0; + + ret = hfi_session_set_property(inst, ptype, &ir); + if (ret) + return ret; + + /* Reset P & B frames */ + ptype = HFI_PROPERTY_CONFIG_VENC_INTRA_PERIOD; + intra_period.pframes = 0; + intra_period.bframes = 0; + + ret = hfi_session_set_property(inst, ptype, &intra_period); + if (ret) + return ret; + + switch (inst->hfi_codec) { + case HFI_VIDEO_CODEC_HEVC: + profile = ctr->profile; + level = ctr->level; + break; + default: + profile = 0; + level = 0; + break; + } + + ret = venus_helper_set_profile_level(inst, profile, level); + if (ret) + return ret; + + /* Prepend SPS and PPS with every I/IDR frame */ + ptype = HFI_PROPERTY_CONFIG_VENC_SYNC_FRAME_SEQUENCE_HEADER; + en.enable = 1; + ret = hfi_session_set_property(inst, ptype, &en); + if (ret) + return ret; + + return 0; +} + +static int ienc_init_session(struct venus_inst *inst) +{ + struct ienc_controls *ctr = &inst->controls.ienc; + u32 width, height; + int ret; + + ret = venus_helper_session_init(inst); + if (ret == -EALREADY) + return 0; + else if (ret) + return ret; + + ret = venus_helper_set_input_resolution(inst, inst->width, + inst->height); + if (ret) + goto deinit; + + width = inst->width; + height = inst->height; + + if (ctr->grid_size) { + width = ctr->grid_size;//512; + height = ctr->grid_size;//512; + } + + ret = venus_helper_set_output_resolution(inst, width, height, + HFI_BUFFER_OUTPUT); + if (ret) + goto deinit; + + ret = venus_helper_set_color_format(inst, inst->fmt_out->pixfmt); + if (ret) + goto deinit; + + ret = ienc_set_properties(inst); + if (ret) + goto deinit; + + return 0; +deinit: + hfi_session_deinit(inst); + return ret; +} + +static int ienc_out_num_buffers(struct venus_inst *inst, unsigned int *num) +{ + struct hfi_buffer_requirements bufreq; + int ret; + + ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq); + if (ret) + return ret; + + *num = bufreq.count_actual; + + return 0; +} + +static int ienc_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, unsigned int *num_planes, + unsigned int sizes[], struct device *alloc_devs[]) +{ + struct venus_inst *inst = vb2_get_drv_priv(q); + unsigned int num, min = 4; + int ret; + + if (*num_planes) { + if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && + *num_planes != inst->fmt_out->num_planes) + return -EINVAL; + + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + *num_planes != inst->fmt_cap->num_planes) + return -EINVAL; + + if (q->type == V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE && + sizes[0] < inst->input_buf_size) + return -EINVAL; + + if (q->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE && + sizes[0] < inst->output_buf_size) + return -EINVAL; + + return 0; + } + + mutex_lock(&inst->lock); + ret = ienc_init_session(inst); + mutex_unlock(&inst->lock); + + if (ret) + return ret; + + switch (q->type) { + case V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE: + *num_planes = inst->fmt_out->num_planes; + + ret = ienc_out_num_buffers(inst, &num); + if (ret) + break; + + num = max(num, min); + *num_buffers = max(*num_buffers, num); + inst->num_input_bufs = *num_buffers; + + sizes[0] = venus_helper_get_framesz(inst->fmt_out->pixfmt, + inst->width, + inst->height); + inst->input_buf_size = sizes[0]; + break; + case V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE: + *num_planes = inst->fmt_cap->num_planes; + *num_buffers = max(*num_buffers, min); + inst->num_output_bufs = *num_buffers; + sizes[0] = venus_helper_get_framesz(inst->fmt_cap->pixfmt, + inst->width, + inst->height); + sizes[0] = max(sizes[0], inst->output_buf_size); + inst->output_buf_size = sizes[0]; + break; + default: + ret = -EINVAL; + break; + } + + return ret; +} + +static int ienc_buf_init(struct vb2_buffer *vb) +{ + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue); + + inst->buf_count++; + + return venus_helper_vb2_buf_init(vb); +} + +static void ienc_release_session(struct venus_inst *inst) +{ + int ret; + + mutex_lock(&inst->lock); + + ret = hfi_session_deinit(inst); + if (ret || inst->session_error) + hfi_session_abort(inst); + + mutex_unlock(&inst->lock); + + venus_pm_load_scale(inst); + INIT_LIST_HEAD(&inst->registeredbufs); + venus_pm_release_core(inst); +} + +static void ienc_buf_cleanup(struct vb2_buffer *vb) +{ + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct venus_buffer *buf = to_venus_buffer(vbuf); + + mutex_lock(&inst->lock); + if (vb->type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) + if (!list_empty(&inst->registeredbufs)) + list_del_init(&buf->reg_list); + mutex_unlock(&inst->lock); + + inst->buf_count--; + if (!inst->buf_count) + ienc_release_session(inst); +} + +static int ienc_verify_conf(struct venus_inst *inst) +{ + enum hfi_version ver = inst->core->res->hfi_version; + struct hfi_buffer_requirements bufreq; + int ret; + + if (!inst->num_input_bufs || !inst->num_output_bufs) + return -EINVAL; + + ret = venus_helper_get_bufreq(inst, HFI_BUFFER_OUTPUT, &bufreq); + if (ret) + return ret; + + if (inst->num_output_bufs < bufreq.count_actual || + inst->num_output_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver)) + return -EINVAL; + + ret = venus_helper_get_bufreq(inst, HFI_BUFFER_INPUT, &bufreq); + if (ret) + return ret; + + if (inst->num_input_bufs < bufreq.count_actual || + inst->num_input_bufs < HFI_BUFREQ_COUNT_MIN(&bufreq, ver)) + return -EINVAL; + + return 0; +} + +static int ienc_start_streaming(struct vb2_queue *q, unsigned int count) +{ + struct venus_inst *inst = vb2_get_drv_priv(q); + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + int ret; + + mutex_lock(&inst->lock); + + v4l2_m2m_update_start_streaming_state(m2m_ctx, q); + + if (V4L2_TYPE_IS_OUTPUT(q->type)) + inst->streamon_out = 1; + else + inst->streamon_cap = 1; + + if (inst->streamon_out && inst->streamon_cap && + inst->state == INST_INIT) { + venus_helper_init_instance(inst); + + inst->sequence_cap = 0; + inst->sequence_out = 0; + + ret = ienc_init_session(inst); + if (ret) + goto bufs_done; + + ret = venus_pm_acquire_core(inst); + if (ret) + goto deinit_sess; + + ret = ienc_set_properties(inst); + if (ret) + goto deinit_sess; + + ret = ienc_verify_conf(inst); + if (ret) + goto deinit_sess; + + ret = venus_helper_set_num_bufs(inst, inst->num_input_bufs, + inst->num_output_bufs, 0); + if (ret) + goto deinit_sess; + + ret = venus_helper_vb2_start_streaming(inst); + if (ret) + goto deinit_sess; + + venus_helper_process_initial_out_bufs(inst); + venus_helper_process_initial_cap_bufs(inst); + } else if (V4L2_TYPE_IS_CAPTURE(q->type) && inst->streamon_cap && + inst->streamon_out) { + ret = venus_helper_vb2_start_streaming(inst); + if (ret) + goto bufs_done; + + venus_helper_process_initial_out_bufs(inst); + venus_helper_process_initial_cap_bufs(inst); + } + + mutex_unlock(&inst->lock); + + return 0; + +deinit_sess: + hfi_session_deinit(inst); +bufs_done: + venus_helper_buffers_done(inst, q->type, VB2_BUF_STATE_QUEUED); + if (V4L2_TYPE_IS_OUTPUT(q->type)) + inst->streamon_out = 0; + else + inst->streamon_cap = 0; + mutex_unlock(&inst->lock); + return ret; +} + +static void ienc_stop_streaming(struct vb2_queue *q) +{ + struct venus_inst *inst = vb2_get_drv_priv(q); + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + int ret = -EINVAL; + + mutex_lock(&inst->lock); + + v4l2_m2m_clear_state(m2m_ctx); + + if (V4L2_TYPE_IS_CAPTURE(q->type)) { + ret = hfi_session_stop(inst); + ret |= hfi_session_unload_res(inst); + ret |= venus_helper_unregister_bufs(inst); + ret |= venus_helper_intbufs_free(inst); + } + + venus_helper_buffers_done(inst, q->type, VB2_BUF_STATE_ERROR); + + if (V4L2_TYPE_IS_OUTPUT(q->type)) + inst->streamon_out = 0; + else + inst->streamon_cap = 0; + + mutex_unlock(&inst->lock); +} + +static void ienc_vb2_buf_queue(struct vb2_buffer *vb) +{ + struct venus_inst *inst = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + + mutex_lock(&inst->lock); + + v4l2_m2m_buf_queue(m2m_ctx, vbuf); + + if (!(inst->streamon_out && inst->streamon_cap)) + goto unlock; + + if (v4l2_m2m_has_stopped(m2m_ctx)) + goto unlock; + + venus_helper_process_buf(vb); + +unlock: + mutex_unlock(&inst->lock); +} + +static const struct vb2_ops ienc_vb2_ops = { + .queue_setup = ienc_queue_setup, + .buf_init = ienc_buf_init, + .buf_cleanup = ienc_buf_cleanup, + .buf_prepare = venus_helper_vb2_buf_prepare, + .start_streaming = ienc_start_streaming, + .stop_streaming = ienc_stop_streaming, + .buf_queue = ienc_vb2_buf_queue, +}; + +static void ienc_buf_done(struct venus_inst *inst, unsigned int buf_type, + u32 tag, u32 bytesused, u32 data_offset, u32 flags, + u32 hfi_flags, u64 timestamp_us) +{ + struct vb2_v4l2_buffer *vbuf; + struct vb2_buffer *vb; + unsigned int type; + struct v4l2_m2m_ctx *m2m_ctx = inst->m2m_ctx; + + if (buf_type == HFI_BUFFER_INPUT) + type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + else + type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + + vbuf = venus_helper_find_buf(inst, type, tag); + if (!vbuf) + return; + + vbuf->flags = flags; + vb = &vbuf->vb2_buf; + + if (type == V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE) { + vb = &vbuf->vb2_buf; + vb2_set_plane_payload(vb, 0, bytesused + data_offset); + vb->planes[0].data_offset = data_offset; + vb->timestamp = timestamp_us * NSEC_PER_USEC; + vbuf->sequence = inst->sequence_cap++; + + if ((!bytesused && m2m_ctx->is_draining) || + (vbuf->flags & V4L2_BUF_FLAG_LAST)) { + vbuf->flags |= V4L2_BUF_FLAG_LAST; + v4l2_m2m_mark_stopped(inst->m2m_ctx); + } + } else { + vbuf->sequence = inst->sequence_out++; + } + + v4l2_m2m_buf_done(vbuf, VB2_BUF_STATE_DONE); +} + +static void ienc_event_notify(struct venus_inst *inst, u32 event, + struct hfi_event_data *data) +{ + struct device *dev = inst->core->dev_enc; + + if (event == EVT_SESSION_ERROR) { + inst->session_error = true; + dev_err(dev, "enc: event session error %x\n", inst->error); + } +} + +static const struct hfi_inst_ops ienc_hfi_ops = { + .buf_done = ienc_buf_done, + .event_notify = ienc_event_notify, +}; + +static void ienc_m2m_device_run(void *priv) +{ +} + +static const struct v4l2_m2m_ops ienc_m2m_ops = { + .device_run = ienc_m2m_device_run, + .job_abort = venus_helper_m2m_job_abort, +}; + +static int m2m_queue_init(void *priv, struct vb2_queue *src_vq, + struct vb2_queue *dst_vq) +{ + struct venus_inst *inst = priv; + int ret; + + src_vq->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE; + src_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF; + src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + src_vq->ops = &ienc_vb2_ops; + src_vq->mem_ops = &vb2_dma_contig_memops; + src_vq->drv_priv = inst; + src_vq->buf_struct_size = sizeof(struct venus_buffer); + src_vq->allow_zero_bytesused = 1; + src_vq->min_buffers_needed = 1; + src_vq->dev = inst->core->dev; + if (inst->core->res->hfi_version == HFI_VERSION_1XX) + src_vq->bidirectional = 1; + ret = vb2_queue_init(src_vq); + if (ret) + return ret; + + dst_vq->type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + dst_vq->io_modes = VB2_MMAP | VB2_USERPTR | VB2_DMABUF; + dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + dst_vq->ops = &ienc_vb2_ops; + dst_vq->mem_ops = &vb2_dma_contig_memops; + dst_vq->drv_priv = inst; + dst_vq->buf_struct_size = sizeof(struct venus_buffer); + dst_vq->allow_zero_bytesused = 1; + dst_vq->min_buffers_needed = 1; + dst_vq->dev = inst->core->dev; + return vb2_queue_init(dst_vq); +} + +static void ienc_inst_init(struct venus_inst *inst) +{ + inst->fmt_cap = &ienc_formats[1]; + inst->fmt_out = &ienc_formats[0]; + inst->width = 1280; + inst->height = ALIGN(720, 32); + inst->out_width = 1280; + inst->out_height = 720; + inst->fps = 1; + inst->timeperframe.numerator = 1; + inst->timeperframe.denominator = 1; + inst->hfi_codec = HFI_VIDEO_CODEC_HEVC; +} + +static int ienc_open(struct file *file) +{ + struct venus_core *core = video_drvdata(file); + struct venus_inst *inst; + int ret; + + inst = kzalloc(sizeof(*inst), GFP_KERNEL); + if (!inst) + return -ENOMEM; + + INIT_LIST_HEAD(&inst->dpbbufs); + INIT_LIST_HEAD(&inst->registeredbufs); + INIT_LIST_HEAD(&inst->internalbufs); + INIT_LIST_HEAD(&inst->list); + mutex_init(&inst->lock); + + inst->core = core; + inst->session_type = VIDC_SESSION_TYPE_ENC; + inst->clk_data.core_id = VIDC_CORE_ID_DEFAULT; + inst->core_acquired = false; + + venus_helper_init_instance(inst); + + ret = pm_runtime_get_sync(core->dev_enc); + if (ret < 0) + goto err_put_sync; + + ret = ienc_ctrl_init(inst); + if (ret) + goto err_put_sync; + + ret = hfi_session_create(inst, &ienc_hfi_ops); + if (ret) + goto err_ctrl_deinit; + + ienc_inst_init(inst); + + /* + * create m2m device for every instance, the m2m context scheduling + * is made by firmware side so we do not need to care about. + */ + inst->m2m_dev = v4l2_m2m_init(&ienc_m2m_ops); + if (IS_ERR(inst->m2m_dev)) { + ret = PTR_ERR(inst->m2m_dev); + goto err_session_destroy; + } + + inst->m2m_ctx = v4l2_m2m_ctx_init(inst->m2m_dev, inst, m2m_queue_init); + if (IS_ERR(inst->m2m_ctx)) { + ret = PTR_ERR(inst->m2m_ctx); + goto err_m2m_release; + } + + v4l2_fh_init(&inst->fh, core->vdev_ienc); + + inst->fh.ctrl_handler = &inst->ctrl_handler; + v4l2_fh_add(&inst->fh); + inst->fh.m2m_ctx = inst->m2m_ctx; + file->private_data = &inst->fh; + + return 0; + +err_m2m_release: + v4l2_m2m_release(inst->m2m_dev); +err_session_destroy: + hfi_session_destroy(inst); +err_ctrl_deinit: + ienc_ctrl_deinit(inst); +err_put_sync: + pm_runtime_put_sync(core->dev_enc); + kfree(inst); + return ret; +} + +static int ienc_close(struct file *file) +{ + struct venus_inst *inst = to_inst(file); + + v4l2_m2m_ctx_release(inst->m2m_ctx); + v4l2_m2m_release(inst->m2m_dev); + ienc_ctrl_deinit(inst); + hfi_session_destroy(inst); + mutex_destroy(&inst->lock); + v4l2_fh_del(&inst->fh); + v4l2_fh_exit(&inst->fh); + + pm_runtime_put_sync(inst->core->dev_enc); + + kfree(inst); + return 0; +} + +static const struct v4l2_file_operations ienc_fops = { + .owner = THIS_MODULE, + .open = ienc_open, + .release = ienc_close, + .unlocked_ioctl = video_ioctl2, + .poll = v4l2_m2m_fop_poll, + .mmap = v4l2_m2m_fop_mmap, +}; + +static int ienc_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct video_device *vdev; + struct venus_core *core; + int ret; + + if (!dev->parent) + return -EPROBE_DEFER; + + core = dev_get_drvdata(dev->parent); + if (!core) + return -EPROBE_DEFER; + + if (!IS_V6(core)) + return -ENODEV; + + platform_set_drvdata(pdev, core); + + if (core->pm_ops->venc_get) { + ret = core->pm_ops->venc_get(dev); + if (ret) + return ret; + } + + vdev = video_device_alloc(); + if (!vdev) + return -ENOMEM; + + strscpy(vdev->name, "qcom-venus-image-encoder", sizeof(vdev->name)); + vdev->release = video_device_release; + vdev->fops = &ienc_fops; + vdev->ioctl_ops = &ienc_ioctl_ops; + vdev->vfl_dir = VFL_DIR_M2M; + vdev->v4l2_dev = &core->v4l2_dev; + vdev->device_caps = V4L2_CAP_VIDEO_M2M_MPLANE | V4L2_CAP_STREAMING; + + ret = video_register_device(vdev, VFL_TYPE_VIDEO, -1); + if (ret) + goto err_vdev_release; + + core->vdev_ienc = vdev; + core->dev_ienc = dev; + + video_set_drvdata(vdev, core); + pm_runtime_enable(dev); + + return 0; + +err_vdev_release: + video_device_release(vdev); + return ret; +} + +static int ienc_remove(struct platform_device *pdev) +{ + struct venus_core *core = dev_get_drvdata(pdev->dev.parent); + + video_unregister_device(core->vdev_ienc); + pm_runtime_disable(core->dev_ienc); + + if (core->pm_ops->venc_put) + core->pm_ops->venc_put(core->dev_ienc); + + return 0; +} + +static __maybe_unused int ienc_runtime_suspend(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; + + if (pm_ops->venc_power) + ret = pm_ops->venc_power(dev, POWER_OFF); + + return ret; +} + +static __maybe_unused int ienc_runtime_resume(struct device *dev) +{ + struct venus_core *core = dev_get_drvdata(dev); + const struct venus_pm_ops *pm_ops = core->pm_ops; + int ret = 0; + + if (pm_ops->venc_power) + ret = pm_ops->venc_power(dev, POWER_ON); + + return ret; +} + +static const struct dev_pm_ops ienc_pm_ops = { + SET_SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, + pm_runtime_force_resume) + SET_RUNTIME_PM_OPS(ienc_runtime_suspend, ienc_runtime_resume, NULL) +}; + +static const struct of_device_id ienc_dt_match[] = { + { .compatible = "venus-image-encoder" }, + { } +}; +MODULE_DEVICE_TABLE(of, ienc_dt_match); + +static struct platform_driver qcom_venus_ienc_driver = { + .probe = ienc_probe, + .remove = ienc_remove, + .driver = { + .name = "qcom-venus-image-encoder", + .of_match_table = ienc_dt_match, + .pm = &ienc_pm_ops, + }, +}; +module_platform_driver(qcom_venus_ienc_driver); + +MODULE_ALIAS("platform:qcom-venus-image-encoder"); +MODULE_DESCRIPTION("Qualcomm Venus image encoder driver"); +MODULE_LICENSE("GPL v2"); diff --git a/drivers/media/platform/qcom/venus/ienc.h b/drivers/media/platform/qcom/venus/ienc.h new file mode 100644 index 000000000000..6d050e0c3b7f --- /dev/null +++ b/drivers/media/platform/qcom/venus/ienc.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved. + * Copyright (C) 2017 Linaro Ltd. + */ +#ifndef __VENUS_VENC_H__ +#define __VENUS_VENC_H__ + +struct venus_inst; + +int ienc_ctrl_init(struct venus_inst *inst); +void ienc_ctrl_deinit(struct venus_inst *inst); + +#endif diff --git a/drivers/media/platform/qcom/venus/ienc_ctrls.c b/drivers/media/platform/qcom/venus/ienc_ctrls.c new file mode 100644 index 000000000000..e925b20834b0 --- /dev/null +++ b/drivers/media/platform/qcom/venus/ienc_ctrls.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (c) 2012-2016, The Linux Foundation. All rights reserved. + * Copyright (C) 2017 Linaro Ltd. + */ +#include +#include + +#include "core.h" +#include "ienc.h" + +static int ienc_op_s_ctrl(struct v4l2_ctrl *ctrl) +{ + struct venus_inst *inst = ctrl_to_inst(ctrl); + struct ienc_controls *ctr = &inst->controls.ienc; + + switch (ctrl->id) { + case V4L2_CID_MPEG_VIDEO_HEVC_PROFILE: + ctr->profile = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_HEVC_LEVEL: + ctr->level = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY: + ctr->image_quality = ctrl->val; + break; + case V4L2_CID_MPEG_VIDEO_HEIC_GRID_SIZE: + ctr->grid_size = ctrl->val; + break; + default: + return -EINVAL; + } + + return 0; +} + +static const struct v4l2_ctrl_ops ienc_ctrl_ops = { + .s_ctrl = ienc_op_s_ctrl, +}; + +int ienc_ctrl_init(struct venus_inst *inst) +{ + int ret; + + ret = v4l2_ctrl_handler_init(&inst->ctrl_handler, 3); + if (ret) + return ret; + + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &ienc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_PROFILE, + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE, + ~(1 << V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE), + V4L2_MPEG_VIDEO_HEVC_PROFILE_MAIN_STILL_PICTURE); + + v4l2_ctrl_new_std_menu(&inst->ctrl_handler, &ienc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEVC_LEVEL, + V4L2_MPEG_VIDEO_HEVC_LEVEL_6_2, + 0, V4L2_MPEG_VIDEO_HEVC_LEVEL_1); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &ienc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_CONSTANT_QUALITY, 0, 100, 1, 80); + + v4l2_ctrl_new_std(&inst->ctrl_handler, &ienc_ctrl_ops, + V4L2_CID_MPEG_VIDEO_HEIC_GRID_SIZE, 0, 512, 512, 0); + + ret = inst->ctrl_handler.error; + if (ret) + goto err; + + ret = v4l2_ctrl_handler_setup(&inst->ctrl_handler); + if (ret) + goto err; + + return 0; +err: + v4l2_ctrl_handler_free(&inst->ctrl_handler); + return ret; +} + +void ienc_ctrl_deinit(struct venus_inst *inst) +{ + v4l2_ctrl_handler_free(&inst->ctrl_handler); +}