From patchwork Fri Mar 19 18:06:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405082 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C284EC433E1 for ; Fri, 19 Mar 2021 18:07:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8CFA461990 for ; Fri, 19 Mar 2021 18:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230296AbhCSSGt (ORCPT ); Fri, 19 Mar 2021 14:06:49 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230294AbhCSSGn (ORCPT ); Fri, 19 Mar 2021 14:06:43 -0400 IronPort-SDR: 43pE442BPGkLcen9E+TN2OMQCh+4VnbC8ZpzPOafB91+IcBmWbKtX5eakYUTIHxbTQdGBtOATr AhQAwPP4GLTw== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036002" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036002" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:42 -0700 IronPort-SDR: 3TATCr0aMGc9/ye7O0+eCl1LkzqwKOQAIfZ5TIQ85BsnGX08H+kyWnHcVA7tvbFW3nKcRH8WPT FhA/yD7JCEJQ== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605690" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:39 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 01/10] dt-bindings: media: Add bindings for Keem Bay Camera Date: Fri, 19 Mar 2021 18:06:23 +0000 Message-Id: <20210319180632.585-2-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Martina Krasteva - Add dt-bindings documentation for Intel Keem Bay Camera driver. - Add MAINTAINERS entry for Intel Keem Bay Camera binding documentation. Co-developed-by: Gjorgji Rosikopulos Signed-off-by: Gjorgji Rosikopulos Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- .../bindings/media/intel,keembay-camera.yaml | 98 ++++++++++++++++++++++ MAINTAINERS | 8 ++ 2 files changed, 106 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/intel,keembay-camera.yaml diff --git a/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml b/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml new file mode 100644 index 000000000000..78242b05228d --- /dev/null +++ b/Documentation/devicetree/bindings/media/intel,keembay-camera.yaml @@ -0,0 +1,98 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +# Copyright (C) 2021 Intel Corporation +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/media/intel,keembay-camera.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Intel Keem Bay camera subsystem + +maintainers: + - Paul J. Murphy + - Daniele Alessandrelli + +properties: + compatible: + const: intel,keembay-camera + memory-region: + $ref: /schemas/types.yaml#/definitions/phandle + + ports: + type: object + $ref: /schemas/graph.yaml#/properties/ports + + "#address-cells": + const: 1 + "#size-cells": + const: 0 + + properties: + port@[0-5]: + type: object + additionalProperties: false + $ref: /schemas/graph.yaml#/properties/port + description: + The port number matches the D-PHY number (D-PHY#0 - D-PHY#5). + + properties: + endpoint: + type: object + + properties: + data-lanes: + $ref: video-interfaces.yaml#/properties/data-lanes + description: + Six two-lane d-phys (D-PHY#0 - D-PHY#5) are available, which + can be used by six RX controllers (RX-CTRL#0 - RX-CTRL#5). + RX-CTRL#0, RX-CTRL#2, RX-CTRL#4 can be connected to two + D-PHY's and will be able to work with 3 and 4 lanes. In this + case the RX-CTRLs mapped to those D-PHYs cannot be used. + + Clock and data lanes are defined as follows + D-PHY#0 - clock - 0, data - 1, 2 + D-PHY#1 - clock - 3, data - 4, 5 + D-PHY#2 - clock - 6, data - 7, 8 + D-PHY#3 - clock - 9, data - 10, 11 + D-PHY#4 - clock - 12, data - 13, 14 + D-PHY#5 - clock - 15, data - 16, 17 + + required: + - data-lanes + + required: + - reg + - endpoint + + required: + - "#address-cells" + - "#size-cells" + - port@[0-5] + +required: + - compatible + - ports + + +additionalProperties: false + +examples: + - | + keembay_camera { + compatible = "intel,keembay-camera"; + memory-region = <&mem>; + + ports { + #address-cells = <1>; + #size-cells = <0>; + + port@4 { + reg = <4>; + + cam: endpoint { + remote-endpoint = <&imx334>; + data-lanes = <13 14 16 17>; + }; + }; + }; + }; +... diff --git a/MAINTAINERS b/MAINTAINERS index 08f9c2b7f3b3..c3f583ef8e46 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1965,6 +1965,14 @@ M: Lennert Buytenhek L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) S: Maintained +ARM/INTEL KEEM BAY CAMERA SUBSYSTEM +M: Paul J. Murphy +M: Daniele Alessandrelli +L: linux-media@vger.kernel.org +S: Maintained +T: git git://linuxtv.org/media_tree.git +F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml + ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT M: Lennert Buytenhek L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) From patchwork Fri Mar 19 18:06:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405081 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SPF_HELO_NONE, SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDB9DC433E3 for ; Fri, 19 Mar 2021 18:07:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BA9246198B for ; Fri, 19 Mar 2021 18:07:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbhCSSGt (ORCPT ); Fri, 19 Mar 2021 14:06:49 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230295AbhCSSGp (ORCPT ); Fri, 19 Mar 2021 14:06:45 -0400 IronPort-SDR: ymxDHgaioXJHa6SVj9LX1YxdCrJm5MlfsXXgqo5cO7u1GLSqbXF5qUgUi9YF/A2oMJOgDyEeck nTh9cQ4Cn8YQ== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036008" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036008" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:45 -0700 IronPort-SDR: yl+nWywZ6m6i2Y5MXcBjBwTNHdRVrnj7CVoYhamMxMSeGMrCSUMu0d42QJiL5mmKLQRvFxC9ed qoEoDQX2gczg== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605703" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:42 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 02/10] media: Keem Bay Camera: Keem Bay camera driver Date: Fri, 19 Mar 2021 18:06:24 +0000 Message-Id: <20210319180632.585-3-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Gjorgji Rosikopulos Keem Bay camera driver is a v4l2 media device which implements media controller interface. It has one sub-device (ISP sub-device) endpoint which can be connected to a remote CSI2 TX endpoint. The camera supports up to 6 ports, each of which is separate CSI2-PHY interface. Camera driver uses Xlink for communication with remote VPU camera. The Xlink camera protocol uses one control channel for pipeline management and configuration, one ISP channel for ISP control and one channel for each endpoint. Signed-off-by: Gjorgji Rosikopulos Co-developed-by: Martina Krasteva Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- MAINTAINERS | 1 + drivers/media/platform/Kconfig | 1 + drivers/media/platform/Makefile | 2 + drivers/media/platform/keembay-camera/Kconfig | 11 + drivers/media/platform/keembay-camera/Makefile | 3 + .../platform/keembay-camera/keembay-cam-xlink.c | 54 ++++ .../platform/keembay-camera/keembay-cam-xlink.h | 31 +++ .../media/platform/keembay-camera/keembay-camera.c | 287 +++++++++++++++++++++ .../media/platform/keembay-camera/keembay-camera.h | 43 +++ .../media/platform/keembay-camera/keembay-isp.c | 53 ++++ .../media/platform/keembay-camera/keembay-isp.h | 90 +++++++ 11 files changed, 576 insertions(+) create mode 100644 drivers/media/platform/keembay-camera/Kconfig create mode 100644 drivers/media/platform/keembay-camera/Makefile create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.c create mode 100644 drivers/media/platform/keembay-camera/keembay-cam-xlink.h create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.c create mode 100644 drivers/media/platform/keembay-camera/keembay-camera.h create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.c create mode 100644 drivers/media/platform/keembay-camera/keembay-isp.h diff --git a/MAINTAINERS b/MAINTAINERS index c3f583ef8e46..76082714a76f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1972,6 +1972,7 @@ L: linux-media@vger.kernel.org S: Maintained T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml +F: drivers/media/platform/keembay-camera/ ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT M: Lennert Buytenhek diff --git a/drivers/media/platform/Kconfig b/drivers/media/platform/Kconfig index 1ddb5d6354cf..6dedbf54d89a 100644 --- a/drivers/media/platform/Kconfig +++ b/drivers/media/platform/Kconfig @@ -171,6 +171,7 @@ source "drivers/media/platform/xilinx/Kconfig" source "drivers/media/platform/rcar-vin/Kconfig" source "drivers/media/platform/atmel/Kconfig" source "drivers/media/platform/sunxi/Kconfig" +source "drivers/media/platform/keembay-camera/Kconfig" config VIDEO_TI_CAL tristate "TI CAL (Camera Adaptation Layer) driver" diff --git a/drivers/media/platform/Makefile b/drivers/media/platform/Makefile index 9d4d6370908d..f67db3de5864 100644 --- a/drivers/media/platform/Makefile +++ b/drivers/media/platform/Makefile @@ -84,3 +84,5 @@ obj-$(CONFIG_VIDEO_QCOM_VENUS) += qcom/venus/ obj-y += sunxi/ obj-$(CONFIG_VIDEO_MESON_GE2D) += meson/ge2d/ + +obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-camera/ diff --git a/drivers/media/platform/keembay-camera/Kconfig b/drivers/media/platform/keembay-camera/Kconfig new file mode 100644 index 000000000000..691348ed5d4a --- /dev/null +++ b/drivers/media/platform/keembay-camera/Kconfig @@ -0,0 +1,11 @@ +config VIDEO_INTEL_KEEMBAY_CAMERA + tristate "Intel Keem Bay camera subsystem" + depends on VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API + depends on XLINK_CORE && HAS_DMA && OF + select VIDEOBUF2_DMA_CONTIG + select V4L2_FWNODE + help + This is a V4L2 driver for the Intel Keem Bay VPU camera interface. + + To compile this driver as a module, choose M here: the module + will be called keembay_cam diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile new file mode 100644 index 000000000000..ea19b61e6469 --- /dev/null +++ b/drivers/media/platform/keembay-camera/Makefile @@ -0,0 +1,3 @@ +keembay-cam-objs = keembay-camera.o keembay-cam-xlink.o keembay-isp.o + +obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c new file mode 100644 index 000000000000..e34516de0ff9 --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c @@ -0,0 +1,54 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Intel Keem Bay camera xlink + * + * Copyright (C) 2021 Intel Corporation + */ +#include +#include + +#include "keembay-cam-xlink.h" + +/** + * kmb_cam_xlink_init - Initialize xlink for VPU camera communication + * @xlink_cam: Pointer to xlink camera handle + * @dev: Client device of the xlink + * + * Perform initialization and establish connection with xlink VPUIP device + * + * Return: 0 if successful, error code otherwise + */ +int kmb_cam_xlink_init(struct kmb_xlink_cam *xlink_cam, struct device *dev) +{ + int ret; + + /* Connect to the device before opening channels */ + memset(&xlink_cam->handle, 0, sizeof(xlink_cam->handle)); + xlink_cam->handle.dev_type = VPUIP_DEVICE; + ret = xlink_connect(&xlink_cam->handle); + if (ret) { + dev_err(xlink_cam->dev, "Failed to connect: %d\n", ret); + return ret; + } + + ida_init(&xlink_cam->channel_ids); + xlink_cam->ctrl_chan_refcnt = 0; + + mutex_init(&xlink_cam->lock); + xlink_cam->dev = dev; + + return 0; +} + +/** + * kmb_cam_xlink_cleanup - Cleanup xlink camera communication + * @xlink_cam: Pointer to xlink camera handle + * + * Return: 0 if successful, error code otherwise + */ +void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam) +{ + /* Disconnect from the device after closing channels */ + xlink_disconnect(&xlink_cam->handle); + ida_destroy(&xlink_cam->channel_ids); +} diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h new file mode 100644 index 000000000000..d219d4298427 --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Intel Keem Bay camera xlink + * + * Copyright (C) 2021 Intel Corporation + */ +#ifndef KEEMBAY_CAM_XLINK_H +#define KEEMBAY_CAM_XLINK_H + +#include + +/** + * struct kmb_xlink_cam - KMB Camera xlink communication + * @dev: Device client of the xlink + * @lock: Mutex to serialize access to kmb xlink communication channels + * @handle: Xlink handle + * @ctrl_chan_refcnt: Main control channel reference count + * @channel_ids: Channel IDs. Each channel should have unique ID + */ +struct kmb_xlink_cam { + struct device *dev; + struct mutex lock; + struct xlink_handle handle; + unsigned int ctrl_chan_refcnt; + struct ida channel_ids; +}; + +int kmb_cam_xlink_init(struct kmb_xlink_cam *xlink_cam, struct device *dev); +void kmb_cam_xlink_cleanup(struct kmb_xlink_cam *xlink_cam); + +#endif /* KEEMBAY_CAM_XLINK_H */ diff --git a/drivers/media/platform/keembay-camera/keembay-camera.c b/drivers/media/platform/keembay-camera/keembay-camera.c new file mode 100644 index 000000000000..bc23df53473e --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-camera.c @@ -0,0 +1,287 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Intel Keem Bay camera driver. + * + * Copyright (C) 2021 Intel Corporation + */ +#include +#include +#include +#include +#include + +#include + +#include "keembay-camera.h" + +#define KMB_CAM_NUM_PORTS 6 + +/* RX-CTRL to data lanes mapping + * 2-lanes + * RX-CTRL#0 - {1, 2} + * RX-CTRL#1 - {4, 5} + * RX-CTRL#2 - {7, 8} + * RX-CTRL#3 - {10, 11} + * RX-CTRL#4 - {13, 14} + * RX-CTRL#5 - {16, 17} + * 4-lanes + * RX-CTRL#0 - {1, 2, 4, 5} + * RX-CTRL#2 - {7, 8, 10, 11} + * RX-CTRL#4 - {13, 14, 16, 17} + */ +static const u8 rx_ctrl[KMB_CAM_NUM_PORTS][2][4] = { + { { 1, 2 }, { 1, 2, 4, 5 } }, + { { 4, 5 }, {} }, + { { 7, 8 }, { 7, 8, 10, 11 } }, + { { 10, 11 }, {} }, + { { 13, 14 }, { 13, 14, 16, 17 } }, + { { 16, 17 }, {} } +}; + +static int get_rx_id(const u8 data_lanes[], + const unsigned short num_data_lanes) +{ + unsigned int i, j; + + for (i = 0; i < KMB_CAM_NUM_PORTS; i++) { + for (j = 0; j < ARRAY_SIZE(rx_ctrl[i]); j++) { + if (!memcmp(data_lanes, rx_ctrl[i][j], + num_data_lanes * sizeof(u8))) + return i; + } + } + + return -EINVAL; +} + +static int kmb_cam_bound(struct v4l2_async_notifier *n, + struct v4l2_subdev *sd, + struct v4l2_async_subdev *asd) +{ + struct v4l2_device *v4l2_dev = n->v4l2_dev; + struct kmb_camera *kmb_cam = + container_of(v4l2_dev, struct kmb_camera, v4l2_dev); + struct kmb_camera_receiver *receiver = + container_of(asd, struct kmb_camera_receiver, asd); + int ret; + + ret = kmb_isp_init(&receiver->isp, kmb_cam->dev, + &receiver->csi2_config, &kmb_cam->xlink_cam); + if (ret < 0) + return ret; + + ret = kmb_isp_register_entities(&receiver->isp, &kmb_cam->v4l2_dev); + if (ret < 0) + goto error_isp_cleanup; + + ret = media_create_pad_link(&sd->entity, 0, + &receiver->isp.subdev.entity, + KMB_ISP_SINK_PAD_SENSOR, + MEDIA_LNK_FL_IMMUTABLE | + MEDIA_LNK_FL_ENABLED); + if (ret < 0) { + dev_err(kmb_cam->dev, "Fail to link %s->%s entities", + sd->entity.name, receiver->isp.subdev.entity.name); + goto error_unregister_entities; + } + + return 0; + +error_unregister_entities: + kmb_isp_unregister_entities(&receiver->isp); +error_isp_cleanup: + kmb_isp_cleanup(&receiver->isp); + + return ret; +} + +static int kmb_cam_complete(struct v4l2_async_notifier *n) +{ + return v4l2_device_register_subdev_nodes(n->v4l2_dev); +} + +static void kmb_cam_unbind(struct v4l2_async_notifier *n, + struct v4l2_subdev *sd, + struct v4l2_async_subdev *asd) +{ + struct kmb_camera_receiver *receiver = + container_of(asd, struct kmb_camera_receiver, asd); + + kmb_isp_unregister_entities(&receiver->isp); + kmb_isp_cleanup(&receiver->isp); +} + +static const struct v4l2_async_notifier_operations notifier_ops = { + .bound = kmb_cam_bound, + .complete = kmb_cam_complete, + .unbind = kmb_cam_unbind +}; + +static int kmb_cam_parse_nodes(struct kmb_camera *kmb_cam, + struct v4l2_async_notifier *n) +{ + struct fwnode_handle *fwnode = NULL; + unsigned int i; + int ret; + + for (i = 0; i < KMB_CAM_NUM_PORTS; i++) { + struct v4l2_fwnode_endpoint ep_data = { + .bus_type = V4L2_MBUS_CSI2_DPHY, + }; + struct kmb_camera_receiver *receiver; + int rx_id; + + fwnode = fwnode_graph_get_endpoint_by_id(dev_fwnode(kmb_cam->dev), + i, 0, + FWNODE_GRAPH_ENDPOINT_NEXT); + if (!fwnode) + continue; + + ret = v4l2_fwnode_endpoint_parse(fwnode, &ep_data); + if (ret < 0) { + dev_err(kmb_cam->dev, + "No endpoint to parse in this fwnode"); + goto error_fwnode_handle_put; + } + + rx_id = get_rx_id(ep_data.bus.mipi_csi2.data_lanes, + ep_data.bus.mipi_csi2.num_data_lanes); + if (rx_id < 0) { + dev_err(kmb_cam->dev, "Invalid RX ID"); + ret = rx_id; + goto error_fwnode_handle_put; + } + + receiver = + v4l2_async_notifier_add_fwnode_remote_subdev(&kmb_cam->v4l2_notifier, + fwnode, + struct kmb_camera_receiver); + if (IS_ERR(receiver)) { + ret = PTR_ERR(receiver); + goto error_fwnode_handle_put; + } + + receiver->csi2_config.rx_id = rx_id; + receiver->csi2_config.num_lanes = + ep_data.bus.mipi_csi2.num_data_lanes; + + fwnode_handle_put(fwnode); + } + + return 0; + +error_fwnode_handle_put: + fwnode_handle_put(fwnode); + + return ret; +} + +static int kmb_cam_probe(struct platform_device *pdev) +{ + struct device *dev = &pdev->dev; + struct v4l2_device *v4l2_dev; + struct kmb_camera *kmb_cam; + int ret; + + kmb_cam = devm_kzalloc(dev, sizeof(*kmb_cam), GFP_KERNEL); + if (!kmb_cam) + return -ENOMEM; + + kmb_cam->dev = dev; + + platform_set_drvdata(pdev, kmb_cam); + + ret = kmb_cam_xlink_init(&kmb_cam->xlink_cam, dev); + if (ret < 0) + return ret; + + strscpy(kmb_cam->media_dev.model, "Keem Bay camera", + sizeof(kmb_cam->media_dev.model)); + kmb_cam->media_dev.dev = dev; + kmb_cam->media_dev.hw_revision = 0; + media_device_init(&kmb_cam->media_dev); + + v4l2_dev = &kmb_cam->v4l2_dev; + v4l2_dev->mdev = &kmb_cam->media_dev; + strscpy(v4l2_dev->name, "keembay-camera", sizeof(v4l2_dev->name)); + + ret = v4l2_device_register(dev, &kmb_cam->v4l2_dev); + if (ret < 0) { + dev_err(kmb_cam->dev, "Fail to register v4l2_device: %d", ret); + goto error_xlink_cleanup; + } + + ret = of_reserved_mem_device_init(dev); + if (ret) + dev_info(dev, "Default CMA memory region will be used!"); + + v4l2_async_notifier_init(&kmb_cam->v4l2_notifier); + ret = kmb_cam_parse_nodes(kmb_cam, &kmb_cam->v4l2_notifier); + if (ret < 0) { + dev_err(kmb_cam->dev, "Fail to parse nodes: %d", ret); + goto error_async_notifier_cleanup; + } + + kmb_cam->v4l2_notifier.ops = ¬ifier_ops; + ret = v4l2_async_notifier_register(&kmb_cam->v4l2_dev, + &kmb_cam->v4l2_notifier); + if (ret < 0) { + dev_err(kmb_cam->dev, "Could not register notifier! %d", ret); + goto error_async_notifier_cleanup; + } + + ret = media_device_register(&kmb_cam->media_dev); + if (ret < 0) { + dev_err(kmb_cam->dev, "Fail to register media device %d", ret); + goto error_async_notifier_unregister; + } + + return 0; + +error_async_notifier_unregister: + v4l2_async_notifier_unregister(&kmb_cam->v4l2_notifier); +error_async_notifier_cleanup: + v4l2_async_notifier_cleanup(&kmb_cam->v4l2_notifier); + v4l2_device_unregister(&kmb_cam->v4l2_dev); +error_xlink_cleanup: + kmb_cam_xlink_cleanup(&kmb_cam->xlink_cam); + + return ret; +} + +static int kmb_cam_remove(struct platform_device *pdev) +{ + struct kmb_camera *kmb_cam = platform_get_drvdata(pdev); + + v4l2_async_notifier_unregister(&kmb_cam->v4l2_notifier); + v4l2_async_notifier_cleanup(&kmb_cam->v4l2_notifier); + + media_device_unregister(&kmb_cam->media_dev); + media_device_cleanup(&kmb_cam->media_dev); + v4l2_device_unregister(&kmb_cam->v4l2_dev); + + kmb_cam_xlink_cleanup(&kmb_cam->xlink_cam); + + return 0; +} + +static const struct of_device_id kmb_cam_dt_match[] = { + {.compatible = "intel,keembay-camera"}, + {} +}; +MODULE_DEVICE_TABLE(of, kmb_cam_dt_match); + +static struct platform_driver keembay_camera_driver = { + .probe = kmb_cam_probe, + .remove = kmb_cam_remove, + .driver = { + .name = "keembay-camera", + .owner = THIS_MODULE, + .of_match_table = kmb_cam_dt_match, + } +}; + +module_platform_driver(keembay_camera_driver); + +MODULE_DESCRIPTION("Intel Keem Bay camera"); +MODULE_LICENSE("GPL"); diff --git a/drivers/media/platform/keembay-camera/keembay-camera.h b/drivers/media/platform/keembay-camera/keembay-camera.h new file mode 100644 index 000000000000..c108c60fdc73 --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-camera.h @@ -0,0 +1,43 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Intel Keem Bay camera driver. + * + * Copyright (C) 2021 Intel Corporation + */ +#ifndef KEEMBAY_CAMERA_H +#define KEEMBAY_CAMERA_H + +#include + +#include "keembay-cam-xlink.h" +#include "keembay-isp.h" + +/** + * struct kmb_camera_receiver - Keem Bay camera receiver + * @asd: V4L2 asynchronous sub-device + * @csi2_config: CSI-2 configuration + * @isp: ISP device + */ +struct kmb_camera_receiver { + struct v4l2_async_subdev asd; + struct kmb_isp_csi2_config csi2_config; + struct kmb_isp isp; +}; + +/** + * struct kmb_cam - Keem Bay camera media device + * @dev: Pointer to basic device structure + * @media_dev: Media device + * @v4l2_dev: V4L2 device + * @v4l2_notifier: V4L2 async notifier + * @xlink_cam: Xlink camera communication handler + */ +struct kmb_camera { + struct device *dev; + struct media_device media_dev; + struct v4l2_device v4l2_dev; + struct v4l2_async_notifier v4l2_notifier; + struct kmb_xlink_cam xlink_cam; +}; + +#endif /* KEEMBAY_CAMERA_H */ diff --git a/drivers/media/platform/keembay-camera/keembay-isp.c b/drivers/media/platform/keembay-camera/keembay-isp.c new file mode 100644 index 000000000000..79f6212e37d9 --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-isp.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Intel Keem Bay camera ISP driver. + * + * Copyright (C) 2021 Intel Corporation + */ +#include "keembay-isp.h" + +/** + * kmb_isp_init - Initialize Kmb isp subdevice + * @kmb_isp: Pointer to kmb isp device + * @dev: Pointer to camera device for which isp will be associated with + * @csi2_config: Csi2 configuration + * @xlink_cam: Xlink camera communication handle + * + * Return: 0 if successful, error code otherwise. + */ +int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev, + struct kmb_isp_csi2_config *csi2_config, + struct kmb_xlink_cam *xlink_cam) +{ + return 0; +} + +/** + * kmb_isp_cleanup - Cleanup kmb isp sub-device resourcess allocated in init + * @kmb_isp: Pointer to kmb isp sub-device + */ +void kmb_isp_cleanup(struct kmb_isp *kmb_isp) +{ } + +/** + * kmb_isp_register_entities - Register entities + * @kmb_isp: pointer to kmb isp device + * @v4l2_dev: pointer to V4L2 device drivers + * + * Register all entities in the pipeline and create + * links between them. + * + * Return: 0 if successful, error code otherwise. + */ +int kmb_isp_register_entities(struct kmb_isp *kmb_isp, + struct v4l2_device *v4l2_dev) +{ + return 0; +} + +/** + * kmb_isp_unregister_entities - Unregister this media's entities + * @kmb_isp: pointer to kmb isp device + */ +void kmb_isp_unregister_entities(struct kmb_isp *kmb_isp) +{ } diff --git a/drivers/media/platform/keembay-camera/keembay-isp.h b/drivers/media/platform/keembay-camera/keembay-isp.h new file mode 100644 index 000000000000..35af6c644676 --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-isp.h @@ -0,0 +1,90 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Intel Keem Bay camera ISP driver. + * + * Copyright (C) 2021 Intel Corporation + */ +#ifndef KEEMBAY_ISP_H +#define KEEMBAY_ISP_H + +#include +#include + +#define KMB_ISP_DRV_NAME "keembay-camera-isp" + +#define KMB_ISP_SINK_PAD_SENSOR 0 +#define KMB_ISP_SINK_PAD_CFG 1 +#define KMB_ISP_SRC_PAD_VID 2 +#define KMB_ISP_PADS_NUM 3 + +/** + * struct kmb_isp_csi2_config - Isp csi2 configuration + * @rx_id: Source port id + * @num_lanes: Number of physical lanes + */ +struct kmb_isp_csi2_config { + u32 rx_id; + u32 num_lanes; +}; + +/** + * struct kmb_isp - Keem Bay camera ISP device structure + * @dev: Pointer to basic device structure + * @lock: Mutex serilizing access to ISP device + * @thread: Pointer to worker thread data + * @xlink_cam: Xlink camera communication handler + * @msg_phy_addr: ISP channel physical CMA address + * @msg_vaddr: ISP channel virtual CMA address + * @cfg_q_lock: Mutex to serialize access to isp cfg bufferss queue + * @isp_cfgs_queue: Isp cfg buffers queue + * @isp_streaming: Flag to indicate ISP state + * @source_streaming: Flag to indicate source state + * @source_stopped: Completion to wait until VPU source is stopped + * @subdev: V4L2 sub-device + * @pads: Array of supported isp pads + * @active_pad_fmt: Array holding active pad formats + * @try_pad_fmt: Array holding try pad formats + * @csi2_config: CSI2 configuration + * @source_fmt: Pointer to isp source format + * @sequence: frame sequence number + */ +struct kmb_isp { + struct device *dev; + struct mutex lock; + struct task_struct *thread; + + struct kmb_xlink_cam *xlink_cam; + + dma_addr_t msg_phy_addr; + void *msg_vaddr; + + struct mutex cfg_q_lock; + struct list_head isp_cfgs_queue; + + bool isp_streaming; + bool source_streaming; + struct completion source_stopped; + + struct v4l2_subdev subdev; + struct media_pad pads[KMB_ISP_PADS_NUM]; + + struct v4l2_subdev_format active_pad_fmt[KMB_ISP_PADS_NUM]; + + struct v4l2_subdev_format try_pad_fmt[KMB_ISP_PADS_NUM]; + + struct kmb_isp_csi2_config csi2_config; + const struct kmb_isp_source_format *source_fmt; + + u32 sequence; +}; + +int kmb_isp_init(struct kmb_isp *kmb_isp, struct device *dev, + struct kmb_isp_csi2_config *csi2_config, + struct kmb_xlink_cam *xlink_cam); +void kmb_isp_cleanup(struct kmb_isp *kmb_isp); + +int kmb_isp_register_entities(struct kmb_isp *kmb_isp, + struct v4l2_device *v4l2_dev); +void kmb_isp_unregister_entities(struct kmb_isp *kmb_isp); + +#endif /* KEEMBAY_ISP_H */ From patchwork Fri Mar 19 18:06:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405080 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-21.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0C4EDC433E3 for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D02EA6198B for ; Fri, 19 Mar 2021 18:07:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230245AbhCSSHW (ORCPT ); Fri, 19 Mar 2021 14:07:22 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230294AbhCSSGv (ORCPT ); Fri, 19 Mar 2021 14:06:51 -0400 IronPort-SDR: Q6Tn0H46jE9neoBo2//Jlh2x5BzUhAhNEUFDsupXlWc4vqjWeToUqNUAlcNvYCv8vY9fC7TAXz 8um17rdf5VOA== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036017" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036017" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:50 -0700 IronPort-SDR: QShnhjiEL+CQeUpdsD07gK8tj/RoLfulFn6XIYarFL6uUhxJ8iEkKKEpO6MegwraToyBfh0aYC ZUmHlariyzLw== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605729" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:48 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 04/10] uapi: Keem Bay ISP Parameters data types Date: Fri, 19 Mar 2021 18:06:26 +0000 Message-Id: <20210319180632.585-5-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Gjorgji Rosikopulos ISP parameters passed to the “keembay-metadata-params” metadata output video node Signed-off-by: Gjorgji Rosikopulos Co-developed-by: Ivan Dimitrov Signed-off-by: Ivan Dimitrov Co-developed-by: Martina Krasteva Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- MAINTAINERS | 1 + include/uapi/linux/keembay-isp-ctl.h | 796 +++++++++++++++++++++++++++++++++++ 2 files changed, 797 insertions(+) create mode 100644 include/uapi/linux/keembay-isp-ctl.h diff --git a/MAINTAINERS b/MAINTAINERS index 76082714a76f..955f9f6a195d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -1973,6 +1973,7 @@ S: Maintained T: git git://linuxtv.org/media_tree.git F: Documentation/devicetree/bindings/media/intel,keembay-camera.yaml F: drivers/media/platform/keembay-camera/ +F: include/uapi/linux/keembay-isp-ctl.h ARM/IP FABRICS DOUBLE ESPRESSO MACHINE SUPPORT M: Lennert Buytenhek diff --git a/include/uapi/linux/keembay-isp-ctl.h b/include/uapi/linux/keembay-isp-ctl.h new file mode 100644 index 000000000000..86e1654067f0 --- /dev/null +++ b/include/uapi/linux/keembay-isp-ctl.h @@ -0,0 +1,796 @@ +/* SPDX-License-Identifier: GPL-2.0-only WITH Linux-syscall-note */ +/* + * Intel Keem Bay camera control parameters and statistics + * + * Copyright (C) 2021 Intel Corporation + */ +#ifndef KEEMBAY_ISP_CTL_H +#define KEEMBAY_ISP_CTL_H + +#include +#include + +#define KMB_CAM_MAX_EXPOSURES 3 + +/* Table max sizes */ +#define KMB_CAM_LCA_MESH_SIZE (32 * 24) +#define KMB_CAM_SHARPEN_RADIAL_SIZE 256 +#define KMB_CAM_LUT3D_SIZE (16 * 16 * 16 * 4) +#define KMB_CAM_LSC_SIZE ((64 + 4) * 64 * 4) +#define KMB_CAM_GAMMA_SIZE (512 * 4) +#define KMB_CAM_STATIC_DEFECT_SIZE 1000 +#define KMB_CAM_CHROMA_LUT_SIZE 1024 +#define KMB_CAM_WARP_MESH_SIZE (480 * 270) +#define KMB_CAM_HDR_TM_LUTS_SIZE (((67 * 16) + 63) & ~63) + +/* Statistics max sizes */ +#define KMB_CAM_AE_AWB_STATS_SIZE (64 * 64) +#define KMB_CAM_AF_STATS_SIZE (64 * 64) +#define KMB_CAM_HIST_LUMA_SIZE 256 +#define KMB_CAM_HIST_RGB_SIZE (3 * 128) +#define KMB_CAM_FLICKER_ROWS_SIZE (65535 + 1) /* align to 64 */ +#define MAX_DHZ_AIRLIGHT_STATS_SIZE (4 * 128) + +/** + * enum kmb_vpu_isp_bayer_order - KMB sensor Bayer arrangement format types + * + * @KMB_ISP_BAYER_ORDER_GRBG: Gr R B Gr + * @KMB_ISP_BAYER_ORDER_RGGB: R Gr Gr B + * @KMB_ISP_BAYER_ORDER_GBRG: Gr B R Gr + * @KMB_ISP_BAYER_ORDER_BGGR: B Gr Gr R + */ +enum kmb_isp_bayer_order { + KMB_ISP_BAYER_ORDER_GRBG = 0, + KMB_ISP_BAYER_ORDER_RGGB = 1, + KMB_ISP_BAYER_ORDER_GBRG = 2, + KMB_ISP_BAYER_ORDER_BGGR = 3, +} __packed; + +/** + * struct kmb_blc_params - KMB Black Level Correction parameters + * + * @coeff1: Black level correction coefficient parameter. Range [0 - 4096] + * @coeff2: Black level correction coefficient parameter. Range [0 - 4096] + * @coeff3: Black level correction coefficient parameter. Range [0 - 4096] + * @coeff4: Black level correction coefficient parameter. Range [0 - 4096] + */ +struct kmb_blc_params { + __u32 coeff1; + __u32 coeff2; + __u32 coeff3; + __u32 coeff4; +} __packed; + +/** + * struct kmb_sigma_dns_params - KMB Sigma Denoise parameters + * + * @noise: Sigma denoise noise parameter. Range [0 - 65535] + * @threshold1: Sigma denoise min threshold1 parameter. Range [0 - 255] + * @threshold2: Sigma denoise max threshold2 parameter. Range [0 - 255] + * @threshold3: Sigma denoise min threshold3 parameter. Range [0 - 255] + * @threshold4: Sigma denoise max threshold4 parameter. Range [0 - 255] + * @threshold5: Sigma denoise min threshold5 parameter. Range [0 - 255] + * @threshold6: Sigma denoise max threshold6 parameter. Range [0 - 255] + * @threshold7: Sigma denoise min threshold7 parameter. Range [0 - 255] + * @threshold8: Sigma denoise max threshold8 parameter. Range [0 - 255] + */ +struct kmb_sigma_dns_params { + __u32 noise; + __u32 threshold1; + __u32 threshold2; + __u32 threshold3; + __u32 threshold4; + __u32 threshold5; + __u32 threshold6; + __u32 threshold7; + __u32 threshold8; +} __packed; + +/** + * struct kmb_lsc_params - KMB Lens Shading Correction parameters + * + * @threshold: Lens shading correction threshold parameter + * @width: Lens shading correction gain1 parameter. Range [1 - 64]. + * Must be a multiple of 4 + * @height: Lens shading correction gain2 parameter. Range [1 - 64]. + * Must be an even number + * @gain_mesh: Lens shading correction gain mesh table + */ +struct kmb_lsc_params { + __u32 threshold; + __u32 width; + __u32 height; + __u8 gain_mesh[KMB_CAM_LSC_SIZE]; +} __packed; + +/** + * struct kmb_raw_params - KMB Raw parameters + * + * @awb_stats_en: Enable AE/AWB stats output + * @awb_rgb_hist_en: Enable RGB histogram output + * @af_stats_en: Enable AF stats output + * @luma_hist_en: Enable Luma histogram output + * @flicker_accum_en: Enable flicker detection row accumulation output + * @bad_pixel_fix_en: Enable Hot/Cold pixel suppression + * @grgb_imb_en: Enable Gr/Gb imbalance correction + * @mono_imbalance_en: Enable mono imbalance correction + * @gain1: Raw gain1 parameter + * @gain2: Raw gain2 parameter + * @gain3: Raw gain3 parameter + * @gain4: Raw gain4 parameter + * @stop1: Raw stop1 parameter + * @stop2: Raw stop2 parameter + * @stop3: Raw stop3 parameter + * @stop4: Raw stop4 parameter + * @threshold1: Raw threshold1 parameter + * @alpha1: Raw alpha1 parameter. Range [0 - 15] + * @alpha2: Raw alpha2 parameter. Range [0 - 15] + * @alpha3: Raw alpha3 parameter. Range [0 - 15] + * @alpha4: Raw alpha4 parameter. Range [0 - 15] + * @threshold2: Raw threshold2 parameter. Range [0 - 2047] + * @static_defect_size: static_defect_map size parameter. Range [0 - 65536] + * @static_defect_map: Static defect map + * @start_row: Raw start_row parameter + * @end_row: Raw end_row parameter + */ +struct kmb_raw_params { + __u32 awb_stats_en; + __u32 awb_rgb_hist_en; + __u32 af_stats_en; + __u32 luma_hist_en; + __u32 flicker_accum_en; + __u32 bad_pixel_fix_en; + __u32 grgb_imb_en; + __u32 mono_imbalance_en; + __u32 gain1; + __u32 gain2; + __u32 gain3; + __u32 gain4; + __u32 stop1; + __u32 stop2; + __u32 stop3; + __u32 stop4; + __u32 threshold1; + __u32 alpha1; + __u32 alpha2; + __u32 alpha3; + __u32 alpha4; + __u32 threshold2; + __u32 static_defect_size; + __u8 static_defect_map[KMB_CAM_STATIC_DEFECT_SIZE]; + __u32 start_row; + __u32 end_row; +} __packed; + +/** + * struct kmb_ae_awb_params - KMB AE/AWB statistics parameters + * + * @start_x: AE/AWB start_x parameter + * @start_y: AE/AWB start_y parameter + * @width: AE/AWB width parameter + * @height: AE/AWB height parameter + * @skip_x: AE/AWB skip_x parameter + * @skip_y: AE/AWB skip_y parameter + * @patches_x: AE/AWB patches_x parameter + * @patches_y: AE/AWB patches_y parameter + * @threshold1: AE/AWB threshold1 parameter + * @threshold2: AE/AWB threshold2 parameter + */ +struct kmb_ae_awb_params { + __u32 start_x; + __u32 start_y; + __u32 width; + __u32 height; + __u32 skip_x; + __u32 skip_y; + __u32 patches_x; + __u32 patches_y; + __u16 threshold1; + __u16 threshold2; +} __packed; + +/** + * struct kmb_af_params - KMB Auto Focus parameters + * + * @start_x: AF start_x parameter + * @start_y: AF start_y parameter + * @width: AF width parameter + * @height: AF height parameter + * @patches_x: AF patches_x parameter + * @patches_y: AF patches_y parameter + * @coeff: AF filter coeff parameter + * @threshold1: AF filer threshold1 parameter + * @threshold2: AF filer threshold2 parameter + * @coeffs1: AF filter coeffs1 parameter + * @coeffs2: AF filter coeffs2 parameter + */ +struct kmb_af_params { + __u32 start_x; + __u32 start_y; + __u32 width; + __u32 height; + __u32 patches_x; + __u32 patches_y; + __s32 coeff; + __s32 threshold1; + __s32 threshold2; + __s32 coeffs1[11]; + __s32 coeffs2[11]; +} __packed; + +/** + * struct kmb_hist_params - KMB Hist parameters + * + * @start_x: Hist start_x parameter. Range [0 - 1] + * @start_y: Hist start_y parameter. Range [0 - 1] + * @end_x: Hist end_x parameter. Range [0 - 1] + * @end_y: Hist end_y parameter. Range [0 - 1] + * @matrix: Hist matrix parameter. Range [0.0 - 8.0] + * @weight: Hist weight parameter. Range [0.0 - 1.0] + */ +struct kmb_hist_params { + __u32 start_x; + __u32 start_y; + __u32 end_x; + __u32 end_y; + __u16 matrix[9]; + __u16 weight[3]; +} __packed; + +/** + * struct kmb_lca_params - KMB Lateral Chromatic Aberration parameters + * + * @coeff: LCA coeff parameter + */ +struct kmb_lca_params { + __u8 coeff[KMB_CAM_LCA_MESH_SIZE]; +} __packed; + +/** + * struct kmb_debayer_params - KMB Debayer parameters + * + * @coeff1: Filter coeff1 parameter + * @multiplier1: Filter multiplier1 parameter + * @multiplier2: Filter multiplier2 parameter + * @coeff2: Filter coeff2 parameter + * @coeff3: Filter coeff3 parameter + * @coeff4: Filter coeff4 parameter + */ +struct kmb_debayer_params { + __s32 coeff1; + __u32 multiplier1; + __u32 multiplier2; + __s32 coeff2; + __s32 coeff3; + __s32 coeff4; +} __packed; + +/** + * struct kmb_hdr_params - KMB HDR parameters + * + * @ratio: HDR ratio parameter. Range [0 - 65536] + * @scale: HDR scale parameter + * @offset1: HDR offset1 parameter. Range [-4095 - 0] + * @slope1: HDR slope1 parameter. Range [0 - 4095] + * @offset2: HDR offset2 parameter. Range [-4095 - 0] + * @slope2: HDR slope2 parameter. Range [0 - 4095] + * @offset3: HDR offset3 parameter. Range [0 - 4095] + * @slope3: HDR slope3 parameter. Range [0 - 4095] + * @offset4: HDR offset4 parameter. Range [0 - 4095] + * @gain1: HDR gain1 parameter. Range [0 - 4095] + * @blur1: HDR blur1 parameter. Range [0.0 - 255.0] + * @blur2: HDR blur2 parameter. Range [0 - 255] + * @contrast1: HDR contrast1 parameter + * @contrast2: HDR contrast2 parameter + * @enable1: HDR enable1 parameter + * @enable2: HDR enable2 parameter + * @offset5: HDR offset5 parameter + * @gain2: HDR gain2 parameter + * @offset6: HDR offset6 parameter. Range [0 - 1024] + * @strength: HDR strength parameter. Range [0 - 65536] + * @tm_lut: HDR tm lut parameter + * @offset7: HDR offset7 parameter. Range [0 - 65536] + * @shift: HDR shift parameter + * @field1: HDR filed1 parameter + * @field2: HDR field2 parameter + * @gain3: HDR gain3 parameter. Range [0 - 255] + * @min: HDR min parameter. Range [0 - 4095] + */ +struct kmb_hdr_params { + __u32 ratio[2]; + __u32 scale[3]; + __s32 offset1; + __u32 slope1; + __s32 offset2; + __u32 slope2; + __s32 offset3; + __u32 slope3; + __s32 offset4; + __u32 gain1; + __u32 blur1[3]; + __u32 blur2[5]; + __u32 contrast1; + __u32 contrast2; + __u32 enable1; + __u32 enable2; + __s32 offset5; + __u32 gain2; + __s32 offset6; + __u32 strength; + __u8 tm_lut[KMB_CAM_HDR_TM_LUTS_SIZE]; + __u16 offset7; + __u32 shift; + __u16 field1; + __u16 field2; + __u8 gain3; + __u16 min; +} __packed; + +/** + * struct kmb_dog_dns_params - KMB Difference-of-Gaussians DNS parameters + * + * @threshold: Filter threshold parameter. Range [0 - 255] + * @strength: Filter strength parameter. Range [0 - 255] + * @coeffs11: Filter coeffs11 parameter. Range [0 - 1023] + * @coeffs15: Filter coeffs15 parameter. Range [0 - 1023] + */ +struct kmb_dog_dns_params { + __u32 threshold; + __u32 strength; + __u8 coeffs11[6]; + __u8 coeffs15[8]; +} __packed; + +/** + * struct kmb_luma_dns_params - KMB Luma DNS parameters + * + * @threshold: Luma DNS threshold parameter. Range [0 - 32768] + * @slope: Luma DNS slope parameter. Range [0 - 2048] + * @shift: Luma DNS shift parameter. Range [0 - 255] + * @alpha: Luma DNS alpha parameter. Range [0 - 127] + * @weight: Luma DNS weight parameter. Range [0 - 4294967295] + * @per_pixel_alpha_en: Enable adapt alpha + * @gain_bypass_en: Enable gain bypass + */ +struct kmb_luma_dns_params { + __u32 threshold; + __u32 slope; + __u32 shift; + __u32 alpha; + __u32 weight; + __u32 per_pixel_alpha_en; + __u32 gain_bypass_en; +} __packed; + +/** + * struct kmb_sharpen_params - KMB Sharpen parameters + * + * @coeffs1: Filter coeffs1 parameter. Range [0 - 255] + * @coeffs2: Filter coeffs2 parameter. Range [0 - 255] + * @coeffs3: Filter coeffs3 parameter. Range [0 - 255] + * @shift: Filter shift parameter. Range [0 - 255] + * @gain1: Filter gain1 parameter. Range [0 - 2047] + * @gain2: Filter gain2 parameter. Range [0 - 2047] + * @gain3: Filter gain3 parameter. Range [0 - 2047] + * @gain4: Filter gain4 parameter. Range [0 - 2047] + * @gain5: Filter gain5 parameter. Range [0 - 255] + * @stops1: Filter stops1 parameter. Range [0 - 4095] + * @gains: Filter gains parameter. Range [0 - 256] + * @stops2: Filter stops2 parameter. Range [0 - 4095] + * @overshoot: Filter overshoot parameter. Range [0 - 256] + * @undershoot: Filter undershoot parameter. Range [0 - 256] + * @alpha: Filter alpha parameter. Range [0 - 256] + * @gain6: Filter gain6 parameter. Range [0 - 255] + * @offset: Filter offset parameter. Range [0 - 1023] + * @radial_lut: Sharpen radial LUT parameter. Range [0 - 255] + */ +struct kmb_sharpen_params { + __u16 coeffs1[6]; + __u16 coeffs2[6]; + __u16 coeffs3[6]; + __u32 shift; + __u32 gain1; + __u32 gain2; + __u32 gain3; + __u32 gain4; + __u32 gain5; + __u32 stops1[3]; + __u32 gains[3]; + __u32 stops2[4]; + __u32 overshoot; + __u32 undershoot; + __u32 alpha; + __u32 gain6; + __u32 offset; + __u8 radial_lut[KMB_CAM_SHARPEN_RADIAL_SIZE]; +} __packed; + +/** + * struct kmb_chroma_gen_params - KMB Chroma GEN parameters + * + * @epsilon: Chroma GEN epsilon parameter. Range [0 - 255] + * @coeff1: Chroma GEN coeff1 parameter. Range [0 - 1024] + * @coeff2: Chroma GEN coeff2 parameter. Range [0 - 1024] + * @coeff3: Chroma GEN coeff3 parameter. Range [0 - 1024] + * @coeff4: Chroma GEN coeff4 parameter. Range [0 - 255] + * @coeff5: Chroma GEN coeff5 parameter. Range [0 - 255] + * @coeff6: Chroma GEN coeff6 parameter. Range [0 - 255] + * @strength1: Chroma GEN strength1 parameter. Range [0 - 255] + * @strength2: Chroma GEN strength2 parameter. Range [0 - 255] + * @coeffs: Chroma GEN coeffs parameter . Range [0 - 255] + * @offset1: Chroma GEN offset1 parameter. Range [0 - 255] + * @slope1: Chroma GEN slope1 parameter. Range [0 - 255] + * @slope2: Chroma GEN slope2 parameter. Range [0 - 255] + * @offset2: Chroma GEN offset2 parameter. Range [0 - 255] + * @limit: Chroma GEN limit parameter. Range [0 - 767] + */ +struct kmb_chroma_gen_params { + __u32 epsilon; + __u32 coeff1; + __u32 coeff2; + __u32 coeff3; + __u32 coeff4; + __u32 coeff5; + __u32 coeff6; + __u32 strength1; + __u32 strength2; + __u32 coeffs[3]; + __s32 offset1; + __u32 slope1; + __u32 slope2; + __s32 offset2; + __u32 limit; +} __packed; + +/** + * struct kmb_median_params - KMB Median parameters + * + * @size: Filter size parameter. Range [1;3;5;7] + * @slope: Filter slope parameter. Range [0 - 128] + * @offset: Filter offset parameter. Range [-32 - 32] + */ +struct kmb_median_params { + __u32 size; + __u32 slope; + __s32 offset; +} __packed; + +/** + * struct kmb_chroma_dns_params - KMB Chroma Denoise parameters + * + * @limit: Filter limit parameter + * @enable: Filter enable parameter + * @threshold1: Filter threshold1 parameter + * @threshold2: Filter threshold2 parameter + * @threshold3: Filter threshold3 parameter. Range [0 - 255] + * @threshold4: Filter threshold4 parameter. Range [0 - 255] + * @threshold5: Filter threshold5 parameter. Range [0 - 255] + * @threshold6: Filter threshold6 parameter. Range [0 - 255] + * @threshold7: Filter threshold7 parameter. Range [0 - 255] + * @threshold8: Filter threshold8 parameter. Range [0 - 255] + * @slope1: Filter slope1 parameter. Range [0 - 255] + * @offset1: Filter offset1 parameter. Range [0 - 255] + * @slope2: Filter slope2 parameter. Range [0 - 255] + * @offset2: Filter offset2 parameter. Range [0 - 255] + * @grey1: Filter grey1 parameter. Range [0 - 255] + * @grey2: Filter grey2 parameter. Range [0 - 255] + * @grey3: Filter grey3 parameter. Range [0 - 255] + * @coeff1: Filter coeff1 parameter. Range [0 - 255] + * @coeff2: Filter coeff2 parameter. Range [0 - 255] + * @coeff3: Filter coeff3 parameter. Range [0 - 255] + */ +struct kmb_chroma_dns_params { + __u32 limit; + __u32 enable; + __u32 threshold1; + __u32 threshold2; + __u32 threshold3; + __u32 threshold4; + __u32 threshold5; + __u32 threshold6; + __u32 threshold7; + __u32 threshold8; + __u32 slope1; + __s32 offset1; + __u32 slope2; + __s32 offset2; + __u32 grey1; + __u32 grey2; + __u32 grey3; + __u32 coeff1; + __u32 coeff2; + __u32 coeff3; +} __packed; + +/** + * struct kmb_color_comb_params - KMB Color Combine parameters + * + * @matrix: Color combine matrix parameter. Range [-8.0 - 8.0] + * @offsets:Color combine offsets parameter. Range [-1.0 - 1.0] + * @coeff1: Color combine coeff1 parameter. Range [0 - 1023] + * @coeff2: Color combine coeff2 parameter. Range [0 - 1023] + * @coeff3: Color combine coeff3 parameter. Range [0 - 1023] + * @lut_3d: Color combine LUT 3D parameter. Range [0 - 4095] + * @enable: Color combine enable parameter + * @weight1: Color combine weight1 parameter. Range [0 - 255] + * @weight2: Color combine weight2 parameter. Range [0 - 255] + * @weight3: Color combine weight3 parameter. Range [0 - 255] + * @limit1: Color combine limit1 parameter. Range [0 - 32766] + * @limit2: Color combine limit2 parameter. Range [-32766 - 0] + * @offset1: Color combine offset1 parameter. Range [0 - 8192] + * @offset2: Color combine offset2 parameter. Range [0 - 8192] + */ +struct kmb_color_comb_params { + __u16 matrix[9]; + __u16 offsets[3]; + __u32 coeff1; + __u32 coeff2; + __u32 coeff3; + __u8 lut_3d[KMB_CAM_LUT3D_SIZE]; + __u32 enable; + __u32 weight1; + __u32 weight2; + __u32 weight3; + __u32 limit1; + __s32 limit2; + __s32 offset1; + __s32 offset2; +} __packed; + +/** + * struct kmb_lut_params - KMB lut parameters + * + * @size: Lut size parameter. Range [0 - 128] + * @table: Lut table parameter + * @matriix: Lut matrix parameter + * @offsets: Lut offsets pparameter + */ +struct kmb_lut_params { + __u32 size; + __u8 table[KMB_CAM_GAMMA_SIZE]; + __u16 matrix[3 * 3]; + __u16 offsets[3]; +} __packed; + +/** + * struct kmb_tnf_params - KMB Temporal Noise Filter parameters + * + * @factor: Filter factor parameter. Range [0 - 255] + * @gain: Filter gain parameter. Range [0 - 4095] + * @offset1: Filter offset1 parameter. Range [0 - 8192] + * @slope1: Filter slope1 parameter. Range [0 - 512] + * @offset2: Filter offset2 parameter. Range [0 - 8192] + * @slope2: Filter slope2 parameter. Range [0 - 512] + * @min1: Filter min1 parameter. Range [0 - 65535] + * @min2: Filter min2 parameter. Range [0 - 65535] + * @value: Filter value parameter. Range [0 - 255] + * @enable: Filter enable parameter + * @chroma_lut0: First chroma LUT. Range [0 - 4095] + * @chroma_lut1: Second chroma LUT. Range [0 - 4095] + */ +struct kmb_tnf_params { + __u32 factor; + __u32 gain; + __u32 offset1; + __u32 slope1; + __u32 offset2; + __u32 slope2; + __u32 min1; + __u32 min2; + __u32 value; + __u32 enable; + __u8 chroma_lut0[KMB_CAM_CHROMA_LUT_SIZE]; + __u8 chroma_lut1[KMB_CAM_CHROMA_LUT_SIZE]; +} __packed; + +/** + * struct kmb_dehaze_params - KMB dehaze parameters + * + * @gain1: Dehaze gain1 parameter. Range [0 - 1023] + * @min: Dehaze min parameter. Range [0 - 255] + * @strength1: Dehaze strength1 parameter. Range [0 - 255] + * @strength2: Dehaze strength2 parameter. Range [0 - 65535] + * @gain2: Dehaze gain2 parameter. Range [0 - 255] + * @saturation: Dehaze saturation parameter. Range [0 - 127] + * @value1: Dehaze value1 parameter. Range [0 - 4095] + * @value2: Dehaze value2 parameter. Range [0 - 4095] + * @value3: Dehaze value3 parameter. Range [0 - 4095] + * @filter: Dehaze filter parameter. Range [0 - 255] + */ +struct kmb_dehaze_params { + __u32 gain1; + __u32 min; + __u32 strength1; + __u32 strength2; + __u32 gain2; + __u32 saturation; + __u32 value1; + __u32 value2; + __u32 value3; + __u32 filter[3]; +} __packed; + +/** + * struct kmb_warp_params - KMB Warp filter parameters + * + * @type: Warp filter type parameter. Range [0 - 1] + * @relative: Warp filter relative parameter. Range [0 - 1] + * @format: Warp filter format parameter. Range [0 - 1] + * @position: Warp filter position parameter + * @mesh_grid: Warp mesh grid + * @width: Warp filter width parameter + * @height: Warp filter height parameter + * @stride: Warp filter stride parameter + * @enable: Warp filter enable parameter + * @matrix: Warp matrix parameter + * @mode: Warp filter mode parameter. Range [0 - 1] + * @values: Warp filter values parameter. Range [0 - 128] + */ +struct kmb_warp_params { + __u8 type; + __u8 relative; + __u8 format; + __u8 position; + __u8 mesh_grid[KMB_CAM_WARP_MESH_SIZE]; + __u16 width; + __u16 height; + __u32 stride; + __u8 enable; + __u32 matrix[9]; + __u8 mode; + __u16 values[3]; +} __packed; + +/** + * struct kmb_isp_params_flags - Bits to indicate which params + * need to be updated + * + * @blc: 1 = update, 0 = do not update. + * @sigma_dns: 1 = update, 0 = do not update. + * @lsc: 1 = update, 0 = do not update. + * @raw: 1 = update, 0 = do not update. + * @ae_awb: 1 = update, 0 = do not update. + * @af: 1 = update, 0 = do not update. + * @histogram: 1 = update, 0 = do not update. + * @lca: 1 = update, 0 = do not update. + * @debayer: 1 = update, 0 = do not update. + * @dog_dns: 1 = update, 0 = do not update. + * @luma_dns: 1 = update, 0 = do not update. + * @sharpen: 1 = update, 0 = do not update. + * @chroma_gen: 1 = update, 0 = do not update. + * @median: 1 = update, 0 = do not update. + * @chroma_dns: 1 = update, 0 = do not update. + * @color_comb: 1 = update, 0 = do not update. + * @hdr: 1 = update, 0 = do not update. + * @lut: 1 = update, 0 = do not update. + * @tnf: 1 = update, 0 = do not update. + * @dehaze: 1 = update, 0 = do not update. + * @warp: 1 = update, 0 = do not update. + * @reserved: reserved for future use and for alignment + */ +struct kmb_isp_params_flags { + __u32 blc:1; + __u32 sigma_dns:1; + __u32 lsc:1; + __u32 raw:1; + __u32 ae_awb:1; + __u32 af:1; + __u32 histogram:1; + __u32 lca:1; + __u32 debayer:1; + __u32 dog_dns:1; + __u32 luma_dns:1; + __u32 sharpen:1; + __u32 chroma_gen:1; + __u32 median:1; + __u32 chroma_dns:1; + __u32 color_comb:1; + __u32 hdr:1; + __u32 lut:1; + __u32 tnf:1; + __u32 dehaze:1; + __u32 warp:1; + __u32 reserved:11; +} __packed; + +/** + * struct kmb_isp_params - KMB ISP parameters structure + * + * @update: Select which parameters to apply, see kmb_vpu_isp_params_flags + * @blc: Black Level correction parameters + * @sigma_dns: Sigma denoise parameters + * @lsc: Lens Shading Correction parameters + * @raw: Raw parameters + * @ae_awb: Auto exposure/Auto white balance parameters + * @af: Auto focus parameters + * @histogram: Histogram parameters + * @lca: Lateral Chromatic Aberration filter parameters + * @debayer: SIPP Bayer demosaicing filter parameters + * @dog_dns: Difference-of-Gaussians filter parameters + * @luma_dns: Luma denoise parameters + * @sharpen: Sharpen filter parameters + * @chroma_gen: Chroma GEN parameters + * @median: Median hardware filter parameters + * @chroma_dns: Chroma Denoise hardware filter parameters + * @color_comb: Color Combine parameters + * @hdr: HDR parameters applied only in HDR mode + * @lut: LUT parameters + * @tnf: Temporal Noise Filter parameters + * @dehaze: Dehaze parameters + * @warp: Warp filter parameters + * + * Each struct represents a filter and its settings which are applied on the raw + * image. + */ +struct kmb_isp_params { + struct kmb_isp_params_flags update; + struct kmb_blc_params blc[KMB_CAM_MAX_EXPOSURES]; + struct kmb_sigma_dns_params sigma_dns[KMB_CAM_MAX_EXPOSURES]; + struct kmb_lsc_params lsc; + struct kmb_raw_params raw; + struct kmb_ae_awb_params ae_awb; + struct kmb_af_params af; + struct kmb_hist_params histogram; + struct kmb_lca_params lca; + struct kmb_debayer_params debayer; + struct kmb_dog_dns_params dog_dns; + struct kmb_luma_dns_params luma_dns; + struct kmb_sharpen_params sharpen; + struct kmb_chroma_gen_params chroma_gen; + struct kmb_median_params median; + struct kmb_chroma_dns_params chroma_dns; + struct kmb_color_comb_params color_comb; + struct kmb_hdr_params hdr; + struct kmb_lut_params lut; + struct kmb_tnf_params tnf; + struct kmb_dehaze_params dehaze; + struct kmb_warp_params warp; +} __packed; + +/** + * struct kmb_isp_stats_flags - Bits to indicate which stats need to be updated + * + * @ae_awb: 1 = updated, 0 = not updated. + * @af: 1 = updated, 0 = not updated. + * @luma_hist: 1 = updated, 0 = not updated. + * @rgb_hist: 1 = updated, 0 = not updated. + * @flicker_rows: 1 = updated, 0 = not updated. + * @dehaze: 1 = updated, 0 = not updated. + * @reserved: reserved for future use and for alignment + */ +struct kmb_isp_stats_flags { + __u32 ae_awb:1; + __u32 af:1; + __u32 luma_hist:1; + __u32 rgb_hist:1; + __u32 flicker_rows:1; + __u32 dehaze:1; + __u32 reserved:26; +} __packed; + +/** + * struct kmb_isp_stats - KMB ISP raw statistics + * + * @exposure: Array with exposure statistics blocks. When HDR mode is not + * enable only statistics with index 0 are valid. Included stats: + * @exposure.ae_awb_stats: Raw AE/AWB statistics. + * @exposure.af_stats: Raw AF statistics. + * @exposure.hist_luma: Luma histogram. + * @exposure.hist_rgb: RGB histogram. + * @exposure.flicker_rows: Flicker detection rows. + * @dehaze: Dehaze statistics it is collected after HDR Fusion in HDR case. + * @update: Select which stats to update, see kmb_vpu_isp_stats_flags + */ +struct kmb_isp_stats { + struct { + __u8 ae_awb_stats[KMB_CAM_AE_AWB_STATS_SIZE]; + __u8 af_stats[KMB_CAM_AF_STATS_SIZE]; + __u8 hist_luma[KMB_CAM_HIST_LUMA_SIZE]; + __u8 hist_rgb[KMB_CAM_HIST_RGB_SIZE]; + __u8 flicker_rows[KMB_CAM_FLICKER_ROWS_SIZE]; + } exposure[KMB_CAM_MAX_EXPOSURES]; + __u8 dehaze[MAX_DHZ_AIRLIGHT_STATS_SIZE]; + struct kmb_isp_stats_flags update; +} __packed; + +#endif /* KEEMBAY_ISP_CTL_H */ From patchwork Fri Mar 19 18:06:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405079 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0803C433ED for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id AF1F06198F for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230315AbhCSSHZ (ORCPT ); Fri, 19 Mar 2021 14:07:25 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230028AbhCSSHA (ORCPT ); Fri, 19 Mar 2021 14:07:00 -0400 IronPort-SDR: T2HQz3YArmexpOXgjLPMFs8CzOVSU0Y3nw/QqzTcy8NQqoj5YQuvJDnGfydr77N7a/H4FQGhDp DTENcwVB4EPw== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036036" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036036" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:59 -0700 IronPort-SDR: 17XftZ/KoeSdf/1ks/gh9Gm4ZqWCUhAgzadCU5i5ZatCznxgJ0e/JGFTptbCvzOBMVoSLvOrXH W7nOBs24icYQ== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605763" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:06:56 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 07/10] media: Keem Bay Camera: Add pipeline support Date: Fri, 19 Mar 2021 18:06:29 +0000 Message-Id: <20210319180632.585-8-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Martina Krasteva Keem Bay pipeline object is responsible for handling of all pipeline management operations. It handles pipeline lifecycle states and configuration. Co-developed-by: Gjorgji Rosikopulos Signed-off-by: Gjorgji Rosikopulos Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- .../platform/keembay-camera/keembay-cam-xlink.c | 133 +++++++++++++++++++ .../platform/keembay-camera/keembay-cam-xlink.h | 7 + .../platform/keembay-camera/keembay-pipeline.c | 145 ++++++++++++++++++++- 3 files changed, 279 insertions(+), 6 deletions(-) diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c index 49a0937bc9fc..5b403874c4a1 100644 --- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.c +++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.c @@ -192,3 +192,136 @@ int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id, return written_size; } + +/** + * kmb_cam_xlink_open_ctrl_channel - Open xlink control channel for communication + * @xlink_cam: Pointer to xlink camera handle + * + * There is only one control channel for xlink camera communication. + * NOTE: The channel is serialized and reference counted. + * + * Return: 0 if successful, error code otherwise + */ +int kmb_cam_xlink_open_ctrl_channel(struct kmb_xlink_cam *xlink_cam) +{ + int ret; + + mutex_lock(&xlink_cam->lock); + + if (xlink_cam->ctrl_chan_refcnt) { + xlink_cam->ctrl_chan_refcnt++; + mutex_unlock(&xlink_cam->lock); + return 0; + } + + ret = xlink_open_channel(&xlink_cam->handle, + KMB_CAM_XLINK_CTRL_CHAN_ID, RXB_TXB, + KMB_CAM_XLINK_CH_MAX_DATA_SIZE, + KMB_CAM_XLINK_CH_TIMEOUT_MS); + if (ret) { + dev_err(xlink_cam->dev, "Failed to open xlink control channel %d", ret); + mutex_unlock(&xlink_cam->lock); + return -ENODEV; + } + + xlink_cam->ctrl_chan_refcnt++; + + mutex_unlock(&xlink_cam->lock); + + return 0; +} + +/** + * kmb_cam_xlink_close_ctrl_channel - Close xlink control channel + * @xlink_cam: Pointer to xlink camera handle + * + * There is only one control channel for xlink camera communication. + * NOTE: The channel is serialized and reference counted. + * + * Return: 0 if successful, error code otherwise + */ +void kmb_cam_xlink_close_ctrl_channel(struct kmb_xlink_cam *xlink_cam) +{ + int ret; + + mutex_lock(&xlink_cam->lock); + + if (WARN_ON(!xlink_cam->ctrl_chan_refcnt)) { + mutex_unlock(&xlink_cam->lock); + return; + } + + if (--xlink_cam->ctrl_chan_refcnt) { + mutex_unlock(&xlink_cam->lock); + return; + } + + ret = xlink_close_channel(&xlink_cam->handle, KMB_CAM_XLINK_CTRL_CHAN_ID); + if (ret) + dev_err(xlink_cam->dev, "Failed to close xlink channel %d", ret); + + mutex_unlock(&xlink_cam->lock); +} + +/** + * kmb_cam_xlink_write_ctrl_msg - Write xlink control message + * @xlink_cam: Pointer to xlink camera handle + * @ctrl_paddr: Physical address of the control message + * @ctrl_type: Control message type + * @expected_result: Control message expected result + * + * For each control message there is ack from the VPU camera. + * This function check the error against expected result. + * NOTE: Because there is only one control channel, the msg/ack + * is sequence serliazed. + * + * Return: 0 if successful, error code otherwise + */ +int kmb_cam_xlink_write_ctrl_msg(struct kmb_xlink_cam *xlink_cam, + dma_addr_t ctrl_paddr, u32 ctrl_type, + u32 expected_result) +{ + size_t init_evt_size = sizeof(struct kmb_ic_ev); + struct kmb_ic_ev init_evt; + int ret; + + mutex_lock(&xlink_cam->lock); + + memset(&init_evt, 0, sizeof(init_evt)); + init_evt.ctrl = ctrl_type; + init_evt.ev_info.user_data_base_addr01 = ctrl_paddr; + ret = xlink_write_volatile(&xlink_cam->handle, + KMB_CAM_XLINK_CTRL_CHAN_ID, + (u8 *)&init_evt, + init_evt_size); + if (ret) { + dev_err(xlink_cam->dev, "Error ret %d ctrl type %d", + ret, ctrl_type); + ret = -ENODEV; + goto error_unlock; + } + + ret = xlink_read_data_to_buffer(&xlink_cam->handle, + KMB_CAM_XLINK_CTRL_CHAN_ID, + (u8 *)&init_evt, + (u32 *)&init_evt_size); + if (ret) { + dev_err(xlink_cam->dev, "Error read ack ret %d", ret); + ret = -ENODEV; + goto error_unlock; + } + if (init_evt.ctrl != expected_result) { + dev_err(xlink_cam->dev, "Error ctrl type %d evt ctrl %d", + ctrl_type, init_evt.ctrl); + ret = -EINVAL; + goto error_unlock; + } + + mutex_unlock(&xlink_cam->lock); + + return 0; + +error_unlock: + mutex_unlock(&xlink_cam->lock); + return ret; +} diff --git a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h index d9a78d847a4b..45e2c003cd33 100644 --- a/drivers/media/platform/keembay-camera/keembay-cam-xlink.h +++ b/drivers/media/platform/keembay-camera/keembay-cam-xlink.h @@ -39,4 +39,11 @@ int kmb_cam_xlink_write_msg(struct kmb_xlink_cam *xlink_cam, int chan_id, int kmb_cam_xlink_read_msg(struct kmb_xlink_cam *xlink_cam, int chan_id, u8 *message, u32 msg_size); +int kmb_cam_xlink_open_ctrl_channel(struct kmb_xlink_cam *xlink_cam); +void kmb_cam_xlink_close_ctrl_channel(struct kmb_xlink_cam *xlink_cam); + +int kmb_cam_xlink_write_ctrl_msg(struct kmb_xlink_cam *xlink_cam, + dma_addr_t ctrl_paddr, u32 ctrl_type, + u32 expected_result); + #endif /* KEEMBAY_CAM_XLINK_H */ diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c index 1ebaa900e4dc..78b2fffa42ee 100644 --- a/drivers/media/platform/keembay-camera/keembay-pipeline.c +++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c @@ -2,13 +2,64 @@ /* * Intel Keem Bay camera pipeline. * - * Copyright (C) 2020 Intel Corporation + * Copyright (C) 2021 Intel Corporation */ +#include +#include +#include +#include + #include +#include "keembay-cam-xlink.h" #include "keembay-pipeline.h" #include "keembay-vpu-cmd.h" +static void kmb_pipe_print_config(struct kmb_pipeline *pipe) +{ + struct kmb_pipe_config_evs *cfg = pipe->pipe_cfg; + struct device *dev = pipe->dev; + unsigned int i; + + dev_dbg(dev, "\tpipe_id %u\n", cfg->pipe_id); + dev_dbg(dev, "\tpipe_type %u\n", cfg->pipe_type); + dev_dbg(dev, "\tsrc_type %u\n", cfg->src_type); + dev_dbg(dev, "\tpipe_trans_hub %u\n", cfg->pipe_trans_hub); + dev_dbg(dev, "\tin_isp_res %ux%u\n", + cfg->in_isp_res.w, cfg->in_isp_res.h); + dev_dbg(dev, "\tout_isp_res %ux%u\n", + cfg->out_isp_res.w, cfg->out_isp_res.h); + dev_dbg(dev, "\tin_isp_stride %u\n", cfg->in_isp_stride); + dev_dbg(dev, "\tin_exp_offsets[0] %u\n\tin_exp_offsets[1] %u\n" + "\tin_exp_offsets[2] %u\n", + cfg->in_exp_offsets[0], cfg->in_exp_offsets[1], + cfg->in_exp_offsets[2]); + + for (i = 0; i < PIPE_OUTPUT_ID_MAX; i++) { + dev_dbg(dev, "\tOUTPUT ID: %d\n", i); + dev_dbg(dev, "\t\tout_min_res %ux%u\n", + cfg->out_min_res[i].w, cfg->out_min_res[i].h); + dev_dbg(dev, "\t\tout_max_res %ux%u\n", + cfg->out_max_res[i].w, cfg->out_max_res[i].h); + } + + for (i = 0; i < PIPE_OUTPUT_ID_MAX; i++) { + dev_dbg(dev, "\tpipe_xlink_chann: %d\n", i); + dev_dbg(dev, "\t\tid: %u %ux%u\n", + cfg->pipe_xlink_chann[i].id, + cfg->pipe_xlink_chann[i].frm_res.w, + cfg->pipe_xlink_chann[i].frm_res.h); + } + + dev_dbg(dev, "\tkeep_aspect_ratio %u\n", cfg->keep_aspect_ratio); + dev_dbg(dev, "\tin_data_width %u\n", cfg->in_data_width); + dev_dbg(dev, "\tin_data_packed %u\n", cfg->in_data_packed); + dev_dbg(dev, "\tout_data_width %u\n", cfg->out_data_width); + dev_dbg(dev, "\tinternal_memory_addr 0x%llx\n", + cfg->internal_memory_addr); + dev_dbg(dev, "\tinternal_memory_size %u\n", cfg->internal_memory_size); +} + /** * kmb_pipe_init - Initialize KMB Pipeline * @pipe: pointer to pipeline object @@ -20,6 +71,20 @@ int kmb_pipe_init(struct kmb_pipeline *pipe, struct device *dev, struct kmb_xlink_cam *xlink_cam) { + pipe->pipe_cfg = dma_alloc_coherent(dev, + sizeof(*pipe->pipe_cfg), + &pipe->pipe_cfg_paddr, 0); + if (!pipe->pipe_cfg) + return -ENOMEM; + + mutex_init(&pipe->lock); + pipe->pending = 0; + pipe->streaming = 0; + pipe->state = KMB_PIPE_STATE_UNCONFIGURED; + + pipe->dev = dev; + pipe->xlink_cam = xlink_cam; + return 0; } @@ -28,7 +93,10 @@ int kmb_pipe_init(struct kmb_pipeline *pipe, struct device *dev, * @pipe: pointer to pipeline object */ void kmb_pipe_cleanup(struct kmb_pipeline *pipe) -{ } +{ + dma_free_coherent(pipe->dev, sizeof(struct kmb_pipe_config_evs), + pipe->pipe_cfg, pipe->pipe_cfg_paddr); +} /** * kmb_pipe_request - Request a pipeline @@ -38,7 +106,13 @@ void kmb_pipe_cleanup(struct kmb_pipeline *pipe) */ int kmb_pipe_request(struct kmb_pipeline *pipe) { - return 0; + int ret; + + ret = kmb_cam_xlink_open_ctrl_channel(pipe->xlink_cam); + if (ret < 0) + dev_err(pipe->dev, "Failed to request control channel"); + + return ret; } /** @@ -46,7 +120,9 @@ int kmb_pipe_request(struct kmb_pipeline *pipe) * @pipe: pointer to pipeline object */ void kmb_pipe_release(struct kmb_pipeline *pipe) -{ } +{ + kmb_cam_xlink_close_ctrl_channel(pipe->xlink_cam); +} /** * kmb_pipe_config_dest - Configure pipeline destination information @@ -56,7 +132,23 @@ void kmb_pipe_release(struct kmb_pipeline *pipe) */ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id, struct kmb_channel_cfg *channel_cfg) -{ } +{ + mutex_lock(&pipe->lock); + + channel_cfg->frm_res.w = + clamp_val(channel_cfg->frm_res.w, + pipe->pipe_cfg->out_min_res[output_id].w, + pipe->pipe_cfg->out_max_res[output_id].w); + + channel_cfg->frm_res.h = + clamp_val(channel_cfg->frm_res.h, + pipe->pipe_cfg->out_min_res[output_id].h, + pipe->pipe_cfg->out_max_res[output_id].h); + + pipe->pipe_cfg->pipe_xlink_chann[output_id] = *channel_cfg; + + mutex_unlock(&pipe->lock); +} /** * kmb_pipe_config_src - Configure pipeline source information @@ -72,5 +164,46 @@ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id, int kmb_pipe_config_src(struct kmb_pipeline *pipe, struct kmb_pipe_config_evs *pipe_cfg) { - return 0; + int ret = 0; + + mutex_lock(&pipe->lock); + + switch (pipe->state) { + case KMB_PIPE_STATE_CONFIGURED: + case KMB_PIPE_STATE_UNCONFIGURED: + /* Initialize pipeline configuration and counters */ + pipe->pending = 0; + pipe->streaming = 0; + + /* Store pipeline configuration */ + *pipe->pipe_cfg = *pipe_cfg; + + /* + * For some reason vpu firmware is returning config pipe as + * result for config pipe control. + */ + ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam, + pipe->pipe_cfg_paddr, + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE, + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE); + if (ret < 0) { + dev_err(pipe->dev, "Failed to reconfigure pipeline!"); + break; + } + kmb_pipe_print_config(pipe); + + pipe->state = KMB_PIPE_STATE_CONFIGURED; + break; + case KMB_PIPE_STATE_BUILT: + dev_err(pipe->dev, "Invalid state transition, already built"); + break; + default: + dev_err(pipe->dev, + "Config pipe in invalid state %d", pipe->state); + ret = -EINVAL; + break; + } + + mutex_unlock(&pipe->lock); + return ret; } From patchwork Fri Mar 19 18:06:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405078 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.9 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,UNWANTED_LANGUAGE_BODY, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2D96C433EC for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 9BDE26199A for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230319AbhCSSHZ (ORCPT ); Fri, 19 Mar 2021 14:07:25 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230304AbhCSSHD (ORCPT ); Fri, 19 Mar 2021 14:07:03 -0400 IronPort-SDR: L+tcqYdtRlJz/RkfD63notuOlfN/S9PvWJHRMSzxNqSdshPc6hwK8y/cYiT+l1VMOGj0UwkmJw /2Yvex7uQxkw== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036047" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036047" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:07:02 -0700 IronPort-SDR: S8tOmrMQOO6vH8GTMpYUMoSWyUGX648XK8zHXferbUPBWaDMLLY9Rc+QVpg/lBtJBcipB+Hfga xP0RXVDSQncg== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605781" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:07:00 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 08/10] media: Keem Bay Camera: Add capture video node Date: Fri, 19 Mar 2021 18:06:30 +0000 Message-Id: <20210319180632.585-9-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Martina Krasteva Capture video node implements v4l2 capture interface and XLink VPU Camera buffer pool operations. Build and set stream pipeline operations are also executed from capture video node. Resolution depends on remote entity pad connected to this video node. Co-developed-by: Gjorgji Rosikopulos Signed-off-by: Gjorgji Rosikopulos Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- .../platform/keembay-camera/keembay-pipeline.c | 192 +++++ .../platform/keembay-camera/keembay-pipeline.h | 4 + .../media/platform/keembay-camera/keembay-video.c | 884 ++++++++++++++++++++- .../media/platform/keembay-camera/keembay-video.h | 51 +- 4 files changed, 1096 insertions(+), 35 deletions(-) diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.c b/drivers/media/platform/keembay-camera/keembay-pipeline.c index 78b2fffa42ee..0050361ef3c0 100644 --- a/drivers/media/platform/keembay-camera/keembay-pipeline.c +++ b/drivers/media/platform/keembay-camera/keembay-pipeline.c @@ -60,6 +60,39 @@ static void kmb_pipe_print_config(struct kmb_pipeline *pipe) dev_dbg(dev, "\tinternal_memory_size %u\n", cfg->internal_memory_size); } +static unsigned int kmb_pipe_get_pending(struct media_entity *entity) +{ + struct media_device *mdev = entity->graph_obj.mdev; + unsigned int num_vdevs = 0; + struct media_entity *next; + struct media_graph graph; + int ret; + + /* Walk through graph to count the connected video node entities */ + mutex_lock(&mdev->graph_mutex); + + ret = media_graph_walk_init(&graph, mdev); + if (ret) { + mutex_unlock(&mdev->graph_mutex); + return -EINVAL; + } + + media_graph_walk_start(&graph, entity); + + while ((next = media_graph_walk_next(&graph))) { + if (!is_media_entity_v4l2_video_device(next)) + continue; + + num_vdevs++; + } + + mutex_unlock(&mdev->graph_mutex); + + media_graph_walk_cleanup(&graph); + + return num_vdevs; +} + /** * kmb_pipe_init - Initialize KMB Pipeline * @pipe: pointer to pipeline object @@ -207,3 +240,162 @@ int kmb_pipe_config_src(struct kmb_pipeline *pipe, mutex_unlock(&pipe->lock); return ret; } + +/** + * kmb_pipe_prepare - Prepare VPU pipeline for streaming + * @pipe: pointer to pipeline object + * + * Prepare pipeline for streaming by sending negotiated configuration to VPU + * and changing state to BUILT. + * + * Return: 0 if successful, error code otherwise. + */ +int kmb_pipe_prepare(struct kmb_pipeline *pipe) +{ + int ret = 0; + + mutex_lock(&pipe->lock); + + /* build only if all outputs are configured */ + switch (pipe->state) { + case KMB_PIPE_STATE_UNCONFIGURED: + /* Call config and continue */ + ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam, + pipe->pipe_cfg_paddr, + KMB_IC_EVENT_TYPE_CONFIG_ISP_PIPE, + KMB_IC_EVENT_TYPE_SUCCESSFUL); + if (ret < 0) { + dev_err(pipe->dev, "Failed to reconfigure pipeline!"); + break; + } + fallthrough; + case KMB_PIPE_STATE_CONFIGURED: + ret = kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam, + pipe->pipe_cfg_paddr, + KMB_IC_EVENT_TYPE_BUILD_ISP_PIPE, + KMB_IC_EVENT_TYPE_SUCCESSFUL); + if (ret < 0) { + dev_err(pipe->dev, "Failed to build pipeline!"); + break; + } + pipe->state = KMB_PIPE_STATE_BUILT; + break; + case KMB_PIPE_STATE_BUILT: + /* Pipeline is already built ignore */ + break; + default: + dev_err(pipe->dev, + "Build pipe in invalid state %d", pipe->state); + ret = -EINVAL; + break; + } + + mutex_unlock(&pipe->lock); + + return ret; +} + +static int kmb_pipe_s_stream(struct kmb_pipeline *pipe, + struct media_entity *entity, int enable) +{ + struct v4l2_subdev *subdev; + struct media_pad *remote; + int ret; + + remote = media_entity_remote_pad(entity->pads); + if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) + return -EINVAL; + + subdev = media_entity_to_v4l2_subdev(remote->entity); + if (!subdev) + return -EINVAL; + + ret = v4l2_subdev_call(subdev, video, s_stream, enable); + if (ret < 0 && ret != -ENOIOCTLCMD) + dev_err(pipe->dev, "Cannot set stream %d", enable); + + return ret != -ENOIOCTLCMD ? ret : 0; +} + +/** + * kmb_pipe_stop - Set stream off and stop media pipeline + * @pipe: KMB pipeline object + * @entity: media entity + */ +void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity) +{ + mutex_lock(&pipe->lock); + + if (WARN_ON(!pipe->streaming)) { + dev_err(pipe->dev, "Calling stop on already stopped pipeline"); + mutex_unlock(&pipe->lock); + return; + } + + if (pipe->state == KMB_PIPE_STATE_STREAMING) { + kmb_pipe_s_stream(pipe, entity, 0); + media_pipeline_stop(entity); + pipe->state = KMB_PIPE_STATE_BUILT; + } + + if (pipe->state == KMB_PIPE_STATE_BUILT || + pipe->state == KMB_PIPE_STATE_CONFIGURED) { + kmb_cam_xlink_write_ctrl_msg(pipe->xlink_cam, + pipe->pipe_cfg_paddr, + KMB_IC_EVENT_TYPE_DELETE_ISP_PIPE, + KMB_IC_EVENT_TYPE_SUCCESSFUL); + + pipe->state = KMB_PIPE_STATE_UNCONFIGURED; + } + + pipe->streaming--; + + mutex_unlock(&pipe->lock); +} + +/** + * kmb_pipe_run - Run media pipeline and start streaming + * @pipe: KMB pipeline object + * @entity: media entity + * + * Return: 0 if successful, error code otherwise. + */ +int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity) +{ + int ret = 0; + + mutex_lock(&pipe->lock); + + if (!pipe->streaming) + pipe->pending = kmb_pipe_get_pending(entity); + + pipe->streaming++; + + if (pipe->streaming != pipe->pending) + goto done_unlock; + + if (pipe->state != KMB_PIPE_STATE_BUILT) { + ret = -EINVAL; + goto done_unlock; + } + + ret = media_pipeline_start(entity, &pipe->media_pipe); + if (ret < 0) { + dev_err(pipe->dev, "Failed to start media pipeline"); + goto done_unlock; + } + + ret = kmb_pipe_s_stream(pipe, entity, 1); + if (ret < 0 && ret != -ENOIOCTLCMD) { + mutex_unlock(&pipe->lock); + kmb_pipe_stop(pipe, entity); + return ret; + } + + pipe->state = KMB_PIPE_STATE_STREAMING; + +done_unlock: + mutex_unlock(&pipe->lock); + + return ret; +} diff --git a/drivers/media/platform/keembay-camera/keembay-pipeline.h b/drivers/media/platform/keembay-camera/keembay-pipeline.h index 83ff94d11b34..60ba99e9a73c 100644 --- a/drivers/media/platform/keembay-camera/keembay-pipeline.h +++ b/drivers/media/platform/keembay-camera/keembay-pipeline.h @@ -68,4 +68,8 @@ void kmb_pipe_config_dest(struct kmb_pipeline *pipe, unsigned int output_id, int kmb_pipe_config_src(struct kmb_pipeline *pipe, struct kmb_pipe_config_evs *pipe_cfg); +int kmb_pipe_prepare(struct kmb_pipeline *pipe); +int kmb_pipe_run(struct kmb_pipeline *pipe, struct media_entity *entity); +void kmb_pipe_stop(struct kmb_pipeline *pipe, struct media_entity *entity); + #endif /* KEEMBAY_PIPELINE_H */ diff --git a/drivers/media/platform/keembay-camera/keembay-video.c b/drivers/media/platform/keembay-camera/keembay-video.c index 02f4d97e16fb..a92cfbeffea9 100644 --- a/drivers/media/platform/keembay-camera/keembay-video.c +++ b/drivers/media/platform/keembay-camera/keembay-video.c @@ -2,9 +2,816 @@ /* * Intel Keem Bay camera Video node. * - * Copyright (C) 2018-2020 Intel Corporation + * Copyright (C) 2021 Intel Corporation */ +#include +#include +#include +#include + +#include +#include +#include +#include + +#include "keembay-cam-xlink.h" +#include "keembay-pipeline.h" #include "keembay-video.h" +#include "keembay-vpu-frame.h" + +#define KMB_CAM_VIDEO_NAME "keembay-video" + +/* Xlink data channel size and timeout */ +#define KMB_VID_CH_DATA_SIZE 1024 +#define KMB_VID_CH_TIMEOUT_MS 5000 + +#define KMB_VID_MIN_WIDTH 16 +#define KMB_VID_MIN_HEIGHT 16 +#define KMB_VID_MAX_WIDTH U16_MAX +#define KMB_VID_MAX_HEIGHT U16_MAX +#define KMB_VID_STEP_WIDTH 8 +#define KMB_VID_STEP_HEIGHT 8 + +#define to_kmb_video_buf(vbuf) container_of(vbuf, struct kmb_frame_buffer, vb) + +/* Kmb video format info structure */ +struct kmb_video_fmt_info { + const char *description; + u32 code; + u32 pixelformat; + enum kmb_frame_types type; + u32 colorspace; + unsigned int planes; + unsigned int bpp; + unsigned int h_subsample; + unsigned int v_subsample; + bool contiguous_memory; +}; + +/* Supported video formats */ +static const struct kmb_video_fmt_info video_formats[] = { + { + .description = "NV12", + .code = MEDIA_BUS_FMT_YUYV8_1_5X8, + .pixelformat = V4L2_PIX_FMT_NV12, + .type = KMB_FRAME_TYPE_NV12, + .colorspace = V4L2_COLORSPACE_SRGB, + .planes = 2, + .bpp = 8, + .h_subsample = 1, + .v_subsample = 2, + .contiguous_memory = true, + }, + { + .description = "Planar YUV 4:2:0", + .code = MEDIA_BUS_FMT_UYYVYY8_0_5X24, + .pixelformat = V4L2_PIX_FMT_YUV420, + .type = KMB_FRAME_TYPE_YUV420P, + .colorspace = V4L2_COLORSPACE_SRGB, + .planes = 3, + .bpp = 8, + .h_subsample = 2, + .v_subsample = 2, + .contiguous_memory = false, + }, + { + .description = "Planar YUV 4:4:4", + .code = MEDIA_BUS_FMT_YUV8_1X24, + .pixelformat = V4L2_PIX_FMT_YUV444, + .type = KMB_FRAME_TYPE_YUV444P, + .colorspace = V4L2_COLORSPACE_SRGB, + .planes = 3, + .bpp = 8, + .h_subsample = 1, + .v_subsample = 1, + .contiguous_memory = false, + }, + { + .description = "RAW 8 Garyscale", + .code = MEDIA_BUS_FMT_Y8_1X8, + .pixelformat = V4L2_PIX_FMT_GREY, + .type = KMB_FRAME_TYPE_RAW8, + .colorspace = V4L2_COLORSPACE_RAW, + .planes = 1, + .bpp = 8, + .h_subsample = 1, + .v_subsample = 1, + .contiguous_memory = false, + }, + { + .description = "RAW 10 Grayscale", + .code = MEDIA_BUS_FMT_Y10_1X10, + .pixelformat = V4L2_PIX_FMT_Y10, + .type = KMB_FRAME_TYPE_RAW10, + .colorspace = V4L2_COLORSPACE_RAW, + .planes = 1, + .bpp = 10, + .h_subsample = 1, + .v_subsample = 1, + .contiguous_memory = false, + } +}; + +static const struct kmb_video_fmt_info * +kmb_video_get_fmt_info_by_code(u32 code) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(video_formats); i++) + if (video_formats[i].code == code) + return &video_formats[i]; + + return NULL; +} + +static const struct kmb_video_fmt_info * +kmb_video_get_fmt_info_by_pixfmt(u32 pix_fmt) +{ + unsigned int i; + + for (i = 0; i < ARRAY_SIZE(video_formats); i++) + if (video_formats[i].pixelformat == pix_fmt) + return &video_formats[i]; + + return NULL; +} + +/* Buffer processing operations */ +static void kmb_video_insert_buf(struct kmb_video *kmb_vid, + struct kmb_frame_buffer *buf) +{ + INIT_LIST_HEAD(&buf->list); + + mutex_lock(&kmb_vid->dma_lock); + list_add_tail(&buf->list, &kmb_vid->dma_queue); + mutex_unlock(&kmb_vid->dma_lock); +} + +static void __kmb_video_buf_discard(struct kmb_video *kmb_vid, + struct kmb_frame_buffer *buf) +{ + lockdep_assert_held(&kmb_vid->dma_lock); + + list_del(&buf->list); + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_ERROR); +} + +static int kmb_video_process_buf(struct kmb_video *kmb_vid, + struct kmb_frame_buffer *buf) +{ + const struct kmb_video_fmt_info *info = kmb_vid->active_fmt.info; + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix; + struct kmb_vpu_frame_buffer rt_frame_buf; + int ret; + + lockdep_assert_held(&kmb_vid->lock); + + memset(&rt_frame_buf, 0, sizeof(rt_frame_buf)); + rt_frame_buf.spec.bpp = info->bpp; + rt_frame_buf.spec.type = info->type; + rt_frame_buf.spec.width = pix->width; + rt_frame_buf.spec.height = pix->height; + rt_frame_buf.spec.stride = pix->plane_fmt[0].bytesperline; + rt_frame_buf.p1 = buf->addr[0]; + + /* Planes not used by the VPU should be set with addr 0 */ + if (pix->num_planes > 1) + rt_frame_buf.p2 = buf->addr[1]; + if (pix->num_planes > 2) + rt_frame_buf.p3 = buf->addr[2]; + + ret = kmb_cam_xlink_write_msg(kmb_vid->xlink_cam, + kmb_vid->chan_id, + (u8 *)&rt_frame_buf, + sizeof(rt_frame_buf)); + if (ret < 0) { + dev_err(kmb_vid->dma_dev, "Error on buffer queue %d", ret); + return ret; + } + + return 0; +} + +static void kmb_video_process_all_bufs(struct kmb_video *kmb_vid) +{ + struct kmb_frame_buffer *buf; + struct list_head *next; + struct list_head *pos; + int ret; + + mutex_lock(&kmb_vid->dma_lock); + + /* Discard buf is removing buffer from the list */ + list_for_each_safe(pos, next, &kmb_vid->dma_queue) { + buf = list_entry(pos, struct kmb_frame_buffer, list); + + ret = kmb_video_process_buf(kmb_vid, buf); + if (ret) { + dev_err(&kmb_vid->video->dev, + "Cannot process output buf 0x%pad", + &buf->addr[0]); + __kmb_video_buf_discard(kmb_vid, buf); + continue; + } + } + + mutex_unlock(&kmb_vid->dma_lock); +} + +static int kmb_video_queue_output_buf(struct kmb_video *kmb_vid, + struct kmb_frame_buffer *buf) +{ + int ret = 0; + + kmb_video_insert_buf(kmb_vid, buf); + + mutex_lock(&kmb_vid->dma_lock); + + /* Process buffers only when device is streaming */ + if (vb2_is_streaming(&kmb_vid->vb2_q)) { + ret = kmb_video_process_buf(kmb_vid, buf); + if (ret) { + dev_err(&kmb_vid->video->dev, + "Fail to process output buf 0x%pad", + &buf->addr[0]); + __kmb_video_buf_discard(kmb_vid, buf); + } + } + + mutex_unlock(&kmb_vid->dma_lock); + + return ret; +} + +static void kmb_video_release_all_bufs(struct kmb_video *kmb_vid, + enum vb2_buffer_state state) +{ + struct list_head *next = NULL; + struct list_head *pos = NULL; + struct kmb_frame_buffer *buf; + + mutex_lock(&kmb_vid->dma_lock); + list_for_each_safe(pos, next, &kmb_vid->dma_queue) { + buf = list_entry(pos, struct kmb_frame_buffer, list); + list_del(&buf->list); + vb2_buffer_done(&buf->vb.vb2_buf, state); + } + mutex_unlock(&kmb_vid->dma_lock); +} + +static void kmb_video_remove_buf(struct kmb_video *kmb_vid, + struct kmb_frame_buffer *buf) +{ + mutex_lock(&kmb_vid->dma_lock); + list_del(&buf->list); + mutex_unlock(&kmb_vid->dma_lock); +} + +static struct kmb_frame_buffer * +kmb_video_find_buf_by_addr(struct kmb_video *kmb_vid, uint64_t addr) +{ + struct kmb_frame_buffer *buf = NULL; + struct list_head *node = NULL; + + mutex_lock(&kmb_vid->dma_lock); + + list_for_each(node, &kmb_vid->dma_queue) { + buf = list_entry(node, struct kmb_frame_buffer, list); + if (buf->addr[0] == addr) { + mutex_unlock(&kmb_vid->dma_lock); + return buf; + } + } + + mutex_unlock(&kmb_vid->dma_lock); + + return NULL; +} + +static void kmb_video_fmt_info_to_pix(const struct kmb_video_fmt_info *info, + struct v4l2_mbus_framefmt *mbus_fmt, + struct v4l2_pix_format_mplane *pix) +{ + u32 bytesperline; + u32 sizeimage; + u32 v_sub = 1; + u32 h_sub = 1; + unsigned int i; + + pix->width = mbus_fmt->width; + pix->height = mbus_fmt->height; + + pix->pixelformat = info->pixelformat; + pix->colorspace = info->colorspace; + pix->num_planes = info->planes; + + for (i = 0; i < pix->num_planes; i++) { + bytesperline = pix->width * info->bpp / 8 / h_sub; + + if (pix->plane_fmt[i].bytesperline < bytesperline) + pix->plane_fmt[i].bytesperline = bytesperline; + + sizeimage = pix->plane_fmt[i].bytesperline * + pix->height / v_sub; + + if (pix->plane_fmt[i].sizeimage < sizeimage) + pix->plane_fmt[i].sizeimage = sizeimage; + + h_sub = info->h_subsample; + v_sub = info->v_subsample; + } +} + +static int kmb_video_get_subdev_fmt(struct kmb_video *kmb_vid, + struct v4l2_pix_format_mplane *pix) +{ + const struct kmb_video_fmt_info *fmt_info; + struct v4l2_subdev_format sd_fmt; + struct v4l2_subdev *subdev; + struct media_pad *remote; + int ret; + + remote = media_entity_remote_pad(&kmb_vid->pad); + if (!remote || !is_media_entity_v4l2_subdev(remote->entity)) + return -EINVAL; + + subdev = media_entity_to_v4l2_subdev(remote->entity); + if (!subdev) + return -EINVAL; + + memset(&sd_fmt, 0, sizeof(sd_fmt)); + sd_fmt.pad = remote->index; + sd_fmt.which = V4L2_SUBDEV_FORMAT_ACTIVE; + ret = v4l2_subdev_call(subdev, pad, get_fmt, NULL, &sd_fmt); + if (ret < 0) + return ret; + + fmt_info = kmb_video_get_fmt_info_by_code(sd_fmt.format.code); + if (!fmt_info) + return -EINVAL; + + kmb_video_fmt_info_to_pix(fmt_info, &sd_fmt.format, pix); + + return 0; +} + +static int kmb_video_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, + unsigned int *num_planes, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(q); + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix; + unsigned int i; + + if (kmb_vid->active_fmt.info->contiguous_memory) { + *num_planes = 1; + for (i = 0; i < pix->num_planes; i++) + sizes[0] += pix->plane_fmt[i].sizeimage; + } else { + *num_planes = pix->num_planes; + for (i = 0; i < pix->num_planes; i++) + sizes[i] = pix->plane_fmt[i].sizeimage; + } + + return 0; +} + +static int kmb_video_buffer_prepare(struct vb2_buffer *vb) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue); + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix; + unsigned int size_image = 0; + unsigned int i; + + if (kmb_vid->active_fmt.info->contiguous_memory) { + for (i = 0; i < pix->num_planes; i++) + size_image += pix->plane_fmt[i].sizeimage; + + vb2_set_plane_payload(vb, 0, size_image); + } else { + for (i = 0; i < pix->num_planes; i++) + vb2_set_plane_payload(vb, i, + pix->plane_fmt[i].sizeimage); + } + + return 0; +} + +static int kmb_video_buf_init(struct vb2_buffer *vb) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf); + struct v4l2_pix_format_mplane *pix = &kmb_vid->active_fmt.pix; + unsigned int i; + + if (kmb_vid->active_fmt.info->contiguous_memory) { + buf->addr[0] = vb2_dma_contig_plane_dma_addr(vb, 0); + for (i = 1; i < pix->num_planes; i++) { + buf->addr[i] = buf->addr[i - 1] + + pix->plane_fmt[i - 1].sizeimage; + } + } else { + for (i = 0; i < pix->num_planes; i++) + buf->addr[i] = vb2_dma_contig_plane_dma_addr(vb, i); + } + + return 0; +} + +static void kmb_video_buf_queue(struct vb2_buffer *vb) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_frame_buffer *buf = to_kmb_video_buf(vbuf); + int ret; + + ret = kmb_video_queue_output_buf(kmb_vid, buf); + if (ret) + dev_err(kmb_vid->dma_dev, "Fail output buf queue %d", ret); +} + +static int kmb_video_worker_thread(void *video) +{ + struct kmb_vpu_frame_buffer rt_frame_buf; + struct kmb_video *kmb_vid = video; + struct kmb_frame_buffer *buf = NULL; + bool stopped = false; + int ret; + + set_freezable(); + + while (!kthread_should_stop()) { + try_to_freeze(); + + if (stopped) { + set_current_state(TASK_INTERRUPTIBLE); + schedule(); + continue; + } + + memset(&rt_frame_buf, 0, sizeof(rt_frame_buf)); + ret = kmb_cam_xlink_read_msg(kmb_vid->xlink_cam, + kmb_vid->chan_id, + (u8 *)&rt_frame_buf, + sizeof(rt_frame_buf)); + if (ret < 0) { + stopped = true; + /* Continue here to enter in freeze state */ + continue; + } + + buf = kmb_video_find_buf_by_addr(kmb_vid, rt_frame_buf.p1); + if (buf) { + kmb_video_remove_buf(kmb_vid, buf); + + buf->vb.vb2_buf.timestamp = rt_frame_buf.ts; + vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_DONE); + } else { + dev_err(kmb_vid->dma_dev, "Ouch cannot find buff %llx", + rt_frame_buf.p1); + } + } + + return 0; +} + +static int kmb_video_worker_start(struct kmb_video *kmb_vid) +{ + int ret; + + ret = kmb_cam_xlink_open_channel(kmb_vid->xlink_cam, kmb_vid->chan_id); + if (ret) + return ret; + + kmb_vid->thread = kthread_run(kmb_video_worker_thread, + kmb_vid, "kmb_vnode_thread"); + if (IS_ERR(kmb_vid->thread)) { + dev_err(&kmb_vid->video->dev, "Cannot start thread"); + ret = -ENOMEM; + kmb_vid->thread = NULL; + goto error_close_xlink_channel; + } + + return 0; + +error_close_xlink_channel: + kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id); + + return ret; +} + +static int kmb_video_worker_stop(struct kmb_video *kmb_vid) +{ + int ret; + + /* + * Xlink has no functionality to unblock read volatile function, + * only way to unblock is to close the channel. + */ + kmb_cam_xlink_close_channel(kmb_vid->xlink_cam, kmb_vid->chan_id); + if (!kmb_vid->thread) { + dev_warn(&kmb_vid->video->dev, "No thread running"); + return 0; + } + + ret = kthread_stop(kmb_vid->thread); + if (ret < 0) + dev_err(&kmb_vid->video->dev, "Thread stop failed %d", ret); + + kmb_vid->thread = NULL; + + return ret; +} + +static int kmb_video_capture_start_streaming(struct vb2_queue *q, + unsigned int count) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(q); + int ret; + + ret = kmb_pipe_prepare(kmb_vid->pipe); + if (ret < 0) + goto error_discard_all_bufs; + + ret = kmb_video_worker_start(kmb_vid); + if (ret < 0) + goto error_pipeline_stop; + + /* Process all pending buffers after worker is started */ + kmb_video_process_all_bufs(kmb_vid); + + /* + * Run the pipeline after all buffers are provided for processing, + * the main reason is to not skip any frame from the source. + */ + ret = kmb_pipe_run(kmb_vid->pipe, &kmb_vid->video->entity); + if (ret < 0) + goto error_pipeline_stop; + + return 0; + +error_pipeline_stop: + kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity); +error_discard_all_bufs: + kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_QUEUED); + + return ret; +} + +static void kmb_video_capture_stop_streaming(struct vb2_queue *q) +{ + struct kmb_video *kmb_vid = vb2_get_drv_priv(q); + + kmb_pipe_stop(kmb_vid->pipe, &kmb_vid->video->entity); + + kmb_video_worker_stop(kmb_vid); + + kmb_video_release_all_bufs(kmb_vid, VB2_BUF_STATE_ERROR); +} + +/* driver-specific operations */ +static const struct vb2_ops kmb_video_vb2_q_capture_ops = { + .queue_setup = kmb_video_queue_setup, + .buf_prepare = kmb_video_buffer_prepare, + .buf_init = kmb_video_buf_init, + .buf_queue = kmb_video_buf_queue, + .start_streaming = kmb_video_capture_start_streaming, + .stop_streaming = kmb_video_capture_stop_streaming, +}; + +static int kmb_video_querycap(struct file *file, void *fh, + struct v4l2_capability *cap) +{ + cap->bus_info[0] = 0; + strscpy(cap->driver, KMB_CAM_VIDEO_NAME, sizeof(cap->driver)); + strscpy(cap->card, KMB_CAM_VIDEO_NAME, sizeof(cap->card)); + + return 0; +} + +static int kmb_video_enum_fmt(struct file *file, void *fh, + struct v4l2_fmtdesc *f) +{ + const struct kmb_video_fmt_info *info; + + if (!V4L2_TYPE_IS_MULTIPLANAR(f->type)) + return -EINVAL; + + if (f->mbus_code) { + if (f->index != 0) + return -EINVAL; + + info = kmb_video_get_fmt_info_by_code(f->mbus_code); + if (!info) + return -EINVAL; + } else { + info = &video_formats[f->index]; + if (!info) + return -EINVAL; + } + + f->pixelformat = info->pixelformat; + f->mbus_code = info->code; + strscpy(f->description, info->description, sizeof(f->description)); + + return 0; +} + +static int kmb_video_enum_framesizes(struct file *file, void *fh, + struct v4l2_frmsizeenum *fsize) +{ + const struct kmb_video_fmt_info *info; + + if (fsize->index != 0) + return -EINVAL; + + info = kmb_video_get_fmt_info_by_pixfmt(fsize->pixel_format); + if (!info) + return -EINVAL; + + fsize->type = V4L2_FRMSIZE_TYPE_STEPWISE; + + fsize->stepwise.min_width = KMB_VID_MIN_WIDTH; + fsize->stepwise.max_width = KMB_VID_MAX_WIDTH; + fsize->stepwise.step_width = KMB_VID_STEP_WIDTH; + fsize->stepwise.min_height = KMB_VID_MIN_HEIGHT; + fsize->stepwise.max_height = KMB_VID_MAX_HEIGHT; + fsize->stepwise.step_height = KMB_VID_STEP_HEIGHT; + + return 0; +} + +static int kmb_video_try_fmt(struct file *file, void *fh, + struct v4l2_format *f) +{ + const struct kmb_video_fmt_info *info; + struct v4l2_mbus_framefmt mbus_fmt; + + info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat); + if (!info) + info = &video_formats[0]; + + mbus_fmt.width = f->fmt.pix_mp.width; + mbus_fmt.height = f->fmt.pix_mp.height; + kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp); + + return 0; +} + +static int kmb_video_set_fmt(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct kmb_video *kmb_vid = video_drvdata(file); + const struct kmb_video_fmt_info *info; + struct v4l2_mbus_framefmt mbus_fmt; + + info = kmb_video_get_fmt_info_by_pixfmt(f->fmt.pix_mp.pixelformat); + if (!info) + info = &video_formats[0]; + + mbus_fmt.width = f->fmt.pix_mp.width; + mbus_fmt.height = f->fmt.pix_mp.height; + kmb_video_fmt_info_to_pix(info, &mbus_fmt, &f->fmt.pix_mp); + + kmb_vid->active_fmt.pix = f->fmt.pix_mp; + kmb_vid->active_fmt.info = info; + + return 0; +} + +static int kmb_video_get_fmt(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct kmb_video *kmb_vid = video_drvdata(file); + + f->fmt.pix_mp = kmb_vid->active_fmt.pix; + + return 0; +} + +static int kmb_video_check_format(struct kmb_video *kmb_vid) +{ + int ret; + struct v4l2_pix_format_mplane pix; + + ret = kmb_video_get_subdev_fmt(kmb_vid, &pix); + if (ret < 0) + return ret; + + if (kmb_vid->active_fmt.pix.pixelformat != pix.pixelformat || + kmb_vid->active_fmt.pix.height != pix.height || + kmb_vid->active_fmt.pix.width != pix.width || + kmb_vid->active_fmt.pix.num_planes != pix.num_planes) { + dev_err(&kmb_vid->video->dev, "Pix fmt mismatch:\n\t" + "pix_fmt %u %u\n\theight %u %u\n\twidth %u %u\n\t" + "num_planes %u %u", + kmb_vid->active_fmt.pix.pixelformat, pix.pixelformat, + kmb_vid->active_fmt.pix.height, pix.height, + kmb_vid->active_fmt.pix.width, pix.width, + kmb_vid->active_fmt.pix.num_planes, pix.num_planes); + ret = -EINVAL; + } + + return ret; +} + +static int kmb_video_streamon(struct file *file, void *fh, + enum v4l2_buf_type type) +{ + struct kmb_video *kmb_vid = video_drvdata(file); + int ret; + + if (type != kmb_vid->vb2_q.type) + return -EINVAL; + + ret = kmb_video_check_format(kmb_vid); + if (ret < 0) + return ret; + + return vb2_streamon(&kmb_vid->vb2_q, type); +} + +/* V4L2 ioctl operations */ +static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = { + .vidioc_querycap = kmb_video_querycap, + .vidioc_enum_fmt_vid_cap = kmb_video_enum_fmt, + .vidioc_enum_framesizes = kmb_video_enum_framesizes, + .vidioc_g_fmt_vid_cap_mplane = kmb_video_get_fmt, + .vidioc_try_fmt_vid_cap_mplane = kmb_video_try_fmt, + .vidioc_s_fmt_vid_cap_mplane = kmb_video_set_fmt, + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_streamon = kmb_video_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, + .vidioc_expbuf = vb2_ioctl_expbuf, +}; + +static int kmb_video_open(struct file *file) +{ + struct kmb_video *kmb_vid = video_drvdata(file); + struct v4l2_mbus_framefmt fmt; + int ret; + + mutex_lock(&kmb_vid->lock); + ret = v4l2_fh_open(file); + if (ret) { + mutex_unlock(&kmb_vid->lock); + return ret; + } + + INIT_LIST_HEAD(&kmb_vid->dma_queue); + + ret = kmb_pipe_request(kmb_vid->pipe); + if (ret < 0) + goto error_fh_release; + + /* Fill default format. */ + memset(&fmt, 0, sizeof(fmt)); + kmb_video_fmt_info_to_pix(&video_formats[0], &fmt, + &kmb_vid->active_fmt.pix); + kmb_vid->active_fmt.info = &video_formats[0]; + + mutex_unlock(&kmb_vid->lock); + + return 0; + +error_fh_release: + _vb2_fop_release(file, NULL); + mutex_unlock(&kmb_vid->lock); + + return ret; +} + +static int kmb_video_release(struct file *file) +{ + struct kmb_video *kmb_vid = video_drvdata(file); + int ret; + + mutex_lock(&kmb_vid->lock); + + kmb_pipe_release(kmb_vid->pipe); + + ret = _vb2_fop_release(file, NULL); + + mutex_unlock(&kmb_vid->lock); + + return ret; +} + +/* FS operations for V4L2 device */ +static const struct v4l2_file_operations kmb_vid_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = video_ioctl2, + .open = kmb_video_open, + .release = kmb_video_release, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; /** * kmb_video_init - Initialize entity @@ -15,7 +822,63 @@ */ int kmb_video_init(struct kmb_video *kmb_vid, const char *name) { + int ret; + + kmb_vid->video = video_device_alloc(); + if (!kmb_vid->video) { + dev_err(&kmb_vid->video->dev, + "Failed to allocate video device"); + return -ENOMEM; + } + + mutex_init(&kmb_vid->lock); + mutex_init(&kmb_vid->dma_lock); + + kmb_vid->video->fops = &kmb_vid_fops; + kmb_vid->video->ioctl_ops = &kmb_vid_ioctl_ops; + kmb_vid->video->minor = -1; + kmb_vid->video->release = video_device_release; + kmb_vid->video->vfl_type = VFL_TYPE_VIDEO; + kmb_vid->video->lock = &kmb_vid->lock; + kmb_vid->video->queue = &kmb_vid->vb2_q; + video_set_drvdata(kmb_vid->video, kmb_vid); + snprintf(kmb_vid->video->name, sizeof(kmb_vid->video->name), + "kmb_video %s", name); + + kmb_vid->vb2_q.drv_priv = kmb_vid; + kmb_vid->vb2_q.ops = &kmb_video_vb2_q_capture_ops; + kmb_vid->vb2_q.type = V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE; + kmb_vid->vb2_q.buf_struct_size = sizeof(struct kmb_frame_buffer); + kmb_vid->vb2_q.io_modes = VB2_MMAP | VB2_DMABUF; + kmb_vid->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; + kmb_vid->vb2_q.mem_ops = &vb2_dma_contig_memops; + kmb_vid->vb2_q.dev = kmb_vid->dma_dev; + kmb_vid->vb2_q.lock = &kmb_vid->lock; + kmb_vid->vb2_q.min_buffers_needed = 1; + + kmb_vid->pad.flags = MEDIA_PAD_FL_SINK; + kmb_vid->video->device_caps = V4L2_CAP_VIDEO_CAPTURE_MPLANE | + V4L2_CAP_STREAMING | V4L2_CAP_IO_MC; + + ret = media_entity_pads_init(&kmb_vid->video->entity, 1, &kmb_vid->pad); + if (ret < 0) + goto error_mutex_destroy; + + ret = vb2_queue_init(&kmb_vid->vb2_q); + if (ret < 0) { + dev_err(&kmb_vid->video->dev, "Failed to init vb2 queue"); + goto error_video_cleanup; + } + return 0; + +error_video_cleanup: + kmb_video_cleanup(kmb_vid); +error_mutex_destroy: + mutex_destroy(&kmb_vid->lock); + mutex_destroy(&kmb_vid->dma_lock); + + return ret; } /** @@ -23,7 +886,11 @@ int kmb_video_init(struct kmb_video *kmb_vid, const char *name) * @kmb_vid: pointer to kmb video device */ void kmb_video_cleanup(struct kmb_video *kmb_vid) -{ } +{ + media_entity_cleanup(&kmb_vid->video->entity); + mutex_destroy(&kmb_vid->lock); + mutex_destroy(&kmb_vid->dma_lock); +} /** * kmb_video_register - Register V4L2 device @@ -35,7 +902,14 @@ void kmb_video_cleanup(struct kmb_video *kmb_vid) int kmb_video_register(struct kmb_video *kmb_vid, struct v4l2_device *v4l2_dev) { - return 0; + int ret; + + kmb_vid->video->v4l2_dev = v4l2_dev; + ret = video_register_device(kmb_vid->video, VFL_TYPE_VIDEO, -1); + if (ret < 0) + dev_err(&kmb_vid->video->dev, "Failed to register video device"); + + return ret; } /** @@ -43,4 +917,6 @@ int kmb_video_register(struct kmb_video *kmb_vid, * @kmb_vid: pointer to kmb video device */ void kmb_video_unregister(struct kmb_video *kmb_vid) -{ } +{ + video_unregister_device(kmb_vid->video); +} diff --git a/drivers/media/platform/keembay-camera/keembay-video.h b/drivers/media/platform/keembay-camera/keembay-video.h index 2aebbb37424b..de25dfe3d684 100644 --- a/drivers/media/platform/keembay-camera/keembay-video.h +++ b/drivers/media/platform/keembay-camera/keembay-video.h @@ -16,7 +16,7 @@ /** * struct kmb_frame_buffer - KMB frame buffer structure * @vb: Video buffer for v4l2 - * @addr: Array of dma buffer plane address + * @addr: Array of dma buffer plane addresses * @list: Frame buffer list */ struct kmb_frame_buffer { @@ -28,50 +28,39 @@ struct kmb_frame_buffer { /** * struct kmb_video - KMB Video device structure * @lock: Mutex serializing kmb video device ops - * @video_lock: Mutex serializing video operations * @video: Pointer to V4L2 sub-device + * @vb2_q: Video buffer queue * @pad: Media pad graph objects * @dma_dev: Pointer to dma device + * @dma_queue: DMA buffers queue + * @dma_lock: Mutex serializing dma queue ops + * @active_fmt: Active format + * @active_fmt.pix: Mplane active pixel format + * @active_fmt.info: Active kmb format info * @pipe: Pointer to kmb media pipeline - * @chan: Pointer to xlink channel + * @xlink_cam: Pointer to xlink camera communication handler + * @chan_id: Channel ID + * @thread: Pointer to worker thread data */ struct kmb_video { - struct mutex lock; /* Lock protecting kmb video device */ - struct mutex video_lock; /* Lock serializing video device operations */ + struct mutex lock; struct video_device *video; + struct vb2_queue vb2_q; struct media_pad pad; + struct device *dma_dev; - struct kmb_pipeline *pipe; - struct kmb_xlink_cam *xlink_cam; - unsigned int chan_id; -}; + struct list_head dma_queue; + struct mutex dma_lock; -/** - * struct kmb_video_fh - KMB video file handler - * @fh: V4L2 file handler - * @kmb_vid: Pointer to KMB video device - * @lock: Mutex serializing access to fh - * @vb2_lock: Mutex serializing access to vb2 queue - * @vb2_q: Video buffer queue - * @active_fmt: Active format - @pix: Mplane active pixel format - @info: Active kmb format info - * @contiguous_memory: Flag to enable contiguous memory allocation - * @dma_queue: DMA buffers queue - * @thread: Pointer to worker thread data - */ -struct kmb_video_fh { - struct v4l2_fh fh; - struct kmb_video *kmb_vid; - struct mutex lock; /* Lock protecting fh operations */ - struct mutex vb2_lock; /* Lock protecting video buffer queue */ - struct vb2_queue vb2_q; struct { struct v4l2_pix_format_mplane pix; const struct kmb_video_fmt_info *info; } active_fmt; - bool contiguous_memory; - struct list_head dma_queue; + + struct kmb_pipeline *pipe; + struct kmb_xlink_cam *xlink_cam; + unsigned int chan_id; + struct task_struct *thread; }; From patchwork Fri Mar 19 18:06:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Krasteva, Martina" X-Patchwork-Id: 405077 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A66BEC433EB for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8182D61997 for ; Fri, 19 Mar 2021 18:07:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230320AbhCSSH0 (ORCPT ); Fri, 19 Mar 2021 14:07:26 -0400 Received: from mga09.intel.com ([134.134.136.24]:57280 "EHLO mga09.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230285AbhCSSHG (ORCPT ); Fri, 19 Mar 2021 14:07:06 -0400 IronPort-SDR: Nw6gXpnEF0tx2ue+mB2qSN+kw07pRLkWBPN4AzV+gXoKla50t7K0n0coJnBCWGv36N5d0FvlYX Xzx2dWOD8VpQ== X-IronPort-AV: E=McAfee;i="6000,8403,9928"; a="190036056" X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="190036056" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:07:05 -0700 IronPort-SDR: rElUgfs7LyiAKG9gI2du4OUNK+miMjJ2L7dDa8plhROo9cTNzDdWs1Ne9sxQZIsAYvj/CWtpAu b1ismd4+YYgA== X-IronPort-AV: E=Sophos;i="5.81,262,1610438400"; d="scan'208";a="413605805" Received: from mkrastex-mobl.ger.corp.intel.com ([10.104.88.55]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 19 Mar 2021 11:07:02 -0700 From: Martina Krasteva To: linux-media@vger.kernel.org Cc: mchehab@kernel.org, robh+dt@kernel.org, devicetree@vger.kernel.org, sakari.ailus@linux.intel.com, daniele.alessandrelli@linux.intel.com, paul.j.murphy@linux.intel.com, gjorgjix.rosikopulos@linux.intel.com, martinax.krasteva@linux.intel.com Subject: [PATCH 09/10] media: Keem Bay Camera: Add metadata video node Date: Fri, 19 Mar 2021 18:06:31 +0000 Message-Id: <20210319180632.585-10-martinax.krasteva@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20210319180632.585-1-martinax.krasteva@linux.intel.com> References: <20210319180632.585-1-martinax.krasteva@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org From: Gjorgji Rosikopulos Metadata video node implements output and capture meta type interface. - Output video node is used to provide isp parameters for processing. Each buffer internally has real vpu isp params structure allocated. User space params are copied on every qbuf based on update flags. Since vpu need every time all parameters to be provided, params are copied on every qbuf. Based on update flags they are copied from userspace buffer or last buffer processed. To reduce coping of the tables, they are allocated separately in table buffer pool. The tables are copied only when there is update from the userspace, otherwise they are only reference from last processed frame. This is possible because vpu interface has separate address for each table. - Capture video node is used to provide statistics to userspace. Capture video node statistics memory addresses are copied to isp params before processing, and corresponding update flags are set based on statistics availability. Signed-off-by: Gjorgji Rosikopulos Co-developed-by: Ivan Dimitrov Signed-off-by: Ivan Dimitrov Co-developed-by: Martina Krasteva Signed-off-by: Martina Krasteva Acked-by: Paul J. Murphy Acked-by: Daniele Alessandrelli --- drivers/media/platform/keembay-camera/Makefile | 4 +- .../platform/keembay-camera/keembay-metadata.c | 1823 +++++++++++++++++++- .../platform/keembay-camera/keembay-metadata.h | 14 +- .../keembay-camera/keembay-params-defaults.c | 326 ++++ .../keembay-camera/keembay-params-defaults.h | 38 + 5 files changed, 2194 insertions(+), 11 deletions(-) create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.c create mode 100644 drivers/media/platform/keembay-camera/keembay-params-defaults.h diff --git a/drivers/media/platform/keembay-camera/Makefile b/drivers/media/platform/keembay-camera/Makefile index 8b3ad715c5c4..1b949cf009ef 100644 --- a/drivers/media/platform/keembay-camera/Makefile +++ b/drivers/media/platform/keembay-camera/Makefile @@ -1,5 +1,5 @@ keembay-cam-objs = keembay-camera.o keembay-pipeline.o \ - keembay-cam-xlink.o keembay-isp.o \ - keembay-metadata.o keembay-video.o + keembay-cam-xlink.o keembay-params-defaults.o \ + keembay-isp.o keembay-metadata.o keembay-video.o obj-$(CONFIG_VIDEO_INTEL_KEEMBAY_CAMERA) += keembay-cam.o diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.c b/drivers/media/platform/keembay-camera/keembay-metadata.c index a1df746d9582..8807e3f322c5 100644 --- a/drivers/media/platform/keembay-camera/keembay-metadata.c +++ b/drivers/media/platform/keembay-camera/keembay-metadata.c @@ -4,17 +4,1818 @@ * * Copyright (C) 2021 Intel Corporation */ + +#include +#include + +#include +#include +#include +#include + +#include "keembay-pipeline.h" #include "keembay-metadata.h" +#define KMB_CAM_METADATA_STATS_NAME "keembay-metadata-stats" +#define KMB_CAM_METADATA_PARAMS_NAME "keembay-metadata-params" + +#define KMB_TABLE_ALIGN 64 + +/* Table names map */ +static const char *table_name[KMB_METADATA_TABLE_MAX] = { + "LSC", + "StaticDefect", + "LCA", + "HDR", + "Sharpness", + "Color cumb", + "LUT", + "TNF1", + "TNF2", + "Dehaze", + "Warp", +}; + +static void +kmb_metadata_copy_blc(struct kmb_vpu_blc_params *dst, + struct kmb_blc_params *src) +{ + int i; + + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) { + dst[i].coeff1 = src[i].coeff1; + dst[i].coeff2 = src[i].coeff2; + dst[i].coeff3 = src[i].coeff3; + dst[i].coeff4 = src[i].coeff4; + } +} + +static void +kmb_metadata_copy_sigma_dns(struct kmb_vpu_sigma_dns_params *dst, + struct kmb_sigma_dns_params *src) +{ + int i; + + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) { + dst[i].noise = src[i].noise; + dst[i].threshold1 = src[i].threshold1; + dst[i].threshold2 = src[i].threshold2; + dst[i].threshold3 = src[i].threshold3; + dst[i].threshold4 = src[i].threshold4; + dst[i].threshold5 = src[i].threshold5; + dst[i].threshold6 = src[i].threshold6; + dst[i].threshold7 = src[i].threshold7; + dst[i].threshold8 = src[i].threshold8; + } +} + +static void +kmb_metadata_copy_lsc(struct kmb_vpu_lsc_params *dst, + struct kmb_lsc_params *src) +{ + dst->threshold = src->threshold; + dst->width = src->width; + dst->height = src->height; +} + +static void +kmb_metadata_copy_raw(struct kmb_vpu_raw_params *dst, + struct kmb_raw_params *src) +{ + dst->awb_stats_en = src->awb_stats_en; + dst->awb_rgb_hist_en = src->awb_rgb_hist_en; + dst->af_stats_en = src->af_stats_en; + dst->luma_hist_en = src->luma_hist_en; + dst->flicker_accum_en = src->flicker_accum_en; + dst->bad_pixel_fix_en = src->bad_pixel_fix_en; + dst->grgb_imb_en = src->grgb_imb_en; + dst->mono_imbalance_en = src->mono_imbalance_en; + dst->gain1 = src->gain1; + dst->gain2 = src->gain2; + dst->gain3 = src->gain3; + dst->gain4 = src->gain4; + dst->stop1 = src->stop1; + dst->stop2 = src->stop2; + dst->stop3 = src->stop3; + dst->stop4 = src->stop4; + dst->threshold1 = src->threshold1; + dst->alpha1 = src->alpha1; + dst->alpha2 = src->alpha2; + dst->alpha3 = src->alpha3; + dst->alpha4 = src->alpha4; + dst->threshold2 = src->threshold2; + dst->static_defect_size = src->static_defect_size; + dst->flicker_first_row_acc = src->start_row; + dst->flicker_last_row_acc = src->end_row; +} + +static void +kmb_metadata_copy_ae_awb(struct kmb_vpu_ae_awb_params *dst, + struct kmb_ae_awb_params *src) +{ + dst->start_x = src->start_x; + dst->start_y = src->start_y; + dst->width = src->width; + dst->height = src->height; + dst->skip_x = src->skip_x; + dst->skip_y = src->skip_y; + dst->patches_x = src->patches_x; + dst->patches_y = src->patches_y; + dst->threshold1 = src->threshold1; + dst->threshold2 = src->threshold2; +} + +static void +kmb_metadata_copy_af(struct kmb_vpu_af_params *dst, + struct kmb_af_params *src) +{ + int i; + + dst->start_x = src->start_x; + dst->start_y = src->start_y; + dst->width = src->width; + dst->height = src->height; + dst->patches_x = src->patches_x; + dst->patches_y = src->patches_y; + dst->coeff = src->coeff; + dst->threshold1 = src->threshold1; + dst->threshold2 = src->threshold2; + + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) { + dst->coeffs1[i] = src->coeffs1[i]; + dst->coeffs2[i] = src->coeffs2[i]; + } +} + +static void +kmb_metadata_copy_histogram(struct kmb_vpu_hist_params *dst, + struct kmb_hist_params *src) +{ + int i; + + dst->start_x = src->start_x; + dst->start_y = src->start_y; + dst->end_x = src->end_x; + dst->end_y = src->end_y; + + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++) + dst->matrix[i] = src->matrix[i]; + + for (i = 0; i < ARRAY_SIZE(dst->weight); i++) + dst->weight[i] = src->weight[i]; +} + +static void +kmb_metadata_copy_debayer(struct kmb_vpu_debayer_params *dst, + struct kmb_debayer_params *src) +{ + dst->coeff1 = src->coeff1; + dst->multiplier1 = src->multiplier1; + dst->multiplier2 = src->multiplier2; + dst->coeff2 = src->coeff2; + dst->coeff3 = src->coeff3; + dst->coeff4 = src->coeff4; +} + +static void +kmb_metadata_copy_dog_dns(struct kmb_vpu_dog_dns_params *dst, + struct kmb_dog_dns_params *src) +{ + int i; + + dst->threshold = src->threshold; + dst->strength = src->strength; + + for (i = 0; i < ARRAY_SIZE(dst->coeffs11); i++) + dst->coeffs11[i] = src->coeffs11[i]; + + for (i = 0; i < ARRAY_SIZE(dst->coeffs15); i++) + dst->coeffs15[i] = src->coeffs15[i]; +} + +static void +kmb_metadata_copy_luma_dns(struct kmb_vpu_luma_dns_params *dst, + struct kmb_luma_dns_params *src) +{ + dst->threshold = src->threshold; + dst->slope = src->slope; + dst->shift = src->shift; + dst->alpha = src->alpha; + dst->weight = src->weight; + dst->per_pixel_alpha_en = src->per_pixel_alpha_en; + dst->gain_bypass_en = src->gain_bypass_en; +} + +static void +kmb_metadata_copy_sharpen(struct kmb_vpu_sharpen_params *dst, + struct kmb_sharpen_params *src) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dst->coeffs1); i++) { + dst->coeffs1[i] = src->coeffs1[i]; + dst->coeffs2[i] = src->coeffs2[i]; + dst->coeffs3[i] = src->coeffs3[i]; + } + + dst->shift = src->shift; + dst->gain1 = src->gain1; + dst->gain2 = src->gain2; + dst->gain3 = src->gain3; + dst->gain4 = src->gain4; + dst->gain5 = src->gain5; + + for (i = 0; i < ARRAY_SIZE(dst->stops1); i++) { + dst->stops1[i] = src->stops1[i]; + dst->gains[i] = src->gains[i]; + } + + for (i = 0; i < ARRAY_SIZE(dst->stops2); i++) + dst->stops2[i] = src->stops2[i]; + + dst->overshoot = src->overshoot; + dst->undershoot = src->undershoot; + dst->alpha = src->alpha; + dst->gain6 = src->gain6; + dst->offset = src->offset; +} + +static void +kmb_metadata_copy_chroma_gen(struct kmb_vpu_chroma_gen_params *dst, + struct kmb_chroma_gen_params *src) +{ + int i; + + dst->epsilon = src->epsilon; + dst->coeff1 = src->coeff1; + dst->coeff2 = src->coeff2; + dst->coeff3 = src->coeff3; + dst->coeff4 = src->coeff4; + dst->coeff5 = src->coeff5; + dst->coeff6 = src->coeff6; + dst->strength1 = src->strength1; + dst->strength2 = src->strength2; + + for (i = 0; i < ARRAY_SIZE(dst->coeffs); i++) + dst->coeffs[i] = src->coeffs[i]; + + dst->offset1 = src->offset1; + dst->slope1 = src->slope1; + dst->slope2 = src->slope2; + dst->offset2 = src->offset2; + dst->limit = src->limit; +} + +static void +kmb_metadata_copy_median(struct kmb_vpu_median_params *dst, + struct kmb_median_params *src) +{ + dst->size = src->size; + dst->slope = src->slope; + dst->offset = src->offset; +} + +static void +kmb_metadata_copy_chroma_dns(struct kmb_vpu_chroma_dns_params *dst, + struct kmb_chroma_dns_params *src) +{ + dst->limit = src->limit; + dst->enable = src->enable; + dst->threshold1 = src->threshold1; + dst->threshold2 = src->threshold2; + dst->threshold3 = src->threshold3; + dst->threshold4 = src->threshold4; + dst->threshold5 = src->threshold5; + dst->threshold6 = src->threshold6; + dst->threshold7 = src->threshold7; + dst->threshold8 = src->threshold8; + dst->slope1 = src->slope1; + dst->offset1 = src->offset1; + dst->slope2 = src->slope2; + dst->offset2 = src->offset2; + dst->grey1 = src->grey1; + dst->grey2 = src->grey2; + dst->grey3 = src->grey3; + dst->coeff1 = src->coeff1; + dst->coeff2 = src->coeff2; + dst->coeff3 = src->coeff3; +} + +static void +kmb_metadata_copy_color_comb(struct kmb_vpu_color_comb_params *dst, + struct kmb_color_comb_params *src) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++) + dst->matrix[i] = src->matrix[i]; + + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++) + dst->offsets[i] = src->offsets[i]; + + dst->coeff1 = src->coeff1; + dst->coeff2 = src->coeff2; + dst->coeff3 = src->coeff3; + dst->enable = src->enable; + dst->weight1 = src->weight1; + dst->weight2 = src->weight2; + dst->weight3 = src->weight3; + dst->limit1 = src->limit1; + dst->limit2 = src->limit2; + dst->offset1 = src->offset1; + dst->offset2 = src->offset2; +} + +static void +kmb_metadata_copy_hdr(struct kmb_vpu_hdr_params *dst, + struct kmb_hdr_params *src) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(dst->ratio); i++) + dst->ratio[i] = src->ratio[i]; + + for (i = 0; i < ARRAY_SIZE(dst->scale); i++) + dst->scale[i] = src->scale[i]; + + dst->offset1 = src->offset1; + dst->slope1 = src->slope1; + dst->offset2 = src->offset2; + dst->slope2 = src->slope2; + dst->offset3 = src->offset3; + dst->slope3 = src->slope3; + dst->offset4 = src->offset4; + dst->gain1 = src->gain1; + + for (i = 0; i < ARRAY_SIZE(dst->blur1); i++) + dst->blur1[i] = src->blur1[i]; + + for (i = 0; i < ARRAY_SIZE(dst->blur2); i++) + dst->blur2[i] = src->blur2[i]; + + dst->contrast1 = src->contrast1; + dst->contrast2 = src->contrast2; + dst->enable1 = src->enable1; + dst->enable2 = src->enable2; + dst->offset5 = src->offset5; + dst->gain2 = src->gain2; + dst->offset6 = src->offset6; + dst->strength = src->strength; + dst->offset7 = src->offset7; + dst->shift = src->shift; + dst->field1 = src->field1; + dst->field2 = src->field2; + dst->gain3 = src->gain3; + dst->min = src->min; +} + +static void +kmb_metadata_copy_lut(struct kmb_vpu_lut_params *dst, + struct kmb_lut_params *src) +{ + int i; + + dst->size = src->size; + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++) + dst->matrix[i] = src->matrix[i]; + + for (i = 0; i < ARRAY_SIZE(dst->offsets); i++) + dst->offsets[i] = src->offsets[i]; +} + +static void +kmb_metadata_copy_tnf(struct kmb_vpu_tnf_params *dst, + struct kmb_tnf_params *src) +{ + dst->factor = src->factor; + dst->gain = src->gain; + dst->offset1 = src->offset1; + dst->slope1 = src->slope1; + dst->offset2 = src->offset2; + dst->slope2 = src->slope2; + dst->min1 = src->min1; + dst->min2 = src->min2; + dst->value = src->value; + dst->enable = src->enable; +} + +static void +kmb_metadata_copy_dehaze(struct kmb_vpu_dehaze_params *dst, + struct kmb_dehaze_params *src) +{ + int i; + + dst->gain1 = src->gain1; + dst->min = src->min; + dst->strength1 = src->strength1; + dst->strength2 = src->strength2; + dst->gain2 = src->gain2; + dst->saturation = src->saturation; + dst->value1 = src->value1; + dst->value2 = src->value2; + dst->value3 = src->value3; + + for (i = 0; i < ARRAY_SIZE(dst->filter); i++) + dst->filter[i] = src->filter[i]; +} + +static void +kmb_metadata_copy_warp(struct kmb_vpu_warp_params *dst, + struct kmb_warp_params *src) +{ + int i; + + dst->type = src->type; + dst->relative = src->relative; + dst->format = src->format; + dst->position = src->position; + dst->width = src->width; + dst->height = src->height; + dst->stride = src->stride; + dst->enable = src->enable; + + for (i = 0; i < ARRAY_SIZE(dst->matrix); i++) + dst->matrix[i] = src->matrix[i]; + + dst->mode = src->mode; + + for (i = 0; i < ARRAY_SIZE(dst->values); i++) + dst->values[i] = src->values[i]; +} + +/* VPU Params tables */ +static struct kmb_metadata_table * +kmb_metadata_cpalloc_table(struct kmb_metadata *kmb_meta, + enum kmb_metadata_table_type type, + size_t src_table_size) +{ + struct kmb_metadata_table *table; + + lockdep_assert_held(&kmb_meta->lock); + + /* First create pool if needed */ + if (!kmb_meta->table_pool[type]) { + kmb_meta->table_pool[type] = + dma_pool_create(table_name[type], + kmb_meta->dma_dev, + src_table_size + sizeof(*table), + KMB_TABLE_ALIGN, 0); + if (!kmb_meta->table_pool[type]) { + dev_err(kmb_meta->dma_dev, + "Fail to create %s pool", table_name[type]); + return NULL; + } + } + + table = kmalloc(sizeof(*table), GFP_KERNEL); + if (!table) + return NULL; + + kref_init(&table->refcount); + table->pool = kmb_meta->table_pool[type]; + + table->cpu_addr = dma_pool_alloc(kmb_meta->table_pool[type], + GFP_KERNEL, + &table->dma_addr); + if (!table->cpu_addr) { + kfree(table); + return NULL; + } + + return table; +} + +static void kmb_metadata_free_table(struct kref *ref) +{ + struct kmb_metadata_table *table = + container_of(ref, struct kmb_metadata_table, refcount); + + dma_pool_free(table->pool, table->cpu_addr, table->dma_addr); + kfree(table); +} + +static void +kmb_metadata_release_tables(struct kmb_metadata_buf *meta_buf) +{ + int i; + + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) { + if (meta_buf->params.tab[i]) { + kref_put(&meta_buf->params.tab[i]->refcount, + kmb_metadata_free_table); + meta_buf->params.tab[i] = NULL; + } + } +} + +static void +kmb_metadata_destroy_table_pools(struct kmb_metadata *kmb_meta) +{ + int i; + + /* Release allocated pools during streaming */ + for (i = 0; i < KMB_METADATA_TABLE_MAX; i++) { + dma_pool_destroy(kmb_meta->table_pool[i]); + kmb_meta->table_pool[i] = NULL; + } +} + +static dma_addr_t +kmb_metadata_get_table_addr(struct kmb_metadata_buf *meta_buf, + enum kmb_metadata_table_type type) +{ + struct kmb_metadata_table *table = meta_buf->params.tab[type]; + + if (!table) + return 0; + + return table->dma_addr; +} + +static struct kmb_metadata_table * +kmb_metadata_create_table(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + enum kmb_metadata_table_type type, + size_t user_table_size) +{ + struct kmb_metadata_table *table; + + lockdep_assert_held(&kmb_meta->lock); + + table = kmb_metadata_cpalloc_table(kmb_meta, + type, + user_table_size); + if (!table) + return NULL; + + if (meta_buf->params.tab[type]) + kref_put(&meta_buf->params.tab[type]->refcount, + kmb_metadata_free_table); + + meta_buf->params.tab[type] = table; + + return table; +} + +static int +kmb_metadata_copy_table_usr(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + enum kmb_metadata_table_type type, + u8 *user_table, size_t user_table_size) +{ + struct kmb_metadata_table *table; + + table = kmb_metadata_create_table(kmb_meta, meta_buf, + type, user_table_size); + if (!table) + return -ENOMEM; + + memcpy(table->cpu_addr, user_table, user_table_size); + + return 0; +} + +static int kmb_metadata_create_default_table(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + enum kmb_metadata_table_type type, + u8 *user_table, + size_t user_table_size) +{ + struct kmb_metadata_table *table; + + table = kmb_metadata_create_table(kmb_meta, meta_buf, + type, user_table_size); + if (!table) + return -ENOMEM; + + memset(table->cpu_addr, 0, user_table_size); + + return 0; +} + +static void +kmb_metadata_copy_table_vpu(struct kmb_metadata_buf *meta_buf, + struct kmb_metadata_buf *last_meta_buf, + enum kmb_metadata_table_type type) +{ + /* Do nothing if params are the same */ + if (WARN_ON(meta_buf->params.isp == last_meta_buf->params.isp)) + return; + + meta_buf->params.tab[type] = last_meta_buf->params.tab[type]; + if (meta_buf->params.tab[type]) + kref_get(&meta_buf->params.tab[type]->refcount); +} + +static void +kmb_metadata_fill_blc(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.blc) { + kmb_metadata_copy_blc(params->blc, user_params->blc); + } else if (last_params) { + if (last_params != params) + memcpy(params->blc, last_params->blc, + sizeof(params->blc)); + } else { + memcpy(params->blc, def_params->blc, sizeof(params->blc)); + } +} + +static void +kmb_metadata_fill_signma_dns(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.sigma_dns) { + kmb_metadata_copy_sigma_dns(params->sigma_dns, + user_params->sigma_dns); + } else if (last_params) { + if (last_params != params) + memcpy(params->sigma_dns, last_params->sigma_dns, + sizeof(params->sigma_dns)); + } else { + memcpy(params->sigma_dns, def_params->sigma_dns, + sizeof(params->sigma_dns)); + } +} + +static void +kmb_metadata_fill_ae_awb(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.ae_awb) { + kmb_metadata_copy_ae_awb(¶ms->ae_awb, + &user_params->ae_awb); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->ae_awb, &last_params->ae_awb, + sizeof(params->ae_awb)); + } else { + memcpy(¶ms->ae_awb, def_params->ae_awb, + sizeof(params->ae_awb)); + } +} + +static void +kmb_metadata_fill_af(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.af) { + kmb_metadata_copy_af(¶ms->af, &user_params->af); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->af, &last_params->af, + sizeof(params->af)); + } else { + memcpy(¶ms->af, def_params->af, sizeof(params->af)); + } +} + +static void +kmb_metadata_fill_histogram(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.histogram) { + kmb_metadata_copy_histogram(¶ms->histogram, + &user_params->histogram); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->histogram, &last_params->histogram, + sizeof(params->histogram)); + } else { + memcpy(¶ms->histogram, def_params->histogram, + sizeof(params->histogram)); + } +} + +static void +kmb_metadata_fill_debayer(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.debayer) { + kmb_metadata_copy_debayer(¶ms->debayer, + &user_params->debayer); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->debayer, &last_params->debayer, + sizeof(params->debayer)); + } else { + memcpy(¶ms->debayer, def_params->debayer, + sizeof(params->debayer)); + } +} + +static void +kmb_metadata_fill_dog_dns(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.dog_dns) { + kmb_metadata_copy_dog_dns(¶ms->dog_dns, + &user_params->dog_dns); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->dog_dns, &last_params->dog_dns, + sizeof(params->dog_dns)); + } else { + memcpy(¶ms->dog_dns, def_params->dog_dns, + sizeof(params->dog_dns)); + } +} + +static void +kmb_metadata_fill_luma_dns(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.luma_dns) { + kmb_metadata_copy_luma_dns(¶ms->luma_dns, + &user_params->luma_dns); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->luma_dns, &last_params->luma_dns, + sizeof(params->luma_dns)); + } else { + memcpy(¶ms->luma_dns, def_params->luma_dns, + sizeof(params->luma_dns)); + } +} + +static void +kmb_metadata_fill_chroma_gen(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.chroma_gen) { + kmb_metadata_copy_chroma_gen(¶ms->chroma_gen, + &user_params->chroma_gen); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->chroma_gen, &last_params->chroma_gen, + sizeof(params->chroma_gen)); + } else { + memcpy(¶ms->chroma_gen, def_params->chroma_gen, + sizeof(params->chroma_gen)); + } +} + +static void +kmb_metadata_fill_median(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.median) { + kmb_metadata_copy_median(¶ms->median, + &user_params->median); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->median, &last_params->median, + sizeof(params->median)); + } else { + memcpy(¶ms->median, def_params->median, + sizeof(params->median)); + } +} + +static void +kmb_metadata_fill_chroma_dns(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.chroma_dns) { + kmb_metadata_copy_chroma_dns(¶ms->chroma_dns, + &user_params->chroma_dns); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->chroma_dns, &last_params->chroma_dns, + sizeof(params->chroma_dns)); + } else { + memcpy(¶ms->chroma_dns, def_params->chroma_dns, + sizeof(params->chroma_dns)); + } +} + +static void +kmb_metadata_fill_dehaze(struct kmb_vpu_isp_params *params, + struct kmb_isp_params *user_params, + struct kmb_vpu_isp_params *last_params, + struct kmb_vpu_isp_params_defaults *def_params) +{ + if (user_params->update.dehaze) { + kmb_metadata_copy_dehaze(¶ms->dehaze, + &user_params->dehaze); + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->dehaze, &last_params->dehaze, + sizeof(params->dehaze)); + } else { + memcpy(¶ms->dehaze, def_params->dehaze, + sizeof(params->dehaze)); + } +} + +static int +kmb_metadata_fill_lsc(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.lsc) { + kmb_metadata_copy_lsc(¶ms->lsc, + &user_params->lsc); + if (params->lsc.width && params->lsc.height) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LSC, + user_params->lsc.gain_mesh, + params->lsc.width * + params->lsc.height); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->lsc, &last_params->lsc, + sizeof(params->lsc)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_LSC); + } else { + memcpy(¶ms->lsc, def_params->lsc, sizeof(params->lsc)); + kmb_metadata_create_default_table(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LSC, + user_params->lsc.gain_mesh, + ARRAY_SIZE(user_params->lsc.gain_mesh)); + } + + if (params->lsc.width && params->lsc.height) { + params->lsc.addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_LSC); + if (!params->lsc.addr) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_raw(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.raw) { + kmb_metadata_copy_raw(¶ms->raw, + &user_params->raw); + if (params->raw.static_defect_size) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_SDEFECT, + user_params->raw.static_defect_map, + params->raw.static_defect_size); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->raw, &last_params->raw, + sizeof(params->raw)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_SDEFECT); + } else { + memcpy(¶ms->raw, def_params->raw, sizeof(params->raw)); + kmb_metadata_create_default_table(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_SDEFECT, + user_params->raw.static_defect_map, + ARRAY_SIZE(user_params->raw.static_defect_map)); + } + + if (params->raw.static_defect_size) { + params->raw.static_defect_addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_SDEFECT); + if (!params->raw.static_defect_addr) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_lca(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.lca) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LCA, + user_params->lca.coeff, + ARRAY_SIZE(user_params->lca.coeff)); + if (ret < 0) + return ret; + } else if (last_params) { + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_LCA); + } else { + kmb_metadata_create_default_table(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LCA, + user_params->lca.coeff, + ARRAY_SIZE(user_params->lca.coeff)); + } + + params->lca.addr = kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_LCA); + if (!params->lca.addr) + ret = -EINVAL; + + return ret; +} + +static int +kmb_metadata_fill_sharpen(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.sharpen) { + kmb_metadata_copy_sharpen(¶ms->sharpen, + &user_params->sharpen); + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_SHARP, + user_params->sharpen.radial_lut, + ARRAY_SIZE(user_params->sharpen.radial_lut)); + if (ret < 0) + return ret; + + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->sharpen, &last_params->sharpen, + sizeof(params->sharpen)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_SHARP); + } else { + memcpy(¶ms->sharpen, def_params->sharpen, + sizeof(params->sharpen)); + + kmb_metadata_create_default_table(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_SHARP, + user_params->sharpen.radial_lut, + ARRAY_SIZE(user_params->sharpen.radial_lut)); + } + + params->sharpen.addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_SHARP); + if (!params->sharpen.addr) + ret = -EINVAL; + + return ret; +} + +static int +kmb_metadata_fill_color_comb(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + struct kmb_color_comb_params *col = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.color_comb) { + col = &user_params->color_comb; + kmb_metadata_copy_color_comb(¶ms->color_comb, + &user_params->color_comb); + if (params->color_comb.enable) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_COLOR_CUMB, + col->lut_3d, + ARRAY_SIZE(col->lut_3d)); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->color_comb, &last_params->color_comb, + sizeof(params->color_comb)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_COLOR_CUMB); + } else { + memcpy(¶ms->color_comb, def_params->color_comb, + sizeof(params->color_comb)); + } + + if (params->color_comb.enable) { + params->color_comb.addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_COLOR_CUMB); + if (!params->color_comb.addr) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_hdr(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.hdr) { + kmb_metadata_copy_hdr(¶ms->hdr, + &user_params->hdr); + if (params->hdr.enable1 || params->hdr.enable2) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_HDR, + user_params->hdr.tm_lut, + ARRAY_SIZE(user_params->hdr.tm_lut)); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->hdr, &last_params->hdr, + sizeof(params->hdr)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_HDR); + } else { + memcpy(¶ms->hdr, def_params->hdr, sizeof(params->hdr)); + } + + if (params->hdr.enable1 || params->hdr.enable2) { + params->hdr.luts_addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_HDR); + if (!params->hdr.luts_addr) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_lut(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.lut) { + kmb_metadata_copy_lut(¶ms->lut, &user_params->lut); + if (params->lut.size) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LUT, + user_params->lut.table, + ARRAY_SIZE(user_params->lut.table)); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->lut, &last_params->lut, + sizeof(params->lut)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_LUT); + } else { + memcpy(¶ms->lut, def_params->lut, sizeof(params->lut)); + kmb_metadata_create_default_table(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_LUT, + user_params->lut.table, + ARRAY_SIZE(user_params->lut.table)); + } + + if (params->lut.size) { + params->lut.addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_LUT); + if (!params->lut.size) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_warp(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.warp) { + kmb_metadata_copy_warp(¶ms->warp, &user_params->warp); + if (params->warp.enable) { + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_WARP, + user_params->warp.mesh_grid, + ARRAY_SIZE(user_params->warp.mesh_grid)); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->warp, &last_params->warp, + sizeof(params->warp)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_WARP); + } else { + memcpy(¶ms->warp, def_params->warp, sizeof(params->warp)); + } + + if (params->warp.enable) { + params->warp.addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_WARP); + if (!params->warp.addr) + ret = -EINVAL; + } + + return ret; +} + +static int +kmb_metadata_fill_tnf(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + struct kmb_tnf_params *tnf = NULL; + int ret = 0; + + if (last_buf) + last_params = last_buf->params.isp; + + if (user_params->update.tnf) { + kmb_metadata_copy_tnf(¶ms->tnf, &user_params->tnf); + if (params->tnf.enable) { + tnf = &user_params->tnf; + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_TNF0, + tnf->chroma_lut0, + ARRAY_SIZE(tnf->chroma_lut0)); + if (ret < 0) + return ret; + + ret = kmb_metadata_copy_table_usr(kmb_meta, + meta_buf, + KMB_METADATA_TABLE_TNF1, + tnf->chroma_lut1, + ARRAY_SIZE(tnf->chroma_lut1)); + if (ret < 0) + return ret; + } + } else if (last_params) { + if (last_params != params) + memcpy(¶ms->tnf, &last_params->tnf, + sizeof(params->tnf)); + + if (kmb_meta->last_buf && meta_buf != kmb_meta->last_buf) { + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_TNF0); + kmb_metadata_copy_table_vpu(meta_buf, last_buf, + KMB_METADATA_TABLE_TNF1); + } + } else { + memcpy(¶ms->tnf, def_params->tnf, sizeof(params->tnf)); + } + + if (params->tnf.enable) { + params->tnf.lut0_addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_TNF0); + if (!params->tnf.lut0_addr) + return -EINVAL; + + params->tnf.lut1_addr = + kmb_metadata_get_table_addr(meta_buf, + KMB_METADATA_TABLE_TNF1); + if (!params->tnf.lut1_addr) + return -EINVAL; + } + + return ret; +} + +/* Fill static functions for conversions here */ +static int kmb_metadata_fill_isp_params(struct kmb_metadata *kmb_meta, + struct kmb_metadata_buf *meta_buf, + struct kmb_isp_params *user_params) +{ + struct kmb_vpu_isp_params *params = meta_buf->params.isp; + struct kmb_metadata_buf *last_buf = kmb_meta->last_buf; + struct kmb_vpu_isp_params *last_params = NULL; + struct kmb_vpu_isp_params_defaults *def_params = &kmb_meta->def; + int ret; + + if (last_buf) + last_params = last_buf->params.isp; + + kmb_metadata_fill_blc(params, user_params, last_params, def_params); + + kmb_metadata_fill_signma_dns(params, user_params, last_params, + def_params); + + kmb_metadata_fill_ae_awb(params, user_params, last_params, def_params); + + kmb_metadata_fill_af(params, user_params, last_params, def_params); + + kmb_metadata_fill_histogram(params, user_params, last_params, + def_params); + + kmb_metadata_fill_debayer(params, user_params, last_params, + def_params); + + kmb_metadata_fill_dog_dns(params, user_params, last_params, + def_params); + + kmb_metadata_fill_luma_dns(params, user_params, last_params, + def_params); + + kmb_metadata_fill_chroma_gen(params, user_params, last_params, + def_params); + + kmb_metadata_fill_median(params, user_params, last_params, def_params); + + kmb_metadata_fill_chroma_dns(params, user_params, last_params, + def_params); + + kmb_metadata_fill_dehaze(params, user_params, last_params, def_params); + + /* Copy params with tables */ + ret = kmb_metadata_fill_lsc(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_raw(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_lca(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_sharpen(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_color_comb(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_hdr(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_lut(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_warp(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + ret = kmb_metadata_fill_tnf(kmb_meta, meta_buf, user_params); + if (ret < 0) + goto error_release_tables; + + /* Store last buffer */ + kmb_meta->last_buf = meta_buf; + + return 0; + +error_release_tables: + kmb_metadata_release_tables(meta_buf); + return ret; +} + +static int kmb_metadata_queue_setup(struct vb2_queue *q, + unsigned int *num_buffers, + unsigned int *num_planes, + unsigned int sizes[], + struct device *alloc_devs[]) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q); + + *num_planes = 1; + sizes[0] = kmb_meta->format.buffersize; + + return 0; +} + +#define to_kmb_meta_buf(vbuf) container_of(vbuf, struct kmb_metadata_buf, vb) + +static int kmb_metadata_buf_params_init(struct vb2_buffer *vb) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf); + + buf->type = KMB_METADATA_PARAMS; + buf->params.isp = dma_alloc_coherent(kmb_meta->dma_dev, + sizeof(*buf->params.isp), + &buf->params.dma_addr_isp, 0); + if (!buf->params.isp) + return -ENOMEM; + + memset(buf->params.isp, 0, sizeof(*buf->params.isp)); + /* + * Table pools will be allocated per need. + * The pools need to be released when last buffer is finished. + * Use table reference count for that purpose + */ + kmb_meta->table_pools_refcnt++; + + return 0; +} + +static int kmb_metadata_buf_params_prepare(struct vb2_buffer *vb) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue); + struct kmb_isp_params *user_params = vb2_plane_vaddr(vb, 0); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf); + + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize); + return kmb_metadata_fill_isp_params(kmb_meta, buf, user_params); +} + +static void kmb_metadata_buf_params_cleanup(struct vb2_buffer *vb) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf); + + if (buf == kmb_meta->last_buf) + kmb_meta->last_buf = NULL; + + kmb_metadata_release_tables(buf); + dma_free_coherent(kmb_meta->dma_dev, sizeof(*buf->params.isp), + buf->params.isp, buf->params.dma_addr_isp); + + /* Destroy allocated table pools on last finish */ + if (kmb_meta->table_pools_refcnt-- == 1) + kmb_metadata_destroy_table_pools(kmb_meta); +} + +static int kmb_metadata_buf_stats_init(struct vb2_buffer *vb) +{ + dma_addr_t stats_addr = vb2_dma_contig_plane_dma_addr(vb, 0); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf); + int i; + + buf->type = KMB_METADATA_STATS; + memset(&buf->stats.raw, 0, sizeof(buf->stats.raw)); + buf->stats.dehaze_stats_addr = 0; + + /* Fill statistics addresses */ + for (i = 0; i < KMB_CAM_MAX_EXPOSURES; i++) { + buf->stats.raw[i].ae_awb_stats_addr = stats_addr + + offsetof(struct kmb_isp_stats, + exposure[i].ae_awb_stats[0]); + + buf->stats.raw[i].af_stats_addr = stats_addr + + offsetof(struct kmb_isp_stats, + exposure[i].af_stats[0]); + + buf->stats.raw[i].hist_luma_addr = stats_addr + + offsetof(struct kmb_isp_stats, + exposure[i].hist_luma[0]); + + buf->stats.raw[i].hist_rgb_addr = stats_addr + + offsetof(struct kmb_isp_stats, + exposure[i].hist_rgb[0]); + + buf->stats.raw[i].flicker_rows_addr = stats_addr + + offsetof(struct kmb_isp_stats, + exposure[i].flicker_rows[0]); + } + + buf->stats.dehaze_stats_addr = stats_addr + + offsetof(struct kmb_isp_stats, dehaze); + + return 0; +} + +static int kmb_metadata_buf_stats_prepare(struct vb2_buffer *vb) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue); + + vb2_set_plane_payload(vb, 0, kmb_meta->format.buffersize); + + return 0; +} + +static void kmb_metadata_buf_queue(struct vb2_buffer *vb) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_v4l2_buffer *vbuf = to_vb2_v4l2_buffer(vb); + struct kmb_metadata_buf *buf = to_kmb_meta_buf(vbuf); + int ret; + + ret = kmb_meta->queue_ops->queue(kmb_meta->priv, buf); + if (ret) + dev_err(&kmb_meta->video.dev, "Fail metadata queue %d", ret); +} + +static int kmb_metadata_start_streaming(struct vb2_queue *q, + unsigned int count) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q); + int ret; + + ret = kmb_pipe_prepare(kmb_meta->pipe); + if (ret < 0) + goto error_discard_all_bufs; + + ret = kmb_pipe_run(kmb_meta->pipe, &kmb_meta->video.entity); + if (ret < 0) + goto error_pipeline_stop; + + return 0; + +error_pipeline_stop: + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity); +error_discard_all_bufs: + kmb_meta->queue_ops->flush(kmb_meta->priv); + return 0; +} + +static void kmb_metadata_stop_streaming(struct vb2_queue *q) +{ + struct kmb_metadata *kmb_meta = vb2_get_drv_priv(q); + + kmb_pipe_stop(kmb_meta->pipe, &kmb_meta->video.entity); + + kmb_meta->queue_ops->flush(kmb_meta->priv); +} + +/* driver-specific operations */ +static struct vb2_ops kmb_meta_params_vb2_q_ops = { + .queue_setup = kmb_metadata_queue_setup, + .buf_init = kmb_metadata_buf_params_init, + .buf_prepare = kmb_metadata_buf_params_prepare, + .buf_cleanup = kmb_metadata_buf_params_cleanup, + .start_streaming = kmb_metadata_start_streaming, + .stop_streaming = kmb_metadata_stop_streaming, + .buf_queue = kmb_metadata_buf_queue, +}; + +static struct vb2_ops kmb_meta_stats_vb2_q_ops = { + .queue_setup = kmb_metadata_queue_setup, + .buf_init = kmb_metadata_buf_stats_init, + .buf_prepare = kmb_metadata_buf_stats_prepare, + .start_streaming = kmb_metadata_start_streaming, + .stop_streaming = kmb_metadata_stop_streaming, + .buf_queue = kmb_metadata_buf_queue, +}; + +#define to_kmb_meta_dev(vdev) container_of(vdev, struct kmb_metadata, video) + +static int kmb_metadata_querycap(struct file *file, void *fh, + struct v4l2_capability *cap) +{ + struct v4l2_fh *vfh = file->private_data; + struct kmb_metadata *kmb_meta = + to_kmb_meta_dev(vfh->vdev); + + cap->bus_info[0] = 0; + strscpy(cap->driver, kmb_meta->video.name, sizeof(cap->driver)); + snprintf(cap->bus_info, sizeof(cap->bus_info), "platform:%s", + kmb_meta->video.name); + + return 0; +} + +static int kmb_metadata_get_fmt(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct v4l2_fh *vfh = file->private_data; + struct kmb_metadata *kmb_meta = + to_kmb_meta_dev(vfh->vdev); + + f->fmt.meta = kmb_meta->format; + + return 0; +} + +static int kmb_metadata_try_fmt_cap(struct file *file, void *fh, + struct v4l2_format *f) +{ + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_STATS; + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_stats)) + f->fmt.meta.buffersize = sizeof(struct kmb_isp_stats); + + return 0; +} + +static int kmb_metadata_set_fmt_cap(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct v4l2_fh *vfh = file->private_data; + struct kmb_metadata *kmb_meta = + to_kmb_meta_dev(vfh->vdev); + int ret; + + ret = kmb_metadata_try_fmt_cap(file, fh, f); + if (ret < 0) + return ret; + + kmb_meta->format = f->fmt.meta; + + return 0; +} + +static int kmb_metadata_try_fmt_out(struct file *file, void *fh, + struct v4l2_format *f) +{ + f->fmt.meta.dataformat = V4L2_META_FMT_KMB_PARAMS; + if (f->fmt.meta.buffersize < sizeof(struct kmb_isp_params)) + f->fmt.meta.buffersize = sizeof(struct kmb_isp_params); + + return 0; +} + +static int kmb_metadata_set_fmt_out(struct file *file, void *fh, + struct v4l2_format *f) +{ + struct v4l2_fh *vfh = file->private_data; + struct kmb_metadata *kmb_meta = + to_kmb_meta_dev(vfh->vdev); + int ret; + + ret = kmb_metadata_try_fmt_out(file, fh, f); + if (ret < 0) + return ret; + + kmb_meta->format = f->fmt.meta; + + return 0; +} + +/* V4L2 ioctl operations */ +static const struct v4l2_ioctl_ops kmb_vid_ioctl_ops = { + .vidioc_querycap = kmb_metadata_querycap, + .vidioc_g_fmt_meta_out = kmb_metadata_get_fmt, + .vidioc_s_fmt_meta_out = kmb_metadata_set_fmt_out, + .vidioc_try_fmt_meta_out = kmb_metadata_try_fmt_out, + .vidioc_g_fmt_meta_cap = kmb_metadata_get_fmt, + .vidioc_s_fmt_meta_cap = kmb_metadata_set_fmt_cap, + .vidioc_try_fmt_meta_cap = kmb_metadata_try_fmt_cap, + .vidioc_reqbufs = vb2_ioctl_reqbufs, + .vidioc_querybuf = vb2_ioctl_querybuf, + .vidioc_qbuf = vb2_ioctl_qbuf, + .vidioc_dqbuf = vb2_ioctl_dqbuf, + .vidioc_streamon = vb2_ioctl_streamon, + .vidioc_streamoff = vb2_ioctl_streamoff, +}; + +static int kmb_metadata_open(struct file *file) +{ + struct kmb_metadata *kmb_meta = video_drvdata(file); + int ret; + + mutex_lock(&kmb_meta->lock); + + ret = v4l2_fh_open(file); + if (ret) { + mutex_unlock(&kmb_meta->lock); + return ret; + } + + ret = kmb_pipe_request(kmb_meta->pipe); + if (ret < 0) + goto error_fh_release; + + mutex_unlock(&kmb_meta->lock); + + return 0; + +error_fh_release: + _vb2_fop_release(file, NULL); + mutex_unlock(&kmb_meta->lock); + return ret; +} + +static int kmb_metadata_release(struct file *file) +{ + struct kmb_metadata *kmb_meta = video_drvdata(file); + int ret; + + mutex_lock(&kmb_meta->lock); + + kmb_pipe_release(kmb_meta->pipe); + + ret = _vb2_fop_release(file, NULL); + + mutex_unlock(&kmb_meta->lock); + + return ret; +} + +/* V4L2 file operations */ +static const struct v4l2_file_operations kmb_vid_output_fops = { + .owner = THIS_MODULE, + .unlocked_ioctl = video_ioctl2, + .open = kmb_metadata_open, + .release = kmb_metadata_release, + .poll = vb2_fop_poll, + .mmap = vb2_fop_mmap, +}; + /** - * kmb_video_init - Initialize entity + * kmb_metadata_init - Initialize entity * @kmb_meta: pointer to kmb isp config device * * Return: 0 if successful, error code otherwise. */ int kmb_metadata_init(struct kmb_metadata *kmb_meta) { + int ret; + + mutex_init(&kmb_meta->lock); + + kmb_meta->table_pools_refcnt = 0; + memset(kmb_meta->table_pool, 0, sizeof(kmb_meta->table_pool)); + + kmb_meta->video.fops = &kmb_vid_output_fops; + kmb_meta->video.ioctl_ops = &kmb_vid_ioctl_ops; + kmb_meta->video.minor = -1; + kmb_meta->video.release = video_device_release; + kmb_meta->video.vfl_type = VFL_TYPE_VIDEO; + kmb_meta->video.lock = &kmb_meta->lock; + kmb_meta->video.queue = &kmb_meta->vb2_q; + video_set_drvdata(&kmb_meta->video, kmb_meta); + + kmb_meta->vb2_q.drv_priv = kmb_meta; + kmb_meta->vb2_q.buf_struct_size = sizeof(struct kmb_metadata_buf); + kmb_meta->vb2_q.io_modes = VB2_DMABUF | VB2_MMAP; + kmb_meta->vb2_q.timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY; + kmb_meta->vb2_q.dev = kmb_meta->dma_dev; + kmb_meta->vb2_q.lock = &kmb_meta->lock; + kmb_meta->vb2_q.min_buffers_needed = 1; + + /* Initialize per type variables */ + kmb_meta->video.device_caps = V4L2_CAP_STREAMING; + if (kmb_meta->type == KMB_METADATA_PARAMS) { + kmb_meta->video.device_caps |= V4L2_CAP_META_OUTPUT; + kmb_meta->video.vfl_dir = VFL_DIR_TX; + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name), + KMB_CAM_METADATA_PARAMS_NAME); + + kmb_meta->vb2_q.ops = &kmb_meta_params_vb2_q_ops; + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops; + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_OUTPUT; + + kmb_meta->pad.flags = MEDIA_PAD_FL_SOURCE; + + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_PARAMS; + kmb_meta->format.buffersize = sizeof(struct kmb_isp_params); + } else { + kmb_meta->video.device_caps |= V4L2_CAP_META_CAPTURE; + kmb_meta->video.vfl_dir = VFL_DIR_RX; + + snprintf(kmb_meta->video.name, sizeof(kmb_meta->video.name), + KMB_CAM_METADATA_STATS_NAME); + + kmb_meta->vb2_q.ops = &kmb_meta_stats_vb2_q_ops; + kmb_meta->vb2_q.mem_ops = &vb2_dma_contig_memops; + kmb_meta->vb2_q.type = V4L2_BUF_TYPE_META_CAPTURE; + + kmb_meta->pad.flags = MEDIA_PAD_FL_SINK; + + kmb_meta->format.dataformat = V4L2_META_FMT_KMB_STATS; + kmb_meta->format.buffersize = sizeof(struct kmb_isp_stats); + } + + ret = media_entity_pads_init(&kmb_meta->video.entity, + 1, &kmb_meta->pad); + if (ret < 0) + goto error_mutex_destroy; + + ret = vb2_queue_init(&kmb_meta->vb2_q); + if (ret < 0) { + dev_err(&kmb_meta->video.dev, "Error vb2 queue init"); + goto error_metadata_cleanup; + } + + kmb_params_get_defaults(&kmb_meta->def); + return 0; + +error_metadata_cleanup: + kmb_metadata_cleanup(kmb_meta); +error_mutex_destroy: + mutex_destroy(&kmb_meta->lock); + + return ret; } /** @@ -22,7 +1823,10 @@ int kmb_metadata_init(struct kmb_metadata *kmb_meta) * @kmb_meta: pointer to kmb isp config device */ void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta) -{ } +{ + media_entity_cleanup(&kmb_meta->video.entity); + mutex_destroy(&kmb_meta->lock); +} /** * kmb_metadata_register - Register V4L2 device @@ -34,7 +1838,15 @@ void kmb_metadata_cleanup(struct kmb_metadata *kmb_meta) int kmb_metadata_register(struct kmb_metadata *kmb_meta, struct v4l2_device *v4l2_dev) { - return 0; + int ret; + + kmb_meta->video.v4l2_dev = v4l2_dev; + + ret = video_register_device(&kmb_meta->video, VFL_TYPE_VIDEO, -1); + if (ret < 0) + dev_err(&kmb_meta->video.dev, "Failed to register video device"); + + return ret; } /** @@ -42,4 +1854,7 @@ int kmb_metadata_register(struct kmb_metadata *kmb_meta, * @kmb_meta: pointer to kmb isp config device */ void kmb_metadata_unregister(struct kmb_metadata *kmb_meta) -{ } +{ + mutex_destroy(&kmb_meta->lock); + video_unregister_device(&kmb_meta->video); +} diff --git a/drivers/media/platform/keembay-camera/keembay-metadata.h b/drivers/media/platform/keembay-camera/keembay-metadata.h index 88e85d3caba0..ab77ed11bd15 100644 --- a/drivers/media/platform/keembay-camera/keembay-metadata.h +++ b/drivers/media/platform/keembay-camera/keembay-metadata.h @@ -12,6 +12,7 @@ #include #include "keembay-vpu-isp.h" +#include "keembay-params-defaults.h" /** * enum kmb_metadata_table_type - Keembay metadata table type @@ -68,12 +69,12 @@ struct kmb_metadata_table { * @vb: Video buffer for v4l2 * @type: Metadata type * @stats: Statistics physical addresses - * @raw: VPU raw statistics physical addresses - * @dehaze_stats_addr: VPU dehaze statistics physical address + * @stats.raw: VPU raw statistics physical addresses + * @stats.dehaze_stats_addr: VPU dehaze statistics physical address * @params: VPU ISP parameters - * @isp: VPU ISP parameters virtual address - * @dma_addr_isp: VPU ISP parameters physical address - * @tab: Metadata tables + * @params.isp: VPU ISP parameters virtual address + * @params.dma_addr_isp: VPU ISP parameters physical address + * @params.tab: Metadata tables * @list: List for buffer queue */ struct kmb_metadata_buf { @@ -118,6 +119,7 @@ struct kmb_metabuf_queue_ops { * @table_pool: ISP tables dma pool * @last_buf: Pointer to last enqueued buffer * @format: Active format + * @def: Default ISP params */ struct kmb_metadata { struct mutex lock; @@ -138,6 +140,8 @@ struct kmb_metadata { struct kmb_metadata_buf *last_buf; struct v4l2_meta_format format; + + struct kmb_vpu_isp_params_defaults def; }; int kmb_metadata_init(struct kmb_metadata *kmb_meta); diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.c b/drivers/media/platform/keembay-camera/keembay-params-defaults.c new file mode 100644 index 000000000000..a2dd7888375e --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.c @@ -0,0 +1,326 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Intel Keem Bay camera ISP parameter defaults. + * + * Copyright (C) 2021 Intel Corporation + */ +#include +#include + +#include "keembay-params-defaults.h" + +static const struct kmb_vpu_blc_params blc_default[KMB_VPU_MAX_EXPOSURES] = { + { + .coeff1 = 800, + .coeff2 = 800, + .coeff3 = 800, + .coeff4 = 800, + }, + { + .coeff1 = 800, + .coeff2 = 800, + .coeff3 = 800, + .coeff4 = 800, + }, + { + .coeff1 = 800, + .coeff2 = 800, + .coeff3 = 800, + .coeff4 = 800, + } + +}; + +static const struct kmb_vpu_sigma_dns_params + sigma_dns_default[KMB_VPU_MAX_EXPOSURES] = { 0 }; + +static const struct kmb_vpu_lsc_params lsc_default = { + .threshold = 2048, + .width = 64, + .height = 44, + .reserved = { 0 }, +}; + +static const struct kmb_vpu_raw_params raw_default = { + .awb_stats_en = 0, + .awb_rgb_hist_en = 0, + .af_stats_en = 0, + .luma_hist_en = 0, + .flicker_accum_en = 0, + .bad_pixel_fix_en = 0, + .grgb_imb_en = 1, + .mono_imbalance_en = 0, + .gain1 = 269, + .gain2 = 452, + .gain3 = 634, + .gain4 = 269, + .stop1 = 400, + .stop2 = 450, + .stop3 = 700, + .stop4 = 800, + .threshold1 = 128, + .alpha1 = 12, + .alpha2 = 12, + .alpha3 = 12, + .alpha4 = 12, + .threshold2 = 53, + .static_defect_size = 1, + .reserved = { 0 }, + .flicker_first_row_acc = 0, + .flicker_last_row_acc = 0, +}; + +static const struct kmb_vpu_ae_awb_params ae_awb_default = { + .start_x = 0, + .start_y = 0, + .width = 100, + .height = 98, + .skip_x = 100, + .skip_y = 98, + .patches_x = 38, + .patches_y = 22, + .threshold1 = 0, + .threshold2 = 4095, +}; + +static const struct kmb_vpu_af_params af_default = { + .start_x = 0, + .start_y = 0, + .width = 192, + .height = 144, + .patches_x = 20, + .patches_y = 15, + .coeff = 0, + .threshold1 = 0, + .threshold2 = 0, + .coeffs1 = {31, 19, -32, 31, 63, 31, -50, -35, 35, -70, 35}, + .coeffs2 = {35, 11, -29, 8, 17, 8, 78, -39, 119, -238, 119}, +}; + +static const struct kmb_vpu_hist_params histogram_default = { + .start_x = 0, + .start_y = 0, + .end_x = 3839, + .end_y = 2156, + .matrix = {1719, 0, 0, 0, 1024, 0, 0, 0, 2414}, + .weight = {64, 128, 64}, +}; + +// only address - nothing to init... +static const struct kmb_vpu_lca_params lca_default = { 0 }; + +static const struct kmb_vpu_debayer_params debayer_default = { + .coeff1 = 51, + .multiplier1 = 13107, + .multiplier2 = 13107, + .coeff2 = 77, + .coeff3 = 150, + .coeff4 = 29, +}; + +static const struct kmb_vpu_dog_dns_params dog_dns_default = { + .threshold = 0, + .strength = 0, + .coeffs11 = {0, 0, 0, 0, 0, 255}, + .coeffs15 = {0, 0, 0, 0, 0, 0, 0, 255}, + .reserved = { 0 }, +}; + +static const struct kmb_vpu_luma_dns_params luma_dns_default = { + .threshold = 13094, + .slope = 967, + .shift = 7, + .alpha = 50, + .weight = 0, + .per_pixel_alpha_en = 0, + .gain_bypass_en = 0, + .reserved = { 0 }, +}; + +static const struct kmb_vpu_sharpen_params sharpen_default = { + .coeffs1 = {0, 0, 0, 4, 182, 396}, + .coeffs2 = {0, 0, 0, 1, 141, 740}, + .coeffs3 = {0, 0, 2, 42, 246, 444}, + .shift = 15, + .gain1 = 3396, + .gain2 = 3378, + .gain3 = 3270, + .gain4 = 3400, + .gain5 = 207, + .stops1 = {20, 40, 605}, + .gains = {10, 120, 60}, + .stops2 = {11, 100, 2500, 4000}, + .overshoot = 359, + .undershoot = 146, + .alpha = 36, + .gain6 = 128, + .offset = 637, +}; + +static const struct kmb_vpu_chroma_gen_params chroma_gen_default = { + .epsilon = 2, + .coeff1 = 426, + .coeff2 = 767, + .coeff3 = 597, + .coeff4 = 77, + .coeff5 = 150, + .coeff6 = 29, + .strength1 = 0, + .strength2 = 32, + .coeffs = {33, 59, 71}, + .offset1 = 2, + .slope1 = 230, + .slope2 = 256, + .offset2 = 0, + .limit = 767, +}; + +static const struct kmb_vpu_median_params median_default = { + .size = 7, + .slope = 32, + .offset = -19, +}; + +static const struct kmb_vpu_chroma_dns_params chroma_dns_default = { + .limit = 255, + .enable = 0, + .threshold1 = 30, + .threshold2 = 30, + .threshold3 = 30, + .threshold4 = 30, + .threshold5 = 45, + .threshold6 = 45, + .threshold7 = 45, + .threshold8 = 45, + .slope1 = 77, + .offset1 = -15, + .slope2 = 255, + .offset2 = 127, + .grey1 = 421, + .grey2 = 758, + .grey3 = 590, + .coeff1 = 52, + .coeff2 = 32, + .coeff3 = 19, +}; + +static const struct kmb_vpu_color_comb_params color_comb_default = { + .matrix = {1303, 65427, 65367, 65172, 1463, 65462, 55, 65034, 1459}, + .offsets = { 0 }, + .coeff1 = 615, + .coeff2 = 342, + .coeff3 = 439, + .reserved = { 0 }, + .enable = 0, + .weight1 = 85, + .weight2 = 86, + .weight3 = 85, + .limit1 = 512, + .limit2 = -8192, + .offset1 = 0, + .offset2 = 0, +}; + +static const struct kmb_vpu_hdr_params hdr_default = { + .ratio = {256, 256}, + .scale = {262143, 262143, 262143}, + .offset1 = -3275, + .slope1 = 320, + .offset2 = -3685, + .slope2 = 641, + .offset3 = -4054, + .slope3 = 4095, + .offset4 = 3686, + .gain1 = 16, + .blur1 = {0, 0, 255}, + .blur2 = {0, 0, 0, 0, 255}, + .contrast1 = 20, + .contrast2 = 16, + .enable1 = 0, + .enable2 = 0, + .offset5 = 0, + .offset6 = 0, + .strength = 0, + .reserved1 = { 0 }, + .offset7 = 15, + .shift = 1702133760, + .field1 = 16, + .field2 = 123, + .gain3 = 0, + .min = 0, + .reserved2 = { 0 }, +}; + +static const struct kmb_vpu_lut_params lut_default = { + .size = 512, + .reserved = { 0 }, + .matrix = {262, 516, 100, 3945, 3799, 449, 449, 3720, 4023}, + .offsets = {256, 2048, 2048}, +}; + +static const struct kmb_vpu_tnf_params tnf_default = { + .factor = 179, + .gain = 0, + .offset1 = 217, + .slope1 = 162, + .offset2 = 299, + .slope2 = 121, + .min1 = 0, + .min2 = 40, + .value = 128, + .enable = 0, +}; + +static const struct kmb_vpu_dehaze_params dehaze_default = { + .gain1 = 512, + .min = 70, + .strength1 = 0, + .strength2 = 0, + .gain2 = 128, + .saturation = 127, + .value1 = 2048, + .value2 = 2048, + .value3 = 2048, + .filter = {0, 0, 255}, +}; + +static const struct kmb_vpu_warp_params warp_default = { + .type = 0, + .relative = 0, + .format = 0, + .position = 0, + .reserved = { 0 }, + .width = 8, + .height = 4, + .stride = 128, + .enable = 0, + .matrix = {1, 0, 0, 0, 1, 0, 0, 0, 1}, + .mode = 1, + .values = {0, 128, 128}, +}; + +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults) +{ + defaults->blc = blc_default; + defaults->sigma_dns = sigma_dns_default; + defaults->lsc = &lsc_default; + defaults->raw = &raw_default; + defaults->ae_awb = &ae_awb_default; + defaults->af = &af_default; + defaults->histogram = &histogram_default; + defaults->lca = &lca_default; + defaults->debayer = &debayer_default; + defaults->dog_dns = &dog_dns_default; + defaults->luma_dns = &luma_dns_default; + defaults->sharpen = &sharpen_default; + defaults->chroma_gen = &chroma_gen_default; + defaults->median = &median_default; + defaults->chroma_dns = &chroma_dns_default; + defaults->color_comb = &color_comb_default; + defaults->hdr = &hdr_default; + defaults->lut = &lut_default; + defaults->tnf = &tnf_default; + defaults->dehaze = &dehaze_default; + defaults->warp = &warp_default; +} + diff --git a/drivers/media/platform/keembay-camera/keembay-params-defaults.h b/drivers/media/platform/keembay-camera/keembay-params-defaults.h new file mode 100644 index 000000000000..d6134d64be7c --- /dev/null +++ b/drivers/media/platform/keembay-camera/keembay-params-defaults.h @@ -0,0 +1,38 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Intel Keem Bay camera ISP parameter defaults. + * + * Copyright (C) 2021 Intel Corporation + */ +#ifndef KEEMBAY_DEFAULTS_H +#define KEEMBAY_DEFAULTS_H + +#include "keembay-vpu-isp.h" + +struct kmb_vpu_isp_params_defaults { + const struct kmb_vpu_blc_params *blc; + const struct kmb_vpu_sigma_dns_params *sigma_dns; + const struct kmb_vpu_lsc_params *lsc; + const struct kmb_vpu_raw_params *raw; + const struct kmb_vpu_ae_awb_params *ae_awb; + const struct kmb_vpu_af_params *af; + const struct kmb_vpu_hist_params *histogram; + const struct kmb_vpu_lca_params *lca; + const struct kmb_vpu_debayer_params *debayer; + const struct kmb_vpu_dog_dns_params *dog_dns; + const struct kmb_vpu_luma_dns_params *luma_dns; + const struct kmb_vpu_sharpen_params *sharpen; + const struct kmb_vpu_chroma_gen_params *chroma_gen; + const struct kmb_vpu_median_params *median; + const struct kmb_vpu_chroma_dns_params *chroma_dns; + const struct kmb_vpu_color_comb_params *color_comb; + const struct kmb_vpu_hdr_params *hdr; + const struct kmb_vpu_lut_params *lut; + const struct kmb_vpu_tnf_params *tnf; + const struct kmb_vpu_dehaze_params *dehaze; + const struct kmb_vpu_warp_params *warp; +}; + +void kmb_params_get_defaults(struct kmb_vpu_isp_params_defaults *defaults); + +#endif /* KEEMBAY_DEFAULTS_H */