From patchwork Fri Jun 30 20:43:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698230 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D19CCEB64DC for ; Fri, 30 Jun 2023 20:43:51 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232599AbjF3Unu (ORCPT ); Fri, 30 Jun 2023 16:43:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53308 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232590AbjF3Uns (ORCPT ); Fri, 30 Jun 2023 16:43:48 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D17B3C0B for ; Fri, 30 Jun 2023 13:43:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157827; x=1719693827; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=x+LP+4lfNYxrrkLWfRafgulLicRZ0Ly2LjAWyJO66Mc=; b=a9cS1vNDziTIRziszEk7BVwR4w83L7WCCUzD5j3XmVBRQe4VckzdrC0k H7n61skK6khpA4mULpc6BrPTl317KDNqXrCEbY1y+5h0WZfWCuBqsDWor ILhFbWYT6MBT6EYPqHTuojU3K/UMPO5LWGjVf6qjbUGUIJgnG0JHdG1Lo bGhln9vtKBWnPguETHgrmvRyKxjlmKBuAUi49Y+EURYyU179hXYFewTeg CkEO7GaNw+bIsFHkA8SOun8tmYU1KGiv9io9EeZeKN6yPWy03/Bno8LwA KezqdvFxXdx63GAK4UULiLLjb25V/75wTGRgWkTUnJuh+LVrqKENu0SHM g==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866688" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866688" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:47 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987156" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987156" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:46 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 7595112075E; Fri, 30 Jun 2023 23:43:41 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 1/7] media: mc: Check pad flag validity Date: Fri, 30 Jun 2023 23:43:32 +0300 Message-Id: <20230630204338.126583-2-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Check the validity of pad flags on entity init. Exactly one of the flags must be set. Signed-off-by: Sakari Ailus --- drivers/media/mc/mc-entity.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 83468d4a440b..4991281dcccc 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -197,6 +197,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, struct media_device *mdev = entity->graph_obj.mdev; struct media_pad *iter; unsigned int i = 0; + int ret = 0; if (num_pads >= MEDIA_ENTITY_MAX_PADS) return -E2BIG; @@ -210,6 +211,14 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, media_entity_for_each_pad(entity, iter) { iter->entity = entity; iter->index = i++; + + if (hweight32(iter->flags & (MEDIA_PAD_FL_SINK | + MEDIA_PAD_FL_SOURCE)) + != 1) { + ret = -EINVAL; + break; + } + if (mdev) media_gobj_create(mdev, MEDIA_GRAPH_PAD, &iter->graph_obj); @@ -218,7 +227,7 @@ int media_entity_pads_init(struct media_entity *entity, u16 num_pads, if (mdev) mutex_unlock(&mdev->graph_mutex); - return 0; + return ret; } EXPORT_SYMBOL_GPL(media_entity_pads_init); From patchwork Fri Jun 30 20:43:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF6ACC0015E for ; Fri, 30 Jun 2023 20:43:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232613AbjF3Unw (ORCPT ); Fri, 30 Jun 2023 16:43:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232623AbjF3Unu (ORCPT ); Fri, 30 Jun 2023 16:43:50 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 669C23C29 for ; Fri, 30 Jun 2023 13:43:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157829; x=1719693829; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r/SIjrwUReb/l+aDUKCinWddolNalKr3CmKEm3N7G3M=; b=T5ljduepHRFJenSWEQLo5wSc5xrlmuHMqAJ6Ht/O8mdqTdGCO7M2jxoU AFfjeHv+Od9DZ/lFlPObRb3pC19Lmoy82FlHbNZLaJmIIxyKH34IkHzG4 DjCdO/asJyyEBDBIqWIp9ZY8jSUkUrb8HV1f8TwxAYR1FkmFLRie/r8wY /jLo4rgh8tEz1lYsjotR/5LfzQGoC20Zsbb8+PaZr9/+9nRJwrwG6lC4z RW54ev002Lhubo/QNj9z8UEO1Uss2clopa1XKgl7P0CPKII6bkoP8ac08 +lelZAnyQDZLiUE7t7svkvcpAI2LvkzKxPNYsni1RZ1Chzmt2ey03c9U5 g==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866695" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866695" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:49 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987171" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987171" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:48 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 5814A12078C; Fri, 30 Jun 2023 23:43:43 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 2/7] media: mc: Add INTERNAL pad flag Date: Fri, 30 Jun 2023 23:43:33 +0300 Message-Id: <20230630204338.126583-3-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Internal source pads will be used as routing endpoints in V4L2 [GS]_ROUTING IOCTLs, to indicate that the stream begins in the entity. Also prevent creating links to pads that have been flagged as internal. Signed-off-by: Sakari Ailus --- Documentation/userspace-api/media/glossary.rst | 6 ++++++ Documentation/userspace-api/media/mediactl/media-types.rst | 6 ++++++ drivers/media/mc/mc-entity.c | 6 +++++- include/uapi/linux/media.h | 1 + 4 files changed, 18 insertions(+), 1 deletion(-) diff --git a/Documentation/userspace-api/media/glossary.rst b/Documentation/userspace-api/media/glossary.rst index 96a360edbf3b..f7b99a4527c7 100644 --- a/Documentation/userspace-api/media/glossary.rst +++ b/Documentation/userspace-api/media/glossary.rst @@ -173,6 +173,12 @@ Glossary An integrated circuit that integrates all components of a computer or other electronic systems. +_media-glossary-stream: + Stream + A distinct flow of data (image data or metadata) over a media pipeline + from source to sink. A source may be e.g. an image sensor and a sink + e.g. a memory buffer. + V4L2 API **V4L2 userspace API** diff --git a/Documentation/userspace-api/media/mediactl/media-types.rst b/Documentation/userspace-api/media/mediactl/media-types.rst index 0ffeece1e0c8..28941da27790 100644 --- a/Documentation/userspace-api/media/mediactl/media-types.rst +++ b/Documentation/userspace-api/media/mediactl/media-types.rst @@ -361,6 +361,7 @@ Types and flags used to represent the media graph elements .. _MEDIA-PAD-FL-SINK: .. _MEDIA-PAD-FL-SOURCE: .. _MEDIA-PAD-FL-MUST-CONNECT: +.. _MEDIA-PAD-FL-INTERNAL: .. flat-table:: Media pad flags :header-rows: 0 @@ -382,6 +383,11 @@ Types and flags used to represent the media graph elements when this flag isn't set; the absence of the flag doesn't imply there is none. + * - ``MEDIA_PAD_FL_INTERNAL`` + - The internal flag indicates an internal pad that has no external + connections. Such a pad shall not be connected with a link. The + internal flag indicates that the :ref:``stream + `` either starts or ends in the entity. One and only one of ``MEDIA_PAD_FL_SINK`` and ``MEDIA_PAD_FL_SOURCE`` must be set for every pad. diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 4991281dcccc..03a9e0b8ebab 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1071,7 +1071,8 @@ int media_get_pad_index(struct media_entity *entity, u32 pad_type, for (i = 0; i < entity->num_pads; i++) { if ((entity->pads[i].flags & - (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE)) != pad_type) + (MEDIA_PAD_FL_SINK | MEDIA_PAD_FL_SOURCE | + MEDIA_PAD_FL_INTERNAL)) != pad_type) continue; if (entity->pads[i].sig_type == sig_type) @@ -1094,6 +1095,9 @@ media_create_pad_link(struct media_entity *source, u16 source_pad, return -EINVAL; if (WARN_ON(!(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE))) return -EINVAL; + if (WARN_ON(source->pads[source_pad].flags & MEDIA_PAD_FL_SOURCE && + source->pads[source_pad].flags & MEDIA_PAD_FL_INTERNAL)) + return -EINVAL; if (WARN_ON(!(sink->pads[sink_pad].flags & MEDIA_PAD_FL_SINK))) return -EINVAL; diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index 1c80b1d6bbaf..80cfd12a43fc 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -208,6 +208,7 @@ struct media_entity_desc { #define MEDIA_PAD_FL_SINK (1U << 0) #define MEDIA_PAD_FL_SOURCE (1U << 1) #define MEDIA_PAD_FL_MUST_CONNECT (1U << 2) +#define MEDIA_PAD_FL_INTERNAL (1U << 3) struct media_pad_desc { __u32 entity; /* entity ID */ From patchwork Fri Jun 30 20:43:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E29BEB64DA for ; Fri, 30 Jun 2023 20:43:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232614AbjF3Unw (ORCPT ); Fri, 30 Jun 2023 16:43:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53292 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232608AbjF3Unv (ORCPT ); Fri, 30 Jun 2023 16:43:51 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7EE523C02 for ; Fri, 30 Jun 2023 13:43:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157830; x=1719693830; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=kJd8KnHGQhFKx0lw/mrsMv6ORjD8mulHwgk+qIF3LoQ=; b=hFUFGY/heFJvm+qVEoWXceo0snCx/VLrmn0/kVJjBgNONIC1KpTlP3/6 +njVl8tWPwzNcJ2J8nf+fKR389/Ke7dHKYL4gf4qS0NXDjx3h9OcRuVnR oSaod/untRm6QmAIcHl6kiSmHWwzW9QSBfuscrwMATzcXAjW8NAMntglL SK5w7wUHQQ5SuD4ELj0D2TQOA431y2ITWsR7NiFK4qLNyUQJczZXRhZax 02+BjkFsWVYKN91o1kQYkvBXPi9SqEoEPJhWS2Pn+Oq9mxtg/H6krzLjx ZPFUBX1bHfr+N2Q4CqihgCMXRDnkElRm5Lix+RIQHwW7g2n5GFPfQ7EfU Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866698" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866698" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:50 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987182" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987182" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:49 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id DF6801207EB; Fri, 30 Jun 2023 23:43:44 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 3/7] media: v4l: subdev: Support INTERNAL pads in routing IOCTLs Date: Fri, 30 Jun 2023 23:43:34 +0300 Message-Id: <20230630204338.126583-4-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Take the new INTERNAL pad flag into account in validating routing IOCTL argument. Effectively this is a SINK pad in this respect. Due to the union there's no need to check for the particular name. Signed-off-by: Sakari Ailus --- drivers/media/v4l2-core/v4l2-subdev.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/media/v4l2-core/v4l2-subdev.c b/drivers/media/v4l2-core/v4l2-subdev.c index 2ec179cd1264..36886a9047c4 100644 --- a/drivers/media/v4l2-core/v4l2-subdev.c +++ b/drivers/media/v4l2-core/v4l2-subdev.c @@ -1787,7 +1787,8 @@ int v4l2_subdev_routing_validate(struct v4l2_subdev *sd, /* Validate the sink and source pad numbers. */ if (route->sink_pad >= sd->entity.num_pads || - !(sd->entity.pads[route->sink_pad].flags & MEDIA_PAD_FL_SINK)) { + !(sd->entity.pads[route->sink_pad].flags & + MEDIA_PAD_FL_SINK)) { dev_dbg(sd->dev, "route %u sink (%u) is not a sink pad\n", i, route->sink_pad); goto out; From patchwork Fri Jun 30 20:43:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4B6D8EB64DC for ; Fri, 30 Jun 2023 20:43:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232631AbjF3Unz (ORCPT ); Fri, 30 Jun 2023 16:43:55 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53354 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229919AbjF3Unw (ORCPT ); Fri, 30 Jun 2023 16:43:52 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0466EA2 for ; Fri, 30 Jun 2023 13:43:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157831; x=1719693831; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NTFlhRtgV9s619y/uGT6ElayJF4/jK9alQChh5RF9dI=; b=QVdGfE2vAX0lUVmmmbc8jFZfPlMxZdeWokxonWyk9x05JicMkQ9MsuVW Q2zAvSnX1I7NZp1dX1hgNpywbnu7diAXeunLpK/GQfhQsA3DDK5esIrE+ f/9oCovmRd5SuN4X7CeAZqr8Mhna8iGi++kgmYfNUVEehy2rqjFLTccma jn/u3cxMl/sHXxJ2S7+VuFP1K+ShXTgUFistjcqQ8kog3/RIdQswVhJ0F 6b9hfdDVXln6BLR5Fk0hRysmNgyUT73oVFdszF0JT7WNfaU0aKZC/Xk3q +1WFASaXG5sM88yXRrTftLR1Y/R4FAZ6lI7yqKYot++3ClGLtDKXKVtGb Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866702" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866702" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:51 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987191" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987191" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:50 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 9E67B1203D3; Fri, 30 Jun 2023 23:43:46 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 4/7] media: uapi: v4l: Document source routes Date: Fri, 30 Jun 2023 23:43:35 +0300 Message-Id: <20230630204338.126583-5-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Document how internal pads are used on source routes. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-subdev.rst | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/dev-subdev.rst b/Documentation/userspace-api/media/v4l/dev-subdev.rst index a4f1df7093e8..5a46c9a9d352 100644 --- a/Documentation/userspace-api/media/v4l/dev-subdev.rst +++ b/Documentation/userspace-api/media/v4l/dev-subdev.rst @@ -551,6 +551,26 @@ A stream at a specific point in the media pipeline is identified by the sub-device and a (pad, stream) pair. For sub-devices that do not support multiplexed streams the 'stream' field is always 0. +.. _v4l2-subdev-source-routes: + +Source routes +^^^^^^^^^^^^^ + +Cases where a single sub-device pad is a source of multiple streams are special +as there is no external sink pad for such a route. In those cases, the sources +of the streams are indicated by source routes that have an internal source pad +as the sink pad of such a route. Internal source pads have the +:ref:`MEDIA_PAD_FL_INTERNAL ` and ``MEDIA_PAD_FL_SINK`` +pad flags set. + +Internal source pads have all the properties of a sink pad in such case, +including formats and selections. The format in this case is the source format +of the stream. An internal pad always has a single stream only (0). + +Generally source routes are not modifiable but they can be activated and +deactivated using the :ref:`V4L2_SUBDEV_ROUTE_FL_ACTIVE +` flag, depending on driver capabilities. + Interaction between routes, streams, formats and selections ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From patchwork Fri Jun 30 20:43:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698228 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B78FEB64DD for ; Fri, 30 Jun 2023 20:43:57 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232095AbjF3Un4 (ORCPT ); Fri, 30 Jun 2023 16:43:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53372 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232628AbjF3Unz (ORCPT ); Fri, 30 Jun 2023 16:43:55 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67B3D3C02 for ; Fri, 30 Jun 2023 13:43:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157834; x=1719693834; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=B/UTzExT+AA5PHDSza/sBvtJLnsA/4Al/m8pE6Z6jdk=; b=Dv+jBTckvcdO0gXiIBXWEPTDMWMjS0jokstDXJ7xLoXugtbvYXJ5Pq5M Io1TyQW7iRoqSdtfHdyyViOOQkc8PZazyA1bUmB7k+i0Ja4CXjL3hyByF IwqcQbobV/ZS1UpTh8xT2qpxuN9NL0HrP5zqCKAamcgO2WYs3IqV0BwN1 NWlVd4gTG6actPbFYU+EXSlLmwwAOpca+VH7mV23HfK9vl8ttWH9bjQ6t 5PfCtrOLbcFRmSv4AvGBUUTlSRgRDWch8f8lzNQqZlxbg6jg3CjrkI9J1 3KjZBNBsqK34RNdW40rJRtOKbiEFwIxonvcqM2/EV98n4K8aZ4VVr8KbQ w==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866718" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866718" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:54 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987212" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987212" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:53 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id E621412075E; Fri, 30 Jun 2023 23:43:47 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 5/7] media: uapi: Add generic serial metadata mbus formats Date: Fri, 30 Jun 2023 23:43:36 +0300 Message-Id: <20230630204338.126583-6-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Add generic serial metadata mbus formats. These formats describe data width and packing but not the content itself. The reason for specifying such formats is that the formats as such are fairly device specific but they are still handled by CSI-2 receiver drivers that should not be aware of device specific formats. What makes generic metadata formats possible is that these formats are parsed by software only, after capturing the data to system memory. Signed-off-by: Sakari Ailus --- .../media/v4l/subdev-formats.rst | 257 ++++++++++++++++++ include/uapi/linux/media-bus-format.h | 9 + 2 files changed, 266 insertions(+) diff --git a/Documentation/userspace-api/media/v4l/subdev-formats.rst b/Documentation/userspace-api/media/v4l/subdev-formats.rst index a3a35eeed708..c615da08502d 100644 --- a/Documentation/userspace-api/media/v4l/subdev-formats.rst +++ b/Documentation/userspace-api/media/v4l/subdev-formats.rst @@ -8234,3 +8234,260 @@ The following table lists the existing metadata formats. both sides of the link and the bus format is a fixed metadata format that is not configurable from userspace. Width and height will be set to 0 for this format. + +Generic Serial Metadata Formats +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Generic serial metadata formats are used on serial busses where the actual data +content is more or less device specific but the data is transmitted and received +by multiple devices that do not process the data in any way, simply writing +it to system memory for processing in software at the end of the pipeline. + +The more specific variant describing the actual data is used on the internal +source pad of the originating sub-device. + +"b" in an array cell signifies a byte of data, followed by the number of byte +and finally the bit number in subscript. "p" indicates a padding bit. + +.. _media-bus-format-generic-meta: + +.. cssclass: longtable + +.. flat-table:: Generic Serial Metadata Formats + :header-rows: 2 + :stub-columns: 0 + + * - Identifier + - Code + - + - :cspan:`23` Data organization + * - + - + - Bit + - 23 + - 22 + - 21 + - 20 + - 19 + - 18 + - 17 + - 16 + - 15 + - 14 + - 13 + - 12 + - 11 + - 10 + - 9 + - 8 + - 7 + - 6 + - 5 + - 4 + - 3 + - 2 + - 1 + - 0 + * .. _MEDIA-BUS-FMT-META-8: + + - MEDIA_BUS_FMT_META_8 + - 0x8001 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + * .. _MEDIA-BUS-FMT-META-10: + + - MEDIA_BUS_FMT_META_10 + - 0x8002 + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + * .. _MEDIA-BUS-FMT-META-12: + + - MEDIA_BUS_FMT_META_12 + - 0x8003 + - + - + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-14: + + - MEDIA_BUS_FMT_META_14 + - 0x8004 + - + - + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-16: + + - MEDIA_BUS_FMT_META_16 + - 0x8005 + - + - + - + - + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-20: + + - MEDIA_BUS_FMT_META_20 + - 0x8007 + - + - + - + - + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + * .. _MEDIA-BUS-FMT-META-24: + + - MEDIA_BUS_FMT_META_24 + - 0x8009 + - + - b0\ :sub:`7` + - b0\ :sub:`6` + - b0\ :sub:`5` + - b0\ :sub:`4` + - b0\ :sub:`3` + - b0\ :sub:`2` + - b0\ :sub:`1` + - b0\ :sub:`0` + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p + - p diff --git a/include/uapi/linux/media-bus-format.h b/include/uapi/linux/media-bus-format.h index a03c543cb072..9ee031397372 100644 --- a/include/uapi/linux/media-bus-format.h +++ b/include/uapi/linux/media-bus-format.h @@ -173,4 +173,13 @@ */ #define MEDIA_BUS_FMT_METADATA_FIXED 0x7001 +/* Generic line based metadata formats for serial buses. Next is 0x8008. */ +#define MEDIA_BUS_FMT_META_8 0x8001 +#define MEDIA_BUS_FMT_META_10 0x8002 +#define MEDIA_BUS_FMT_META_12 0x8003 +#define MEDIA_BUS_FMT_META_14 0x8004 +#define MEDIA_BUS_FMT_META_16 0x8005 +#define MEDIA_BUS_FMT_META_20 0x8006 +#define MEDIA_BUS_FMT_META_24 0x8007 + #endif /* __LINUX_MEDIA_BUS_FORMAT_H */ From patchwork Fri Jun 30 20:43:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 931EFEB64DC for ; Fri, 30 Jun 2023 20:44:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232634AbjF3Un7 (ORCPT ); Fri, 30 Jun 2023 16:43:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53418 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232623AbjF3Un6 (ORCPT ); Fri, 30 Jun 2023 16:43:58 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0C1F03C0B for ; Fri, 30 Jun 2023 13:43:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157836; x=1719693836; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6req4PQTPGl6xKGwlf8PUjvyjR6Ca39HsmUDCvOt2Zs=; b=mOHeXV8KTcrAMlpwe/0Q1B141MsaFRsoawnRpLqI//i6jiuRw1DRqCxA ye16mIlIw2j3i98kPtgP3p8WdWmwHqcKs45f8sN/KxyETxFtx+Qa60OK/ 3ftWcipogyqXQ/JNIKNQ+P4fy9LozHeWpt6+TihCnCviuKvOPNYt8l1K7 3Z0NsOk/zOcBBoHhAc+D6QAUc32SnbDs2ojj92zDlLeQyQu7t7U9wogmC KwEZu6GDiLWIkRwVoe6N8rUmigEh7I0Q4MQvKjRPL3bIH0FfVz0OkcAju +ZxY5bdfvfUNwsen1ZNIZkH5oblKAui417p64g46dsjJgOJQOJH8akG0K g==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866726" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866726" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:55 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987225" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987225" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:54 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id D254812078C; Fri, 30 Jun 2023 23:43:49 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 6/7] media: uapi: Add generic 8-bit metadata format definitions Date: Fri, 30 Jun 2023 23:43:37 +0300 Message-Id: <20230630204338.126583-7-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Generic 8-bit metadata formats define the in-memory data layout but not the format of the data itself. The reasoning for having such formats is to allow CSI-2 receiver drivers to receive and DMA drivers to write the data to memory without knowing a large number of device specific formats. These formats may be used only in conjunction of a Media controller pipeline where the internal pad of the source sub-device defines the specific format of the data (using an mbus code). Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/meta-formats.rst | 1 + .../media/v4l/metafmt-generic.rst | 331 ++++++++++++++++++ drivers/media/v4l2-core/v4l2-ioctl.c | 8 + include/uapi/linux/videodev2.h | 9 + 4 files changed, 349 insertions(+) create mode 100644 Documentation/userspace-api/media/v4l/metafmt-generic.rst diff --git a/Documentation/userspace-api/media/v4l/meta-formats.rst b/Documentation/userspace-api/media/v4l/meta-formats.rst index 0bb61fc5bc00..919f595576b9 100644 --- a/Documentation/userspace-api/media/v4l/meta-formats.rst +++ b/Documentation/userspace-api/media/v4l/meta-formats.rst @@ -19,3 +19,4 @@ These formats are used for the :ref:`metadata` interface only. metafmt-vsp1-hgo metafmt-vsp1-hgt metafmt-vivid + metafmt-generic diff --git a/Documentation/userspace-api/media/v4l/metafmt-generic.rst b/Documentation/userspace-api/media/v4l/metafmt-generic.rst new file mode 100644 index 000000000000..a27bfc721edf --- /dev/null +++ b/Documentation/userspace-api/media/v4l/metafmt-generic.rst @@ -0,0 +1,331 @@ +.. SPDX-License-Identifier: GPL-2.0 OR GFDL-1.1-no-invariants-or-later + +************************************************************************************************************************************************************************************************************************************************************************************************************************** +V4L2_META_FMT_GENERIC_8 ('MET8'), V4L2_META_FMT_GENERIC_CSI2_10 ('MC1A'), V4L2_META_FMT_GENERIC_CSI2_12 ('MC1C'), V4L2_META_FMT_GENERIC_CSI2_14 ('MC1E'), V4L2_META_FMT_GENERIC_CSI2_16 ('MC1G'), V4L2_META_FMT_GENERIC_CSI2_20 ('MC1K'), V4L2_META_FMT_GENERIC_CSI2_24 ('MC1O'), V4L2_META_FMT_GENERIC_CSI2_2_24 ('MC2O') +************************************************************************************************************************************************************************************************************************************************************************************************************************** + + +Generic line-based metadata formats + + +Description +=========== + +These generic line-based metadata formats define the memory layout of the data +without defining the format or meaning of the metadata itself. These formats may +only be used with a Media controller pipeline where the more specific format is +defined in an :ref:`internal source pad ` of the source +sub-device. See also :ref:`source routes `. + +.. _v4l2-meta-fmt-generic-8: + +V4L2_META_FMT_GENERIC_8 +----------------------- + +The V4L2_META_FMT_GENERIC_8 format is a plain 8-bit metadata format. + +This format is also used on CSI-2 on both 8 bits per sample as well as on +16 bits per sample when two bytes of metadata are packed into one sample. + +**Byte Order Of V4L2_META_FMT_GENERIC_8.** +Each cell is one byte. "M" denotes a byte of metadata. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + * - start + 4: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + +.. _v4l2-meta-fmt-generic-csi2-10: + +V4L2_META_FMT_GENERIC_CSI2_10 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_10 contains packed 8-bit generic metadata, 10 bits +for each 8 bits of data. Every four bytes of metadata is followed by a single +byte of padding. The way the data is stored follows the CSI-2 specification. + +This format is also used on CSI-2 on 20 bits per sample format that packs two +bytes of metadata into one sample. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_10.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 5: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-12: + +V4L2_META_FMT_GENERIC_CSI2_12 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_12 contains packed 8-bit generic metadata, 12 bits +for each 8 bits of data. Every four bytes of metadata is followed by two bytes +of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_12.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-14: + +V4L2_META_FMT_GENERIC_CSI2_14 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_14 contains packed 8-bit generic metadata, 14 bits +for each 8 bits of data. Every four bytes of metadata is followed by three +bytes of padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_14.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - M\ :sub:`20` + - M\ :sub:`30` + - p + - p + - p + * - start + 7: + - M\ :sub:`01` + - M\ :sub:`11` + - M\ :sub:`21` + - M\ :sub:`31` + - p + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-16: + +V4L2_META_FMT_GENERIC_CSI2_16 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_16 contains packed 8-bit generic metadata, 16 bits +for each 8 bits of data. Every byte of metadata is followed by one byte of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_16.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + * - start + 8: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + +.. _v4l2-meta-fmt-generic-csi2-20: + +V4L2_META_FMT_GENERIC_CSI2_20 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_20 contains packed 8-bit generic metadata, 20 bits +for each 8 bits of data. Every byte of metadata is followed by alternating one +and two bytes of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_20.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - M\ :sub:`30` + - p + - p + * - start + 10: + - M\ :sub:`01` + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-24: + +V4L2_META_FMT_GENERIC_CSI2_24 +----------------------------- + +V4L2_META_FMT_GENERIC_CSI2_24 contains packed 8-bit generic metadata, 24 bits +for each 8 bits of data. Every byte of metadata is followed by two bytes of +padding. The way the data is stored follows the CSI-2 specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}|p{1.2cm}|p{.8cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - p + - p + - M\ :sub:`10` + - p + - p + - M\ :sub:`20` + - p + - p + - M\ :sub:`30` + - p + - p + * - start + 12: + - M\ :sub:`01` + - p + - p + - M\ :sub:`11` + - p + - p + - M\ :sub:`21` + - p + - p + - M\ :sub:`31` + - p + - p + +.. _v4l2-meta-fmt-generic-csi2-2-24: + +V4L2_META_FMT_GENERIC_CSI2_2_24 +------------------------------- + +V4L2_META_FMT_GENERIC_CSI2_2_24 contains packed 8-bit generic metadata, 24 bits +for each two times 8 bits of data. Every two bytes of metadata are followed by +one byte of padding. The way the data is stored follows the CSI-2 +specification. + +This format is little endian. + +**Byte Order Of V4L2_META_FMT_GENERIC_CSI2_2_24.** +Each cell is one byte. "M" denotes a byte of metadata and "p" a byte of padding. + +.. tabularcolumns:: |p{2.4cm}|p{1.2cm}|p{1.2cm}|p{.8cm}|p{1.2cm}|p{1.2cm}|p{.8cm}| + +.. flat-table:: + :header-rows: 0 + :stub-columns: 0 + :widths: 12 8 8 8 8 8 8 + + * - start + 0: + - M\ :sub:`00` + - M\ :sub:`10` + - p + - M\ :sub:`20` + - M\ :sub:`30` + - p + * - start + 6: + - M\ :sub:`01` + - M\ :sub:`11` + - p + - M\ :sub:`21` + - M\ :sub:`31` + - p + diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index a858acea6547..7781b0bd3e7d 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -1451,6 +1451,14 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt) case V4L2_PIX_FMT_Y210: descr = "10-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y212: descr = "12-bit YUYV Packed"; break; case V4L2_PIX_FMT_Y216: descr = "16-bit YUYV Packed"; break; + case V4L2_META_FMT_GENERIC_8: descr = "8-bit Generic Metadata"; break; + case V4L2_META_FMT_GENERIC_CSI2_10: descr = "8b Generic Meta, 10b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_12: descr = "8b Generic Meta, 12b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_14: descr = "8b Generic Meta, 14b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_16: descr = "8b Generic Meta, 16b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_20: descr = "8b Generic Meta, 20b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_24: descr = "8b Generic Meta, 24b CSI-2"; break; + case V4L2_META_FMT_GENERIC_CSI2_2_24: descr = "2x8b Generic Meta, 24b CSI-2"; break; default: /* Compressed formats */ diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 5d8bd754c69f..13d404699842 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -834,6 +834,15 @@ struct v4l2_pix_format { #define V4L2_META_FMT_RK_ISP1_PARAMS v4l2_fourcc('R', 'K', '1', 'P') /* Rockchip ISP1 3A Parameters */ #define V4L2_META_FMT_RK_ISP1_STAT_3A v4l2_fourcc('R', 'K', '1', 'S') /* Rockchip ISP1 3A Statistics */ +#define V4L2_META_FMT_GENERIC_8 v4l2_fourcc('M', 'E', 'T', '8') /* Generic 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_10 v4l2_fourcc('M', 'C', '1', 'A') /* 10-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_12 v4l2_fourcc('M', 'C', '1', 'C') /* 12-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_14 v4l2_fourcc('M', 'C', '1', 'E') /* 14-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_16 v4l2_fourcc('M', 'C', '1', 'G') /* 16-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_20 v4l2_fourcc('M', 'C', '1', 'K') /* 20-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_24 v4l2_fourcc('M', 'C', '1', 'O') /* 24-bit CSI-2 packed 8-bit metadata */ +#define V4L2_META_FMT_GENERIC_CSI2_2_24 v4l2_fourcc('M', 'C', '2', 'O') /* 2 bytes of 8-bit metadata, 24-bit CSI-2 packed */ + /* priv field value to indicates that subsequent fields are valid. */ #define V4L2_PIX_FMT_PRIV_MAGIC 0xfeedcafe From patchwork Fri Jun 30 20:43:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sakari Ailus X-Patchwork-Id: 698227 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46C37EB64DA for ; Fri, 30 Jun 2023 20:44:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232636AbjF3UoA (ORCPT ); Fri, 30 Jun 2023 16:44:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53420 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229919AbjF3Un6 (ORCPT ); Fri, 30 Jun 2023 16:43:58 -0400 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 39DD03C07 for ; Fri, 30 Jun 2023 13:43:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1688157837; x=1719693837; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4gMiUwZY0bZW9YmfgwN4PSttOidZn4uGQM8Cmq2XO8k=; b=VSmLqOY1Tn1POTsLIk1TWr2h3Aic3xFCoBDQj50LNwu7tVe03oI36TDH j+DW10dCWO2riKJiXH8146rXCNnmva4cFi2MeF19QuTGYrMBAs1ajkwsY f3VaJnBzlvq0RbVFbfADqCx5j4AL4EuXMXz35hSJl8HxcM7aQqLglQRSA aG6P+i78qEQLMzsVybaAabVEm5Qhu35zO990unmb+JNCFj0Cb/0WDL06e yqa28uURfc6awqbd8EsDfuuP+AaOMlbmpKeJ4z0TwDv7MplupjT2Zc5o/ cWK6xjv65QYeHJzaFm8kfpI3eY0nkAoKvMlg+gsXZ+BVWkukKg63vJ+bo w==; X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="448866733" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="448866733" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:56 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10757"; a="841987232" X-IronPort-AV: E=Sophos;i="6.01,171,1684825200"; d="scan'208";a="841987232" Received: from turnipsi.fi.intel.com (HELO kekkonen.fi.intel.com) ([10.237.72.44]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 30 Jun 2023 13:43:55 -0700 Received: from svinhufvud.ger.corp.intel.com (localhost [IPv6:::1]) by kekkonen.fi.intel.com (Postfix) with ESMTP id 66E9A1207EB; Fri, 30 Jun 2023 23:43:51 +0300 (EEST) From: Sakari Ailus To: linux-media@vger.kernel.org Cc: Laurent Pinchart , tomi.valkeinen@ideasonboard.com, bingbu.cao@intel.com, hongju.wang@intel.com, hverkuil@xs4all.nl Subject: [PATCH 7/7] media: v4l: Support line-based metadata capture Date: Fri, 30 Jun 2023 23:43:38 +0300 Message-Id: <20230630204338.126583-8-sakari.ailus@linux.intel.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230630204338.126583-1-sakari.ailus@linux.intel.com> References: <20230630204338.126583-1-sakari.ailus@linux.intel.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org many camera sensors, among other devices, transmit embedded data and image data for each CSI-2 frame. This embedded data typically contains register configuration of the sensor that has been used to capture the image data of the same frame. The embedded data is received by the CSI-2 receiver and has the same properties as the image data, including that it is line based: it has width, height and bytesperline (stride). Add these fields to struct v4l2_meta_format and document them. Also add V4L2_FMT_FLAG_META_LINE_BASED to tell a given format is line-based i.e. these fields of struct v4l2_meta_format are valid for it. Signed-off-by: Sakari Ailus --- .../userspace-api/media/v4l/dev-meta.rst | 15 +++++++++++++++ .../userspace-api/media/v4l/vidioc-enum-fmt.rst | 7 +++++++ .../media/videodev2.h.rst.exceptions | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 5 +++-- include/uapi/linux/videodev2.h | 10 ++++++++++ 5 files changed, 36 insertions(+), 2 deletions(-) diff --git a/Documentation/userspace-api/media/v4l/dev-meta.rst b/Documentation/userspace-api/media/v4l/dev-meta.rst index 0e7e1ee1471a..4b24bae6e171 100644 --- a/Documentation/userspace-api/media/v4l/dev-meta.rst +++ b/Documentation/userspace-api/media/v4l/dev-meta.rst @@ -65,3 +65,18 @@ to 0. - ``buffersize`` - Maximum buffer size in bytes required for data. The value is set by the driver. + * - __u32 + - ``width`` + - Width of a line of metadata in samples. Valid when :c:type`v4l2_fmtdesc` + flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``height`` + - Number of rows of metadata. Valid when :c:type`v4l2_fmtdesc` flag + ``V4L2_FMT_FLAG_META_LINE_BASED`` is set, otherwise zero. See + :c:func:`VIDIOC_ENUM_FMT`. + * - __u32 + - ``bytesperline`` + - Offset in bytes between the beginning of two consecutive lines. Valid + when :c:type`v4l2_fmtdesc` flag ``V4L2_FMT_FLAG_META_LINE_BASED`` is + set, otherwise zero. See :c:func:`VIDIOC_ENUM_FMT`. diff --git a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst index 000c154b0f98..6d7664345a4e 100644 --- a/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst +++ b/Documentation/userspace-api/media/v4l/vidioc-enum-fmt.rst @@ -227,6 +227,13 @@ the ``mbus_code`` field is handled differently: The application can ask to configure the quantization of the capture device when calling the :ref:`VIDIOC_S_FMT ` ioctl with :ref:`V4L2_PIX_FMT_FLAG_SET_CSC ` set. + * - ``V4L2_FMT_FLAG_META_LINE_BASED`` + - 0x0200 + - The metadata format is line-based. In this case the ``width``, + ``height`` and ``bytesperline`` fields of :c:type:`v4l2_meta_format` are + valid. The buffer consists of ``height`` lines, each having ``width`` + bytes of data and offset between the beginning of each two consecutive + lines is ``bytesperline``. Return Value ============ diff --git a/Documentation/userspace-api/media/videodev2.h.rst.exceptions b/Documentation/userspace-api/media/videodev2.h.rst.exceptions index 2a589d34b80e..5d623bf03bc5 100644 --- a/Documentation/userspace-api/media/videodev2.h.rst.exceptions +++ b/Documentation/userspace-api/media/videodev2.h.rst.exceptions @@ -211,6 +211,7 @@ replace define V4L2_FMT_FLAG_CSC_XFER_FUNC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_YCBCR_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_HSV_ENC fmtdesc-flags replace define V4L2_FMT_FLAG_CSC_QUANTIZATION fmtdesc-flags +replace define V4L2_FMT_FLAG_META_LINE_BASED fmtdesc-flags # V4L2 timecode types replace define V4L2_TC_TYPE_24FPS timecode-type diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 7781b0bd3e7d..2c7a7a6059cc 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -343,8 +343,9 @@ static void v4l_print_format(const void *arg, bool write_only) case V4L2_BUF_TYPE_META_OUTPUT: meta = &p->fmt.meta; pixelformat = meta->dataformat; - pr_cont(", dataformat=%p4cc, buffersize=%u\n", - &pixelformat, meta->buffersize); + pr_cont(", dataformat=%p4cc, buffersize=%u, width=%u, height=%u, bytesperline=%u\n", + &pixelformat, meta->buffersize, meta->width, + meta->height, meta->bytesperline); break; } } diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index 13d404699842..7485920f1ac9 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -873,6 +873,7 @@ struct v4l2_fmtdesc { #define V4L2_FMT_FLAG_CSC_YCBCR_ENC 0x0080 #define V4L2_FMT_FLAG_CSC_HSV_ENC V4L2_FMT_FLAG_CSC_YCBCR_ENC #define V4L2_FMT_FLAG_CSC_QUANTIZATION 0x0100 +#define V4L2_FMT_FLAG_META_LINE_BASED 0x0200 /* Frame Size and frame rate enumeration */ /* @@ -2407,10 +2408,19 @@ struct v4l2_sdr_format { * struct v4l2_meta_format - metadata format definition * @dataformat: little endian four character code (fourcc) * @buffersize: maximum size in bytes required for data + * @width: number of bytes of data per line (valid for line based + * formats only, see format documentation) + * @height: number of lines of data per buffer (valid for line based + * formats only) + * @bytesperline: offset between the beginnings of two adjacent lines in + * bytes (valid for line based formats only) */ struct v4l2_meta_format { __u32 dataformat; __u32 buffersize; + __u32 width; + __u32 height; + __u32 bytesperline; } __attribute__ ((packed)); /**