From patchwork Fri Sep 13 21:46:47 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 828761 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDA941553A1 for ; Fri, 13 Sep 2024 21:47:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264040; cv=none; b=qCzq3zpYZQDCkyYyjK7uWNEeT/v3jPMX53VmCN6aTJ/RoiC7DUBzrVnaHBXExTFpvfYGW6e3Mrxa2wS4dymCDM03Lt6uIUeUKPeuUo0jO+a6KRD7bsyamMgupf6NuKIq20O6jQM+8fy8v5qzE82Hx7i65fC8GlBkC09tlvYWgCE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264040; c=relaxed/simple; bh=H4YgJl+ixXKsioffdNbdZUQ1TkUacypExs2wltDE3OU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aKTD4aURglKMlGMRnd7XEQILTD23EIXzYU0ou8CS3wxGm6wnw7u3saqHQBh8qK9ij6FPqkHBKqTW0AgPBaVIljHXl3cGoRVL5lXlwH4vVjL5foQO0kzJY7Bb/mW9PpLT+oYor/quw/bIo9i6J5b1eBGJOVOyZNSCXw4gMTD1xSw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=jbrHOSjS; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="jbrHOSjS" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 06C781583; Fri, 13 Sep 2024 23:45:48 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263949; bh=H4YgJl+ixXKsioffdNbdZUQ1TkUacypExs2wltDE3OU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jbrHOSjSzzc/+CzcH0awOF7N0nPTEVHmjqQo0b5o0aK05xyyqZeOCGXmxKZgcOkQw KiGRerjE+uYzK8sWgKg0e/6TPWzwWDewMewlFjIeZGGz+eTq1Cnt8yaTnTBT8vPm4p ZUj3NdC2SFCe+v5ezbVSqiY9YKWtSHjzLt8iT9TQ= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 02/10] media: media-device: Introduce media device context Date: Fri, 13 Sep 2024 23:46:47 +0200 Message-ID: <20240913214657.1502838-3-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new type in the media-fh.h header that represent a media device context. A media device context is allocated when the media device is open and released when the last reference to it is put. A new pair of media_device_ops is added to allow device drivers to allocate and release a media context. The media context groups together the media entity contexts that are associated with it to form an isolated execution context. Provide helpers in mc-device.c for drivers and for the v4l2-core to handle media device contexts and to bind/unbind entity contexts to it. Once an entity context has been bound to a media device context it is possible to retrieve it by using a pointer to the entity the device is represented by. Signed-off-by: Jacopo Mondi --- drivers/media/mc/mc-device.c | 168 ++++++++++++++++++++++++++++ drivers/media/mc/mc-entity.c | 1 + include/media/media-device.h | 210 +++++++++++++++++++++++++++++++++++ include/media/media-fh.h | 4 + 4 files changed, 383 insertions(+) diff --git a/drivers/media/mc/mc-device.c b/drivers/media/mc/mc-device.c index 6616757640ec..1b24d5e74ffb 100644 --- a/drivers/media/mc/mc-device.c +++ b/drivers/media/mc/mc-device.c @@ -12,7 +12,9 @@ #include #include #include +#include #include +#include #include #include #include @@ -49,11 +51,31 @@ static int media_device_open(struct media_devnode *devnode, struct file *filp) { struct media_device *mdev = to_media_device(devnode); struct media_device_fh *fh; + int ret; fh = kzalloc(sizeof(*fh), GFP_KERNEL); if (!fh) return -ENOMEM; + if (mdev->ops && mdev->ops->alloc_context) { + if (WARN_ON(!mdev->ops->destroy_context)) { + kfree(fh); + return -EINVAL; + } + + ret = mdev->ops->alloc_context(mdev, &fh->context); + if (ret) { + kfree(fh); + return ret; + } + + /* + * Make sure the driver implementing alloc_context has + * called media_device_init_context + */ + WARN_ON(!fh->context->initialized); + } + fh->fh.ref = devnode->ref; filp->private_data = &fh->fh; @@ -78,6 +100,8 @@ static int media_device_close(struct file *filp) spin_unlock_irq(&mdev->fh_list_lock); } + media_device_context_put(fh->context); + kfree(fh); return 0; @@ -885,6 +909,150 @@ void media_device_unregister(struct media_device *mdev) } EXPORT_SYMBOL_GPL(media_device_unregister); +/* ----------------------------------------------------------------------------- + * Context handling + */ + +static void media_device_release_context(struct kref *refcount) +{ + struct media_device_context *context = + container_of(refcount, struct media_device_context, refcount); + + /* + * All the associated entity contexts should have been released if we + * get here. + */ + WARN_ON(!list_empty(&context->contexts)); + + context->mdev->ops->destroy_context(context); +} + +struct media_device_context * +media_device_context_get(struct media_device_context *ctx) +{ + if (!ctx) + return ERR_PTR(-EINVAL); + + kref_get(&ctx->refcount); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get); + +void media_device_context_put(struct media_device_context *ctx) +{ + if (!ctx) + return; + + kref_put(&ctx->refcount, media_device_release_context); +} +EXPORT_SYMBOL_GPL(media_device_context_put); + +struct media_device_context *media_device_context_get_from_fd(unsigned int fd) +{ + struct media_device_context *ctx; + struct file *filp = fget(fd); + struct media_device_fh *fh; + + if (!filp) + return NULL; + + fh = media_device_fh(filp); + ctx = media_device_context_get(fh->context); + fput(filp); + + return ctx; +} +EXPORT_SYMBOL_GPL(media_device_context_get_from_fd); + +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx) +{ + ctx->mdev = mdev; + INIT_LIST_HEAD(&ctx->contexts); + mutex_init(&ctx->lock); + kref_init(&ctx->refcount); + + ctx->initialized = true; + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_init_context); + +void media_device_cleanup_context(struct media_device_context *ctx) +{ + mutex_destroy(&ctx->lock); + list_del_init(&ctx->contexts); +} +EXPORT_SYMBOL_GPL(media_device_cleanup_context); + +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context) +{ + struct media_entity_context *entry; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + + /* Make sure the entity has not been bound already. */ + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry == context) + return -EINVAL; + } + + list_add_tail(&context->list, &mdev_context->contexts); + context->mdev_context = media_device_context_get(mdev_context); + + return 0; +} +EXPORT_SYMBOL_GPL(media_device_bind_context); + +int media_device_unbind_context(struct media_entity_context *context) +{ + struct media_device_context *mdev_context = context->mdev_context; + struct media_entity_context *entry; + struct media_entity_context *tmp; + + if (WARN_ON(!mdev_context || !context)) + return -EINVAL; + + guard(mutex)(&mdev_context->lock); + list_for_each_entry_safe(entry, tmp, &mdev_context->contexts, list) { + if (entry != context) + continue; + + list_del(&entry->list); + media_device_context_put(mdev_context); + entry->mdev_context = NULL; + + return 0; + } + + WARN(true, "Media entity context is not bound to any media context\n"); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(media_device_unbind_context); + +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity) +{ + struct media_entity_context *entry; + + guard(mutex)(&mdev_context->lock); + + list_for_each_entry(entry, &mdev_context->contexts, list) { + if (entry->entity == entity) + return media_entity_context_get(entry); + } + + return ERR_PTR(-EINVAL); +} +EXPORT_SYMBOL(media_device_get_entity_context); + #if IS_ENABLED(CONFIG_PCI) void media_device_pci_init(struct media_device *mdev, struct pci_dev *pci_dev, diff --git a/drivers/media/mc/mc-entity.c b/drivers/media/mc/mc-entity.c index 8ce72d72bc1d..4108d3da4cb0 100644 --- a/drivers/media/mc/mc-entity.c +++ b/drivers/media/mc/mc-entity.c @@ -1704,5 +1704,6 @@ EXPORT_SYMBOL_GPL(media_entity_init_context); void media_entity_cleanup_context(struct media_entity_context *ctx) { + media_device_unbind_context(ctx); } EXPORT_SYMBOL_GPL(media_entity_cleanup_context); diff --git a/include/media/media-device.h b/include/media/media-device.h index dd937552daf7..a73f650f122f 100644 --- a/include/media/media-device.h +++ b/include/media/media-device.h @@ -18,10 +18,72 @@ #include #include +#include struct ida; struct media_device; +/** + * struct media_device_context - Media device context + * @mdev: The media device this context is associated with + * @refcount: The kref reference counter + * @lock: Protects the entities contexts list + * @contexts: List of entity contexts associated with this media device context + * @initialized: Flag set to true by media_device_init_context() + * + * A media device context is created every time the media device gets opened by + * userspace. It is then uniquely identified for applications by the numerical + * file descriptor returned by a successful call to open() and is associated + * with an instance of :c:type:`struct media_device_fh`. + * + * Media device contexts are ref-counted and thus freed once the last reference + * to them is released. + * + * A media device context groups together the media entity contexts registered + * on a video device or v4l2 subdevice that has been associated with a media + * device context. The association between a media entity context and media + * device context is called 'bounding', and the result of bounding is to create + * an 'execution context' independent from other execution contexts. + * + * An entity context is bound to a media device context by a call to the + * VIDIOC_BIND_CONTEXT ioctl on video devices and by a call to + * VIDIOC_SUBDEV_BIND_CONTEXT on subdevices by userspace. The bounding operation + * groups together entity contexts to the same media device context. As video + * devices and v4l2 subdevices devnodes can be opened multiple times, each file + * descriptor resulting from a successful open() call can be bound to a + * different media device context. + * + * Creating execution contexts by bounding video entity contexts to a media + * device context allows userspace to effectively multiplex the usage of a + * media graph and of the device nodes that are part of it. + * + * In order to create an execution context userspace should: + * 1) Open the media device to create a media device context identified by the + * file descriptor returned by a successful 'open()' call + * 2) Open the video device or v4l2 subdevice and bind the file descriptors to + * the media device context by calling the VIDIOC_BIND_CONTEXT and + * VIDIOC_SUBDEV_BIND_CONTEXT ioctls + * + * All devices bound to the same media device context are now part of the same + * execution context. From this point on all the operations performed on a file + * descriptor bound to a media device context are independent from operations + * performed on a file descriptor bound to a different execution context. + * + * Binding an entity context to a media device context increases the media + * device context reference count. This guarantees that references to media + * device context are valid as long as there are valid entity contexts that + * refers to it. Symmetrically, unbinding an entity context from a media + * device context decreases the media device context reference count. + */ +struct media_device_context { + struct media_device *mdev; + struct kref refcount; + /* Protects the 'contexts' list */ + struct mutex lock; + struct list_head contexts; + bool initialized; +}; + /** * struct media_entity_notify - Media Entity Notify * @@ -63,6 +125,13 @@ struct media_entity_notify { * And once a buffer is queued, then the driver can complete * or delete objects from the request before req_queue exits. * @release: Release the resources of the media device. + * @alloc_context: Allocate a media device context. The operation allows drivers to + * allocate a driver-specific structure that embeds a + * media_device_context instance as first member where to store + * driver-specific information that are global to all device + * contexts part of media device context. Returns 0 on success a + * negative error code otherwise. + * @destroy_context: Release a media device context. */ struct media_device_ops { int (*link_notify)(struct media_link *link, u32 flags, @@ -72,6 +141,9 @@ struct media_device_ops { int (*req_validate)(struct media_request *req); void (*req_queue)(struct media_request *req); void (*release)(struct media_device *mdev); + int (*alloc_context)(struct media_device *mdev, + struct media_device_context **ctx); + void (*destroy_context)(struct media_device_context *ctx); }; /** @@ -331,6 +403,144 @@ int __must_check __media_device_register(struct media_device *mdev, */ void media_device_unregister(struct media_device *mdev); +/* ----------------------------------------------------------------------------- + * media device context handling + */ + +/** + * media_device_context_get - Increase the media device context reference count + * and return a reference to it + * @ctx: The media device context + */ +struct media_device_context * +media_device_context_get(struct media_device_context *ctx); + +/** + * media_device_context_put - Decrease the media device context reference count + * @ctx: The media device context + */ +void media_device_context_put(struct media_device_context *ctx); + +/** + * media_device_context_get_from_fd - Get the media device context associated with a + * numerical file descriptor + * + * @fd: the numerical file descriptor + * + * A media device context is created whenever the media device devnode is opened + * by userspace. It is then associated uniquely with a numerical file descriptor + * which is unique in the userspace process context. + * + * This function allows to retrieve the media device associated with such + * numerical file descriptor and increases the media device context reference + * count to guarantee the returned reference stays valid at least until the + * caller does not call media_device_context_put(). + * + * Caller of this function are required to put the returned media device context + * once they are done with it. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * which need to get the media device contexts associated to a numerical file + * descriptor. + */ +struct media_device_context *media_device_context_get_from_fd(unsigned int fd); + +/** + * media_device_init_context - Initialize the media device context + * + * @mdev: The media device this context belongs to + * @ctx: The media device context to initialize + * + * Initialize the fields of a media device context. Device drivers that support + * multi context operations shall call this function in their implementation of + * media_device_operations.alloc_context() + */ +int media_device_init_context(struct media_device *mdev, + struct media_device_context *ctx); + +/** + * media_device_cleanup_context - Cleanup the media device context + * + * @ctx: The media device context to clean up + * + * Cleanup a media device context. Device drivers that support multi context + * operations shall call this function in their implementation of + * media_device_operations.destroy_context() before releasing the memory allocated + * by media_device_operations.alloc_context(). + */ +void media_device_cleanup_context(struct media_device_context *ctx); + +/** + * media_device_bind_context - Bind an entity context to a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @context: the entity context to bind + * + * This function creates a mapping entry in the media device context that + * associates an entity context to the media entity it belongs to and stores it + * in a linked list so that they can be retrieved later. + * + * Binding an entity context to a media device context increases the media + * device context refcount. + * + * The intended caller of this function is the VIDIOC_BIND_CONTEXT ioctl handler + * that binds a newly created context to a media device context. + */ +int media_device_bind_context(struct media_device_context *mdev_context, + struct media_entity_context *context); + +/** + * media_device_unbind_context - Unbind an entity context from a media device + * context + * + * @context: the entity context to unbind + * + * An entity context is unbound from a media device context when the file handle + * it is associated with gets closed. + * + * Unbinding an entity context from a media device context decreases the media + * device context refcount. + * + * Returns 0 if the context was bound to a media device context, -EINVAL + * otherwise. + */ +int media_device_unbind_context(struct media_entity_context *context); + +/** + * media_device_get_entity_context - Get the entity context associated with + * a media entity in a media device context + * + * @mdev_context: pointer to struct &media_device_context + * @entity: pointer to struct &media_entity that the entity context is + * associated with + * + * An entity context is uniquely associated with a media device context after it + * has been bound to it by a call to the VIDIOC_BIND_CONTEXT ioctl. This helper + * function retrieves the entity context associated with a media device context + * for a specific entity that represents a video device or a v4l2 subdevice. + * + * The reference count of the returned entity context is increased to guarantee + * the returned reference stays valid until the caller does not call + * media_entity_context_put(). + * + * Drivers are not expected to call this function directly but should instead + * use the helpers provided by the video_device and v4l2_subdevice layers, + * video_device_context_get() and v4l2_subdev_get_context() respectively. + * Drivers are always required to decrease the returned context reference count + * by calling video_device_context_put() and v4l2_subdev_put_context(). + * + * If no entity context has been associated with the media device context + * provided as first argument an error pointer is returned. Drivers are + * required to always check the value returned by this function. + */ +struct media_entity_context * +media_device_get_entity_context(struct media_device_context *mdev_context, + struct media_entity *entity); + +/*------------------------------------------------------------------------------ + * Media entity handling + */ + /** * media_device_register_entity() - registers a media entity inside a * previously registered media device. diff --git a/include/media/media-fh.h b/include/media/media-fh.h index 6f00744b81d6..70331d3e3470 100644 --- a/include/media/media-fh.h +++ b/include/media/media-fh.h @@ -13,6 +13,8 @@ #include +struct media_device_context; + /** * struct media_device_fh - File handle specific information on MC * @@ -22,6 +24,8 @@ struct media_device_fh { struct media_devnode_fh fh; struct list_head mdev_list; + + struct media_device_context *context; }; static inline struct media_device_fh *media_device_fh(struct file *filp) From patchwork Fri Sep 13 21:46:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 828760 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C95346BFD4 for ; Fri, 13 Sep 2024 21:47:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; cv=none; b=aA+BZM02bHNGQVnBacU9Uab+HfGP8W/2V4053+giL4vpMPBKAng69s164aIyuYYHtaUWKU3UNt4JKJcVOrQf/rkuVgi6cWiapJInAzqqAXcEjkLf0RS8/Nad/iDo9Itg0u1gykH9xr1dZo25tQv/EHr7nL8Gh7dqO4OnT1eNrks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264044; c=relaxed/simple; bh=C/tLRufzYECRe+GhTy0TGewSXXIqGyGCVTzqUTapgGA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=J2p/zbGk6Zv7M1uWK23I4wpo4qtjWlTEQyKJgfH8TlDpnOcepY1SPdZiJ8GflqdG3q03Oeir9la4v6CqKlDEmVzLDwd4oCqX2mhdYRahs/d+OEV+K6UXD6VHvc6ejBT4JGnYRYqqwjZHWxv7eHQYrvK+9MlEEmdPGFW3UoZULhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=PZIymVBY; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="PZIymVBY" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 989841961; Fri, 13 Sep 2024 23:45:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263949; bh=C/tLRufzYECRe+GhTy0TGewSXXIqGyGCVTzqUTapgGA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=PZIymVBY0tIA3OpULT+ZeT9fear0PwP0kLVBu5GrxIv9zKNPHqGPYXuUqdJEqN1t4 wT9dP0Z1Mgg1pk3iK7Gn61PPA/dttUv9BfZQLWlSihWyhnlfWmXn/2mgP7uOYouwJb rE1JqqZVD/ld3KGInnCUiEDtEgHKWJ8gzpgtn/Vk= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 04/10] media: v4l2-ioctl: Introduce VIDIOC_BIND_CONTEXT Date: Fri, 13 Sep 2024 23:46:49 +0200 Message-ID: <20240913214657.1502838-5-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce a new ioctl in V4L2 to allocate a video device context and associate it with a media device context. The ioctl is valid only if support for MEDIA_CONTROLLER is compiled in as it calls into the entity ops to let driver allocate a new context and binds the newly created context with the media context associated with the file descriptor provided by userspace as the new ioctl argument. The newly allocated video context is then stored in the v4l2-fh that represent the open file handle on which the ioctl has been called. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 10 +++++ drivers/media/v4l2-core/v4l2-fh.c | 1 + drivers/media/v4l2-core/v4l2-ioctl.c | 64 ++++++++++++++++++++++++++++ include/media/v4l2-ioctl.h | 5 +++ include/uapi/linux/videodev2.h | 11 +++++ 5 files changed, 91 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 0fd6ff3f00f6..41719b009c1e 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -628,6 +628,10 @@ static void determine_valid_ioctls(struct video_device *vdev) __set_bit(_IOC_NR(VIDIOC_ENUM_FREQ_BANDS), valid_ioctls); if (is_vid) { +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif + /* video specific ioctls */ if ((is_rx && (ops->vidioc_enum_fmt_vid_cap || ops->vidioc_enum_fmt_vid_overlay)) || @@ -681,12 +685,18 @@ static void determine_valid_ioctls(struct video_device *vdev) SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_cap); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_cap); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } else if (is_meta && is_tx) { /* metadata output specific ioctls */ SET_VALID_IOCTL(ops, VIDIOC_ENUM_FMT, vidioc_enum_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_G_FMT, vidioc_g_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_S_FMT, vidioc_s_fmt_meta_out); SET_VALID_IOCTL(ops, VIDIOC_TRY_FMT, vidioc_try_fmt_meta_out); +#ifdef CONFIG_MEDIA_CONTROLLER + __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls); +#endif } if (is_vbi) { /* vbi specific ioctls */ diff --git a/drivers/media/v4l2-core/v4l2-fh.c b/drivers/media/v4l2-core/v4l2-fh.c index 90eec79ee995..f7af444d2344 100644 --- a/drivers/media/v4l2-core/v4l2-fh.c +++ b/drivers/media/v4l2-core/v4l2-fh.c @@ -93,6 +93,7 @@ int v4l2_fh_release(struct file *filp) struct v4l2_fh *fh = filp->private_data; if (fh) { + video_device_context_put(fh->context); v4l2_fh_del(fh); v4l2_fh_exit(fh); kfree(fh); diff --git a/drivers/media/v4l2-core/v4l2-ioctl.c b/drivers/media/v4l2-core/v4l2-ioctl.c index 4119b23bb954..0e37e777d21f 100644 --- a/drivers/media/v4l2-core/v4l2-ioctl.c +++ b/drivers/media/v4l2-core/v4l2-ioctl.c @@ -9,6 +9,7 @@ */ #include +#include #include #include #include @@ -349,6 +350,13 @@ static void v4l_print_format(const void *arg, bool write_only) } } +static void v4l_print_context(const void *arg, bool write_only) +{ + const struct v4l2_context *c = arg; + + pr_cont("context=%u\n", c->context_fd); +} + static void v4l_print_framebuffer(const void *arg, bool write_only) { const struct v4l2_framebuffer *p = arg; @@ -2110,6 +2118,61 @@ static int v4l_overlay(const struct v4l2_ioctl_ops *ops, return ops->vidioc_overlay(file, fh, *(unsigned int *)arg); } +static int v4l_bind_context(const struct v4l2_ioctl_ops *ops, + struct file *file, void *fh, void *arg) +{ + struct video_device *vdev = video_devdata(file); + struct media_device_context *mdev_context; + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vdev->flags) ? fh : NULL; + struct v4l2_context *c = arg; + int ret; + + /* + * TODO: do not __set_bit(_IOC_NR(VIDIOC_BIND_CONTEXT), valid_ioctls) + * if V4L2_FL_USES_V4L2_FH isn't set or the driver does not implement + * alloc_context and destroy_context. + */ + if (!vfh) + return -ENOTTY; + + if (!vdev->entity.ops || !vdev->entity.ops->alloc_context || + !vdev->entity.ops->destroy_context) + return -ENOTTY; + + mdev_context = media_device_context_get_from_fd(c->context_fd); + if (!mdev_context) + return -EINVAL; + + /* Let the driver allocate the per-file handle context. */ + ret = vdev->entity.ops->alloc_context(&vdev->entity, + (struct media_entity_context **) + &vfh->context); + if (ret) + goto err_put_mdev_context; + + /* + * Bind the newly created video device context to the media device + * context identified by the file descriptor. + */ + ret = media_device_bind_context(mdev_context, + (struct media_entity_context *) + vfh->context); + if (ret) + goto err_put_context; + + media_device_context_put(mdev_context); + + return 0; + +err_put_context: + video_device_context_put(vfh->context); +err_put_mdev_context: + media_device_context_put(mdev_context); + + return ret; +} + static int v4l_reqbufs(const struct v4l2_ioctl_ops *ops, struct file *file, void *fh, void *arg) { @@ -2853,6 +2916,7 @@ static const struct v4l2_ioctl_info v4l2_ioctls[] = { IOCTL_INFO(VIDIOC_ENUM_FMT, v4l_enum_fmt, v4l_print_fmtdesc, 0), IOCTL_INFO(VIDIOC_G_FMT, v4l_g_fmt, v4l_print_format, 0), IOCTL_INFO(VIDIOC_S_FMT, v4l_s_fmt, v4l_print_format, INFO_FL_PRIO), + IOCTL_INFO(VIDIOC_BIND_CONTEXT, v4l_bind_context, v4l_print_context, 0), IOCTL_INFO(VIDIOC_REQBUFS, v4l_reqbufs, v4l_print_requestbuffers, INFO_FL_PRIO | INFO_FL_QUEUE), IOCTL_INFO(VIDIOC_QUERYBUF, v4l_querybuf, v4l_print_buffer, INFO_FL_QUEUE | INFO_FL_CLEAR(v4l2_buffer, length)), IOCTL_INFO(VIDIOC_G_FBUF, v4l_stub_g_fbuf, v4l_print_framebuffer, 0), diff --git a/include/media/v4l2-ioctl.h b/include/media/v4l2-ioctl.h index edb733f21604..a70d2e446351 100644 --- a/include/media/v4l2-ioctl.h +++ b/include/media/v4l2-ioctl.h @@ -18,6 +18,7 @@ #include struct v4l2_fh; +struct video_device_context; /** * struct v4l2_ioctl_ops - describe operations for each V4L2 ioctl @@ -406,6 +407,10 @@ struct v4l2_ioctl_ops { int (*vidioc_try_fmt_meta_out)(struct file *file, void *fh, struct v4l2_format *f); + /* Context handlers */ + int (*vidioc_bind_context)(struct file *file, void *fh, + struct video_device_context *c); + /* Buffer handlers */ int (*vidioc_reqbufs)(struct file *file, void *fh, struct v4l2_requestbuffers *b); diff --git a/include/uapi/linux/videodev2.h b/include/uapi/linux/videodev2.h index f8157c9c9da6..0b7f12fd798e 100644 --- a/include/uapi/linux/videodev2.h +++ b/include/uapi/linux/videodev2.h @@ -1035,6 +1035,14 @@ struct v4l2_jpegcompression { * always use APP0 */ }; +/* + * V I D E O D E V I C E C O N T E X T + */ + +struct v4l2_context { + __u32 context_fd; +}; + /* * M E M O R Y - M A P P I N G B U F F E R S */ @@ -2758,6 +2766,9 @@ struct v4l2_create_buffers { #define VIDIOC_QUERY_EXT_CTRL _IOWR('V', 103, struct v4l2_query_ext_ctrl) +/* Context handling */ +#define VIDIOC_BIND_CONTEXT _IOW('V', 104, struct v4l2_context) + /* Reminder: when adding new ioctls please add support for them to drivers/media/v4l2-core/v4l2-compat-ioctl32.c as well! */ From patchwork Fri Sep 13 21:46:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 828759 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 327EA1527AC for ; Fri, 13 Sep 2024 21:47:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; cv=none; b=AV+8+oFHDt8bJU08TlUn1LKUshWigEfsHDodhxSA1zGOLKDqq7jugtdVIrmzv9NzhyHknkd118YYNwJd59vaYfexdHLUYDSZ/x8cej1CIlPftEpjlZZ7wDLv/gcJrebCz4lvdpujGvzcbWz832UgJG7MlHR6ecELzjjhEewn3L4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264047; c=relaxed/simple; bh=I9XkynIK6QYsUmsKZc85wfx8aP+41+aqdO1VT7BsRyg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YNUG8ERnfRXZfBkevA2cVoWAMyc173znPRKc+iisnCJIJvQkEch7ocff+s3aRdNQKIs5pHAN4sb1/v5IhdV8cmWAplw2Yt5z0fTxlzrPa3RZRhe0KYR1OnCc3m6cU0WBxA8wJbfzST4GY1AFc4umpTeiREJRjyAaT0hutOaj2i0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=UzsxQsWp; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="UzsxQsWp" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 320311A6F; Fri, 13 Sep 2024 23:45:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=I9XkynIK6QYsUmsKZc85wfx8aP+41+aqdO1VT7BsRyg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=UzsxQsWpn/b3CDiEBYJkcPTwqgaAALbGpbP0BQQfnobwEROqwyWIPf7cnyBUcXo4f 2BoUl4aQLoF9BleLz8n5rxHPcZD0tNUoNy5l0aZB7zGaAv3fi5sJRPr4KGEqC0+f4X VQ028sNS/GUCb86JnCDW2T1vd2al7FNgXffs2Wfk= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 06/10] media: v4l2-dev: Add video_device_context_from_file() Date: Fri, 13 Sep 2024 23:46:51 +0200 Message-ID: <20240913214657.1502838-7-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Introduce an helper function to get a video_device_context from file. If no context has been bound to the file, return the video device default context. As a video device context lifetime is bound to the one of the file handle it is associated with, and the intended user of this helper are the v4l2_ioctl_ops handlers that operates on an open file handle, the context returned by this function is guaranteed to be valid for the whole function scope of the caller. Signed-off-by: Jacopo Mondi --- drivers/media/v4l2-core/v4l2-dev.c | 15 +++++++++++++++ include/media/v4l2-dev.h | 30 ++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/drivers/media/v4l2-core/v4l2-dev.c b/drivers/media/v4l2-core/v4l2-dev.c index 73462191fd17..d92989d76cf0 100644 --- a/drivers/media/v4l2-core/v4l2-dev.c +++ b/drivers/media/v4l2-core/v4l2-dev.c @@ -1181,6 +1181,21 @@ void video_unregister_device(struct video_device *vdev) } EXPORT_SYMBOL(video_unregister_device); +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vfd) +{ + struct v4l2_fh *vfh = + test_bit(V4L2_FL_USES_V4L2_FH, &vfd->flags) ? filp->private_data + : NULL; + + /* If the file handle has been bound to a context return it. */ + if (vfh && vfh->context) + return vfh->context; + + return vfd->default_context; +} +EXPORT_SYMBOL_GPL(video_device_context_from_file); + #if defined(CONFIG_MEDIA_CONTROLLER) __must_check int video_device_pipeline_start(struct video_device *vdev, diff --git a/include/media/v4l2-dev.h b/include/media/v4l2-dev.h index b79b14e36a2a..32e61f9f9cb5 100644 --- a/include/media/v4l2-dev.h +++ b/include/media/v4l2-dev.h @@ -766,6 +766,36 @@ int video_device_init_context(struct video_device *vdev, */ void video_device_cleanup_context(struct video_device_context *ctx); +/** + * video_device_context_from_file - Get the video device context associated with + * an open file handle + * @filp: The file handle + * @vdev: The video device + * + * If a video device context has been bound to an open file handle by a call + * to the VIDIOC_BIND_CONTEXT ioctl this function returns it, otherwise returns + * the default video device context. + * + * The intended callers of this helper are the driver-specific v4l2_ioctl_ops + * handlers, which receive as first argument a file pointer. By using this + * helper drivers can get the context associated with such file, if any. + * Otherwise, if the video device has not been bound to any media device + * context, the default video device context is returned. + * + * As video device contexts are reference counted and their lifetime is + * guaranteed to be at least the one of the file handle they are associated + * with, callers of this function are guaranteed to always receive a valid + * context reference by this function. The reference will remain valid for the + * whole function scope of the caller that has received the open file handle as + * argument. Likewise, accessing the media context from the video device context + * returned by this function is always safe within the same function scope. + * + * This function does not increase the reference count of the returned video + * device context. + */ +struct video_device_context * +video_device_context_from_file(struct file *filp, struct video_device *vdev); + #endif /* CONFIG_MEDIA_CONTROLLER */ #endif /* _V4L2_DEV_H */ From patchwork Fri Sep 13 21:46:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 828758 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50E6A1494DC for ; Fri, 13 Sep 2024 21:47:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; cv=none; b=niSgTuNjQAimzTRvqQ/ocS+38nSu7FA71xmuqJeyFUuoOT+RxY3VDwMdeekj8WvpFm88GuplTYoJRD/KXdb+TXxjyhaLmdHYt4SrujKewVUupmgNMihCDaRDrVQ2fGLLCPXdCbrMNb9qFy05Ec61IEmdaKS0RN5bxBVMumcaIww= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264051; c=relaxed/simple; bh=JkL1mUCZoVEqCrviXrBtMAXA0r6wMGrZjDYOteOnxVk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F9OCdJIT+HgRTvHgbjt58yOwEtAmtgDNWafsbCZupwSw2xMHxqOGZbUHMZ/V0WwB7lWnsSbzLwe2Fyq2d73sYCIn2hlyZ3XruMv6iLNeNIg9QQX6ploF8F7Me6l8lanqPrbWmTGh2aTlSQdt3PT6oDPl0d4NntI8sQJdw0rnRgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=OYTnzKUZ; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="OYTnzKUZ" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id BC7862590; Fri, 13 Sep 2024 23:45:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263950; bh=JkL1mUCZoVEqCrviXrBtMAXA0r6wMGrZjDYOteOnxVk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OYTnzKUZSYI7BxfCrL9jdhEHkJs0xV0gCAjWrCLdxZsHdKhN7nBdlv1QAEo22Dg1I IoT1teSLOgzNuzMcqp3YvUAeSjyZEwj7BOH7UCN3I9ExvPjfWECqSKw9ElX0c6gx2o dulyIgWliZFH4feMuqQv+D69M4WhCMKZLF0rZGKE= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 08/10] videobuf2-v4l2: Support vb2_queue embedded in a context Date: Fri, 13 Sep 2024 23:46:53 +0200 Message-ID: <20240913214657.1502838-9-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Support device drivers that implement multi-context operations in the videobuf2 core by providing an helper to retrieve the vb2_queue from the context associated with an open file handle. If no context is associated with a file handle, retrieve it from the video device default context, created by the core for multi-context aware drivers. Fall-back to use the vb2_queue from the video_device to support existing drivers which are not context aware. Signed-off-by: Jacopo Mondi --- .../media/common/videobuf2/videobuf2-v4l2.c | 129 +++++++++++------- 1 file changed, 81 insertions(+), 48 deletions(-) diff --git a/drivers/media/common/videobuf2/videobuf2-v4l2.c b/drivers/media/common/videobuf2/videobuf2-v4l2.c index c7a54d82a55e..a221436718c2 100644 --- a/drivers/media/common/videobuf2/videobuf2-v4l2.c +++ b/drivers/media/common/videobuf2/videobuf2-v4l2.c @@ -976,27 +976,46 @@ EXPORT_SYMBOL_GPL(vb2_poll); * and so they simplify the driver code. */ +/* + * Helper to get the vb2 queue either from: + * 1) The video context bound to the open file handle + * 2) The default context for context-aware drivers if userspace has not bound + * a context to the file handle + * 3) From the video device for non-context aware drivers + */ +static struct vb2_queue *get_vb2_queue(struct file *file, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_from_file(file, vdev); + + return ctx ? &ctx->queue + : vdev->default_context ? &vdev->default_context->queue + : vdev->queue; +} + /* vb2 ioctl helpers */ int vb2_ioctl_reqbufs(struct file *file, void *priv, struct v4l2_requestbuffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, p->type); + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->type); u32 flags = p->flags; - fill_buf_caps(vdev->queue, &p->capabilities); - validate_memory_flags(vdev->queue, p->memory, &flags); + fill_buf_caps(q, &p->capabilities); + validate_memory_flags(q, p->memory, &flags); p->flags = flags; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_core_reqbufs(vdev->queue, p->memory, p->flags, &p->count); + res = vb2_core_reqbufs(q, p->memory, p->flags, &p->count); /* If count == 0, then the owner has released all buffers and he is no longer owner of the queue. Otherwise we have a new owner. */ if (res == 0) - vdev->queue->owner = p->count ? file->private_data : NULL; + q->owner = p->count ? file->private_data : NULL; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_reqbufs); @@ -1005,12 +1024,13 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, struct v4l2_create_buffers *p) { struct video_device *vdev = video_devdata(file); - int res = vb2_verify_memory_type(vdev->queue, p->memory, + struct vb2_queue *q = get_vb2_queue(file, vdev); + int res = vb2_verify_memory_type(q, p->memory, p->format.type); - p->index = vdev->queue->num_buffers; - fill_buf_caps(vdev->queue, &p->capabilities); - validate_memory_flags(vdev->queue, p->memory, &p->flags); + p->index = q->num_buffers; + fill_buf_caps(q, &p->capabilities); + validate_memory_flags(q, p->memory, &p->flags); /* * If count == 0, then just check if memory and type are valid. * Any -EBUSY result from vb2_verify_memory_type can be mapped to 0. @@ -1019,12 +1039,12 @@ int vb2_ioctl_create_bufs(struct file *file, void *priv, return res != -EBUSY ? res : 0; if (res) return res; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - res = vb2_create_bufs(vdev->queue, p); + res = vb2_create_bufs(q, p); if (res == 0) - vdev->queue->owner = file->private_data; + q->owner = file->private_data; return res; } EXPORT_SYMBOL_GPL(vb2_ioctl_create_bufs); @@ -1033,69 +1053,76 @@ int vb2_ioctl_prepare_buf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_prepare_buf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_prepare_buf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_prepare_buf); int vb2_ioctl_querybuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); /* No need to call vb2_queue_is_busy(), anyone can query buffers. */ - return vb2_querybuf(vdev->queue, p); + return vb2_querybuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_querybuf); int vb2_ioctl_qbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_qbuf(vdev->queue, vdev->v4l2_dev->mdev, p); + return vb2_qbuf(q, vdev->v4l2_dev->mdev, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_qbuf); int vb2_ioctl_dqbuf(struct file *file, void *priv, struct v4l2_buffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_dqbuf(vdev->queue, p, file->f_flags & O_NONBLOCK); + return vb2_dqbuf(q, p, file->f_flags & O_NONBLOCK); } EXPORT_SYMBOL_GPL(vb2_ioctl_dqbuf); int vb2_ioctl_streamon(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamon(vdev->queue, i); + return vb2_streamon(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamon); int vb2_ioctl_streamoff(struct file *file, void *priv, enum v4l2_buf_type i) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_streamoff(vdev->queue, i); + return vb2_streamoff(q, i); } EXPORT_SYMBOL_GPL(vb2_ioctl_streamoff); int vb2_ioctl_expbuf(struct file *file, void *priv, struct v4l2_exportbuffer *p) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) return -EBUSY; - return vb2_expbuf(vdev->queue, p); + return vb2_expbuf(q, p); } EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); @@ -1104,20 +1131,22 @@ EXPORT_SYMBOL_GPL(vb2_ioctl_expbuf); int vb2_fop_mmap(struct file *file, struct vm_area_struct *vma) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_mmap(vdev->queue, vma); + return vb2_mmap(q, vma); } EXPORT_SYMBOL_GPL(vb2_fop_mmap); int _vb2_fop_release(struct file *file, struct mutex *lock) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); if (lock) mutex_lock(lock); - if (file->private_data == vdev->queue->owner) { - vb2_queue_release(vdev->queue); - vdev->queue->owner = NULL; + if (file->private_data == q->owner) { + vb2_queue_release(q); + q->owner = NULL; } if (lock) mutex_unlock(lock); @@ -1128,7 +1157,8 @@ EXPORT_SYMBOL_GPL(_vb2_fop_release); int vb2_fop_release(struct file *file) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; return _vb2_fop_release(file, lock); } @@ -1138,19 +1168,20 @@ ssize_t vb2_fop_write(struct file *file, const char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_WRITE)) + if (!(q->io_modes & VB2_WRITE)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - err = vb2_write(vdev->queue, buf, count, ppos, - file->f_flags & O_NONBLOCK); - if (vdev->queue->fileio) - vdev->queue->owner = file->private_data; + err = vb2_write(q, buf, count, ppos, + file->f_flags & O_NONBLOCK); + if (q->fileio) + q->owner = file->private_data; exit: if (lock) mutex_unlock(lock); @@ -1162,20 +1193,21 @@ ssize_t vb2_fop_read(struct file *file, char __user *buf, size_t count, loff_t *ppos) { struct video_device *vdev = video_devdata(file); - struct mutex *lock = vdev->queue->lock ? vdev->queue->lock : vdev->lock; + struct vb2_queue *q = get_vb2_queue(file, vdev); + struct mutex *lock = q->lock ? q->lock : vdev->lock; int err = -EBUSY; - if (!(vdev->queue->io_modes & VB2_READ)) + if (!(q->io_modes & VB2_READ)) return -EINVAL; if (lock && mutex_lock_interruptible(lock)) return -ERESTARTSYS; - if (vb2_queue_is_busy(vdev->queue, file)) + if (vb2_queue_is_busy(q, file)) goto exit; - vdev->queue->owner = file->private_data; - err = vb2_read(vdev->queue, buf, count, ppos, + q->owner = file->private_data; + err = vb2_read(q, buf, count, ppos, file->f_flags & O_NONBLOCK); - if (!vdev->queue->fileio) - vdev->queue->owner = NULL; + if (!q->fileio) + q->owner = NULL; exit: if (lock) mutex_unlock(lock); @@ -1186,7 +1218,7 @@ EXPORT_SYMBOL_GPL(vb2_fop_read); __poll_t vb2_fop_poll(struct file *file, poll_table *wait) { struct video_device *vdev = video_devdata(file); - struct vb2_queue *q = vdev->queue; + struct vb2_queue *q = get_vb2_queue(file, vdev); struct mutex *lock = q->lock ? q->lock : vdev->lock; __poll_t res; void *fileio; @@ -1202,7 +1234,7 @@ __poll_t vb2_fop_poll(struct file *file, poll_table *wait) fileio = q->fileio; - res = vb2_poll(vdev->queue, file, wait); + res = vb2_poll(q, file, wait); /* If fileio was started, then we have a new queue owner. */ if (!fileio && q->fileio) @@ -1218,8 +1250,9 @@ unsigned long vb2_fop_get_unmapped_area(struct file *file, unsigned long addr, unsigned long len, unsigned long pgoff, unsigned long flags) { struct video_device *vdev = video_devdata(file); + struct vb2_queue *q = get_vb2_queue(file, vdev); - return vb2_get_unmapped_area(vdev->queue, addr, len, pgoff, flags); + return vb2_get_unmapped_area(q, addr, len, pgoff, flags); } EXPORT_SYMBOL_GPL(vb2_fop_get_unmapped_area); #endif From patchwork Fri Sep 13 21:46:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jacopo Mondi X-Patchwork-Id: 828757 Received: from perceval.ideasonboard.com (perceval.ideasonboard.com [213.167.242.64]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE9FC155321 for ; Fri, 13 Sep 2024 21:47:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.167.242.64 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264055; cv=none; b=ZNJVkxErl4CD6elGQpCqohJpvtSK1n7lLX2yD2KxNvsYXVom10dng8jB7XtNLtYu7V99cSOp49KV2ncs7C5/afVEYue7Ir6Mf/PZp0iJHQ7Cihn6q2iFUnFsnp4MsDh+ZNa7AHhsf6t1xo3XnKdY2OTIKq7echtROPnb6hNNhR8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726264055; c=relaxed/simple; bh=UjeQVJLvchVgUkD23gOYlJ7aYEUTepJcGMF+imJjm7A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dC7z6MX9/d7xd7v0AuuD66wP60mCM0n30SJjZUxPS7o42VrClY192c4DfxyfUZFfi5KPXe3XlwfqJjAa9uyEZTNEOP4qn3wohn3eYnXeHYlTlnOj2XO2OA9tuZD+XNx4at3Bl+afebZh9FpXnOMTTZVf2HZy4BGcALuBVYacu0w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com; spf=pass smtp.mailfrom=ideasonboard.com; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b=X0CmsT88; arc=none smtp.client-ip=213.167.242.64 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ideasonboard.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=ideasonboard.com header.i=@ideasonboard.com header.b="X0CmsT88" Received: from ideasonboard.com (213-229-8-243.static.upcbusiness.at [213.229.8.243]) by perceval.ideasonboard.com (Postfix) with ESMTPSA id 5470C2598; Fri, 13 Sep 2024 23:45:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ideasonboard.com; s=mail; t=1726263951; bh=UjeQVJLvchVgUkD23gOYlJ7aYEUTepJcGMF+imJjm7A=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X0CmsT88crHPHV+70UFkLx4bfWissfunGauLCTF+nUBT1OLQrOzxXgl5v3MsBC5a+ TumUJtpP2+X/oet1RLBwwlgMm7IBOXloEGhVxi3x9YyUUTFQm7q1C7ZSn4AldS8szT pv1g83TAQuaZ17xrE4undLJqb34Gu3PwpNlxAG/E= From: Jacopo Mondi To: linux-media@vger.kernel.org Cc: Jacopo Mondi , Laurent Pinchart Subject: [PATCH 10/10] media: pispbe: Add support for multi-context Date: Fri, 13 Sep 2024 23:46:55 +0200 Message-ID: <20240913214657.1502838-11-jacopo.mondi@ideasonboard.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> References: <20240913214657.1502838-1-jacopo.mondi@ideasonboard.com> Precedence: bulk X-Mailing-List: linux-media@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add support for context multiplexing to the PiSP BackEnd driver. Move information that used to be per-node to the video context and global information to the media context. Implement media and video context allocation and release operations to allocate and initialize a context and initialize the video queue there contained. Remove the per-node video device format and the buffer queues and port the driver to use the ones in the video device context. The operations that used to work with a pispbe_node now operates on a pispbe_context. - v4l2 ioctl ops: receive a file pointer, retrieve the associated context from the open file handle - vb2 ops: receive a queue, retrieve the context from the queue - internal driver ops: given a media device contex retrieve the video context from the other video devices when assembling a job Signed-off-by: Jacopo Mondi --- .../platform/raspberrypi/pisp_be/pisp_be.c | 509 +++++++++++++----- 1 file changed, 370 insertions(+), 139 deletions(-) diff --git a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c index 555c28dddc4b..34fbe2b730b0 100644 --- a/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c +++ b/drivers/media/platform/raspberrypi/pisp_be/pisp_be.c @@ -151,6 +151,70 @@ static const struct pispbe_node_description node_desc[PISPBE_NUM_NODES] = { * Structure to describe a single node /dev/video which represents a single * input or output queue to the PiSP Back End device. */ + +struct pispbe_media_context { + struct media_device_context mdev_context; + u32 streaming_map; +}; + +static struct pispbe_media_context * +pispbe_media_context(struct media_device_context *mdev_context) +{ + return container_of(mdev_context, struct pispbe_media_context, + mdev_context); +} + +struct pispbe_node; +struct pispbe_entity_context { + struct video_device_context vdev_context; + + struct v4l2_format format; + struct list_head ready_queue; + const struct pisp_be_format *pisp_format; + struct pispbe_dev *pispbe; + struct pispbe_node *node; +}; + +static struct pispbe_entity_context * +pispbe_entity_context(struct video_device_context *ctx) +{ + return container_of(ctx, struct pispbe_entity_context, vdev_context); +} + +static struct pispbe_entity_context * +pispbe_entity_context_from_queue(struct vb2_queue *queue) +{ + return pispbe_entity_context(video_device_context_from_queue(queue)); +} + +static struct pispbe_entity_context * +pispbe_entity_context_from_file(struct file *file, struct video_device *vfd) +{ + return pispbe_entity_context(video_device_context_from_file(file, vfd)); +} + +static struct pispbe_entity_context * +pispbe_get_dev_context(struct pispbe_media_context *pispbe_mdev_context, + struct video_device *vdev) +{ + struct video_device_context *ctx = + video_device_context_get(&pispbe_mdev_context->mdev_context, + vdev); + + return ctx ? pispbe_entity_context(ctx) : NULL; +} + +void pispbe_put_dev_context(struct pispbe_entity_context *ctx) +{ + video_device_context_put(&ctx->vdev_context); +} + +static struct pispbe_media_context * +pispbe_media_context_from_dev(struct pispbe_entity_context *context) +{ + return pispbe_media_context(context->vdev_context.base.mdev_context); +} + struct pispbe_node { unsigned int id; int vfl_dir; @@ -162,14 +226,13 @@ struct pispbe_node { struct pispbe_dev *pispbe; /* Video device lock */ struct mutex node_lock; - /* vb2_queue lock */ - struct mutex queue_lock; - struct list_head ready_queue; - struct vb2_queue queue; - struct v4l2_format format; - const struct pisp_be_format *pisp_format; }; +static struct pispbe_node *pispbe_node_from_vdev(struct video_device *vdev) +{ + return container_of(vdev, struct pispbe_node, vfd); +} + /* For logging only, use the entity name with "pispbe" and separator removed */ #define NODE_NAME(node) \ (node_desc[(node)->id].ent_name + sizeof(PISPBE_NAME)) @@ -216,15 +279,15 @@ struct pispbe_dev { struct pispbe_node node[PISPBE_NUM_NODES]; dma_addr_t config_dma_addr; unsigned int sequence; - u32 streaming_map; struct pispbe_job queued_job, running_job; - /* protects "hw_busy" flag, streaming_map and job_queue */ + /* protects "hw_busy" flag and job_queue */ spinlock_t hw_lock; bool hw_busy; /* non-zero if a job is queued or is being started */ struct list_head job_queue; int irq; u32 hw_version; u8 done, started; + struct media_pipeline pipe; }; static u32 pispbe_rd(struct pispbe_dev *pispbe, unsigned int offset) @@ -308,30 +371,31 @@ struct pispbe_buffer { }; static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, - struct pispbe_node *node) + struct pispbe_entity_context *context) { - unsigned int num_planes = node->format.fmt.pix_mp.num_planes; + struct v4l2_format *format = &context->format; + unsigned int num_planes = format->fmt.pix_mp.num_planes; unsigned int plane_factor = 0; unsigned int size; unsigned int p; - if (!buf || !node->pisp_format) + if (!buf || !context->pisp_format) return 0; /* * Determine the base plane size. This will not be the same - * as node->format.fmt.pix_mp.plane_fmt[0].sizeimage for a single + * as format->fmt.pix_mp.plane_fmt[0].sizeimage for a single * plane buffer in an mplane format. */ - size = node->format.fmt.pix_mp.plane_fmt[0].bytesperline * - node->format.fmt.pix_mp.height; + size = format->fmt.pix_mp.plane_fmt[0].bytesperline * + format->fmt.pix_mp.height; for (p = 0; p < num_planes && p < PISPBE_MAX_PLANES; p++) { addr[p] = vb2_dma_contig_plane_dma_addr(&buf->vb.vb2_buf, p); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } - for (; p < PISPBE_MAX_PLANES && node->pisp_format->plane_factor[p]; p++) { + for (; p < PISPBE_MAX_PLANES && context->pisp_format->plane_factor[p]; p++) { /* * Calculate the address offset of this plane as needed * by the hardware. This is specifically for non-mplane @@ -339,7 +403,7 @@ static int pispbe_get_planes_addr(dma_addr_t addr[3], struct pispbe_buffer *buf, * for the V4L2_PIX_FMT_YUV420 format. */ addr[p] = addr[0] + ((size * plane_factor) >> 3); - plane_factor += node->pisp_format->plane_factor[p]; + plane_factor += context->pisp_format->plane_factor[p]; } return num_planes; @@ -355,11 +419,13 @@ static dma_addr_t pispbe_get_addr(struct pispbe_buffer *buf) static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, struct pispbe_job_descriptor *job, + struct pispbe_media_context *mdev_context, struct pispbe_buffer *buf[PISPBE_NUM_NODES]) { struct pispbe_hw_enables *hw_en = &job->hw_enables; struct pisp_be_tiles_config *config = job->config; dma_addr_t *addrs = job->hw_dma_addrs; + struct pispbe_entity_context *ctx; int ret; /* Take a copy of the "enable" bitmaps so we can modify them. */ @@ -370,8 +436,10 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * Main input first. There are 3 address pointers, corresponding to up * to 3 planes. */ - ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], - &pispbe->node[MAIN_INPUT_NODE]); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[MAIN_INPUT_NODE].vfd); + ret = pispbe_get_planes_addr(addrs, buf[MAIN_INPUT_NODE], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) { /* Shouldn't happen, we have validated an input is available. */ dev_warn(pispbe->dev, "ISP-BE missing input\n"); @@ -427,9 +495,11 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, /* Main image output channels. */ for (unsigned int i = 0; i < PISP_BACK_END_NUM_OUTPUTS; i++) { + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + i].vfd); ret = pispbe_get_planes_addr(addrs + 7 + 3 * i, - buf[OUTPUT0_NODE + i], - &pispbe->node[OUTPUT0_NODE + i]); + buf[OUTPUT0_NODE + i], ctx); + pispbe_put_dev_context(ctx); if (ret <= 0) hw_en->rgb_enables &= ~(PISP_BE_RGB_ENABLE_OUTPUT0 << i); } @@ -450,35 +520,46 @@ static void pispbe_xlate_addrs(struct pispbe_dev *pispbe, * * Returns 0 if a job has been successfully prepared, < 0 otherwise. */ -static int pispbe_prepare_job(struct pispbe_dev *pispbe) +static int pispbe_prepare_job(struct pispbe_dev *pispbe, + struct pispbe_entity_context *context) { + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); struct pispbe_buffer *buf[PISPBE_NUM_NODES] = {}; struct pispbe_job_descriptor *job; + struct pispbe_entity_context *ctx; unsigned int streaming_map; unsigned int config_index; - struct pispbe_node *node; scoped_guard(spinlock_irq, &pispbe->hw_lock) { static const u32 mask = BIT(CONFIG_NODE) | BIT(MAIN_INPUT_NODE); - if ((pispbe->streaming_map & mask) != mask) + if ((mdev_context->streaming_map & mask) != mask) return -ENODEV; /* * Take a copy of streaming_map: nodes activated after this * point are ignored when preparing this job. */ - streaming_map = pispbe->streaming_map; + streaming_map = mdev_context->streaming_map; } job = kzalloc(sizeof(*job), GFP_KERNEL); if (!job) return -ENOMEM; - node = &pispbe->node[CONFIG_NODE]; - buf[CONFIG_NODE] = list_first_entry_or_null(&node->ready_queue, + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[CONFIG_NODE].vfd); + if (!ctx) { + kfree(job); + return -ENODEV; + } + + buf[CONFIG_NODE] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (!buf[CONFIG_NODE]) { kfree(job); return -ENODEV; @@ -530,12 +611,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) ignore_buffers = true; } - node = &pispbe->node[i]; - /* Pull a buffer from each V4L2 queue to form the queued job */ - buf[i] = list_first_entry_or_null(&node->ready_queue, + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx && !ignore_buffers) + goto err_return_buffers; + + buf[i] = list_first_entry_or_null(&ctx->ready_queue, struct pispbe_buffer, ready_list); + pispbe_put_dev_context(ctx); + if (buf[i]) { list_del(&buf[i]->ready_list); job->buffers[i] = buf[i]; @@ -546,7 +632,7 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) } /* Convert buffers to DMA addresses for the hardware */ - pispbe_xlate_addrs(pispbe, job, buf); + pispbe_xlate_addrs(pispbe, job, mdev_context, buf); spin_lock(&pispbe->hw_lock); list_add_tail(&job->queue, &pispbe->job_queue); @@ -556,13 +642,17 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) err_return_buffers: for (unsigned int i = 0; i < PISPBE_NUM_NODES; i++) { - struct pispbe_node *n = &pispbe->node[i]; - if (!buf[i]) continue; /* Return the buffer to the ready_list queue */ - list_add(&buf[i]->ready_list, &n->ready_queue); + ctx = pispbe_get_dev_context(mdev_context, + &pispbe->node[i].vfd); + if (!ctx) + continue; + + list_add(&buf[i]->ready_list, &ctx->ready_queue); + pispbe_put_dev_context(ctx); } kfree(job); @@ -570,7 +660,9 @@ static int pispbe_prepare_job(struct pispbe_dev *pispbe) return -ENODEV; } -static void pispbe_schedule(struct pispbe_dev *pispbe, bool clear_hw_busy) +static void pispbe_schedule(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, + bool clear_hw_busy) { struct pispbe_job_descriptor *job; @@ -674,18 +766,20 @@ static irqreturn_t pispbe_isr(int irq, void *dev) } /* check if there's more to do before going to sleep */ - pispbe_schedule(pispbe, can_queue_another); + pispbe_schedule(pispbe, NULL, can_queue_another); return IRQ_HANDLED; } static int pisp_be_validate_config(struct pispbe_dev *pispbe, + struct pispbe_media_context *mdev_context, struct pisp_be_tiles_config *config) { u32 bayer_enables = config->config.global.bayer_enables; u32 rgb_enables = config->config.global.rgb_enables; + struct pispbe_entity_context *context; + struct v4l2_pix_format_mplane *mp; struct device *dev = pispbe->dev; - struct v4l2_format *fmt; unsigned int bpl, size; if (!(bayer_enables & PISP_BE_BAYER_ENABLE_INPUT) == @@ -702,36 +796,50 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, } /* Ensure output config strides and buffer sizes match the V4L2 formats. */ - fmt = &pispbe->node[TDN_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[TDN_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_TDN_OUTPUT) { bpl = config->config.tdn_output_format.stride; size = bpl * config->config.tdn_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on tdn_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on tdn_output\n", __func__); return -EINVAL; } } - fmt = &pispbe->node[STITCH_OUTPUT_NODE].format; + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[STITCH_OUTPUT_NODE].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + if (bayer_enables & PISP_BE_BAYER_ENABLE_STITCH_OUTPUT) { bpl = config->config.stitch_output_format.stride; size = bpl * config->config.stitch_output_format.height; - if (fmt->fmt.pix_mp.plane_fmt[0].bytesperline < bpl) { + if (mp->plane_fmt[0].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on stitch_output\n", __func__); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[0].sizeimage < size) { + if (mp->plane_fmt[0].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on stitch_output\n", __func__); return -EINVAL; @@ -746,8 +854,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_WALLPAPER_ROLL) continue; /* TODO: Size checks for wallpaper formats */ - fmt = &pispbe->node[OUTPUT0_NODE + j].format; - for (unsigned int i = 0; i < fmt->fmt.pix_mp.num_planes; i++) { + context = pispbe_get_dev_context(mdev_context, + &pispbe->node[OUTPUT0_NODE + j].vfd); + if (!context) + return -EINVAL; + + mp = &context->format.fmt.pix_mp; + pispbe_put_dev_context(context); + + for (unsigned int i = 0; i < mp->num_planes; i++) { bpl = !i ? config->config.output_format[j].image.stride : config->config.output_format[j].image.stride2; size = bpl * config->config.output_format[j].image.height; @@ -756,13 +871,15 @@ static int pisp_be_validate_config(struct pispbe_dev *pispbe, PISP_IMAGE_FORMAT_SAMPLING_420) size >>= 1; - if (fmt->fmt.pix_mp.plane_fmt[i].bytesperline < bpl) { + if (mp->plane_fmt[i].bytesperline < bpl) { dev_dbg(dev, "%s: bpl mismatch on output %d\n", __func__, j); + dev_dbg(dev, "Expected %u, got %u\n", + bpl, mp->plane_fmt[i].bytesperline); return -EINVAL; } - if (fmt->fmt.pix_mp.plane_fmt[i].sizeimage < size) { + if (mp->plane_fmt[i].sizeimage < size) { dev_dbg(dev, "%s: size mismatch on output\n", __func__); return -EINVAL; @@ -777,10 +894,12 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, unsigned int *nplanes, unsigned int sizes[], struct device *alloc_devs[]) { + struct pispbe_entity_context *context = pispbe_entity_context_from_queue(q); + struct v4l2_format *format = &context->format; struct pispbe_node *node = vb2_get_drv_priv(q); struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; if (*nplanes) { if (*nplanes != num_planes) @@ -788,8 +907,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (sizes[i] < size) return -EINVAL; @@ -801,8 +920,8 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, *nplanes = num_planes; for (unsigned int i = 0; i < *nplanes; i++) { unsigned int size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; sizes[i] = size; } @@ -815,15 +934,21 @@ static int pispbe_node_queue_setup(struct vb2_queue *q, unsigned int *nbuffers, static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) { - struct pispbe_node *node = vb2_get_drv_priv(vb->vb2_queue); + struct vb2_queue *q = vb->vb2_queue; + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct v4l2_format *format = &context->format; + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; unsigned int num_planes = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.num_planes : 1; + format->fmt.pix_mp.num_planes : 1; for (unsigned int i = 0; i < num_planes; i++) { unsigned long size = NODE_IS_MPLANE(node) ? - node->format.fmt.pix_mp.plane_fmt[i].sizeimage : - node->format.fmt.meta.buffersize; + format->fmt.pix_mp.plane_fmt[i].sizeimage : + format->fmt.meta.buffersize; if (vb2_plane_size(vb, i) < size) { dev_dbg(pispbe->dev, @@ -841,7 +966,7 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) memcpy(dst, src, sizeof(struct pisp_be_tiles_config)); - return pisp_be_validate_config(pispbe, dst); + return pisp_be_validate_config(pispbe, mdev_context, dst); } return 0; @@ -849,53 +974,67 @@ static int pispbe_node_buffer_prepare(struct vb2_buffer *vb) static void pispbe_node_buffer_queue(struct vb2_buffer *buf) { - struct vb2_v4l2_buffer *vbuf = - container_of(buf, struct vb2_v4l2_buffer, vb2_buf); - struct pispbe_buffer *buffer = - container_of(vbuf, struct pispbe_buffer, vb); - struct pispbe_node *node = vb2_get_drv_priv(buf->vb2_queue); + struct vb2_v4l2_buffer *vbuf = container_of(buf, struct vb2_v4l2_buffer, + vb2_buf); + struct pispbe_buffer *buffer = container_of(vbuf, struct pispbe_buffer, + vb); + struct vb2_queue *q = buf->vb2_queue; + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); - list_add_tail(&buffer->ready_list, &node->ready_queue); + list_add_tail(&buffer->ready_list, &context->ready_queue); /* * Every time we add a buffer, check if there's now some work for the hw * to do. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); } static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_buffer *buf, *tmp; int ret; + ret = video_device_context_pipeline_start(&context->vdev_context, + &pispbe->pipe); + if (ret) + return ret; + ret = pm_runtime_resume_and_get(pispbe->dev); if (ret < 0) goto err_return_buffers; spin_lock_irq(&pispbe->hw_lock); - node->pispbe->streaming_map |= BIT(node->id); + mdev_context->streaming_map |= BIT(node->id); node->pispbe->sequence = 0; spin_unlock_irq(&pispbe->hw_lock); dev_dbg(pispbe->dev, "%s: for node %s (count %u)\n", __func__, NODE_NAME(node), count); dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - node->pispbe->streaming_map); + mdev_context->streaming_map); /* Maybe we're ready to run. */ - if (!pispbe_prepare_job(pispbe)) - pispbe_schedule(pispbe, false); + if (!pispbe_prepare_job(pispbe, context)) + pispbe_schedule(pispbe, mdev_context, false); return 0; err_return_buffers: - list_for_each_entry_safe(buf, tmp, &node->ready_queue, ready_list) { + list_for_each_entry_safe(buf, tmp, &context->ready_queue, ready_list) { list_del(&buf->ready_list); vb2_buffer_done(&buf->vb.vb2_buf, VB2_BUF_STATE_QUEUED); } @@ -905,7 +1044,11 @@ static int pispbe_node_start_streaming(struct vb2_queue *q, unsigned int count) static void pispbe_node_stop_streaming(struct vb2_queue *q) { - struct pispbe_node *node = vb2_get_drv_priv(q); + struct pispbe_entity_context *context = + pispbe_entity_context_from_queue(q); + struct pispbe_media_context *mdev_context = + pispbe_media_context_from_dev(context); + struct pispbe_node *node = context->node; struct pispbe_dev *pispbe = node->pispbe; struct pispbe_buffer *buf; @@ -920,7 +1063,7 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) */ dev_dbg(pispbe->dev, "%s: for node %s\n", __func__, NODE_NAME(node)); do { - buf = list_first_entry_or_null(&node->ready_queue, + buf = list_first_entry_or_null(&context->ready_queue, struct pispbe_buffer, ready_list); if (buf) { @@ -929,13 +1072,13 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) } } while (buf); - vb2_wait_for_all_buffers(&node->queue); + vb2_wait_for_all_buffers(&context->vdev_context.queue); spin_lock_irq(&pispbe->hw_lock); - pispbe->streaming_map &= ~BIT(node->id); + mdev_context->streaming_map &= ~BIT(node->id); /* Release all jobs once all nodes have stopped streaming. */ - if (pispbe->streaming_map == 0) { + if (mdev_context->streaming_map == 0) { struct pispbe_job_descriptor *job, *temp; list_for_each_entry_safe(job, temp, &pispbe->job_queue, queue) { @@ -948,8 +1091,10 @@ static void pispbe_node_stop_streaming(struct vb2_queue *q) pm_runtime_mark_last_busy(pispbe->dev); pm_runtime_put_autosuspend(pispbe->dev); + video_device_pipeline_stop(context->vdev_context.vdev); + dev_dbg(pispbe->dev, "Nodes streaming now 0x%x\n", - pispbe->streaming_map); + mdev_context->streaming_map); } static const struct vb2_ops pispbe_node_queue_ops = { @@ -990,6 +1135,8 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (!NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -998,7 +1145,7 @@ static int pispbe_node_g_fmt_vid_cap(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get capture format for node %s\n", NODE_NAME(node)); @@ -1010,6 +1157,8 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (NODE_IS_CAPTURE(node) || NODE_IS_META(node)) { dev_dbg(pispbe->dev, @@ -1018,7 +1167,7 @@ static int pispbe_node_g_fmt_vid_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for node %s\n", NODE_NAME(node)); @@ -1030,6 +1179,8 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); if (!NODE_IS_META(node) || NODE_IS_CAPTURE(node)) { dev_dbg(pispbe->dev, @@ -1038,7 +1189,7 @@ static int pispbe_node_g_fmt_meta_out(struct file *file, void *priv, return -EINVAL; } - *f = node->format; + *f = context->format; dev_dbg(pispbe->dev, "Get output format for meta node %s\n", NODE_NAME(node)); @@ -1206,17 +1357,19 @@ static int pispbe_node_s_fmt_vid_cap(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_cap(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set capture format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1229,17 +1382,19 @@ static int pispbe_node_s_fmt_vid_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_vid_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); + context->format = *f; + context->pisp_format = pispbe_find_fmt(f->fmt.pix_mp.pixelformat); dev_dbg(pispbe->dev, "Set output format for node %s to %p4cc\n", NODE_NAME(node), &f->fmt.pix_mp.pixelformat); @@ -1252,17 +1407,19 @@ static int pispbe_node_s_fmt_meta_out(struct file *file, void *priv, { struct pispbe_node *node = video_drvdata(file); struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); int ret; ret = pispbe_node_try_fmt_meta_out(file, priv, f); if (ret < 0) return ret; - if (vb2_is_busy(&node->queue)) + if (vb2_is_busy(&context->vdev_context.queue)) return -EBUSY; - node->format = *f; - node->pisp_format = &meta_out_supported_formats[0]; + context->format = *f; + context->pisp_format = &meta_out_supported_formats[0]; dev_dbg(pispbe->dev, "Set output format for meta node %s to %p4cc\n", NODE_NAME(node), &f->fmt.meta.dataformat); @@ -1274,8 +1431,10 @@ static int pispbe_node_enum_fmt(struct file *file, void *priv, struct v4l2_fmtdesc *f) { struct pispbe_node *node = video_drvdata(file); + struct pispbe_entity_context *context = + pispbe_entity_context_from_file(file, &node->vfd); - if (f->type != node->queue.type) + if (f->type != context->vdev_context.queue.type) return -EINVAL; if (NODE_IS_META(node)) { @@ -1349,24 +1508,16 @@ static const struct v4l2_ioctl_ops pispbe_node_ioctl_ops = { .vidioc_streamoff = vb2_ioctl_streamoff, }; -static const struct video_device pispbe_videodev = { - .name = PISPBE_NAME, - .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ - .fops = &pispbe_fops, - .ioctl_ops = &pispbe_node_ioctl_ops, - .minor = -1, - .release = video_device_release_empty, -}; - -static void pispbe_node_def_fmt(struct pispbe_node *node) +static void pispbe_node_def_fmt(struct pispbe_node *node, + struct pispbe_entity_context *context) { + struct v4l2_format *format = &context->format; + if (NODE_IS_META(node) && NODE_IS_OUTPUT(node)) { /* Config node */ - struct v4l2_format *f = &node->format; - - f->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; - f->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); - f->type = node->buf_type; + format->fmt.meta.dataformat = V4L2_META_FMT_RPI_BE_CFG; + format->fmt.meta.buffersize = sizeof(struct pisp_be_tiles_config); + format->type = node->buf_type; } else { struct v4l2_format f = { .fmt.pix_mp.pixelformat = V4L2_PIX_FMT_YUV420, @@ -1375,36 +1526,39 @@ static void pispbe_node_def_fmt(struct pispbe_node *node) .type = node->buf_type, }; pispbe_try_format(&f, node); - node->format = f; + *format = f; } - node->pisp_format = pispbe_find_fmt(node->format.fmt.pix_mp.pixelformat); + context->pisp_format = pispbe_find_fmt(format->fmt.pix_mp.pixelformat); } -/* - * Initialise a struct pispbe_node and register it as /dev/video - * to represent one of the PiSP Back End's input or output streams. - */ -static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +static int pispbe_alloc_entity_context(struct media_entity *entity, + struct media_entity_context **ctx) { - bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); - struct pispbe_node *node = &pispbe->node[id]; - struct media_entity *entity = &node->vfd.entity; - struct video_device *vdev = &node->vfd; - struct vb2_queue *q = &node->queue; + struct video_device *vdev = container_of(entity, struct video_device, + entity); + struct pispbe_node *node = pispbe_node_from_vdev(vdev); + struct pispbe_dev *pispbe = node->pispbe; + struct pispbe_entity_context *context; + struct vb2_queue *q; int ret; - node->id = id; - node->pispbe = pispbe; - node->buf_type = node_desc[id].buf_type; + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_entity_context *)*ctx; - mutex_init(&node->node_lock); - mutex_init(&node->queue_lock); - INIT_LIST_HEAD(&node->ready_queue); + video_device_init_context(vdev, &context->vdev_context); + + context->pispbe = pispbe; + context->node = node; + + INIT_LIST_HEAD(&context->ready_queue); - node->format.type = node->buf_type; - pispbe_node_def_fmt(node); + context->format.type = node->buf_type; + pispbe_node_def_fmt(node, context); + q = &context->vdev_context.queue; q->type = node->buf_type; q->io_modes = VB2_MMAP | VB2_DMABUF; q->mem_ops = &vb2_dma_contig_memops; @@ -1414,41 +1568,92 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) q->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_MONOTONIC; q->dev = pispbe->dev; /* get V4L2 to handle node->queue locking */ - q->lock = &node->queue_lock; + q->lock = &context->vdev_context.queue_lock; ret = vb2_queue_init(q); if (ret < 0) { dev_err(pispbe->dev, "vb2_queue_init failed\n"); - goto err_mutex_destroy; + goto err_cleanup; } + return 0; + +err_cleanup: + kfree(*ctx); + return ret; +} + +static void pispbe_free_entity_context(struct media_entity_context *ctx) +{ + struct pispbe_entity_context *context = + (struct pispbe_entity_context *)ctx; + + list_del_init(&context->ready_queue); + video_device_cleanup_context(&context->vdev_context); + kfree(context); +} + +static const struct media_entity_operations pispbe_media_entity_ops = { + .alloc_context = pispbe_alloc_entity_context, + .destroy_context = pispbe_free_entity_context, +}; + +static const struct video_device pispbe_videodev = { + .name = PISPBE_NAME, + .vfl_dir = VFL_DIR_M2M, /* gets overwritten */ + .fops = &pispbe_fops, + .ioctl_ops = &pispbe_node_ioctl_ops, + .minor = -1, + .release = video_device_release_empty, +}; + +/* + * Initialise a struct pispbe_node and register it as /dev/video + * to represent one of the PiSP Back End's input or output streams. + */ +static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) +{ + bool output = NODE_DESC_IS_OUTPUT(&node_desc[id]); + struct pispbe_node *node = &pispbe->node[id]; + struct media_entity *entity = &node->vfd.entity; + struct video_device *vdev = &node->vfd; + int ret; + + node->id = id; + node->pispbe = pispbe; + node->buf_type = node_desc[id].buf_type; + + mutex_init(&node->node_lock); + *vdev = pispbe_videodev; /* default initialization */ strscpy(vdev->name, node_desc[id].ent_name, sizeof(vdev->name)); vdev->v4l2_dev = &pispbe->v4l2_dev; vdev->vfl_dir = output ? VFL_DIR_TX : VFL_DIR_RX; /* get V4L2 to serialise our ioctls */ vdev->lock = &node->node_lock; - vdev->queue = &node->queue; vdev->device_caps = V4L2_CAP_STREAMING | node_desc[id].caps; + vdev->entity.ops = &pispbe_media_entity_ops; + node->pad.flags = output ? MEDIA_PAD_FL_SOURCE : MEDIA_PAD_FL_SINK; ret = media_entity_pads_init(entity, 1, &node->pad); if (ret) { dev_err(pispbe->dev, "Failed to register media pads for %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } + video_set_drvdata(vdev, node); + ret = video_register_device(vdev, VFL_TYPE_VIDEO, PISPBE_VIDEO_NODE_OFFSET); if (ret) { dev_err(pispbe->dev, "Failed to register video %s device node\n", NODE_NAME(node)); - goto err_unregister_queue; + goto err_mutex_destroy; } - video_set_drvdata(vdev, node); if (output) ret = media_create_pad_link(entity, 0, &pispbe->sd.entity, @@ -1468,11 +1673,8 @@ static int pispbe_init_node(struct pispbe_dev *pispbe, unsigned int id) err_unregister_video_dev: video_unregister_device(&node->vfd); -err_unregister_queue: - vb2_queue_release(&node->queue); err_mutex_destroy: mutex_destroy(&node->node_lock); - mutex_destroy(&node->queue_lock); return ret; } @@ -1484,6 +1686,38 @@ static const struct v4l2_subdev_ops pispbe_sd_ops = { .pad = &pispbe_pad_ops, }; +static int pispbe_media_alloc_context(struct media_device *mdev, + struct media_device_context **ctx) +{ + struct pispbe_media_context *context; + + *ctx = kzalloc(sizeof(*context), GFP_KERNEL); + if (!*ctx) + return -ENOMEM; + context = (struct pispbe_media_context *)*ctx; + + media_device_init_context(mdev, &context->mdev_context); + + context->streaming_map = 0; + + return 0; +} + +static void pispbe_media_destroy_context(struct media_device_context *ctx) +{ + struct pispbe_media_context *context = + (struct pispbe_media_context *)ctx; + + context->streaming_map = 0; + media_device_cleanup_context(&context->mdev_context); + kfree(context); +} + +static const struct media_device_ops pispbe_media_device_ops = { + .alloc_context = pispbe_media_alloc_context, + .destroy_context = pispbe_media_destroy_context, +}; + static int pispbe_init_subdev(struct pispbe_dev *pispbe) { struct v4l2_subdev *sd = &pispbe->sd; @@ -1527,6 +1761,7 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) mdev = &pispbe->mdev; mdev->hw_revision = pispbe->hw_version; mdev->dev = pispbe->dev; + mdev->ops = &pispbe_media_device_ops; strscpy(mdev->model, PISPBE_NAME, sizeof(mdev->model)); media_device_init(mdev); @@ -1570,10 +1805,8 @@ static int pispbe_init_devices(struct pispbe_dev *pispbe) err_unregister_mdev: media_device_unregister(mdev); err_unregister_nodes: - while (num_regist-- > 0) { + while (num_regist-- > 0) video_unregister_device(&pispbe->node[num_regist].vfd); - vb2_queue_release(&pispbe->node[num_regist].queue); - } v4l2_device_unregister_subdev(&pispbe->sd); media_entity_cleanup(&pispbe->sd.entity); err_unregister_v4l2: @@ -1601,9 +1834,7 @@ static void pispbe_destroy_devices(struct pispbe_dev *pispbe) for (int i = PISPBE_NUM_NODES - 1; i >= 0; i--) { video_unregister_device(&pispbe->node[i].vfd); - vb2_queue_release(&pispbe->node[i].queue); mutex_destroy(&pispbe->node[i].node_lock); - mutex_destroy(&pispbe->node[i].queue_lock); } media_device_cleanup(&pispbe->mdev);