From patchwork Fri Dec 2 17:02:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 630503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4448C47089 for ; Fri, 2 Dec 2022 17:03:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234265AbiLBRDO (ORCPT ); Fri, 2 Dec 2022 12:03:14 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234232AbiLBRDD (ORCPT ); Fri, 2 Dec 2022 12:03:03 -0500 Received: from mail-ej1-x630.google.com (mail-ej1-x630.google.com [IPv6:2a00:1450:4864:20::630]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 821BD2BB22 for ; Fri, 2 Dec 2022 09:03:02 -0800 (PST) Received: by mail-ej1-x630.google.com with SMTP id bj12so12920835ejb.13 for ; Fri, 02 Dec 2022 09:03:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=bNwN70fEkttO9/UpjS90pNFzNVmjhBb547h7/recR+g=; b=YMYF3FKLRKlD+VLDPSP4cm/n9VDbH+eOQhMJ4Ub0qET1/4YDbnMaYFow1EP3RGfI+m vyNQwoImPxvn4FEnFouVF4PoTlzo+tCYB4JJvcJyL2Jw440PlHUJbuIcJTc6gtRPIlbT 7m7jbmZvRpcEM+ytuQZkODS4ts/U3+6fqeuOc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bNwN70fEkttO9/UpjS90pNFzNVmjhBb547h7/recR+g=; b=jcvHErAVhdJgLYrGBT/XKNK4J1sBJgEvJn5SoVQpfa7yRP/dSEBGeZPDfZ4aFKQ8Jd 4Hp1fWhpujk/CC9FTdrBTqqYa4NEP/djPSdktaz1wN6zsKC4F+eemqkv9WyPfbRZII9b QEU3wSUkY+13//vp+OY8N7buOSgGdsRh5FxtKa7/Ia1ej6ULoxQvn/iGfZaalUUlCbLh 56bQe4m71C1CgGxP9XQ1IwtcxWKE4o+FEbnfwLRvO8boHWylm9rfz9RYRrx6fhmI4vz/ wv+4akNIt8XOSfsk01c75a7Dh+GHeB7G3DHHssP95gJlVLJ040Vvh4KTe0YrsS/BVCfn EL2w== X-Gm-Message-State: ANoB5pnRG65HvEjxtm9jc+ncBjWADZnOB2A/Ytmd4C55ioErnyrvAwkO J4umslaL4GpB2Ng1qndiF5nU8g== X-Google-Smtp-Source: AA0mqf7j0ojwl/0ab78IwDUCpSYN3+EkBoWKs2AM8Wssw53FrEnbQ+LP+OZh9FaM8Gz3BMpLN0e6OQ== X-Received: by 2002:a17:906:5586:b0:7ad:902c:d158 with SMTP id y6-20020a170906558600b007ad902cd158mr47425492ejp.121.1670000581051; Fri, 02 Dec 2022 09:03:01 -0800 (PST) Received: from alco.roam.corp.google.com (80.71.134.83.ipv4.parknet.dk. [80.71.134.83]) by smtp.gmail.com with ESMTPSA id sa22-20020a170906edb600b0073d7ab84375sm3249133ejb.92.2022.12.02.09.03.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Dec 2022 09:03:00 -0800 (PST) From: Ricardo Ribalda Date: Fri, 02 Dec 2022 18:02:42 +0100 Subject: [PATCH RESEND v2 2/8] media: uvc: Allow quirking by entity guid MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v2-2-0d7978a817cc@chromium.org> References: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> To: Mauro Carvalho Chehab , Laurent Pinchart Cc: "hn.chen" , linux-media@vger.kernel.org, Ricardo Ribalda , linux-kernel@vger.kernel.org X-Mailer: b4 0.11.0-dev-696ae X-Developer-Signature: v=1; a=openpgp-sha256; l=1791; i=ribalda@chromium.org; h=from:subject:message-id; bh=es8VifXL1MkLaxdmrIj9wvoFjqnH8ap0KliznPX0ArQ=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjii+4UFyy5wEL7JacPovRxgY/wDg86ABUbzcK85TJ WJqFrIeJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCY4ovuAAKCRDRN9E+zzrEiHZ9D/ 4yXuaKvxm2kV9m+nvZzNTV1+awqBpG5QYMl05OQ1Oww/Pq5x7PBGr0GQgVM4IauG2bLsRv6BKdFSAs z/H+qXgVE6q+ObvX1yOphZgnRXAuYZv+Jfy5o3Q1gQNjgw0glnps8A/gHuRf7PEmq0aWuNYRKsEjMB gl/cT+EP8p9dSNX9e7mWK44C4z9dsWfmR12O8urSGzRzR98LOeEED1ywLmh4dklEJId4WjexIOmvhU gmBL9Ir+GUgGk5oVY3VwyzOi2RgVxVhb/WqgwVNe1VuRGINTN/bVE2ckUzey56teTB/+JJeNp7LBo1 WkxaGSUqVvACvKaNZEavwD6p9a7tP5Pyuk25zQrAhlT2DbljorwNyZa55B1b7K4lll7EzVBQ+HwBUd LRy893zuzDEWmtJGgcoK3dGXoaG2VSw9plEXXmSqFZbUQaK0pRDw8bPLhdlRitgpchX9WWcVNQCXsz t8FquJoH4sbmjRR2wxCWLeIFHiwTHTZwJ/eMQEWdjptRspwhpTyGqp2w9rjhxvM8wzrAgEepCNsS6a /3NMoA7ws+dnZ9urdLId4T+nCP70rUtzhB+9uuPKlcIoH77AQBac/aWleL+IjjjIxFGRJhakl9XHD0 7U+mdJMfjOq8KDm+v1elgwnafSKwNHNc7QQwICG6oMjE6VfMcpNT9kFzvt/w== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org When an IP is shared by multiple devices its erratas will be shared by all of them. Instead of creating a long list of device quirks, or waiting for the users to report errors in their hardware lets add a routine to add quirks based on the entity guid. Tested-by: HungNien Chen Signed-off-by: Ricardo Ribalda --- drivers/media/usb/uvc/uvc_driver.c | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c index 9c05776f11d1..c63ecfd4617d 100644 --- a/drivers/media/usb/uvc/uvc_driver.c +++ b/drivers/media/usb/uvc/uvc_driver.c @@ -1493,6 +1493,28 @@ static int uvc_parse_control(struct uvc_device *dev) return 0; } +static const struct uvc_entity_quirk { + u8 guid[16]; + u32 quirks; +} uvc_entity_quirk[] = { +}; + +static void uvc_entity_quirks(struct uvc_device *dev) +{ + struct uvc_entity *entity; + int i; + + list_for_each_entry(entity, &dev->entities, list) { + for (i = 0; i < ARRAY_SIZE(uvc_entity_quirk); i++) { + if (memcmp(entity->guid, uvc_entity_quirk[i].guid, + sizeof(entity->guid)) == 0) { + dev->quirks |= uvc_entity_quirk[i].quirks; + break; + } + } + } +} + /* ----------------------------------------------------------------------------- * Privacy GPIO */ @@ -2452,6 +2474,9 @@ static int uvc_probe(struct usb_interface *intf, goto error; } + /* Apply entity based quirks */ + uvc_entity_quirks(dev); + dev_info(&dev->udev->dev, "Found UVC %u.%02x device %s (%04x:%04x)\n", dev->uvc_version >> 8, dev->uvc_version & 0xff, udev->product ? udev->product : "", From patchwork Fri Dec 2 17:02:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 630502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC003C47089 for ; Fri, 2 Dec 2022 17:03:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234344AbiLBRDS (ORCPT ); Fri, 2 Dec 2022 12:03:18 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33526 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234247AbiLBRDG (ORCPT ); Fri, 2 Dec 2022 12:03:06 -0500 Received: from mail-ed1-x534.google.com (mail-ed1-x534.google.com [IPv6:2a00:1450:4864:20::534]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BA47510B7E for ; Fri, 2 Dec 2022 09:03:04 -0800 (PST) Received: by mail-ed1-x534.google.com with SMTP id z92so7318437ede.1 for ; Fri, 02 Dec 2022 09:03:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=lx0RTwbeF07JW5ZW7wkjZkyoNyHqWh3JjIKMCtW0xFs=; b=Q0BXyVxGC04ZBN43ZmgHVsV2ENEz3uNs5dN0tnRmaOqfDcdwgb1mIuFXyb1sTmKXMC 9UWDUYn9wSJfi3oRQuSgJFtf4CL8YicgTCkK6Q/LZgIwVe5Ifj5P0eHEj8We7j221b2o ekkFERis8zlYXF6hONPczkPBZgvS0Dla31oIc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lx0RTwbeF07JW5ZW7wkjZkyoNyHqWh3JjIKMCtW0xFs=; b=BCCOFXHr72rRyOE8N3Kjt9DPOVN1BsXFTNs2kuBZzTMvm3vPbRIPAJwPtRDBRetfp/ XOCB0XbyYOiIf1RrNHd7PHe8hJaAdM3zSIoESx4E1Ux11m+urCR37f27N0B97YneP0Y+ uZvnOE30iDZ/1c6vuElZqxyf5u71cFdsrh40Y+/BBxr8dQrdjgMMP3nDswZ8Jd8vAeRT QLa8/TuHZc9a9TkwQ4stgOM9+Em3uQLP0acerWLwNFqrHBlkw7WiLIhm4+crG1ArFTQp nmC4gkutglQ2FU5iao+tQk9fyhEAXYGJ2dihb0V6qAyyOkreqPA5LT/OPdiJYpo0F0Qy HXcw== X-Gm-Message-State: ANoB5plERCi+x4BFMWkAhh6nQt8tUOdEvIEWjNdOFSpCfRmZb6snXuVu OrnxbUAnI8twZva4YJYTG+YG+w== X-Google-Smtp-Source: AA0mqf4q0sbghPWT/ao8+lEAWXg07S1DilOZGtE48QLxxBSoVjHWdW4/whl/ERYkyo973iRwUd/xPA== X-Received: by 2002:a05:6402:5519:b0:461:e3f2:38bc with SMTP id fi25-20020a056402551900b00461e3f238bcmr55588823edb.149.1670000583256; Fri, 02 Dec 2022 09:03:03 -0800 (PST) Received: from alco.roam.corp.google.com (80.71.134.83.ipv4.parknet.dk. [80.71.134.83]) by smtp.gmail.com with ESMTPSA id sa22-20020a170906edb600b0073d7ab84375sm3249133ejb.92.2022.12.02.09.03.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Dec 2022 09:03:02 -0800 (PST) From: Ricardo Ribalda Date: Fri, 02 Dec 2022 18:02:44 +0100 Subject: [PATCH RESEND v2 4/8] media: uvcvideo: Quirk for invalid dev_sof in Logi C922 MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v2-4-0d7978a817cc@chromium.org> References: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> To: Mauro Carvalho Chehab , Laurent Pinchart Cc: "hn.chen" , linux-media@vger.kernel.org, Ricardo Ribalda , linux-kernel@vger.kernel.org X-Mailer: b4 0.11.0-dev-696ae X-Developer-Signature: v=1; a=openpgp-sha256; l=4938; i=ribalda@chromium.org; h=from:subject:message-id; bh=CXXqj68dbKHxKKYxVvCObfRHe2J96ojRxuGidUgj7F4=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjii+7hkWADKT+kFyE0yUtny4/lV/mcvdYK9d63u6m ndKhSOGJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCY4ovuwAKCRDRN9E+zzrEiNkxEA CCRQKBg62it0U3/7hc9gW6lfXAWNdhksJcn/l0I99NsevhUmMPhWe9tREUHjlZhiBibsEf6wqNDqWD c3ZhdrRU6H4t4oBEi0EhmU6KXLs2MfZHnNAW3iF+yqYqzrNipqh/FP3HGjByPdsZuyhnaXtPMyhlpw PdvVFETjuEQlwpT1SRUYW0Mibal9x56jERM9tAVU8NMDVETUpX6o4lfaYH9W1IApWJ1MEa/c9M10ON 16qyJ2Jr6aoylbB5ji2bG0UDwrktJoOYRZB2ze7vCTJD2CVvvbIn5950+kbexh/gNYG7cNg2gMyqxz LRaVkVe04lUh7DxFaeoAgmqoLmCLFGCInYhuyG21SlSiFtpiooPHuThhuiv1wXlvOksEEnTcvY7Lr0 dFP8MFAmrWELd7Be5MvWRw3Iu/2VE8xGYQvbrJstaSrGSowhfh0xgzA+OpXI2UjTIw9J+Qp97hxEmS TiMxjb00faZFwnago/YfIr778vU4TXv2Pdxy2oI94jZWHLRgYbe1//nu1fa3rOrQcaGQrByHagN+7r JSVAE5ZI7x04VE90O+TqMX34Y8qSNhyNlhDgOfu28OpxbOp0Mymu9sC/14Jd2dEWuL4FBst2LKF5GM qh4bukTXXuiJRNjIrQVFvizT/q5BlpdSrQLuVoAJRLMNhwSDaDwWd3c7s2RQ== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Logitech C922 internal sof does not increases at a stable rate of 1KHz. This causes that the device_sof and the host_sof run at different rates, breaking the clock domain conversion algorithm. Eg: 30 (6) [-] none 30 614400 B 21.245557 21.395214 34.133 fps ts mono/SoE 31 (7) [-] none 31 614400 B 21.275327 21.427246 33.591 fps ts mono/SoE 32 (0) [-] none 32 614400 B 21.304739 21.459256 34.000 fps ts mono/SoE 33 (1) [-] none 33 614400 B 21.334324 21.495274 33.801 fps ts mono/SoE * 34 (2) [-] none 34 614400 B 21.529237 21.527297 5.130 fps ts mono/SoE * 35 (3) [-] none 35 614400 B 21.649416 21.559306 8.321 fps ts mono/SoE 36 (4) [-] none 36 614400 B 21.678789 21.595320 34.045 fps ts mono/SoE ... 99 (3) [-] none 99 614400 B 23.542226 23.696352 33.541 fps ts mono/SoE 100 (4) [-] none 100 614400 B 23.571578 23.728404 34.069 fps ts mono/SoE 101 (5) [-] none 101 614400 B 23.601425 23.760420 33.504 fps ts mono/SoE * 102 (6) [-] none 102 614400 B 23.798324 23.796428 5.079 fps ts mono/SoE * 103 (7) [-] none 103 614400 B 23.916271 23.828450 8.478 fps ts mono/SoE 104 (0) [-] none 104 614400 B 23.945720 23.860479 33.957 fps ts mono/SoE Instead of disabling completely the hardware timestamping for such hardware we take the assumption that the package handling jitter is under 2ms and use the host_sof as dev_sof. For the hardware tested, it provides a much better timestamping than a pure software solution: https://ibb.co/D1HJJ4x https://ibb.co/8s9dBdk https://ibb.co/QC9MgVK This bug in the camera firmware has been confirmed by the vendor. lsusb -v Bus 001 Device 044: ID 046d:085c Logitech, Inc. C922 Pro Stream Webcam Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.00 bDeviceClass 239 Miscellaneous Device bDeviceSubClass 2 bDeviceProtocol 1 Interface Association bMaxPacketSize0 64 idVendor 0x046d Logitech, Inc. idProduct 0x085c C922 Pro Stream Webcam bcdDevice 0.16 iManufacturer 0 iProduct 2 C922 Pro Stream Webcam iSerial 1 80B912DF bNumConfigurations 1 Signed-off-by: Ricardo Ribalda --- drivers/media/usb/uvc/uvc_driver.c | 9 +++++++++ drivers/media/usb/uvc/uvc_video.c | 11 +++++++++-- drivers/media/usb/uvc/uvcvideo.h | 1 + 3 files changed, 19 insertions(+), 2 deletions(-) diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c index 80ef0f0e04b0..4512316c8748 100644 --- a/drivers/media/usb/uvc/uvc_driver.c +++ b/drivers/media/usb/uvc/uvc_driver.c @@ -2880,6 +2880,15 @@ static const struct usb_device_id uvc_ids[] = { .bInterfaceSubClass = 1, .bInterfaceProtocol = 0, .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_RESTORE_CTRLS_ON_INIT) }, + /* Logitech HD Pro Webcam C922 */ + { .match_flags = USB_DEVICE_ID_MATCH_DEVICE + | USB_DEVICE_ID_MATCH_INT_INFO, + .idVendor = 0x046d, + .idProduct = 0x085c, + .bInterfaceClass = USB_CLASS_VIDEO, + .bInterfaceSubClass = 1, + .bInterfaceProtocol = 0, + .driver_info = UVC_INFO_QUIRK(UVC_QUIRK_INVALID_DEVICE_SOF) }, /* Chicony CNF7129 (Asus EEE 100HE) */ { .match_flags = USB_DEVICE_ID_MATCH_DEVICE | USB_DEVICE_ID_MATCH_INT_INFO, diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index a5b184e71ad7..d387d6335344 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -529,13 +529,20 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, * data packages of the same frame contains the same sof. In that case * only the first one will match the host_sof. */ - dev_sof = get_unaligned_le16(&data[header_size - 2]); + if (stream->dev->quirks & UVC_QUIRK_INVALID_DEVICE_SOF) + dev_sof = usb_get_current_frame_number(stream->dev->udev); + else + dev_sof = get_unaligned_le16(&data[header_size - 2]); + if (dev_sof == stream->clock.last_sof) return; stream->clock.last_sof = dev_sof; - host_sof = usb_get_current_frame_number(stream->dev->udev); + if (stream->dev->quirks & UVC_QUIRK_INVALID_DEVICE_SOF) + host_sof = dev_sof; + else + host_sof = usb_get_current_frame_number(stream->dev->udev); time = uvc_video_get_time(); /* diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h index f395b67fe95a..e41289605d0e 100644 --- a/drivers/media/usb/uvc/uvcvideo.h +++ b/drivers/media/usb/uvc/uvcvideo.h @@ -213,6 +213,7 @@ #define UVC_QUIRK_FORCE_Y8 0x00000800 #define UVC_QUIRK_FORCE_BPP 0x00001000 #define UVC_QUIRK_IGNORE_EMPTY_TS 0x00002000 +#define UVC_QUIRK_INVALID_DEVICE_SOF 0x00004000 /* Format flags */ #define UVC_FMT_FLAG_COMPRESSED 0x00000001 From patchwork Fri Dec 2 17:02:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 630501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 669EBC47088 for ; Fri, 2 Dec 2022 17:03:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234369AbiLBRDX (ORCPT ); Fri, 2 Dec 2022 12:03:23 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234271AbiLBRDH (ORCPT ); Fri, 2 Dec 2022 12:03:07 -0500 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 307392D5 for ; Fri, 2 Dec 2022 09:03:06 -0800 (PST) Received: by mail-ed1-x535.google.com with SMTP id m19so7238128edj.8 for ; Fri, 02 Dec 2022 09:03:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=HzFzDnnX9pi7g8WQvEiGHdhzPZgy4R7i/X22w+LEdMI=; b=ZJW37MUF8AZ6gWULeaZCcqG6uUEz6GiDhcUhr3cuJ0TXqjmGQGQ/KaERArPSMmZYVQ v5NuRaE9CAuAEJk4+fiZAb3B1M/Uhb+ROQ6H7e+5zjdE42H2+rFoSmDtlIcorXvviC/o BoTkA6Z7SzYy+iuJ9UYXh4+8sw3+x7MODEjyU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HzFzDnnX9pi7g8WQvEiGHdhzPZgy4R7i/X22w+LEdMI=; b=an+e/X0m81meAksdmSC95TxXVtev7Oltrwjog5IsmTFaH/UCtBzD+U68XWr4WLlr9c Fa2dP4DyT/gFKvfDutWj8f8+9aOV3NuKK+TxY1PGkgDbp1p5DO/WQujnVgdOmLbEZFSs VHrEr0tBS5vEEeCKAognxH7pVMYH4M7WHRBV+vOslk7Zr/x0VTK7mnf+ZN+CtFGtsjvf T3OAdF88aWXhiJnrlmHB6IwH8JK9XJJk5ko89IQeAbekrSBaRsHaXiQH2bVW32HreHCk 4yonlfLJbgfWune+gpznynpUxSEYDatrdrDYj+h92e7kvBINXuVhRp+NGzocEFlm3CO5 JHfQ== X-Gm-Message-State: ANoB5pl3NSFYDNDbw+S0bIqubJtMYUUkivieDZ2X2ZpoX4g4MpRALI04 GqTJ4VAccwmbkqM+4BmORJKPXA== X-Google-Smtp-Source: AA0mqf7lHA+y3zQtGSgQQEOsOTynBhg3rVkN8KO3h1NcrWakisbHptVDJ4C48OnjG+xDUKY+RrqzHA== X-Received: by 2002:a05:6402:5405:b0:468:4cd4:f133 with SMTP id ev5-20020a056402540500b004684cd4f133mr50745505edb.18.1670000584730; Fri, 02 Dec 2022 09:03:04 -0800 (PST) Received: from alco.roam.corp.google.com (80.71.134.83.ipv4.parknet.dk. [80.71.134.83]) by smtp.gmail.com with ESMTPSA id sa22-20020a170906edb600b0073d7ab84375sm3249133ejb.92.2022.12.02.09.03.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Dec 2022 09:03:04 -0800 (PST) From: Ricardo Ribalda Date: Fri, 02 Dec 2022 18:02:46 +0100 Subject: [PATCH RESEND v2 6/8] media: uvcvideo: Allow hw clock updates with buffers not full MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v2-6-0d7978a817cc@chromium.org> References: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> To: Mauro Carvalho Chehab , Laurent Pinchart Cc: "hn.chen" , linux-media@vger.kernel.org, Ricardo Ribalda , linux-kernel@vger.kernel.org X-Mailer: b4 0.11.0-dev-696ae X-Developer-Signature: v=1; a=openpgp-sha256; l=1769; i=ribalda@chromium.org; h=from:subject:message-id; bh=3Fs3H+ze+0F22wwrtRMwFmFB8rGrRyFOzQuhxGsMqbQ=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjii++aFV/k9m1GdsfCWkMCRxITd6LDxE6DJa5Uc8e mePmB3eJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCY4ovvgAKCRDRN9E+zzrEiGCRD/ 9rwsjQhtVhBi0S2pt0k3Ny+IWzN/fublTh3yfjMsIAlrEucBUf88I9uq7z7D9OGqfqOs9R54/imZnS IRo1BrgKSqPK12ZuHcEDh/f2LP69JM9q9F9SYw2RN2mG8zi8YwO/BT5OP2qSFgLZLHTikUSuWoHuaF wUFN60AKltKKp48MCA/KTTH+XH28nx91yVyWJ5QaN2UzNIhOoA1U270tXxAsgwjxiw+68DghRTXqJ6 84ericULGTZtwenLdIlkHcw6ggOSIhJLTJPsT+J9aqizin7S/Dizv4pM8JaOlrynZHl/AWqc4+TT1F nC9h3DocTnNLizd3/vj/+hLFGnszvcZZJFwdhSrlDfkCNUXPBExLBTHn4s819WDGori6goKfZsj/zW vf7euVAUB3AcojasWJCoipCGIsGpT8fXw/24ko2BrSp4da1c9dm5QYItjdWE+nu5fHS9qNQZzAEhBx +x9750sF/rm7sAPW7IZ/uxRmrmLfHOYbkkCT/eAPUWVqzfFO0Xc3md0yuR6twJ1qX0LAjPzk5TT8Z8 F+ClhgplynV1MT0IVTeW/sO747rJZY1rRlu7gJGtpIV5dCkaFemOnlAy/+mxu3cEpSsoCw4JqFxtRf PUp4TWs3VGwSKH9rEs/pIAQIoZh0AFj/T7fpozfVUWYI8JMKAF1MxqRmsw0g== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org With UVC 1.5 we get as little as one clock sample per frame. Which means that it takes 32 frames to move from the software timestamp to the hardware timestamp method. This results in abrupt changes in the timestamping after 32 frames (~1 second), resulting in noticeable artifacts when used for encoding. With this patch we modify the update algorithm to work with whatever amount of values are available. Tested-by: HungNien Chen Signed-off-by: Ricardo Ribalda --- drivers/media/usb/uvc/uvc_video.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index 75c32e232f5d..7c6448c6d706 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -742,10 +742,10 @@ void uvc_video_clock_update(struct uvc_streaming *stream, spin_lock_irqsave(&clock->lock, flags); - if (clock->count < clock->size) + if (clock->count < 2) goto done; - first = &clock->samples[clock->head]; + first = &clock->samples[(clock->head - clock->count) % clock->size]; last = &clock->samples[(clock->head - 1) % clock->size]; /* First step, PTS to SOF conversion. */ @@ -760,6 +760,14 @@ void uvc_video_clock_update(struct uvc_streaming *stream, if (y2 < y1) y2 += 2048 << 16; + /* + * Have at least 1/4 of a second of timestamps before we + * try to do any calculation. Otherwise we do not have enough + * precission. + */ + if ((y2 - y1) < (256 << 16)) + goto done; + y = (u64)(y2 - y1) * (1ULL << 31) + (u64)y1 * (u64)x2 - (u64)y2 * (u64)x1; y = div_u64(y, x2 - x1); From patchwork Fri Dec 2 17:02:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 630500 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E319EC4321E for ; Fri, 2 Dec 2022 17:03:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234385AbiLBRD0 (ORCPT ); Fri, 2 Dec 2022 12:03:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35138 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234283AbiLBRDJ (ORCPT ); Fri, 2 Dec 2022 12:03:09 -0500 Received: from mail-ej1-x62b.google.com (mail-ej1-x62b.google.com [IPv6:2a00:1450:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3F53FA80AD for ; Fri, 2 Dec 2022 09:03:08 -0800 (PST) Received: by mail-ej1-x62b.google.com with SMTP id n21so12938107ejb.9 for ; Fri, 02 Dec 2022 09:03:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=/Z0mke9x4z8GIO16lOl+BgRgkcVeUrWg/yLKPpvbpk0=; b=KRuFz2z0S00H+Jjqi+ULpUWR9oqEccGYNJsAXZkGZBhcf65dYLS+1T60tut2/084H9 yFv8Ka0pIlw9bYdvYfPfIwQnaIMHbWOkBl+Jq/ice6ApLUxFH2OyJayzcbyfsoSRShMP mPkqZWZDlmcJ9vI5y6DfbmeuD9vPMDD0NZ0Ss= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Z0mke9x4z8GIO16lOl+BgRgkcVeUrWg/yLKPpvbpk0=; b=eT0eSifYJ36G/RrUt9iwrU0fPV/QVsWQ40PSP3OfbORhLtUY6SmW4+adbOSt0wp8R+ 65eDoPgE0zq2HGstqdfkg3L1lUgPuvRYp56La8hdf8hh2/Av3pbeZFVcAcfa+Wp5irx8 93adszqWYB2cFeSsTpJPPgD1+1+pjzWF34jNtRvIIL4q73aD9TP75q6cS3KlqX76IBfq Tvat6sNG3Sq0tA6nFi12hM6noeLxm/g/MXgM2ok1qoE3mtVFdt74P6gQ45+YKC8oyGb7 UhrCdAr4JlKlKS6yScaBajMiNvlAjOoFah8RIz9dsmjpjgHREFg2n9rAjkCnC3LfYYF9 rncQ== X-Gm-Message-State: ANoB5pmIrM6lxmSImmqiDyw4trMP8fl8FtAOxKFK8WaepI7hjMUOwYsd HtLS1yIa+HDnG5q9SW/AAKcNww== X-Google-Smtp-Source: AA0mqf4r6lyjL/nTXmoNE4mmFDtudjSLLgswHdy173L99fa3FCvYicPNl45W4w9oXR2zBxUxINOE7w== X-Received: by 2002:a17:906:706:b0:7ac:2e16:eb05 with SMTP id y6-20020a170906070600b007ac2e16eb05mr63176435ejb.26.1670000586670; Fri, 02 Dec 2022 09:03:06 -0800 (PST) Received: from alco.roam.corp.google.com (80.71.134.83.ipv4.parknet.dk. [80.71.134.83]) by smtp.gmail.com with ESMTPSA id sa22-20020a170906edb600b0073d7ab84375sm3249133ejb.92.2022.12.02.09.03.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Dec 2022 09:03:06 -0800 (PST) From: Ricardo Ribalda Date: Fri, 02 Dec 2022 18:02:48 +0100 Subject: [PATCH RESEND v2 8/8] media: uvcvideo: Fix hw timestampt handling for slow FPS MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v2-8-0d7978a817cc@chromium.org> References: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v2-0-0d7978a817cc@chromium.org> To: Mauro Carvalho Chehab , Laurent Pinchart Cc: "hn.chen" , linux-media@vger.kernel.org, Ricardo Ribalda , linux-kernel@vger.kernel.org X-Mailer: b4 0.11.0-dev-696ae X-Developer-Signature: v=1; a=openpgp-sha256; l=2142; i=ribalda@chromium.org; h=from:subject:message-id; bh=ZqBE4dWRk0InSyAxEr/auCSM1bp+38IChW6qVFv0hng=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjii/Bd0vLNfrMM135jh9PhB+8ydixKJHN5VNmGQ1w 6MdQC8WJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCY4ovwQAKCRDRN9E+zzrEiGdGD/ 9DoAVTWVQznsEgzOwpYpRyWMCIQBxtwlwKzaq1BnTIWye0goRJf5CjrA9bRJu+Z6+rx/+lp/RTMufc MVg5aMuukcsBzqLmyag5x0oDtpufG/8q8n2V/I+eejUQIFhJ/WgRBeRArQaT9Ib0AKH9ZXcZt/uMTa yGrm+vuxIPwgV1SuSPB5rIxtxwyUAfiq7qlxJArP79mrahtKd78OmPqqKX5q0h64kJzurNStsVKwbw qigdhxx3mcd8ABG20FwkR2+uV72DJqtQxmTcFtiNkThYx9dAVGxnF+hi70ALb9Dt0KpiB+eXme+mzQ xpLSqnfBKJ0kUOikWg+horEpNYRqVMGp5DWgZ6Z0FDz6JfeDzu5/MbrQwUfr8MocQajT2SS5/+S7OI PHexHL/gL++hSerdMcUMIX0G3RxG+sAJHjcA6G97N0Qinm/yb6WuYKEDAp/PtldnBNrbFCe9aCEIbJ UmRo+SfzDzWrRpf1gxmME/bl68ZyQzBIiZMhajxdBi7VazfWPFNfmNcOKR/94iaIn/S1wmMpFCZUZK TG3CY6KD/rbVr64gJDE/XQMagdmzJFdtwG2F0SGjMp621qyhW6nIA3ZFpAhJreZkPAdoKlHaet+tAm XP6RlzuWoO5Ab/c4qButbsLioCEdGQd1KqvgIlt3Z0r9bJEbSeajw8QWfEmg== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org In UVC 1.5, when working with FPS under 32, there is a chance that the circular buffer contains two dev_sof overflows, but the clock interpolator is only capable of handle a single overflow. Remove all the samples from the circular buffer that are two overflows old. Tested-by: HungNien Chen Signed-off-by: Ricardo Ribalda --- drivers/media/usb/uvc/uvc_video.c | 15 +++++++++++++++ drivers/media/usb/uvc/uvcvideo.h | 1 + 2 files changed, 16 insertions(+) diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index c81a8362d582..6b6bd521d6c2 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -471,6 +471,20 @@ static void uvc_video_clock_add_sample(struct uvc_clock *clock, spin_lock_irqsave(&clock->lock, flags); + /* Delete last overflows */ + if (clock->head == clock->last_sof_overflow) + clock->last_sof_overflow = -1; + + /* Handle overflows */ + if (clock->count > 0 && clock->last_sof > sample->dev_sof) { + /* Remove data from the last^2 overflows */ + if (clock->last_sof_overflow != -1) + clock->count = (clock->head - clock->last_sof_overflow) + % clock->count; + clock->last_sof_overflow = clock->head; + } + + /* Add sample */ memcpy(&clock->samples[clock->head], sample, sizeof(*sample)); clock->last_sof = sample->dev_sof; clock->head = (clock->head + 1) % clock->size; @@ -594,6 +608,7 @@ static void uvc_video_clock_reset(struct uvc_clock *clock) clock->head = 0; clock->count = 0; clock->last_sof = -1; + clock->last_sof_overflow = -1; clock->sof_offset = -1; } diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h index 14daa7111953..d8c520ce5a86 100644 --- a/drivers/media/usb/uvc/uvcvideo.h +++ b/drivers/media/usb/uvc/uvcvideo.h @@ -647,6 +647,7 @@ struct uvc_streaming { unsigned int head; unsigned int count; unsigned int size; + unsigned int last_sof_overflow; u16 last_sof; u16 sof_offset;