From patchwork Tue Sep 20 14:08:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 608147 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F77FC54EE9 for ; Tue, 20 Sep 2022 14:08:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231502AbiITOIf (ORCPT ); Tue, 20 Sep 2022 10:08:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41654 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231416AbiITOIb (ORCPT ); Tue, 20 Sep 2022 10:08:31 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37EA156BB9 for ; Tue, 20 Sep 2022 07:08:29 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id y17so6465887ejo.6 for ; Tue, 20 Sep 2022 07:08:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date; bh=GNRZeqqBofwFFgvrtZo64pHwe4vN8QcPuwanc3P7m18=; b=dnl1UAEIow118xOMF3d6RV2uDS3Ampdbfn1MZ3MdrPkxuKps/M901LSwgUR2f7gSnx Aqs9MQURHmRNMKI3ptqiA5ssM3VAnUCI2/E+Gqnc9KxiVfJ7pqRU6QsdWr5LYXTyvyVS avsZfKcyJwxWSCWuBvLqgXVSUib0DJUHZ5MBY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date; bh=GNRZeqqBofwFFgvrtZo64pHwe4vN8QcPuwanc3P7m18=; b=4ivT3TYnvsoLO48tA4YYCMsesI/r8JUu+ozOEodo1ZSg8NcdJfzs+qJfVezcDdzpK5 CUNHvjU5EGAN7pAyL09psiAMhNJhOFFMumpVc/2WdckFvlxUNdD330feUWuuSlLt1UQJ nnzh63v5Y+OKazOQlfk72UnJSNVheSX3PSHUxl3P0m6lkVVNpeWCDEn+jgPlJ1GwRnI9 NAPIbm3BEcuTsHYOpvqvbTT+9WtmV4hE6oz8SH2FQIm6aa8/C/7QI1y+mSY97UNz2Tr+ 8BURAiaxM3D5xc3C+A8q+nezAdBJn+6h3AIGnTQ00jQBb0ki7jIXNkMk+RC56bkKifOv 9e3Q== X-Gm-Message-State: ACrzQf1L0uf0iS1vUXF3KSe72Hw77HHP25hk5F82a0WSUczePPYOpja/ m8b6NdWTeBQYuvUBp1DjcOfJ5g== X-Google-Smtp-Source: AMsMyM5NcNM0Hllkv6SuG/1QMVy1P2GJGEUX/BF/5aVADEeqmQHmsphKLxgXR1Wte8bFdAGpZWm95A== X-Received: by 2002:a17:907:3f09:b0:780:330d:d907 with SMTP id hq9-20020a1709073f0900b00780330dd907mr17967603ejc.23.1663682907241; Tue, 20 Sep 2022 07:08:27 -0700 (PDT) Received: from alco.roam.corp.google.com ([2620:0:1059:10:935d:52b0:7461:88e1]) by smtp.gmail.com with ESMTPSA id j17-20020a17090623f100b0077016f4c6d4sm936564ejg.55.2022.09.20.07.08.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Sep 2022 07:08:26 -0700 (PDT) From: Ricardo Ribalda Date: Tue, 20 Sep 2022 16:08:07 +0200 Subject: [PATCH v1 1/8] media: uvc: Extend documentation of uvc_video_clock_decode() MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v1-1-e9c14b258404@chromium.org> References: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> To: Laurent Pinchart , Mauro Carvalho Chehab Cc: "hn.chen" , Ricardo Ribalda , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org X-Mailer: b4 0.11.0-dev-d93f8 X-Developer-Signature: v=1; a=openpgp-sha256; l=962; i=ribalda@chromium.org; h=from:subject:message-id; bh=HA7gMmyIlso0y4JghGpn78Jy/w8ZP1mfGIQm1eY7vlA=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjKclLuC0+GprVRxBvAJv8LhmrYKAks0uKapIO0d7+ k/bas+eJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCYynJSwAKCRDRN9E+zzrEiK66D/ 0eAwIh96q1kC3kaW1GM7CZDtt98KXc5Fyz2oK4QdJdaRA2/zxVxQ3lnLFw6cvs/XI4nawjYU4prYEt b6gYhY7vPvLyoWbigdRjcwSxxj9Bn9B+p34lMimLLV+sTw0MVlDdf84xLmSxeFffg+89JaJl62Cs+o Gc3notu6stl92B4UP6bsueh/XsifvFj/EqGYO5crJBUawtuF2CHrJomFRvNfmul72kockWrm12nFSS nmga38/SDIz+IyubUOfV1feuhA/VC/Q45Lith1ezwQNgUFCLPcAVJ6hQj+DeWnWL0sJQyV73NwO3yq tRgZaHwHz/VjBBQ8IpVB7e38+a4v+mQILoO58jRaT/NYeHqLS9HpA4DoyyrxlCdxw6sFAsi11CZ6GX HkwdVX2A7CBgEQGXfjHLsYjWoy0kP/EDlECWDtoCwVx2eWO/gKPopJI7rzY+j2GLPRgqNyPKJHK5zI 83WXnMdvWu2RgBRsO8BFonmUgEl1Gka2T4/g2l0HgSJTYcHxDLuGX0yDNqDBltuKJIlREgyz6AoeO3 CptJ/WVpT6rEE04NDCvX5xXe0Ze/zP7C9rHkegbbj7epu7TC+f/n0LglqBG9ousJZTb78lmoOlcqKz pPtsHvkicSFlDeD6F/qKk6kCuxYkHQr2CxPb39ALNuFQrcXI7wFXqOAu37rg== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Make a explicit reference to uvc 1.5, explaining how the algorithm supports the different behaviour of uvc 1.1 and 1.5. Tested-by: hn.chen Signed-off-by: Ricardo Ribalda diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index 170a008f4006..ab56e65ca324 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -517,6 +517,9 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, /* * To limit the amount of data, drop SCRs with an SOF identical to the * previous one. + * This filtering is also needed for supporting UVC 1.5. Where all the + * data packages of the same frame contains the same sof. In that case + * only the first one will match the host_sof. */ dev_sof = get_unaligned_le16(&data[header_size - 2]); if (dev_sof == stream->clock.last_sof) From patchwork Tue Sep 20 14:08:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 608146 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 98135ECAAD8 for ; Tue, 20 Sep 2022 14:08:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231503AbiITOIr (ORCPT ); Tue, 20 Sep 2022 10:08:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41714 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231366AbiITOIe (ORCPT ); Tue, 20 Sep 2022 10:08:34 -0400 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [IPv6:2a00:1450:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 296055A151 for ; Tue, 20 Sep 2022 07:08:32 -0700 (PDT) Received: by mail-ej1-x62e.google.com with SMTP id u9so6473000ejy.5 for ; Tue, 20 Sep 2022 07:08:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date; bh=p6VhhvjRu0kK7K1Hcnt66imAt+uopG+auRNJJJNMyOY=; b=CJsiqSTfMdfC7jQHgYXCN+C4as3ChCCL+3/PsNJJU0SwHyNnYpb5DMzKtdxvreeIMX OJuoWX6zpmgy5pBj3YuJavTlMgGY9ig+B30MLBTBwVnpx9IzTBeFThnBSa7Gmsj8eq+R jHfwBXqxcyQuEJuTLM3TjOZq0JCj3D9JFlKh4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date; bh=p6VhhvjRu0kK7K1Hcnt66imAt+uopG+auRNJJJNMyOY=; b=ypkQcEr2/+dVU+aMKRJWrZk6jNKzUvoqC/l99+PNaJ46w9xyN5H1t01SPVFN+ZuRRF 6eJgILG0p2QDuMh4R3WPXAwb1jwjl0ois0V9UREXchI5gIxUDSofVZ9AaUBX++SihfX1 Y25z4WrKroDjLo2m76oI8SBweYx/Pa/9InX79sCiRNYSfZXSMT/avXS8A9RS7gVYS71G 5I/qLkd1AC6MTBIG+oOXul7cvtWl4plGJ1dq91GiVVZNHU62P/L6FxAVGgaTpI/Jgbre xt8lCCyMXfPc7rij4A/DMW+3eWe1rJ93nC4cVec7yjfi5UZFFrcRQgjfynKhhayMc/Wx b5lQ== X-Gm-Message-State: ACrzQf0ewxEqs4sCUTjqhMnik/diHtX96OX6qBOYYNiflt++2nHfoJes Vfi+cVPRctN+WNSjKaF3PpKx2w== X-Google-Smtp-Source: AMsMyM4iT+6qCsM7u6VxivAii1dUx8n1OMlP6yZDc1sGIaV/za1Roa/PTfYtH1jJtFJeUp73C0Qa2g== X-Received: by 2002:a17:907:9610:b0:77b:a7cd:839b with SMTP id gb16-20020a170907961000b0077ba7cd839bmr17379946ejc.60.1663682910391; Tue, 20 Sep 2022 07:08:30 -0700 (PDT) Received: from alco.roam.corp.google.com ([2620:0:1059:10:935d:52b0:7461:88e1]) by smtp.gmail.com with ESMTPSA id j17-20020a17090623f100b0077016f4c6d4sm936564ejg.55.2022.09.20.07.08.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Sep 2022 07:08:29 -0700 (PDT) From: Ricardo Ribalda Date: Tue, 20 Sep 2022 16:08:09 +0200 Subject: [PATCH v1 3/8] media: uvc: Create UVC_QUIRK_IGNORE_EMPTY_TS quirk MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v1-3-e9c14b258404@chromium.org> References: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> To: Laurent Pinchart , Mauro Carvalho Chehab Cc: "hn.chen" , Ricardo Ribalda , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org X-Mailer: b4 0.11.0-dev-d93f8 X-Developer-Signature: v=1; a=openpgp-sha256; l=34637; i=ribalda@chromium.org; h=from:subject:message-id; bh=qf+BX1U2ysQFPEHaekF+yNvYD+9tgnIh4nUntliIbnQ=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjKclP9kstBm6PLELEAkutsd3Vx4UGCMs7s0fgxI0O HzRmRdqJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCYynJTwAKCRDRN9E+zzrEiA0zEA CGaxKpPCHNMbq6rJtuzH2BGi8mV3p4pBdhulj0yKJyAmN4ywWFwo2XKPHJL1qlyiceFqlIqWuW0xXF qxY8MKqxYG/ddwXCHQSYqn12P/Es7mL6rWyNIrU2pgbeopk+Skb6DB9uLGHTXKPRoYLQv2bwcqeTte Z1n+sTxoQw9997ZPJYIiEGhxVJBPeKuEYMLqHF6Ezn+zLEsRTrZw4UHzBShqhRtBZ3kTlQ+7CuszzY wSs3vfJ66SRkUBRy58CrghWu6x/VO9CegGiPuCyHwt/tQQciHgBuW+4pbVBX+nTDukoSbfAz8ntj7G j1HCAKN0rJqiKQAF0Y/jwZgWGL7Na0mo32GoJtG+PTdSsbGS2SpmmjhP9mwTOFZIXmo/IewiyLj1bt K3vRPWxibAv3b98aWO4P8iOwuzQOl/QoICfPxsKpXIfLfTm0l0+OcSBGQ4AtdjufYeaHuIhxpQqhMY 7TgwGXzHEjpVXTs0/Vs4TliF2TZXS7+XNn4dS9dS8EJA+Nz4qftSImRF29EpOQ7diNCtD5ElIAs8kw uC7vwWtpQlumeZUhrK5ys7okniM2L0/FToSCnwl7rR2HxS07JVDQEab0J0BX2f0HKItlLZ5jKksHPp Gl3npveGARPXWZoBaU7JNzrJKKQahrD3au4+tTvTWCHhW3a066CXjYod5ggg== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Some Sunplus cameras took a borderline interpretation of the UVC 1.5 standard, and fill the PTS and SCR fields with invalid data if the package does not contain data. "STC must be captured when the first video data of a video frame is put on the USB bus." Eg: buffer: 0xa7755c00 len 000012 header:0x8c stc 00000000 sof 0000 pts 00000000 buffer: 0xa7755c00 len 000012 header:0x8c stc 00000000 sof 0000 pts 00000000 buffer: 0xa7755c00 len 000668 header:0x8c stc 73779dba sof 070c pts 7376d37a This borderline/buggy interpretation has been implemented in a variety of devices, from directly Sunplus and from other OEMs that rebrand Sunplus products. Luckily we can identify the affected modules by looking at the guid of one of the extension units: VideoControl Interface Descriptor: guidExtensionCode {82066163-7050-ab49-b8cc-b3855e8d221d} This patch adds a new quirk to take care of this. Complete lsusb -v of one of the affected cameras: Bus 001 Device 003: ID 1bcf:2a01 Sunplus Innovation Technology Inc. Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.01 bDeviceClass 239 Miscellaneous Device bDeviceSubClass 2 ? bDeviceProtocol 1 Interface Association bMaxPacketSize0 64 idVendor 0x1bcf Sunplus Innovation Technology Inc. idProduct 0x2a01 bcdDevice 0.02 iManufacturer 1 SunplusIT Inc iProduct 2 HanChen Wise Camera iSerial 3 01.00.00 bNumConfigurations 1 Configuration Descriptor: bLength 9 bDescriptorType 2 wTotalLength 935 bNumInterfaces 4 bConfigurationValue 1 iConfiguration 0 bmAttributes 0x80 (Bus Powered) MaxPower 500mA Interface Association: bLength 8 bDescriptorType 11 bFirstInterface 0 bInterfaceCount 2 bFunctionClass 14 Video bFunctionSubClass 3 Video Interface Collection bFunctionProtocol 0 iFunction 4 HanChen Wise Camera Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 0 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 1 Video Control bInterfaceProtocol 1 iInterface 4 HanChen Wise Camera VideoControl Interface Descriptor: bLength 13 bDescriptorType 36 bDescriptorSubtype 1 (HEADER) bcdUVC 1.50 wTotalLength 111 dwClockFrequency 48.000000MHz bInCollection 1 baInterfaceNr( 0) 1 VideoControl Interface Descriptor: bLength 18 bDescriptorType 36 bDescriptorSubtype 2 (INPUT_TERMINAL) bTerminalID 1 wTerminalType 0x0201 Camera Sensor bAssocTerminal 0 iTerminal 0 wObjectiveFocalLengthMin 0 wObjectiveFocalLengthMax 0 wOcularFocalLength 0 bControlSize 3 bmControls 0x0030000e Auto-Exposure Mode Auto-Exposure Priority Exposure Time (Absolute) VideoControl Interface Descriptor: bLength 13 bDescriptorType 36 bDescriptorSubtype 5 (PROCESSING_UNIT) bUnitID 2 bSourceID 1 wMaxMultiplier 16384 bControlSize 3 bmControls 0x0000157f Brightness Contrast Hue Saturation Sharpness Gamma White Balance Temperature Backlight Compensation Power Line Frequency White Balance Temperature, Auto iProcessing 0 bmVideoStandards 0x 0 VideoControl Interface Descriptor: bLength 29 bDescriptorType 36 bDescriptorSubtype 6 (EXTENSION_UNIT) bUnitID 3 guidExtensionCode {0fb885c3-68c2-4547-90f7-8f47579d95fc} bNumControl 5 bNrPins 1 baSourceID( 0) 2 bControlSize 4 bmControls( 0) 0x1f bmControls( 1) 0x00 bmControls( 2) 0x00 bmControls( 3) 0x00 iExtension 0 VideoControl Interface Descriptor: bLength 29 bDescriptorType 36 bDescriptorSubtype 6 (EXTENSION_UNIT) bUnitID 4 guidExtensionCode {63610682-5070-49ab-b8cc-b3855e8d221d} bNumControl 20 bNrPins 1 baSourceID( 0) 3 bControlSize 4 bmControls( 0) 0xff bmControls( 1) 0xff bmControls( 2) 0x71 bmControls( 3) 0x00 iExtension 0 VideoControl Interface Descriptor: bLength 9 bDescriptorType 36 bDescriptorSubtype 3 (OUTPUT_TERMINAL) bTerminalID 5 wTerminalType 0x0101 USB Streaming bAssocTerminal 0 bSourceID 4 iTerminal 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x87 EP 7 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0010 1x 16 bytes bInterval 8 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 0 bNumEndpoints 0 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 VideoStreaming Interface Descriptor: bLength 15 bDescriptorType 36 bDescriptorSubtype 1 (INPUT_HEADER) bNumFormats 2 wTotalLength 335 bEndPointAddress 129 bmInfo 0 bTerminalLink 5 bStillCaptureMethod 1 bTriggerSupport 0 bTriggerUsage 0 bControlSize 1 bmaControls( 0) 11 bmaControls( 1) 11 VideoStreaming Interface Descriptor: bLength 11 bDescriptorType 36 bDescriptorSubtype 6 (FORMAT_MJPEG) bFormatIndex 1 bNumFrameDescriptors 7 bFlags 1 Fixed-size samples: Yes bDefaultFrameIndex 1 bAspectRatioX 0 bAspectRatioY 0 bmInterlaceFlags 0x00 Interlaced stream or variable: No Fields per frame: 1 fields Field 1 first: No Field pattern: Field 1 only bCopyProtect 0 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 1 bmCapabilities 0x01 Still image supported wWidth 1920 wHeight 1080 dwMinBitRate 995328000 dwMaxBitRate 995328000 dwMaxVideoFrameBufferSize 4147200 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 2 bmCapabilities 0x01 Still image supported wWidth 1280 wHeight 720 dwMinBitRate 442368000 dwMaxBitRate 442368000 dwMaxVideoFrameBufferSize 1843200 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 3 bmCapabilities 0x01 Still image supported wWidth 640 wHeight 480 dwMinBitRate 147456000 dwMaxBitRate 147456000 dwMaxVideoFrameBufferSize 614400 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 4 bmCapabilities 0x01 Still image supported wWidth 640 wHeight 360 dwMinBitRate 110592000 dwMaxBitRate 110592000 dwMaxVideoFrameBufferSize 460800 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 5 bmCapabilities 0x01 Still image supported wWidth 352 wHeight 288 dwMinBitRate 48660480 dwMaxBitRate 48660480 dwMaxVideoFrameBufferSize 202752 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 6 bmCapabilities 0x01 Still image supported wWidth 320 wHeight 240 dwMinBitRate 36864000 dwMaxBitRate 36864000 dwMaxVideoFrameBufferSize 153600 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 7 (FRAME_MJPEG) bFrameIndex 7 bmCapabilities 0x01 Still image supported wWidth 176 wHeight 144 dwMinBitRate 12165120 dwMaxBitRate 12165120 dwMaxVideoFrameBufferSize 50688 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 6 bDescriptorType 36 bDescriptorSubtype 13 (COLORFORMAT) bColorPrimaries 1 (BT.709,sRGB) bTransferCharacteristics 1 (BT.709) bMatrixCoefficients 4 (SMPTE 170M (BT.601)) VideoStreaming Interface Descriptor: bLength 27 bDescriptorType 36 bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED) bFormatIndex 2 bNumFrameDescriptors 2 guidFormat {32595559-0000-0010-8000-00aa00389b71} bBitsPerPixel 16 bDefaultFrameIndex 1 bAspectRatioX 0 bAspectRatioY 0 bmInterlaceFlags 0x00 Interlaced stream or variable: No Fields per frame: 2 fields Field 1 first: No Field pattern: Field 1 only bCopyProtect 0 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 5 (FRAME_UNCOMPRESSED) bFrameIndex 1 bmCapabilities 0x01 Still image supported wWidth 640 wHeight 480 dwMinBitRate 147456000 dwMaxBitRate 147456000 dwMaxVideoFrameBufferSize 614400 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 30 bDescriptorType 36 bDescriptorSubtype 5 (FRAME_UNCOMPRESSED) bFrameIndex 2 bmCapabilities 0x01 Still image supported wWidth 640 wHeight 360 dwMinBitRate 110592000 dwMaxBitRate 110592000 dwMaxVideoFrameBufferSize 460800 dwDefaultFrameInterval 333333 bFrameIntervalType 1 dwFrameInterval( 0) 333333 VideoStreaming Interface Descriptor: bLength 6 bDescriptorType 36 bDescriptorSubtype 13 (COLORFORMAT) bColorPrimaries 1 (BT.709,sRGB) bTransferCharacteristics 1 (BT.709) bMatrixCoefficients 4 (SMPTE 170M (BT.601)) Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 1 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x00c0 1x 192 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 2 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0180 1x 384 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 3 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 4 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0280 1x 640 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 5 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0320 1x 800 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 6 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x03b0 1x 944 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 7 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0a80 2x 640 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 8 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0b20 2x 800 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 9 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0be0 2x 992 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 10 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x13c0 3x 960 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 1 bAlternateSetting 11 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x81 EP 1 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x13fc 3x 1020 bytes bInterval 1 Interface Association: bLength 8 bDescriptorType 11 bFirstInterface 2 bInterfaceCount 2 bFunctionClass 14 Video bFunctionSubClass 3 Video Interface Collection bFunctionProtocol 0 iFunction 11 HanChen IR Camera Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 2 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 1 Video Control bInterfaceProtocol 1 iInterface 11 HanChen IR Camera VideoControl Interface Descriptor: bLength 13 bDescriptorType 36 bDescriptorSubtype 1 (HEADER) bcdUVC 1.50 wTotalLength 111 dwClockFrequency 48.000000MHz bInCollection 1 baInterfaceNr( 0) 3 VideoControl Interface Descriptor: bLength 18 bDescriptorType 36 bDescriptorSubtype 2 (INPUT_TERMINAL) bTerminalID 1 wTerminalType 0x0201 Camera Sensor bAssocTerminal 0 iTerminal 0 wObjectiveFocalLengthMin 0 wObjectiveFocalLengthMax 0 wOcularFocalLength 0 bControlSize 3 bmControls 0x00300000 VideoControl Interface Descriptor: bLength 13 bDescriptorType 36 bDescriptorSubtype 5 (PROCESSING_UNIT) bUnitID 2 bSourceID 1 wMaxMultiplier 16384 bControlSize 3 bmControls 0x00000000 iProcessing 0 bmVideoStandards 0x 0 VideoControl Interface Descriptor: bLength 29 bDescriptorType 36 bDescriptorSubtype 6 (EXTENSION_UNIT) bUnitID 7 guidExtensionCode {0f3f95dc-2632-4c4e-92c9-a04782f43bc8} bNumControl 4 bNrPins 1 baSourceID( 0) 2 bControlSize 4 bmControls( 0) 0xe0 bmControls( 1) 0x01 bmControls( 2) 0x00 bmControls( 3) 0x00 iExtension 0 VideoControl Interface Descriptor: bLength 29 bDescriptorType 36 bDescriptorSubtype 6 (EXTENSION_UNIT) bUnitID 4 guidExtensionCode {63610682-5070-49ab-b8cc-b3855e8d221d} bNumControl 27 bNrPins 1 baSourceID( 0) 7 bControlSize 4 bmControls( 0) 0xff bmControls( 1) 0xff bmControls( 2) 0x77 bmControls( 3) 0x1f iExtension 0 VideoControl Interface Descriptor: bLength 9 bDescriptorType 36 bDescriptorSubtype 3 (OUTPUT_TERMINAL) bTerminalID 5 wTerminalType 0x0101 USB Streaming bAssocTerminal 0 bSourceID 4 iTerminal 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x84 EP 4 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0010 1x 16 bytes bInterval 8 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 3 bAlternateSetting 0 bNumEndpoints 0 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 VideoStreaming Interface Descriptor: bLength 14 bDescriptorType 36 bDescriptorSubtype 1 (INPUT_HEADER) bNumFormats 1 wTotalLength 101 bEndPointAddress 130 bmInfo 0 bTerminalLink 5 bStillCaptureMethod 1 bTriggerSupport 0 bTriggerUsage 0 bControlSize 1 bmaControls( 0) 27 VideoStreaming Interface Descriptor: bLength 27 bDescriptorType 36 bDescriptorSubtype 4 (FORMAT_UNCOMPRESSED) bFormatIndex 1 bNumFrameDescriptors 1 guidFormat {00000032-0002-0010-8000-00aa00389b71} bBitsPerPixel 8 bDefaultFrameIndex 1 bAspectRatioX 0 bAspectRatioY 0 bmInterlaceFlags 0x00 Interlaced stream or variable: No Fields per frame: 2 fields Field 1 first: No Field pattern: Field 1 only bCopyProtect 0 VideoStreaming Interface Descriptor: bLength 54 bDescriptorType 36 bDescriptorSubtype 5 (FRAME_UNCOMPRESSED) bFrameIndex 1 bmCapabilities 0x01 Still image supported wWidth 624 wHeight 352 dwMinBitRate 52715520 dwMaxBitRate 52715520 dwMaxVideoFrameBufferSize 439296 dwDefaultFrameInterval 333333 bFrameIntervalType 7 dwFrameInterval( 0) 333333 dwFrameInterval( 1) 400000 dwFrameInterval( 2) 500000 dwFrameInterval( 3) 666666 dwFrameInterval( 4) 1000000 dwFrameInterval( 5) 1333333 dwFrameInterval( 6) 2000000 VideoStreaming Interface Descriptor: bLength 6 bDescriptorType 36 bDescriptorSubtype 13 (COLORFORMAT) bColorPrimaries 1 (BT.709,sRGB) bTransferCharacteristics 1 (BT.709) bMatrixCoefficients 4 (SMPTE 170M (BT.601)) Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 3 bAlternateSetting 1 bNumEndpoints 1 bInterfaceClass 14 Video bInterfaceSubClass 2 Video Streaming bInterfaceProtocol 1 iInterface 0 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x82 EP 2 IN bmAttributes 5 Transfer Type Isochronous Synch Type Asynchronous Usage Type Data wMaxPacketSize 0x0a80 2x 640 bytes bInterval 1 Binary Object Store Descriptor: bLength 5 bDescriptorType 15 wTotalLength 33 bNumDeviceCaps 1 Platform Device Capability: bLength 28 bDescriptorType 16 bDevCapabilityType 5 bReserved 0 PlatformCapabilityUUID {d8dd60df-4589-4cc7-9cd2-659d9e648a9f} CapabilityData[0] 0x00 CapabilityData[1] 0x00 CapabilityData[2] 0x00 CapabilityData[3] 0x0a CapabilityData[4] 0x6a CapabilityData[5] 0x03 CapabilityData[6] 0x01 CapabilityData[7] 0x00 Device Status: 0x0000 (Bus Powered) Tested-by: hn.chen Signed-off-by: Ricardo Ribalda diff --git a/drivers/media/usb/uvc/uvc_driver.c b/drivers/media/usb/uvc/uvc_driver.c index c63ecfd4617d..80ef0f0e04b0 100644 --- a/drivers/media/usb/uvc/uvc_driver.c +++ b/drivers/media/usb/uvc/uvc_driver.c @@ -1497,6 +1497,17 @@ static const struct uvc_entity_quirk { u8 guid[16]; u32 quirks; } uvc_entity_quirk[] = { + /* + * Some SunPlus uvc 1.5 device firmware expects that packages with + * no frame data are ignored by the host. + * Therefore it does not clear the PTS/SCR bits in the header, and + * breaks the timestamp decode algorithm. + */ + { + .guid = {0x82, 0x06, 0x61, 0x63, 0x70, 0x50, 0xab, 0x49, + 0xb8, 0xcc, 0xb3, 0x85, 0x5e, 0x8d, 0x22, 0x1d}, + .quirks = UVC_QUIRK_IGNORE_EMPTY_TS, + }, }; static void uvc_entity_quirks(struct uvc_device *dev) diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index ab56e65ca324..a5b184e71ad7 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -500,6 +500,14 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, if (len < header_size) return; + /* + * Ignore the hardware timestamp on frames with no data on + * miss-behaving devices. + */ + if (stream->dev->quirks & UVC_QUIRK_IGNORE_EMPTY_TS && + len == header_size) + return; + /* * Extract the timestamps: * diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h index 24c911aeebce..f395b67fe95a 100644 --- a/drivers/media/usb/uvc/uvcvideo.h +++ b/drivers/media/usb/uvc/uvcvideo.h @@ -212,6 +212,7 @@ #define UVC_QUIRK_RESTORE_CTRLS_ON_INIT 0x00000400 #define UVC_QUIRK_FORCE_Y8 0x00000800 #define UVC_QUIRK_FORCE_BPP 0x00001000 +#define UVC_QUIRK_IGNORE_EMPTY_TS 0x00002000 /* Format flags */ #define UVC_FMT_FLAG_COMPRESSED 0x00000001 From patchwork Tue Sep 20 14:08:12 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 608144 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC3FFC6FA8E for ; Tue, 20 Sep 2022 14:08:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231579AbiITOIx (ORCPT ); Tue, 20 Sep 2022 10:08:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41686 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231521AbiITOIi (ORCPT ); Tue, 20 Sep 2022 10:08:38 -0400 Received: from mail-ed1-x52d.google.com (mail-ed1-x52d.google.com [IPv6:2a00:1450:4864:20::52d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D707A57267 for ; Tue, 20 Sep 2022 07:08:34 -0700 (PDT) Received: by mail-ed1-x52d.google.com with SMTP id b35so4109860edf.0 for ; Tue, 20 Sep 2022 07:08:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date; bh=2m5NjiSQ3UFcFg+RUZgWJGP2l7gs/3+ArugwLslOKeE=; b=Qog8O021pF2YpCNSfKIFSrzglOkUJfYjp/+iykdEDhA2rUzeTskHBghLNIn667tEIx YRjgd/2WjIW8lAE5Te5Yd/KuQ6nhDQh3n/Gy4igXH4sV1xPYrK3VLZWa7iDQxRFkKDny LNH7AtkR12pvioQ8Xepwi/BClUJDOUtspbh0A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date; bh=2m5NjiSQ3UFcFg+RUZgWJGP2l7gs/3+ArugwLslOKeE=; b=n6qa5ZvfM/VkI9y6gJisUXuQWYKKKyFCMh5UeWcc5JU5mTP1rsfhOt32wOLREEeQUM tTOxJQ5cytMZeeosp1RWkWzkdiec3RrV6ouzNo8OqaZbsLo1tPz2jd+XrEjUuwdhWMKh Cl5zYFWRHkmMkufG6FtF7XpWfYpLRXUsiRTUdD8ioalkOnkpGIMQz42mBcmkUjStkCW6 iHVGYUOqxdeU9HfsOorLU1FzS+rzPljKfeJBPwV75iCjp52LJxYxbdXNznVGBNsFzTZs OvGzWEsC3bYTA2FmseJYUhgsom3q8IWBcfzCbgFwQzfe4AVcEzZxg7zQG8eLeRpmgpus 5muw== X-Gm-Message-State: ACrzQf3lQMKuLoQMfaTMbEPbq+iyLpFAfGzpuwvA0ypvrAi3Iz9Axp7k P7L6zl38bAF4F8Yi+2AyOtLNvg== X-Google-Smtp-Source: AMsMyM7Kttfu3Wsv8rxi19u3YeiT1tiXz9wHrvoFQFz4kKSYvS3PuNb1O521uebjBc0XlYZE2HrKoQ== X-Received: by 2002:a05:6402:1b06:b0:44e:a073:1dd8 with SMTP id by6-20020a0564021b0600b0044ea0731dd8mr19527488edb.391.1663682913102; Tue, 20 Sep 2022 07:08:33 -0700 (PDT) Received: from alco.roam.corp.google.com ([2620:0:1059:10:935d:52b0:7461:88e1]) by smtp.gmail.com with ESMTPSA id j17-20020a17090623f100b0077016f4c6d4sm936564ejg.55.2022.09.20.07.08.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Sep 2022 07:08:32 -0700 (PDT) From: Ricardo Ribalda Date: Tue, 20 Sep 2022 16:08:12 +0200 Subject: [PATCH v1 6/8] media: uvcvideo: Allow hw clock updates with buffers not full MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v1-6-e9c14b258404@chromium.org> References: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> To: Laurent Pinchart , Mauro Carvalho Chehab Cc: "hn.chen" , Ricardo Ribalda , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org X-Mailer: b4 0.11.0-dev-d93f8 X-Developer-Signature: v=1; a=openpgp-sha256; l=1139; i=ribalda@chromium.org; h=from:subject:message-id; bh=e4J+nrDAhgtu8XgKC9q34sJueku2ku3RgCFGTHQq+rM=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjKclVPstkH2tqWY0nEWxBnkdg51HwzIUCHCT2SJIh 90evtTuJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCYynJVQAKCRDRN9E+zzrEiMh4EA CMsLkuVpPkdk5Dw96HZtyoD4QFh88Ekmm5Ugj05gyKL5+RNVrwIuaEvTQgneLKQ5i2ARyq2ZdnWmFt Lx/d+P7OXGibbi38Jqllt19Jv7qtFI0eD/ldJ1FuLDPsz/sYBgrD4yNQqHlSTlHoX6rN6FHCKVpG+f QXJeagc1ChxU9gIb+P0lGvomrAq1qIO3RHjipwd2d4Qnyr5GQ7hXxlw/jmHK0HpbJQXG8wp7iotWVU bhwQY9WXeHJEZ6K4I345bFFpeQGQw7y+koAtpdO2wcCdYT4NnIVMBZELONHgKilYgd9mA3skOzMRir u4j1W0faE/Ml5Wq8BnrtUNimXrfLGebTUNTVl7bxrARfeH7H/Caa/VAuEesHrH2OICMEsxVlwzM/hW J4XRPzEaoTgMlRBouzs6x+YbOMSb/l/SHIEQxywDiRSOom87Dfvacqxrdmgczi8e+AWtsUW6/wkhpE EY5EqMze+HehXjoPJvHKYAwf/SPtbeNSH06aK8LTRrF46l5QR9AY6+nTt8eViEH5b2HG0O7wFG2nFN MjsbObx7yVgT1OPC2rG9YRCrmjfYrpvu2wm9oUFW5iBs/jJjE0zOPXiLh8aXJc4tweTNMN8zDAixm1 4kKV9lMekTN3+T72giWH24O5uVNd0PTBlP/VRrDq+wARMBD0qUj1h298kOvA== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org With UVC 1.5 we get as little as one clock sample per frame. Which means that it takes 32 frames to move from the software timestamp to the hardware timestamp method. This results in abrupt changes in the timestamping after 32 frames (~1 second), resulting in noticeable artifacts when used for encoding. With this patch we modify the update algorithm to work with whatever amount of values are available. Signed-off-by: Ricardo Ribalda diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index 75c32e232f5d..7c7fb478aef7 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -742,10 +742,10 @@ void uvc_video_clock_update(struct uvc_streaming *stream, spin_lock_irqsave(&clock->lock, flags); - if (clock->count < clock->size) + if (clock->count < 2) goto done; - first = &clock->samples[clock->head]; + first = &clock->samples[(clock->head - clock->count) % clock->size]; last = &clock->samples[(clock->head - 1) % clock->size]; /* First step, PTS to SOF conversion. */ From patchwork Tue Sep 20 14:08:13 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 608145 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7304ECAAD8 for ; Tue, 20 Sep 2022 14:08:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231528AbiITOIu (ORCPT ); Tue, 20 Sep 2022 10:08:50 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:41704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231529AbiITOIi (ORCPT ); Tue, 20 Sep 2022 10:08:38 -0400 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [IPv6:2a00:1450:4864:20::62a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A6D385AA07 for ; Tue, 20 Sep 2022 07:08:35 -0700 (PDT) Received: by mail-ej1-x62a.google.com with SMTP id a26so6485374ejc.4 for ; Tue, 20 Sep 2022 07:08:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date; bh=NCc5XnE86A35upfQb7w9Zg49ZtxeysSLRX/7yyxgtR0=; b=BX1eXrPvlIVcwDO3uOoPlFa+ImKnuEQCQV/DWcnsoCT6sqrex4Hu7kkrwCgLMyWJtx UNJcjUQV2YD8jravMv0yw7c/xqBO25g8hMAACLntYpoJoUXKUELs4ZOkieYcfdHHBhFd l/quskDGx3sQyd4FHgPSVZrexFzLbKQLo3caM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date; bh=NCc5XnE86A35upfQb7w9Zg49ZtxeysSLRX/7yyxgtR0=; b=bFCBUxyBamhmTwRedrAyZuPhpKrSPXaUIe9h8PKlbTn3Ossp/8qkT6QdLUjAZAuBCN xuvXVs7RqgfGLNOQ6spPo5POuVvmJkDSv4hZyDDhGZNKwEhV8l6Zm3ma40I2ZMFhLulk F7oSFSGOLWIknJ/5xuAz/ZVnDP/Wnr3VKHGRd93aRXrcGk557ebSeEFFucvwe4RF1zwk 8h6cSuUrefWAlseD+dT9PoGs9i7KoGgVqwEIAqkI6A6jaleBnBD99oDeVkLP6YGdHKAJ ZDVenYstmoh7kt3lwnC1b2RCstu8ttCO+xh6QQ3F0tSaCv4SaZVwHM7teNk+6oLdfhTK o+aw== X-Gm-Message-State: ACrzQf26bCJWkuycyARBxv4Ax57HTKPzKgkHd2NG0ggccPv3eS6muaeM eLZzoigNR2Dp3ywLPihYs0rchw== X-Google-Smtp-Source: AMsMyM7PE6lyS3+0vtSumMvg+ryWzwkFIVOxy/qOiei/xbbb0im3D3s8lTsiQHWWrV9HXqQr9yDzsQ== X-Received: by 2002:a17:906:cc4a:b0:779:ed37:b5a3 with SMTP id mm10-20020a170906cc4a00b00779ed37b5a3mr16861886ejb.626.1663682914022; Tue, 20 Sep 2022 07:08:34 -0700 (PDT) Received: from alco.roam.corp.google.com ([2620:0:1059:10:935d:52b0:7461:88e1]) by smtp.gmail.com with ESMTPSA id j17-20020a17090623f100b0077016f4c6d4sm936564ejg.55.2022.09.20.07.08.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 20 Sep 2022 07:08:33 -0700 (PDT) From: Ricardo Ribalda Date: Tue, 20 Sep 2022 16:08:13 +0200 Subject: [PATCH v1 7/8] media: uvcvideo: Refactor clock circular buffer MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v1-7-e9c14b258404@chromium.org> References: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v1-0-e9c14b258404@chromium.org> To: Laurent Pinchart , Mauro Carvalho Chehab Cc: "hn.chen" , Ricardo Ribalda , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org X-Mailer: b4 0.11.0-dev-d93f8 X-Developer-Signature: v=1; a=openpgp-sha256; l=5683; i=ribalda@chromium.org; h=from:subject:message-id; bh=z/+kzQHc3YxeEXHUqT6czLGAa5vbV9no6M5D99w9Aks=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBjKclW0N/GT6a158BpvExijGgXwYoWPIwxcszDBMi7 C5iAMHCJAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCYynJVgAKCRDRN9E+zzrEiGC4EA CKtFH0hEXzzHQtBofaYIEP0M3h7dxYuzsCo0upAQgd5jzreQXQPSfq4+hIHvfBqlKEWYO8j1Kl5BVw jcs9KGfYF3H+Oxk2tAlNHpvh/MehKyHnE9HtNbQjkU9OEOcoEQElSleKv9meMo7eKZ0t59K9tlVNRv 0z/MaIVQCFJ8slQ5YxkqP6OGJoUCqcr4w4fVYvGKQf758AFB5AHRdyoFC8zZR7ARU7MnUNnQRnqHu1 4FQ2rxxukpuiuzwZaiyBqT0IWVohLEk/1X16na43YkMdBqhFODfxt6M0W4CtmWXINvx+5pvYf4RKgF tPsK3IFGY8sY2LJ6fRu3pLcRHahGF2lxvxpR6f4C+ZsJSU96txF693nooIQcwqQVuZwbBDIfPDv5iF Wgex9vKVDIeh7Jt44sQXJ6yYFZ/TtaDahxhL1lR+mZcZBVTsHtPxvFpHNphehj/n/m+h1cAesTsJBg 2T1GpsRCbJDPHZCPUvjhJnHpucaRohuofM+LXHcihW40X8TC/ku/Gd7psBf3ytCyKwc1f4zJQCiC5f qJCD6ZOure5118fJouhI0V5F7HVrAbpGMd3eAuDv6UxWfJdU4BipA6TiNoNCwrTUApUlvIVLtGfp8F vNjD/8l2RRUQvxsMubsWoJQyBHKULQNpoNWmGS4a+QCVT3vK791zMsyVcJsQ== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Move the addition to the buffer into its own function and remove the stream dependency to all the clock functions. Signed-off-by: Ricardo Ribalda diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index 7c7fb478aef7..9e7ab7ef8624 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -464,18 +464,29 @@ static inline ktime_t uvc_video_get_time(void) return ktime_get_real(); } +static void uvc_video_clock_add_sample(struct uvc_clock *clock, + const struct uvc_clock_sample *sample) +{ + unsigned long flags; + + spin_lock_irqsave(&clock->lock, flags); + + memcpy(&clock->samples[clock->head], sample, sizeof(*sample)); + clock->last_sof = sample->dev_sof; + clock->head = (clock->head + 1) % clock->size; + clock->count = min(clock->count + 1, clock->size); + + spin_unlock_irqrestore(&clock->lock, flags); +} + static void uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, const u8 *data, int len) { - struct uvc_clock_sample *sample; + struct uvc_clock_sample sample; unsigned int header_size; bool has_pts = false; bool has_scr = false; - unsigned long flags; - ktime_t time; - u16 host_sof; - u16 dev_sof; switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) { case UVC_STREAM_PTS | UVC_STREAM_SCR: @@ -530,20 +541,19 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, * only the first one will match the host_sof. */ if (stream->dev->quirks & UVC_QUIRK_INVALID_DEVICE_SOF) - dev_sof = usb_get_current_frame_number(stream->dev->udev); + sample.dev_sof = + usb_get_current_frame_number(stream->dev->udev); else - dev_sof = get_unaligned_le16(&data[header_size - 2]); + sample.dev_sof = get_unaligned_le16(&data[header_size - 2]); - if (dev_sof == stream->clock.last_sof) + if (sample.dev_sof == stream->clock.last_sof) return; - stream->clock.last_sof = dev_sof; - if (stream->dev->quirks & UVC_QUIRK_INVALID_DEVICE_SOF) - host_sof = dev_sof; + sample.host_sof = sample.dev_sof; else - host_sof = usb_get_current_frame_number(stream->dev->udev); - time = uvc_video_get_time(); + sample.host_sof = usb_get_current_frame_number(stream->dev->udev); + sample.host_time = uvc_video_get_time(); /* * The UVC specification allows device implementations that can't obtain @@ -566,46 +576,29 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, * the 8 LSBs of the delta are kept. */ if (stream->clock.sof_offset == (u16)-1) { - u16 delta_sof = (host_sof - dev_sof) & 255; + u16 delta_sof = (sample.host_sof - sample.dev_sof) & 255; if (delta_sof >= 10) stream->clock.sof_offset = delta_sof; else stream->clock.sof_offset = 0; } - dev_sof = (dev_sof + stream->clock.sof_offset) & 2047; - - spin_lock_irqsave(&stream->clock.lock, flags); - - sample = &stream->clock.samples[stream->clock.head]; - sample->dev_stc = get_unaligned_le32(&data[header_size - 6]); - sample->dev_sof = dev_sof; - sample->host_sof = host_sof; - sample->host_time = time; - - /* Update the sliding window head and count. */ - stream->clock.head = (stream->clock.head + 1) % stream->clock.size; + sample.dev_sof = (sample.dev_sof + stream->clock.sof_offset) & 2047; + sample.dev_stc = get_unaligned_le32(&data[header_size - 6]); - if (stream->clock.count < stream->clock.size) - stream->clock.count++; - - spin_unlock_irqrestore(&stream->clock.lock, flags); + uvc_video_clock_add_sample(&stream->clock, &sample); } -static void uvc_video_clock_reset(struct uvc_streaming *stream) +static void uvc_video_clock_reset(struct uvc_clock *clock) { - struct uvc_clock *clock = &stream->clock; - clock->head = 0; clock->count = 0; clock->last_sof = -1; clock->sof_offset = -1; } -static int uvc_video_clock_init(struct uvc_streaming *stream) +static int uvc_video_clock_init(struct uvc_clock *clock) { - struct uvc_clock *clock = &stream->clock; - spin_lock_init(&clock->lock); clock->size = 32; @@ -614,15 +607,15 @@ static int uvc_video_clock_init(struct uvc_streaming *stream) if (clock->samples == NULL) return -ENOMEM; - uvc_video_clock_reset(stream); + uvc_video_clock_reset(clock); return 0; } -static void uvc_video_clock_cleanup(struct uvc_streaming *stream) +static void uvc_video_clock_cleanup(struct uvc_clock *clock) { - kfree(stream->clock.samples); - stream->clock.samples = NULL; + kfree(clock->samples); + clock->samples = NULL; } /* @@ -2077,7 +2070,7 @@ int uvc_video_resume(struct uvc_streaming *stream, int reset) stream->frozen = 0; - uvc_video_clock_reset(stream); + uvc_video_clock_reset(&stream->clock); if (!uvc_queue_streaming(&stream->queue)) return 0; @@ -2211,7 +2204,7 @@ int uvc_video_start_streaming(struct uvc_streaming *stream) { int ret; - ret = uvc_video_clock_init(stream); + ret = uvc_video_clock_init(&stream->clock); if (ret < 0) return ret; @@ -2229,7 +2222,7 @@ int uvc_video_start_streaming(struct uvc_streaming *stream) error_video: usb_set_interface(stream->dev->udev, stream->intfnum, 0); error_commit: - uvc_video_clock_cleanup(stream); + uvc_video_clock_cleanup(&stream->clock); return ret; } @@ -2257,5 +2250,5 @@ void uvc_video_stop_streaming(struct uvc_streaming *stream) usb_clear_halt(stream->dev->udev, pipe); } - uvc_video_clock_cleanup(stream); + uvc_video_clock_cleanup(&stream->clock); }