From patchwork Wed Mar 15 12:35:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Ribalda X-Patchwork-Id: 663861 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FCAFC6FD1D for ; Wed, 15 Mar 2023 12:37:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232684AbjCOMhX (ORCPT ); Wed, 15 Mar 2023 08:37:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232584AbjCOMhG (ORCPT ); Wed, 15 Mar 2023 08:37:06 -0400 Received: from mail-ed1-x529.google.com (mail-ed1-x529.google.com [IPv6:2a00:1450:4864:20::529]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F0869FBF5 for ; Wed, 15 Mar 2023 05:36:12 -0700 (PDT) Received: by mail-ed1-x529.google.com with SMTP id fd5so41012961edb.7 for ; Wed, 15 Mar 2023 05:36:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1678883760; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=Ks+I0iL9S/eJzdcQJQgtcbFNoYKHK8Hr97Z4+vCos3E=; b=jNDOTpVguRqmPsNwZMHd8LjaCyFGqB0h/vgUFZNsVbc4vJM5fkjfVSXBLB5tuMQGc9 AibuVUnU3RCV0RHGJwrd0V6/7frhHnpG0bb+ycHjnnJZhR+dRqn4Igxn1LhJZdpU/5VD 4LqGKTW1xyuwuq3F+MH8bNjeZBHxbem17nF60= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678883760; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ks+I0iL9S/eJzdcQJQgtcbFNoYKHK8Hr97Z4+vCos3E=; b=C4L0OcG4XEH6QNuQPcnG1c3+T4Ev99+aEllW7QHlzD2TWPMgcLZ699eKpM3dgTuBwG OJLZ88mdAxos3EvnOUEHxNIXQD3yuZnsiTITKM6eTSV1fVBP0POXtl45JvE3EcV+mdJC R13hjw+Ss3pOP17d41m2x2RqjnanoRWW6I3+XGrDLAO/oDFf5YNHAq/86543COo0VbgC GgwI4NiAnnjhX0EX7uYkN7umpkZX+x9bsGTB8qpKNJkp8oKprZygESIXn3qBSlLDjHeP b2B/y+M9kIhnXmUiCdLoa5QFGexCewBuoZJBIYDqQ4osRMF3PhOI4Ay+qv3nE3OUoEtn 0r7Q== X-Gm-Message-State: AO0yUKXFU94c6doPSqcPAnuxeifxqeoJ5aISDdlPFbFuuQcioq4mUbZb XwbPAm7UZF4LXJoY+IB3hcWN+A== X-Google-Smtp-Source: AK7set+6iMJqwTCKxbtSpa/DEZpAg+l9W6d0SLzFYfPsd3q00iUx64y4/CiIhyx5TsTvO7Flmon4+A== X-Received: by 2002:a05:6402:404:b0:4fc:b1f2:42eb with SMTP id q4-20020a056402040400b004fcb1f242ebmr2219182edv.20.1678883759997; Wed, 15 Mar 2023 05:35:59 -0700 (PDT) Received: from alco.roam.corp.google.com ([2620:0:1059:10:491c:f627:27f0:1c4]) by smtp.gmail.com with ESMTPSA id w3-20020a50c443000000b004c30e2fc6e5sm2363256edf.65.2023.03.15.05.35.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 15 Mar 2023 05:35:59 -0700 (PDT) From: Ricardo Ribalda Date: Wed, 15 Mar 2023 13:35:38 +0100 Subject: [PATCH v8 2/6] media: uvcvideo: Ignore empty TS packets MIME-Version: 1.0 Message-Id: <20220920-resend-hwtimestamp-v8-2-0edaca2e2ab3@chromium.org> References: <20220920-resend-hwtimestamp-v8-0-0edaca2e2ab3@chromium.org> In-Reply-To: <20220920-resend-hwtimestamp-v8-0-0edaca2e2ab3@chromium.org> To: Laurent Pinchart , Mauro Carvalho Chehab Cc: "hn.chen" , linux-kernel@vger.kernel.org, linux-media@vger.kernel.org, Sergey Senozhatsky , Ricardo Ribalda X-Mailer: b4 0.11.0-dev-696ae X-Developer-Signature: v=1; a=openpgp-sha256; l=3922; i=ribalda@chromium.org; h=from:subject:message-id; bh=+e1rJhTRZYFtyKcR4/N8/nQKunH8/Dqv1jSdJGbbaL4=; b=owEBbQKS/ZANAwAKAdE30T7POsSIAcsmYgBkEbujR7E2zx9xw86qmRztrnP4mXdUpFSOlP0hN7K3 xBV763+JAjMEAAEKAB0WIQREDzjr+/4oCDLSsx7RN9E+zzrEiAUCZBG7owAKCRDRN9E+zzrEiC5rD/ 9mI8NjUQ/dqMqieVdH8Zk2i68yUANex7arMlWP0u2OXtFOEHQ7eXRkzeo7w4fED2RBgpacUy//ZktZ YL5rIdili429BjEKDVLOXKTdVpmoX7B1JC6a95aNGxGZiPmhoc/P89+5VWBDdrkPtj/fBC617rdkxN hzyoI1Vcvb019dJpAJcFWsyC6UCz1ujkjy6sSAusz5jOsn9jWphBxtFX39NZxZpQ/a2f6h7Lf4ReGo KBUyJgLEDUg4F4IIRUL7omOfxh0uRALD0BDFiM5TU0TG22DP1327g9n7c85c/51J8Y9oI91iSVvs0Q ztD44jnNYI/re0K8wMBJ+H0zkywdpFuSh7BedIJB1I2yCnx6AvWzNdOH4kT6suz5CIRAcqRcUtiPbB nnRI3LqMYccQx0k6mCifj/BNnx7pdpuBquU3Sx+26NRrjzOgLlUsuqcP/M3nJMsO2W/ZOIkD4ShWc6 Q8ZS073PF7uPr0qnszaZUoUiQcA5iLSFy1d6zuHGY0hMBDDdZ1UewdHUq2Tuts9mJDeNjoPUYeCW0F C9havLCEWEeP2JHDAPqIbfJA4srKlBTJA6F6Wv9UcNbyBMQ2tL1gqsLb0flPJyBldRpSu0A0i1I/Tc y5juGqG781g0qU9eCGcE0qPydxI6JXyN+Ec/zQkQIfeqqxanzodtayRQX1cA== X-Developer-Key: i=ribalda@chromium.org; a=openpgp; fpr=9EC3BB66E2FC129A6F90B39556A0D81F9F782DA9 Precedence: bulk List-ID: X-Mailing-List: linux-media@vger.kernel.org Some SunplusIT cameras took a borderline interpretation of the UVC 1.5 standard, and fill the PTS and SCR fields with invalid data if the package does not contain data. "STC must be captured when the first video data of a video frame is put on the USB bus." Eg: buffer: 0xa7755c00 len 000012 header:0x8c stc 00000000 sof 0000 pts 00000000 buffer: 0xa7755c00 len 000012 header:0x8c stc 00000000 sof 0000 pts 00000000 buffer: 0xa7755c00 len 000668 header:0x8c stc 73779dba sof 070c pts 7376d37a This borderline/buggy interpretation has been implemented in a variety of devices, from directly SunplusIT and from other OEMs that rebrand SunplusIT products. So quirking based on VID:PID will be problematic. All the affected modules have the following extension unit: VideoControl Interface Descriptor: guidExtensionCode {82066163-7050-ab49-b8cc-b3855e8d221d} But the vendor plans to use that GUID in the future and fix the bug, this means that we should use heuristic to figure out the broken packets. This patch takes care of this. lsusb of one of the affected cameras: Bus 001 Device 003: ID 1bcf:2a01 Sunplus Innovation Technology Inc. Device Descriptor: bLength 18 bDescriptorType 1 bcdUSB 2.01 bDeviceClass 239 Miscellaneous Device bDeviceSubClass 2 ? bDeviceProtocol 1 Interface Association bMaxPacketSize0 64 idVendor 0x1bcf Sunplus Innovation Technology Inc. idProduct 0x2a01 bcdDevice 0.02 iManufacturer 1 SunplusIT Inc iProduct 2 HanChen Wise Camera iSerial 3 01.00.00 bNumConfigurations 1 Tested-by: HungNien Chen Reviewed-by: Sergey Senozhatsky Signed-off-by: Ricardo Ribalda --- drivers/media/usb/uvc/uvc_video.c | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/drivers/media/usb/uvc/uvc_video.c b/drivers/media/usb/uvc/uvc_video.c index 4ff4ab4471fe..1f416c494acc 100644 --- a/drivers/media/usb/uvc/uvc_video.c +++ b/drivers/media/usb/uvc/uvc_video.c @@ -478,6 +478,7 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, ktime_t time; u16 host_sof; u16 dev_sof; + u32 dev_stc; switch (data[1] & (UVC_STREAM_PTS | UVC_STREAM_SCR)) { case UVC_STREAM_PTS | UVC_STREAM_SCR: @@ -526,6 +527,23 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, if (dev_sof == stream->clock.last_sof) return; + dev_stc = get_unaligned_le32(&data[header_size - 6]); + + /* + * STC (Source Time Clock) is the clock used by the camera. The UVC 1.5 + * standard states that it "must be captured when the first video data + * of a video frame is put on the USB bus". + * Most of the vendors, clear the `UVC_STREAM_SCR` bit when the data is + * not valid, other vendors always set the `UVC_STREAM_SCR` bit and + * expect that the driver only samples the stc if there is data on the + * packet. + * Ignore all the hardware timestamp information if there is no data + * and stc and sof are zero. + */ + if (buf && buf->bytesused == 0 && len == header_size && + dev_stc == 0 && dev_sof == 0) + return; + stream->clock.last_sof = dev_sof; host_sof = usb_get_current_frame_number(stream->dev->udev); @@ -564,7 +582,7 @@ uvc_video_clock_decode(struct uvc_streaming *stream, struct uvc_buffer *buf, spin_lock_irqsave(&stream->clock.lock, flags); sample = &stream->clock.samples[stream->clock.head]; - sample->dev_stc = get_unaligned_le32(&data[header_size - 6]); + sample->dev_stc = dev_stc; sample->dev_sof = dev_sof; sample->host_sof = host_sof; sample->host_time = time;