From patchwork Tue Jun 25 08:19:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 167686 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp5248050ilk; Tue, 25 Jun 2019 01:19:32 -0700 (PDT) X-Google-Smtp-Source: APXvYqzI3xtWe/3SGYxLkD6/tyCwad5TCrDKRwDZIy2J+yyq7QkLBjlZEgcZLCOsVy2BqCrwqCBK X-Received: by 2002:a17:90a:1b4a:: with SMTP id q68mr30275384pjq.61.1561450772372; Tue, 25 Jun 2019 01:19:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561450772; cv=none; d=google.com; s=arc-20160816; b=Zta1vOQNVI0Brwk40UI+soLvruMKnvVhVXk7h6Dg1zp272Iyx0cTTpN3Sj4Gtg/19j +kaC96pZw8cbj5BYZeVbsiRzMT2VRDxwA6fO8+6fW54o5vLqYSXzGL/KcCcS9cj+CT3v 5cK27rxSdHAAehilOvfPTacX7V1BT8qcTHOoRQrmP4qWuqEY7+wKTPB+k7fBgOdkAWjm 7MTCMB0VQ4/He3ID/Y+nJgY/DOlKUiY2tVNSIWxvIw/al8B7dP3wpijLSUI5Zr86pCSr V3EprdOehzNNH/NOFCB+ue0mYkVm9/OE8fWKT0jkkCTC0k2iRQHRXvIXSWldmai/1bGn 4yjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=qlum8tDaCUK/L3UhA2gI/wD/Iovib4Hd/J6byQUl0xU=; b=gEuRrku8F1Q9TQkWD/p8QMBmg2h3UaY4eMxBoCvWeKFiTO4/TE78G9cY86fVdJu101 h4lzKdS5BxT9uwBS9VasAILnrrrftjToVJbuofRa28zrsRGzGTsdhseGXWsEy8ETaEvE Lunvd7UutrPVJaVU9Xlo5eY2fmcCxMZ5fmEPkin7avWxZuOhPsYVBYIBBkpeHpEYL0py 1AvLfKsqfDBI8YMs1fUBMf+PStgSQStYMJ2XoHnDim1NYCoEtPU61NljJ4xznmb+ySNK Spy2/jop0qHjmH2xnj8uCXaCVtlCUEAEst6vVpXEUv0arpzt1NRo2vRrFbRMw3l0Q/OA EJPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@zx2c4.com header.s=mail header.b=wxbzTAZz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=zx2c4.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w9si2511605pgr.250.2019.06.25.01.19.32; Tue, 25 Jun 2019 01:19:32 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@zx2c4.com header.s=mail header.b=wxbzTAZz; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=zx2c4.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729657AbfFYITa (ORCPT + 30 others); Tue, 25 Jun 2019 04:19:30 -0400 Received: from frisell.zx2c4.com ([192.95.5.64]:38083 "EHLO frisell.zx2c4.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726783AbfFYIT3 (ORCPT ); Tue, 25 Jun 2019 04:19:29 -0400 Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTP id dd2a6a86; Tue, 25 Jun 2019 07:45:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=zx2c4.com; h=from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; s=mail; bh=6AhJklfhAlrrPr4iFlW6cSyKn 9g=; b=wxbzTAZzfeJMO5yikXjo33XmrOOOrvW9zvzUN2wlDmd9dcQLdyTN6kaPp F7/BoBGmiH6QdGIzjqqo/GAd2KlTqqiZAiErX82LC62TrRMpKOmlB7x7AnKRmvPa 6HyXJtnd3i4JVvty6nbErfK4iJrhzzy03nzk1qf6P6GdtCTWrGV65rlMYgxoIYf6 LnG936mY0JWAfT7zu0sdwMO0gtRxw4FbNpg9uFoU2LXOwKNRRvnV3sy1iVOvGFkl /jjs3W6J0FE0esRbcTiffDwedGaWb8ByE0SQDpVmzBDDxpCx5Zmd7WrdBh5VBSdK OSq3CaXn6RKWvl6BnkjU5sPv2CzsA== Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id fd7a8702 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Tue, 25 Jun 2019 07:45:34 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org Cc: "Jason A. Donenfeld" , Arnd Bergmann , Thomas Gleixner Subject: [PATCH 1/2] timekeeping: add missing non-_ns functions for fast accessors Date: Tue, 25 Jun 2019 10:19:11 +0200 Message-Id: <20190625081912.14813-2-Jason@zx2c4.com> In-Reply-To: <20190625081912.14813-1-Jason@zx2c4.com> References: <20190625081912.14813-1-Jason@zx2c4.com> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Previously there was no analogue to get proper ktime_t versions of the fast variety of ktime invocations. This commit makes the interface uniform with the other accessors. Signed-off-by: Jason A. Donenfeld Cc: Arnd Bergmann Cc: Thomas Gleixner --- Documentation/core-api/timekeeping.rst | 7 +++- include/linux/timekeeping.h | 28 ++++++++++++-- kernel/time/timekeeping.c | 52 +++++++++++++------------- 3 files changed, 55 insertions(+), 32 deletions(-) -- 2.21.0 diff --git a/Documentation/core-api/timekeeping.rst b/Documentation/core-api/timekeeping.rst index 20ee447a50f3..29d38a86faac 100644 --- a/Documentation/core-api/timekeeping.rst +++ b/Documentation/core-api/timekeeping.rst @@ -94,7 +94,7 @@ different format depending on what is required by the user: down the seconds to the full seconds of the last timer tick using the respective reference. -Coarse and fast_ns access +Coarse and fast access ------------------------- Some additional variants exist for more specialized cases: @@ -129,6 +129,11 @@ Some additional variants exist for more specialized cases: up to several microseconds on older hardware with an external clocksource. +.. c:function:: ktime_t ktime_get_mono_fast( void ) + ktime_t ktime_get_raw_fast( void ) + ktime_t ktime_get_boottime_fast( void ) + ktime_t ktime_get_real_fast( void ) + .. c:function:: u64 ktime_get_mono_fast_ns( void ) u64 ktime_get_raw_fast_ns( void ) u64 ktime_get_boot_fast_ns( void ) diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h index b27e2ffa96c1..b8a3a129258c 100644 --- a/include/linux/timekeeping.h +++ b/include/linux/timekeeping.h @@ -174,10 +174,30 @@ static inline u64 ktime_get_raw_ns(void) return ktime_to_ns(ktime_get_raw()); } -extern u64 ktime_get_mono_fast_ns(void); -extern u64 ktime_get_raw_fast_ns(void); -extern u64 ktime_get_boot_fast_ns(void); -extern u64 ktime_get_real_fast_ns(void); +extern ktime_t ktime_get_mono_fast(void); +extern ktime_t ktime_get_raw_fast(void); +extern ktime_t ktime_get_boottime_fast(void); +extern ktime_t ktime_get_real_fast(void); + +static inline u64 ktime_get_mono_fast_ns(void) +{ + return ktime_to_ns(ktime_get_mono_fast()); +} + +static inline u64 ktime_get_raw_fast_ns(void) +{ + return ktime_to_ns(ktime_get_raw_fast()); +} + +static inline u64 ktime_get_boot_fast_ns(void) +{ + return ktime_to_ns(ktime_get_boottime_fast()); +} + +static inline u64 ktime_get_real_fast_ns(void) +{ + return ktime_to_ns(ktime_get_real_fast()); +} /* * timespec64/time64_t interfaces utilizing the ktime based ones diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index d911c8470149..db0081a14b90 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -443,41 +443,40 @@ static void update_fast_timekeeper(const struct tk_read_base *tkr, * of the following timestamps. Callers need to be aware of that and * deal with it. */ -static __always_inline u64 __ktime_get_fast_ns(struct tk_fast *tkf) +static __always_inline ktime_t __ktime_get_fast(struct tk_fast *tkf) { struct tk_read_base *tkr; unsigned int seq; - u64 now; + ktime_t now; do { seq = raw_read_seqcount_latch(&tkf->seq); tkr = tkf->base + (seq & 0x01); - now = ktime_to_ns(tkr->base); - - now += timekeeping_delta_to_ns(tkr, + now = ktime_add_ns(tkr->base, + timekeeping_delta_to_ns(tkr, clocksource_delta( tk_clock_read(tkr), tkr->cycle_last, - tkr->mask)); + tkr->mask))); } while (read_seqcount_retry(&tkf->seq, seq)); return now; } -u64 ktime_get_mono_fast_ns(void) +ktime_t ktime_get_mono_fast(void) { - return __ktime_get_fast_ns(&tk_fast_mono); + return __ktime_get_fast(&tk_fast_mono); } -EXPORT_SYMBOL_GPL(ktime_get_mono_fast_ns); +EXPORT_SYMBOL_GPL(ktime_get_mono_fast); -u64 ktime_get_raw_fast_ns(void) +ktime_t ktime_get_raw_fast(void) { - return __ktime_get_fast_ns(&tk_fast_raw); + return __ktime_get_fast(&tk_fast_raw); } -EXPORT_SYMBOL_GPL(ktime_get_raw_fast_ns); +EXPORT_SYMBOL_GPL(ktime_get_raw_fast); /** - * ktime_get_boot_fast_ns - NMI safe and fast access to boot clock. + * ktime_get_boottime_fast - NMI safe and fast access to boot clock. * * To keep it NMI safe since we're accessing from tracing, we're not using a * separate timekeeper with updates to monotonic clock and boot offset @@ -497,47 +496,46 @@ EXPORT_SYMBOL_GPL(ktime_get_raw_fast_ns); * partially updated. Since the tk->offs_boot update is a rare event, this * should be a rare occurrence which postprocessing should be able to handle. */ -u64 notrace ktime_get_boot_fast_ns(void) +ktime_t notrace ktime_get_boottime_fast(void) { struct timekeeper *tk = &tk_core.timekeeper; - return (ktime_get_mono_fast_ns() + ktime_to_ns(tk->offs_boot)); + return ktime_add(ktime_get_mono_fast(), tk->offs_boot); } -EXPORT_SYMBOL_GPL(ktime_get_boot_fast_ns); +EXPORT_SYMBOL_GPL(ktime_get_boottime_fast); /* - * See comment for __ktime_get_fast_ns() vs. timestamp ordering + * See comment for __ktime_get_fast() vs. timestamp ordering */ -static __always_inline u64 __ktime_get_real_fast_ns(struct tk_fast *tkf) +static __always_inline ktime_t __ktime_get_real_fast(struct tk_fast *tkf) { struct tk_read_base *tkr; unsigned int seq; - u64 now; + ktime_t now; do { seq = raw_read_seqcount_latch(&tkf->seq); tkr = tkf->base + (seq & 0x01); - now = ktime_to_ns(tkr->base_real); - - now += timekeeping_delta_to_ns(tkr, + now = ktime_add_ns(tkr->base_real, + timekeeping_delta_to_ns(tkr, clocksource_delta( tk_clock_read(tkr), tkr->cycle_last, - tkr->mask)); + tkr->mask))); } while (read_seqcount_retry(&tkf->seq, seq)); return now; } /** - * ktime_get_real_fast_ns: - NMI safe and fast access to clock realtime. + * ktime_get_real_fast: - NMI safe and fast access to clock realtime. */ -u64 ktime_get_real_fast_ns(void) +ktime_t ktime_get_real_fast(void) { - return __ktime_get_real_fast_ns(&tk_fast_mono); + return __ktime_get_real_fast(&tk_fast_mono); } -EXPORT_SYMBOL_GPL(ktime_get_real_fast_ns); +EXPORT_SYMBOL_GPL(ktime_get_real_fast); /** * halt_fast_timekeeper - Prevent fast timekeeper from accessing clocksource. From patchwork Tue Jun 25 08:19:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Jason A. Donenfeld" X-Patchwork-Id: 167687 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp5248083ilk; Tue, 25 Jun 2019 01:19:34 -0700 (PDT) X-Google-Smtp-Source: APXvYqzzxnmRGkzLEriFx72MAyArzVRftDSmFbcxoO0SvaLZFfz8pgwOvHRxilVZRzRUIcs2RiAJ X-Received: by 2002:a63:1d10:: with SMTP id d16mr19474825pgd.446.1561450774590; Tue, 25 Jun 2019 01:19:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561450774; cv=none; d=google.com; s=arc-20160816; b=G//J/uQwhvPRGfRR6Ami6hwyZRPNNmAInUbCXYSZxYJNYRyUaW/2B7CZ5aeM37UK+K DDV41JaKdxgOT5Rkyyx4EutRrwQAvflW3mF3Qa/u8N+LxpB8T53y+xL8sWhrSMZqQgMG MZVEmFdnwcAqLnjSNIDj1oIQxVOJqPPxx/Quzkt0ApFCXGvMEfjVkTTTCYNCsBkC2eay 8Pi2+pvw7+Yek7DXNVMPOazESn2tnibiOn1ML43YWUIZgs6UGIa/APOL5FZzMEvYdANh kJwAmsSCKxa9IFXxUnmZGpoqAcRlqtz8SrM1Cq7YXq7IGHKPo0k3en+AkjhRS1fYF9bt QMyQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=5dREE2F8exbHguvjR1Rzxut8VoLhzBStF+V+XYqujGE=; b=p6OlAei5teIAxSA5tLd+PqPlqOLbk2jBa2PuU4FCXNGSvQVpz9uz1CQb4T6c4h2bt1 LleqlH4AAOyohK9bFSEWB0CB5RZVL+LUJPegP2T2n64hCF6sRFXTRVMrL5+wjMxHFqsT AaN4UIncxK9y21mdxRBuTentZ6DSJ0JUh/PB+rvoOJnUhjF9TqY2gTkan1RJhfY3Bdrm oRSPaAVdluttvdwK605ltxS2auQeOgK2BurqijyrLTWjJYyae9guY/htOeL3cHjirDes Lc4toQJnZiZboyNLIYYkdjH4SmTPG6rzvWvPLr3HAonfq60XeSyFQUJjLTWKUqkqi9LI U4KQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@zx2c4.com header.s=mail header.b=ihxT0NBf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=zx2c4.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id gn3si11828636plb.321.2019.06.25.01.19.34; Tue, 25 Jun 2019 01:19:34 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@zx2c4.com header.s=mail header.b=ihxT0NBf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=zx2c4.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729773AbfFYITc (ORCPT + 30 others); Tue, 25 Jun 2019 04:19:32 -0400 Received: from frisell.zx2c4.com ([192.95.5.64]:38083 "EHLO frisell.zx2c4.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728818AbfFYITb (ORCPT ); Tue, 25 Jun 2019 04:19:31 -0400 Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTP id c4d01081; Tue, 25 Jun 2019 07:45:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=zx2c4.com; h=from:to:cc :subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; s=mail; bh=2oUXYO/ZlKc88Ej+IHrly3uiZ o4=; b=ihxT0NBfTWjQVuwgzsSOVSXLDTPuyZmoqIMZ/CbKrraGLF3w8/Ta5qADf MP+RhCOK3Ros+msDJZPsYYPKNvEpRV1pn7kOvBYT3/9+jOodbq9oDa5B4VWhhqIx iZBxZ9gTDvUDgPMX2WQz70MbAWFWAL9TliPuUgXTy3yrs28q1Et1VhT57JpPsJ25 Y+qGXajCocedM013PSvSLIQXXVqNTkQJwpw4tX4ukiRV15TQY3CVVZ4iLgzwIZJ5 Vl5jLSiIrU/N6+klyYQc1sxo2DJgcR2CQeyHPvruyqPtXvRI60DWdDwv7vBVuhlq 6+4KrNqXguyC+HkFq/3kBY/V2qrPQ== Received: by frisell.zx2c4.com (ZX2C4 Mail Server) with ESMTPSA id b724ab95 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Tue, 25 Jun 2019 07:45:35 +0000 (UTC) From: "Jason A. Donenfeld" To: linux-kernel@vger.kernel.org Cc: "Jason A. Donenfeld" , Arnd Bergmann , Thomas Gleixner Subject: [PATCH 2/2] timekeeping: rename fast getter functions to be consistent Date: Tue, 25 Jun 2019 10:19:12 +0200 Message-Id: <20190625081912.14813-3-Jason@zx2c4.com> In-Reply-To: <20190625081912.14813-1-Jason@zx2c4.com> References: <20190625081912.14813-1-Jason@zx2c4.com> MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The _fast_ family of functions put the _fast_ modifier in the wrong place and confuse tai with clocktai and boot with boottime. Also, no other functions use _mono, making that the default unlabeled one. This commit normalizes these to match the rest of timekeeping.h. Signed-off-by: Jason A. Donenfeld Cc: Arnd Bergmann Cc: Thomas Gleixner --- Documentation/core-api/timekeeping.rst | 18 ++++++++-------- Documentation/trace/ftrace.rst | 2 +- drivers/base/power/runtime.c | 12 +++++------ drivers/gpu/drm/i915/i915_perf.c | 2 +- fs/pstore/platform.c | 2 +- include/linux/pm_runtime.h | 2 +- include/linux/timekeeping.h | 24 ++++++++++----------- kernel/bpf/helpers.c | 2 +- kernel/debug/kdb/kdb_main.c | 2 +- kernel/events/core.c | 4 ++-- kernel/rcu/rcuperf.c | 6 +++--- kernel/rcu/srcutree.c | 6 +++--- kernel/time/timekeeping.c | 30 +++++++++++++------------- kernel/trace/trace.c | 6 +++--- kernel/watchdog_hld.c | 2 +- 15 files changed, 60 insertions(+), 60 deletions(-) -- 2.21.0 diff --git a/Documentation/core-api/timekeeping.rst b/Documentation/core-api/timekeeping.rst index 29d38a86faac..3a9eac43f619 100644 --- a/Documentation/core-api/timekeeping.rst +++ b/Documentation/core-api/timekeeping.rst @@ -129,15 +129,15 @@ Some additional variants exist for more specialized cases: up to several microseconds on older hardware with an external clocksource. -.. c:function:: ktime_t ktime_get_mono_fast( void ) - ktime_t ktime_get_raw_fast( void ) - ktime_t ktime_get_boottime_fast( void ) - ktime_t ktime_get_real_fast( void ) - -.. c:function:: u64 ktime_get_mono_fast_ns( void ) - u64 ktime_get_raw_fast_ns( void ) - u64 ktime_get_boot_fast_ns( void ) - u64 ktime_get_real_fast_ns( void ) +.. c:function:: ktime_t ktime_get_fast( void ) + ktime_t ktime_get_fast_raw( void ) + ktime_t ktime_get_fast_boottime( void ) + ktime_t ktime_get_fast_real( void ) + +.. c:function:: u64 ktime_get_fast_ns( void ) + u64 ktime_get_fast_raw_ns( void ) + u64 ktime_get_fast_boottime_ns( void ) + u64 ktime_get_fast_real_ns( void ) These variants are safe to call from any context, including from a non-maskable interrupt (NMI) during a timekeeper update, and diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index f60079259669..6ec0e2a59419 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -482,7 +482,7 @@ of ftrace. Here is a list of some of the key files: Also on 32-bit systems, it's possible that the 64-bit boot offset sees a partial update. These effects are rare and post processing should be able to handle them. See comments in the - ktime_get_boot_fast_ns() function for more information. + ktime_get_fast_boottime_ns() function for more information. To set a clock, simply echo the clock name into this file:: diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c index 952a1e7057c7..218bc89c20ba 100644 --- a/drivers/base/power/runtime.c +++ b/drivers/base/power/runtime.c @@ -71,11 +71,11 @@ static void update_pm_runtime_accounting(struct device *dev) last = dev->power.accounting_timestamp; - now = ktime_get_mono_fast_ns(); + now = ktime_get_fast_ns(); dev->power.accounting_timestamp = now; /* - * Because ktime_get_mono_fast_ns() is not monotonic during + * Because ktime_get_fast_ns() is not monotonic during * timekeeping updates, ensure that 'now' is after the last saved * timesptamp. */ @@ -174,7 +174,7 @@ u64 pm_runtime_autosuspend_expiration(struct device *dev) expires = READ_ONCE(dev->power.last_busy); expires += (u64)autosuspend_delay * NSEC_PER_MSEC; - if (expires > ktime_get_mono_fast_ns()) + if (expires > ktime_get_fast_ns()) return expires; /* Expires in the future */ return 0; @@ -938,7 +938,7 @@ static enum hrtimer_restart pm_suspend_timer_fn(struct hrtimer *timer) * If 'expires' is after the current time, we've been called * too early. */ - if (expires > 0 && expires < ktime_get_mono_fast_ns()) { + if (expires > 0 && expires < ktime_get_fast_ns()) { dev->power.timer_expires = 0; rpm_suspend(dev, dev->power.timer_autosuspends ? (RPM_ASYNC | RPM_AUTO) : RPM_ASYNC); @@ -974,7 +974,7 @@ int pm_schedule_suspend(struct device *dev, unsigned int delay) /* Other scheduled or pending requests need to be canceled. */ pm_runtime_cancel_pending(dev); - expires = ktime_get_mono_fast_ns() + (u64)delay * NSEC_PER_MSEC; + expires = ktime_get_fast_ns() + (u64)delay * NSEC_PER_MSEC; dev->power.timer_expires = expires; dev->power.timer_autosuspends = 0; hrtimer_start(&dev->power.suspend_timer, expires, HRTIMER_MODE_ABS); @@ -1378,7 +1378,7 @@ void pm_runtime_enable(struct device *dev) /* About to enable runtime pm, set accounting_timestamp to now */ if (!dev->power.disable_depth) - dev->power.accounting_timestamp = ktime_get_mono_fast_ns(); + dev->power.accounting_timestamp = ktime_get_fast_ns(); } else { dev_warn(dev, "Unbalanced %s!\n", __func__); } diff --git a/drivers/gpu/drm/i915/i915_perf.c b/drivers/gpu/drm/i915/i915_perf.c index dc4ce694c06a..211b10cec5e2 100644 --- a/drivers/gpu/drm/i915/i915_perf.c +++ b/drivers/gpu/drm/i915/i915_perf.c @@ -478,7 +478,7 @@ static bool oa_buffer_check_unlocked(struct drm_i915_private *dev_priv) */ hw_tail &= ~(report_size - 1); - now = ktime_get_mono_fast_ns(); + now = ktime_get_fast_ns(); /* Update the aged tail * diff --git a/fs/pstore/platform.c b/fs/pstore/platform.c index 3d7024662d29..531e48d96096 100644 --- a/fs/pstore/platform.c +++ b/fs/pstore/platform.c @@ -378,7 +378,7 @@ void pstore_record_init(struct pstore_record *record, record->psi = psinfo; /* Report zeroed timestamp if called before timekeeping has resumed. */ - record->time = ns_to_timespec64(ktime_get_real_fast_ns()); + record->time = ns_to_timespec64(ktime_get_fast_real_ns()); } /* diff --git a/include/linux/pm_runtime.h b/include/linux/pm_runtime.h index 22af69d237a6..e904d93c6cfd 100644 --- a/include/linux/pm_runtime.h +++ b/include/linux/pm_runtime.h @@ -104,7 +104,7 @@ static inline bool pm_runtime_callbacks_present(struct device *dev) static inline void pm_runtime_mark_last_busy(struct device *dev) { - WRITE_ONCE(dev->power.last_busy, ktime_get_mono_fast_ns()); + WRITE_ONCE(dev->power.last_busy, ktime_get_fast_ns()); } static inline bool pm_runtime_is_irq_safe(struct device *dev) diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h index b8a3a129258c..e06a5989b7c5 100644 --- a/include/linux/timekeeping.h +++ b/include/linux/timekeeping.h @@ -174,29 +174,29 @@ static inline u64 ktime_get_raw_ns(void) return ktime_to_ns(ktime_get_raw()); } -extern ktime_t ktime_get_mono_fast(void); -extern ktime_t ktime_get_raw_fast(void); -extern ktime_t ktime_get_boottime_fast(void); -extern ktime_t ktime_get_real_fast(void); +extern ktime_t ktime_get_fast(void); +extern ktime_t ktime_get_fast_raw(void); +extern ktime_t ktime_get_fast_boottime(void); +extern ktime_t ktime_get_fast_real(void); -static inline u64 ktime_get_mono_fast_ns(void) +static inline u64 ktime_get_fast_ns(void) { - return ktime_to_ns(ktime_get_mono_fast()); + return ktime_to_ns(ktime_get_fast()); } -static inline u64 ktime_get_raw_fast_ns(void) +static inline u64 ktime_get_fast_raw_ns(void) { - return ktime_to_ns(ktime_get_raw_fast()); + return ktime_to_ns(ktime_get_fast_raw()); } -static inline u64 ktime_get_boot_fast_ns(void) +static inline u64 ktime_get_fast_boottime_ns(void) { - return ktime_to_ns(ktime_get_boottime_fast()); + return ktime_to_ns(ktime_get_fast_boottime()); } -static inline u64 ktime_get_real_fast_ns(void) +static inline u64 ktime_get_fast_real_ns(void) { - return ktime_to_ns(ktime_get_real_fast()); + return ktime_to_ns(ktime_get_fast_real()); } /* diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c index 5e28718928ca..df3617962dfd 100644 --- a/kernel/bpf/helpers.c +++ b/kernel/bpf/helpers.c @@ -143,7 +143,7 @@ const struct bpf_func_proto bpf_get_numa_node_id_proto = { BPF_CALL_0(bpf_ktime_get_ns) { /* NMI safe access to clock monotonic */ - return ktime_get_mono_fast_ns(); + return ktime_get_fast_ns(); } const struct bpf_func_proto bpf_ktime_get_ns_proto = { diff --git a/kernel/debug/kdb/kdb_main.c b/kernel/debug/kdb/kdb_main.c index 9ecfa37c7fbf..59dce544cfea 100644 --- a/kernel/debug/kdb/kdb_main.c +++ b/kernel/debug/kdb/kdb_main.c @@ -2491,7 +2491,7 @@ static int kdb_kill(int argc, const char **argv) */ static void kdb_sysinfo(struct sysinfo *val) { - u64 uptime = ktime_get_mono_fast_ns(); + u64 uptime = ktime_get_fast_ns(); memset(val, 0, sizeof(*val)); val->uptime = div_u64(uptime, NSEC_PER_SEC); diff --git a/kernel/events/core.c b/kernel/events/core.c index e2d014395fc6..2f708be06aad 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10666,12 +10666,12 @@ static int perf_event_set_clock(struct perf_event *event, clockid_t clk_id) switch (clk_id) { case CLOCK_MONOTONIC: - event->clock = &ktime_get_mono_fast_ns; + event->clock = &ktime_get_fast_ns; nmi_safe = true; break; case CLOCK_MONOTONIC_RAW: - event->clock = &ktime_get_raw_fast_ns; + event->clock = &ktime_get_fast_raw_ns; nmi_safe = true; break; diff --git a/kernel/rcu/rcuperf.c b/kernel/rcu/rcuperf.c index 7a6890b23c5f..665056894dea 100644 --- a/kernel/rcu/rcuperf.c +++ b/kernel/rcu/rcuperf.c @@ -375,7 +375,7 @@ rcu_perf_writer(void *arg) if (holdoff) schedule_timeout_uninterruptible(holdoff * HZ); - t = ktime_get_mono_fast_ns(); + t = ktime_get_fast_ns(); if (atomic_inc_return(&n_rcu_perf_writer_started) >= nrealwriters) { t_rcu_perf_writer_started = t; if (gp_exp) { @@ -390,7 +390,7 @@ rcu_perf_writer(void *arg) if (writer_holdoff) udelay(writer_holdoff); wdp = &wdpp[i]; - *wdp = ktime_get_mono_fast_ns(); + *wdp = ktime_get_fast_ns(); if (gp_async) { retry: if (!rhp) @@ -415,7 +415,7 @@ rcu_perf_writer(void *arg) cur_ops->sync(); } rcu_perf_writer_state = RTWS_IDLE; - t = ktime_get_mono_fast_ns(); + t = ktime_get_fast_ns(); *wdp = t - *wdp; i_max = i; if (!started && diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index 9b761e546de8..21e06a227e0a 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -179,7 +179,7 @@ static int init_srcu_struct_fields(struct srcu_struct *ssp, bool is_static) ssp->sda = alloc_percpu(struct srcu_data); init_srcu_struct_nodes(ssp, is_static); ssp->srcu_gp_seq_needed_exp = 0; - ssp->srcu_last_gp_end = ktime_get_mono_fast_ns(); + ssp->srcu_last_gp_end = ktime_get_fast_ns(); smp_store_release(&ssp->srcu_gp_seq_needed, 0); /* Init done. */ return ssp->sda ? 0 : -ENOMEM; } @@ -530,7 +530,7 @@ static void srcu_gp_end(struct srcu_struct *ssp) idx = rcu_seq_state(ssp->srcu_gp_seq); WARN_ON_ONCE(idx != SRCU_STATE_SCAN2); cbdelay = srcu_get_delay(ssp); - ssp->srcu_last_gp_end = ktime_get_mono_fast_ns(); + ssp->srcu_last_gp_end = ktime_get_fast_ns(); rcu_seq_end(&ssp->srcu_gp_seq); gpseq = rcu_seq_current(&ssp->srcu_gp_seq); if (ULONG_CMP_LT(ssp->srcu_gp_seq_needed_exp, gpseq)) @@ -779,7 +779,7 @@ static bool srcu_might_be_idle(struct srcu_struct *ssp) */ /* First, see if enough time has passed since the last GP. */ - t = ktime_get_mono_fast_ns(); + t = ktime_get_fast_ns(); if (exp_holdoff == 0 || time_in_range_open(t, ssp->srcu_last_gp_end, ssp->srcu_last_gp_end + exp_holdoff)) diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c index db0081a14b90..ae0ab8911e0c 100644 --- a/kernel/time/timekeeping.c +++ b/kernel/time/timekeeping.c @@ -391,7 +391,7 @@ static inline u64 timekeeping_cycles_to_ns(const struct tk_read_base *tkr, u64 c * So if a NMI hits the update of base[0] then it will use base[1] * which is still consistent. In the worst case this can result is a * slightly wrong timestamp (a few nanoseconds). See - * @ktime_get_mono_fast_ns. + * @ktime_get_fast_ns. */ static void update_fast_timekeeper(const struct tk_read_base *tkr, struct tk_fast *tkf) @@ -412,7 +412,7 @@ static void update_fast_timekeeper(const struct tk_read_base *tkr, } /** - * ktime_get_mono_fast_ns - Fast NMI safe access to clock monotonic + * ktime_get_fast_ns - Fast NMI safe access to clock monotonic * * This timestamp is not guaranteed to be monotonic across an update. * The timestamp is calculated by: @@ -463,20 +463,20 @@ static __always_inline ktime_t __ktime_get_fast(struct tk_fast *tkf) return now; } -ktime_t ktime_get_mono_fast(void) +ktime_t ktime_get_fast(void) { return __ktime_get_fast(&tk_fast_mono); } -EXPORT_SYMBOL_GPL(ktime_get_mono_fast); +EXPORT_SYMBOL_GPL(ktime_get_fast); -ktime_t ktime_get_raw_fast(void) +ktime_t ktime_get_fast_raw(void) { return __ktime_get_fast(&tk_fast_raw); } -EXPORT_SYMBOL_GPL(ktime_get_raw_fast); +EXPORT_SYMBOL_GPL(ktime_get_fast_raw); /** - * ktime_get_boottime_fast - NMI safe and fast access to boot clock. + * ktime_get_fast_boottime - NMI safe and fast access to boot clock. * * To keep it NMI safe since we're accessing from tracing, we're not using a * separate timekeeper with updates to monotonic clock and boot offset @@ -496,19 +496,19 @@ EXPORT_SYMBOL_GPL(ktime_get_raw_fast); * partially updated. Since the tk->offs_boot update is a rare event, this * should be a rare occurrence which postprocessing should be able to handle. */ -ktime_t notrace ktime_get_boottime_fast(void) +ktime_t notrace ktime_get_fast_boottime(void) { struct timekeeper *tk = &tk_core.timekeeper; - return ktime_add(ktime_get_mono_fast(), tk->offs_boot); + return ktime_add(ktime_get_fast(), tk->offs_boot); } -EXPORT_SYMBOL_GPL(ktime_get_boottime_fast); +EXPORT_SYMBOL_GPL(ktime_get_fast_boottime); /* * See comment for __ktime_get_fast() vs. timestamp ordering */ -static __always_inline ktime_t __ktime_get_real_fast(struct tk_fast *tkf) +static __always_inline ktime_t __ktime_get_fast_real(struct tk_fast *tkf) { struct tk_read_base *tkr; unsigned int seq; @@ -529,13 +529,13 @@ static __always_inline ktime_t __ktime_get_real_fast(struct tk_fast *tkf) } /** - * ktime_get_real_fast: - NMI safe and fast access to clock realtime. + * ktime_get_fast_real: - NMI safe and fast access to clock realtime. */ -ktime_t ktime_get_real_fast(void) +ktime_t ktime_get_fast_real(void) { - return __ktime_get_real_fast(&tk_fast_mono); + return __ktime_get_fast_real(&tk_fast_mono); } -EXPORT_SYMBOL_GPL(ktime_get_real_fast); +EXPORT_SYMBOL_GPL(ktime_get_fast_real); /** * halt_fast_timekeeper - Prevent fast timekeeper from accessing clocksource. diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 83e08b78dbee..840a4d024015 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1336,9 +1336,9 @@ static struct { { trace_clock_counter, "counter", 0 }, { trace_clock_jiffies, "uptime", 0 }, { trace_clock, "perf", 1 }, - { ktime_get_mono_fast_ns, "mono", 1 }, - { ktime_get_raw_fast_ns, "mono_raw", 1 }, - { ktime_get_boot_fast_ns, "boot", 1 }, + { ktime_get_fast_ns, "mono", 1 }, + { ktime_get_fast_raw_ns, "mono_raw", 1 }, + { ktime_get_fast_boottime_ns, "boot", 1 }, ARCH_TRACE_CLOCKS }; diff --git a/kernel/watchdog_hld.c b/kernel/watchdog_hld.c index 247bf0b1582c..044fc850b9a9 100644 --- a/kernel/watchdog_hld.c +++ b/kernel/watchdog_hld.c @@ -75,7 +75,7 @@ void watchdog_update_hrtimer_threshold(u64 period) static bool watchdog_check_timestamp(void) { - ktime_t delta, now = ktime_get_mono_fast_ns(); + ktime_t delta, now = ktime_get_fast_ns(); delta = now - __this_cpu_read(last_timestamp); if (delta < watchdog_hrtimer_sample_threshold) {