From patchwork Tue May 16 19:44:33 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel Lezcano X-Patchwork-Id: 99892 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp2184529qge; Tue, 16 May 2017 12:46:50 -0700 (PDT) X-Received: by 10.98.66.82 with SMTP id p79mr13978225pfa.124.1494964010783; Tue, 16 May 2017 12:46:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1494964010; cv=none; d=google.com; s=arc-20160816; b=ey3FP2JbyRpLKS2LTAoqC/vnS3vp2TmS/pB0ltXE7EBA6SA4W9SytGQpvfNaNPl9bu HuoASmm/NQ7TzCZOJ5mdr50EfYkXVf/dfeY3ZW8YCMoKABFQkgV3VDDPBftp6HA0adZA ZopuhmWqhRoPAGiTFs8mSUR9Ahuhv0Gmdw7z8lBOv9KBkgS/J54p/66WIKkjCt7kMAGc a/nXud5Utb0YHVdub0gStJ4s+cNpRxePUj95+PtGrp9g7fMJUhe/Zg67ydUGPUUfe9jg O2A/lqY7swomPPV3BxZVt8aJ8qhNEXB00k0CBAMitjPkC5J5d+3bWk4XGGzS+UDj3hxw J2ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=eOcDyYVrpZsSDRDf7rR0QgMnwKs5OdoUyPqaPbgnjEc=; b=C0dq1TJZxf10jzmohxtKWnMPZ0WrIf2/I8iZ1NEHlPLMsBemmtZ/k3EgFjFcqXKgnb Jn3ud04uB1mWatQB8MHmxhdCzbxErbZGFdqlJIaz8LV8VVE1PpFQi+5nWXP16UgoZK1d MsJ8ZuD328YEULXSlEmaVWTKYnuUWOilO5Vyy+L5Aw1DBhRnTevbOSIn1I7zntSeHLzG jWdA0LeCEbQs2D+UjBASENLLOp+Ob83dQAJS7Inv9MkCMmxWCvZtNMMPb7zAhT/43Tpc +XEzN6jeGFMtCktj2FOdjwe/GvJ2sAmf+nCBOKsVcbxShaGZEkP8wFBzr8/xXHdJYzpn 5dDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 91si191780pla.131.2017.05.16.12.46.50; Tue, 16 May 2017 12:46:50 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752810AbdEPTqD (ORCPT + 25 others); Tue, 16 May 2017 15:46:03 -0400 Received: from mail-wm0-f42.google.com ([74.125.82.42]:34534 "EHLO mail-wm0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752120AbdEPTqA (ORCPT ); Tue, 16 May 2017 15:46:00 -0400 Received: by mail-wm0-f42.google.com with SMTP id u65so38270360wmu.1 for ; Tue, 16 May 2017 12:45:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=eOcDyYVrpZsSDRDf7rR0QgMnwKs5OdoUyPqaPbgnjEc=; b=aSRoNKgTNkwkDVmgbDr8xk6NdM7LubPIkWCSp6Em8H84v6MnsSUJdfQkzErH3qN5vD 4E5jRSdo2WO0RPK1h52EYI73AT/2+XWGl/ArWulSa1HvB1EHrh4otueFNFR+vJ6q8Pia 6UQgC4iCcRbTDGj1nk3d4auFtO5m+VNc9DGTg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=eOcDyYVrpZsSDRDf7rR0QgMnwKs5OdoUyPqaPbgnjEc=; b=XWBZHEaXGtfrItvAV7JdwILPEohdV7XEBr9YPYGID9wnhS4x2vJpdsmYYzHDEWa5A2 qDKfXduLrzj+APEq8l1WG++FvIoC48M1TMkmEwkBhYSBdclHb/HBimjTajFvhqS65obw +xhxtRbhp6u+Hh+lSYuDWdDwQDEhxadf/2KN2jcrtSurhwEEgSizoQUkXbqHqzK47SzE JgsYokbjLu+YoYWUPVLKv6PoAMEg0c1nMuRGXybWewjdngARZ442OHqhDVs43Jy2vJjp c5NB9Aae1a25u1Xr4rnZLyFp4rQvx37QFoq3986hcDiLiSJ3sWdq/pbnT6utbKKuAk/d rJpQ== X-Gm-Message-State: AODbwcAoLvIxbwhP/G5wlWvF6QJr2F2Y2LipwFDHY4FmgtF5vyu+6weU Xb+UqzM7g60Pv95I X-Received: by 10.28.71.201 with SMTP id m70mr8742798wmi.89.1494963949047; Tue, 16 May 2017 12:45:49 -0700 (PDT) Received: from localhost.localdomain ([2a01:e35:879a:6cd0:78a3:5e81:ca22:9f9b]) by smtp.gmail.com with ESMTPSA id x64sm3219599wrb.10.2017.05.16.12.45.47 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 16 May 2017 12:45:48 -0700 (PDT) From: Daniel Lezcano To: tglx@linutronix.de Cc: Peter Zijlstra , "Rafael J . Wysocki" , Vincent Guittot , Nicolas Pitre , Christoph Hellwig , Hannes Reinecke , Jens Axboe , linux-kernel@vger.kernel.org (open list) Subject: [PATCH V10 2/3] irq: Track the interrupt timings Date: Tue, 16 May 2017 21:44:33 +0200 Message-Id: <1494963880-16160-2-git-send-email-daniel.lezcano@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1494963880-16160-1-git-send-email-daniel.lezcano@linaro.org> References: <1494963880-16160-1-git-send-email-daniel.lezcano@linaro.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The interrupt framework gives a lot of information about each interrupt. It does not keep track of when those interrupts occur though. This patch provides a mean to record the timestamp for each interrupt occurrences in a per-CPU circular buffer to help with the prediction of the next occurrence using a statistical model. Each CPU can store IRQ_TIMINGS_SIZE events , the current value of IRQ_TIMINGS_SIZE is 32. Each event is encoded into a single u64, where the 48 bits are used for the timestamp and the next 16 bits are for the irq number. A static key is introduced so when the irq prediction is switched off at runtime, we can reduce the overhead near to zero. It results in most of the code in internals.h for inline reason and a very few in the new file timings.c. The latter will contain more in the next patch which will provide the statistical model for the next event prediction. Note this code is by default *not* compiled in the kernel. Signed-off-by: Daniel Lezcano Acked-by: Nicolas Pitre Cc: Peter Zijlstra Cc: Rafael J. Wysocki Cc: Vincent Guittot --- V9: - Changed indentation level by inverting the static key condition - Encoded interrupt and timestamp into a u64 variable - Boolean enable instead of refcount for the static key V8: - Replaced percpu field in the irqdesc by a percpu array containing the timings and the associated irq. The function irq_timings_get_next() is no longer needed, so it is removed - Removed all unused code resulting from the conversion irqdesc->percpu timings storage V7: - Mentionned in the irq_timings_get_next() function description, the function must be called inside a rcu read locked section V6: - Renamed handle_irq_timings to record_irq_time - Stored the event time instead of the interval time - Removed the 'timestamp' field from the timings structure - Moved _handle_irq_timings content inside record_irq_time V5: - Changed comment about 'deterministic' as the comment is confusing - Added license comment in the header - Replaced irq_timings_get/put by irq_timings_enable/disable - Moved IRQS_TIMINGS check in the handle_timings inline function - Dropped 'if !prev' as it is pointless - Stored time interval in nsec basis with u64 instead of u32 - Removed redundant store - Removed the math V4: - Added a static key - Added more comments for irq_timings_get_next() - Unified some function names to be prefixed by 'irq_timings_...' - Fixed a rebase error V3: - Replaced ktime_get() by local_clock() - Shared irq are not handled - Simplified code by adding the timing in the irqdesc struct - Added a function to browse the irq timings V2: - Fixed kerneldoc comment - Removed data field from the struct irq timing - Changed the lock section comment - Removed semi-colon style with empty stub - Replaced macro by static inline - Fixed static functions declaration RFC: - initial posting --- include/linux/interrupt.h | 5 +++ kernel/irq/Kconfig | 3 ++ kernel/irq/Makefile | 1 + kernel/irq/handle.c | 2 ++ kernel/irq/internals.h | 84 +++++++++++++++++++++++++++++++++++++++++++++++ kernel/irq/manage.c | 3 ++ kernel/irq/timings.c | 30 +++++++++++++++++ 7 files changed, 128 insertions(+) create mode 100644 kernel/irq/timings.c -- 2.7.4 diff --git a/include/linux/interrupt.h b/include/linux/interrupt.h index 89defd5..4c9d3ca 100644 --- a/include/linux/interrupt.h +++ b/include/linux/interrupt.h @@ -712,6 +712,11 @@ static inline void init_irq_proc(void) } #endif +#ifdef CONFIG_IRQ_TIMINGS +void irq_timings_enable(void); +void irq_timings_disable(void); +#endif + struct seq_file; int show_interrupts(struct seq_file *p, void *v); int arch_show_interrupts(struct seq_file *p, int prec); diff --git a/kernel/irq/Kconfig b/kernel/irq/Kconfig index 3bbfd6a..38e551d 100644 --- a/kernel/irq/Kconfig +++ b/kernel/irq/Kconfig @@ -81,6 +81,9 @@ config GENERIC_MSI_IRQ_DOMAIN config HANDLE_DOMAIN_IRQ bool +config IRQ_TIMINGS + bool + config IRQ_DOMAIN_DEBUG bool "Expose hardware/virtual IRQ mapping via debugfs" depends on IRQ_DOMAIN && DEBUG_FS diff --git a/kernel/irq/Makefile b/kernel/irq/Makefile index 1d3ee31..efb5f14 100644 --- a/kernel/irq/Makefile +++ b/kernel/irq/Makefile @@ -10,3 +10,4 @@ obj-$(CONFIG_PM_SLEEP) += pm.o obj-$(CONFIG_GENERIC_MSI_IRQ) += msi.o obj-$(CONFIG_GENERIC_IRQ_IPI) += ipi.o obj-$(CONFIG_SMP) += affinity.o +obj-$(CONFIG_IRQ_TIMINGS) += timings.o diff --git a/kernel/irq/handle.c b/kernel/irq/handle.c index d3f2490..eb4d3e8 100644 --- a/kernel/irq/handle.c +++ b/kernel/irq/handle.c @@ -138,6 +138,8 @@ irqreturn_t __handle_irq_event_percpu(struct irq_desc *desc, unsigned int *flags unsigned int irq = desc->irq_data.irq; struct irqaction *action; + record_irq_time(desc); + for_each_action_of_desc(desc, action) { irqreturn_t res; diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h index bc226e7..df51b5e0 100644 --- a/kernel/irq/internals.h +++ b/kernel/irq/internals.h @@ -8,6 +8,7 @@ #include #include #include +#include #ifdef CONFIG_SPARSE_IRQ # define IRQ_BITMAP_BITS (NR_IRQS + 8196) @@ -57,6 +58,7 @@ enum { IRQS_WAITING = 0x00000080, IRQS_PENDING = 0x00000200, IRQS_SUSPENDED = 0x00000800, + IRQS_TIMINGS = 0x00001000, }; #include "debug.h" @@ -226,3 +228,85 @@ irq_pm_install_action(struct irq_desc *desc, struct irqaction *action) { } static inline void irq_pm_remove_action(struct irq_desc *desc, struct irqaction *action) { } #endif + +#ifdef CONFIG_IRQ_TIMINGS + +#define IRQ_TIMINGS_SHIFT 5 +#define IRQ_TIMINGS_SIZE (1 << IRQ_TIMINGS_SHIFT) +#define IRQ_TIMINGS_MASK (IRQ_TIMINGS_SIZE - 1) + +struct irq_timings { + u64 values[IRQ_TIMINGS_SIZE]; /* our circular buffer */ + unsigned int count; /* Number of interruptions since last inspection */ +}; + +DECLARE_PER_CPU(struct irq_timings, irq_timings); + +static inline void remove_timings(struct irq_desc *desc) +{ + desc->istate &= ~IRQS_TIMINGS; +} + +static inline void setup_timings(struct irq_desc *desc, struct irqaction *act) +{ + /* + * We don't need the measurement because the idle code already + * knows the next expiry event. + */ + if (act->flags & __IRQF_TIMER) + return; + + desc->istate |= IRQS_TIMINGS; +} + +extern void irq_timings_enable(void); +extern void irq_timings_disable(void); + +extern struct static_key_false irq_timing_enabled; + +/* + * The interrupt number and the timestamp are encoded into a single + * u64 variable to optimize the size. + * 48 bit time stamp and 16 bit IRQ number is way sufficient. + * Who cares an IRQ after 78 hours of idle time? + */ +static inline u64 irq_timing_encode(u64 timestamp, int irq) +{ + return (timestamp << 16) | irq; +} + +static inline void irq_timing_decode(u64 value, u64 *timestamp, int *irq) +{ + *timestamp = value >> 16; + *irq = value & U16_MAX; +} + +/* + * The function record_irq_time is only called in one place in the + * interrupts handler. We want this function always inline so the code + * inside is embedded in the function and the static key branching + * code can act at the higher level. Without the explicit + * __always_inline we can end up with a function call and a small + * overhead in the hotpath for nothing. + */ +static __always_inline void record_irq_time(struct irq_desc *desc) +{ + if (!static_branch_likely(&irq_timing_enabled)) + return; + + if (desc->istate & IRQS_TIMINGS) { + struct irq_timings *timings = this_cpu_ptr(&irq_timings); + + timings->values[timings->count & IRQ_TIMINGS_MASK] = + irq_timing_encode(local_clock(), + irq_desc_get_irq(desc)); + + timings->count++; + } +} +#else +static inline void remove_timings(struct irq_desc *desc) {} +static inline void setup_timings(struct irq_desc *desc, + struct irqaction *act) {}; +static inline void record_irq_time(struct irq_desc *desc) {} +#endif /* CONFIG_IRQ_TIMINGS */ diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c index 057fde8..5dfa63d 100644 --- a/kernel/irq/manage.c +++ b/kernel/irq/manage.c @@ -1372,6 +1372,8 @@ __setup_irq(unsigned int irq, struct irq_desc *desc, struct irqaction *new) raw_spin_unlock_irqrestore(&desc->lock, flags); + setup_timings(desc, new); + /* * Strictly no need to wake it up, but hung_task complains * when no hard interrupt wakes the thread up. @@ -1500,6 +1502,7 @@ static struct irqaction *__free_irq(unsigned int irq, void *dev_id) irq_settings_clr_disable_unlazy(desc); irq_shutdown(desc); irq_release_resources(desc); + remove_timings(desc); } #ifdef CONFIG_SMP diff --git a/kernel/irq/timings.c b/kernel/irq/timings.c new file mode 100644 index 0000000..56cf687 --- /dev/null +++ b/kernel/irq/timings.c @@ -0,0 +1,30 @@ +/* + * linux/kernel/irq/timings.c + * + * Copyright (C) 2016, Linaro Ltd - Daniel Lezcano + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ +#include +#include +#include +#include + +#include "internals.h" + +DEFINE_STATIC_KEY_FALSE(irq_timing_enabled); + +DEFINE_PER_CPU(struct irq_timings, irq_timings); + +void irq_timings_enable(void) +{ + static_branch_enable(&irq_timing_enabled); +} + +void irq_timings_disable(void) +{ + static_branch_disable(&irq_timing_enabled); +}