From patchwork Mon Nov 28 13:20:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 059DBC43217 for ; Mon, 28 Nov 2022 13:13:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231432AbiK1NNr (ORCPT ); Mon, 28 Nov 2022 08:13:47 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39028 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231624AbiK1NNf (ORCPT ); Mon, 28 Nov 2022 08:13:35 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90105120A3; Mon, 28 Nov 2022 05:13:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641213; x=1701177213; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=SGRJsWlMFBDt9YHKWDWh4VtdInjeycHfz68CW0YMoNE=; b=V03l0WTHV5S3FfrqNguzqJB+2yysCVjerBdkNqpNDS0sQrIsS8sZLIaN sSyY2JCBkkPJILUo6kgK304SCUfnjw/Dp4bTn8MgUagOG88dI4R9QQ3pA +9Ym90KFbjrggAafgIiATCA/E7bLKHEHiFn6wHUEXat2Eh/7AW01OpNkB Hp9Y8aH3r/CR5X9ZjydRdr4p2PB/dCa/eLjAHKNVk1DWptAwI+Q9a/ylf m9qfr6Gwcas3FbwWWARjUrG8GmcHV2TT4lfA/xlh1s4Vm33LIMghHmnes 6wI1QXQmUmCLC1Khr1ydkrDAjT5ioPmo0nDJV1apxSmlupUG0bmFcSSgY w==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117071" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117071" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381332" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381332" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:31 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 03/22] sched/core: Initialize the IPC class of a new task Date: Mon, 28 Nov 2022 05:20:41 -0800 Message-Id: <20221128132100.30253-4-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org New tasks shall start life as unclassified. They will be classified by hardware when they run. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * None --- kernel/sched/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 78b2d5cabcc5..8dd43ee05534 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4372,6 +4372,9 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p) p->se.prev_sum_exec_runtime = 0; p->se.nr_migrations = 0; p->se.vruntime = 0; +#ifdef CONFIG_IPC_CLASSES + p->ipcc = IPC_CLASS_UNCLASSIFIED; +#endif INIT_LIST_HEAD(&p->se.group_node); #ifdef CONFIG_FAIR_GROUP_SCHED From patchwork Mon Nov 28 13:20:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC9B0C4708C for ; Mon, 28 Nov 2022 13:13:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231638AbiK1NNt (ORCPT ); Mon, 28 Nov 2022 08:13:49 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40018 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231510AbiK1NNh (ORCPT ); Mon, 28 Nov 2022 08:13:37 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 43DDB1CFCF; Mon, 28 Nov 2022 05:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641215; x=1701177215; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=C8o1EqWi47EMS3NLEboiqwaO6Cqj8PqboSa/beXp5NY=; b=CIquFBhoaOO+JcnTLi7Rt1q5gZb9/AhDXDPV2ooUICbRVFq3AHQFlfSD WoyyIfAMrneTWaWYNTpYjfhLsB+y/nk+cVRBnNqKoOp2lOgePfoZGw9Eo 9ysUk4bmTWUaPYoxRJ/QPmaY4QPZ7LCBei/RpGL09Wzzq62QTeGvXUQH0 VEWBACqkmUvgwONsKSMr/FNFreA2nu+2lwo2Gjv6GjtpINguY9QUm++bO St+BlwWawy1G8AhdGmg491/qACjerpzgug7iF9Tz4rIk2fSJL0eQ1Y0zn kU2NA5JTmZF5apbH9lPXpX6NvQiyrZHSNES3n2f1Rh8P7w0QwrjnBT+K9 g==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117084" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117084" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381335" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381335" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:32 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 04/22] sched/core: Add user_tick as argument to scheduler_tick() Date: Mon, 28 Nov 2022 05:20:42 -0800 Message-Id: <20221128132100.30253-5-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Differentiate between user and kernel ticks so that the scheduler updates the IPC class of the current task during the latter. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * None --- include/linux/sched.h | 2 +- kernel/sched/core.c | 2 +- kernel/time/timer.c | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index 47ae3557ba07..ddabc7449edd 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -293,7 +293,7 @@ enum { TASK_COMM_LEN = 16, }; -extern void scheduler_tick(void); +extern void scheduler_tick(bool user_tick); #define MAX_SCHEDULE_TIMEOUT LONG_MAX diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 8dd43ee05534..8bb6f597c42b 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5487,7 +5487,7 @@ static inline u64 cpu_resched_latency(struct rq *rq) { return 0; } * This function gets called by the timer code, with HZ frequency. * We call it with interrupts disabled. */ -void scheduler_tick(void) +void scheduler_tick(bool user_tick) { int cpu = smp_processor_id(); struct rq *rq = cpu_rq(cpu); diff --git a/kernel/time/timer.c b/kernel/time/timer.c index 63a8ce7177dd..e15e24105891 100644 --- a/kernel/time/timer.c +++ b/kernel/time/timer.c @@ -2073,7 +2073,7 @@ void update_process_times(int user_tick) if (in_irq()) irq_work_tick(); #endif - scheduler_tick(); + scheduler_tick(user_tick); if (IS_ENABLED(CONFIG_POSIX_TIMERS)) run_posix_cpu_timers(); } From patchwork Mon Nov 28 13:20:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67B92C433FE for ; Mon, 28 Nov 2022 13:13:49 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231699AbiK1NNs (ORCPT ); Mon, 28 Nov 2022 08:13:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39380 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231470AbiK1NNg (ORCPT ); Mon, 28 Nov 2022 08:13:36 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30DEFE66; Mon, 28 Nov 2022 05:13:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641215; x=1701177215; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=IPfAz2o04h3wMx9MPOZMNtyWtq/ctVtFFpyHR+YbVE4=; b=MFqN/Seue60Te17UU6g1FX7dvMqWh65ZFIrG2uEvFCXNdZg+iMovFME0 aoiqctpQnp2v+2eGPxRaiZVWzq3FsUh1kE2qwAw4DpOXYdCoPbnEEpKuu K7iRKfzBJQnovxtDG9zZ560tt6wogH3U4R/kjTND/cWPopURCK4PFB1rC GA3marC4rnpWf9pQODn8DBQHUNMdZboRjCj/wlKvAfRtDaIY1IWvgrDhC iAqtbXGDZSzo0IIq3qcOvy+Ui+NeQadxix1PtZEk2Jf+yo4baPCIpwYSL fxHOHSXSmOwfde7g6Ihtw8E4IQ4LGqPbTeRdBBabhKapvm7wzJbO5r1Kf w==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117095" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117095" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:32 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381339" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381339" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:32 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 05/22] sched/core: Update the IPC class of the current task Date: Mon, 28 Nov 2022 05:20:43 -0800 Message-Id: <20221128132100.30253-6-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org When supported, hardware monitors the instruction stream to classify the current task. Hence, at userspace tick, we are ready to read the most recent classification result for the current task. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Removed argument smt_siblings_idle from call to arch_ipcc_update(). * Used the new IPCC interfaces names. --- kernel/sched/core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 8bb6f597c42b..2cd409536b72 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5496,6 +5496,9 @@ void scheduler_tick(bool user_tick) unsigned long thermal_pressure; u64 resched_latency; + if (sched_ipcc_enabled() && user_tick) + arch_update_ipcc(curr); + arch_scale_freq_tick(); sched_clock_tick(); From patchwork Mon Nov 28 13:20:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0421BC433FE for ; Mon, 28 Nov 2022 13:13:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231652AbiK1NN5 (ORCPT ); Mon, 28 Nov 2022 08:13:57 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231577AbiK1NNi (ORCPT ); Mon, 28 Nov 2022 08:13:38 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 812071D678; Mon, 28 Nov 2022 05:13:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641217; x=1701177217; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=GsviGa0DcVIC2LcEIZUUyDmrwHqAyGZci2nKyeSjNto=; b=Mpkr6eFStFskEuiJLekkFdqLeBG5DoXW4DtoVyy0Wywdo0aoFwILAJ4I bk/HdXFqz0Llb7RMabI7fawAxTeOmeYJPr2BjYSKwKXTE5jj/nLXdUm4J /pMf7EY18eE5kB4XjjUVWu4cqbFSkGuQgf6/spZDo1HtJkCw24MpxsABa jmPrWoQgas7ObviX9c7Y9RhK4tSLHGVukeyIbFX0otcEj9cIGGRIjebPn nL7RadxxVaqJMuU84hYaU+86Bt6ENYV9BoCCQyChvKBswqWomcHx2e4k9 vzOvq5Q4PrEEvmPjhOqd7RX7Zi3rHd1td8fNlYS2R55oGBsxICA9UE/fK w==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117125" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117125" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381349" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381349" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:33 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 08/22] sched/fair: Use IPC class to pick the busiest group Date: Mon, 28 Nov 2022 05:20:46 -0800 Message-Id: <20221128132100.30253-9-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org As it iterates, update_sd_pick_busiest() keeps on selecting as busiest sched groups of identical priority. Since both groups have the same priority, either group is a good choice. The IPCC score of the tasks placed a sched group can break this tie. Pick as busiest the sched group that yields a higher IPCC score after load balancing. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Added a comment to clarify why sched_asym_prefer() needs a tie breaker only in update_sd_pick_busiest(). (PeterZ) * Renamed functions for accuracy: sched_asym_class_prefer() >> sched_asym_ipcc_prefer() sched_asym_class_pick() >> sched_asym_ipcc_pick() * Reworded commit message for clarity. --- kernel/sched/fair.c | 75 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 75 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e333f9623b3a..e8b181c31842 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9176,6 +9176,63 @@ static void update_sg_lb_stats_scores(struct sg_lb_ipcc_stats *sgcs, sgs->ipcc_score_before = before; } +/** + * sched_asym_ipcc_prefer - Select a sched group based on its IPCC score + * @a: Load balancing statistics of @sg_a + * @b: Load balancing statistics of @sg_b + * + * Returns: true if preferring @a has a higher IPCC score than @b after + * balancing load. Returns false otherwise. + */ +static bool sched_asym_ipcc_prefer(struct sg_lb_stats *a, + struct sg_lb_stats *b) +{ + if (!sched_ipcc_enabled()) + return false; + + /* @a increases overall throughput after load balance. */ + if (a->ipcc_score_after > b->ipcc_score_after) + return true; + + /* + * If @a and @b yield the same overall throughput, pick @a if + * its current throughput is lower than that of @b. + */ + if (a->ipcc_score_after == b->ipcc_score_after) + return a->ipcc_score_before < b->ipcc_score_before; + + return false; +} + +/** + * sched_asym_ipcc_pick - Select a sched group based on its IPCC score + * @a: A scheduling group + * @b: A second scheduling group + * @a_stats: Load balancing statistics of @a + * @b_stats: Load balancing statistics of @b + * + * Returns: true if @a has the same priority and @a has tasks with IPCC classes + * that yield higher overall throughput after load balance. + * Returns false otherwise. + */ +static bool sched_asym_ipcc_pick(struct sched_group *a, + struct sched_group *b, + struct sg_lb_stats *a_stats, + struct sg_lb_stats *b_stats) +{ + /* + * Only use the class-specific preference selection if both sched + * groups have the same priority. We are not looking at a specific + * CPU. We do not care about the idle state of the groups' + * preferred CPU. + */ + if (arch_asym_cpu_priority(a->asym_prefer_cpu, false) != + arch_asym_cpu_priority(b->asym_prefer_cpu, false)) + return false; + + return sched_asym_ipcc_prefer(a_stats, b_stats); +} + #else /* CONFIG_IPC_CLASSES */ static void update_sg_lb_ipcc_stats(struct sg_lb_ipcc_stats *sgcs, struct rq *rq) @@ -9193,6 +9250,14 @@ static void update_sg_lb_stats_scores(struct sg_lb_ipcc_stats *sgcs, { } +static bool sched_asym_ipcc_pick(struct sched_group *a, + struct sched_group *b, + struct sg_lb_stats *a_stats, + struct sg_lb_stats *b_stats) +{ + return false; +} + #endif /* CONFIG_IPC_CLASSES */ /** @@ -9452,6 +9517,16 @@ static bool update_sd_pick_busiest(struct lb_env *env, sds->busiest->asym_prefer_cpu, false)) return false; + + /* + * Unlike other callers of sched_asym_prefer(), here both @sg + * and @sds::busiest have tasks running. When they have equal + * priority, their IPC class scores can be used to select a + * better busiest. + */ + if (sched_asym_ipcc_pick(sds->busiest, sg, &sds->busiest_stat, sgs)) + return false; + break; case group_misfit_task: From patchwork Mon Nov 28 13:20:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A366C433FE for ; Mon, 28 Nov 2022 13:14:02 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231776AbiK1NN7 (ORCPT ); Mon, 28 Nov 2022 08:13:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39688 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231653AbiK1NNj (ORCPT ); Mon, 28 Nov 2022 08:13:39 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7C4391D67C; Mon, 28 Nov 2022 05:13:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641218; x=1701177218; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=Ts4gZATy16m9TfKQnjzi/8+7PtA65oHBAtgxhfYL63g=; b=dUhTCKb9+Deq3nD5cKNThUR5G3vwq71Bg7saPFZ5um7Wlua7/F/upQPJ RRhTDsz7xqH2nud8wqpz2x+cjS2FUh9DA8y5OaMabpdhwpPlnpD+JebNT I64o+SdsjNXkSJafO02AOp2dNx4l6jQ/wOeGCh/5CTDSkjxgYaL176zvp RnQblt2WgZVSPuYjmIeacSm+RFs6KtICOWyLzMp3JWyV/hBo2RohGJinx tSRsslZnuK83Ho8zgLGyo7HBvbJ3tt7NJQXXrVcDzP2cwqKLlcEIqvrUM 2Ok9NSyUMCJER11DT73mRPUUPMv8NYriZnKCr3Z2Y6oYG6y5TC/jOOWoY Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117148" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117148" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:33 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381357" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381357" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:33 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 10/22] thermal: intel: hfi: Introduce Intel Thread Director classes Date: Mon, 28 Nov 2022 05:20:48 -0800 Message-Id: <20221128132100.30253-11-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org On Intel hybrid parts, each type of CPU has specific performance and energy efficiency capabilities. The Intel Thread Director technology extends the Hardware Feedback Interface (HFI) to provide performance and energy efficiency data for advanced classes of instructions. Add support to parse and parse per-class capabilities. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Removed a now obsolete comment. --- drivers/thermal/intel/intel_hfi.c | 30 ++++++++++++++++++++++++------ 1 file changed, 24 insertions(+), 6 deletions(-) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index a0640f762dc5..df4dc50e19fb 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -79,7 +79,7 @@ union cpuid6_edx { * @ee_cap: Energy efficiency capability * * Capabilities of a logical processor in the HFI table. These capabilities are - * unitless. + * unitless and specific to each HFI class. */ struct hfi_cpu_data { u8 perf_cap; @@ -91,7 +91,8 @@ struct hfi_cpu_data { * @perf_updated: Hardware updated performance capabilities * @ee_updated: Hardware updated energy efficiency capabilities * - * Properties of the data in an HFI table. + * Properties of the data in an HFI table. There exists one header per each + * HFI class. */ struct hfi_hdr { u8 perf_updated; @@ -129,16 +130,21 @@ struct hfi_instance { /** * struct hfi_features - Supported HFI features + * @nr_classes: Number of classes supported * @nr_table_pages: Size of the HFI table in 4KB pages * @cpu_stride: Stride size to locate the capability data of a logical * processor within the table (i.e., row stride) + * @class_stride: Stride size to locate a class within the capability + * data of a logical processor or the HFI table header * @hdr_size: Size of the table header * * Parameters and supported features that are common to all HFI instances */ struct hfi_features { + unsigned int nr_classes; unsigned int nr_table_pages; unsigned int cpu_stride; + unsigned int class_stride; unsigned int hdr_size; }; @@ -325,8 +331,8 @@ static void init_hfi_cpu_index(struct hfi_cpu_info *info) } /* - * The format of the HFI table depends on the number of capabilities that the - * hardware supports. Keep a data structure to navigate the table. + * The format of the HFI table depends on the number of capabilities and classes + * that the hardware supports. Keep a data structure to navigate the table. */ static void init_hfi_instance(struct hfi_instance *hfi_instance) { @@ -507,18 +513,30 @@ static __init int hfi_parse_features(void) /* The number of 4KB pages required by the table */ hfi_features.nr_table_pages = edx.split.table_pages + 1; + /* + * Capability fields of an HFI class are grouped together. Classes are + * contiguous in memory. Hence, use the number of supported features to + * locate a specific class. + */ + hfi_features.class_stride = nr_capabilities; + + /* For now, use only one class of the HFI table */ + hfi_features.nr_classes = 1; + /* * The header contains change indications for each supported feature. * The size of the table header is rounded up to be a multiple of 8 * bytes. */ - hfi_features.hdr_size = DIV_ROUND_UP(nr_capabilities, 8) * 8; + hfi_features.hdr_size = DIV_ROUND_UP(nr_capabilities * + hfi_features.nr_classes, 8) * 8; /* * Data of each logical processor is also rounded up to be a multiple * of 8 bytes. */ - hfi_features.cpu_stride = DIV_ROUND_UP(nr_capabilities, 8) * 8; + hfi_features.cpu_stride = DIV_ROUND_UP(nr_capabilities * + hfi_features.nr_classes, 8) * 8; return 0; } From patchwork Mon Nov 28 13:20:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39316C4167D for ; Mon, 28 Nov 2022 13:14:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231791AbiK1NOC (ORCPT ); Mon, 28 Nov 2022 08:14:02 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39876 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229896AbiK1NNj (ORCPT ); Mon, 28 Nov 2022 08:13:39 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8F1A51DA53; Mon, 28 Nov 2022 05:13:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641218; x=1701177218; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=xRdgNgGdXQiORGcgdnIBmgL4e1VMD8FuTc66jRXlDMQ=; b=eP3if2/17XG58Aa8gCs/yoHPvTQLjlMwsRb/GHZZrNwMSJXhaeb6x5+r uUDoskJFU7j9fkN1QVb7UR4UUy7SsC7JH3IIRwqG7T6gFXG90nEA43f64 mpc+3oJeU8kKtKxxhS+zq1JAfrT7X9cL76ewDU7rMMKQoctRL7iXmgbIm DM/HTwt5k0aVt5fwSJOcEuqaia37mB6umldvwazWGZQM1lpP3DKqg4OjR 9NAsMwIROJJb0MkzT+O2TCop5NfHvBjRkl3QMfAAM7a3T0hgiEbSjbF2M SPBJukrmxYQBZKfWUb3tpNHLQf/+6OMfoPp5EFbuR0SApUEwFuFgs3p9s Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117159" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117159" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381363" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381363" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:33 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 11/22] thermal: intel: hfi: Store per-CPU IPCC scores Date: Mon, 28 Nov 2022 05:20:49 -0800 Message-Id: <20221128132100.30253-12-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The scheduler reads the IPCC scores when balancing load. These reads can be quite frequent. Hardware can also update the HFI table frequently. Concurrent access may cause a lot of contention. It gets worse as the number of CPUs increases. Instead, create separate per-CPU IPCC scores that the scheduler can read without the HFI table lock. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Ricardo Neri --- Changes since v1: * Added this patch. --- drivers/thermal/intel/intel_hfi.c | 38 +++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index df4dc50e19fb..56dba967849c 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -29,6 +29,7 @@ #include #include #include +#include #include #include #include @@ -172,6 +173,35 @@ static struct workqueue_struct *hfi_updates_wq; #define HFI_UPDATE_INTERVAL HZ #define HFI_MAX_THERM_NOTIFY_COUNT 16 +#ifdef CONFIG_IPC_CLASSES +static int __percpu *hfi_ipcc_scores; + +static int alloc_hfi_ipcc_scores(void) +{ + hfi_ipcc_scores = __alloc_percpu(sizeof(*hfi_ipcc_scores) * + hfi_features.nr_classes, + sizeof(*hfi_ipcc_scores)); + + return !hfi_ipcc_scores; +} + +static void set_hfi_ipcc_score(void *caps, int cpu) +{ + int i, *hfi_class = per_cpu_ptr(hfi_ipcc_scores, cpu); + + for (i = 0; i < hfi_features.nr_classes; i++) { + struct hfi_cpu_data *class_caps; + + class_caps = caps + i * hfi_features.class_stride; + WRITE_ONCE(hfi_class[i], class_caps->perf_cap); + } +} + +#else +static int alloc_hfi_ipcc_scores(void) { return 0; } +static void set_hfi_ipcc_score(void *caps, int cpu) { } +#endif /* CONFIG_IPC_CLASSES */ + static void get_hfi_caps(struct hfi_instance *hfi_instance, struct thermal_genl_cpu_caps *cpu_caps) { @@ -194,6 +224,8 @@ static void get_hfi_caps(struct hfi_instance *hfi_instance, cpu_caps[i].efficiency = caps->ee_cap << 2; ++i; + + set_hfi_ipcc_score(caps, cpu); } raw_spin_unlock_irq(&hfi_instance->table_lock); } @@ -572,8 +604,14 @@ void __init intel_hfi_init(void) if (!hfi_updates_wq) goto err_nomem; + if (alloc_hfi_ipcc_scores()) + goto err_ipcc; + return; +err_ipcc: + destroy_workqueue(hfi_updates_wq); + err_nomem: for (j = 0; j < i; ++j) { hfi_instance = &hfi_instances[j]; From patchwork Mon Nov 28 13:20:51 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 46D4AC46467 for ; Mon, 28 Nov 2022 13:14:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231600AbiK1NOK (ORCPT ); Mon, 28 Nov 2022 08:14:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39690 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231531AbiK1NNk (ORCPT ); Mon, 28 Nov 2022 08:13:40 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8FDEFA46F; Mon, 28 Nov 2022 05:13:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641219; x=1701177219; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=ElamVYUpgDqJdxkmls7IkTtE4f7JXYRsemQvg1xBFFc=; b=Ku9+ujnj0jt1DV4ZEqWbFcflA/Y5vkn34TyYe0Bz1h7aC2yrSAKha9K5 N0lM62DkG2hcWWgGMPqRYervSHCKX1FgnHYN3bZzBOWw/nCdF4cZAAabx yZ2Dn00nr8wrnuvhUKP5tlO6rpzbEBdpJCn/lt6G0gFV1IrD6WR2uxqNw jhRlK+Lk6xMXvkEsVtrRQkUG8Kn269QlSWABtxPaP6abUCeGp3gdoEbmN GsQL5cZzxMf4HiBs8/QNTD1JqmCWHnCDH1xGSPH1/BxS8Usx1NMF1s3vj Vy5wNR1uow/QWbttI5ioYBPHhIvvXGgfpnz/YNrnp6OPd0tXLEH2GRQYB Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117179" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117179" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:34 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381369" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381369" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:34 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 13/22] thermal: intel: hfi: Update the IPC class of the current task Date: Mon, 28 Nov 2022 05:20:51 -0800 Message-Id: <20221128132100.30253-14-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Use Intel Thread Director classification to update the IPC class of a task. Implement the needed scheduler interfaces. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Adjusted the result the classification of Intel Thread Director to start at class 1. Class 0 for the scheduler means that the task is unclassified. * Redefined union hfi_thread_feedback_char_msr to ensure all bit-fields are packed. (PeterZ) * Removed CONFIG_INTEL_THREAD_DIRECTOR. (PeterZ) * Shortened the names of the functions that implement IPC classes. * Removed argument smt_siblings_idle from intel_hfi_update_ipcc(). (PeterZ) --- arch/x86/include/asm/topology.h | 8 +++++++ drivers/thermal/intel/intel_hfi.c | 37 +++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) diff --git a/arch/x86/include/asm/topology.h b/arch/x86/include/asm/topology.h index 458c891a8273..cf46a3aea283 100644 --- a/arch/x86/include/asm/topology.h +++ b/arch/x86/include/asm/topology.h @@ -227,4 +227,12 @@ void init_freq_invariance_cppc(void); #define arch_init_invariance_cppc init_freq_invariance_cppc #endif +#if defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) +int intel_hfi_has_ipc_classes(void); +void intel_hfi_update_ipcc(struct task_struct *curr); + +#define arch_has_ipc_classes intel_hfi_has_ipc_classes +#define arch_update_ipcc intel_hfi_update_ipcc +#endif /* defined(CONFIG_IPC_CLASSES) && defined(CONFIG_INTEL_HFI_THERMAL) */ + #endif /* _ASM_X86_TOPOLOGY_H */ diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 56dba967849c..f85394b532a7 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -74,6 +74,17 @@ union cpuid6_edx { u32 full; }; +#ifdef CONFIG_IPC_CLASSES +union hfi_thread_feedback_char_msr { + struct { + u64 classid : 8; + u64 __reserved : 55; + u64 valid : 1; + } split; + u64 full; +}; +#endif + /** * struct hfi_cpu_data - HFI capabilities per CPU * @perf_cap: Performance capability @@ -176,6 +187,32 @@ static struct workqueue_struct *hfi_updates_wq; #ifdef CONFIG_IPC_CLASSES static int __percpu *hfi_ipcc_scores; +int intel_hfi_has_ipc_classes(void) +{ + return cpu_feature_enabled(X86_FEATURE_ITD); +} + +void intel_hfi_update_ipcc(struct task_struct *curr) +{ + union hfi_thread_feedback_char_msr msr; + + /* We should not be here if ITD is not supported. */ + if (!cpu_feature_enabled(X86_FEATURE_ITD)) { + pr_warn_once("task classification requested but not supported!"); + return; + } + + rdmsrl(MSR_IA32_HW_FEEDBACK_CHAR, msr.full); + if (!msr.split.valid) + return; + + /* + * 0 is a valid classification for Intel Thread Director. A scheduler + * IPCC class of 0 means that the task is unclassified. Adjust. + */ + curr->ipcc = msr.split.classid + 1; +} + static int alloc_hfi_ipcc_scores(void) { hfi_ipcc_scores = __alloc_percpu(sizeof(*hfi_ipcc_scores) * From patchwork Mon Nov 28 13:20:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6BAEC4167D for ; Mon, 28 Nov 2022 13:14:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231815AbiK1NON (ORCPT ); Mon, 28 Nov 2022 08:14:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40118 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231567AbiK1NNk (ORCPT ); Mon, 28 Nov 2022 08:13:40 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A62372DCD; Mon, 28 Nov 2022 05:13:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641219; x=1701177219; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=6edsBDhndxufd+3EwJXKfsHyRlYl4SM2zX6wq8H/56U=; b=OzSNHXalsHBm3jZdROfgz1/kUfd36h+u4RkznczD5jae31akEp2HLtoI GU+8kO2KdFMWV/HLxBZHebThJKcj54O8vgTIp6AIv3RjhGDVpcWya+MQ8 YujLMP2fiO6j9iWyTx3yhzoG49/IJ85wd8/UgYwGCOmpMzHVID0HzgDCO 6o7mGpKPv3+u7ia2XzUBjywjqdp4RZbUJG6Ur0LyrhvbAH9eoPw+g5uus gZcbH+OewhalZbqqqkwGIsd/OlZusxkcJDFvV8r0Qt/Yv1NQ7GIvDQGJQ N4kZyvB1OpvwGWwM0hwA9zun0qqIl7cz6tgPTlL07cHkHyEFYOH6a86WR g==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117214" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117214" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381384" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381384" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:35 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 16/22] thermal: intel: hfi: Enable the Intel Thread Director Date: Mon, 28 Nov 2022 05:20:54 -0800 Message-Id: <20221128132100.30253-17-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Enable Intel Thread Director from the CPU hotplug callback: globally from CPU0 and then enable the thread-classification hardware in each logical processor individually. Also, initialize the number of classes supported. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * None --- arch/x86/include/asm/msr-index.h | 2 ++ drivers/thermal/intel/intel_hfi.c | 30 ++++++++++++++++++++++++++++-- 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/msr-index.h b/arch/x86/include/asm/msr-index.h index 37ff47552bcb..96303330223b 100644 --- a/arch/x86/include/asm/msr-index.h +++ b/arch/x86/include/asm/msr-index.h @@ -1075,6 +1075,8 @@ /* Hardware Feedback Interface */ #define MSR_IA32_HW_FEEDBACK_PTR 0x17d0 #define MSR_IA32_HW_FEEDBACK_CONFIG 0x17d1 +#define MSR_IA32_HW_FEEDBACK_THREAD_CONFIG 0x17d4 +#define MSR_IA32_HW_FEEDBACK_CHAR 0x17d2 /* x2APIC locked status */ #define MSR_IA32_XAPIC_DISABLE_STATUS 0xBD diff --git a/drivers/thermal/intel/intel_hfi.c b/drivers/thermal/intel/intel_hfi.c index 1b3fd704ae9a..8287bfd7d6b6 100644 --- a/drivers/thermal/intel/intel_hfi.c +++ b/drivers/thermal/intel/intel_hfi.c @@ -50,6 +50,8 @@ /* Hardware Feedback Interface MSR configuration bits */ #define HW_FEEDBACK_PTR_VALID_BIT BIT(0) #define HW_FEEDBACK_CONFIG_HFI_ENABLE_BIT BIT(0) +#define HW_FEEDBACK_CONFIG_ITD_ENABLE_BIT BIT(1) +#define HW_FEEDBACK_THREAD_CONFIG_ENABLE_BIT BIT(0) /* CPUID detection and enumeration definitions for HFI */ @@ -74,6 +76,15 @@ union cpuid6_edx { u32 full; }; +union cpuid6_ecx { + struct { + u32 dont_care0:8; + u32 nr_classes:8; + u32 dont_care1:16; + } split; + u32 full; +}; + #ifdef CONFIG_IPC_CLASSES union hfi_thread_feedback_char_msr { struct { @@ -495,6 +506,11 @@ void intel_hfi_online(unsigned int cpu) init_hfi_cpu_index(info); + if (cpu_feature_enabled(X86_FEATURE_ITD)) { + msr_val = HW_FEEDBACK_THREAD_CONFIG_ENABLE_BIT; + wrmsrl(MSR_IA32_HW_FEEDBACK_THREAD_CONFIG, msr_val); + } + /* * Now check if the HFI instance of the package/die of @cpu has been * initialized (by checking its header). In such case, all we have to @@ -550,6 +566,10 @@ void intel_hfi_online(unsigned int cpu) */ rdmsrl(MSR_IA32_HW_FEEDBACK_CONFIG, msr_val); msr_val |= HW_FEEDBACK_CONFIG_HFI_ENABLE_BIT; + + if (cpu_feature_enabled(X86_FEATURE_ITD)) + msr_val |= HW_FEEDBACK_CONFIG_ITD_ENABLE_BIT; + wrmsrl(MSR_IA32_HW_FEEDBACK_CONFIG, msr_val); unlock: @@ -629,8 +649,14 @@ static __init int hfi_parse_features(void) */ hfi_features.class_stride = nr_capabilities; - /* For now, use only one class of the HFI table */ - hfi_features.nr_classes = 1; + if (cpu_feature_enabled(X86_FEATURE_ITD)) { + union cpuid6_ecx ecx; + + ecx.full = cpuid_ecx(CPUID_HFI_LEAF); + hfi_features.nr_classes = ecx.split.nr_classes; + } else { + hfi_features.nr_classes = 1; + } /* * The header contains change indications for each supported feature. From patchwork Mon Nov 28 13:20:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629379 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C6E68C4167D for ; Mon, 28 Nov 2022 13:14:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231831AbiK1NOT (ORCPT ); Mon, 28 Nov 2022 08:14:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231599AbiK1NNl (ORCPT ); Mon, 28 Nov 2022 08:13:41 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BF74122D; Mon, 28 Nov 2022 05:13:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641220; x=1701177220; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=p7BCBvy/ijiGArW5gb4LxSYPxM/Arci+QhS/bMOw9bo=; b=gSFrBgEqXr5rBXnI3IFSYy+cx+9uhnT4QhrkD3L0uF9xdrzRceWmNDjh ZKMKJJarF4v+ACLXe8Sh03JqoLVY8TjXdK2yU0sQZONt8nuVXGDSe4apo rMCVuxYe9m8XJw9xn4W49uNfPB2J2Sk9DZ0V6QfNNjlR37RAMisrOlJST iWwWrHksfFikdxNVa8UqgYtvEXKkVh2EaXDlzCMrFVJ3eLAYqq6KZqXws c5xI2L11dv4buBjsJwsTvn0AyDd0VoIaq6Vz4is3dg1nQBE+rKPV+VXBV 2h9YaQnS9bR5/oDopZ3nPh2pflANk57kPXVFFnnMCKzINW1Zl8GrG2FFQ Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117226" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117226" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381387" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381387" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:35 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 17/22] sched/task_struct: Add helpers for IPC classification Date: Mon, 28 Nov 2022 05:20:55 -0800 Message-Id: <20221128132100.30253-18-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org The unprocessed classification that hardware provides for a task may not be usable by the scheduler: the classification may change too frequently or architectures may want to consider extra factors. For instance, some processors with Intel Thread Director need to consider the state of the SMT siblings of a core. Provide per-task helper variables that architectures can use to post- process the classification that hardware provides. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Used bit-fields to fit all the IPC class data in 4 bytes. (PeterZ) * Shortened names of the helpers. * Renamed helpers with the ipcc_ prefix. * Reworded commit message for clarity --- include/linux/sched.h | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/include/linux/sched.h b/include/linux/sched.h index ddabc7449edd..8a99aa316c37 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1532,7 +1532,17 @@ struct task_struct { * A hardware-defined classification of task based on the number * of instructions per cycle. */ - unsigned int ipcc; + unsigned int ipcc : 9; + /* + * A candidate classification that arch-specific implementations + * qualify for correctness. + */ + unsigned int ipcc_tmp : 9; + /* + * Counter to filter out transient the candidate classification + * of a task + */ + unsigned int ipcc_cntr : 14; #endif /* From patchwork Mon Nov 28 13:20:56 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629378 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1092BC4167D for ; Mon, 28 Nov 2022 13:14:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230502AbiK1NOU (ORCPT ); Mon, 28 Nov 2022 08:14:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39894 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231607AbiK1NNl (ORCPT ); Mon, 28 Nov 2022 08:13:41 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C07FE1CFC2; Mon, 28 Nov 2022 05:13:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641220; x=1701177220; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=KWVB7nlsf20aLmkJw9PMc9fnASyxtCkpqLu/IPZ9sIY=; b=gNCDRFydNHxGrZ32vgvgI8pBRG7LjpNHtdZzMzyuZ8fXB50Uwp9lS3kA zRoRQPUFn5de46HRr7ZQ0XtAZjS5JTFIKMxG6YaJQ8yhdeAb692U3a5lz ddTIozmiUIT/bUIamOMSsrzn2+clHiq2Bbf27x66B0kySRWFuc+HrSSI3 AUQXSVS4riThDleOtdzqdRFCp1EU8JYHP/JptWOUl42lRUMyceLqsS+cS WlehERQ4rOdZkTyepAbsEHN1ulF3wM+Q7NBexfVzod03maujcwqaCd/fT MQl2rA2qzsFcYqON4Vzc9K56Rj4InE42wLT0QjnYCqvOH+kp1+kp0uZ6x A==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117238" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117238" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381390" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381390" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:35 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 18/22] sched/core: Initialize helpers of task classification Date: Mon, 28 Nov 2022 05:20:56 -0800 Message-Id: <20221128132100.30253-19-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Just as tasks start life unclassified, initialize the classification auxiliar variables. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * None --- kernel/sched/core.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 2cd409536b72..0406b07c51a0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4374,6 +4374,8 @@ static void __sched_fork(unsigned long clone_flags, struct task_struct *p) p->se.vruntime = 0; #ifdef CONFIG_IPC_CLASSES p->ipcc = IPC_CLASS_UNCLASSIFIED; + p->ipcc_tmp = IPC_CLASS_UNCLASSIFIED; + p->ipcc_cntr = 0; #endif INIT_LIST_HEAD(&p->se.group_node); From patchwork Mon Nov 28 13:20:59 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ricardo Neri X-Patchwork-Id: 629377 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AF62C433FE for ; Mon, 28 Nov 2022 13:14:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231847AbiK1NOY (ORCPT ); Mon, 28 Nov 2022 08:14:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:40170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231666AbiK1NNm (ORCPT ); Mon, 28 Nov 2022 08:13:42 -0500 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 907011B9C4; Mon, 28 Nov 2022 05:13:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1669641221; x=1701177221; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=PIe0SGbn3SQDW/BsVmI7VngYYmBv3E8AzHVNV7RbEPw=; b=E/Bedru4SRJxz0kyZ8401LjnEyu0QpOhrf3kb9UrBjy1Xr5fxQb/8Y2k znOEIgvlr+EQq7fnSoGKQ9HS+kbJuBc3cvjqN9/zDtWDt/gr5pySlWN5X 0rJycqYZZu+wdBpBKCT/M6ky265Kq+lHdjd1JxCz7D9mgoPRJqc7mTs2O QflTBSm/RsqFXxttcvIUT6XTO5HcNjyung4F2iygT9MBaxN5JbpAzjRcz fh+bGDm8l3q5wzjoHUwN4pdvQBa/cCIRA/i+TWr+AX4hieC6YdFoldlu2 6qTAEBSoiYgWZTibirPYMYVlJ/VVZ/7OgTjmlwodhbpsgTM+brpWv09gm w==; X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="401117273" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="401117273" Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 28 Nov 2022 05:13:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10544"; a="749381402" X-IronPort-AV: E=Sophos;i="5.96,200,1665471600"; d="scan'208";a="749381402" Received: from ranerica-svr.sc.intel.com ([172.25.110.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Nov 2022 05:13:36 -0800 From: Ricardo Neri To: "Peter Zijlstra (Intel)" , Juri Lelli , Vincent Guittot Cc: Ricardo Neri , "Ravi V. Shankar" , Ben Segall , Daniel Bristot de Oliveira , Dietmar Eggemann , Len Brown , Mel Gorman , "Rafael J. Wysocki" , Srinivas Pandruvada , Steven Rostedt , Tim Chen , Valentin Schneider , x86@kernel.org, "Joel Fernandes (Google)" , linux-kernel@vger.kernel.org, linux-pm@vger.kernel.org, Ricardo Neri , "Tim C . Chen" Subject: [PATCH v2 21/22] x86/hreset: Configure history reset Date: Mon, 28 Nov 2022 05:20:59 -0800 Message-Id: <20221128132100.30253-22-ricardo.neri-calderon@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> References: <20221128132100.30253-1-ricardo.neri-calderon@linux.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Configure the MSR that controls the behavior of HRESET on each logical processor. Cc: Ben Segall Cc: Daniel Bristot de Oliveira Cc: Dietmar Eggemann Cc: Joel Fernandes (Google) Cc: Len Brown Cc: Mel Gorman Cc: Rafael J. Wysocki Cc: Srinivas Pandruvada Cc: Steven Rostedt Cc: Tim C. Chen Cc: Valentin Schneider Cc: x86@kernel.org Cc: linux-pm@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Ricardo Neri --- Changes since v1: * Marked hardware_history_features as __ro_after_init instead of __read_mostly. (PeterZ) --- arch/x86/kernel/cpu/common.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 73cc546e024d..f8630da2a6dd 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -412,6 +412,26 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) cr4_clear_bits(X86_CR4_UMIP); } +static u32 hardware_history_features __ro_after_init; + +static __always_inline void setup_hreset(struct cpuinfo_x86 *c) +{ + if (!cpu_feature_enabled(X86_FEATURE_HRESET)) + return; + + /* + * Use on all CPUs the hardware history features that the boot + * CPU supports. + */ + if (c == &boot_cpu_data) + hardware_history_features = cpuid_ebx(0x20); + + if (!hardware_history_features) + return; + + wrmsrl(MSR_IA32_HW_HRESET_ENABLE, hardware_history_features); +} + /* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask = X86_CR4_SMEP | X86_CR4_SMAP | X86_CR4_UMIP | @@ -1844,10 +1864,11 @@ static void identify_cpu(struct cpuinfo_x86 *c) /* Disable the PN if appropriate */ squash_the_stupid_serial_number(c); - /* Set up SMEP/SMAP/UMIP */ + /* Set up SMEP/SMAP/UMIP/HRESET */ setup_smep(c); setup_smap(c); setup_umip(c); + setup_hreset(c); /* Enable FSGSBASE instructions if available. */ if (cpu_has(c, X86_FEATURE_FSGSBASE)) {