From patchwork Tue Dec 8 04:16:55 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 339690 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp3368021jai; Mon, 7 Dec 2020 20:18:50 -0800 (PST) X-Google-Smtp-Source: ABdhPJzaQGzdIgJcYuzpbImONXmHO4ROAgloTzO6vd08Dnja2O+4MCQLLaUfBBityO2l9vlfTFF8 X-Received: by 2002:a50:f40e:: with SMTP id r14mr22201491edm.5.1607401130049; Mon, 07 Dec 2020 20:18:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1607401130; cv=none; d=google.com; s=arc-20160816; b=O2kz0j6i520aWYnTbVl4sdDJnLD9ZE4VYxxMSABrSYA8fI1PZsRH2hd5ODlciQNEit gU7MfZVxEd5GWlGbS8Uj5ZmMfiDk36aKkJZuZ8m+O8MC0m1zpxA0gwJe2CfRL2aRw9zA uN8C6tBhZfd7TtmN5/BGAzZh7krlx30JfmJsVKKIO/aLTc4e4Q8J3u7VADYTLHOwQvY+ YHj0UHMvnh4QEJMX5X7YHGK+IM1+x2deuZbz3k0NkR4HREsS+A7B9kGow5RPmbyIzi2F /YQFWwtruT/Djf+64i5vUf6BwPA7bhmo1TdHpWfYfIz383lH7wHONGmP7WqZnyH9fvgI 9VFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=kvuVE1MVkJSAH2h3Ge77/KZl7+7QpoDOc2ID4uw5A4o=; b=E2hhpQ+SwDeZ8yWe7XAuHJHPIEPmtsvpy5XPxgbuENywKyUSuGXJNM/1O6Nkbm+/7d kiNotTiZOK4BcuG2Ss6zhoB1aLbtUOuSRwF4I5X183ADS8v7NBbG/DPtN7b3PfwHR/km RioGItA4Ho49l2//H/OYIs96SFi4noRrTLhhd48zzIuJe/FSXB9Xa8E3n4WU23vGQUEw bXHypo2+9Mn8bdTTK3WM6xwSjaG0XP2kCTjUWSgWaRrRJwT1YvifnICI748XP56t7Wuv lb7+oSwuBj4R/BpuSacja+DT6SNq/RSUmWCkMUeuW3CZWsBAoZKTnAJFPME7Md9bYWOE +oXQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uUXuveKM; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id pw18si4674329ejb.163.2020.12.07.20.18.49; Mon, 07 Dec 2020 20:18:50 -0800 (PST) Received-SPF: pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uUXuveKM; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726379AbgLHERs (ORCPT + 8 others); Mon, 7 Dec 2020 23:17:48 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726359AbgLHERr (ORCPT ); Mon, 7 Dec 2020 23:17:47 -0500 Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com [IPv6:2607:f8b0:4864:20::541]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49EA4C061794 for ; Mon, 7 Dec 2020 20:17:07 -0800 (PST) Received: by mail-pg1-x541.google.com with SMTP id f17so11024109pge.6 for ; Mon, 07 Dec 2020 20:17:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kvuVE1MVkJSAH2h3Ge77/KZl7+7QpoDOc2ID4uw5A4o=; b=uUXuveKMZvI7RSNnbtIfAMYE2JD4w2JUi5rfH6bEZG5jIzkbEFvfTDsI5RK/GTJhiT EawXZybNQ0ZdvWDRnulgl++8ip40kRwtC0nyJJA9SpyKMEfOLUQN5enjZVrTaneuwCwV lWiQKI8/bHcXcVy/DJS8YC7P8RNfVR2SPSQvtrpVU+AhMbL2plpWmKZWl0+qv5/kObWF plwmAcPDuYtNukan8pLEZ9PdzbvCu3i/K94iIdJ1FcqmYrUlzISlqPuLAI/GTokGUMsf bphMFys20p+R8ekF4al57DAUZAZuFK14GV8mpZzjgqo8DaDbt0FA+TURqTvwF6YPNKGB 2QFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kvuVE1MVkJSAH2h3Ge77/KZl7+7QpoDOc2ID4uw5A4o=; b=FVxOWrU+YeFf742drM0zH80W2OdsI1hocerJ5HmwcX+Uh/jBmy3G7g367mmiSYqfrO Y1XP2xW8Lk2R7K7Dcs4KVSORTJkfR5noTv6hm8RCKEXVtGs1oRDfr4UZHZI0qYKdTBHb Ewv3xYhYruSr8rSHBueSCMyzyktwv/JwuzPmbE2fANRVuGgThAkt6di9YrD6hch7EH+v DDyqPxEhgJbUuOZ74sq+KB5X//DQBWVC9GzfJeaJeITriYzjTML18LJk2D99YtDe0+dd /Y3s3yAirpV8h7Xq+XwtG8WCHUKBZI+DcTyqc3RcpIXuHu+K56811UvuMlBawaLnkPrp k3CQ== X-Gm-Message-State: AOAM5322XmtCX+4ze8XyWK5uBQmjhLXccY/1NiujO+dFZEVhbBVpte0v p96MLT10h899UfTciKSnIjRf/A== X-Received: by 2002:a17:90b:16cd:: with SMTP id iy13mr2237290pjb.182.1607401026662; Mon, 07 Dec 2020 20:17:06 -0800 (PST) Received: from localhost ([122.172.136.109]) by smtp.gmail.com with ESMTPSA id u3sm1116335pjf.52.2020.12.07.20.17.05 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:17:05 -0800 (PST) From: Viresh Kumar To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , "Rafael J. Wysocki" , Viresh Kumar Cc: linux-kernel@vger.kernel.org, Quentin Perret , Lukasz Luba , "Rafael J . Wysocki" , linux-pm@vger.kernel.org Subject: [PATCH V5 1/3] sched/core: Move schedutil_cpu_util() to core.c Date: Tue, 8 Dec 2020 09:46:55 +0530 Message-Id: X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org There is nothing schedutil specific in schedutil_cpu_util(), move it to core.c and define it only for CONFIG_SMP. Acked-by: Rafael J. Wysocki Signed-off-by: Viresh Kumar --- kernel/sched/core.c | 108 +++++++++++++++++++++++++++++++ kernel/sched/cpufreq_schedutil.c | 106 ------------------------------ kernel/sched/sched.h | 12 +--- 3 files changed, 109 insertions(+), 117 deletions(-) -- 2.25.0.rc1.19.g042ed3e048af diff --git a/kernel/sched/core.c b/kernel/sched/core.c index d2003a7d5ab5..b81265aec4a0 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5117,6 +5117,114 @@ struct task_struct *idle_task(int cpu) return cpu_rq(cpu)->idle; } +#ifdef CONFIG_SMP +/* + * This function computes an effective utilization for the given CPU, to be + * used for frequency selection given the linear relation: f = u * f_max. + * + * The scheduler tracks the following metrics: + * + * cpu_util_{cfs,rt,dl,irq}() + * cpu_bw_dl() + * + * Where the cfs,rt and dl util numbers are tracked with the same metric and + * synchronized windows and are thus directly comparable. + * + * The cfs,rt,dl utilization are the running times measured with rq->clock_task + * which excludes things like IRQ and steal-time. These latter are then accrued + * in the irq utilization. + * + * The DL bandwidth number otoh is not a measured metric but a value computed + * based on the task model parameters and gives the minimal utilization + * required to meet deadlines. + */ +unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, + unsigned long max, enum schedutil_type type, + struct task_struct *p) +{ + unsigned long dl_util, util, irq; + struct rq *rq = cpu_rq(cpu); + + if (!uclamp_is_used() && + type == FREQUENCY_UTIL && rt_rq_is_runnable(&rq->rt)) { + return max; + } + + /* + * Early check to see if IRQ/steal time saturates the CPU, can be + * because of inaccuracies in how we track these -- see + * update_irq_load_avg(). + */ + irq = cpu_util_irq(rq); + if (unlikely(irq >= max)) + return max; + + /* + * Because the time spend on RT/DL tasks is visible as 'lost' time to + * CFS tasks and we use the same metric to track the effective + * utilization (PELT windows are synchronized) we can directly add them + * to obtain the CPU's actual utilization. + * + * CFS and RT utilization can be boosted or capped, depending on + * utilization clamp constraints requested by currently RUNNABLE + * tasks. + * When there are no CFS RUNNABLE tasks, clamps are released and + * frequency will be gracefully reduced with the utilization decay. + */ + util = util_cfs + cpu_util_rt(rq); + if (type == FREQUENCY_UTIL) + util = uclamp_rq_util_with(rq, util, p); + + dl_util = cpu_util_dl(rq); + + /* + * For frequency selection we do not make cpu_util_dl() a permanent part + * of this sum because we want to use cpu_bw_dl() later on, but we need + * to check if the CFS+RT+DL sum is saturated (ie. no idle time) such + * that we select f_max when there is no idle time. + * + * NOTE: numerical errors or stop class might cause us to not quite hit + * saturation when we should -- something for later. + */ + if (util + dl_util >= max) + return max; + + /* + * OTOH, for energy computation we need the estimated running time, so + * include util_dl and ignore dl_bw. + */ + if (type == ENERGY_UTIL) + util += dl_util; + + /* + * There is still idle time; further improve the number by using the + * irq metric. Because IRQ/steal time is hidden from the task clock we + * need to scale the task numbers: + * + * max - irq + * U' = irq + --------- * U + * max + */ + util = scale_irq_capacity(util, irq, max); + util += irq; + + /* + * Bandwidth required by DEADLINE must always be granted while, for + * FAIR and RT, we use blocked utilization of IDLE CPUs as a mechanism + * to gracefully reduce the frequency when no tasks show up for longer + * periods of time. + * + * Ideally we would like to set bw_dl as min/guaranteed freq and util + + * bw_dl as requested freq. However, cpufreq is not yet ready for such + * an interface. So, we only do the latter for now. + */ + if (type == FREQUENCY_UTIL) + util += cpu_bw_dl(rq); + + return min(max, util); +} +#endif /* CONFIG_SMP */ + /** * find_process_by_pid - find a process with a matching PID value. * @pid: the pid in question. diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index e254745a82cb..2d44befb322b 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -169,112 +169,6 @@ static unsigned int get_next_freq(struct sugov_policy *sg_policy, return cpufreq_driver_resolve_freq(policy, freq); } -/* - * This function computes an effective utilization for the given CPU, to be - * used for frequency selection given the linear relation: f = u * f_max. - * - * The scheduler tracks the following metrics: - * - * cpu_util_{cfs,rt,dl,irq}() - * cpu_bw_dl() - * - * Where the cfs,rt and dl util numbers are tracked with the same metric and - * synchronized windows and are thus directly comparable. - * - * The cfs,rt,dl utilization are the running times measured with rq->clock_task - * which excludes things like IRQ and steal-time. These latter are then accrued - * in the irq utilization. - * - * The DL bandwidth number otoh is not a measured metric but a value computed - * based on the task model parameters and gives the minimal utilization - * required to meet deadlines. - */ -unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, - unsigned long max, enum schedutil_type type, - struct task_struct *p) -{ - unsigned long dl_util, util, irq; - struct rq *rq = cpu_rq(cpu); - - if (!uclamp_is_used() && - type == FREQUENCY_UTIL && rt_rq_is_runnable(&rq->rt)) { - return max; - } - - /* - * Early check to see if IRQ/steal time saturates the CPU, can be - * because of inaccuracies in how we track these -- see - * update_irq_load_avg(). - */ - irq = cpu_util_irq(rq); - if (unlikely(irq >= max)) - return max; - - /* - * Because the time spend on RT/DL tasks is visible as 'lost' time to - * CFS tasks and we use the same metric to track the effective - * utilization (PELT windows are synchronized) we can directly add them - * to obtain the CPU's actual utilization. - * - * CFS and RT utilization can be boosted or capped, depending on - * utilization clamp constraints requested by currently RUNNABLE - * tasks. - * When there are no CFS RUNNABLE tasks, clamps are released and - * frequency will be gracefully reduced with the utilization decay. - */ - util = util_cfs + cpu_util_rt(rq); - if (type == FREQUENCY_UTIL) - util = uclamp_rq_util_with(rq, util, p); - - dl_util = cpu_util_dl(rq); - - /* - * For frequency selection we do not make cpu_util_dl() a permanent part - * of this sum because we want to use cpu_bw_dl() later on, but we need - * to check if the CFS+RT+DL sum is saturated (ie. no idle time) such - * that we select f_max when there is no idle time. - * - * NOTE: numerical errors or stop class might cause us to not quite hit - * saturation when we should -- something for later. - */ - if (util + dl_util >= max) - return max; - - /* - * OTOH, for energy computation we need the estimated running time, so - * include util_dl and ignore dl_bw. - */ - if (type == ENERGY_UTIL) - util += dl_util; - - /* - * There is still idle time; further improve the number by using the - * irq metric. Because IRQ/steal time is hidden from the task clock we - * need to scale the task numbers: - * - * max - irq - * U' = irq + --------- * U - * max - */ - util = scale_irq_capacity(util, irq, max); - util += irq; - - /* - * Bandwidth required by DEADLINE must always be granted while, for - * FAIR and RT, we use blocked utilization of IDLE CPUs as a mechanism - * to gracefully reduce the frequency when no tasks show up for longer - * periods of time. - * - * Ideally we would like to set bw_dl as min/guaranteed freq and util + - * bw_dl as requested freq. However, cpufreq is not yet ready for such - * an interface. So, we only do the latter for now. - */ - if (type == FREQUENCY_UTIL) - util += cpu_bw_dl(rq); - - return min(max, util); -} - static unsigned long sugov_get_util(struct sugov_cpu *sg_cpu) { struct rq *rq = cpu_rq(sg_cpu->cpu); diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index df80bfcea92e..0db6bcf0881f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2484,7 +2484,6 @@ static inline unsigned long capacity_orig_of(int cpu) { return cpu_rq(cpu)->cpu_capacity_orig; } -#endif /** * enum schedutil_type - CPU utilization type @@ -2501,8 +2500,6 @@ enum schedutil_type { ENERGY_UTIL, }; -#ifdef CONFIG_CPU_FREQ_GOV_SCHEDUTIL - unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, unsigned long max, enum schedutil_type type, struct task_struct *p); @@ -2533,14 +2530,7 @@ static inline unsigned long cpu_util_rt(struct rq *rq) { return READ_ONCE(rq->avg_rt.util_avg); } -#else /* CONFIG_CPU_FREQ_GOV_SCHEDUTIL */ -static inline unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, - unsigned long max, enum schedutil_type type, - struct task_struct *p) -{ - return 0; -} -#endif /* CONFIG_CPU_FREQ_GOV_SCHEDUTIL */ +#endif #ifdef CONFIG_HAVE_SCHED_AVG_IRQ static inline unsigned long cpu_util_irq(struct rq *rq) From patchwork Tue Dec 8 04:16:56 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 339692 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp3368044jai; Mon, 7 Dec 2020 20:18:52 -0800 (PST) X-Google-Smtp-Source: ABdhPJyPS1BNL4LBvda5yW2hou0mVyuvjcpqzwZiZLgF3dNkkGQxp+9SYOPoKrAvso2k/uWnlURl X-Received: by 2002:a17:906:fa88:: with SMTP id lt8mr21904314ejb.408.1607401132321; Mon, 07 Dec 2020 20:18:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1607401132; cv=none; d=google.com; s=arc-20160816; b=XgKcTv99hjH5L7R5kvx0SVWcrl7tACJAQcvIYMpM/3QFPfJ2X/FoAWx9npFP3Fy911 BItT6/r5Cd36wSvsxl0MH9rRutadQQqo3L7Ixi1BwMw8YxZ/OFetg2GTUqcbf0rFIxhw 7okBU9TomZxfVhUZhvNF+CyNiOWJs0Wdef+dIEVq7HEve4IuvtxByHXmlxVGRP5FyEz2 f1FatoOgoETrbW9nBN1vjIboCaIPHBddBpoFwnOGmMdRfoHIbgmhBwN2wDQPYQ6CCZ9r J/ReQDMlQu4RBtIJPEuJmpcyLwN7P65tKLgapRoqfZKVe51kc53R0x5qUQxR4UasyhVx q8HQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=BgroMCzBx2iJ+H86ym6L2CJTtyoziwkrOuTzWQtpDQI=; b=gpHRopPCW0he4P5JE71oFpniNYAkdwGT1A9EB76R9ezq0LqtQABH54sgKlfyNp+Dme 757B0hKvsD7cbuoNCnNsgGbHLJDrAu43KPZZXGadSJbOYKI6C+ZwpgNRwLjUV3WqyGQu N0QtlNaQOndU4HusrDmtC7IxoELwz5o/B3fHdc2S8HXkqlm/0hd9lmFfTV631erEUPDd aDmVJsJE/zYNQX8YmI6Lxe/WqRusMOBJdpUJhXAB31hnxNqN0PSeCMCBOXO6mJHxPvRQ hiC4/Fl+OpdLTlgIxbb8BPxG6ZQrXuHEB6GrZTPvo8KuaQsNRdDe4yjou7PQNsPE0j7Z 4ptQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uQtBmECu; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id pw18si4674329ejb.163.2020.12.07.20.18.52; Mon, 07 Dec 2020 20:18:52 -0800 (PST) Received-SPF: pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uQtBmECu; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726779AbgLHESY (ORCPT + 8 others); Mon, 7 Dec 2020 23:18:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56210 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726794AbgLHESY (ORCPT ); Mon, 7 Dec 2020 23:18:24 -0500 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8A14EC0617A6 for ; Mon, 7 Dec 2020 20:17:10 -0800 (PST) Received: by mail-pl1-x643.google.com with SMTP id t6so757879plq.1 for ; Mon, 07 Dec 2020 20:17:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BgroMCzBx2iJ+H86ym6L2CJTtyoziwkrOuTzWQtpDQI=; b=uQtBmECumfix2CVr91vbg+9llcJiypEeUz0345IueFtDzmXZlPAjtJyohrI28xhdRP jRrbaOGvoUJY7kqpFCK3qB+vjR5NxuMl9GocAKXYLkVTtsr0bnewBp31OGYx/DpxvV24 NX9Y26SZAanZhM4fSLvXgyhslSULTS+xX0YktbEWQV3MwOx5h+NI1H+PgrNKOZhYJAFL hK2/U/2UVpZ3Wd5Bf3OW8/E/jSVRf98I+1nsusXozTrJ0P3evMaU10teFpxOeOe9tGuS N9FTQVfI2x/wOK6XO9Zk6x1X7FTj9cU3+nrQrQwFlcDPdMvSd7gW5SmCffbVFf1Kx1tJ jaBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BgroMCzBx2iJ+H86ym6L2CJTtyoziwkrOuTzWQtpDQI=; b=lTsvBgqT/z9xpqIajtbvGUaV4/CNqgQcqCbiH3Na1PEC+HbEP6glFBoqtJ7hN3PcZ1 j034Rgpg+WPuva+Okm9iMbs9XMSj03cSCWzk7wYl43XgYijiOmIeM4WXWT6iyVq1us27 2KiKC8FNsHhXCE99CpoFgG888D8SpgR1OkGIhHuCRXRTEiVco/Oy60F1/T7xA/DHvb+V B1oSFL2WTft0Y5DHMhy200LhLdPZrxy7hlWJr+VG2DjvkVVIFAsCH+DiCxZyYwcFe//O bbUjG/Z+0UVOCSeWLFoWlGQgCeY/tegULlPk8KHmVU9uHWK2XqS0Ye8DG41lEr4YAhu0 XEIg== X-Gm-Message-State: AOAM530BVW6tltNXWCALN3R/qqvC/8RtTwm/fiSrENniFMitoMzQPnU1 c/yeu/VmO2hC2oTbzH5wznDPbg== X-Received: by 2002:a17:902:bb92:b029:d9:e9bf:b775 with SMTP id m18-20020a170902bb92b02900d9e9bfb775mr19582388pls.24.1607401030082; Mon, 07 Dec 2020 20:17:10 -0800 (PST) Received: from localhost ([122.172.136.109]) by smtp.gmail.com with ESMTPSA id x128sm16035595pfx.52.2020.12.07.20.17.09 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:17:09 -0800 (PST) From: Viresh Kumar To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , "Rafael J. Wysocki" , Viresh Kumar Cc: linux-kernel@vger.kernel.org, Quentin Perret , Lukasz Luba , "Rafael J . Wysocki" , linux-pm@vger.kernel.org Subject: [PATCH V5 2/3] sched/core: Rename schedutil_cpu_util() and allow rest of the kernel to use it Date: Tue, 8 Dec 2020 09:46:56 +0530 Message-Id: X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org There is nothing schedutil specific in schedutil_cpu_util(), rename it to effective_cpu_util(). Also create and expose another wrapper sched_cpu_util() which can be used by other parts of the kernel, like thermal core (that will be done in a later commit). Acked-by: Rafael J. Wysocki Signed-off-by: Viresh Kumar --- include/linux/sched.h | 5 +++++ kernel/sched/core.c | 10 ++++++++-- kernel/sched/cpufreq_schedutil.c | 2 +- kernel/sched/fair.c | 6 +++--- kernel/sched/sched.h | 10 +++++----- 5 files changed, 22 insertions(+), 11 deletions(-) -- 2.25.0.rc1.19.g042ed3e048af diff --git a/include/linux/sched.h b/include/linux/sched.h index 063cd120b459..3a2fe4fe6155 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -1926,6 +1926,11 @@ extern long sched_getaffinity(pid_t pid, struct cpumask *mask); #define TASK_SIZE_OF(tsk) TASK_SIZE #endif +#ifdef CONFIG_SMP +/* Returns effective CPU energy utilization, as seen by the scheduler */ +unsigned long sched_cpu_util(int cpu, unsigned long max); +#endif /* CONFIG_SMP */ + #ifdef CONFIG_RSEQ /* diff --git a/kernel/sched/core.c b/kernel/sched/core.c index b81265aec4a0..0a24b93dd7f3 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -5138,8 +5138,8 @@ struct task_struct *idle_task(int cpu) * based on the task model parameters and gives the minimal utilization * required to meet deadlines. */ -unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, - unsigned long max, enum schedutil_type type, +unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, + unsigned long max, enum cpu_util_type type, struct task_struct *p) { unsigned long dl_util, util, irq; @@ -5223,6 +5223,12 @@ unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, return min(max, util); } + +unsigned long sched_cpu_util(int cpu, unsigned long max) +{ + return effective_cpu_util(cpu, cpu_util_cfs(cpu_rq(cpu)), max, + ENERGY_UTIL, NULL); +} #endif /* CONFIG_SMP */ /** diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 2d44befb322b..e71627a3792b 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -178,7 +178,7 @@ static unsigned long sugov_get_util(struct sugov_cpu *sg_cpu) sg_cpu->max = max; sg_cpu->bw_dl = cpu_bw_dl(rq); - return schedutil_cpu_util(sg_cpu->cpu, util, max, FREQUENCY_UTIL, NULL); + return effective_cpu_util(sg_cpu->cpu, util, max, FREQUENCY_UTIL, NULL); } /** diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 290f9e38378c..0e1c8eb7ad53 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -6499,7 +6499,7 @@ compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) * is already enough to scale the EM reported power * consumption at the (eventually clamped) cpu_capacity. */ - sum_util += schedutil_cpu_util(cpu, util_cfs, cpu_cap, + sum_util += effective_cpu_util(cpu, util_cfs, cpu_cap, ENERGY_UTIL, NULL); /* @@ -6509,7 +6509,7 @@ compute_energy(struct task_struct *p, int dst_cpu, struct perf_domain *pd) * NOTE: in case RT tasks are running, by default the * FREQUENCY_UTIL's utilization can be max OPP. */ - cpu_util = schedutil_cpu_util(cpu, util_cfs, cpu_cap, + cpu_util = effective_cpu_util(cpu, util_cfs, cpu_cap, FREQUENCY_UTIL, tsk); max_util = max(max_util, cpu_util); } @@ -6607,7 +6607,7 @@ static int find_energy_efficient_cpu(struct task_struct *p, int prev_cpu) * IOW, placing the task there would make the CPU * overutilized. Take uclamp into account to see how * much capacity we can get out of the CPU; this is - * aligned with schedutil_cpu_util(). + * aligned with sched_cpu_util(). */ util = uclamp_rq_util_with(cpu_rq(cpu), util, p); if (!fits_capacity(util, cpu_cap)) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 0db6bcf0881f..495e7ea278ce 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2486,22 +2486,22 @@ static inline unsigned long capacity_orig_of(int cpu) } /** - * enum schedutil_type - CPU utilization type + * enum cpu_util_type - CPU utilization type * @FREQUENCY_UTIL: Utilization used to select frequency * @ENERGY_UTIL: Utilization used during energy calculation * * The utilization signals of all scheduling classes (CFS/RT/DL) and IRQ time * need to be aggregated differently depending on the usage made of them. This - * enum is used within schedutil_freq_util() to differentiate the types of + * enum is used within effective_cpu_util() to differentiate the types of * utilization expected by the callers, and adjust the aggregation accordingly. */ -enum schedutil_type { +enum cpu_util_type { FREQUENCY_UTIL, ENERGY_UTIL, }; -unsigned long schedutil_cpu_util(int cpu, unsigned long util_cfs, - unsigned long max, enum schedutil_type type, +unsigned long effective_cpu_util(int cpu, unsigned long util_cfs, + unsigned long max, enum cpu_util_type type, struct task_struct *p); static inline unsigned long cpu_bw_dl(struct rq *rq) From patchwork Tue Dec 8 04:16:57 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 339691 Delivered-To: patch@linaro.org Received: by 2002:a02:85a7:0:0:0:0:0 with SMTP id d36csp3368039jai; Mon, 7 Dec 2020 20:18:52 -0800 (PST) X-Google-Smtp-Source: ABdhPJyjcFVeqL4ullw/pb0ilX2lSKZzlWGa7841rk3OplJtYznDP6Peu/1a0rSId7Y5h4+UxP/t X-Received: by 2002:a17:906:b56:: with SMTP id v22mr17825931ejg.145.1607401131950; Mon, 07 Dec 2020 20:18:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1607401131; cv=none; d=google.com; s=arc-20160816; b=V12B/EherLduHfPrAo3IgfG8ivCLhuw+sJ4XKNEEW2vv4wEJniFhzLkQ9LdDrdYtrD TvE/uAf+FAsO/u861tC7yvSY1aXGpCtHLSUuCaUfQMXnPI6UsX9sEiwg+Njz0XkpCU8/ qFMjJ91QdVgLwjBXG2eo9rSKQzy6b0vYQC/AS3tdBUbVigSTiHmGX3eQQHOhnkfMKMMQ Pxo9y9nYpNFNh8WnexwrunlH0275+JPlaFTxiM+LYFcbdicUwZlktovKTyQ573LyzQYa 6nt4VNYEHS8qGtiSvb7LuZ3HHyebl1nJxoDW3OR98QAfcWVpZeFyzdBv5G5fOn9c3yOk DsuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=HS1iJkm0hv8Ke2csA68ZLm2xcERjTLMbURsBR9ZOsB4=; b=mPyzgrOrTIDVvuGXPX3IeZRsGPA9LJNFSVtatlrrWPzfnPvdd6PdWA2LAF/cBaN5eJ gEP+bvemHL/v4IUhS7AolkyvHDj/yUqELM8hh2DzdmPiWXXZZjzqCp3qRvD6zIxc8pFx 7BKUxLFcuiR6WZtGOYFPm6JwY2Z+ADCyTymfe1S5N1gPoWeRfpU4taEhxRYuj8itEG2i lZfnvQ03KZdNEB1F3U5kJcx6eYThOUnIrquhL69X4sovSEWxZlak9SdtwgQTx1i0jmXV yjygJBw+zMhK6u7pljpTl2A1+T+5Q3bn6QU0d7i5a8FoA2kOdC7L4L/YOkpyi2sRTLll 8c3w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qX8hLtO9; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id pw18si4674329ejb.163.2020.12.07.20.18.51; Mon, 07 Dec 2020 20:18:51 -0800 (PST) Received-SPF: pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qX8hLtO9; spf=pass (google.com: domain of linux-pm-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-pm-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726825AbgLHESY (ORCPT + 8 others); Mon, 7 Dec 2020 23:18:24 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726779AbgLHESY (ORCPT ); Mon, 7 Dec 2020 23:18:24 -0500 Received: from mail-pf1-x441.google.com (mail-pf1-x441.google.com [IPv6:2607:f8b0:4864:20::441]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 02DC2C0617A7 for ; Mon, 7 Dec 2020 20:17:14 -0800 (PST) Received: by mail-pf1-x441.google.com with SMTP id 11so6405742pfu.4 for ; Mon, 07 Dec 2020 20:17:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HS1iJkm0hv8Ke2csA68ZLm2xcERjTLMbURsBR9ZOsB4=; b=qX8hLtO9+Pw8qEOWiWZoSgX/K+1uf38NjGAhipQoXgrGUrkt4De+MR5KgTa2Zz/T18 Km9nrQPYzrsy+RKyjVeuCKznbAqpjaTg520f8diuzJZO17WY/g4lbHdbJgcLWMPebRoo IER9UcGkZXBz/gvfORKEuB6lc0j3vYooS0rCVi8tpQDsd6QKpt+O2X7WDoZrOtxiDjzh VMu8ECkUrg26j8n3UF3ROb1fJij8Y4qt5P1vj+OSMTR0BAOzq0/zFCPumP8vgaUELfNU v/HJvXorvVx0mm5yeLaQXU3MvAZzKLJ3bJtBpJ4LtiUhswCc+wAmE8m1Ygg9ZDDXUpSz GBFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HS1iJkm0hv8Ke2csA68ZLm2xcERjTLMbURsBR9ZOsB4=; b=oRBEgyoNIm9Fy4GqQI8V+tI6pA2VkkMCicQx9+1vuktVsTo21/h6nIeTHwFeS8l1SZ uf+VdkJ1sedkhIqTsAXftH9x0RkVS2MiRosuI3BX+PFB5iDMGVBo0RvfvM64BiHDGp05 olQQOJRk/fPxpNN7ksDCcUfvxM1ydn8F8cVxz9dNt9AAASDp7uVB5FUr69N+X4z3z2IN Ns4WesvxLucnGS3/ZEAuxnTsJeGZqkGWtoHVOaNAUewkshJbG8SVtxSr8uIa9H+PLFmE xUphdu4K2ZhKmyap4Zq4NKRb30KpubCoQBcvc2nZcTsQPafAf4BS5ttz6ZYVUcGJL4ui B/aw== X-Gm-Message-State: AOAM533rxFRhv9L8tyzQE9J042ryQ6m8ecQWXxBzs98LfXfkjaEdhw2F tWY+saD9YriBssbbTEYHBAdH+A== X-Received: by 2002:a62:be13:0:b029:19a:4c0:efe3 with SMTP id l19-20020a62be130000b029019a04c0efe3mr19199070pff.75.1607401033563; Mon, 07 Dec 2020 20:17:13 -0800 (PST) Received: from localhost ([122.172.136.109]) by smtp.gmail.com with ESMTPSA id s30sm14195891pgl.39.2020.12.07.20.17.12 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 Dec 2020 20:17:12 -0800 (PST) From: Viresh Kumar To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Amit Daniel Kachhap , Daniel Lezcano , Viresh Kumar , Javi Merino , Zhang Rui , Amit Kucheria Cc: linux-kernel@vger.kernel.org, Quentin Perret , Lukasz Luba , linux-pm@vger.kernel.org Subject: [PATCH V5 3/3] thermal: cpufreq_cooling: Reuse sched_cpu_util() for SMP platforms Date: Tue, 8 Dec 2020 09:46:57 +0530 Message-Id: <9c255c83d78d58451abc06848001faef94c87a12.1607400596.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.25.0.rc1.19.g042ed3e048af In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Several parts of the kernel are already using the effective CPU utilization (as seen by the scheduler) to get the current load on the CPU, do the same here instead of depending on the idle time of the CPU, which isn't that accurate comparatively. This is also the right thing to do as it makes the cpufreq governor (schedutil) align better with the cpufreq_cooling driver, as the power requested by cpufreq_cooling governor will exactly match the next frequency requested by the schedutil governor since they are both using the same metric to calculate load. This was tested on ARM Hikey6220 platform with hackbench, sysbench and schbench. None of them showed any regression or significant improvements. Schbench is the most important ones out of these as it creates the scenario where the utilization numbers provide a better estimate of the future. Scenario 1: The CPUs were mostly idle in the previous polling window of the IPA governor as the tasks were sleeping and here are the details from traces (load is in %): Old: thermal_power_cpu_get_power: cpus=00000000,000000ff freq=1200000 total_load=203 load={{0x35,0x1,0x0,0x31,0x0,0x0,0x64,0x0}} dynamic_power=1339 New: thermal_power_cpu_get_power: cpus=00000000,000000ff freq=1200000 total_load=600 load={{0x60,0x46,0x45,0x45,0x48,0x3b,0x61,0x44}} dynamic_power=3960 Here, the "Old" line gives the load and requested_power (dynamic_power here) numbers calculated using the idle time based implementation, while "New" is based on the CPU utilization from scheduler. As can be clearly seen, the load and requested_power numbers are simply incorrect in the idle time based approach and the numbers collected from CPU's utilization are much closer to the reality. Scenario 2: The CPUs were busy in the previous polling window of the IPA governor: Old: thermal_power_cpu_get_power: cpus=00000000,000000ff freq=1200000 total_load=800 load={{0x64,0x64,0x64,0x64,0x64,0x64,0x64,0x64}} dynamic_power=5280 New: thermal_power_cpu_get_power: cpus=00000000,000000ff freq=1200000 total_load=708 load={{0x4d,0x5c,0x5c,0x5b,0x5c,0x5c,0x51,0x5b}} dynamic_power=4672 As can be seen, the idle time based load is 100% for all the CPUs as it took only the last window into account, but in reality the CPUs aren't that loaded as shown by the utilization numbers. Reviewed-by: Lukasz Luba Signed-off-by: Viresh Kumar --- drivers/thermal/cpufreq_cooling.c | 69 ++++++++++++++++++++++++------- 1 file changed, 55 insertions(+), 14 deletions(-) -- 2.25.0.rc1.19.g042ed3e048af diff --git a/drivers/thermal/cpufreq_cooling.c b/drivers/thermal/cpufreq_cooling.c index cc2959f22f01..cee59b95633b 100644 --- a/drivers/thermal/cpufreq_cooling.c +++ b/drivers/thermal/cpufreq_cooling.c @@ -76,7 +76,9 @@ struct cpufreq_cooling_device { struct em_perf_domain *em; struct cpufreq_policy *policy; struct list_head node; +#ifndef CONFIG_SMP struct time_in_idle *idle_time; +#endif struct freq_qos_request qos_req; }; @@ -132,14 +134,25 @@ static u32 cpu_power_to_freq(struct cpufreq_cooling_device *cpufreq_cdev, } /** - * get_load() - get load for a cpu since last updated - * @cpufreq_cdev: &struct cpufreq_cooling_device for this cpu - * @cpu: cpu number - * @cpu_idx: index of the cpu in time_in_idle* + * get_load() - get load for a cpu + * @cpufreq_cdev: struct cpufreq_cooling_device for the cpu + * @cpu: cpu number + * @cpu_idx: index of the cpu in time_in_idle array * * Return: The average load of cpu @cpu in percentage since this * function was last called. */ +#ifdef CONFIG_SMP +static u32 get_load(struct cpufreq_cooling_device *cpufreq_cdev, int cpu, + int cpu_idx) +{ + unsigned long max = arch_scale_cpu_capacity(cpu); + unsigned long util; + + util = sched_cpu_util(cpu, max); + return (util * 100) / max; +} +#else /* !CONFIG_SMP */ static u32 get_load(struct cpufreq_cooling_device *cpufreq_cdev, int cpu, int cpu_idx) { @@ -161,6 +174,7 @@ static u32 get_load(struct cpufreq_cooling_device *cpufreq_cdev, int cpu, return load; } +#endif /* CONFIG_SMP */ /** * get_dynamic_power() - calculate the dynamic power @@ -346,6 +360,36 @@ static inline bool em_is_sane(struct cpufreq_cooling_device *cpufreq_cdev, } #endif /* CONFIG_THERMAL_GOV_POWER_ALLOCATOR */ +#ifdef CONFIG_SMP +static inline int allocate_idle_time(struct cpufreq_cooling_device *cpufreq_cdev) +{ + return 0; +} + +static inline void free_idle_time(struct cpufreq_cooling_device *cpufreq_cdev) +{ +} +#else +static int allocate_idle_time(struct cpufreq_cooling_device *cpufreq_cdev) +{ + unsigned int num_cpus = cpumask_weight(cpufreq_cdev->policy->related_cpus); + + cpufreq_cdev->idle_time = kcalloc(num_cpus, + sizeof(*cpufreq_cdev->idle_time), + GFP_KERNEL); + if (!cpufreq_cdev->idle_time) + return -ENOMEM; + + return 0; +} + +static void free_idle_time(struct cpufreq_cooling_device *cpufreq_cdev) +{ + kfree(cpufreq_cdev->idle_time); + cpufreq_cdev->idle_time = NULL; +} +#endif /* CONFIG_SMP */ + static unsigned int get_state_freq(struct cpufreq_cooling_device *cpufreq_cdev, unsigned long state) { @@ -487,7 +531,7 @@ __cpufreq_cooling_register(struct device_node *np, struct thermal_cooling_device *cdev; struct cpufreq_cooling_device *cpufreq_cdev; char dev_name[THERMAL_NAME_LENGTH]; - unsigned int i, num_cpus; + unsigned int i; struct device *dev; int ret; struct thermal_cooling_device_ops *cooling_ops; @@ -498,7 +542,6 @@ __cpufreq_cooling_register(struct device_node *np, return ERR_PTR(-ENODEV); } - if (IS_ERR_OR_NULL(policy)) { pr_err("%s: cpufreq policy isn't valid: %p\n", __func__, policy); return ERR_PTR(-EINVAL); @@ -516,12 +559,10 @@ __cpufreq_cooling_register(struct device_node *np, return ERR_PTR(-ENOMEM); cpufreq_cdev->policy = policy; - num_cpus = cpumask_weight(policy->related_cpus); - cpufreq_cdev->idle_time = kcalloc(num_cpus, - sizeof(*cpufreq_cdev->idle_time), - GFP_KERNEL); - if (!cpufreq_cdev->idle_time) { - cdev = ERR_PTR(-ENOMEM); + + ret = allocate_idle_time(cpufreq_cdev); + if (ret) { + cdev = ERR_PTR(ret); goto free_cdev; } @@ -581,7 +622,7 @@ __cpufreq_cooling_register(struct device_node *np, remove_ida: ida_simple_remove(&cpufreq_ida, cpufreq_cdev->id); free_idle_time: - kfree(cpufreq_cdev->idle_time); + free_idle_time(cpufreq_cdev); free_cdev: kfree(cpufreq_cdev); return cdev; @@ -674,7 +715,7 @@ void cpufreq_cooling_unregister(struct thermal_cooling_device *cdev) thermal_cooling_device_unregister(cdev); freq_qos_remove_request(&cpufreq_cdev->qos_req); ida_simple_remove(&cpufreq_ida, cpufreq_cdev->id); - kfree(cpufreq_cdev->idle_time); + free_idle_time(cpufreq_cdev); kfree(cpufreq_cdev); } EXPORT_SYMBOL_GPL(cpufreq_cooling_unregister);