From patchwork Thu Oct 24 06:45:27 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Viresh Kumar X-Patchwork-Id: 177368 Delivered-To: patch@linaro.org Received: by 2002:a92:409a:0:0:0:0:0 with SMTP id d26csp1773085ill; Wed, 23 Oct 2019 23:48:04 -0700 (PDT) X-Google-Smtp-Source: APXvYqy0y1TmptcwtySog/p9G1GeFHhybmt6m1QJ4LvRMI9SlRMi1WQeSLLt/Fc5FnQTx18SLV2z X-Received: by 2002:a50:ec0e:: with SMTP id g14mr16904326edr.274.1571899683877; Wed, 23 Oct 2019 23:48:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1571899683; cv=none; d=google.com; s=arc-20160816; b=KvxHwMvQiO4PCzb4N2/oAqzOBjzVDZJoZhOBMnnH7BAo+E3Yf0lSFp7IIp6p76xrOk hkWAlUg6lO9dTEqXieGk5V3Xrk0j5NrVpzj8h+qNSf4qH4oMf++VZ/c7tzx7actZIx31 3s+c0rhZdhYjVN8rqIoCau4uqiZk48xrhfHG/olaP8sLjZIKCAQJbUzmYGJNmEbZ9dMZ LRt8tFeOZik0ixImD7Lr6PUySRtkn+ctpi+NamFwYBQbI8+wS3i/Sxm6NCHURKl09Et9 WnoqpPJgmoUjQ/r+xKK0BQz1fRVItA6/4/UM23Rj6sEfhuPir7ChYtVCt6no0u0zMKso TeRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=NgFTobKzYLIoybimGSOXWg5eOtygrhwDEBEYyn3XxEY=; b=fHRhL4vIzPC8JKyDsnIOVPOw0RZHr+fc0o++x2pWKdJMp9rmBXhqCSOlryPENwdN61 poUdtymJECURERK/PnynEZIYNQNIObQQNeBzH3e1QcVXwU2NxAgWLK+3S4NIqLi1BrLY gb3VHd8CQkJkHXnu6wc83tltIIUXeGpeV4M7a1jeUYJw0ohUydLhtRHzIlh9aUWc/GEH aazTJs+gNb2jMEsW5N/ZTZSx3htgMHcG2lEmibirkmjf9I5x6w4MpEIFGR7X+YN8Br1S auyHNQSuf5TgjjhcmS/xBYtH989DRVLFo9TpQM5AgkX7cS+jA20uhIK9NLZg7KaEPSJr aW8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=whQQndDh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id w14si14025678ejv.112.2019.10.23.23.48.03; Wed, 23 Oct 2019 23:48:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=whQQndDh; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2437769AbfJXGqE (ORCPT + 26 others); Thu, 24 Oct 2019 02:46:04 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:37089 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727736AbfJXGqD (ORCPT ); Thu, 24 Oct 2019 02:46:03 -0400 Received: by mail-pg1-f194.google.com with SMTP id p1so13632365pgi.4 for ; Wed, 23 Oct 2019 23:46:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=NgFTobKzYLIoybimGSOXWg5eOtygrhwDEBEYyn3XxEY=; b=whQQndDhmoaZK9OecPknNrAyqb6J4Qy15F9F+r54pyj8/muqFAkT4vfgH6jCOH6K/l brQ6bB7gE1MNTawZQCct3jWf0VaVfXaLzHESz7uVzlgPu/2dmuAC7IvO61y/BG5iVWWT RZflyRHfwYYxJDDUkJqdwZDsZJlSFQ2DyAlkDAC3We5tIANqzQNu3D66tVwu2PZLr8gF zxVWntqQuoZSIcT8PcD0u5vdpHQh7LHNOLj82mWDmFV5oIjI3l4e10lQrgvhSWvocGV+ A1xqlQhfAsc336vPQaN5B7P5L41ZUy4bNPnSkYCH6R/1TR/G1XiPkgDOp6hY/qkTv3yO KmVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=NgFTobKzYLIoybimGSOXWg5eOtygrhwDEBEYyn3XxEY=; b=thWbVwsAJnR5SYPF6QiMwL4dcw4Zmyg050SMhMOvkhMl+r4LAAsIr2xbBGXGMZQ2qW NdAG8tvfZiUVSmKHetmZR/Rewtkh5khU2x2ABQDAyEHhbe8ZJlXs7RMm4hnUwii4007a 6nSa88WDYT+22oX7OSz3GBW1RXYx9fEYIjg0Ha6YB7lgA5J0ZFroce+BgcfbVk2hs3qS rmDDE9QENQiiguzvbD/q4n5aPwIbI74UnmVGODUGWQ5v7iP9IIEgWcIy4Q5g3oXDXQmm JQlH05a60kGpqgqUQpR9wu7PseqG4MYH+t6HVpdsqiedui5kuSDmigWkn2MaqMKFhiIA Tfjw== X-Gm-Message-State: APjAAAU3RggDDMMz95F5K/iZwP1j31aeZMLSjaqFefadJgsMuwizeLAM MX7oitCOMhK0fl6qcubZUY6ZTw== X-Received: by 2002:a63:4553:: with SMTP id u19mr14508400pgk.436.1571899562379; Wed, 23 Oct 2019 23:46:02 -0700 (PDT) Received: from localhost ([122.172.151.112]) by smtp.gmail.com with ESMTPSA id l72sm2536946pjb.7.2019.10.23.23.46.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 Oct 2019 23:46:01 -0700 (PDT) From: Viresh Kumar To: Ingo Molnar , Peter Zijlstra , Vincent Guittot , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman Cc: Viresh Kumar , linux-kernel@vger.kernel.org Subject: [PATCH] sched/fair: Make sched-idle cpu selection consistent throughout Date: Thu, 24 Oct 2019 12:15:27 +0530 Message-Id: <5eba2fb4af9ebc7396101bb9bd6c8aa9c8af0710.1571899508.git.viresh.kumar@linaro.org> X-Mailer: git-send-email 2.21.0.rc0.269.g1a574e7a288b MIME-Version: 1.0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org There are instances where we keep searching for an idle CPU despite having a sched-idle cpu already (in find_idlest_group_cpu(), select_idle_smt() and select_idle_cpu() and then there are places where we don't necessarily do that and return a sched-idle cpu as soon as we find one (in select_idle_sibling()). This looks a bit inconsistent and it may be worth having the same policy everywhere. On the other hand, choosing a sched-idle cpu over a idle one shall be beneficial from performance point of view as well, as we don't need to get the cpu online from a deep idle state which is quite a time consuming process and delays the scheduling of the newly wakeup task. This patch tries to simplify code around sched-idle cpu selection and make it consistent throughout. FWIW, tests were done with the help of rt-app (8 SCHED_OTHER and 5 SCHED_IDLE tasks, not bound to any cpu) on ARM platform (octa-core), and no significant difference in scheduling latency of SCHED_OTHER tasks was found. Signed-off-by: Viresh Kumar --- kernel/sched/fair.c | 34 ++++++++++++---------------------- 1 file changed, 12 insertions(+), 22 deletions(-) -- 2.21.0.rc0.269.g1a574e7a288b diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index a81c36472822..bb367f48c1ef 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -5545,7 +5545,7 @@ find_idlest_group_cpu(struct sched_group *group, struct task_struct *p, int this unsigned int min_exit_latency = UINT_MAX; u64 latest_idle_timestamp = 0; int least_loaded_cpu = this_cpu; - int shallowest_idle_cpu = -1, si_cpu = -1; + int shallowest_idle_cpu = -1; int i; /* Check if we have any choice: */ @@ -5554,6 +5554,9 @@ find_idlest_group_cpu(struct sched_group *group, struct task_struct *p, int this /* Traverse only the allowed CPUs */ for_each_cpu_and(i, sched_group_span(group), p->cpus_ptr) { + if (sched_idle_cpu(i)) + return i; + if (available_idle_cpu(i)) { struct rq *rq = cpu_rq(i); struct cpuidle_state *idle = idle_get_state(rq); @@ -5576,12 +5579,7 @@ find_idlest_group_cpu(struct sched_group *group, struct task_struct *p, int this latest_idle_timestamp = rq->idle_stamp; shallowest_idle_cpu = i; } - } else if (shallowest_idle_cpu == -1 && si_cpu == -1) { - if (sched_idle_cpu(i)) { - si_cpu = i; - continue; - } - + } else if (shallowest_idle_cpu == -1) { load = cpu_load(cpu_rq(i)); if (load < min_load) { min_load = load; @@ -5590,11 +5588,7 @@ find_idlest_group_cpu(struct sched_group *group, struct task_struct *p, int this } } - if (shallowest_idle_cpu != -1) - return shallowest_idle_cpu; - if (si_cpu != -1) - return si_cpu; - return least_loaded_cpu; + return shallowest_idle_cpu != -1 ? shallowest_idle_cpu : least_loaded_cpu; } static inline int find_idlest_cpu(struct sched_domain *sd, struct task_struct *p, @@ -5747,7 +5741,7 @@ static int select_idle_core(struct task_struct *p, struct sched_domain *sd, int */ static int select_idle_smt(struct task_struct *p, int target) { - int cpu, si_cpu = -1; + int cpu; if (!static_branch_likely(&sched_smt_present)) return -1; @@ -5755,13 +5749,11 @@ static int select_idle_smt(struct task_struct *p, int target) for_each_cpu(cpu, cpu_smt_mask(target)) { if (!cpumask_test_cpu(cpu, p->cpus_ptr)) continue; - if (available_idle_cpu(cpu)) + if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) return cpu; - if (si_cpu == -1 && sched_idle_cpu(cpu)) - si_cpu = cpu; } - return si_cpu; + return -1; } #else /* CONFIG_SCHED_SMT */ @@ -5790,7 +5782,7 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t u64 time, cost; s64 delta; int this = smp_processor_id(); - int cpu, nr = INT_MAX, si_cpu = -1; + int cpu, nr = INT_MAX; this_sd = rcu_dereference(*this_cpu_ptr(&sd_llc)); if (!this_sd) @@ -5818,13 +5810,11 @@ static int select_idle_cpu(struct task_struct *p, struct sched_domain *sd, int t for_each_cpu_wrap(cpu, sched_domain_span(sd), target) { if (!--nr) - return si_cpu; + return -1; if (!cpumask_test_cpu(cpu, p->cpus_ptr)) continue; - if (available_idle_cpu(cpu)) + if (available_idle_cpu(cpu) || sched_idle_cpu(cpu)) break; - if (si_cpu == -1 && sched_idle_cpu(cpu)) - si_cpu = cpu; } time = cpu_clock(this) - time;