From patchwork Mon Jul 1 15:47:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 168215 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp3080063ilk; Mon, 1 Jul 2019 08:47:10 -0700 (PDT) X-Google-Smtp-Source: APXvYqzVXsfBDgVgY7dSTHrJBXEEAzeXU7ezycsgeJlGP3DkCBHkP5tm+2IggM+dtqbyTc/3XX+Y X-Received: by 2002:a17:90a:ac11:: with SMTP id o17mr32307065pjq.134.1561996030457; Mon, 01 Jul 2019 08:47:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561996030; cv=none; d=google.com; s=arc-20160816; b=EJebYiI+JFQnZFQC1dX+N5hPij+eYzUZ1L0v/Q8hi/H/IXyrjgeBeVqmctzaqJlskB ckQh4VF5vKdFDVfg/K2Nf7opK+DCMMUczWAiQjsf08U8Ln9/JLY6HiEyjC2EoeskBAPV aSeO1CORtqgnkU6HiMZwTuXEiQIDLQ6NvpMCRMMKF/Q9IdoQ6cLfWNv/iAU3NkLiL6s/ QUCP1HJTB7C3Mzm8V24a4L/RIvlAAv7tKDOq1qGQYbbOUxGwZxTHaQmmfqndc1UNfsl9 c4wA9wKYsudqKn11KcGeX0axwmEOX5hYDunZvsqPEKsRA61764bc6y9M97Ial4JPVtM1 g+lA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature; bh=OYbnfUR9dCYFLINXoIyX0GbGtAYWpqvIvnH71iWYfBU=; b=YAk9SzcdlLn1ciJhgrSIV65WfoucAd0XE4stSD7GFmUdx8/C9odAfbCX4yu4Byq2IR Z6jIQiIeC/NjGWgpC03FISjIUo2WQAnpYY7wYMT6zPyId/qzJbEMQkW27mfdzX9tGot0 G/dtjP3JIvreY49+RHI4kX7YNylSO5Rx1cxjZfrUhNRqdjHrwQyPtEFyK2NOAv2dim4d 7I1O2aYAvd6bGgYpiiJ35IFbzy/Y484mh7Wvl31+59g2z4bZBSUmLzpz/BXtcj6KHMnj cyExM3MGgz8ahD0XoK9BqcFZwdiy0rw323whU4MRV7liobVe0uypCpUBnwndxb5KkEJy pQyQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="bbSa/XL2"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id k14si10928820pjq.53.2019.07.01.08.47.10; Mon, 01 Jul 2019 08:47:10 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="bbSa/XL2"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729924AbfGAPrI (ORCPT + 30 others); Mon, 1 Jul 2019 11:47:08 -0400 Received: from mail-wr1-f65.google.com ([209.85.221.65]:34760 "EHLO mail-wr1-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727373AbfGAPrI (ORCPT ); Mon, 1 Jul 2019 11:47:08 -0400 Received: by mail-wr1-f65.google.com with SMTP id u18so6246287wru.1 for ; Mon, 01 Jul 2019 08:47:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=OYbnfUR9dCYFLINXoIyX0GbGtAYWpqvIvnH71iWYfBU=; b=bbSa/XL2D0XSiOrgG+7e7u10xLBjPFGH5IozlqzrBvyBI056vpYcrC92X37b1DgR9k Ve2dh3k0Nw9Li5iqNW86i/48IN5qelgvuHjjy+/3h5zZL27kV6yfVPPSb7ZcLxWK/EiW x7jhjXpF6/1PMa/tX7EFJ2BH3ymhO9bgr7t8/1z+Svw5mOeNx6sI39Xe4a5ps4EZPh8y YYQUHLpJwx2iV06SC0gCPjMKhqsdZ/6JQPKuStigw840BMBBHHt//m+G2bXkvkvpZhWa Ya1chKqoOBgirMJ5hKSr7R+gg5+9DkhC6f+qvMBqB37ssg62kO0ARe66hp00uTS+Suds N4sw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=OYbnfUR9dCYFLINXoIyX0GbGtAYWpqvIvnH71iWYfBU=; b=oHwKTziaAcCzdoX6NAtt0xVNG0XpXJH7ga9k2ZOnsCUtqsPNd7AP2aBPrlnfhF2MuP 6FIPO3fyXC+vkI3X4DrWj8CaE7EN9nwn4IHkPrX5iTk2/KQb1qRe3L0vNFaBWQsNkwbG ynLdDaxfiLp2O3hdqVTub/VJD1BAynIcXGhORTsKmc4+BmMu4vThLEAORs/rGTCJxkeo c2xmQZWvqyeqxdYYjKPcOqwmJdWMbdPXEs6IpHkBwlYha7W28lMpS/+osK5vYTdcPHIe q24Oi0x+fHbaxXDDqePdSnpboxbA+4y8EmnUIGFkGWDw7fYIEieaCn7I1ur30Wi1NZJZ XWwA== X-Gm-Message-State: APjAAAU4E+ydIh8+hZ0RQ5C6K4zicqlU1B0ZoW5TPa4rhrrn2d4U6Woe upA0N4gkmZJCChT+Ckcw2GytbuzV1PI= X-Received: by 2002:a5d:5302:: with SMTP id e2mr19063405wrv.347.1561996026093; Mon, 01 Jul 2019 08:47:06 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:5dbd:30b:8a71:a020]) by smtp.gmail.com with ESMTPSA id y16sm8300300wru.28.2019.07.01.08.47.05 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 01 Jul 2019 08:47:05 -0700 (PDT) From: Vincent Guittot To: linux-kernel@vger.kernel.org, mingo@redhat.com, peterz@infradead.org Cc: Vincent Guittot Subject: [PATCH v2] sched/fair: fix imbalance due to CPU affinity Date: Mon, 1 Jul 2019 17:47:02 +0200 Message-Id: <1561996022-28829-1-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The load_balance() has a dedicated mecanism to detect when an imbalance is due to CPU affinity and must be handled at parent level. In this case, the imbalance field of the parent's sched_group is set. The description of sg_imbalanced() gives a typical example of two groups of 4 CPUs each and 4 tasks each with a cpumask covering 1 CPU of the first group and 3 CPUs of the second group. Something like: { 0 1 2 3 } { 4 5 6 7 } * * * * But the load_balance fails to fix this UC on my octo cores system made of 2 clusters of quad cores. Whereas the load_balance is able to detect that the imbalanced is due to CPU affinity, it fails to fix it because the imbalance field is cleared before letting parent level a chance to run. In fact, when the imbalance is detected, the load_balance reruns without the CPU with pinned tasks. But there is no other running tasks in the situation described above and everything looks balanced this time so the imbalance field is immediately cleared. The imbalance field should not be cleared if there is no other task to move when the imbalance is detected. Signed-off-by: Vincent Guittot --- Sorry, I sent the patch before rebasing it on top of sched-tip and it might conlfict when applying because it was on top of my ongoing rework of load_balance This version is rebased on top of latest shced-tip/sched/core kernel/sched/fair.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) -- 2.7.4 diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index b798fe7..fff5632 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -8992,9 +8992,10 @@ static int load_balance(int this_cpu, struct rq *this_rq, out_balanced: /* * We reach balance although we may have faced some affinity - * constraints. Clear the imbalance flag if it was set. + * constraints. Clear the imbalance flag only if other tasks got + * a chance to move and fix the imbalance. */ - if (sd_parent) { + if (sd_parent && !(env.flags & LBF_ALL_PINNED)) { int *group_imbalance = &sd_parent->groups->sgc->imbalance; if (*group_imbalance)