From patchwork Fri May 25 13:12:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Vincent Guittot X-Patchwork-Id: 136859 Delivered-To: patch@linaro.org Received: by 2002:a2e:9706:0:0:0:0:0 with SMTP id r6-v6csp3573240lji; Fri, 25 May 2018 06:12:44 -0700 (PDT) X-Google-Smtp-Source: AB8JxZp9ALnW79AKTNfuNFRxGqtnYGHI625a/HLAT3wSascGy8w95NNUtC82JR6Q9iaivPdWot8q X-Received: by 2002:a17:902:758a:: with SMTP id j10-v6mr2617860pll.109.1527253964282; Fri, 25 May 2018 06:12:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527253964; cv=none; d=google.com; s=arc-20160816; b=h9OKd1JHgFZJT067h6HmlYUC6h1P/RSj6MpR4y/4u7f0CsWJQW8y5UhC/VBIQYpduw 6LViHegsBGnZ+SaofsJ1caLLpU724xfKeFyIdXFCLjaPkE/35oUkBdioECbEdflTpaEf +wwhSDEe3ZwJyQiQbFYBC9VuxPfqijVysazzktEo7D9oF2wC8Qcx7M6TKYcq2GUXLLv6 qZw75bRFkr9E3dPpbjaT7z9fhDRwl4T4UDgqgF32YR4U6YEA9c+JzlxWzu0s9yHCbSwq QiQHwQrftRM4be8GiOT+qooH1Zs28jL0rGCHRiWVde2Nme//F05cTJAAKrX4lPpot2ye y9ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=ytrOmvgHwomxa/tqsSAGDD/EGv0uRVRxeCJC1xQG+dc=; b=hHpZQNgh31N8I57ew+Mu7AzByAb/apgfOFV8Y6TWNamHd3SX6+TBk+YuR2kEWgZXIH YqYOc//ZIgA6BlbryIbWd5hlsZz48vLgGsd0jF///2tmsb7a/arYFzW0befXDj6+I5Hn /TxwqXAW0f63RT5lm6ICApGacdFjBjU+3+mtfxpvFLqHQaAkH9gdtIhMwO8Ut+iahZfO 4ufwKFSk5mFZCU7SqT47/ZZGGAM5WMe+eF4bw+ybS6xEFDRn7CXQfAtyUqVZyq5Y1MAV aDNz8E8kSGP95SHsR/pvLP5XTLtjzk5YGsBBAciw2qStsnXFaRAWyRQQipTi6hz4ttLt 9E+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fr4eS8Y/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a1-v6si18669649pgn.425.2018.05.25.06.12.43; Fri, 25 May 2018 06:12:44 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fr4eS8Y/; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S934520AbeEYNMl (ORCPT + 30 others); Fri, 25 May 2018 09:12:41 -0400 Received: from mail-wm0-f66.google.com ([74.125.82.66]:51731 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752581AbeEYNMj (ORCPT ); Fri, 25 May 2018 09:12:39 -0400 Received: by mail-wm0-f66.google.com with SMTP id j4-v6so14325063wme.1 for ; Fri, 25 May 2018 06:12:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=ytrOmvgHwomxa/tqsSAGDD/EGv0uRVRxeCJC1xQG+dc=; b=fr4eS8Y/FwnUMLlic6rF3VqqMZ5yPWGOJHM6ip4v27yuO40jaaOeWUwH2gxa5Hc5GT 2edEx+oj1MX8JSAFbtm9UCaKIrOAeCKmh3LxlPJBDw/EAvFMLTcIGVkTtRA9Txzc7JFE 2ki5r996GDQEScpXKHEuTaFDcjQrPUTqpeno8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=ytrOmvgHwomxa/tqsSAGDD/EGv0uRVRxeCJC1xQG+dc=; b=lajfDTzVANgvccKdS75mpkGbcC2q/cqBsuxnTw4YrmlpgBbYeWnOAQu0wEe9ktBKrd zlqBwI8HXemOhrhmq5MBg8D3Zyt9hTnhGnKA7RUnFd7zQi4j/L54esZrekVmWWUXIFB7 kpbV2xktc93nQCaZaw3CkOdTGb8th+Nit8jMH6XAFMaW+e4sA03IkYyMuaACLrCuJ5n6 NLq02EUGQfK92S1KzdWzLG7BPF52kpHENV/g+F6VH1zQb+s0xs6R3wHJZHmW5LBKVNXG Y227RAtF0KxlVhNCv0JMV6CW9N84Ycd11PNBEww9ENccCae5ajIqGTiWPwo/MkoLttw+ B0XA== X-Gm-Message-State: ALKqPwdmVUOvaZTDgS4yJH4BgQ1V+QGgsONhy6VYpcdKGNSlVK+fujaD qgGHMoTHvpu9YVChKSrPZIqfU6rLZDM= X-Received: by 2002:a1c:5ec9:: with SMTP id s192-v6mr1614230wmb.54.1527253958282; Fri, 25 May 2018 06:12:38 -0700 (PDT) Received: from localhost.localdomain ([2a01:e0a:f:6020:a860:64b4:335b:c763]) by smtp.gmail.com with ESMTPSA id 4-v6sm9690948wmg.8.2018.05.25.06.12.37 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 25 May 2018 06:12:37 -0700 (PDT) From: Vincent Guittot To: peterz@infradead.org, mingo@kernel.org, linux-kernel@vger.kernel.org, rjw@rjwysocki.net Cc: juri.lelli@redhat.com, dietmar.eggemann@arm.com, Morten.Rasmussen@arm.com, viresh.kumar@linaro.org, valentin.schneider@arm.com, quentin.perret@arm.com, Vincent Guittot Subject: [PATCH v5 00/10] track CPU utilization Date: Fri, 25 May 2018 15:12:21 +0200 Message-Id: <1527253951-22709-1-git-send-email-vincent.guittot@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patchset initially tracked only the utilization of RT rq. During OSPM summit, it has been discussed the opportunity to extend it in order to get an estimate of the utilization of the CPU. - Patches 1-3 correspond to the content of patchset v4 and add utilization tracking for rt_rq. When both cfs and rt tasks compete to run on a CPU, we can see some frequency drops with schedutil governor. In such case, the cfs_rq's utilization doesn't reflect anymore the utilization of cfs tasks but only the remaining part that is not used by rt tasks. We should monitor the stolen utilization and take it into account when selecting OPP. This patchset doesn't change the OPP selection policy for RT tasks but only for CFS tasks A rt-app use case which creates an always running cfs thread and a rt threads that wakes up periodically with both threads pinned on same CPU, show lot of frequency switches of the CPU whereas the CPU never goes idles during the test. I can share the json file that I used for the test if someone is interested in. For a 15 seconds long test on a hikey 6220 (octo core cortex A53 platfrom), the cpufreq statistics outputs (stats are reset just before the test) : $ cat /sys/devices/system/cpu/cpufreq/policy0/stats/total_trans without patchset : 1230 with patchset : 14 If we replace the cfs thread of rt-app by a sysbench cpu test, we can see performance improvements: - Without patchset : Test execution summary: total time: 15.0009s total number of events: 4903 total time taken by event execution: 14.9972 per-request statistics: min: 1.23ms avg: 3.06ms max: 13.16ms approx. 95 percentile: 12.73ms Threads fairness: events (avg/stddev): 4903.0000/0.00 execution time (avg/stddev): 14.9972/0.00 - With patchset: Test execution summary: total time: 15.0014s total number of events: 7694 total time taken by event execution: 14.9979 per-request statistics: min: 1.23ms avg: 1.95ms max: 10.49ms approx. 95 percentile: 10.39ms Threads fairness: events (avg/stddev): 7694.0000/0.00 execution time (avg/stddev): 14.9979/0.00 The performance improvement is 56% for this use case. - Patches 4-5 add utilization tracking for dl_rq in order to solve similar problem as with rt_rq - Patches 6 uses dl and rt utilization in the scale_rt_capacity() and remove dl and rt from sched_rt_avg_update - Patches 7-8 add utilization tracking for interrupt and use it select OPP A test with iperf on hikey 6220 gives: w/o patchset w/ patchset Tx 276 Mbits/sec 304 Mbits/sec +10% Rx 299 Mbits/sec 328 Mbits/sec +09% 8 iterations of iperf -c server_address -r -t 5 stdev is lower than 1% Only WFI idle state is enable (shallowest arm idle state) - Patches 9 removes the unused sched_avg_update code - Patch 10 removes the unused sched_time_avg_ms Change since v3: - add support of periodic update of blocked utilization - rebase on lastest tip/sched/core Change since v2: - move pelt code into a dedicated pelt.c file - rebase on load tracking changes Change since v1: - Only a rebase. I have addressed the comments on previous version in patch 1/2 Vincent Guittot (10): sched/pelt: Move pelt related code in a dedicated file sched/rt: add rt_rq utilization tracking cpufreq/schedutil: add rt utilization tracking sched/dl: add dl_rq utilization tracking cpufreq/schedutil: get max utilization sched: remove rt and dl from sched_avg sched/irq: add irq utilization tracking cpufreq/schedutil: take into account interrupt sched: remove rt_avg code proc/sched: remove unused sched_time_avg_ms include/linux/sched/sysctl.h | 1 - kernel/sched/Makefile | 2 +- kernel/sched/core.c | 38 +--- kernel/sched/cpufreq_schedutil.c | 24 ++- kernel/sched/deadline.c | 7 +- kernel/sched/fair.c | 381 +++---------------------------------- kernel/sched/pelt.c | 395 +++++++++++++++++++++++++++++++++++++++ kernel/sched/pelt.h | 63 +++++++ kernel/sched/rt.c | 10 +- kernel/sched/sched.h | 57 ++++-- kernel/sysctl.c | 8 - 11 files changed, 563 insertions(+), 423 deletions(-) create mode 100644 kernel/sched/pelt.c create mode 100644 kernel/sched/pelt.h -- 2.7.4