From patchwork Tue Aug 9 23:47:38 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: John Stultz X-Patchwork-Id: 73591 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp134430qga; Tue, 9 Aug 2016 16:47:44 -0700 (PDT) X-Received: by 10.66.161.225 with SMTP id xv1mr1793623pab.20.1470786464412; Tue, 09 Aug 2016 16:47:44 -0700 (PDT) Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e185si45119184pfe.238.2016.08.09.16.47.44; Tue, 09 Aug 2016 16:47:44 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932592AbcHIXrl (ORCPT + 27 others); Tue, 9 Aug 2016 19:47:41 -0400 Received: from mail-oi0-f41.google.com ([209.85.218.41]:36108 "EHLO mail-oi0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750909AbcHIXrk (ORCPT ); Tue, 9 Aug 2016 19:47:40 -0400 Received: by mail-oi0-f41.google.com with SMTP id f189so38810363oig.3 for ; Tue, 09 Aug 2016 16:47:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=yoOBfTvh4R+JCf/Ue3H7cJvXYJE5HbOW8KUOAqO2kqI=; b=CJtNtlhDPObhIHNZwdGGdyJeiMGNrqZU821gUiXIQJ3jBMwMTuGPxngqq9pK2vV/vS n0j3bOhA4NmBWxHggqRrIP4eZkavGOmL8YI57soFSS9vOm8SH+LfgLvUAvmuxoRSdItw pO/7WGYQ9wMfzo7Df8ni/VJrCQALj4JxGbUq4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=yoOBfTvh4R+JCf/Ue3H7cJvXYJE5HbOW8KUOAqO2kqI=; b=Klz1sUXjN4hLOdfC/ZadN8Y0sohqDit/CLcdh3jmzkNklI7bHHQ+f2niDtksQq0gl5 mPCM1o3jXWALTWPUyWxT1CDNqMW9Bt6Tz7/horLHnZyAjrLbNyiu3dvUIfSX37B/0RY2 /AG5iVp4Mdb8wKzCgHLFrS4NJKSLF38d13d6rAsTTaUeg0QkFFku3sgnCKzXTp93rN6S n5zLaDIeKJ04V0AdmgagMqZ/qHm1NHHtzmckuvKKmk4tCVm2uIoZbGDltEs7rmYLEJhy MJUNEWRa5wNNEKi1kaLkspcwNlkbcGzkbwEgxcWhcCFmO2/JUN60zv+TcAhWK6cG1hfV RXSw== X-Gm-Message-State: AEkoouski3HzNrK/2ZfLBGZwfAA40wu7RbV7Gg60SglaPHHVEd9/52I6EGP/Kun7Dyw8BnP4ODaYn4LBKIwRhyGk X-Received: by 10.202.175.87 with SMTP id y84mr480323oie.170.1470786459043; Tue, 09 Aug 2016 16:47:39 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.93.11 with HTTP; Tue, 9 Aug 2016 16:47:38 -0700 (PDT) In-Reply-To: <20160809095112.GG30192@twins.programming.kicks-ass.net> References: <20160809095112.GG30192@twins.programming.kicks-ass.net> From: John Stultz Date: Tue, 9 Aug 2016 16:47:38 -0700 Message-ID: Subject: Re: [PATCH v2] locking/percpu-rwsem: Optimize readers and reduce global impact To: Peter Zijlstra Cc: Oleg Nesterov , "Paul E. McKenney" , Ingo Molnar , lkml , Tejun Heo , Dmitry Shmidt , Rom Lemarchand , Colin Cross , Todd Kjos Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Aug 9, 2016 at 2:51 AM, Peter Zijlstra wrote: > > Currently the percpu-rwsem switches to (global) atomic ops while a > writer is waiting; which could be quite a while and slows down > releasing the readers. > > This patch cures this problem by ordering the reader-state vs > reader-count (see the comments in __percpu_down_read() and > percpu_down_write()). This changes a global atomic op into a full > memory barrier, which doesn't have the global cacheline contention. > > This also enables using the percpu-rwsem with rcu_sync disabled in order > to bias the implementation differently, reducing the writer latency by > adding some cost to readers. So this by itself doesn't help us much, but including the following from Oleg does help quite a bit: thanks -john diff --git a/kernel/cgroup.c b/kernel/cgroup.c index db27804..9e9200b 100644 --- a/kernel/cgroup.c +++ b/kernel/cgroup.c @@ -5394,6 +5394,8 @@ int __init cgroup_init(void) BUG_ON(cgroup_init_cftypes(NULL, cgroup_dfl_base_files)); BUG_ON(cgroup_init_cftypes(NULL, cgroup_legacy_base_files)); + rcu_sync_enter(&cgroup_threadgroup_rwsem.rss); + mutex_lock(&cgroup_mutex); /* Add init_css_set to the hash table */