From patchwork Tue Aug 28 20:13:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amit Pundir X-Patchwork-Id: 145368 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp1589455ljw; Tue, 28 Aug 2018 13:14:09 -0700 (PDT) X-Google-Smtp-Source: ANB0VdaORpMoyroaO0VzKckqAcQuxVOv1ZUL6uLeuCw8tTz0vAGWVjJe1ERUo+HFfAmZ0C5vjQcj X-Received: by 2002:a63:f111:: with SMTP id f17-v6mr2812413pgi.87.1535487249640; Tue, 28 Aug 2018 13:14:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535487249; cv=none; d=google.com; s=arc-20160816; b=H7QKDoE1dP/Rx9I+NjwROdvMXKDbcMzQnAm//QmhgdO2RVe+J5ru7+K9f2na4su9Kz ht66CVlSh4PcrSagj+2PF+MQIs/o0msA3+4hmFvqALVCf9R7sdMR7/a+aNEbmf9FYmu8 lO53FUZkLShRgWqXcMOOogWTjKRyd3J1bgA8+6f4vxZuTlkwh2oMBso269A7bPGaLNRm /lp4VzbOPK9jhNTfctgvoQ67JFL95nWoLkzx6jHbQV3dZQmDJsmd7q4geijX40SjrwKU bJ4h4W/pq8ixzQwpNejM5iUGnDMuW2XVrdWNtDEKCKVRJ1uBoHCQ5k8JV95nar8b9vjx GEDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=FwrE8QRLzDSVDpybvsbsFCrkig0rVq395QXSR7cbD5s=; b=ZKDe13gRWu2dFAg2RsssBCnd0x+Dg0aXw/v5de9Ty6mwam5cHV/Lfx48XhHDreYUw+ rCvhp0di7NZwYF65pOn3tWu6zLL0w8v+loR5qgxJlt+qtMTsKL12Z+A7sOxdlkavB4wD sQwf/+YY7PP9abRiWhWFzB9yZrdoV+1OTi3TFPMqDq5TMUKrZUDYwtq9wLCth/QCxPEC XX52XnD9k4lqd0zUSz0jeLTgEdP/ktdtKCZjqgskV+keM4t4ZDWpTjKSpEhRfZA78eJA 3oatz7Oi0zMK1EGwrWosJRVsOLpkygwkvLH6bOWQDJHkhHI7aFzNqCpE5USIMv0LHsOe NKXw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="AW/Use2+"; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y10-v6si1906969pgf.312.2018.08.28.13.14.09; Tue, 28 Aug 2018 13:14:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="AW/Use2+"; spf=pass (google.com: best guess record for domain of stable-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=stable-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727193AbeH2AHX (ORCPT + 13 others); Tue, 28 Aug 2018 20:07:23 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:44779 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726961AbeH2AHW (ORCPT ); Tue, 28 Aug 2018 20:07:22 -0400 Received: by mail-pg1-f195.google.com with SMTP id r1-v6so1229741pgp.11 for ; Tue, 28 Aug 2018 13:14:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=FwrE8QRLzDSVDpybvsbsFCrkig0rVq395QXSR7cbD5s=; b=AW/Use2++byq9Zs1yN1cgxL7cGmtdB3V4GWhkO4Y7QuSFVjNvc79/QCBj6/Cv8Buqt 2qQWz7aA/Ysu797K33TFeOlTOTARTzSDvnzvW5dfEN/vQFZEMjGdZILQCOemUICO7u6Q j5CzNlLWdEUZqE58HqJLpG9Etgp9nGBhx0wws= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=FwrE8QRLzDSVDpybvsbsFCrkig0rVq395QXSR7cbD5s=; b=E7r1nso0rM0lNZXs7cc0ENHk9QN2xaTXJDaqVDrtMRCrFJF9D14FyWLK/ugwsUn4xI oetD2BtgkLFczbj5D7SFUQ72BBOm58K+cPQvz8vw5qgWUjR/orTCRS1T51UeX0JcX8lz /dWRNj5mIJ9YOGpS5j1xUmQooilDW64VwXAgJkgUtBVpofISqMnpObi0JhDI4auzxjvD ZoFrmhQhRdEhyNNw0rWKxxROtryHuMYG1ONBZxHrKO9PuRz/8W7liZxIjbbcLJ3rCH1D xnOb/coyS07W0XDwUg28zwB8H4AZoaXtmEY0wCFbaOINimN502h/gYbHuS2sq3ojC7dv ZdVQ== X-Gm-Message-State: APzg51DaOJQJyNg1VHyHsr67xl7Cr0tANWuLgeYlvaPqhfND3yUJuTru dKSecCQZQImQ6QfMZ067HVyPmQ== X-Received: by 2002:a65:650f:: with SMTP id x15-v6mr2939261pgv.127.1535487247818; Tue, 28 Aug 2018 13:14:07 -0700 (PDT) Received: from localhost.localdomain ([49.207.48.21]) by smtp.gmail.com with ESMTPSA id t86-v6sm3098181pfe.109.2018.08.28.13.14.04 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 28 Aug 2018 13:14:06 -0700 (PDT) From: Amit Pundir To: Greg KH Cc: Stable , Prateek Sood , Peter Zijlstra , Linus Torvalds , Thomas Gleixner , sramana@codeaurora.org, Ingo Molnar Subject: [PATCH for-4.9.y 12/14] locking/osq_lock: Fix osq_lock queue corruption Date: Wed, 29 Aug 2018 01:43:23 +0530 Message-Id: <1535487205-26280-13-git-send-email-amit.pundir@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1535487205-26280-1-git-send-email-amit.pundir@linaro.org> References: <1535487205-26280-1-git-send-email-amit.pundir@linaro.org> Sender: stable-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: stable@vger.kernel.org From: Prateek Sood commit 50972fe78f24f1cd0b9d7bbf1f87d2be9e4f412e upstream. Fix ordering of link creation between node->prev and prev->next in osq_lock(). A case in which the status of optimistic spin queue is CPU6->CPU2 in which CPU6 has acquired the lock. tail v ,-. <- ,-. |6| |2| `-' -> `-' At this point if CPU0 comes in to acquire osq_lock, it will update the tail count. CPU2 CPU0 ---------------------------------- tail v ,-. <- ,-. ,-. |6| |2| |0| `-' -> `-' `-' After tail count update if CPU2 starts to unqueue itself from optimistic spin queue, it will find an updated tail count with CPU0 and update CPU2 node->next to NULL in osq_wait_next(). unqueue-A tail v ,-. <- ,-. ,-. |6| |2| |0| `-' `-' `-' unqueue-B ->tail != curr && !node->next If reordering of following stores happen then prev->next where prev being CPU2 would be updated to point to CPU0 node: tail v ,-. <- ,-. ,-. |6| |2| |0| `-' `-' -> `-' osq_wait_next() node->next <- 0 xchg(node->next, NULL) tail v ,-. <- ,-. ,-. |6| |2| |0| `-' `-' `-' unqueue-C At this point if next instruction WRITE_ONCE(next->prev, prev); in CPU2 path is committed before the update of CPU0 node->prev = prev then CPU0 node->prev will point to CPU6 node. tail v----------. v ,-. <- ,-. ,-. |6| |2| |0| `-' `-' `-' `----------^ At this point if CPU0 path's node->prev = prev is committed resulting in change of CPU0 prev back to CPU2 node. CPU2 node->next is NULL currently, tail v ,-. <- ,-. <- ,-. |6| |2| |0| `-' `-' `-' `----------^ so if CPU0 gets into unqueue path of osq_lock it will keep spinning in infinite loop as condition prev->next == node will never be true. Signed-off-by: Prateek Sood [ Added pictures, rewrote comments. ] Signed-off-by: Peter Zijlstra (Intel) Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Thomas Gleixner Cc: sramana@codeaurora.org Link: http://lkml.kernel.org/r/1500040076-27626-1-git-send-email-prsood@codeaurora.org Signed-off-by: Ingo Molnar Signed-off-by: Amit Pundir --- To be applied on 4.4.y as well. Build tested on v4.4.153. kernel/locking/osq_lock.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) -- 2.7.4 diff --git a/kernel/locking/osq_lock.c b/kernel/locking/osq_lock.c index 05a37857ab55..8d7047ecef4e 100644 --- a/kernel/locking/osq_lock.c +++ b/kernel/locking/osq_lock.c @@ -104,6 +104,19 @@ bool osq_lock(struct optimistic_spin_queue *lock) prev = decode_cpu(old); node->prev = prev; + + /* + * osq_lock() unqueue + * + * node->prev = prev osq_wait_next() + * WMB MB + * prev->next = node next->prev = prev // unqueue-C + * + * Here 'node->prev' and 'next->prev' are the same variable and we need + * to ensure these stores happen in-order to avoid corrupting the list. + */ + smp_wmb(); + WRITE_ONCE(prev->next, node); /*