From patchwork Fri Feb 14 23:23:00 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212847 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53A06C3F68F for ; Fri, 14 Feb 2020 23:23:15 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 21D97217F4 for ; Fri, 14 Feb 2020 23:23:15 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="h8uRaeIR" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728343AbgBNXXO (ORCPT ); Fri, 14 Feb 2020 18:23:14 -0500 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:17213 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728176AbgBNXXM (ORCPT ); Fri, 14 Feb 2020 18:23:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722593; x=1613258593; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=j3Iz4zet15tTZehEn1q0m7zvUCktveIQzobSO7Tbh5A=; b=h8uRaeIRKXw3Vk0FQN9W2P2da1CbF74prdqapqBbBhtVYsC9992I310a wMJxPCwkWnUFYYSZK2nbw+nIlxOYkv9nRXm2y0USMZpaa4MyhhAioNI6T Arg+49/vtVUWGQJULIGbSS+5QZLrLpIfu9yzt2e30izmohytApurhIg4o 8=; IronPort-SDR: XqbB6njxZYLKkIjkrKS2XRIKw7AMuDayPVSyoFyViFpl0hxCLi0sqh45jyTtWEAAW59D1maMOt C5x6jnIh1OpA== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="25191815" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-2b-baacba05.us-west-2.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 14 Feb 2020 23:23:11 +0000 Received: from EX13MTAUWA001.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan2.pdx.amazon.com [10.170.41.162]) by email-inbound-relay-2b-baacba05.us-west-2.amazon.com (Postfix) with ESMTPS id 48800A2065; Fri, 14 Feb 2020 23:23:09 +0000 (UTC) Received: from EX13D10UWA002.ant.amazon.com (10.43.160.228) by EX13MTAUWA001.ant.amazon.com (10.43.160.58) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:23:01 +0000 Received: from EX13MTAUWA001.ant.amazon.com (10.43.160.58) by EX13D10UWA002.ant.amazon.com (10.43.160.228) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:23:00 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.160.118) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 14 Feb 2020 23:23:00 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id E28634028E; Fri, 14 Feb 2020 23:23:00 +0000 (UTC) Date: Fri, 14 Feb 2020 23:23:00 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 02/12] xenbus: add freeze/thaw/restore callbacks support Message-ID: <4693a477cca9a4f56ec369853b7cf982aaf30e7e.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Munehisa Kamata Since commit b3e96c0c7562 ("xen: use freeze/restore/thaw PM events for suspend/resume/chkpt"), xenbus uses PMSG_FREEZE, PMSG_THAW and PMSG_RESTORE events for Xen suspend. However, they're actually assigned to xenbus_dev_suspend(), xenbus_dev_cancel() and xenbus_dev_resume() respectively, and only suspend and resume callbacks are supported at driver level. To support PM suspend and PM hibernation, modify the bus level PM callbacks to invoke not only device driver's suspend/resume but also freeze/thaw/restore. Note that we'll use freeze/restore callbacks even for PM suspend whereas suspend/resume callbacks are normally used in the case, becausae the existing xenbus device drivers already have suspend/resume callbacks specifically designed for Xen suspend. So we can allow the device drivers to keep the existing callbacks wihtout modification. [Anchal Changelog: Refactored the callbacks code] Signed-off-by: Agarwal Anchal Signed-off-by: Munehisa Kamata --- drivers/xen/xenbus/xenbus_probe.c | 99 +++++++++++++++++++++++++------ include/xen/xenbus.h | 3 + 2 files changed, 84 insertions(+), 18 deletions(-) diff --git a/drivers/xen/xenbus/xenbus_probe.c b/drivers/xen/xenbus/xenbus_probe.c index 5b471889d723..0fa8eeee68c2 100644 --- a/drivers/xen/xenbus/xenbus_probe.c +++ b/drivers/xen/xenbus/xenbus_probe.c @@ -49,6 +49,7 @@ #include #include #include +#include #include #include @@ -597,27 +598,44 @@ int xenbus_dev_suspend(struct device *dev) struct xenbus_driver *drv; struct xenbus_device *xdev = container_of(dev, struct xenbus_device, dev); - + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); DPRINTK("%s", xdev->nodename); if (dev->driver == NULL) return 0; drv = to_xenbus_driver(dev->driver); - if (drv->suspend) - err = drv->suspend(xdev); - if (err) - pr_warn("suspend %s failed: %i\n", dev_name(dev), err); + + if (xen_suspend) { + if (drv->suspend) + err = drv->suspend(xdev); + } else { + if (drv->freeze) { + err = drv->freeze(xdev); + if (!err) { + free_otherend_watch(xdev); + free_otherend_details(xdev); + return 0; + } + } + } + + if (err) { + pr_warn("%s %s failed: %i\n", xen_suspend ? + "suspend" : "freeze", dev_name(dev), err); + return err; + } + return 0; } EXPORT_SYMBOL_GPL(xenbus_dev_suspend); int xenbus_dev_resume(struct device *dev) { - int err; + int err = 0; struct xenbus_driver *drv; struct xenbus_device *xdev = container_of(dev, struct xenbus_device, dev); - + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); DPRINTK("%s", xdev->nodename); if (dev->driver == NULL) @@ -625,24 +643,32 @@ int xenbus_dev_resume(struct device *dev) drv = to_xenbus_driver(dev->driver); err = talk_to_otherend(xdev); if (err) { - pr_warn("resume (talk_to_otherend) %s failed: %i\n", + pr_warn("%s (talk_to_otherend) %s failed: %i\n", + xen_suspend ? "resume" : "restore", dev_name(dev), err); return err; } - xdev->state = XenbusStateInitialising; + if (xen_suspend) { + xdev->state = XenbusStateInitialising; + if (drv->resume) + err = drv->resume(xdev); + } else { + if (drv->restore) + err = drv->restore(xdev); + } - if (drv->resume) { - err = drv->resume(xdev); - if (err) { - pr_warn("resume %s failed: %i\n", dev_name(dev), err); - return err; - } + if (err) { + pr_warn("%s %s failed: %i\n", + xen_suspend ? "resume" : "restore", + dev_name(dev), err); + return err; } err = watch_otherend(xdev); if (err) { - pr_warn("resume (watch_otherend) %s failed: %d.\n", + pr_warn("%s (watch_otherend) %s failed: %d.\n", + xen_suspend ? "resume" : "restore", dev_name(dev), err); return err; } @@ -653,8 +679,45 @@ EXPORT_SYMBOL_GPL(xenbus_dev_resume); int xenbus_dev_cancel(struct device *dev) { - /* Do nothing */ - DPRINTK("cancel"); + int err = 0; + struct xenbus_driver *drv; + struct xenbus_device *xdev + = container_of(dev, struct xenbus_device, dev); + bool xen_suspend = xen_suspend_mode_is_xen_suspend(); + + if (xen_suspend) { + /* Do nothing */ + DPRINTK("cancel"); + return 0; + } + + DPRINTK("%s", xdev->nodename); + + if (dev->driver == NULL) + return 0; + drv = to_xenbus_driver(dev->driver); + err = talk_to_otherend(xdev); + if (err) { + pr_warn("thaw (talk_to_otherend) %s failed: %d.\n", + dev_name(dev), err); + return err; + } + + if (drv->thaw) { + err = drv->thaw(xdev); + if (err) { + pr_warn("thaw %s failed: %i\n", dev_name(dev), err); + return err; + } + } + + err = watch_otherend(xdev); + if (err) { + pr_warn("thaw (watch_otherend) %s failed: %d.\n", + dev_name(dev), err); + return err; + } + return 0; } EXPORT_SYMBOL_GPL(xenbus_dev_cancel); diff --git a/include/xen/xenbus.h b/include/xen/xenbus.h index 869c816d5f8c..20261d5f4e78 100644 --- a/include/xen/xenbus.h +++ b/include/xen/xenbus.h @@ -100,6 +100,9 @@ struct xenbus_driver { int (*remove)(struct xenbus_device *dev); int (*suspend)(struct xenbus_device *dev); int (*resume)(struct xenbus_device *dev); + int (*freeze)(struct xenbus_device *dev); + int (*thaw)(struct xenbus_device *dev); + int (*restore)(struct xenbus_device *dev); int (*uevent)(struct xenbus_device *, struct kobj_uevent_env *); struct device_driver driver; int (*read_otherend_details)(struct xenbus_device *dev); From patchwork Fri Feb 14 23:24:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212846 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B95CBC3B1BF for ; Fri, 14 Feb 2020 23:24:54 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 8E5B32081E for ; Fri, 14 Feb 2020 23:24:54 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="Mgk0Dq1O" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727996AbgBNXYt (ORCPT ); Fri, 14 Feb 2020 18:24:49 -0500 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:19684 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727649AbgBNXYs (ORCPT ); Fri, 14 Feb 2020 18:24:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722688; x=1613258688; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=O4TyplSL+UdPLY/NuUPDprwnMfLeEuUi3w9c5rcDMEE=; b=Mgk0Dq1O8Jq52EqeAClb4ctgTMPGRrN41Qi5hucpS+1YlNy3qcH83x7y bji9MIg3zxk+pxyD9APV8BvbcaPPUlj5F0DnuoYdWiAbiXGJR685fvpc/ jpt2mmteyKVUldijdhKZs7dGZhqKnlc/d5q2F/jPyNXH44EkXbKe1XUDB I=; IronPort-SDR: jucGW3i6+D+Zt1UnEu8igkzhVY7ZncQ6ZmzcjtF2QWs4I5Qu8TXOsW70zoTfaFnk99NzPxn2lB 9Cxg1E1dRtaQ== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="26558627" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 14 Feb 2020 23:24:45 +0000 Received: from EX13MTAUWB001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com (Postfix) with ESMTPS id 15478A2998; Fri, 14 Feb 2020 23:24:38 +0000 (UTC) Received: from EX13D05UWB003.ant.amazon.com (10.43.161.26) by EX13MTAUWB001.ant.amazon.com (10.43.161.207) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:24:19 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D05UWB003.ant.amazon.com (10.43.161.26) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:24:18 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 14 Feb 2020 23:24:18 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id DCDD44028E; Fri, 14 Feb 2020 23:24:18 +0000 (UTC) Date: Fri, 14 Feb 2020 23:24:18 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 04/12] x86/xen: add system core suspend and resume callbacks Message-ID: <49b41926cf0d2b50a9632d425cd09257853ca73b.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Munehisa Kamata Add Xen PVHVM specific system core callbacks for PM suspend and hibernation support. The callbacks suspend and resume Xen primitives,like shared_info, pvclock and grant table. Note that Xen suspend can handle them in a different manner, but system core callbacks are called from the context. So if the callbacks are called from Xen suspend context, return immediately. Signed-off-by: Agarwal Anchal Signed-off-by: Munehisa Kamata --- arch/x86/xen/enlighten_hvm.c | 1 + arch/x86/xen/suspend.c | 53 ++++++++++++++++++++++++++++++++++++ include/xen/xen-ops.h | 3 ++ 3 files changed, 57 insertions(+) diff --git a/arch/x86/xen/enlighten_hvm.c b/arch/x86/xen/enlighten_hvm.c index 75b1ec7a0fcd..138e71786e03 100644 --- a/arch/x86/xen/enlighten_hvm.c +++ b/arch/x86/xen/enlighten_hvm.c @@ -204,6 +204,7 @@ static void __init xen_hvm_guest_init(void) if (xen_feature(XENFEAT_hvm_callback_vector)) xen_have_vector_callback = 1; + xen_setup_syscore_ops(); xen_hvm_smp_init(); WARN_ON(xen_cpuhp_setup(xen_cpu_up_prepare_hvm, xen_cpu_dead_hvm)); xen_unplug_emulated_devices(); diff --git a/arch/x86/xen/suspend.c b/arch/x86/xen/suspend.c index 1d83152c761b..784c4484100b 100644 --- a/arch/x86/xen/suspend.c +++ b/arch/x86/xen/suspend.c @@ -2,17 +2,22 @@ #include #include #include +#include +#include #include #include +#include #include #include +#include #include #include #include #include #include +#include #include "xen-ops.h" #include "mmu.h" @@ -82,3 +87,51 @@ void xen_arch_suspend(void) on_each_cpu(xen_vcpu_notify_suspend, NULL, 1); } + +static int xen_syscore_suspend(void) +{ + struct xen_remove_from_physmap xrfp; + int ret; + + /* Xen suspend does similar stuffs in its own logic */ + if (xen_suspend_mode_is_xen_suspend()) + return 0; + + xrfp.domid = DOMID_SELF; + xrfp.gpfn = __pa(HYPERVISOR_shared_info) >> PAGE_SHIFT; + + ret = HYPERVISOR_memory_op(XENMEM_remove_from_physmap, &xrfp); + if (!ret) + HYPERVISOR_shared_info = &xen_dummy_shared_info; + + return ret; +} + +static void xen_syscore_resume(void) +{ + /* Xen suspend does similar stuffs in its own logic */ + if (xen_suspend_mode_is_xen_suspend()) + return; + + /* No need to setup vcpu_info as it's already moved off */ + xen_hvm_map_shared_info(); + + pvclock_resume(); + + gnttab_resume(); +} + +/* + * These callbacks will be called with interrupts disabled and when having only + * one CPU online. + */ +static struct syscore_ops xen_hvm_syscore_ops = { + .suspend = xen_syscore_suspend, + .resume = xen_syscore_resume +}; + +void __init xen_setup_syscore_ops(void) +{ + if (xen_hvm_domain()) + register_syscore_ops(&xen_hvm_syscore_ops); +} diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 6c36e161dfd1..3b3992b5b0c2 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -43,6 +43,9 @@ int xen_setup_shutdown_event(void); bool xen_suspend_mode_is_xen_suspend(void); bool xen_suspend_mode_is_pm_suspend(void); bool xen_suspend_mode_is_pm_hibernation(void); + +void xen_setup_syscore_ops(void); + extern unsigned long *xen_contiguous_bitmap; #if defined(CONFIG_XEN_PV) || defined(CONFIG_ARM) || defined(CONFIG_ARM64) From patchwork Fri Feb 14 23:25:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F0D04C35242 for ; Fri, 14 Feb 2020 23:25:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BF165217F4 for ; Fri, 14 Feb 2020 23:25:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="r8tTtiOf" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728258AbgBNXZx (ORCPT ); Fri, 14 Feb 2020 18:25:53 -0500 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:17717 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728022AbgBNXZx (ORCPT ); Fri, 14 Feb 2020 18:25:53 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722752; x=1613258752; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=65wFrGrYmXjd0UW/pA31otT2ZrHynjVNpCVt+utpQWU=; b=r8tTtiOf76Futw7LkvLm87BeP5zwH1loaG6laKLHpNY3hcH0HRYSK53z ka9iC1Vf+mNS2Z/nJAGAKYhbgkZ4NId4DiIa6VBAU8tFgOmzh3D8kbgl9 iFQgyMuB3eSFqYafLs0PPw7wYU1i/KCsxPlaJ1gLVTGhhFis99ZAptAjS Y=; IronPort-SDR: bL7Fsnk+QmgPzaLi2dVu7FU9qWoIvjANaCWDzTME9a1GTC9kcbC0NMrraFxDGeXgb6TxgZFuDt MZ9MFEE9KDJQ== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="25192150" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-807d4a99.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 14 Feb 2020 23:25:50 +0000 Received: from EX13MTAUEE002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-807d4a99.us-east-1.amazon.com (Postfix) with ESMTPS id 2FCECA2466; Fri, 14 Feb 2020 23:25:42 +0000 (UTC) Received: from EX13D08UEE004.ant.amazon.com (10.43.62.182) by EX13MTAUEE002.ant.amazon.com (10.43.62.24) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:25:35 +0000 Received: from EX13MTAUEE002.ant.amazon.com (10.43.62.24) by EX13D08UEE004.ant.amazon.com (10.43.62.182) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:25:35 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.62.224) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 14 Feb 2020 23:25:35 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id D719E4028E; Fri, 14 Feb 2020 23:25:34 +0000 (UTC) Date: Fri, 14 Feb 2020 23:25:34 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation Message-ID: <890c404c585d7790514527f0c021056a7be6e748.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Munehisa Kamata Signed-off-by: Munehisa Kamata --- drivers/block/xen-blkfront.c | 119 ++++++++++++++++++++++++++++++++--- 1 file changed, 112 insertions(+), 7 deletions(-) diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c index 478120233750..d715ed3cb69a 100644 --- a/drivers/block/xen-blkfront.c +++ b/drivers/block/xen-blkfront.c @@ -47,6 +47,8 @@ #include #include #include +#include +#include #include #include @@ -79,6 +81,8 @@ enum blkif_state { BLKIF_STATE_DISCONNECTED, BLKIF_STATE_CONNECTED, BLKIF_STATE_SUSPENDED, + BLKIF_STATE_FREEZING, + BLKIF_STATE_FROZEN }; struct grant { @@ -220,6 +224,7 @@ struct blkfront_info struct list_head requests; struct bio_list bio_list; struct list_head info_list; + struct completion wait_backend_disconnected; }; static unsigned int nr_minors; @@ -261,6 +266,7 @@ static DEFINE_SPINLOCK(minor_lock); static int blkfront_setup_indirect(struct blkfront_ring_info *rinfo); static void blkfront_gather_backend_features(struct blkfront_info *info); static int negotiate_mq(struct blkfront_info *info); +static void __blkif_free(struct blkfront_info *info); static int get_id_from_freelist(struct blkfront_ring_info *rinfo) { @@ -995,6 +1001,7 @@ static int xlvbd_init_blk_queue(struct gendisk *gd, u16 sector_size, info->sector_size = sector_size; info->physical_sector_size = physical_sector_size; blkif_set_queue_limits(info); + init_completion(&info->wait_backend_disconnected); return 0; } @@ -1218,6 +1225,8 @@ static void xlvbd_release_gendisk(struct blkfront_info *info) /* Already hold rinfo->ring_lock. */ static inline void kick_pending_request_queues_locked(struct blkfront_ring_info *rinfo) { + if (unlikely(rinfo->dev_info->connected == BLKIF_STATE_FREEZING)) + return; if (!RING_FULL(&rinfo->ring)) blk_mq_start_stopped_hw_queues(rinfo->dev_info->rq, true); } @@ -1341,8 +1350,6 @@ static void blkif_free_ring(struct blkfront_ring_info *rinfo) static void blkif_free(struct blkfront_info *info, int suspend) { - unsigned int i; - /* Prevent new requests being issued until we fix things up. */ info->connected = suspend ? BLKIF_STATE_SUSPENDED : BLKIF_STATE_DISCONNECTED; @@ -1350,6 +1357,13 @@ static void blkif_free(struct blkfront_info *info, int suspend) if (info->rq) blk_mq_stop_hw_queues(info->rq); + __blkif_free(info); +} + +static void __blkif_free(struct blkfront_info *info) +{ + unsigned int i; + for (i = 0; i < info->nr_rings; i++) blkif_free_ring(&info->rinfo[i]); @@ -1553,8 +1567,10 @@ static irqreturn_t blkif_interrupt(int irq, void *dev_id) struct blkfront_ring_info *rinfo = (struct blkfront_ring_info *)dev_id; struct blkfront_info *info = rinfo->dev_info; - if (unlikely(info->connected != BLKIF_STATE_CONNECTED)) - return IRQ_HANDLED; + if (unlikely(info->connected != BLKIF_STATE_CONNECTED)) { + if (info->connected != BLKIF_STATE_FREEZING) + return IRQ_HANDLED; + } spin_lock_irqsave(&rinfo->ring_lock, flags); again: @@ -2020,6 +2036,7 @@ static int blkif_recover(struct blkfront_info *info) struct bio *bio; unsigned int segs; + bool frozen = info->connected == BLKIF_STATE_FROZEN; blkfront_gather_backend_features(info); /* Reset limits changed by blk_mq_update_nr_hw_queues(). */ blkif_set_queue_limits(info); @@ -2046,6 +2063,9 @@ static int blkif_recover(struct blkfront_info *info) kick_pending_request_queues(rinfo); } + if (frozen) + return 0; + list_for_each_entry_safe(req, n, &info->requests, queuelist) { /* Requeue pending requests (flush or discard) */ list_del_init(&req->queuelist); @@ -2359,6 +2379,7 @@ static void blkfront_connect(struct blkfront_info *info) return; case BLKIF_STATE_SUSPENDED: + case BLKIF_STATE_FROZEN: /* * If we are recovering from suspension, we need to wait * for the backend to announce it's features before @@ -2476,12 +2497,37 @@ static void blkback_changed(struct xenbus_device *dev, break; case XenbusStateClosed: - if (dev->state == XenbusStateClosed) + if (dev->state == XenbusStateClosed) { + if (info->connected == BLKIF_STATE_FREEZING) { + __blkif_free(info); + info->connected = BLKIF_STATE_FROZEN; + complete(&info->wait_backend_disconnected); + break; + } + break; + } + + /* + * We may somehow receive backend's Closed again while thawing + * or restoring and it causes thawing or restoring to fail. + * Ignore such unexpected state anyway. + */ + if (info->connected == BLKIF_STATE_FROZEN && + dev->state == XenbusStateInitialised) { + dev_dbg(&dev->dev, + "ignore the backend's Closed state: %s", + dev->nodename); + break; + } /* fall through */ case XenbusStateClosing: - if (info) - blkfront_closing(info); + if (info) { + if (info->connected == BLKIF_STATE_FREEZING) + xenbus_frontend_closed(dev); + else + blkfront_closing(info); + } break; } } @@ -2625,6 +2671,62 @@ static void blkif_release(struct gendisk *disk, fmode_t mode) mutex_unlock(&blkfront_mutex); } +static int blkfront_freeze(struct xenbus_device *dev) +{ + unsigned int i; + struct blkfront_info *info = dev_get_drvdata(&dev->dev); + struct blkfront_ring_info *rinfo; + /* This would be reasonable timeout as used in xenbus_dev_shutdown() */ + unsigned int timeout = 5 * HZ; + int err = 0; + + info->connected = BLKIF_STATE_FREEZING; + + blk_mq_freeze_queue(info->rq); + blk_mq_quiesce_queue(info->rq); + + for (i = 0; i < info->nr_rings; i++) { + rinfo = &info->rinfo[i]; + + gnttab_cancel_free_callback(&rinfo->callback); + flush_work(&rinfo->work); + } + + /* Kick the backend to disconnect */ + xenbus_switch_state(dev, XenbusStateClosing); + + /* + * We don't want to move forward before the frontend is diconnected + * from the backend cleanly. + */ + timeout = wait_for_completion_timeout(&info->wait_backend_disconnected, + timeout); + if (!timeout) { + err = -EBUSY; + xenbus_dev_error(dev, err, "Freezing timed out;" + "the device may become inconsistent state"); + } + + return err; +} + +static int blkfront_restore(struct xenbus_device *dev) +{ + struct blkfront_info *info = dev_get_drvdata(&dev->dev); + int err = 0; + + err = talk_to_blkback(dev, info); + blk_mq_unquiesce_queue(info->rq); + blk_mq_unfreeze_queue(info->rq); + + if (err) + goto out; + blk_mq_update_nr_hw_queues(&info->tag_set, info->nr_rings); + +out: + return err; +} + static const struct block_device_operations xlvbd_block_fops = { .owner = THIS_MODULE, @@ -2647,6 +2749,9 @@ static struct xenbus_driver blkfront_driver = { .resume = blkfront_resume, .otherend_changed = blkback_changed, .is_ready = blkfront_is_ready, + .freeze = blkfront_freeze, + .thaw = blkfront_restore, + .restore = blkfront_restore }; static void purge_persistent_grants(struct blkfront_info *info) From patchwork Fri Feb 14 23:26:24 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212844 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13DA5C35242 for ; Fri, 14 Feb 2020 23:26:51 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id C8FF5217F4 for ; Fri, 14 Feb 2020 23:26:50 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="K7YnIgqv" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728119AbgBNX0u (ORCPT ); Fri, 14 Feb 2020 18:26:50 -0500 Received: from smtp-fw-9101.amazon.com ([207.171.184.25]:9208 "EHLO smtp-fw-9101.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727963AbgBNX0t (ORCPT ); Fri, 14 Feb 2020 18:26:49 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722809; x=1613258809; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=UMKkqskcj+uJc+bMo+rVrL+B1mUIrmErlOz9JcigkDg=; b=K7YnIgqvLokD5Zq7gJtR+xt+KWcEPo58+XvVX/HcROgC7lM147PcEho/ jcGOZyfAyaEYDax4nMIyx5cLiHe8+yfGtZbgsUOrCFf5yubwIbQ185lFa n7rb7LVujZxHf78bVUGxpcwxQE7uCS4bLl6jqNLBxq8O6vFpV5AytflBN s=; IronPort-SDR: XLi6hZdvWyf7uNHazz/iulkhBXgI093HQ6dQRzAdNaXJql5DDSrkG+d7cAh5YRJHw6o1DuT9Es 275DPTN1UEvw== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="16798132" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-38ae4ad2.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9101.sea19.amazon.com with ESMTP; 14 Feb 2020 23:26:47 +0000 Received: from EX13MTAUWB001.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1d-38ae4ad2.us-east-1.amazon.com (Postfix) with ESMTPS id 46059A272A; Fri, 14 Feb 2020 23:26:39 +0000 (UTC) Received: from EX13D07UWB004.ant.amazon.com (10.43.161.196) by EX13MTAUWB001.ant.amazon.com (10.43.161.249) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:26:25 +0000 Received: from EX13MTAUWB001.ant.amazon.com (10.43.161.207) by EX13D07UWB004.ant.amazon.com (10.43.161.196) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:26:25 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.161.249) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 14 Feb 2020 23:26:24 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id 9BC794028E; Fri, 14 Feb 2020 23:26:24 +0000 (UTC) Date: Fri, 14 Feb 2020 23:26:24 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 08/12] xen/time: introduce xen_{save,restore}_steal_clock Message-ID: <489caa869095f77ed8db188d36e858533d7b4d7c.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Munehisa Kamata Currently, steal time accounting code in scheduler expects steal clock callback to provide monotonically increasing value. If the accounting code receives a smaller value than previous one, it uses a negative value to calculate steal time and results in incorrectly updated idle and steal time accounting. This breaks userspace tools which read /proc/stat. top - 08:05:35 up 2:12, 3 users, load average: 0.00, 0.07, 0.23 Tasks: 80 total, 1 running, 79 sleeping, 0 stopped, 0 zombie Cpu(s): 0.0%us, 0.0%sy, 0.0%ni,30100.0%id, 0.0%wa, 0.0%hi, 0.0%si,-1253874204672.0%st This can actually happen when a Xen PVHVM guest gets restored from hibernation, because such a restored guest is just a fresh domain from Xen perspective and the time information in runstate info starts over from scratch. This patch introduces xen_save_steal_clock() which saves current values in runstate info into per-cpu variables. Its couterpart, xen_restore_steal_clock(), sets offset if it found the current values in runstate info are smaller than previous ones. xen_steal_clock() is also modified to use the offset to ensure that scheduler only sees monotonically increasing number. Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal --- drivers/xen/time.c | 29 ++++++++++++++++++++++++++++- include/xen/xen-ops.h | 2 ++ 2 files changed, 30 insertions(+), 1 deletion(-) diff --git a/drivers/xen/time.c b/drivers/xen/time.c index 0968859c29d0..3560222cc0dd 100644 --- a/drivers/xen/time.c +++ b/drivers/xen/time.c @@ -23,6 +23,9 @@ static DEFINE_PER_CPU(struct vcpu_runstate_info, xen_runstate); static DEFINE_PER_CPU(u64[4], old_runstate_time); +static DEFINE_PER_CPU(u64, xen_prev_steal_clock); +static DEFINE_PER_CPU(u64, xen_steal_clock_offset); + /* return an consistent snapshot of 64-bit time/counter value */ static u64 get64(const u64 *p) { @@ -149,7 +152,7 @@ bool xen_vcpu_stolen(int vcpu) return per_cpu(xen_runstate, vcpu).state == RUNSTATE_runnable; } -u64 xen_steal_clock(int cpu) +static u64 __xen_steal_clock(int cpu) { struct vcpu_runstate_info state; @@ -157,6 +160,30 @@ u64 xen_steal_clock(int cpu) return state.time[RUNSTATE_runnable] + state.time[RUNSTATE_offline]; } +u64 xen_steal_clock(int cpu) +{ + return __xen_steal_clock(cpu) + per_cpu(xen_steal_clock_offset, cpu); +} + +void xen_save_steal_clock(int cpu) +{ + per_cpu(xen_prev_steal_clock, cpu) = xen_steal_clock(cpu); +} + +void xen_restore_steal_clock(int cpu) +{ + u64 steal_clock = __xen_steal_clock(cpu); + + if (per_cpu(xen_prev_steal_clock, cpu) > steal_clock) { + /* Need to update the offset */ + per_cpu(xen_steal_clock_offset, cpu) = + per_cpu(xen_prev_steal_clock, cpu) - steal_clock; + } else { + /* Avoid unnecessary steal clock warp */ + per_cpu(xen_steal_clock_offset, cpu) = 0; + } +} + void xen_setup_runstate_info(int cpu) { struct vcpu_register_runstate_memory_area area; diff --git a/include/xen/xen-ops.h b/include/xen/xen-ops.h index 3b3992b5b0c2..12b3f4474a05 100644 --- a/include/xen/xen-ops.h +++ b/include/xen/xen-ops.h @@ -37,6 +37,8 @@ void xen_time_setup_guest(void); void xen_manage_runstate_time(int action); void xen_get_runstate_snapshot(struct vcpu_runstate_info *res); u64 xen_steal_clock(int cpu); +void xen_save_steal_clock(int cpu); +void xen_restore_steal_clock(int cpu); int xen_setup_shutdown_event(void); From patchwork Fri Feb 14 23:27:18 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212843 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D3357C3F68F for ; Fri, 14 Feb 2020 23:27:53 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id AAD372086A for ; Fri, 14 Feb 2020 23:27:53 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="wIWeAYgn" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727963AbgBNX1s (ORCPT ); Fri, 14 Feb 2020 18:27:48 -0500 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:1035 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727649AbgBNX1s (ORCPT ); Fri, 14 Feb 2020 18:27:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722867; x=1613258867; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=w2YudPijii+dzE2cW4maD6E6FQY1WRpU8HmwPnvoM5k=; b=wIWeAYgn40YOrYt2FC6c5BTIphd+RIWYLpUy4SbFnDl1Cp9drlyhvVc6 N6VynUKymRW5+/IXaVCtxQa6ACgoS7pFMti42jWpUJ9wV9sdUlKLIfV9c JA7UZLw1SXo1k6n10q7e3xqgNSG/14gZHHn0S6xD+tF38IR+IZUBgLjbu 0=; IronPort-SDR: 4QzLk7agU2cCkHMRbCkf+yxdh7O3HX/h13ILndMQD5DBXAC5QIGb3xBDBxJz+ZI2EvA/bpTqBv mUC+a3+dOYhg== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="26558954" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 14 Feb 2020 23:27:45 +0000 Received: from EX13MTAUEB002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1d-74cf8b49.us-east-1.amazon.com (Postfix) with ESMTPS id 9E2B3C0845; Fri, 14 Feb 2020 23:27:38 +0000 (UTC) Received: from EX13D08UEB001.ant.amazon.com (10.43.60.245) by EX13MTAUEB002.ant.amazon.com (10.43.60.12) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:27:25 +0000 Received: from EX13MTAUEB002.ant.amazon.com (10.43.60.12) by EX13D08UEB001.ant.amazon.com (10.43.60.245) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:27:25 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.60.234) with Microsoft SMTP Server id 15.0.1367.3 via Frontend Transport; Fri, 14 Feb 2020 23:27:19 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id F07DD4028E; Fri, 14 Feb 2020 23:27:18 +0000 (UTC) Date: Fri, 14 Feb 2020 23:27:18 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 10/12] xen: Introduce wrapper for save/restore sched clock offset Message-ID: <305f20fbf503b637b1a08b7bec543f49271a725a.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org Introduce wrappers for save/restore xen_sched_clock_offset to be used by PM hibernation code to avoid system instability during resume. Signed-off-by: Anchal Agarwal --- arch/x86/xen/time.c | 15 +++++++++++++-- arch/x86/xen/xen-ops.h | 2 ++ 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c index 8cf632dda605..eeb6d3d2eaab 100644 --- a/arch/x86/xen/time.c +++ b/arch/x86/xen/time.c @@ -379,12 +379,23 @@ static const struct pv_time_ops xen_time_ops __initconst = { static struct pvclock_vsyscall_time_info *xen_clock __read_mostly; static u64 xen_clock_value_saved; +/*This is needed to maintain a monotonic clock value during PM hibernation */ +void xen_save_sched_clock_offset(void) +{ + xen_clock_value_saved = xen_clocksource_read() - xen_sched_clock_offset; +} + +void xen_restore_sched_clock_offset(void) +{ + xen_sched_clock_offset = xen_clocksource_read() - xen_clock_value_saved; +} + void xen_save_time_memory_area(void) { struct vcpu_register_time_memory_area t; int ret; - xen_clock_value_saved = xen_clocksource_read() - xen_sched_clock_offset; + xen_save_sched_clock_offset(); if (!xen_clock) return; @@ -426,7 +437,7 @@ void xen_restore_time_memory_area(void) out: /* Need pvclock_resume() before using xen_clocksource_read(). */ pvclock_resume(); - xen_sched_clock_offset = xen_clocksource_read() - xen_clock_value_saved; + xen_restore_sched_clock_offset(); } static void xen_setup_vsyscall_time_info(void) diff --git a/arch/x86/xen/xen-ops.h b/arch/x86/xen/xen-ops.h index d84c357994bd..9f49124df033 100644 --- a/arch/x86/xen/xen-ops.h +++ b/arch/x86/xen/xen-ops.h @@ -72,6 +72,8 @@ void xen_save_time_memory_area(void); void xen_restore_time_memory_area(void); void xen_init_time_ops(void); void xen_hvm_init_time_ops(void); +void xen_save_sched_clock_offset(void); +void xen_restore_sched_clock_offset(void); irqreturn_t xen_debug_interrupt(int irq, void *dev_id); From patchwork Fri Feb 14 23:28:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anchal Agarwal X-Patchwork-Id: 212842 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.4 required=3.0 tests=DKIMWL_WL_HIGH, DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39A1DC35242 for ; Fri, 14 Feb 2020 23:28:38 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 0C18B222C4 for ; Fri, 14 Feb 2020 23:28:38 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=amazon.com header.i=@amazon.com header.b="m0a62/lK" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728172AbgBNX2h (ORCPT ); Fri, 14 Feb 2020 18:28:37 -0500 Received: from smtp-fw-33001.amazon.com ([207.171.190.10]:1236 "EHLO smtp-fw-33001.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727976AbgBNX2h (ORCPT ); Fri, 14 Feb 2020 18:28:37 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1581722917; x=1613258917; h=date:from:to:subject:message-id:references:mime-version: in-reply-to; bh=XFd2LTl25ne3kMWrL/YPna7uVttvsKNxoFQR8TCvE9E=; b=m0a62/lKKLCqMlQYPcsCumDyvtuTauNQVD6L5UwkutcOiVZvd078XFM8 DlDNWxn5Xvj1kG7gqOKOJNpPw3+ZVsKfayPQd8MhIHUb68E8Jh42wok7P HOkZIEbFOF/8UjAerb3YePdKAakmk+Ikxw7g+Qo/HpVw3ZFOFQtqklxMz w=; IronPort-SDR: Uq4xL82dqRbjGX7NdFvnD6eSDrEZWhgkJv2ITzh9uXV0XvYsU1YnSahYNjBZ0qyDvhYKBxVhJc Teid1lqAMnhQ== X-IronPort-AV: E=Sophos;i="5.70,442,1574121600"; d="scan'208";a="26559030" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1e-303d0b0e.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-33001.sea14.amazon.com with ESMTP; 14 Feb 2020 23:28:35 +0000 Received: from EX13MTAUEB002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan3.iad.amazon.com [10.40.159.166]) by email-inbound-relay-1e-303d0b0e.us-east-1.amazon.com (Postfix) with ESMTPS id 3AFE1A23FF; Fri, 14 Feb 2020 23:28:27 +0000 (UTC) Received: from EX13D08UEB002.ant.amazon.com (10.43.60.107) by EX13MTAUEB002.ant.amazon.com (10.43.60.12) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:28:07 +0000 Received: from EX13MTAUEA002.ant.amazon.com (10.43.61.77) by EX13D08UEB002.ant.amazon.com (10.43.60.107) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Fri, 14 Feb 2020 23:28:07 +0000 Received: from dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (172.22.96.68) by mail-relay.amazon.com (10.43.61.169) with Microsoft SMTP Server id 15.0.1236.3 via Frontend Transport; Fri, 14 Feb 2020 23:28:07 +0000 Received: by dev-dsk-anchalag-2a-9c2d1d96.us-west-2.amazon.com (Postfix, from userid 4335130) id 500DF4028E; Fri, 14 Feb 2020 23:28:07 +0000 (UTC) Date: Fri, 14 Feb 2020 23:28:07 +0000 From: Anchal Agarwal To: , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH v3 12/12] PM / hibernate: update the resume offset on SNAPSHOT_SET_SWAP_AREA Message-ID: <4659d20be8f27e40ef39adfa06b0c759c2d6cd78.1581721799.git.anchalag@amazon.com> References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-pm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pm@vger.kernel.org From: Aleksei Besogonov The SNAPSHOT_SET_SWAP_AREA is supposed to be used to set the hibernation offset on a running kernel to enable hibernating to a swap file. However, it doesn't actually update the swsusp_resume_block variable. As a result, the hibernation fails at the last step (after all the data is written out) in the validation of the swap signature in mark_swapfiles(). Before this patch, the command line processing was the only place where swsusp_resume_block was set. Signed-off-by: Aleksei Besogonov Signed-off-by: Munehisa Kamata Signed-off-by: Anchal Agarwal --- kernel/power/user.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/kernel/power/user.c b/kernel/power/user.c index 77438954cc2b..d396e313cb7b 100644 --- a/kernel/power/user.c +++ b/kernel/power/user.c @@ -374,8 +374,12 @@ static long snapshot_ioctl(struct file *filp, unsigned int cmd, if (swdev) { offset = swap_area.offset; data->swap = swap_type_of(swdev, offset, NULL); - if (data->swap < 0) + if (data->swap < 0) { error = -ENODEV; + } else { + swsusp_resume_device = swdev; + swsusp_resume_block = offset; + } } else { data->swap = -1; error = -EINVAL;