From patchwork Thu May 8 21:18:28 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Wei Huang X-Patchwork-Id: 29866 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f200.google.com (mail-ie0-f200.google.com [209.85.223.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 20EE420534 for ; Thu, 8 May 2014 21:23:25 +0000 (UTC) Received: by mail-ie0-f200.google.com with SMTP id lx4sf15072450iec.3 for ; Thu, 08 May 2014 14:23:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:cc:subject:precedence:list-id:list-unsubscribe:list-post :list-help:list-subscribe:mime-version:sender:errors-to :x-original-sender:x-original-authentication-results:mailing-list :list-archive:content-type:content-transfer-encoding; bh=CAbswFV9BkonxE9T/IMDJlx3lxj8EJcLv3OE8/IRiaE=; b=DWzAhUSGqfb2X9tvsmya5ORAmSD6o3TNGs7IUIiiSrULQzmbkIxqpVH/F2gyW3QHvd M1N2mWGETrMHxdV77pIt8dXZYejxigTPpODAH4xxPbyT4QMYSfJgkAqyv3ef9bSfsK1V iY7G3KZBj2IRslzhzquUN5tUMxJK2ZfLyd9N8Q53KhLVzRr/5A+ii8D3DsSHX5ePLk0v Krth36Eoy/NTZgbdvZ4ZU9tWVV5PK3bsHZdqUO8WnfkOrA+xYTxMJtRFp37YQW8XELRY nh5DBVcetGVwv5fznpeco5mSa1FhESVVgygDWmrHYX6B/fTtTQKWr8+9yl53ILi+I/a5 Xd4w== X-Gm-Message-State: ALoCoQk+9rA00CZsIHL8vWp0f5f0OjNmwr62StlaqGDcvpKatCOo2lpeANiV3EeuWrPBHLLg2nle X-Received: by 10.182.137.233 with SMTP id ql9mr3014578obb.46.1399584204513; Thu, 08 May 2014 14:23:24 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.88.105 with SMTP id s96ls67753qgd.25.gmail; Thu, 08 May 2014 14:23:24 -0700 (PDT) X-Received: by 10.58.107.65 with SMTP id ha1mr4721608veb.1.1399584204385; Thu, 08 May 2014 14:23:24 -0700 (PDT) Received: from mail-ve0-f171.google.com (mail-ve0-f171.google.com [209.85.128.171]) by mx.google.com with ESMTPS id eb17si384886veb.22.2014.05.08.14.23.24 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 08 May 2014 14:23:24 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.128.171; Received: by mail-ve0-f171.google.com with SMTP id oz11so4073629veb.2 for ; Thu, 08 May 2014 14:23:24 -0700 (PDT) X-Received: by 10.220.191.134 with SMTP id dm6mr4739078vcb.16.1399584204279; Thu, 08 May 2014 14:23:24 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp26203vcb; Thu, 8 May 2014 14:23:23 -0700 (PDT) X-Received: by 10.58.25.3 with SMTP id y3mr2473636vef.48.1399584203846; Thu, 08 May 2014 14:23:23 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id gq7si384672vdc.14.2014.05.08.14.23.23 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Thu, 08 May 2014 14:23:23 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WiVlQ-0000fq-2R; Thu, 08 May 2014 21:22:00 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WiVlN-0000ee-SO for xen-devel@lists.xen.org; Thu, 08 May 2014 21:21:58 +0000 Received: from [85.158.143.35:15884] by server-2.bemta-4.messagelabs.com id 1F/18-06539-575FB635; Thu, 08 May 2014 21:21:57 +0000 X-Env-Sender: w1.huang@samsung.com X-Msg-Ref: server-15.tower-21.messagelabs.com!1399584112!3727915!1 X-Originating-IP: [203.254.224.33] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMjAzLjI1NC4yMjQuMzMgPT4gMzY3NDY5\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 30457 invoked from network); 8 May 2014 21:21:54 -0000 Received: from mailout3.samsung.com (HELO mailout3.samsung.com) (203.254.224.33) by server-15.tower-21.messagelabs.com with DES-CBC3-SHA encrypted SMTP; 8 May 2014 21:21:54 -0000 Received: from epcpsbgm1.samsung.com (epcpsbgm1 [203.254.230.26]) by mailout3.samsung.com (Oracle Communications Messaging Server 7u4-24.01(7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTP id <0N5900M3MY0FQR70@mailout3.samsung.com> for xen-devel@lists.xen.org; Fri, 09 May 2014 06:21:51 +0900 (KST) X-AuditID: cbfee61a-b7f2b6d000006c4d-5f-536bf56fe131 Received: from epmmp1.local.host ( [203.254.227.16]) by epcpsbgm1.samsung.com (EPCPMTA) with SMTP id 21.32.27725.F65FB635; Fri, 09 May 2014 06:21:51 +0900 (KST) Received: from weihp.spa.sarc.sas ([105.140.31.10]) by mmp1.samsung.com (Oracle Communications Messaging Server 7u4-24.01 (7.0.4.24.0) 64bit (built Nov 17 2011)) with ESMTPA id <0N59002IGXZLDK90@mmp1.samsung.com>; Fri, 09 May 2014 06:21:50 +0900 (KST) From: Wei Huang To: xen-devel@lists.xen.org Date: Thu, 08 May 2014 16:18:28 -0500 Message-id: <1399583908-21755-7-git-send-email-w1.huang@samsung.com> X-Mailer: git-send-email 1.7.9.5 In-reply-to: <1399583908-21755-1-git-send-email-w1.huang@samsung.com> References: <1399583908-21755-1-git-send-email-w1.huang@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrGLMWRmVeSWpSXmKPExsVy+t9jAd38r9nBBnNfs1ssfryL3eJNbweL xcVrr5gsbvTeYrP4uechm8X0P3fYLPZ9XsVi8fPORUaLr82rGC1en/vIbLHk42IWi45/09gc eDxeT57A6LH9iYjHnWt72DyO7v7N5NG3ZRWjx/otV1k8Tt+axRbAHsVlk5Kak1mWWqRvl8CV cWHqLfaC7ycYK+6/rWlgfN7G2MXIwSEhYCLRdjKzi5ETyBSTuHBvPVsXIxeHkMAiRokfbx8z gySEBJqZJOa3i4LYbAJqEqcu/mcBsUUEpCWufb7MCNLALPCRUeLwrD6whLBAgETP0SXsIDaL gKrEw+2PGEFsXgEXiU1nXrNCLFaQmDPJBiTMKeAqcX/aPEaIXS4Sr669Y53AyLuAkWEVo2hq QXJBcVJ6rqFecWJucWleul5yfu4mRnBoPpPawbiyweIQowAHoxIP74sp2cFCrIllxZW5hxgl OJiVRHhfLAMK8aYkVlalFuXHF5XmpBYfYpTmYFES5z3Qah0oJJCeWJKanZpakFoEk2Xi4JRq YAyan50aKfA/X9fW65j876Lbmq+fHbru2bVGqr7/RJPuJ5//jkemGgkLvjgheX9i0B0uo6hJ u7505G3eKLX5zHevNP9Iy2N6XcLxWwPCMz/H3LeIn6I18VbxJ7vHnAnNsrPZTb11Mvdwu91v jajR8HstpbmFM1HmyQXntIAbVW/SdX4YWWc3KbEUZyQaajEXFScCAL9mMGFJAgAA Cc: keir@xen.org, ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, andrew.cooper3@citrix.com, julien.grall@linaro.org, tim@xen.org, jaeyong.yoo@samsung.com, jbeulich@suse.com, ian.jackson@eu.citrix.com, yjhyun.yoo@samsung.com Subject: [Xen-devel] [RFC v3 6/6] xen/arm: Implement toolstack for xl restore/save/migration X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: w1.huang@samsung.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: This patch implements xl save/restore operation in xc_arm_migrate.c and make it compilable with existing design. The operation is also used by migration. The overall process of save/restore is the following: 1) save guest parameters; 2) save memory; 3) save HVM states. Signed-off-by: Alexey Sokolov Signed-off-by: Wei Huang --- config/arm32.mk | 1 + config/arm64.mk | 1 + tools/libxc/Makefile | 5 + tools/libxc/xc_arm_migrate.c | 653 ++++++++++++++++++++++++++++++++++++++++++ tools/libxc/xc_dom_arm.c | 4 +- tools/libxc/xc_resume.c | 20 +- tools/libxl/libxl.h | 3 - tools/misc/Makefile | 4 +- 8 files changed, 677 insertions(+), 14 deletions(-) create mode 100644 tools/libxc/xc_arm_migrate.c diff --git a/config/arm32.mk b/config/arm32.mk index aa79d22..01374c9 100644 --- a/config/arm32.mk +++ b/config/arm32.mk @@ -1,6 +1,7 @@ CONFIG_ARM := y CONFIG_ARM_32 := y CONFIG_ARM_$(XEN_OS) := y +CONFIG_MIGRATE := y CONFIG_XEN_INSTALL_SUFFIX := diff --git a/config/arm64.mk b/config/arm64.mk index 15b57a4..7ac3b65 100644 --- a/config/arm64.mk +++ b/config/arm64.mk @@ -1,6 +1,7 @@ CONFIG_ARM := y CONFIG_ARM_64 := y CONFIG_ARM_$(XEN_OS) := y +CONFIG_MIGRATE := y CONFIG_XEN_INSTALL_SUFFIX := diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile index a74b19e..9aaa6ff 100644 --- a/tools/libxc/Makefile +++ b/tools/libxc/Makefile @@ -43,8 +43,13 @@ CTRL_SRCS-$(CONFIG_MiniOS) += xc_minios.c GUEST_SRCS-y := GUEST_SRCS-y += xg_private.c xc_suspend.c ifeq ($(CONFIG_MIGRATE),y) +ifeq ($(CONFIG_X86),y) GUEST_SRCS-y += xc_domain_restore.c xc_domain_save.c GUEST_SRCS-y += xc_offline_page.c xc_compression.c +endif +ifeq ($(CONFIG_ARM),y) +GUEST_SRCS-y += xc_arm_migrate.c +endif else GUEST_SRCS-y += xc_nomigrate.c endif diff --git a/tools/libxc/xc_arm_migrate.c b/tools/libxc/xc_arm_migrate.c new file mode 100644 index 0000000..776e373 --- /dev/null +++ b/tools/libxc/xc_arm_migrate.c @@ -0,0 +1,653 @@ +/* + * Copyright (c) 2014 Samsung Electronics. + * + * This program is free software; you can redistribute it and/or modify it + * under the terms and conditions of the GNU General Public License, + * version 2, as published by the Free Software Foundation. + * + * This program is distributed in the hope it will be useful, but WITHOUT + * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or + * FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for + * more details. + * + * You should have received a copy of the GNU General Public License along with + * this program; if not, write to the Free Software Foundation, Inc., 59 Temple + * Place - Suite 330, Boston, MA 02111-1307 USA. + */ + +#include +#include +#include +#include + +#include +#include +#include +#include "xc_bitops.h" +#include "xg_private.h" + +#define DEF_MAX_ITERS 29 /* limit us to 30 times round loop */ +#define DEF_MAX_FACTOR 3 /* never send more than 3x p2m_size */ +#define DEF_MIN_DIRTY_PER_ITER 50 /* dirty page count to define last iter */ +#define DEF_PROGRESS_RATE 50 /* progress bar update rate */ + +/* Guest params to save */ +typedef struct guest_params +{ + unsigned long console_pfn; + unsigned long store_pfn; + uint32_t flags; + xen_pfn_t start_gpfn; + xen_pfn_t max_gpfn; + uint32_t max_vcpu_id; +} guest_params_t; + +/***********************************/ +/* Misc Support */ +/***********************************/ +static int suspend_and_state(int (*suspend)(void*), void *data, + xc_interface *xch, int dom) +{ + xc_dominfo_t info; + + if ( !(*suspend)(data) ) + { + PERROR("Suspend request failed"); + return -1; + } + + if ( (xc_domain_getinfo(xch, dom, 1, &info) != 1) || info.domid != dom || + !info.shutdown || (info.shutdown_reason != SHUTDOWN_suspend) ) + { + PERROR("Domain is not in suspended state after suspend attempt"); + return -1; + } + + return 0; +} + +/***********************************/ +/* Guest Memory */ +/***********************************/ +static int save_guest_memory(xc_interface *xch, int io_fd, uint32_t dom, + struct save_callbacks *callbacks, + uint32_t max_iters, uint32_t max_factor, + guest_params_t *params) +{ + int live = !!(params->flags & XCFLAGS_LIVE); + int debug = !!(params->flags & XCFLAGS_DEBUG); + xen_pfn_t i; + char reportbuf[80]; + int iter = 0; + int last_iter = !live; + int total_dirty = 0; /* total number of dirty pages */ + int prev_iter_dirty = 0; + int count = 0; + char *page = NULL; + xen_pfn_t *busy_pages = 0; + int busy_pages_count = 0; + int busy_pages_max = 256; + DECLARE_HYPERCALL_BUFFER(unsigned long, to_send); + xen_pfn_t start = params->start_gpfn; + const xen_pfn_t end = params->max_gpfn; + const xen_pfn_t mem_size = end - start; + int rc = -1, frc; + + if ( debug ) + { + IPRINTF("Saving memory: start=0x%llx end=0x%llx!\n", + (unsigned long long)start, (unsigned long long)end); + } + + /* Note: to_send will be deallocated at the end of this function. */ + to_send = xc_hypercall_buffer_alloc_pages(xch, to_send, + NRPAGES(bitmap_size(mem_size))); + if ( !to_send ) + { + PERROR("Can't allocate to_send array!\n"); + goto out; + } + /* send all pages on first iter */ + memset(to_send, 0xff, bitmap_size(mem_size)); + + if ( live ) + { + if ( xc_shadow_control(xch, dom, XEN_DOMCTL_SHADOW_OP_ENABLE_LOGDIRTY, + NULL, 0, NULL, 0, NULL) < 0 ) + { + PERROR("Can't enable log-dirty mode!\n"); + goto out; + } + + max_iters = max_iters ? : DEF_MAX_ITERS; + max_factor = max_factor ? : DEF_MAX_FACTOR; + } + + for ( ; ; ) + { + int current_iter_dirty = 0; /* num of dirty pages in current iter */ + + iter++; + snprintf(reportbuf, sizeof(reportbuf), + "Saving memory: iter %d, last sent %u", iter, + prev_iter_dirty); + + xc_report_progress_start(xch, reportbuf, mem_size); + + if ( (iter > 1 && prev_iter_dirty < DEF_MIN_DIRTY_PER_ITER) || + (iter == max_iters) || + (total_dirty >= mem_size*max_factor) ) + { + last_iter = 1; + } + + if ( last_iter ) + { + if ( suspend_and_state(callbacks->suspend, callbacks->data, xch, + dom) ) + { + PERROR("Domain appears not to have suspended"); + goto out; + } + } + + if ( live && iter > 1 ) + { + frc = xc_shadow_control(xch, dom, XEN_DOMCTL_SHADOW_OP_CLEAN, + HYPERCALL_BUFFER(to_send), mem_size, + NULL, 0, NULL); + if ( frc != mem_size ) + { + PERROR("Can't peek bitmap of guest VM's dirty pages"); + goto out; + } + } + + /* busy_pages allocation */ + busy_pages = malloc(sizeof(xen_pfn_t) * busy_pages_max); + + for ( i = start; i < end; ++i ) + { + if ( test_bit(i - start, to_send) ) + { + page = xc_map_foreign_range(xch, dom, PAGE_SIZE, PROT_READ, i); + if ( !page ) + { + /* This page is mapped elsewhere, should be resent later */ + busy_pages[busy_pages_count] = i; + busy_pages_count++; + if ( busy_pages_count >= busy_pages_max ) + { + busy_pages_max += 256; + busy_pages = realloc(busy_pages, sizeof(xen_pfn_t) * + busy_pages_max); + } + continue; + } + + if ( write_exact(io_fd, &i, sizeof(i)) || + write_exact(io_fd, page, PAGE_SIZE) ) + { + PERROR("Write Error: guest memory gpfn and content"); + free(busy_pages); /* must do here */ + goto out; + } + + if ( (i % DEF_PROGRESS_RATE) == 0 ) + xc_report_progress_step(xch, i - start, mem_size); + + count++; + munmap(page, PAGE_SIZE); + current_iter_dirty++; + } + } + + while ( busy_pages_count ) + { + /* Send busy pages */ + busy_pages_count--; + i = busy_pages[busy_pages_count]; + if ( test_bit(i - start, to_send) ) + { + page = xc_map_foreign_range(xch, dom, PAGE_SIZE, PROT_READ, i); + if ( !page ) + { + IPRINTF("WARNING: 2nd attempt to save page failed at " + "pfn=0x%llx", (unsigned long long)i); + continue; + } + + if ( write_exact(io_fd, &i, sizeof(i)) || + write_exact(io_fd, page, PAGE_SIZE) ) + { + PERROR("Write Error: guest memory gpfn and content"); + free(busy_pages); /* must do here */ + goto out; + } + + count++; + munmap(page, PAGE_SIZE); + current_iter_dirty++; + } + } + free(busy_pages); + + xc_report_progress_step(xch, mem_size, mem_size); + + prev_iter_dirty = current_iter_dirty; + total_dirty += current_iter_dirty; + + if ( last_iter ) + { + if ( live ) + { + if ( xc_shadow_control(xch, dom, XEN_DOMCTL_SHADOW_OP_OFF, + NULL, 0, NULL, 0, NULL) < 0 ) + PERROR("Can't disable log-dirty mode"); + } + break; + } + } + + i = (xen_pfn_t) -1; /* end page marker */ + if ( write_exact(io_fd, &i, sizeof(i)) ) + { + PERROR("Write Error: end page mark"); + goto out; + } + /* flush last write and check for errno (must do, see fsync()). */ + if ( fsync(io_fd) && errno != EINVAL ) + { + PERROR("Write Error: flushing stream"); + goto out; + } + + rc = 0; + out: + /* clean up */ + if ( page ) + munmap(page, PAGE_SIZE); + if ( to_send ) + xc_hypercall_buffer_free_pages(xch, to_send, + NRPAGES(bitmap_size(mem_size))); + return rc; +} + +static int restore_guest_memory(xc_interface *xch, int io_fd, uint32_t dom, + guest_params_t *params) +{ + xen_pfn_t gpfn; + xen_pfn_t start = params->start_gpfn; + xen_pfn_t end = params->max_gpfn; + int count = 0; + char *page = NULL; + int rc = -1; + + /* TODO: batch operation */ + for ( gpfn = start; gpfn < end; ++gpfn ) + { + if ( xc_domain_populate_physmap_exact(xch, dom, 1, 0, 0, &gpfn) ) + { + PERROR("Can't populate guest physical memory"); + goto out; + } + } + + while ( 1 ) + { + if ( read_exact(io_fd, &gpfn, sizeof(gpfn)) ) + { + PERROR("Read Error: guest memory gpfn (count=%d)", count); + goto out; + } + + /* end of guest pages */ + if ( gpfn == (xen_pfn_t) -1 ) + break; + + if ( gpfn < start || gpfn >= end ) + { + PERROR("gpfn 0x%llx doesn't belong to guest RAM addr space " + "[0x%llx, 0x%llx]", gpfn, start, end); + goto out; + } + + if ( !(page = xc_map_foreign_range(xch, dom, PAGE_SIZE, + PROT_READ | PROT_WRITE, gpfn)) ) + { + PERROR("Can't map guest page to dom0 (gpfn=0x%llx)", gpfn); + goto out; + } + + if ( read_exact(io_fd, page, PAGE_SIZE) ) + { + PERROR("Read Error: guest memory content (gpfn=0x%llx)", gpfn); + goto out; + } + + munmap(page, PAGE_SIZE); + count++; + } + + rc = 0; + out: + if ( page ) + munmap(page, PAGE_SIZE); + + return rc; +} + +/***********************************/ +/* HVM Context */ +/***********************************/ +static int save_guest_hvm_ctxt(xc_interface *xch, int io_fd, uint32_t dom) +{ + uint32_t ctxt_size = 0; + uint8_t *hvm_ctxt = NULL; + uint32_t rec_size; + int rc = -1; + + /* Figure out HVM context size and allocate a local buffer */ + if ( (ctxt_size = xc_domain_hvm_getcontext(xch, dom, 0, 0)) == -1 ) + { + PERROR("Can't get HVM context size"); + goto out; + } + + if ( (hvm_ctxt = malloc(ctxt_size)) == NULL ) + { + PERROR("Can't allocate memory for HVM context"); + goto out; + } + + /* Retrieve HVM context from Xen and save it */ + if ( (rec_size = xc_domain_hvm_getcontext(xch, dom, hvm_ctxt, + ctxt_size)) == -1 ) + { + PERROR("Can't receive HVM context"); + goto out; + } + + if ( write_exact(io_fd, &rec_size, sizeof(uint32_t)) || + write_exact(io_fd, hvm_ctxt, rec_size) ) + { + PERROR("Write Error: HVM context"); + goto out; + } + + rc = 0; +out: + free (hvm_ctxt); + return rc; +} + +static int restore_guest_hvm_ctxt(xc_interface *xch, int io_fd, uint32_t dom) +{ + uint32_t rec_size; + uint32_t ctxt_size = 0; + uint8_t *hvm_ctxt = NULL; + int frc = 0; + int rc = -1; + + if ( read_exact(io_fd, &rec_size, sizeof(uint32_t)) ) + { + PERROR("Read Error: HVM context size"); + goto out; + } + + if ( !rec_size ) + { + PERROR("No HVM context"); + goto out; + } + + if ( (ctxt_size = xc_domain_hvm_getcontext(xch, dom, 0, 0)) != rec_size ) + { + PERROR("Stored HVM context size isn't matched with current system"); + goto out; + } + + if ( !(hvm_ctxt = malloc(ctxt_size)) ) + { + PERROR("Can't allocate memory"); + goto out; + } + + if ( read_exact(io_fd, hvm_ctxt, ctxt_size) ) + { + PERROR("Read Error: HVM context"); + goto out; + } + + if ( (frc = xc_domain_hvm_setcontext(xch, dom, hvm_ctxt, ctxt_size)) ) + { + PERROR("Can't set HVM context"); + goto out; + } + + rc = 0; +out: + free (hvm_ctxt); + return rc; +} + +/***********************************/ +/* Guest Parameters */ +/***********************************/ +static int save_guest_params(xc_interface *xch, int io_fd, uint32_t dom, + uint32_t flags, guest_params_t *params) +{ + xc_dominfo_t dom_info; + + /* retrieve domain info */ + if ( (xc_domain_getinfo(xch, dom, 1, &dom_info ) != 1) || + dom_info.domid != dom) + { + PERROR("Can't get domain info for dom %d", dom); + return -1; + } + + /* start and max of gpfn */ + params->start_gpfn = (GUEST_RAM_BASE >> PAGE_SHIFT); + params->max_gpfn = (GUEST_RAM_BASE >> PAGE_SHIFT) + dom_info.nr_pages; + + /* console pfn */ + if ( xc_get_hvm_param(xch, dom, HVM_PARAM_CONSOLE_PFN, + ¶ms->console_pfn) ) + { + PERROR("Can't get console gpfn"); + return -1; + } + + if ( xc_get_hvm_param(xch, dom, HVM_PARAM_STORE_PFN, ¶ms->store_pfn) ) + { + PERROR("Can't get store gpfn"); + return -1; + } + + params->max_vcpu_id = dom_info.max_vcpu_id; + params->flags = flags; + + if ( write_exact(io_fd, params, sizeof(*params)) ) + { + PERROR("Write Error: guest params"); + return -1; + } + + return 0; +} + +static int restore_guest_params(xc_interface *xch, int io_fd, uint32_t dom, + guest_params_t *params) +{ + xen_pfn_t nr_pfns; + unsigned int maxmemkb; + + if ( read_exact(io_fd, params, sizeof(*params)) ) + { + PERROR("Read Error: guest params"); + return -1; + } + + nr_pfns = params->max_gpfn - params->start_gpfn; + maxmemkb = (unsigned int) nr_pfns << (PAGE_SHIFT - 10); + + if ( xc_domain_setmaxmem(xch, dom, maxmemkb) ) + { + PERROR("Can't set memory map"); + return -1; + } + + if ( xc_domain_max_vcpus(xch, dom, params->max_vcpu_id + 1) ) + { + PERROR("Can't set max vcpu number for domain"); + return -1; + } + + return 0; +} + +/* Configure guest parameters after restore */ +static int set_guest_params(xc_interface *xch, int io_fd, uint32_t dom, + guest_params_t *params, + unsigned int console_evtchn, domid_t console_domid, + unsigned int store_evtchn, domid_t store_domid) +{ + int rc = 0; + + if ( (rc = xc_clear_domain_page(xch, dom, params->console_pfn)) ) + { + PERROR("Can't clear console page"); + return rc; + } + + if ( (rc = xc_clear_domain_page(xch, dom, params->store_pfn)) ) + { + PERROR("Can't clear xenstore page"); + return rc; + } + + if ( (rc = xc_dom_gnttab_hvm_seed(xch, dom, params->console_pfn, + params->store_pfn, console_domid, + store_domid)) ) + { + PERROR("Can't grant console and xenstore pages"); + return rc; + } + + if ( (rc = xc_set_hvm_param(xch, dom, HVM_PARAM_CONSOLE_PFN, + params->console_pfn)) ) + { + PERROR("Can't set console gpfn"); + return rc; + } + + if ( (rc = xc_set_hvm_param(xch, dom, HVM_PARAM_STORE_PFN, + params->store_pfn)) ) + { + PERROR("Can't set xenstore gpfn"); + return rc; + } + + if ( (rc = xc_set_hvm_param(xch, dom, HVM_PARAM_CONSOLE_EVTCHN, + console_evtchn)) ) + { + PERROR("Can't set console event channel"); + return rc; + } + + if ( (rc = xc_set_hvm_param(xch, dom, HVM_PARAM_STORE_EVTCHN, + store_evtchn)) ) + { + PERROR("Can't set xenstore event channel"); + return rc; + } + + return 0; +} + +/***********************************/ +/* Main Entries */ +/***********************************/ +int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, + uint32_t max_iters, uint32_t max_factor, uint32_t flags, + struct save_callbacks *callbacks, int hvm, + unsigned long vm_generationid_addr) +{ + guest_params_t params; + + if ( save_guest_params(xch, io_fd, dom, flags, ¶ms) ) + { + PERROR("Can't save guest params"); + return -1; + } + + if ( save_guest_memory(xch, io_fd, dom, callbacks, max_iters, max_factor, + ¶ms) ) + { + PERROR("Can't save guest memory"); + return -1; + } + + if ( save_guest_hvm_ctxt(xch, io_fd, dom) ) + { + PERROR("Cant' save guest HVM context"); + return -1; + } + + return 0; +} + +int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom, + unsigned int store_evtchn, unsigned long *store_gpfn, + domid_t store_domid, unsigned int console_evtchn, + unsigned long *console_gpfn, domid_t console_domid, + unsigned int hvm, unsigned int pae, int superpages, + int no_incr_generationid, int checkpointed_stream, + unsigned long *vm_generationid_addr, + struct restore_callbacks *callbacks) +{ + guest_params_t params; + + if ( restore_guest_params(xch, io_fd, dom, ¶ms) ) + { + PERROR("Can't restore guest params"); + return -1; + } + + if ( restore_guest_memory(xch, io_fd, dom, ¶ms) ) + { + PERROR("Can't restore memory"); + return -1; + } + + if ( set_guest_params(xch, io_fd, dom, ¶ms, console_evtchn, + console_domid, store_evtchn, store_domid) ) + { + PERROR("Can't setup guest params"); + return -1; + } + + /* setup console and store PFNs to caller */ + *console_gpfn = params.console_pfn; + *store_gpfn = params.store_pfn; + + /* restore HVM context of guest VM */ + if ( restore_guest_hvm_ctxt(xch, io_fd, dom) ) + { + PERROR("Can't restore HVM context"); + return -1; + } + + return 0; +} + +/* + * Local variables: + * mode: C + * c-set-style: "BSD" + * c-basic-offset: 4 + * tab-width: 4 + * indent-tabs-mode: nil + * End: + */ diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c index 60ac51a..d9e50d2 100644 --- a/tools/libxc/xc_dom_arm.c +++ b/tools/libxc/xc_dom_arm.c @@ -348,7 +348,9 @@ int arch_setup_meminit(struct xc_dom_image *dom) modbase += dtb_size; } - return 0; + return xc_domain_setmaxmem(dom->xch, dom->guest_domid, + (dom->total_pages + NR_MAGIC_PAGES) + << (PAGE_SHIFT - 10)); } int arch_setup_bootearly(struct xc_dom_image *dom) diff --git a/tools/libxc/xc_resume.c b/tools/libxc/xc_resume.c index 18b4818..6d05d4e 100644 --- a/tools/libxc/xc_resume.c +++ b/tools/libxc/xc_resume.c @@ -18,11 +18,15 @@ #include "xg_private.h" #include "xg_save_restore.h" -#if defined(__i386__) || defined(__x86_64__) +#if defined(__i386__) || defined(__x86_64__) || defined(__arm__) || \ + defined(__aarch64__) +/* Only required for x86 */ +#if defined(__i386__) || defined(__x86_64__) #include #include #include +#endif static int modify_returncode(xc_interface *xch, uint32_t domid) { @@ -33,7 +37,7 @@ static int modify_returncode(xc_interface *xch, uint32_t domid) struct domain_info_context *dinfo = &_dinfo; int rc; - if ( xc_domain_getinfo(xch, domid, 1, &info) != 1 ) + if ( xc_domain_getinfo(xch, domid, 1, &info) != 1 || info.domid != domid ) { PERROR("Could not get domain info"); return -1; @@ -65,23 +69,23 @@ static int modify_returncode(xc_interface *xch, uint32_t domid) if ( (rc = xc_vcpu_getcontext(xch, domid, 0, &ctxt)) != 0 ) return rc; +#if defined(__i386__) || defined(__x86_64__) SET_FIELD(&ctxt, user_regs.eax, 1); +#elif defined(__arm__) || defined(__aarch64__) + ctxt.c.user_regs.r0_usr = 1; +#endif if ( (rc = xc_vcpu_setcontext(xch, domid, 0, &ctxt)) != 0 ) return rc; return 0; } - -#else - +#else /* any architecture other than x86 or ARM */ static int modify_returncode(xc_interface *xch, uint32_t domid) { return 0; - } - -#endif +#endif static int xc_domain_resume_cooperative(xc_interface *xch, uint32_t domid) { diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 84f9c0e..ce22aa0 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -448,9 +448,6 @@ * - libxl_domain_resume * - libxl_domain_remus_start */ -#if defined(__arm__) || defined(__aarch64__) -#define LIBXL_HAVE_NO_SUSPEND_RESUME 1 -#endif /* * LIBXL_HAVE_DEVICE_PCI_SEIZE diff --git a/tools/misc/Makefile b/tools/misc/Makefile index 69b1817..f4ea7ab 100644 --- a/tools/misc/Makefile +++ b/tools/misc/Makefile @@ -11,7 +11,7 @@ HDRS = $(wildcard *.h) TARGETS-y := xenperf xenpm xen-tmem-list-parse gtraceview gtracestat xenlockprof xenwatchdogd xencov TARGETS-$(CONFIG_X86) += xen-detect xen-hvmctx xen-hvmcrash xen-lowmemd xen-mfndump -TARGETS-$(CONFIG_MIGRATE) += xen-hptool +TARGETS-$(CONFIG_X86) += xen-hptool TARGETS := $(TARGETS-y) SUBDIRS := $(SUBDIRS-y) @@ -23,7 +23,7 @@ INSTALL_BIN := $(INSTALL_BIN-y) INSTALL_SBIN-y := xen-bugtool xen-python-path xenperf xenpm xen-tmem-list-parse gtraceview \ gtracestat xenlockprof xenwatchdogd xen-ringwatch xencov INSTALL_SBIN-$(CONFIG_X86) += xen-hvmctx xen-hvmcrash xen-lowmemd xen-mfndump -INSTALL_SBIN-$(CONFIG_MIGRATE) += xen-hptool +INSTALL_SBIN-$(CONFIG_X86) += xen-hptool INSTALL_SBIN := $(INSTALL_SBIN-y) INSTALL_PRIVBIN-y := xenpvnetboot