From patchwork Tue Jun 10 09:57:54 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 31617 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ie0-f199.google.com (mail-ie0-f199.google.com [209.85.223.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id B5AF12054B for ; Tue, 10 Jun 2014 09:59:57 +0000 (UTC) Received: by mail-ie0-f199.google.com with SMTP id rd18sf49845iec.10 for ; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=WYw98LeX+A6tiwck98wRXiIdblJVkQVyibZcSr6adCE=; b=DF2aWlkDHv144SdZubEFmva+0tmZWMaykU7fEP5pILrY//BgIH/sOFC3Uq+XO9sUS8 RLsMsZ6nq7Qmk50S4KyuKlx0m6N3d8BWVbq/7cN7Q4r5c+tovrFau1TG+CpYXHwa8Rtz oqgJvK6PCpHfSCo43kg6+blVquDqhiKpKiSVd/deYovFZ7M0MOE5RiCis3JurANPnAbN F5cB4gBGL9cF1ukPobkDhF8qyQ38uuRKj7avo3NMIbQUJ0G3bl93tlqIVix6Asi24AMa 12tVTC2JN78bI1J9d+YzILvZ6tmqdirpHw8Dy2E5DkMdPWtt2zD8FGH/VaGbIuZHM1vg CdKQ== X-Gm-Message-State: ALoCoQnO4BXwLu1pKuv/TdtjIUvqk/FVsr71or4YfzEJB6wD2CER//acuTol7u73oD+3uRpk0pKM X-Received: by 10.182.120.129 with SMTP id lc1mr15132273obb.21.1402394397329; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.93.131 with SMTP id d3ls1985168qge.52.gmail; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) X-Received: by 10.221.40.193 with SMTP id tr1mr31421526vcb.31.1402394397158; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) Received: from mail-vc0-f178.google.com (mail-vc0-f178.google.com [209.85.220.178]) by mx.google.com with ESMTPS id kh9si3326894vdb.21.2014.06.10.02.59.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 10 Jun 2014 02:59:57 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) client-ip=209.85.220.178; Received: by mail-vc0-f178.google.com with SMTP id ij19so3958671vcb.9 for ; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) X-Received: by 10.58.208.228 with SMTP id mh4mr17408780vec.22.1402394397074; Tue, 10 Jun 2014 02:59:57 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.221.54.6 with SMTP id vs6csp212645vcb; Tue, 10 Jun 2014 02:59:56 -0700 (PDT) X-Received: by 10.58.29.106 with SMTP id j10mr32078331veh.31.1402394396752; Tue, 10 Jun 2014 02:59:56 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id zc15si12868229vdb.79.2014.06.10.02.59.56 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Tue, 10 Jun 2014 02:59:56 -0700 (PDT) Received-SPF: none (google.com: xen-devel-bounces@lists.xen.org does not designate permitted sender hosts) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WuIoi-0007jR-Gt; Tue, 10 Jun 2014 09:58:08 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WuIog-0007h3-FW for xen-devel@lists.xen.org; Tue, 10 Jun 2014 09:58:06 +0000 Received: from [85.158.139.211:27864] by server-1.bemta-5.messagelabs.com id 50/43-01663-DA6D6935; Tue, 10 Jun 2014 09:58:05 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-5.tower-206.messagelabs.com!1402394280!9063329!3 X-Originating-IP: [66.165.176.89] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni44OSA9PiAyMDMwMDc=\n X-StarScan-Received: X-StarScan-Version: 6.11.3; banners=-,-,- X-VirusChecked: Checked Received: (qmail 6861 invoked from network); 10 Jun 2014 09:58:03 -0000 Received: from smtp.citrix.com (HELO SMTP.CITRIX.COM) (66.165.176.89) by server-5.tower-206.messagelabs.com with RC4-SHA encrypted SMTP; 10 Jun 2014 09:58:03 -0000 X-IronPort-AV: E=Sophos; i="4.98,1008,1392163200"; d="scan'208"; a="141868038" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO01.CITRIX.COM with ESMTP; 10 Jun 2014 09:58:00 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.3.181.6; Tue, 10 Jun 2014 05:57:59 -0400 Received: from marilith-n13-p0.uk.xensource.com ([10.80.229.115] helo=marilith-n13.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WuIoZ-000207-4e; Tue, 10 Jun 2014 09:57:59 +0000 From: Ian Campbell To: Date: Tue, 10 Jun 2014 10:57:54 +0100 Message-ID: <1402394278-9850-2-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1402394127.29980.52.camel@kazak.uk.xensource.com> References: <1402394127.29980.52.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH 2/6] tools: arm: allocate superpages to guests. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.220.178 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Tries to allocate as many large (1G or 2M) pages as it can to the guest and tries to align to the next larger size on each attempt so that we can attempt to allocate even larger pages on the next iteration (this is currently only exercised via a compile time debug option, which is left in place under a #if 0). Since ARM page tables are consistent at each level there is a common helper function which tries to allocate a levels worth of pages. The exception to this consistency is level 0 which does not support table mappings (0.5TB superpages!). Signed-off-by: Ian Campbell --- tools/libxc/xc_dom_arm.c | 103 +++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 92 insertions(+), 11 deletions(-) diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c index 75f8363..da68ec3 100644 --- a/tools/libxc/xc_dom_arm.c +++ b/tools/libxc/xc_dom_arm.c @@ -30,6 +30,13 @@ #define CONSOLE_PFN_OFFSET 0 #define XENSTORE_PFN_OFFSET 1 +#define LPAE_SHIFT 9 + +#define PFN_4K_SHIFT (0) +#define PFN_2M_SHIFT (PFN_4K_SHIFT+LPAE_SHIFT) +#define PFN_1G_SHIFT (PFN_2M_SHIFT+LPAE_SHIFT) +#define PFN_512G_SHIFT (PFN_1G_SHIFT+LPAE_SHIFT) + /* get guest IO ABI protocol */ const char *xc_domain_get_native_protocol(xc_interface *xch, uint32_t domid) @@ -249,11 +256,57 @@ static int set_mode(xc_interface *xch, domid_t domid, char *guest_type) return rc; } +#define min_t(type,x,y) \ + ({ type __x = (x); type __y = (y); __x < __y ? __x: __y; }) + +/* >0: success, *nr_pfns set to number actually populated + * 0: didn't try with this pfn shift (e.g. misaligned base etc) + * <0: ERROR + */ +static int populate_one_size(struct xc_dom_image *dom, int pfn_shift, + xen_pfn_t base_pfn, xen_pfn_t *nr_pfns) +{ + uint64_t mask = ((uint64_t)1<<(pfn_shift))-1; + uint64_t next_mask = ((uint64_t)1<<(LPAE_SHIFT))-1; + int nr, i, count = min_t(int, 1024*1024, *nr_pfns >> pfn_shift); + xen_pfn_t extents[count]; + + /* No level zero super pages with current hardware */ + if ( pfn_shift == PFN_512G_SHIFT ) + return 0; + + /* Nothing to allocate */ + if ( !count ) + return 0; + + /* base is misaligned for this level */ + if ( mask & base_pfn ) + return 0; + + /* align to the end of a super page at this level */ + if ( count & next_mask ) + count &= next_mask; + + for ( i = 0 ; i < count ; i ++ ) + extents[i] = base_pfn + (i<xch, dom->guest_domid, count, + pfn_shift, 0, extents); + if ( nr <= 0 ) return nr; + DOMPRINTF("%s: populated %#x/%#x entries with shift %d", + __FUNCTION__, nr, count, pfn_shift); + + *nr_pfns = nr << pfn_shift; + + return 1; +} + static int populate_guest_memory(struct xc_dom_image *dom, xen_pfn_t base_pfn, xen_pfn_t nr_pfns) { - int rc; + int rc = 0; xen_pfn_t allocsz, pfn; + int debug_iters = 0; DOMPRINTF("%s: populating RAM @ %016"PRIx64"-%016"PRIx64" (%"PRId64"MB)", __FUNCTION__, @@ -261,21 +314,49 @@ static int populate_guest_memory(struct xc_dom_image *dom, (uint64_t)(base_pfn + nr_pfns) << XC_PAGE_SHIFT, (uint64_t)nr_pfns >> (20-XC_PAGE_SHIFT)); - for ( pfn = 0; pfn < nr_pfns; pfn++ ) - dom->p2m_host[pfn] = base_pfn + pfn; - - for ( pfn = rc = allocsz = 0; (pfn < nr_pfns) && !rc; pfn += allocsz ) + for ( pfn = 0; pfn < nr_pfns; pfn += allocsz ) { allocsz = nr_pfns - pfn; - if ( allocsz > 1024*1024 ) - allocsz = 1024*1024; - rc = xc_domain_populate_physmap_exact( - dom->xch, dom->guest_domid, allocsz, - 0, 0, &dom->p2m_host[pfn]); + if ( debug_iters++ > 4 ) + abort(); +#if 0 /* Enable this to exercise/debug the code which tries to realign + * to a superpage boundary, by misaligning at the start. */ + if ( pfn == 0 ) + { + allocsz = 1; + rc = populate_one_size(dom, PFN_4K_SHIFT, base_pfn + pfn, &allocsz); + if (rc < 0) break; + if (rc > 0) continue; + /* Failed to allocate a single page? */ + break; + } +#endif + + rc = populate_one_size(dom, PFN_512G_SHIFT, base_pfn + pfn, &allocsz); + if (rc < 0) break; + if (rc > 0) continue; + + rc = populate_one_size(dom, PFN_1G_SHIFT, base_pfn + pfn, &allocsz); + if (rc < 0) break; + if (rc > 0) continue; + + rc = populate_one_size(dom, PFN_2M_SHIFT, base_pfn + pfn, &allocsz); + if (rc < 0) break; + if (rc > 0) continue; + + rc = populate_one_size(dom, PFN_4K_SHIFT, base_pfn + pfn, &allocsz); + if (rc < 0) break; + + assert(rc > 0); /* Must have tried to allocate some 4k pages! */ } - return rc; + DOMPRINTF("%s: rc=%d", __FUNCTION__, rc); + + for ( pfn = 0; pfn < nr_pfns; pfn++ ) + dom->p2m_host[pfn] = base_pfn + pfn; + + return rc < 0 ? rc : 0; } int arch_setup_meminit(struct xc_dom_image *dom)