From patchwork Thu May 28 18:49:10 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christoffer Dall X-Patchwork-Id: 49120 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-la0-f69.google.com (mail-la0-f69.google.com [209.85.215.69]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id 9614422B3A for ; Thu, 28 May 2015 18:50:50 +0000 (UTC) Received: by laat2 with SMTP id t2sf13535613laa.2 for ; Thu, 28 May 2015 11:50:49 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:subject:date:message-id:cc :precedence:list-id:list-unsubscribe:list-archive:list-post :list-help:list-subscribe:mime-version:content-type :content-transfer-encoding:sender:errors-to:x-original-sender :x-original-authentication-results:mailing-list; bh=p2N5y0KVcCsv0+DBLs+l6DgSwT3r2JSXTKZYuylBYbE=; b=RMR2BiFTmzUUNGQ/9dlNJiYXal2rDAwu0ZaWSbzUhwii7bPKj9EbwXlT/NZpvuAcwv /Fdw5Hm2DLx/s/9LuTj4PVrbD4cJxsv/lomxRRE3pircv3O9yM9NIXKqUOkyWfVD/sfp M42Vwj7yuiTxuf4JoIuH1RotTLsDZoMQEyRsCjDjP5fAg+RCNzjdgeOie6inMZTCL9xg EewXjxO6Atvs67kEUv6XKX2SMS9dlPG7QXhGo0gW74nbK63xTvMwkRL/4nSwn92da9UU mge9H3RxfqTouIUVmYza/3yOLAOGvJl2aXRRqtNeDMtq4S5dLTxJjQM8Uo1ndUw6XZ/e BKpA== X-Gm-Message-State: ALoCoQk/lPrS1JBj9TRHxghx++XiavQ7TL/AO5JRfGDpOhThL19h5gL/kYseZ0dqKE/VUvvkH1In X-Received: by 10.180.216.12 with SMTP id om12mr8905103wic.1.1432839049577; Thu, 28 May 2015 11:50:49 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.43.36 with SMTP id t4ls250814lal.99.gmail; Thu, 28 May 2015 11:50:49 -0700 (PDT) X-Received: by 10.152.9.201 with SMTP id c9mr4285204lab.104.1432839049414; Thu, 28 May 2015 11:50:49 -0700 (PDT) Received: from mail-la0-f44.google.com (mail-la0-f44.google.com. [209.85.215.44]) by mx.google.com with ESMTPS id wx3si2583116lbb.142.2015.05.28.11.50.49 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 May 2015 11:50:49 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.44 as permitted sender) client-ip=209.85.215.44; Received: by labpy14 with SMTP id py14so27329028lab.0 for ; Thu, 28 May 2015 11:50:49 -0700 (PDT) X-Received: by 10.112.204.72 with SMTP id kw8mr4243292lbc.88.1432839049161; Thu, 28 May 2015 11:50:49 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.108.230 with SMTP id hn6csp1377008lbb; Thu, 28 May 2015 11:50:48 -0700 (PDT) X-Received: by 10.70.102.11 with SMTP id fk11mr8103723pdb.144.1432839047374; Thu, 28 May 2015 11:50:47 -0700 (PDT) Received: from bombadil.infradead.org (bombadil.infradead.org. [2001:1868:205::9]) by mx.google.com with ESMTPS id zb14si4808736pac.209.2015.05.28.11.50.46 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 28 May 2015 11:50:47 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 2001:1868:205::9 as permitted sender) client-ip=2001:1868:205::9; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.80.1 #2 (Red Hat Linux)) id 1Yy2re-00043z-T1; Thu, 28 May 2015 18:49:10 +0000 Received: from mail-la0-f47.google.com ([209.85.215.47]) by bombadil.infradead.org with esmtps (Exim 4.80.1 #2 (Red Hat Linux)) id 1Yy2rY-0003sT-4I for linux-arm-kernel@lists.infradead.org; Thu, 28 May 2015 18:49:05 +0000 Received: by lagv1 with SMTP id v1so39012176lag.3 for ; Thu, 28 May 2015 11:48:41 -0700 (PDT) X-Received: by 10.112.29.36 with SMTP id g4mr4327509lbh.56.1432838921268; Thu, 28 May 2015 11:48:41 -0700 (PDT) Received: from localhost.localdomain (188-178-240-98-static.dk.customer.tdc.net. [188.178.240.98]) by mx.google.com with ESMTPSA id yq5sm756117lab.32.2015.05.28.11.48.39 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 28 May 2015 11:48:40 -0700 (PDT) From: Christoffer Dall To: kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org Subject: [PATCH v2] arm/arm64: KVM: Properly account for guest CPU time Date: Thu, 28 May 2015 20:49:10 +0200 Message-Id: <1432838950-28774-1-git-send-email-christoffer.dall@linaro.org> X-Mailer: git-send-email 2.1.2.330.g565301e.dirty X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20150528_114904_487227_8298F960 X-CRM114-Status: GOOD ( 15.85 ) X-Spam-Score: -0.7 (/) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-0.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [209.85.215.47 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.215.47 listed in wl.mailspike.net] -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders Cc: Marc Zyngier , borntraeger@de.ibm.com, Christoffer Dall , Paolo Bonzini X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: christoffer.dall@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.215.44 as permitted sender) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Until now we have been calling kvm_guest_exit after re-enabling interrupts when we come back from the guest, but this has the unfortunate effect that CPU time accounting done in the context of timer interrupts occurring while the guest is running doesn't properly notice that the time since the last tick was spent in the guest. Inspired by the comment in the x86 code, move the kvm_guest_exit() call below the local_irq_enable() call and change __kvm_guest_exit() to kvm_guest_exit(), because we are now calling this function with interrupts enabled. We have to now explicitly disable preemption and not enable preemption before we've called kvm_guest_exit(), since otherwise we could be preempted and everything happening before we eventually get scheduled again would be accounted for as guest time. At the same time, move the trace_kvm_exit() call outside of the atomic section, since there is no reason for us to do that with interrupts disabled. Signed-off-by: Christoffer Dall --- This patch is based on kvm/queue, because it has the kvm_guest_enter/exit rework recently posted by Christian Borntraeger. I hope I got the logic of this right, there were 2 slightly worrying facts about this: First, we now enable and disable and enable interrupts on each exit path, but I couldn't see any performance overhead on hackbench - yes the only benchmark we care about. Second, looking at the ppc and mips code, they seem to also call kvm_guest_exit() before enabling interrupts, so I don't understand how guest CPU time accounting works on those architectures. Changes since v1: - Tweak comment and commit text based on Marc's feedback. - Explicitly disable preemption and enable it only after kvm_guest_exit(). arch/arm/kvm/arm.c | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c index e41cb11..fe8028d 100644 --- a/arch/arm/kvm/arm.c +++ b/arch/arm/kvm/arm.c @@ -532,6 +532,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) kvm_vgic_flush_hwstate(vcpu); kvm_timer_flush_hwstate(vcpu); + preempt_disable(); local_irq_disable(); /* @@ -544,6 +545,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) if (ret <= 0 || need_new_vmid_gen(vcpu->kvm)) { local_irq_enable(); + preempt_enable(); kvm_timer_sync_hwstate(vcpu); kvm_vgic_sync_hwstate(vcpu); continue; @@ -559,8 +561,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) ret = kvm_call_hyp(__kvm_vcpu_run, vcpu); vcpu->mode = OUTSIDE_GUEST_MODE; - __kvm_guest_exit(); - trace_kvm_exit(kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); + /* + * Back from guest + *************************************************************/ + /* * We may have taken a host interrupt in HYP mode (ie * while executing the guest). This interrupt is still @@ -574,8 +578,17 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run) local_irq_enable(); /* - * Back from guest - *************************************************************/ + * We do local_irq_enable() before calling kvm_guest_exit() so + * that if a timer interrupt hits while running the guest we + * account that tick as being spent in the guest. We enable + * preemption after calling kvm_guest_exit() so that if we get + * preempted we make sure ticks after that is not counted as + * guest time. + */ + kvm_guest_exit(); + trace_kvm_exit(kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); + preempt_enable(); + kvm_timer_sync_hwstate(vcpu); kvm_vgic_sync_hwstate(vcpu);