From patchwork Wed Oct 31 12:21:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 149790 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp6726263ljp; Wed, 31 Oct 2018 05:27:26 -0700 (PDT) X-Google-Smtp-Source: AJdET5do8A3QrERtNuGbxUbvRv1LkMxZM55lqPxwMsMtT08ZXGz2SZyDBspspQdbn6fJnVdcYJgc X-Received: by 2002:a0c:a585:: with SMTP id z5mr2459146qvz.52.1540988846232; Wed, 31 Oct 2018 05:27:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540988846; cv=none; d=google.com; s=arc-20160816; b=QTBIJjafW9/tY9do/F8wBRmzQDxNfHu/ik3xXXM9lL+gyaWR+IsZnPJ8dZ59BC8Iv0 qnrUp9TvHb0I/lIEAcBiO7Gmx/Yyo5SrMELX97scXdwP36ny75HRTLKLxi35MyQeZ8KL eBSFmoB8puDKcCIgkUeHOQUytDfvVcrTsSCkrsD8c1pOIMM79JPnWnrairFl1XL01tp9 gIMeRU2efpDu3ry8zp2A03VTstzI9tqTYPYZKFC4embWtXD2w5mG2HlWId1sJknX0p1p hFyk9kuiWCV0AYk8q84Y06gA9QvcqlQHlwnV29afnOke80iylt3PYjsmGtfRXZNV9W6u Lv0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=q/yjo8z+g7lwII7MAY80uwlsqNxP5aVjMUM/ksqnyvc=; b=lpI5dAnfoQ/y0h1c/kWedOrjtQRtAGk92hC18l0SvSOjrSJV1A9InwMltUyCcZTF+w gSfkXNu68hiEpv/vyd9h4MzeCYWv1BqiTcWfs6n/0qsPMHcZP4QTyU6VbTf+OkTMGgsq 2fekjotxnUDAVVZQzzIM8vYPZjFF+QLnKum5L9Zn50CQGn1P10CmaTN8jnz9yEhro3B5 yFZg7mI3X8DKUIQdXuhWcdfRWi2kozVC+aB8OUs44aaOQ8nv1VhM+EB9ej1jbxKo5i+E qmNkHqvM9kNeC5wzv6p/NTOSknm0ltjIvpyURzZ7SkpnpYVSxg2KNNTIO3HLAvtNDqnC yALQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XbtsLZBb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id l25-v6si7253184qtj.77.2018.10.31.05.27.26 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 31 Oct 2018 05:27:26 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=XbtsLZBb; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59164 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gHpan-0005xs-MI for patch@linaro.org; Wed, 31 Oct 2018 08:27:25 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48918) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gHpVg-0000pd-8L for qemu-devel@nongnu.org; Wed, 31 Oct 2018 08:22:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gHpVd-0001rE-1j for qemu-devel@nongnu.org; Wed, 31 Oct 2018 08:22:06 -0400 Received: from mail-wm1-x334.google.com ([2a00:1450:4864:20::334]:53756) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gHpVY-0001Ei-IV for qemu-devel@nongnu.org; Wed, 31 Oct 2018 08:22:02 -0400 Received: by mail-wm1-x334.google.com with SMTP id v24-v6so5087175wmh.3 for ; Wed, 31 Oct 2018 05:21:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=q/yjo8z+g7lwII7MAY80uwlsqNxP5aVjMUM/ksqnyvc=; b=XbtsLZBbUDXxTOZGWz73kzMKoAJWHQS6KGA01KvPELR0F+kDJNmogqkZLQdwvinWwC Kb5181SqUxBliOMyzjH5Ti/zyHQRgRcvjtLn9HWKBx01ibsoKhFsBVrMMX5XyZ0SxsiK JcpWF7gjgcKWe8AqWHZKOZZwTVHiJWN5x9JLE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=q/yjo8z+g7lwII7MAY80uwlsqNxP5aVjMUM/ksqnyvc=; b=t5+30K0SbpLsYGvCtD+7MkPoK0bNthB80tX/ew3cTx+xpMsAZsDvoMnvzEPrteBLMt krchYZ0LnPyVnHqfACc3oFgkZyPJ0dPXA80n/PhAyfe6xBdsf43rXP0k9MJijeU+rN+M 5oH/LxeFqVdXhGLmhJE4e2pJYckSWZlW78bV4yqv8Sj1eHF42wq2kLoJ6VjoJOtVMmV0 NYYMLGXiOshdWJJaI2QMjZ3GdPMkHRW1SJrIf8iHPECT/x/Aytf9iR1BKorvfrvfYWNo qjTgwzarh109kr3QBXtKH3JOxzohuLidwmOUU83JoD1wwRgX4ObI0YGr4jtEXNHA9j77 21Ew== X-Gm-Message-State: AGRZ1gILyRhAMKGcaordz25Ac0HDLKQQga7/xlXCJc8j6rueAd1zTgte /zu+wwW8IGClZjIdoUaLWtSQ4Kzseyc= X-Received: by 2002:a1c:1d12:: with SMTP id d18-v6mr2115254wmd.31.1540988488611; Wed, 31 Oct 2018 05:21:28 -0700 (PDT) Received: from cloudburst.twiddle.net.lan (79-69-241-110.dynamic.dsl.as9105.com. [79.69.241.110]) by smtp.gmail.com with ESMTPSA id v2-v6sm13450362wru.17.2018.10.31.05.21.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 31 Oct 2018 05:21:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 31 Oct 2018 12:21:18 +0000 Message-Id: <20181031122119.1669-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181031122119.1669-1-richard.henderson@linaro.org> References: <20181031122119.1669-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::334 Subject: [Qemu-devel] [PULL 09/10] cputlb: Filter flushes on already clean tlbs X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Especially for guests with large numbers of tlbs, like ARM or PPC, we may well not use all of them in between flush operations. Remember which tlbs have been used since the last flush, and avoid any useless flushing. Tested-by: Emilio G. Cota Reviewed-by: Emilio G. Cota Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h | 7 ++++++- accel/tcg/cputlb.c | 35 +++++++++++++++++++++++++---------- 2 files changed, 31 insertions(+), 11 deletions(-) -- 2.17.2 diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index c7b501d627..ca0fea8b27 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -166,7 +166,12 @@ typedef struct CPUTLBCommon { * mmu_idx may be discarded. Protected by tlb_c.lock. */ uint16_t pending_flush; - + /* + * Within dirty, for each bit N, modifications have been made to + * mmu_idx N since the last time that mmu_idx was flushed. + * Protected by tlb_c.lock. + */ + uint16_t dirty; /* * Statistics. These are not lock protected, but are read and * written atomically. This allows the monitor to print a snapshot diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index e60628c350..f6c37bc4db 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -79,6 +79,9 @@ void tlb_init(CPUState *cpu) CPUArchState *env = cpu->env_ptr; qemu_spin_init(&env->tlb_c.lock); + + /* Ensure that cpu_reset performs a full flush. */ + env->tlb_c.dirty = ALL_MMUIDX_BITS; } /* flush_all_helper: run fn across all cpus @@ -129,31 +132,40 @@ static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx) static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; - unsigned long mmu_idx_bitmask = data.host_int; - int mmu_idx; + uint16_t asked = data.host_int; + uint16_t all_dirty, work, to_clean; assert_cpu_is_self(cpu); - tlb_debug("mmu_idx:0x%04lx\n", mmu_idx_bitmask); + tlb_debug("mmu_idx:0x%04" PRIx16 "\n", asked); qemu_spin_lock(&env->tlb_c.lock); - env->tlb_c.pending_flush &= ~mmu_idx_bitmask; - for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - if (test_bit(mmu_idx, &mmu_idx_bitmask)) { - tlb_flush_one_mmuidx_locked(env, mmu_idx); - } + all_dirty = env->tlb_c.dirty; + to_clean = asked & all_dirty; + all_dirty &= ~to_clean; + env->tlb_c.dirty = all_dirty; + + for (work = to_clean; work != 0; work &= work - 1) { + int mmu_idx = ctz32(work); + tlb_flush_one_mmuidx_locked(env, mmu_idx); } + qemu_spin_unlock(&env->tlb_c.lock); cpu_tb_jmp_cache_clear(cpu); - if (mmu_idx_bitmask == ALL_MMUIDX_BITS) { + if (to_clean == ALL_MMUIDX_BITS) { atomic_set(&env->tlb_c.full_flush_count, env->tlb_c.full_flush_count + 1); } else { atomic_set(&env->tlb_c.part_flush_count, - env->tlb_c.part_flush_count + ctpop16(mmu_idx_bitmask)); + env->tlb_c.part_flush_count + ctpop16(to_clean)); + if (to_clean != asked) { + atomic_set(&env->tlb_c.elide_flush_count, + env->tlb_c.elide_flush_count + + ctpop16(asked & ~to_clean)); + } } } @@ -581,6 +593,9 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, */ qemu_spin_lock(&env->tlb_c.lock); + /* Note that the tlb is no longer clean. */ + env->tlb_c.dirty |= 1 << mmu_idx; + /* Make sure there's no cached translation for the new page. */ tlb_flush_vtlb_page_locked(env, mmu_idx, vaddr_page);