From patchwork Sat Aug 3 18:47:28 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 170477 Delivered-To: patch@linaro.org Received: by 2002:a92:512:0:0:0:0:0 with SMTP id q18csp2341275ile; Sat, 3 Aug 2019 11:49:31 -0700 (PDT) X-Google-Smtp-Source: APXvYqxizRAAmbVQ0sIKEGyZt4Qk3Vg1zhRfn7ZOFlfEZfPepl7na4el/YA9MsLtn446gZVx6EBi X-Received: by 2002:a17:906:19d3:: with SMTP id h19mr10116883ejd.300.1564858171873; Sat, 03 Aug 2019 11:49:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1564858171; cv=none; d=google.com; s=arc-20160816; b=QE8I7WGWcjw3ygbQePvnK7Bkja9EShMQ85qUqC0Wu3Ch7+uDaIOQFhjbpxGampneBq DOxtcKKA/JiUPIA8VAeTdWD3jqVsEEzfOyWluGVOg8z0vE5ust7uow4XqtIh+lgMrEoh E1tg606NFtIEaDBdB4OArIt5XEW20ISG8+/+TmnXYRytawjJbs6PhRdZ2UDkwik1dN6S b0FmEr+nYtqHU8PyuzePWASZPYL0SzdKVuULLuhXHhXpqgLU82X5jf7eNmuoFwDRAVml 4Rl7VmCyx55puJDew8qw4oiEtnA8aX8HE8APKj0maRp9EpynXomxdocXnoa8/v8Zre6M v6RQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=OzH9oAANiLbbz3MZqOd0740wSKDkRIpOzsZLTONmMZs=; b=R/g9SyEZBVi8Gkm8IBo/EPMngoxNULdAm6AGF2oKfp0XbfpDkbghxhNCGjsB6LQ/is EVztMn2iDMiuhywhBQ1R1tvEA2zqvmXeAaPgOlw3NcByb/Fgs0kYcLinEy4gZVuKxPsE CkJ9I7j/Z/9q4anqO8/Qcp7E51tk+GrhQlMNNfHG3u4YcCJ+gv7S/ABeJ6R1OVWSxPht AaiW1ioN/Db9v1zLeZA87GrEOHhUldhA/M2V27QqZKLJ6YZZ7FgHZgN9ysa6dQXTgNv1 ez8pcvB3InDSrsYHu183XQkK1sPDS6Bz3CakdCZEhHLtxVGfUYN+xxtuqb3KXp202gOQ KLsg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=POoyQ7p+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q3si23533895ejt.89.2019.08.03.11.49.31 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Sat, 03 Aug 2019 11:49:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=POoyQ7p+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htz5u-0008BB-Td for patch@linaro.org; Sat, 03 Aug 2019 14:49:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:60495) by lists.gnu.org with esmtp (Exim 4.86_2) (envelope-from ) id 1htz4a-0005vH-3K for qemu-devel@nongnu.org; Sat, 03 Aug 2019 14:48:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1htz4Y-00075V-OO for qemu-devel@nongnu.org; Sat, 03 Aug 2019 14:48:08 -0400 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:40878) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1htz4Y-00074y-J2 for qemu-devel@nongnu.org; Sat, 03 Aug 2019 14:48:06 -0400 Received: by mail-pf1-x443.google.com with SMTP id p184so37618535pfp.7 for ; Sat, 03 Aug 2019 11:48:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=OzH9oAANiLbbz3MZqOd0740wSKDkRIpOzsZLTONmMZs=; b=POoyQ7p+raLFWdXdMAzpyzaP2JaDBIv6nyVaZb9BfgIy2h7Mrrv6IJN3cSSEJntlw5 fyjBeZs76EWn/Or6CbrU6PLJ1BCKSmRURYCh4ha3iDIM+5H4TlNZPQdCdwiX5W9Fp0Ee aswnR4wlqWtWQXx2paH4aGQY1c09aAtCPGHlqIQFNTe5xZgzADjy6HMgjLbaiCAlDmRs hhe/82vasY2whCT2XBPfo/RsXzWAh3RH9J/ywRIinWZ58/xgy3dkpNzVgCKgV8FvkYcC 4iXcMs8vs3kMvuW0wVWCymak7Ouftt5En+eVh42Dr1EM1IHppVJKxWULZeme/RoHJKWT a2uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=OzH9oAANiLbbz3MZqOd0740wSKDkRIpOzsZLTONmMZs=; b=G8Vwueu4DCcV72hxMCKddVYazxtV8S5YDmepDfcefUUOu+X6syHDkLlHzckFCKlL/a K61eU/cIVD+EQSSkNQjM1VUjj5MiuQoGo2PuiLPvE7S6UM3UAMpeLX0H10osPkVTmzCF lWrTJlth9pqod6gU4rcnLvmyDd0OyZEFkSdmBDAZlcu6fU0U2dAcR4pHRgGREWqaLHkW JlQj06HQDigybmgcE3pkGdau5ZGvoc6ccYwrF2cW/DVeVFcqS35kT2ye1WIMywirn++4 lLrbmKrXDBfnz9nRErHyTOMIZzGM8VDWgOcKfX3Sx91qjyuRpA4cgAb20FQz41L8PRCL QuSg== X-Gm-Message-State: APjAAAWG7EZI1rfZsQBlinFBFpdz6SNc1gEdUoKgwLO4+CiOknTCPjmx mLSqjI5CKeNw44tc6PG56DcKCpjo4g8= X-Received: by 2002:a63:8a49:: with SMTP id y70mr35691142pgd.271.1564858085297; Sat, 03 Aug 2019 11:48:05 -0700 (PDT) Received: from localhost.localdomain (97-113-7-119.tukw.qwest.net. [97.113.7.119]) by smtp.gmail.com with ESMTPSA id q69sm15405454pjb.0.2019.08.03.11.48.04 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Sat, 03 Aug 2019 11:48:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Sat, 3 Aug 2019 11:47:28 -0700 Message-Id: <20190803184800.8221-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20190803184800.8221-1-richard.henderson@linaro.org> References: <20190803184800.8221-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::443 Subject: [Qemu-devel] [PATCH v3 02/34] cputlb: Add tlb_flush_asid_by_mmuidx and friends X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, qemu-arm@nongnu.org, alex.bennee@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Since we have remembered ASIDs, we can further minimize flushing by comparing against the one we want to flush. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- include/exec/exec-all.h | 16 ++++++++++++ include/qom/cpu.h | 2 ++ accel/tcg/cputlb.c | 55 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 73 insertions(+) -- 2.17.1 diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 9c77aa5bf9..0d890e1e60 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -240,6 +240,22 @@ void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *cpu, uint16_t idxmap); */ void tlb_set_asid_for_mmuidx(CPUState *cpu, uint32_t asid, uint16_t idxmap, uint16_t dep_idxmap); +/** + * tlb_flush_asid_by_mmuidx: + * @cpu: Originating CPU of the flush + * @asid: Address Space Identifier + * @idxmap: bitmap of MMU indexes to flush if asid matches + * + * For each mmu index, if @asid matches the value previously saved via + * tlb_set_asid_for_mmuidx, flush the index. + */ +void tlb_flush_asid_by_mmuidx(CPUState *cpu, uint32_t asid, uint16_t idxmap); +/* Similarly, broadcasting to all cpus. */ +void tlb_flush_asid_by_mmuidx_all_cpus(CPUState *cpu, uint32_t asid, + uint16_t idxmap); +/* Similarly, waiting for the broadcast to complete. */ +void tlb_flush_asid_by_mmuidx_all_cpus_synced(CPUState *cpu, uint32_t asid, + uint16_t idxmap); /** * tlb_set_page_with_attrs: * @cpu: CPU to add this TLB entry for diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 5ee0046b62..c072dd4c47 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -285,12 +285,14 @@ typedef union { unsigned long host_ulong; void *host_ptr; vaddr target_ptr; + uint64_t uint64; } run_on_cpu_data; #define RUN_ON_CPU_HOST_PTR(p) ((run_on_cpu_data){.host_ptr = (p)}) #define RUN_ON_CPU_HOST_INT(i) ((run_on_cpu_data){.host_int = (i)}) #define RUN_ON_CPU_HOST_ULONG(ul) ((run_on_cpu_data){.host_ulong = (ul)}) #define RUN_ON_CPU_TARGET_PTR(v) ((run_on_cpu_data){.target_ptr = (v)}) +#define RUN_ON_CPU_UINT64(i) ((run_on_cpu_data){.uint64 = (i)}) #define RUN_ON_CPU_NULL RUN_ON_CPU_HOST_PTR(NULL) typedef void (*run_on_cpu_func)(CPUState *cpu, run_on_cpu_data data); diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index c68f57755b..62baaa9ca6 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -540,6 +540,61 @@ void tlb_flush_page_all_cpus_synced(CPUState *src, target_ulong addr) tlb_flush_page_by_mmuidx_all_cpus_synced(src, addr, ALL_MMUIDX_BITS); } +static void tlb_flush_asid_by_mmuidx_async_work(CPUState *cpu, + run_on_cpu_data data) +{ + CPUTLB *tlb = cpu_tlb(cpu); + uint32_t asid = data.uint64; + uint16_t idxmap = data.uint64 >> 32; + uint16_t to_flush = 0, work; + + assert_cpu_is_self(cpu); + + for (work = idxmap; work != 0; work &= work - 1) { + int mmu_idx = ctz32(work); + if (tlb->d[mmu_idx].asid == asid) { + to_flush |= 1 << mmu_idx; + } + } + + if (to_flush) { + tlb_flush_by_mmuidx_async_work(cpu, RUN_ON_CPU_HOST_INT(to_flush)); + } +} + +void tlb_flush_asid_by_mmuidx(CPUState *cpu, uint32_t asid, uint16_t idxmap) +{ + uint64_t asid_idx = deposit64(asid, 32, 32, idxmap); + + if (cpu->created && !qemu_cpu_is_self(cpu)) { + async_run_on_cpu(cpu, tlb_flush_asid_by_mmuidx_async_work, + RUN_ON_CPU_UINT64(asid_idx)); + } else { + tlb_flush_asid_by_mmuidx_async_work(cpu, RUN_ON_CPU_UINT64(asid_idx)); + } +} + +void tlb_flush_asid_by_mmuidx_all_cpus(CPUState *src_cpu, + uint32_t asid, uint16_t idxmap) +{ + uint64_t asid_idx = deposit64(asid, 32, 32, idxmap); + + flush_all_helper(src_cpu, tlb_flush_asid_by_mmuidx_async_work, + RUN_ON_CPU_UINT64(asid_idx)); + tlb_flush_asid_by_mmuidx_async_work(src_cpu, RUN_ON_CPU_UINT64(asid_idx)); +} + +void tlb_flush_asid_by_mmuidx_all_cpus_synced(CPUState *src_cpu, + uint32_t asid, uint16_t idxmap) +{ + uint64_t asid_idx = deposit64(asid, 32, 32, idxmap); + + flush_all_helper(src_cpu, tlb_flush_asid_by_mmuidx_async_work, + RUN_ON_CPU_UINT64(asid_idx)); + async_safe_run_on_cpu(src_cpu, tlb_flush_asid_by_mmuidx_async_work, + RUN_ON_CPU_UINT64(asid_idx)); +} + void tlb_set_asid_for_mmuidx(CPUState *cpu, uint32_t asid, uint16_t idxmap, uint16_t depmap) {