From patchwork Fri Jan 27 10:34:41 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92591 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp140165obz; Fri, 27 Jan 2017 02:43:00 -0800 (PST) X-Received: by 10.237.42.12 with SMTP id c12mr7140232qtd.284.1485513780290; Fri, 27 Jan 2017 02:43:00 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id p129si677434qkb.59.2017.01.27.02.42.59 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:43:00 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44248 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3zd-0005tH-Ck for patch@linaro.org; Fri, 27 Jan 2017 05:42:57 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47866) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sB-0007qA-7P for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3s8-00028r-MJ for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:15 -0500 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:35329) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3s8-00027U-DD for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:12 -0500 Received: by mail-wm0-x231.google.com with SMTP id r126so110787455wmr.0 for ; Fri, 27 Jan 2017 02:35:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ku419Yr6buTPZKGzHurO/1BGW+RmH6rQEgEnpJ4bXm4=; b=acOaWtBsu+FWkkWRwtdFQgQCmKtkGpVIBwdDeW5htzUdMk+d4tWsrcTkTuMoZZBEfJ /j4vh8tQAH/cx7rt2O7DLHsjkxXZYIoSUspRV6BzQsz/QxUfdA1Wd1+DBBxtanpQ605w 2zZyuHzzG0Dr3SukwfXhCYYAtJ+KZN8Z54vw8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ku419Yr6buTPZKGzHurO/1BGW+RmH6rQEgEnpJ4bXm4=; b=Duw01I634JcJ4j3RmyKmgrZSJab63s4RBg3S98zyifr8KkVMLefTO03f/Dbj8yZpRx E0VCV4SXVahlq6r6CBLlGxi8QE663IK2irLUtKfWIn+boMMB8QLSC4ne0iikhS7MQBlw HuPTm+qcPS0g0ScrAB2VIjMtCAwQeug7mRlN6NRy/7mR7LPtL/Z0Ll2iIQ3RjYERq/3Y +/IVg7u3jAP4LnUv5gbtBRT5Kl62pGZa/PGaKcc15X+LBbe5vSX8Fj7Ey4DSZNuL+6bZ BmvTmAWmHOBr0vJAt+4gIACR3AoIIPbELnBFhOn9eZzYphV76PUMPw6km8/D7/UJ9Ecf j4NA== X-Gm-Message-State: AIkVDXJtYKLh0RWsM1cTOckJqdODxvyUlX/jn5dZRRNnTWwr4f8xrUfhHf0oDXEH/L4/fCr0 X-Received: by 10.28.95.87 with SMTP id t84mr2289786wmb.135.1485513310912; Fri, 27 Jan 2017 02:35:10 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id p7sm7166393wrc.2.2017.01.27.02.35.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:08 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 97B0E3E049E; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:41 +0000 Message-Id: <20170127103505.18606-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [PATCH v8 01/25] docs: new design document multi-thread-tcg.txt X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This documents the current design for upgrading TCG emulation to take advantage of modern CPUs by running a thread-per-CPU. The document goes through the various areas of the code affected by such a change and proposes design requirements for each part of the solution. The text marked with (Current solution[s]) to document what the current approaches being used are. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v1 - initial version v2 - update discussion on locks - bit more detail on vCPU scheduling - explicitly mention Translation Blocks - emulated hardware state already covered by iomutex - a few minor rewords v3 - mention this covers system-mode - describe main main-loop and lookup hot-path - mention multi-concurrent-reader lookups - enumerate reasons for invalidation - add more details on lookup structures - describe the softmmu hot-path better - mention store-after-load barrier problem v4 - mention some cross-over between linux-user/system emulation - various minor grammar and scanning fixes - fix reference to tb_ctx.htbale - describe the solution for hot-path - more detail on TB flushing and invalidation - add (Current solution) following design requirements - more detail on iothread/BQL mutex - mention implicit memory barriers - add links to current LL/SC and cmpxchg patch sets - add TLB flag setting as an additional requirement v6 - remove DRAFTING, update copyright dates - document current solutions to each design requirement - tb_lock() serialisation for codegen/patch - cputlb changes to defer cross-vCPU flushes - cputlb atomic updates for slow-path - BQL usage for hardware serialisation - cmpxchg as initial atomic/synchronisation support mechanism v7 - minor format fix - include target-mips in list of MB aware front-ends - mention BQL around IRQ raising - update with notes on _all_cpus and the wait flag --- docs/multi-thread-tcg.txt | 350 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 350 insertions(+) create mode 100644 docs/multi-thread-tcg.txt -- 2.11.0 diff --git a/docs/multi-thread-tcg.txt b/docs/multi-thread-tcg.txt new file mode 100644 index 0000000000..a99b4564c6 --- /dev/null +++ b/docs/multi-thread-tcg.txt @@ -0,0 +1,350 @@ +Copyright (c) 2015-2016 Linaro Ltd. + +This work is licensed under the terms of the GNU GPL, version 2 or +later. See the COPYING file in the top-level directory. + +Introduction +============ + +This document outlines the design for multi-threaded TCG system-mode +emulation. The current user-mode emulation mirrors the thread +structure of the translated executable. Some of the work will be +applicable to both system and linux-user emulation. + +The original system-mode TCG implementation was single threaded and +dealt with multiple CPUs with simple round-robin scheduling. This +simplified a lot of things but became increasingly limited as systems +being emulated gained additional cores and per-core performance gains +for host systems started to level off. + +vCPU Scheduling +=============== + +We introduce a new running mode where each vCPU will run on its own +user-space thread. This will be enabled by default for all FE/BE +combinations that have had the required work done to support this +safely. + +In the general case of running translated code there should be no +inter-vCPU dependencies and all vCPUs should be able to run at full +speed. Synchronisation will only be required while accessing internal +shared data structures or when the emulated architecture requires a +coherent representation of the emulated machine state. + +Shared Data Structures +====================== + +Main Run Loop +------------- + +Even when there is no code being generated there are a number of +structures associated with the hot-path through the main run-loop. +These are associated with looking up the next translation block to +execute. These include: + + tb_jmp_cache (per-vCPU, cache of recent jumps) + tb_ctx.htable (global hash table, phys address->tb lookup) + +As TB linking only occurs when blocks are in the same page this code +is critical to performance as looking up the next TB to execute is the +most common reason to exit the generated code. + +DESIGN REQUIREMENT: Make access to lookup structures safe with +multiple reader/writer threads. Minimise any lock contention to do it. + +The hot-path avoids using locks where possible. The tb_jmp_cache is +updated with atomic accesses to ensure consistent results. The fall +back QHT based hash table is also designed for lockless lookups. Locks +are only taken when code generation is required or TranslationBlocks +have their block-to-block jumps patched. + +Global TCG State +---------------- + +We need to protect the entire code generation cycle including any post +generation patching of the translated code. This also implies a shared +translation buffer which contains code running on all cores. Any +execution path that comes to the main run loop will need to hold a +mutex for code generation. This also includes times when we need flush +code or entries from any shared lookups/caches. Structures held on a +per-vCPU basis won't need locking unless other vCPUs will need to +modify them. + +DESIGN REQUIREMENT: Add locking around all code generation and TB +patching. + +(Current solution) + +Mainly as part of the linux-user work all code generation is +serialised with a tb_lock(). For the SoftMMU tb_lock() also takes the +place of mmap_lock() in linux-user. + +Translation Blocks +------------------ + +Currently the whole system shares a single code generation buffer +which when full will force a flush of all translations and start from +scratch again. Some operations also force a full flush of translations +including: + + - debugging operations (breakpoint insertion/removal) + - some CPU helper functions + +This is done with the async_safe_run_on_cpu() mechanism to ensure all +vCPUs are quiescent when changes are being made to shared global +structures. + +More granular translation invalidation events are typically due +to a change of the state of a physical page: + + - code modification (self modify code, patching code) + - page changes (new page mapping in linux-user mode) + +While setting the invalid flag in a TranslationBlock will stop it +being used when looked up in the hot-path there are a number of other +book-keeping structures that need to be safely cleared. + +Any TranslationBlocks which have been patched to jump directly to the +now invalid blocks need the jump patches reversing so they will return +to the C code. + +There are a number of look-up caches that need to be properly updated +including the: + + - jump lookup cache + - the physical-to-tb lookup hash table + - the global page table + +The global page table (l1_map) which provides a multi-level look-up +for PageDesc structures which contain pointers to the start of a +linked list of all Translation Blocks in that page (see page_next). + +Both the jump patching and the page cache involve linked lists that +the invalidated TranslationBlock needs to be removed from. + +DESIGN REQUIREMENT: Safely handle invalidation of TBs + - safely patch/revert direct jumps + - remove central PageDesc lookup entries + - ensure lookup caches/hashes are safely updated + +(Current solution) + +The direct jump themselves are updated atomically by the TCG +tb_set_jmp_target() code. Modification to the linked lists that allow +searching for linked pages are done under the protect of the +tb_lock(). + +The global page table is protected by the tb_lock() in system-mode and +mmap_lock() in linux-user mode. + +The lookup caches are updated atomically and the lookup hash uses QHT +which is designed for concurrent safe lookup. + + +Memory maps and TLBs +-------------------- + +The memory handling code is fairly critical to the speed of memory +access in the emulated system. The SoftMMU code is designed so the +hot-path can be handled entirely within translated code. This is +handled with a per-vCPU TLB structure which once populated will allow +a series of accesses to the page to occur without exiting the +translated code. It is possible to set flags in the TLB address which +will ensure the slow-path is taken for each access. This can be done +to support: + + - Memory regions (dividing up access to PIO, MMIO and RAM) + - Dirty page tracking (for code gen, SMC detection, migration and display) + - Virtual TLB (for translating guest address->real address) + +When the TLB tables are updated by a vCPU thread other than their own +we need to ensure it is done in a safe way so no inconsistent state is +seen by the vCPU thread. + +Some operations require updating a number of vCPUs TLBs at the same +time in a synchronised manner. + +DESIGN REQUIREMENTS: + + - TLB Flush All/Page + - can be across-vCPUs + - cross vCPU TLB flush may need other vCPU brought to halt + - change may need to be visible to the calling vCPU immediately + - TLB Flag Update + - usually cross-vCPU + - want change to be visible as soon as possible + - TLB Update (update a CPUTLBEntry, via tlb_set_page_with_attrs) + - This is a per-vCPU table - by definition can't race + - updated by its own thread when the slow-path is forced + +(Current solution) + +We have updated cputlb.c to defer operations when a cross-vCPU +operation with async_run_on_cpu() which ensures each vCPU sees a +coherent state when it next runs its work (in a few instructions +time). + +A new set up operations (tlb_flush_*_all_cpus) take an additional flag +which when set will force synchronisation by setting the source vCPUs +work as "safe work" and exiting the cpu run loop. This ensure by the +time execution restarts all flush operations have completed. + +TLB flag updates are all done atomically and are also protected by the +tb_lock() which is used by the functions that update the TLB in bulk. + +(Known limitation) + +Not really a limitation but the wait mechanism is overly strict for +some architectures which only need flushes completed by a barrier +instruction. This could be a future optimisation. + +Emulated hardware state +----------------------- + +Currently thanks to KVM work any access to IO memory is automatically +protected by the global iothread mutex, also known as the BQL (Big +Qemu Lock). Any IO region that doesn't use global mutex is expected to +do its own locking. + +However IO memory isn't the only way emulated hardware state can be +modified. Some architectures have model specific registers that +trigger hardware emulation features. Generally any translation helper +that needs to update more than a single vCPUs of state should take the +BQL. + +As the BQL, or global iothread mutex is shared across the system we +push the use of the lock as far down into the TCG code as possible to +minimise contention. + +(Current solution) + +MMIO access automatically serialises hardware emulation by way of the +BQL. Currently ARM targets serialise all ARM_CP_IO register accesses +and also defer the reset/startup of vCPUs to the vCPU context by way +of async_run_on_cpu(). + +Updates to interrupt state are also protected by the BQL as they can +often be cross vCPU. + +Memory Consistency +================== + +Between emulated guests and host systems there are a range of memory +consistency models. Even emulating weakly ordered systems on strongly +ordered hosts needs to ensure things like store-after-load re-ordering +can be prevented when the guest wants to. + +Memory Barriers +--------------- + +Barriers (sometimes known as fences) provide a mechanism for software +to enforce a particular ordering of memory operations from the point +of view of external observers (e.g. another processor core). They can +apply to any memory operations as well as just loads or stores. + +The Linux kernel has an excellent write-up on the various forms of +memory barrier and the guarantees they can provide [1]. + +Barriers are often wrapped around synchronisation primitives to +provide explicit memory ordering semantics. However they can be used +by themselves to provide safe lockless access by ensuring for example +a change to a signal flag will only be visible once the changes to +payload are. + +DESIGN REQUIREMENT: Add a new tcg_memory_barrier op + +This would enforce a strong load/store ordering so all loads/stores +complete at the memory barrier. On single-core non-SMP strongly +ordered backends this could become a NOP. + +Aside from explicit standalone memory barrier instructions there are +also implicit memory ordering semantics which comes with each guest +memory access instruction. For example all x86 load/stores come with +fairly strong guarantees of sequential consistency where as ARM has +special variants of load/store instructions that imply acquire/release +semantics. + +In the case of a strongly ordered guest architecture being emulated on +a weakly ordered host the scope for a heavy performance impact is +quite high. + +DESIGN REQUIREMENTS: Be efficient with use of memory barriers + - host systems with stronger implied guarantees can skip some barriers + - merge consecutive barriers to the strongest one + +(Current solution) + +The system currently has a tcg_gen_mb() which will add memory barrier +operations if code generation is being done in a parallel context. The +tcg_optimize() function attempts to merge barriers up to their +strongest form before any load/store operations. The solution was +originally developed and tested for linux-user based systems. All +backends have been converted to emit fences when required. So far the +following front-ends have been updated to emit fences when required: + + - target-i386 + - target-arm + - target-aarch64 + - target-alpha + - target-mips + +Memory Control and Maintenance +------------------------------ + +This includes a class of instructions for controlling system cache +behaviour. While QEMU doesn't model cache behaviour these instructions +are often seen when code modification has taken place to ensure the +changes take effect. + +Synchronisation Primitives +-------------------------- + +There are two broad types of synchronisation primitives found in +modern ISAs: atomic instructions and exclusive regions. + +The first type offer a simple atomic instruction which will guarantee +some sort of test and conditional store will be truly atomic w.r.t. +other cores sharing access to the memory. The classic example is the +x86 cmpxchg instruction. + +The second type offer a pair of load/store instructions which offer a +guarantee that an region of memory has not been touched between the +load and store instructions. An example of this is ARM's ldrex/strex +pair where the strex instruction will return a flag indicating a +successful store only if no other CPU has accessed the memory region +since the ldrex. + +Traditionally TCG has generated a series of operations that work +because they are within the context of a single translation block so +will have completed before another CPU is scheduled. However with +the ability to have multiple threads running to emulate multiple CPUs +we will need to explicitly expose these semantics. + +DESIGN REQUIREMENTS: + - Support classic atomic instructions + - Support load/store exclusive (or load link/store conditional) pairs + - Generic enough infrastructure to support all guest architectures +CURRENT OPEN QUESTIONS: + - How problematic is the ABA problem in general? + +(Current solution) + +The TCG provides a number of atomic helpers (tcg_gen_atomic_*) which +can be used directly or combined to emulate other instructions like +ARM's ldrex/strex instructions. While they are susceptible to the ABA +problem so far common guests have not implemented patterns where +this may be a problem - typically presenting a locking ABI which +assumes cmpxchg like semantics. + +The code also includes a fall-back for cases where multi-threaded TCG +ops can't work (e.g. guest atomic width > host atomic width). In this +case an EXCP_ATOMIC exit occurs and the instruction is emulated with +an exclusive lock which ensures all emulation is serialised. + +While the atomic helpers look good enough for now there may be a need +to look at solutions that can more closely model the guest +architectures semantics. + +========== + +[1] https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/plain/Documentation/memory-barriers.txt From patchwork Fri Jan 27 10:34:42 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92585 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp138597obz; Fri, 27 Jan 2017 02:38:01 -0800 (PST) X-Received: by 10.237.49.225 with SMTP id 88mr7004453qth.120.1485513481741; Fri, 27 Jan 2017 02:38:01 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id d44si3231966qte.186.2017.01.27.02.38.01 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:38:01 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44223 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3uo-00011d-Vu for patch@linaro.org; Fri, 27 Jan 2017 05:37:59 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47828) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3s9-0007oj-4J for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:14 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3s6-00026W-21 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:13 -0500 Received: from mail-wm0-x233.google.com ([2a00:1450:400c:c09::233]:34744) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3s5-00025U-Oh for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:10 -0500 Received: by mail-wm0-x233.google.com with SMTP id f73so12546767wmf.1 for ; Fri, 27 Jan 2017 02:35:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KdkItYknUErEkF+W57vro7JnHc9Ila5EvXDRMvVx84Q=; b=T+gitzJfISP1uCM6RoAvkib/Jslsx1T9cqsSmfBXlV9smzMa5wG2P+9ZuNgr4nwlaF 1JOHpU+VzCHhYX+hMHtfZfnT/ZW6sb60gAZqktasq5pbUIzdot/nQLXpPg7gdml66xWg ARU11EhYuIRgf+mNHNHejhATgv9awLWjrTd0g= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KdkItYknUErEkF+W57vro7JnHc9Ila5EvXDRMvVx84Q=; b=Jmhcmn+y7gFAC8jM8Cc85VZZy5z6aS2FmF1MGTsGy45iPYyizVyiFSWyMa2QmWeFjJ OdIJZMLbdfqPhtDxOM0Ka88Hb5Q2H78yUa7/ahKi2gvSMZM27PjRyJHaCfp85/85A+7m 1DDkPL5s42S1FBkdMe8hxXM0Hmm/NGdLgm95xibKv9HaGvqbZs2npeY6CYkQhLpcjj78 0c9pjIajYAjNOLgikwRjpU8906QOFEADPrqNyx3nm1hfsjPY7EB22Phy0Cz3msCNPrKF nTwz7xctaSJklmqvO7L/y0AlR+hMov2X3eh64NYClRchCjE2UudjzjaTW1tuy39/ItvC ecgQ== X-Gm-Message-State: AIkVDXJx/a6yJ0SrrCawcQhjSTDo8PmSgYW7HpFjhOV29cxl5wy+GbT51AFNlerImcVA92la X-Received: by 10.28.130.212 with SMTP id e203mr2330722wmd.104.1485513308543; Fri, 27 Jan 2017 02:35:08 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id k195sm3149652wmd.7.2017.01.27.02.35.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:06 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id A82C93E2A53; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:42 +0000 Message-Id: <20170127103505.18606-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::233 Subject: [Qemu-devel] [PATCH v8 02/25] mttcg: translate-all: Enable locking debug in a debug build X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , Pranith Kumar , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Pranith Kumar Enable tcg lock debug asserts in a debug build by default instead of relying on DEBUG_LOCKING. None of the other DEBUG_* macros have asserts, so this patch removes DEBUG_LOCKING and enable these asserts in a debug build. CC: Richard Henderson Signed-off-by: Pranith Kumar [AJB: tweak ifdefs so can be early in series] Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- translate-all.c | 52 ++++++++++++++++------------------------------------ 1 file changed, 16 insertions(+), 36 deletions(-) -- 2.11.0 diff --git a/translate-all.c b/translate-all.c index 20262938bb..055436a676 100644 --- a/translate-all.c +++ b/translate-all.c @@ -59,7 +59,6 @@ /* #define DEBUG_TB_INVALIDATE */ /* #define DEBUG_TB_FLUSH */ -/* #define DEBUG_LOCKING */ /* make various TB consistency checks */ /* #define DEBUG_TB_CHECK */ @@ -74,20 +73,10 @@ * access to the memory related structures are protected with the * mmap_lock. */ -#ifdef DEBUG_LOCKING -#define DEBUG_MEM_LOCKS 1 -#else -#define DEBUG_MEM_LOCKS 0 -#endif - #ifdef CONFIG_SOFTMMU #define assert_memory_lock() do { /* nothing */ } while (0) #else -#define assert_memory_lock() do { \ - if (DEBUG_MEM_LOCKS) { \ - g_assert(have_mmap_lock()); \ - } \ - } while (0) +#define assert_memory_lock() tcg_debug_assert(have_mmap_lock()) #endif #define SMC_BITMAP_USE_THRESHOLD 10 @@ -169,10 +158,18 @@ static void page_table_config_init(void) assert(v_l2_levels >= 0); } +#ifdef CONFIG_USER_ONLY +#define assert_tb_locked() tcg_debug_assert(have_tb_lock) +#define assert_tb_unlocked() tcg_debug_assert(!have_tb_lock) +#else +#define assert_tb_locked() do { /* nothing */ } while (0) +#define assert_tb_unlocked() do { /* nothing */ } while (0) +#endif + void tb_lock(void) { #ifdef CONFIG_USER_ONLY - assert(!have_tb_lock); + assert_tb_unlocked(); qemu_mutex_lock(&tcg_ctx.tb_ctx.tb_lock); have_tb_lock++; #endif @@ -181,7 +178,7 @@ void tb_lock(void) void tb_unlock(void) { #ifdef CONFIG_USER_ONLY - assert(have_tb_lock); + assert_tb_locked(); have_tb_lock--; qemu_mutex_unlock(&tcg_ctx.tb_ctx.tb_lock); #endif @@ -197,23 +194,6 @@ void tb_lock_reset(void) #endif } -#ifdef DEBUG_LOCKING -#define DEBUG_TB_LOCKS 1 -#else -#define DEBUG_TB_LOCKS 0 -#endif - -#ifdef CONFIG_SOFTMMU -#define assert_tb_lock() do { /* nothing */ } while (0) -#else -#define assert_tb_lock() do { \ - if (DEBUG_TB_LOCKS) { \ - g_assert(have_tb_lock); \ - } \ - } while (0) -#endif - - static TranslationBlock *tb_find_pc(uintptr_t tc_ptr); void cpu_gen_init(void) @@ -847,7 +827,7 @@ static TranslationBlock *tb_alloc(target_ulong pc) { TranslationBlock *tb; - assert_tb_lock(); + assert_tb_locked(); if (tcg_ctx.tb_ctx.nb_tbs >= tcg_ctx.code_gen_max_blocks) { return NULL; @@ -862,7 +842,7 @@ static TranslationBlock *tb_alloc(target_ulong pc) /* Called with tb_lock held. */ void tb_free(TranslationBlock *tb) { - assert_tb_lock(); + assert_tb_locked(); /* In practice this is mostly used for single use temporary TB Ignore the hard cases and just back up if this TB happens to @@ -1104,7 +1084,7 @@ void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr) uint32_t h; tb_page_addr_t phys_pc; - assert_tb_lock(); + assert_tb_locked(); atomic_set(&tb->invalid, true); @@ -1419,7 +1399,7 @@ static void tb_invalidate_phys_range_1(tb_page_addr_t start, tb_page_addr_t end) #ifdef CONFIG_SOFTMMU void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end) { - assert_tb_lock(); + assert_tb_locked(); tb_invalidate_phys_range_1(start, end); } #else @@ -1462,7 +1442,7 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, #endif /* TARGET_HAS_PRECISE_SMC */ assert_memory_lock(); - assert_tb_lock(); + assert_tb_locked(); p = page_find(start >> TARGET_PAGE_BITS); if (!p) { From patchwork Fri Jan 27 10:34:43 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92587 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp138717obz; Fri, 27 Jan 2017 02:38:19 -0800 (PST) X-Received: by 10.200.36.81 with SMTP id d17mr6532988qtd.99.1485513499563; Fri, 27 Jan 2017 02:38:19 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id s26si3228834qte.294.2017.01.27.02.38.19 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:38:19 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44225 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3v6-0001Yw-Rt for patch@linaro.org; Fri, 27 Jan 2017 05:38:16 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47870) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sB-0007qJ-Dx for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3s7-00027C-4e for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:15 -0500 Received: from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:38424) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3s6-00026u-V9 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:11 -0500 Received: by mail-wm0-x235.google.com with SMTP id r144so129825816wme.1 for ; Fri, 27 Jan 2017 02:35:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y2+v1flgVFTMeQ/XkdeqfH3M+QZGMjHF/xurefWed+s=; b=e3rm/Dh/SwAsGwX1VXrPbvLLbSSa5wMo060c5xzLH3AGIRFVvjwzZUeUUtpgjr6AqW 39rHOfV4mXlTQSbZZa4yOnNIBIyjowKA0IbMna594FhVuTCZUakJZeL+HRnj5ZWzckwk i7EhflFPv24ulpjljbU6B4ym7XlD5pNuz1voo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y2+v1flgVFTMeQ/XkdeqfH3M+QZGMjHF/xurefWed+s=; b=tsqRO2kgExhoJ0eqtHbU04I1mHGd+3fbXI8MA5VvAkpUZRObWDOQIJfjBwgZ+9aPgu dqSQAWPofxfc61Rnwbh+ngj94CAA+Qdf4TUNpTgdw6Usn1eKiGsV0FsHozj4lFTy4FnX lv8Z/c5lzFV2mOeLWcRG39rkSrLg8aS9Pectq2OSm62ys0RWo/ckGs+bonfr3TAR0OtH usSR5eBn4FYI4JS6P934PwhrBBdAVvW5YgvwiHRtSPnXHVQ424S2GHNDzPhjPg76jrVo kTXUOlcxcnGeR7Gcaj0RaTj6ezAnpbHtexyVqoreD+0OGzOOOIyAi3QZAaxbJrG7mzqn 3VIg== X-Gm-Message-State: AIkVDXIGN07K2utxTysucoJk9nsqoWf9zhmeaUGW425MdgBsjiLqLzZSlyC+3Hp93uT9Reev X-Received: by 10.223.154.165 with SMTP id a34mr6688121wrc.193.1485513309895; Fri, 27 Jan 2017 02:35:09 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id q1sm3168171wmd.6.2017.01.27.02.35.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:07 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id B8FE43E36FC; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:43 +0000 Message-Id: <20170127103505.18606-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::235 Subject: [Qemu-devel] [PATCH v8 03/25] mttcg: Add missing tb_lock/unlock() in cpu_exec_step() X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , Pranith Kumar , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Pranith Kumar The recent patch enabling lock assertions uncovered the missing lock acquisition in cpu_exec_step(). This patch adds them. CC: Richard Henderson CC: Alex Bennée Signed-off-by: Pranith Kumar --- cpu-exec.c | 4 ++++ 1 file changed, 4 insertions(+) -- 2.11.0 diff --git a/cpu-exec.c b/cpu-exec.c index 4188fed3c6..1b8685dc21 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -233,14 +233,18 @@ static void cpu_exec_step(CPUState *cpu) uint32_t flags; cpu_get_tb_cpu_state(env, &pc, &cs_base, &flags); + tb_lock(); tb = tb_gen_code(cpu, pc, cs_base, flags, 1 | CF_NOCACHE | CF_IGNORE_ICOUNT); tb->orig_tb = NULL; + tb_unlock(); /* execute the generated code */ trace_exec_tb_nocache(tb, pc); cpu_tb_exec(cpu, tb); + tb_lock(); tb_phys_invalidate(tb, -1); tb_free(tb); + tb_unlock(); } void cpu_exec_step_atomic(CPUState *cpu) From patchwork Fri Jan 27 10:34:44 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92589 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp140034obz; Fri, 27 Jan 2017 02:42:35 -0800 (PST) X-Received: by 10.200.56.187 with SMTP id f56mr6857093qtc.234.1485513755796; Fri, 27 Jan 2017 02:42:35 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id j66si3232453qkd.306.2017.01.27.02.42.35 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:42:35 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44246 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3zF-000535-5h for patch@linaro.org; Fri, 27 Jan 2017 05:42:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47858) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sA-0007pp-NX for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3s9-00029G-E1 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:14 -0500 Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]:35335) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3s9-00028i-8H for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:13 -0500 Received: by mail-wm0-x234.google.com with SMTP id r126so110787979wmr.0 for ; Fri, 27 Jan 2017 02:35:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ol0eMdbsQVWkGjfZbuc2uH0Q/SjOQV113ODhhcKxIFI=; b=FTUj1Hozgkoew6kcbO1Ofh9TwDERirR80TW8Mr1rkIQb4soTONH/AbiNsD3tvX9PFI nzHLfJjvuchdu65uOSeYpdOxN3fwSfqoGqOPj2BOjUewbDgRchv14yTPoq75Q3MK7mvs 5FB7Jt4Wx8dBIKwbAsdmYlrNf4mVYN2lBjHnU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ol0eMdbsQVWkGjfZbuc2uH0Q/SjOQV113ODhhcKxIFI=; b=nWXiFlPVSAJTdSGHXenjYc8Vo2XEXGjcNbED2avanLWbKBwRrC/pFErDzE1vPFyOke srBxr8jn6nHcHs/h6q/9i+EoFc6fGFOpBXe3LLwxMRx7AMkKojGJTWL8vqBdx8qyIBCB CSVQ8z48zqcmj4FwOhcKZFoHWBqst1DcIrIbu8qbPLOIFC5TA4zSCFOqWIBs0qMxofpl qWakg5ZBMUP/WD+18CUyYbx66b+hY80syT/vdi26vh+oHy+nIAR3SXCZ1DNArxWECS7N xmMiCk05b+uizzIjM3R71aiAqdMF31WPwtmTMkQu82CazN653KJiy4c3pf67Tzq+MR9E bdFg== X-Gm-Message-State: AIkVDXKwKDQj6CZtPN89RMrXE58TxnVTk8ydldZ9Tgj76WWGTGmbWShKZxtuSdEn6DOr9lvc X-Received: by 10.28.111.78 with SMTP id k75mr2603892wmc.71.1485513312079; Fri, 27 Jan 2017 02:35:12 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g40sm7155805wrg.19.2017.01.27.02.35.06 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:08 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id C9AD33E3713; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:44 +0000 Message-Id: <20170127103505.18606-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::234 Subject: [Qemu-devel] [PATCH v8 04/25] tcg: move TCG_MO/BAR types into own file X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We'll be using the memory ordering definitions to define values for both the host and guest. To avoid fighting with circular header dependencies just move these types into their own minimal header. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- tcg/tcg-mo.h | 45 +++++++++++++++++++++++++++++++++++++++++++++ tcg/tcg.h | 18 +----------------- 2 files changed, 46 insertions(+), 17 deletions(-) create mode 100644 tcg/tcg-mo.h -- 2.11.0 diff --git a/tcg/tcg-mo.h b/tcg/tcg-mo.h new file mode 100644 index 0000000000..429b022561 --- /dev/null +++ b/tcg/tcg-mo.h @@ -0,0 +1,45 @@ +/* + * Tiny Code Generator for QEMU + * + * Copyright (c) 2008 Fabrice Bellard + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef TCG_MO_H +#define TCG_MO_H + +typedef enum { + /* Used to indicate the type of accesses on which ordering + is to be ensured. Modeled after SPARC barriers. */ + TCG_MO_LD_LD = 0x01, + TCG_MO_ST_LD = 0x02, + TCG_MO_LD_ST = 0x04, + TCG_MO_ST_ST = 0x08, + TCG_MO_ALL = 0x0F, /* OR of the above */ + + /* Used to indicate the kind of ordering which is to be ensured by the + instruction. These types are derived from x86/aarch64 instructions. + It should be noted that these are different from C11 semantics. */ + TCG_BAR_LDAQ = 0x10, /* Following ops will not come forward */ + TCG_BAR_STRL = 0x20, /* Previous ops will not be delayed */ + TCG_BAR_SC = 0x30, /* No ops cross barrier; OR of the above */ +} TCGBar; + +#endif /* TCG_MO_H */ diff --git a/tcg/tcg.h b/tcg/tcg.h index 631c6f69b1..f946452049 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -29,6 +29,7 @@ #include "cpu.h" #include "exec/tb-context.h" #include "qemu/bitops.h" +#include "tcg-mo.h" #include "tcg-target.h" /* XXX: make safe guess about sizes */ @@ -498,23 +499,6 @@ static inline intptr_t QEMU_ARTIFICIAL GET_TCGV_PTR(TCGv_ptr t) #define TCG_CALL_DUMMY_TCGV MAKE_TCGV_I32(-1) #define TCG_CALL_DUMMY_ARG ((TCGArg)(-1)) -typedef enum { - /* Used to indicate the type of accesses on which ordering - is to be ensured. Modeled after SPARC barriers. */ - TCG_MO_LD_LD = 0x01, - TCG_MO_ST_LD = 0x02, - TCG_MO_LD_ST = 0x04, - TCG_MO_ST_ST = 0x08, - TCG_MO_ALL = 0x0F, /* OR of the above */ - - /* Used to indicate the kind of ordering which is to be ensured by the - instruction. These types are derived from x86/aarch64 instructions. - It should be noted that these are different from C11 semantics. */ - TCG_BAR_LDAQ = 0x10, /* Following ops will not come forward */ - TCG_BAR_STRL = 0x20, /* Previous ops will not be delayed */ - TCG_BAR_SC = 0x30, /* No ops cross barrier; OR of the above */ -} TCGBar; - /* Conditions. Note that these are laid out for easy manipulation by the functions below: bit 0 is used for inverting; From patchwork Fri Jan 27 10:34:45 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92590 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp140042obz; Fri, 27 Jan 2017 02:42:38 -0800 (PST) X-Received: by 10.55.0.148 with SMTP id t20mr7015343qkg.299.1485513758011; Fri, 27 Jan 2017 02:42:38 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id x65si3235101qke.229.2017.01.27.02.42.37 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:42:38 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44247 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3zH-00054o-4N for patch@linaro.org; Fri, 27 Jan 2017 05:42:35 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47903) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sC-0007qS-Hz for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sA-00029u-Rz for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:16 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:38457) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sA-00029W-Il for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:14 -0500 Received: by mail-wm0-x236.google.com with SMTP id r144so129828323wme.1 for ; Fri, 27 Jan 2017 02:35:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8jg6RZj0CTHSYLn9yq0CoQ7TigGXymIvPOsBoLSL1Rg=; b=cADirWdPfnv/OwgiqwoP7ysQXXaOnPHxT8evmQ/7zrWiQIDAyVdFNV0B0Fr53lXCIS A2H4/YM9pXJZrQiihoyij/SVCm0HlDRSWbssMO3xQGuOwhTfFcwUW2gBQOLzO98rJwjH zo6frxJD9ocRTtvLJucCodxEBTCkExvriVQtQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8jg6RZj0CTHSYLn9yq0CoQ7TigGXymIvPOsBoLSL1Rg=; b=qmZPdWfyicUpuQzSKraZ7EiXUeE0yr01Kd0ciDsjfSgQz+/u+APGqQdw9kY6dVNz9P Y7hAkvshLgGynP6L9oVwM5efO7v5t2QWleARL23Ie4nCOSSdIm+h8OLVu7fAIGk0lSFT 1JXsBHum+EDcA9tWkf1erHhb3rsVQ20o2Xbg6f5WIMrwAVRA4s3GACjDv8BoZoNUUJ36 qT6xIS5/kOo+vYlSG1XekKsjaxhft9GRDQmEg0eyOdLIC/m06ZpDSp2eFDl/AXQ+O9jI f+EAM0bF0JmFdBmRiG6BwVW2/HlsanOyWsxNsPh/sRbCNzBP2OEgAprhn7kSm+sRTY1e qaoA== X-Gm-Message-State: AIkVDXKc1+Ewc24cpYwqSTbu9ObjcP8J/5MTx5F5CbRmyIQEBFoBC8Q+sewFrszLfeBpZeza X-Received: by 10.28.144.66 with SMTP id s63mr2589467wmd.134.1485513313209; Fri, 27 Jan 2017 02:35:13 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 123sm3147520wml.6.2017.01.27.02.35.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:09 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id DF95D3E3744; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:45 +0000 Message-Id: <20170127103505.18606-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 05/25] tcg: add options for enabling MTTCG X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , KONRAD Frederic , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: KONRAD Frederic We know there will be cases where MTTCG won't work until additional work is done in the front/back ends to support. It will however be useful to be able to turn it on. As a result MTTCG will default to off unless the combination is supported. However the user can turn it on for the sake of testing. Signed-off-by: KONRAD Frederic [AJB: move to -accel tcg,thread=multi|single, defaults] Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v1: - merge with add mttcg option. - update commit message v2: - machine_init->opts_init v3: - moved from -tcg to -accel tcg,thread=single|multi - fix checkpatch warnings v4: - make mttcg_enabled extern, qemu_tcg_mttcg_enabled() now just macro - qemu_tcg_configure now propagates Error instead of exiting - better error checking of thread=foo - use CONFIG flags for default_mttcg_enabled() - disable mttcg with icount, error if both forced on v7 - explicitly disable MTTCG for TCG_OVERSIZED_GUEST - use check_tcg_memory_orders_compatible() instead of CONFIG_MTTCG_HOST - change CONFIG_MTTCG_TARGET to TARGET_SUPPORTS_MTTCG v8 - fix missing include tcg.h - change mismatched MOs to a warning instead of error --- cpus.c | 72 +++++++++++++++++++++++++++++++++++++++++++++++++++ include/qom/cpu.h | 9 +++++++ include/sysemu/cpus.h | 2 ++ qemu-options.hx | 20 ++++++++++++++ tcg/tcg.h | 9 +++++++ vl.c | 49 ++++++++++++++++++++++++++++++++++- 6 files changed, 160 insertions(+), 1 deletion(-) -- 2.11.0 diff --git a/cpus.c b/cpus.c index 71a82e5004..76b6e04332 100644 --- a/cpus.c +++ b/cpus.c @@ -25,6 +25,7 @@ /* Needed early for CONFIG_BSD etc. */ #include "qemu/osdep.h" #include "qemu-common.h" +#include "qemu/config-file.h" #include "cpu.h" #include "monitor/monitor.h" #include "qapi/qmp/qerror.h" @@ -45,6 +46,7 @@ #include "qemu/main-loop.h" #include "qemu/bitmap.h" #include "qemu/seqlock.h" +#include "tcg.h" #include "qapi-event.h" #include "hw/nmi.h" #include "sysemu/replay.h" @@ -150,6 +152,76 @@ typedef struct TimersState { } TimersState; static TimersState timers_state; +bool mttcg_enabled; + +/* + * We default to false if we know other options have been enabled + * which are currently incompatible with MTTCG. Otherwise when each + * guest (target) has been updated to support: + * - atomic instructions + * - memory ordering primitives (barriers) + * they can set the appropriate CONFIG flags in ${target}-softmmu.mak + * + * Once a guest architecture has been converted to the new primitives + * there are two remaining limitations to check. + * + * - The guest can't be oversized (e.g. 64 bit guest on 32 bit host) + * - The host must have a stronger memory order than the guest + * + * It may be possible in future to support strong guests on weak hosts + * but that will require tagging all load/stores in a guest with their + * implicit memory order requirements which would likely slow things + * down a lot. + */ + +static bool check_tcg_memory_orders_compatible(void) +{ +#if defined(TCG_DEFAULT_MO) && defined(TCG_TARGET_DEFAULT_MO) + return (TCG_DEFAULT_MO & ~TCG_TARGET_DEFAULT_MO) == 0; +#else + return false; +#endif +} + +static bool default_mttcg_enabled(void) +{ + QemuOpts *icount_opts = qemu_find_opts_singleton("icount"); + const char *rr = qemu_opt_get(icount_opts, "rr"); + + if (rr || TCG_OVERSIZED_GUEST) { + return false; + } else { +#ifdef TARGET_SUPPORTS_MTTCG + return check_tcg_memory_orders_compatible(); +#else + return false; +#endif + } +} + +void qemu_tcg_configure(QemuOpts *opts, Error **errp) +{ + const char *t = qemu_opt_get(opts, "thread"); + if (t) { + if (strcmp(t, "multi") == 0) { + if (TCG_OVERSIZED_GUEST) { + error_setg(errp, "No MTTCG when guest word size > hosts"); + } else { + if (!check_tcg_memory_orders_compatible()) { + error_report("Guest requires stronger MO that host"); + error_printf("Results will likely be unpredictable"); + } + mttcg_enabled = true; + } + } else if (strcmp(t, "single") == 0) { + mttcg_enabled = false; + } else { + error_setg(errp, "Invalid 'thread' setting %s", t); + } + } else { + mttcg_enabled = default_mttcg_enabled(); + } +} int64_t cpu_get_icount_raw(void) { diff --git a/include/qom/cpu.h b/include/qom/cpu.h index ca4d0fb1b4..11db2015a4 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -412,6 +412,15 @@ extern struct CPUTailQ cpus; extern __thread CPUState *current_cpu; /** + * qemu_tcg_mttcg_enabled: + * Check whether we are running MultiThread TCG or not. + * + * Returns: %true if we are in MTTCG mode %false otherwise. + */ +extern bool mttcg_enabled; +#define qemu_tcg_mttcg_enabled() (mttcg_enabled) + +/** * cpu_paging_enabled: * @cpu: The CPU whose state is to be inspected. * diff --git a/include/sysemu/cpus.h b/include/sysemu/cpus.h index 3728a1ea7e..a73b5d4bce 100644 --- a/include/sysemu/cpus.h +++ b/include/sysemu/cpus.h @@ -36,4 +36,6 @@ extern int smp_threads; void list_cpus(FILE *f, fprintf_function cpu_fprintf, const char *optarg); +void qemu_tcg_configure(QemuOpts *opts, Error **errp); + #endif diff --git a/qemu-options.hx b/qemu-options.hx index 588e5beab3..47ac537285 100644 --- a/qemu-options.hx +++ b/qemu-options.hx @@ -95,6 +95,26 @@ STEXI Select CPU model (@code{-cpu help} for list and additional feature selection) ETEXI +DEF("accel", HAS_ARG, QEMU_OPTION_accel, + "-accel [accel=]accelerator[,thread=single|multi]\n" + " select accelerator ('-accel help for list')\n" + " thread=single|multi (enable multi-threaded TCG)", QEMU_ARCH_ALL) +STEXI +@item -accel @var{name}[,prop=@var{value}[,...]] +@findex -accel +This is used to enable an accelerator. Depending on the target architecture, +kvm, xen, or tcg can be available. By default, tcg is used. If there is more +than one accelerator specified, the next one is used if the previous one fails +to initialize. +@table @option +@item thread=single|multi +Controls number of TCG threads. When the TCG is multi-threaded there will be one +thread per vCPU therefor taking advantage of additional host cores. The default +is to enable multi-threading where both the back-end and front-ends support it and +no incompatible TCG features have been enabled (e.g. icount/replay). +@end table +ETEXI + DEF("smp", HAS_ARG, QEMU_OPTION_smp, "-smp [cpus=]n[,maxcpus=cpus][,cores=cores][,threads=threads][,sockets=sockets]\n" " set the number of CPUs to 'n' [default=1]\n" diff --git a/tcg/tcg.h b/tcg/tcg.h index f946452049..4c7f258220 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -80,6 +80,15 @@ typedef uint64_t tcg_target_ulong; #error unsupported #endif +/* Oversized TCG guests make things like MTTCG hard + * as we can't use atomics for cputlb updates. + */ +#if TARGET_LONG_BITS > TCG_TARGET_REG_BITS +#define TCG_OVERSIZED_GUEST 1 +#else +#define TCG_OVERSIZED_GUEST 0 +#endif + #if TCG_TARGET_NB_REGS <= 32 typedef uint32_t TCGRegSet; #elif TCG_TARGET_NB_REGS <= 64 diff --git a/vl.c b/vl.c index 68e8c003d1..eabd7e04f8 100644 --- a/vl.c +++ b/vl.c @@ -299,6 +299,26 @@ static QemuOptsList qemu_machine_opts = { }, }; +static QemuOptsList qemu_accel_opts = { + .name = "accel", + .implied_opt_name = "accel", + .head = QTAILQ_HEAD_INITIALIZER(qemu_accel_opts.head), + .merge_lists = true, + .desc = { + { + .name = "accel", + .type = QEMU_OPT_STRING, + .help = "Select the type of accelerator", + }, + { + .name = "thread", + .type = QEMU_OPT_STRING, + .help = "Enable/disable multi-threaded TCG", + }, + { /* end of list */ } + }, +}; + static QemuOptsList qemu_boot_opts = { .name = "boot-opts", .implied_opt_name = "order", @@ -2973,7 +2993,8 @@ int main(int argc, char **argv, char **envp) const char *boot_once = NULL; DisplayState *ds; int cyls, heads, secs, translation; - QemuOpts *hda_opts = NULL, *opts, *machine_opts, *icount_opts = NULL; + QemuOpts *opts, *machine_opts; + QemuOpts *hda_opts = NULL, *icount_opts = NULL, *accel_opts = NULL; QemuOptsList *olist; int optind; const char *optarg; @@ -3028,6 +3049,7 @@ int main(int argc, char **argv, char **envp) qemu_add_opts(&qemu_trace_opts); qemu_add_opts(&qemu_option_rom_opts); qemu_add_opts(&qemu_machine_opts); + qemu_add_opts(&qemu_accel_opts); qemu_add_opts(&qemu_mem_opts); qemu_add_opts(&qemu_smp_opts); qemu_add_opts(&qemu_boot_opts); @@ -3723,6 +3745,26 @@ int main(int argc, char **argv, char **envp) qdev_prop_register_global(&kvm_pit_lost_tick_policy); break; } + case QEMU_OPTION_accel: + accel_opts = qemu_opts_parse_noisily(qemu_find_opts("accel"), + optarg, true); + optarg = qemu_opt_get(accel_opts, "accel"); + + olist = qemu_find_opts("machine"); + if (strcmp("kvm", optarg) == 0) { + qemu_opts_parse_noisily(olist, "accel=kvm", false); + } else if (strcmp("xen", optarg) == 0) { + qemu_opts_parse_noisily(olist, "accel=xen", false); + } else if (strcmp("tcg", optarg) == 0) { + qemu_opts_parse_noisily(olist, "accel=tcg", false); + } else { + if (!is_help_option(optarg)) { + error_printf("Unknown accelerator: %s", optarg); + } + error_printf("Supported accelerators: kvm, xen, tcg\n"); + exit(1); + } + break; case QEMU_OPTION_usb: olist = qemu_find_opts("machine"); qemu_opts_parse_noisily(olist, "usb=on", false); @@ -4031,6 +4073,8 @@ int main(int argc, char **argv, char **envp) replay_configure(icount_opts); + qemu_tcg_configure(accel_opts, &error_fatal); + machine_class = select_machine(); set_memory_options(&ram_slots, &maxram_size, machine_class); @@ -4395,6 +4439,9 @@ int main(int argc, char **argv, char **envp) if (!tcg_enabled()) { error_report("-icount is not allowed with hardware virtualization"); exit(1); + } else if (qemu_tcg_mttcg_enabled()) { + error_report("-icount does not currently work with MTTCG"); + exit(1); } configure_icount(icount_opts, &error_abort); qemu_opts_del(icount_opts); From patchwork Fri Jan 27 10:34:46 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92586 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp138601obz; Fri, 27 Jan 2017 02:38:02 -0800 (PST) X-Received: by 10.55.182.65 with SMTP id g62mr7637782qkf.6.1485513482350; Fri, 27 Jan 2017 02:38:02 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id w204si3220507qka.302.2017.01.27.02.38.01 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:38:02 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44224 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3up-000133-PD for patch@linaro.org; Fri, 27 Jan 2017 05:37:59 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47901) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sC-0007qR-HU for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sB-0002AR-Kv for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:16 -0500 Received: from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:36043) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sB-00029y-G9 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:15 -0500 Received: by mail-wm0-x235.google.com with SMTP id c85so110457304wmi.1 for ; Fri, 27 Jan 2017 02:35:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZI58x1tTKAUx1hXutBDgrwqAewo271LU8BBdXl8gsgc=; b=Xo2BdkmTfKRLqaMCF7Bw8XpTzTN+rb6dp4CaFqq05k1w5oNDL4cY/r0zD7C09mvlD1 aQH0eFxvRCkJCpVmtQC61SUbIT71YihzUBWrxLwILOGs95JN0c7ShHdLN/OZucyDDRWZ xeU2lmbCARCQcVJaybi6OkGxUZBHEEbPjf4nI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZI58x1tTKAUx1hXutBDgrwqAewo271LU8BBdXl8gsgc=; b=NX8ywo3paqwZyMuGtWFSoEg4ezNd8Ft01C4BqJVy8ZzqMy03MFu2C3fGQSlm/XKWIs 5mM27jr0zRRd5GemfmCT7jwu1Zx3EPxmgCGc9oKczMGAfxoov5BMfn0Yoq4QA4QVOEtV 7nslgTtO7DzHNFr0FZBPTKLkdhx5zTvAHuWyFJqz5RmNYk5UD0pWSMi9kv0GJjC3ikV9 1rFYd336WvlezKtuicQVdqJQWkWepRt7OqIbPs16mo8sBXQ2Tezd9IXnWkIojtCD+K95 C0Iw0yuJt6D1FKP+gTitTSPqfhhRT3+zkM40vDQmBBVknYncUZPauTIXqFo3FgYo1dxO 9lBA== X-Gm-Message-State: AIkVDXJLxGrsAnmWeeWaM4p28z68rwab7twCgvVAxlv1Kw3HPFmcJWsObVopLpEvZIOMnBVl X-Received: by 10.28.169.135 with SMTP id s129mr2385987wme.24.1485513314450; Fri, 27 Jan 2017 02:35:14 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g75sm3142922wme.5.2017.01.27.02.35.07 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:09 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id F03873E3745; Fri, 27 Jan 2017 10:35:05 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:46 +0000 Message-Id: <20170127103505.18606-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::235 Subject: [Qemu-devel] [PATCH v8 06/25] tcg: add kick timer for single-threaded vCPU emulation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Currently we rely on the side effect of the main loop grabbing the iothread_mutex to give any long running basic block chains a kick to ensure the next vCPU is scheduled. As this code is being re-factored and rationalised we now do it explicitly here. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v2 - re-base fixes - get_ticks_per_sec() -> NANOSECONDS_PER_SEC v3 - add define for TCG_KICK_FREQ - fix checkpatch warning v4 - wrap next calc in inline qemu_tcg_next_kick() instead of macro v5 - move all kick code into own section - use global for timer - add helper functions to start/stop timer - stop timer when all cores paused v7 - checkpatch > 80 char fix --- cpus.c | 61 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) -- 2.11.0 diff --git a/cpus.c b/cpus.c index 76b6e04332..a98925105c 100644 --- a/cpus.c +++ b/cpus.c @@ -767,6 +767,53 @@ void configure_icount(QemuOpts *opts, Error **errp) } /***********************************************************/ +/* TCG vCPU kick timer + * + * The kick timer is responsible for moving single threaded vCPU + * emulation on to the next vCPU. If more than one vCPU is running a + * timer event with force a cpu->exit so the next vCPU can get + * scheduled. + * + * The timer is removed if all vCPUs are idle and restarted again once + * idleness is complete. + */ + +static QEMUTimer *tcg_kick_vcpu_timer; + +static void qemu_cpu_kick_no_halt(void); + +#define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10) + +static inline int64_t qemu_tcg_next_kick(void) +{ + return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD; +} + +static void kick_tcg_thread(void *opaque) +{ + timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick()); + qemu_cpu_kick_no_halt(); +} + +static void start_tcg_kick_timer(void) +{ + if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) { + tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, + kick_tcg_thread, NULL); + timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick()); + } +} + +static void stop_tcg_kick_timer(void) +{ + if (tcg_kick_vcpu_timer) { + timer_del(tcg_kick_vcpu_timer); + tcg_kick_vcpu_timer = NULL; + } +} + + +/***********************************************************/ void hw_error(const char *fmt, ...) { va_list ap; @@ -1020,9 +1067,12 @@ static void qemu_wait_io_event_common(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { while (all_cpu_threads_idle()) { + stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); } + start_tcg_kick_timer(); + while (iothread_requesting_mutex) { qemu_cond_wait(&qemu_io_proceeded_cond, &qemu_global_mutex); } @@ -1222,6 +1272,15 @@ static void deal_with_unplugged_cpus(void) } } +/* Single-threaded TCG + * + * In the single-threaded case each vCPU is simulated in turn. If + * there is more than a single vCPU we create a simple timer to kick + * the vCPU and ensure we don't get stuck in a tight loop in one vCPU. + * This is done explicitly rather than relying on side-effects + * elsewhere. + */ + static void *qemu_tcg_cpu_thread_fn(void *arg) { CPUState *cpu = arg; @@ -1248,6 +1307,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) } } + start_tcg_kick_timer(); + /* process any pending work */ atomic_mb_set(&exit_request, 1); From patchwork Fri Jan 27 10:34:47 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92593 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp141451obz; Fri, 27 Jan 2017 02:47:05 -0800 (PST) X-Received: by 10.55.155.88 with SMTP id d85mr7398754qke.90.1485514025807; Fri, 27 Jan 2017 02:47:05 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t25si3259063qkt.36.2017.01.27.02.47.05 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:47:05 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44268 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX43b-00018E-5X for patch@linaro.org; Fri, 27 Jan 2017 05:47:03 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47937) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sE-0007qT-IT for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sD-0002BW-Db for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:18 -0500 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:37258) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sD-0002Ar-56 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:17 -0500 Received: by mail-wm0-x22b.google.com with SMTP id c206so129999731wme.0 for ; Fri, 27 Jan 2017 02:35:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=6ke/+PrlBWT7mDFDKNOItj6FUDXHPfUdOHagOiF1agk=; b=SNWWOoseZcPtXTdgQHppk6Bi23yf3qkGpZQuFG3Cw0MkxBnjTM+cHLYeSF5Fpwk/pS hXq+kijYt0335BcRUXsvWMYy5boLPP7WNMxVzsDicp3fNTDo+5nsBWfvvwbN7/61ScaL LT7Wk0QsasUuz/V9Wt/P5esT1O4wddwx5L5Fs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=6ke/+PrlBWT7mDFDKNOItj6FUDXHPfUdOHagOiF1agk=; b=g+b8xgOfH816uDGU7+cvM++KanWs8DU7B3nZNT/OPgFC07sHv3jMc6GEBWx5q+5dGv jYoWjCrDd3x6P82KUCSuceCRx6VVgyi980PFNlleOMnjlx8z5RL2F1Xeedz16Co+OF49 pY1kAWAJ3wbtsecDKbsW4FbY356QVu1nG1RPQ5bID4/Kcd8ZbEhul3h7b7Y3HFmgHa9/ 717lHDoTgZTaTM/CF5uHSU9wYdt+1tnUcuFDF+V9s5R8NVuoFQtx4sEZpP9k95Vl8ARr 5HB3fGXWwhJT/5VZ9lq/vpu+vQsjsnf5koqL9Doq2hqwIWRvvQ9NmWUPw7WeUvuOeu3t S5ZA== X-Gm-Message-State: AIkVDXIfKJXgSFoAevmH0p6iONUBCQhHTFR6VYjjNUeBvZw4MB/atExTyCLNjoMwTxdqqLzy X-Received: by 10.28.58.204 with SMTP id h195mr2443413wma.116.1485513316003; Fri, 27 Jan 2017 02:35:16 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 186sm3075202wmw.24.2017.01.27.02.35.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:11 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 0F4E63E374A; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:47 +0000 Message-Id: <20170127103505.18606-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22b Subject: [Qemu-devel] [PATCH v8 07/25] tcg: rename tcg_current_cpu to tcg_current_rr_cpu X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" ..and make the definition local to cpus. In preparation for MTTCG the concept of a global tcg_current_cpu will no longer make sense. However we still need to keep track of it in the single-threaded case to be able to exit quickly when required. qemu_cpu_kick_no_halt() moves and becomes qemu_cpu_kick_rr_cpu() to emphasise its use-case. qemu_cpu_kick now kicks the relevant cpu as well as qemu_kick_rr_cpu() which will become a no-op in MTTCG. For the time being the setting of the global exit_request remains. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v4: - keep global exit_request setting for now - fix merge conflicts v5: - merge conflicts with kick changes --- cpu-exec-common.c | 1 - cpu-exec.c | 3 --- cpus.c | 41 ++++++++++++++++++++++------------------- include/exec/exec-all.h | 1 - 4 files changed, 22 insertions(+), 24 deletions(-) -- 2.11.0 diff --git a/cpu-exec-common.c b/cpu-exec-common.c index 767d9c6f0c..e2bc053372 100644 --- a/cpu-exec-common.c +++ b/cpu-exec-common.c @@ -24,7 +24,6 @@ #include "exec/memory-internal.h" bool exit_request; -CPUState *tcg_current_cpu; /* exit the current TB, but without causing any exception to be raised */ void cpu_loop_exit_noexc(CPUState *cpu) diff --git a/cpu-exec.c b/cpu-exec.c index 1b8685dc21..f9e836c8dd 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -609,7 +609,6 @@ int cpu_exec(CPUState *cpu) return EXCP_HALTED; } - atomic_mb_set(&tcg_current_cpu, cpu); rcu_read_lock(); if (unlikely(atomic_mb_read(&exit_request))) { @@ -668,7 +667,5 @@ int cpu_exec(CPUState *cpu) /* fail safe : never use current_cpu outside cpu_exec() */ current_cpu = NULL; - /* Does not need atomic_mb_set because a spurious wakeup is okay. */ - atomic_set(&tcg_current_cpu, NULL); return ret; } diff --git a/cpus.c b/cpus.c index a98925105c..6d64199831 100644 --- a/cpus.c +++ b/cpus.c @@ -779,8 +779,7 @@ void configure_icount(QemuOpts *opts, Error **errp) */ static QEMUTimer *tcg_kick_vcpu_timer; - -static void qemu_cpu_kick_no_halt(void); +static CPUState *tcg_current_rr_cpu; #define TCG_KICK_PERIOD (NANOSECONDS_PER_SECOND / 10) @@ -789,10 +788,23 @@ static inline int64_t qemu_tcg_next_kick(void) return qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) + TCG_KICK_PERIOD; } +/* Kick the currently round-robin scheduled vCPU */ +static void qemu_cpu_kick_rr_cpu(void) +{ + CPUState *cpu; + atomic_mb_set(&exit_request, 1); + do { + cpu = atomic_mb_read(&tcg_current_rr_cpu); + if (cpu) { + cpu_exit(cpu); + } + } while (cpu != atomic_mb_read(&tcg_current_rr_cpu)); +} + static void kick_tcg_thread(void *opaque) { timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick()); - qemu_cpu_kick_no_halt(); + qemu_cpu_kick_rr_cpu(); } static void start_tcg_kick_timer(void) @@ -812,7 +824,6 @@ static void stop_tcg_kick_timer(void) } } - /***********************************************************/ void hw_error(const char *fmt, ...) { @@ -1323,6 +1334,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) } for (; cpu != NULL && !exit_request; cpu = CPU_NEXT(cpu)) { + atomic_mb_set(&tcg_current_rr_cpu, cpu); qemu_clock_enable(QEMU_CLOCK_VIRTUAL, (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0); @@ -1342,6 +1354,8 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) } } /* for cpu.. */ + /* Does not need atomic_mb_set because a spurious wakeup is okay. */ + atomic_set(&tcg_current_rr_cpu, NULL); /* Pairs with smp_wmb in qemu_cpu_kick. */ atomic_mb_set(&exit_request, 0); @@ -1420,24 +1434,13 @@ static void qemu_cpu_kick_thread(CPUState *cpu) #endif } -static void qemu_cpu_kick_no_halt(void) -{ - CPUState *cpu; - /* Ensure whatever caused the exit has reached the CPU threads before - * writing exit_request. - */ - atomic_mb_set(&exit_request, 1); - cpu = atomic_mb_read(&tcg_current_cpu); - if (cpu) { - cpu_exit(cpu); - } -} - void qemu_cpu_kick(CPUState *cpu) { qemu_cond_broadcast(cpu->halt_cond); if (tcg_enabled()) { - qemu_cpu_kick_no_halt(); + cpu_exit(cpu); + /* Also ensure current RR cpu is kicked */ + qemu_cpu_kick_rr_cpu(); } else { if (hax_enabled()) { /* @@ -1485,7 +1488,7 @@ void qemu_mutex_lock_iothread(void) atomic_dec(&iothread_requesting_mutex); } else { if (qemu_mutex_trylock(&qemu_global_mutex)) { - qemu_cpu_kick_no_halt(); + qemu_cpu_kick_rr_cpu(); qemu_mutex_lock(&qemu_global_mutex); } atomic_dec(&iothread_requesting_mutex); diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index bbc9478a50..3cbd359dd7 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -404,7 +404,6 @@ bool memory_region_is_unassigned(MemoryRegion *mr); extern int singlestep; /* cpu-exec.c, accessed with atomic_mb_read/atomic_mb_set */ -extern CPUState *tcg_current_cpu; extern bool exit_request; #endif From patchwork Fri Jan 27 10:34:48 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92588 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp138860obz; Fri, 27 Jan 2017 02:38:48 -0800 (PST) X-Received: by 10.55.209.157 with SMTP id o29mr7203376qkl.154.1485513528239; Fri, 27 Jan 2017 02:38:48 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id i50si3245933qta.73.2017.01.27.02.38.47 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:38:48 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44226 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3vZ-0001tT-7E for patch@linaro.org; Fri, 27 Jan 2017 05:38:45 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47968) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sH-0007sO-EE for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sE-0002C6-Kz for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:21 -0500 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:38494) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sE-0002Bg-84 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:18 -0500 Received: by mail-wm0-x231.google.com with SMTP id r144so129831062wme.1 for ; Fri, 27 Jan 2017 02:35:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xHvyzn82dYDpng+iumgPX7m0T5v9O+kfarbaqnZD84A=; b=P3goHR6K7BEuJxsGKDJxWgOzYrzKRCi5CV38t/S916YX45KFdI0tc0TCiF9iDpFiLY fhtDp66bw7LWD8fLug91QV9xL8yaSqcOkriF5e66+6S3warNkSL2eS4neKF/I00PcrG8 W1g9xK9W1H6r9P3lxclwkCAu11SoKrQwNo/Fk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xHvyzn82dYDpng+iumgPX7m0T5v9O+kfarbaqnZD84A=; b=XG2Z0EhALpUoKJbU21k30nqam3GVF4VsC/ywfsZiY4JeR6IX+fGeOZ0bUJOEd2fTQZ K0VQTysvXiDW8gMh5VauGFRnb7QklB0IHuKhT8aYZn4lK+/xyvpJTsyhkDUDIWDD1BAN kiuG297F+WHqa+iXxAD/js5V+4JWdv5MkppRTGzaJOJgP3qTwcKgbgm3ioRQXiSZ59LV f9UZdbLkPpWxZysT2ZtfiVrhR31l3aL3+7v+fLwhwe8vVNysMTcRlsolta6uF2tR3a45 LRTK3aRJA8/fWA1ZCRUmpnXWBsRfuZj7glfEmHJVwhszmOsD6tGODIFRZnPb94M0CyYG WrrA== X-Gm-Message-State: AIkVDXJFNNRZq9+Hya/KEJj4NrFe6wAMKZPm0kVngIW8uFi9tNY6Xyo7NimPD2jt3b9E0Ql+ X-Received: by 10.223.147.100 with SMTP id 91mr6743955wro.60.1485513316969; Fri, 27 Jan 2017 02:35:16 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id w70sm7112903wrc.47.2017.01.27.02.35.08 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:11 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 31B803E374F; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:48 +0000 Message-Id: <20170127103505.18606-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [PATCH v8 08/25] tcg: drop global lock during TCG code execution X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Eduardo Habkost , Peter Crosthwaite , Jan Kiszka , "Michael S. Tsirkin" , Alexander Graf , "open list:Overall" , "Emilio G . Cota" , "open list:PowerPC" , David Gibson , Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:ARM cores" , KONRAD Frederic Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Jan Kiszka This finally allows TCG to benefit from the iothread introduction: Drop the global mutex while running pure TCG CPU code. Reacquire the lock when entering MMIO or PIO emulation, or when leaving the TCG loop. We have to revert a few optimization for the current TCG threading model, namely kicking the TCG thread in qemu_mutex_lock_iothread and not kicking it in qemu_cpu_kick. We also need to disable RAM block reordering until we have a more efficient locking mechanism at hand. Still, a Linux x86 UP guest and my Musicpal ARM model boot fine here. These numbers demonstrate where we gain something: 20338 jan 20 0 331m 75m 6904 R 99 0.9 0:50.95 qemu-system-arm 20337 jan 20 0 331m 75m 6904 S 20 0.9 0:26.50 qemu-system-arm The guest CPU was fully loaded, but the iothread could still run mostly independent on a second core. Without the patch we don't get beyond 32206 jan 20 0 330m 73m 7036 R 82 0.9 1:06.00 qemu-system-arm 32204 jan 20 0 330m 73m 7036 S 21 0.9 0:17.03 qemu-system-arm We don't benefit significantly, though, when the guest is not fully loading a host CPU. Signed-off-by: Jan Kiszka Message-Id: <1439220437-23957-10-git-send-email-fred.konrad@greensocs.com> [FK: Rebase, fix qemu_devices_reset deadlock, rm address_space_* mutex] Signed-off-by: KONRAD Frederic [EGC: fixed iothread lock for cpu-exec IRQ handling] Signed-off-by: Emilio G. Cota [AJB: -smp single-threaded fix, clean commit msg, BQL fixes] Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v8: - merged in BQL fixes for PPC target: ppc_set_irq - merged in BQL fixes for ARM target: ARM_CP_IO helpers - merged in BQL fixes for ARM target: arm_call_el_change_hook v5 (ajb, base patches): - added an assert to BQL unlock/lock functions instead of hanging - ensure all cpu->interrupt_requests *modifications* protected by BQL - add a re-read on cpu->interrupt_request for correctness - BQL fixes for: - assert BQL held for PPC hypercalls (emulate_spar_hypercall) - SCLP service calls on s390x - merge conflict with kick timer patch v4 (ajb, base patches): - protect cpu->interrupt updates with BQL - fix wording io_mem_notdirty calls - s/we/with/ v3 (ajb, base-patches): - stale iothread_unlocks removed (cpu_exit/resume_from_signal deals with it in the longjmp). - fix re-base conflicts v2 (ajb): - merge with tcg: grab iothread lock in cpu-exec interrupt handling - use existing fns for tracking lock state - lock iothread for mem_region - add assert on mem region modification - ensure smm_helper holds iothread - Add JK s-o-b - Fix-up FK s-o-b annotation v1 (ajb, base-patches): - SMP failure now fixed by previous commit Changes from Fred Konrad (mttcg-v7 via paolo): * Rebase on the current HEAD. * Fixes a deadlock in qemu_devices_reset(). * Remove the mutex in address_space_* --- cpu-exec.c | 20 ++++++++++++++++++-- cpus.c | 28 +++++----------------------- cputlb.c | 21 ++++++++++++++++++++- exec.c | 12 +++++++++--- hw/core/irq.c | 1 + hw/i386/kvmvapic.c | 4 ++-- hw/intc/arm_gicv3_cpuif.c | 3 +++ hw/ppc/ppc.c | 16 +++++++++++++++- hw/ppc/spapr.c | 3 +++ include/qom/cpu.h | 1 + memory.c | 2 ++ qom/cpu.c | 10 ++++++++++ target/arm/helper.c | 6 ++++++ target/arm/op_helper.c | 43 +++++++++++++++++++++++++++++++++++++++---- target/i386/smm_helper.c | 7 +++++++ target/s390x/misc_helper.c | 5 ++++- translate-all.c | 9 +++++++-- translate-common.c | 21 +++++++++++---------- 18 files changed, 163 insertions(+), 49 deletions(-) -- 2.11.0 diff --git a/cpu-exec.c b/cpu-exec.c index f9e836c8dd..f42a128bdf 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -29,6 +29,7 @@ #include "qemu/rcu.h" #include "exec/tb-hash.h" #include "exec/log.h" +#include "qemu/main-loop.h" #if defined(TARGET_I386) && !defined(CONFIG_USER_ONLY) #include "hw/i386/apic.h" #endif @@ -388,8 +389,10 @@ static inline bool cpu_handle_halt(CPUState *cpu) if ((cpu->interrupt_request & CPU_INTERRUPT_POLL) && replay_interrupt()) { X86CPU *x86_cpu = X86_CPU(cpu); + qemu_mutex_lock_iothread(); apic_poll_irq(x86_cpu->apic_state); cpu_reset_interrupt(cpu, CPU_INTERRUPT_POLL); + qemu_mutex_unlock_iothread(); } #endif if (!cpu_has_work(cpu)) { @@ -443,7 +446,9 @@ static inline bool cpu_handle_exception(CPUState *cpu, int *ret) #else if (replay_exception()) { CPUClass *cc = CPU_GET_CLASS(cpu); + qemu_mutex_lock_iothread(); cc->do_interrupt(cpu); + qemu_mutex_unlock_iothread(); cpu->exception_index = -1; } else if (!replay_has_interrupt()) { /* give a chance to iothread in replay mode */ @@ -469,9 +474,11 @@ static inline void cpu_handle_interrupt(CPUState *cpu, TranslationBlock **last_tb) { CPUClass *cc = CPU_GET_CLASS(cpu); - int interrupt_request = cpu->interrupt_request; - if (unlikely(interrupt_request)) { + if (unlikely(atomic_read(&cpu->interrupt_request))) { + int interrupt_request; + qemu_mutex_lock_iothread(); + interrupt_request = cpu->interrupt_request; if (unlikely(cpu->singlestep_enabled & SSTEP_NOIRQ)) { /* Mask out external interrupts for this step. */ interrupt_request &= ~CPU_INTERRUPT_SSTEP_MASK; @@ -526,7 +533,12 @@ static inline void cpu_handle_interrupt(CPUState *cpu, the program flow was changed */ *last_tb = NULL; } + + /* If we exit via cpu_loop_exit/longjmp it is reset in cpu_exec */ + qemu_mutex_unlock_iothread(); } + + if (unlikely(atomic_read(&cpu->exit_request) || replay_has_interrupt())) { atomic_set(&cpu->exit_request, 0); cpu->exception_index = EXCP_INTERRUPT; @@ -656,8 +668,12 @@ int cpu_exec(CPUState *cpu) g_assert(cpu == current_cpu); g_assert(cc == CPU_GET_CLASS(cpu)); #endif /* buggy compiler */ + cpu->can_do_io = 1; tb_lock_reset(); + if (qemu_mutex_iothread_locked()) { + qemu_mutex_unlock_iothread(); + } } } /* for(;;) */ diff --git a/cpus.c b/cpus.c index 6d64199831..c48bc8d5b3 100644 --- a/cpus.c +++ b/cpus.c @@ -1026,8 +1026,6 @@ static void qemu_kvm_init_cpu_signals(CPUState *cpu) #endif /* _WIN32 */ static QemuMutex qemu_global_mutex; -static QemuCond qemu_io_proceeded_cond; -static unsigned iothread_requesting_mutex; static QemuThread io_thread; @@ -1041,7 +1039,6 @@ void qemu_init_cpu_loop(void) qemu_init_sigbus(); qemu_cond_init(&qemu_cpu_cond); qemu_cond_init(&qemu_pause_cond); - qemu_cond_init(&qemu_io_proceeded_cond); qemu_mutex_init(&qemu_global_mutex); qemu_thread_get_self(&io_thread); @@ -1084,10 +1081,6 @@ static void qemu_tcg_wait_io_event(CPUState *cpu) start_tcg_kick_timer(); - while (iothread_requesting_mutex) { - qemu_cond_wait(&qemu_io_proceeded_cond, &qemu_global_mutex); - } - CPU_FOREACH(cpu) { qemu_wait_io_event_common(cpu); } @@ -1248,9 +1241,11 @@ static int tcg_cpu_exec(CPUState *cpu) cpu->icount_decr.u16.low = decr; cpu->icount_extra = count; } + qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); ret = cpu_exec(cpu); cpu_exec_end(cpu); + qemu_mutex_lock_iothread(); #ifdef CONFIG_PROFILER tcg_time += profile_getclock() - ti; #endif @@ -1478,27 +1473,14 @@ bool qemu_mutex_iothread_locked(void) void qemu_mutex_lock_iothread(void) { - atomic_inc(&iothread_requesting_mutex); - /* In the simple case there is no need to bump the VCPU thread out of - * TCG code execution. - */ - if (!tcg_enabled() || qemu_in_vcpu_thread() || - !first_cpu || !first_cpu->created) { - qemu_mutex_lock(&qemu_global_mutex); - atomic_dec(&iothread_requesting_mutex); - } else { - if (qemu_mutex_trylock(&qemu_global_mutex)) { - qemu_cpu_kick_rr_cpu(); - qemu_mutex_lock(&qemu_global_mutex); - } - atomic_dec(&iothread_requesting_mutex); - qemu_cond_broadcast(&qemu_io_proceeded_cond); - } + g_assert(!qemu_mutex_iothread_locked()); + qemu_mutex_lock(&qemu_global_mutex); iothread_locked = true; } void qemu_mutex_unlock_iothread(void) { + g_assert(qemu_mutex_iothread_locked()); iothread_locked = false; qemu_mutex_unlock(&qemu_global_mutex); } diff --git a/cputlb.c b/cputlb.c index 6c39927455..1cc9d9da51 100644 --- a/cputlb.c +++ b/cputlb.c @@ -18,6 +18,7 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" #include "cpu.h" #include "exec/exec-all.h" #include "exec/memory.h" @@ -495,6 +496,7 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, hwaddr physaddr = iotlbentry->addr; MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); uint64_t val; + bool locked = false; physaddr = (physaddr & TARGET_PAGE_MASK) + addr; cpu->mem_io_pc = retaddr; @@ -503,7 +505,16 @@ static uint64_t io_readx(CPUArchState *env, CPUIOTLBEntry *iotlbentry, } cpu->mem_io_vaddr = addr; + + if (mr->global_locking) { + qemu_mutex_lock_iothread(); + locked = true; + } memory_region_dispatch_read(mr, physaddr, &val, size, iotlbentry->attrs); + if (locked) { + qemu_mutex_unlock_iothread(); + } + return val; } @@ -514,15 +525,23 @@ static void io_writex(CPUArchState *env, CPUIOTLBEntry *iotlbentry, CPUState *cpu = ENV_GET_CPU(env); hwaddr physaddr = iotlbentry->addr; MemoryRegion *mr = iotlb_to_region(cpu, physaddr, iotlbentry->attrs); + bool locked = false; physaddr = (physaddr & TARGET_PAGE_MASK) + addr; if (mr != &io_mem_rom && mr != &io_mem_notdirty && !cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } - cpu->mem_io_vaddr = addr; cpu->mem_io_pc = retaddr; + + if (mr->global_locking) { + qemu_mutex_lock_iothread(); + locked = true; + } memory_region_dispatch_write(mr, physaddr, val, size, iotlbentry->attrs); + if (locked) { + qemu_mutex_unlock_iothread(); + } } /* Return true if ADDR is present in the victim tlb, and has been copied diff --git a/exec.c b/exec.c index f2bed92b64..87cf0db91e 100644 --- a/exec.c +++ b/exec.c @@ -2133,9 +2133,9 @@ static void check_watchpoint(int offset, int len, MemTxAttrs attrs, int flags) } cpu->watchpoint_hit = wp; - /* The tb_lock will be reset when cpu_loop_exit or - * cpu_loop_exit_noexc longjmp back into the cpu_exec - * main loop. + /* Both tb_lock and iothread_mutex will be reset when + * cpu_loop_exit or cpu_loop_exit_noexc longjmp + * back into the cpu_exec main loop. */ tb_lock(); tb_check_watchpoint(cpu); @@ -2370,8 +2370,14 @@ static void io_mem_init(void) memory_region_init_io(&io_mem_rom, NULL, &unassigned_mem_ops, NULL, NULL, UINT64_MAX); memory_region_init_io(&io_mem_unassigned, NULL, &unassigned_mem_ops, NULL, NULL, UINT64_MAX); + + /* io_mem_notdirty calls tb_invalidate_phys_page_fast, + * which can be called without the iothread mutex. + */ memory_region_init_io(&io_mem_notdirty, NULL, ¬dirty_mem_ops, NULL, NULL, UINT64_MAX); + memory_region_clear_global_locking(&io_mem_notdirty); + memory_region_init_io(&io_mem_watch, NULL, &watch_mem_ops, NULL, NULL, UINT64_MAX); } diff --git a/hw/core/irq.c b/hw/core/irq.c index 49ff2e64fe..b98d1d69f5 100644 --- a/hw/core/irq.c +++ b/hw/core/irq.c @@ -22,6 +22,7 @@ * THE SOFTWARE. */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" #include "qemu-common.h" #include "hw/irq.h" #include "qom/object.h" diff --git a/hw/i386/kvmvapic.c b/hw/i386/kvmvapic.c index 702e281dc8..b3c49b2c61 100644 --- a/hw/i386/kvmvapic.c +++ b/hw/i386/kvmvapic.c @@ -451,8 +451,8 @@ static void patch_instruction(VAPICROMState *s, X86CPU *cpu, target_ulong ip) resume_all_vcpus(); if (!kvm_enabled()) { - /* tb_lock will be reset when cpu_loop_exit_noexc longjmps - * back into the cpu_exec loop. */ + /* Both tb_lock and iothread_mutex will be reset when + * longjmps back into the cpu_exec loop. */ tb_lock(); tb_gen_code(cs, current_pc, current_cs_base, current_flags, 1); cpu_loop_exit_noexc(cs); diff --git a/hw/intc/arm_gicv3_cpuif.c b/hw/intc/arm_gicv3_cpuif.c index a9ee7fddf9..2624d8d909 100644 --- a/hw/intc/arm_gicv3_cpuif.c +++ b/hw/intc/arm_gicv3_cpuif.c @@ -14,6 +14,7 @@ #include "qemu/osdep.h" #include "qemu/bitops.h" +#include "qemu/main-loop.h" #include "trace.h" #include "gicv3_internal.h" #include "cpu.h" @@ -733,6 +734,8 @@ void gicv3_cpuif_update(GICv3CPUState *cs) ARMCPU *cpu = ARM_CPU(cs->cpu); CPUARMState *env = &cpu->env; + g_assert(qemu_mutex_iothread_locked()); + trace_gicv3_cpuif_update(gicv3_redist_affid(cs), cs->hppi.irq, cs->hppi.grp, cs->hppi.prio); diff --git a/hw/ppc/ppc.c b/hw/ppc/ppc.c index 8945869009..59c3faa6c8 100644 --- a/hw/ppc/ppc.c +++ b/hw/ppc/ppc.c @@ -62,7 +62,16 @@ void ppc_set_irq(PowerPCCPU *cpu, int n_IRQ, int level) { CPUState *cs = CPU(cpu); CPUPPCState *env = &cpu->env; - unsigned int old_pending = env->pending_interrupts; + unsigned int old_pending; + bool locked = false; + + /* We may already have the BQL if coming from the reset path */ + if (!qemu_mutex_iothread_locked()) { + locked = true; + qemu_mutex_lock_iothread(); + } + + old_pending = env->pending_interrupts; if (level) { env->pending_interrupts |= 1 << n_IRQ; @@ -80,9 +89,14 @@ void ppc_set_irq(PowerPCCPU *cpu, int n_IRQ, int level) #endif } + LOG_IRQ("%s: %p n_IRQ %d level %d => pending %08" PRIx32 "req %08x\n", __func__, env, n_IRQ, level, env->pending_interrupts, CPU(cpu)->interrupt_request); + + if (locked) { + qemu_mutex_unlock_iothread(); + } } /* PowerPC 6xx / 7xx internal IRQ controller */ diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c index a642e663d4..745743d64b 100644 --- a/hw/ppc/spapr.c +++ b/hw/ppc/spapr.c @@ -1012,6 +1012,9 @@ static void emulate_spapr_hypercall(PowerPCCPU *cpu) { CPUPPCState *env = &cpu->env; + /* The TCG path should also be holding the BQL at this point */ + g_assert(qemu_mutex_iothread_locked()); + if (msr_pr) { hcall_dprintf("Hypercall made with MSR[PR]=1\n"); env->gpr[3] = H_PRIVILEGE; diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 11db2015a4..1a06ae5938 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -325,6 +325,7 @@ struct CPUState { bool unplug; bool crash_occurred; bool exit_request; + /* updates protected by BQL */ uint32_t interrupt_request; int singlestep_enabled; int64_t icount_extra; diff --git a/memory.c b/memory.c index 2bfc37f65c..7d7b285e41 100644 --- a/memory.c +++ b/memory.c @@ -917,6 +917,8 @@ void memory_region_transaction_commit(void) AddressSpace *as; assert(memory_region_transaction_depth); + assert(qemu_mutex_iothread_locked()); + --memory_region_transaction_depth; if (!memory_region_transaction_depth) { if (memory_region_update_pending) { diff --git a/qom/cpu.c b/qom/cpu.c index 7f575879f6..bd77c05cd0 100644 --- a/qom/cpu.c +++ b/qom/cpu.c @@ -113,9 +113,19 @@ static void cpu_common_get_memory_mapping(CPUState *cpu, error_setg(errp, "Obtaining memory mappings is unsupported on this CPU."); } +/* Resetting the IRQ comes from across the code base so we take the + * BQL here if we need to. cpu_interrupt assumes it is held.*/ void cpu_reset_interrupt(CPUState *cpu, int mask) { + bool need_lock = !qemu_mutex_iothread_locked(); + + if (need_lock) { + qemu_mutex_lock_iothread(); + } cpu->interrupt_request &= ~mask; + if (need_lock) { + qemu_mutex_unlock_iothread(); + } } void cpu_exit(CPUState *cpu) diff --git a/target/arm/helper.c b/target/arm/helper.c index 7111c8cf18..84d789be93 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -6693,6 +6693,12 @@ void arm_cpu_do_interrupt(CPUState *cs) arm_cpu_do_interrupt_aarch32(cs); } + /* Hooks may change global state so BQL should be held, also the + * BQL needs to be held for any modification of + * cs->interrupt_request. + */ + g_assert(qemu_mutex_iothread_locked()); + arm_call_el_change_hook(cpu); if (!kvm_enabled()) { diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index ba796d898e..e1a883c595 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -18,6 +18,7 @@ */ #include "qemu/osdep.h" #include "qemu/log.h" +#include "qemu/main-loop.h" #include "cpu.h" #include "exec/helper-proto.h" #include "internals.h" @@ -487,7 +488,9 @@ void HELPER(cpsr_write_eret)(CPUARMState *env, uint32_t val) */ env->regs[15] &= (env->thumb ? ~1 : ~3); + qemu_mutex_lock_iothread(); arm_call_el_change_hook(arm_env_get_cpu(env)); + qemu_mutex_unlock_iothread(); } /* Access to user mode registers from privileged modes. */ @@ -735,28 +738,58 @@ void HELPER(set_cp_reg)(CPUARMState *env, void *rip, uint32_t value) { const ARMCPRegInfo *ri = rip; - ri->writefn(env, ri, value); + if (ri->type & ARM_CP_IO) { + qemu_mutex_lock_iothread(); + ri->writefn(env, ri, value); + qemu_mutex_unlock_iothread(); + } else { + ri->writefn(env, ri, value); + } } uint32_t HELPER(get_cp_reg)(CPUARMState *env, void *rip) { const ARMCPRegInfo *ri = rip; + uint32_t res; - return ri->readfn(env, ri); + if (ri->type & ARM_CP_IO) { + qemu_mutex_lock_iothread(); + res = ri->readfn(env, ri); + qemu_mutex_unlock_iothread(); + } else { + res = ri->readfn(env, ri); + } + + return res; } void HELPER(set_cp_reg64)(CPUARMState *env, void *rip, uint64_t value) { const ARMCPRegInfo *ri = rip; - ri->writefn(env, ri, value); + if (ri->type & ARM_CP_IO) { + qemu_mutex_lock_iothread(); + ri->writefn(env, ri, value); + qemu_mutex_unlock_iothread(); + } else { + ri->writefn(env, ri, value); + } } uint64_t HELPER(get_cp_reg64)(CPUARMState *env, void *rip) { const ARMCPRegInfo *ri = rip; + uint64_t res; + + if (ri->type & ARM_CP_IO) { + qemu_mutex_lock_iothread(); + res = ri->readfn(env, ri); + qemu_mutex_unlock_iothread(); + } else { + res = ri->readfn(env, ri); + } - return ri->readfn(env, ri); + return res; } void HELPER(msr_i_pstate)(CPUARMState *env, uint32_t op, uint32_t imm) @@ -989,7 +1022,9 @@ void HELPER(exception_return)(CPUARMState *env) cur_el, new_el, env->pc); } + qemu_mutex_lock_iothread(); arm_call_el_change_hook(arm_env_get_cpu(env)); + qemu_mutex_unlock_iothread(); return; diff --git a/target/i386/smm_helper.c b/target/i386/smm_helper.c index 4dd6a2c544..f051a77c4a 100644 --- a/target/i386/smm_helper.c +++ b/target/i386/smm_helper.c @@ -18,6 +18,7 @@ */ #include "qemu/osdep.h" +#include "qemu/main-loop.h" #include "cpu.h" #include "exec/helper-proto.h" #include "exec/log.h" @@ -42,11 +43,14 @@ void helper_rsm(CPUX86State *env) #define SMM_REVISION_ID 0x00020000 #endif +/* Called with iothread lock taken */ void cpu_smm_update(X86CPU *cpu) { CPUX86State *env = &cpu->env; bool smm_enabled = (env->hflags & HF_SMM_MASK); + g_assert(qemu_mutex_iothread_locked()); + if (cpu->smram) { memory_region_set_enabled(cpu->smram, smm_enabled); } @@ -333,7 +337,10 @@ void helper_rsm(CPUX86State *env) } env->hflags2 &= ~HF2_SMM_INSIDE_NMI_MASK; env->hflags &= ~HF_SMM_MASK; + + qemu_mutex_lock_iothread(); cpu_smm_update(cpu); + qemu_mutex_unlock_iothread(); qemu_log_mask(CPU_LOG_INT, "SMM: after RSM\n"); log_cpu_state_mask(CPU_LOG_INT, CPU(cpu), CPU_DUMP_CCOP); diff --git a/target/s390x/misc_helper.c b/target/s390x/misc_helper.c index c9604ea9c7..3cb942e8bb 100644 --- a/target/s390x/misc_helper.c +++ b/target/s390x/misc_helper.c @@ -25,6 +25,7 @@ #include "exec/helper-proto.h" #include "sysemu/kvm.h" #include "qemu/timer.h" +#include "qemu/main-loop.h" #include "exec/address-spaces.h" #ifdef CONFIG_KVM #include @@ -109,11 +110,13 @@ void program_interrupt(CPUS390XState *env, uint32_t code, int ilen) /* SCLP service call */ uint32_t HELPER(servc)(CPUS390XState *env, uint64_t r1, uint64_t r2) { + qemu_mutex_lock_iothread(); int r = sclp_service_call(env, r1, r2); if (r < 0) { program_interrupt(env, -r, 4); - return 0; + r = 0; } + qemu_mutex_unlock_iothread(); return r; } diff --git a/translate-all.c b/translate-all.c index 055436a676..41b36f04c6 100644 --- a/translate-all.c +++ b/translate-all.c @@ -55,6 +55,7 @@ #include "translate-all.h" #include "qemu/bitmap.h" #include "qemu/timer.h" +#include "qemu/main-loop.h" #include "exec/log.h" /* #define DEBUG_TB_INVALIDATE */ @@ -1521,7 +1522,7 @@ void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end, #ifdef CONFIG_SOFTMMU /* len must be <= 8 and start must be a multiple of len. * Called via softmmu_template.h when code areas are written to with - * tb_lock held. + * iothread mutex not held. */ void tb_invalidate_phys_page_fast(tb_page_addr_t start, int len) { @@ -1723,7 +1724,10 @@ void tb_check_watchpoint(CPUState *cpu) #ifndef CONFIG_USER_ONLY /* in deterministic execution mode, instructions doing device I/Os - must be at the end of the TB */ + * must be at the end of the TB. + * + * Called by softmmu_template.h, with iothread mutex not held. + */ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr) { #if defined(TARGET_MIPS) || defined(TARGET_SH4) @@ -1935,6 +1939,7 @@ void dump_opcount_info(FILE *f, fprintf_function cpu_fprintf) void cpu_interrupt(CPUState *cpu, int mask) { + g_assert(qemu_mutex_iothread_locked()); cpu->interrupt_request |= mask; cpu->tcg_exit_req = 1; } diff --git a/translate-common.c b/translate-common.c index 5e989cdf70..d504dd0d33 100644 --- a/translate-common.c +++ b/translate-common.c @@ -21,6 +21,7 @@ #include "qemu-common.h" #include "qom/cpu.h" #include "sysemu/cpus.h" +#include "qemu/main-loop.h" uintptr_t qemu_real_host_page_size; intptr_t qemu_real_host_page_mask; @@ -30,6 +31,7 @@ intptr_t qemu_real_host_page_mask; static void tcg_handle_interrupt(CPUState *cpu, int mask) { int old_mask; + g_assert(qemu_mutex_iothread_locked()); old_mask = cpu->interrupt_request; cpu->interrupt_request |= mask; @@ -40,17 +42,16 @@ static void tcg_handle_interrupt(CPUState *cpu, int mask) */ if (!qemu_cpu_is_self(cpu)) { qemu_cpu_kick(cpu); - return; - } - - if (use_icount) { - cpu->icount_decr.u16.high = 0xffff; - if (!cpu->can_do_io - && (mask & ~old_mask) != 0) { - cpu_abort(cpu, "Raised interrupt while not in I/O function"); - } } else { - cpu->tcg_exit_req = 1; + if (use_icount) { + cpu->icount_decr.u16.high = 0xffff; + if (!cpu->can_do_io + && (mask & ~old_mask) != 0) { + cpu_abort(cpu, "Raised interrupt while not in I/O function"); + } + } else { + cpu->tcg_exit_req = 1; + } } } From patchwork Fri Jan 27 10:34:49 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92607 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp174924qgi; Fri, 27 Jan 2017 02:56:00 -0800 (PST) X-Received: by 10.200.41.3 with SMTP id y3mr6727317qty.205.1485514560694; Fri, 27 Jan 2017 02:56:00 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u66si3269011qki.107.2017.01.27.02.56.00 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:56:00 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44317 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4CE-00015a-6V for patch@linaro.org; Fri, 27 Jan 2017 05:55:58 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47977) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sI-0007tS-MS for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sG-0002Cc-Ka for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:22 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:38515) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sG-0002CR-Ce for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:20 -0500 Received: by mail-wm0-x236.google.com with SMTP id r144so129832612wme.1 for ; Fri, 27 Jan 2017 02:35:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TQfblUXwf0JxZg1iBmpt4TOHhWrXsGEbuOo5hPD+gWc=; b=YfE8hCzt+CN6svC3O5FijR5swyE2zQR7uukts/tbOSc9n4yn+Vae9IT/pfX9jPevhI Y+YsqDglXpdOHbYDz3uIcHpcbu5CGC/Vq0PKTD/ilQJHcbDP75BXhQ4+tJTeYsyCMC85 FAerNzxqMtpgxhEl2ivJiRm19lv0B7KxQVp3s= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TQfblUXwf0JxZg1iBmpt4TOHhWrXsGEbuOo5hPD+gWc=; b=FMuSdnn1u0Vm64dYOPbE7tavheFL1MLiegsE5zgUxJgPp4frMV1a77IyqYny/4KYZi QvNfpnzBVc+ndTYmcUiyOBsQ10A5djBPLP9+mKQx1F1UqF8L8JJV8CFiVGsjdN+f6nLy OCj0PW0Uf1aFobTUwHU3ET+48iEmtp5601JkOpu9/4ZEOT3mxBByjsNggoLT2kUfcENM BPiTe8MVzjO+mSPYtAxRI/bcm8PrcjtgrAZv01gnenkhDu9+PcRYBHmyCrFMNRPL3UXH KDVZuNzeHm5FqRTxsOW+o1r4qVoaZYSFsurC+DVQWmNo2+qJXC2En05Yqqgm4+05I6W0 JtIg== X-Gm-Message-State: AIkVDXLLMXH5Exj0629+Gcr7UzGIEs7xyakibwiX7K828Pvfh71lzG82d25x/RihV8qbD5Zm X-Received: by 10.28.111.78 with SMTP id k75mr2604354wmc.71.1485513319310; Fri, 27 Jan 2017 02:35:19 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id c133sm3135458wmd.13.2017.01.27.02.35.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:14 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 4544B3E3751; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:49 +0000 Message-Id: <20170127103505.18606-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 09/25] tcg: remove global exit_request X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" There are now only two uses of the global exit_request left. The first ensures we exit the run_loop when we first start to process pending work and in the kick handler. This is just as easily done by setting the first_cpu->exit_request flag. The second use is in the round robin kick routine. The global exit_request ensured every vCPU would set its local exit_request and cause a full exit of the loop. Now the iothread isn't being held while running we can just rely on the kick handler to push us out as intended. We lightly re-factor the main vCPU thread to ensure cpu->exit_requests cause us to exit the main loop and process any IO requests that might come along. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v5 - minor merge conflict with kick patch v4 - moved to after iothread unlocking patch - needed to remove kick exit_request as well. - remove extraneous cpu->exit_request check - remove stray exit_request setting - remove needless atomic operation --- cpu-exec-common.c | 2 -- cpu-exec.c | 9 ++------- cpus.c | 18 ++++++++++-------- include/exec/exec-all.h | 3 --- 4 files changed, 12 insertions(+), 20 deletions(-) -- 2.11.0 diff --git a/cpu-exec-common.c b/cpu-exec-common.c index e2bc053372..0504a9457b 100644 --- a/cpu-exec-common.c +++ b/cpu-exec-common.c @@ -23,8 +23,6 @@ #include "exec/exec-all.h" #include "exec/memory-internal.h" -bool exit_request; - /* exit the current TB, but without causing any exception to be raised */ void cpu_loop_exit_noexc(CPUState *cpu) { diff --git a/cpu-exec.c b/cpu-exec.c index f42a128bdf..cc09c1fc37 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -565,9 +565,8 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb, /* Something asked us to stop executing * chained TBs; just continue round the main * loop. Whatever requested the exit will also - * have set something else (eg exit_request or - * interrupt_request) which we will handle - * next time around the loop. But we need to + * have set something else (eg interrupt_request) which we + * will handle next time around the loop. But we need to * ensure the tcg_exit_req read in generated code * comes before the next read of cpu->exit_request * or cpu->interrupt_request. @@ -623,10 +622,6 @@ int cpu_exec(CPUState *cpu) rcu_read_lock(); - if (unlikely(atomic_mb_read(&exit_request))) { - cpu->exit_request = 1; - } - cc->cpu_exec_enter(cpu); /* Calculate difference between guest clock and host clock. diff --git a/cpus.c b/cpus.c index c48bc8d5b3..18daf41dae 100644 --- a/cpus.c +++ b/cpus.c @@ -792,7 +792,6 @@ static inline int64_t qemu_tcg_next_kick(void) static void qemu_cpu_kick_rr_cpu(void) { CPUState *cpu; - atomic_mb_set(&exit_request, 1); do { cpu = atomic_mb_read(&tcg_current_rr_cpu); if (cpu) { @@ -1315,11 +1314,11 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) start_tcg_kick_timer(); - /* process any pending work */ - atomic_mb_set(&exit_request, 1); - cpu = first_cpu; + /* process any pending work */ + cpu->exit_request = 1; + while (1) { /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ qemu_account_warp_timer(); @@ -1328,7 +1327,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) cpu = first_cpu; } - for (; cpu != NULL && !exit_request; cpu = CPU_NEXT(cpu)) { + while (cpu && !cpu->exit_request) { atomic_mb_set(&tcg_current_rr_cpu, cpu); qemu_clock_enable(QEMU_CLOCK_VIRTUAL, @@ -1348,12 +1347,15 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) break; } - } /* for cpu.. */ + cpu = CPU_NEXT(cpu); + } /* while (cpu && !cpu->exit_request).. */ + /* Does not need atomic_mb_set because a spurious wakeup is okay. */ atomic_set(&tcg_current_rr_cpu, NULL); - /* Pairs with smp_wmb in qemu_cpu_kick. */ - atomic_mb_set(&exit_request, 0); + if (cpu && cpu->exit_request) { + atomic_mb_set(&cpu->exit_request, 0); + } handle_icount_deadline(); diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index 3cbd359dd7..bd4622ac5d 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -403,7 +403,4 @@ bool memory_region_is_unassigned(MemoryRegion *mr); /* vl.c */ extern int singlestep; -/* cpu-exec.c, accessed with atomic_mb_read/atomic_mb_set */ -extern bool exit_request; - #endif From patchwork Fri Jan 27 10:34:50 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92603 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp143016obz; Fri, 27 Jan 2017 02:51:39 -0800 (PST) X-Received: by 10.55.26.159 with SMTP id l31mr7622925qkh.164.1485514299212; Fri, 27 Jan 2017 02:51:39 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id r30si3266577qkr.104.2017.01.27.02.51.38 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:51:39 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44293 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX480-0005kD-IH for patch@linaro.org; Fri, 27 Jan 2017 05:51:36 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:47976) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sI-0007tR-MO for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sF-0002CK-KT for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:22 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:37279) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sF-0002CG-ER for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:19 -0500 Received: by mail-wm0-x236.google.com with SMTP id c206so130001436wme.0 for ; Fri, 27 Jan 2017 02:35:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zHeDdh7LS/qfaOBi0Wrx/3dwOaXRkrY7AoKsyWB2WWc=; b=UZg4ABG8ocEwXHci+ip2YAb7EqwzW3WgPUvjkRA5ZXCUpLN/gWXzIMiqdEpadlAGbL a3emFl8K53q63f1V7QBPFImUyMVjGMp+qolJ70WoR9j7UyWxmYicl/pqft1936f4Ys+/ 19mUXDRJPiReHI0UcoNqmunDaI8Z33AAHvrvw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zHeDdh7LS/qfaOBi0Wrx/3dwOaXRkrY7AoKsyWB2WWc=; b=nU9ak0uMwhrk3dBR+rs7SAhGgGAXYRfgseKsGRw56gqPG0IpgNGTZDUGLBh7BzU22t qSb2xunyDvQVxg7WQoukrDHfScjbzHpjmYt16GzDSlp88Ny6iiVn6Zq9UyfIQwuqBTUv oYJfNBnQTYv4LSNhtUS3tpeAA3qtvK1QBBgqdOqQFoIuXRGzTupbVwxtnpF2tZIRD6Cn SJfehqc0FRoo16CLfpOdYT0NO6YYYnjUfwBQm0N3fFB6ve0q593Pm7Rk2XWdR9tCW9Xr NwcAtS6DTJ8ZVu66MA0oEZrdvJgEpszbSDDY4uMRX5rXQaTCfs8ce12LUOkl2LWDmYSn REPA== X-Gm-Message-State: AIkVDXJyYnUkSonJb7xOChU8mLAQGaFwo9ISEvjbZXDNYlE0wFNymLMnjd5+ipiSMI9g6bo1 X-Received: by 10.223.134.7 with SMTP id 7mr6602177wrv.45.1485513318430; Fri, 27 Jan 2017 02:35:18 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 186sm3075283wmw.24.2017.01.27.02.35.10 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:14 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 55E6E3E3752; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:50 +0000 Message-Id: <20170127103505.18606-11-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 10/25] tcg: enable tb_lock() for SoftMMU X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" tb_lock() has long been used for linux-user mode to protect code generation. By enabling it now we prepare for MTTCG and ensure all code generation is serialised by this lock. The other major structure that needs protecting is the l1_map and its PageDesc structures. For the SoftMMU case we also use tb_lock() to protect these structures instead of linux-user mmap_lock() which as the name suggests serialises updates to the structure as a result of guest mmap operations. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v4 - split from main tcg: enable thread-per-vCPU patch v7 - fixed up with Pranith's tcg_debug_assert() changes --- translate-all.c | 15 +-------------- 1 file changed, 1 insertion(+), 14 deletions(-) -- 2.11.0 diff --git a/translate-all.c b/translate-all.c index 41b36f04c6..87e9d00d14 100644 --- a/translate-all.c +++ b/translate-all.c @@ -75,7 +75,7 @@ * mmap_lock. */ #ifdef CONFIG_SOFTMMU -#define assert_memory_lock() do { /* nothing */ } while (0) +#define assert_memory_lock() tcg_debug_assert(have_tb_lock) #else #define assert_memory_lock() tcg_debug_assert(have_mmap_lock()) #endif @@ -135,9 +135,7 @@ TCGContext tcg_ctx; bool parallel_cpus; /* translation block context */ -#ifdef CONFIG_USER_ONLY __thread int have_tb_lock; -#endif static void page_table_config_init(void) { @@ -159,40 +157,29 @@ static void page_table_config_init(void) assert(v_l2_levels >= 0); } -#ifdef CONFIG_USER_ONLY #define assert_tb_locked() tcg_debug_assert(have_tb_lock) #define assert_tb_unlocked() tcg_debug_assert(!have_tb_lock) -#else -#define assert_tb_locked() do { /* nothing */ } while (0) -#define assert_tb_unlocked() do { /* nothing */ } while (0) -#endif void tb_lock(void) { -#ifdef CONFIG_USER_ONLY assert_tb_unlocked(); qemu_mutex_lock(&tcg_ctx.tb_ctx.tb_lock); have_tb_lock++; -#endif } void tb_unlock(void) { -#ifdef CONFIG_USER_ONLY assert_tb_locked(); have_tb_lock--; qemu_mutex_unlock(&tcg_ctx.tb_ctx.tb_lock); -#endif } void tb_lock_reset(void) { -#ifdef CONFIG_USER_ONLY if (have_tb_lock) { qemu_mutex_unlock(&tcg_ctx.tb_ctx.tb_lock); have_tb_lock = 0; } -#endif } static TranslationBlock *tb_find_pc(uintptr_t tc_ptr); From patchwork Fri Jan 27 10:34:51 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92592 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp140732obz; Fri, 27 Jan 2017 02:44:44 -0800 (PST) X-Received: by 10.237.53.201 with SMTP id d9mr7240124qte.235.1485513884929; Fri, 27 Jan 2017 02:44:44 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id k66si3235396qkb.260.2017.01.27.02.44.44 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:44:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44253 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX41K-0007bW-31 for patch@linaro.org; Fri, 27 Jan 2017 05:44:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48006) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sK-0007ul-07 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sI-0002DA-9k for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:24 -0500 Received: from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a]:36091) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sI-0002Cm-15 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:22 -0500 Received: by mail-wm0-x22a.google.com with SMTP id c85so110461001wmi.1 for ; Fri, 27 Jan 2017 02:35:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=oMzbNIcR3PzgDq8Ft3UX1jit2B5ZhreuJZDkuz5staw=; b=QBjPyt7Loi5kKKKte3EvEVUjl3EJEHQR2dl7fOFYpFntLgRIezRrBDcRJf1QGUMJr+ DqYhW9Po6B7JQQSo7uQzk9GS0iw+Ba2nH6UWgTq/hTpC9BDzWDT+8eSZwWtj2v4piGji Aqc2OshHv2wG8qJEIjW/ypRttyx1mWbCjbrD0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=oMzbNIcR3PzgDq8Ft3UX1jit2B5ZhreuJZDkuz5staw=; b=Gdrei5SiA5/tsv25sNKigHEL3o2HYL03dIscvTSu3LLHLSXT+3q3mNNwk4VR3jcrSJ oaxUBjqUk4GgzSmFiZD+zDyYCrVga6zOBsAluGHGuhyYoBuM2Dv9EwgUX7EWjKx12t4X JmYr4qOrPaaqAEpNhKWBrdBJh+3kjJg7B0xhNVphkalrwSmRU8wMOUjLWKEArSBffPs7 Pgqnvn2dEzYJLVhBBlSwhmyZBepmVcwN1Svd/GDpYpXaW6cisECkJGhVTJSlEBhe07oX ai6mfYatpso31TCGPRGitVbwm4QnB51MjtE6E6dU7t2KGVG/A7QtOg/Z4si0QFU3kHNB iITA== X-Gm-Message-State: AIkVDXLbtY/dNyMBwnt45fQpEos1TXPAYD1qCCAcbSNRQLwFmVGJX02GrIMDc1h0uuhPAJGW X-Received: by 10.28.128.150 with SMTP id b144mr2643809wmd.119.1485513320725; Fri, 27 Jan 2017 02:35:20 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id b87sm3181051wmi.0.2017.01.27.02.35.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:16 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 67CAE3E3753; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:51 +0000 Message-Id: <20170127103505.18606-12-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22a Subject: [Qemu-devel] [PATCH v8 11/25] tcg: enable thread-per-vCPU X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , KONRAD Frederic , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" There are a couple of changes that occur at the same time here: - introduce a single vCPU qemu_tcg_cpu_thread_fn One of these is spawned per vCPU with its own Thread and Condition variables. qemu_tcg_rr_cpu_thread_fn is the new name for the old single threaded function. - the TLS current_cpu variable is now live for the lifetime of MTTCG vCPU threads. This is for future work where async jobs need to know the vCPU context they are operating in. The user to switch on multi-thread behaviour and spawn a thread per-vCPU. For a simple test kvm-unit-test like: ./arm/run ./arm/locking-test.flat -smp 4 -accel tcg,thread=multi Will now use 4 vCPU threads and have an expected FAIL (instead of the unexpected PASS) as the default mode of the test has no protection when incrementing a shared variable. We enable the parallel_cpus flag to ensure we generate correct barrier and atomic code if supported by the front and backends. As each back end and front end is updated they can add CONFIG_MTTCG_TARGET and CONFIG_MTTCG_HOST to their respective make configurations so default_mttcg_enabled does the right thing. Signed-off-by: KONRAD Frederic Signed-off-by: Paolo Bonzini [AJB: Some fixes, conditionally, commit rewording] Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v1 (ajb): - fix merge conflicts - maintain single-thread approach v2 - re-base fixes (no longer has tb_find_fast lock tweak ahead) - remove bogus break condition on cpu->stop/stopped - only process exiting cpus exit_request - handle all cpus idle case (fixes shutdown issues) - sleep on EXCP_HALTED in mttcg mode (prevent crash on start-up) - move icount timer into helper v3 - update the commit message - rm kick_timer tweaks (move to earlier tcg_current_cpu tweaks) - ensure linux-user clears cpu->exit_request in loop - purging of global exit_request and tcg_current_cpu in earlier patches - fix checkpatch warnings v4 - don't break loop on stopped, we may never schedule next in RR mode - make sure we flush iorequests of current cpu if we exited on one - add tcg_cpu_exec_start/end wraps for async work functions - stop killing of current_cpu on loop exit - set current_cpu in the single thread function - remove sleep special case, add qemu_tcg_should_sleep() for mttcg - no need to atomic set cpu->exit_request going into the loop - removed extraneous setting of exit_request - split tb_lock() part of patch - rename single thread fn to qemu_tcg_rr_cpu_thread_fn v5 - enable parallel_cpus for MTTCG (for barriers/atomics) - expand on CONFIG_ flags in commit message v7 - move parallel_cpus down into the mttcg leg - minor ws merge fix --- cpu-exec.c | 5 --- cpus.c | 134 +++++++++++++++++++++++++++++++++++++++++++++++-------------- 2 files changed, 103 insertions(+), 36 deletions(-) -- 2.11.0 diff --git a/cpu-exec.c b/cpu-exec.c index cc09c1fc37..ef328087be 100644 --- a/cpu-exec.c +++ b/cpu-exec.c @@ -396,7 +396,6 @@ static inline bool cpu_handle_halt(CPUState *cpu) } #endif if (!cpu_has_work(cpu)) { - current_cpu = NULL; return true; } @@ -540,7 +539,6 @@ static inline void cpu_handle_interrupt(CPUState *cpu, if (unlikely(atomic_read(&cpu->exit_request) || replay_has_interrupt())) { - atomic_set(&cpu->exit_request, 0); cpu->exception_index = EXCP_INTERRUPT; cpu_loop_exit(cpu); } @@ -675,8 +673,5 @@ int cpu_exec(CPUState *cpu) cc->cpu_exec_exit(cpu); rcu_read_unlock(); - /* fail safe : never use current_cpu outside cpu_exec() */ - current_cpu = NULL; - return ret; } diff --git a/cpus.c b/cpus.c index 18daf41dae..ecd1ec08d3 100644 --- a/cpus.c +++ b/cpus.c @@ -808,7 +808,7 @@ static void kick_tcg_thread(void *opaque) static void start_tcg_kick_timer(void) { - if (!tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) { + if (!mttcg_enabled && !tcg_kick_vcpu_timer && CPU_NEXT(first_cpu)) { tcg_kick_vcpu_timer = timer_new_ns(QEMU_CLOCK_VIRTUAL, kick_tcg_thread, NULL); timer_mod(tcg_kick_vcpu_timer, qemu_tcg_next_kick()); @@ -1062,27 +1062,34 @@ static void qemu_tcg_destroy_vcpu(CPUState *cpu) static void qemu_wait_io_event_common(CPUState *cpu) { + atomic_mb_set(&cpu->thread_kicked, false); if (cpu->stop) { cpu->stop = false; cpu->stopped = true; qemu_cond_broadcast(&qemu_pause_cond); } process_queued_cpu_work(cpu); - cpu->thread_kicked = false; +} + +static bool qemu_tcg_should_sleep(CPUState *cpu) +{ + if (mttcg_enabled) { + return cpu_thread_is_idle(cpu); + } else { + return all_cpu_threads_idle(); + } } static void qemu_tcg_wait_io_event(CPUState *cpu) { - while (all_cpu_threads_idle()) { + while (qemu_tcg_should_sleep(cpu)) { stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); } start_tcg_kick_timer(); - CPU_FOREACH(cpu) { - qemu_wait_io_event_common(cpu); - } + qemu_wait_io_event_common(cpu); } static void qemu_kvm_wait_io_event(CPUState *cpu) @@ -1153,6 +1160,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) qemu_thread_get_self(cpu->thread); cpu->thread_id = qemu_get_thread_id(); cpu->can_do_io = 1; + current_cpu = cpu; sigemptyset(&waitset); sigaddset(&waitset, SIG_IPI); @@ -1161,9 +1169,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) cpu->created = true; qemu_cond_signal(&qemu_cpu_cond); - current_cpu = cpu; while (1) { - current_cpu = NULL; qemu_mutex_unlock_iothread(); do { int sig; @@ -1174,7 +1180,6 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) exit(1); } qemu_mutex_lock_iothread(); - current_cpu = cpu; qemu_wait_io_event_common(cpu); } @@ -1286,7 +1291,7 @@ static void deal_with_unplugged_cpus(void) * elsewhere. */ -static void *qemu_tcg_cpu_thread_fn(void *arg) +static void *qemu_tcg_rr_cpu_thread_fn(void *arg) { CPUState *cpu = arg; @@ -1308,6 +1313,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) /* process any pending work */ CPU_FOREACH(cpu) { + current_cpu = cpu; qemu_wait_io_event_common(cpu); } } @@ -1329,6 +1335,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) while (cpu && !cpu->exit_request) { atomic_mb_set(&tcg_current_rr_cpu, cpu); + current_cpu = cpu; qemu_clock_enable(QEMU_CLOCK_VIRTUAL, (cpu->singlestep_enabled & SSTEP_NOTIMER) == 0); @@ -1340,7 +1347,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) cpu_handle_guest_debug(cpu); break; } - } else if (cpu->stop || cpu->stopped) { + } else if (cpu->stop) { if (cpu->unplug) { cpu = CPU_NEXT(cpu); } @@ -1359,7 +1366,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) handle_icount_deadline(); - qemu_tcg_wait_io_event(QTAILQ_FIRST(&cpus)); + qemu_tcg_wait_io_event(cpu ? cpu : QTAILQ_FIRST(&cpus)); deal_with_unplugged_cpus(); } @@ -1406,6 +1413,64 @@ static void CALLBACK dummy_apc_func(ULONG_PTR unused) } #endif +/* Multi-threaded TCG + * + * In the multi-threaded case each vCPU has its own thread. The TLS + * variable current_cpu can be used deep in the code to find the + * current CPUState for a given thread. + */ + +static void *qemu_tcg_cpu_thread_fn(void *arg) +{ + CPUState *cpu = arg; + + rcu_register_thread(); + + qemu_mutex_lock_iothread(); + qemu_thread_get_self(cpu->thread); + + cpu->thread_id = qemu_get_thread_id(); + cpu->created = true; + cpu->can_do_io = 1; + current_cpu = cpu; + qemu_cond_signal(&qemu_cpu_cond); + + /* process any pending work */ + cpu->exit_request = 1; + + while (1) { + if (cpu_can_run(cpu)) { + int r; + r = tcg_cpu_exec(cpu); + switch (r) { + case EXCP_DEBUG: + cpu_handle_guest_debug(cpu); + break; + case EXCP_HALTED: + /* during start-up the vCPU is reset and the thread is + * kicked several times. If we don't ensure we go back + * to sleep in the halted state we won't cleanly + * start-up when the vCPU is enabled. + * + * cpu->halted should ensure we sleep in wait_io_event + */ + g_assert(cpu->halted); + break; + default: + /* Ignore everything else? */ + break; + } + } + + handle_icount_deadline(); + + atomic_mb_set(&cpu->exit_request, 0); + qemu_tcg_wait_io_event(cpu); + } + + return NULL; +} + static void qemu_cpu_kick_thread(CPUState *cpu) { #ifndef _WIN32 @@ -1436,7 +1501,7 @@ void qemu_cpu_kick(CPUState *cpu) qemu_cond_broadcast(cpu->halt_cond); if (tcg_enabled()) { cpu_exit(cpu); - /* Also ensure current RR cpu is kicked */ + /* NOP unless doing single-thread RR */ qemu_cpu_kick_rr_cpu(); } else { if (hax_enabled()) { @@ -1512,13 +1577,6 @@ void pause_all_vcpus(void) if (qemu_in_vcpu_thread()) { cpu_stop_current(); - if (!kvm_enabled()) { - CPU_FOREACH(cpu) { - cpu->stop = false; - cpu->stopped = true; - } - return; - } } while (!all_vcpus_paused()) { @@ -1567,29 +1625,43 @@ void cpu_remove_sync(CPUState *cpu) static void qemu_tcg_init_vcpu(CPUState *cpu) { char thread_name[VCPU_THREAD_NAME_SIZE]; - static QemuCond *tcg_halt_cond; - static QemuThread *tcg_cpu_thread; + static QemuCond *single_tcg_halt_cond; + static QemuThread *single_tcg_cpu_thread; - /* share a single thread for all cpus with TCG */ - if (!tcg_cpu_thread) { + if (qemu_tcg_mttcg_enabled() || !single_tcg_cpu_thread) { cpu->thread = g_malloc0(sizeof(QemuThread)); cpu->halt_cond = g_malloc0(sizeof(QemuCond)); qemu_cond_init(cpu->halt_cond); - tcg_halt_cond = cpu->halt_cond; - snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG", + + if (qemu_tcg_mttcg_enabled()) { + /* create a thread per vCPU with TCG (MTTCG) */ + parallel_cpus = true; + snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG", cpu->cpu_index); - qemu_thread_create(cpu->thread, thread_name, qemu_tcg_cpu_thread_fn, - cpu, QEMU_THREAD_JOINABLE); + + qemu_thread_create(cpu->thread, thread_name, qemu_tcg_cpu_thread_fn, + cpu, QEMU_THREAD_JOINABLE); + + } else { + /* share a single thread for all cpus with TCG */ + snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "ALL CPUs/TCG"); + qemu_thread_create(cpu->thread, thread_name, + qemu_tcg_rr_cpu_thread_fn, + cpu, QEMU_THREAD_JOINABLE); + + single_tcg_halt_cond = cpu->halt_cond; + single_tcg_cpu_thread = cpu->thread; + } #ifdef _WIN32 cpu->hThread = qemu_thread_get_handle(cpu->thread); #endif while (!cpu->created) { qemu_cond_wait(&qemu_cpu_cond, &qemu_global_mutex); } - tcg_cpu_thread = cpu->thread; } else { - cpu->thread = tcg_cpu_thread; - cpu->halt_cond = tcg_halt_cond; + /* For non-MTTCG cases we share the thread */ + cpu->thread = single_tcg_cpu_thread; + cpu->halt_cond = single_tcg_halt_cond; } } From patchwork Fri Jan 27 10:34:52 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92595 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp141612obz; Fri, 27 Jan 2017 02:47:36 -0800 (PST) X-Received: by 10.55.138.196 with SMTP id m187mr7410426qkd.214.1485514056176; Fri, 27 Jan 2017 02:47:36 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id c41si3240847qta.309.2017.01.27.02.47.35 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:47:36 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44272 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX445-0001Uu-Hu for patch@linaro.org; Fri, 27 Jan 2017 05:47:33 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48048) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sM-0007xv-Fp for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sI-0002DU-PQ for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:26 -0500 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:34764) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sI-0002D6-JX for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:22 -0500 Received: by mail-wm0-x231.google.com with SMTP id f73so12548277wmf.1 for ; Fri, 27 Jan 2017 02:35:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vp8ExxcF58dNAqavMaLO29LPlP1rgbh1h1jw9l7kNOk=; b=I9xeptTo0yQMJ8RRqhOGfo1wqE8lZv+Og9mcEANZ8BDixpSHjlv/7/N4TnI19PzTPb Kerksy4Hkbb6amkIlD+V2WJwedV/cMlRkgylMQRq9vyZMdCW86eZBq4dAVSFOhkYIYqE jLxIVcM+X3uA+KibqF/BsGSdZNimbzbtVzl20= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vp8ExxcF58dNAqavMaLO29LPlP1rgbh1h1jw9l7kNOk=; b=fRlWbd3aFg4Eirx590r1r0GZOZbydP8KACykHhECyP2CRF6FOQfhFQ8NuVcv5S+oyb PKzafUxNqavdZXvDvQelVAGN1ams05snl3Ns/Z4+cm2Z5E2+Kk28KM3kMKbAUjSeY0hM 3dPhbyIaJ4OH8CAy+r+q4sNWLyh3esnqmL9fAUgnW8cPMaBZbV9MaBENMiJAeWMZXLfX ViIeMIcsFF5X/qvvx7/EuX5iVdQ1ELTg0+0/cY+uMojw2EhMWSs5nMhteRa4TKeQ1Njd G1djG8dryLa5/pc5E6wX56o1Qaf+DZFTa9nyx6Vwl5raqv5gnaVMLFSAwB7x6+jE63Hk ONYA== X-Gm-Message-State: AIkVDXKUGvLiBzutI5cOz5mR5xUnRF4/sU20YV66bvKBAJlMI2h2+M73rkNEzyjlNkX4BI0I X-Received: by 10.223.138.9 with SMTP id w9mr6909383wrw.50.1485513321582; Fri, 27 Jan 2017 02:35:21 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g71sm3150622wmc.9.2017.01.27.02.35.11 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:16 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 781593E3763; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:52 +0000 Message-Id: <20170127103505.18606-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [PATCH v8 12/25] tcg: handle EXCP_ATOMIC exception for system emulation X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , Pranith Kumar , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Pranith Kumar The patch enables handling atomic code in the guest. This should be preferably done in cpu_handle_exception(), but the current assumptions regarding when we can execute atomic sections cause a deadlock. Signed-off-by: Pranith Kumar [AJB: tweak title] Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- cpus.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.11.0 diff --git a/cpus.c b/cpus.c index ecd1ec08d3..e3d9f3fe21 100644 --- a/cpus.c +++ b/cpus.c @@ -1346,6 +1346,11 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) if (r == EXCP_DEBUG) { cpu_handle_guest_debug(cpu); break; + } else if (r == EXCP_ATOMIC) { + qemu_mutex_unlock_iothread(); + cpu_exec_step_atomic(cpu); + qemu_mutex_lock_iothread(); + break; } } else if (cpu->stop) { if (cpu->unplug) { @@ -1456,6 +1461,10 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) */ g_assert(cpu->halted); break; + case EXCP_ATOMIC: + qemu_mutex_unlock_iothread(); + cpu_exec_step_atomic(cpu); + qemu_mutex_lock_iothread(); default: /* Ignore everything else? */ break; From patchwork Fri Jan 27 10:34:53 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92625 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp151491obz; Fri, 27 Jan 2017 03:14:02 -0800 (PST) X-Received: by 10.55.91.197 with SMTP id p188mr1730103qkb.197.1485515642633; Fri, 27 Jan 2017 03:14:02 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id t20si3286610qtc.316.2017.01.27.03.14.02 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:14:02 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44421 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4Tg-00030R-0I for patch@linaro.org; Fri, 27 Jan 2017 06:14:00 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49551) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40K-0007cI-37 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40J-0004eq-3H for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:40 -0500 Received: from mail-wm0-x229.google.com ([2a00:1450:400c:c09::229]:35318) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40I-0004eU-SN for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:38 -0500 Received: by mail-wm0-x229.google.com with SMTP id r126so111091921wmr.0 for ; Fri, 27 Jan 2017 02:43:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J1xpBQjVD3DTVxiV/CC9qQCH9HfTxWdvl37XvN6Emzw=; b=bbEmBMWUG0AREtgvHW9+5BRwqB+BNZ7Abu8w0FnnTyrV1F3yRRQDgYO9ItoF8YGnWO I119F2JL0pvca+ZqELwR55q4ALyknCy1xXvPCAnl9DjSH2v+n7qUXmSv2t66p29bA25o PaBinAaXQd6h9MdlcIugNcelcQiAy4jyYbvtQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J1xpBQjVD3DTVxiV/CC9qQCH9HfTxWdvl37XvN6Emzw=; b=bEzRgYqhzf799AXpFujAHOeKI5Lz8W5obbaZkLo4l1aOKKV0MXtsM7Nct+omoe3/g9 FOO7zivC1/rt50zZ9UC2E5FTIWhJUJoJt1OHgaPsWh3r3nKA123+9OqptgUYALDtFR4H h/XKz/8qx9jVGsSK3udBUl7bybZ7XuU4kQBgaNAgxKOkPaZjLY+Eg7i5MkW4OLjDxUkW IhxKPheZKq2LEQXNo8DHo6N60Zn7RWXF7swKrDI9B6f36mCIl1ZUjgYVLJtPPdxyVky7 P3rVi1hvkjcV6bS0essFgii6B6ECuwaLezo3tuk7UHIXTvDwbaKQ3TINA6cn5vV+OCXG 7eCA== X-Gm-Message-State: AIkVDXIHUk3r5+lAOpHPak8gy0RZ34J+66KpwYLmLqR51ZqXfQs6c3SYLv3lxQRGPCu/o0WD X-Received: by 10.223.160.114 with SMTP id l47mr6666186wrl.73.1485513817799; Fri, 27 Jan 2017 02:43:37 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id y97sm3144084wmh.24.2017.01.27.02.43.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:33 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 88DFF3E3765; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:53 +0000 Message-Id: <20170127103505.18606-14-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::229 Subject: [Qemu-devel] [PATCH v8 13/25] cputlb: add assert_cpu_is_self checks X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SoftMMU the TLB flushes are an example of a task that can be triggered on one vCPU by another. To deal with this properly we need to use safe work to ensure these changes are done safely. The new assert can be enabled while debugging to catch these cases. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- cputlb.c | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index 1cc9d9da51..af0e65cd2c 100644 --- a/cputlb.c +++ b/cputlb.c @@ -58,6 +58,12 @@ } \ } while (0) +#define assert_cpu_is_self(this_cpu) do { \ + if (DEBUG_TLB_GATE) { \ + g_assert(!cpu->created || qemu_cpu_is_self(cpu)); \ + } \ + } while (0) + /* statistics */ int tlb_flush_count; @@ -70,6 +76,9 @@ void tlb_flush(CPUState *cpu) { CPUArchState *env = cpu->env_ptr; + assert_cpu_is_self(cpu); + tlb_debug("(count: %d)\n", tlb_flush_count++); + memset(env->tlb_table, -1, sizeof(env->tlb_table)); memset(env->tlb_v_table, -1, sizeof(env->tlb_v_table)); memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache)); @@ -77,13 +86,13 @@ void tlb_flush(CPUState *cpu) env->vtlb_index = 0; env->tlb_flush_addr = -1; env->tlb_flush_mask = 0; - tlb_flush_count++; } static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) { CPUArchState *env = cpu->env_ptr; + assert_cpu_is_self(cpu); tlb_debug("start\n"); for (;;) { @@ -128,6 +137,7 @@ void tlb_flush_page(CPUState *cpu, target_ulong addr) int i; int mmu_idx; + assert_cpu_is_self(cpu); tlb_debug("page :" TARGET_FMT_lx "\n", addr); /* Check if we need to flush due to large pages. */ @@ -165,6 +175,7 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...) va_start(argp, addr); + assert_cpu_is_self(cpu); tlb_debug("addr "TARGET_FMT_lx"\n", addr); /* Check if we need to flush due to large pages. */ @@ -253,6 +264,8 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) int mmu_idx; + assert_cpu_is_self(cpu); + env = cpu->env_ptr; for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { unsigned int i; @@ -284,6 +297,8 @@ void tlb_set_dirty(CPUState *cpu, target_ulong vaddr) int i; int mmu_idx; + assert_cpu_is_self(cpu); + vaddr &= TARGET_PAGE_MASK; i = (vaddr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { @@ -343,6 +358,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, unsigned vidx = env->vtlb_index++ % CPU_VTLB_SIZE; int asidx = cpu_asidx_from_attrs(cpu, attrs); + assert_cpu_is_self(cpu); assert(size >= TARGET_PAGE_SIZE); if (size != TARGET_PAGE_SIZE) { tlb_add_large_page(env, vaddr, size); From patchwork Fri Jan 27 10:34:54 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92612 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp176199qgi; Fri, 27 Jan 2017 03:00:16 -0800 (PST) X-Received: by 10.200.37.125 with SMTP id 58mr7281374qtn.232.1485514816614; Fri, 27 Jan 2017 03:00:16 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id n2si3273494qtn.164.2017.01.27.03.00.16 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:00:16 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44342 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4GM-0005Ql-70 for patch@linaro.org; Fri, 27 Jan 2017 06:00:14 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48017) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sK-0007vu-TI for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sK-0002Dq-2V for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:24 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:38549) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sJ-0002Di-Sc for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:23 -0500 Received: by mail-wm0-x236.google.com with SMTP id r144so129835080wme.1 for ; Fri, 27 Jan 2017 02:35:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1KXA5KwXZSok3ABWG8NWl4fzC0thcYoHp+qAltY0myc=; b=d3UArYWykxog43Y/LWqV26HPC8fYOFQSxJQO7rj2W38qeVyhqPHOdB/3TfG1eGlANY u4nHslRq1IUNU64LBxA3o8wgywv0STcadmwFW/Vim7m9zushc8ueYBn0OHtRewL+D8s7 tnFGLJGTrfYJVdnlEm6TaslophXLzmFfNskHg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1KXA5KwXZSok3ABWG8NWl4fzC0thcYoHp+qAltY0myc=; b=fc2uerVQ84dEg50K/N10q9qsGtqbKWgsh7JVjr+2GX1s5fqA+IPrzCV6QOlv+E990e KY+TX2bONDOF0DJFBBgIa2XtIjmxQydXl9otIpR30jB+N7jLOcxCzxmEb1ihDQDCM6Pc pH1MMEyBpL8KIWXGtA5BJKXiSYj/A6u7zJ7t6Oy5vEK35Z2BD1Wbigx5eBAc2rbjKvbX 4prWQRfuCn3EeDakZfC7SUcbbevb9NgvJGLmtS/IEUOaAoiDftJxkWdcD1VhvncsA9vk 8h7uEYwTkxygzBbxowYIM8lNA3nqPv3+axAeA0YPpCd30TN39z69+/kXLAxzL4Z321cH Y2Yg== X-Gm-Message-State: AIkVDXLqyHWmX5eNyRv8oN2AMDZTLwizfrX9wNA2w6RH46xPFBgRwdMlYumPZNBE7dJe15fh X-Received: by 10.28.154.70 with SMTP id c67mr2731818wme.90.1485513322866; Fri, 27 Jan 2017 02:35:22 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g5sm7175286wrd.0.2017.01.27.02.35.12 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:17 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 994313E3766; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:54 +0000 Message-Id: <20170127103505.18606-15-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 14/25] cputlb: tweak qemu_ram_addr_from_host_nofail reporting X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This moves the helper function closer to where it is called and updates the error message to report via error_report instead of the deprecated fprintf. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- cputlb.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index af0e65cd2c..94fa9977c5 100644 --- a/cputlb.c +++ b/cputlb.c @@ -246,18 +246,6 @@ void tlb_reset_dirty_range(CPUTLBEntry *tlb_entry, uintptr_t start, } } -static inline ram_addr_t qemu_ram_addr_from_host_nofail(void *ptr) -{ - ram_addr_t ram_addr; - - ram_addr = qemu_ram_addr_from_host(ptr); - if (ram_addr == RAM_ADDR_INVALID) { - fprintf(stderr, "Bad ram pointer %p\n", ptr); - abort(); - } - return ram_addr; -} - void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) { CPUArchState *env; @@ -469,6 +457,18 @@ static void report_bad_exec(CPUState *cpu, target_ulong addr) log_cpu_state_mask(LOG_GUEST_ERROR, cpu, CPU_DUMP_FPU | CPU_DUMP_CCOP); } +static inline ram_addr_t qemu_ram_addr_from_host_nofail(void *ptr) +{ + ram_addr_t ram_addr; + + ram_addr = qemu_ram_addr_from_host(ptr); + if (ram_addr == RAM_ADDR_INVALID) { + error_report("Bad ram pointer %p", ptr); + abort(); + } + return ram_addr; +} + /* NOTE: this function can trigger an exception */ /* NOTE2: the returned address is not exactly the physical address: it * is actually a ram_addr_t (in system mode; the user mode emulation From patchwork Fri Jan 27 10:34:55 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92594 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp141490obz; Fri, 27 Jan 2017 02:47:12 -0800 (PST) X-Received: by 10.200.49.167 with SMTP id h36mr6708460qte.11.1485514032600; Fri, 27 Jan 2017 02:47:12 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id y190si778762qke.241.2017.01.27.02.47.12 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:47:12 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44270 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX43h-0001DA-TW for patch@linaro.org; Fri, 27 Jan 2017 05:47:09 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48050) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sM-0007y4-Gj for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sL-0002EG-5y for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:26 -0500 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:36114) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sK-0002E3-TA for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:25 -0500 Received: by mail-wm0-x22b.google.com with SMTP id c85so110462826wmi.1 for ; Fri, 27 Jan 2017 02:35:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=erpZnd0pitnHmgzHfCuX3/9CIVLvrSFtg5nbi7i7aX8=; b=C+VcGO4GRH+8jmfwXDB02n0ENOjrgdCsDigED+LPu57h29MO3kv1IH0Hh3VFfn6yws WI7RVMoaOvScuqgv5wpE7laT3hFGoQu/stvHfvc75VzSFr+LnEI4XySkblXiv7qEM00J VWdr3HwxGNGQ6yA5aKqtEfvJHK4dpH+wRmSE4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=erpZnd0pitnHmgzHfCuX3/9CIVLvrSFtg5nbi7i7aX8=; b=eVwF6PnGqdPk6XgrXr6K2GTReKtRH3t0Q8eWQHHBnDHSjPecrrApOlDCKhR1g6rP3G 00B2BfR8/7XC9/UYKGs8IrjgsM7spJA02sbJqg0sffkoKtb1y6CBu38I1YI6t06MYbRv wVk0UwFXHJj1JxTkTBoe8zndgyBVcjvldUCaw6aJqa7ERV3kHjUt2V/rILMzVS8pAnK+ TsCfM6mR97/NxpC0cCbBqQW/0VoOhhgCbVW93ioETFBflj+W9FStnTSD8N8EpBH3vSbS xxgK4NtXsaDKf7G9rwNCGnImw0Dl3VJDoXJcNHTt7Pyk+KooAkiYVrczAAjtmJRebP/W XscQ== X-Gm-Message-State: AIkVDXLucNqHA5QHCnn9+lJEpsozFt9gGb7o7OlaTtTG3fXFo/XL+JL/J5gWYJcUED2RBQ8o X-Received: by 10.28.23.66 with SMTP id 63mr2550657wmx.46.1485513323769; Fri, 27 Jan 2017 02:35:23 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id r5sm3067553wme.23.2017.01.27.02.35.13 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:17 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id ABF6B3E3767; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:55 +0000 Message-Id: <20170127103505.18606-16-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22b Subject: [Qemu-devel] [PATCH v8 15/25] cputlb: introduce tlb_flush_* async work. X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Crosthwaite , "open list:Overall" , Paolo Bonzini , KONRAD Frederic , =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: KONRAD Frederic Some architectures allow to flush the tlb of other VCPUs. This is not a problem when we have only one thread for all VCPUs but it definitely needs to be an asynchronous work when we are in true multithreaded work. We take the tb_lock() when doing this to avoid racing with other threads which may be invalidating TB's at the same time. The alternative would be to use proper atomic primitives to clear the tlb entries en-mass. This patch doesn't do anything to protect other cputlb function being called in MTTCG mode making cross vCPU changes. Signed-off-by: KONRAD Frederic [AJB: remove need for g_malloc on defer, make check fixes, tb_lock] Signed-off-by: Alex Bennée --- v8 - fix merge failure mentioning global flush v6 (base patches) - don't use cmpxchg_bool (we drop it later anyway) - use RUN_ON_CPU macros instead of inlines - bug out of tlb_flush if !tcg_enabled() (MacOSX make check failure) v5 (base patches) - take tb_lock() for memset - ensure tb_flush_page properly asyncs work for other vCPUs - use run_on_cpu_data v4 (base_patches) - brought forward from arm enabling series - restore pending_tlb_flush flag v1 - Remove tlb_flush_all just do the check in tlb_flush. - remove the need to g_malloc - tlb_flush calls direct if !cpu->created --- cputlb.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++-- include/exec/exec-all.h | 1 + include/qom/cpu.h | 6 +++++ 3 files changed, 71 insertions(+), 2 deletions(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index 94fa9977c5..5dfd3c3ba9 100644 --- a/cputlb.c +++ b/cputlb.c @@ -64,6 +64,10 @@ } \ } while (0) +/* run_on_cpu_data.target_ptr should always be big enough for a + * target_ulong even on 32 bit builds */ +QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data)); + /* statistics */ int tlb_flush_count; @@ -72,13 +76,22 @@ int tlb_flush_count; * flushing more entries than required is only an efficiency issue, * not a correctness issue. */ -void tlb_flush(CPUState *cpu) +static void tlb_flush_nocheck(CPUState *cpu) { CPUArchState *env = cpu->env_ptr; + /* The QOM tests will trigger tlb_flushes without setting up TCG + * so we bug out here in that case. + */ + if (!tcg_enabled()) { + return; + } + assert_cpu_is_self(cpu); tlb_debug("(count: %d)\n", tlb_flush_count++); + tb_lock(); + memset(env->tlb_table, -1, sizeof(env->tlb_table)); memset(env->tlb_v_table, -1, sizeof(env->tlb_v_table)); memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache)); @@ -86,6 +99,27 @@ void tlb_flush(CPUState *cpu) env->vtlb_index = 0; env->tlb_flush_addr = -1; env->tlb_flush_mask = 0; + + tb_unlock(); + + atomic_mb_set(&cpu->pending_tlb_flush, false); +} + +static void tlb_flush_global_async_work(CPUState *cpu, run_on_cpu_data data) +{ + tlb_flush_nocheck(cpu); +} + +void tlb_flush(CPUState *cpu) +{ + if (cpu->created && !qemu_cpu_is_self(cpu)) { + if (atomic_cmpxchg(&cpu->pending_tlb_flush, false, true) == true) { + async_run_on_cpu(cpu, tlb_flush_global_async_work, + RUN_ON_CPU_NULL); + } + } else { + tlb_flush_nocheck(cpu); + } } static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) @@ -95,6 +129,8 @@ static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) assert_cpu_is_self(cpu); tlb_debug("start\n"); + tb_lock(); + for (;;) { int mmu_idx = va_arg(argp, int); @@ -109,6 +145,8 @@ static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) } memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache)); + + tb_unlock(); } void tlb_flush_by_mmuidx(CPUState *cpu, ...) @@ -131,13 +169,15 @@ static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) } } -void tlb_flush_page(CPUState *cpu, target_ulong addr) +static void tlb_flush_page_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; + target_ulong addr = (target_ulong) data.target_ptr; int i; int mmu_idx; assert_cpu_is_self(cpu); + tlb_debug("page :" TARGET_FMT_lx "\n", addr); /* Check if we need to flush due to large pages. */ @@ -167,6 +207,18 @@ void tlb_flush_page(CPUState *cpu, target_ulong addr) tb_flush_jmp_cache(cpu, addr); } +void tlb_flush_page(CPUState *cpu, target_ulong addr) +{ + tlb_debug("page :" TARGET_FMT_lx "\n", addr); + + if (!qemu_cpu_is_self(cpu)) { + async_run_on_cpu(cpu, tlb_flush_page_async_work, + RUN_ON_CPU_TARGET_PTR(addr)); + } else { + tlb_flush_page_async_work(cpu, RUN_ON_CPU_TARGET_PTR(addr)); + } +} + void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...) { CPUArchState *env = cpu->env_ptr; @@ -213,6 +265,16 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...) tb_flush_jmp_cache(cpu, addr); } +void tlb_flush_page_all(target_ulong addr) +{ + CPUState *cpu; + + CPU_FOREACH(cpu) { + async_run_on_cpu(cpu, tlb_flush_page_async_work, + RUN_ON_CPU_TARGET_PTR(addr)); + } +} + /* update the TLBs so that writes to code in the virtual page 'addr' can be detected */ void tlb_protect_code(ram_addr_t ram_addr) diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index bd4622ac5d..e43cb68355 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -158,6 +158,7 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr, void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr); void probe_write(CPUArchState *env, target_ulong addr, int mmu_idx, uintptr_t retaddr); +void tlb_flush_page_all(target_ulong addr); #else static inline void tlb_flush_page(CPUState *cpu, target_ulong addr) { diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 1a06ae5938..7f1d6a81a0 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -398,6 +398,12 @@ struct CPUState { bool hax_vcpu_dirty; struct hax_vcpu_state *hax_vcpu; + + /* The pending_tlb_flush flag is set and cleared atomically to + * avoid potential races. The aim of the flag is to avoid + * unnecessary flushes. + */ + bool pending_tlb_flush; }; QTAILQ_HEAD(CPUTailQ, CPUState); From patchwork Fri Jan 27 10:34:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92643 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp158815obz; Fri, 27 Jan 2017 03:34:42 -0800 (PST) X-Received: by 10.55.181.6 with SMTP id e6mr7272182qkf.298.1485516882822; Fri, 27 Jan 2017 03:34:42 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id d4si3332738qtb.136.2017.01.27.03.34.41 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:34:42 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44561 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4nf-0005ru-Qk for patch@linaro.org; Fri, 27 Jan 2017 06:34:39 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49667) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40S-0007m0-Vn for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:52 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40P-0004jW-PD for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:49 -0500 Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]:36042) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40P-0004jA-CU for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:45 -0500 Received: by mail-wm0-x234.google.com with SMTP id c85so110762917wmi.1 for ; Fri, 27 Jan 2017 02:43:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=eqbiWhpWvfG2Fr+WD5ep5foynmjb4+QIrw4otQhAiMI=; b=F4EQWxrnTNSGXBAXeOTqrUPdXIWyPpsNksZooTgA2sRRc0/3DZz8AXe6Bnt8aC6kcf CQrh5HKEe/Ivf9tMBHUk++1HBZBxYTb00kbwyX2ZsatHcfpIoCDVSappkCOwjoJ00KPk kUu1sjgvylVxjmpN9/qXczcnf7kc9TY7pZeiw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=eqbiWhpWvfG2Fr+WD5ep5foynmjb4+QIrw4otQhAiMI=; b=G76NjlJOEKXmfCH92zOe1paGzxAtI/E+Md/DL6aBfZBfsCZzY9C4uGqVjU4zbnKMmo m8R5smuarEK85we/OXv15mFAaM9/mQc212DXl81G49WRrBFdIa5Es8WvWfo6t8XcfVc5 2MTyfT0S3uLaxFyqouKaCa0QZ+BGsoPF203AiZ1HKoj/yuWqhnY6NvBFWWYX6yPGG1Lx gU19tRXa5R85fnBl9Zxr1sGcXte2mr+Fs8s+vMkz8Or08fXmVLiOb8WHYFl3OG3QKkN3 CGshDI3/Lhw/C8zYaCYbo4UilsUeS90DN6vnwzkrDXslZwenTQM1Mlr/OcKymRwR4Odc i8Jw== X-Gm-Message-State: AIkVDXLVDprFzDpeyg5wGVkdTAcQ3LYa4bmF6jCTyT5ot4Rs6prPiSKf28xFgwe5HutXkFs5 X-Received: by 10.223.171.22 with SMTP id q22mr7550392wrc.70.1485513823132; Fri, 27 Jan 2017 02:43:43 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 8sm8743326wmg.1.2017.01.27.02.43.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:38 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id C3A763E3768; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:56 +0000 Message-Id: <20170127103505.18606-17-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::234 Subject: [Qemu-devel] [PATCH v8 16/25] cputlb and arm/sparc targets: convert mmuidx flushes from varg to bitmap X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , Peter Crosthwaite , Mark Cave-Ayland , "open list:Overall" , "open list:ARM" , Paolo Bonzini , =?utf-8?q?Alex_Benn=C3=A9e?= , Artyom Tarasenko , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" While the vargs approach was flexible the original MTTCG ended up having munge the bits to a bitmap so the data could be used in deferred work helpers. Instead of hiding that in cputlb we push the change to the API to make it take a bitmap of MMU indexes instead. This change is fairly mechanical but as storing the actual index is useful for cases like the current running context. As a result the constants are renamed to ARMMMUBit_foo and a couple of helper functions added to convert between a single bit and a scalar index. Signed-off-by: Alex Bennée --- cputlb.c | 60 +++++------- include/exec/exec-all.h | 13 +-- target/arm/cpu.h | 41 +++++--- target/arm/helper.c | 227 ++++++++++++++++++++++----------------------- target/arm/translate-a64.c | 14 +-- target/arm/translate.c | 24 +++-- target/arm/translate.h | 4 +- target/sparc/ldst_helper.c | 8 +- 8 files changed, 194 insertions(+), 197 deletions(-) -- 2.11.0 Reviewed-by: Artyom Tarasenko diff --git a/cputlb.c b/cputlb.c index 5dfd3c3ba9..97e5c12de8 100644 --- a/cputlb.c +++ b/cputlb.c @@ -122,26 +122,25 @@ void tlb_flush(CPUState *cpu) } } -static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) +static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) { CPUArchState *env = cpu->env_ptr; + unsigned long mmu_idx_bitmask = idxmap; + int mmu_idx; assert_cpu_is_self(cpu); tlb_debug("start\n"); tb_lock(); - for (;;) { - int mmu_idx = va_arg(argp, int); - - if (mmu_idx < 0) { - break; - } + for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { - tlb_debug("%d\n", mmu_idx); + if (test_bit(mmu_idx, &mmu_idx_bitmask)) { + tlb_debug("%d\n", mmu_idx); - memset(env->tlb_table[mmu_idx], -1, sizeof(env->tlb_table[0])); - memset(env->tlb_v_table[mmu_idx], -1, sizeof(env->tlb_v_table[0])); + memset(env->tlb_table[mmu_idx], -1, sizeof(env->tlb_table[0])); + memset(env->tlb_v_table[mmu_idx], -1, sizeof(env->tlb_v_table[0])); + } } memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache)); @@ -149,12 +148,9 @@ static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, va_list argp) tb_unlock(); } -void tlb_flush_by_mmuidx(CPUState *cpu, ...) +void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) { - va_list argp; - va_start(argp, cpu); - v_tlb_flush_by_mmuidx(cpu, argp); - va_end(argp); + v_tlb_flush_by_mmuidx(cpu, idxmap); } static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) @@ -219,13 +215,11 @@ void tlb_flush_page(CPUState *cpu, target_ulong addr) } } -void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...) +void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) { CPUArchState *env = cpu->env_ptr; - int i, k; - va_list argp; - - va_start(argp, addr); + unsigned long mmu_idx_bitmap = idxmap; + int i, page, mmu_idx; assert_cpu_is_self(cpu); tlb_debug("addr "TARGET_FMT_lx"\n", addr); @@ -236,31 +230,23 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...) TARGET_FMT_lx "/" TARGET_FMT_lx ")\n", env->tlb_flush_addr, env->tlb_flush_mask); - v_tlb_flush_by_mmuidx(cpu, argp); - va_end(argp); + v_tlb_flush_by_mmuidx(cpu, idxmap); return; } addr &= TARGET_PAGE_MASK; - i = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - - for (;;) { - int mmu_idx = va_arg(argp, int); + page = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); - if (mmu_idx < 0) { - break; - } - - tlb_debug("idx %d\n", mmu_idx); - - tlb_flush_entry(&env->tlb_table[mmu_idx][i], addr); + for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { + if (test_bit(mmu_idx, &mmu_idx_bitmap)) { + tlb_flush_entry(&env->tlb_table[mmu_idx][page], addr); - /* check whether there are vltb entries that need to be flushed */ - for (k = 0; k < CPU_VTLB_SIZE; k++) { - tlb_flush_entry(&env->tlb_v_table[mmu_idx][k], addr); + /* check whether there are vltb entries that need to be flushed */ + for (i = 0; i < CPU_VTLB_SIZE; i++) { + tlb_flush_entry(&env->tlb_v_table[mmu_idx][i], addr); + } } } - va_end(argp); tb_flush_jmp_cache(cpu, addr); } diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index e43cb68355..a6c17ed74a 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -106,21 +106,22 @@ void tlb_flush(CPUState *cpu); * tlb_flush_page_by_mmuidx: * @cpu: CPU whose TLB should be flushed * @addr: virtual address of page to be flushed - * @...: list of MMU indexes to flush, terminated by a negative value + * @idxmap: bitmap of MMU indexes to flush * * Flush one page from the TLB of the specified CPU, for the specified * MMU indexes. */ -void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, ...); +void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, + uint16_t idxmap); /** * tlb_flush_by_mmuidx: * @cpu: CPU whose TLB should be flushed - * @...: list of MMU indexes to flush, terminated by a negative value + * @idxmap: bitmap of MMU indexes to flush * * Flush all entries from the TLB of the specified CPU, for the specified * MMU indexes. */ -void tlb_flush_by_mmuidx(CPUState *cpu, ...); +void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap); /** * tlb_set_page_with_attrs: * @cpu: CPU to add this TLB entry for @@ -169,11 +170,11 @@ static inline void tlb_flush(CPUState *cpu) } static inline void tlb_flush_page_by_mmuidx(CPUState *cpu, - target_ulong addr, ...) + target_ulong addr, uint16_t idxmap) { } -static inline void tlb_flush_by_mmuidx(CPUState *cpu, ...) +static inline void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) { } #endif diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 151a5d754e..274ef17562 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -1954,27 +1954,40 @@ static inline bool arm_excp_unmasked(CPUState *cs, unsigned int excp_idx, * of the AT/ATS operations. * The values used are carefully arranged to make mmu_idx => EL lookup easy. */ -typedef enum ARMMMUIdx { - ARMMMUIdx_S12NSE0 = 0, - ARMMMUIdx_S12NSE1 = 1, - ARMMMUIdx_S1E2 = 2, - ARMMMUIdx_S1E3 = 3, - ARMMMUIdx_S1SE0 = 4, - ARMMMUIdx_S1SE1 = 5, - ARMMMUIdx_S2NS = 6, +typedef enum ARMMMUBitMap { + ARMMMUBit_S12NSE0 = 1 << 0, + ARMMMUBit_S12NSE1 = 1 << 1, + ARMMMUBit_S1E2 = 1 << 2, + ARMMMUBit_S1E3 = 1 << 3, + ARMMMUBit_S1SE0 = 1 << 4, + ARMMMUBit_S1SE1 = 1 << 5, + ARMMMUBit_S2NS = 1 << 6, /* Indexes below here don't have TLBs and are used only for AT system * instructions or for the first stage of an S12 page table walk. */ - ARMMMUIdx_S1NSE0 = 7, - ARMMMUIdx_S1NSE1 = 8, -} ARMMMUIdx; + ARMMMUBit_S1NSE0 = 1 << 7, + ARMMMUBit_S1NSE1 = 1 << 8, +} ARMMMUBitMap; -#define MMU_USER_IDX 0 +typedef int ARMMMUIdx; + +static inline ARMMMUIdx arm_mmu_bit_to_idx(ARMMMUBitMap bit) +{ + g_assert(ctpop16(bit) == 1); + return ctz32(bit); +} + +static inline ARMMMUBitMap arm_mmu_idx_to_bit(ARMMMUIdx idx) +{ + return 1 << idx; +} + +#define MMU_USER_IDX (1 << 0) /* Return the exception level we're running at if this is our mmu_idx */ static inline int arm_mmu_idx_to_el(ARMMMUIdx mmu_idx) { - assert(mmu_idx < ARMMMUIdx_S2NS); + assert(mmu_idx < arm_mmu_bit_to_idx(ARMMMUBit_S2NS)); return mmu_idx & 3; } @@ -1984,7 +1997,7 @@ static inline int cpu_mmu_index(CPUARMState *env, bool ifetch) int el = arm_current_el(env); if (el < 2 && arm_is_secure_below_el3(env)) { - return ARMMMUIdx_S1SE0 + el; + return arm_mmu_bit_to_idx(ARMMMUBit_S1SE0) + el; } return el; } diff --git a/target/arm/helper.c b/target/arm/helper.c index 84d789be93..0c6fb4add7 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -578,8 +578,8 @@ static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, { CPUState *cs = ENV_GET_CPU(env); - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S12NSE1, ARMMMUIdx_S12NSE0, - ARMMMUIdx_S2NS, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0 + | ARMMMUBit_S2NS); } static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -588,8 +588,8 @@ static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *other_cs; CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, ARMMMUIdx_S2NS, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0 | ARMMMUBit_S2NS); } } @@ -611,7 +611,7 @@ static void tlbiipas2_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 40); - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S2NS, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S2NS); } static void tlbiipas2_is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -627,7 +627,7 @@ static void tlbiipas2_is_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 40); CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S2NS, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S2NS); } } @@ -636,7 +636,7 @@ static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, { CPUState *cs = ENV_GET_CPU(env); - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S1E2, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S1E2); } static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -645,7 +645,7 @@ static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *other_cs; CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S1E2, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E2); } } @@ -655,7 +655,7 @@ static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S1E2, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S1E2); } static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -665,7 +665,7 @@ static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S1E2, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E2); } } @@ -2100,7 +2100,7 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { int access_type = ri->opc2 & 1; uint64_t par64; - ARMMMUIdx mmu_idx; + ARMMMUBitMap mmu_bit; int el = arm_current_el(env); bool secure = arm_is_secure_below_el3(env); @@ -2109,13 +2109,13 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) /* stage 1 current state PL1: ATS1CPR, ATS1CPW */ switch (el) { case 3: - mmu_idx = ARMMMUIdx_S1E3; + mmu_bit = ARMMMUBit_S1E3; break; case 2: - mmu_idx = ARMMMUIdx_S1NSE1; + mmu_bit = ARMMMUBit_S1NSE1; break; case 1: - mmu_idx = secure ? ARMMMUIdx_S1SE1 : ARMMMUIdx_S1NSE1; + mmu_bit = secure ? ARMMMUBit_S1SE1 : ARMMMUBit_S1NSE1; break; default: g_assert_not_reached(); @@ -2125,13 +2125,13 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) /* stage 1 current state PL0: ATS1CUR, ATS1CUW */ switch (el) { case 3: - mmu_idx = ARMMMUIdx_S1SE0; + mmu_bit = ARMMMUBit_S1SE0; break; case 2: - mmu_idx = ARMMMUIdx_S1NSE0; + mmu_bit = ARMMMUBit_S1NSE0; break; case 1: - mmu_idx = secure ? ARMMMUIdx_S1SE0 : ARMMMUIdx_S1NSE0; + mmu_bit = secure ? ARMMMUBit_S1SE0 : ARMMMUBit_S1NSE0; break; default: g_assert_not_reached(); @@ -2139,17 +2139,17 @@ static void ats_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) break; case 4: /* stage 1+2 NonSecure PL1: ATS12NSOPR, ATS12NSOPW */ - mmu_idx = ARMMMUIdx_S12NSE1; + mmu_bit = ARMMMUBit_S12NSE1; break; case 6: /* stage 1+2 NonSecure PL0: ATS12NSOUR, ATS12NSOUW */ - mmu_idx = ARMMMUIdx_S12NSE0; + mmu_bit = ARMMMUBit_S12NSE0; break; default: g_assert_not_reached(); } - par64 = do_ats_write(env, value, access_type, mmu_idx); + par64 = do_ats_write(env, value, access_type, arm_mmu_bit_to_idx(mmu_bit)); A32_BANKED_CURRENT_REG_SET(env, par, par64); } @@ -2160,7 +2160,8 @@ static void ats1h_write(CPUARMState *env, const ARMCPRegInfo *ri, int access_type = ri->opc2 & 1; uint64_t par64; - par64 = do_ats_write(env, value, access_type, ARMMMUIdx_S2NS); + par64 = do_ats_write(env, value, access_type, + arm_mmu_bit_to_idx(ARMMMUBit_S2NS)); A32_BANKED_CURRENT_REG_SET(env, par, par64); } @@ -2185,26 +2186,26 @@ static void ats_write64(CPUARMState *env, const ARMCPRegInfo *ri, case 0: switch (ri->opc1) { case 0: /* AT S1E1R, AT S1E1W */ - mmu_idx = secure ? ARMMMUIdx_S1SE1 : ARMMMUIdx_S1NSE1; + mmu_idx = secure ? ARMMMUBit_S1SE1 : ARMMMUBit_S1NSE1; break; case 4: /* AT S1E2R, AT S1E2W */ - mmu_idx = ARMMMUIdx_S1E2; + mmu_idx = ARMMMUBit_S1E2; break; case 6: /* AT S1E3R, AT S1E3W */ - mmu_idx = ARMMMUIdx_S1E3; + mmu_idx = ARMMMUBit_S1E3; break; default: g_assert_not_reached(); } break; case 2: /* AT S1E0R, AT S1E0W */ - mmu_idx = secure ? ARMMMUIdx_S1SE0 : ARMMMUIdx_S1NSE0; + mmu_idx = secure ? ARMMMUBit_S1SE0 : ARMMMUBit_S1NSE0; break; case 4: /* AT S12E1R, AT S12E1W */ - mmu_idx = secure ? ARMMMUIdx_S1SE1 : ARMMMUIdx_S12NSE1; + mmu_idx = secure ? ARMMMUBit_S1SE1 : ARMMMUBit_S12NSE1; break; case 6: /* AT S12E0R, AT S12E0W */ - mmu_idx = secure ? ARMMMUIdx_S1SE0 : ARMMMUIdx_S12NSE0; + mmu_idx = secure ? ARMMMUBit_S1SE0 : ARMMMUBit_S12NSE0; break; default: g_assert_not_reached(); @@ -2499,8 +2500,8 @@ static void vttbr_write(CPUARMState *env, const ARMCPRegInfo *ri, /* Accesses to VTTBR may change the VMID so we must flush the TLB. */ if (raw_read(env, ri) != value) { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S12NSE1, ARMMMUIdx_S12NSE0, - ARMMMUIdx_S2NS, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0 | + ARMMMUBit_S2NS); raw_write(env, ri, value); } } @@ -2859,9 +2860,9 @@ static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *cs = CPU(cpu); if (arm_is_secure_below_el3(env)) { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S1SE1, ARMMMUIdx_S1SE0, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); } else { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S12NSE1, ARMMMUIdx_S12NSE0, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0); } } @@ -2873,10 +2874,10 @@ static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPU_FOREACH(other_cs) { if (sec) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S1SE1, ARMMMUIdx_S1SE0, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); } else { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0); } } } @@ -2892,13 +2893,13 @@ static void tlbi_aa64_alle1_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *cs = CPU(cpu); if (arm_is_secure_below_el3(env)) { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S1SE1, ARMMMUIdx_S1SE0, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); } else { if (arm_feature(env, ARM_FEATURE_EL2)) { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S12NSE1, ARMMMUIdx_S12NSE0, - ARMMMUIdx_S2NS, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0 | + ARMMMUBit_S2NS); } else { - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S12NSE1, ARMMMUIdx_S12NSE0, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0); } } } @@ -2909,7 +2910,7 @@ static void tlbi_aa64_alle2_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMCPU *cpu = arm_env_get_cpu(env); CPUState *cs = CPU(cpu); - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S1E2, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S1E2); } static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -2918,7 +2919,7 @@ static void tlbi_aa64_alle3_write(CPUARMState *env, const ARMCPRegInfo *ri, ARMCPU *cpu = arm_env_get_cpu(env); CPUState *cs = CPU(cpu); - tlb_flush_by_mmuidx(cs, ARMMMUIdx_S1E3, -1); + tlb_flush_by_mmuidx(cs, ARMMMUBit_S1E3); } static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -2934,13 +2935,13 @@ static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPU_FOREACH(other_cs) { if (sec) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S1SE1, ARMMMUIdx_S1SE0, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); } else if (has_el2) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, ARMMMUIdx_S2NS, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0 | ARMMMUBit_S2NS); } else { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0); } } } @@ -2951,7 +2952,7 @@ static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *other_cs; CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S1E2, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E2); } } @@ -2961,7 +2962,7 @@ static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *other_cs; CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUIdx_S1E3, -1); + tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E3); } } @@ -2978,11 +2979,11 @@ static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t pageaddr = sextract64(value << 12, 0, 56); if (arm_is_secure_below_el3(env)) { - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S1SE1, - ARMMMUIdx_S1SE0, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S1SE1 | + ARMMMUBit_S1SE0); } else { - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0); } } @@ -2997,7 +2998,7 @@ static void tlbi_aa64_vae2_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *cs = CPU(cpu); uint64_t pageaddr = sextract64(value << 12, 0, 56); - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S1E2, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S1E2); } static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3011,7 +3012,7 @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, CPUState *cs = CPU(cpu); uint64_t pageaddr = sextract64(value << 12, 0, 56); - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S1E3, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S1E3); } static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3023,11 +3024,11 @@ static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, CPU_FOREACH(other_cs) { if (sec) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S1SE1, - ARMMMUIdx_S1SE0, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1SE1 | + ARMMMUBit_S1SE0); } else { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S12NSE1, - ARMMMUIdx_S12NSE0, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0); } } } @@ -3039,7 +3040,7 @@ static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t pageaddr = sextract64(value << 12, 0, 56); CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S1E2, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E2); } } @@ -3050,7 +3051,7 @@ static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t pageaddr = sextract64(value << 12, 0, 56); CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S1E3, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E3); } } @@ -3073,7 +3074,7 @@ static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 48); - tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUIdx_S2NS, -1); + tlb_flush_page_by_mmuidx(cs, pageaddr, ARMMMUBit_S2NS); } static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3089,7 +3090,7 @@ static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 48); CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUIdx_S2NS, -1); + tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S2NS); } } @@ -6709,41 +6710,33 @@ void arm_cpu_do_interrupt(CPUState *cs) /* Return the exception level which controls this address translation regime */ static inline uint32_t regime_el(CPUARMState *env, ARMMMUIdx mmu_idx) { - switch (mmu_idx) { - case ARMMMUIdx_S2NS: - case ARMMMUIdx_S1E2: + ARMMMUBitMap bit = arm_mmu_idx_to_bit(mmu_idx); + if (bit & (ARMMMUBit_S2NS | ARMMMUBit_S1E2)) { return 2; - case ARMMMUIdx_S1E3: + } else if (bit & ARMMMUBit_S1E3) { return 3; - case ARMMMUIdx_S1SE0: + } else if (bit & ARMMMUBit_S1SE0) { return arm_el_is_aa64(env, 3) ? 1 : 3; - case ARMMMUIdx_S1SE1: - case ARMMMUIdx_S1NSE0: - case ARMMMUIdx_S1NSE1: + } else if (bit & (ARMMMUBit_S1SE1 | ARMMMUBit_S1NSE0 | ARMMMUBit_S1NSE1)) { return 1; - default: - g_assert_not_reached(); } + + g_assert_not_reached(); } /* Return true if this address translation regime is secure */ static inline bool regime_is_secure(CPUARMState *env, ARMMMUIdx mmu_idx) { - switch (mmu_idx) { - case ARMMMUIdx_S12NSE0: - case ARMMMUIdx_S12NSE1: - case ARMMMUIdx_S1NSE0: - case ARMMMUIdx_S1NSE1: - case ARMMMUIdx_S1E2: - case ARMMMUIdx_S2NS: + ARMMMUBitMap bit = arm_mmu_idx_to_bit(mmu_idx); + + if (bit & (ARMMMUBit_S12NSE0 | ARMMMUBit_S12NSE1 | ARMMMUBit_S1NSE0 | + ARMMMUBit_S1NSE1 | ARMMMUBit_S1E2 | ARMMMUBit_S2NS)) { return false; - case ARMMMUIdx_S1E3: - case ARMMMUIdx_S1SE0: - case ARMMMUIdx_S1SE1: + } else if (bit & (ARMMMUBit_S1E3 | ARMMMUBit_S1SE0 | ARMMMUBit_S1SE1)) { return true; - default: - g_assert_not_reached(); } + + g_assert_not_reached(); } /* Return the SCTLR value which controls this address translation regime */ @@ -6756,7 +6749,7 @@ static inline uint32_t regime_sctlr(CPUARMState *env, ARMMMUIdx mmu_idx) static inline bool regime_translation_disabled(CPUARMState *env, ARMMMUIdx mmu_idx) { - if (mmu_idx == ARMMMUIdx_S2NS) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)) { return (env->cp15.hcr_el2 & HCR_VM) == 0; } return (regime_sctlr(env, mmu_idx) & SCTLR_M) == 0; @@ -6771,7 +6764,7 @@ static inline bool regime_translation_big_endian(CPUARMState *env, /* Return the TCR controlling this translation regime */ static inline TCR *regime_tcr(CPUARMState *env, ARMMMUIdx mmu_idx) { - if (mmu_idx == ARMMMUIdx_S2NS) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)) { return &env->cp15.vtcr_el2; } return &env->cp15.tcr_el[regime_el(env, mmu_idx)]; @@ -6786,8 +6779,9 @@ uint32_t arm_regime_tbi0(CPUARMState *env, ARMMMUIdx mmu_idx) /* For EL0 and EL1, TBI is controlled by stage 1's TCR, so convert * a stage 1+2 mmu index into the appropriate stage 1 mmu index. */ - if (mmu_idx == ARMMMUIdx_S12NSE0 || mmu_idx == ARMMMUIdx_S12NSE1) { - mmu_idx += ARMMMUIdx_S1NSE0; + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE0) || + mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE1)) { + mmu_idx += arm_mmu_bit_to_idx(ARMMMUBit_S1NSE0); } tcr = regime_tcr(env, mmu_idx); @@ -6809,8 +6803,9 @@ uint32_t arm_regime_tbi1(CPUARMState *env, ARMMMUIdx mmu_idx) /* For EL0 and EL1, TBI is controlled by stage 1's TCR, so convert * a stage 1+2 mmu index into the appropriate stage 1 mmu index. */ - if (mmu_idx == ARMMMUIdx_S12NSE0 || mmu_idx == ARMMMUIdx_S12NSE1) { - mmu_idx += ARMMMUIdx_S1NSE0; + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE0) || + mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE1)) { + mmu_idx += arm_mmu_bit_to_idx(ARMMMUBit_S1NSE0); } tcr = regime_tcr(env, mmu_idx); @@ -6827,7 +6822,7 @@ uint32_t arm_regime_tbi1(CPUARMState *env, ARMMMUIdx mmu_idx) static inline uint64_t regime_ttbr(CPUARMState *env, ARMMMUIdx mmu_idx, int ttbrn) { - if (mmu_idx == ARMMMUIdx_S2NS) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)) { return env->cp15.vttbr_el2; } if (ttbrn == 0) { @@ -6857,8 +6852,9 @@ static inline bool regime_using_lpae_format(CPUARMState *env, * on whether the long or short descriptor format is in use. */ bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx) { - if (mmu_idx == ARMMMUIdx_S12NSE0 || mmu_idx == ARMMMUIdx_S12NSE1) { - mmu_idx += ARMMMUIdx_S1NSE0; + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE0) || + mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE1)) { + mmu_idx += arm_mmu_bit_to_idx(ARMMMUBit_S1NSE0); } return regime_using_lpae_format(env, mmu_idx); @@ -6866,15 +6862,14 @@ bool arm_s1_regime_using_lpae_format(CPUARMState *env, ARMMMUIdx mmu_idx) static inline bool regime_is_user(CPUARMState *env, ARMMMUIdx mmu_idx) { - switch (mmu_idx) { - case ARMMMUIdx_S1SE0: - case ARMMMUIdx_S1NSE0: + ARMMMUBitMap bit = arm_mmu_idx_to_bit(mmu_idx); + + if (bit & (ARMMMUBit_S1SE0 | ARMMMUBit_S1NSE0)) { return true; - default: - return false; - case ARMMMUIdx_S12NSE0: - case ARMMMUIdx_S12NSE1: + } else if (bit & (ARMMMUBit_S12NSE0 | ARMMMUBit_S12NSE1)) { g_assert_not_reached(); + } else { + return false; } } @@ -7004,7 +6999,7 @@ static int get_S1prot(CPUARMState *env, ARMMMUIdx mmu_idx, bool is_aa64, bool have_wxn; int wxn = 0; - assert(mmu_idx != ARMMMUIdx_S2NS); + assert(mmu_idx != ARMMMUBit_S2NS); user_rw = simple_ap_to_rw_prot_is_user(ap, true); if (is_user) { @@ -7096,14 +7091,15 @@ static hwaddr S1_ptw_translate(CPUARMState *env, ARMMMUIdx mmu_idx, uint32_t *fsr, ARMMMUFaultInfo *fi) { - if ((mmu_idx == ARMMMUIdx_S1NSE0 || mmu_idx == ARMMMUIdx_S1NSE1) && - !regime_translation_disabled(env, ARMMMUIdx_S2NS)) { + if ((mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S1NSE0) || + mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S1NSE1)) && + !regime_translation_disabled(env, arm_mmu_bit_to_idx(ARMMMUBit_S2NS))) { target_ulong s2size; hwaddr s2pa; int s2prot; int ret; - ret = get_phys_addr_lpae(env, addr, 0, ARMMMUIdx_S2NS, &s2pa, + ret = get_phys_addr_lpae(env, addr, 0, ARMMMUBit_S2NS, &s2pa, &txattrs, &s2prot, &s2size, fsr, fi); if (ret) { fi->s2addr = addr; @@ -7546,7 +7542,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, level = 0; addrsize = 64; if (el > 1) { - if (mmu_idx != ARMMMUIdx_S2NS) { + if (mmu_idx != ARMMMUBit_S2NS) { tbi = extract64(tcr->raw_tcr, 20, 1); } } else { @@ -7583,7 +7579,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, t0sz = extract32(tcr->raw_tcr, 0, 6); t0sz = MIN(t0sz, 39); t0sz = MAX(t0sz, 16); - } else if (mmu_idx != ARMMMUIdx_S2NS) { + } else if (mmu_idx != ARMMMUBit_S2NS) { /* AArch32 stage 1 translation. */ t0sz = extract32(tcr->raw_tcr, 0, 3); } else { @@ -7677,7 +7673,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, goto do_fault; } - if (mmu_idx != ARMMMUIdx_S2NS) { + if (mmu_idx != ARMMMUBit_S2NS) { /* The starting level depends on the virtual address size (which can * be up to 48 bits) and the translation granule size. It indicates * the number of strides (stride bits at a time) needed to @@ -7777,7 +7773,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, attrs = extract64(descriptor, 2, 10) | (extract64(descriptor, 52, 12) << 10); - if (mmu_idx == ARMMMUIdx_S2NS) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)) { /* Stage 2 table descriptors do not include any attribute fields */ break; } @@ -7805,7 +7801,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, target_ulong address, ap = extract32(attrs, 4, 2); xn = extract32(attrs, 12, 1); - if (mmu_idx == ARMMMUIdx_S2NS) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)) { ns = true; *prot = get_S2prot(env, ap, xn); } else { @@ -7834,7 +7830,7 @@ do_fault: /* Long-descriptor format IFSR/DFSR value */ *fsr = (1 << 9) | (fault_type << 2) | level; /* Tag the error as S2 for failed S1 PTW at S2 or ordinary S2. */ - fi->stage2 = fi->s1ptw || (mmu_idx == ARMMMUIdx_S2NS); + fi->stage2 = fi->s1ptw || (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S2NS)); return true; } @@ -8103,7 +8099,8 @@ static bool get_phys_addr(CPUARMState *env, target_ulong address, target_ulong *page_size, uint32_t *fsr, ARMMMUFaultInfo *fi) { - if (mmu_idx == ARMMMUIdx_S12NSE0 || mmu_idx == ARMMMUIdx_S12NSE1) { + if (mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE0) || + mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S12NSE1)) { /* Call ourselves recursively to do the stage 1 and then stage 2 * translations. */ @@ -8113,17 +8110,17 @@ static bool get_phys_addr(CPUARMState *env, target_ulong address, int ret; ret = get_phys_addr(env, address, access_type, - mmu_idx + ARMMMUIdx_S1NSE0, &ipa, attrs, + mmu_idx + ARMMMUBit_S1NSE0, &ipa, attrs, prot, page_size, fsr, fi); /* If S1 fails or S2 is disabled, return early. */ - if (ret || regime_translation_disabled(env, ARMMMUIdx_S2NS)) { + if (ret || regime_translation_disabled(env, ARMMMUBit_S2NS)) { *phys_ptr = ipa; return ret; } /* S1 is done. Now do S2 translation. */ - ret = get_phys_addr_lpae(env, ipa, access_type, ARMMMUIdx_S2NS, + ret = get_phys_addr_lpae(env, ipa, access_type, ARMMMUBit_S2NS, phys_ptr, attrs, &s2_prot, page_size, fsr, fi); fi->s2addr = ipa; @@ -8134,7 +8131,7 @@ static bool get_phys_addr(CPUARMState *env, target_ulong address, /* * For non-EL2 CPUs a stage1+stage2 translation is just stage 1. */ - mmu_idx += ARMMMUIdx_S1NSE0; + mmu_idx += arm_mmu_bit_to_idx(ARMMMUBit_S1NSE0); } } @@ -8148,7 +8145,7 @@ static bool get_phys_addr(CPUARMState *env, target_ulong address, /* Fast Context Switch Extension. This doesn't exist at all in v8. * In v7 and earlier it affects all stage 1 translations. */ - if (address < 0x02000000 && mmu_idx != ARMMMUIdx_S2NS + if (address < 0x02000000 && mmu_idx != arm_mmu_bit_to_idx(ARMMMUBit_S2NS) && !arm_feature(env, ARM_FEATURE_V8)) { if (regime_el(env, mmu_idx) == 3) { address += env->cp15.fcseidr_s; diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index d0352e2045..88a4df6959 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -106,14 +106,14 @@ static inline ARMMMUIdx get_a64_user_mem_index(DisasContext *s) /* Return the mmu_idx to use for A64 "unprivileged load/store" insns: * if EL1, access as if EL0; otherwise access at current EL */ - switch (s->mmu_idx) { - case ARMMMUIdx_S12NSE1: - return ARMMMUIdx_S12NSE0; - case ARMMMUIdx_S1SE1: - return ARMMMUIdx_S1SE0; - case ARMMMUIdx_S2NS: + ARMMMUBitMap bit = arm_mmu_idx_to_bit(s->mmu_idx); + if (bit & ARMMMUBit_S12NSE1) { + return ARMMMUBit_S12NSE0; + } else if (bit & ARMMMUBit_S1SE1) { + return ARMMMUBit_S1SE0; + } else if (bit & ARMMMUBit_S2NS) { g_assert_not_reached(); - default: + } else { return s->mmu_idx; } } diff --git a/target/arm/translate.c b/target/arm/translate.c index c9186b6195..dc67887918 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -109,19 +109,17 @@ static inline ARMMMUIdx get_a32_user_mem_index(DisasContext *s) * if PL2, UNPREDICTABLE (we choose to implement as if PL0) * otherwise, access as if at PL0. */ - switch (s->mmu_idx) { - case ARMMMUIdx_S1E2: /* this one is UNPREDICTABLE */ - case ARMMMUIdx_S12NSE0: - case ARMMMUIdx_S12NSE1: - return ARMMMUIdx_S12NSE0; - case ARMMMUIdx_S1E3: - case ARMMMUIdx_S1SE0: - case ARMMMUIdx_S1SE1: - return ARMMMUIdx_S1SE0; - case ARMMMUIdx_S2NS: - default: - g_assert_not_reached(); - } + ARMMMUBitMap bit = arm_mmu_idx_to_bit(s->mmu_idx); + if (bit & (ARMMMUBit_S1E2 | /* this one is UNPREDICTABLE */ + ARMMMUBit_S12NSE0 | + ARMMMUBit_S12NSE1)) { + return arm_mmu_bit_to_idx(ARMMMUBit_S12NSE0); + } else if (bit & (ARMMMUBit_S1E3 | + ARMMMUBit_S1SE0 | + ARMMMUBit_S1SE1)) { + return arm_mmu_bit_to_idx(ARMMMUBit_S1SE0); + } + g_assert_not_reached(); } static inline TCGv_i32 load_cpu_offset(int offset) diff --git a/target/arm/translate.h b/target/arm/translate.h index 285e96f087..8011b8562d 100644 --- a/target/arm/translate.h +++ b/target/arm/translate.h @@ -100,8 +100,8 @@ static inline int default_exception_el(DisasContext *s) * exceptions can only be routed to ELs above 1, so we target the higher of * 1 or the current EL. */ - return (s->mmu_idx == ARMMMUIdx_S1SE0 && s->secure_routed_to_el3) - ? 3 : MAX(1, s->current_el); + return (s->mmu_idx == arm_mmu_bit_to_idx(ARMMMUBit_S1SE0) + && s->secure_routed_to_el3) ? 3 : MAX(1, s->current_el); } /* target-specific extra values for is_jmp */ diff --git a/target/sparc/ldst_helper.c b/target/sparc/ldst_helper.c index 2c05d6af75..57968d9143 100644 --- a/target/sparc/ldst_helper.c +++ b/target/sparc/ldst_helper.c @@ -1768,13 +1768,15 @@ void helper_st_asi(CPUSPARCState *env, target_ulong addr, target_ulong val, case 1: env->dmmu.mmu_primary_context = val; env->immu.mmu_primary_context = val; - tlb_flush_by_mmuidx(CPU(cpu), MMU_USER_IDX, MMU_KERNEL_IDX, -1); + tlb_flush_by_mmuidx(CPU(cpu), + (1 << MMU_USER_IDX) | (1 << MMU_KERNEL_IDX)); break; case 2: env->dmmu.mmu_secondary_context = val; env->immu.mmu_secondary_context = val; - tlb_flush_by_mmuidx(CPU(cpu), MMU_USER_SECONDARY_IDX, - MMU_KERNEL_SECONDARY_IDX, -1); + tlb_flush_by_mmuidx(CPU(cpu), + (1 << MMU_USER_SECONDARY_IDX) | + (1 << MMU_KERNEL_SECONDARY_IDX)); break; default: cpu_unassigned_access(cs, addr, true, false, 1, size); From patchwork Fri Jan 27 10:34:57 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92638 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp187029qgi; Fri, 27 Jan 2017 03:29:04 -0800 (PST) X-Received: by 10.55.178.132 with SMTP id b126mr7203793qkf.225.1485516544821; Fri, 27 Jan 2017 03:29:04 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id d39si3315380qtf.224.2017.01.27.03.29.04 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:29:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44528 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4iD-0008Nn-CT for patch@linaro.org; Fri, 27 Jan 2017 06:29:01 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49523) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40J-0007bJ-0I for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40H-0004dk-A9 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:39 -0500 Received: from mail-wm0-x22f.google.com ([2a00:1450:400c:c09::22f]:36931) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40H-0004cx-0Y for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:37 -0500 Received: by mail-wm0-x22f.google.com with SMTP id c206so130356319wme.0 for ; Fri, 27 Jan 2017 02:43:36 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kzGiKrvslo18E91aONEhVqBi6bz2lY/TTgH5q2P/rEM=; b=gel96Rhr92t4B/R71l1do/+D1V4KnpILfOLOLrE5MVa1ZjsuCpSMVoc+Fp6SSPnOL8 D3tUC3c7Ue4WFin6XkUBZ04UPK6iGAjZZ6rBqLlSewGwCmyZjTe/DSdJEmO7ru0xh31k pcZlt4dCOQhAt8GAndQDq7d4V11hQtUCOUPS4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kzGiKrvslo18E91aONEhVqBi6bz2lY/TTgH5q2P/rEM=; b=tOFSmw7rY5/DHhZMXMdZGZE/ZkzdtKO/VTUSHN+lAZgQzujkpApNEv8cci7Nf9BW13 76DBNrFO7MF48tuNsTRiIIsdOTwZRoQM0v6pn9VlgsQGqwy8JYzO9KK9eI6kmaKfkrjX 1PXezh/t9wAJmy6yMRs66TjN6uzFYzqnjZHHlIPT9mZr2SouXJh6JQSgbTrU9v2cKJkx pbfQsWyfn0VHuhIA7/yOwiLcoIGNqmIIFCDvfXMn3NrG70Qzq+evsG1AoJTtgIvXKlfr 6FwQ4Zz6jBU+iK7zlWv+LhECLYGuDyNtYuc2iQV9CHNCYHyYnl/1tWL+ZBSl9Ru3zZCx QoTQ== X-Gm-Message-State: AIkVDXLW1xPGSZtXK0SKvWy/w9CcaZiqjYLLbqemUE5LYaAKY5PktECeDcUchlJrhTKR9V1F X-Received: by 10.28.64.213 with SMTP id n204mr2746598wma.12.1485513815872; Fri, 27 Jan 2017 02:43:35 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id h3sm7222129wrb.31.2017.01.27.02.43.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:33 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id D5B593E3769; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:57 +0000 Message-Id: <20170127103505.18606-18-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22f Subject: [Qemu-devel] [PATCH v8 17/25] cputlb: add tlb_flush_by_mmuidx async routines X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This converts the remaining TLB flush routines to use async work when detecting a cross-vCPU flush. The only minor complication is having to serialise the var_list of MMU indexes into a form that can be punted to an asynchronous job. The pending_tlb_flush field on QOM's CPU structure also becomes a bitfield rather than a boolean. Signed-off-by: Alex Bennée --- v7 - un-merged from the atomic cputlb patch in the last series - fix long line reported by checkpatch v8 - re-base merge/fixes --- cputlb.c | 110 +++++++++++++++++++++++++++++++++++++++++++----------- include/qom/cpu.h | 2 +- 2 files changed, 89 insertions(+), 23 deletions(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index 97e5c12de8..c50254be26 100644 --- a/cputlb.c +++ b/cputlb.c @@ -68,6 +68,11 @@ * target_ulong even on 32 bit builds */ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data)); +/* We currently can't handle more than 16 bits in the MMUIDX bitmask. + */ +QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16); +#define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1) + /* statistics */ int tlb_flush_count; @@ -102,7 +107,7 @@ static void tlb_flush_nocheck(CPUState *cpu) tb_unlock(); - atomic_mb_set(&cpu->pending_tlb_flush, false); + atomic_mb_set(&cpu->pending_tlb_flush, 0); } static void tlb_flush_global_async_work(CPUState *cpu, run_on_cpu_data data) @@ -113,7 +118,8 @@ static void tlb_flush_global_async_work(CPUState *cpu, run_on_cpu_data data) void tlb_flush(CPUState *cpu) { if (cpu->created && !qemu_cpu_is_self(cpu)) { - if (atomic_cmpxchg(&cpu->pending_tlb_flush, false, true) == true) { + if (atomic_mb_read(&cpu->pending_tlb_flush) != ALL_MMUIDX_BITS) { + atomic_mb_set(&cpu->pending_tlb_flush, ALL_MMUIDX_BITS); async_run_on_cpu(cpu, tlb_flush_global_async_work, RUN_ON_CPU_NULL); } @@ -122,17 +128,18 @@ void tlb_flush(CPUState *cpu) } } -static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) +static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; - unsigned long mmu_idx_bitmask = idxmap; + unsigned long mmu_idx_bitmask = data.host_int; int mmu_idx; assert_cpu_is_self(cpu); - tlb_debug("start\n"); tb_lock(); + tlb_debug("start: mmu_idx:0x%04lx\n", mmu_idx_bitmask); + for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { if (test_bit(mmu_idx, &mmu_idx_bitmask)) { @@ -145,12 +152,30 @@ static inline void v_tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) memset(cpu->tb_jmp_cache, 0, sizeof(cpu->tb_jmp_cache)); + tlb_debug("done\n"); + tb_unlock(); } void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) { - v_tlb_flush_by_mmuidx(cpu, idxmap); + tlb_debug("mmu_idx: 0x%" PRIx16 "\n", idxmap); + + if (!qemu_cpu_is_self(cpu)) { + uint16_t pending_flushes = idxmap; + pending_flushes &= ~atomic_mb_read(&cpu->pending_tlb_flush); + + if (pending_flushes) { + tlb_debug("reduced mmu_idx: 0x%" PRIx16 "\n", pending_flushes); + + atomic_or(&cpu->pending_tlb_flush, pending_flushes); + async_run_on_cpu(cpu, tlb_flush_by_mmuidx_async_work, + RUN_ON_CPU_HOST_INT(pending_flushes)); + } + } else { + tlb_flush_by_mmuidx_async_work(cpu, + RUN_ON_CPU_HOST_INT(idxmap)); + } } static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) @@ -215,27 +240,26 @@ void tlb_flush_page(CPUState *cpu, target_ulong addr) } } -void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) +/* As we are going to hijack the bottom bits of the page address for a + * mmuidx bit mask we need to fail to build if we can't do that + */ +QEMU_BUILD_BUG_ON(NB_MMU_MODES > TARGET_PAGE_BITS_MIN); + +static void tlb_flush_page_by_mmuidx_async_work(CPUState *cpu, + run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; - unsigned long mmu_idx_bitmap = idxmap; - int i, page, mmu_idx; + target_ulong addr_and_mmuidx = (target_ulong) data.target_ptr; + target_ulong addr = addr_and_mmuidx & TARGET_PAGE_MASK; + unsigned long mmu_idx_bitmap = addr_and_mmuidx & ALL_MMUIDX_BITS; + int page = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); + int mmu_idx; + int i; assert_cpu_is_self(cpu); - tlb_debug("addr "TARGET_FMT_lx"\n", addr); - - /* Check if we need to flush due to large pages. */ - if ((addr & env->tlb_flush_mask) == env->tlb_flush_addr) { - tlb_debug("forced full flush (" - TARGET_FMT_lx "/" TARGET_FMT_lx ")\n", - env->tlb_flush_addr, env->tlb_flush_mask); - - v_tlb_flush_by_mmuidx(cpu, idxmap); - return; - } - addr &= TARGET_PAGE_MASK; - page = (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); + tlb_debug("page:%d addr:"TARGET_FMT_lx" mmu_idx:0x%lx\n", + page, addr, mmu_idx_bitmap); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { if (test_bit(mmu_idx, &mmu_idx_bitmap)) { @@ -251,6 +275,48 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) tb_flush_jmp_cache(cpu, addr); } +static void tlb_check_page_and_flush_by_mmuidx_async_work(CPUState *cpu, + run_on_cpu_data data) +{ + CPUArchState *env = cpu->env_ptr; + target_ulong addr_and_mmuidx = (target_ulong) data.target_ptr; + target_ulong addr = addr_and_mmuidx & TARGET_PAGE_MASK; + unsigned long mmu_idx_bitmap = addr_and_mmuidx & ALL_MMUIDX_BITS; + + tlb_debug("addr:"TARGET_FMT_lx" mmu_idx: %04lx\n", addr, mmu_idx_bitmap); + + /* Check if we need to flush due to large pages. */ + if ((addr & env->tlb_flush_mask) == env->tlb_flush_addr) { + tlb_debug("forced full flush (" + TARGET_FMT_lx "/" TARGET_FMT_lx ")\n", + env->tlb_flush_addr, env->tlb_flush_mask); + + tlb_flush_by_mmuidx_async_work(cpu, + RUN_ON_CPU_HOST_INT(mmu_idx_bitmap)); + } else { + tlb_flush_page_by_mmuidx_async_work(cpu, data); + } +} + +void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) +{ + target_ulong addr_and_mmu_idx; + + tlb_debug("addr: "TARGET_FMT_lx" mmu_idx:%" PRIx16 "\n", addr, idxmap); + + /* This should already be page aligned */ + addr_and_mmu_idx = addr & TARGET_PAGE_MASK; + addr_and_mmu_idx |= idxmap; + + if (!qemu_cpu_is_self(cpu)) { + async_run_on_cpu(cpu, tlb_check_page_and_flush_by_mmuidx_async_work, + RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); + } else { + tlb_check_page_and_flush_by_mmuidx_async_work( + cpu, RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); + } +} + void tlb_flush_page_all(target_ulong addr) { CPUState *cpu; diff --git a/include/qom/cpu.h b/include/qom/cpu.h index 7f1d6a81a0..d996e5a0f4 100644 --- a/include/qom/cpu.h +++ b/include/qom/cpu.h @@ -403,7 +403,7 @@ struct CPUState { * avoid potential races. The aim of the flag is to avoid * unnecessary flushes. */ - bool pending_tlb_flush; + uint16_t pending_tlb_flush; }; QTAILQ_HEAD(CPUTailQ, CPUState); From patchwork Fri Jan 27 10:34:58 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92641 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp158288obz; Fri, 27 Jan 2017 03:33:24 -0800 (PST) X-Received: by 10.237.59.213 with SMTP id s21mr6983447qte.146.1485516804159; Fri, 27 Jan 2017 03:33:24 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id m29si3321631qki.224.2017.01.27.03.33.23 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:33:24 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44557 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4mP-0004Lg-CB for patch@linaro.org; Fri, 27 Jan 2017 06:33:21 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49639) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40P-0007hw-1M for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40N-0004in-IL for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:45 -0500 Received: from mail-wm0-x22a.google.com ([2a00:1450:400c:c09::22a]:35353) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40N-0004iQ-9h for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:43 -0500 Received: by mail-wm0-x22a.google.com with SMTP id r126so111094477wmr.0 for ; Fri, 27 Jan 2017 02:43:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0tVFYz/nzL0By9w3sLfPvjvA6MQu3IYvxpDN14+1TYM=; b=hWSuHR+v5mnspNC0ut3AmGW8nR4GDywfTQGkIvS6fB5TPuipnu6bShPhTIFoPVAEVw DDRe1t483bX24hI3VpVb5eBr8aXTrgJmWWPFGJznOPODX9noti7Sn4+pmkEhvMaYFTqJ hA6Sxde7JGB0aebCtPbi0+4wXMZ+DZrmoAFLA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0tVFYz/nzL0By9w3sLfPvjvA6MQu3IYvxpDN14+1TYM=; b=oewDgiGhA87Q+BDn1PUIHRwzRhmpkRFSW8SbW4PQk7/ZhPyOj3yS2d+nWNx9azhOgt M0SAiqAu8g4IKEM9obu3cDC+R3G6HnSKOOdSHNf7pmK6eO+pOpD0tLA7fyGuqhxDO2nf nROQNmlqmpsTOiivBE+J6hgR+C/419mn3PdIgtiO5ZggQ7zITQPsYfMsfGYE4EG7HzGJ mwzpmoWlVAdhyHo15ebYMdZLPZruR5WQh7ND/31iAV8FI11jILwc7HeCGw29m/eqz8Kh GxOtuTIk1rKnqfVFzLIMiYAVb+F4r8tKDRgVZJNKGeAJg2JES94C7doXAVw8ci+0WFwC 58Rw== X-Gm-Message-State: AIkVDXI8lqmn6izjFk4SrXmWfkgS3XoVvTTHoD3opdQjopEUqzs4Gc9ybXwwTFKxJgFNQx5E X-Received: by 10.28.224.10 with SMTP id x10mr2625108wmg.134.1485513822124; Fri, 27 Jan 2017 02:43:42 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id q1sm3204756wmd.6.2017.01.27.02.43.36 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:38 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id E770A3E376A; Fri, 27 Jan 2017 10:35:06 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:58 +0000 Message-Id: <20170127103505.18606-19-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22a Subject: [Qemu-devel] [PATCH v8 18/25] cputlb: atomically update tlb fields used by tlb_reset_dirty X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The main use case for tlb_reset_dirty is to set the TLB_NOTDIRTY flags in TLB entries to force the slow-path on writes. This is used to mark page ranges containing code which has been translated so it can be invalidated if written to. To do this safely we need to ensure the TLB entries in question for all vCPUs are updated before we attempt to run the code otherwise a race could be introduced. To achieve this we atomically set the flag in tlb_reset_dirty_range and take care when setting it when the TLB entry is filled. On 32 bit systems attempting to emulate 64 bit guests we don't even bother as we might not have the atomic primitives available. MTTCG is disabled in this case and can't be forced on. The copy_tlb_helper function helps keep the atomic semantics in one place to avoid confusion. The dirty helper function is made static as it isn't used outside of cputlb. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v6 - use TARGET_PAGE_BITS_MIN - use run_on_cpu helpers v7 - fix tlb_debug fmt for 32bit build - un-merged the mmuidx async work which got mashed in last round - introduced copy_tlb_helper function and made TCG_OVERSIZED_GUEST aware --- cputlb.c | 120 +++++++++++++++++++++++++++++++++++++++----------- include/exec/cputlb.h | 2 - 2 files changed, 95 insertions(+), 27 deletions(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index c50254be26..65003350e3 100644 --- a/cputlb.c +++ b/cputlb.c @@ -342,32 +342,90 @@ void tlb_unprotect_code(ram_addr_t ram_addr) cpu_physical_memory_set_dirty_flag(ram_addr, DIRTY_MEMORY_CODE); } -static bool tlb_is_dirty_ram(CPUTLBEntry *tlbe) -{ - return (tlbe->addr_write & (TLB_INVALID_MASK|TLB_MMIO|TLB_NOTDIRTY)) == 0; -} -void tlb_reset_dirty_range(CPUTLBEntry *tlb_entry, uintptr_t start, +/* + * Dirty write flag handling + * + * When the TCG code writes to a location it looks up the address in + * the TLB and uses that data to compute the final address. If any of + * the lower bits of the address are set then the slow path is forced. + * There are a number of reasons to do this but for normal RAM the + * most usual is detecting writes to code regions which may invalidate + * generated code. + * + * Because we want other vCPUs to respond to changes straight away we + * update the te->addr_write field atomically. If the TLB entry has + * been changed by the vCPU in the mean time we skip the update. + * + * As this function uses atomic accesses we also need to ensure + * updates to tlb_entries follow the same access rules. We don't need + * to worry about this for oversized guests as MTTCG is disabled for + * them. + */ + +static void tlb_reset_dirty_range(CPUTLBEntry *tlb_entry, uintptr_t start, uintptr_t length) { - uintptr_t addr; +#if TCG_OVERSIZED_GUEST + uintptr_t addr = tlb_entry->addr_write; - if (tlb_is_dirty_ram(tlb_entry)) { - addr = (tlb_entry->addr_write & TARGET_PAGE_MASK) + tlb_entry->addend; + if ((addr & (TLB_INVALID_MASK | TLB_MMIO | TLB_NOTDIRTY)) == 0) { + addr &= TARGET_PAGE_MASK; + addr += tlb_entry->addend; if ((addr - start) < length) { tlb_entry->addr_write |= TLB_NOTDIRTY; } } +#else + /* paired with atomic_mb_set in tlb_set_page_with_attrs */ + uintptr_t orig_addr = atomic_mb_read(&tlb_entry->addr_write); + uintptr_t addr = orig_addr; + + if ((addr & (TLB_INVALID_MASK | TLB_MMIO | TLB_NOTDIRTY)) == 0) { + addr &= TARGET_PAGE_MASK; + addr += atomic_read(&tlb_entry->addend); + if ((addr - start) < length) { + uintptr_t notdirty_addr = orig_addr | TLB_NOTDIRTY; + atomic_cmpxchg(&tlb_entry->addr_write, orig_addr, notdirty_addr); + } + } +#endif +} + +/* For atomic correctness when running MTTCG we need to use the right + * primitives when copying entries */ +static inline void copy_tlb_helper(CPUTLBEntry *d, CPUTLBEntry *s, + bool atomic_set) +{ +#if TCG_OVERSIZED_GUEST + *d = *s; +#else + if (atomic_set) { + d->addr_read = s->addr_read; + d->addr_code = s->addr_code; + atomic_set(&d->addend, atomic_read(&s->addend)); + /* Pairs with flag setting in tlb_reset_dirty_range */ + atomic_mb_set(&d->addr_write, atomic_read(&s->addr_write)); + } else { + d->addr_read = s->addr_read; + d->addr_write = atomic_read(&s->addr_write); + d->addr_code = s->addr_code; + d->addend = atomic_read(&s->addend); + } +#endif } +/* This is a cross vCPU call (i.e. another vCPU resetting the flags of + * the target vCPU). As such care needs to be taken that we don't + * dangerously race with another vCPU update. The only thing actually + * updated is the target TLB entry ->addr_write flags. + */ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) { CPUArchState *env; int mmu_idx; - assert_cpu_is_self(cpu); - env = cpu->env_ptr; for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { unsigned int i; @@ -455,7 +513,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, target_ulong address; target_ulong code_address; uintptr_t addend; - CPUTLBEntry *te; + CPUTLBEntry *te, *tv, tn; hwaddr iotlb, xlat, sz; unsigned vidx = env->vtlb_index++ % CPU_VTLB_SIZE; int asidx = cpu_asidx_from_attrs(cpu, attrs); @@ -490,41 +548,50 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, index = (vaddr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); te = &env->tlb_table[mmu_idx][index]; - /* do not discard the translation in te, evict it into a victim tlb */ - env->tlb_v_table[mmu_idx][vidx] = *te; + tv = &env->tlb_v_table[mmu_idx][vidx]; + + /* addr_write can race with tlb_reset_dirty_range */ + copy_tlb_helper(tv, te, true); + env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index]; /* refill the tlb */ env->iotlb[mmu_idx][index].addr = iotlb - vaddr; env->iotlb[mmu_idx][index].attrs = attrs; - te->addend = addend - vaddr; + + /* Now calculate the new entry */ + tn.addend = addend - vaddr; if (prot & PAGE_READ) { - te->addr_read = address; + tn.addr_read = address; } else { - te->addr_read = -1; + tn.addr_read = -1; } if (prot & PAGE_EXEC) { - te->addr_code = code_address; + tn.addr_code = code_address; } else { - te->addr_code = -1; + tn.addr_code = -1; } + + tn.addr_write = -1; if (prot & PAGE_WRITE) { if ((memory_region_is_ram(section->mr) && section->readonly) || memory_region_is_romd(section->mr)) { /* Write access calls the I/O callback. */ - te->addr_write = address | TLB_MMIO; + tn.addr_write = address | TLB_MMIO; } else if (memory_region_is_ram(section->mr) && cpu_physical_memory_is_clean( memory_region_get_ram_addr(section->mr) + xlat)) { - te->addr_write = address | TLB_NOTDIRTY; + tn.addr_write = address | TLB_NOTDIRTY; } else { - te->addr_write = address; + tn.addr_write = address; } - } else { - te->addr_write = -1; } + + /* Pairs with flag setting in tlb_reset_dirty_range */ + copy_tlb_helper(te, &tn, true); + /* atomic_mb_set(&te->addr_write, write_address); */ } /* Add a new TLB entry, but without specifying the memory @@ -687,10 +754,13 @@ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index, if (cmp == page) { /* Found entry in victim tlb, swap tlb and iotlb. */ CPUTLBEntry tmptlb, *tlb = &env->tlb_table[mmu_idx][index]; + + copy_tlb_helper(&tmptlb, tlb, false); + copy_tlb_helper(tlb, vtlb, true); + copy_tlb_helper(vtlb, &tmptlb, true); + CPUIOTLBEntry tmpio, *io = &env->iotlb[mmu_idx][index]; CPUIOTLBEntry *vio = &env->iotlb_v[mmu_idx][vidx]; - - tmptlb = *tlb; *tlb = *vtlb; *vtlb = tmptlb; tmpio = *io; *io = *vio; *vio = tmpio; return true; } diff --git a/include/exec/cputlb.h b/include/exec/cputlb.h index d454c005b7..3f941783c5 100644 --- a/include/exec/cputlb.h +++ b/include/exec/cputlb.h @@ -23,8 +23,6 @@ /* cputlb.c */ void tlb_protect_code(ram_addr_t ram_addr); void tlb_unprotect_code(ram_addr_t ram_addr); -void tlb_reset_dirty_range(CPUTLBEntry *tlb_entry, uintptr_t start, - uintptr_t length); extern int tlb_flush_count; #endif From patchwork Fri Jan 27 10:34:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92605 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp143173obz; Fri, 27 Jan 2017 02:52:10 -0800 (PST) X-Received: by 10.237.53.56 with SMTP id a53mr7612550qte.78.1485514330096; Fri, 27 Jan 2017 02:52:10 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id s49si3255994qtb.234.2017.01.27.02.52.09 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:52:10 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44296 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX48V-00069w-4m for patch@linaro.org; Fri, 27 Jan 2017 05:52:07 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48112) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sP-00081m-9C for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sN-0002FM-5y for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:29 -0500 Received: from mail-wm0-x232.google.com ([2a00:1450:400c:c09::232]:35442) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sM-0002Eq-Tm for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:27 -0500 Received: by mail-wm0-x232.google.com with SMTP id r126so110796511wmr.0 for ; Fri, 27 Jan 2017 02:35:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=y0qb/ZNL0M+Z6lcBvCeXxRcVaqF7l0wGhpJK9E+gUpY=; b=ZaxL4cQKMK94uqbclTI0/w6mii55kzMa/3taZ/eMsY/6l1l4Gacu4eo0lxRwmGaKNa 8jAYKEgNRC4WIwD0nIOlLSQcVw5gCsO5d1Oy8HIiZhOn+qgvr3M+oLmgqrj0jhnrpJ7u hQ2JS4F2DBKBMFKW5e4jE1zuH62P8grN+a+Rs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y0qb/ZNL0M+Z6lcBvCeXxRcVaqF7l0wGhpJK9E+gUpY=; b=PkjU6RjBMSEWe5fuSecBau63/M2Glle4qUKFwNRzogEm0r+O51Tzl2CFQgmbiBj51A GR8Uyiivvou7xBsnGnInb22fXVLjLkzmFoqJwQCbzZD44y/fu6Wb40ehqFSm3kjR3AkC B232MjCsJNNfV/0iuWIv3n4DDYTO8jpQY7t1EqYv2FxD4Tc6zX6xoXNz0nubqZPrXVqL c1jW4/Ov8SXfEcUTbdUoKm8RJSG7WzIKZr6G0ylz6xU6l6kWZKl545GaHyICol9D8TpO TgcZ4x9pmn7+St+MUwFNof73Lm9bAdWEA2xgU1+lW+iZGTI+V61uhkUk9xHIJZb55Wch +ryg== X-Gm-Message-State: AIkVDXKTNuDILRUhiWoNpRNYvrfYGn6y5TTVSRLj5MXohEbH1Tq2P38sLEL2Wfe3ZQexGs/s X-Received: by 10.28.148.76 with SMTP id w73mr2560295wmd.43.1485513325673; Fri, 27 Jan 2017 02:35:25 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id g40sm7156445wrg.19.2017.01.27.02.35.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:19 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 04A273E376B; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:34:59 +0000 Message-Id: <20170127103505.18606-20-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::232 Subject: [Qemu-devel] [PATCH v8 19/25] cputlb: introduce tlb_flush_*_all_cpus[_synced] X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Paolo Bonzini , Richard Henderson , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:Overall" , Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This introduces support to the cputlb API for flushing all CPUs TLBs with one call. This avoids the need for target helpers to iterate through the vCPUs themselves. An additional variant of the API (_synced) do not return from the caller and will cause the work to be scheduled as "safe work". The result will be all the flush operations will be complete by the time the originating vCPU starts executing again. It is up to the caller to ensure enough state has been saved so execution can be restarted at the next appropriate instruction. Some guest architectures can defer completion of flush operations until later. If they later schedule work using the async_safe_work mechanism they can be sure other vCPUs will have flushed their TLBs by the point execution returns from the safe work. Signed-off-by: Alex Bennée --- v7 - some checkpatch long line fixes v8 - change from varg to bitmap calling convention - add _synced variants, re-factored helper --- cputlb.c | 110 +++++++++++++++++++++++++++++++++++++++++++--- include/exec/exec-all.h | 114 ++++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 215 insertions(+), 9 deletions(-) -- 2.11.0 diff --git a/cputlb.c b/cputlb.c index 65003350e3..7f9a54f253 100644 --- a/cputlb.c +++ b/cputlb.c @@ -73,6 +73,25 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data)); QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16); #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1) +/* flush_all_helper: run fn across all cpus + * + * If the wait flag is set then the src cpu's helper will be queued as + * "safe" work and the loop exited creating a synchronisation point + * where all queued work will be finished before execution starts + * again. + */ +static void flush_all_helper(CPUState *src, run_on_cpu_func fn, + run_on_cpu_data d) +{ + CPUState *cpu; + + CPU_FOREACH(cpu) { + if (cpu != src) { + async_run_on_cpu(cpu, fn, d); + } + } +} + /* statistics */ int tlb_flush_count; @@ -128,6 +147,19 @@ void tlb_flush(CPUState *cpu) } } +void tlb_flush_all_cpus(CPUState *src_cpu) +{ + flush_all_helper(src_cpu, tlb_flush_global_async_work, RUN_ON_CPU_NULL); + tlb_flush_global_async_work(src_cpu, RUN_ON_CPU_NULL); +} + +void QEMU_NORETURN tlb_flush_all_cpus_synced(CPUState *src_cpu) +{ + flush_all_helper(src_cpu, tlb_flush_global_async_work, RUN_ON_CPU_NULL); + tlb_flush_global_async_work(src_cpu, RUN_ON_CPU_NULL); + cpu_loop_exit(src_cpu); +} + static void tlb_flush_by_mmuidx_async_work(CPUState *cpu, run_on_cpu_data data) { CPUArchState *env = cpu->env_ptr; @@ -178,6 +210,30 @@ void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) } } +void tlb_flush_by_mmuidx_all_cpus(CPUState *src_cpu, uint16_t idxmap) +{ + const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work; + + tlb_debug("mmu_idx: 0x%"PRIx16"\n", idxmap); + + flush_all_helper(src_cpu, fn, RUN_ON_CPU_HOST_INT(idxmap)); + fn(src_cpu, RUN_ON_CPU_HOST_INT(idxmap)); +} + +void QEMU_NORETURN tlb_flush_by_mmuidx_all_cpus_synced(CPUState *src_cpu, + uint16_t idxmap) +{ + const run_on_cpu_func fn = tlb_flush_by_mmuidx_async_work; + + tlb_debug("mmu_idx: 0x%"PRIx16"\n", idxmap); + + flush_all_helper(src_cpu, fn, RUN_ON_CPU_HOST_INT(idxmap)); + async_safe_run_on_cpu(src_cpu, fn, RUN_ON_CPU_HOST_INT(idxmap)); + cpu_loop_exit(src_cpu); +} + + + static inline void tlb_flush_entry(CPUTLBEntry *tlb_entry, target_ulong addr) { if (addr == (tlb_entry->addr_read & @@ -317,14 +373,56 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) } } -void tlb_flush_page_all(target_ulong addr) +void tlb_flush_page_by_mmuidx_all_cpus(CPUState *src_cpu, target_ulong addr, + uint16_t idxmap) { - CPUState *cpu; + const run_on_cpu_func fn = tlb_check_page_and_flush_by_mmuidx_async_work; + target_ulong addr_and_mmu_idx; - CPU_FOREACH(cpu) { - async_run_on_cpu(cpu, tlb_flush_page_async_work, - RUN_ON_CPU_TARGET_PTR(addr)); - } + tlb_debug("addr: "TARGET_FMT_lx" mmu_idx:%"PRIx16"\n", addr, idxmap); + + /* This should already be page aligned */ + addr_and_mmu_idx = addr & TARGET_PAGE_MASK; + addr_and_mmu_idx |= idxmap; + + flush_all_helper(src_cpu, fn, RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); + fn(src_cpu, RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); +} + +void QEMU_NORETURN tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *src_cpu, + target_ulong addr, + uint16_t idxmap) +{ + const run_on_cpu_func fn = tlb_check_page_and_flush_by_mmuidx_async_work; + target_ulong addr_and_mmu_idx; + + tlb_debug("addr: "TARGET_FMT_lx" mmu_idx:%"PRIx16"\n", addr, idxmap); + + /* This should already be page aligned */ + addr_and_mmu_idx = addr & TARGET_PAGE_MASK; + addr_and_mmu_idx |= idxmap; + + flush_all_helper(src_cpu, fn, RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); + async_safe_run_on_cpu(src_cpu, fn, RUN_ON_CPU_TARGET_PTR(addr_and_mmu_idx)); + cpu_loop_exit(src_cpu); +} + +void tlb_flush_page_all_cpus(CPUState *src, target_ulong addr) +{ + const run_on_cpu_func fn = tlb_flush_page_async_work; + + flush_all_helper(src, fn, RUN_ON_CPU_TARGET_PTR(addr)); + fn(src, RUN_ON_CPU_TARGET_PTR(addr)); +} + +void QEMU_NORETURN tlb_flush_page_all_cpus_synced(CPUState *src, + target_ulong addr) +{ + const run_on_cpu_func fn = tlb_flush_page_async_work; + + flush_all_helper(src, fn, RUN_ON_CPU_TARGET_PTR(addr)); + async_safe_run_on_cpu(src, fn, RUN_ON_CPU_TARGET_PTR(addr)); + cpu_loop_exit(src); } /* update the TLBs so that writes to code in the virtual page 'addr' diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h index a6c17ed74a..cffed39fe9 100644 --- a/include/exec/exec-all.h +++ b/include/exec/exec-all.h @@ -93,6 +93,27 @@ void cpu_address_space_init(CPUState *cpu, AddressSpace *as, int asidx); */ void tlb_flush_page(CPUState *cpu, target_ulong addr); /** + * tlb_flush_page_all_cpus: + * @cpu: src CPU of the flush + * @addr: virtual address of page to be flushed + * + * Flush one page from the TLB of the specified CPU, for all + * MMU indexes. + */ +void tlb_flush_page_all_cpus(CPUState *src, target_ulong addr); +/** + * tlb_flush_page_all_cpus_synced: + * @cpu: src CPU of the flush + * @addr: virtual address of page to be flushed + * + * Flush one page from the TLB of the specified CPU, for all + * MMU indexes like tlb_flush_page_all_cpus except this function does + * not return and will restart the run-loop once all CPUs have + * executed the flush. + */ +void QEMU_NORETURN tlb_flush_page_all_cpus_synced(CPUState *src, + target_ulong addr); +/** * tlb_flush: * @cpu: CPU whose TLB should be flushed * @@ -103,6 +124,19 @@ void tlb_flush_page(CPUState *cpu, target_ulong addr); */ void tlb_flush(CPUState *cpu); /** + * tlb_flush_all_cpus: + * @cpu: src CPU of the flush + */ +void tlb_flush_all_cpus(CPUState *src_cpu); +/** + * tlb_flush_all_cpus_synced: + * @cpu: src CPU of the flush + * + * Like tlb_flush_all_cpus except this function does not return and + * will restart the run-loop once all CPUs have executed the flush. + */ +void QEMU_NORETURN tlb_flush_all_cpus_synced(CPUState *src_cpu); +/** * tlb_flush_page_by_mmuidx: * @cpu: CPU whose TLB should be flushed * @addr: virtual address of page to be flushed @@ -114,8 +148,34 @@ void tlb_flush(CPUState *cpu); void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap); /** + * tlb_flush_page_by_mmuidx_all_cpus: + * @cpu: Originating CPU of the flush + * @addr: virtual address of page to be flushed + * @idxmap: bitmap of MMU indexes to flush + * + * Flush one page from the TLB of all CPUs, for the specified + * MMU indexes. + */ +void tlb_flush_page_by_mmuidx_all_cpus(CPUState *cpu, target_ulong addr, + uint16_t idxmap); +/** + * tlb_flush_page_by_mmuidx_all_cpus_synced: + * @cpu: Originating CPU of the flush + * @addr: virtual address of page to be flushed + * @idxmap: bitmap of MMU indexes to flush + * + * Flush one page from the TLB of all CPUs, for the specified MMU + * indexes like tlb_flush_page_by_mmuidx_all_cpus except this function + * does not return and will restart the run-loop once all CPUs have + * executed the flush. + */ +void QEMU_NORETURN tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *cpu, + target_ulong addr, + uint16_t idxmap); +/** * tlb_flush_by_mmuidx: * @cpu: CPU whose TLB should be flushed + * @wait: If true ensure synchronisation by exiting the cpu_loop * @idxmap: bitmap of MMU indexes to flush * * Flush all entries from the TLB of the specified CPU, for the specified @@ -123,6 +183,27 @@ void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, */ void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap); /** + * tlb_flush_by_mmuidx_all_cpus: + * @cpu: Originating CPU of the flush + * @idxmap: bitmap of MMU indexes to flush + * + * Flush all entries from all TLBs of all CPUs, for the specified + * MMU indexes. + */ +void tlb_flush_by_mmuidx_all_cpus(CPUState *cpu, uint16_t idxmap); +/** + * tlb_flush_by_mmuidx_all_cpus_synced: + * @cpu: Originating CPU of the flush + * @idxmap: bitmap of MMU indexes to flush + * + * Flush all entries from all TLBs of all CPUs, for the specified + * MMU indexes like tlb_flush_by_mmuidx_all_cpus except this function + * does not return and will restart the run-loop once all CPUs have + * executed the flush. + */ +void QEMU_NORETURN tlb_flush_by_mmuidx_all_cpus_synced(CPUState *cpu, + uint16_t idxmap); +/** * tlb_set_page_with_attrs: * @cpu: CPU to add this TLB entry for * @vaddr: virtual address of page to add entry for @@ -159,16 +240,26 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr, void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr); void probe_write(CPUArchState *env, target_ulong addr, int mmu_idx, uintptr_t retaddr); -void tlb_flush_page_all(target_ulong addr); #else static inline void tlb_flush_page(CPUState *cpu, target_ulong addr) { } - +static inline void tlb_flush_page_all_cpus(CPUState *src, target_ulong addr) +{ +} +static inline void tlb_flush_page_all_cpus_synced(CPUState *src, + target_ulong addr) +{ +} static inline void tlb_flush(CPUState *cpu) { } - +static inline void tlb_flush_all_cpus(CPUState *src_cpu) +{ +} +static inline void tlb_flush_all_cpus_synced(CPUState *src_cpu) +{ +} static inline void tlb_flush_page_by_mmuidx(CPUState *cpu, target_ulong addr, uint16_t idxmap) { @@ -177,6 +268,23 @@ static inline void tlb_flush_page_by_mmuidx(CPUState *cpu, static inline void tlb_flush_by_mmuidx(CPUState *cpu, uint16_t idxmap) { } +static inline void tlb_flush_page_by_mmuidx_all_cpus(CPUState *cpu, + target_ulong addr, + uint16_t idxmap) +{ +} +static inline void tlb_flush_page_by_mmuidx_all_cpus_synced(CPUState *cpu, + target_ulong addr, + uint16_t idxmap) +{ +} +static inline void tlb_flush_by_mmuidx_all_cpus(CPUState *cpu, uint16_t idxmap) +{ +} +static inline void tlb_flush_by_mmuidx_all_cpus_synced(CPUState *cpu, + uint16_t idxmap) +{ +} #endif #define CODE_GEN_ALIGN 16 /* must be >= of the size of a icache line */ From patchwork Fri Jan 27 10:35:00 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92596 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp142290obz; Fri, 27 Jan 2017 02:49:37 -0800 (PST) X-Received: by 10.55.166.17 with SMTP id p17mr6994095qke.111.1485514177455; Fri, 27 Jan 2017 02:49:37 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id v201si3268137qka.33.2017.01.27.02.49.36 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 02:49:37 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44278 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX462-0003xr-R3 for patch@linaro.org; Fri, 27 Jan 2017 05:49:34 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48072) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX3sN-000805-MF for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:28 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX3sM-0002Ew-F2 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:27 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:37344) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX3sM-0002Ec-5w for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:35:26 -0500 Received: by mail-wm0-x236.google.com with SMTP id c206so130006196wme.0 for ; Fri, 27 Jan 2017 02:35:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wteZ9HPk/LB0YnUKNcFFZ3fdS6sGUP564feMjuMOMqs=; b=iR0+hJhqzMdPIG0GlgL8uP2kTvFtjJ8GaXIitLI5upPZLwD4/OzmUPIaCB5qOj7uTM u7Wfen74zdeblzVcXxfkBIGF9srYkjIUFN5deOsaKIA9f05DHqcSEBokB12IRUwaRvTO Yndv63D6wtJ0Osy0jjLnxg+P+XJIU5DNLM5Og= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wteZ9HPk/LB0YnUKNcFFZ3fdS6sGUP564feMjuMOMqs=; b=WyycqL9QxGa62saeN/daiwTLu3b7j7taBH/BZSn2JqFPKh2neABXAHGGvgwu7tPHKK s7ri56L3pDJL4kh+oaXa9YuMrvLxnjAG/ofZEbkYaw+FeIeFDMI3XN/aFqk4eqM08HM1 Hy6A8WhC29gBSmiGhwrEFyE5/ErOywC3xyr7BDclOrWuDhuJO8MnlmwQtsZ7DRN6yxeH dguBzviFTCkXwrosrjp1FJ6M70u5UCbhCsHnRrbo8F6TFi97IYDGJihzn51grAWuPN2b m0rODhGYb58yTf2yzJIJ4hSEzhg2720Uwvi5Sd+orOmZmiZ1VpalpLvxEVM5H62ufPG5 nPJg== X-Gm-Message-State: AIkVDXJlWWmnYRZRUNntTAyOBMWAuRiTruQZRmAXY1bNNKe9XupGV5WOZGiHio5qxDPujOmo X-Received: by 10.223.171.66 with SMTP id r2mr6686616wrc.71.1485513325000; Fri, 27 Jan 2017 02:35:25 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id d42sm7169848wrd.7.2017.01.27.02.35.14 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:35:19 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 156453E376C; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:00 +0000 Message-Id: <20170127103505.18606-21-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 20/25] target-arm/powerctl: defer cpu reset work to CPU context X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" When switching a new vCPU on we want to complete a bunch of the setup work before we start scheduling the vCPU thread. To do this cleanly we defer vCPU setup to async work which will run the vCPUs execution context as the thread is woken up. The scheduling of the work will kick the vCPU awake. This avoids potential races in MTTCG system emulation. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- v7 - add const to static mode_for_el[] array - fix checkpatch long lines --- target/arm/arm-powerctl.c | 146 ++++++++++++++++++++++++++++------------------ 1 file changed, 88 insertions(+), 58 deletions(-) -- 2.11.0 diff --git a/target/arm/arm-powerctl.c b/target/arm/arm-powerctl.c index fbb7a15daa..082788e3a4 100644 --- a/target/arm/arm-powerctl.c +++ b/target/arm/arm-powerctl.c @@ -48,11 +48,87 @@ CPUState *arm_get_cpu_by_id(uint64_t id) return NULL; } +struct cpu_on_info { + uint64_t entry; + uint64_t context_id; + uint32_t target_el; + bool target_aa64; +}; + + +static void arm_set_cpu_on_async_work(CPUState *target_cpu_state, + run_on_cpu_data data) +{ + ARMCPU *target_cpu = ARM_CPU(target_cpu_state); + struct cpu_on_info *info = (struct cpu_on_info *) data.host_ptr; + + /* Initialize the cpu we are turning on */ + cpu_reset(target_cpu_state); + target_cpu->powered_off = false; + target_cpu_state->halted = 0; + + if (info->target_aa64) { + if ((info->target_el < 3) && arm_feature(&target_cpu->env, + ARM_FEATURE_EL3)) { + /* + * As target mode is AArch64, we need to set lower + * exception level (the requested level 2) to AArch64 + */ + target_cpu->env.cp15.scr_el3 |= SCR_RW; + } + + if ((info->target_el < 2) && arm_feature(&target_cpu->env, + ARM_FEATURE_EL2)) { + /* + * As target mode is AArch64, we need to set lower + * exception level (the requested level 1) to AArch64 + */ + target_cpu->env.cp15.hcr_el2 |= HCR_RW; + } + + target_cpu->env.pstate = aarch64_pstate_mode(info->target_el, true); + } else { + /* We are requested to boot in AArch32 mode */ + static const uint32_t mode_for_el[] = { 0, + ARM_CPU_MODE_SVC, + ARM_CPU_MODE_HYP, + ARM_CPU_MODE_SVC }; + + cpsr_write(&target_cpu->env, mode_for_el[info->target_el], CPSR_M, + CPSRWriteRaw); + } + + if (info->target_el == 3) { + /* Processor is in secure mode */ + target_cpu->env.cp15.scr_el3 &= ~SCR_NS; + } else { + /* Processor is not in secure mode */ + target_cpu->env.cp15.scr_el3 |= SCR_NS; + } + + /* We check if the started CPU is now at the correct level */ + assert(info->target_el == arm_current_el(&target_cpu->env)); + + if (info->target_aa64) { + target_cpu->env.xregs[0] = info->context_id; + target_cpu->env.thumb = false; + } else { + target_cpu->env.regs[0] = info->context_id; + target_cpu->env.thumb = info->entry & 1; + info->entry &= 0xfffffffe; + } + + /* Start the new CPU at the requested address */ + cpu_set_pc(target_cpu_state, info->entry); + g_free(info); +} + int arm_set_cpu_on(uint64_t cpuid, uint64_t entry, uint64_t context_id, uint32_t target_el, bool target_aa64) { CPUState *target_cpu_state; ARMCPU *target_cpu; + struct cpu_on_info *info; DPRINTF("cpu %" PRId64 " (EL %d, %s) @ 0x%" PRIx64 " with R0 = 0x%" PRIx64 "\n", cpuid, target_el, target_aa64 ? "aarch64" : "aarch32", entry, @@ -109,64 +185,18 @@ int arm_set_cpu_on(uint64_t cpuid, uint64_t entry, uint64_t context_id, return QEMU_ARM_POWERCTL_INVALID_PARAM; } - /* Initialize the cpu we are turning on */ - cpu_reset(target_cpu_state); - target_cpu->powered_off = false; - target_cpu_state->halted = 0; - - if (target_aa64) { - if ((target_el < 3) && arm_feature(&target_cpu->env, ARM_FEATURE_EL3)) { - /* - * As target mode is AArch64, we need to set lower - * exception level (the requested level 2) to AArch64 - */ - target_cpu->env.cp15.scr_el3 |= SCR_RW; - } - - if ((target_el < 2) && arm_feature(&target_cpu->env, ARM_FEATURE_EL2)) { - /* - * As target mode is AArch64, we need to set lower - * exception level (the requested level 1) to AArch64 - */ - target_cpu->env.cp15.hcr_el2 |= HCR_RW; - } - - target_cpu->env.pstate = aarch64_pstate_mode(target_el, true); - } else { - /* We are requested to boot in AArch32 mode */ - static uint32_t mode_for_el[] = { 0, - ARM_CPU_MODE_SVC, - ARM_CPU_MODE_HYP, - ARM_CPU_MODE_SVC }; - - cpsr_write(&target_cpu->env, mode_for_el[target_el], CPSR_M, - CPSRWriteRaw); - } - - if (target_el == 3) { - /* Processor is in secure mode */ - target_cpu->env.cp15.scr_el3 &= ~SCR_NS; - } else { - /* Processor is not in secure mode */ - target_cpu->env.cp15.scr_el3 |= SCR_NS; - } - - /* We check if the started CPU is now at the correct level */ - assert(target_el == arm_current_el(&target_cpu->env)); - - if (target_aa64) { - target_cpu->env.xregs[0] = context_id; - target_cpu->env.thumb = false; - } else { - target_cpu->env.regs[0] = context_id; - target_cpu->env.thumb = entry & 1; - entry &= 0xfffffffe; - } - - /* Start the new CPU at the requested address */ - cpu_set_pc(target_cpu_state, entry); - - qemu_cpu_kick(target_cpu_state); + /* To avoid racing with a CPU we are just kicking off we do the + * final bit of preparation for the work in the target CPUs + * context. + */ + info = g_new(struct cpu_on_info, 1); + info->entry = entry; + info->context_id = context_id; + info->target_el = target_el; + info->target_aa64 = target_aa64; + + async_run_on_cpu(target_cpu_state, arm_set_cpu_on_async_work, + RUN_ON_CPU_HOST_PTR(info)); /* We are good to go */ return QEMU_ARM_POWERCTL_RET_SUCCESS; From patchwork Fri Jan 27 10:35:01 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92637 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp186770qgi; Fri, 27 Jan 2017 03:28:21 -0800 (PST) X-Received: by 10.55.25.19 with SMTP id k19mr7948198qkh.105.1485516501098; Fri, 27 Jan 2017 03:28:21 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 125si3329321qkk.68.2017.01.27.03.28.20 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:28:21 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44521 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4hW-0007cl-Hk for patch@linaro.org; Fri, 27 Jan 2017 06:28:18 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49524) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40J-0007bK-0T for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:40 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40I-0004eN-0S for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:39 -0500 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:35981) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40H-0004db-QU for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:37 -0500 Received: by mail-wm0-x22b.google.com with SMTP id c85so110758373wmi.1 for ; Fri, 27 Jan 2017 02:43:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/0U3sUBP4xqPfuNeLa547JecXLGhyPmtSpeOtWjU9bA=; b=N5cGg8WLze8OIomn8N8OS3cDJK77CoFc7ufgY90abA9uztToGukkZZXTW+reZAcd3u HWN3qOZwn1iHErbq01XdtHbVklyP0n08stkJ0i3sssVDnZ6ijD9i1go8axMsUYp90Lu3 /We02zRk5Wtb0jZ5VxUx8QKsmKs2WTLA/aGh0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/0U3sUBP4xqPfuNeLa547JecXLGhyPmtSpeOtWjU9bA=; b=WXL7uKAeaLsSXQFWeUgFuD7ztb4TKa+RSKAPiPnRwEcxZs1Eo3Gpz2S8aw0+uKvFvR nYA1jS+uxi5BtXaCxfeXndbrAxUY76/BYAA8OPNls6Vh8khCrZ3pqlbV6AOtAvRP8g0I b3LfpD3PdMjZzSdPRN9lZnf8O7FT6at87gtf5oh9huMHpsQBRa4Ei/X9YK0Q9/0epsVx LwP6uvHAJc2ITTPakHppPoBs7Tq49qhd4esdTqd+sCNz4mYcGKjbKcHhSk0ZEs6XI69n g80BGZvdqzxmmbUlum2FI9xJolJNhmMfM6rTUJd4Uz1mrZfo3mm/nZzQy7AmDf6NRuD2 A8pQ== X-Gm-Message-State: AIkVDXKWzT1v96f8+tmqBT91r6WxRSqjJRvyZtAf9fQOgHdCI+BfQdJCk8xhUqigsDXeMGgp X-Received: by 10.223.150.183 with SMTP id u52mr6681583wrb.180.1485513816682; Fri, 27 Jan 2017 02:43:36 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id p49sm7164475wrb.10.2017.01.27.02.43.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:33 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 27FCF3E376D; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:01 +0000 Message-Id: <20170127103505.18606-22-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::22b Subject: [Qemu-devel] [PATCH v8 21/25] target-arm: don't generate WFE/YIELD calls for MTTCG X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The WFE and YIELD instructions are really only hints and in TCG's case they were useful to move the scheduling on from one vCPU to the next. In the parallel context (MTTCG) this just causes an unnecessary cpu_exit and contention of the BQL. Signed-off-by: Alex Bennée Reviewed-by: Richard Henderson --- target/arm/op_helper.c | 7 +++++++ target/arm/translate-a64.c | 8 ++++++-- target/arm/translate.c | 20 ++++++++++++++++---- 3 files changed, 29 insertions(+), 6 deletions(-) -- 2.11.0 diff --git a/target/arm/op_helper.c b/target/arm/op_helper.c index e1a883c595..abfa7cdd39 100644 --- a/target/arm/op_helper.c +++ b/target/arm/op_helper.c @@ -436,6 +436,13 @@ void HELPER(yield)(CPUARMState *env) ARMCPU *cpu = arm_env_get_cpu(env); CPUState *cs = CPU(cpu); + /* When running in MTTCG we don't generate jumps to the yield and + * WFE helpers as it won't affect the scheduling of other vCPUs. + * If we wanted to more completely model WFE/SEV so we don't busy + * spin unnecessarily we would need to do something more involved. + */ + g_assert(!parallel_cpus); + /* This is a non-trappable hint instruction that generally indicates * that the guest is currently busy-looping. Yield control back to the * top level loop so that a more deserving VCPU has a chance to run. diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 88a4df6959..05162f335e 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1342,10 +1342,14 @@ static void handle_hint(DisasContext *s, uint32_t insn, s->is_jmp = DISAS_WFI; return; case 1: /* YIELD */ - s->is_jmp = DISAS_YIELD; + if (!parallel_cpus) { + s->is_jmp = DISAS_YIELD; + } return; case 2: /* WFE */ - s->is_jmp = DISAS_WFE; + if (!parallel_cpus) { + s->is_jmp = DISAS_WFE; + } return; case 4: /* SEV */ case 5: /* SEVL */ diff --git a/target/arm/translate.c b/target/arm/translate.c index dc67887918..444a24c2b6 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -4343,20 +4343,32 @@ static void gen_exception_return(DisasContext *s, TCGv_i32 pc) gen_rfe(s, pc, load_cpu_field(spsr)); } +/* + * For WFI we will halt the vCPU until an IRQ. For WFE and YIELD we + * only call the helper when running single threaded TCG code to ensure + * the next round-robin scheduled vCPU gets a crack. In MTTCG mode we + * just skip this instruction. Currently the SEV/SEVL instructions + * which are *one* of many ways to wake the CPU from WFE are not + * implemented so we can't sleep like WFI does. + */ static void gen_nop_hint(DisasContext *s, int val) { switch (val) { case 1: /* yield */ - gen_set_pc_im(s, s->pc); - s->is_jmp = DISAS_YIELD; + if (!parallel_cpus) { + gen_set_pc_im(s, s->pc); + s->is_jmp = DISAS_YIELD; + } break; case 3: /* wfi */ gen_set_pc_im(s, s->pc); s->is_jmp = DISAS_WFI; break; case 2: /* wfe */ - gen_set_pc_im(s, s->pc); - s->is_jmp = DISAS_WFE; + if (!parallel_cpus) { + gen_set_pc_im(s, s->pc); + s->is_jmp = DISAS_WFE; + } break; case 4: /* sev */ case 5: /* sevl */ From patchwork Fri Jan 27 10:35:02 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92621 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp149960obz; Fri, 27 Jan 2017 03:09:51 -0800 (PST) X-Received: by 10.55.141.194 with SMTP id p185mr7128665qkd.209.1485515390977; Fri, 27 Jan 2017 03:09:50 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u45si3308539qtc.5.2017.01.27.03.09.50 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:09:50 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44399 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4Pc-00075A-96 for patch@linaro.org; Fri, 27 Jan 2017 06:09:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49519) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40I-0007bG-VX for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:39 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40F-0004cb-TK for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:39 -0500 Received: from mail-wm0-x234.google.com ([2a00:1450:400c:c09::234]:35291) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40F-0004bO-Mr for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:35 -0500 Received: by mail-wm0-x234.google.com with SMTP id r126so111089993wmr.0 for ; Fri, 27 Jan 2017 02:43:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y7zYZvE7n9D0dBB1IqY7SsdefFQddJx3kVYUDcVdxF0=; b=S6N03AI+JTd1DENHGQ2QR6BJsOF4Qty26Tyi1wzj2oEret54ObOUEvKYmOvKeg+F0Z CvYKVnOw7kqLcFyjRxC9vzbrXVFruSpYO0ax9DX7rMPFFaygrq80KD6nYJ+oPn+vot5l xTnD944AWpHoMFhBcCWNXZy8uFnL5proRaWfU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y7zYZvE7n9D0dBB1IqY7SsdefFQddJx3kVYUDcVdxF0=; b=rETDEOP1bXVIg6kf2bR+GGDMOSAtb9+e2c4wh9oh6xdluNM4z9UuuiiEjO133ZuWL3 x0bpWDoEMKvMDCdacVEp0Djk/OAdD9OfzNUjXr3yiLQxImelc9axRsSPCJX/iIBp8J3s SRybbrh/9gFBcrIoR96pI2NLHL3O43C5BCuBW5R4pURucoQwiOYVkoHmqQgf8C8dwO2r Gpj6NtmY4S5MYt843WaNA100o6SM3SX3AWFdAqeRuD+MCaO16ugKImL51jhysM051XsI Hq7ik+5BQnt5kfLtpbJpo+ByH1ENR5KuBg2OeAfQWjMub3+g9BNSt/F4lYrzbEIw4NTA oEOA== X-Gm-Message-State: AIkVDXKvvQ7EC5AhrGe5xAAJ8nCVXWnFQluxNlCgFeisfGZXftmf2/s23mZQRfEoPfmgQhuH X-Received: by 10.28.169.135 with SMTP id s129mr2415433wme.24.1485513814607; Fri, 27 Jan 2017 02:43:34 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id e6sm7183648wrc.30.2017.01.27.02.43.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:33 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 38BDC3E376F; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:02 +0000 Message-Id: <20170127103505.18606-23-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::234 Subject: [Qemu-devel] [PATCH v8 22/25] target-arm/cpu.h: make ARM_CP defined consistent X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is a purely mechanical change to make the ARM_CP flags neatly align and use a consistent format so it is easier to see which bit each flag is. Signed-off-by: Alex Bennée --- target/arm/cpu.h | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) -- 2.11.0 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 274ef17562..f56a96c675 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -1398,20 +1398,20 @@ static inline uint64_t cpreg_to_kvm_id(uint32_t cpregid) * need to be surrounded by gen_io_start()/gen_io_end(). In particular, * registers which implement clocks or timers require this. */ -#define ARM_CP_SPECIAL 1 -#define ARM_CP_CONST 2 -#define ARM_CP_64BIT 4 -#define ARM_CP_SUPPRESS_TB_END 8 -#define ARM_CP_OVERRIDE 16 -#define ARM_CP_ALIAS 32 -#define ARM_CP_IO 64 -#define ARM_CP_NO_RAW 128 -#define ARM_CP_NOP (ARM_CP_SPECIAL | (1 << 8)) -#define ARM_CP_WFI (ARM_CP_SPECIAL | (2 << 8)) -#define ARM_CP_NZCV (ARM_CP_SPECIAL | (3 << 8)) -#define ARM_CP_CURRENTEL (ARM_CP_SPECIAL | (4 << 8)) -#define ARM_CP_DC_ZVA (ARM_CP_SPECIAL | (5 << 8)) -#define ARM_LAST_SPECIAL ARM_CP_DC_ZVA +#define ARM_CP_SPECIAL (1 << 0) +#define ARM_CP_CONST (1 << 1) +#define ARM_CP_64BIT (1 << 2) +#define ARM_CP_SUPPRESS_TB_END (1 << 3) +#define ARM_CP_OVERRIDE (1 << 4) +#define ARM_CP_ALIAS (1 << 5) +#define ARM_CP_IO (1 << 6) +#define ARM_CP_NO_RAW (1 << 7) +#define ARM_CP_NOP (ARM_CP_SPECIAL | (1 << 8)) +#define ARM_CP_WFI (ARM_CP_SPECIAL | (2 << 8)) +#define ARM_CP_NZCV (ARM_CP_SPECIAL | (3 << 8)) +#define ARM_CP_CURRENTEL (ARM_CP_SPECIAL | (4 << 8)) +#define ARM_CP_DC_ZVA (ARM_CP_SPECIAL | (5 << 8)) +#define ARM_LAST_SPECIAL ARM_CP_DC_ZVA /* Used only as a terminator for ARMCPRegInfo lists */ #define ARM_CP_SENTINEL 0xffff /* Mask of only the flag bits in a type field */ From patchwork Fri Jan 27 10:35:03 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92636 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp155792obz; Fri, 27 Jan 2017 03:26:23 -0800 (PST) X-Received: by 10.55.163.80 with SMTP id m77mr2263359qke.157.1485516383496; Fri, 27 Jan 2017 03:26:23 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id f37si3301555qte.335.2017.01.27.03.26.22 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:26:23 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44497 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4fc-0005f7-Ns for patch@linaro.org; Fri, 27 Jan 2017 06:26:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49575) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40L-0007e1-Nf for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:42 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40L-0004hT-22 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:41 -0500 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:38210) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40K-0004gi-SD for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:40 -0500 Received: by mail-wm0-x231.google.com with SMTP id r144so130192458wme.1 for ; Fri, 27 Jan 2017 02:43:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jTdkzqovOaa/vHi9qgvcJFTj3jqkbsRN9xvWUg//VFM=; b=DT9DKIu5/HSoPhLJDzqo+ihoEkx/KZ0lFmNZhRCg/FvOA1kV1xJT1UNggY3zTaxHKh WpMvrzQQzJEekkp+cDc5TajsK/ftF/+OVWu8WJ6RRacCMJ8Y5Jh1AxNtkVgo8QfNuzR/ R6BP5Pz6OhsqxJfvkPBp51W5GuhtnngvfFtWk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jTdkzqovOaa/vHi9qgvcJFTj3jqkbsRN9xvWUg//VFM=; b=oYJsV8HvU7saKSFJuHsYT9TNGk8+CmBVzI2nE9K/3bGXLnlU34pWAKflE0mFLGKQhT iIAQP4dTSGVI1VSlg2kCRqKoTZJ5FdeeRa/I8ZHCQlMprgBTYxoOVhy1dfnODgk9pRz2 rGVDOo69eWd+1E0VKplPcvCC3BXpprNT/jhTnqAEBg9anMpHTqJJz2qq5G1dCsQDCGvi 5nvlCcrixqIo6oOG81rFAFebwWRGf++WusmOWioKcENqNLnwlQiJLZ4n+7GLKaziEiTq JIgrWvTP0lNy5uyQrSF/MGPJIYo3/wbOALt7nfiZO0MHeonVX9WzyNVJPm65Ci6Wb0B2 xOTA== X-Gm-Message-State: AIkVDXJUuSq3p9qo1UCTcQ+2UpQEl8ESksdq4MnlDdBVb2MsNuQw73w/YW7fBTQXH/IWVlqJ X-Received: by 10.223.172.17 with SMTP id v17mr6811481wrc.115.1485513819773; Fri, 27 Jan 2017 02:43:39 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id q5sm7241288wrd.32.2017.01.27.02.43.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:38 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 4B4413E3770; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:03 +0000 Message-Id: <20170127103505.18606-24-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [PATCH v8 23/25] target-arm: introduce ARM_CP_EXIT_PC X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Some helpers may trigger an immediate exit of the cpu_loop. If this happens the PC need to be rectified to ensure the restart will begin on the next instruction. Signed-off-by: Alex Bennée --- target/arm/cpu.h | 3 ++- target/arm/translate-a64.c | 4 ++++ target/arm/translate.c | 4 ++++ 3 files changed, 10 insertions(+), 1 deletion(-) -- 2.11.0 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index f56a96c675..1b0670ae11 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -1411,7 +1411,8 @@ static inline uint64_t cpreg_to_kvm_id(uint32_t cpregid) #define ARM_CP_NZCV (ARM_CP_SPECIAL | (3 << 8)) #define ARM_CP_CURRENTEL (ARM_CP_SPECIAL | (4 << 8)) #define ARM_CP_DC_ZVA (ARM_CP_SPECIAL | (5 << 8)) -#define ARM_LAST_SPECIAL ARM_CP_DC_ZVA +#define ARM_CP_EXIT_PC (ARM_CP_SPECIAL | (6 << 8)) +#define ARM_LAST_SPECIAL ARM_CP_EXIT_PC /* Used only as a terminator for ARMCPRegInfo lists */ #define ARM_CP_SENTINEL 0xffff /* Mask of only the flag bits in a type field */ diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 05162f335e..a3f37d8bec 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -1561,6 +1561,10 @@ static void handle_sys(DisasContext *s, uint32_t insn, bool isread, tcg_rt = cpu_reg(s, rt); gen_helper_dc_zva(cpu_env, tcg_rt); return; + case ARM_CP_EXIT_PC: + /* The helper may exit the cpu_loop so ensure PC is correct */ + gen_a64_set_pc_im(s->pc); + break; default: break; } diff --git a/target/arm/translate.c b/target/arm/translate.c index 444a24c2b6..7bd18cd25d 100644 --- a/target/arm/translate.c +++ b/target/arm/translate.c @@ -7508,6 +7508,10 @@ static int disas_coproc_insn(DisasContext *s, uint32_t insn) gen_set_pc_im(s, s->pc); s->is_jmp = DISAS_WFI; return 0; + case ARM_CP_EXIT_PC: + /* The helper may exit the cpu_loop so ensure PC is correct */ + gen_set_pc_im(s, s->pc); + break; default: break; } From patchwork Fri Jan 27 10:35:04 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92639 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp157848obz; Fri, 27 Jan 2017 03:32:10 -0800 (PST) X-Received: by 10.55.27.65 with SMTP id b62mr7978956qkb.202.1485516623279; Fri, 27 Jan 2017 03:30:23 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id c41si3312614qta.309.2017.01.27.03.30.22 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:30:23 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44533 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4jU-0000yF-EB for patch@linaro.org; Fri, 27 Jan 2017 06:30:20 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49604) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40M-0007f3-IL for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40K-0004gR-7r for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:42 -0500 Received: from mail-wm0-x236.google.com ([2a00:1450:400c:c09::236]:35326) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40J-0004fG-V5 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:40 -0500 Received: by mail-wm0-x236.google.com with SMTP id r126so111092589wmr.0 for ; Fri, 27 Jan 2017 02:43:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vJ8bA4QnVdYGx0XF2sQEVnOEiIGJknl4LKjMC1hSUTs=; b=KDiOOf6Eqlw+8BZ57Nrs6ryUMeukr5XpukNzt0vhg11xNynZ3PQfr1wylJNC9ZQ/jb scJyGgVa6PPQVyKAnYzLH0sjgM0Cj3VPVIVaZH8wP6V8PmqmSF6BOVd30kky1KSjYfOv lQf9xwHG5jWgy1ZRDMbnMzSpVcDz38CUjcuP8= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vJ8bA4QnVdYGx0XF2sQEVnOEiIGJknl4LKjMC1hSUTs=; b=UGL09gzb41FQeJ/FX849AzL+1cDgZuim19IdQvCiVlTcqreWdNYYDCTcPY+1TRMFzJ Uuixd7sWZOCMuUYsnNPrF3uEgsFCG6CYZUX4R476d9boDfUqI9yHSgbgXdJTyokUfdzj xUWsTetwQUS87xz8OcsWN8bGpafPvAlFsBzN8OmoCC1mT8M7RR3anp29oqufr4FWFVqa Il7tJaHv7Q/F1UXYAp8NDNFEYyVcCPf+4JZ0Rrh+NDVe0CaurMhiG5DsIN4uxVKB8mfY AX9g1MSph8Sr45ZMB5zVXTa2/tJ8bjD/iP8qCSjXZDi1thgv1eNaPaEW4wb3wdLfnNgf RPmA== X-Gm-Message-State: AIkVDXJU8pcVKb1+7+MpTSvl+QswRC+IEFEIyCHA9ngakTRDpdNckprvvLgFL6xEA9YKMHfG X-Received: by 10.223.155.157 with SMTP id d29mr6690373wrc.74.1485513818602; Fri, 27 Jan 2017 02:43:38 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id w70sm7148760wrc.47.2017.01.27.02.43.33 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:33 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 5BEE73E008E; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:04 +0000 Message-Id: <20170127103505.18606-25-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::236 Subject: [Qemu-devel] [PATCH v8 24/25] target-arm: ensure all cross vCPUs TLB flushes complete X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Previously flushes on other vCPUs would only get serviced when they exited their TranslationBlocks. While this isn't overly problematic it violates the semantics of TLB flush from the point of view of source vCPU. To solve this we call the cputlb *_all_cpus() functions to do the flushes and ask it to ensure all flushes are completed before we start the next instruction. As this involves exiting the cpu_loop we need to ensure the PC is saved before the tlb helper functions are called. Signed-off-by: Alex Bennée --- v8 - fixup merge for bitmap based API - fixup new _synced API --- target/arm/helper.c | 194 +++++++++++++++++++++++----------------------------- 1 file changed, 84 insertions(+), 110 deletions(-) -- 2.11.0 diff --git a/target/arm/helper.c b/target/arm/helper.c index 0c6fb4add7..b3eb4d7430 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -536,41 +536,33 @@ static void tlbimvaa_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbiall_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush(other_cs); - } + tlb_flush_all_cpus_synced(cs); } static void tlbiasid_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush(other_cs); - } + tlb_flush_all_cpus_synced(cs); } static void tlbimva_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_page(other_cs, value & TARGET_PAGE_MASK); - } + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); } static void tlbimvaa_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_page(other_cs, value & TARGET_PAGE_MASK); - } + tlb_flush_page_all_cpus_synced(cs, value & TARGET_PAGE_MASK); } static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -585,12 +577,10 @@ static void tlbiall_nsnh_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbiall_nsnh_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S12NSE1 | ARMMMUBit_S12NSE0 | ARMMMUBit_S2NS); - } } static void tlbiipas2_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -617,7 +607,7 @@ static void tlbiipas2_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbiipas2_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr; if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { @@ -626,9 +616,7 @@ static void tlbiipas2_is_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 40); - CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S2NS); - } + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUBit_S2NS); } static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -642,11 +630,9 @@ static void tlbiall_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbiall_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E2); - } + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S1E2); } static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -661,12 +647,10 @@ static void tlbimva_hyp_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbimva_hyp_is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr = value & ~MAKE_64BIT_MASK(0, 12); - CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E2); - } + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUBit_S1E2); } static const ARMCPRegInfo cp_reginfo[] = { @@ -1335,14 +1319,16 @@ static const ARMCPRegInfo v7_cp_reginfo[] = { static const ARMCPRegInfo v7mp_cp_reginfo[] = { /* 32 bit TLB invalidates, Inner Shareable */ { .name = "TLBIALLIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .type = ARM_CP_NO_RAW, .access = PL1_W, .writefn = tlbiall_is_write }, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL1_W, + .writefn = tlbiall_is_write }, { .name = "TLBIMVAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL1_W, .writefn = tlbimva_is_write }, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL1_W, + .writefn = tlbimva_is_write }, { .name = "TLBIASIDIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .type = ARM_CP_NO_RAW, .access = PL1_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL1_W, .writefn = tlbiasid_is_write }, { .name = "TLBIMVAAIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .type = ARM_CP_NO_RAW, .access = PL1_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL1_W, .writefn = tlbimvaa_is_write }, REGINFO_SENTINEL }; @@ -2856,8 +2842,7 @@ static CPAccessResult aa64_cacheop_access(CPUARMState *env, static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - ARMCPU *cpu = arm_env_get_cpu(env); - CPUState *cs = CPU(cpu); + CPUState *cs = ENV_GET_CPU(env); if (arm_is_secure_below_el3(env)) { tlb_flush_by_mmuidx(cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); @@ -2869,16 +2854,15 @@ static void tlbi_aa64_vmalle1_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbi_aa64_vmalle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { + CPUState *cs = ENV_GET_CPU(env); bool sec = arm_is_secure_below_el3(env); - CPUState *other_cs; - CPU_FOREACH(other_cs) { - if (sec) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); - } else { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | - ARMMMUBit_S12NSE0); - } + if (sec) { + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S1SE1 + | ARMMMUBit_S1SE0); + } else { + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S12NSE1 + | ARMMMUBit_S12NSE0); } } @@ -2931,39 +2915,34 @@ static void tlbi_aa64_alle1is_write(CPUARMState *env, const ARMCPRegInfo *ri, */ bool sec = arm_is_secure_below_el3(env); bool has_el2 = arm_feature(env, ARM_FEATURE_EL2); - CPUState *other_cs; - - CPU_FOREACH(other_cs) { - if (sec) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1SE1 | ARMMMUBit_S1SE0); - } else if (has_el2) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | - ARMMMUBit_S12NSE0 | ARMMMUBit_S2NS); - } else { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S12NSE1 | - ARMMMUBit_S12NSE0); - } + CPUState *cs = ENV_GET_CPU(env); + + if (sec) { + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S1SE1 + | ARMMMUBit_S1SE0); + } else if (has_el2) { + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S12NSE1 + | ARMMMUBit_S12NSE0 | ARMMMUBit_S2NS); + } else { + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S12NSE1 + | ARMMMUBit_S12NSE0); } } static void tlbi_aa64_alle2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E2); - } + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S1E2); } static void tlbi_aa64_alle3is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); - CPU_FOREACH(other_cs) { - tlb_flush_by_mmuidx(other_cs, ARMMMUBit_S1E3); - } + tlb_flush_by_mmuidx_all_cpus_synced(cs, ARMMMUBit_S1E3); } static void tlbi_aa64_vae1_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3018,41 +2997,38 @@ static void tlbi_aa64_vae3_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbi_aa64_vae1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { + ARMCPU *cpu = arm_env_get_cpu(env); + CPUState *cs = CPU(cpu); bool sec = arm_is_secure_below_el3(env); - CPUState *other_cs; uint64_t pageaddr = sextract64(value << 12, 0, 56); - CPU_FOREACH(other_cs) { - if (sec) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1SE1 | - ARMMMUBit_S1SE0); - } else { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S12NSE1 | - ARMMMUBit_S12NSE0); - } + if (sec) { + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUBit_S1SE1 | + ARMMMUBit_S1SE0); + } else { + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, + ARMMMUBit_S12NSE1 | + ARMMMUBit_S12NSE0); } } static void tlbi_aa64_vae2is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr = sextract64(value << 12, 0, 56); - CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E2); - } + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUBit_S1E2); } static void tlbi_aa64_vae3is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr = sextract64(value << 12, 0, 56); - CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S1E3); - } + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUBit_S1E3); } static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3080,7 +3056,7 @@ static void tlbi_aa64_ipas2e1_write(CPUARMState *env, const ARMCPRegInfo *ri, static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, uint64_t value) { - CPUState *other_cs; + CPUState *cs = ENV_GET_CPU(env); uint64_t pageaddr; if (!arm_feature(env, ARM_FEATURE_EL2) || !(env->cp15.scr_el3 & SCR_NS)) { @@ -3089,9 +3065,7 @@ static void tlbi_aa64_ipas2e1is_write(CPUARMState *env, const ARMCPRegInfo *ri, pageaddr = sextract64(value << 12, 0, 48); - CPU_FOREACH(other_cs) { - tlb_flush_page_by_mmuidx(other_cs, pageaddr, ARMMMUBit_S2NS); - } + tlb_flush_page_by_mmuidx_all_cpus_synced(cs, pageaddr, ARMMMUBit_S2NS); } static CPAccessResult aa64_zva_access(CPUARMState *env, const ARMCPRegInfo *ri, @@ -3248,27 +3222,27 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { /* TLBI operations */ { .name = "TLBI_VMALLE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_ASIDE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 2, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vmalle1is_write }, { .name = "TLBI_VAAE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 3, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VAALE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .access = PL1_W, .type = ARM_CP_NO_RAW, + .access = PL1_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae1is_write }, { .name = "TLBI_VMALLE1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 0, @@ -3296,19 +3270,19 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbi_aa64_vae1_write }, { .name = "TLBI_IPAS2E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_ipas2e1is_write }, { .name = "TLBI_IPAS2LE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_ipas2e1is_write }, { .name = "TLBI_ALLE1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_VMALLS12E1IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_alle1is_write }, { .name = "TLBI_IPAS2E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, @@ -3324,7 +3298,7 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbi_aa64_alle1_write }, { .name = "TLBI_VMALLS12E1", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 6, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_alle1is_write }, #ifndef CONFIG_USER_ONLY /* 64 bit address translation operations */ @@ -3370,7 +3344,7 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { { .name = "TLBIMVALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 5, .type = ARM_CP_NO_RAW, .access = PL1_W, .writefn = tlbimva_is_write }, { .name = "TLBIMVAALIS", .cp = 15, .opc1 = 0, .crn = 8, .crm = 3, .opc2 = 7, - .type = ARM_CP_NO_RAW, .access = PL1_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL1_W, .writefn = tlbimvaa_is_write }, { .name = "TLBIMVAL", .cp = 15, .opc1 = 0, .crn = 8, .crm = 7, .opc2 = 5, .type = ARM_CP_NO_RAW, .access = PL1_W, .writefn = tlbimva_write }, @@ -3381,7 +3355,7 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbimva_hyp_write }, { .name = "TLBIMVALHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbimva_hyp_is_write }, { .name = "TLBIIPAS2", .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 1, @@ -3389,7 +3363,7 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbiipas2_write }, { .name = "TLBIIPAS2IS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbiipas2_is_write }, { .name = "TLBIIPAS2L", .cp = 15, .opc1 = 4, .crn = 8, .crm = 4, .opc2 = 5, @@ -3397,7 +3371,7 @@ static const ARMCPRegInfo v8_cp_reginfo[] = { .writefn = tlbiipas2_write }, { .name = "TLBIIPAS2LIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 0, .opc2 = 5, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbiipas2_is_write }, /* 32 bit cache operations */ { .name = "ICIALLUIS", .cp = 15, .opc1 = 0, .crn = 7, .crm = 1, .opc2 = 0, @@ -3737,7 +3711,7 @@ static const ARMCPRegInfo el2_cp_reginfo[] = { .writefn = tlbiall_nsnh_write }, { .name = "TLBIALLNSNHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 4, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbiall_nsnh_is_write }, { .name = "TLBIALLH", .cp = 15, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, .type = ARM_CP_NO_RAW, .access = PL2_W, @@ -3749,7 +3723,7 @@ static const ARMCPRegInfo el2_cp_reginfo[] = { .type = ARM_CP_NO_RAW, .access = PL2_W, .writefn = tlbimva_hyp_write }, { .name = "TLBIMVAHIS", .cp = 15, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbimva_hyp_is_write }, { .name = "TLBI_ALLE2", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 7, .opc2 = 0, @@ -3765,15 +3739,15 @@ static const ARMCPRegInfo el2_cp_reginfo[] = { .writefn = tlbi_aa64_vae2_write }, { .name = "TLBI_ALLE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_alle2is_write }, { .name = "TLBI_VAE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 1, - .type = ARM_CP_NO_RAW, .access = PL2_W, + .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .access = PL2_W, .writefn = tlbi_aa64_vae2is_write }, { .name = "TLBI_VALE2IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 4, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL2_W, .type = ARM_CP_NO_RAW, + .access = PL2_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae2is_write }, #ifndef CONFIG_USER_ONLY /* Unlike the other EL2-related AT operations, these must @@ -3960,15 +3934,15 @@ static const ARMCPRegInfo el3_cp_reginfo[] = { .resetvalue = 0 }, { .name = "TLBI_ALLE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 0, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_alle3is_write }, { .name = "TLBI_VAE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 1, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae3is_write }, { .name = "TLBI_VALE3IS", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 3, .opc2 = 5, - .access = PL3_W, .type = ARM_CP_NO_RAW, + .access = PL3_W, .type = ARM_CP_NO_RAW | ARM_CP_EXIT_PC, .writefn = tlbi_aa64_vae3is_write }, { .name = "TLBI_ALLE3", .state = ARM_CP_STATE_AA64, .opc0 = 1, .opc1 = 6, .crn = 8, .crm = 7, .opc2 = 0, From patchwork Fri Jan 27 10:35:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 92640 Delivered-To: patch@linaro.org Received: by 10.182.3.34 with SMTP id 2csp158048obz; Fri, 27 Jan 2017 03:32:44 -0800 (PST) X-Received: by 10.55.162.138 with SMTP id l132mr7392369qke.204.1485516764932; Fri, 27 Jan 2017 03:32:44 -0800 (PST) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id q58si3333538qta.119.2017.01.27.03.32.44 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 27 Jan 2017 03:32:44 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44552 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX4lm-0003WU-8f for patch@linaro.org; Fri, 27 Jan 2017 06:32:42 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49609) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1cX40M-0007fV-VB for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:44 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1cX40L-0004hn-Q5 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:43 -0500 Received: from mail-wm0-x230.google.com ([2a00:1450:400c:c09::230]:38218) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1cX40L-0004hR-K6 for qemu-devel@nongnu.org; Fri, 27 Jan 2017 05:43:41 -0500 Received: by mail-wm0-x230.google.com with SMTP id r144so130192985wme.1 for ; Fri, 27 Jan 2017 02:43:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=MMvQVwwMrO8E2KpPpfeKpPjfgHAC1cH1xyqcJDBpDi0=; b=eQyhj7nyi0uIB3FJLIdhuC6YLbWjFZTdM+OWGFUcJYkI1mXmD7RD+nBWkYDI4GA8G0 mMjFuhRmRnbdH3E6s0ykfsK/HHu+a10PxuPl/nn6QYQhCtWtjlP1oZKAzEqFpeCyKafn 2ScBtNwrwrfENjWJ068zbnvmfhSFBW5XDnFG4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=MMvQVwwMrO8E2KpPpfeKpPjfgHAC1cH1xyqcJDBpDi0=; b=UB88BRtB3BjTtRWAdJ0P0M6c4tJbyWCfZAqVgGwxdDJHcXVvf9vKk6UtG/z5XP7Xz9 Zm2+LbtvrTYkTer0m3UBGp+9yyVr+0ZkPQQpv0Wr5PK+3MtzdIHpkHZyiYEwspAZJG5j WCbC586OnrMSqbCf/F2DPjmFBcTVAn1YUERc5Q//nofB/Pgsd1nVb+j3UVDpm1wwBbec 6CeAj8s2KYYLP1WBT5WaD9qBLgfJZ50jVNwxUY25qjjmcGYfWvks85rxLraP51fofuyS EZu8ckjOSbxIqsRus2CU2pKt9Y5gd252sh8R5N89mpQf9cIT2E4RaAwlUmec/vnf6Zt2 wsXQ== X-Gm-Message-State: AIkVDXLTYXnq6erxgKW+LW1Q+lG3XgBrW+yLoZSKbZWM6bxAWuH0z46EhPJ2YwtSjtCPd/Lb X-Received: by 10.28.19.78 with SMTP id 75mr2763691wmt.108.1485513820533; Fri, 27 Jan 2017 02:43:40 -0800 (PST) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id y80sm7257167wrb.12.2017.01.27.02.43.34 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 27 Jan 2017 02:43:38 -0800 (PST) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 6E7BF3E3771; Fri, 27 Jan 2017 10:35:07 +0000 (GMT) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: Date: Fri, 27 Jan 2017 10:35:05 +0000 Message-Id: <20170127103505.18606-26-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170127103505.18606-1-alex.bennee@linaro.org> References: <20170127103505.18606-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2a00:1450:400c:c09::230 Subject: [Qemu-devel] [PATCH v8 25/25] tcg: enable MTTCG by default for ARM on x86 hosts X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , "open list:ARM" , =?utf-8?q?Alex_Benn=C3=A9e?= , "open list:All patches CC here" , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This enables the multi-threaded system emulation by default for ARMv7 and ARMv8 guests using the x86_64 TCG backend. This is because on the guest side: - The ARM translate.c/translate-64.c have been converted to - use MTTCG safe atomic primitives - emit the appropriate barrier ops - The ARM machine has been updated to - hold the BQL when modifying shared cross-vCPU state - defer cpu_reset to async safe work All the host backends support the barrier and atomic primitives but need to provide same-or-better support for normal load/store operations. Signed-off-by: Alex Bennée --- v7 - drop configure check for backend - declare backend memory order for x86 - declare guest memory order for ARM - add configure snippet to set TARGET_SUPPORTS_MTTCG --- configure | 6 ++++++ target/arm/cpu.h | 3 +++ tcg/i386/tcg-target.h | 16 ++++++++++++++++ 3 files changed, 25 insertions(+) -- 2.11.0 diff --git a/configure b/configure index 86fd833feb..9f2a665f5b 100755 --- a/configure +++ b/configure @@ -5879,6 +5879,7 @@ mkdir -p $target_dir echo "# Automatically generated by configure - do not modify" > $config_target_mak bflt="no" +mttcg="no" interp_prefix1=$(echo "$interp_prefix" | sed "s/%M/$target_name/g") gdb_xml_files="" @@ -5897,11 +5898,13 @@ case "$target_name" in arm|armeb) TARGET_ARCH=arm bflt="yes" + mttcg="yes" gdb_xml_files="arm-core.xml arm-vfp.xml arm-vfp3.xml arm-neon.xml" ;; aarch64) TARGET_BASE_ARCH=arm bflt="yes" + mttcg="yes" gdb_xml_files="aarch64-core.xml aarch64-fpu.xml arm-core.xml arm-vfp.xml arm-vfp3.xml arm-neon.xml" ;; cris) @@ -6066,6 +6069,9 @@ if test "$target_bigendian" = "yes" ; then fi if test "$target_softmmu" = "yes" ; then echo "CONFIG_SOFTMMU=y" >> $config_target_mak + if test "$mttcg" = "yes" ; then + echo "TARGET_SUPPORTS_MTTCG=y" >> $config_target_mak + fi fi if test "$target_user_only" = "yes" ; then echo "CONFIG_USER_ONLY=y" >> $config_target_mak diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 1b0670ae11..47a42ec6d6 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -29,6 +29,9 @@ # define TARGET_LONG_BITS 32 #endif +/* ARM processors have a weak memory model */ +#define TCG_DEFAULT_MO (0) + #define CPUArchState struct CPUARMState #include "qemu-common.h" diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 21d96ec35c..536190f647 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -165,4 +165,20 @@ static inline void flush_icache_range(uintptr_t start, uintptr_t stop) { } +/* This defines the natural memory order supported by this + * architecture before guarantees made by various barrier + * instructions. + * + * The x86 has a pretty strong memory ordering which only really + * allows for some stores to be re-ordered after loads. + */ +#include "tcg-mo.h" + +static inline int get_tcg_target_mo(void) +{ + return TCG_MO_ALL & ~TCG_MO_LD_ST; +} + +#define TCG_TARGET_DEFAULT_MO get_tcg_target_mo() + #endif