From patchwork Mon Aug 28 18:55:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717740 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347353wru; Mon, 28 Aug 2023 11:57:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFFpRNbYl17axPBtndB+ZsA+Lu9+j7MVgB2C78kNg5cCVCh38aNxeW5J+ILesSEBF+PreO0 X-Received: by 2002:a05:622a:1049:b0:400:9390:d129 with SMTP id f9-20020a05622a104900b004009390d129mr37295802qte.36.1693249021567; Mon, 28 Aug 2023 11:57:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249021; cv=none; d=google.com; s=arc-20160816; b=VAKprhwp1UdmwD1/F/nW/m4K17AVk/6NCByK5xIvdwx4Q7tIC+j8NZq0nlOzMP9oQz SfVJeKLRClSj9ZxAYGmryqreOXElX0c3r8czzd4tyBohvZzhwocpWeKDlPA2tMofyniT MRB/evBIe/e9fSK605DNK1EwhXjDyvdDPQGFXz6A1IXoeu7kDGFtgk9DCnpjhVFDcaid Re4eDcdYY6dqxgJ5ayS+rwLPPDLpb8MXI8oarnEs9Hag/YR4asJUKl9ivkFtYoAu14mu +Ap1OmhgjOLIJKO/T0yv2I12Bm2whOt0h9lH6LkVb2+tV53Y5Me9C08hqEWfpe2DZJwO dwnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=03eKjHBSqhHdZ2C0Bhm1xZE8o9z6xsoYmMLEvVPH1DU=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=MVFhvKUiyYUnTS/fagtWUcYeJ/eTFMXKB9rfV4pYQSvOnp1QLbXQwiD8gC7Z7I4JzA UoXe/qVB8txJhJIA7shcvnQ1a5l9fn3x+PZ4dbxdtOq/cY8yYIOXHDQfE1fy/YyrwQfb wz1dS/1Ow0ei1GpjEm62TnSud5qO9D7h+p2rBpKhTuboja/IZbJIHHlB/u5ck27mQrkS IPFNzTPRRNHtiEaFJkPzmwNP9g0T6wykaTKEVg6bO22f726fxUUKwJjJ/RsHIZyIUiov fwi2f1pZbBKzMu16LrCCnNZ5r5PGnkEWiL1NX6pp0p3SnYH7ivRMFW3eBfhC/CDI9Arg seAQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V+4+WRs+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f3-20020a05622a104300b00403c9478d96si5337273qte.344.2023.08.28.11.57.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=V+4+WRs+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP8-0000ql-If; Mon, 28 Aug 2023 14:56:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP3-0000oV-BB for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP0-0002QW-R8 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-26d1a17ce06so1725245a91.0 for ; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248953; x=1693853753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=03eKjHBSqhHdZ2C0Bhm1xZE8o9z6xsoYmMLEvVPH1DU=; b=V+4+WRs+wA91WtdmjuCmZv3m/bypY/4YkVMNzRqFJ0UWrTlNPYFZDRyvJUfuvpKsBT OgYD76e5SRD9GHYXTBB7vvGE4ILru+OFnoBy+OW/7iEvOVRx0Z1AJJGMJ+lhFWgv4Ndk 7qhEpVoNb4R9eh7NIZrnqKJblISeg/hsjdgSFmBAfrRbmsRMwRaAfVMkkLZ2v4AITgW4 5HnK2DsdE2psAAPBqRuwqKGRwUH0HEHmoOkSBrDweiqB9XaPTgC2yPhF+5AgoEq0Tun8 83DzEgyFD5TOSVGxcDRG6UNRN5UQs0qA+gsnH1eeqcstD9xltohyhq7NaGUmdhieA5mT dDnQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248953; x=1693853753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=03eKjHBSqhHdZ2C0Bhm1xZE8o9z6xsoYmMLEvVPH1DU=; b=P3nMveWNkQGdXqzmVSw7SHQstd5pA/wNi1hKh7A+eI/FqTgBh33S9VWeYuU7UaGoCB qnQfEnWwoR2G3ucmirFDRzO/bdiK+gmrrwojBqLdzfz+EpFCuxpMj8ZDuvubiCdsThke 1ne1Sv8Bs9GiqnQm7QSWy8hdYxFLeN2clvNHzSKqR8jz2fYHJifjk8S1w8e2DzxfMRD5 WytNffWFJ1qrnsLRg6OLfSqLZSGJOw7Y1jvPRiMqAujqKKRNUgkKNaJIaZWN61HFGzKY ONraQCew9r0p0/oOyWbhow3FZudGJtPZvxw5mbfjRycW0brCnQb2P+jpYMeF0JdYOBPZ 9Cog== X-Gm-Message-State: AOJu0Yz/YJVQ1XguCqukDjo74mlZccv/iQ+l+lmHTEzvzWQxV2u1c/Qo YaI7b2vndJ96gGMqksII88EU844YJ5QGbDFh9Us= X-Received: by 2002:a17:90a:a389:b0:268:4314:2dc6 with SMTP id x9-20020a17090aa38900b0026843142dc6mr20076489pjp.37.1693248953198; Mon, 28 Aug 2023 11:55:53 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 01/10] accel/tcg: Simplify tlb_plugin_lookup Date: Mon, 28 Aug 2023 11:55:41 -0700 Message-Id: <20230828185550.573653-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Now that we defer address space update and tlb_flush until the next async_run_on_cpu, the plugin run at the end of the instruction no longer has to contend with a flushed tlb. Therefore, delete SavedIOTLB entirely. Properly return false from tlb_plugin_lookup when we do not have a tlb match. Fixes a bug in which SavedIOTLB had stale data, because there were multiple i/o accesses within a single insn. Signed-off-by: Richard Henderson --- include/hw/core/cpu.h | 13 ------- include/qemu/typedefs.h | 1 - accel/tcg/cputlb.c | 79 ++++++++++++----------------------------- 3 files changed, 23 insertions(+), 70 deletions(-) diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h index fdcbe87352..20e4f64d72 100644 --- a/include/hw/core/cpu.h +++ b/include/hw/core/cpu.h @@ -227,17 +227,6 @@ struct CPUWatchpoint { QTAILQ_ENTRY(CPUWatchpoint) entry; }; -#ifdef CONFIG_PLUGIN -/* - * For plugins we sometime need to save the resolved iotlb data before - * the memory regions get moved around by io_writex. - */ -typedef struct SavedIOTLB { - MemoryRegionSection *section; - hwaddr mr_offset; -} SavedIOTLB; -#endif - struct KVMState; struct kvm_run; @@ -409,8 +398,6 @@ struct CPUState { #ifdef CONFIG_PLUGIN GArray *plugin_mem_cbs; - /* saved iotlb data from io_writex */ - SavedIOTLB saved_iotlb; #endif /* TODO Move common fields from CPUArchState here. */ diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h index 834b0e47a0..5abdbc3874 100644 --- a/include/qemu/typedefs.h +++ b/include/qemu/typedefs.h @@ -129,7 +129,6 @@ typedef struct QString QString; typedef struct RAMBlock RAMBlock; typedef struct Range Range; typedef struct ReservedRegion ReservedRegion; -typedef struct SavedIOTLB SavedIOTLB; typedef struct SHPCDevice SHPCDevice; typedef struct SSIBus SSIBus; typedef struct TCGHelperInfo TCGHelperInfo; diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index c643d66190..fcf6ccebff 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1364,21 +1364,6 @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, } } -/* - * Save a potentially trashed CPUTLBEntryFull for later lookup by plugin. - * This is read by tlb_plugin_lookup if the fulltlb entry doesn't match - * because of the side effect of io_writex changing memory layout. - */ -static void save_iotlb_data(CPUState *cs, MemoryRegionSection *section, - hwaddr mr_offset) -{ -#ifdef CONFIG_PLUGIN - SavedIOTLB *saved = &cs->saved_iotlb; - saved->section = section; - saved->mr_offset = mr_offset; -#endif -} - static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, vaddr addr, uintptr_t retaddr, MMUAccessType access_type, MemOp op) @@ -1398,12 +1383,6 @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, cpu_io_recompile(cpu, retaddr); } - /* - * The memory_region_dispatch may trigger a flush/resize - * so for plugins we save the iotlb_data just in case. - */ - save_iotlb_data(cpu, section, mr_offset); - { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); @@ -1438,12 +1417,6 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, } cpu->mem_io_pc = retaddr; - /* - * The memory_region_dispatch may trigger a flush/resize - * so for plugins we save the iotlb_data just in case. - */ - save_iotlb_data(cpu, section, mr_offset); - { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs); @@ -1726,45 +1699,39 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, vaddr addr, * in the softmmu lookup code (or helper). We don't handle re-fills or * checking the victim table. This is purely informational. * - * This almost never fails as the memory access being instrumented - * should have just filled the TLB. The one corner case is io_writex - * which can cause TLB flushes and potential resizing of the TLBs - * losing the information we need. In those cases we need to recover - * data from a copy of the CPUTLBEntryFull. As long as this always occurs - * from the same thread (which a mem callback will be) this is safe. + * The one corner case is i/o write, which can cause changes to the + * address space. Those changes, and the corresponding tlb flush, + * should be delayed until the next TB, so even then this ought not fail. + * But check, Just in Case. */ - bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, bool is_store, struct qemu_plugin_hwaddr *data) { CPUArchState *env = cpu->env_ptr; CPUTLBEntry *tlbe = tlb_entry(env, mmu_idx, addr); uintptr_t index = tlb_index(env, mmu_idx, addr); - uint64_t tlb_addr = is_store ? tlb_addr_write(tlbe) : tlbe->addr_read; + MMUAccessType access_type = is_store ? MMU_DATA_STORE : MMU_DATA_LOAD; + uint64_t tlb_addr = tlb_read_idx(tlbe, access_type); - if (likely(tlb_hit(tlb_addr, addr))) { - /* We must have an iotlb entry for MMIO */ - if (tlb_addr & TLB_MMIO) { - CPUTLBEntryFull *full; - full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; - data->is_io = true; - data->v.io.section = - iotlb_to_section(cpu, full->xlat_section, full->attrs); - data->v.io.offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; - } else { - data->is_io = false; - data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); - } - return true; - } else { - SavedIOTLB *saved = &cpu->saved_iotlb; - data->is_io = true; - data->v.io.section = saved->section; - data->v.io.offset = saved->mr_offset; - return true; + if (unlikely(!tlb_hit(tlb_addr, addr))) { + return false; } -} + /* We must have an iotlb entry for MMIO */ + if (tlb_addr & TLB_MMIO) { + CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; + hwaddr xlat = full->xlat_section; + + data->is_io = true; + data->v.io.offset = (xlat & TARGET_PAGE_MASK) + addr; + data->v.io.section = + iotlb_to_section(cpu, xlat & ~TARGET_PAGE_MASK, full->attrs); + } else { + data->is_io = false; + data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); + } + return true; +} #endif /* From patchwork Mon Aug 28 18:55:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717739 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347346wru; Mon, 28 Aug 2023 11:57:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFrCtZQ1y5Bsc6qeqxwkfiozguOIKaX2CeOaTUhqDZLwudAQJGZJIoTJnsWE7qBHcpCkaUx X-Received: by 2002:a05:6102:7b0:b0:44d:55a4:227a with SMTP id x16-20020a05610207b000b0044d55a4227amr16819765vsg.14.1693249020958; Mon, 28 Aug 2023 11:57:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249020; cv=none; d=google.com; s=arc-20160816; b=EWp73FfHId3IfCnAr35svomkkowcGZWppcmfg3iiVawgREEAF26oih/MFVPRQAZOuu H6wGValoOD3A7U2BlDXt1mcuSfiTGI9oQ3eTOrXtkdfXbiLF/A/xg1Wy99AnNEBxpd5T YXuQBsfyXMwm/+uN1aDL2jQLFxXgsNtdzXicPpMobNJaYrCa2yXMV4cyEstKVEi72DYD xIQA8DdfafUt7AAkT7OxPY7U4DXGoNSzyjTdANKVm3w1IsAXBCGffGlDXJqW6blv2LCi pEYSlYqAUsphwt62fxnu62wGIjXjI1FcNXvC1W/J7xCERn21PQA90WExExV2FMR9vElm xKcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=7K3agjAj/5Z9V2az8Pltf/TZABQcnS/GDuSWWPrto4M=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=Ff2flRqqGnRDAgd0TJuR/Na+aQ7tQY0X1jpUkum/cjG953M/x2O6r9MxKDtfqBkT1G RRuWSYEjc98uLkvaXA6WD4HIAON1bIYuRC1l7q6NhaHGFL7IQvHvIrMGbbeQtmWmgybW PeKtCtzi7pDz0VxFMFHf8yYGFJQ99MOF2Thb2i5i7UBfT/tZiaKFw8vy1SQGRmlFGgoI KrRJJU2A2y4ClHrXnZ1+rR16fQN6QPuabyqCWxKvx9BIKmhwlmaPa6QN7++CofNBPWJN FUHXPIykOkp2yVf5OnvXMvuxV0mi0BZecWG6hr4ZpiBmr3X4YddIL7xHxC5u7shxXJNc wl+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FdiXYEtA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f18-20020a0cf3d2000000b0063d456fcd44si5010994qvm.515.2023.08.28.11.57.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FdiXYEtA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP9-0000qu-7J; Mon, 28 Aug 2023 14:56:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP4-0000ot-7S for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: from mail-oa1-x2b.google.com ([2001:4860:4864:20::2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP1-0002Qz-Ac for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:57 -0400 Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-1c8d895a602so2016419fac.2 for ; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248954; x=1693853754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7K3agjAj/5Z9V2az8Pltf/TZABQcnS/GDuSWWPrto4M=; b=FdiXYEtAuIOeWw0FEEU4pTlgXX6bL0Bucwv+mbpNY3N7olA45QBU3ycKPPhDlTqKYL +LUCx6C2CRvewBDUnNccnmTafAnasLzTlyz6rGxRszJ52N/hZVE/GdPei/MhACYKPyst NqFbMUI6nJXf21SaZO7UtZVBkiIeSQDpTfuwenfPw442F4NsV8J3euXN6oUYkfIm9ykA JJOahdO0azAo75+T21ikOUxPBKx+/xwLfusSDYjyvG4U7iB38fwPBSn3wUbi//JB9tqk 2Zxytxxyw8bB2kyOklbhhsVU0zoheysg9hCGX7rnfjL3fwj21k3blKQungl6rA9ywk4q mWKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248954; x=1693853754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7K3agjAj/5Z9V2az8Pltf/TZABQcnS/GDuSWWPrto4M=; b=LVYMT0Me/Vc/0mDtZTs1Y1ojVZ9hu9Hmo81x/C34ngbnLi/MNG6/m2tZKFH06EG5e2 pInovv97vIr+v7sXKcxFd9K2OIMSpnqt6oiSvi3lnjVigzFgKwcbBFxjqsuFp9KmD5N/ BYzbyIi22gQeJPCkluJhI+ZGR7JGPmzdP84FP8pcLQx2sKot1UREieif7Bln5aoNaGnd vveZXjJAKtbt2yGcka1NB7wPXm3phEF+891mebsTSAAgEWXUTTCauncNumUlrnqPEu90 r7VVMY57hEAQwRn6HYXgDYSAExub1VCgYt/fdhJsfXDmO0DLsRcRHp2s4IsTqGDH+4KC 9WLg== X-Gm-Message-State: AOJu0YxCkqMsVQrvIrIZZVa0CQgm8sPrVU2WKK17jYFKG/OHC9B3i/lR NBJEoXK7/wdqgNt8xWmUXPFNeSPjf6nqiugNNqQ= X-Received: by 2002:a05:6870:4706:b0:1c8:bfd1:ecc0 with SMTP id b6-20020a056870470600b001c8bfd1ecc0mr12897140oaq.0.1693248954008; Mon, 28 Aug 2023 11:55:54 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 02/10] accel/tcg: Split out io_prepare and io_failed Date: Mon, 28 Aug 2023 11:55:42 -0700 Message-Id: <20230828185550.573653-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::2b; envelope-from=richard.henderson@linaro.org; helo=mail-oa1-x2b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org These are common code from io_readx and io_writex. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 77 +++++++++++++++++++++++++++------------------- 1 file changed, 45 insertions(+), 32 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index fcf6ccebff..17987f74e5 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1264,7 +1264,7 @@ void tlb_set_page_full(CPUState *cpu, int mmu_idx, * (non-page-aligned) vaddr of the eventual memory access to get * the MemoryRegion offset for the access. Note that the vaddr we * subtract here is that of the page base, and not the same as the - * vaddr we add back in io_readx()/io_writex()/get_page_addr_code(). + * vaddr we add back in io_prepare()/get_page_addr_code(). */ desc->fulltlb[index] = *full; full = &desc->fulltlb[index]; @@ -1364,37 +1364,60 @@ static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, } } -static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, vaddr addr, uintptr_t retaddr, - MMUAccessType access_type, MemOp op) +static MemoryRegionSection * +io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, + MemTxAttrs attrs, vaddr addr, uintptr_t retaddr) { CPUState *cpu = env_cpu(env); - hwaddr mr_offset; MemoryRegionSection *section; - MemoryRegion *mr; - uint64_t val; - MemTxResult r; + hwaddr mr_offset; - section = iotlb_to_section(cpu, full->xlat_section, full->attrs); - mr = section->mr; - mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; + section = iotlb_to_section(cpu, xlat, attrs); + mr_offset = (xlat & TARGET_PAGE_MASK) + addr; cpu->mem_io_pc = retaddr; if (!cpu->can_do_io) { cpu_io_recompile(cpu, retaddr); } + *out_offset = mr_offset; + return section; +} + +static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, + unsigned size, MMUAccessType access_type, int mmu_idx, + MemTxResult response, uintptr_t retaddr, + MemoryRegionSection *section, hwaddr mr_offset) +{ + hwaddr physaddr = (mr_offset + + section->offset_within_address_space - + section->offset_within_region); + + cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, + mmu_idx, full->attrs, response, retaddr); +} + +static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, + int mmu_idx, vaddr addr, uintptr_t retaddr, + MMUAccessType access_type, MemOp op) +{ + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxResult r; + uint64_t val; + + section = io_prepare(&mr_offset, env, full->xlat_section, + full->attrs, addr, retaddr); + mr = section->mr; + { QEMU_IOTHREAD_LOCK_GUARD(); r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); } if (r != MEMTX_OK) { - hwaddr physaddr = mr_offset + - section->offset_within_address_space - - section->offset_within_region; - - cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), access_type, - mmu_idx, full->attrs, r, retaddr); + io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, + r, retaddr, section, mr_offset); } return val; } @@ -1403,19 +1426,14 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, uint64_t val, vaddr addr, uintptr_t retaddr, MemOp op) { - CPUState *cpu = env_cpu(env); - hwaddr mr_offset; MemoryRegionSection *section; + hwaddr mr_offset; MemoryRegion *mr; MemTxResult r; - section = iotlb_to_section(cpu, full->xlat_section, full->attrs); + section = io_prepare(&mr_offset, env, full->xlat_section, + full->attrs, addr, retaddr); mr = section->mr; - mr_offset = (full->xlat_section & TARGET_PAGE_MASK) + addr; - if (!cpu->can_do_io) { - cpu_io_recompile(cpu, retaddr); - } - cpu->mem_io_pc = retaddr; { QEMU_IOTHREAD_LOCK_GUARD(); @@ -1423,13 +1441,8 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, } if (r != MEMTX_OK) { - hwaddr physaddr = mr_offset + - section->offset_within_address_space - - section->offset_within_region; - - cpu_transaction_failed(cpu, physaddr, addr, memop_size(op), - MMU_DATA_STORE, mmu_idx, full->attrs, r, - retaddr); + io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, + r, retaddr, section, mr_offset); } } From patchwork Mon Aug 28 18:55:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717748 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347740wru; Mon, 28 Aug 2023 11:58:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHfEBiWQpT/SFL0WX1bpG9Sy9HsMPE2ulGSag56EZUDk8XQXE3LDW+3B8mA4axGekHTLvx3 X-Received: by 2002:a0c:f4d2:0:b0:64f:72f7:6209 with SMTP id o18-20020a0cf4d2000000b0064f72f76209mr12588801qvm.11.1693249084329; Mon, 28 Aug 2023 11:58:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249084; cv=none; d=google.com; s=arc-20160816; b=g5utOVnOgS4pSROk4/bdhn5hbuYH1vE81QE5lAKMd/7wCpwb7sn5ONqFTRCdMn+l32 B++RYr+9GRR9H15sNFuum+iPdX/btLaRMsDeSwgfxpXqfTsaQqw7s81V0+cnn5/x+ynT ujUTQycJrczvceIUpYDlnQ5QgCUcu5Qgy2wThWFh9XQaEsDtBHggFhzTZeS9yeqICaCX vyd3aVVCt4Ihaj71PBUt7nMzupb8oKVC0HFzFWVbPKlR5CR4KZ206AoD2Vuk6ubGkgEY BtxKtR1W7fS9mze9jsrS1h7HBhr0reeaVJIdVx4tzOOrkc+bKP9z0G2DX9f5F1Vs5rlm 511w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=PXC5tTcUi0lmpfhMw90VKckGWB1iYRWY7/22QuukN5Y=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=Bc0MHVeykgIUPaDaHLdmIKghO4P4CY/6knDrcu9frRyUXw3Lz+0WywE+TGI/qm2uy8 97GnnE5QIWYtxhXSPhYmGGDRfFLByaqHVQTw/i2Oh2EgPqK3whTks6haOFnvyGU5vjoP E+MyFJ6RJ+8oXoCUjr4xQDA1RPymwiaoe6NViztYYJYbSH19j8sMRmh1jdr8LOtkzgMB CDsW5guC029RauNIby4VO8690yFN+idyM9WnzsTrJIdSPNv3Mg8rHOY98QwyencUBRK8 xJOHsuPeAPpY4hAY1d7T2uSp96HVWAFGQjVBZQ5U7DAn3xaRZ8UPx8hGGiQ+R5j5XMgN Ei5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Kd1cLMLD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x4-20020a0cb204000000b0064f43efc841si5077662qvd.446.2023.08.28.11.58.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:58:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Kd1cLMLD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP6-0000pP-45; Mon, 28 Aug 2023 14:56:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP4-0000p1-R3 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: from mail-pg1-x52d.google.com ([2607:f8b0:4864:20::52d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP2-0002RV-Hk for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:58 -0400 Received: by mail-pg1-x52d.google.com with SMTP id 41be03b00d2f7-564b8e60ce9so1664984a12.2 for ; Mon, 28 Aug 2023 11:55:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248955; x=1693853755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PXC5tTcUi0lmpfhMw90VKckGWB1iYRWY7/22QuukN5Y=; b=Kd1cLMLDbJlPQlmcLcuLYjtkG7ROMUJoSVcjc1Y2yY5ZivWdgP8FQHmF9bOonW8V/I J3mTuwSYQdYiU0uXQ9XowQI1ar6WrwLCtrAW/HJhlfEqfsCyB8QKztt7fnuoKYgDDs/O nq7jao+hCs/DKd4gHQosqLRrouZHv5QzCJNf3Rh90S8kAxXBLNg1gV0k9rGpYBzVBQWZ RQfvulioYJmPjzISsCbsKxhYgwofAgx5wfKRNxwEW3KYhqXB1iUhmoD+/FATUmHYvFYb MPFdODnk+sJO841ANvbNzh0EgbQACY1Pfi5c7BnCFT1SdTqWjzKXGF4Up9M8qQgkMxAZ 0MRA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248955; x=1693853755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PXC5tTcUi0lmpfhMw90VKckGWB1iYRWY7/22QuukN5Y=; b=XV7TAwt8+0WYiha2uY8rSibtFhaQ3P2ALFGZs05gRkqOVyd8igikFvt9Uxj2j/RjM0 tscKccj+TWp8iWtHXWzlHxYssyxNGIwHp9O+sGi2liACoLixtro6ujJ4KKD+dxBjzu61 rgTp77EzZ9Mx1HNACDsJra37DzCNNEpkJawDxDC9OGGYh18v9dPZnR9QUoTCNzF/ofJj 0P7J8di2lLj8YKh4CbyaphwAosyQ8aWGCZx8WYAZRROoQORgt5NaoOG5DGHvy9uoX/tw VoV4j3iLy436jo28rX8yz0KPxtqChNnAwCQWYVX//OWFr9aajz0MqsXvcSFrABR8sF5a ZzJQ== X-Gm-Message-State: AOJu0Ywf7/Nml3O7ZqL4TvGrhPswdif5Y19EMHESyGBHY/pfkLOCkYMB rOh7VVBMs6ZFqcTOWN8nS4YfmzaSh56oV8sQU8o= X-Received: by 2002:a17:90a:2f47:b0:268:fc26:73a9 with SMTP id s65-20020a17090a2f4700b00268fc2673a9mr19645961pjd.40.1693248955001; Mon, 28 Aug 2023 11:55:55 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 03/10] accel/tcg: Use CPUTLBEntryFull.phys_addr in io_failed Date: Mon, 28 Aug 2023 11:55:43 -0700 Message-Id: <20230828185550.573653-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52d; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Since the introduction of CPUTLBEntryFull, we can recover the full cpu address space physical address without having to examine the MemoryRegionSection. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 17987f74e5..fd6c956214 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,13 +1385,9 @@ io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, - MemTxResult response, uintptr_t retaddr, - MemoryRegionSection *section, hwaddr mr_offset) + MemTxResult response, uintptr_t retaddr) { - hwaddr physaddr = (mr_offset + - section->offset_within_address_space - - section->offset_within_region); - + hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, mmu_idx, full->attrs, response, retaddr); } @@ -1417,7 +1413,7 @@ static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, if (r != MEMTX_OK) { io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, - r, retaddr, section, mr_offset); + r, retaddr); } return val; } @@ -1442,7 +1438,7 @@ static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, if (r != MEMTX_OK) { io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, - r, retaddr, section, mr_offset); + r, retaddr); } } From patchwork Mon Aug 28 18:55:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717745 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347546wru; Mon, 28 Aug 2023 11:57:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEkSUun43hMz+NbRYd9fTaJ5QWLyeYLA1oftxiVanw7toU+lerL4512UnVcnQfnTljrSNh8 X-Received: by 2002:a05:620a:2007:b0:76f:19fd:5063 with SMTP id c7-20020a05620a200700b0076f19fd5063mr1184855qka.34.1693249051453; Mon, 28 Aug 2023 11:57:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249051; cv=none; d=google.com; s=arc-20160816; b=wFckeksb3jldq1LTWNMfgnOoEKIXlhMPR93vCGqgsgh789eJNRNxxel/nTi5xD5VPO t//P8PUHkgrPZnnWmK1W2iLzgxyoGLHyC0W0SP7NUV/rrykQ86Vj+ZhqrgWdSuoly1GA Xzma5nsc1zS3QkBdMSZUVvVKgoLsB9JidTtqrbbD/AsNbfoNHkgZhlBf3OdE89td56UX IyfcNJ/73FdOJqIByZxRP7rLYgSeuloWMsH+lNvSCaD0rS0j+eEL9u/0VoQpYY3KE7ps nUMteWYLUbj8XXCqGOP0U5h8h69gY+69xw/URvvR2tzXUBtZoi6yuWd2eaa3iNVxZFRr y9oQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=7fAz4IDlZXVLjSd/SH1PWPNAHXfaCucW3CyJSYpT2lM=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=K7X0iisyv/dUfY2vEkCVl0nThMw7bw2vXGmW0XyXHPAeVpDEMynbBKSkt92+nwnDXp 9VaHzwOCLG1VFsiLDT8K5HvByd5FEINVPtDx2/FJnveE4qr39aWgC5wqGPVySn/kc0x8 iF9YtBqWEqh42RaLG5z6jSbFaqOs2ABDbR4kch3HkbtXibcDWBKS17hTpikUYguBM55P x1ns67yqzOmEs7l0E3EhgcrdWfQQM5IGDRp3cznpIK2FWSbJWu4SfdRmVpgxkfVWWf6I OS7Gy0mSWc39VOMntlzhESsCNpJOEinIhkOVHyL78cguaDau3usl+ZYnn5W0Y0v8ruhy kZbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E4xlrPgE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id bj33-20020a05620a192100b0076cc7de6bc2si5513516qkb.442.2023.08.28.11.57.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=E4xlrPgE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP6-0000pX-Q3; Mon, 28 Aug 2023 14:56:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP5-0000pN-UW for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:59 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP3-0002Rj-KD for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:55:59 -0400 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-56a55c0f8b1so1702578a12.0 for ; Mon, 28 Aug 2023 11:55:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248956; x=1693853756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7fAz4IDlZXVLjSd/SH1PWPNAHXfaCucW3CyJSYpT2lM=; b=E4xlrPgE61OLpG+Lg2pYYPFU10HAxkOaVX5Qhs+/nyHFH20ZK74yUdx6Jv+GLDg9NI 2coQT4DL2VRySF6Og5j2of+lFXce+Y1DwP9299VXmTdfmVkGkdTb8dLiaAm+g286db3K H/etax29NYtIkQFLcxNEqISPLMFcyHdwWo24x7q7mn5Zsm93D+xX04pQ+ZBAAAARwWb3 LmhTLFJ8PnlTdIxTw4mx0SHxOfE9U3AYhdzkL11hROr6IfirNK/rUVZWwiiJ9hkVj8hO b7u004dRc2Pw1K3y/H14NFDsSrZHGpBd6mADd9A3DEHhFJvef0z46hhDJoqtksUxQb7o LfVQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248956; x=1693853756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7fAz4IDlZXVLjSd/SH1PWPNAHXfaCucW3CyJSYpT2lM=; b=QfAO/P1M1ctzgJJLLBrPYsqASkZnmyDhBOEVhe2DBol/tJdjJM+KaMSg/K8HQ21E3X xKl4VXWAFujh239CiXnou1K4vaHOueWYmOV2I6D/FPR2Ts1+j2W9crMkIpwjyot1RLBu Bi3jz3LAnRUdDqSFbnVwQcpGFTuBWPCEauJSfFcEBqQEYqqUolUYkfkqHfVQGuvCjYbm mIZQ7Om0i0LSDw+9hOegL6aSArCoLvRHevEKIb85iKaZE6LxwmC2FgOw/UEtkkLX4tHb ji07glYO1XVY5LDqL2dZmc6oqjraVUNoDktShpkYbvFmtmO/WWlzIpv0tXokOrGXXRuM n1Rg== X-Gm-Message-State: AOJu0YxlKU4x68pOh8Xz10weYsvGBGZG/q7YXYMiIrodc/VTGYoIl7q0 CtOfbxc2AP34X6fz+2ZjhuRFcZljS8Q+U/mT+wg= X-Received: by 2002:a17:90a:db08:b0:26b:3f10:820f with SMTP id g8-20020a17090adb0800b0026b3f10820fmr19825065pjv.36.1693248956126; Mon, 28 Aug 2023 11:55:56 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 04/10] plugin: Simplify struct qemu_plugin_hwaddr Date: Mon, 28 Aug 2023 11:55:44 -0700 Message-Id: <20230828185550.573653-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x531.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Rather than saving MemoryRegionSection and offset, save phys_addr and MemoryRegion. This matches up much closer with the plugin api. Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- include/qemu/plugin-memory.h | 11 ++--------- accel/tcg/cputlb.c | 16 +++++++++------- plugins/api.c | 27 ++++++--------------------- 3 files changed, 17 insertions(+), 37 deletions(-) diff --git a/include/qemu/plugin-memory.h b/include/qemu/plugin-memory.h index 43165f2452..71c1123308 100644 --- a/include/qemu/plugin-memory.h +++ b/include/qemu/plugin-memory.h @@ -15,15 +15,8 @@ struct qemu_plugin_hwaddr { bool is_io; bool is_store; - union { - struct { - MemoryRegionSection *section; - hwaddr offset; - } io; - struct { - void *hostaddr; - } ram; - } v; + hwaddr phys_addr; + MemoryRegion *mr; }; /** diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index fd6c956214..b1dc213675 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1721,23 +1721,25 @@ bool tlb_plugin_lookup(CPUState *cpu, vaddr addr, int mmu_idx, uintptr_t index = tlb_index(env, mmu_idx, addr); MMUAccessType access_type = is_store ? MMU_DATA_STORE : MMU_DATA_LOAD; uint64_t tlb_addr = tlb_read_idx(tlbe, access_type); + CPUTLBEntryFull *full; if (unlikely(!tlb_hit(tlb_addr, addr))) { return false; } + full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; + data->phys_addr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); + /* We must have an iotlb entry for MMIO */ if (tlb_addr & TLB_MMIO) { - CPUTLBEntryFull *full = &env_tlb(env)->d[mmu_idx].fulltlb[index]; - hwaddr xlat = full->xlat_section; - + MemoryRegionSection *section = + iotlb_to_section(cpu, full->xlat_section & ~TARGET_PAGE_MASK, + full->attrs); data->is_io = true; - data->v.io.offset = (xlat & TARGET_PAGE_MASK) + addr; - data->v.io.section = - iotlb_to_section(cpu, xlat & ~TARGET_PAGE_MASK, full->attrs); + data->mr = section->mr; } else { data->is_io = false; - data->v.ram.hostaddr = (void *)((uintptr_t)addr + tlbe->addend); + data->mr = NULL; } return true; } diff --git a/plugins/api.c b/plugins/api.c index 2078b16edb..5521b0ad36 100644 --- a/plugins/api.c +++ b/plugins/api.c @@ -316,22 +316,7 @@ uint64_t qemu_plugin_hwaddr_phys_addr(const struct qemu_plugin_hwaddr *haddr) { #ifdef CONFIG_SOFTMMU if (haddr) { - if (!haddr->is_io) { - RAMBlock *block; - ram_addr_t offset; - void *hostaddr = haddr->v.ram.hostaddr; - - block = qemu_ram_block_from_host(hostaddr, false, &offset); - if (!block) { - error_report("Bad host ram pointer %p", haddr->v.ram.hostaddr); - abort(); - } - - return block->offset + offset + block->mr->addr; - } else { - MemoryRegionSection *mrs = haddr->v.io.section; - return mrs->offset_within_address_space + haddr->v.io.offset; - } + return haddr->phys_addr; } #endif return 0; @@ -341,13 +326,13 @@ const char *qemu_plugin_hwaddr_device_name(const struct qemu_plugin_hwaddr *h) { #ifdef CONFIG_SOFTMMU if (h && h->is_io) { - MemoryRegionSection *mrs = h->v.io.section; - if (!mrs->mr->name) { - unsigned long maddr = 0xffffffff & (uintptr_t) mrs->mr; - g_autofree char *temp = g_strdup_printf("anon%08lx", maddr); + MemoryRegion *mr = h->mr; + if (!mr->name) { + unsigned maddr = (uintptr_t)mr; + g_autofree char *temp = g_strdup_printf("anon%08x", maddr); return g_intern_string(temp); } else { - return g_intern_string(mrs->mr->name); + return g_intern_string(mr->name); } } else { return g_intern_static_string("RAM"); From patchwork Mon Aug 28 18:55:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717743 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347435wru; Mon, 28 Aug 2023 11:57:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEOYKutL/wLZ2Wb3U3IX2FNQsxygiwlfXuEXoLH8hOv+zdncdWSkFvu56LF2cYubpLMszY1 X-Received: by 2002:a05:620a:29c9:b0:76c:97fd:c113 with SMTP id s9-20020a05620a29c900b0076c97fdc113mr32566660qkp.15.1693249034781; Mon, 28 Aug 2023 11:57:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249034; cv=none; d=google.com; s=arc-20160816; b=r3krY4c8MgwsdhHkXQOyeNE9EZ9rMIDoW+1y7MGsPinBrmmTRXsPviwK31SRRnQDF3 XS8MRWm0PfX/LnvUouCYUiTTVo5XpHPOL6cBAO9ah9pmGYtzk24oJ6HC/VLvXNSlQpuW /8aVjWt/SGyiQ0ln/AEeW2nIOBKD6Fiw4vJWWoO8Nc4rF2nCjOdHQJ9eRgHm3tSaaYY0 F2PRsOHcOMsgxJ/XtO9rIk22QfE/c52YXdYzUw+FV/UolWD4GMAfUkaQkhE7hBp9DuLU Qvd0qdLqyj1zbOglIMcHFtPlon9aneBDl5BrDU3HIKMA0yOaCZeY+DYEPZx/NI1WOzg3 cOqw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=zs4ouB+IX/j6ZyzVg7pnpseLfh71GLbrppugYTGLrDA=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=XDlVQeZOrLrIRB13XtLwC49JNWRccfY8hZ89x2U+7jtdptJme6qakfJ/aBlQJjCxr6 NzkDuJ5Pe/o17L3oEjDxqs/7ytzcfB3qewiKd8876uuqp6wqdXr8lCPfBdLbTqnw6fDR nQvP2RDj2TILQGayCP/Saynq01ibnpYq/9Nj0NbaTPc98XDoQjOBrdXHz+Iv7PdO2uNm WSbwgIx9TsHrkK0fuQNl8MZt0nmyXa3f3ZSZPN5wA7O3jkoAL1dnjEBJSa8pLsvUW+Xj xXD55kY/Njj+u72+C3n/EdWAYdOGwOTRU7vC6uGtGTB5RPs0oGM5yDVF5Iajd0G8NC2C 24Ug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lN7aAqol; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id to4-20020a05620a4c8400b0076cb682f501si2935552qkn.777.2023.08.28.11.57.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lN7aAqol; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP8-0000qm-Sg; Mon, 28 Aug 2023 14:56:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP7-0000po-9D for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP5-0002S2-3K for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26b44247123so2254482a91.2 for ; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248957; x=1693853757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zs4ouB+IX/j6ZyzVg7pnpseLfh71GLbrppugYTGLrDA=; b=lN7aAqolA/8tcvFHR0Zvln/tX1gsiAtzxgz8PfQaTFRmO4Q2G1WExradIaVLUIKdHL lnWGtKE0VB4kOMgeg5CMLaKdkUQ8q+sfyrG5hF6moKywqNAx4TaQQJTI15iSsVwFbG+7 rIduSO6Jx2kCErACxo/K4oiuS8QRlBEnPOUKneSYWL1hMbeEmgFcsmnnbaxVU680J17n kOQuW+aGs19UGzBY2gSMA4jr0Fw87cgiLsDFPlOTUYJPTWlnItLcmW+dBfsn0bYuC+cO etuExAJEsDvP9ox0qw2WQ0QkRAy0/HSKy1945M4j/9yyN6/SoO4p97VSsT0hgmLMZBXA g3jQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248957; x=1693853757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zs4ouB+IX/j6ZyzVg7pnpseLfh71GLbrppugYTGLrDA=; b=V+U5b9noFalV2T2A32Y82ZdKVqrkFMa0GccHckzQ3u8FBtx7/NECWrtHtUPoy7rerV 19PZ7/6AsPhXZUlakNEQ2vn/g862kFfYYGGTg9Fnpe08393okogMxpKNgpE6tVEl0Ztq fhgdKb1d3c9H0tuvEHgulbILk1PkjEJ2be8eIEc+3gGDYnubezoHVXIOZXBcZZB/BD14 nFU6xZ958zWHgAXaEOK70aKycVaqQ3e5kkOGGIfGMNDgRVotlUz6C5PMQR61h874Dw+f Ej7kbZa7S9IibVYTKYvxdY4JLbPIPK1hl4QotLX8jvJS65PQhlTq+4wl053P85Yh7yAG y46g== X-Gm-Message-State: AOJu0YzE1ZhzN9H+IRq35X0BO8nra1rDpX1ixLkmjZdLhxbbXFXJ3dUE trDiY1W/AgnUMU0hTfbEsePkKSZJhElt4BnxBC8= X-Received: by 2002:a17:90b:4015:b0:262:e3aa:fd73 with SMTP id ie21-20020a17090b401500b00262e3aafd73mr25314055pjb.17.1693248957185; Mon, 28 Aug 2023 11:55:57 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 05/10] accel/tcg: Merge cpu_transaction_failed into io_failed Date: Mon, 28 Aug 2023 11:55:45 -0700 Message-Id: <20230828185550.573653-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Push computation down into the if statements to the point the data is used. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- accel/tcg/cputlb.c | 33 +++++++++++++-------------------- 1 file changed, 13 insertions(+), 20 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b1dc213675..d2e4c4459d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1347,23 +1347,6 @@ static inline void cpu_unaligned_access(CPUState *cpu, vaddr addr, mmu_idx, retaddr); } -static inline void cpu_transaction_failed(CPUState *cpu, hwaddr physaddr, - vaddr addr, unsigned size, - MMUAccessType access_type, - int mmu_idx, MemTxAttrs attrs, - MemTxResult response, - uintptr_t retaddr) -{ - CPUClass *cc = CPU_GET_CLASS(cpu); - - if (!cpu->ignore_memory_transaction_failures && - cc->tcg_ops->do_transaction_failed) { - cc->tcg_ops->do_transaction_failed(cpu, physaddr, addr, size, - access_type, mmu_idx, attrs, - response, retaddr); - } -} - static MemoryRegionSection * io_prepare(hwaddr *out_offset, CPUArchState *env, hwaddr xlat, MemTxAttrs attrs, vaddr addr, uintptr_t retaddr) @@ -1387,9 +1370,19 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, unsigned size, MMUAccessType access_type, int mmu_idx, MemTxResult response, uintptr_t retaddr) { - hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); - cpu_transaction_failed(env_cpu(env), physaddr, addr, size, access_type, - mmu_idx, full->attrs, response, retaddr); + CPUState *cpu = env_cpu(env); + + if (!cpu->ignore_memory_transaction_failures) { + CPUClass *cc = CPU_GET_CLASS(cpu); + + if (cc->tcg_ops->do_transaction_failed) { + hwaddr physaddr = full->phys_addr | (addr & ~TARGET_PAGE_MASK); + + cc->tcg_ops->do_transaction_failed(cpu, physaddr, addr, size, + access_type, mmu_idx, + full->attrs, response, retaddr); + } + } } static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, From patchwork Mon Aug 28 18:55:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717744 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347536wru; Mon, 28 Aug 2023 11:57:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGcxJRXwYnDcjlA8MWPZZyG0E8P/UThr2jLUms/RPW8jgo1fDcc9LhyDKmbHNuAJpbbNNeN X-Received: by 2002:a0c:e251:0:b0:64f:518d:ac96 with SMTP id x17-20020a0ce251000000b0064f518dac96mr727554qvl.12.1693249050493; Mon, 28 Aug 2023 11:57:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249050; cv=none; d=google.com; s=arc-20160816; b=UnbYcfYX2+IRIwSTRlgllXGohUE9sCUyd6Hzj+bsR+MzymWYjbdJ3vzGULtu3sCRXz 3SYrUAGgiEi4clK+XVajoepQvildU5PMDiHcC/3I1FAJT5aKhv/9jGQiq0KJzFToyiV2 hwDIsgNMgoKBQIUHUIHmA9GR+f5qWjI26kvWJd1pcqsK68KAQlGNp7ELUEZ7scT8V0nd A82Ht8KJWPVyggt5RzciURf6lAOPeJ+fPdRLlUgzOouW4knF9PFiPFmErnGGvJp1ST6/ NgPU+IdQb/CJnm0zyZugvLdOExH+AfGRG0btXqJSDwOiFSd1P7kyMthjpuYURYxDUfxk sVNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=kes4dWAfnCF0sbsmuI+kEptn1hS/yp74fg4t9duR770=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=VfGAdTWkWokwiiCXWlJ5V4HPUHhpSIiP5W66bY3QkMwIfNpaI1wNrc4NaBDxibg5WL EuSNfhw25GWLv2Jv+Napl9U21yZnvRjEuOMV/A4KadkjRAtU6OV5NuKn6/YQcgtTuqIi ORZOkMHl6cfQFQE8XBv4d2ja7Vcer6bDW49r+J2OaCQGxuzgjr5f+za+Ri2RNc2Bx72d AAgvLkYxzu+AtFnq5zTgMnyDJSZSnbxIu4/qO6gUQykezXNoHgL7U2IOBWT57cwq8RT0 O0fKhi7QupD2PuAL28+m836C5bh63+sN7H66GoOoMXJPlHUvu4PIgwso0McHa15Lq0cV hwHw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PVlt6LU7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h3-20020a0cf443000000b0064facf0cf5dsi2809440qvm.49.2023.08.28.11.57.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=PVlt6LU7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahP9-0000r1-Bn; Mon, 28 Aug 2023 14:56:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP7-0000pr-D3 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: from mail-pj1-x1032.google.com ([2607:f8b0:4864:20::1032]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP5-0002SC-9Q for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:01 -0400 Received: by mail-pj1-x1032.google.com with SMTP id 98e67ed59e1d1-268bc714ce0so3097399a91.0 for ; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248958; x=1693853758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=kes4dWAfnCF0sbsmuI+kEptn1hS/yp74fg4t9duR770=; b=PVlt6LU7ElyNwkbZTuIdBKWHN+yZ6AbiKL44WN+tpuTzvnCE7I7CoIGJ56vbugw01Q MX9ThoXSE2u0HkHbIJOGiTbabNx0NVFe3bv77/NJHQks6qAOdLc4vHJTHoicedeyyp7p +qeTA/UiRchYgzr5zFpuVfxIROdOHKXvW5nfbxJOMlXw2L/7v6j9yQhPlZQQmWulQJIA 3QQudAU2YgC/BL8+POFnEBIa3cTzFxvmUiUZBDsvDNUGzBQil9Tx0OakHg8mjvo51Q1i aH7hVo86Uid02hMEue3P3tJMZVwSBio9idOU+EqjN1vspAiyjeRPtuPIoCAb6UQWMwIz x+Ng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248958; x=1693853758; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kes4dWAfnCF0sbsmuI+kEptn1hS/yp74fg4t9duR770=; b=d37QsIAX1/RDrrpw5OdYKqxB+n95e5vV3OTDL4c7HEm4zqYbOBGVCELSIvLqnf5p2i bXnum9WeTjf82MR98kQXLxb5nWHpmgmolnZgy389mj6P15mIST7xiUiNnmjuH0Tc13gK 2HrdmZ4lzbmBsa61uLrnoimJDdIbEAsjxZAbNPdM0O5x+8AaU0p4eXJp/jxahxQyXJ8V geQIO+ZcNEOsRdjD5QhsEXub6SUvTHutXvxFGGcriVU9t6YF8RUZ+xFmQBPYsB5goyrd x5BAvgvtohM5MC5iMjPDvMMnvaId+U7WoTQSaWKQae5qonLgcb8J+R8PiHnomVNdLYy7 Mmaw== X-Gm-Message-State: AOJu0YxUaHmkfbny2W6HqPFQgSK16aoZMkdVDpD0KoNyxB9sj/60WJaD odP3on/jOr4Nnt+AXtgdbmnD2cV35vk/vgg5P0I= X-Received: by 2002:a17:90a:d185:b0:268:808:8e82 with SMTP id fu5-20020a17090ad18500b0026808088e82mr683519pjb.1.1693248958105; Mon, 28 Aug 2023 11:55:58 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 06/10] accel/tcg: Replace direct use of io_readx/io_writex in do_{ld, st}_1 Date: Mon, 28 Aug 2023 11:55:46 -0700 Message-Id: <20230828185550.573653-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1032; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1032.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index d2e4c4459d..6a7f6bf701 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2339,7 +2339,8 @@ static uint8_t do_ld_1(CPUArchState *env, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - return io_readx(env, p->full, mmu_idx, p->addr, ra, type, MO_UB); + QEMU_IOTHREAD_LOCK_GUARD(); + return do_ld_mmio_beN(env, p->full, 0, p->addr, 1, mmu_idx, type, ra); } else { return *(uint8_t *)p->haddr; } @@ -2854,7 +2855,8 @@ static void do_st_1(CPUArchState *env, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - io_writex(env, p->full, mmu_idx, val, p->addr, ra, MO_UB); + QEMU_IOTHREAD_LOCK_GUARD(); + do_st_mmio_leN(env, p->full, val, p->addr, 1, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ } else { From patchwork Mon Aug 28 18:55:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717746 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347633wru; Mon, 28 Aug 2023 11:57:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGlpeiDTQ647wZkbrw94wJbaMJg+Xw+2P8Nmkb8SbPwzCXRc09eb/gU8N5WpGiAMSifaPuE X-Received: by 2002:a05:622a:120a:b0:403:ef20:5142 with SMTP id y10-20020a05622a120a00b00403ef205142mr34817717qtx.40.1693249066638; Mon, 28 Aug 2023 11:57:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249066; cv=none; d=google.com; s=arc-20160816; b=eMJUN1ZzjO6TPPTOghdMduM7TS0lnUIGEYqhKiEwkEFoKutsbvaIpwRxJcdir7K3E8 lDLgS+u/FmzJq5v6EcQWiQw55fwpbeDqO0n33z5jMy61flnhDCCOHrQcVSbkLWIQYe8s HOKHGnOZDhWQXw+0Dl9FqO9tjVeS0/U44TeZ7414roPRAet1Pv6LnJ5C1o0Er5OYPh9w sfiEfJEeZfK27D+0ipQE5kyR0uAz/YxWi6FwEmymNfhqM3olJ8seayQui1pLNQa1vB6C 42B9KeYwwLFWpdypsSxTBHtjIR9i8cohoc+FQlZsKdfeKYBiIgk7hUNO23pLe7WT+Lxl fl9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8mz6E+/wbBE5eFwC0E1gYDTq5fpYP+E2DwEXnYpjQuQ=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=lcplyHfHRenuFULE18fz2eUnwKP806W61/aWtS5DHlqfCcF/j2aCbxA57THSfBxMM3 fvmdg+HcOO/2k1yDte+5zO4/FI/MPjb1/ptbBA5YWnDw8eXiPsNY3eWkTFmRgC721Krg lHTKkCHkVQnlB9MYq0eIpD+bI3xkqtqVBBSblUMIzwJfPeHIVCx4YrtfVBokLIlE1Qo/ Espx5XDUHmezYAuIeboLv/iGryZZVPgMGY1DBP/xMVBeO2lTbiTeoHMj9WSgmcgWh4x3 c0rJbSZ7Xmp5l2f+KPoxnLZ1TglWK4rCQIn2iTS+591tIWjyQfddZJ10W3nxxrezl3J7 IVnw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ev9AcvLo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a17-20020a05622a065100b004120bff5c3csi4956196qtb.236.2023.08.28.11.57.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ev9AcvLo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPB-0000rp-FR; Mon, 28 Aug 2023 14:56:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP8-0000qn-Uo for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:02 -0400 Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP6-0002SP-Hf for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:02 -0400 Received: by mail-pj1-x1036.google.com with SMTP id 98e67ed59e1d1-26d54d3d984so1646919a91.1 for ; Mon, 28 Aug 2023 11:56:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248959; x=1693853759; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8mz6E+/wbBE5eFwC0E1gYDTq5fpYP+E2DwEXnYpjQuQ=; b=ev9AcvLorCgsfvAPYLODTxwYRpY0ernGMzjQB++gjzdCOF7m/7rUDNg8MtMbyTUjpG sEQoIiYdyYa/VdZuCFiin5MtkDuIY7PIiDrz7ayWegFUQRwSLQOWwSZySn1ibVk9GGLN b/NgkvRG7k8kcx7gTBm8m447g9RLjaSxdj907kleSk4IdghmJK8x1e8CSe4vTEj+c7Yq 4XW2AsVLMtcY+0B3gETglvPay6SszOF158e89Q6vQSrT9nbuRQu/hCKcXdKS/hS5f+bi iBqmQKxSAjbQlcXyn3QCMJxpJ4mSnJzs9cecIJGKmc03QTwobRvbCUd8/tKVBXjGs7kS gegQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248959; x=1693853759; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8mz6E+/wbBE5eFwC0E1gYDTq5fpYP+E2DwEXnYpjQuQ=; b=AJwr0nPKL2k6DTHFLB8CevzXLgpqtNv425UCqv5pPX15wXaCEYjY/bW1S0Id0arNKn Uq3vtTzb71CRCwTBjz6+zyzN4McbNq7/2wAV7j0gSteKEdTmfdhfmClzyKv/XLyfYUrn fe76HAFuoOMVtnODpuz07qyzrfAjP2Tv355GtLyTeDiC0pBcAIgPij5TuGcyzqHtpa+T u9ygfEyDMsiorvON8kpdhSpiG42FkTMNzIfPY2l4MsFs7on1xPY8IaK9jwYSRXmWwu6c 3mKFA+6+LhDeo93lj80xvn6pfY8RqbBbsjbdu9N6cqE3AJXEvc4Bx59b4EkYAmK23jCT Es7A== X-Gm-Message-State: AOJu0YyDfVdi/mmWmldmRqRHnkpkDBi+TX50jqyDHcwIViqLUC+cmqPN 1xwC1VzS/wARUAkzYbQcf95ap/Jo5RJZxhFlTWU= X-Received: by 2002:a17:90b:4a46:b0:26d:4421:854d with SMTP id lb6-20020a17090b4a4600b0026d4421854dmr20741968pjb.37.1693248959019; Mon, 28 Aug 2023 11:55:59 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 07/10] accel/tcg: Merge io_readx into do_ld_mmio_beN Date: Mon, 28 Aug 2023 11:55:47 -0700 Message-Id: <20230828185550.573653-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1036; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1036.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Avoid multiple calls to io_prepare for unaligned acceses. One call to do_ld_mmio_beN will never cross pages. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 84 +++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 54 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 6a7f6bf701..f4772b5c5d 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,32 +1385,6 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, } } -static uint64_t io_readx(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, vaddr addr, uintptr_t retaddr, - MMUAccessType access_type, MemOp op) -{ - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxResult r; - uint64_t val; - - section = io_prepare(&mr_offset, env, full->xlat_section, - full->attrs, addr, retaddr); - mr = section->mr; - - { - QEMU_IOTHREAD_LOCK_GUARD(); - r = memory_region_dispatch_read(mr, mr_offset, &val, op, full->attrs); - } - - if (r != MEMTX_OK) { - io_failed(env, full, addr, memop_size(op), access_type, mmu_idx, - r, retaddr); - } - return val; -} - static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, int mmu_idx, uint64_t val, vaddr addr, uintptr_t retaddr, MemOp op) @@ -2059,40 +2033,42 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, uint64_t ret_be, vaddr addr, int size, int mmu_idx, MMUAccessType type, uintptr_t ra) { - uint64_t t; + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + do { + MemOp this_mop; + unsigned this_size; + uint64_t val; + MemTxResult r; + /* Read aligned pieces up to 8 bytes. */ - switch ((size | (int)addr) & 7) { - case 1: - case 3: - case 5: - case 7: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_UB); - ret_be = (ret_be << 8) | t; - size -= 1; - addr += 1; - break; - case 2: - case 6: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUW); - ret_be = (ret_be << 16) | t; - size -= 2; - addr += 2; - break; - case 4: - t = io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUL); - ret_be = (ret_be << 32) | t; - size -= 4; - addr += 4; - break; - case 0: - return io_readx(env, full, mmu_idx, addr, ra, type, MO_BEUQ); - default: - qemu_build_not_reached(); + this_mop = ctz32(size | (int)addr | 8); + this_size = 1 << this_mop; + this_mop |= MO_BE; + + r = memory_region_dispatch_read(mr, mr_offset, &val, this_mop, attrs); + if (unlikely(r != MEMTX_OK)) { + io_failed(env, full, addr, this_size, type, mmu_idx, r, ra); } + if (this_size == 8) { + return val; + } + + ret_be = (ret_be << (this_size * 8)) | val; + addr += this_size; + mr_offset += this_size; + size -= this_size; } while (size); + return ret_be; } From patchwork Mon Aug 28 18:55:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717747 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347687wru; Mon, 28 Aug 2023 11:57:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEzY40V8YiiPQ7LEFkMUyw5GdkJVd+/4fcpyHosHxxN2Ej7y1TryXHKhsuZjCzZvxR0SO5v X-Received: by 2002:a05:620a:25ce:b0:76c:e0c5:2f6c with SMTP id y14-20020a05620a25ce00b0076ce0c52f6cmr30670142qko.38.1693249075323; Mon, 28 Aug 2023 11:57:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249075; cv=none; d=google.com; s=arc-20160816; b=cKEHW89n4xlr/ChYnVK3fJer8oFYYGT0yu/WCNjLDs2ROEsaJsS9qulPD+gF8b9h5D IvynY2Iuq8WolGo0Xv6aQGuGxlYB2sjsB+y6cQbZJglioBCF6bJkO0qbcAvdhLiGHn3O 8iqswXocilIpgUR/mKhk2FtRGlAAKhjO053zYP+b7f+m9DF9LBJ4UUb+akKbW5bly1yo Z9CL3JDi6Ou+6Md2PVHhhH2GRToAxKuvbeW6MDN161ZoEp7cP8c8wbkk0HYDAy4xSGQN z8ON98Kft1jTaS8Jmmm+FS82gpbmlnWe4Nb40SVKca01AL03b8xlzPu2iBEDUAPwRLUG HRhw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Q52YryY77MZ2LmC1AwinyEGqkvB+2AbO6eV6neuXedY=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=LXii9Ugwujqv9zpOoN4MWY7JUsVyNPk5p8iImZdHFST9twAz94r/ZNldHx9jLnLyj9 XecGgzsAnSSbHN2CiTthHTZkQHShq+HbZiINJnk7XSzrCjkV+uAr3fHTDQz19DJlEx7U ReSDbhNtQs25z3xgDORvJ5Z+CohwPvJ8EDebzVJUTwvia5xEiTQlu3k/W8XBhOHG2w9n PQ8yq4G4RdRWVxxP/Rwx5/uEG76zUmrIgXq4prP/whB1gzKgusaX+JcOguDyco8yK+eA BBUYSSdcBzS0R2V0GNCoVOXXTxmIyKsFmtvpjAvxfQyhaPLlQBo5mlFmPeCBIbSaBD6s PaSQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gDR4y1GM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n10-20020a05620a294a00b0076d9fa3e6c8si3132319qkp.409.2023.08.28.11.57.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gDR4y1GM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPB-0000rs-H0; Mon, 28 Aug 2023 14:56:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahP9-0000rQ-L6 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:03 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP7-0002Sa-CZ for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:03 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26ef24b8e5aso2282742a91.0 for ; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248960; x=1693853760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Q52YryY77MZ2LmC1AwinyEGqkvB+2AbO6eV6neuXedY=; b=gDR4y1GM4p6ymFuZdvFKhpQUY8CeIu7itJ0Mo7YMk0FABEwHdlpawQbZVADkHdH/5T z6wQvAOJAkkwHewMYqcS57bXmoHKD3twileIqCmBpWxNxE05df77N07bvBps1heApkI0 WE3awanwadjzqIEKoT6wLEveu8/XtyXVq5eaxXNWmUslrB35xXvt+TCMsr9ap3Fvechm cPqZCEVvHKk3ybhsv33VCOHCBdrme0on7O8iJxpuq87fG3LINaHOF5Jv9gibolZKWvrV Z6ImsIHfwotvWbMEE8gFehq7iOeI2Isgptzy+GdvIpxfxnmv09Wq52c66NvUHC0YV49H Zfwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248960; x=1693853760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Q52YryY77MZ2LmC1AwinyEGqkvB+2AbO6eV6neuXedY=; b=Izv2/t9bfY9+5+w7imj/LA0YBPj9UpccrJ9/BkfHm1oJUkSEonYD4T5sUs1rYLj4RA Qr+wnq7f5jS+9YSNB/tsHDKFxq0ffpU5V9qCVhxg3dwy04xWXefXiq5YyaJR5k+iNKMq FXHYCaK5Ij7Cpd2Sw1gSfMgEmkjrw461kHG3sQWAHJ+TUbh0LeTv9YgI4HGcDlwMkmTs LZ2qIVfuPtIG2gryjQIVuGUvwE1l0UOs0/MOKWjryGplAchBhXyVLQdGHfJNvgWkpj6p /cR8yhO2pt3RvM8X98mGu81aTO1bGIEZrxRX40BkUV/JBPyRMMcSU6fOQp+VuDxxdwJ5 wWgw== X-Gm-Message-State: AOJu0YxWnIO393GSBk6FPO/6szUL+dvMP64EBbF1ounRvEuspw9kooYE wNcxwn+IljAC1MaqPnS1s1ssXNGM7FycCh9Zltg= X-Received: by 2002:a17:90a:b395:b0:26f:6a0c:8210 with SMTP id e21-20020a17090ab39500b0026f6a0c8210mr21014529pjr.41.1693248959965; Mon, 28 Aug 2023 11:55:59 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.55.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:55:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 08/10] accel/tcg: Merge io_writex into do_st_mmio_leN Date: Mon, 28 Aug 2023 11:55:48 -0700 Message-Id: <20230828185550.573653-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Avoid multiple calls to io_prepare for unaligned acceses. One call to do_st_mmio_leN will never cross pages. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 82 +++++++++++++++++----------------------------- 1 file changed, 30 insertions(+), 52 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index f4772b5c5d..cf80add4b3 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1385,30 +1385,6 @@ static void io_failed(CPUArchState *env, CPUTLBEntryFull *full, vaddr addr, } } -static void io_writex(CPUArchState *env, CPUTLBEntryFull *full, - int mmu_idx, uint64_t val, vaddr addr, - uintptr_t retaddr, MemOp op) -{ - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxResult r; - - section = io_prepare(&mr_offset, env, full->xlat_section, - full->attrs, addr, retaddr); - mr = section->mr; - - { - QEMU_IOTHREAD_LOCK_GUARD(); - r = memory_region_dispatch_write(mr, mr_offset, val, op, full->attrs); - } - - if (r != MEMTX_OK) { - io_failed(env, full, addr, memop_size(op), MMU_DATA_STORE, mmu_idx, - r, retaddr); - } -} - /* Return true if ADDR is present in the victim tlb, and has been copied back to the main tlb. */ static bool victim_tlb_hit(CPUArchState *env, size_t mmu_idx, size_t index, @@ -2679,39 +2655,41 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, uint64_t val_le, vaddr addr, int size, int mmu_idx, uintptr_t ra) { + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; + tcg_debug_assert(size > 0 && size <= 8); + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + do { + MemOp this_mop; + unsigned this_size; + MemTxResult r; + /* Store aligned pieces up to 8 bytes. */ - switch ((size | (int)addr) & 7) { - case 1: - case 3: - case 5: - case 7: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_UB); - val_le >>= 8; - size -= 1; - addr += 1; - break; - case 2: - case 6: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUW); - val_le >>= 16; - size -= 2; - addr += 2; - break; - case 4: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUL); - val_le >>= 32; - size -= 4; - addr += 4; - break; - case 0: - io_writex(env, full, mmu_idx, val_le, addr, ra, MO_LEUQ); - return 0; - default: - qemu_build_not_reached(); + this_mop = ctz32(size | (int)addr | 8); + this_size = 1 << this_mop; + this_mop |= MO_LE; + + r = memory_region_dispatch_write(mr, mr_offset, val_le, + this_mop, attrs); + if (unlikely(r != MEMTX_OK)) { + io_failed(env, full, addr, this_size, MMU_DATA_STORE, + mmu_idx, r, ra); } + if (this_size == 8) { + return 0; + } + + val_le >>= this_size * 8; + addr += this_size; + mr_offset += this_size; + size -= this_size; } while (size); return val_le; From patchwork Mon Aug 28 18:55:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717741 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347365wru; Mon, 28 Aug 2023 11:57:04 -0700 (PDT) X-Google-Smtp-Source: AGHT+IECyBOP9QVI3Y3f5N3MM4bhK/U+tadOaF6qMzb3qaRwG2dgBQSAoW/aiFZq26W1W0KTMo16 X-Received: by 2002:a67:f8d3:0:b0:44d:4c28:55ca with SMTP id c19-20020a67f8d3000000b0044d4c2855camr21112803vsp.16.1693249024246; Mon, 28 Aug 2023 11:57:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249024; cv=none; d=google.com; s=arc-20160816; b=DIbihsHF/mCO/4QTG2UShw6XCYynvbV+OROKwWsXZLwH2aKK9ndh80C9lxyHuUyySZ JRZtvvwTAUNOJ34ze082NYekeoA92iWyxGj35XIYBgP60UWjdwvhF7owz5ukXXW6Yu5z sh4x5ThvDC5RPcgE9DEO8zyB5fIGEk3vke3RMDKLqx+VEHWgAdJPGAp60z3QdXvPhJfq +P5ZNSiEOkkM3OWiVzjXA5DcW0ednts2Eo6KwONkxPJM8Jj7hRLS3orh++k85iKWjN9v H++7iUi7VkifAvG8Agc+psA+CwM0JkrlXcyG/Ron9QhyBTNbOI4NcmZQ1LlAyUEY417W qifA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=cS4vlm8y6MqQ8ErQLX0t/96y+gcpjdiVv6UfrIIxZbo=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=KX2ByjB9xuuVT4z3jJ5OVFnyJgLXiRhz6sJKsI5VvqK5jYT0HkT8TmhFVHb18b0KmD D4i8SOkRjy1NtPqbAeUG/4Qcvt3uAB0wCsqUbIjwVZkSPxodrlMFUknfjvfy3Azmtulh dVknqXawzO9LA/42Urwelbn90zT5vtH4XuQjg86G4mNJc/33gRs8BMkd66QCdBgIQp2D TWN608ShLEeXwsHNgvNihqV3k7ywf4HowcgM/TUvVjnqn7oARVB/h1CVPBcksGjyIBS9 q9qAmo9+tXfhhwQj3IlX8+8jzVzXj8uFPLHmQkiwe89Usn2FrCY0L8tcu9KYiRWjSgbQ 9UUw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EKjbNHz4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z19-20020a0cf253000000b0064ca647d174si5183085qvl.88.2023.08.28.11.57.04 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:04 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EKjbNHz4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPC-0000sc-2c; Mon, 28 Aug 2023 14:56:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahPA-0000rb-J9 for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:04 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP8-0002Sn-1z for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:04 -0400 Received: by mail-pj1-x102f.google.com with SMTP id 98e67ed59e1d1-26d50a832a9so1857731a91.3 for ; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248961; x=1693853761; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cS4vlm8y6MqQ8ErQLX0t/96y+gcpjdiVv6UfrIIxZbo=; b=EKjbNHz4SXq67JnJFEotkal0Y0+t1ZmKBS6vctZQ/TqHPTRP/MLDQW+jW/xCgqM9zs 0yYOQjmO/j9nKf479bbUWaqfSVcQVL63KD8nd9dEBGaWPjnhc7arqpasVxmee87ypz4f mdnPnzSO+bX1JvPvPxJxItmzNu3v/gDeLJFsEfhzXbz2r73mNDYg4Kbaoij0Fd3htMI3 1xRpkzjknjPsWEUiDGODr2thB2Q27MMr7wyKma2W5cfRs0Lm8CKyWFpDVevmYFTZ3vS4 e0QTVvqMW4Z6Vw66C15olGS0lUQD94VXSVlJcCBs5PHKkTHj6R/p0+HxHw10NNBgEoi6 KK9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248961; x=1693853761; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cS4vlm8y6MqQ8ErQLX0t/96y+gcpjdiVv6UfrIIxZbo=; b=Vn+RcMfyBDCvqwh4PVstCXGwUgbMEOfO5AUoS1JYVHaXIkcButLES+rLBvf+oL6dU/ e5qvPpFc6BY2GovoGH+wyBTvyD9khid5s8gNHvvVeaGpvYRxB5Gcns6OlaOMzbDgOlYq mJfgTVnI5fAcuRT6vOrI98r2U1L9xJzlI6Qm8xFlzAB+cvW2qBujBf5DYLAj5fAtEopO kz5KnRUvtLVvaTllDzHenrSXKS7M7WfuVQL4pGF+EHIvbarGwAxDpX9eVZ0LiXF4ZNNp KvyYD8gxgrwzuizU+Ul17WX6T/a8l6+6bakyBrFiMmVp3YN/Z6KJT4GZWr/0evT0k4VT 11wg== X-Gm-Message-State: AOJu0YzvmfHkZeuYvDKYfF7rAj42WwAxq1l0z2e/XNfhPwko4UcdZIEY mStrO1SR6+XQ2tw+DOCe9U7hI8VhKI0hRig95oo= X-Received: by 2002:a17:90b:3110:b0:26b:36dc:2f08 with SMTP id gc16-20020a17090b311000b0026b36dc2f08mr20714299pjb.46.1693248960851; Mon, 28 Aug 2023 11:56:00 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.56.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:56:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 09/10] accel/tcg: Introduce do_ld16_mmio_beN Date: Mon, 28 Aug 2023 11:55:49 -0700 Message-Id: <20230828185550.573653-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Split out int_ld_mmio_beN, to be used by both do_ld_mmio_beN and do_ld16_mmio_beN. Move the locks down into the two functions, since each one now covers all accesses to once page. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 91 ++++++++++++++++++++++++++++++---------------- 1 file changed, 59 insertions(+), 32 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index cf80add4b3..8a7e386415 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2005,21 +2005,11 @@ static void *atomic_mmu_lookup(CPUArchState *env, vaddr addr, MemOpIdx oi, * Load @size bytes from @addr, which is memory-mapped i/o. * The bytes are concatenated in big-endian order with @ret_be. */ -static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, - uint64_t ret_be, vaddr addr, int size, - int mmu_idx, MMUAccessType type, uintptr_t ra) +static uint64_t int_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, MMUAccessType type, uintptr_t ra, + MemoryRegion *mr, hwaddr mr_offset) { - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxAttrs attrs; - - tcg_debug_assert(size > 0 && size <= 8); - - attrs = full->attrs; - section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); - mr = section->mr; - do { MemOp this_mop; unsigned this_size; @@ -2031,7 +2021,8 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, this_size = 1 << this_mop; this_mop |= MO_BE; - r = memory_region_dispatch_read(mr, mr_offset, &val, this_mop, attrs); + r = memory_region_dispatch_read(mr, mr_offset, &val, + this_mop, full->attrs); if (unlikely(r != MEMTX_OK)) { io_failed(env, full, addr, this_size, type, mmu_idx, r, ra); } @@ -2048,6 +2039,56 @@ static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, return ret_be; } +static uint64_t do_ld_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, MMUAccessType type, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + ret = int_ld_mmio_beN(env, full, ret_be, addr, size, mmu_idx, + type, ra, mr, mr_offset); + qemu_mutex_unlock_iothread(); + + return ret; +} + +static Int128 do_ld16_mmio_beN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t ret_be, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t a, b; + + tcg_debug_assert(size > 8 && size <= 16); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + a = int_ld_mmio_beN(env, full, ret_be, addr, size - 8, mmu_idx, + MMU_DATA_LOAD, ra, mr, mr_offset); + b = int_ld_mmio_beN(env, full, ret_be, addr + size - 8, 8, mmu_idx, + MMU_DATA_LOAD, ra, mr, mr_offset + size - 8); + qemu_mutex_unlock_iothread(); + + return int128_make128(b, a); +} + /** * do_ld_bytes_beN * @p: translation parameters @@ -2190,7 +2231,6 @@ static uint64_t do_ld_beN(CPUArchState *env, MMULookupPageData *p, unsigned tmp, half_size; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_ld_mmio_beN(env, p->full, ret_be, p->addr, p->size, mmu_idx, type, ra); } @@ -2241,12 +2281,7 @@ static Int128 do_ld16_beN(CPUArchState *env, MMULookupPageData *p, MemOp atom; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - a = do_ld_mmio_beN(env, p->full, a, p->addr, size - 8, - mmu_idx, MMU_DATA_LOAD, ra); - b = do_ld_mmio_beN(env, p->full, 0, p->addr + 8, 8, - mmu_idx, MMU_DATA_LOAD, ra); - return int128_make128(b, a); + return do_ld16_mmio_beN(env, p->full, a, p->addr, size, mmu_idx, ra); } /* @@ -2291,7 +2326,6 @@ static uint8_t do_ld_1(CPUArchState *env, MMULookupPageData *p, int mmu_idx, MMUAccessType type, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_ld_mmio_beN(env, p->full, 0, p->addr, 1, mmu_idx, type, ra); } else { return *(uint8_t *)p->haddr; @@ -2304,7 +2338,6 @@ static uint16_t do_ld_2(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint16_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 2, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap16(ret); @@ -2325,7 +2358,6 @@ static uint32_t do_ld_4(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint32_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 4, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap32(ret); @@ -2346,7 +2378,6 @@ static uint64_t do_ld_8(CPUArchState *env, MMULookupPageData *p, int mmu_idx, uint64_t ret; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); ret = do_ld_mmio_beN(env, p->full, 0, p->addr, 8, mmu_idx, type, ra); if ((memop & MO_BSWAP) == MO_LE) { ret = bswap64(ret); @@ -2505,12 +2536,8 @@ static Int128 do_ld16_mmu(CPUArchState *env, vaddr addr, crosspage = mmu_lookup(env, addr, oi, ra, MMU_DATA_LOAD, &l); if (likely(!crosspage)) { if (unlikely(l.page[0].flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - a = do_ld_mmio_beN(env, l.page[0].full, 0, addr, 8, - l.mmu_idx, MMU_DATA_LOAD, ra); - b = do_ld_mmio_beN(env, l.page[0].full, 0, addr + 8, 8, - l.mmu_idx, MMU_DATA_LOAD, ra); - ret = int128_make128(b, a); + ret = do_ld16_mmio_beN(env, l.page[0].full, 0, addr, 16, + l.mmu_idx, ra); if ((l.memop & MO_BSWAP) == MO_LE) { ret = bswap128(ret); } From patchwork Mon Aug 28 18:55:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 717742 Delivered-To: patch@linaro.org Received: by 2002:a5d:688a:0:b0:317:ecd7:513f with SMTP id h10csp1347427wru; Mon, 28 Aug 2023 11:57:14 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEapWwwUgJWm4GIAkYYcnDyj3qylBDQRmP0YAv0UgWGwaf+3nIPHgUs/BNgX/ZrvSao9EuF X-Received: by 2002:a05:622a:1041:b0:40e:13e5:9663 with SMTP id f1-20020a05622a104100b0040e13e59663mr29421074qte.47.1693249033920; Mon, 28 Aug 2023 11:57:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1693249033; cv=none; d=google.com; s=arc-20160816; b=RQnwb0qQEiN5sfRqqM9hVJdILhef+Inc6WzL6xDuO4AVmHXvY3enS1CT3lkQ3hYRt9 Rb/+GYaU7AXrHINi9rYKgafrIedeE62A4y+S8wLlyIt9UBnYR/OVLRr3cwcd0FS3ccIT SQnTIPyJBp0KSc1npILBajGTAvGNrFQqZ3m3JBTlgEH5ArwwvuEwmIbrzMbeqzz/UXOi msohkQJrPsYztX0+GG4xMlWiQnl302NaMBKUyffVGLW96THWdhJGEkBavNa7+LdbU8oH ggLsc+4YAdndxVoiGbA3oj6MNSdUY7lqbf/h+anbNcVZ3IXveXPrBblTZ0gSZj4TtGLA xOBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=OOzEj3zxNXy+t9npCtUx1T/2sNWvJU9XlZEuj/GyRA4=; fh=Vvliz9ENSk/yCY42nG87aL5dW+Zo0lduSx74U3lH2V8=; b=fy96r/mzqRvx8zAx1ZAMMlZS6eEyCxvim1vSpGpyJFC699PGBseF8HEQo+BCi2ogQU h067S5xnaOp68mTNfMta6yodHiZXCYpkv+Rq7yK6KWwWN7EC3DBMIGE3kSaBVjVr/Oe3 r2OEbefuxnHneJRM+qAbQyI59gWdjABtPRKgtSNVEI2zrqatSbfgqvSd6eRY2Z/6s5Vp AhcmVslUFv6xI4vOmr6ZSD7O/rI+KRIcA/0z2Q46g4jNtBG15jMuKYRdp6+IJ7VBZmMP dXIBZjhL26DvMUY3j+0djkX7BrEnZD3z7vmQ8zR0PGze+HhWwmzor4lEMZa9sAI/VPEr zvlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lMgrsc0O; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z17-20020ac87f91000000b0040697512e99si3147430qtj.49.2023.08.28.11.57.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Aug 2023 11:57:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lMgrsc0O; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qahPC-0000sl-Ug; Mon, 28 Aug 2023 14:56:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qahPB-0000sG-Nk for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:05 -0400 Received: from mail-pj1-x102b.google.com ([2607:f8b0:4864:20::102b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qahP9-0002Sv-8d for qemu-devel@nongnu.org; Mon, 28 Aug 2023 14:56:05 -0400 Received: by mail-pj1-x102b.google.com with SMTP id 98e67ed59e1d1-26f6b2c8e80so2265934a91.1 for ; Mon, 28 Aug 2023 11:56:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1693248962; x=1693853762; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OOzEj3zxNXy+t9npCtUx1T/2sNWvJU9XlZEuj/GyRA4=; b=lMgrsc0OYG2gn7Jt4IJ6jHnLSC4cB+MpYs4BDa3LAlXfDCT7ZiJ1q3UQ921seQJXgp dem+w35mkzct0tDALgYZGZE9uZRP8jqR7pWveiehiJx+Wv8/PYWYJvnNmmPQuiwK915+ 4u6Sr2615UDsua+RjMoOrzaoHXRwBeZFPPVQk6bMWnalq+MvfTBZ21KbKaPilF8SUi1Y akg+OR7iWQwxCEj/4Sufy/PvTZxMoeax25hpZIgvBAGK7EJIDgsMhASKX3imzwV97T+i 8MWi75xZ84CLFlpNy/fPX5MLjuRMHAX1gY1VcBrKvraBrQKEGa8rxGJ90fXirXXOJedF s8bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693248962; x=1693853762; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OOzEj3zxNXy+t9npCtUx1T/2sNWvJU9XlZEuj/GyRA4=; b=D7heM06TF5N6SNlhO5T0SZcHqTjagdAgZFQ7nUvJXOUE9QDCBcxl/MHbkJG91qbfVL 8AYb6IgTQUMxAA81uYWqHDnFzV4Fk9Rt5sydhW+f3v2+pDYxubNA63fUJn2FrVh3fQ/6 umP4i2Sqkmhtla5NfD4OmdlP7BU/2mYRJbFwc/wB7GdD2l6WsseyOAc+/N+s8QqnX8A/ uPe8Ndi+ibPArfgC5zehimvILyeFwUVkBRUhJmcFXrevLdWBfX4r172afCOXa3VpDMbl JQV/xCzXM9wk+bkLZtD2GiZz4TVhbZBdG93PqxfN8csghGTu1tWFBW1/FPuJ4N5t4xfe UNDw== X-Gm-Message-State: AOJu0YySZE0WUEx10nq9K2h8RqjKQ/+BcSiCtHVxYbrx1RDA1KD6aR/6 LMXlorxaG/ErhSRcQ7fsFtnP80SxkMbpHYQYcbs= X-Received: by 2002:a17:90b:1a89:b0:268:ca76:64a with SMTP id ng9-20020a17090b1a8900b00268ca76064amr21315364pjb.49.1693248961827; Mon, 28 Aug 2023 11:56:01 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id 4-20020a17090a190400b002680dfd368dsm7932835pjg.51.2023.08.28.11.56.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 28 Aug 2023 11:56:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: alex.bennee@linaro.org Subject: [PATCH 10/10] accel/tcg: Introduce do_st16_mmio_leN Date: Mon, 28 Aug 2023 11:55:50 -0700 Message-Id: <20230828185550.573653-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230828185550.573653-1-richard.henderson@linaro.org> References: <20230828185550.573653-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102b; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Split out int_st_mmio_leN, to be used by both do_st_mmio_leN and do_st16_mmio_leN. Move the locks down into the two functions, since each one now covers all accesses to once page. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 88 ++++++++++++++++++++++++++++++---------------- 1 file changed, 58 insertions(+), 30 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 8a7e386415..ff253e78d2 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2678,21 +2678,11 @@ Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, * The bytes to store are extracted in little-endian order from @val_le; * return the bytes of @val_le beyond @p->size that have not been stored. */ -static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, - uint64_t val_le, vaddr addr, int size, - int mmu_idx, uintptr_t ra) +static uint64_t int_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra, + MemoryRegion *mr, hwaddr mr_offset) { - MemoryRegionSection *section; - hwaddr mr_offset; - MemoryRegion *mr; - MemTxAttrs attrs; - - tcg_debug_assert(size > 0 && size <= 8); - - attrs = full->attrs; - section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); - mr = section->mr; - do { MemOp this_mop; unsigned this_size; @@ -2704,7 +2694,7 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, this_mop |= MO_LE; r = memory_region_dispatch_write(mr, mr_offset, val_le, - this_mop, attrs); + this_mop, full->attrs); if (unlikely(r != MEMTX_OK)) { io_failed(env, full, addr, this_size, MMU_DATA_STORE, mmu_idx, r, ra); @@ -2722,6 +2712,56 @@ static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, return val_le; } +static uint64_t do_st_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + uint64_t val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + hwaddr mr_offset; + MemoryRegion *mr; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 0 && size <= 8); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + ret = int_st_mmio_leN(env, full, val_le, addr, size, mmu_idx, + ra, mr, mr_offset); + qemu_mutex_unlock_iothread(); + + return ret; +} + +static uint64_t do_st16_mmio_leN(CPUArchState *env, CPUTLBEntryFull *full, + Int128 val_le, vaddr addr, int size, + int mmu_idx, uintptr_t ra) +{ + MemoryRegionSection *section; + MemoryRegion *mr; + hwaddr mr_offset; + MemTxAttrs attrs; + uint64_t ret; + + tcg_debug_assert(size > 8 && size <= 16); + + attrs = full->attrs; + section = io_prepare(&mr_offset, env, full->xlat_section, attrs, addr, ra); + mr = section->mr; + + qemu_mutex_lock_iothread(); + int_st_mmio_leN(env, full, int128_getlo(val_le), addr, 8, + mmu_idx, ra, mr, mr_offset); + ret = int_st_mmio_leN(env, full, int128_gethi(val_le), addr + 8, + size - 8, mmu_idx, ra, mr, mr_offset + 8); + qemu_mutex_unlock_iothread(); + + return ret; +} + /* * Wrapper for the above. */ @@ -2733,7 +2773,6 @@ static uint64_t do_st_leN(CPUArchState *env, MMULookupPageData *p, unsigned tmp, half_size; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); return do_st_mmio_leN(env, p->full, val_le, p->addr, p->size, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { @@ -2788,11 +2827,8 @@ static uint64_t do_st16_leN(CPUArchState *env, MMULookupPageData *p, MemOp atom; if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); - do_st_mmio_leN(env, p->full, int128_getlo(val_le), - p->addr, 8, mmu_idx, ra); - return do_st_mmio_leN(env, p->full, int128_gethi(val_le), - p->addr + 8, size - 8, mmu_idx, ra); + return do_st16_mmio_leN(env, p->full, val_le, p->addr, + size, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { return int128_gethi(val_le) >> ((size - 8) * 8); } @@ -2836,7 +2872,6 @@ static void do_st_1(CPUArchState *env, MMULookupPageData *p, uint8_t val, int mmu_idx, uintptr_t ra) { if (unlikely(p->flags & TLB_MMIO)) { - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 1, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2852,7 +2887,6 @@ static void do_st_2(CPUArchState *env, MMULookupPageData *p, uint16_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap16(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 2, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2872,7 +2906,6 @@ static void do_st_4(CPUArchState *env, MMULookupPageData *p, uint32_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap32(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 4, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -2892,7 +2925,6 @@ static void do_st_8(CPUArchState *env, MMULookupPageData *p, uint64_t val, if ((memop & MO_BSWAP) != MO_LE) { val = bswap64(val); } - QEMU_IOTHREAD_LOCK_GUARD(); do_st_mmio_leN(env, p->full, val, p->addr, 8, mmu_idx, ra); } else if (unlikely(p->flags & TLB_DISCARD_WRITE)) { /* nothing */ @@ -3020,11 +3052,7 @@ static void do_st16_mmu(CPUArchState *env, vaddr addr, Int128 val, if ((l.memop & MO_BSWAP) != MO_LE) { val = bswap128(val); } - a = int128_getlo(val); - b = int128_gethi(val); - QEMU_IOTHREAD_LOCK_GUARD(); - do_st_mmio_leN(env, l.page[0].full, a, addr, 8, l.mmu_idx, ra); - do_st_mmio_leN(env, l.page[0].full, b, addr + 8, 8, l.mmu_idx, ra); + do_st16_mmio_leN(env, l.page[0].full, val, addr, 16, l.mmu_idx, ra); } else if (unlikely(l.page[0].flags & TLB_DISCARD_WRITE)) { /* nothing */ } else {