From patchwork Fri Nov 23 14:45:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 151882 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp2211934ljp; Fri, 23 Nov 2018 06:50:37 -0800 (PST) X-Google-Smtp-Source: AFSGD/XjPfoMWIcjQvsE0cCaz2jhgujaKHHM4j+1RTob9esGpUM8oJouJ0odGF5jtVJtYSwjy4vs X-Received: by 2002:a81:c50d:: with SMTP id k13mr15344219ywi.42.1542984637654; Fri, 23 Nov 2018 06:50:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542984637; cv=none; d=google.com; s=arc-20160816; b=UuQC1J5XUYZ2SGV55ORdota6N+vhfua8nxW5vh7bWuz1kAzYtJHscKEgWdIkJrU+pH vonmkI95pPlN4+J4LKDMOUCCwkxaGmYRgVK7wPyrqMMX+UA6YxI/XXigfIzD1btIFqei KDSGDRX5hvKXUN0aBGOHT41FwOaqgo0wVSquUHPwgfVl8a9Cuq3yKbmp5vM8RO2woc7j nwJc22B2UOJ7Ebvg+Yidy7NnGhqGlveKycosFL2/jzvpFjMdJ850NMZQ9ZF2E/EzmEzR o8rVIxhxAcsupw3jxrMTS76IkiwEhii9keFJ4zLynNG/zYek8wD+D9ig5uL7Rzz1LNeh GMLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=SISXkLKbIiz24VL0MBfPmZUjp9/oTXXI5t5wyAowjGh3U71GkO9S/EeuU79NTtZayU KXGMJebauS+TPuxd+YKsLP7J6ZRt07OW+MYBfJqVAaR4O72lMgdvrDJWQJbJbnbJtabb rBQdOTToNGWcqYMCTLsb41caei0mTs73OV0PRM8D1OlvnJ0ZSnINXQXbC/Ba74CxyODW 7JaCCsfInmNlOYg5t8axoEa/WTGWZv50Sa5YnTCVIEwb1V3eMIRSFIErc/PbgP9IoJmp ncU8oRkGYb0VsUyir2uJBhR2kJEBGk3FPR7jdFDQcUCYAi4Mp6u930GhaV0yrxDXf+r5 SAgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FlGMdLNP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id v65si19975500ywf.79.2018.11.23.06.50.37 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 23 Nov 2018 06:50:37 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=FlGMdLNP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52774 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gQCmy-0004NQ-Oz for patch@linaro.org; Fri, 23 Nov 2018 09:50:36 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:43985) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gQCio-00085Q-42 for qemu-devel@nongnu.org; Fri, 23 Nov 2018 09:46:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gQCim-0003L7-0b for qemu-devel@nongnu.org; Fri, 23 Nov 2018 09:46:18 -0500 Received: from mail-wm1-x341.google.com ([2a00:1450:4864:20::341]:55872) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gQCil-0003JF-M5 for qemu-devel@nongnu.org; Fri, 23 Nov 2018 09:46:15 -0500 Received: by mail-wm1-x341.google.com with SMTP id y139so12074837wmc.5 for ; Fri, 23 Nov 2018 06:46:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=FlGMdLNPO95E0P+uMYKw323F+PHb4hxhD2IsOtPI0rtna6KxRZcROdYXrhwG2TsWfw 5ghRcxSXdGYyqynzFqnr3IxU01N9kGcvVgnGvu2ocn7RHzZKZ7prUuCQeypryzcQ35Zj 7PpYuuwWPAh0nAwYEdJZmc1e514Se28xSDIeQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=XXIoOe/cezZhOReKPwYJdYOpc7ojwFAWBJIgpj9iROvv8uS5j1kQwKjFBPEdONyEry tdpxA9C38ZJS4+fcpGzU/AChb7Oia9zF2xeV9M92pChFeWUDzt1wKp8PuW5wYP9y1pSl Rzhu9kYhdlhZ3QBmskfwBA1PAecPLHFHnes2EkHZuAOyr3wAHFL5ghA0BzA3F2sbJt/B RVP+I1VQTDfcuVa+1gBCWWMn/gFObZ0/7hFaDm9R5yTEfzAmKkUQCr47PYqORfNbrdO1 BxWLNmkW5KUA3KlBdy23dOEq4mUo+/L/KQ532pDS20FWcW2gjgv/Wtr8z/kw9cz6EzBh 1iPQ== X-Gm-Message-State: AGRZ1gKODgLq5j1jRJZH6eMLcuzm3g9UqiskXLK+ZYMIW7B11PH+5zwe oPdHzVqt9+JN3MdsArIbwdkwyYj560toZQ== X-Received: by 2002:a1c:a13:: with SMTP id 19mr14069153wmk.81.1542984374230; Fri, 23 Nov 2018 06:46:14 -0800 (PST) Received: from cloudburst.twiddle.net ([195.77.246.50]) by smtp.gmail.com with ESMTPSA id p74sm10339630wmd.29.2018.11.23.06.46.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 23 Nov 2018 06:46:13 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 23 Nov 2018 15:45:35 +0100 Message-Id: <20181123144558.5048-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181123144558.5048-1-richard.henderson@linaro.org> References: <20181123144558.5048-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::341 Subject: [Qemu-devel] [PATCH for-4.0 v2 14/37] tcg/aarch64: Use TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alistair.Francis@wdc.com Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 191 +++++++++++++++++------------------ 2 files changed, 93 insertions(+), 100 deletions(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 9aea1d1771..d1bd77c41d 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -146,7 +146,7 @@ static inline void flush_icache_range(uintptr_t start, uintptr_t stop) void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t); #ifdef CONFIG_SOFTMMU -#define TCG_TARGET_NEED_LDST_LABELS +#define TCG_TARGET_NEED_LDST_OOL_LABELS #endif #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 403f5caf14..8edea527f7 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -145,18 +145,6 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, ct->ct |= TCG_CT_REG; ct->u.regs |= 0xffffffff00000000ull; break; - case 'l': /* qemu_ld / qemu_st address, data_reg */ - ct->ct |= TCG_CT_REG; - ct->u.regs = 0xffffffffu; -#ifdef CONFIG_SOFTMMU - /* x0 and x1 will be overwritten when reading the tlb entry, - and x2, and x3 for helper args, better to avoid using them. */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X0); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X1); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X2); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X3); -#endif - break; case 'A': /* Valid for arithmetic immediate (positive or negative). */ ct->ct |= TCG_CT_CONST_AIMM; break; @@ -1378,7 +1366,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, } #ifdef CONFIG_SOFTMMU -#include "tcg-ldst.inc.c" +#include "tcg-ldst-ool.inc.c" /* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, * TCGMemOpIdx oi, uintptr_t ra) @@ -1391,6 +1379,12 @@ static void * const qemu_ld_helpers[16] = { [MO_BEUW] = helper_be_lduw_mmu, [MO_BEUL] = helper_be_ldul_mmu, [MO_BEQ] = helper_be_ldq_mmu, + + [MO_SB] = helper_ret_ldsb_mmu, + [MO_LESW] = helper_le_ldsw_mmu, + [MO_LESL] = helper_le_ldsl_mmu, + [MO_BESW] = helper_be_ldsw_mmu, + [MO_BESL] = helper_be_ldsl_mmu, }; /* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, @@ -1407,67 +1401,6 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; -static inline void tcg_out_adr(TCGContext *s, TCGReg rd, void *target) -{ - ptrdiff_t offset = tcg_pcrel_diff(s, target); - tcg_debug_assert(offset == sextract64(offset, 0, 21)); - tcg_out_insn(s, 3406, ADR, rd, offset); -} - -static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X2, oi); - tcg_out_adr(s, TCG_REG_X3, lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); - if (opc & MO_SIGN) { - tcg_out_sxt(s, lb->type, size, lb->datalo_reg, TCG_REG_X0); - } else { - tcg_out_mov(s, size == MO_64, lb->datalo_reg, TCG_REG_X0); - } - - tcg_out_goto(s, lb->raddr); -} - -static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_mov(s, size == MO_64, TCG_REG_X2, lb->datalo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X3, oi); - tcg_out_adr(s, TCG_REG_X4, lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); - tcg_out_goto(s, lb->raddr); -} - -static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, - TCGType ext, TCGReg data_reg, TCGReg addr_reg, - tcg_insn_unit *raddr, tcg_insn_unit *label_ptr) -{ - TCGLabelQemuLdst *label = new_ldst_label(s); - - label->is_ld = is_ld; - label->oi = oi; - label->type = ext; - label->datalo_reg = data_reg; - label->addrlo_reg = addr_reg; - label->raddr = raddr; - label->label_ptr[0] = label_ptr; -} - /* * Load and compare a TLB entry, emitting the conditional jump to the * slow path on failure. Returns the register for the host addend. @@ -1644,19 +1577,22 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi, TCGType ext) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_ld_direct(s, memop, ext, data_reg, - base, otype, addr_reg); - add_qemu_ldst_label(s, true, oi, ext, data_reg, addr_reg, - s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + /* Ignore the requested "ext". We get the same correct result from + * a 16-bit sign-extended to 64-bit as we do sign-extended to 32-bit, + * and we create fewer out-of-line thunks. + */ + bool is_64 = (memop & MO_SIGN) || ((memop & MO_SIZE) == MO_64); + + tcg_debug_assert(data_reg == TCG_REG_X0); + tcg_debug_assert(addr_reg == TCG_REG_X1); + + add_ldst_ool_label(s, true, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_ld_direct(s, memop, ext, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1671,18 +1607,18 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); - add_qemu_ldst_label(s, false, oi, (memop & MO_SIZE)== MO_64, - data_reg, addr_reg, s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + bool is_64 = (memop & MO_SIZE) == MO_64; + + tcg_debug_assert(addr_reg == TCG_REG_X1); + tcg_debug_assert(data_reg == TCG_REG_X2); + + add_ldst_ool_label(s, false, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_st_direct(s, memop, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1693,6 +1629,52 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, #endif /* CONFIG_SOFTMMU */ } +#ifdef CONFIG_SOFTMMU +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is_64, TCGMemOpIdx oi) +{ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + const TCGMemOp memop = get_memop(oi); + const unsigned mem_index = get_mmuidx(oi); + const TCGReg addr_reg = TCG_REG_X1; + const TCGReg data_reg = is_ld ? TCG_REG_X0 : TCG_REG_X2; + tcg_insn_unit * const thunk = s->code_ptr; + tcg_insn_unit *label; + TCGReg base, arg; + + base = tcg_out_tlb_read(s, addr_reg, memop, &label, mem_index, is_ld, + TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7); + + /* TLB Hit */ + if (is_ld) { + tcg_out_qemu_ld_direct(s, memop, is_64, data_reg, + base, otype, addr_reg); + } else { + tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); + } + tcg_out_insn(s, 3207, RET, TCG_REG_LR); + + /* TLB Miss */ + reloc_pc19(label, s->code_ptr); + + tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); + /* addr_reg and data_reg are already in place. */ + arg = is_ld ? TCG_REG_X2 : TCG_REG_X3; + tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); + tcg_out_mov(s, TCG_TYPE_PTR, arg++, TCG_REG_LR); + + if (is_ld) { + tcg_out_goto_long(s, qemu_ld_helpers[memop & (MO_BSWAP | MO_SSIZE)], + TCG_REG_X7); + } else { + tcg_out_goto_long(s, qemu_st_helpers[memop & (MO_BSWAP | MO_SIZE)], + TCG_REG_X7); + } + + return thunk; +} +#endif + static tcg_insn_unit *tb_ret_addr; static void tcg_out_op(TCGContext *s, TCGOpcode opc, @@ -2262,10 +2244,12 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) static const TCGTargetOpDef w_w = { .args_ct_str = { "w", "w" } }; static const TCGTargetOpDef w_r = { .args_ct_str = { "w", "r" } }; static const TCGTargetOpDef w_wr = { .args_ct_str = { "w", "wr" } }; - static const TCGTargetOpDef r_l = { .args_ct_str = { "r", "l" } }; static const TCGTargetOpDef r_rA = { .args_ct_str = { "r", "rA" } }; static const TCGTargetOpDef rZ_r = { .args_ct_str = { "rZ", "r" } }; - static const TCGTargetOpDef lZ_l = { .args_ct_str = { "lZ", "l" } }; +#ifdef CONFIG_SOFTMMU + static const TCGTargetOpDef a_b = { .args_ct_str = { "a", "b" } }; + static const TCGTargetOpDef c_b = { .args_ct_str = { "c", "b" } }; +#endif static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } }; static const TCGTargetOpDef w_w_w = { .args_ct_str = { "w", "w", "w" } }; static const TCGTargetOpDef w_w_wZ = { .args_ct_str = { "w", "w", "wZ" } }; @@ -2397,10 +2381,19 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_i32: case INDEX_op_qemu_ld_i64: - return &r_l; +#ifdef CONFIG_SOFTMMU + return &a_b; +#else + return &r_r; +#endif + case INDEX_op_qemu_st_i32: case INDEX_op_qemu_st_i64: - return &lZ_l; +#ifdef CONFIG_SOFTMMU + return &c_b; +#else + return &r_r; +#endif case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: