From patchwork Mon Nov 12 21:44:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 150886 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp3652526ljp; Mon, 12 Nov 2018 13:58:49 -0800 (PST) X-Google-Smtp-Source: AJdET5cQu+UxD2WrPyu7V25rjw7dxfM330AM8qUxx9IqEv1jY9MMT2Dyfk2/gGalHsRYl9pNjJuP X-Received: by 2002:aed:35c5:: with SMTP id d5mr2595000qte.212.1542059928999; Mon, 12 Nov 2018 13:58:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542059928; cv=none; d=google.com; s=arc-20160816; b=hwK4nVrkI+qyfjIWcDJirXErF2aUbpgZ5RWJqIYWiFwZm7MsDVbbvOh4DxzD/qmJTI AgygkCCrdgZX0IJz7oLtTQ/ql+HAfnTj3PPMHdAFb2FRR88L2CuPfvAxwI1sUgFJtkUd 7+e0r3UmgIjrNaWSb6SCIvpg5roK7d19r5zJgBm4MwZVDSmqnH7K40gsf/y9PQsQuPIU GpR5sbjApfJugX0WpPEwgxNOpZtNB9tDWHlRm5Fb6AYQYfROsvXJ9yneKYuxpb0Em4ik BpP0XA7995sE2w3OdNL7MTofrq6zn7douCzRvWT4lrdvvVp5b1t8RlgpH90BBFjcbMsr 2Tpw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=QmLV0q16ujInb4IZH24vdEF6uC9PwtphTCsxZqNSagMFnBQwe7bAfcxIlNkDFkj7n+ lnkBLN0TqLqq8U5F0IQZt/d7bpmjksscVQ+/rs3zugFrSg5l3umVdBRlVVEBFLArc28n YODM2yVGjhA5YRKmDi8P8a0Sj23CYvpGarSvGbVHRawDZ1Be4Wbt15kSkyfN+rPfV2Zw 5YQZCMn532sIK8dLUQQJmHMJFCo1/8QHweuOHQOQ9WkurUx/0cRA8yLTYKQLv6k0FFQT jHrsJVB2E5xHJK59T9J8aOtreNeQi1PlNsFxi5dhmrJBmyBr99HobYZl5Cg1K6j3uzt8 fI8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=QhQeqh1D; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id q90si143171qvq.51.2018.11.12.13.58.48 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 12 Nov 2018 13:58:48 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=QhQeqh1D; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50862 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMKEK-0005ps-9c for patch@linaro.org; Mon, 12 Nov 2018 16:58:48 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:54243) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gMK3K-0003zA-Lv for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:29 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gMK3I-000531-Rp for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:26 -0500 Received: from mail-wr1-x441.google.com ([2a00:1450:4864:20::441]:33398) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gMK3G-00050p-JS for qemu-devel@nongnu.org; Mon, 12 Nov 2018 16:47:23 -0500 Received: by mail-wr1-x441.google.com with SMTP id u9-v6so11033081wrr.0 for ; Mon, 12 Nov 2018 13:47:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=QhQeqh1DTuKDkKbZiY12ONFneb9FqwUYrwh7Z7bZBBaaPgTuRLnWXzlMgzAD063jUb HoOjIXgh1hltdszIbrQ4B2C74rNzaupBRvC1npEON8Lwy1OdSg2SpVLt9JLwPWDGChPt jNO39cIeZbVSmx9TtzKvaURr0IHFHD5G+F7xM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UrR/IldFdFkWy6ywHmFI8AClyOk1jWWQ8mWiO+LW0Iw=; b=r6mlT+KoXaPBM5rLwBi8XiOndSnqVJ5u9ZryHCv6mJ02E88pt2gb2VTU5Xnr0VqiEv nK+dUdBT4Ifk8/eeeZLC7DF6O0cnWVru3JOQYyQduIdhGNhFrb63wn9IwujKwKX9TrUi VpHQWqtAxkuPdfxu2FpxxNFri6BNxG/P2NjMILCH3IWxBClnvrku63Kom0bsSvpt8scw RAKRZupyWThVVoqOf56uJxQUAyEO9BkdZhG0YWtQfR/JDUarOpfJH7vHMP0hyjly1+BM 1/5dWNg5HomTlagOtUppPymrJApSCVSTSmeBMMe8ApgxFsj0xeJoZckbc6JugBflSDVP TdOQ== X-Gm-Message-State: AGRZ1gI1vAugLYlnwNvYn29OmPS1HkR9KdXbRPRUZ8YCi98GnCcfnGtd baCFL10eKhS/pVig8EatpuJxULGoEFdhpg== X-Received: by 2002:adf:e983:: with SMTP id h3-v6mr2533904wrm.58.1542059239783; Mon, 12 Nov 2018 13:47:19 -0800 (PST) Received: from cloudburst.twiddle.net (178.red-213-99-154.dynamicip.rima-tde.net. [213.99.154.178]) by smtp.gmail.com with ESMTPSA id y123-v6sm3946907wme.38.2018.11.12.13.47.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 12 Nov 2018 13:47:19 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 12 Nov 2018 22:44:58 +0100 Message-Id: <20181112214503.22941-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20181112214503.22941-1-richard.henderson@linaro.org> References: <20181112214503.22941-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:4864:20::441 Subject: [Qemu-devel] [PATCH for-4.0 12/17] tcg/aarch64: Use TCG_TARGET_NEED_LDST_OOL_LABELS X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 191 +++++++++++++++++------------------ 2 files changed, 93 insertions(+), 100 deletions(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 9aea1d1771..d1bd77c41d 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -146,7 +146,7 @@ static inline void flush_icache_range(uintptr_t start, uintptr_t stop) void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t); #ifdef CONFIG_SOFTMMU -#define TCG_TARGET_NEED_LDST_LABELS +#define TCG_TARGET_NEED_LDST_OOL_LABELS #endif #define TCG_TARGET_NEED_POOL_LABELS diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 403f5caf14..8edea527f7 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -145,18 +145,6 @@ static const char *target_parse_constraint(TCGArgConstraint *ct, ct->ct |= TCG_CT_REG; ct->u.regs |= 0xffffffff00000000ull; break; - case 'l': /* qemu_ld / qemu_st address, data_reg */ - ct->ct |= TCG_CT_REG; - ct->u.regs = 0xffffffffu; -#ifdef CONFIG_SOFTMMU - /* x0 and x1 will be overwritten when reading the tlb entry, - and x2, and x3 for helper args, better to avoid using them. */ - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X0); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X1); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X2); - tcg_regset_reset_reg(ct->u.regs, TCG_REG_X3); -#endif - break; case 'A': /* Valid for arithmetic immediate (positive or negative). */ ct->ct |= TCG_CT_CONST_AIMM; break; @@ -1378,7 +1366,7 @@ static void tcg_out_cltz(TCGContext *s, TCGType ext, TCGReg d, } #ifdef CONFIG_SOFTMMU -#include "tcg-ldst.inc.c" +#include "tcg-ldst-ool.inc.c" /* helper signature: helper_ret_ld_mmu(CPUState *env, target_ulong addr, * TCGMemOpIdx oi, uintptr_t ra) @@ -1391,6 +1379,12 @@ static void * const qemu_ld_helpers[16] = { [MO_BEUW] = helper_be_lduw_mmu, [MO_BEUL] = helper_be_ldul_mmu, [MO_BEQ] = helper_be_ldq_mmu, + + [MO_SB] = helper_ret_ldsb_mmu, + [MO_LESW] = helper_le_ldsw_mmu, + [MO_LESL] = helper_le_ldsl_mmu, + [MO_BESW] = helper_be_ldsw_mmu, + [MO_BESL] = helper_be_ldsl_mmu, }; /* helper signature: helper_ret_st_mmu(CPUState *env, target_ulong addr, @@ -1407,67 +1401,6 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; -static inline void tcg_out_adr(TCGContext *s, TCGReg rd, void *target) -{ - ptrdiff_t offset = tcg_pcrel_diff(s, target); - tcg_debug_assert(offset == sextract64(offset, 0, 21)); - tcg_out_insn(s, 3406, ADR, rd, offset); -} - -static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X2, oi); - tcg_out_adr(s, TCG_REG_X3, lb->raddr); - tcg_out_call(s, qemu_ld_helpers[opc & (MO_BSWAP | MO_SIZE)]); - if (opc & MO_SIGN) { - tcg_out_sxt(s, lb->type, size, lb->datalo_reg, TCG_REG_X0); - } else { - tcg_out_mov(s, size == MO_64, lb->datalo_reg, TCG_REG_X0); - } - - tcg_out_goto(s, lb->raddr); -} - -static void tcg_out_qemu_st_slow_path(TCGContext *s, TCGLabelQemuLdst *lb) -{ - TCGMemOpIdx oi = lb->oi; - TCGMemOp opc = get_memop(oi); - TCGMemOp size = opc & MO_SIZE; - - reloc_pc19(lb->label_ptr[0], s->code_ptr); - - tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); - tcg_out_mov(s, TARGET_LONG_BITS == 64, TCG_REG_X1, lb->addrlo_reg); - tcg_out_mov(s, size == MO_64, TCG_REG_X2, lb->datalo_reg); - tcg_out_movi(s, TCG_TYPE_I32, TCG_REG_X3, oi); - tcg_out_adr(s, TCG_REG_X4, lb->raddr); - tcg_out_call(s, qemu_st_helpers[opc & (MO_BSWAP | MO_SIZE)]); - tcg_out_goto(s, lb->raddr); -} - -static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, - TCGType ext, TCGReg data_reg, TCGReg addr_reg, - tcg_insn_unit *raddr, tcg_insn_unit *label_ptr) -{ - TCGLabelQemuLdst *label = new_ldst_label(s); - - label->is_ld = is_ld; - label->oi = oi; - label->type = ext; - label->datalo_reg = data_reg; - label->addrlo_reg = addr_reg; - label->raddr = raddr; - label->label_ptr[0] = label_ptr; -} - /* * Load and compare a TLB entry, emitting the conditional jump to the * slow path on failure. Returns the register for the host addend. @@ -1644,19 +1577,22 @@ static void tcg_out_qemu_ld(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi, TCGType ext) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 1, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_ld_direct(s, memop, ext, data_reg, - base, otype, addr_reg); - add_qemu_ldst_label(s, true, oi, ext, data_reg, addr_reg, - s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + /* Ignore the requested "ext". We get the same correct result from + * a 16-bit sign-extended to 64-bit as we do sign-extended to 32-bit, + * and we create fewer out-of-line thunks. + */ + bool is_64 = (memop & MO_SIGN) || ((memop & MO_SIZE) == MO_64); + + tcg_debug_assert(data_reg == TCG_REG_X0); + tcg_debug_assert(addr_reg == TCG_REG_X1); + + add_ldst_ool_label(s, true, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_ld_direct(s, memop, ext, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1671,18 +1607,18 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, TCGMemOpIdx oi) { TCGMemOp memop = get_memop(oi); - const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; -#ifdef CONFIG_SOFTMMU - unsigned mem_index = get_mmuidx(oi); - tcg_insn_unit *label_ptr; - TCGReg base; - base = tcg_out_tlb_read(s, addr_reg, memop, &label_ptr, mem_index, 0, - TCG_REG_X0, TCG_REG_X1, TCG_REG_X2, TCG_REG_X3); - tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); - add_qemu_ldst_label(s, false, oi, (memop & MO_SIZE)== MO_64, - data_reg, addr_reg, s->code_ptr, label_ptr); +#ifdef CONFIG_SOFTMMU + bool is_64 = (memop & MO_SIZE) == MO_64; + + tcg_debug_assert(addr_reg == TCG_REG_X1); + tcg_debug_assert(data_reg == TCG_REG_X2); + + add_ldst_ool_label(s, false, is_64, oi, R_AARCH64_JUMP26, 0); + tcg_out_insn(s, 3206, BL, 0); #else /* !CONFIG_SOFTMMU */ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + if (USE_GUEST_BASE) { tcg_out_qemu_st_direct(s, memop, data_reg, TCG_REG_GUEST_BASE, otype, addr_reg); @@ -1693,6 +1629,52 @@ static void tcg_out_qemu_st(TCGContext *s, TCGReg data_reg, TCGReg addr_reg, #endif /* CONFIG_SOFTMMU */ } +#ifdef CONFIG_SOFTMMU +static tcg_insn_unit *tcg_out_qemu_ldst_ool(TCGContext *s, bool is_ld, + bool is_64, TCGMemOpIdx oi) +{ + const TCGType otype = TARGET_LONG_BITS == 64 ? TCG_TYPE_I64 : TCG_TYPE_I32; + const TCGMemOp memop = get_memop(oi); + const unsigned mem_index = get_mmuidx(oi); + const TCGReg addr_reg = TCG_REG_X1; + const TCGReg data_reg = is_ld ? TCG_REG_X0 : TCG_REG_X2; + tcg_insn_unit * const thunk = s->code_ptr; + tcg_insn_unit *label; + TCGReg base, arg; + + base = tcg_out_tlb_read(s, addr_reg, memop, &label, mem_index, is_ld, + TCG_REG_X4, TCG_REG_X5, TCG_REG_X6, TCG_REG_X7); + + /* TLB Hit */ + if (is_ld) { + tcg_out_qemu_ld_direct(s, memop, is_64, data_reg, + base, otype, addr_reg); + } else { + tcg_out_qemu_st_direct(s, memop, data_reg, base, otype, addr_reg); + } + tcg_out_insn(s, 3207, RET, TCG_REG_LR); + + /* TLB Miss */ + reloc_pc19(label, s->code_ptr); + + tcg_out_mov(s, TCG_TYPE_PTR, TCG_REG_X0, TCG_AREG0); + /* addr_reg and data_reg are already in place. */ + arg = is_ld ? TCG_REG_X2 : TCG_REG_X3; + tcg_out_movi(s, TCG_TYPE_I32, arg++, oi); + tcg_out_mov(s, TCG_TYPE_PTR, arg++, TCG_REG_LR); + + if (is_ld) { + tcg_out_goto_long(s, qemu_ld_helpers[memop & (MO_BSWAP | MO_SSIZE)], + TCG_REG_X7); + } else { + tcg_out_goto_long(s, qemu_st_helpers[memop & (MO_BSWAP | MO_SIZE)], + TCG_REG_X7); + } + + return thunk; +} +#endif + static tcg_insn_unit *tb_ret_addr; static void tcg_out_op(TCGContext *s, TCGOpcode opc, @@ -2262,10 +2244,12 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) static const TCGTargetOpDef w_w = { .args_ct_str = { "w", "w" } }; static const TCGTargetOpDef w_r = { .args_ct_str = { "w", "r" } }; static const TCGTargetOpDef w_wr = { .args_ct_str = { "w", "wr" } }; - static const TCGTargetOpDef r_l = { .args_ct_str = { "r", "l" } }; static const TCGTargetOpDef r_rA = { .args_ct_str = { "r", "rA" } }; static const TCGTargetOpDef rZ_r = { .args_ct_str = { "rZ", "r" } }; - static const TCGTargetOpDef lZ_l = { .args_ct_str = { "lZ", "l" } }; +#ifdef CONFIG_SOFTMMU + static const TCGTargetOpDef a_b = { .args_ct_str = { "a", "b" } }; + static const TCGTargetOpDef c_b = { .args_ct_str = { "c", "b" } }; +#endif static const TCGTargetOpDef r_r_r = { .args_ct_str = { "r", "r", "r" } }; static const TCGTargetOpDef w_w_w = { .args_ct_str = { "w", "w", "w" } }; static const TCGTargetOpDef w_w_wZ = { .args_ct_str = { "w", "w", "wZ" } }; @@ -2397,10 +2381,19 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_qemu_ld_i32: case INDEX_op_qemu_ld_i64: - return &r_l; +#ifdef CONFIG_SOFTMMU + return &a_b; +#else + return &r_r; +#endif + case INDEX_op_qemu_st_i32: case INDEX_op_qemu_st_i64: - return &lZ_l; +#ifdef CONFIG_SOFTMMU + return &c_b; +#else + return &r_r; +#endif case INDEX_op_deposit_i32: case INDEX_op_deposit_i64: