From patchwork Mon Jan 28 15:59:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156790 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3618315jaa; Mon, 28 Jan 2019 08:26:09 -0800 (PST) X-Google-Smtp-Source: ALg8bN4S/BwNoE1Kx+vJo5plZW3hWjzvzFhIrIGL9rO7UmS3kyQS8B8FppS3uJ6TcomUjo6ps7EK X-Received: by 2002:a5d:4e0b:: with SMTP id p11mr24466983wrt.227.1548692769692; Mon, 28 Jan 2019 08:26:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692769; cv=none; d=google.com; s=arc-20160816; b=YwuAud3K2gGIJTrVqNPUnohTHxflX7ctnXylp9r+S3Xq9d7u8u2JynNe+ql/4PO3Zx oMLt2VHcGDCXQ6YxqfPMaPDiyYntwxuGvF42TcfCx7yXgriXr/TEOQ3WaFdhrPG1BOSD 2uIT/E/9bIwO83SvqD9Cl3VDBKOvkXoQ5f/du5W6coDgv5ALeAwvQPkXS4jzb5JMy5Wz 7om9d3eQyzn3YrRX3/ejne4YfPr4UYBAt51+22/tNTHVXqoJq4+UYMAeFPyW+jMmHRjS ktG6c3pnmyHygvlCG34rjfRdPMNjazsVO2AzrYUQM8V6MVkFEYo4V8BwCJgt3C1okz5M SYGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=kY0UDeSFY7+trlsasRDaSGN5M2nMcaBMhIbok6UiKjh0pebRqbG/8HBSTnHsaVTwU0 g0uGZauVGjhV4yrOUxXhOOFD8ebmvzwXWFqjGfiWj+ccZNmMxHQtzQV1vYtRWqhhqtw6 abpWvNTEJ6I8O1NQnQpExrVkeTm4fDweC78es2h/mB/aHwBGudLLoIEcr6OPDl2ov3NG IDsdRo59aEZtk1wM7JbauqQ7yPsYqKdTRWI55+6Tpe8ZKLDO8O6fgwWUrGMTnZh9fi2e E7jO1z9Ii6zRmtIIQRWGpChWnOcnmcoXXqp6G5Y3J57BpdNi1ZA73rpgDMyCxD2LK4Ol 30mQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iD0lRo0N; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 128si47670013wmd.69.2019.01.28.08.26.09 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:26:09 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iD0lRo0N; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34559 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9jc-0000kw-FC for patch@linaro.org; Mon, 28 Jan 2019 11:26:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33678) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K8-0005FS-Dv for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vu-S1 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:48 -0500 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]:33948) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007qH-Kb for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pf1-x429.google.com with SMTP id h3so8199392pfg.1 for ; Mon, 28 Jan 2019 07:59:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=iD0lRo0NTGl8UGklwhXcEVKqgrfNVel0H+NI9dwnyQLWic4MSgzk+61vg29RpVFFHQ jgL5qTCWmTQbwF+ePP+hmGF1cD5BagfcBmO6bzVxVzP2ERMnK3xFVIXEtdYIR2ybJgNw +eDYa+/gmmEc2AWJPbYE+crqTaBMO5j56lNHU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=sTVGlLQaa/g9+WOqsixZQx0z4/STN+T3+b7eXZm0iNK0mJxWqhdn9dwDGFyKyZtPQV /7rCRlbFqxMZ+MVz9c5DtQ5WED0ARjnoAPnssXftwpNT3sTCGuFE63Tn7rEJdPb9krYw w4TiIwV51r//a/l6nYQ4cgbD3bVZjPW/lE5Wq8PbdG7E3A/bhcNiiRmiDpdDVW3aJNyy LZfuf2shbGgPEykcAJLbTcBo2TA7ewgsvtZtsIvKfnJFsVF4MvcvJyLYzRn4+b6gfgPC sGZHwzmnFA6CxiP0f+p/l/3kHuJ6FEe97EtfUyKdzfAR/JeBzuypJZTU+hJJE2sZVDcR YYDg== X-Gm-Message-State: AJcUukeGpAruSAR3tKOkIBoihZhNvSxFXQkYaF+AEAAS1M79EA6XWys9 OQsTbu6DJH6/UdvbM5ZB4Nh59D7xgXM= X-Received: by 2002:a63:2141:: with SMTP id s1mr20473994pgm.148.1548691175739; Mon, 28 Jan 2019 07:59:35 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:34 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:02 -0800 Message-Id: <20190128155907.20607-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::429 Subject: [Qemu-devel] [PULL 18/23] tcg/riscv: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Alistair Francis Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.h | 2 +- tcg/riscv/tcg-target.inc.c | 126 ++++++++++++++++--------------------- 2 files changed, 56 insertions(+), 72 deletions(-) -- 2.17.2 diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 1eb032626c..83b123ca03 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -33,7 +33,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/riscv/tcg-target.inc.c b/tcg/riscv/tcg-target.inc.c index 6cf8de32b5..b785f4acb7 100644 --- a/tcg/riscv/tcg-target.inc.c +++ b/tcg/riscv/tcg-target.inc.c @@ -958,6 +958,17 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; +/* We don't support oversize guests */ +QEMU_BUILD_BUG_ON(TCG_TARGET_REG_BITS < TARGET_LONG_BITS); + +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect tlb_mask to be "near" tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) - + offsetof(CPUArchState, tlb_mask) >= 0x800); + static void tcg_out_tlb_load(TCGContext *s, TCGReg addrl, TCGReg addrh, TCGMemOpIdx oi, tcg_insn_unit **label_ptr, bool is_load) @@ -965,94 +976,67 @@ static void tcg_out_tlb_load(TCGContext *s, TCGReg addrl, TCGMemOp opc = get_memop(oi); unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - target_ulong mask; + tcg_target_long compare_mask; int mem_index = get_mmuidx(oi); - int cmp_off - = (is_load - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - RISCVInsn load_cmp_op = (TARGET_LONG_BITS == 64 ? OPC_LD : - TCG_TARGET_REG_BITS == 64 ? OPC_LWU : OPC_LW); - RISCVInsn load_add_op = TCG_TARGET_REG_BITS == 64 ? OPC_LD : OPC_LW; - TCGReg base = TCG_AREG0; + int mask_off, table_off; + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; - /* We don't support oversize guests */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - g_assert_not_reached(); + mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + table_off = offsetof(CPUArchState, tlb_table[mem_index]); + if (table_off > 0x7ff) { + int mask_hi = mask_off - sextreg(mask_off, 0, 12); + int table_hi = table_off - sextreg(table_off, 0, 12); + + if (likely(mask_hi == table_hi)) { + mask_base = table_base = TCG_REG_TMP1; + tcg_out_opc_upper(s, OPC_LUI, mask_base, mask_hi); + tcg_out_opc_reg(s, OPC_ADD, mask_base, mask_base, TCG_AREG0); + mask_off -= mask_hi; + table_off -= mask_hi; + } else { + mask_base = TCG_REG_TMP0; + table_base = TCG_REG_TMP1; + tcg_out_opc_upper(s, OPC_LUI, mask_base, mask_hi); + tcg_out_opc_reg(s, OPC_ADD, mask_base, mask_base, TCG_AREG0); + table_off -= mask_off; + mask_off -= mask_hi; + tcg_out_opc_imm(s, OPC_ADDI, table_base, mask_base, mask_off); + } } + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, table_base, table_off); + + tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP2, addrl, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP0); + tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); + + /* Load the tlb comparator and the addend. */ + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP0, TCG_REG_TMP2, + is_load ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP2, TCG_REG_TMP2, + offsetof(CPUTLBEntry, addend)); + /* We don't support unaligned accesses. */ if (a_bits < s_bits) { a_bits = s_bits; } - mask = (target_ulong)TARGET_PAGE_MASK | ((1 << a_bits) - 1); - - - /* Compensate for very large offsets. */ - if (add_off >= 0x1000) { - int adj; - base = TCG_REG_TMP2; - if (cmp_off <= 2 * 0xfff) { - adj = 0xfff; - tcg_out_opc_imm(s, OPC_ADDI, base, TCG_AREG0, adj); - } else { - adj = cmp_off - sextreg(cmp_off, 0, 12); - tcg_debug_assert(add_off - adj >= -0x1000 - && add_off - adj < 0x1000); - - tcg_out_opc_upper(s, OPC_LUI, base, adj); - tcg_out_opc_reg(s, OPC_ADD, base, base, TCG_AREG0); - } - add_off -= adj; - cmp_off -= adj; - } - - /* Extract the page index. */ - if (CPU_TLB_BITS + CPU_TLB_ENTRY_BITS < 12) { - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, addrl, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP0, TCG_REG_TMP0, - MAKE_64BIT_MASK(CPU_TLB_ENTRY_BITS, CPU_TLB_BITS)); - } else if (TARGET_PAGE_BITS >= 12) { - tcg_out_opc_upper(s, OPC_LUI, TCG_REG_TMP0, - MAKE_64BIT_MASK(TARGET_PAGE_BITS, CPU_TLB_BITS)); - tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP0, TCG_REG_TMP0, addrl); - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, TCG_REG_TMP0, - CPU_TLB_BITS - CPU_TLB_ENTRY_BITS); - } else { - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, addrl, TARGET_PAGE_BITS); - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP0, TCG_REG_TMP0, - MAKE_64BIT_MASK(0, CPU_TLB_BITS)); - tcg_out_opc_imm(s, OPC_SLLI, TCG_REG_TMP0, TCG_REG_TMP0, - CPU_TLB_ENTRY_BITS); - } - - /* Add that to the base address to index the tlb. */ - tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, base, TCG_REG_TMP0); - base = TCG_REG_TMP2; - - /* Load the tlb comparator and the addend. */ - tcg_out_ldst(s, load_cmp_op, TCG_REG_TMP0, base, cmp_off); - tcg_out_ldst(s, load_add_op, TCG_REG_TMP2, base, add_off); - /* Clear the non-page, non-alignment bits from the address. */ - if (mask == sextreg(mask, 0, 12)) { - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addrl, mask); + compare_mask = (tcg_target_long)TARGET_PAGE_MASK | ((1 << a_bits) - 1); + if (compare_mask == sextreg(compare_mask, 0, 12)) { + tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addrl, compare_mask); } else { - tcg_out_movi(s, TCG_TYPE_REG, TCG_REG_TMP1, mask); + tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_TMP1, compare_mask); tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP1, TCG_REG_TMP1, addrl); - } + } /* Compare masked address with the TLB entry. */ label_ptr[0] = s->code_ptr; tcg_out_opc_branch(s, OPC_BNE, TCG_REG_TMP0, TCG_REG_TMP1, 0); /* NOP to allow patching later */ tcg_out_opc_imm(s, OPC_ADDI, TCG_REG_ZERO, TCG_REG_ZERO, 0); - /* TODO: Move this out of line - * see: - * https://lists.nongnu.org/archive/html/qemu-devel/2018-11/msg02234.html - */ /* TLB Hit - translate address using addend. */ if (TCG_TARGET_REG_BITS > TARGET_LONG_BITS) {