From patchwork Mon Jan 28 15:58:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156768 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3597768jaa; Mon, 28 Jan 2019 08:08:51 -0800 (PST) X-Google-Smtp-Source: ALg8bN78m03jQR+bUadcMPGAFXes9qDfdBjeGkxgVqIh1O8wGzrrhjZHarQ8eSBmev9yVtcdRrAf X-Received: by 2002:a1c:1902:: with SMTP id 2mr18225670wmz.150.1548691731373; Mon, 28 Jan 2019 08:08:51 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691731; cv=none; d=google.com; s=arc-20160816; b=Cti6J6itKzAuZdls8pCNgM059KL4I5Nc4qdn8CLDriOfJ+t0VBZgc2NU5s7DKD7mPF 2AEpekyOI1sIVENK4Z40Xx7UULPBUbZrVnD4KVTpSYQgTuO7VDTeyryMRrkNkUXajCl9 mle+SIH+9/XqUMpeIBaOQrlqjBrWQsAaVo8b7my/gtQ4Z6AdrPCgiAwSfvoJKIJ8ym8L 6INvm6QxU6GP1q3H0I76N+GTTj7JzCNn0tTxn8XWVraOEfX+3PtFZxLa0SPLqNZWlZLo Vgx3ljZ0ImDOlElGv2MgQRUnbtnO1gfkoYFZGs7Ofz88ehEuj3qkDKV/eVXhaiznzOTg 1TwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=po3fGKZcEsraXkqBMSX5FPnIKtuOJRDnAJ2/AeZX7VI=; b=Bgko5g8KGJ5b1us9Yxebr9RbGMpaTkSH4YDPqcSp7FQUTeCT+IOFRePb75Guw9Wb/u 6t2g0suX767KitRM/7uMiosFGEFbIHdUGO2iFB0e86T1ZUq6yhF/Vg6L1jjM6ORT4SjW Ton/OxHS85ZaDTs0w5dmZrEK6vA8y3yJcFD5TpzuAjD0xNVlDcGr/hjC2edOiPuc67cQ tTcVInjcF4mMmwkgk7BNEdT7teygxazlVRgDSx3e4J7f7UQUQK7sX9MdRWIPmjpRpbOQ MQmyM8bU52rMLbQpITDxCGKdnCzt5hy+2G5pd3BFgc/IVHVgHBGGxh7RwV5CITCs7g0q zexA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=CjassVpi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h70si84729613wrh.88.2019.01.28.08.08.51 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:08:51 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=CjassVpi; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34265 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Ss-0002kr-7V for patch@linaro.org; Mon, 28 Jan 2019 11:08:50 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33339) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jc-0004l7-Un for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:17 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jb-0007hU-Vw for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:16 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:44105) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jb-0007ga-3x for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:15 -0500 Received: by mail-pl1-x62d.google.com with SMTP id e11so7904565plt.11 for ; Mon, 28 Jan 2019 07:59:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=po3fGKZcEsraXkqBMSX5FPnIKtuOJRDnAJ2/AeZX7VI=; b=CjassVpiSvAdhz+HmuArDM2Ss1pfNvazmr2x0K+2arepsS0fG9/dE9FRCJymAn8+ZP 1MtLwisO68oULAdE5vz84aakOoVqTp4gNGx1FmJuLLic4CF3jM6QHNELJXob/pLch3BH hL903L7NcknKEnAloaVikYfkHqeAYHH8kRZS4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=po3fGKZcEsraXkqBMSX5FPnIKtuOJRDnAJ2/AeZX7VI=; b=b+Ec9xchRbcK4jdkPCfBFeZDhpVCelFSukNC8WlJzwTkC6n6ubLgxsnfxJlJI8wswy yVVn1O1bydSDPsgNLwlW1eEtenqlPNKNLXu8eVZ+TnU+rwXdAQodH2dYrvR/8/H1rbB0 kCshqtBLD26Qpn/kLVH1Cn/qDZx+j2gj/6XZarRZPXfazmUt81ZkFet+gV/9wavmsi+I /kaPBDMtznI0HH2FKAhp59wtl0MLUxWXdkiROHXCOyO1OuVjiTm2uoQSCaPQa7PaCSGf OIjJzQ9C4c0jGVd/AfqZjGOdIeDQUCyjThaetF8WlEI3cwQQYWOi4HIzcgevOdcOvP0x imbw== X-Gm-Message-State: AJcUukfOsn46sMybR+L+O2MsJvA2nDZLAu2Eu4xVZGoYokUS7C3EU26P zioHf8JjMtupVQX6phxVs0Ny8NUUHao= X-Received: by 2002:a17:902:8607:: with SMTP id f7mr21746826plo.123.1548691152396; Mon, 28 Jan 2019 07:59:12 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:11 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:45 -0800 Message-Id: <20190128155907.20607-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::62d Subject: [Qemu-devel] [PULL 01/23] tcg: Add logical simplifications during gvec expand X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We handle many of these during integer expansion, and the rest of them during integer optimization. Reviewed-by: David Gibson Signed-off-by: Richard Henderson --- tcg/tcg-op-gvec.c | 35 ++++++++++++++++++++++++++++++----- 1 file changed, 30 insertions(+), 5 deletions(-) -- 2.17.2 diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index 61c25f5784..ec231b78fb 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -1840,7 +1840,12 @@ void tcg_gen_gvec_and(unsigned vece, uint32_t dofs, uint32_t aofs, .opc = INDEX_op_and_vec, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; - tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + + if (aofs == bofs) { + tcg_gen_gvec_mov(vece, dofs, aofs, oprsz, maxsz); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } } void tcg_gen_gvec_or(unsigned vece, uint32_t dofs, uint32_t aofs, @@ -1853,7 +1858,12 @@ void tcg_gen_gvec_or(unsigned vece, uint32_t dofs, uint32_t aofs, .opc = INDEX_op_or_vec, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; - tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + + if (aofs == bofs) { + tcg_gen_gvec_mov(vece, dofs, aofs, oprsz, maxsz); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } } void tcg_gen_gvec_xor(unsigned vece, uint32_t dofs, uint32_t aofs, @@ -1866,7 +1876,12 @@ void tcg_gen_gvec_xor(unsigned vece, uint32_t dofs, uint32_t aofs, .opc = INDEX_op_xor_vec, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; - tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + + if (aofs == bofs) { + tcg_gen_gvec_dup8i(dofs, oprsz, maxsz, 0); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } } void tcg_gen_gvec_andc(unsigned vece, uint32_t dofs, uint32_t aofs, @@ -1879,7 +1894,12 @@ void tcg_gen_gvec_andc(unsigned vece, uint32_t dofs, uint32_t aofs, .opc = INDEX_op_andc_vec, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; - tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + + if (aofs == bofs) { + tcg_gen_gvec_dup8i(dofs, oprsz, maxsz, 0); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } } void tcg_gen_gvec_orc(unsigned vece, uint32_t dofs, uint32_t aofs, @@ -1892,7 +1912,12 @@ void tcg_gen_gvec_orc(unsigned vece, uint32_t dofs, uint32_t aofs, .opc = INDEX_op_orc_vec, .prefer_i64 = TCG_TARGET_REG_BITS == 64, }; - tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + + if (aofs == bofs) { + tcg_gen_gvec_dup8i(dofs, oprsz, maxsz, -1); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } } static const GVecGen2s gop_ands = { From patchwork Mon Jan 28 15:58:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156770 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3530618oct; Mon, 28 Jan 2019 08:10:27 -0800 (PST) X-Google-Smtp-Source: ALg8bN6V6y+WaKHPdvMS4sa12cpEhIUxu96NUp2vRUc3jBcKRU7HuhpuH2EsGXOf0ToBcWDnD0K5 X-Received: by 2002:a5d:5208:: with SMTP id j8mr23173380wrv.188.1548691827458; Mon, 28 Jan 2019 08:10:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691827; cv=none; d=google.com; s=arc-20160816; b=AjgkHZyDV7Mbv54H+3FhLcFfm9Bkq8pRZHN8OP63mIDd9tt5DgkZe1zLs/b6hD/eBq RrsQuQr9zwlbm8Xpwx5O4kyTlhG/KMft5U3MFZF1ayWB4B+U3T8u0NxtdPKV2joYpFnC QbF2c/aQDTC5BAfkfkwwGLhXWgGMMv8AQVgGIIi2LY4AY6/gIF/cbudlDU3qEw+yrsir bCZXlaJjMEe/VY6n47MwxHKQzJeRFdfzz9exZX1+2yyqrSfb2ozB9B0RvhKT540/fTD9 HN/bcQeqUj4iUdZ8+wPvPKc47jwJLIeompPzkTkgtaBC7mfCUfjNg/mfhUpWn8maRPmv aTnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=DCNcv2k0t88K5ivbES4v5MrwSiEW1I9azwgO1d1vYhU=; b=rYtJBoeifpt03Es7cPe+TrvHMIacAn6AezfTJ7YqzZdvaV2TmiZI4Us1aGir3tBmS3 nbSkqw7h3zVbTWewd8ClWTUpXuHpS6nYAgxRLMv1LD6W6XZE3/fLZDgHqGYTWrijlPcV gf+E1IEMa8V6nCxmo6USMOOGq+R7K6XLeM3FKszrODOVE6dbLcUt8uNqEew1a73qoCwk U8zEK5VIA00+Qo1OIaXwD+7UC06QfOPIDwq1Q3oF5wTZSwGHKG/ZMj+UhE9M/IgK1RZZ Ra7WpNa3CVC5e5KYVftt0u96CQD5fjzy89xQqhRgnCxbbNnuHsReDRz3SHhSYRjfSFLX FQvg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BeucWBR1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u84si48533708wmg.158.2019.01.28.08.10.26 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:10:27 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BeucWBR1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34281 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9UP-0003w1-Bj for patch@linaro.org; Mon, 28 Jan 2019 11:10:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33342) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jd-0004lL-76 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:19 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jc-0007hb-1J for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:17 -0500 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:34960) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jb-0007h2-5K for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:15 -0500 Received: by mail-pl1-x642.google.com with SMTP id p8so7917958plo.2 for ; Mon, 28 Jan 2019 07:59:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=DCNcv2k0t88K5ivbES4v5MrwSiEW1I9azwgO1d1vYhU=; b=BeucWBR1Due/Y6REYG6dCD3liqlFfCfxhjbBDsSQNlb/5oKY4EBWlvOcER2tk7D7Xj hpK0WWiqZp/fnmWWHGxWi2sPxra4xx0pDZTz0qJvuNev5JXBTcpSuCVQstabeSFsizTJ ov3fcw2YqmjNqFt4l3Qv7VWjgQmjrHvvDMkiY= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=DCNcv2k0t88K5ivbES4v5MrwSiEW1I9azwgO1d1vYhU=; b=i3sEb4Gi3lnJV+E12jKJW6WG3LG79oY3Hwqv/O7t3WegC63SO2mFyFIQ3KT+KmAmEQ t03CNkE9imWDbpd0ySFi+iRQKB6qEI/j9ASZ+SwQhC5ueyeoJeCguIKfpRigSOcURvMq oc0dqapveH4V9WXbJ9qyfL6gLhUN69wMMPWuHO0GK40t8iV99CtlBwxjhmERPBZ5E1Yp XbhQBjyZyyUdKaPBps+RHU0RmBPc0hnrtLINHxek2QZ/xufSqP9QCCQlGGTAbAcoUeXS lB0OgXEf5wvOAfGcYRZANeRZbaYufE+86uzTsbebzDLI2FA2JliCBIw5sNbUkf6ybf99 FOtQ== X-Gm-Message-State: AJcUukcvH9YSqcSieU9EEdexs9m2NFoDF8GwFR2y1vZiAPexN/n5w87i ZFwFlFrMSbAbyQfvgfH8chD/EVZZvXM= X-Received: by 2002:a17:902:f24:: with SMTP id 33mr22540189ply.65.1548691153779; Mon, 28 Jan 2019 07:59:13 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.12 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:13 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:46 -0800 Message-Id: <20190128155907.20607-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::642 Subject: [Qemu-devel] [PULL 02/23] tcg: Add gvec expanders for nand, nor, eqv X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: David Gibson Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 3 +++ tcg/tcg-op-gvec.h | 6 +++++ tcg/tcg-op.h | 3 +++ accel/tcg/tcg-runtime-gvec.c | 33 +++++++++++++++++++++++ tcg/tcg-op-gvec.c | 51 ++++++++++++++++++++++++++++++++++++ tcg/tcg-op-vec.c | 21 +++++++++++++++ 6 files changed, 117 insertions(+) -- 2.17.2 diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index 1bd39d136d..835ddfebb2 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -211,6 +211,9 @@ DEF_HELPER_FLAGS_4(gvec_or, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_xor, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_andc, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_orc, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_nand, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_nor, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_eqv, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ands, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(gvec_xors, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) diff --git a/tcg/tcg-op-gvec.h b/tcg/tcg-op-gvec.h index ff43a29a0b..d65b9d9d4c 100644 --- a/tcg/tcg-op-gvec.h +++ b/tcg/tcg-op-gvec.h @@ -242,6 +242,12 @@ void tcg_gen_gvec_andc(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz); void tcg_gen_gvec_orc(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_nand(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_nor(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_eqv(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); void tcg_gen_gvec_andi(unsigned vece, uint32_t dofs, uint32_t aofs, int64_t c, uint32_t oprsz, uint32_t maxsz); diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h index 7007ec0d4d..f6ef1cd690 100644 --- a/tcg/tcg-op.h +++ b/tcg/tcg-op.h @@ -962,6 +962,9 @@ void tcg_gen_or_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_xor_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_andc_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_orc_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_nand_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_nor_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_eqv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_not_vec(unsigned vece, TCGv_vec r, TCGv_vec a); void tcg_gen_neg_vec(unsigned vece, TCGv_vec r, TCGv_vec a); diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c index 90340e56e0..d1802467d5 100644 --- a/accel/tcg/tcg-runtime-gvec.c +++ b/accel/tcg/tcg-runtime-gvec.c @@ -512,6 +512,39 @@ void HELPER(gvec_orc)(void *d, void *a, void *b, uint32_t desc) clear_high(d, oprsz, desc); } +void HELPER(gvec_nand)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(vec64)) { + *(vec64 *)(d + i) = ~(*(vec64 *)(a + i) & *(vec64 *)(b + i)); + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_nor)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(vec64)) { + *(vec64 *)(d + i) = ~(*(vec64 *)(a + i) | *(vec64 *)(b + i)); + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_eqv)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(vec64)) { + *(vec64 *)(d + i) = ~(*(vec64 *)(a + i) ^ *(vec64 *)(b + i)); + } + clear_high(d, oprsz, desc); +} + void HELPER(gvec_ands)(void *d, void *a, uint64_t b, uint32_t desc) { intptr_t oprsz = simd_oprsz(desc); diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index ec231b78fb..81689d02f7 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -1920,6 +1920,57 @@ void tcg_gen_gvec_orc(unsigned vece, uint32_t dofs, uint32_t aofs, } } +void tcg_gen_gvec_nand(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g = { + .fni8 = tcg_gen_nand_i64, + .fniv = tcg_gen_nand_vec, + .fno = gen_helper_gvec_nand, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + + if (aofs == bofs) { + tcg_gen_gvec_not(vece, dofs, aofs, oprsz, maxsz); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } +} + +void tcg_gen_gvec_nor(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g = { + .fni8 = tcg_gen_nor_i64, + .fniv = tcg_gen_nor_vec, + .fno = gen_helper_gvec_nor, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + + if (aofs == bofs) { + tcg_gen_gvec_not(vece, dofs, aofs, oprsz, maxsz); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } +} + +void tcg_gen_gvec_eqv(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g = { + .fni8 = tcg_gen_eqv_i64, + .fniv = tcg_gen_eqv_vec, + .fno = gen_helper_gvec_eqv, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + + if (aofs == bofs) { + tcg_gen_gvec_dup8i(dofs, oprsz, maxsz, -1); + } else { + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g); + } +} + static const GVecGen2s gop_ands = { .fni8 = tcg_gen_and_i64, .fniv = tcg_gen_and_vec, diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c index cefba3d185..d77fdf7c1d 100644 --- a/tcg/tcg-op-vec.c +++ b/tcg/tcg-op-vec.c @@ -275,6 +275,27 @@ void tcg_gen_orc_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) } } +void tcg_gen_nand_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + /* TODO: Add TCG_TARGET_HAS_nand_vec when adding a backend supports it. */ + tcg_gen_and_vec(0, r, a, b); + tcg_gen_not_vec(0, r, r); +} + +void tcg_gen_nor_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + /* TODO: Add TCG_TARGET_HAS_nor_vec when adding a backend supports it. */ + tcg_gen_or_vec(0, r, a, b); + tcg_gen_not_vec(0, r, r); +} + +void tcg_gen_eqv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + /* TODO: Add TCG_TARGET_HAS_eqv_vec when adding a backend supports it. */ + tcg_gen_xor_vec(0, r, a, b); + tcg_gen_not_vec(0, r, r); +} + void tcg_gen_not_vec(unsigned vece, TCGv_vec r, TCGv_vec a) { if (TCG_TARGET_HAS_not_vec) { From patchwork Mon Jan 28 15:58:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156767 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3595135jaa; Mon, 28 Jan 2019 08:06:48 -0800 (PST) X-Google-Smtp-Source: ALg8bN4J76VQ4qF44fmCZ0mEQB9N7avcdFA1InshyBtKuxB74f6O11/Q/aSMboewV1b7tg/DFi7o X-Received: by 2002:adf:fa83:: with SMTP id h3mr23858232wrr.173.1548691608581; Mon, 28 Jan 2019 08:06:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691608; cv=none; d=google.com; s=arc-20160816; b=FoLqRXH8O7GAHVkfA7JydNCrPUZzdlsbyfC6UmPPtGkwqZ8rtXfBt6eWsKVpEJx36Z lxjUmjGfNJlrClSePDCa55Vr4zzUfbBfCXeGT8OS1FRY93DuzwgRxxzxasOMv73+1tT7 ba/CEzx+Onl+yBc5jSs2FYWrVsnCPZPMDk+skh3gVWPZ3UavbNHi63f+IeRu1zSanWwj jeY9ra9PgWfRdhs+ZF7V7jvLxrJ1+riU+GWuZyTQIUOP9rG5HcLylhc1LO7xPoYcG9rS YlR7ckVSbfE6qHOFTbCaIvk1Llc5VNPUM8qLTFOviqEOGXMQiYChBCfyUgdktvn0X8pU 3UWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=13baR0fYl6duiBYqIpKDqOz4EjDaN2cM4w2ffhmehW8=; b=fDE88RtTpByZI8w3JTpbeOIkQTAfI+y5yFFlQMa+29IT45DBncd2s5ggIBELsIbvDr JkxTzC+TNWBN6t/d1Lhdr8PUsmiO4gKK3AZ+rxqrqBXHuxDDDUOWI7r3buFX230aQOwv sfJJBrR8BY4L6z/TgtBt9oELh392hk4TQuXUiozGmnHtFXpj/mG0lODastbvXpPg8KRo VKMDPUbe3/lIaIlCSea2Xpxm4dsnICZU3g2dSkBD3CSegherS1ieGd+QA7/CD81OuHtR bqeI9ENX9Rv4n4Bd8iniHqeugnl9xvNE9rnfaTHUGvbxDH3jEYzzfjGoPWAIg/89Ad5F PPUQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=TvvOZbnN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m12si47354990wmd.167.2019.01.28.08.06.48 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:06:48 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=TvvOZbnN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34257 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Qt-00026o-Bw for patch@linaro.org; Mon, 28 Jan 2019 11:06:47 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33360) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jf-0004ne-1q for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:20 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jd-0007hv-5J for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:18 -0500 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:43918) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jc-0007hR-U9 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:17 -0500 Received: by mail-pg1-x535.google.com with SMTP id v28so7381336pgk.10 for ; Mon, 28 Jan 2019 07:59:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=13baR0fYl6duiBYqIpKDqOz4EjDaN2cM4w2ffhmehW8=; b=TvvOZbnNY38w1YroBEn0r5BAh0m9V5jySacb2qNJx3U+O5j9FPeoZjIem11l3mklFq iGLV6AE02iXDUT5mt9U/plGe9Mjhg7g7RCE6RLPLO1VXYW/jpapA3dcLhaZ4Ybpf+kv2 KhWa61PDDijJfBIWI8w1EqRk8lBHPUQB4OAZc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=13baR0fYl6duiBYqIpKDqOz4EjDaN2cM4w2ffhmehW8=; b=QqJY8OgYV0XeEijuOh+lnQMX1Tm7S9dBpLCwti2OUF7OvrsglA8FDSU/vy7XFz0WRO XuxT/GsDOWLSr74Z2ckpDncB98qxihqO3ZnTnxf/zhxnwZfmU4JU2z6ujfRaFnsqWft7 I0wFd/3Z+zUS99goo8dCkaVIs1WIY1r2Z5nz2ptgbiC0vsHDy3dum8tAjLrBnf0KQJeH PeqYZ/VD5/4qlYAaMadRIaIROEZVjl4/GEpUenNTSfIHqQTX+TxMQb62FJ4iEMEo0xac SPqHHNsTm0DdfW6ZSZEtUD0FD0tKLyQaT9ZR/nHnWd8kVFr97Y4k1O1bpPinTF2abYhf n9pg== X-Gm-Message-State: AJcUukfAFmKMbGGZbPRSYG9UojQQHsnrAoI6IDR3y+DlZGlY2/HWwvgI Se4ti4NW5wOCsdMQ2O5qBlSS6gx5PYc= X-Received: by 2002:a62:ca05:: with SMTP id n5mr22467747pfg.154.1548691155192; Mon, 28 Jan 2019 07:59:15 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.13 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:14 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:47 -0800 Message-Id: <20190128155907.20607-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::535 Subject: [Qemu-devel] [PULL 03/23] tcg: Add write_aofs to GVecGen4 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This allows writing 2 output, 3 input operations. Signed-off-by: Richard Henderson --- tcg/tcg-op-gvec.h | 2 ++ tcg/tcg-op-gvec.c | 27 +++++++++++++++++++-------- 2 files changed, 21 insertions(+), 8 deletions(-) -- 2.17.2 diff --git a/tcg/tcg-op-gvec.h b/tcg/tcg-op-gvec.h index d65b9d9d4c..2cb447112e 100644 --- a/tcg/tcg-op-gvec.h +++ b/tcg/tcg-op-gvec.h @@ -181,6 +181,8 @@ typedef struct { uint8_t vece; /* Prefer i64 to v64. */ bool prefer_i64; + /* Write aofs as a 2nd dest operand. */ + bool write_aofs; } GVecGen4; void tcg_gen_gvec_2(uint32_t dofs, uint32_t aofs, diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index 81689d02f7..c10d3d7b26 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -665,7 +665,7 @@ static void expand_3_i32(uint32_t dofs, uint32_t aofs, /* Expand OPSZ bytes worth of three-operand operations using i32 elements. */ static void expand_4_i32(uint32_t dofs, uint32_t aofs, uint32_t bofs, - uint32_t cofs, uint32_t oprsz, + uint32_t cofs, uint32_t oprsz, bool write_aofs, void (*fni)(TCGv_i32, TCGv_i32, TCGv_i32, TCGv_i32)) { TCGv_i32 t0 = tcg_temp_new_i32(); @@ -680,6 +680,9 @@ static void expand_4_i32(uint32_t dofs, uint32_t aofs, uint32_t bofs, tcg_gen_ld_i32(t3, cpu_env, cofs + i); fni(t0, t1, t2, t3); tcg_gen_st_i32(t0, cpu_env, dofs + i); + if (write_aofs) { + tcg_gen_st_i32(t1, cpu_env, aofs + i); + } } tcg_temp_free_i32(t3); tcg_temp_free_i32(t2); @@ -769,7 +772,7 @@ static void expand_3_i64(uint32_t dofs, uint32_t aofs, /* Expand OPSZ bytes worth of three-operand operations using i64 elements. */ static void expand_4_i64(uint32_t dofs, uint32_t aofs, uint32_t bofs, - uint32_t cofs, uint32_t oprsz, + uint32_t cofs, uint32_t oprsz, bool write_aofs, void (*fni)(TCGv_i64, TCGv_i64, TCGv_i64, TCGv_i64)) { TCGv_i64 t0 = tcg_temp_new_i64(); @@ -784,6 +787,9 @@ static void expand_4_i64(uint32_t dofs, uint32_t aofs, uint32_t bofs, tcg_gen_ld_i64(t3, cpu_env, cofs + i); fni(t0, t1, t2, t3); tcg_gen_st_i64(t0, cpu_env, dofs + i); + if (write_aofs) { + tcg_gen_st_i64(t1, cpu_env, aofs + i); + } } tcg_temp_free_i64(t3); tcg_temp_free_i64(t2); @@ -880,7 +886,7 @@ static void expand_3_vec(unsigned vece, uint32_t dofs, uint32_t aofs, /* Expand OPSZ bytes worth of four-operand operations using host vectors. */ static void expand_4_vec(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, uint32_t oprsz, - uint32_t tysz, TCGType type, + uint32_t tysz, TCGType type, bool write_aofs, void (*fni)(unsigned, TCGv_vec, TCGv_vec, TCGv_vec, TCGv_vec)) { @@ -896,6 +902,9 @@ static void expand_4_vec(unsigned vece, uint32_t dofs, uint32_t aofs, tcg_gen_ld_vec(t3, cpu_env, cofs + i); fni(vece, t0, t1, t2, t3); tcg_gen_st_vec(t0, cpu_env, dofs + i); + if (write_aofs) { + tcg_gen_st_vec(t1, cpu_env, aofs + i); + } } tcg_temp_free_vec(t3); tcg_temp_free_vec(t2); @@ -1187,7 +1196,7 @@ void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, */ some = QEMU_ALIGN_DOWN(oprsz, 32); expand_4_vec(g->vece, dofs, aofs, bofs, cofs, some, - 32, TCG_TYPE_V256, g->fniv); + 32, TCG_TYPE_V256, g->write_aofs, g->fniv); if (some == oprsz) { break; } @@ -1200,18 +1209,20 @@ void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, /* fallthru */ case TCG_TYPE_V128: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, - 16, TCG_TYPE_V128, g->fniv); + 16, TCG_TYPE_V128, g->write_aofs, g->fniv); break; case TCG_TYPE_V64: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, - 8, TCG_TYPE_V64, g->fniv); + 8, TCG_TYPE_V64, g->write_aofs, g->fniv); break; case 0: if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_4_i64(dofs, aofs, bofs, cofs, oprsz, g->fni8); + expand_4_i64(dofs, aofs, bofs, cofs, oprsz, + g->write_aofs, g->fni8); } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_4_i32(dofs, aofs, bofs, cofs, oprsz, g->fni4); + expand_4_i32(dofs, aofs, bofs, cofs, oprsz, + g->write_aofs, g->fni4); } else { assert(g->fno != NULL); tcg_gen_gvec_4_ool(dofs, aofs, bofs, cofs, From patchwork Mon Jan 28 15:58:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156772 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3532416oct; Mon, 28 Jan 2019 08:11:54 -0800 (PST) X-Google-Smtp-Source: ALg8bN63yKp+d6FUSQ0EL4gm1fOVWMmBa+7+A9srWTJokc6poqmR8pvK7MdSpedljI42VfGtqbPN X-Received: by 2002:a1c:2d89:: with SMTP id t131mr18276186wmt.97.1548691914287; Mon, 28 Jan 2019 08:11:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691914; cv=none; d=google.com; s=arc-20160816; b=ghAjkXjFK2Oc4RnhAdalEKzilcnPToVU0cGVTErEe/z/w1dSUMIfLv/kvw0Ke3gBCG Ual4hd9adlB3X+UadwjGHNdOnfnRB9kQZb0M7FxQ689Qd1XQ1hWjFUvzRWvcV+xAmY3O 9CEutQCbN501vVGlhapNWjVUMawH0tNkNsUf6ynbZS7zirEA4Vcd39MoYtCrkDQ97/Ct 17MGSJ8saufSNw8UddSrCRQn1ZyoTh6lxM4ZX/EgVHrAEx9yHjdB3z2RHwFWHkQ4sOFv OrTLwv3YBT3VNSk2jxzE/nFPp1Tfg91y249j6Me2V2NVX/yThNFQEB9UPfvwUsMLjQHh egtg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=dzhcNTPgwlc4WINupPShWOs/UFMSDVTx9oQgzH2yGAI=; b=YYIJPIVh0F90eb3n5XwaTzhaJJwNYmMV1U7J4mIPG/T8o5AAuTsVzOj3bTLvmzvSu5 ugOm/6gUf/9cfubUEDXzwf2SGiolb3yQmzamgWLEP6WzEVMypiwDVW6BAM9Ci4Y1P/QB yuwP+qlHjXK6eFpWYamnkcv5DuUU6JfxTJoWjlBkOe0RwMgEharGBZL4XO5/0bJ1RdoW pHvFw34AN+yl90FW1Y5wKTget22L+XJPZJnoBupA3Y2TXJs2Yeaf8tgVoDmS/LcY1Aq2 t/1euks8QZe3IMmVo82ZIUah+Yd4IkrriF9cPpDeIJ06W/M72le3NIsHk7t7oHHOeGc8 nALw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SzpofAkk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y18si11980807wmi.81.2019.01.28.08.11.53 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:11:54 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SzpofAkk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34327 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Vp-00059I-22 for patch@linaro.org; Mon, 28 Jan 2019 11:11:53 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jg-0004pU-NH for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:22 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jf-0007is-8m for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:20 -0500 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]:42636) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Je-0007iK-W3 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:19 -0500 Received: by mail-pl1-x632.google.com with SMTP id y1so7917332plp.9 for ; Mon, 28 Jan 2019 07:59:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dzhcNTPgwlc4WINupPShWOs/UFMSDVTx9oQgzH2yGAI=; b=SzpofAkkQWeEjVmpSxNNMxFoZnTwdVxr82r/JuL3+kUuw8hSfJ1UGX1/HToWulzbQT F1FipXSzaWh8tvMd1+UZoH62ZcbUeciA+rzqOesfTQ7S83niCEzGn+Tvemv6cNJer9dF M6OTq/RLShooHbSpCOgXJ06NXbDjOHWLqwDe4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dzhcNTPgwlc4WINupPShWOs/UFMSDVTx9oQgzH2yGAI=; b=Kwb2fslsTS3tYYN2bwp+M+H05mblU7UNIRFoB0/cpWpLuMGOLNEDETUjdfIfSNBJNv nGJjpK3MXhHe4dnyyDQFsasCD1JD20BkKhCYjIyGVEgqeR61iKlno301+dlj0qnPqkfq gNmcxx44caGr/wZPLft2I1IS8+T8UftFbkIQe/xL9K1RFx6XprnkIoNhnEGIDDF8O+jW oZxb1EuWk8iF7GqhNe7xrsmQs9mXT+UHE3m8+s4xvQ9hVTCWy739ruqRTftU0Xs32QjR MRng7ANsKQD5WlX2AI8Q+GrOYELD2Obl2Ic7z3PdoB+oHuyt+THBtXRGNCyU8iIlMSqW vc2w== X-Gm-Message-State: AJcUukeLdts3KozldSxOeumHjRgKMjehGYCFwl+21Kj+TAUJCKEz0IWt YBmc5jI4lehaA80Bi7kcN7FW0u5L9GQ= X-Received: by 2002:a17:902:654a:: with SMTP id d10mr21840556pln.324.1548691156490; Mon, 28 Jan 2019 07:59:16 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.15 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:15 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:48 -0800 Message-Id: <20190128155907.20607-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::632 Subject: [Qemu-devel] [PULL 04/23] tcg: Add opcodes for vector saturated arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 1 + tcg/i386/tcg-target.h | 1 + tcg/tcg-op.h | 4 ++ tcg/tcg-opc.h | 4 ++ tcg/tcg.h | 1 + tcg/tcg-op-gvec.c | 84 ++++++++++++++++++++++++++++++---------- tcg/tcg-op-vec.c | 34 ++++++++++++++-- tcg/tcg.c | 5 +++ tcg/README | 9 +++++ 9 files changed, 119 insertions(+), 24 deletions(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index f966a4fcb3..98556bcf22 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -135,6 +135,7 @@ typedef enum { #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 +#define TCG_TARGET_HAS_sat_vec 0 #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index f378d29568..44381062e6 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -185,6 +185,7 @@ extern bool have_avx2; #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 +#define TCG_TARGET_HAS_sat_vec 0 #define TCG_TARGET_deposit_i32_valid(ofs, len) \ (((ofs) == 0 && (len) == 8) || ((ofs) == 8 && (len) == 8) || \ diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h index f6ef1cd690..4a93d730e8 100644 --- a/tcg/tcg-op.h +++ b/tcg/tcg-op.h @@ -967,6 +967,10 @@ void tcg_gen_nor_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_eqv_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_not_vec(unsigned vece, TCGv_vec r, TCGv_vec a); void tcg_gen_neg_vec(unsigned vece, TCGv_vec r, TCGv_vec a); +void tcg_gen_ssadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_usadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_sssub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_ussub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_shli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i); void tcg_gen_shri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i); diff --git a/tcg/tcg-opc.h b/tcg/tcg-opc.h index 7a8a3edb5b..94b2ed80af 100644 --- a/tcg/tcg-opc.h +++ b/tcg/tcg-opc.h @@ -222,6 +222,10 @@ DEF(add_vec, 1, 2, 0, IMPLVEC) DEF(sub_vec, 1, 2, 0, IMPLVEC) DEF(mul_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_mul_vec)) DEF(neg_vec, 1, 1, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_neg_vec)) +DEF(ssadd_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) +DEF(usadd_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) +DEF(sssub_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) +DEF(ussub_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) DEF(and_vec, 1, 2, 0, IMPLVEC) DEF(or_vec, 1, 2, 0, IMPLVEC) diff --git a/tcg/tcg.h b/tcg/tcg.h index f8ec265027..5590dc512a 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -183,6 +183,7 @@ typedef uint64_t TCGRegSet; #define TCG_TARGET_HAS_shs_vec 0 #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_mul_vec 0 +#define TCG_TARGET_HAS_sat_vec 0 #else #define TCG_TARGET_MAYBE_vec 1 #endif diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index c10d3d7b26..0a33f51065 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -1678,10 +1678,22 @@ void tcg_gen_gvec_ssadd(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz) { static const GVecGen3 g[4] = { - { .fno = gen_helper_gvec_ssadd8, .vece = MO_8 }, - { .fno = gen_helper_gvec_ssadd16, .vece = MO_16 }, - { .fno = gen_helper_gvec_ssadd32, .vece = MO_32 }, - { .fno = gen_helper_gvec_ssadd64, .vece = MO_64 } + { .fniv = tcg_gen_ssadd_vec, + .fno = gen_helper_gvec_ssadd8, + .opc = INDEX_op_ssadd_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_ssadd_vec, + .fno = gen_helper_gvec_ssadd16, + .opc = INDEX_op_ssadd_vec, + .vece = MO_16 }, + { .fniv = tcg_gen_ssadd_vec, + .fno = gen_helper_gvec_ssadd32, + .opc = INDEX_op_ssadd_vec, + .vece = MO_32 }, + { .fniv = tcg_gen_ssadd_vec, + .fno = gen_helper_gvec_ssadd64, + .opc = INDEX_op_ssadd_vec, + .vece = MO_64 }, }; tcg_debug_assert(vece <= MO_64); tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); @@ -1691,16 +1703,28 @@ void tcg_gen_gvec_sssub(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz) { static const GVecGen3 g[4] = { - { .fno = gen_helper_gvec_sssub8, .vece = MO_8 }, - { .fno = gen_helper_gvec_sssub16, .vece = MO_16 }, - { .fno = gen_helper_gvec_sssub32, .vece = MO_32 }, - { .fno = gen_helper_gvec_sssub64, .vece = MO_64 } + { .fniv = tcg_gen_sssub_vec, + .fno = gen_helper_gvec_sssub8, + .opc = INDEX_op_sssub_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_sssub_vec, + .fno = gen_helper_gvec_sssub16, + .opc = INDEX_op_sssub_vec, + .vece = MO_16 }, + { .fniv = tcg_gen_sssub_vec, + .fno = gen_helper_gvec_sssub32, + .opc = INDEX_op_sssub_vec, + .vece = MO_32 }, + { .fniv = tcg_gen_sssub_vec, + .fno = gen_helper_gvec_sssub64, + .opc = INDEX_op_sssub_vec, + .vece = MO_64 }, }; tcg_debug_assert(vece <= MO_64); tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); } -static void tcg_gen_vec_usadd32_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) +static void tcg_gen_usadd_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) { TCGv_i32 max = tcg_const_i32(-1); tcg_gen_add_i32(d, a, b); @@ -1708,7 +1732,7 @@ static void tcg_gen_vec_usadd32_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) tcg_temp_free_i32(max); } -static void tcg_gen_vec_usadd32_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) +static void tcg_gen_usadd_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) { TCGv_i64 max = tcg_const_i64(-1); tcg_gen_add_i64(d, a, b); @@ -1720,20 +1744,30 @@ void tcg_gen_gvec_usadd(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz) { static const GVecGen3 g[4] = { - { .fno = gen_helper_gvec_usadd8, .vece = MO_8 }, - { .fno = gen_helper_gvec_usadd16, .vece = MO_16 }, - { .fni4 = tcg_gen_vec_usadd32_i32, + { .fniv = tcg_gen_usadd_vec, + .fno = gen_helper_gvec_usadd8, + .opc = INDEX_op_usadd_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_usadd_vec, + .fno = gen_helper_gvec_usadd16, + .opc = INDEX_op_usadd_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_usadd_i32, + .fniv = tcg_gen_usadd_vec, .fno = gen_helper_gvec_usadd32, + .opc = INDEX_op_usadd_vec, .vece = MO_32 }, - { .fni8 = tcg_gen_vec_usadd32_i64, + { .fni8 = tcg_gen_usadd_i64, + .fniv = tcg_gen_usadd_vec, .fno = gen_helper_gvec_usadd64, + .opc = INDEX_op_usadd_vec, .vece = MO_64 } }; tcg_debug_assert(vece <= MO_64); tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); } -static void tcg_gen_vec_ussub32_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) +static void tcg_gen_ussub_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) { TCGv_i32 min = tcg_const_i32(0); tcg_gen_sub_i32(d, a, b); @@ -1741,7 +1775,7 @@ static void tcg_gen_vec_ussub32_i32(TCGv_i32 d, TCGv_i32 a, TCGv_i32 b) tcg_temp_free_i32(min); } -static void tcg_gen_vec_ussub32_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) +static void tcg_gen_ussub_i64(TCGv_i64 d, TCGv_i64 a, TCGv_i64 b) { TCGv_i64 min = tcg_const_i64(0); tcg_gen_sub_i64(d, a, b); @@ -1753,13 +1787,23 @@ void tcg_gen_gvec_ussub(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz) { static const GVecGen3 g[4] = { - { .fno = gen_helper_gvec_ussub8, .vece = MO_8 }, - { .fno = gen_helper_gvec_ussub16, .vece = MO_16 }, - { .fni4 = tcg_gen_vec_ussub32_i32, + { .fniv = tcg_gen_ussub_vec, + .fno = gen_helper_gvec_ussub8, + .opc = INDEX_op_ussub_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_ussub_vec, + .fno = gen_helper_gvec_ussub16, + .opc = INDEX_op_ussub_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_ussub_i32, + .fniv = tcg_gen_ussub_vec, .fno = gen_helper_gvec_ussub32, + .opc = INDEX_op_ussub_vec, .vece = MO_32 }, - { .fni8 = tcg_gen_vec_ussub32_i64, + { .fni8 = tcg_gen_ussub_i64, + .fniv = tcg_gen_ussub_vec, .fno = gen_helper_gvec_ussub64, + .opc = INDEX_op_ussub_vec, .vece = MO_64 } }; tcg_debug_assert(vece <= MO_64); diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c index d77fdf7c1d..675aa09258 100644 --- a/tcg/tcg-op-vec.c +++ b/tcg/tcg-op-vec.c @@ -386,7 +386,8 @@ void tcg_gen_cmp_vec(TCGCond cond, unsigned vece, } } -void tcg_gen_mul_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +static void do_op3(unsigned vece, TCGv_vec r, TCGv_vec a, + TCGv_vec b, TCGOpcode opc) { TCGTemp *rt = tcgv_vec_temp(r); TCGTemp *at = tcgv_vec_temp(a); @@ -399,11 +400,36 @@ void tcg_gen_mul_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) tcg_debug_assert(at->base_type >= type); tcg_debug_assert(bt->base_type >= type); - can = tcg_can_emit_vec_op(INDEX_op_mul_vec, type, vece); + can = tcg_can_emit_vec_op(opc, type, vece); if (can > 0) { - vec_gen_3(INDEX_op_mul_vec, type, vece, ri, ai, bi); + vec_gen_3(opc, type, vece, ri, ai, bi); } else { tcg_debug_assert(can < 0); - tcg_expand_vec_op(INDEX_op_mul_vec, type, vece, ri, ai, bi); + tcg_expand_vec_op(opc, type, vece, ri, ai, bi); } } + +void tcg_gen_mul_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_mul_vec); +} + +void tcg_gen_ssadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_ssadd_vec); +} + +void tcg_gen_usadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_usadd_vec); +} + +void tcg_gen_sssub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_sssub_vec); +} + +void tcg_gen_ussub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_ussub_vec); +} diff --git a/tcg/tcg.c b/tcg/tcg.c index f34f52fbdb..6363f0cb29 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -1607,6 +1607,11 @@ bool tcg_op_supported(TCGOpcode op) case INDEX_op_shrv_vec: case INDEX_op_sarv_vec: return have_vec && TCG_TARGET_HAS_shv_vec; + case INDEX_op_ssadd_vec: + case INDEX_op_usadd_vec: + case INDEX_op_sssub_vec: + case INDEX_op_ussub_vec: + return have_vec && TCG_TARGET_HAS_sat_vec; default: tcg_debug_assert(op > INDEX_op_last_generic && op < NB_OPS); diff --git a/tcg/README b/tcg/README index d22ee084b8..dd9fd9c86c 100644 --- a/tcg/README +++ b/tcg/README @@ -554,6 +554,15 @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32. Similarly, v0 = -v1. +* ssadd_vec: +* sssub_vec: +* usadd_vec: +* ussub_vec: + + Signed and unsigned saturating addition and subtraction. If the true + result is not representable within the element type, the element is + set to the minimum or maximum value for the type. + * and_vec v0, v1, v2 * or_vec v0, v1, v2 * xor_vec v0, v1, v2 From patchwork Mon Jan 28 15:58:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156775 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3534309oct; Mon, 28 Jan 2019 08:13:26 -0800 (PST) X-Google-Smtp-Source: ALg8bN6EZ5k+PCrTipSWH8CeWgsoAlwEHxspTpIkP0zKIXPhtfKaN9mfRA1MI0imHCYBFmu8XRoy X-Received: by 2002:adf:fa05:: with SMTP id m5mr22014663wrr.155.1548692006625; Mon, 28 Jan 2019 08:13:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692006; cv=none; d=google.com; s=arc-20160816; b=R1iPI+nEw/c3685vxo9H1XjpEN8DNCM5JcKhln2TMU/Ly3YW55vhDi709CdnWe+x2F WvuxWcf/K4kIApQeDq4YN2jhEuH6FBGWi5VJb8h7TZb5Mie6uHR8jjL2qhDucarCcm6U iuBttLw0RJL3BhFWtSGnKwuL3U7oXEbNd29bqR7BWzxT8OtYL4iGkcoNwo92yqRAfs0q X3pymrVoZPaxoMdwTzYeLS/rIwu3hHyYxVbsaAZvJVSQ6J+uMcVOrzKYL67X1GYyZWrE JS8Uk2K82H6pV/4w9QyuyHNJAoca+DrlDyZ8uIEtAI6R69beoqpqETVjImzZG7t0bVxm kDhg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=M6EvWlWNkm0G2PqzxG3n4iDr1nxjrPLmEESLZejWtew=; b=qdf39dgDE7xR20EQ0MbpYVRc+Zswtfy3qTJ9dbSx2InlmL1jiRIvuUmibdRXJie9ld SExyUDFsTBa41pGlAVRhejaDimli81Zdz0gr7uMBLFpAS0Mu9kQnYE+P41fvOxXx0jqA r09uDpVvhF4LEP+EFc7AOu8MEKBVMm410KE2QYvxF5Ip6uqvE4zpGXTV7XvJyxUUbRP+ hLF7RTjw/5ElEDTInQm6CQuRfMTs+ZrG9jE3Z/RQKfPToCM/VvlHprhIDrqyfSpJYZm8 rQDHXA9Xd7+usUMuOL27NrvOnIOTypCE+cDqqwDNECKJ18x4l/TojlXYG4XK1RaQ/Q1Z B3jA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Yew8QDUd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g6si78280550wro.307.2019.01.28.08.13.26 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:13:26 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Yew8QDUd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34339 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9XJ-0006Ke-AL for patch@linaro.org; Mon, 28 Jan 2019 11:13:25 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33399) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jh-0004qz-R2 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:23 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jf-0007j1-To for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:21 -0500 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:38959) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jf-0007in-LU for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:19 -0500 Received: by mail-pg1-x535.google.com with SMTP id w6so7382890pgl.6 for ; Mon, 28 Jan 2019 07:59:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=M6EvWlWNkm0G2PqzxG3n4iDr1nxjrPLmEESLZejWtew=; b=Yew8QDUd9u1hCZXPk2QAyyPj9eJOFzdT8xodW9HQ3o//vz2y4Lm+Q9Uft4GoF2BDvZ l6ofdLqNUYNFH5IxqqIiOdMsNJ78lQAS9jBNXL32fmLo2WyxNe3wczIZq0GZPVavjCft 0uV5qoXaY5BuWr8qOPK5F4sfV24tXUZmmre6A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=M6EvWlWNkm0G2PqzxG3n4iDr1nxjrPLmEESLZejWtew=; b=bb4ApMrOuxssXYc860yHAc7kcuoGpaOB7JSE3cXy6aFwQ5Tihm88UyvUPKIRRIkNEv QnoX0dwBMWku0bxs9flQh6ls1h+ZwDEOXBZcqpyZC+AJ0osFLXzZzQpbNeiSuFbIP/Em SVyCsOWLzczvEBgDDzvdCd8gn4QoZ4FGkAXTyZnxP0qNx93w4Splyd34Kd7TC2JsiJhk 6DEbv1ZRGy4vjxnUu5igGjP2dC97P/6VzhLcLqvtBashVUAS5xYm2Ksp8Kp93TG08mvK zanFeb7hrtOgFvjHc2vlepv+MAPEf/Z5e9llmGwXhCK+K3EREShkMJFhqb5v+Uk4lU1r rfGg== X-Gm-Message-State: AJcUuke4Uj0jf4urZhEp8ppKruv+BMkzCLxbHCkU+kknScWF5Qyv3d3V NNmQ6KiaC+F1E/pBJgdcnz2sVtGo+PE= X-Received: by 2002:a62:546:: with SMTP id 67mr21931165pff.99.1548691158027; Mon, 28 Jan 2019 07:59:18 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.16 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:17 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:49 -0800 Message-Id: <20190128155907.20607-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::535 Subject: [Qemu-devel] [PULL 05/23] tcg: Add opcodes for vector minmax arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- accel/tcg/tcg-runtime.h | 20 ++++ tcg/aarch64/tcg-target.h | 1 + tcg/i386/tcg-target.h | 1 + tcg/tcg-op-gvec.h | 10 ++ tcg/tcg-op.h | 4 + tcg/tcg-opc.h | 4 + tcg/tcg.h | 1 + accel/tcg/tcg-runtime-gvec.c | 224 +++++++++++++++++++++++++++++++++++ tcg/tcg-op-gvec.c | 108 +++++++++++++++++ tcg/tcg-op-vec.c | 20 ++++ tcg/tcg.c | 5 + tcg/README | 10 ++ 12 files changed, 408 insertions(+) -- 2.17.2 diff --git a/accel/tcg/tcg-runtime.h b/accel/tcg/tcg-runtime.h index 835ddfebb2..dfe325625c 100644 --- a/accel/tcg/tcg-runtime.h +++ b/accel/tcg/tcg-runtime.h @@ -200,6 +200,26 @@ DEF_HELPER_FLAGS_4(gvec_ussub16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ussub32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ussub64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smin8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smin16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smin32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smin64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_smax8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smax16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smax32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smax64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_umin8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umin16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umin32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umin64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_umax8, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umax16, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umax32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umax64, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(gvec_neg8, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_neg16, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_neg32, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 98556bcf22..545a6eec75 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -136,6 +136,7 @@ typedef enum { #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 #define TCG_TARGET_HAS_sat_vec 0 +#define TCG_TARGET_HAS_minmax_vec 0 #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 44381062e6..7bd7eae672 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -186,6 +186,7 @@ extern bool have_avx2; #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 #define TCG_TARGET_HAS_sat_vec 0 +#define TCG_TARGET_HAS_minmax_vec 0 #define TCG_TARGET_deposit_i32_valid(ofs, len) \ (((ofs) == 0 && (len) == 8) || ((ofs) == 8 && (len) == 8) || \ diff --git a/tcg/tcg-op-gvec.h b/tcg/tcg-op-gvec.h index 2cb447112e..4734eef7de 100644 --- a/tcg/tcg-op-gvec.h +++ b/tcg/tcg-op-gvec.h @@ -234,6 +234,16 @@ void tcg_gen_gvec_usadd(unsigned vece, uint32_t dofs, uint32_t aofs, void tcg_gen_gvec_ussub(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +/* Min/max. */ +void tcg_gen_gvec_smin(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_umin(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_smax(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); +void tcg_gen_gvec_umax(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz); + void tcg_gen_gvec_and(unsigned vece, uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz); void tcg_gen_gvec_or(unsigned vece, uint32_t dofs, uint32_t aofs, diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h index 4a93d730e8..2d98868d8f 100644 --- a/tcg/tcg-op.h +++ b/tcg/tcg-op.h @@ -971,6 +971,10 @@ void tcg_gen_ssadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_usadd_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_sssub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_ussub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_smin_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_umin_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_smax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); +void tcg_gen_umax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b); void tcg_gen_shli_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i); void tcg_gen_shri_vec(unsigned vece, TCGv_vec r, TCGv_vec a, int64_t i); diff --git a/tcg/tcg-opc.h b/tcg/tcg-opc.h index 94b2ed80af..4e0238ad1a 100644 --- a/tcg/tcg-opc.h +++ b/tcg/tcg-opc.h @@ -226,6 +226,10 @@ DEF(ssadd_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) DEF(usadd_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) DEF(sssub_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) DEF(ussub_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_sat_vec)) +DEF(smin_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_minmax_vec)) +DEF(umin_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_minmax_vec)) +DEF(smax_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_minmax_vec)) +DEF(umax_vec, 1, 2, 0, IMPLVEC | IMPL(TCG_TARGET_HAS_minmax_vec)) DEF(and_vec, 1, 2, 0, IMPLVEC) DEF(or_vec, 1, 2, 0, IMPLVEC) diff --git a/tcg/tcg.h b/tcg/tcg.h index 5590dc512a..045c24a357 100644 --- a/tcg/tcg.h +++ b/tcg/tcg.h @@ -184,6 +184,7 @@ typedef uint64_t TCGRegSet; #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_mul_vec 0 #define TCG_TARGET_HAS_sat_vec 0 +#define TCG_TARGET_HAS_minmax_vec 0 #else #define TCG_TARGET_MAYBE_vec 1 #endif diff --git a/accel/tcg/tcg-runtime-gvec.c b/accel/tcg/tcg-runtime-gvec.c index d1802467d5..9358749741 100644 --- a/accel/tcg/tcg-runtime-gvec.c +++ b/accel/tcg/tcg-runtime-gvec.c @@ -1028,3 +1028,227 @@ void HELPER(gvec_ussub64)(void *d, void *a, void *b, uint32_t desc) } clear_high(d, oprsz, desc); } + +void HELPER(gvec_smin8)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int8_t)) { + int8_t aa = *(int8_t *)(a + i); + int8_t bb = *(int8_t *)(b + i); + int8_t dd = aa < bb ? aa : bb; + *(int8_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smin16)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int16_t)) { + int16_t aa = *(int16_t *)(a + i); + int16_t bb = *(int16_t *)(b + i); + int16_t dd = aa < bb ? aa : bb; + *(int16_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smin32)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int32_t)) { + int32_t aa = *(int32_t *)(a + i); + int32_t bb = *(int32_t *)(b + i); + int32_t dd = aa < bb ? aa : bb; + *(int32_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smin64)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int64_t)) { + int64_t aa = *(int64_t *)(a + i); + int64_t bb = *(int64_t *)(b + i); + int64_t dd = aa < bb ? aa : bb; + *(int64_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smax8)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int8_t)) { + int8_t aa = *(int8_t *)(a + i); + int8_t bb = *(int8_t *)(b + i); + int8_t dd = aa > bb ? aa : bb; + *(int8_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smax16)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int16_t)) { + int16_t aa = *(int16_t *)(a + i); + int16_t bb = *(int16_t *)(b + i); + int16_t dd = aa > bb ? aa : bb; + *(int16_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smax32)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int32_t)) { + int32_t aa = *(int32_t *)(a + i); + int32_t bb = *(int32_t *)(b + i); + int32_t dd = aa > bb ? aa : bb; + *(int32_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_smax64)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(int64_t)) { + int64_t aa = *(int64_t *)(a + i); + int64_t bb = *(int64_t *)(b + i); + int64_t dd = aa > bb ? aa : bb; + *(int64_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umin8)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint8_t)) { + uint8_t aa = *(uint8_t *)(a + i); + uint8_t bb = *(uint8_t *)(b + i); + uint8_t dd = aa < bb ? aa : bb; + *(uint8_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umin16)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint16_t)) { + uint16_t aa = *(uint16_t *)(a + i); + uint16_t bb = *(uint16_t *)(b + i); + uint16_t dd = aa < bb ? aa : bb; + *(uint16_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umin32)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint32_t)) { + uint32_t aa = *(uint32_t *)(a + i); + uint32_t bb = *(uint32_t *)(b + i); + uint32_t dd = aa < bb ? aa : bb; + *(uint32_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umin64)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint64_t)) { + uint64_t aa = *(uint64_t *)(a + i); + uint64_t bb = *(uint64_t *)(b + i); + uint64_t dd = aa < bb ? aa : bb; + *(uint64_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umax8)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint8_t)) { + uint8_t aa = *(uint8_t *)(a + i); + uint8_t bb = *(uint8_t *)(b + i); + uint8_t dd = aa > bb ? aa : bb; + *(uint8_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umax16)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint16_t)) { + uint16_t aa = *(uint16_t *)(a + i); + uint16_t bb = *(uint16_t *)(b + i); + uint16_t dd = aa > bb ? aa : bb; + *(uint16_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umax32)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint32_t)) { + uint32_t aa = *(uint32_t *)(a + i); + uint32_t bb = *(uint32_t *)(b + i); + uint32_t dd = aa > bb ? aa : bb; + *(uint32_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} + +void HELPER(gvec_umax64)(void *d, void *a, void *b, uint32_t desc) +{ + intptr_t oprsz = simd_oprsz(desc); + intptr_t i; + + for (i = 0; i < oprsz; i += sizeof(uint64_t)) { + uint64_t aa = *(uint64_t *)(a + i); + uint64_t bb = *(uint64_t *)(b + i); + uint64_t dd = aa > bb ? aa : bb; + *(uint64_t *)(d + i) = dd; + } + clear_high(d, oprsz, desc); +} diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index 0a33f51065..3ee44fcb75 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -1810,6 +1810,114 @@ void tcg_gen_gvec_ussub(unsigned vece, uint32_t dofs, uint32_t aofs, tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); } +void tcg_gen_gvec_smin(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g[4] = { + { .fniv = tcg_gen_smin_vec, + .fno = gen_helper_gvec_smin8, + .opc = INDEX_op_smin_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_smin_vec, + .fno = gen_helper_gvec_smin16, + .opc = INDEX_op_smin_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_smin_i32, + .fniv = tcg_gen_smin_vec, + .fno = gen_helper_gvec_smin32, + .opc = INDEX_op_smin_vec, + .vece = MO_32 }, + { .fni8 = tcg_gen_smin_i64, + .fniv = tcg_gen_smin_vec, + .fno = gen_helper_gvec_smin64, + .opc = INDEX_op_smin_vec, + .vece = MO_64 } + }; + tcg_debug_assert(vece <= MO_64); + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); +} + +void tcg_gen_gvec_umin(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g[4] = { + { .fniv = tcg_gen_umin_vec, + .fno = gen_helper_gvec_umin8, + .opc = INDEX_op_umin_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_umin_vec, + .fno = gen_helper_gvec_umin16, + .opc = INDEX_op_umin_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_umin_i32, + .fniv = tcg_gen_umin_vec, + .fno = gen_helper_gvec_umin32, + .opc = INDEX_op_umin_vec, + .vece = MO_32 }, + { .fni8 = tcg_gen_umin_i64, + .fniv = tcg_gen_umin_vec, + .fno = gen_helper_gvec_umin64, + .opc = INDEX_op_umin_vec, + .vece = MO_64 } + }; + tcg_debug_assert(vece <= MO_64); + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); +} + +void tcg_gen_gvec_smax(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g[4] = { + { .fniv = tcg_gen_smax_vec, + .fno = gen_helper_gvec_smax8, + .opc = INDEX_op_smax_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_smax_vec, + .fno = gen_helper_gvec_smax16, + .opc = INDEX_op_smax_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_smax_i32, + .fniv = tcg_gen_smax_vec, + .fno = gen_helper_gvec_smax32, + .opc = INDEX_op_smax_vec, + .vece = MO_32 }, + { .fni8 = tcg_gen_smax_i64, + .fniv = tcg_gen_smax_vec, + .fno = gen_helper_gvec_smax64, + .opc = INDEX_op_smax_vec, + .vece = MO_64 } + }; + tcg_debug_assert(vece <= MO_64); + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); +} + +void tcg_gen_gvec_umax(unsigned vece, uint32_t dofs, uint32_t aofs, + uint32_t bofs, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen3 g[4] = { + { .fniv = tcg_gen_umax_vec, + .fno = gen_helper_gvec_umax8, + .opc = INDEX_op_umax_vec, + .vece = MO_8 }, + { .fniv = tcg_gen_umax_vec, + .fno = gen_helper_gvec_umax16, + .opc = INDEX_op_umax_vec, + .vece = MO_16 }, + { .fni4 = tcg_gen_umax_i32, + .fniv = tcg_gen_umax_vec, + .fno = gen_helper_gvec_umax32, + .opc = INDEX_op_umax_vec, + .vece = MO_32 }, + { .fni8 = tcg_gen_umax_i64, + .fniv = tcg_gen_umax_vec, + .fno = gen_helper_gvec_umax64, + .opc = INDEX_op_umax_vec, + .vece = MO_64 } + }; + tcg_debug_assert(vece <= MO_64); + tcg_gen_gvec_3(dofs, aofs, bofs, oprsz, maxsz, &g[vece]); +} + /* Perform a vector negation using normal negation and a mask. Compare gen_subv_mask above. */ static void gen_negv_mask(TCGv_i64 d, TCGv_i64 b, TCGv_i64 m) diff --git a/tcg/tcg-op-vec.c b/tcg/tcg-op-vec.c index 675aa09258..36f35022ac 100644 --- a/tcg/tcg-op-vec.c +++ b/tcg/tcg-op-vec.c @@ -433,3 +433,23 @@ void tcg_gen_ussub_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) { do_op3(vece, r, a, b, INDEX_op_ussub_vec); } + +void tcg_gen_smin_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_smin_vec); +} + +void tcg_gen_umin_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_umin_vec); +} + +void tcg_gen_smax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_smax_vec); +} + +void tcg_gen_umax_vec(unsigned vece, TCGv_vec r, TCGv_vec a, TCGv_vec b) +{ + do_op3(vece, r, a, b, INDEX_op_umax_vec); +} diff --git a/tcg/tcg.c b/tcg/tcg.c index 6363f0cb29..20a5d8f315 100644 --- a/tcg/tcg.c +++ b/tcg/tcg.c @@ -1612,6 +1612,11 @@ bool tcg_op_supported(TCGOpcode op) case INDEX_op_sssub_vec: case INDEX_op_ussub_vec: return have_vec && TCG_TARGET_HAS_sat_vec; + case INDEX_op_smin_vec: + case INDEX_op_umin_vec: + case INDEX_op_smax_vec: + case INDEX_op_umax_vec: + return have_vec && TCG_TARGET_HAS_minmax_vec; default: tcg_debug_assert(op > INDEX_op_last_generic && op < NB_OPS); diff --git a/tcg/README b/tcg/README index dd9fd9c86c..603f4df659 100644 --- a/tcg/README +++ b/tcg/README @@ -554,6 +554,16 @@ E.g. VECL=1 -> 64 << 1 -> v128, and VECE=2 -> 1 << 2 -> i32. Similarly, v0 = -v1. +* smin_vec: +* umin_vec: + + Similarly, v0 = MIN(v1, v2), for signed and unsigned element types. + +* smax_vec: +* umax_vec: + + Similarly, v0 = MAX(v1, v2), for signed and unsigned element types. + * ssadd_vec: * sssub_vec: * usadd_vec: From patchwork Mon Jan 28 15:58:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156779 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3609665jaa; Mon, 28 Jan 2019 08:17:57 -0800 (PST) X-Google-Smtp-Source: ALg8bN5lgJ0fSPS3JoMGCDE/pc/pPl68wkL86itW5n0BikTEFCOIo8jzx0XkebOTYC1WA8ysJ1/I X-Received: by 2002:a1c:9c15:: with SMTP id f21mr16637829wme.94.1548692277639; Mon, 28 Jan 2019 08:17:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692277; cv=none; d=google.com; s=arc-20160816; b=lPPJ5yVJKOH6lfpx2zFxqNapOsWJF7q9a8imXAZhi1QnCyTTqf6Dfn/RLCaQprZKIz pdR3dn09ynqTCvYk08hmk8S3VeMsx2c7AfZIj/R8VeiQzd7VkLhN7ulFaGz5nDRWTfLu /007ek6wYeNPBUp039wLnhvqYzkfCJiPsa3RDQsGGpq+uDQMzHSzahQm+N10+UewuJ8n BwJeJg/jgLE7C5oTEEus6mXRgaPktMONjC5S5yXdtQkWsqZZe8tod+JURCN6afY5iQDo zfGgffZzQlPZG8oC/aSbcFwFm+/VJtZRZlvSaDWPmJ4msaXeOLyi0WlyAryDerHGyVrw lCZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=v8SxN2gYQGPbndIagmg7wvgmLMyWGfk0uvvElxIvszQ=; b=CNl15T0yEtGzFPRHg2UtiA2ebLfCUxMY9jPvuUxOhKCQMGOOnLKoJpqZGOSt61NhBL VinwxFBcahvzzMVsZBoSz3csOQv4MNnU3QfzFwqxCPjDR414Md2oAnDo8RNwidiHaZOv TzGZoOoGyinUhTqKhplaTY7qz9wc1LpiLigmOMG6cceizetp709SrgSe9LVe6ZgKAvRq oJ3yT3ISFvSpFE/DK5cqOSmyTPIWkQRIr4aP/04Qfg3Gtp9q42O96MFvGxOn4opgDFOc FjMyxKZ4DgHf1p65aDaZ26nHuE05HBXZlNvXQqtsjoKWnkK0G7eRzOUHXZqp8FN6eqi2 tB9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Rq8qHZZm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m8si60942699wrx.128.2019.01.28.08.17.57 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:17:57 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Rq8qHZZm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34404 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9bg-0001TQ-C4 for patch@linaro.org; Mon, 28 Jan 2019 11:17:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33413) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jj-0004sV-B7 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:25 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jh-0007jc-DD for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:23 -0500 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]:37800) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jh-0007jK-4m for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:21 -0500 Received: by mail-pg1-x534.google.com with SMTP id c25so7390876pgb.4 for ; Mon, 28 Jan 2019 07:59:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=v8SxN2gYQGPbndIagmg7wvgmLMyWGfk0uvvElxIvszQ=; b=Rq8qHZZmKj8AUZYxM3cEjAJCsRNf2aiar05XZv1MKm3XC2Hx4F3gj3Z4Bx3I4xlLI1 mTuVq9vQV4t3LyMyphdr2sIfHI8nTt6lfia15yLOya+EBylZKPyAOmF1pdqeQfwAtWzn 79TkPkIYfRjFiwJDkPZOhbLhSt5jGfCutYSVE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=v8SxN2gYQGPbndIagmg7wvgmLMyWGfk0uvvElxIvszQ=; b=Z0X19kD2CFpdCvwq83ZrJk4IpswVBCWyAiZZHDGO17fGPFyhR8ceR8sOFqj1RIsttx jA69WdMQBbDvRBLVrOP+dvt6quYc6wMdErO+D9sNYVguYtsYCjr/rbMjpEbyAG2u+Oj7 b46KrYVVyOziVDgUftXx1G0nAfYnHjwr2XTvi6jRMBzDlZApl5CY+t5SPEBF7oij5+RC VRmWO3FpPGJyXXpqmcQBQetdr5pxnLZAlc7dNg+dpgVUS843e90UGttqKsLuqP+fOjw+ FaJDq71gd+MLV40q9ApZe2pqp0SgrCgSxjWGf4mc4Y2cT826wDFSGRmKlWFPbRbkEtba b/Fw== X-Gm-Message-State: AJcUukfRKET21gZy3KXS49iT/LYObCxDCXw4DcBK8HIoVMxNMaj27Sjv k4vpqG7CrZ8H8u1mn0KRmh6tufTyZuM= X-Received: by 2002:a65:63d3:: with SMTP id n19mr20564735pgv.179.1548691159562; Mon, 28 Jan 2019 07:59:19 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.18 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:18 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:50 -0800 Message-Id: <20190128155907.20607-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::534 Subject: [Qemu-devel] [PULL 06/23] tcg/i386: Split subroutines out of tcg_expand_vec_op X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This routine was becoming too large. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 459 +++++++++++++++++++------------------- 1 file changed, 232 insertions(+), 227 deletions(-) -- 2.17.2 diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 1b4e3b80e1..6f4c16326f 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -3079,253 +3079,258 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece) } } +static void expand_vec_shi(TCGType type, unsigned vece, bool shr, + TCGv_vec v0, TCGv_vec v1, TCGArg imm) +{ + TCGv_vec t1, t2; + + tcg_debug_assert(vece == MO_8); + + t1 = tcg_temp_new_vec(type); + t2 = tcg_temp_new_vec(type); + + /* Unpack to W, shift, and repack. Tricky bits: + (1) Use punpck*bw x,x to produce DDCCBBAA, + i.e. duplicate in other half of the 16-bit lane. + (2) For right-shift, add 8 so that the high half of + the lane becomes zero. For left-shift, we must + shift up and down again. + (3) Step 2 leaves high half zero such that PACKUSWB + (pack with unsigned saturation) does not modify + the quantity. */ + vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, + tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(v1)); + vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, + tcgv_vec_arg(t2), tcgv_vec_arg(v1), tcgv_vec_arg(v1)); + + if (shr) { + tcg_gen_shri_vec(MO_16, t1, t1, imm + 8); + tcg_gen_shri_vec(MO_16, t2, t2, imm + 8); + } else { + tcg_gen_shli_vec(MO_16, t1, t1, imm + 8); + tcg_gen_shli_vec(MO_16, t2, t2, imm + 8); + tcg_gen_shri_vec(MO_16, t1, t1, 8); + tcg_gen_shri_vec(MO_16, t2, t2, 8); + } + + vec_gen_3(INDEX_op_x86_packus_vec, type, MO_8, + tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(t2)); + tcg_temp_free_vec(t1); + tcg_temp_free_vec(t2); +} + +static void expand_vec_sari(TCGType type, unsigned vece, + TCGv_vec v0, TCGv_vec v1, TCGArg imm) +{ + TCGv_vec t1, t2; + + switch (vece) { + case MO_8: + /* Unpack to W, shift, and repack, as in expand_vec_shi. */ + t1 = tcg_temp_new_vec(type); + t2 = tcg_temp_new_vec(type); + vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, + tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(v1)); + vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, + tcgv_vec_arg(t2), tcgv_vec_arg(v1), tcgv_vec_arg(v1)); + tcg_gen_sari_vec(MO_16, t1, t1, imm + 8); + tcg_gen_sari_vec(MO_16, t2, t2, imm + 8); + vec_gen_3(INDEX_op_x86_packss_vec, type, MO_8, + tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(t2)); + tcg_temp_free_vec(t1); + tcg_temp_free_vec(t2); + break; + + case MO_64: + if (imm <= 32) { + /* We can emulate a small sign extend by performing an arithmetic + * 32-bit shift and overwriting the high half of a 64-bit logical + * shift (note that the ISA says shift of 32 is valid). + */ + t1 = tcg_temp_new_vec(type); + tcg_gen_sari_vec(MO_32, t1, v1, imm); + tcg_gen_shri_vec(MO_64, v0, v1, imm); + vec_gen_4(INDEX_op_x86_blend_vec, type, MO_32, + tcgv_vec_arg(v0), tcgv_vec_arg(v0), + tcgv_vec_arg(t1), 0xaa); + tcg_temp_free_vec(t1); + } else { + /* Otherwise we will need to use a compare vs 0 to produce + * the sign-extend, shift and merge. + */ + t1 = tcg_const_zeros_vec(type); + tcg_gen_cmp_vec(TCG_COND_GT, MO_64, t1, t1, v1); + tcg_gen_shri_vec(MO_64, v0, v1, imm); + tcg_gen_shli_vec(MO_64, t1, t1, 64 - imm); + tcg_gen_or_vec(MO_64, v0, v0, t1); + tcg_temp_free_vec(t1); + } + break; + + default: + g_assert_not_reached(); + } +} + +static void expand_vec_mul(TCGType type, unsigned vece, + TCGv_vec v0, TCGv_vec v1, TCGv_vec v2) +{ + TCGv_vec t1, t2, t3, t4; + + tcg_debug_assert(vece == MO_8); + + /* + * Unpack v1 bytes to words, 0 | x. + * Unpack v2 bytes to words, y | 0. + * This leaves the 8-bit result, x * y, with 8 bits of right padding. + * Shift logical right by 8 bits to clear the high 8 bytes before + * using an unsigned saturated pack. + * + * The difference between the V64, V128 and V256 cases is merely how + * we distribute the expansion between temporaries. + */ + switch (type) { + case TCG_TYPE_V64: + t1 = tcg_temp_new_vec(TCG_TYPE_V128); + t2 = tcg_temp_new_vec(TCG_TYPE_V128); + tcg_gen_dup16i_vec(t2, 0); + vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, + tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(t2)); + vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, + tcgv_vec_arg(t2), tcgv_vec_arg(t2), tcgv_vec_arg(v2)); + tcg_gen_mul_vec(MO_16, t1, t1, t2); + tcg_gen_shri_vec(MO_16, t1, t1, 8); + vec_gen_3(INDEX_op_x86_packus_vec, TCG_TYPE_V128, MO_8, + tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(t1)); + tcg_temp_free_vec(t1); + tcg_temp_free_vec(t2); + break; + + case TCG_TYPE_V128: + case TCG_TYPE_V256: + t1 = tcg_temp_new_vec(type); + t2 = tcg_temp_new_vec(type); + t3 = tcg_temp_new_vec(type); + t4 = tcg_temp_new_vec(type); + tcg_gen_dup16i_vec(t4, 0); + vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, + tcgv_vec_arg(t1), tcgv_vec_arg(v1), tcgv_vec_arg(t4)); + vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, + tcgv_vec_arg(t2), tcgv_vec_arg(t4), tcgv_vec_arg(v2)); + vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, + tcgv_vec_arg(t3), tcgv_vec_arg(v1), tcgv_vec_arg(t4)); + vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, + tcgv_vec_arg(t4), tcgv_vec_arg(t4), tcgv_vec_arg(v2)); + tcg_gen_mul_vec(MO_16, t1, t1, t2); + tcg_gen_mul_vec(MO_16, t3, t3, t4); + tcg_gen_shri_vec(MO_16, t1, t1, 8); + tcg_gen_shri_vec(MO_16, t3, t3, 8); + vec_gen_3(INDEX_op_x86_packus_vec, type, MO_8, + tcgv_vec_arg(v0), tcgv_vec_arg(t1), tcgv_vec_arg(t3)); + tcg_temp_free_vec(t1); + tcg_temp_free_vec(t2); + tcg_temp_free_vec(t3); + tcg_temp_free_vec(t4); + break; + + default: + g_assert_not_reached(); + } +} + +static void expand_vec_cmp(TCGType type, unsigned vece, TCGv_vec v0, + TCGv_vec v1, TCGv_vec v2, TCGCond cond) +{ + enum { + NEED_SWAP = 1, + NEED_INV = 2, + NEED_BIAS = 4 + }; + static const uint8_t fixups[16] = { + [0 ... 15] = -1, + [TCG_COND_EQ] = 0, + [TCG_COND_NE] = NEED_INV, + [TCG_COND_GT] = 0, + [TCG_COND_LT] = NEED_SWAP, + [TCG_COND_LE] = NEED_INV, + [TCG_COND_GE] = NEED_SWAP | NEED_INV, + [TCG_COND_GTU] = NEED_BIAS, + [TCG_COND_LTU] = NEED_BIAS | NEED_SWAP, + [TCG_COND_LEU] = NEED_BIAS | NEED_INV, + [TCG_COND_GEU] = NEED_BIAS | NEED_SWAP | NEED_INV, + }; + TCGv_vec t1, t2; + uint8_t fixup; + + fixup = fixups[cond & 15]; + tcg_debug_assert(fixup != 0xff); + + if (fixup & NEED_INV) { + cond = tcg_invert_cond(cond); + } + if (fixup & NEED_SWAP) { + t1 = v1, v1 = v2, v2 = t1; + cond = tcg_swap_cond(cond); + } + + t1 = t2 = NULL; + if (fixup & NEED_BIAS) { + t1 = tcg_temp_new_vec(type); + t2 = tcg_temp_new_vec(type); + tcg_gen_dupi_vec(vece, t2, 1ull << ((8 << vece) - 1)); + tcg_gen_sub_vec(vece, t1, v1, t2); + tcg_gen_sub_vec(vece, t2, v2, t2); + v1 = t1; + v2 = t2; + cond = tcg_signed_cond(cond); + } + + tcg_debug_assert(cond == TCG_COND_EQ || cond == TCG_COND_GT); + /* Expand directly; do not recurse. */ + vec_gen_4(INDEX_op_cmp_vec, type, vece, + tcgv_vec_arg(v0), tcgv_vec_arg(v1), tcgv_vec_arg(v2), cond); + + if (t1) { + tcg_temp_free_vec(t1); + if (t2) { + tcg_temp_free_vec(t2); + } + } + if (fixup & NEED_INV) { + tcg_gen_not_vec(vece, v0, v0); + } +} + void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece, TCGArg a0, ...) { va_list va; - TCGArg a1, a2; - TCGv_vec v0, t1, t2, t3, t4; + TCGArg a2; + TCGv_vec v0, v1, v2; va_start(va, a0); v0 = temp_tcgv_vec(arg_temp(a0)); + v1 = temp_tcgv_vec(arg_temp(va_arg(va, TCGArg))); + a2 = va_arg(va, TCGArg); switch (opc) { case INDEX_op_shli_vec: case INDEX_op_shri_vec: - tcg_debug_assert(vece == MO_8); - a1 = va_arg(va, TCGArg); - a2 = va_arg(va, TCGArg); - /* Unpack to W, shift, and repack. Tricky bits: - (1) Use punpck*bw x,x to produce DDCCBBAA, - i.e. duplicate in other half of the 16-bit lane. - (2) For right-shift, add 8 so that the high half of - the lane becomes zero. For left-shift, we must - shift up and down again. - (3) Step 2 leaves high half zero such that PACKUSWB - (pack with unsigned saturation) does not modify - the quantity. */ - t1 = tcg_temp_new_vec(type); - t2 = tcg_temp_new_vec(type); - vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, - tcgv_vec_arg(t1), a1, a1); - vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, - tcgv_vec_arg(t2), a1, a1); - if (opc == INDEX_op_shri_vec) { - vec_gen_3(INDEX_op_shri_vec, type, MO_16, - tcgv_vec_arg(t1), tcgv_vec_arg(t1), a2 + 8); - vec_gen_3(INDEX_op_shri_vec, type, MO_16, - tcgv_vec_arg(t2), tcgv_vec_arg(t2), a2 + 8); - } else { - vec_gen_3(INDEX_op_shli_vec, type, MO_16, - tcgv_vec_arg(t1), tcgv_vec_arg(t1), a2 + 8); - vec_gen_3(INDEX_op_shli_vec, type, MO_16, - tcgv_vec_arg(t2), tcgv_vec_arg(t2), a2 + 8); - vec_gen_3(INDEX_op_shri_vec, type, MO_16, - tcgv_vec_arg(t1), tcgv_vec_arg(t1), 8); - vec_gen_3(INDEX_op_shri_vec, type, MO_16, - tcgv_vec_arg(t2), tcgv_vec_arg(t2), 8); - } - vec_gen_3(INDEX_op_x86_packus_vec, type, MO_8, - a0, tcgv_vec_arg(t1), tcgv_vec_arg(t2)); - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); + expand_vec_shi(type, vece, opc == INDEX_op_shri_vec, v0, v1, a2); break; case INDEX_op_sari_vec: - a1 = va_arg(va, TCGArg); - a2 = va_arg(va, TCGArg); - if (vece == MO_8) { - /* Unpack to W, shift, and repack, as above. */ - t1 = tcg_temp_new_vec(type); - t2 = tcg_temp_new_vec(type); - vec_gen_3(INDEX_op_x86_punpckl_vec, type, MO_8, - tcgv_vec_arg(t1), a1, a1); - vec_gen_3(INDEX_op_x86_punpckh_vec, type, MO_8, - tcgv_vec_arg(t2), a1, a1); - vec_gen_3(INDEX_op_sari_vec, type, MO_16, - tcgv_vec_arg(t1), tcgv_vec_arg(t1), a2 + 8); - vec_gen_3(INDEX_op_sari_vec, type, MO_16, - tcgv_vec_arg(t2), tcgv_vec_arg(t2), a2 + 8); - vec_gen_3(INDEX_op_x86_packss_vec, type, MO_8, - a0, tcgv_vec_arg(t1), tcgv_vec_arg(t2)); - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); - break; - } - tcg_debug_assert(vece == MO_64); - /* MO_64: If the shift is <= 32, we can emulate the sign extend by - performing an arithmetic 32-bit shift and overwriting the high - half of the result (note that the ISA says shift of 32 is valid). */ - if (a2 <= 32) { - t1 = tcg_temp_new_vec(type); - vec_gen_3(INDEX_op_sari_vec, type, MO_32, tcgv_vec_arg(t1), a1, a2); - vec_gen_3(INDEX_op_shri_vec, type, MO_64, a0, a1, a2); - vec_gen_4(INDEX_op_x86_blend_vec, type, MO_32, - a0, a0, tcgv_vec_arg(t1), 0xaa); - tcg_temp_free_vec(t1); - break; - } - /* Otherwise we will need to use a compare vs 0 to produce the - sign-extend, shift and merge. */ - t1 = tcg_temp_new_vec(type); - t2 = tcg_const_zeros_vec(type); - vec_gen_4(INDEX_op_cmp_vec, type, MO_64, - tcgv_vec_arg(t1), tcgv_vec_arg(t2), a1, TCG_COND_GT); - tcg_temp_free_vec(t2); - vec_gen_3(INDEX_op_shri_vec, type, MO_64, a0, a1, a2); - vec_gen_3(INDEX_op_shli_vec, type, MO_64, - tcgv_vec_arg(t1), tcgv_vec_arg(t1), 64 - a2); - vec_gen_3(INDEX_op_or_vec, type, MO_64, a0, a0, tcgv_vec_arg(t1)); - tcg_temp_free_vec(t1); + expand_vec_sari(type, vece, v0, v1, a2); break; case INDEX_op_mul_vec: - tcg_debug_assert(vece == MO_8); - a1 = va_arg(va, TCGArg); - a2 = va_arg(va, TCGArg); - switch (type) { - case TCG_TYPE_V64: - t1 = tcg_temp_new_vec(TCG_TYPE_V128); - t2 = tcg_temp_new_vec(TCG_TYPE_V128); - tcg_gen_dup16i_vec(t2, 0); - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t1), a1, tcgv_vec_arg(t2)); - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t2), tcgv_vec_arg(t2), a2); - tcg_gen_mul_vec(MO_16, t1, t1, t2); - tcg_gen_shri_vec(MO_16, t1, t1, 8); - vec_gen_3(INDEX_op_x86_packus_vec, TCG_TYPE_V128, MO_8, - a0, tcgv_vec_arg(t1), tcgv_vec_arg(t1)); - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); - break; - - case TCG_TYPE_V128: - t1 = tcg_temp_new_vec(TCG_TYPE_V128); - t2 = tcg_temp_new_vec(TCG_TYPE_V128); - t3 = tcg_temp_new_vec(TCG_TYPE_V128); - t4 = tcg_temp_new_vec(TCG_TYPE_V128); - tcg_gen_dup16i_vec(t4, 0); - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t1), a1, tcgv_vec_arg(t4)); - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t2), tcgv_vec_arg(t4), a2); - vec_gen_3(INDEX_op_x86_punpckh_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t3), a1, tcgv_vec_arg(t4)); - vec_gen_3(INDEX_op_x86_punpckh_vec, TCG_TYPE_V128, MO_8, - tcgv_vec_arg(t4), tcgv_vec_arg(t4), a2); - tcg_gen_mul_vec(MO_16, t1, t1, t2); - tcg_gen_mul_vec(MO_16, t3, t3, t4); - tcg_gen_shri_vec(MO_16, t1, t1, 8); - tcg_gen_shri_vec(MO_16, t3, t3, 8); - vec_gen_3(INDEX_op_x86_packus_vec, TCG_TYPE_V128, MO_8, - a0, tcgv_vec_arg(t1), tcgv_vec_arg(t3)); - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); - tcg_temp_free_vec(t3); - tcg_temp_free_vec(t4); - break; - - case TCG_TYPE_V256: - t1 = tcg_temp_new_vec(TCG_TYPE_V256); - t2 = tcg_temp_new_vec(TCG_TYPE_V256); - t3 = tcg_temp_new_vec(TCG_TYPE_V256); - t4 = tcg_temp_new_vec(TCG_TYPE_V256); - tcg_gen_dup16i_vec(t4, 0); - /* a1: A[0-7] ... D[0-7]; a2: W[0-7] ... Z[0-7] - t1: extends of B[0-7], D[0-7] - t2: extends of X[0-7], Z[0-7] - t3: extends of A[0-7], C[0-7] - t4: extends of W[0-7], Y[0-7]. */ - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V256, MO_8, - tcgv_vec_arg(t1), a1, tcgv_vec_arg(t4)); - vec_gen_3(INDEX_op_x86_punpckl_vec, TCG_TYPE_V256, MO_8, - tcgv_vec_arg(t2), tcgv_vec_arg(t4), a2); - vec_gen_3(INDEX_op_x86_punpckh_vec, TCG_TYPE_V256, MO_8, - tcgv_vec_arg(t3), a1, tcgv_vec_arg(t4)); - vec_gen_3(INDEX_op_x86_punpckh_vec, TCG_TYPE_V256, MO_8, - tcgv_vec_arg(t4), tcgv_vec_arg(t4), a2); - /* t1: BX DZ; t2: AW CY. */ - tcg_gen_mul_vec(MO_16, t1, t1, t2); - tcg_gen_mul_vec(MO_16, t3, t3, t4); - tcg_gen_shri_vec(MO_16, t1, t1, 8); - tcg_gen_shri_vec(MO_16, t3, t3, 8); - /* a0: AW BX CY DZ. */ - vec_gen_3(INDEX_op_x86_packus_vec, TCG_TYPE_V256, MO_8, - a0, tcgv_vec_arg(t1), tcgv_vec_arg(t3)); - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); - tcg_temp_free_vec(t3); - tcg_temp_free_vec(t4); - break; - - default: - g_assert_not_reached(); - } + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_mul(type, vece, v0, v1, v2); break; case INDEX_op_cmp_vec: - { - enum { - NEED_SWAP = 1, - NEED_INV = 2, - NEED_BIAS = 4 - }; - static const uint8_t fixups[16] = { - [0 ... 15] = -1, - [TCG_COND_EQ] = 0, - [TCG_COND_NE] = NEED_INV, - [TCG_COND_GT] = 0, - [TCG_COND_LT] = NEED_SWAP, - [TCG_COND_LE] = NEED_INV, - [TCG_COND_GE] = NEED_SWAP | NEED_INV, - [TCG_COND_GTU] = NEED_BIAS, - [TCG_COND_LTU] = NEED_BIAS | NEED_SWAP, - [TCG_COND_LEU] = NEED_BIAS | NEED_INV, - [TCG_COND_GEU] = NEED_BIAS | NEED_SWAP | NEED_INV, - }; - - TCGCond cond; - uint8_t fixup; - - a1 = va_arg(va, TCGArg); - a2 = va_arg(va, TCGArg); - cond = va_arg(va, TCGArg); - fixup = fixups[cond & 15]; - tcg_debug_assert(fixup != 0xff); - - if (fixup & NEED_INV) { - cond = tcg_invert_cond(cond); - } - if (fixup & NEED_SWAP) { - TCGArg t; - t = a1, a1 = a2, a2 = t; - cond = tcg_swap_cond(cond); - } - - t1 = t2 = NULL; - if (fixup & NEED_BIAS) { - t1 = tcg_temp_new_vec(type); - t2 = tcg_temp_new_vec(type); - tcg_gen_dupi_vec(vece, t2, 1ull << ((8 << vece) - 1)); - tcg_gen_sub_vec(vece, t1, temp_tcgv_vec(arg_temp(a1)), t2); - tcg_gen_sub_vec(vece, t2, temp_tcgv_vec(arg_temp(a2)), t2); - a1 = tcgv_vec_arg(t1); - a2 = tcgv_vec_arg(t2); - cond = tcg_signed_cond(cond); - } - - tcg_debug_assert(cond == TCG_COND_EQ || cond == TCG_COND_GT); - vec_gen_4(INDEX_op_cmp_vec, type, vece, a0, a1, a2, cond); - - if (fixup & NEED_BIAS) { - tcg_temp_free_vec(t1); - tcg_temp_free_vec(t2); - } - if (fixup & NEED_INV) { - tcg_gen_not_vec(vece, v0, v0); - } - } + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_cmp(type, vece, v0, v1, v2, va_arg(va, TCGArg)); break; default: From patchwork Mon Jan 28 15:58:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156778 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3535910oct; Mon, 28 Jan 2019 08:14:43 -0800 (PST) X-Google-Smtp-Source: ALg8bN4P0pwOGcFBQodg+wjfnMwCJbUeteEaMEXfuSyMq11MbeCIQ0psueixEtnKjXsHWMXHU0wP X-Received: by 2002:a1c:7dd7:: with SMTP id y206mr17650424wmc.50.1548692083708; Mon, 28 Jan 2019 08:14:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692083; cv=none; d=google.com; s=arc-20160816; b=aYsYh4oscq+JZaLOntLemro2cyjZP6bMXxlOLV60IqxR7qQqLqQahwcyKkNU3fhTHe cAq3JincggLsBQTl9j/ugkETwsEF/6bfcz8dy3Kq90jLVN6rgA1/TU8T6R3KKVrOdFSa yl+uvRV9/I77tWXo5fbR+7Lso/EhgeBec1HjrHwhoeFj7DN6rjlfL7eVLVrI18/QPJ+O IExIf9T09/CgFhVcj5ySFN3zTcxbu8IFzBTWuaS3e58KCfI5BAH5EGD0geplxQslm/vo IDUkmg08y2yxXup6u2QHpxDsRlEhAYnOC6o6bxYK4Qxjf1yKVygh7a/b/HZdaYJHpFNu ihFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=TqlRQyNLGQUtXCq+DygG9z0Ggqd2FQuVBMC93AMwhZk=; b=p2Y1fNgUwbBHJG+8w3Q/TwoXnjM7zWZwf1lIVos3j8lCslmm7etkjxfzacepnlFxCv 4vyb59PVdF7erinYfCqXTE+W/FtATfIopCePFl+szaidQOPEOD00rWtDGpyb1vAqABJ/ ovgwBCoJ1CMulTeSerm8/V6XqTZD86v4UhcGFpJfPXvsibRGc4KbS3E2gXeUFhtqHjhe uKGBwby9XN4b9zNGOfO9e3XClvjNDqeQ9I+Lp/UjyAAKE0zhwmwCkaSmiyNHyMBypjkc R3ULBNTTzAAn9DDGcQkh5ejaNn7enDrIrkT8uvOgPTJaCoRJOx6Vlkh1cnaq/etIr3e8 NYYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=dHE4EOHR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w75si7312750wmf.95.2019.01.28.08.14.43 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:14:43 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=dHE4EOHR; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34347 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9YX-0007QN-Nb for patch@linaro.org; Mon, 28 Jan 2019 11:14:41 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33418) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jj-0004so-I4 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:24 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Ji-0007k3-HG for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:23 -0500 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]:45044) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Ji-0007jl-C2 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:22 -0500 Received: by mail-pg1-x531.google.com with SMTP id t13so7370193pgr.11 for ; Mon, 28 Jan 2019 07:59:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=TqlRQyNLGQUtXCq+DygG9z0Ggqd2FQuVBMC93AMwhZk=; b=dHE4EOHRX8bUaF/vz2SDQZ/5+nBpKPGLolTFmRTubzYryYImF0kMxGArJoF46kNpHh vtplG45+1vYafXKTZ7XM989Iu/wySprjTMntHvzBl8+Z4H+9FnhBEfMEKweIvlhR5mhO AfLxwvhFq8drkgMU9uIiFFfgMypufdrekh4kk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=TqlRQyNLGQUtXCq+DygG9z0Ggqd2FQuVBMC93AMwhZk=; b=APSkwq1TufG0yizkURpCbvz72eD2ZPM5wIo8gl6hJ+CmieLJXUZx53c/2CEw7Bzlx6 ZeagEWaZZeO7Y+5V5FBYwlanYm19IvapnPsDmKAeTIXdwt/E4syUj4INjFanj6xMhnpD xkf7pVQ2RqsXtK2RO8vW+HMafj5SeE/Uf/zHyKEYKNj/iDV9qi4gE8rK0eJVPk9u++Gk mwcYZZHFsgwrODK1qdDSeSt9v361CXruuV3Y526H1lWsm+mjyXA5BFZsNvJXdYEP/N3U SB6syDyy2kaExiIPYTnn37yAhsGbtkdsfg4hWz7nV8IUSX1/jIGyx/HNZ++kR53TLims jAFQ== X-Gm-Message-State: AJcUukeKFH+0IGG/HT/S5eSBYio+W9cSm0FyI8qccxfsNIx3JBMqab4F tU+Hi9cc59O6SY7sdwuk5dtH6Riu1Gk= X-Received: by 2002:a62:8a51:: with SMTP id y78mr22176191pfd.35.1548691160951; Mon, 28 Jan 2019 07:59:20 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.19 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:20 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:51 -0800 Message-Id: <20190128155907.20607-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::531 Subject: [Qemu-devel] [PULL 07/23] tcg/i386: Implement vector saturating arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Only MO_8 and MO_16 are implemented, since that's all the instruction set provides. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 2 +- tcg/i386/tcg-target.inc.c | 42 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 43 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 7bd7eae672..efbd5a6fc9 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -185,7 +185,7 @@ extern bool have_avx2; #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 -#define TCG_TARGET_HAS_sat_vec 0 +#define TCG_TARGET_HAS_sat_vec 1 #define TCG_TARGET_HAS_minmax_vec 0 #define TCG_TARGET_deposit_i32_valid(ofs, len) \ diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index 6f4c16326f..a5791dfaa5 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -377,6 +377,10 @@ static inline int tcg_target_const_match(tcg_target_long val, TCGType type, #define OPC_PADDW (0xfd | P_EXT | P_DATA16) #define OPC_PADDD (0xfe | P_EXT | P_DATA16) #define OPC_PADDQ (0xd4 | P_EXT | P_DATA16) +#define OPC_PADDSB (0xec | P_EXT | P_DATA16) +#define OPC_PADDSW (0xed | P_EXT | P_DATA16) +#define OPC_PADDUB (0xdc | P_EXT | P_DATA16) +#define OPC_PADDUW (0xdd | P_EXT | P_DATA16) #define OPC_PAND (0xdb | P_EXT | P_DATA16) #define OPC_PANDN (0xdf | P_EXT | P_DATA16) #define OPC_PBLENDW (0x0e | P_EXT3A | P_DATA16) @@ -408,6 +412,10 @@ static inline int tcg_target_const_match(tcg_target_long val, TCGType type, #define OPC_PSUBW (0xf9 | P_EXT | P_DATA16) #define OPC_PSUBD (0xfa | P_EXT | P_DATA16) #define OPC_PSUBQ (0xfb | P_EXT | P_DATA16) +#define OPC_PSUBSB (0xe8 | P_EXT | P_DATA16) +#define OPC_PSUBSW (0xe9 | P_EXT | P_DATA16) +#define OPC_PSUBUB (0xd8 | P_EXT | P_DATA16) +#define OPC_PSUBUW (0xd9 | P_EXT | P_DATA16) #define OPC_PUNPCKLBW (0x60 | P_EXT | P_DATA16) #define OPC_PUNPCKLWD (0x61 | P_EXT | P_DATA16) #define OPC_PUNPCKLDQ (0x62 | P_EXT | P_DATA16) @@ -2591,9 +2599,21 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, static int const add_insn[4] = { OPC_PADDB, OPC_PADDW, OPC_PADDD, OPC_PADDQ }; + static int const ssadd_insn[4] = { + OPC_PADDSB, OPC_PADDSW, OPC_UD2, OPC_UD2 + }; + static int const usadd_insn[4] = { + OPC_PADDSB, OPC_PADDSW, OPC_UD2, OPC_UD2 + }; static int const sub_insn[4] = { OPC_PSUBB, OPC_PSUBW, OPC_PSUBD, OPC_PSUBQ }; + static int const sssub_insn[4] = { + OPC_PSUBSB, OPC_PSUBSW, OPC_UD2, OPC_UD2 + }; + static int const ussub_insn[4] = { + OPC_PSUBSB, OPC_PSUBSW, OPC_UD2, OPC_UD2 + }; static int const mul_insn[4] = { OPC_UD2, OPC_PMULLW, OPC_PMULLD, OPC_UD2 }; @@ -2631,9 +2651,21 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_add_vec: insn = add_insn[vece]; goto gen_simd; + case INDEX_op_ssadd_vec: + insn = ssadd_insn[vece]; + goto gen_simd; + case INDEX_op_usadd_vec: + insn = usadd_insn[vece]; + goto gen_simd; case INDEX_op_sub_vec: insn = sub_insn[vece]; goto gen_simd; + case INDEX_op_sssub_vec: + insn = sssub_insn[vece]; + goto gen_simd; + case INDEX_op_ussub_vec: + insn = ussub_insn[vece]; + goto gen_simd; case INDEX_op_mul_vec: insn = mul_insn[vece]; goto gen_simd; @@ -3007,6 +3039,10 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_or_vec: case INDEX_op_xor_vec: case INDEX_op_andc_vec: + case INDEX_op_ssadd_vec: + case INDEX_op_usadd_vec: + case INDEX_op_sssub_vec: + case INDEX_op_ussub_vec: case INDEX_op_cmp_vec: case INDEX_op_x86_shufps_vec: case INDEX_op_x86_blend_vec: @@ -3074,6 +3110,12 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece) } return 1; + case INDEX_op_ssadd_vec: + case INDEX_op_usadd_vec: + case INDEX_op_sssub_vec: + case INDEX_op_ussub_vec: + return vece <= MO_16; + default: return 0; } From patchwork Mon Jan 28 15:58:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156765 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3591176jaa; Mon, 28 Jan 2019 08:03:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN7vsa0ZpapxaiWSoJhR9oJrXielpPCcgcZxUEc22DoSw4NsrKismNj5eStNISfr0aEG+Rrs X-Received: by 2002:a1c:a00f:: with SMTP id j15mr16941171wme.84.1548691421482; Mon, 28 Jan 2019 08:03:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691421; cv=none; d=google.com; s=arc-20160816; b=vgKKtByCrIL6GL8GdS1DxwiKU3tyW72K9gjOBH3J00kp6SV30azYlxAmdSUpaxW2lL SuKL4dUeO21H3GQONNxJFd87x7A81z1fhlKki7f0FdUG3EbxGZ4Pa+YDG565eKdVhhwk XwluYrUVf2W0WHu/ygaiY9t1TqLs7Q5z+O+NWJ+Rgk9EI/Bw7eToOEX7lzb8/nuIcnkh 9C2Jo86oQIbmk/2TdJZOYspiQP25UnohBvNBtQ26fzmBIbKtoyNc6hYNYZRh4nRJR61j PNXPIibnZubHbCeUxsxAnlVTifUCftXzyBiR47sibkNvhxdTVpNl9D3k5UtMH0R0m3BR 2ziA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=E3cZlO8FHbEZnWarBcLjNkcP2FXc/lN8oOBh/1vDfvY=; b=zl18ppmdSbTAuZ5p2DpSKWvzrkzr/m0KTDvID6THXLQ4Iat5Twt/Z3YJU1G0YvaJ8O x14D28xlYDDrwp2awSlz46mQiSsF3kXp6IE8z/ajxzGh/lZs6PJScEC1rM2gcfW2EjYQ LWXrFIVIpmDv22kywS/b9ZvvVYH4itQd1f9JGPITLtWmTrLfGnLlwQZR4EBpcPDaqqlW T+Pazj3rshdLj/HtLSb5xiRB0riR7w96jHOPsPC81ILkZrqqmieOeQtLoW57vHKWTE5F eJmcp8Y/RPjB+NZ9UYZy8xvlXV7oQkAbIVxsG+Sp+kJoYFaH1AKsEqPLTHXIDMWVkDhH svrA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WkpNgZm3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t6si80581659wrw.74.2019.01.28.08.03.41 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:03:41 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WkpNgZm3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34208 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Nq-0007xg-CF for patch@linaro.org; Mon, 28 Jan 2019 11:03:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33440) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jl-0004uI-5h for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jk-0007kd-12 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:25 -0500 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]:45045) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jj-0007k9-SA for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:23 -0500 Received: by mail-pg1-x532.google.com with SMTP id t13so7370214pgr.11 for ; Mon, 28 Jan 2019 07:59:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=E3cZlO8FHbEZnWarBcLjNkcP2FXc/lN8oOBh/1vDfvY=; b=WkpNgZm3NuVIMd5QZC29lLKA3LxvKzsXHQKYN8SlU2VpQpm8XN8Yk/Dyi6wR6FZNaZ eTLz/q3zUel13zJwH9f5qrBRQJ0H6dQLDWpdLdIKwdHHFvNJXE2JpYD0JTkSVyjIBfxR BnTObPnS5UlQeINo8NVNohB6+tKVVEyP3bKvk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=E3cZlO8FHbEZnWarBcLjNkcP2FXc/lN8oOBh/1vDfvY=; b=IbbKo9GIvhmHRoGaPuO1qHlN5R2MYt5zJnYxWTFFeHiGYTX0BtkmaIL2EQswBE9yRF Y2xrZ3K/EeVHNlGCLynpt6T498xxOxMZSkWHcAeUN3pqaTiTs0SbZO+g+VzZXpt3c5hx /Th3cFqO8MwFIzfjme0ES/0bDh93CORfnL8B2rrZ45hoE8vvfID5px1Fp8FY6wMV8jc1 BSsGNbpGbgWh+P/VfOkZQDpLfmestJEZVEldCI97e8cfoNdpQneVNb35Mt0TEPeTD9Fe ORT/6YKKdxLE5DQ/CrNCkm7XoAsWGMvRe7iQ/PFlsOuKfPH1zmob6i9WoeBazMmL+gFP cGdQ== X-Gm-Message-State: AJcUukcwK/W+BHQ9WdytBD1kILgb4W3cT+sAYacPRYrBW7ePP25kCdvO oqrNUN5zAXA3DCuBW9Anuzu3pyK5nMU= X-Received: by 2002:a63:8ac4:: with SMTP id y187mr20457389pgd.446.1548691162125; Mon, 28 Jan 2019 07:59:22 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.21 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:21 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:52 -0800 Message-Id: <20190128155907.20607-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::532 Subject: [Qemu-devel] [PULL 08/23] tcg/i386: Implement vector minmax arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The avx instruction set does not directly provide MO_64. We can still implement 64-bit with comparison and vpblendvb. Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 2 +- tcg/i386/tcg-target.inc.c | 81 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 82 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index efbd5a6fc9..7995fe3eab 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -186,7 +186,7 @@ extern bool have_avx2; #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 #define TCG_TARGET_HAS_sat_vec 1 -#define TCG_TARGET_HAS_minmax_vec 0 +#define TCG_TARGET_HAS_minmax_vec 1 #define TCG_TARGET_deposit_i32_valid(ofs, len) \ (((ofs) == 0 && (len) == 8) || ((ofs) == 8 && (len) == 8) || \ diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index a5791dfaa5..dbf8253718 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -392,6 +392,18 @@ static inline int tcg_target_const_match(tcg_target_long val, TCGType type, #define OPC_PCMPGTW (0x65 | P_EXT | P_DATA16) #define OPC_PCMPGTD (0x66 | P_EXT | P_DATA16) #define OPC_PCMPGTQ (0x37 | P_EXT38 | P_DATA16) +#define OPC_PMAXSB (0x3c | P_EXT38 | P_DATA16) +#define OPC_PMAXSW (0xee | P_EXT | P_DATA16) +#define OPC_PMAXSD (0x3d | P_EXT38 | P_DATA16) +#define OPC_PMAXUB (0xde | P_EXT | P_DATA16) +#define OPC_PMAXUW (0x3e | P_EXT38 | P_DATA16) +#define OPC_PMAXUD (0x3f | P_EXT38 | P_DATA16) +#define OPC_PMINSB (0x38 | P_EXT38 | P_DATA16) +#define OPC_PMINSW (0xea | P_EXT | P_DATA16) +#define OPC_PMINSD (0x39 | P_EXT38 | P_DATA16) +#define OPC_PMINUB (0xda | P_EXT | P_DATA16) +#define OPC_PMINUW (0x3a | P_EXT38 | P_DATA16) +#define OPC_PMINUD (0x3b | P_EXT38 | P_DATA16) #define OPC_PMOVSXBW (0x20 | P_EXT38 | P_DATA16) #define OPC_PMOVSXWD (0x23 | P_EXT38 | P_DATA16) #define OPC_PMOVSXDQ (0x25 | P_EXT38 | P_DATA16) @@ -2638,6 +2650,18 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, static int const packus_insn[4] = { OPC_PACKUSWB, OPC_PACKUSDW, OPC_UD2, OPC_UD2 }; + static int const smin_insn[4] = { + OPC_PMINSB, OPC_PMINSW, OPC_PMINSD, OPC_UD2 + }; + static int const smax_insn[4] = { + OPC_PMAXSB, OPC_PMAXSW, OPC_PMAXSD, OPC_UD2 + }; + static int const umin_insn[4] = { + OPC_PMINUB, OPC_PMINUW, OPC_PMINUD, OPC_UD2 + }; + static int const umax_insn[4] = { + OPC_PMAXUB, OPC_PMAXUW, OPC_PMAXUD, OPC_UD2 + }; TCGType type = vecl + TCG_TYPE_V64; int insn, sub; @@ -2678,6 +2702,18 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_xor_vec: insn = OPC_PXOR; goto gen_simd; + case INDEX_op_smin_vec: + insn = smin_insn[vece]; + goto gen_simd; + case INDEX_op_umin_vec: + insn = umin_insn[vece]; + goto gen_simd; + case INDEX_op_smax_vec: + insn = smax_insn[vece]; + goto gen_simd; + case INDEX_op_umax_vec: + insn = umax_insn[vece]; + goto gen_simd; case INDEX_op_x86_punpckl_vec: insn = punpckl_insn[vece]; goto gen_simd; @@ -3043,6 +3079,10 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_usadd_vec: case INDEX_op_sssub_vec: case INDEX_op_ussub_vec: + case INDEX_op_smin_vec: + case INDEX_op_umin_vec: + case INDEX_op_smax_vec: + case INDEX_op_umax_vec: case INDEX_op_cmp_vec: case INDEX_op_x86_shufps_vec: case INDEX_op_x86_blend_vec: @@ -3115,6 +3155,11 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece) case INDEX_op_sssub_vec: case INDEX_op_ussub_vec: return vece <= MO_16; + case INDEX_op_smin_vec: + case INDEX_op_smax_vec: + case INDEX_op_umin_vec: + case INDEX_op_umax_vec: + return vece <= MO_32 ? 1 : -1; default: return 0; @@ -3343,6 +3388,25 @@ static void expand_vec_cmp(TCGType type, unsigned vece, TCGv_vec v0, } } +static void expand_vec_minmax(TCGType type, unsigned vece, + TCGCond cond, bool min, + TCGv_vec v0, TCGv_vec v1, TCGv_vec v2) +{ + TCGv_vec t1 = tcg_temp_new_vec(type); + + tcg_debug_assert(vece == MO_64); + + tcg_gen_cmp_vec(cond, vece, t1, v1, v2); + if (min) { + TCGv_vec t2; + t2 = v1, v1 = v2, v2 = t2; + } + vec_gen_4(INDEX_op_x86_vpblendvb_vec, type, vece, + tcgv_vec_arg(v0), tcgv_vec_arg(v1), + tcgv_vec_arg(v2), tcgv_vec_arg(t1)); + tcg_temp_free_vec(t1); +} + void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece, TCGArg a0, ...) { @@ -3375,6 +3439,23 @@ void tcg_expand_vec_op(TCGOpcode opc, TCGType type, unsigned vece, expand_vec_cmp(type, vece, v0, v1, v2, va_arg(va, TCGArg)); break; + case INDEX_op_smin_vec: + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_minmax(type, vece, TCG_COND_GT, true, v0, v1, v2); + break; + case INDEX_op_smax_vec: + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_minmax(type, vece, TCG_COND_GT, false, v0, v1, v2); + break; + case INDEX_op_umin_vec: + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_minmax(type, vece, TCG_COND_GTU, true, v0, v1, v2); + break; + case INDEX_op_umax_vec: + v2 = temp_tcgv_vec(arg_temp(a2)); + expand_vec_minmax(type, vece, TCG_COND_GTU, false, v0, v1, v2); + break; + default: break; } From patchwork Mon Jan 28 15:58:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156769 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3598800jaa; Mon, 28 Jan 2019 08:09:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN63db1nyZHcrRtzMTKp7KV5xsdbW9mH4YDcU9k8lhG25HlsgeFqtXezum82IGmKA/XGNuWC X-Received: by 2002:a7b:cd85:: with SMTP id y5mr17014085wmj.129.1548691781016; Mon, 28 Jan 2019 08:09:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691781; cv=none; d=google.com; s=arc-20160816; b=Vn6zjRlq58wPHyEV5SCYVSEsaFyRbYGyJNdwdlCke+ZKXobZOLDohNq0Bxz75zBmnJ ml9Yn1QC3/wOGaWT3B9m4Ao+bemW5AKRWumhjSOw0tInTSfLa+do/8gFlwIQ4/EgzQui cipPUuxVSUoQodlA0gOGcpe8OmSg3z0d4GK71GxNtnDDPR6TdHd/etTHNrU3EfvArkhS WDAltdsD02mPjFQlpbBgZxXvUq+R33GK0l0Tc95Iya0WM85usY5n4GZAvO91+iz8982O spRe4PNFVNlZnLDXqrWFPAv8Ev27kuiAFqWbuPg6skatjKLPzCT6ZYZISDoZ/mtMoyrP myvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=bjBWCHW6CsJ6peflKASsBY6j3Bp2s4dyGDxZEiyy4vc=; b=Y7eIjwZU6XEZ8E5xHac9ZvuEinMREuEh31QGRi7Eo7jtziT5eaHIH3GJ6zOc/LoTl+ MhAT0cnSpFThmK6xfLSciRSEtfwm8dndYKE1bLu21Lqx9Br8DC+tsT9lHdd/qxWUtBph S8s9zqpuXx+1JVsTN7loETwy4yjg34biEo8Nigff429BMFup11CX2nCJCf0d6xMEEDZc c68iBHEp+/KlRo35kLjZA0i1Mzz7VHQZJu2etqgd/UwHpqXuOiOtHVS0zyJ6GgZCX/W9 koa6QKPBl0iGD5rfwj64lBhz64gOI970tablwSstxM9I5jCdwIy8vhiXLuoh3UA3KdSF +kIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="ViPpwfi/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y62si40807790wmg.13.2019.01.28.08.09.40 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:09:41 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b="ViPpwfi/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34275 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Tg-0003O9-0J for patch@linaro.org; Mon, 28 Jan 2019 11:09:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33456) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Jm-0004vg-GH for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:27 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jl-0007m9-4p for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:26 -0500 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]:43910) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jk-0007ka-Tj for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:25 -0500 Received: by mail-pg1-x52c.google.com with SMTP id v28so7381494pgk.10 for ; Mon, 28 Jan 2019 07:59:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=bjBWCHW6CsJ6peflKASsBY6j3Bp2s4dyGDxZEiyy4vc=; b=ViPpwfi/ASvNUaxPhC+rJKQjqg0Rkfr9Dh8dAaLxj1csG4RoeG1xgQWLhyQ2/l4hEV xrLrnKM4vwyijmleLBog847X555NxbE0PJDa0JsxllygV0Zkg2CE01/zddmVQPnbJBkp em6eNBF28efl0UZtmxDGci/cxKR4e0r9oXNDw= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=bjBWCHW6CsJ6peflKASsBY6j3Bp2s4dyGDxZEiyy4vc=; b=N0dL3+qzkBZdGU9dlb9E1OGEzUtw3TiH8KwEfBNc+QEmLTUNUWV1QlZFd/Ik1FKV3n 7KkVnMUTgZO4qnq13GJMJ7VCOEjFUkj6ml251R8lKEJTaVFIzmXkqPOjcRSavoxzs+Wm rUecrZuaz/MQbD48JJBKH0McEeIyt+N+zNpW1HVfRjEMMx3hdiRsoMoFAMG/ceJrYjmG KV9lHmF+Zq2KoX0OweBTXuz1TYkl2Jt4mMPIGbM3/0pLzgsPyrdKRpY89oESotMajhXl XOYrlphLvhvGAko9u2czkYjY+XF6SGSUFSvvum/saJ+PKYjvrKK6lXWQBqqvh4FrQx1a YuNg== X-Gm-Message-State: AJcUukdwED8OCeyunQH1M6T2CHfGkZa9PmnwWUGNd+60C7cy1IOZk1Bx 21bBTH1uyrk83f9o2bt34bIsIvlFRTs= X-Received: by 2002:a63:68c4:: with SMTP id d187mr20146108pgc.11.1548691163307; Mon, 28 Jan 2019 07:59:23 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.22 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:22 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:53 -0800 Message-Id: <20190128155907.20607-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::52c Subject: [Qemu-devel] [PULL 09/23] tcg/aarch64: Implement vector saturating arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 545a6eec75..a1884543d0 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -135,7 +135,7 @@ typedef enum { #define TCG_TARGET_HAS_shv_vec 0 #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 -#define TCG_TARGET_HAS_sat_vec 0 +#define TCG_TARGET_HAS_sat_vec 1 #define TCG_TARGET_HAS_minmax_vec 0 #define TCG_TARGET_DEFAULT_MO (0) diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index 0562e0aa40..b2b011f130 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -528,6 +528,10 @@ typedef enum { I3616_CMHI = 0x2e203400, I3616_CMHS = 0x2e203c00, I3616_CMEQ = 0x2e208c00, + I3616_SQADD = 0x0e200c00, + I3616_SQSUB = 0x0e202c00, + I3616_UQADD = 0x2e200c00, + I3616_UQSUB = 0x2e202c00, /* AdvSIMD two-reg misc. */ I3617_CMGT0 = 0x0e208800, @@ -2137,6 +2141,18 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_orc_vec: tcg_out_insn(s, 3616, ORN, is_q, 0, a0, a1, a2); break; + case INDEX_op_ssadd_vec: + tcg_out_insn(s, 3616, SQADD, is_q, vece, a0, a1, a2); + break; + case INDEX_op_sssub_vec: + tcg_out_insn(s, 3616, SQSUB, is_q, vece, a0, a1, a2); + break; + case INDEX_op_usadd_vec: + tcg_out_insn(s, 3616, UQADD, is_q, vece, a0, a1, a2); + break; + case INDEX_op_ussub_vec: + tcg_out_insn(s, 3616, UQSUB, is_q, vece, a0, a1, a2); + break; case INDEX_op_not_vec: tcg_out_insn(s, 3617, NOT, is_q, 0, a0, a1); break; @@ -2207,6 +2223,10 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece) case INDEX_op_shli_vec: case INDEX_op_shri_vec: case INDEX_op_sari_vec: + case INDEX_op_ssadd_vec: + case INDEX_op_sssub_vec: + case INDEX_op_usadd_vec: + case INDEX_op_ussub_vec: return 1; case INDEX_op_mul_vec: return vece < MO_64; @@ -2386,6 +2406,10 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_xor_vec: case INDEX_op_andc_vec: case INDEX_op_orc_vec: + case INDEX_op_ssadd_vec: + case INDEX_op_sssub_vec: + case INDEX_op_usadd_vec: + case INDEX_op_ussub_vec: return &w_w_w; case INDEX_op_not_vec: case INDEX_op_neg_vec: From patchwork Mon Jan 28 15:58:54 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156783 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3613080jaa; Mon, 28 Jan 2019 08:21:04 -0800 (PST) X-Google-Smtp-Source: ALg8bN6dmOGHh2Putkq084YVHFVuL6SLhlFbavDLHg9cbOmT9ZlybBh9Sx4D8G/aVZiSvGXYkt+H X-Received: by 2002:adf:e284:: with SMTP id v4mr21669303wri.26.1548692463940; Mon, 28 Jan 2019 08:21:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692463; cv=none; d=google.com; s=arc-20160816; b=jGjjJ+6rEy+K1YEuckN+AUO/UkYNzu1sQ75DhbQ4PzcIZrZ/JeVaSBhExtLHNu8ce/ z+dE0dQBlmV5NUeDtbRV9kOtvDi56o5IXdelskTopfnm0Z5/60xnFtKbH0M45HLadjRz TSHxzPpULdTeoZhHAwOjUKjlG6ZQVfdgIxhtpUi01J4CznrfkZsoqcA5QmvNlz9VV5ET uopInkUc8hoA8ShEaS62L3Ptc3+6py4aKr7rpXBz/i3oDmDVswHx/oBY2Jwc7fRp35Yh HpQTdUz0EzFKJGSZFRed6CaDBDTjXf8gFpCZopVGX7DWh6pm4M5YSbsl8tWNT9EUiL8c AJsw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=1RVpvwteSuePPKo6wNsZRQ/0lJBH3U2L6QXC6WM1wUc=; b=NRf6aQNWnEoDS8J5j+K+hxFeD38v+RaRbL5/AWzazFFfIPFS87d7ubK57rqZIhHdel Vzjbr4CH98UebshGQaN7I7NMV4FjPUZsjfAqp4JtGSmP9bU+T2RDftbujdS0sEjqEy8s udcJjCC73vMfZ99mqqbEu3LSqpyfZkxth7TsVJfZDlu4q6rlTMBwuBYhluvw7VQun3Xw ArTqggjKEBSki2ANLpclUrLkDG0zYfjvEhfWVZ8tPeyVf6qfC4+Cp6VxJgO//0Z+4/f6 grreiJA7R/wnFS5socyb2xz1ZbomsrGnSwdr0MYhDRntLmA8zea38cQt1pcQFCHW35jI LiPg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=b8ph8Mkv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v1si77014323wrq.443.2019.01.28.08.21.03 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:21:03 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=b8ph8Mkv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34430 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9eg-0003ql-Kl for patch@linaro.org; Mon, 28 Jan 2019 11:21:02 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33509) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K1-000583-20 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:41 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9Jw-0007qC-F9 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:39 -0500 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:40214) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9Jp-0007mw-VV for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:31 -0500 Received: by mail-pl1-x635.google.com with SMTP id u18so7917175plq.7 for ; Mon, 28 Jan 2019 07:59:25 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=1RVpvwteSuePPKo6wNsZRQ/0lJBH3U2L6QXC6WM1wUc=; b=b8ph8MkvD61zEpPq732b6uuUxIbtM9zeg7aZidBt3qAvXlKBhfK3FgJlQDof5jYGAP PoGOpA9msWA9jUEMFfOW1AWqXY+hVT0aY4WV44+9SjEyYRQqjyOeCMwlZZkCTBVQQHUy PK/gBtF+2R4A7/TOCQ6S8USwNVBXEOvCHED9Q= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=1RVpvwteSuePPKo6wNsZRQ/0lJBH3U2L6QXC6WM1wUc=; b=TrrbYDqVMYPJ3grtp+5TWtEhb3qC11x1QMjnSN+j6STw+jNgaBymwRMZE7Z1qtKtjT U0JLElVPEnQZEtY+WjVlK12yNA6rMHKW0i+IJUQZXwGs0SiCD6fx9ibqlMOxulv9JiJy rSsr7Q8oNR911DsMkoj2vqDc/oBga9HEhUzBNax9YHa11QvPXI7nAyBaboZvsa2n55SC YqqIVVQRxlQt0pcFVno+2+ad3gSLpECmFZ3t/Mj5knWKTMs4OzUN+fTp6icWfq7KqAXk Q/2yoD/cl72Im4N4uomrxkCJ/fY6SKr3LyyMfoLiA87XZSOG2vJmq0yEjHr69xmlsmnz r6Qw== X-Gm-Message-State: AJcUukclFRgpe9hjxNLZybkiL10zcRTd/0FzrGyRHKpD/LKx1N6n07iC BmdeAYAkvxRwH01uXFWkKTcKflXh7z4= X-Received: by 2002:a17:902:690c:: with SMTP id j12mr22120166plk.206.1548691164649; Mon, 28 Jan 2019 07:59:24 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:24 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:54 -0800 Message-Id: <20190128155907.20607-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::635 Subject: [Qemu-devel] [PULL 10/23] tcg/aarch64: Implement vector minmax arithmetic X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 24 ++++++++++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index a1884543d0..2d93cf404e 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -136,7 +136,7 @@ typedef enum { #define TCG_TARGET_HAS_cmp_vec 1 #define TCG_TARGET_HAS_mul_vec 1 #define TCG_TARGET_HAS_sat_vec 1 -#define TCG_TARGET_HAS_minmax_vec 0 +#define TCG_TARGET_HAS_minmax_vec 1 #define TCG_TARGET_DEFAULT_MO (0) #define TCG_TARGET_HAS_MEMORY_BSWAP 1 diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index b2b011f130..ee0d5819af 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -528,8 +528,12 @@ typedef enum { I3616_CMHI = 0x2e203400, I3616_CMHS = 0x2e203c00, I3616_CMEQ = 0x2e208c00, + I3616_SMAX = 0x0e206400, + I3616_SMIN = 0x0e206c00, I3616_SQADD = 0x0e200c00, I3616_SQSUB = 0x0e202c00, + I3616_UMAX = 0x2e206400, + I3616_UMIN = 0x2e206c00, I3616_UQADD = 0x2e200c00, I3616_UQSUB = 0x2e202c00, @@ -2153,6 +2157,18 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_ussub_vec: tcg_out_insn(s, 3616, UQSUB, is_q, vece, a0, a1, a2); break; + case INDEX_op_smax_vec: + tcg_out_insn(s, 3616, SMAX, is_q, vece, a0, a1, a2); + break; + case INDEX_op_smin_vec: + tcg_out_insn(s, 3616, SMIN, is_q, vece, a0, a1, a2); + break; + case INDEX_op_umax_vec: + tcg_out_insn(s, 3616, UMAX, is_q, vece, a0, a1, a2); + break; + case INDEX_op_umin_vec: + tcg_out_insn(s, 3616, UMIN, is_q, vece, a0, a1, a2); + break; case INDEX_op_not_vec: tcg_out_insn(s, 3617, NOT, is_q, 0, a0, a1); break; @@ -2227,6 +2243,10 @@ int tcg_can_emit_vec_op(TCGOpcode opc, TCGType type, unsigned vece) case INDEX_op_sssub_vec: case INDEX_op_usadd_vec: case INDEX_op_ussub_vec: + case INDEX_op_smax_vec: + case INDEX_op_smin_vec: + case INDEX_op_umax_vec: + case INDEX_op_umin_vec: return 1; case INDEX_op_mul_vec: return vece < MO_64; @@ -2410,6 +2430,10 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_sssub_vec: case INDEX_op_usadd_vec: case INDEX_op_ussub_vec: + case INDEX_op_smax_vec: + case INDEX_op_smin_vec: + case INDEX_op_umax_vec: + case INDEX_op_umin_vec: return &w_w_w; case INDEX_op_not_vec: case INDEX_op_neg_vec: From patchwork Mon Jan 28 15:58:55 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156780 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3611500jaa; Mon, 28 Jan 2019 08:19:36 -0800 (PST) X-Google-Smtp-Source: ALg8bN6oZwzAVhTGP2m2AN9SNa+Mg0WEe2t2jaMVA8FuirHspI+RtMF/uLwcZrsoeTb3/0/Q2pN+ X-Received: by 2002:a1c:7fca:: with SMTP id a193mr17854606wmd.36.1548692376270; Mon, 28 Jan 2019 08:19:36 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692376; cv=none; d=google.com; s=arc-20160816; b=zKivfzdSnAPWjLqP53XbYXifhTpJtE/G4Jj3cjR24oDuF5I2yzSOP/OIZ3R/uSZdgv yCNXYff8Q5TzraaLg41ioD2rS0brt6+Ya5fyPiCvYr2x6UXgNNbArBklxKFUFALIw7Ry Gd3JQuNUlSaMOqtsb8pEIrGfkZUxxVlR2nRH2NU0QCZ/oFkbA43It/RkQMeKm0Tq/HrH WxqBE+8rOiZ+u+ADxPe9XGZ8Um2ddNhcxs+qXMTur6OxUErDEacYFFyX4bP5v5sW1hf9 5dNPuI/mSFzf2Mk7nLoBsZOGZcInPtNIdB4cppxT94NywZk+EI5wY+x38hfslI2kW1E2 x5KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=AGAWWFtB255WL3QD01Pm/ADB1O2A0MpRDbzf5uqk7D0=; b=QpGfI4DtjAqbfmEXBTZ+tDygj/tYYezOeM8y8Dc5bh3ZctvP4YgcftMUOqDIJkOzAw TrefAeEsizIxFvJtNoQaKamiacRkyTgleKjojUDhPwoWNk3KwI54J1DPJMP2VN+XlHGM Urt8JookZnbtNweMS6hQNChxyqht0XRRzqVDeGLmoeiy0vSqt1H/LZtmvAUl6jOv9ZAT Hjadtz3sT1Zu9Kf1eM4bOVP+9D1ki9E6vOxiOfqbXe/1GAEdKjOF1+QAbtKDWV7FVGnt cTsGzQrAn00KegNt3Nrwni1kK8e3kUWhNkCO5PrrQjuj00NSDlqu28IknPijn2crs6t+ 9qGg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Znk9xUOk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v8si83631589wrd.56.2019.01.28.08.19.36 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:19:36 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Znk9xUOk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34420 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9dH-0002ff-61 for patch@linaro.org; Mon, 28 Jan 2019 11:19:35 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33602) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K6-0005D3-2X for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vV-Kd for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:38256) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007oG-CS for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x636.google.com with SMTP id e5so7924208plb.5 for ; Mon, 28 Jan 2019 07:59:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AGAWWFtB255WL3QD01Pm/ADB1O2A0MpRDbzf5uqk7D0=; b=Znk9xUOkNSDt5PqCRG1bXq2BZ5vUMPjzxYdjhNrsfsyo5pT7kXcf8+iuGb+qmS+sQY mkGQobm8gXV85HSNh0Pdr4IG6oEvuPy7oz1rQfUQhsHC9JC79GlreEI+iMTikbQ5thq+ I5BWArl8z9IE5JJUPQWedIdMMV1neCKWYSgbg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AGAWWFtB255WL3QD01Pm/ADB1O2A0MpRDbzf5uqk7D0=; b=c2NuwlPcdn3SHpKG/zczhXZwroUYKgvajj+Txcr4hsodx/owbtnDU/SoH8aJMOtTjy ze46Fl7tmogM8WqHUzs6onMvBHSimN+B1JPWph1rPx0rrZHJMgyZQbgbe0qgLulPWKIa 6H0lNzK6vwpDfIumPwNf5M9xqBJG2TCWCLxqByJlKPFbwxaLJyzngvQDEREuhW40/Fh+ IscsRILuutcl3bQbBMWF7sUqzFzntLURJHbpI13Au/IjrZ9kMc1BF3S486RHqSXqkIGl NrR0Nc/a3XKRKdFsZSSwx6QBRHewZkrDZpHZtaWXjuSinh/N5nG4Ya6uaurM+zPZyayb 6scA== X-Gm-Message-State: AJcUukcjUZtsEzejwPLEV8pwixUQZYz35JtVP5YGuG7YImFkRu3g273k 45KAv7UVaMx53izIsFpowWHCPuXS+uU= X-Received: by 2002:a17:902:b112:: with SMTP id q18mr22373102plr.255.1548691166105; Mon, 28 Jan 2019 07:59:26 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.24 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:25 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:55 -0800 Message-Id: <20190128155907.20607-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::636 Subject: [Qemu-devel] [PULL 11/23] cputlb: do not evict empty entries to the vtlb X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" Currently we evict an entry to the victim TLB when it doesn't match the current address. But it could be that there's no match because the current entry is empty (i.e. all -1's, for instance via tlb_flush). Do not evict the entry to the vtlb in that case. This change will help us keep track of the TLB's use rate, which we'll use to implement a policy for dynamic TLB sizing. Tested-by: Alex Bennée Reviewed-by: Alex Bennée Reviewed-by: Richard Henderson Signed-off-by: Emilio G. Cota Message-Id: <20190116170114.26802-2-cota@braap.org> Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index af6bd8ccf9..10f1150c62 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -224,6 +224,15 @@ static inline bool tlb_hit_page_anyprot(CPUTLBEntry *tlb_entry, tlb_hit_page(tlb_entry->addr_code, page); } +/** + * tlb_entry_is_empty - return true if the entry is not in use + * @te: pointer to CPUTLBEntry + */ +static inline bool tlb_entry_is_empty(const CPUTLBEntry *te) +{ + return te->addr_read == -1 && te->addr_write == -1 && te->addr_code == -1; +} + /* Called with tlb_c.lock held */ static inline void tlb_flush_entry_locked(CPUTLBEntry *tlb_entry, target_ulong page) @@ -591,7 +600,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, * Only evict the old entry to the victim tlb if it's for a * different page; otherwise just overwrite the stale data. */ - if (!tlb_hit_page_anyprot(te, vaddr_page)) { + if (!tlb_hit_page_anyprot(te, vaddr_page) && !tlb_entry_is_empty(te)) { unsigned vidx = env->tlb_d[mmu_idx].vindex++ % CPU_VTLB_SIZE; CPUTLBEntry *tv = &env->tlb_v_table[mmu_idx][vidx]; From patchwork Mon Jan 28 15:58:56 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156784 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3615041jaa; Mon, 28 Jan 2019 08:22:58 -0800 (PST) X-Google-Smtp-Source: ALg8bN7JgFTk05iaV9nXCgq8dK3tzPB8F/TYqwAiT8vTWN2FIXYthAWPu0b8uSJCUTUqO5Anh5xr X-Received: by 2002:a1c:27c6:: with SMTP id n189mr17409828wmn.108.1548692578019; Mon, 28 Jan 2019 08:22:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692578; cv=none; d=google.com; s=arc-20160816; b=fPwPvbUbJqXdVQUNcueIOqcaro7ZkBnpT7zgm2hxGaHnKZs3DpIVFmIbmE3ULf5AOE Yg0ekjh11Y56340F/JWY7ijfwLl/logT3GS4R3Hy7jqcmfoK28rsQzXzlnfi2XE/53IY S2lLAnUxPKfYuTmbkxkYDDzcF+w4FYr6HkqsnxYhRIebtUPiSIt7JQAKcsKxYQvJjKDf RX/PIs1TeLx+w6yz1VuZduGJmknFDTXUnSPc8919g4WjxeDyGLQfWn16UvTfDFvDgtZ1 XEZKz3Fq2qic9dok/1kVf+zK/kxmzGj/ElpoCTPVlD9WXxFsacSUA1Fpxh41xaFXAlTq nVIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=HLjl+n4SV78hr3c2b29Cbyc0sFVyUwvUdC3hlSZ/Kjs=; b=HGfxW6F+xPEM4moZrhrQDbZX6A8QzkXeNq2ppoWN5PSYUV0Pt3EZjmE9XRlz91ZdBx YL0Ca4Ox3FkA3sgFDwKrOznTERE/tO/Axwia4TZXduc2G69H1ikzsm3ZN7Sib13KpiZB 0Yiwn7v4B2ZonmYOWW6xJ1py01zCTsjpu45qTcn+yzC3N8PgTDtF8Wug0XYmp3nme5PJ wUULKHWyP625flktSCXu9S+tDUtxYdP0nAsUytymWM6YdFzVHOBYOxSkUvt7fDRjBMaA k9TSeyNPFpqPQSxM1ZBFivacHe5dmb5jzPeBDi/8U+FYfLw6e58iSxSg6cIuli0mxVl6 pt/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Wsl5eyml; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f18si46843525wme.168.2019.01.28.08.22.57 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:22:58 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Wsl5eyml; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34483 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9gW-0005Xt-Ln for patch@linaro.org; Mon, 28 Jan 2019 11:22:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33711) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K9-0005H3-ML for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:54 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007w1-U9 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:49 -0500 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:42631) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007oZ-J4 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x62c.google.com with SMTP id y1so7917554plp.9 for ; Mon, 28 Jan 2019 07:59:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HLjl+n4SV78hr3c2b29Cbyc0sFVyUwvUdC3hlSZ/Kjs=; b=Wsl5eymlZ5IJ7ff6CpNu9dj0Sy4kQzHF7fEQ6AszjWvldx+g0WGxvEGukq3zEnARr8 56NmLWDSMRs+wdwJnIQ1WoaNcxEokDJzVxZtX105ci7sgNItNI24bh5BZudIbNnIdZoP 7EUHqmgG8ynHIPhWQeRB7yxQzm2g9TsKehzfQ= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HLjl+n4SV78hr3c2b29Cbyc0sFVyUwvUdC3hlSZ/Kjs=; b=DPufaI42JwSh6OHShLnsBWZRaajKGr88ooovxjxNwf0rws+3mTfE4Zaj2bBeIAMkAB TR2aOLfjCAcaNopjXK5myABOEFMSVouPtELqShH0XRLXL1NCxkzjantA8FE/weo2ivEh X5FSFzf+48jdNKmjMJSmOQrdWEzGRnceYgCeXQOIZ8QK2EbQKTtvSe97PSX4QLxASvrn /T+V8ZoIfnZiebFhlQ7u5Mn483w5rzLqmciQ/d9RPPdsfKlYLNjjgx+G0YuBOBMSdm6x gdpco7312mOh+fr6iFf8c+BiyyB+mfUXfT4lDhLd4sijJSuSO3LvIHuXNPapuF29F26M VvLw== X-Gm-Message-State: AJcUukfGSwPjWN9McShK0i8KBDqJ1XO7+HbpC/z2fzpYakjHcVP+GdKh 4lpPLES/6xCl0C5nJRSsiek6p97h5/g= X-Received: by 2002:a17:902:7b88:: with SMTP id w8mr22175100pll.320.1548691167764; Mon, 28 Jan 2019 07:59:27 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:27 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:56 -0800 Message-Id: <20190128155907.20607-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::62c Subject: [Qemu-devel] [PULL 12/23] tcg: introduce dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" Disabled in all TCG backends for now. Tested-by: Alex Bennée Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Message-Id: <20190116170114.26802-3-cota@braap.org> Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h | 57 ++++++++++- include/exec/cpu_ldst.h | 21 ++++ tcg/aarch64/tcg-target.h | 1 + tcg/arm/tcg-target.h | 1 + tcg/i386/tcg-target.h | 1 + tcg/mips/tcg-target.h | 1 + tcg/ppc/tcg-target.h | 1 + tcg/riscv/tcg-target.h | 1 + tcg/s390/tcg-target.h | 1 + tcg/sparc/tcg-target.h | 1 + tcg/tci/tcg-target.h | 1 + accel/tcg/cputlb.c | 202 ++++++++++++++++++++++++++++++++++++++- 12 files changed, 282 insertions(+), 7 deletions(-) -- 2.17.2 diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index 6a60f94a41..191a1e021f 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -67,6 +67,28 @@ typedef uint64_t target_ulong; #define CPU_TLB_ENTRY_BITS 5 #endif +#if TCG_TARGET_IMPLEMENTS_DYN_TLB +#define CPU_TLB_DYN_MIN_BITS 6 +#define CPU_TLB_DYN_DEFAULT_BITS 8 + + +# if HOST_LONG_BITS == 32 +/* Make sure we do not require a double-word shift for the TLB load */ +# define CPU_TLB_DYN_MAX_BITS (32 - TARGET_PAGE_BITS) +# else /* HOST_LONG_BITS == 64 */ +/* + * Assuming TARGET_PAGE_BITS==12, with 2**22 entries we can cover 2**(22+12) == + * 2**34 == 16G of address space. This is roughly what one would expect a + * TLB to cover in a modern (as of 2018) x86_64 CPU. For instance, Intel + * Skylake's Level-2 STLB has 16 1G entries. + * Also, make sure we do not size the TLB past the guest's address space. + */ +# define CPU_TLB_DYN_MAX_BITS \ + MIN(22, TARGET_VIRT_ADDR_SPACE_BITS - TARGET_PAGE_BITS) +# endif + +#else /* !TCG_TARGET_IMPLEMENTS_DYN_TLB */ + /* TCG_TARGET_TLB_DISPLACEMENT_BITS is used in CPU_TLB_BITS to ensure that * the TLB is not unnecessarily small, but still small enough for the * TLB lookup instruction sequence used by the TCG target. @@ -98,6 +120,7 @@ typedef uint64_t target_ulong; NB_MMU_MODES <= 8 ? 3 : 4)) #define CPU_TLB_SIZE (1 << CPU_TLB_BITS) +#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ typedef struct CPUTLBEntry { /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address @@ -141,6 +164,18 @@ typedef struct CPUIOTLBEntry { MemTxAttrs attrs; } CPUIOTLBEntry; +/** + * struct CPUTLBWindow + * @begin_ns: host time (in ns) at the beginning of the time window + * @max_entries: maximum number of entries observed in the window + * + * See also: tlb_mmu_resize_locked() + */ +typedef struct CPUTLBWindow { + int64_t begin_ns; + size_t max_entries; +} CPUTLBWindow; + typedef struct CPUTLBDesc { /* * Describe a region covering all of the large pages allocated @@ -152,6 +187,10 @@ typedef struct CPUTLBDesc { target_ulong large_page_mask; /* The next index to use in the tlb victim table. */ size_t vindex; +#if TCG_TARGET_IMPLEMENTS_DYN_TLB + CPUTLBWindow window; + size_t n_used_entries; +#endif } CPUTLBDesc; /* @@ -176,6 +215,20 @@ typedef struct CPUTLBCommon { size_t elide_flush_count; } CPUTLBCommon; +#if TCG_TARGET_IMPLEMENTS_DYN_TLB +# define CPU_TLB \ + /* tlb_mask[i] contains (n_entries - 1) << CPU_TLB_ENTRY_BITS */ \ + uintptr_t tlb_mask[NB_MMU_MODES]; \ + CPUTLBEntry *tlb_table[NB_MMU_MODES]; +# define CPU_IOTLB \ + CPUIOTLBEntry *iotlb[NB_MMU_MODES]; +#else +# define CPU_TLB \ + CPUTLBEntry tlb_table[NB_MMU_MODES][CPU_TLB_SIZE]; +# define CPU_IOTLB \ + CPUIOTLBEntry iotlb[NB_MMU_MODES][CPU_TLB_SIZE]; +#endif + /* * The meaning of each of the MMU modes is defined in the target code. * Note that NB_MMU_MODES is not yet defined; we can only reference it @@ -184,9 +237,9 @@ typedef struct CPUTLBCommon { #define CPU_COMMON_TLB \ CPUTLBCommon tlb_c; \ CPUTLBDesc tlb_d[NB_MMU_MODES]; \ - CPUTLBEntry tlb_table[NB_MMU_MODES][CPU_TLB_SIZE]; \ + CPU_TLB \ CPUTLBEntry tlb_v_table[NB_MMU_MODES][CPU_VTLB_SIZE]; \ - CPUIOTLBEntry iotlb[NB_MMU_MODES][CPU_TLB_SIZE]; \ + CPU_IOTLB \ CPUIOTLBEntry iotlb_v[NB_MMU_MODES][CPU_VTLB_SIZE]; #else diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 959068495a..83b2907d86 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -135,6 +135,21 @@ static inline target_ulong tlb_addr_write(const CPUTLBEntry *entry) #endif } +#if TCG_TARGET_IMPLEMENTS_DYN_TLB +/* Find the TLB index corresponding to the mmu_idx + address pair. */ +static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, + target_ulong addr) +{ + uintptr_t size_mask = env->tlb_mask[mmu_idx] >> CPU_TLB_ENTRY_BITS; + + return (addr >> TARGET_PAGE_BITS) & size_mask; +} + +static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx) +{ + return (env->tlb_mask[mmu_idx] >> CPU_TLB_ENTRY_BITS) + 1; +} +#else /* Find the TLB index corresponding to the mmu_idx + address pair. */ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, target_ulong addr) @@ -142,6 +157,12 @@ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, return (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); } +static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx) +{ + return CPU_TLB_SIZE; +} +#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ + /* Find the TLB entry corresponding to the mmu_idx + address pair. */ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, target_ulong addr) diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 2d93cf404e..68868a27eb 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -15,6 +15,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #undef TCG_TARGET_STACK_GROWSUP typedef enum { diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 16172f73a3..c5a7064bdc 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -60,6 +60,7 @@ extern int arm_arch; #undef TCG_TARGET_STACK_GROWSUP #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 typedef enum { TCG_REG_R0 = 0, diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 7995fe3eab..94aa4cef7c 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -27,6 +27,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 31 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #ifdef __x86_64__ # define TCG_TARGET_REG_BITS 64 diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 5cb8672470..8600eefd9a 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -37,6 +37,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 52c1bb04b1..b51854b5cf 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -34,6 +34,7 @@ #define TCG_TARGET_NB_REGS 32 #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 typedef enum { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3, diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 60918cacb4..1eb032626c 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -33,6 +33,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h index 853ed6e7aa..394b545369 100644 --- a/tcg/s390/tcg-target.h +++ b/tcg/s390/tcg-target.h @@ -27,6 +27,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 2 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 19 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 typedef enum TCGReg { TCG_REG_R0 = 0, diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h index a0ed2a3342..dc0a227890 100644 --- a/tcg/sparc/tcg-target.h +++ b/tcg/sparc/tcg-target.h @@ -29,6 +29,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 086f34e69a..816dc4697c 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -43,6 +43,7 @@ #define TCG_TARGET_INTERPRETER 1 #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 #if UINTPTR_MAX == UINT32_MAX # define TCG_TARGET_REG_BITS 32 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 10f1150c62..a3a1614f0e 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -74,6 +74,187 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data)); QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16); #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1) +#if TCG_TARGET_IMPLEMENTS_DYN_TLB +static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx) +{ + return env->tlb_mask[mmu_idx] + (1 << CPU_TLB_ENTRY_BITS); +} + +static void tlb_window_reset(CPUTLBWindow *window, int64_t ns, + size_t max_entries) +{ + window->begin_ns = ns; + window->max_entries = max_entries; +} + +static void tlb_dyn_init(CPUArchState *env) +{ + int i; + + for (i = 0; i < NB_MMU_MODES; i++) { + CPUTLBDesc *desc = &env->tlb_d[i]; + size_t n_entries = 1 << CPU_TLB_DYN_DEFAULT_BITS; + + tlb_window_reset(&desc->window, get_clock_realtime(), 0); + desc->n_used_entries = 0; + env->tlb_mask[i] = (n_entries - 1) << CPU_TLB_ENTRY_BITS; + env->tlb_table[i] = g_new(CPUTLBEntry, n_entries); + env->iotlb[i] = g_new(CPUIOTLBEntry, n_entries); + } +} + +/** + * tlb_mmu_resize_locked() - perform TLB resize bookkeeping; resize if necessary + * @env: CPU that owns the TLB + * @mmu_idx: MMU index of the TLB + * + * Called with tlb_lock_held. + * + * We have two main constraints when resizing a TLB: (1) we only resize it + * on a TLB flush (otherwise we'd have to take a perf hit by either rehashing + * the array or unnecessarily flushing it), which means we do not control how + * frequently the resizing can occur; (2) we don't have access to the guest's + * future scheduling decisions, and therefore have to decide the magnitude of + * the resize based on past observations. + * + * In general, a memory-hungry process can benefit greatly from an appropriately + * sized TLB, since a guest TLB miss is very expensive. This doesn't mean that + * we just have to make the TLB as large as possible; while an oversized TLB + * results in minimal TLB miss rates, it also takes longer to be flushed + * (flushes can be _very_ frequent), and the reduced locality can also hurt + * performance. + * + * To achieve near-optimal performance for all kinds of workloads, we: + * + * 1. Aggressively increase the size of the TLB when the use rate of the + * TLB being flushed is high, since it is likely that in the near future this + * memory-hungry process will execute again, and its memory hungriness will + * probably be similar. + * + * 2. Slowly reduce the size of the TLB as the use rate declines over a + * reasonably large time window. The rationale is that if in such a time window + * we have not observed a high TLB use rate, it is likely that we won't observe + * it in the near future. In that case, once a time window expires we downsize + * the TLB to match the maximum use rate observed in the window. + * + * 3. Try to keep the maximum use rate in a time window in the 30-70% range, + * since in that range performance is likely near-optimal. Recall that the TLB + * is direct mapped, so we want the use rate to be low (or at least not too + * high), since otherwise we are likely to have a significant amount of + * conflict misses. + */ +static void tlb_mmu_resize_locked(CPUArchState *env, int mmu_idx) +{ + CPUTLBDesc *desc = &env->tlb_d[mmu_idx]; + size_t old_size = tlb_n_entries(env, mmu_idx); + size_t rate; + size_t new_size = old_size; + int64_t now = get_clock_realtime(); + int64_t window_len_ms = 100; + int64_t window_len_ns = window_len_ms * 1000 * 1000; + bool window_expired = now > desc->window.begin_ns + window_len_ns; + + if (desc->n_used_entries > desc->window.max_entries) { + desc->window.max_entries = desc->n_used_entries; + } + rate = desc->window.max_entries * 100 / old_size; + + if (rate > 70) { + new_size = MIN(old_size << 1, 1 << CPU_TLB_DYN_MAX_BITS); + } else if (rate < 30 && window_expired) { + size_t ceil = pow2ceil(desc->window.max_entries); + size_t expected_rate = desc->window.max_entries * 100 / ceil; + + /* + * Avoid undersizing when the max number of entries seen is just below + * a pow2. For instance, if max_entries == 1025, the expected use rate + * would be 1025/2048==50%. However, if max_entries == 1023, we'd get + * 1023/1024==99.9% use rate, so we'd likely end up doubling the size + * later. Thus, make sure that the expected use rate remains below 70%. + * (and since we double the size, that means the lowest rate we'd + * expect to get is 35%, which is still in the 30-70% range where + * we consider that the size is appropriate.) + */ + if (expected_rate > 70) { + ceil *= 2; + } + new_size = MAX(ceil, 1 << CPU_TLB_DYN_MIN_BITS); + } + + if (new_size == old_size) { + if (window_expired) { + tlb_window_reset(&desc->window, now, desc->n_used_entries); + } + return; + } + + g_free(env->tlb_table[mmu_idx]); + g_free(env->iotlb[mmu_idx]); + + tlb_window_reset(&desc->window, now, 0); + /* desc->n_used_entries is cleared by the caller */ + env->tlb_mask[mmu_idx] = (new_size - 1) << CPU_TLB_ENTRY_BITS; + env->tlb_table[mmu_idx] = g_try_new(CPUTLBEntry, new_size); + env->iotlb[mmu_idx] = g_try_new(CPUIOTLBEntry, new_size); + /* + * If the allocations fail, try smaller sizes. We just freed some + * memory, so going back to half of new_size has a good chance of working. + * Increased memory pressure elsewhere in the system might cause the + * allocations to fail though, so we progressively reduce the allocation + * size, aborting if we cannot even allocate the smallest TLB we support. + */ + while (env->tlb_table[mmu_idx] == NULL || env->iotlb[mmu_idx] == NULL) { + if (new_size == (1 << CPU_TLB_DYN_MIN_BITS)) { + error_report("%s: %s", __func__, strerror(errno)); + abort(); + } + new_size = MAX(new_size >> 1, 1 << CPU_TLB_DYN_MIN_BITS); + env->tlb_mask[mmu_idx] = (new_size - 1) << CPU_TLB_ENTRY_BITS; + + g_free(env->tlb_table[mmu_idx]); + g_free(env->iotlb[mmu_idx]); + env->tlb_table[mmu_idx] = g_try_new(CPUTLBEntry, new_size); + env->iotlb[mmu_idx] = g_try_new(CPUIOTLBEntry, new_size); + } +} + +static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx) +{ + tlb_mmu_resize_locked(env, mmu_idx); + memset(env->tlb_table[mmu_idx], -1, sizeof_tlb(env, mmu_idx)); + env->tlb_d[mmu_idx].n_used_entries = 0; +} + +static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx) +{ + env->tlb_d[mmu_idx].n_used_entries++; +} + +static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx) +{ + env->tlb_d[mmu_idx].n_used_entries--; +} + +#else /* !TCG_TARGET_IMPLEMENTS_DYN_TLB */ + +static inline void tlb_dyn_init(CPUArchState *env) +{ +} + +static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx) +{ + memset(env->tlb_table[mmu_idx], -1, sizeof(env->tlb_table[0])); +} + +static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx) +{ +} + +static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx) +{ +} +#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ + void tlb_init(CPUState *cpu) { CPUArchState *env = cpu->env_ptr; @@ -82,6 +263,8 @@ void tlb_init(CPUState *cpu) /* Ensure that cpu_reset performs a full flush. */ env->tlb_c.dirty = ALL_MMUIDX_BITS; + + tlb_dyn_init(env); } /* flush_all_helper: run fn across all cpus @@ -122,7 +305,7 @@ void tlb_flush_counts(size_t *pfull, size_t *ppart, size_t *pelide) static void tlb_flush_one_mmuidx_locked(CPUArchState *env, int mmu_idx) { - memset(env->tlb_table[mmu_idx], -1, sizeof(env->tlb_table[0])); + tlb_table_flush_by_mmuidx(env, mmu_idx); memset(env->tlb_v_table[mmu_idx], -1, sizeof(env->tlb_v_table[0])); env->tlb_d[mmu_idx].large_page_addr = -1; env->tlb_d[mmu_idx].large_page_mask = -1; @@ -234,12 +417,14 @@ static inline bool tlb_entry_is_empty(const CPUTLBEntry *te) } /* Called with tlb_c.lock held */ -static inline void tlb_flush_entry_locked(CPUTLBEntry *tlb_entry, +static inline bool tlb_flush_entry_locked(CPUTLBEntry *tlb_entry, target_ulong page) { if (tlb_hit_page_anyprot(tlb_entry, page)) { memset(tlb_entry, -1, sizeof(*tlb_entry)); + return true; } + return false; } /* Called with tlb_c.lock held */ @@ -250,7 +435,9 @@ static inline void tlb_flush_vtlb_page_locked(CPUArchState *env, int mmu_idx, assert_cpu_is_self(ENV_GET_CPU(env)); for (k = 0; k < CPU_VTLB_SIZE; k++) { - tlb_flush_entry_locked(&env->tlb_v_table[mmu_idx][k], page); + if (tlb_flush_entry_locked(&env->tlb_v_table[mmu_idx][k], page)) { + tlb_n_used_entries_dec(env, mmu_idx); + } } } @@ -267,7 +454,9 @@ static void tlb_flush_page_locked(CPUArchState *env, int midx, midx, lp_addr, lp_mask); tlb_flush_one_mmuidx_locked(env, midx); } else { - tlb_flush_entry_locked(tlb_entry(env, midx, page), page); + if (tlb_flush_entry_locked(tlb_entry(env, midx, page), page)) { + tlb_n_used_entries_dec(env, midx); + } tlb_flush_vtlb_page_locked(env, midx, page); } } @@ -444,8 +633,9 @@ void tlb_reset_dirty(CPUState *cpu, ram_addr_t start1, ram_addr_t length) qemu_spin_lock(&env->tlb_c.lock); for (mmu_idx = 0; mmu_idx < NB_MMU_MODES; mmu_idx++) { unsigned int i; + unsigned int n = tlb_n_entries(env, mmu_idx); - for (i = 0; i < CPU_TLB_SIZE; i++) { + for (i = 0; i < n; i++) { tlb_reset_dirty_range_locked(&env->tlb_table[mmu_idx][i], start1, length); } @@ -607,6 +797,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, /* Evict the old entry into the victim tlb. */ copy_tlb_helper_locked(tv, te); env->iotlb_v[mmu_idx][vidx] = env->iotlb[mmu_idx][index]; + tlb_n_used_entries_dec(env, mmu_idx); } /* refill the tlb */ @@ -658,6 +849,7 @@ void tlb_set_page_with_attrs(CPUState *cpu, target_ulong vaddr, } copy_tlb_helper_locked(te, &tn); + tlb_n_used_entries_inc(env, mmu_idx); qemu_spin_unlock(&env->tlb_c.lock); } From patchwork Mon Jan 28 15:58:57 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156787 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3617808jaa; Mon, 28 Jan 2019 08:25:41 -0800 (PST) X-Google-Smtp-Source: ALg8bN67ShcJxrPKO0PXRtzEdg7HI7JSYVu214r7GgrsujDxRHTtyZAyzi52u+8ShfXHRHNzZQ3R X-Received: by 2002:adf:cd0e:: with SMTP id w14mr23267765wrm.218.1548692741522; Mon, 28 Jan 2019 08:25:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692741; cv=none; d=google.com; s=arc-20160816; b=aSVV2nb4ZINo7BNdWuNmY6BKpoyBD4xU0ezm5Zs3DbawjFYxS62a7ZehrcJTvtYW0Y /Qn8LNroHyY8p48Perq5RkMVuBMtlBVTAhnv93VmP6n5rsHiOYptzJQAOTPATUr3byti vZmSBtB9yzRjNEKOAvRcJ4HsCzUybwGwTrsezuzwOCYcjnMqm2UVjX3nIHjFPASIVQvR zun2LeOGzYABFUuNSHT3vl/HciSKixuZsAWlk7l+mstk7G5fanauJ7KcmVIVJBdyMdfG /piXIqgicpGGRXnmUA5djgR9aJSWYVZ6gyJZHCcH9Y5sZR29xoiuynyoEXOwQYnk+ckm oDTw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=G3R6M8lo+egOdVDHfgVuN8I78ohbr7aGRLQnNFv5cCY=; b=VF2bddu/x3KgnpTgb1OAeEgJEV3qio66T9eh3XUlJCBDHsYmqCR3GDy+W7/0QDk8qY rH31+Y2rmlY9RdnkVn9GS+yAxu6XRJ+hZ3/ccf+OjbB8JNuoHYkSxk4JAowuV789A4rl VRw06RaQtovKwEEsljJz5pu5y5fGlJSo4krSKkHBnjt4atlohKPsqBqWVVVveMhwCYqi 58FRnD+BTLclISRL7KjzD0WidcO0M1hoP4n2EUoKs4XNgrv9turO2G1fSLSmUqmqJVU+ TO7n1gWjBaMA7yMPmj7int9URbfjPSihIODF8eor0wmsrJG4RDxBhrLgu5SyvDxJkU8A bfgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BI8PyEot; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i67si34910582wmi.164.2019.01.28.08.25.41 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:25:41 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=BI8PyEot; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34517 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9jA-0007qz-9r for patch@linaro.org; Mon, 28 Jan 2019 11:25:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33630) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K6-0005Dq-UG for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vO-Ib for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:46 -0500 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]:41075) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007om-7N for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x642.google.com with SMTP id u6so7919431plm.8 for ; Mon, 28 Jan 2019 07:59:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=G3R6M8lo+egOdVDHfgVuN8I78ohbr7aGRLQnNFv5cCY=; b=BI8PyEot8BbJCc0GinTMC85x5fEgOgU38qSj2XRMrC4iaX0hllM5w7Rqn2eX6GmD9t OSzJtYxRiHCgCGszRUKreVsWEZ+oQf356KVDesWsnvclvFnL6HpR1BSXVP1zA1M9iJTJ RbJ6JLOWjnBNZ9apMKCRx522jkoKg0FeTX758= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=G3R6M8lo+egOdVDHfgVuN8I78ohbr7aGRLQnNFv5cCY=; b=mlrt5F3thSmKr15t7SGa14AhplILEaNvLQQumCwaPRUUM+wyaW1LblgLwH513iF+g5 plk95nYFble12ZO4WYooCn5q8SlwRIy3v2TA3OMqcGRqRTrewvNFfstG96lf67RDLBSa tg/a3lV2a8icr4NiobpInUD+ExQBLH8+dOnZfZjCdGioLdfTDT6Kt6QIABPfhO3R8+cQ feqUjVOTNgz8T9hh2j7iLO4NW4giqiEZIiKnnNSvpD+P2PGu0m7lR3vnxVzMn9KjcKgR YtLCDgId8gkrGC3BbtPQ54Q2iUIJhqLXwj1kRf13dmtm8oRFttq5aEpfvD8P7sIwSUwu rmLw== X-Gm-Message-State: AJcUukfw+KtevSUaWO/fMQQYyeROo3Ok3g7eYX2ShnJSWlTV8CisgCtF qZbTQc4y9K45BRYrJivIDjAsQawDHRU= X-Received: by 2002:a17:902:b48b:: with SMTP id y11mr21442992plr.200.1548691169074; Mon, 28 Jan 2019 07:59:29 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:28 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:57 -0800 Message-Id: <20190128155907.20607-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::642 Subject: [Qemu-devel] [PULL 13/23] tcg/i386: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org, "Emilio G. Cota" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: "Emilio G. Cota" As the following experiments show, this series is a net perf gain, particularly for memory-heavy workloads. Experiments are run on an Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz. 1. System boot + shudown, debian aarch64: - Before (v3.1.0): Performance counter stats for './die.sh v3.1.0' (10 runs): 9019.797015 task-clock (msec) # 0.993 CPUs utilized ( +- 0.23% ) 29,910,312,379 cycles # 3.316 GHz ( +- 0.14% ) 54,699,252,014 instructions # 1.83 insn per cycle ( +- 0.08% ) 10,061,951,686 branches # 1115.541 M/sec ( +- 0.08% ) 172,966,530 branch-misses # 1.72% of all branches ( +- 0.07% ) 9.084039051 seconds time elapsed ( +- 0.23% ) - After: Performance counter stats for './die.sh tlb-dyn-v5' (10 runs): 8624.084842 task-clock (msec) # 0.993 CPUs utilized ( +- 0.23% ) 28,556,123,404 cycles # 3.311 GHz ( +- 0.13% ) 51,755,089,512 instructions # 1.81 insn per cycle ( +- 0.05% ) 9,526,513,946 branches # 1104.641 M/sec ( +- 0.05% ) 166,578,509 branch-misses # 1.75% of all branches ( +- 0.19% ) 8.680540350 seconds time elapsed ( +- 0.24% ) That is, a 4.4% perf increase. 2. System boot + shutdown, ubuntu 18.04 x86_64: - Before (v3.1.0): 56100.574751 task-clock (msec) # 1.016 CPUs utilized ( +- 4.81% ) 200,745,466,128 cycles # 3.578 GHz ( +- 5.24% ) 431,949,100,608 instructions # 2.15 insn per cycle ( +- 5.65% ) 77,502,383,330 branches # 1381.490 M/sec ( +- 6.18% ) 844,681,191 branch-misses # 1.09% of all branches ( +- 3.82% ) 55.221556378 seconds time elapsed ( +- 5.01% ) - After: 56603.419540 task-clock (msec) # 1.019 CPUs utilized ( +- 10.19% ) 202,217,930,479 cycles # 3.573 GHz ( +- 10.69% ) 439,336,291,626 instructions # 2.17 insn per cycle ( +- 14.14% ) 80,538,357,447 branches # 1422.853 M/sec ( +- 16.09% ) 776,321,622 branch-misses # 0.96% of all branches ( +- 3.77% ) 55.549661409 seconds time elapsed ( +- 10.44% ) No improvement (within noise range). Note that for this workload, increasing the time window too much can lead to perf degradation, since it flushes the TLB *very* frequently. 3. x86_64 SPEC06int: x86_64-softmmu speedup vs. v3.1.0 for SPEC06int (test set) Host: Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz (Skylake) 5.5 +------------------------------------------------------------------------+ | +-+ | 5 |-+.................+-+...............................tlb-dyn-v5.......+-| | * * | 4.5 |-+.................*.*................................................+-| | * * | 4 |-+.................*.*................................................+-| | * * | 3.5 |-+.................*.*................................................+-| | * * | 3 |-+......+-+*.......*.*................................................+-| | * * * * | 2.5 |-+......*..*.......*.*.................................+-+*...........+-| | * * * * * * | 2 |-+......*..*.......*.*.................................*..*...........+-| | * * * * * * +-+ | 1.5 |-+......*..*.......*.*.................................*..*.*+-+.*+-+.+-| | * * *+-+ * * +-+ *+-+ +-+ +-+ * * * * * * | 1 |++++-+*+*++*+*++*++*+*++*+*+++-+*+*+-++*+-++++-++++-+++*++*+*++*+*++*+++| | * * * * * * * * * * * * * * * * * * * * * * * * * * | 0.5 +------------------------------------------------------------------------+ 400.perlb401.bzip403.g429445.g456.hm462.libq464.h471.omn47483.xalancbgeomean png: https://imgur.com/YRF90f7 That is, a 1.51x average speedup over the baseline, with a max speedup of 5.17x. Here's a different look at the SPEC06int results, using KVM as the baseline: x86_64-softmmu slowdown vs. KVM for SPEC06int (test set) Host: Intel(R) Xeon(R) Gold 6142 CPU @ 2.60GHz (Skylake) 25 +---------------------------------------------------------------------------+ | +-+ +-+ | | * * +-+ v3.1.0 | | * * +-+ tlb-dyn-v5 | | * * * * +-+ | 20 |-+.................*.*.............................*.+-+......*.*........+-| | * * * # # * * | | +-+ * * * # # * * | | * * * * * # # * * | 15 |-+......*.*........*.*.............................*.#.#......*.+-+......+-| | * * * * * # # * #|# | | * * * * +-+ * # # * +-+ | | * * +-+ * * ++-+ +-+ * # # * # # +-+ | | * * +-+ * * * ## *| +-+ * # # * # # +-+ | 10 |-+......*.*..*.+-+.*.*........*.##.......++-+.*.+-+*.#.#......*.#.#.*.*..+-| | * * * +-+ * * * ## +-+ *# # * # #* # # +-+ * # # * * | | * * * # # * * +-+ * ## * +-+ *# # * # #* # # * * * # # *+-+ | | * * * # # * * * +-+ * ## * # # *# # * # #* # # * * * # # * ## | 5 |-+......*.+-+*.#.#.*.*..*.#.#.*.##.*.#.#.*#.#.*.#.#*.#.#.*.*..*.#.#.*.##.+-| | * # #* # # * +-+* # # * ## * # # *# # * # #* # # * * * # # * ## | | * # #* # # * # #* # # * ## * # # *# # * # #* # # * +-+* # # * ## | | ++-+ * # #* # # * # #* # # * ## * # # *# # * # #* # # * # #* # # * ## | |+++*#+#+*+#+#*+#+#+*+#+#*+#+#+*+##+*+#+#+*#+#+*+#+#*+#+#+*+#+#*+#+#+*+##+++| 0 +---------------------------------------------------------------------------+ 400.perlbe401.bzi403.gc429445.go456.h462.libqu464.h471.omne4483.xalancbmgeomean png: https://imgur.com/YzAMNEV After this series, we bring down the average SPEC06int slowdown vs KVM from 11.47x to 7.58x. Tested-by: Alex Bennée Reviewed-by: Alex Bennée Signed-off-by: Emilio G. Cota Message-Id: <20190116170114.26802-4-cota@braap.org> Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 2 +- tcg/i386/tcg-target.inc.c | 28 ++++++++++++++-------------- 2 files changed, 15 insertions(+), 15 deletions(-) -- 2.17.2 diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 94aa4cef7c..eb40312e67 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -27,7 +27,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 31 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #ifdef __x86_64__ # define TCG_TARGET_REG_BITS 64 diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index dbf8253718..4d84aea3a9 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -329,6 +329,7 @@ static inline int tcg_target_const_match(tcg_target_long val, TCGType type, #define OPC_ARITH_GvEv (0x03) /* ... plus (ARITH_FOO << 3) */ #define OPC_ANDN (0xf2 | P_EXT38) #define OPC_ADD_GvEv (OPC_ARITH_GvEv | (ARITH_ADD << 3)) +#define OPC_AND_GvEv (OPC_ARITH_GvEv | (ARITH_AND << 3)) #define OPC_BLENDPS (0x0c | P_EXT3A | P_DATA16) #define OPC_BSF (0xbc | P_EXT) #define OPC_BSR (0xbd | P_EXT) @@ -1641,7 +1642,7 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, } if (TCG_TYPE_PTR == TCG_TYPE_I64) { hrexw = P_REXW; - if (TARGET_PAGE_BITS + CPU_TLB_BITS > 32) { + if (TARGET_PAGE_BITS + CPU_TLB_DYN_MAX_BITS > 32) { tlbtype = TCG_TYPE_I64; tlbrexw = P_REXW; } @@ -1649,6 +1650,15 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, } tcg_out_mov(s, tlbtype, r0, addrlo); + tcg_out_shifti(s, SHIFT_SHR + tlbrexw, r0, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + + tcg_out_modrm_offset(s, OPC_AND_GvEv + trexw, r0, TCG_AREG0, + offsetof(CPUArchState, tlb_mask[mem_index])); + + tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, r0, TCG_AREG0, + offsetof(CPUArchState, tlb_table[mem_index])); + /* If the required alignment is at least as large as the access, simply copy the address and mask. For lesser alignments, check that we don't cross pages for the complete access. */ @@ -1658,20 +1668,10 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, tcg_out_modrm_offset(s, OPC_LEA + trexw, r1, addrlo, s_mask - a_mask); } tlb_mask = (target_ulong)TARGET_PAGE_MASK | a_mask; - - tcg_out_shifti(s, SHIFT_SHR + tlbrexw, r0, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); - tgen_arithi(s, ARITH_AND + trexw, r1, tlb_mask, 0); - tgen_arithi(s, ARITH_AND + tlbrexw, r0, - (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS, 0); - - tcg_out_modrm_sib_offset(s, OPC_LEA + hrexw, r0, TCG_AREG0, r0, 0, - offsetof(CPUArchState, tlb_table[mem_index][0]) - + which); /* cmp 0(r0), r1 */ - tcg_out_modrm_offset(s, OPC_CMP_GvEv + trexw, r1, r0, 0); + tcg_out_modrm_offset(s, OPC_CMP_GvEv + trexw, r1, r0, which); /* Prepare for both the fast path add of the tlb addend, and the slow path function argument setup. */ @@ -1684,7 +1684,7 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, if (TARGET_LONG_BITS > TCG_TARGET_REG_BITS) { /* cmp 4(r0), addrhi */ - tcg_out_modrm_offset(s, OPC_CMP_GvEv, addrhi, r0, 4); + tcg_out_modrm_offset(s, OPC_CMP_GvEv, addrhi, r0, which + 4); /* jne slow_path */ tcg_out_opc(s, OPC_JCC_long + JCC_JNE, 0, 0, 0); @@ -1696,7 +1696,7 @@ static inline void tcg_out_tlb_load(TCGContext *s, TCGReg addrlo, TCGReg addrhi, /* add addend(r0), r1 */ tcg_out_modrm_offset(s, OPC_ADD_GvEv + hrexw, r1, r0, - offsetof(CPUTLBEntry, addend) - which); + offsetof(CPUTLBEntry, addend)); } /* From patchwork Mon Jan 28 15:58:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156782 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3612018jaa; Mon, 28 Jan 2019 08:20:04 -0800 (PST) X-Google-Smtp-Source: ALg8bN7jDjDV6KV77wODSd9EZ0u4NS9VrbIhCd017qDC6+1kzSxhQKzx6b9thw/Ha1QWdFD6/E7c X-Received: by 2002:adf:e5d0:: with SMTP id a16mr22338994wrn.89.1548692404170; Mon, 28 Jan 2019 08:20:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692404; cv=none; d=google.com; s=arc-20160816; b=WsuJH/NZSX9caEDMYMtA9uNoLfluXVVySjhotG+lGjVs8OXIlxqbrujnoUe+aMFbcU +ogVcURWVNmEDkH4xOBPoq2Ds5RNYE8rsBhYFl2skq3+06Waad8ZRS9maDwWJX3qMM0A x8ep9SLyYKlfeB/jIxr8O/ed5BsxUtib+y+1RPowpQDpODCPdSR5wO8j7zRd5FOi0r+d LRKM03Z/w1K4XMMq+53pZwXijD0HLuXC+sOJ0jJ4hLRz14mbpXHlJRj3MB9EjsH9jU8J pN7iOcuFx2a0fazHrqXapAhm7jmjonkVLp5IDOfIneMYNcJ8mABTpWI2kAQNIwxck95P sdSQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=3XUrqLNGaYapg7Nttjm2+IIjxBvlvdpFLTOsovN7rj0=; b=yizhJRhbMNyoThrRVrb5YEtwucfoR37Txv3FGrAv/Br6nXwJrXweX8iJG7hForjw7O IN72MPsPyxZMsqYfRi5mrfk5pHQO/+S4O5wYVXVb3ilMZRDsmrz1K+es80Agl5BUiVEW 5Ndeq8qVTnoSimC09TsquyK/9ppG8lQcQLOW62wM/C/hBxH9UBOAHFVIEC78paHCzpku wwA6qoehzdGH3OIRJi4LW9HxN460qKhQf4uGqrsGjktoBvnA65bDEJQ+oyyg9O1JoeDZ IzFXujqp40fOSO1aO9SYE7WkMznt6BqMn1vNyzy4/sEw2kXCEF5aZn8F0657M3f/VD2E wSIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=imi05m1K; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 63si9903034wrf.260.2019.01.28.08.20.03 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:20:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=imi05m1K; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34432 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9dj-0003tK-1Y for patch@linaro.org; Mon, 28 Jan 2019 11:20:03 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33580) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005Cg-KV for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007v9-5J for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pf1-x443.google.com ([2607:f8b0:4864:20::443]:43172) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K3-0007ot-Rc for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pf1-x443.google.com with SMTP id w73so8164797pfk.10 for ; Mon, 28 Jan 2019 07:59:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3XUrqLNGaYapg7Nttjm2+IIjxBvlvdpFLTOsovN7rj0=; b=imi05m1KCfyb7Nj55UjpaWg3+ts4heZSh+kxTWvzMmNEn/EtncDqzrmZ3uvEh07TKB OeaXa7r8huCr1iYRMcroezD6zZcwDAoEzsP62LLpM5dSWjZb+fEnYokc52wdyAJQ0Aux T0f9LYUhwIdheV8sZgwLscM1oyxtLohE/hRbo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3XUrqLNGaYapg7Nttjm2+IIjxBvlvdpFLTOsovN7rj0=; b=Zym/lT7QeD+sTjFBXRccO1yv7lxuB0ZGqdZTpuyhWx/RN1tXFAlQKKggM/KBF/MM7x XH3EUTawtiUjD2iJCqk6+iIOpYoNIOqTCsrRurVe5JLRnXlVYPkPnuH5lOArAxaz40Wf W+u8x+WDWXnoUreEgZX93EmRzvkYopYD30pmffR+pEFVfKxJ+jXLQNEMtoFLMj46DeWF hoHYe6TWevBJFyBfrKwHfePZx/4K8jL2vhzyoxcn/LUGiqMn5acOcNr0aztNbwQbyWbp 2R8ZAnouw+MNQy40AFElEas1froF0v8OcHUZ/vDmdKdTC+/h6OPau70Rc/NTRgqB0QO9 H9ag== X-Gm-Message-State: AJcUukdYJ4rwHWXLFvubvIkyF/pnM3eHL3kW+O4PiL50lnmjScl3EBIM OExHM3GFMLzuTTsUUnvSYLSaheK+/MM= X-Received: by 2002:a63:fb15:: with SMTP id o21mr20290446pgh.211.1548691170304; Mon, 28 Jan 2019 07:59:30 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:29 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:58 -0800 Message-Id: <20190128155907.20607-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::443 Subject: [Qemu-devel] [PULL 14/23] tcg/aarch64: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Alex Bennée Tested-by: Alex Bennée Signed-off-by: Richard Henderson --- tcg/aarch64/tcg-target.h | 2 +- tcg/aarch64/tcg-target.inc.c | 100 +++++++++++++++++++++-------------- 2 files changed, 60 insertions(+), 42 deletions(-) -- 2.17.2 diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 68868a27eb..5085a81060 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -15,7 +15,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #undef TCG_TARGET_STACK_GROWSUP typedef enum { diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c index ee0d5819af..d57f9e500f 100644 --- a/tcg/aarch64/tcg-target.inc.c +++ b/tcg/aarch64/tcg-target.inc.c @@ -498,6 +498,9 @@ typedef enum { I3510_EON = 0x4a200000, I3510_ANDS = 0x6a000000, + /* Logical shifted register instructions (with a shift). */ + I3502S_AND_LSR = I3510_AND | (1 << 22), + /* AdvSIMD copy */ I3605_DUP = 0x0e000400, I3605_INS = 0x4e001c00, @@ -1448,6 +1451,14 @@ static void add_qemu_ldst_label(TCGContext *s, bool is_ld, TCGMemOpIdx oi, label->label_ptr[0] = label_ptr; } +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect to use a 24-bit unsigned offset from ENV. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table[NB_MMU_MODES - 1]) + > 0xffffff); + /* Load and compare a TLB entry, emitting the conditional jump to the slow path for the failure case, which will be patched later when finalizing the slow path. Generated code returns the host addend in X1, @@ -1456,15 +1467,55 @@ static void tcg_out_tlb_read(TCGContext *s, TCGReg addr_reg, TCGMemOp opc, tcg_insn_unit **label_ptr, int mem_index, bool is_read) { - int tlb_offset = is_read ? - offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write); + int mask_ofs = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_ofs = offsetof(CPUArchState, tlb_table[mem_index]); unsigned a_bits = get_alignment_bits(opc); unsigned s_bits = opc & MO_SIZE; unsigned a_mask = (1u << a_bits) - 1; unsigned s_mask = (1u << s_bits) - 1; - TCGReg base = TCG_AREG0, x3; - uint64_t tlb_mask; + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0, x3; + TCGType mask_type; + uint64_t compare_mask; + + if (table_ofs > 0xfff) { + int table_hi = table_ofs & ~0xfff; + int mask_hi = mask_ofs & ~0xfff; + + table_base = TCG_REG_X1; + if (mask_hi == table_hi) { + mask_base = table_base; + } else if (mask_hi) { + mask_base = TCG_REG_X0; + tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, + mask_base, TCG_AREG0, mask_hi); + } + tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, + table_base, TCG_AREG0, table_hi); + mask_ofs -= mask_hi; + table_ofs -= table_hi; + } + + mask_type = (TARGET_PAGE_BITS + CPU_TLB_DYN_MAX_BITS > 32 + ? TCG_TYPE_I64 : TCG_TYPE_I32); + + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, mask_type, TCG_REG_X0, mask_base, mask_ofs); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_X1, table_base, table_ofs); + + /* Extract the TLB index from the address into X0. */ + tcg_out_insn(s, 3502S, AND_LSR, mask_type == TCG_TYPE_I64, + TCG_REG_X0, TCG_REG_X0, addr_reg, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + + /* Add the tlb_table pointer, creating the CPUTLBEntry address into X1. */ + tcg_out_insn(s, 3502, ADD, 1, TCG_REG_X1, TCG_REG_X1, TCG_REG_X0); + + /* Load the tlb comparator into X0, and the fast path addend into X1. */ + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_X0, TCG_REG_X1, is_read + ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_X1, TCG_REG_X1, + offsetof(CPUTLBEntry, addend)); /* For aligned accesses, we check the first byte and include the alignment bits within the address. For unaligned access, we check that we don't @@ -1476,47 +1527,14 @@ static void tcg_out_tlb_read(TCGContext *s, TCGReg addr_reg, TCGMemOp opc, TCG_REG_X3, addr_reg, s_mask - a_mask); x3 = TCG_REG_X3; } - tlb_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; - - /* Extract the TLB index from the address into X0. - X0 = - addr_reg */ - tcg_out_ubfm(s, TARGET_LONG_BITS == 64, TCG_REG_X0, addr_reg, - TARGET_PAGE_BITS, TARGET_PAGE_BITS + CPU_TLB_BITS); + compare_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; /* Store the page mask part of the address into X3. */ tcg_out_logicali(s, I3404_ANDI, TARGET_LONG_BITS == 64, - TCG_REG_X3, x3, tlb_mask); - - /* Add any "high bits" from the tlb offset to the env address into X2, - to take advantage of the LSL12 form of the ADDI instruction. - X2 = env + (tlb_offset & 0xfff000) */ - if (tlb_offset & 0xfff000) { - tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_X2, base, - tlb_offset & 0xfff000); - base = TCG_REG_X2; - } - - /* Merge the tlb index contribution into X2. - X2 = X2 + (X0 << CPU_TLB_ENTRY_BITS) */ - tcg_out_insn(s, 3502S, ADD_LSL, TCG_TYPE_I64, TCG_REG_X2, base, - TCG_REG_X0, CPU_TLB_ENTRY_BITS); - - /* Merge "low bits" from tlb offset, load the tlb comparator into X0. - X0 = load [X2 + (tlb_offset & 0x000fff)] */ - tcg_out_ldst(s, TARGET_LONG_BITS == 32 ? I3312_LDRW : I3312_LDRX, - TCG_REG_X0, TCG_REG_X2, tlb_offset & 0xfff, - TARGET_LONG_BITS == 32 ? 2 : 3); - - /* Load the tlb addend. Do that early to avoid stalling. - X1 = load [X2 + (tlb_offset & 0xfff) + offsetof(addend)] */ - tcg_out_ldst(s, I3312_LDRX, TCG_REG_X1, TCG_REG_X2, - (tlb_offset & 0xfff) + (offsetof(CPUTLBEntry, addend)) - - (is_read ? offsetof(CPUTLBEntry, addr_read) - : offsetof(CPUTLBEntry, addr_write)), 3); + TCG_REG_X3, x3, compare_mask); /* Perform the address comparison. */ - tcg_out_cmp(s, (TARGET_LONG_BITS == 64), TCG_REG_X0, TCG_REG_X3, 0); + tcg_out_cmp(s, TARGET_LONG_BITS == 64, TCG_REG_X0, TCG_REG_X3, 0); /* If not equal, we jump to the slow path. */ *label_ptr = s->code_ptr; From patchwork Mon Jan 28 15:58:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156781 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3611755jaa; Mon, 28 Jan 2019 08:19:49 -0800 (PST) X-Google-Smtp-Source: ALg8bN63CFCZJU4F14A7w/J2aFmveo0NDpoRN0/RNDFidjLSVtUSNd7B9hGqQ6AsIuKwo1SCDgfE X-Received: by 2002:adf:e9d1:: with SMTP id l17mr20964952wrn.73.1548692389647; Mon, 28 Jan 2019 08:19:49 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692389; cv=none; d=google.com; s=arc-20160816; b=WPhl+3D8jfFcCtjvNIDuehAnj/IHmZCOUBtCxZj8zl+BJKBPZlU1gVuB59Ely11gt0 l9WhlAaRbbZqMTlYV1WelhCGaYZMDX+obzDIjp9eLb1OnWzNoME+VFSFF9ZWVhhSl131 IwAwww5INXk/XPPv5Gm9vqITP017syqBRD06ctnCK0x9Kz8AFR4ku2SRCsq7i2zfyks+ s6dv2nCoaDduStV9om71DJJKxcJsHmR49KkEtW2H/Hn/ViAWyEQBdb0BQdHWNxQih+57 otOVJ64AkvwhGH1I+rewR1op+fQBvsGiXnpaKjMl7aFmGqJplfw+lrNwq/0QJcCzKCaZ WJLw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=GdYICPq4CpG4O+JQQaCNuqVcH+thq20RL0BQO6HO2vwFSvaTDpD2PV6Is3XPGkiQ9E HTvdAphbMNQGZ9yYwBcRvnACyhlPl7U0VDIbABwjAzBFa5XE92peIrX5+3QGWpAwoScJ pPBuT19gRLPxOxWltAV/qwEkBXr7YKMcm8uU2W12EwZarHP11aAs/ZcYSHm9DUheMpLN QqnviTZCUdhF2skgvyieh3bqJT3d83KxHHCRpCTqUAg096Ec/9Nc2bZ2cR42QBdtom/R dM+qykYqLXhC3EUjPfip3Jyw4IpTRhXHhzpruhmhqaBmqcJbQq4zBky+TsS/9q49yFaf GFPw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zsr6VqYx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n1si87612890wre.10.2019.01.28.08.19.49 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:19:49 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=Zsr6VqYx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34424 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9dU-0002ta-Fn for patch@linaro.org; Mon, 28 Jan 2019 11:19:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33655) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K7-0005ET-H1 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:49 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007w7-V6 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:38248) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007pP-OF for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x62d.google.com with SMTP id e5so7924314plb.5 for ; Mon, 28 Jan 2019 07:59:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=Zsr6VqYxkcJI6lX9hZptFADvGzpqquZo6KTtPZ8McOxf+AO6LOEQyRWOHIMCqCbxCg ye/RYQB1EaV1kPcTJYTerPFzmdsrYJmmzQVmmUGb3BdmFEo5zvgiQEoyiaGzm8+5J+kY XqpzTYpTSOgCFqaBP2+dEkmoW4aifem7c1cyk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=UQit8ZLfJm/PCda/410nizx0XvPkSZMgAZZJczgwnZs=; b=LmAhQ+GeqAMKBEU1dzO8kidaD6dhdxF6Ae2jOtZYALfgQmF1daLPQLGOxcE6soCZJv 0Rg4/ZiK8V7gPAziOOuhizphvfbHyRQZzqanrawj8yMYfGJnczGTk2Ukq+7QjZL3lv7W 49ldDgyI+9Jxjjt/mW704uQST+DS2OIWCIxNQV8CY8KPnr++zuRr44tlFMc9t0hSj4Od IFNVmry4EjPA+E8v2jV4LblxX+gAVv/lNueHp6uiBaTtSS5wYaEGy8CfZWKsIieFBF6Z d64s8XwBJq39ZACMWTiLwbhEdWiWRnnrrSuz81D1TYT0Meem7RkdzpsnaBVMGdYFGjYw kzug== X-Gm-Message-State: AJcUukcucnfcRsUFzS8ckeEyXvmzz0m3b+9Duw8VcixG262mxrL2d+3y VqD5YpIJaM+8I8ulmTE4PWXw+gfry0o= X-Received: by 2002:a17:902:a5c3:: with SMTP id t3mr21943192plq.117.1548691171653; Mon, 28 Jan 2019 07:59:31 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.30 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:30 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:58:59 -0800 Message-Id: <20190128155907.20607-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::62d Subject: [Qemu-devel] [PULL 15/23] tcg/ppc: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/ppc/tcg-target.h | 2 +- tcg/ppc/tcg-target.inc.c | 91 ++++++++++++++++++++++------------------ 2 files changed, 52 insertions(+), 41 deletions(-) -- 2.17.2 diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index b51854b5cf..95b735b0bb 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -34,7 +34,7 @@ #define TCG_TARGET_NB_REGS 32 #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3, diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c index 8c1cfdd7ac..773690f1d9 100644 --- a/tcg/ppc/tcg-target.inc.c +++ b/tcg/ppc/tcg-target.inc.c @@ -327,6 +327,7 @@ static int tcg_target_const_match(tcg_target_long val, TCGType type, #define LHZ OPCD( 40) #define LHA OPCD( 42) #define LWZ OPCD( 32) +#define LWZUX XO31( 55) #define STB OPCD( 38) #define STH OPCD( 44) #define STW OPCD( 36) @@ -338,6 +339,7 @@ static int tcg_target_const_match(tcg_target_long val, TCGType type, #define LD XO58( 0) #define LDX XO31( 21) #define LDU XO58( 1) +#define LDUX XO31( 53) #define LWA XO58( 2) #define LWAX XO31(341) @@ -1503,6 +1505,10 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + /* Perform the TLB load and compare. Places the result of the comparison in CR7, loads the addend of the TLB into R3, and returns the register containing the guest address (zero-extended into R4). Clobbers R0 and R2. */ @@ -1513,61 +1519,63 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGMemOp opc, { int cmp_off = (is_read - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - TCGReg base = TCG_AREG0; + ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - /* Extract the page index, shifted into place for tlb index. */ - if (TCG_TARGET_REG_BITS == 64) { - if (TARGET_LONG_BITS == 32) { - /* Zero-extend the address into a place helpful for further use. */ - tcg_out_ext32u(s, TCG_REG_R4, addrlo); - addrlo = TCG_REG_R4; - } else { - tcg_out_rld(s, RLDICL, TCG_REG_R3, addrlo, - 64 - TARGET_PAGE_BITS, 64 - CPU_TLB_BITS); + if (table_off > 0x7fff) { + int mask_hi = mask_off - (int16_t)mask_off; + int table_hi = table_off - (int16_t)table_off; + + table_base = TCG_REG_R4; + if (mask_hi == table_hi) { + mask_base = table_base; + } else if (mask_hi) { + mask_base = TCG_REG_R3; + tcg_out32(s, ADDIS | TAI(mask_base, TCG_AREG0, mask_hi >> 16)); } + tcg_out32(s, ADDIS | TAI(table_base, TCG_AREG0, table_hi >> 16)); + mask_off -= mask_hi; + table_off -= table_hi; } - /* Compensate for very large offsets. */ - if (add_off >= 0x8000) { - int low = (int16_t)cmp_off; - int high = cmp_off - low; - assert((high & 0xffff) == 0); - assert(cmp_off - high == (int16_t)(cmp_off - high)); - assert(add_off - high == (int16_t)(add_off - high)); - tcg_out32(s, ADDIS | TAI(TCG_REG_TMP1, base, high >> 16)); - base = TCG_REG_TMP1; - cmp_off -= high; - add_off -= high; - } + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R4, table_base, table_off); - /* Extraction and shifting, part 2. */ - if (TCG_TARGET_REG_BITS == 32 || TARGET_LONG_BITS == 32) { - tcg_out_rlw(s, RLWINM, TCG_REG_R3, addrlo, - 32 - (TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS), - 32 - (CPU_TLB_BITS + CPU_TLB_ENTRY_BITS), - 31 - CPU_TLB_ENTRY_BITS); + /* Extract the page index, shifted into place for tlb index. */ + if (TCG_TARGET_REG_BITS == 32) { + tcg_out_shri32(s, TCG_REG_TMP1, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); } else { - tcg_out_shli64(s, TCG_REG_R3, TCG_REG_R3, CPU_TLB_ENTRY_BITS); + tcg_out_shri64(s, TCG_REG_TMP1, addrlo, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); } + tcg_out32(s, AND | SAB(TCG_REG_R3, TCG_REG_R3, TCG_REG_TMP1)); - tcg_out32(s, ADD | TAB(TCG_REG_R3, TCG_REG_R3, base)); - - /* Load the tlb comparator. */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_R4, TCG_REG_R3, cmp_off); - tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_TMP1, TCG_REG_R3, cmp_off + 4); + /* Load the TLB comparator. */ + if (cmp_off == 0 && TCG_TARGET_REG_BITS >= TARGET_LONG_BITS) { + uint32_t lxu = (TCG_TARGET_REG_BITS == 32 || TARGET_LONG_BITS == 32 + ? LWZUX : LDUX); + tcg_out32(s, lxu | TAB(TCG_REG_TMP1, TCG_REG_R3, TCG_REG_R4)); } else { - tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP1, TCG_REG_R3, cmp_off); + tcg_out32(s, ADD | TAB(TCG_REG_R3, TCG_REG_R3, TCG_REG_R4)); + if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_TMP1, TCG_REG_R3, cmp_off + 4); + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_R4, TCG_REG_R3, cmp_off); + } else { + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP1, TCG_REG_R3, cmp_off); + } } /* Load the TLB addend for use on the fast path. Do this asap to minimize any load use delay. */ - tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, TCG_REG_R3, add_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_R3, TCG_REG_R3, + offsetof(CPUTLBEntry, addend)); /* Clear the non-page, non-alignment bits from the address */ if (TCG_TARGET_REG_BITS == 32) { @@ -1600,6 +1608,9 @@ static TCGReg tcg_out_tlb_read(TCGContext *s, TCGMemOp opc, if (TARGET_LONG_BITS == 32) { tcg_out_rlw(s, RLWINM, TCG_REG_R0, t, 0, (32 - a_bits) & 31, 31 - TARGET_PAGE_BITS); + /* Zero-extend the address for use in the final address. */ + tcg_out_ext32u(s, TCG_REG_R4, addrlo); + addrlo = TCG_REG_R4; } else if (a_bits == 0) { tcg_out_rld(s, RLDICR, TCG_REG_R0, t, 0, 63 - TARGET_PAGE_BITS); } else { From patchwork Mon Jan 28 15:59:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156771 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3531575oct; Mon, 28 Jan 2019 08:11:15 -0800 (PST) X-Google-Smtp-Source: ALg8bN6AVakAJE5o5oDppxR2dENw5LgBfWqcPy5VUOA3Dld68XVAkz5KbK5AxO/+Frf9O1EJpkwR X-Received: by 2002:adf:c846:: with SMTP id e6mr21577100wrh.243.1548691875111; Mon, 28 Jan 2019 08:11:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691875; cv=none; d=google.com; s=arc-20160816; b=OXb2VmYqfS6/+gNQMZuws6uHSIlvgs/tycGSngDyXh/WOB1HsFWXU78GfhhnSdAFVF fo4V5NizEm8XHzVN+9hcmNFzqNL6Z6Awg9dylVVcYUzIFD2rk4DuC2T38DSCGmmDs/ui LTZ2xR18l9DlnL6k7/mOW2vi2a/XtXkPM1CQ9pnC8wV2vEKxetYgM4e383kh7EecT/Hj f3aPMzxzKRamjxVLOlcvKLPtRL6JD3wTqZQ6+UwN4Ax2cVAnNuixAtow2eoWG79/NbR6 dSs1tRhOTHryqNuqpzgcArMm1cuCa/G+TmVQxEJ8XS4r6gmhUGalki9HG65SDrmEzMw6 1uJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=Dw5nbSHMsvXdy8KVi1NIDw6faRz5GrF8p+oczK/QAi4=; b=HjHAvYpr5cteio0UEdm+cs5bwzNoKlEozPtZJeWFIn0M38/OR6r71vPDkExxx8d7TI GUpgaLny0gIoeuDiOYQOsd9OtOW4zPMQ8xhE72bWcgRFt7F6T8y6jb7WjU3QXB0skUHs KMzwcIvjMAqyfcsboMth+yjQdYGh2heyXJWSTTZqa59j6WstP7zZFEbcPFJ7VRZMo8E3 BunBeg8kR/MCHG6RUu224GGhOmxSyabv2wmpx3p51i05sJDeJaxSYRHf/hGLzIprofpI eWAsxmQ6PTXgt+n//FW4FZ87fN8BMJyvQwI3pVI29ebErf9l9LJtQQU1BU7MOnwEozJt j5ZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=h9jD2xGu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a8si93706804wrg.342.2019.01.28.08.11.14 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:11:15 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=h9jD2xGu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34331 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9VC-0005PV-0Y for patch@linaro.org; Mon, 28 Jan 2019 11:11:14 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33571) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005CU-Be for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007v0-4t for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:35796) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K3-0007pd-Tb for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pg1-x530.google.com with SMTP id s198so7393876pgs.2 for ; Mon, 28 Jan 2019 07:59:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=Dw5nbSHMsvXdy8KVi1NIDw6faRz5GrF8p+oczK/QAi4=; b=h9jD2xGuseFRR0EnpKPcCl8SUbPv8ygnClcIPDwbmnPZa40hje/+acnnY6redy7IvH uCKo59+utqB3FppijyZdntu68hHdxLtxgUxBLMak2hysSEpgrt99eCka/1PsXJTggKht caWWQXX4/6AIN9z3T2y6h+J6pyHGwJQ5SSpYE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=Dw5nbSHMsvXdy8KVi1NIDw6faRz5GrF8p+oczK/QAi4=; b=egOpqYsVFle1IvIJQ6bjSm8+1Vfz9OwF7r0En0kQXwATtvIY4KfuBhB1XOPBADvYpJ Osuh3vVoh3fV2KEj7lOBjNUND9vidZGRE4XVW4W/vSGt553QvZ3ER0POG7T67twxZcTw sNsHg3wsVDHfntCzan8Tn4ctkb5H911m7N9ylbhsrVAmSwyqRjOBaSLTsb7g+ImrW6oE JVw//wA3TVrL3blaL+w+5Sbo67teRAJp8FVunPhM+7peoL/O2qNcwgggVRua/QH0yzLi ggmF0DysaYj8HjQb42U7USZBt2zW3ZNkpliZA6HQkUJukhHGJcpszejm4r1aCFrzPJQI nONg== X-Gm-Message-State: AJcUukfxroGmsIJmzu+vL+zp3txCQUjURIfJRQR7o/dthrNIhCVxvpuq 89GRu0HqAU6hBvCBEiE/FvQAXIgwop8= X-Received: by 2002:a63:5d20:: with SMTP id r32mr20531423pgb.329.1548691173021; Mon, 28 Jan 2019 07:59:33 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.31 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:32 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:00 -0800 Message-Id: <20190128155907.20607-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::530 Subject: [Qemu-devel] [PULL 16/23] tcg/sparc: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/sparc/tcg-target.h | 2 +- tcg/sparc/tcg-target.inc.c | 82 +++++++++++++++++++++++--------------- 2 files changed, 51 insertions(+), 33 deletions(-) -- 2.17.2 diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h index dc0a227890..6020a670c0 100644 --- a/tcg/sparc/tcg-target.h +++ b/tcg/sparc/tcg-target.h @@ -29,7 +29,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/sparc/tcg-target.inc.c b/tcg/sparc/tcg-target.inc.c index 55144c437c..7a61839dc1 100644 --- a/tcg/sparc/tcg-target.inc.c +++ b/tcg/sparc/tcg-target.inc.c @@ -1074,54 +1074,72 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) The result of the TLB comparison is in %[ix]cc. The sanitized address is in the returned register, maybe %o0. The TLB addend is in %o1. */ +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect tlb_mask to be "near" tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) - + offsetof(CPUArchState, tlb_mask) >= (1 << 13)); + static TCGReg tcg_out_tlb_load(TCGContext *s, TCGReg addr, int mem_index, TCGMemOp opc, int which) { + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); + TCGReg base = TCG_AREG0; const TCGReg r0 = TCG_REG_O0; const TCGReg r1 = TCG_REG_O1; const TCGReg r2 = TCG_REG_O2; unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - int tlb_ofs; + tcg_target_long compare_mask; - /* Shift the page number down. */ - tcg_out_arithi(s, r1, addr, TARGET_PAGE_BITS, SHIFT_SRL); + if (!check_fit_i32(table_off, 13)) { + int table_hi; + + base = r1; + if (table_off <= 2 * 0xfff) { + table_hi = 0xfff; + tcg_out_arithi(s, base, TCG_AREG0, table_hi, ARITH_ADD); + } else { + table_hi = table_off & ~0x3ff; + tcg_out_sethi(s, base, table_hi); + tcg_out_arith(s, base, TCG_AREG0, base, ARITH_ADD); + } + mask_off -= table_hi; + table_off -= table_hi; + tcg_debug_assert(check_fit_i32(mask_off, 13)); + } + + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, TCG_TYPE_PTR, r0, base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, r1, base, table_off); + + /* Extract the page index, shifted into place for tlb index. */ + tcg_out_arithi(s, r2, addr, TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS, + SHIFT_SRL); + tcg_out_arith(s, r2, r2, r0, ARITH_AND); + + /* Add the tlb_table pointer, creating the CPUTLBEntry address into R2. */ + tcg_out_arith(s, r2, r2, r1, ARITH_ADD); + + /* Load the tlb comparator and the addend. */ + tcg_out_ld(s, TCG_TYPE_TL, r0, r2, which); + tcg_out_ld(s, TCG_TYPE_PTR, r1, r2, offsetof(CPUTLBEntry, addend)); /* Mask out the page offset, except for the required alignment. We don't support unaligned accesses. */ if (a_bits < s_bits) { a_bits = s_bits; } - tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_T1, - TARGET_PAGE_MASK | ((1 << a_bits) - 1)); - - /* Mask the tlb index. */ - tcg_out_arithi(s, r1, r1, CPU_TLB_SIZE - 1, ARITH_AND); - - /* Mask page, part 2. */ - tcg_out_arith(s, r0, addr, TCG_REG_T1, ARITH_AND); - - /* Shift the tlb index into place. */ - tcg_out_arithi(s, r1, r1, CPU_TLB_ENTRY_BITS, SHIFT_SLL); - - /* Relative to the current ENV. */ - tcg_out_arith(s, r1, TCG_AREG0, r1, ARITH_ADD); - - /* Find a base address that can load both tlb comparator and addend. */ - tlb_ofs = offsetof(CPUArchState, tlb_table[mem_index][0]); - if (!check_fit_ptr(tlb_ofs + sizeof(CPUTLBEntry), 13)) { - if (tlb_ofs & ~0x3ff) { - tcg_out_movi(s, TCG_TYPE_PTR, TCG_REG_T1, tlb_ofs & ~0x3ff); - tcg_out_arith(s, r1, r1, TCG_REG_T1, ARITH_ADD); - } - tlb_ofs &= 0x3ff; + compare_mask = (tcg_target_ulong)TARGET_PAGE_MASK | ((1 << a_bits) - 1); + if (check_fit_tl(compare_mask, 13)) { + tcg_out_arithi(s, r2, addr, compare_mask, ARITH_AND); + } else { + tcg_out_movi(s, TCG_TYPE_TL, r2, compare_mask); + tcg_out_arith(s, r2, addr, r2, ARITH_AND); } - - /* Load the tlb comparator and the addend. */ - tcg_out_ld(s, TCG_TYPE_TL, r2, r1, tlb_ofs + which); - tcg_out_ld(s, TCG_TYPE_PTR, r1, r1, tlb_ofs+offsetof(CPUTLBEntry, addend)); - - /* subcc arg0, arg2, %g0 */ tcg_out_cmp(s, r0, r2, 0); /* If the guest address must be zero-extended, do so now. */ From patchwork Mon Jan 28 15:59:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156776 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3534842oct; Mon, 28 Jan 2019 08:13:53 -0800 (PST) X-Google-Smtp-Source: ALg8bN7WGMNPTfHjnTb6zIFjE758ptZTxuBXtR9cYpgaAabD3Zz35qRnllHnYe24iiLypy9g25n/ X-Received: by 2002:adf:ecc8:: with SMTP id s8mr21772760wro.208.1548692033341; Mon, 28 Jan 2019 08:13:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692033; cv=none; d=google.com; s=arc-20160816; b=NTKvZvk+v1ucg5XJa4sTuNF/o3K9ZJQxBKeuEGra/fKsmgvAY6768jENjMpB8zoQiS Vq/0sxE3P/xp0sQnbPFm4aydPPC/aMTht3K4lWkfjC/tUeS0b6Zr0bpwlTbxbkGI3WEF 8pOxEEeAVcHs/Euil66gdWjS5uJjTndHjfkrQx/KirLhuSwwzYX67NrpYFFkPVTaeY6J jZzjmkREbXwP5g4YeCYDrDX1aATUUKlcf4uMG5yHQpmzz4TgOBbnoUYdmoEcxQKgX+IW At5a8jFIJG5LJVOdC4b97nrQq5lP3lQQy82cAtjRMobiRGAc2rF5G/BLvyQLgJpaD/NF QbHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=u1h6Y+1jeyggZCDYjdrFQDBu8S/E6FnZ3feRJf5H+uM=; b=g4UKpknr/hO95NCKE2xOAn3sYnna2IhZ3q1BIIwrNPwFyVTBIpyt5hgwuVB5+uQrc0 7t+EpO51EkoILJjRt+OC0oHx2WYf2gs3KaCo7EDh5NArjq2zhU3qRLV+a7R4bpdd7CzT DOrQRrsLvhLNfIR2vUikVLIYNoM0NAy3n0fie7KAwk12MKr5HuTaUtPSeZub8MpMLBT/ RYYsd+o6681iv5imOLb7+LBmHc3yMP8NCoCUYa21aS5sZMyJbRD5l6VRD8NkKa7bG9aT CYWdmhyRGA9awZR/rMIdNsPN6xNmHUTgummEh2MAEqeFZCrS+V7NSGHw/DWFM9W3zUnX ebjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=dLSxUdTC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t82si47420342wmb.150.2019.01.28.08.13.53 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:13:53 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=dLSxUdTC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34352 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Xk-0007YU-5e for patch@linaro.org; Mon, 28 Jan 2019 11:13:52 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33564) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005CL-7l for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007v1-4z for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:46138) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K3-0007pt-T9 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pg1-x535.google.com with SMTP id w7so7368116pgp.13 for ; Mon, 28 Jan 2019 07:59:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=u1h6Y+1jeyggZCDYjdrFQDBu8S/E6FnZ3feRJf5H+uM=; b=dLSxUdTCzq5+3zQc76tDGeEPS6ODRGSmjKlJTz+ivzRNlXOioJUkHPnjQKo5pyqUmW IEfPbSW89gv8jfg0OcwbcNXBX2YDpyXxTP8ITtXzArRiN+MVLtdaun1tu+XiAy9geDrH QzARw8OHDuEz50CtckKGb0O1lSK23c90XWFaM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=u1h6Y+1jeyggZCDYjdrFQDBu8S/E6FnZ3feRJf5H+uM=; b=qEjnbBYK2vBXCGOhvNY/7vK564pggNWOz+zmSnbDuy7BeTwg32TVNd8DGvD9lwFkwV FkZfiJyTuLvWgr9jAPzfmCTY4320UfEf1OkYWCBiRhrl9EYrW+dhsI+CLEsCDlsEyAl0 KVteBXnw3wOmrMEi8ZPkaIA1QPGNtZ5tu88VnzUC7+NX0JQXEQsYTkKvRcXpzTWrgp5H bwl3grANcHS9dVLzAdkjjm/mS6qvAywZbspEGRcKNicKlQgvV3dYnqRShHXRy4uQGa6K 1DCuwopbToA1G6Whe+dNDTJwFdiK6J8F1VSMer2WPSRV0FxddizmrrNxoXL7OuMRQoVu hmNw== X-Gm-Message-State: AJcUuke1EuJXOWl8/nPqZh8SFRZ5/0unZ8pwonJbVjqQ8q+M7iC7k3QI mrOZ/In9yjg5l/pDpTuiPIO7vJpwL2A= X-Received: by 2002:a63:b94c:: with SMTP id v12mr20263483pgo.221.1548691174301; Mon, 28 Jan 2019 07:59:34 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.33 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:33 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:01 -0800 Message-Id: <20190128155907.20607-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::535 Subject: [Qemu-devel] [PULL 17/23] tcg/s390: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/s390/tcg-target.h | 2 +- tcg/s390/tcg-target.inc.c | 45 +++++++++++++++++---------------------- 2 files changed, 20 insertions(+), 27 deletions(-) -- 2.17.2 diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h index 394b545369..357528dd97 100644 --- a/tcg/s390/tcg-target.h +++ b/tcg/s390/tcg-target.h @@ -27,7 +27,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 2 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 19 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum TCGReg { TCG_REG_R0 = 0, diff --git a/tcg/s390/tcg-target.inc.c b/tcg/s390/tcg-target.inc.c index 39ecf609a1..7db90b3bae 100644 --- a/tcg/s390/tcg-target.inc.c +++ b/tcg/s390/tcg-target.inc.c @@ -1537,10 +1537,10 @@ static void tcg_out_qemu_st_direct(TCGContext *s, TCGMemOp opc, TCGReg data, #if defined(CONFIG_SOFTMMU) #include "tcg-ldst.inc.c" -/* We're expecting to use a 20-bit signed offset on the tlb memory ops. - Using the offset of the second entry in the last tlb table ensures - that we can index all of the elements of the first entry. */ -QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table[NB_MMU_MODES - 1][1]) +/* We're expecting to use a 20-bit signed offset on the tlb memory ops. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_mask[NB_MMU_MODES - 1]) + > 0x7ffff); +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table[NB_MMU_MODES - 1]) > 0x7ffff); /* Load and compare a TLB entry, leaving the flags set. Loads the TLB @@ -1552,48 +1552,41 @@ static TCGReg tcg_out_tlb_read(TCGContext* s, TCGReg addr_reg, TCGMemOp opc, unsigned a_bits = get_alignment_bits(opc); unsigned s_mask = (1 << s_bits) - 1; unsigned a_mask = (1 << a_bits) - 1; + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); int ofs, a_off; uint64_t tlb_mask; + tcg_out_sh64(s, RSY_SRLG, TCG_REG_R2, addr_reg, TCG_REG_NONE, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + tcg_out_insn(s, RXY, NG, TCG_REG_R2, TCG_AREG0, TCG_REG_NONE, mask_off); + tcg_out_insn(s, RXY, AG, TCG_REG_R2, TCG_AREG0, TCG_REG_NONE, table_off); + /* For aligned accesses, we check the first byte and include the alignment bits within the address. For unaligned access, we check that we don't cross pages using the address of the last byte of the access. */ a_off = (a_bits >= s_bits ? 0 : s_mask - a_mask); tlb_mask = (uint64_t)TARGET_PAGE_MASK | a_mask; - - if (s390_facilities & FACILITY_GEN_INST_EXT) { - tcg_out_risbg(s, TCG_REG_R2, addr_reg, - 64 - CPU_TLB_BITS - CPU_TLB_ENTRY_BITS, - 63 - CPU_TLB_ENTRY_BITS, - 64 + CPU_TLB_ENTRY_BITS - TARGET_PAGE_BITS, 1); - if (a_off) { - tcg_out_insn(s, RX, LA, TCG_REG_R3, addr_reg, TCG_REG_NONE, a_off); - tgen_andi(s, TCG_TYPE_TL, TCG_REG_R3, tlb_mask); - } else { - tgen_andi_risbg(s, TCG_REG_R3, addr_reg, tlb_mask); - } + if ((s390_facilities & FACILITY_GEN_INST_EXT) && a_off == 0) { + tgen_andi_risbg(s, TCG_REG_R3, addr_reg, tlb_mask); } else { - tcg_out_sh64(s, RSY_SRLG, TCG_REG_R2, addr_reg, TCG_REG_NONE, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); tcg_out_insn(s, RX, LA, TCG_REG_R3, addr_reg, TCG_REG_NONE, a_off); - tgen_andi(s, TCG_TYPE_I64, TCG_REG_R2, - (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS); tgen_andi(s, TCG_TYPE_TL, TCG_REG_R3, tlb_mask); } if (is_ld) { - ofs = offsetof(CPUArchState, tlb_table[mem_index][0].addr_read); + ofs = offsetof(CPUTLBEntry, addr_read); } else { - ofs = offsetof(CPUArchState, tlb_table[mem_index][0].addr_write); + ofs = offsetof(CPUTLBEntry, addr_write); } if (TARGET_LONG_BITS == 32) { - tcg_out_mem(s, RX_C, RXY_CY, TCG_REG_R3, TCG_REG_R2, TCG_AREG0, ofs); + tcg_out_insn(s, RX, C, TCG_REG_R3, TCG_REG_R2, TCG_REG_NONE, ofs); } else { - tcg_out_mem(s, 0, RXY_CG, TCG_REG_R3, TCG_REG_R2, TCG_AREG0, ofs); + tcg_out_insn(s, RXY, CG, TCG_REG_R3, TCG_REG_R2, TCG_REG_NONE, ofs); } - ofs = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - tcg_out_mem(s, 0, RXY_LG, TCG_REG_R2, TCG_REG_R2, TCG_AREG0, ofs); + tcg_out_insn(s, RXY, LG, TCG_REG_R2, TCG_REG_R2, TCG_REG_NONE, + offsetof(CPUTLBEntry, addend)); if (TARGET_LONG_BITS == 32) { tgen_ext32u(s, TCG_REG_R3, addr_reg); From patchwork Mon Jan 28 15:59:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156790 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3618315jaa; Mon, 28 Jan 2019 08:26:09 -0800 (PST) X-Google-Smtp-Source: ALg8bN4S/BwNoE1Kx+vJo5plZW3hWjzvzFhIrIGL9rO7UmS3kyQS8B8FppS3uJ6TcomUjo6ps7EK X-Received: by 2002:a5d:4e0b:: with SMTP id p11mr24466983wrt.227.1548692769692; Mon, 28 Jan 2019 08:26:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692769; cv=none; d=google.com; s=arc-20160816; b=YwuAud3K2gGIJTrVqNPUnohTHxflX7ctnXylp9r+S3Xq9d7u8u2JynNe+ql/4PO3Zx oMLt2VHcGDCXQ6YxqfPMaPDiyYntwxuGvF42TcfCx7yXgriXr/TEOQ3WaFdhrPG1BOSD 2uIT/E/9bIwO83SvqD9Cl3VDBKOvkXoQ5f/du5W6coDgv5ALeAwvQPkXS4jzb5JMy5Wz 7om9d3eQyzn3YrRX3/ejne4YfPr4UYBAt51+22/tNTHVXqoJq4+UYMAeFPyW+jMmHRjS ktG6c3pnmyHygvlCG34rjfRdPMNjazsVO2AzrYUQM8V6MVkFEYo4V8BwCJgt3C1okz5M SYGQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=kY0UDeSFY7+trlsasRDaSGN5M2nMcaBMhIbok6UiKjh0pebRqbG/8HBSTnHsaVTwU0 g0uGZauVGjhV4yrOUxXhOOFD8ebmvzwXWFqjGfiWj+ccZNmMxHQtzQV1vYtRWqhhqtw6 abpWvNTEJ6I8O1NQnQpExrVkeTm4fDweC78es2h/mB/aHwBGudLLoIEcr6OPDl2ov3NG IDsdRo59aEZtk1wM7JbauqQ7yPsYqKdTRWI55+6Tpe8ZKLDO8O6fgwWUrGMTnZh9fi2e E7jO1z9Ii6zRmtIIQRWGpChWnOcnmcoXXqp6G5Y3J57BpdNi1ZA73rpgDMyCxD2LK4Ol 30mQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iD0lRo0N; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 128si47670013wmd.69.2019.01.28.08.26.09 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:26:09 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iD0lRo0N; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34559 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9jc-0000kw-FC for patch@linaro.org; Mon, 28 Jan 2019 11:26:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33678) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K8-0005FS-Dv for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vu-S1 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:48 -0500 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]:33948) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007qH-Kb for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pf1-x429.google.com with SMTP id h3so8199392pfg.1 for ; Mon, 28 Jan 2019 07:59:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=iD0lRo0NTGl8UGklwhXcEVKqgrfNVel0H+NI9dwnyQLWic4MSgzk+61vg29RpVFFHQ jgL5qTCWmTQbwF+ePP+hmGF1cD5BagfcBmO6bzVxVzP2ERMnK3xFVIXEtdYIR2ybJgNw +eDYa+/gmmEc2AWJPbYE+crqTaBMO5j56lNHU= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=0sDuYnqvRz5TtD+qB2nm3nVGkjJvEbzT2FXAL0tduOU=; b=sTVGlLQaa/g9+WOqsixZQx0z4/STN+T3+b7eXZm0iNK0mJxWqhdn9dwDGFyKyZtPQV /7rCRlbFqxMZ+MVz9c5DtQ5WED0ARjnoAPnssXftwpNT3sTCGuFE63Tn7rEJdPb9krYw w4TiIwV51r//a/l6nYQ4cgbD3bVZjPW/lE5Wq8PbdG7E3A/bhcNiiRmiDpdDVW3aJNyy LZfuf2shbGgPEykcAJLbTcBo2TA7ewgsvtZtsIvKfnJFsVF4MvcvJyLYzRn4+b6gfgPC sGZHwzmnFA6CxiP0f+p/l/3kHuJ6FEe97EtfUyKdzfAR/JeBzuypJZTU+hJJE2sZVDcR YYDg== X-Gm-Message-State: AJcUukeGpAruSAR3tKOkIBoihZhNvSxFXQkYaF+AEAAS1M79EA6XWys9 OQsTbu6DJH6/UdvbM5ZB4Nh59D7xgXM= X-Received: by 2002:a63:2141:: with SMTP id s1mr20473994pgm.148.1548691175739; Mon, 28 Jan 2019 07:59:35 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.34 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:34 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:02 -0800 Message-Id: <20190128155907.20607-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::429 Subject: [Qemu-devel] [PULL 18/23] tcg/riscv: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Reviewed-by: Alistair Francis Signed-off-by: Richard Henderson --- tcg/riscv/tcg-target.h | 2 +- tcg/riscv/tcg-target.inc.c | 126 ++++++++++++++++--------------------- 2 files changed, 56 insertions(+), 72 deletions(-) -- 2.17.2 diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 1eb032626c..83b123ca03 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -33,7 +33,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/riscv/tcg-target.inc.c b/tcg/riscv/tcg-target.inc.c index 6cf8de32b5..b785f4acb7 100644 --- a/tcg/riscv/tcg-target.inc.c +++ b/tcg/riscv/tcg-target.inc.c @@ -958,6 +958,17 @@ static void * const qemu_st_helpers[16] = { [MO_BEQ] = helper_be_stq_mmu, }; +/* We don't support oversize guests */ +QEMU_BUILD_BUG_ON(TCG_TARGET_REG_BITS < TARGET_LONG_BITS); + +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect tlb_mask to be "near" tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) - + offsetof(CPUArchState, tlb_mask) >= 0x800); + static void tcg_out_tlb_load(TCGContext *s, TCGReg addrl, TCGReg addrh, TCGMemOpIdx oi, tcg_insn_unit **label_ptr, bool is_load) @@ -965,94 +976,67 @@ static void tcg_out_tlb_load(TCGContext *s, TCGReg addrl, TCGMemOp opc = get_memop(oi); unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - target_ulong mask; + tcg_target_long compare_mask; int mem_index = get_mmuidx(oi); - int cmp_off - = (is_load - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - RISCVInsn load_cmp_op = (TARGET_LONG_BITS == 64 ? OPC_LD : - TCG_TARGET_REG_BITS == 64 ? OPC_LWU : OPC_LW); - RISCVInsn load_add_op = TCG_TARGET_REG_BITS == 64 ? OPC_LD : OPC_LW; - TCGReg base = TCG_AREG0; + int mask_off, table_off; + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; - /* We don't support oversize guests */ - if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - g_assert_not_reached(); + mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + table_off = offsetof(CPUArchState, tlb_table[mem_index]); + if (table_off > 0x7ff) { + int mask_hi = mask_off - sextreg(mask_off, 0, 12); + int table_hi = table_off - sextreg(table_off, 0, 12); + + if (likely(mask_hi == table_hi)) { + mask_base = table_base = TCG_REG_TMP1; + tcg_out_opc_upper(s, OPC_LUI, mask_base, mask_hi); + tcg_out_opc_reg(s, OPC_ADD, mask_base, mask_base, TCG_AREG0); + mask_off -= mask_hi; + table_off -= mask_hi; + } else { + mask_base = TCG_REG_TMP0; + table_base = TCG_REG_TMP1; + tcg_out_opc_upper(s, OPC_LUI, mask_base, mask_hi); + tcg_out_opc_reg(s, OPC_ADD, mask_base, mask_base, TCG_AREG0); + table_off -= mask_off; + mask_off -= mask_hi; + tcg_out_opc_imm(s, OPC_ADDI, table_base, mask_base, mask_off); + } } + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP1, table_base, table_off); + + tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP2, addrl, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP0); + tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, TCG_REG_TMP2, TCG_REG_TMP1); + + /* Load the tlb comparator and the addend. */ + tcg_out_ld(s, TCG_TYPE_TL, TCG_REG_TMP0, TCG_REG_TMP2, + is_load ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP2, TCG_REG_TMP2, + offsetof(CPUTLBEntry, addend)); + /* We don't support unaligned accesses. */ if (a_bits < s_bits) { a_bits = s_bits; } - mask = (target_ulong)TARGET_PAGE_MASK | ((1 << a_bits) - 1); - - - /* Compensate for very large offsets. */ - if (add_off >= 0x1000) { - int adj; - base = TCG_REG_TMP2; - if (cmp_off <= 2 * 0xfff) { - adj = 0xfff; - tcg_out_opc_imm(s, OPC_ADDI, base, TCG_AREG0, adj); - } else { - adj = cmp_off - sextreg(cmp_off, 0, 12); - tcg_debug_assert(add_off - adj >= -0x1000 - && add_off - adj < 0x1000); - - tcg_out_opc_upper(s, OPC_LUI, base, adj); - tcg_out_opc_reg(s, OPC_ADD, base, base, TCG_AREG0); - } - add_off -= adj; - cmp_off -= adj; - } - - /* Extract the page index. */ - if (CPU_TLB_BITS + CPU_TLB_ENTRY_BITS < 12) { - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, addrl, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP0, TCG_REG_TMP0, - MAKE_64BIT_MASK(CPU_TLB_ENTRY_BITS, CPU_TLB_BITS)); - } else if (TARGET_PAGE_BITS >= 12) { - tcg_out_opc_upper(s, OPC_LUI, TCG_REG_TMP0, - MAKE_64BIT_MASK(TARGET_PAGE_BITS, CPU_TLB_BITS)); - tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP0, TCG_REG_TMP0, addrl); - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, TCG_REG_TMP0, - CPU_TLB_BITS - CPU_TLB_ENTRY_BITS); - } else { - tcg_out_opc_imm(s, OPC_SRLI, TCG_REG_TMP0, addrl, TARGET_PAGE_BITS); - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP0, TCG_REG_TMP0, - MAKE_64BIT_MASK(0, CPU_TLB_BITS)); - tcg_out_opc_imm(s, OPC_SLLI, TCG_REG_TMP0, TCG_REG_TMP0, - CPU_TLB_ENTRY_BITS); - } - - /* Add that to the base address to index the tlb. */ - tcg_out_opc_reg(s, OPC_ADD, TCG_REG_TMP2, base, TCG_REG_TMP0); - base = TCG_REG_TMP2; - - /* Load the tlb comparator and the addend. */ - tcg_out_ldst(s, load_cmp_op, TCG_REG_TMP0, base, cmp_off); - tcg_out_ldst(s, load_add_op, TCG_REG_TMP2, base, add_off); - /* Clear the non-page, non-alignment bits from the address. */ - if (mask == sextreg(mask, 0, 12)) { - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addrl, mask); + compare_mask = (tcg_target_long)TARGET_PAGE_MASK | ((1 << a_bits) - 1); + if (compare_mask == sextreg(compare_mask, 0, 12)) { + tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_TMP1, addrl, compare_mask); } else { - tcg_out_movi(s, TCG_TYPE_REG, TCG_REG_TMP1, mask); + tcg_out_movi(s, TCG_TYPE_TL, TCG_REG_TMP1, compare_mask); tcg_out_opc_reg(s, OPC_AND, TCG_REG_TMP1, TCG_REG_TMP1, addrl); - } + } /* Compare masked address with the TLB entry. */ label_ptr[0] = s->code_ptr; tcg_out_opc_branch(s, OPC_BNE, TCG_REG_TMP0, TCG_REG_TMP1, 0); /* NOP to allow patching later */ tcg_out_opc_imm(s, OPC_ADDI, TCG_REG_ZERO, TCG_REG_ZERO, 0); - /* TODO: Move this out of line - * see: - * https://lists.nongnu.org/archive/html/qemu-devel/2018-11/msg02234.html - */ /* TLB Hit - translate address using addend. */ if (TCG_TARGET_REG_BITS > TARGET_LONG_BITS) { From patchwork Mon Jan 28 15:59:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156774 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3533954oct; Mon, 28 Jan 2019 08:13:09 -0800 (PST) X-Google-Smtp-Source: ALg8bN51i8hXHq1sZf4oriImLIV0uX1aHn3WQ7O14jEyaa5lkBtpW3iiiohYH3l9OS1OWZwikclw X-Received: by 2002:a5d:4fcb:: with SMTP id h11mr22846264wrw.139.1548691989263; Mon, 28 Jan 2019 08:13:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691989; cv=none; d=google.com; s=arc-20160816; b=mEYBmCbe5K2KDWVUwK/KXwtz9sWwnhLWkbAV6mHohRs/+NebuEaddDCXEgwJ+PjKZx mqD395dsgHdyqD7CI+Ei4P5HXuop+2Qvrn8jSJDED4JPLz/68Igu+qs5e2DihWM5c0Y/ 5wQdyje7PlM9MOLvbDJoau5kLTRUzk6Pmnqbi8ka9YNflvaZlFBIC64U+rBRCEqdq+a7 1m4v1g+TpwT4lX6tHQ5Qk1/ORqmh9I4AD1An12NsNZOzNWMUhf/oXwWE7R3BJRxPUIrU YwY3AJ4RjCP5uCiDfboVqWJaypwHQtjIx0mFNAzbAc0v+2+hIZ4x+inMD7ZAJJAnDz8Z 2/KA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=C2BrQaGdJrcnzidVlPeIc02gojy3eZ+R8AGWPoviy0s=; b=R1squmBoPFXoCAOV7rQaMGweqYG2oF2Slin08Z8OLggy8QofP9XmDJm1BoMzWnct39 S3GRE4Z84bQUDw1Rw/wfFCm21gYQYbvZGfxnmykHc/Yb9PHd7CYBCdT+jIyacV740ZbV eOcuS3faojyD+RRaOeNp2MHanITjh9PbYniz4I90C39XgEaezdVYsD0HAGXb/ABIjChq vvh514qImP3ieHj8uNVChB4vzJDvZn4WsahKd9VE7wiZDF5PShCreYFcle+d/2H3yrg2 3SojvIm77L0NYJGjr+CO0+2fHefC/9F6RBiOIOiaotIYwgPyj0aifDzjiAYv9oki2sPJ Gcjg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kamK8Rk4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l22si46010446wmc.35.2019.01.28.08.13.08 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:13:09 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=kamK8Rk4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34345 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9X2-0006uL-41 for patch@linaro.org; Mon, 28 Jan 2019 11:13:08 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33613) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K6-0005DR-G7 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vb-Km for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:46 -0500 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]:35950) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007qY-8i for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pf1-x42a.google.com with SMTP id b85so8190178pfc.3 for ; Mon, 28 Jan 2019 07:59:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=C2BrQaGdJrcnzidVlPeIc02gojy3eZ+R8AGWPoviy0s=; b=kamK8Rk4a+jCO6iqsF2h6hjY6yq8DGKjuc70zmq6769+hxP3u+XjRPusFjiEQHLVBS Us7YcQlkNEcwCjS2deCjFA4gPotVEU+SR7YeHdYVhRKf5HOTPo55h6Z4Qk146NsrUsqk 2JLw9NKyVWOCIMCZNtlbLsxJW8sfqtCecdGpo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=C2BrQaGdJrcnzidVlPeIc02gojy3eZ+R8AGWPoviy0s=; b=h4hPOiDyvzAG8ZsQz0AeSxbnwXt7Vb/iyeHBNuAI0WdV8X6Pmaa2fni+MvXvaRS4jm li5AfoGE2BvhiRjjy9nB2AruLi8h0S3/TDzWi6H+v5LZxubBZVVn6JajhOkxiN8RlgZ8 OryGBKkvzwpDul446FCKY41g9Ea+jzbyVSWnGGgTw8j005NGaitv02NxRE6Lu2ZQ5/mv xQD3NsfkPGYaSglY3KUFm9yoXg7o3hrRRuj0jvKJEGltrqdBoTQhbe0tmFJfEpigPpjI OYxjy3PFqR+t2pKsa4FKUjK8cdyKiuay6HwjqF5Kg5+jmPm3Jj3ZmnNiHdpm6wggyuhU uGlw== X-Gm-Message-State: AJcUukfjQHf3oy0eSvnjlRR+U82aFzQCRMzNtMQnmc1Uh5/r4TCiDLEo 1qyl5r90pv9tLNelsiRFxM8fDuL0Gcs= X-Received: by 2002:a63:e001:: with SMTP id e1mr14102341pgh.39.1548691177092; Mon, 28 Jan 2019 07:59:37 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.35 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:36 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:03 -0800 Message-Id: <20190128155907.20607-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42a Subject: [Qemu-devel] [PULL 19/23] tcg/arm: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/arm/tcg-target.h | 2 +- tcg/arm/tcg-target.inc.c | 143 +++++++++++++++++++-------------------- 2 files changed, 72 insertions(+), 73 deletions(-) -- 2.17.2 diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index c5a7064bdc..679aaf097e 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -60,7 +60,7 @@ extern int arm_arch; #undef TCG_TARGET_STACK_GROWSUP #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum { TCG_REG_R0 = 0, diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c index 49f57d655e..2245a8aeb9 100644 --- a/tcg/arm/tcg-target.inc.c +++ b/tcg/arm/tcg-target.inc.c @@ -500,6 +500,12 @@ static inline void tcg_out_ldrd_r(TCGContext *s, int cond, TCGReg rt, tcg_out_memop_r(s, cond, INSN_LDRD_REG, rt, rn, rm, 1, 1, 0); } +static inline void tcg_out_ldrd_rwb(TCGContext *s, int cond, TCGReg rt, + TCGReg rn, TCGReg rm) +{ + tcg_out_memop_r(s, cond, INSN_LDRD_REG, rt, rn, rm, 1, 1, 1); +} + static inline void tcg_out_strd_8(TCGContext *s, int cond, TCGReg rt, TCGReg rn, int imm8) { @@ -1229,8 +1235,13 @@ static TCGReg tcg_out_arg_reg64(TCGContext *s, TCGReg argreg, #define TLB_SHIFT (CPU_TLB_ENTRY_BITS + CPU_TLB_BITS) -/* We're expecting to use an 8-bit immediate and to mask. */ -QEMU_BUILD_BUG_ON(CPU_TLB_BITS > 8); +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect to use a 20-bit unsigned offset from ENV. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table[NB_MMU_MODES - 1]) + > 0xfffff); /* Load and compare a TLB entry, leaving the flags set. Returns the register containing the addend of the tlb entry. Clobbers R0, R1, R2, TMP. */ @@ -1238,84 +1249,72 @@ QEMU_BUILD_BUG_ON(CPU_TLB_BITS > 8); static TCGReg tcg_out_tlb_read(TCGContext *s, TCGReg addrlo, TCGReg addrhi, TCGMemOp opc, int mem_index, bool is_load) { - TCGReg base = TCG_AREG0; - int cmp_off = - (is_load - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); - int mask_off; + int cmp_off = (is_load ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - /* V7 generates the following: - * ubfx r0, addrlo, #TARGET_PAGE_BITS, #CPU_TLB_BITS - * add r2, env, #high - * add r2, r2, r0, lsl #CPU_TLB_ENTRY_BITS - * ldr r0, [r2, #cmp] - * ldr r2, [r2, #add] - * movw tmp, #page_align_mask - * bic tmp, addrlo, tmp - * cmp r0, tmp - * - * Otherwise we generate: - * shr tmp, addrlo, #TARGET_PAGE_BITS - * add r2, env, #high - * and r0, tmp, #(CPU_TLB_SIZE - 1) - * add r2, r2, r0, lsl #CPU_TLB_ENTRY_BITS - * ldr r0, [r2, #cmp] - * ldr r2, [r2, #add] - * tst addrlo, #s_mask - * cmpeq r0, tmp, lsl #TARGET_PAGE_BITS - */ - if (use_armv7_instructions) { - tcg_out_extract(s, COND_AL, TCG_REG_R0, addrlo, - TARGET_PAGE_BITS, CPU_TLB_BITS); - } else { - tcg_out_dat_reg(s, COND_AL, ARITH_MOV, TCG_REG_TMP, - 0, addrlo, SHIFT_IMM_LSR(TARGET_PAGE_BITS)); - } + if (table_off > 0xfff) { + int mask_hi = mask_off & ~0xfff; + int table_hi = table_off & ~0xfff; + int rot; - /* Add portions of the offset until the memory access is in range. - * If we plan on using ldrd, reduce to an 8-bit offset; otherwise - * we can use a 12-bit offset. */ - if (use_armv6_instructions && TARGET_LONG_BITS == 64) { - mask_off = 0xff; - } else { - mask_off = 0xfff; - } - while (cmp_off > mask_off) { - int shift = ctz32(cmp_off & ~mask_off) & ~1; - int rot = ((32 - shift) << 7) & 0xf00; - int addend = cmp_off & (0xff << shift); - tcg_out_dat_imm(s, COND_AL, ARITH_ADD, TCG_REG_R2, base, - rot | ((cmp_off >> shift) & 0xff)); - base = TCG_REG_R2; - add_off -= addend; - cmp_off -= addend; - } - - if (!use_armv7_instructions) { - tcg_out_dat_imm(s, COND_AL, ARITH_AND, - TCG_REG_R0, TCG_REG_TMP, CPU_TLB_SIZE - 1); - } - tcg_out_dat_reg(s, COND_AL, ARITH_ADD, TCG_REG_R2, base, - TCG_REG_R0, SHIFT_IMM_LSL(CPU_TLB_ENTRY_BITS)); - - /* Load the tlb comparator. Use ldrd if needed and available, - but due to how the pointer needs setting up, ldm isn't useful. - Base arm5 doesn't have ldrd, but armv5te does. */ - if (use_armv6_instructions && TARGET_LONG_BITS == 64) { - tcg_out_ldrd_8(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); - } else { - tcg_out_ld32_12(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); - if (TARGET_LONG_BITS == 64) { - tcg_out_ld32_12(s, COND_AL, TCG_REG_R1, TCG_REG_R2, cmp_off + 4); + table_base = TCG_REG_R2; + if (mask_hi == table_hi) { + mask_base = table_base; + } else if (mask_hi) { + mask_base = TCG_REG_TMP; + rot = encode_imm(mask_hi); + assert(rot >= 0); + tcg_out_dat_imm(s, COND_AL, ARITH_ADD, mask_base, TCG_AREG0, + rotl(mask_hi, rot) | (rot << 7)); } + rot = encode_imm(table_hi); + assert(rot >= 0); + tcg_out_dat_imm(s, COND_AL, ARITH_ADD, table_base, TCG_AREG0, + rotl(table_hi, rot) | (rot << 7)); + + mask_off -= mask_hi; + table_off -= table_hi; + } + + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_TMP, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_I32, TCG_REG_R2, table_base, table_off); + + /* Extract the tlb index from the address into TMP. */ + tcg_out_dat_reg(s, COND_AL, ARITH_AND, TCG_REG_TMP, TCG_REG_TMP, addrlo, + SHIFT_IMM_LSR(TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS)); + + /* + * Add the tlb_table pointer, creating the CPUTLBEntry address in R2. + * Load the tlb comparator into R0/R1 and the fast path addend into R2. + */ + if (cmp_off == 0) { + if (use_armv6_instructions && TARGET_LONG_BITS == 64) { + tcg_out_ldrd_rwb(s, COND_AL, TCG_REG_R0, TCG_REG_R2, TCG_REG_TMP); + } else { + tcg_out_ld32_rwb(s, COND_AL, TCG_REG_R0, TCG_REG_R2, TCG_REG_TMP); + } + } else { + tcg_out_dat_reg(s, COND_AL, ARITH_ADD, + TCG_REG_R2, TCG_REG_R2, TCG_REG_TMP, 0); + if (use_armv6_instructions && TARGET_LONG_BITS == 64) { + tcg_out_ldrd_8(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); + } else { + tcg_out_ld32_12(s, COND_AL, TCG_REG_R0, TCG_REG_R2, cmp_off); + } + } + if (!use_armv6_instructions && TARGET_LONG_BITS == 64) { + tcg_out_ld32_12(s, COND_AL, TCG_REG_R1, TCG_REG_R2, cmp_off + 4); } /* Load the tlb addend. */ - tcg_out_ld32_12(s, COND_AL, TCG_REG_R2, TCG_REG_R2, add_off); + tcg_out_ld32_12(s, COND_AL, TCG_REG_R2, TCG_REG_R2, + offsetof(CPUTLBEntry, addend)); /* Check alignment. We don't support inline unaligned acceses, but we can easily support overalignment checks. */ From patchwork Mon Jan 28 15:59:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156773 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3533364oct; Mon, 28 Jan 2019 08:12:41 -0800 (PST) X-Google-Smtp-Source: AHgI3IbB4A99UyFDIt6Njwqdo1JTXqI8zd6vZWIpO1Ocit6Nrp0vE1sySx8mnXG3wkw28W6hBgWJ X-Received: by 2002:adf:f3c6:: with SMTP id g6mr18010751wrp.111.1548691961709; Mon, 28 Jan 2019 08:12:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548691961; cv=none; d=google.com; s=arc-20160816; b=M/6mY0Gzw7L01uBn+TXuPD8CqFoIKJi4VM4h8ZQ+++hH1n4vDpQuTDYsagVEdAeRB9 dE8UV/wdjkCUsa4AjszEntKqvJxdHxghTlmwsP9CvXhBqeQmi4ky1VNbnLtv8uVODyXr 8gLvcoTwMIfC8JEMhtEfOOVUb403aWCJSskECxZDRp/fT0m835WLh6MPo4f4wtGaoLfg Kg9P8JOIDCEa/SO1LwUSKmobNNH0+C8DjTmbMBD2jbUlgvQbkMgnAHnhSYMmXODeAcqM b8QexOrP+a4DG3cF/kuxGUvvpkBeb+bw3XpDO2AuZjYxkQtKyYJU74DZQNkkB6BhSkKp p1dA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=+L0wBkxKTUeVI/04FDDLFSjFaBD1pRVqKJYILvbKu24=; b=J9lcSPHwsJUN9AWFAAqwP2Nf5i5FDo8jVPEZbfUOEfy4ilF/d5J7J6OvXi5D2WNuvd yFSPtlZuLeeVcxglwaOpwQDVeAP50yQuSq0PmF/r1yLty0JGQezgFAWQE2JLNTsKQ3nI mllygLXQ2f7XSuEaqI+2RZU7ZyIt56+031njuRTJZXPpObamaNJX4GBLpctQG3h0Ns4+ ovKp/E77Tjf+vt0+ywbU64FCmlZxSwoaD9t/VUXb/9PL4zav+3wuFA9Auke9GlHo5rWU 7Ts3GPjo93RGXHXfp9xZA/3QRi2FdNoIm5RD0R8W3e9Aw+Oi1riiQ/yJZ6QIY1NwqJ4B pJHQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=gnLBcltK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g185si49092566wmd.156.2019.01.28.08.12.41 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:12:41 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=gnLBcltK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34333 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Wa-0005lD-Hq for patch@linaro.org; Mon, 28 Jan 2019 11:12:40 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33582) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005Cj-MU for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:48 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vh-Mf for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]:35951) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007r7-EQ for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pf1-x42b.google.com with SMTP id b85so8190205pfc.3 for ; Mon, 28 Jan 2019 07:59:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+L0wBkxKTUeVI/04FDDLFSjFaBD1pRVqKJYILvbKu24=; b=gnLBcltKUVg11sRq3MtKiFSnchmvi8BpnG0QHM5o0aeSwufE9yDmp+Ppee0Wg03m0U jKMZXcm6LzKLp1wp2x4BkNz59OmAZFLoSK/HthohXhTOhrJ/GRmT3L1GXcg2RpfSSf7X yvhwPmc6Za3duBpjP3A31HzuyIeIl0IMg1EcM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+L0wBkxKTUeVI/04FDDLFSjFaBD1pRVqKJYILvbKu24=; b=XZqMcva5R/40gfU9tXfuBn1OAuOs8W8y2M8pxcLOf06i5Opn+tBM4Xn+/eVzVDHpln xtDGbfRJbVwMjr6MWmK7Mh+glnumnACTW+wmhyShKUkpJZqsrp9E6m1UQkh2t/P+dqzM BuLJoImpu4aOsUpazU+iHW0HctPoWpFqlzt5SGIBPhbPAoMGcxf46iqHktgMvouwCLL8 gKOHEjoQSKeLrN9PRY45AxLTbJpeFhZiTeHcVVLhboTwbYDyo0Fzz6FzB3woGiDoGBa6 s23ijl2YuPbg6jvX3/oq6GrJk9TgMNP1eHY8u8Gniqn7KH2+6Aq+iDCOkPS1EDwh5cvv TGJw== X-Gm-Message-State: AJcUukew72du9Hx2LXf6+13y6Giat6cjQ0hkDEwhfy0rErIHUHUJX2F8 Z02Vy8AdWTmfe6th+tbSRFC7LjpeEik= X-Received: by 2002:a63:9712:: with SMTP id n18mr20015429pge.295.1548691178670; Mon, 28 Jan 2019 07:59:38 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:37 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:04 -0800 Message-Id: <20190128155907.20607-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::42b Subject: [Qemu-devel] [PULL 20/23] tcg/mips: Fix tcg_out_qemu_ld_slow_path X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Patch the branch after it has been emitted rather than before it exists. Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.inc.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/mips/tcg-target.inc.c b/tcg/mips/tcg-target.inc.c index be0bc92e8e..c5d7067f89 100644 --- a/tcg/mips/tcg-target.inc.c +++ b/tcg/mips/tcg-target.inc.c @@ -1343,8 +1343,9 @@ static void tcg_out_qemu_ld_slow_path(TCGContext *s, TCGLabelQemuLdst *l) } } - reloc_pc16(s->code_ptr, l->raddr); tcg_out_opc_br(s, OPC_BEQ, TCG_REG_ZERO, TCG_REG_ZERO); + reloc_pc16(s->code_ptr - 1, l->raddr); + /* delay slot */ if (TCG_TARGET_REG_BITS == 64 && l->type == TCG_TYPE_I32) { /* we always sign-extend 32-bit loads */ From patchwork Mon Jan 28 15:59:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156777 Delivered-To: patch@linaro.org Received: by 2002:ac9:7558:0:0:0:0:0 with SMTP id r24csp3535155oct; Mon, 28 Jan 2019 08:14:08 -0800 (PST) X-Google-Smtp-Source: ALg8bN4SGtb3QldGjRn+L1KfgRySReY8Zl7EYZXQ1hHY6sPFkDkVgj7qta1i+lyU/z5fA7ZvcjsO X-Received: by 2002:adf:a357:: with SMTP id d23mr23487777wrb.195.1548692048003; Mon, 28 Jan 2019 08:14:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692047; cv=none; d=google.com; s=arc-20160816; b=bXrsAFIBkRLtaeA5dnrpbGF7jMbK5pDan3mU181qBPwyNUkHh7xkww0Uo7sjpy1RXL sOLaJ5D1RF7suI8S43kAun1Tq6DXZhkDSA4cz75hicM/DpdjsddvELn1icP3zTjEGwdW 1m6JX3b2yE6bPTgzQo3mdQO6y2YD3aM1NWBs/IlvYpx48nRL0tN+TGU7UeEPNfWZ3P1K vMaWZh103i1xqHj7wzKFe0FuV1viVK69n5hDtADf6XGNunL+2xn7GCScvmY6JAZwWw23 NKp0mBRLZiRNneiXnt1LN6X2ObMstFk340ehULiEmK8Ndn8q5F0agUa7L1ZxuaU/IHMO dH2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=+jpzDaVjPXUUB7/HTmam1U+UM2kIhlzghJN+I1PSa8c=; b=cjTyEKqH56+PMm+NK7EXhw96D+YwWPiBfm7C3rOxG4ARQUUsLHflxU7sun3DHYxY+Z 15Ofaw76unO1mh3U4gO2e4lomjm6DNGANYmAyJAvm/c+ahyDrtQiGtZtMZm/tON+Lxp5 jmIWva4wGFHEdTvgFkKTz3ux8fFTXlqGcrOcPKzdkmsch4aVuV+3GUUNHhPmLgyKu7Lk 0NNMW7hxQrE/AfJQl1olBiVocbKIOcx35c/zdvDX5y9TtLk77wi6d92myzFsn+fiW30d indYCtyP6HT/C+LC4Tw4xVs+akvBzHzaoKpdUUfAhaKBMBLLSYy3jBHiHisi9NvDSlXJ 1Xsw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VDdN6yit; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n9si45854873wmi.179.2019.01.28.08.14.07 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:14:07 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VDdN6yit; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34356 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9Xy-0007hN-Rk for patch@linaro.org; Mon, 28 Jan 2019 11:14:06 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33577) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005CX-G7 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007uq-2d for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:45610) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K3-0007sC-Qe for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:43 -0500 Received: by mail-pl1-x630.google.com with SMTP id a14so7896002plm.12 for ; Mon, 28 Jan 2019 07:59:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=+jpzDaVjPXUUB7/HTmam1U+UM2kIhlzghJN+I1PSa8c=; b=VDdN6yitYF3WKVSIMo0oM0kNa1B/igzWJgJAMC3p5WdgEISqY0O05ArPamC0Gzqqxi kdTzRsd5Z8koVeArXhqmT2DgowkiPSHsQ4NHEZ1vJeILG+kF8Tspkx/61rgGK66t11ck 7cc0ZXPy5YwAbGGXk2ScJuMLKqxqNnEqnBw60= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=+jpzDaVjPXUUB7/HTmam1U+UM2kIhlzghJN+I1PSa8c=; b=UilwNDwqT231d6bDikrmxFK68Vlr09PX5EJ3XtWL6pXhBCVi05I7QK8I2+Rlg2p5mJ IT7AHMdPGe/HwIDNT2Dko/O0pssr38kdKY3ZkIe+2emQvoZWbxoZkLzjJVll8T4N/k5X h+dj2DfpkJH0IHyHrNVZy+8uM9igxt/yvfU80Xduslk7PzVPX01um1/0ENbKmXaaHQVc 4QjwmNPy9THFGKSv/WWe4zYic19o50JBhEnaa5HWIiAkanLesbxA1DRpjz3Xm6y9DDmX wSJLKZU/AfhMvLfoEJv0Xmyj6zwZpuiHhpoBiTo34Uoj9sE+hxJr8yfwntgof+iraxZ7 TMZA== X-Gm-Message-State: AJcUukdh8KdPhF7uJndDq/yJ5zNUyBXfE5UZ7FnlME6Vkr6BdcQr4oZv wK1lpJA9wn50olbVKhhMQdn5S6PP+lA= X-Received: by 2002:a17:902:4:: with SMTP id 4mr22452401pla.20.1548691180160; Mon, 28 Jan 2019 07:59:40 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.38 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:39 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:05 -0800 Message-Id: <20190128155907.20607-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::630 Subject: [Qemu-devel] [PULL 21/23] tcg/mips: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- tcg/mips/tcg-target.h | 2 +- tcg/mips/tcg-target.inc.c | 94 ++++++++++++++++++++++++++------------- 2 files changed, 64 insertions(+), 32 deletions(-) -- 2.17.2 diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 8600eefd9a..40adbe38cb 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -37,7 +37,7 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/mips/tcg-target.inc.c b/tcg/mips/tcg-target.inc.c index c5d7067f89..8a92e916dd 100644 --- a/tcg/mips/tcg-target.inc.c +++ b/tcg/mips/tcg-target.inc.c @@ -1201,8 +1201,19 @@ static int tcg_out_call_iarg_reg2(TCGContext *s, int i, TCGReg al, TCGReg ah) return i; } -/* Perform the tlb comparison operation. The complete host address is - placed in BASE. Clobbers TMP0, TMP1, TMP2, A0. */ +/* We expect tlb_mask to be before tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) < + offsetof(CPUArchState, tlb_mask)); + +/* We expect tlb_mask to be "near" tlb_table. */ +QEMU_BUILD_BUG_ON(offsetof(CPUArchState, tlb_table) - + offsetof(CPUArchState, tlb_mask) >= 0x8000); + +/* + * Perform the tlb comparison operation. + * The complete host address is placed in BASE. + * Clobbers TMP0, TMP1, TMP2, TMP3. + */ static void tcg_out_tlb_load(TCGContext *s, TCGReg base, TCGReg addrl, TCGReg addrh, TCGMemOpIdx oi, tcg_insn_unit *label_ptr[2], bool is_load) @@ -1210,52 +1221,73 @@ static void tcg_out_tlb_load(TCGContext *s, TCGReg base, TCGReg addrl, TCGMemOp opc = get_memop(oi); unsigned s_bits = opc & MO_SIZE; unsigned a_bits = get_alignment_bits(opc); - target_ulong mask; int mem_index = get_mmuidx(oi); - int cmp_off - = (is_load - ? offsetof(CPUArchState, tlb_table[mem_index][0].addr_read) - : offsetof(CPUArchState, tlb_table[mem_index][0].addr_write)); - int add_off = offsetof(CPUArchState, tlb_table[mem_index][0].addend); + int mask_off = offsetof(CPUArchState, tlb_mask[mem_index]); + int table_off = offsetof(CPUArchState, tlb_table[mem_index]); + int add_off = offsetof(CPUTLBEntry, addend); + int cmp_off = (is_load ? offsetof(CPUTLBEntry, addr_read) + : offsetof(CPUTLBEntry, addr_write)); + TCGReg mask_base = TCG_AREG0, table_base = TCG_AREG0; + target_ulong mask; - tcg_out_opc_sa(s, ALIAS_TSRL, TCG_REG_A0, addrl, - TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); - tcg_out_opc_imm(s, OPC_ANDI, TCG_REG_A0, TCG_REG_A0, - (CPU_TLB_SIZE - 1) << CPU_TLB_ENTRY_BITS); - tcg_out_opc_reg(s, ALIAS_PADD, TCG_REG_A0, TCG_REG_A0, TCG_AREG0); + if (table_off > 0x7fff) { + int mask_hi = mask_off - (int16_t)mask_off; + int table_hi = table_off - (int16_t)table_off; - /* Compensate for very large offsets. */ - while (add_off >= 0x8000) { - /* Most target env are smaller than 32k, but a few are larger than 64k, - * so handle an arbitrarily large offset. - */ - tcg_out_opc_imm(s, ALIAS_PADDI, TCG_REG_A0, TCG_REG_A0, 0x7ff0); - cmp_off -= 0x7ff0; - add_off -= 0x7ff0; + table_base = TCG_TMP1; + if (likely(mask_hi == table_hi)) { + mask_base = table_base; + tcg_out_opc_imm(s, OPC_LUI, mask_base, TCG_REG_ZERO, mask_hi >> 16); + tcg_out_opc_reg(s, ALIAS_PADD, mask_base, mask_base, TCG_AREG0); + mask_off -= mask_hi; + table_off -= mask_hi; + } else { + if (mask_hi != 0) { + mask_base = TCG_TMP0; + tcg_out_opc_imm(s, OPC_LUI, + mask_base, TCG_REG_ZERO, mask_hi >> 16); + tcg_out_opc_reg(s, ALIAS_PADD, + mask_base, mask_base, TCG_AREG0); + } + table_off -= mask_off; + mask_off -= mask_hi; + tcg_out_opc_imm(s, ALIAS_PADDI, table_base, mask_base, mask_off); + } } + /* Load tlb_mask[mmu_idx] and tlb_table[mmu_idx]. */ + tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP0, mask_base, mask_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP1, table_base, table_off); + + /* Extract the TLB index from the address into TMP3. */ + tcg_out_opc_sa(s, ALIAS_TSRL, TCG_TMP3, addrl, + TARGET_PAGE_BITS - CPU_TLB_ENTRY_BITS); + tcg_out_opc_reg(s, OPC_AND, TCG_TMP3, TCG_TMP3, TCG_TMP0); + + /* Add the tlb_table pointer, creating the CPUTLBEntry address in TMP3. */ + tcg_out_opc_reg(s, ALIAS_PADD, TCG_TMP3, TCG_TMP3, TCG_TMP1); + /* We don't currently support unaligned accesses. We could do so with mips32r6. */ if (a_bits < s_bits) { a_bits = s_bits; } + /* Mask the page bits, keeping the alignment bits to compare against. */ mask = (target_ulong)TARGET_PAGE_MASK | ((1 << a_bits) - 1); - /* Load the (low half) tlb comparator. Mask the page bits, keeping the - alignment bits to compare against. */ + /* Load the (low-half) tlb comparator. */ if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { - tcg_out_ld(s, TCG_TYPE_I32, TCG_TMP0, TCG_REG_A0, cmp_off + LO_OFF); + tcg_out_ld(s, TCG_TYPE_I32, TCG_TMP0, TCG_TMP3, cmp_off + LO_OFF); tcg_out_movi(s, TCG_TYPE_I32, TCG_TMP1, mask); } else { - tcg_out_ldst(s, - (TARGET_LONG_BITS == 64 ? OPC_LD - : TCG_TARGET_REG_BITS == 64 ? OPC_LWU : OPC_LW), - TCG_TMP0, TCG_REG_A0, cmp_off); + tcg_out_ldst(s, (TARGET_LONG_BITS == 64 ? OPC_LD + : TCG_TARGET_REG_BITS == 64 ? OPC_LWU : OPC_LW), + TCG_TMP0, TCG_TMP3, cmp_off); tcg_out_movi(s, TCG_TYPE_TL, TCG_TMP1, mask); /* No second compare is required here; load the tlb addend for the fast path. */ - tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP2, TCG_REG_A0, add_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP2, TCG_TMP3, add_off); } tcg_out_opc_reg(s, OPC_AND, TCG_TMP1, TCG_TMP1, addrl); @@ -1271,10 +1303,10 @@ static void tcg_out_tlb_load(TCGContext *s, TCGReg base, TCGReg addrl, /* Load and test the high half tlb comparator. */ if (TCG_TARGET_REG_BITS < TARGET_LONG_BITS) { /* delay slot */ - tcg_out_ld(s, TCG_TYPE_I32, TCG_TMP0, TCG_REG_A0, cmp_off + HI_OFF); + tcg_out_ld(s, TCG_TYPE_I32, TCG_TMP0, TCG_TMP3, cmp_off + HI_OFF); /* Load the tlb addend for the fast path. */ - tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP2, TCG_REG_A0, add_off); + tcg_out_ld(s, TCG_TYPE_PTR, TCG_TMP2, TCG_TMP3, add_off); label_ptr[1] = s->code_ptr; tcg_out_opc_br(s, OPC_BNE, addrh, TCG_TMP0); From patchwork Mon Jan 28 15:59:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156785 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3616191jaa; Mon, 28 Jan 2019 08:24:05 -0800 (PST) X-Google-Smtp-Source: ALg8bN51BlEqIbTNaaoQKhpkSaVRIlBIetw7CEm99lK4JI5Nd3jCNfxSxAmq5/49+8lZXjT1Mh/a X-Received: by 2002:a05:6000:f:: with SMTP id h15mr21628100wrx.248.1548692644941; Mon, 28 Jan 2019 08:24:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692644; cv=none; d=google.com; s=arc-20160816; b=q7pZk2Q8yWgiODnZyvdA9urT7DHgis+nu/ZXxYNfJUl7U+6ULa/7NvwFYAJDqMOHyu pmEfTmZzQR7zJsSKhVVAcMQORIQVZ4Lk5RKG1AHYZ9jt2sf/AM3ILCp+MrXD6+berOyh AFNVia2oXoP27F/T5cU5eFZoDrjy9OfyOwJnw/ujJW4sTxSj5ilK0VWgr9t8yXWnwCx/ +4bCC7TVknFD5Z+ZuxOeDGobGbkuGWENrY0a4ZrtFNFeAq5SHD8hRNQCguEDZyA5UKKA 0JDRK+3lcqwCkI7m8y2j6/hQaomwibpSrd2jcqRZtVcraW5EJTo4GB23LAzQFxUpa/jB ECyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=chBVMwzjNPannioQJTQaKUW8hZTcHq/vz45yGlCAQkY=; b=gfdOhKwWNsr87uk9DL60OOr4jFCXHrV6StqDPt0dPMuSwSLJBeqgbhAwLrFCR+Pahq TZ7lltT5pO4hbZrhXPR+x3r5rO5uR/YhIawSGDotXrKSdY3mmiHxtgrHzDJk7NJowFOR bUB87wP4lg7Oj9QOotTGlTcuFL6S/n4dDk9pv9FA8O3AhkEUnZLPdmlq6vZ2ny4nLVEQ oiaZDg1UwAhraD6EN0iUiUDsDeFurIiq/+o/lJozsd83G2MG0o79Nfm4FRUWeuIb9Nk1 PVqK0XGDcy8oXZWgjStLl3dXvJ9K+C+JAo8a9CE8fqhKGy+t+B3ntkcBzrS3DGB2iqjZ mvMg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GJqIJ4cP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 21si6412079wme.120.2019.01.28.08.24.04 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:24:04 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=GJqIJ4cP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34487 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9hb-0006c4-SE for patch@linaro.org; Mon, 28 Jan 2019 11:24:03 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33558) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K5-0005CF-4Z for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:46 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vI-9L for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:45 -0500 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:39755) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007sg-3F for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x636.google.com with SMTP id 101so7918234pld.6 for ; Mon, 28 Jan 2019 07:59:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=chBVMwzjNPannioQJTQaKUW8hZTcHq/vz45yGlCAQkY=; b=GJqIJ4cPIfnXg/ZsxpAQFxcZAJAtnjo7MrL+tRHVGQNFdYU1/WmLk/mV50PGuJBWq2 jE9pU9b0+h3h0CwOiWjStG2HAV3HKbmGNQegyI+Sd6uXgeRSDq85q6qhdsGWeWJqAI8w sTJ6mYIyjQSc91UVyoklsZaGN49UjCe+/LUgE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=chBVMwzjNPannioQJTQaKUW8hZTcHq/vz45yGlCAQkY=; b=YXwYd0FXidFkVmQRoQw7nspndilOUY9np22X2gB2cwp9yw/ro8VyIBCVPd1M8amBxR G0J/U9O52QEH3Fpwi/Qlgw5YsGYESm7WVjHo+s3RKB7XnQ5eih2LOojCCmqT1MrnfUor pB8TAGIx4VQ46ihYz8QBqTHKMWVQhn7NWd79dviOgFs4U3TBvND1xFQrWLQeO6K19DN/ Iq7r2MQK6vXFlAQioSneW8Fka8laNnjenapKYG29Rn4YJ39veurZ1iKAAYwVpS6yNhHJ IrC3TWu103Q93M0oW4XSKc8Z7cUacpzN17xBiLPPKPlK43rtAEw0QtVTkttjausnavpt /vPw== X-Gm-Message-State: AJcUukeIzOgl8fPcekPvXuMbfEqkg1N3lrW36nw/8+1P48iBIFniAe+C r/+1XpGVfZQR5ooVpqbZ8GCmXoeVHqs= X-Received: by 2002:a17:902:161:: with SMTP id 88mr22843803plb.306.1548691181426; Mon, 28 Jan 2019 07:59:41 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.40 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:40 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:06 -0800 Message-Id: <20190128155907.20607-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::636 Subject: [Qemu-devel] [PULL 22/23] tcg/tci: enable dynamic TLB sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is automatic due to TCI using the other softtlb macros. Signed-off-by: Richard Henderson --- tcg/tci/tcg-target.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.17.2 diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index 816dc4697c..d9a28752c1 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -43,7 +43,7 @@ #define TCG_TARGET_INTERPRETER 1 #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 0 +#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #if UINTPTR_MAX == UINT32_MAX # define TCG_TARGET_REG_BITS 32 From patchwork Mon Jan 28 15:59:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 156791 Delivered-To: patch@linaro.org Received: by 2002:a02:48:0:0:0:0:0 with SMTP id 69csp3620986jaa; Mon, 28 Jan 2019 08:28:58 -0800 (PST) X-Google-Smtp-Source: ALg8bN7r5eEIDED6NlM8ryhaiXi2TfsxroU/FGBaSrz+UsbNjNq27X2zmBHzgDI72ymAxdrMGttm X-Received: by 2002:adf:b190:: with SMTP id q16mr22618630wra.95.1548692938094; Mon, 28 Jan 2019 08:28:58 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548692938; cv=none; d=google.com; s=arc-20160816; b=ebDu7Fh7lFJanjVEFs0p7eL0S4ZkVeQaHVrId7TsLTf18Vh52UqhE5rUq+Wzedtg1+ S1+12qcQvf7/EbvVbMlTiY3tRQvlw887xzhdr/4wSBK0K94R6BYuSufdBBGH3UO+Wg2B 8gNUys9D4QTtQO8ELzuG9zQluMk20c5t05Rmt6f+valuY3CmIQnkif9vrYKFY7TidLoO hMV4uCt1SOCX7+pXJ7jfsUeEnyxTEDjxUJh9eJ54X/3ALwUs8uakoNI5DCgC/6vwb6tF PGXd2FYXPZXlmIElxXW4B20fmQBF692aJCyS/GU/yGuHD2lROtxLRGcSHedax9Qtklff 2lnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature; bh=dCPOTyCZ2dNy2CK7aJXEwHKbAHASbHa8+HqlogUniZw=; b=BCIW37pnnnRvLT4TXRX0e2Ob7xNs9gqZHB64pbTD2bHJu6bY9mCLXbi0Kz/U5y8TKn FXNi3RTe4gXoRRpgyGrOVtgy0HPNNSo+GtxAAsnoo4aYLUPWbda/2MAASOotAmqv7/tg 8FpXlvo7ZGn7+WklEqQwzlriYrOnrCJZ2zjYIJvuxY0FCmNGXVyhol3ZP1d1zrQA6anj 2Jb52bqoE3lYeUl29iP2F0MIZIiQqzHcoTqADsRDpsKRBAvDiBtwvs3VodhIwTNOXUf/ jU07yYTeIUjB3/kw/1eNmdxIAUL97iMqyoVM2Emf48me/oAMmjI2z7OGZcaubab4iiTI RzHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=R4D3pRlr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 200si47988409wmm.114.2019.01.28.08.28.57 for (version=TLS1 cipher=AES128-SHA bits=128/128); Mon, 28 Jan 2019 08:28:58 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=R4D3pRlr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1]:34592 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9mK-0003Ct-RS for patch@linaro.org; Mon, 28 Jan 2019 11:28:56 -0500 Received: from eggs.gnu.org ([209.51.188.92]:33669) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1go9K7-0005Ey-Vq for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:50 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1go9K4-0007vp-R1 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:47 -0500 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]:41588) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1go9K4-0007tY-H4 for qemu-devel@nongnu.org; Mon, 28 Jan 2019 10:59:44 -0500 Received: by mail-pl1-x632.google.com with SMTP id u6so7919685plm.8 for ; Mon, 28 Jan 2019 07:59:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dCPOTyCZ2dNy2CK7aJXEwHKbAHASbHa8+HqlogUniZw=; b=R4D3pRlrkUptArePxnF1I5T58sW+/OfXY4XPmEO50HrABlajd02UlEo42J4fQkExia w40X/5nuWcYIi/ECDPEXBOt0SYyf2KqScR/F5HZb6VtXW78WR+sH412Xr11RZFH6vHMY 7mCo4U08I2PiU2Dzbb9vMFz3mp+x0FQ+8b33k= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dCPOTyCZ2dNy2CK7aJXEwHKbAHASbHa8+HqlogUniZw=; b=Y5opRgwtRy3eBV8Sg4LEPr4gA0PcSRhd31rExNvq2tAmWKx/dO3D5BleW5LpY9xxZw os5niDsc2NBvf+Sre2DF3EOOEteZcGupkpmYwiVyXzmkoYcGjcBEahDT15E9ppE3L/W7 ZZmWDmTvgXqM7nHpPeDZwoq70ZIUrRGGWMxx+Xzczw4omBuDZUzdDdOxMJYkiHkTXXz1 mfIWOZWR/MP7dhJa2gq82wCxoKiTJhBnICdFWYULBpIO5gEWKHZ8Bc5Cvds5bN2nBvy6 P2kjMoCFxkxXHBdC0wh4l04bqAC3VTtaA9NJsAMx2x0ayG3mQs64TwySGBR/KuVfXa/+ gHZA== X-Gm-Message-State: AJcUukeY/45zV0sNTvF+hkMEXPeHvYkSGmLE0dBAsCbSlW/smotT3ROI GW5CuuytKSSItdgypxCM0vS2UnZ1mtE= X-Received: by 2002:a17:902:2a0a:: with SMTP id i10mr22043514plb.323.1548691182594; Mon, 28 Jan 2019 07:59:42 -0800 (PST) Received: from cloudburst.twiddle.net (50-233-235-3-static.hfc.comcastbusiness.net. [50.233.235.3]) by smtp.gmail.com with ESMTPSA id p2sm45518687pfp.125.2019.01.28.07.59.41 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Mon, 28 Jan 2019 07:59:41 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Mon, 28 Jan 2019 07:59:07 -0800 Message-Id: <20190128155907.20607-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.2 In-Reply-To: <20190128155907.20607-1-richard.henderson@linaro.org> References: <20190128155907.20607-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::632 Subject: [Qemu-devel] [PULL 23/23] cputlb: Remove static tlb sizing X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now that all tcg backends support TCG_TARGET_IMPLEMENTS_DYN_TLB, remove the define and the old code. Reviewed-by: Alistair Francis Signed-off-by: Richard Henderson --- include/exec/cpu-defs.h | 46 ---------------------------------------- include/exec/cpu_ldst.h | 14 ------------ tcg/aarch64/tcg-target.h | 1 - tcg/arm/tcg-target.h | 1 - tcg/i386/tcg-target.h | 1 - tcg/mips/tcg-target.h | 1 - tcg/ppc/tcg-target.h | 1 - tcg/riscv/tcg-target.h | 1 - tcg/s390/tcg-target.h | 1 - tcg/sparc/tcg-target.h | 1 - tcg/tci/tcg-target.h | 1 - accel/tcg/cputlb.c | 21 ------------------ 12 files changed, 90 deletions(-) -- 2.17.2 diff --git a/include/exec/cpu-defs.h b/include/exec/cpu-defs.h index 191a1e021f..8f2a848bf5 100644 --- a/include/exec/cpu-defs.h +++ b/include/exec/cpu-defs.h @@ -67,11 +67,9 @@ typedef uint64_t target_ulong; #define CPU_TLB_ENTRY_BITS 5 #endif -#if TCG_TARGET_IMPLEMENTS_DYN_TLB #define CPU_TLB_DYN_MIN_BITS 6 #define CPU_TLB_DYN_DEFAULT_BITS 8 - # if HOST_LONG_BITS == 32 /* Make sure we do not require a double-word shift for the TLB load */ # define CPU_TLB_DYN_MAX_BITS (32 - TARGET_PAGE_BITS) @@ -87,41 +85,6 @@ typedef uint64_t target_ulong; MIN(22, TARGET_VIRT_ADDR_SPACE_BITS - TARGET_PAGE_BITS) # endif -#else /* !TCG_TARGET_IMPLEMENTS_DYN_TLB */ - -/* TCG_TARGET_TLB_DISPLACEMENT_BITS is used in CPU_TLB_BITS to ensure that - * the TLB is not unnecessarily small, but still small enough for the - * TLB lookup instruction sequence used by the TCG target. - * - * TCG will have to generate an operand as large as the distance between - * env and the tlb_table[NB_MMU_MODES - 1][0].addend. For simplicity, - * the TCG targets just round everything up to the next power of two, and - * count bits. This works because: 1) the size of each TLB is a largish - * power of two, 2) and because the limit of the displacement is really close - * to a power of two, 3) the offset of tlb_table[0][0] inside env is smaller - * than the size of a TLB. - * - * For example, the maximum displacement 0xFFF0 on PPC and MIPS, but TCG - * just says "the displacement is 16 bits". TCG_TARGET_TLB_DISPLACEMENT_BITS - * then ensures that tlb_table at least 0x8000 bytes large ("not unnecessarily - * small": 2^15). The operand then will come up smaller than 0xFFF0 without - * any particular care, because the TLB for a single MMU mode is larger than - * 0x10000-0xFFF0=16 bytes. In the end, the maximum value of the operand - * could be something like 0xC000 (the offset of the last TLB table) plus - * 0x18 (the offset of the addend field in each TLB entry) plus the offset - * of tlb_table inside env (which is non-trivial but not huge). - */ -#define CPU_TLB_BITS \ - MIN(8, \ - TCG_TARGET_TLB_DISPLACEMENT_BITS - CPU_TLB_ENTRY_BITS - \ - (NB_MMU_MODES <= 1 ? 0 : \ - NB_MMU_MODES <= 2 ? 1 : \ - NB_MMU_MODES <= 4 ? 2 : \ - NB_MMU_MODES <= 8 ? 3 : 4)) - -#define CPU_TLB_SIZE (1 << CPU_TLB_BITS) -#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ - typedef struct CPUTLBEntry { /* bit TARGET_LONG_BITS to TARGET_PAGE_BITS : virtual address bit TARGET_PAGE_BITS-1..4 : Nonzero for accesses that should not @@ -187,10 +150,8 @@ typedef struct CPUTLBDesc { target_ulong large_page_mask; /* The next index to use in the tlb victim table. */ size_t vindex; -#if TCG_TARGET_IMPLEMENTS_DYN_TLB CPUTLBWindow window; size_t n_used_entries; -#endif } CPUTLBDesc; /* @@ -215,19 +176,12 @@ typedef struct CPUTLBCommon { size_t elide_flush_count; } CPUTLBCommon; -#if TCG_TARGET_IMPLEMENTS_DYN_TLB # define CPU_TLB \ /* tlb_mask[i] contains (n_entries - 1) << CPU_TLB_ENTRY_BITS */ \ uintptr_t tlb_mask[NB_MMU_MODES]; \ CPUTLBEntry *tlb_table[NB_MMU_MODES]; # define CPU_IOTLB \ CPUIOTLBEntry *iotlb[NB_MMU_MODES]; -#else -# define CPU_TLB \ - CPUTLBEntry tlb_table[NB_MMU_MODES][CPU_TLB_SIZE]; -# define CPU_IOTLB \ - CPUIOTLBEntry iotlb[NB_MMU_MODES][CPU_TLB_SIZE]; -#endif /* * The meaning of each of the MMU modes is defined in the target code. diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 83b2907d86..d78041d7a0 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -135,7 +135,6 @@ static inline target_ulong tlb_addr_write(const CPUTLBEntry *entry) #endif } -#if TCG_TARGET_IMPLEMENTS_DYN_TLB /* Find the TLB index corresponding to the mmu_idx + address pair. */ static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, target_ulong addr) @@ -149,19 +148,6 @@ static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx) { return (env->tlb_mask[mmu_idx] >> CPU_TLB_ENTRY_BITS) + 1; } -#else -/* Find the TLB index corresponding to the mmu_idx + address pair. */ -static inline uintptr_t tlb_index(CPUArchState *env, uintptr_t mmu_idx, - target_ulong addr) -{ - return (addr >> TARGET_PAGE_BITS) & (CPU_TLB_SIZE - 1); -} - -static inline size_t tlb_n_entries(CPUArchState *env, uintptr_t mmu_idx) -{ - return CPU_TLB_SIZE; -} -#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ /* Find the TLB entry corresponding to the mmu_idx + address pair. */ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 5085a81060..2d93cf404e 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -15,7 +15,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #undef TCG_TARGET_STACK_GROWSUP typedef enum { diff --git a/tcg/arm/tcg-target.h b/tcg/arm/tcg-target.h index 679aaf097e..16172f73a3 100644 --- a/tcg/arm/tcg-target.h +++ b/tcg/arm/tcg-target.h @@ -60,7 +60,6 @@ extern int arm_arch; #undef TCG_TARGET_STACK_GROWSUP #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum { TCG_REG_R0 = 0, diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index eb40312e67..7995fe3eab 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -27,7 +27,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 31 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #ifdef __x86_64__ # define TCG_TARGET_REG_BITS 64 diff --git a/tcg/mips/tcg-target.h b/tcg/mips/tcg-target.h index 40adbe38cb..5cb8672470 100644 --- a/tcg/mips/tcg-target.h +++ b/tcg/mips/tcg-target.h @@ -37,7 +37,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/ppc/tcg-target.h b/tcg/ppc/tcg-target.h index 95b735b0bb..52c1bb04b1 100644 --- a/tcg/ppc/tcg-target.h +++ b/tcg/ppc/tcg-target.h @@ -34,7 +34,6 @@ #define TCG_TARGET_NB_REGS 32 #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 16 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum { TCG_REG_R0, TCG_REG_R1, TCG_REG_R2, TCG_REG_R3, diff --git a/tcg/riscv/tcg-target.h b/tcg/riscv/tcg-target.h index 83b123ca03..60918cacb4 100644 --- a/tcg/riscv/tcg-target.h +++ b/tcg/riscv/tcg-target.h @@ -33,7 +33,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 20 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/s390/tcg-target.h b/tcg/s390/tcg-target.h index 357528dd97..853ed6e7aa 100644 --- a/tcg/s390/tcg-target.h +++ b/tcg/s390/tcg-target.h @@ -27,7 +27,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 2 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 19 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 typedef enum TCGReg { TCG_REG_R0 = 0, diff --git a/tcg/sparc/tcg-target.h b/tcg/sparc/tcg-target.h index 6020a670c0..a0ed2a3342 100644 --- a/tcg/sparc/tcg-target.h +++ b/tcg/sparc/tcg-target.h @@ -29,7 +29,6 @@ #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #define TCG_TARGET_NB_REGS 32 typedef enum { diff --git a/tcg/tci/tcg-target.h b/tcg/tci/tcg-target.h index d9a28752c1..086f34e69a 100644 --- a/tcg/tci/tcg-target.h +++ b/tcg/tci/tcg-target.h @@ -43,7 +43,6 @@ #define TCG_TARGET_INTERPRETER 1 #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 32 -#define TCG_TARGET_IMPLEMENTS_DYN_TLB 1 #if UINTPTR_MAX == UINT32_MAX # define TCG_TARGET_REG_BITS 32 diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index a3a1614f0e..dad9b7796c 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -74,7 +74,6 @@ QEMU_BUILD_BUG_ON(sizeof(target_ulong) > sizeof(run_on_cpu_data)); QEMU_BUILD_BUG_ON(NB_MMU_MODES > 16); #define ALL_MMUIDX_BITS ((1 << NB_MMU_MODES) - 1) -#if TCG_TARGET_IMPLEMENTS_DYN_TLB static inline size_t sizeof_tlb(CPUArchState *env, uintptr_t mmu_idx) { return env->tlb_mask[mmu_idx] + (1 << CPU_TLB_ENTRY_BITS); @@ -235,26 +234,6 @@ static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx) env->tlb_d[mmu_idx].n_used_entries--; } -#else /* !TCG_TARGET_IMPLEMENTS_DYN_TLB */ - -static inline void tlb_dyn_init(CPUArchState *env) -{ -} - -static inline void tlb_table_flush_by_mmuidx(CPUArchState *env, int mmu_idx) -{ - memset(env->tlb_table[mmu_idx], -1, sizeof(env->tlb_table[0])); -} - -static inline void tlb_n_used_entries_inc(CPUArchState *env, uintptr_t mmu_idx) -{ -} - -static inline void tlb_n_used_entries_dec(CPUArchState *env, uintptr_t mmu_idx) -{ -} -#endif /* TCG_TARGET_IMPLEMENTS_DYN_TLB */ - void tlb_init(CPUState *cpu) { CPUArchState *env = cpu->env_ptr;