From patchwork Fri Mar 2 18:56:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 130579 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp1628557lja; Fri, 2 Mar 2018 11:27:50 -0800 (PST) X-Google-Smtp-Source: AG47ELtl31U4fOcQjFvbdVxN72nIHsWmMByevnscDEpd7Totrts5PTVY3DGB0MQdsQuXrOAiHdXC X-Received: by 10.13.237.70 with SMTP id w67mr4432117ywe.414.1520018870097; Fri, 02 Mar 2018 11:27:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1520018870; cv=none; d=google.com; s=arc-20160816; b=chYG3rZLthP06OpwzMDYK4/PRUaBPyYnuO2WhelIxiFIKAClaVzbZkOhG1R37FoiXS HyVe6Xn6d7ORF7C1Nwc0vcLNNlmxF5Y4zT+cPYEvCl/c6f3eUYoqmILfDzW6rC5CRa7d DQVOZyh4ce/gfVWNOuh6o6rbIYUNORfZSCPnyIysUiK+SvfdG/DPCZpW928ZiJgcyUgU LqrD0tlY7YWkFEtwaitqir6pXqiaYQ7ueSePv6sVMYQNl36ffkMEJkLlas5BLctp7ZvI wscLVhTAojh1+ON2RI3UQY7IufSrDVHkGsBfYkXMn6mR8/a/U1vgLQnnKigvK2CUpiS9 IpTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=LcYFVickf/j3KDsUvOF6xdyeiYNt74Fp/yMAi0FYaksnRxJJ2qBFP0iv7St2jOjLac Z38pvsFZQlYDBQnaZHQQhBzsGQ7ihM/8DI5baa8jKStuBwtn8u1VJZ3WXAJgXnnSx40Z +8amy6s3NceGBALjsc/uvoKMximv1zllkSuhK0hzZmt551XYGZQO5BE2XVgUzPQRsQah Onebcpi/VFDmFnBBaLA2SIsi2pCn0d0PftbDb9XpDH6SX+1FiBwaXUBxbfMbETCQCYDa cI6cNXuqVPV/s30cTAxS04E4vLqZrTCmIfYhCEtpd+UdMC/fcc2BTIQ79TVxussJFtZJ pHcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=NMRPIp54; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 204si1173927yws.147.2018.03.02.11.27.49 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 02 Mar 2018 11:27:50 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=NMRPIp54; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37073 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erqLN-0004bb-Dm for patch@linaro.org; Fri, 02 Mar 2018 14:27:49 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53369) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erpr5-0002Cq-46 for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:31 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erpr3-0003Yz-Hs for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:31 -0500 Received: from mail-pl0-x242.google.com ([2607:f8b0:400e:c01::242]:41450) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1erpr3-0003Yp-BJ for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:29 -0500 Received: by mail-pl0-x242.google.com with SMTP id k8-v6so6188890pli.8 for ; Fri, 02 Mar 2018 10:56:29 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=NMRPIp54BwDMko5dUNqbAjVzHIhCuUfBo0QZltytJh68Lmk/51NsxIxjipHJoVytT/ wrtproOGY+Xq0wtTQ4Y++Y6ULgFyZm2+VDsgGLPIekM+Dt3j7TQYQVkb5BuvrlKyqHlf AK6EWtfrTxy0LQ6yIAVsjsk25M0B3x/Stel+w= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=ljOFYaU2maT8GXQZ/EvobbjHFyoRdpPKAMZa3o3bFFsNUk6tlIRpfL3aMm83cIYrrh F/eVqY3RoUa30ij1uDqeCSQtNlw0Mcl1RSn/Yb2VktBuIkdNyEq8SwqEg/AaCs0H8nBa 5ekQWfFqTpcp/0aWoUcu2WYHEeHrGrSr9eGfUTrTkvMFkmCQeWzt3vd67g2ESYpQmaFB GiixU9cabfn/paeAjZMG2o+Tl6R1IxTJKC2SKYhT5yvD9mbi3QLsQGup6RMcM6aJ8G6n TqAw7WToyzk8aaCmVO84DJZdJTdH6xIWIgiwoXR2bDZ12B5Gf3OB8fdvYwfaMuhmVd0I sf4g== X-Gm-Message-State: APf1xPDdllISppAdbc0woxqJV/HBW5IBJZRxH0vpDlgyvpfZdiUvt9u4 7fstrSHDMnPbMGxhMiZKcMtu6J/o8DM= X-Received: by 2002:a17:902:20e5:: with SMTP id v34-v6mr5955773plg.66.1520016987876; Fri, 02 Mar 2018 10:56:27 -0800 (PST) Received: from cloudburst.twiddle.net (97-113-169-147.tukw.qwest.net. [97.113.169.147]) by smtp.gmail.com with ESMTPSA id o2sm10070338pgq.54.2018.03.02.10.56.26 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Mar 2018 10:56:26 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 2 Mar 2018 10:56:20 -0800 Message-Id: <20180302185622.27780-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180302185622.27780-1-richard.henderson@linaro.org> References: <20180302185622.27780-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::242 Subject: [Qemu-devel] [PULL 1/3] tcg: Improve tcg_gen_muli_i32/i64 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Convert multiplication by power of two to left shift. Reviewed-by: Emilio G. Cota Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/tcg-op.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) -- 2.14.3 diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index 3467787323..34b96d68f3 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -277,9 +277,15 @@ void tcg_gen_setcondi_i32(TCGCond cond, TCGv_i32 ret, void tcg_gen_muli_i32(TCGv_i32 ret, TCGv_i32 arg1, int32_t arg2) { - TCGv_i32 t0 = tcg_const_i32(arg2); - tcg_gen_mul_i32(ret, arg1, t0); - tcg_temp_free_i32(t0); + if (arg2 == 0) { + tcg_gen_movi_i32(ret, 0); + } else if (is_power_of_2(arg2)) { + tcg_gen_shli_i32(ret, arg1, ctz32(arg2)); + } else { + TCGv_i32 t0 = tcg_const_i32(arg2); + tcg_gen_mul_i32(ret, arg1, t0); + tcg_temp_free_i32(t0); + } } void tcg_gen_div_i32(TCGv_i32 ret, TCGv_i32 arg1, TCGv_i32 arg2) @@ -1430,9 +1436,15 @@ void tcg_gen_setcondi_i64(TCGCond cond, TCGv_i64 ret, void tcg_gen_muli_i64(TCGv_i64 ret, TCGv_i64 arg1, int64_t arg2) { - TCGv_i64 t0 = tcg_const_i64(arg2); - tcg_gen_mul_i64(ret, arg1, t0); - tcg_temp_free_i64(t0); + if (arg2 == 0) { + tcg_gen_movi_i64(ret, 0); + } else if (is_power_of_2(arg2)) { + tcg_gen_shli_i64(ret, arg1, ctz64(arg2)); + } else { + TCGv_i64 t0 = tcg_const_i64(arg2); + tcg_gen_mul_i64(ret, arg1, t0); + tcg_temp_free_i64(t0); + } } void tcg_gen_div_i64(TCGv_i64 ret, TCGv_i64 arg1, TCGv_i64 arg2) From patchwork Fri Mar 2 18:56:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 130580 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp1628920lja; Fri, 2 Mar 2018 11:28:19 -0800 (PST) X-Google-Smtp-Source: AG47ELurVGvYhpSJpIBRuTfXsMKvTddOIkk0aDdb6TmMb5uWK0tjyL1K252r/fk7wHTsvUV8tbwj X-Received: by 10.129.154.10 with SMTP id r10mr4469051ywg.163.1520018899102; Fri, 02 Mar 2018 11:28:19 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1520018899; cv=none; d=google.com; s=arc-20160816; b=gwAyNcCs6O3AhCCEx9xX/QVess+T2I4xiKAdv6Vy8mxGT9zAZ8tqZ6ESZDTS8pQ7XN h4kXeoUH9jkIKWqFG13ygWZxpRlA1LeQt8y5XqjjW+6JskVo705YlklO+5EnTcCZDmsR R6TForecHCDz+Z4Cfrtr20msFZL5NfHqWJTf52+YkBlwhYl50OkVWmknykRMqJ+ttV1f R4ncJ30H1ZUw4DkncIdCfV1XqSGaiQ5sG9gmfCyrpmenNRI7+KvxBtEZ1r0o0JgQkj6w XKSNUS6+lWefSVEt7z81FER1x7ZvraQkWyYQXwE/5y71OGEP56x+shJOI+YsTD5Hy756 88Gg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=u+9Dloi2SqYP0nqMI10moh9sKIrnTYlQNxQBZ4pe4Ug15VCdTzrbJifPE+IeMcRXwv zYI/ErJ2ZiG9XK6yZMyntb4ULVMRVwcCNEAohlvHYDbniv+8mqY223iph6/FeRU1oK/s Dlq7iBdct39Hx2nsaXLpuKxsuVJsQefURQYXrq4urepcYHZQXKJBXPtcPdOzjhJ1NYJw uPjmbnvEbdktrvz2L/xt0TRrEJLk3bMAJQDsja6IjhODQYTxDwRHgK3KxzcjroKtnJnn URmcm5Q+DZShpad180CBSJKwxoUXd4I9IgR8PZL4YK2vDZS7UzYbZDhybTlRXeTFOAct jDdQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=N1NmFM7j; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u124si1158535ywc.386.2018.03.02.11.28.18 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 02 Mar 2018 11:28:19 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=N1NmFM7j; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37083 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erqLp-0005yo-Nv for patch@linaro.org; Fri, 02 Mar 2018 14:28:17 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53385) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erpr5-0002DN-Mt for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:32 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erpr5-0003Zs-0Y for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:31 -0500 Received: from mail-pl0-x241.google.com ([2607:f8b0:400e:c01::241]:46694) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1erpr4-0003Z5-R6 for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:30 -0500 Received: by mail-pl0-x241.google.com with SMTP id y8-v6so6186057pll.13 for ; Fri, 02 Mar 2018 10:56:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=N1NmFM7j+LQsJTeyZf770wwz5VSUhCXszYzhy+iOz+NxCn8iVSGp8aySBuxyULSgLn dUiRLyoGorjciMzxWeTGuBD+2qNbifXk4zqE/d5tU75Wu1CmU+PhVbUuoYTvbr9RroYn iT/Reqbh4EgbLp52bpnvsASy7te0kFF6wJkCc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=DP6M4UH+EF5UvWkw+NRFL516b3CO0q7fTVLdfvoY9N3aICthpva77ROPA9+JAQ9adP I51DdfS7cNS5LIKlgEWwV8no73Qz89FxPTjPLv0iEiY2xcaM6ZxZN/7CW6lPyqw4JTWD rB5G7vnSh2xyukkWuy5j3pdXu/M5tV2VTvrguSDMkUyJz3+CNpbPCrRu56VMu4TDn/GR 1QVQ6ee9y7i+14mA7NApg+0QKhVrH7eCKzSGleYPWEzgaNrc1onFlEJfPKsAzXtYr7ur 6JjQg9BbaB6QqAm1M7Y9ZOdX34kFRGmDcQauFXa/K2VoUVDQLTVMqc9/B7mxDr8UAwkK wx0g== X-Gm-Message-State: APf1xPC/U1nhuwIWIeQZRCJN4hZbe6mWjS2yOokZTZHNwN0WK7k5gWYm Igo8AWVLXZ3g3WvfSjFD764hvKDjpdw= X-Received: by 2002:a17:902:149:: with SMTP id 67-v6mr6023307plb.73.1520016989207; Fri, 02 Mar 2018 10:56:29 -0800 (PST) Received: from cloudburst.twiddle.net (97-113-169-147.tukw.qwest.net. [97.113.169.147]) by smtp.gmail.com with ESMTPSA id o2sm10070338pgq.54.2018.03.02.10.56.27 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Mar 2018 10:56:28 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 2 Mar 2018 10:56:21 -0800 Message-Id: <20180302185622.27780-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180302185622.27780-1-richard.henderson@linaro.org> References: <20180302185622.27780-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::241 Subject: [Qemu-devel] [PULL 2/3] tcg/i386: Support INDEX_op_dup2_vec for -m32 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Unknown why -m32 was passing with gcc but not clang; it should have failed for both. This would be used for tcg_gen_dup_i64_vec, and visible with the right TB and an aarch64 guest. Reported-by: Max Reitz Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.14.3 diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index fc05909d1d..d7e59e79c5 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -2696,6 +2696,12 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_x86_packus_vec: insn = packus_insn[vece]; goto gen_simd; +#if TCG_TARGET_REG_BITS == 32 + case INDEX_op_dup2_vec: + /* Constraints have already placed both 32-bit inputs in xmm regs. */ + insn = OPC_PUNPCKLDQ; + goto gen_simd; +#endif gen_simd: tcg_debug_assert(insn != OPC_UD2); if (type == TCG_TYPE_V256) { @@ -3045,6 +3051,9 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_x86_vperm2i128_vec: case INDEX_op_x86_punpckl_vec: case INDEX_op_x86_punpckh_vec: +#if TCG_TARGET_REG_BITS == 32 + case INDEX_op_dup2_vec: +#endif return &x_x_x; case INDEX_op_dup_vec: case INDEX_op_shli_vec: From patchwork Fri Mar 2 18:56:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 130581 Delivered-To: patch@linaro.org Received: by 10.46.66.2 with SMTP id p2csp1634281lja; Fri, 2 Mar 2018 11:34:20 -0800 (PST) X-Google-Smtp-Source: AG47ELuj+RKhh6GYA3CmjRKNoEzYhZMd2gHpyz5nKqwMytdqQvT6S4o68YoNNEpfKjqjJvkQ4xNA X-Received: by 10.129.231.7 with SMTP id x7mr4427579ywl.455.1520019260273; Fri, 02 Mar 2018 11:34:20 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1520019260; cv=none; d=google.com; s=arc-20160816; b=T+RI5gPiAnLdJuWzwcWSV9eQR9shJP7k7yjr73rIAR/2WAGagf0YxLzyPnPR/KBpxh XaToR6QslDjCY8GpzTOZEs+1po6ke94UojxO4GEPOQ6FyTnZEGM+rOWOPwqmIn8PB9/I 3xVA1ke1JKmizUSvTQdwKTErVKVcrcVINhZF2xwLPIwIIt8/PvyVLBcikHMXaHLj0bt2 FP6drXGVYTxzsvdCo0fbetRk5jEjAaXPXohyqZ0ByQj4+58lIP+yP2I/1NZ/3y05t+mz hOnWZXWRNEtmNTGwen/lCbwS7Ep+Y+LSDok1aiyisJPQdobe528W4eST13hbYGR5uLm3 PhrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=dlUwQKTOCZ1JMDn/3KeP3YkAOVl3hmct8+iYYhIA/qgL+JgYINV5V96LaRCUixrFKd usRWOFg8EADsCePad7FykBUjo/umFytDFu/dwKPgwKujf8inU8XmD2XusWpwxLVKgTYA hxrIuwi4hcXotbW+bGARLT1axnNuM6dMw2Aluw52HrejYJwgom5CuBOpHrcrgvulpZYl xdJHU5RWxWU/+N9g+C1XRGhclQw+iSuT1j3RFU0FUhwecNUZTzHO+GnIs95HmgwwPAub iYicZ/WjnhdcnCdD3rp6tK1EcCFvOY3QVk1a9JQSirOKhCM0gZgb0TRBcSEQGFie4Hg0 tOhQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UNO+Tqx7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id n5si1161389ywf.538.2018.03.02.11.34.19 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 02 Mar 2018 11:34:20 -0800 (PST) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=UNO+Tqx7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37122 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erqRf-0002Rj-8j for patch@linaro.org; Fri, 02 Mar 2018 14:34:19 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:53404) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1erpr9-0002Hr-MN for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:38 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1erpr6-0003cY-Va for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:35 -0500 Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:43864) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1erpr6-0003bl-Kx for qemu-devel@nongnu.org; Fri, 02 Mar 2018 13:56:32 -0500 Received: by mail-pg0-x242.google.com with SMTP id e9so4123964pgs.10 for ; Fri, 02 Mar 2018 10:56:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=UNO+Tqx7130ZIHrDtblXCJ+Y4OG/91Exki12UZI1u2HAMj1RfIWrGGw7BmlXp6/mi4 omVyWW+MyhH3fDR9kYg2gS6v35/NbY88vZOPhBJy9DiJg+TTfzEgN79TCQjB30cVaUQk pMsA5bZrkHR8Wu+0dUYXtvQJsq+o5oHA7Y5pA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=YMfz/NOTC3KPGIGAPAHfFY7f7fN4qDjAozFdI9hMJNN8HIEKv8snQRP0fziVGeZ3Vi vfg5l61VFtqCtMEyZuNUAxQasplG3NZMgjV6Zf9gtOCUMGRUenD9Iao2O+9lhVAoPz6Y KNQA4Zclv1vBpW21RdZA4+cT9XtUTTvXKfwkJushbESKp/vDEkpluErWecnzXLhR4jDZ EcIngwhUVNWlIxlvTh4E4FY3ZLAuVbKAT6FOBB1yP7mpdH/gi3ArnWgQIQWUkpp/FCDv /69JUV8GVIFwdO29EyJ8aGXE6u2Uj4zSKoMsRL0MftWEa95PA8PDWNbAOZDaiayN04AZ bCKw== X-Gm-Message-State: APf1xPAUIGWL8HCZQsMYmacBDs1OPe60aR4PVeCUSmdIdxqNWAh0ns6d 2Uo1+ex8BNX0GG6fMj3no1naJAhdHpM= X-Received: by 10.101.68.141 with SMTP id l13mr4374892pgq.216.1520016990769; Fri, 02 Mar 2018 10:56:30 -0800 (PST) Received: from cloudburst.twiddle.net (97-113-169-147.tukw.qwest.net. [97.113.169.147]) by smtp.gmail.com with ESMTPSA id o2sm10070338pgq.54.2018.03.02.10.56.29 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Mar 2018 10:56:29 -0800 (PST) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 2 Mar 2018 10:56:22 -0800 Message-Id: <20180302185622.27780-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180302185622.27780-1-richard.henderson@linaro.org> References: <20180302185622.27780-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::242 Subject: [Qemu-devel] [PULL 3/3] tcg: Add choose_vector_size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This unifies 5 copies of checks for supported vector size, and in the process fixes a missing check in tcg_gen_gvec_2s. This lead to an assertion failure for 64-bit vector multiply, which is not available in the AVX instruction set. Suggested-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg-op-gvec.c | 438 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 259 insertions(+), 179 deletions(-) -- 2.14.3 diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index bfe44bba81..22db1590d5 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -351,6 +351,42 @@ static void gen_dup_i64(unsigned vece, TCGv_i64 out, TCGv_i64 in) } } +/* Select a supported vector type for implementing an operation on SIZE + * bytes. If OP is 0, assume that the real operation to be performed is + * required by all backends. Otherwise, make sure than OP can be performed + * on elements of size VECE in the selected type. Do not select V64 if + * PREFER_I64 is true. Return 0 if no vector type is selected. + */ +static TCGType choose_vector_type(TCGOpcode op, unsigned vece, uint32_t size, + bool prefer_i64) +{ + if (TCG_TARGET_HAS_v256 && check_size_impl(size, 32)) { + if (op == 0) { + return TCG_TYPE_V256; + } + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + * It is hard to imagine a case in which v256 is supported + * but v128 is not, but check anyway. + */ + if (tcg_can_emit_vec_op(op, TCG_TYPE_V256, vece) + && (size % 32 == 0 + || tcg_can_emit_vec_op(op, TCG_TYPE_V128, vece))) { + return TCG_TYPE_V256; + } + } + if (TCG_TARGET_HAS_v128 && check_size_impl(size, 16) + && (op == 0 || tcg_can_emit_vec_op(op, TCG_TYPE_V128, vece))) { + return TCG_TYPE_V128; + } + if (TCG_TARGET_HAS_v64 && !prefer_i64 && check_size_impl(size, 8) + && (op == 0 || tcg_can_emit_vec_op(op, TCG_TYPE_V64, vece))) { + return TCG_TYPE_V64; + } + return 0; +} + /* Set OPRSZ bytes at DOFS to replications of IN_32, IN_64 or IN_C. * Only one of IN_32 or IN_64 may be set; * IN_C is used if IN_32 and IN_64 are unset. @@ -376,19 +412,12 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } } - type = 0; - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32)) { - type = TCG_TYPE_V256; - } else if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16)) { - type = TCG_TYPE_V128; - } else if (TCG_TARGET_HAS_v64 && check_size_impl(oprsz, 8) - /* Prefer integer when 64-bit host and no variable dup. */ - && !(TCG_TARGET_REG_BITS == 64 && in_32 == NULL - && (in_64 == NULL || vece == MO_64))) { - type = TCG_TYPE_V64; - } - - /* Implement inline with a vector type, if possible. */ + /* Implement inline with a vector type, if possible. + * Prefer integer when 64-bit host and no variable dup. + */ + type = choose_vector_type(0, vece, oprsz, + (TCG_TARGET_REG_BITS == 64 && in_32 == NULL + && (in_64 == NULL || vece == MO_64))); if (type != 0) { TCGv_vec t_vec = tcg_temp_new_vec(type); @@ -414,21 +443,30 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } i = 0; - if (TCG_TARGET_HAS_v256) { + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ for (; i + 32 <= oprsz; i += 32) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V256); } - } - if (TCG_TARGET_HAS_v128) { + /* fallthru */ + case TCG_TYPE_V128: for (; i + 16 <= oprsz; i += 16) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V128); } - } - if (TCG_TARGET_HAS_v64) { + break; + case TCG_TYPE_V64: for (; i < oprsz; i += 8) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V64); } + break; + default: + g_assert_not_reached(); } + tcg_temp_free_vec(t_vec); goto done; } @@ -484,7 +522,7 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } tcg_temp_free_i64(t_64); goto done; - } + } } /* Otherwise implement out of line. */ @@ -866,49 +904,55 @@ static void expand_4_vec(unsigned vece, uint32_t dofs, uint32_t aofs, void tcg_gen_gvec_2(uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz, const GVecGen2 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs); check_overlap_2(dofs, aofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - /* ??? For maxsz > oprsz, the host may be able to use an opr-sized - operation, zeroing the balance of the register. We can then - use a max-sized store to implement the clearing without an extra - store operation. This is true for aarch64 and x86_64 hosts. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_2_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_2_vec(g->vece, dofs, aofs, oprsz, 16, TCG_TYPE_V128, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_2_vec(g->vece, dofs, aofs, oprsz, 8, TCG_TYPE_V64, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_2_i64(dofs, aofs, oprsz, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_2_i32(dofs, aofs, oprsz, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, g->data, g->fno); - return; + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_2_i64(dofs, aofs, oprsz, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_2_i32(dofs, aofs, oprsz, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -918,53 +962,64 @@ void tcg_gen_gvec_2(uint32_t dofs, uint32_t aofs, void tcg_gen_gvec_2i(uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz, int64_t c, const GVecGen2i *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs); check_overlap_2(dofs, aofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_2i_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, c, g->load_dest, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_2i_vec(g->vece, dofs, aofs, oprsz, 16, TCG_TYPE_V128, c, g->load_dest, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_2i_vec(g->vece, dofs, aofs, oprsz, 8, TCG_TYPE_V64, c, g->load_dest, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_2i_i64(dofs, aofs, oprsz, c, g->load_dest, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_2i_i32(dofs, aofs, oprsz, c, g->load_dest, g->fni4); - } else { - if (g->fno) { - tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, c, g->fno); + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_2i_i64(dofs, aofs, oprsz, c, g->load_dest, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_2i_i32(dofs, aofs, oprsz, c, g->load_dest, g->fni4); } else { - TCGv_i64 tcg_c = tcg_const_i64(c); - tcg_gen_gvec_2i_ool(dofs, aofs, tcg_c, oprsz, maxsz, c, g->fnoi); - tcg_temp_free_i64(tcg_c); + if (g->fno) { + tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, c, g->fno); + } else { + TCGv_i64 tcg_c = tcg_const_i64(c); + tcg_gen_gvec_2i_ool(dofs, aofs, tcg_c, oprsz, + maxsz, c, g->fnoi); + tcg_temp_free_i64(tcg_c); + } + return; } - return; + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -981,37 +1036,30 @@ void tcg_gen_gvec_2s(uint32_t dofs, uint32_t aofs, uint32_t oprsz, type = 0; if (g->fniv) { - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32)) { - type = TCG_TYPE_V256; - } else if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16)) { - type = TCG_TYPE_V128; - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && check_size_impl(oprsz, 8)) { - type = TCG_TYPE_V64; - } + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); } if (type != 0) { TCGv_vec t_vec = tcg_temp_new_vec(type); + uint32_t some; tcg_gen_dup_i64_vec(g->vece, t_vec, c); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ switch (type) { case TCG_TYPE_V256: - { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); - expand_2s_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, - t_vec, g->scalar_first, g->fniv); - if (some == oprsz) { - break; - } - dofs += some; - aofs += some; - oprsz -= some; - maxsz -= some; + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); + expand_2s_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, + t_vec, g->scalar_first, g->fniv); + if (some == oprsz) { + break; } + dofs += some; + aofs += some; + oprsz -= some; + maxsz -= some; /* fallthru */ case TCG_TYPE_V128: @@ -1055,48 +1103,60 @@ void tcg_gen_gvec_2s(uint32_t dofs, uint32_t aofs, uint32_t oprsz, void tcg_gen_gvec_3(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz, const GVecGen3 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs | bofs); check_overlap_3(dofs, aofs, bofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_3_vec(g->vece, dofs, aofs, bofs, some, 32, TCG_TYPE_V256, g->load_dest, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; bofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_3_vec(g->vece, dofs, aofs, bofs, oprsz, 16, TCG_TYPE_V128, g->load_dest, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_3_vec(g->vece, dofs, aofs, bofs, oprsz, 8, TCG_TYPE_V64, g->load_dest, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_3_i64(dofs, aofs, bofs, oprsz, g->load_dest, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_3_i32(dofs, aofs, bofs, oprsz, g->load_dest, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, g->data, g->fno); + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_3_i64(dofs, aofs, bofs, oprsz, g->load_dest, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_3_i32(dofs, aofs, bofs, oprsz, g->load_dest, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, + maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -1106,20 +1166,27 @@ void tcg_gen_gvec_3(uint32_t dofs, uint32_t aofs, uint32_t bofs, void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, uint32_t oprsz, uint32_t maxsz, const GVecGen4 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs | bofs | cofs); check_overlap_4(dofs, aofs, bofs, cofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_4_vec(g->vece, dofs, aofs, bofs, cofs, some, 32, TCG_TYPE_V256, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; @@ -1127,30 +1194,33 @@ void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, cofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, 16, TCG_TYPE_V128, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, 8, TCG_TYPE_V64, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_4_i64(dofs, aofs, bofs, cofs, oprsz, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_4_i32(dofs, aofs, bofs, cofs, oprsz, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_4_ool(dofs, aofs, bofs, cofs, - oprsz, maxsz, g->data, g->fno); - return; + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_4_i64(dofs, aofs, bofs, cofs, oprsz, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_4_i32(dofs, aofs, bofs, cofs, oprsz, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_4_ool(dofs, aofs, bofs, cofs, + oprsz, maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -2155,6 +2225,8 @@ void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs, [TCG_COND_LTU] = ltu_fn, [TCG_COND_LEU] = leu_fn, }; + TCGType type; + uint32_t some; check_size_align(oprsz, maxsz, dofs | aofs | bofs); check_overlap_3(dofs, aofs, bofs, maxsz); @@ -2165,51 +2237,59 @@ void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs, return; } - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V256, vece)) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + /* Implement inline with a vector type, if possible. + * Prefer integer when 64-bit host and 64-bit comparison. + */ + type = choose_vector_type(INDEX_op_cmp_vec, vece, oprsz, + TCG_TARGET_REG_BITS == 64 && vece == MO_64); + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_cmp_vec(vece, dofs, aofs, bofs, some, 32, TCG_TYPE_V256, cond); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; bofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V128, vece)) { + /* fallthru */ + case TCG_TYPE_V128: expand_cmp_vec(vece, dofs, aofs, bofs, oprsz, 16, TCG_TYPE_V128, cond); - } else if (TCG_TARGET_HAS_v64 - && check_size_impl(oprsz, 8) - && (TCG_TARGET_REG_BITS == 32 || vece != MO_64) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V64, vece)) { + break; + case TCG_TYPE_V64: expand_cmp_vec(vece, dofs, aofs, bofs, oprsz, 8, TCG_TYPE_V64, cond); - } else if (vece == MO_64 && check_size_impl(oprsz, 8)) { - expand_cmp_i64(dofs, aofs, bofs, oprsz, cond); - } else if (vece == MO_32 && check_size_impl(oprsz, 4)) { - expand_cmp_i32(dofs, aofs, bofs, oprsz, cond); - } else { - gen_helper_gvec_3 * const *fn = fns[cond]; + break; - if (fn == NULL) { - uint32_t tmp; - tmp = aofs, aofs = bofs, bofs = tmp; - cond = tcg_swap_cond(cond); - fn = fns[cond]; - assert(fn != NULL); + case 0: + if (vece == MO_64 && check_size_impl(oprsz, 8)) { + expand_cmp_i64(dofs, aofs, bofs, oprsz, cond); + } else if (vece == MO_32 && check_size_impl(oprsz, 4)) { + expand_cmp_i32(dofs, aofs, bofs, oprsz, cond); + } else { + gen_helper_gvec_3 * const *fn = fns[cond]; + + if (fn == NULL) { + uint32_t tmp; + tmp = aofs, aofs = bofs, bofs = tmp; + cond = tcg_swap_cond(cond); + fn = fns[cond]; + assert(fn != NULL); + } + tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, 0, fn[vece]); + return; } - tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, 0, fn[vece]); - return; + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); }