From patchwork Thu Mar 15 17:38:53 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 131833 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp1407000ljb; Thu, 15 Mar 2018 10:39:34 -0700 (PDT) X-Google-Smtp-Source: AG47ELsphI8Wbr/WTW6bR8rEY+tZ83PmhaoVZuWjMR1bHlrmu9e6oE/IAcPc4JYWdmsJZRgziXzv X-Received: by 10.55.122.130 with SMTP id v124mr14208709qkc.140.1521135574158; Thu, 15 Mar 2018 10:39:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521135574; cv=none; d=google.com; s=arc-20160816; b=kClUgCgOMhQpJ73As+N4ejcYcnYJvctGeADitbP+7AlvX8hyqSVpstqTCWtFmHdljL 3NMqv3IooFAT7t+IuCk8nSdw2cHBUpLu8T8Q7PuT+TUzKrnBE3v13yPCkf7QLyZQtDP+ FQm4lMgU/DWcza+gKYHDywUyh9hBoaA2oevS4y/M+nZW+ZHk+ToFXylg2yezDeJqTAtH D9YQxO+6qEhdlPsVNJktkTQ8xUx57CoewaUlgoAnbEoXiNZR0G50dlg7XbvjewOoZFF9 I4V8+ek7lrnKBARonU3l2x5ZMmqjzwMZZh1/xIkX/GdyDHv2Ii2iqHAJRiG5YPTpPpyk 4i4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject :content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=fku4iRPo89wVui51GnyeU/p+wOxcqKJUuqCW24cmVZFkcgXwrWI6V70qxcqj+iSGBz GLgWoj30rj7kuw5vFoa2IDPj8a2Dvwo15AP5gv/tgRCzZEPEmwTBZjUJ3wIVQab10T5M uqGH5uMh9it1zexF/xAhztUGT0cDib9JaVct/4BctfNEh2QVkl/9AU8poITANaThbQ81 Zf7vjgS8oUeIKeHChlm7qaPpboQ+ZBSIshPvgIqcZtLkBbayZDuNhzHCwKgbEBv+md8N 78LskHpZk+a36lsBZrBjN5vuwPZ6seb+nmG1NhLO/auPmZhkeCWiH2Hhz5teu8tPzX1W dONQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A70ThDuA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id u23si1378739qtu.306.2018.03.15.10.39.33 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 15 Mar 2018 10:39:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=A70ThDuA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52714 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWqj-0003RJ-J5 for patch@linaro.org; Thu, 15 Mar 2018 13:39:33 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59741) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWqK-0003PO-3M for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:09 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewWqI-00051L-UP for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:08 -0400 Received: from mail-pf0-x244.google.com ([2607:f8b0:400e:c00::244]:36070) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ewWqI-00050s-P2 for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:06 -0400 Received: by mail-pf0-x244.google.com with SMTP id 68so3100101pfx.3 for ; Thu, 15 Mar 2018 10:39:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=A70ThDuAqujtfy8+e8fE9DjlmDbTbzaaGMGwidubucuZ7KV9PsKP8lRsNLOEVcweaa DPtbEEpNeDrUKhcE7zCJoq4YrFn9haQZNxzHZrz7/VofJlutTNp/iwgvCBXOiSVnM7M0 0eOcSRRK6hUDhCyzBFnuKERE3rZNRtfFKA8eI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=37ouaMvY5pTOVrhHJUAjWDoVOcGc1UxIbP09Exhmpzk=; b=WscRwjS0s1ZLONLKuSI6LHsZvn4xwuHewESiCfUQ2XQEYyxlMTIWSwEEMPoUzPNGRs UzSY7cVfGWgy7vi05rlmsK/MAukDg1fz7XUe+MHigAjtbj6dv4kjGBKwUizDhmdNgBka 5oqrj0sBFvDSmEVZ0/Glda4mF7yEr+oxL7+gC5WaU2bRubU0Gznv2E4/so9jF4C56U9z 6zPJGJBLKeCKcS6vCvRR7SU1mqfYSIjahn6K6xK7hZ0HTYtSOdyDlCezGZoNthKgWFVy 9UTL8m84QQEedGxvh8DhVJdf8dEitbIor/BKGptwhv7xniYPbQc12gIOKtAUVVddHX4w lawQ== X-Gm-Message-State: AElRT7GLuKwUvZHP5JrlhoFipb0cwLRns7aqvgAIzynzdwzVNVQixF0t QayY3OGhE95fRE1vxjUfnRB4HGR1JPg= X-Received: by 10.99.156.17 with SMTP id f17mr7327442pge.102.1521135545384; Thu, 15 Mar 2018 10:39:05 -0700 (PDT) Received: from cloudburst.twiddle.net ([182.239.119.25]) by smtp.gmail.com with ESMTPSA id f6sm9684571pgo.58.2018.03.15.10.39.03 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 15 Mar 2018 10:39:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 16 Mar 2018 01:38:53 +0800 Message-Id: <20180315173855.22456-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180315173855.22456-1-richard.henderson@linaro.org> References: <20180315173855.22456-1-richard.henderson@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::244 Subject: [Qemu-devel] [PULL v2 1/3] tcg: Improve tcg_gen_muli_i32/i64 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Convert multiplication by power of two to left shift. Reviewed-by: Emilio G. Cota Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- tcg/tcg-op.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) -- 2.14.3 diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c index 3467787323..34b96d68f3 100644 --- a/tcg/tcg-op.c +++ b/tcg/tcg-op.c @@ -277,9 +277,15 @@ void tcg_gen_setcondi_i32(TCGCond cond, TCGv_i32 ret, void tcg_gen_muli_i32(TCGv_i32 ret, TCGv_i32 arg1, int32_t arg2) { - TCGv_i32 t0 = tcg_const_i32(arg2); - tcg_gen_mul_i32(ret, arg1, t0); - tcg_temp_free_i32(t0); + if (arg2 == 0) { + tcg_gen_movi_i32(ret, 0); + } else if (is_power_of_2(arg2)) { + tcg_gen_shli_i32(ret, arg1, ctz32(arg2)); + } else { + TCGv_i32 t0 = tcg_const_i32(arg2); + tcg_gen_mul_i32(ret, arg1, t0); + tcg_temp_free_i32(t0); + } } void tcg_gen_div_i32(TCGv_i32 ret, TCGv_i32 arg1, TCGv_i32 arg2) @@ -1430,9 +1436,15 @@ void tcg_gen_setcondi_i64(TCGCond cond, TCGv_i64 ret, void tcg_gen_muli_i64(TCGv_i64 ret, TCGv_i64 arg1, int64_t arg2) { - TCGv_i64 t0 = tcg_const_i64(arg2); - tcg_gen_mul_i64(ret, arg1, t0); - tcg_temp_free_i64(t0); + if (arg2 == 0) { + tcg_gen_movi_i64(ret, 0); + } else if (is_power_of_2(arg2)) { + tcg_gen_shli_i64(ret, arg1, ctz64(arg2)); + } else { + TCGv_i64 t0 = tcg_const_i64(arg2); + tcg_gen_mul_i64(ret, arg1, t0); + tcg_temp_free_i64(t0); + } } void tcg_gen_div_i64(TCGv_i64 ret, TCGv_i64 arg1, TCGv_i64 arg2) From patchwork Thu Mar 15 17:38:54 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 131836 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp1409932ljb; Thu, 15 Mar 2018 10:42:31 -0700 (PDT) X-Google-Smtp-Source: AG47ELuYawcetV55259+Wo2wNS2GDSNg4go3ASl0KNVJ67KNBMiFTH0quA/oFeau8+NHT+VXKlyx X-Received: by 10.200.28.90 with SMTP id j26mr14256411qtk.188.1521135751869; Thu, 15 Mar 2018 10:42:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521135751; cv=none; d=google.com; s=arc-20160816; b=tk8gbdaiSkLNRk+zh3PnDyMKDSIQI7ioh726lmmekcZxZZ7LZmoqN5AJB617uRWeD6 q1wVnWPZL9ANiSjDgNlz/pUf+JdYZUqnepbWhCwyUTtD2Fl+FUEAHS1Og5E+pjnEai6d yziSHAID+oOkLkPzn9+AXLCE4yXn05vtwFUOvMmSRGfdvgTVie2rqEuRSJIOmATlnq1R KS5Igosbh7cvAuDmtcf4orbK7DlLJNLMuvn7+KA+EYIlPo/NzO+RAb+7s2FRn+yglx3R 0qhHLPpmZLom7Tu0TBfoEpnMKajXizaLy0/eYvb0kMxTcX3gc82Oq3cLuxWEI9x7fBNK wtNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=IulWmAZYeh+3g5tThEZ2g5TSEKD1T/Yg3wOq9TlWIm4fD5L73LA5RqwzUGBe999bSt SNNx1l13WdTIFQL60gcZ7xYBHhDiUVhOSetwSXuSYTqPeDgGkC4gnYWAeF9Bu4jCUrG+ yz16LlhsgrJbGvGCBulCamg/2c3DvkKkNBa7TwLUNLIwnUhGKB1O1QTr9x6ukbACCgA9 3TGduyvzvfoyO3Unm75nhh4FJI3VAsgYASBpiNzlpc9cuPGKWDuVgnLPjCl2Go1za66k IcO9s1IYgRdZb3SdwhQZmYjhIFKoD2PkMeObeLusfkZLy4/OZnoYg1YoCnu5BYSFC9Eu mHJw== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VLge01cr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id 82si5192162qko.238.2018.03.15.10.42.31 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 15 Mar 2018 10:42:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=VLge01cr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52728 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWtb-0005Jc-BA for patch@linaro.org; Thu, 15 Mar 2018 13:42:31 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59758) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWqM-0003QV-CY for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:12 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewWqL-000535-CM for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:10 -0400 Received: from mail-pf0-x241.google.com ([2607:f8b0:400e:c00::241]:37452) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ewWqL-00052a-6H for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:09 -0400 Received: by mail-pf0-x241.google.com with SMTP id h11so3095123pfn.4 for ; Thu, 15 Mar 2018 10:39:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=VLge01crjOn9yhpsYpxiw8dMPzcnf+05ew8M4EcvWaW9ktju69KETiC18KO0DAZjrY RhMLo7lRST4Ge2afuDDT76pqgfkPWM2MigMOZTdFj8bduX3uLvsUKRPtccDzIebIrSUW bn3b4AqZs7yfMs10j3tgq95d+DR96NxYScAqs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=dDDlrROqkaZQZi5Yc/LAlOF0DHxsXVz9otaL0UYlmQE=; b=dZa8XEpmSmLpfUNwePidwrNAwy7TGesRFjs3k2AD0Ugu1UkRejqD14gRj8om3BHbYD j3YqVbNZoA7xSUIUy1Sw0PFkx8/DovLgDb9ulctg6Irq8Zxg2pKgKGiiMNWWNKEiY8KB Jwybxfzja29pWcrsoZwuZJEpQsotcylOuhbUgMz02VK5hzuZghVQSH/mGrf9AAzgOe2J jX3KhOblFNp/nZRqvpqPXQtaj6oUfdP9aLYpQmZmZ31TQlDvV+J+utM3AcGBcnH0VnSj exI8KLSdleQJqvOinh9pgTp8IWXVNIClmQCvqcBMRpHCxz3WELhirUdfQNDZeCrYFRnN sXTw== X-Gm-Message-State: AElRT7GNqUH0UnA3DHY+V/yfrw6rpppjRCBPSUI+hj3EKeSmEp6uKZ6p f4ZuqKjDVpXn5qUK6q1FKECheKaxwCA= X-Received: by 10.101.97.139 with SMTP id c11mr7309179pgv.435.1521135547788; Thu, 15 Mar 2018 10:39:07 -0700 (PDT) Received: from cloudburst.twiddle.net ([182.239.119.25]) by smtp.gmail.com with ESMTPSA id f6sm9684571pgo.58.2018.03.15.10.39.05 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 15 Mar 2018 10:39:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 16 Mar 2018 01:38:54 +0800 Message-Id: <20180315173855.22456-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180315173855.22456-1-richard.henderson@linaro.org> References: <20180315173855.22456-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c00::241 Subject: [Qemu-devel] [PULL v2 2/3] tcg/i386: Support INDEX_op_dup2_vec for -m32 X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Unknown why -m32 was passing with gcc but not clang; it should have failed for both. This would be used for tcg_gen_dup_i64_vec, and visible with the right TB and an aarch64 guest. Reported-by: Max Reitz Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.inc.c | 9 +++++++++ 1 file changed, 9 insertions(+) -- 2.14.3 diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c index fc05909d1d..d7e59e79c5 100644 --- a/tcg/i386/tcg-target.inc.c +++ b/tcg/i386/tcg-target.inc.c @@ -2696,6 +2696,12 @@ static void tcg_out_vec_op(TCGContext *s, TCGOpcode opc, case INDEX_op_x86_packus_vec: insn = packus_insn[vece]; goto gen_simd; +#if TCG_TARGET_REG_BITS == 32 + case INDEX_op_dup2_vec: + /* Constraints have already placed both 32-bit inputs in xmm regs. */ + insn = OPC_PUNPCKLDQ; + goto gen_simd; +#endif gen_simd: tcg_debug_assert(insn != OPC_UD2); if (type == TCG_TYPE_V256) { @@ -3045,6 +3051,9 @@ static const TCGTargetOpDef *tcg_target_op_def(TCGOpcode op) case INDEX_op_x86_vperm2i128_vec: case INDEX_op_x86_punpckl_vec: case INDEX_op_x86_punpckh_vec: +#if TCG_TARGET_REG_BITS == 32 + case INDEX_op_dup2_vec: +#endif return &x_x_x; case INDEX_op_dup_vec: case INDEX_op_shli_vec: From patchwork Thu Mar 15 17:38:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 131834 Delivered-To: patch@linaro.org Received: by 10.46.84.17 with SMTP id i17csp1407104ljb; Thu, 15 Mar 2018 10:39:40 -0700 (PDT) X-Google-Smtp-Source: AG47ELtzeYOqkTlkvIyO7siMJ0oEUEnz4ehiBcIxEZfk8hMx38XzksrGIEj+eTIYb2NX6EJLyVcS X-Received: by 10.237.41.194 with SMTP id o60mr398951qtd.197.1521135580685; Thu, 15 Mar 2018 10:39:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521135580; cv=none; d=google.com; s=arc-20160816; b=ryqTpyLfYYSX3LYaeS62Wz6ktbyZDZyoI//RhnYkgTJ7e6/6A4kYwQTsEutcFDuZ3S f5Lozd4K/Z5vjXOTVm+w117e72ifdDXoAf58D4W1DS4UxgTfKMWQaqPkEJyCMdjXpEZ/ 43jE0bv9boQCLHlx+PFTosl35EDm2VDRQK4u05rci8sdJouBkfd0AVQC30pJ7FklyEvi cO6PZurVP1F3eAsmw+I+jNLxUPyCJpAXL5zmdG1HxguJTw8EHC3qozxdZ2rvYGMkFtit VRe2tyjTYCEviWP+o3009ow0PYN4+SN7Zh227lYupoicfyDCkMotFqtENFlxRH0TxacT RQrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=iz4YyFpFaMpnVBxE/ECxQRg6es7Ir1bp/OsIZBKPbH8I35JIlvLAR01wjOuR6+qEp9 ipGaHfkFMGvHGpLyyP/KPZlic39f6OmP91KRtNjBxMGxRcXnDEpXxAhGLbiWLpmckgbV KEayaWqZZqABpFxK/04OTr4EZlHTbc/LkQcGKA5nmCCcxEonP8vkwI6vd7R/bW/YAH+y 1BXbJ92wsxAa7tCGOtBfOrifm9n42TksvkCmNirxVgXB+tQ2HwPMNQ1vg5m007rGYoUH sOM2NymBYgkcBYQnZliJplPNgSI9QkKjmIfeUdrlLmwGP+PMhb32ETVyznGCFmVREOMN UeZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=EFliLdOL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id k11si1605025qtb.3.2018.03.15.10.39.40 for (version=TLS1 cipher=AES128-SHA bits=128/128); Thu, 15 Mar 2018 10:39:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=EFliLdOL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52715 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWqq-0003VH-2N for patch@linaro.org; Thu, 15 Mar 2018 13:39:40 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59769) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewWqR-0003Su-1h for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewWqO-00055F-FI for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:15 -0400 Received: from mail-pg0-x242.google.com ([2607:f8b0:400e:c05::242]:34818) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1ewWqO-00054j-6G for qemu-devel@nongnu.org; Thu, 15 Mar 2018 13:39:12 -0400 Received: by mail-pg0-x242.google.com with SMTP id d1so3040123pgv.2 for ; Thu, 15 Mar 2018 10:39:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=EFliLdOLnioF76gtNXskaqG+7ap3SkAEpka/CJV5WVrEmcA8ZNQqV2tE+w0nVfCqr8 J4AS618Bcyu0rWb5X1va+3s4p8DuFEZx+TbfwsdXpQ6ocQZOCi9EIiF/2RTggiNkDTcf SkJqzzcEuiVi9nVDBmULjaVIzDs1xl3MNzyCk= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=QCFOQUN3wD1O4OQciVQZvmcRO2/CGYAHHQkpAk8oWRs=; b=bX55G2H2Ma0pidqAXyGzEqsDLoIrpUZEQQV4Xmfyi+XFYxFOZGq54diTu2/fSu79Ob J++o147neipMsWsnY1nb//YNvfr1zsmWF4n7jOr6m7/+hdfI8qK2lk7joiDrSl2EJp2r WMfH087XUCoizCHvmu8nRUc7VBVtM3t7KzjGhitJfGBCGr/f1xaSM8oOkeC9m9+TeaGd d5wHU/MJS7YotgAfWGDAzCneci+RSD+b4Ufzvq6EnzbqwOepqRNiB39Ozx7xcfRNycje LoKJA/btLOJjVqXpbfrqGQpin+wRmevjMInkPRKHvJcx4Zst5ztsw6TGoXGz/fl4WhX5 LaiQ== X-Gm-Message-State: AElRT7HYzIkNM3kEFK23MrZK8+zb1UsKl5NvGd5BFD+jTvExww/7o7z8 uSf1lkjPmJU3e5ln+xSavI45cbZqRu8= X-Received: by 10.101.97.139 with SMTP id c11mr7258442pgv.449.1521135550339; Thu, 15 Mar 2018 10:39:10 -0700 (PDT) Received: from cloudburst.twiddle.net ([182.239.119.25]) by smtp.gmail.com with ESMTPSA id f6sm9684571pgo.58.2018.03.15.10.39.07 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 15 Mar 2018 10:39:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Fri, 16 Mar 2018 01:38:55 +0800 Message-Id: <20180315173855.22456-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180315173855.22456-1-richard.henderson@linaro.org> References: <20180315173855.22456-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c05::242 Subject: [Qemu-devel] [PULL v2 3/3] tcg: Add choose_vector_size X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This unifies 5 copies of checks for supported vector size, and in the process fixes a missing check in tcg_gen_gvec_2s. This lead to an assertion failure for 64-bit vector multiply, which is not available in the AVX instruction set. Suggested-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/tcg-op-gvec.c | 438 ++++++++++++++++++++++++++++++++---------------------- 1 file changed, 259 insertions(+), 179 deletions(-) -- 2.14.3 diff --git a/tcg/tcg-op-gvec.c b/tcg/tcg-op-gvec.c index bfe44bba81..22db1590d5 100644 --- a/tcg/tcg-op-gvec.c +++ b/tcg/tcg-op-gvec.c @@ -351,6 +351,42 @@ static void gen_dup_i64(unsigned vece, TCGv_i64 out, TCGv_i64 in) } } +/* Select a supported vector type for implementing an operation on SIZE + * bytes. If OP is 0, assume that the real operation to be performed is + * required by all backends. Otherwise, make sure than OP can be performed + * on elements of size VECE in the selected type. Do not select V64 if + * PREFER_I64 is true. Return 0 if no vector type is selected. + */ +static TCGType choose_vector_type(TCGOpcode op, unsigned vece, uint32_t size, + bool prefer_i64) +{ + if (TCG_TARGET_HAS_v256 && check_size_impl(size, 32)) { + if (op == 0) { + return TCG_TYPE_V256; + } + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + * It is hard to imagine a case in which v256 is supported + * but v128 is not, but check anyway. + */ + if (tcg_can_emit_vec_op(op, TCG_TYPE_V256, vece) + && (size % 32 == 0 + || tcg_can_emit_vec_op(op, TCG_TYPE_V128, vece))) { + return TCG_TYPE_V256; + } + } + if (TCG_TARGET_HAS_v128 && check_size_impl(size, 16) + && (op == 0 || tcg_can_emit_vec_op(op, TCG_TYPE_V128, vece))) { + return TCG_TYPE_V128; + } + if (TCG_TARGET_HAS_v64 && !prefer_i64 && check_size_impl(size, 8) + && (op == 0 || tcg_can_emit_vec_op(op, TCG_TYPE_V64, vece))) { + return TCG_TYPE_V64; + } + return 0; +} + /* Set OPRSZ bytes at DOFS to replications of IN_32, IN_64 or IN_C. * Only one of IN_32 or IN_64 may be set; * IN_C is used if IN_32 and IN_64 are unset. @@ -376,19 +412,12 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } } - type = 0; - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32)) { - type = TCG_TYPE_V256; - } else if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16)) { - type = TCG_TYPE_V128; - } else if (TCG_TARGET_HAS_v64 && check_size_impl(oprsz, 8) - /* Prefer integer when 64-bit host and no variable dup. */ - && !(TCG_TARGET_REG_BITS == 64 && in_32 == NULL - && (in_64 == NULL || vece == MO_64))) { - type = TCG_TYPE_V64; - } - - /* Implement inline with a vector type, if possible. */ + /* Implement inline with a vector type, if possible. + * Prefer integer when 64-bit host and no variable dup. + */ + type = choose_vector_type(0, vece, oprsz, + (TCG_TARGET_REG_BITS == 64 && in_32 == NULL + && (in_64 == NULL || vece == MO_64))); if (type != 0) { TCGv_vec t_vec = tcg_temp_new_vec(type); @@ -414,21 +443,30 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } i = 0; - if (TCG_TARGET_HAS_v256) { + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ for (; i + 32 <= oprsz; i += 32) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V256); } - } - if (TCG_TARGET_HAS_v128) { + /* fallthru */ + case TCG_TYPE_V128: for (; i + 16 <= oprsz; i += 16) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V128); } - } - if (TCG_TARGET_HAS_v64) { + break; + case TCG_TYPE_V64: for (; i < oprsz; i += 8) { tcg_gen_stl_vec(t_vec, cpu_env, dofs + i, TCG_TYPE_V64); } + break; + default: + g_assert_not_reached(); } + tcg_temp_free_vec(t_vec); goto done; } @@ -484,7 +522,7 @@ static void do_dup(unsigned vece, uint32_t dofs, uint32_t oprsz, } tcg_temp_free_i64(t_64); goto done; - } + } } /* Otherwise implement out of line. */ @@ -866,49 +904,55 @@ static void expand_4_vec(unsigned vece, uint32_t dofs, uint32_t aofs, void tcg_gen_gvec_2(uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz, const GVecGen2 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs); check_overlap_2(dofs, aofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - /* ??? For maxsz > oprsz, the host may be able to use an opr-sized - operation, zeroing the balance of the register. We can then - use a max-sized store to implement the clearing without an extra - store operation. This is true for aarch64 and x86_64 hosts. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_2_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_2_vec(g->vece, dofs, aofs, oprsz, 16, TCG_TYPE_V128, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_2_vec(g->vece, dofs, aofs, oprsz, 8, TCG_TYPE_V64, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_2_i64(dofs, aofs, oprsz, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_2_i32(dofs, aofs, oprsz, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, g->data, g->fno); - return; + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_2_i64(dofs, aofs, oprsz, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_2_i32(dofs, aofs, oprsz, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -918,53 +962,64 @@ void tcg_gen_gvec_2(uint32_t dofs, uint32_t aofs, void tcg_gen_gvec_2i(uint32_t dofs, uint32_t aofs, uint32_t oprsz, uint32_t maxsz, int64_t c, const GVecGen2i *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs); check_overlap_2(dofs, aofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_2i_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, c, g->load_dest, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_2i_vec(g->vece, dofs, aofs, oprsz, 16, TCG_TYPE_V128, c, g->load_dest, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_2i_vec(g->vece, dofs, aofs, oprsz, 8, TCG_TYPE_V64, c, g->load_dest, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_2i_i64(dofs, aofs, oprsz, c, g->load_dest, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_2i_i32(dofs, aofs, oprsz, c, g->load_dest, g->fni4); - } else { - if (g->fno) { - tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, c, g->fno); + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_2i_i64(dofs, aofs, oprsz, c, g->load_dest, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_2i_i32(dofs, aofs, oprsz, c, g->load_dest, g->fni4); } else { - TCGv_i64 tcg_c = tcg_const_i64(c); - tcg_gen_gvec_2i_ool(dofs, aofs, tcg_c, oprsz, maxsz, c, g->fnoi); - tcg_temp_free_i64(tcg_c); + if (g->fno) { + tcg_gen_gvec_2_ool(dofs, aofs, oprsz, maxsz, c, g->fno); + } else { + TCGv_i64 tcg_c = tcg_const_i64(c); + tcg_gen_gvec_2i_ool(dofs, aofs, tcg_c, oprsz, + maxsz, c, g->fnoi); + tcg_temp_free_i64(tcg_c); + } + return; } - return; + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -981,37 +1036,30 @@ void tcg_gen_gvec_2s(uint32_t dofs, uint32_t aofs, uint32_t oprsz, type = 0; if (g->fniv) { - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32)) { - type = TCG_TYPE_V256; - } else if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16)) { - type = TCG_TYPE_V128; - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && check_size_impl(oprsz, 8)) { - type = TCG_TYPE_V64; - } + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); } if (type != 0) { TCGv_vec t_vec = tcg_temp_new_vec(type); + uint32_t some; tcg_gen_dup_i64_vec(g->vece, t_vec, c); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ switch (type) { case TCG_TYPE_V256: - { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); - expand_2s_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, - t_vec, g->scalar_first, g->fniv); - if (some == oprsz) { - break; - } - dofs += some; - aofs += some; - oprsz -= some; - maxsz -= some; + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); + expand_2s_vec(g->vece, dofs, aofs, some, 32, TCG_TYPE_V256, + t_vec, g->scalar_first, g->fniv); + if (some == oprsz) { + break; } + dofs += some; + aofs += some; + oprsz -= some; + maxsz -= some; /* fallthru */ case TCG_TYPE_V128: @@ -1055,48 +1103,60 @@ void tcg_gen_gvec_2s(uint32_t dofs, uint32_t aofs, uint32_t oprsz, void tcg_gen_gvec_3(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t oprsz, uint32_t maxsz, const GVecGen3 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs | bofs); check_overlap_3(dofs, aofs, bofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_3_vec(g->vece, dofs, aofs, bofs, some, 32, TCG_TYPE_V256, g->load_dest, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; bofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_3_vec(g->vece, dofs, aofs, bofs, oprsz, 16, TCG_TYPE_V128, g->load_dest, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_3_vec(g->vece, dofs, aofs, bofs, oprsz, 8, TCG_TYPE_V64, g->load_dest, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_3_i64(dofs, aofs, bofs, oprsz, g->load_dest, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_3_i32(dofs, aofs, bofs, oprsz, g->load_dest, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, g->data, g->fno); + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_3_i64(dofs, aofs, bofs, oprsz, g->load_dest, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_3_i32(dofs, aofs, bofs, oprsz, g->load_dest, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, + maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -1106,20 +1166,27 @@ void tcg_gen_gvec_3(uint32_t dofs, uint32_t aofs, uint32_t bofs, void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, uint32_t oprsz, uint32_t maxsz, const GVecGen4 *g) { + TCGType type; + uint32_t some; + check_size_align(oprsz, maxsz, dofs | aofs | bofs | cofs); check_overlap_4(dofs, aofs, bofs, cofs, maxsz); - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && g->fniv && check_size_impl(oprsz, 32) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V256, g->vece))) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + type = 0; + if (g->fniv) { + type = choose_vector_type(g->opc, g->vece, oprsz, g->prefer_i64); + } + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_4_vec(g->vece, dofs, aofs, bofs, cofs, some, 32, TCG_TYPE_V256, g->fniv); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; @@ -1127,30 +1194,33 @@ void tcg_gen_gvec_4(uint32_t dofs, uint32_t aofs, uint32_t bofs, uint32_t cofs, cofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && g->fniv && check_size_impl(oprsz, 16) - && (!g->opc || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V128, g->vece))) { + /* fallthru */ + case TCG_TYPE_V128: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, 16, TCG_TYPE_V128, g->fniv); - } else if (TCG_TARGET_HAS_v64 && !g->prefer_i64 - && g->fniv && check_size_impl(oprsz, 8) - && (!g->opc - || tcg_can_emit_vec_op(g->opc, TCG_TYPE_V64, g->vece))) { + break; + case TCG_TYPE_V64: expand_4_vec(g->vece, dofs, aofs, bofs, cofs, oprsz, 8, TCG_TYPE_V64, g->fniv); - } else if (g->fni8 && check_size_impl(oprsz, 8)) { - expand_4_i64(dofs, aofs, bofs, cofs, oprsz, g->fni8); - } else if (g->fni4 && check_size_impl(oprsz, 4)) { - expand_4_i32(dofs, aofs, bofs, cofs, oprsz, g->fni4); - } else { - assert(g->fno != NULL); - tcg_gen_gvec_4_ool(dofs, aofs, bofs, cofs, - oprsz, maxsz, g->data, g->fno); - return; + break; + + case 0: + if (g->fni8 && check_size_impl(oprsz, 8)) { + expand_4_i64(dofs, aofs, bofs, cofs, oprsz, g->fni8); + } else if (g->fni4 && check_size_impl(oprsz, 4)) { + expand_4_i32(dofs, aofs, bofs, cofs, oprsz, g->fni4); + } else { + assert(g->fno != NULL); + tcg_gen_gvec_4_ool(dofs, aofs, bofs, cofs, + oprsz, maxsz, g->data, g->fno); + return; + } + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); } @@ -2155,6 +2225,8 @@ void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs, [TCG_COND_LTU] = ltu_fn, [TCG_COND_LEU] = leu_fn, }; + TCGType type; + uint32_t some; check_size_align(oprsz, maxsz, dofs | aofs | bofs); check_overlap_3(dofs, aofs, bofs, maxsz); @@ -2165,51 +2237,59 @@ void tcg_gen_gvec_cmp(TCGCond cond, unsigned vece, uint32_t dofs, return; } - /* Recall that ARM SVE allows vector sizes that are not a power of 2. - Expand with successively smaller host vector sizes. The intent is - that e.g. oprsz == 80 would be expanded with 2x32 + 1x16. */ - - if (TCG_TARGET_HAS_v256 && check_size_impl(oprsz, 32) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V256, vece)) { - uint32_t some = QEMU_ALIGN_DOWN(oprsz, 32); + /* Implement inline with a vector type, if possible. + * Prefer integer when 64-bit host and 64-bit comparison. + */ + type = choose_vector_type(INDEX_op_cmp_vec, vece, oprsz, + TCG_TARGET_REG_BITS == 64 && vece == MO_64); + switch (type) { + case TCG_TYPE_V256: + /* Recall that ARM SVE allows vector sizes that are not a + * power of 2, but always a multiple of 16. The intent is + * that e.g. size == 80 would be expanded with 2x32 + 1x16. + */ + some = QEMU_ALIGN_DOWN(oprsz, 32); expand_cmp_vec(vece, dofs, aofs, bofs, some, 32, TCG_TYPE_V256, cond); if (some == oprsz) { - goto done; + break; } dofs += some; aofs += some; bofs += some; oprsz -= some; maxsz -= some; - } - - if (TCG_TARGET_HAS_v128 && check_size_impl(oprsz, 16) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V128, vece)) { + /* fallthru */ + case TCG_TYPE_V128: expand_cmp_vec(vece, dofs, aofs, bofs, oprsz, 16, TCG_TYPE_V128, cond); - } else if (TCG_TARGET_HAS_v64 - && check_size_impl(oprsz, 8) - && (TCG_TARGET_REG_BITS == 32 || vece != MO_64) - && tcg_can_emit_vec_op(INDEX_op_cmp_vec, TCG_TYPE_V64, vece)) { + break; + case TCG_TYPE_V64: expand_cmp_vec(vece, dofs, aofs, bofs, oprsz, 8, TCG_TYPE_V64, cond); - } else if (vece == MO_64 && check_size_impl(oprsz, 8)) { - expand_cmp_i64(dofs, aofs, bofs, oprsz, cond); - } else if (vece == MO_32 && check_size_impl(oprsz, 4)) { - expand_cmp_i32(dofs, aofs, bofs, oprsz, cond); - } else { - gen_helper_gvec_3 * const *fn = fns[cond]; + break; - if (fn == NULL) { - uint32_t tmp; - tmp = aofs, aofs = bofs, bofs = tmp; - cond = tcg_swap_cond(cond); - fn = fns[cond]; - assert(fn != NULL); + case 0: + if (vece == MO_64 && check_size_impl(oprsz, 8)) { + expand_cmp_i64(dofs, aofs, bofs, oprsz, cond); + } else if (vece == MO_32 && check_size_impl(oprsz, 4)) { + expand_cmp_i32(dofs, aofs, bofs, oprsz, cond); + } else { + gen_helper_gvec_3 * const *fn = fns[cond]; + + if (fn == NULL) { + uint32_t tmp; + tmp = aofs, aofs = bofs, bofs = tmp; + cond = tcg_swap_cond(cond); + fn = fns[cond]; + assert(fn != NULL); + } + tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, 0, fn[vece]); + return; } - tcg_gen_gvec_3_ool(dofs, aofs, bofs, oprsz, maxsz, 0, fn[vece]); - return; + break; + + default: + g_assert_not_reached(); } - done: if (oprsz < maxsz) { expand_clr(dofs + oprsz, maxsz - oprsz); }