From patchwork Fri Apr 30 20:24:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429837 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp769560jao; Fri, 30 Apr 2021 13:27:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzUwUbMesXT2zHKfEa5WhylIjInBwPogdggrfxXwXAh1RfXrDizS6hIOWHpoaXIY97SyRzp X-Received: by 2002:a5d:9598:: with SMTP id a24mr5422850ioo.24.1619814464104; Fri, 30 Apr 2021 13:27:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814464; cv=none; d=google.com; s=arc-20160816; b=V/bob+3H9bIFA6SkePtOLqRUQG3JjPe63ihwXZruCNBNOhFU4NZCIX6Dz1GB0MlJMX PmQshGqdJllVlQaQxU0Yc9Et9N73s1+VAMG47mfPpApNSt0DqX0LORPMdBSPC8rzuPxr Apu07g32x29aOgB7NmWHMw11+EiWSDK/oLAFk5UOSSxi1hxD8p48k4cjxATkyj0rabyp Q8HmKeZU+jwjQRn7U0mq9g3M/g6CqEpLh+1nOLExz6rvKQKdnAkntaSlaodQJ7kpWrqY hAWctYp3410zWs5pOFnnw4yikxGFoDhQENPpZ0P+bi8sQREpDVIwJgopBL+qDGgboHMj o1wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1UThjLBSb/XmVkkzs/Oev6mcIHSACdRwZzotytFesXI=; b=n+XBhhpmZPDm6kwoLmj6KNbVC65Oi2fVYhHywY1vWpDoQI3mO761CVjkdNOSDc00W1 kS5T9N3vbQaaWn2A8SjD26WX3UrmO3yLbqOs6VifjCxnDLj1Sl5hE5z1Ocb3m9XTPg8p aUG6i8gJyCxyly2N8pcDzTb7lj7sY7f5HXulGua1dmYKhEvk6kQ+N125qfp4VoYTMfDp /d9y56scAbOu1Csybt/IANw8ekxk/bk6HtpZnoR8vsS/Xzv8/inOcG2vtQH0CbFGlgHG eMy1pnvpT8WCrz0cCvrjR2R4JXib9XJdFUd/vC2uBrnW54pwyIIaPehE6mq21huiEm9k G74w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cNLHONW+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g16si3300020jaj.93.2021.04.30.13.27.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:27:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cNLHONW+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54634 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZjj-0000N2-D0 for patch@linaro.org; Fri, 30 Apr 2021 16:27:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51334) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiK-00083f-T3 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:16 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:39536) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiH-0005rY-Kr for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:16 -0400 Received: by mail-pj1-x1034.google.com with SMTP id z6-20020a17090a1706b0290155e8a752d8so2344854pjd.4 for ; Fri, 30 Apr 2021 13:26:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1UThjLBSb/XmVkkzs/Oev6mcIHSACdRwZzotytFesXI=; b=cNLHONW+YFgQKZ2/axzcHwBr4YBzMah9n+gwaeVweoI3j/YXVc8aFIk4mJ4Qfkmdov 7SXTjZdzZgy1/RCowPKv1WInS7KBnMO/18GvHiqGpkIOuMIPXcgaDfY81yoIodvscjok 3zRA5epdBoxXOose2c8A7fjBukt7jn/reAmkF+ki3F1ha83DhaGNIZV8rRD2vrM+zqNk 3QNIuVuS/jy8WksY6W9RJ4kQOaVm/DFaulD/72c/IWOucrw6gEQtx4LdoLFqsLJ8UShB 34JLtGU3zn4CSi9m4aquihhy5magy+NeOXEHZGn1KPAlFzUH+BjBTNp8rJeMafDdLdg/ H4ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1UThjLBSb/XmVkkzs/Oev6mcIHSACdRwZzotytFesXI=; b=Ha6FGcgCoke1MFCj1BNY8RRvmRBDWueZKC6bWOhzWulurKvmwlffM/CG+FZ0LceWOq K6Xpupc9oqcrXIcvNxy51K/WEZlQtZ74WRmhraqIVe+jrlIN5cBMmNEcHbwdlGAqbvnB RTRRyT+COGphNUTudPmse3u33mHAlG/yLFXs+aT5HJ4StOPCQix9tE2rV52phauSohTP LeR1KQn3pEtqo+BJ9gpRkJq6lMgc7rP5rlkWrkkiW8t//Rjbu5ZCLp4jFZYRi9KbgL/i ONVSmwJC2uVB6pYxMZIYGY/Q/SpaTjnyxAYRhxogMJQZZlCOq1cN/GqPkVSoj3tRzpQe DMUw== X-Gm-Message-State: AOAM532AEB3P7mmMTPPQlVo3OON3v1+C9iCXhrc1/ycUYND6HmCB+3E1 Ipf8phXP1TpoEM2NKc+eqMXUySLio9Bycw== X-Received: by 2002:a17:902:501:b029:eb:7a1b:c96c with SMTP id 1-20020a1709020501b02900eb7a1bc96cmr7177337plf.76.1619814371770; Fri, 30 Apr 2021 13:26:11 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 01/82] target/arm: Add ID_AA64ZFR0 fields and isar_feature_aa64_sve2 Date: Fri, 30 Apr 2021 13:24:49 -0700 Message-Id: <20210430202610.1136687-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, =?utf-8?q?Alex_Benn=C3=A9e?= Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Will be used for SVE2 isa subset enablement. Reviewed-by: Alex Bennée Signed-off-by: Richard Henderson --- v2: Do not read zfr0 from kvm unless sve is available. --- target/arm/cpu.h | 16 ++++++++++++++++ target/arm/helper.c | 3 +-- target/arm/kvm64.c | 11 +++++++++++ 3 files changed, 28 insertions(+), 2 deletions(-) -- 2.25.1 diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 616b393253..a6e1fa6333 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -947,6 +947,7 @@ struct ARMCPU { uint64_t id_aa64mmfr2; uint64_t id_aa64dfr0; uint64_t id_aa64dfr1; + uint64_t id_aa64zfr0; } isar; uint64_t midr; uint32_t revidr; @@ -2034,6 +2035,16 @@ FIELD(ID_AA64DFR0, DOUBLELOCK, 36, 4) FIELD(ID_AA64DFR0, TRACEFILT, 40, 4) FIELD(ID_AA64DFR0, MTPMU, 48, 4) +FIELD(ID_AA64ZFR0, SVEVER, 0, 4) +FIELD(ID_AA64ZFR0, AES, 4, 4) +FIELD(ID_AA64ZFR0, BITPERM, 16, 4) +FIELD(ID_AA64ZFR0, BFLOAT16, 20, 4) +FIELD(ID_AA64ZFR0, SHA3, 32, 4) +FIELD(ID_AA64ZFR0, SM4, 40, 4) +FIELD(ID_AA64ZFR0, I8MM, 44, 4) +FIELD(ID_AA64ZFR0, F32MM, 52, 4) +FIELD(ID_AA64ZFR0, F64MM, 56, 4) + FIELD(ID_DFR0, COPDBG, 0, 4) FIELD(ID_DFR0, COPSDBG, 4, 4) FIELD(ID_DFR0, MMAPDBG, 8, 4) @@ -4215,6 +4226,11 @@ static inline bool isar_feature_aa64_ssbs(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64pfr1, ID_AA64PFR1, SSBS) != 0; } +static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper.c b/target/arm/helper.c index 9b1b98705f..f47dd96076 100644 --- a/target/arm/helper.c +++ b/target/arm/helper.c @@ -7561,8 +7561,7 @@ void register_cp_regs_for_features(ARMCPU *cpu) .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 4, .access = PL1_R, .type = ARM_CP_CONST, .accessfn = access_aa64_tid3, - /* At present, only SVEver == 0 is defined anyway. */ - .resetvalue = 0 }, + .resetvalue = cpu->isar.id_aa64zfr0 }, { .name = "ID_AA64PFR5_EL1_RESERVED", .state = ARM_CP_STATE_AA64, .opc0 = 3, .opc1 = 0, .crn = 0, .crm = 4, .opc2 = 5, .access = PL1_R, .type = ARM_CP_CONST, diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c index dff85f6db9..e8729b53fd 100644 --- a/target/arm/kvm64.c +++ b/target/arm/kvm64.c @@ -567,6 +567,17 @@ bool kvm_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf) err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64mmfr2, ARM64_SYS_REG(3, 0, 0, 7, 2)); + /* + * Before v5.1, KVM did not support SVE and did not expose + * ID_AA64ZFR0_EL1 even as RAZ. After v5.1, KVM still does + * not expose the register to "user" requests like this + * unless the host supports SVE. + */ + if (isar_feature_aa64_sve(&ahcf->isar)) { + err |= read_sys_reg64(fdarray[2], &ahcf->isar.id_aa64zfr0, + ARM64_SYS_REG(3, 0, 0, 4, 4)); + } + /* * Note that if AArch32 support is not present in the host, * the AArch32 sysregs are present to be read, but will From patchwork Fri Apr 30 20:24:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429835 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp769147jao; Fri, 30 Apr 2021 13:27:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx3vNJ09bikRkwHjCPzrqmtMi45vbHrKuD3XfMEN1JcTwJHJFVhOzes0Ihlv5cX6xOKXyKH X-Received: by 2002:a9d:764b:: with SMTP id o11mr5092243otl.149.1619814426518; Fri, 30 Apr 2021 13:27:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814426; cv=none; d=google.com; s=arc-20160816; b=cHzhTOb0O1FV5nRSM6NSza0FVmU5VoRDdvJ4PCRUSu5VBhLMrjDTDvUDFt2c5qe6Y1 wVPP2sNN4VwlHxNx8QeXqLAG9ZqljelaeV7E/5PlrQUDpgKODC3bnOnEm8GUYyeVcEaU xlRahCNHpvf6tPT1sS1a/2ZLrfLFeyUbH/EPYVyFaIi8ZU4qqNwnTtHq7YvsEKfAfu9o V9E2J+ETVkD/7FdeAVVnkmpPJzFRwdrRWtuvlRGtiE5Hs2weO7liFuvtR8Bh2X9QFjlz nue2/4EDTTWihC6WzVWzR0rxl+ftC0itiZ+4qqetq5ssgiUeY2JqGg2nm6Oxgo6BfEgQ +Igg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=fI6JN3iS8n6r1R4eIA5hVPbjjVYwF3qnxo33YmutgM5zlS7wgLbEUm32G9FObJ4w6f QK9e9TZRk4q4MukEoGmv8s0WCc+HyLFlW5nL9+lyCSbubuKH320CLFpVmLb2Ux6R6Z+f cPydojTJ2nZ1kC8KTllZuco47eovzzi7RlNIWtsEQr5BE9mWjM8U3VcO2PA76+wVXdeo WmtFs8kPy1RTcL+0Wmp+RltbllbuEnj/gmsQQS4yfuxaMY02reURPZwzovdd6JK2fwaf 6lAdR3uCnS+vjtXf8tO9SHZv+i+3P4uEAY7O1lwBFoojNbnVNC0GJUykYcS8+d3trsTr hOgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xBtxhyUn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f20si5715038otc.182.2021.04.30.13.27.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:27:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xBtxhyUn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54196 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZj7-00008w-OM for patch@linaro.org; Fri, 30 Apr 2021 16:27:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51360) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiL-00084H-In for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:17 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]:35837) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiH-0005rm-Se for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:17 -0400 Received: by mail-pl1-x634.google.com with SMTP id t21so6128883plo.2 for ; Fri, 30 Apr 2021 13:26:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=xBtxhyUnGA/Vl3wdyNEW6W44HqHkHT840Jctgl0XaQFA2oTN1cxq4Jy3HjMnbbU1kX DqYU1XAuI/Q5uhweNDUS7/xhqwjQ6NAGYfiEXf7t5Bw0ZZ3xdKni/+3HyxjgYVOJnf0C 92phgRkunDus7l6ROqDKwxQSh2iDzsc6ik4Cr//ZU/DTvdhX0c5LbIUlFdksxpBA5GCx bL5z6+tu9ERS/9HzaeSDRBxrQG4CC0L/JIVbUt/DuOl9O4+/iB2hiO5FU4xCU4YRaW3Z IIL/87AANPcYpUKqdrlf+/GE4dGYyg6TBJHRaOg2jzLGdi7u3MbNIpU49gLDX0n+cjP6 Ydwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Y5trnwoiyeOfl0r4vLyT45eBorm8d2BCBnFoodmdZyM=; b=hcFBCwXZ8IUPLMlHd8/jT1j/rLU+ZLveCccWbWWjCqBBlwWWwHhSpEG5g09BjHzSTk 7T/HIET6S7algHPkg7RoBO67+jg8U+T3oII08E26Nu1uItUp03wS82sjp22ghXS+/Xx/ KtU3X1EG1SQXxzbs4IbEhW1C/55OA0qT/mpeid1ocx6ebKfV8RKnLpgP4j3eTR5iNh2n Elt04APygyCe2pklhSl4vzWN0o1u5DjdDmKkKO0gdyHWv1xgdKUhIU/oORS6vTJX3R8G +syugtlAg5teKcPSNoiRTekK/X0mQ/GQq3q+Ee2yq/jXW4LUOV58Rt0w36LN/oVDE/46 EFdA== X-Gm-Message-State: AOAM533FdRG0PLPcSnlPwZb2aAvS+uDjo25zQbqO8Zh+/hzB1OB0kuvG 9A4XXa676t5i2bLGXS/BscNI3KEZroSoFA== X-Received: by 2002:a17:902:b687:b029:eb:6491:b3f7 with SMTP id c7-20020a170902b687b02900eb6491b3f7mr7030388pls.38.1619814372402; Fri, 30 Apr 2021 13:26:12 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 02/82] target/arm: Implement SVE2 Integer Multiply - Unpredicated Date: Fri, 30 Apr 2021 13:24:50 -0700 Message-Id: <20210430202610.1136687-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For MUL, we can rely on generic support. For SMULH and UMULH, create some trivial helpers. For PMUL, back in a21bb78e5817, we organized helper_gvec_pmul_b in preparation for this use. Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 ++++ target/arm/sve.decode | 10 ++++ target/arm/translate-sve.c | 50 ++++++++++++++++++++ target/arm/vec_helper.c | 96 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 166 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index ff8148ddc6..2c412ffd3b 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -828,6 +828,16 @@ DEF_HELPER_FLAGS_3(gvec_cgt0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_b, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(gvec_cge0_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_smulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(gvec_umulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_umulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sshl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sshl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_ushl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5c90603358..557706cacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1090,3 +1090,13 @@ ST1_zprz 1110010 .. 00 ..... 100 ... ..... ..... \ @rprr_scatter_store xs=0 esz=3 scale=0 ST1_zprz 1110010 .. 00 ..... 110 ... ..... ..... \ @rprr_scatter_store xs=1 esz=3 scale=0 + +#### SVE2 Support + +### SVE2 Integer Multiply - Unpredicated + +# SVE2 integer multiply vectors (unpredicated) +MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm +SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm +UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm +PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 864ed669c4..f82d7d96f6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5795,3 +5795,53 @@ static bool trans_MOVPRFX_z(DisasContext *s, arg_rpr_esz *a) { return do_movz_zpz(s, a->rd, a->rn, a->pg, a->esz, false); } + +/* + * SVE2 Integer Multiply - Unpredicated + */ + +static bool trans_MUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, tcg_gen_gvec_mul, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool do_sve2_zzz_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_smulh_b, gen_helper_gvec_smulh_h, + gen_helper_gvec_smulh_s, gen_helper_gvec_smulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_UMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_umulh_b, gen_helper_gvec_umulh_h, + gen_helper_gvec_umulh_s, gen_helper_gvec_umulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 3fbeae87cb..40b92100bf 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1985,3 +1985,99 @@ void HELPER(simd_tblx)(void *vd, void *vm, void *venv, uint32_t desc) clear_tail(vd, oprsz, simd_maxsz(desc)); } #endif + +/* + * NxN -> N highpart multiply + * + * TODO: expose this as a generic vector operation. + */ + +void HELPER(gvec_smulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((int32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((int64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_smulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + muls64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 8; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint16_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = ((uint32_t)n[i] * m[i]) >> 16; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = ((uint64_t)n[i] * m[i]) >> 32; + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint64_t discard; + + for (i = 0; i < opr_sz / 8; ++i) { + mulu64(&discard, &d[i], n[i], m[i]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} From patchwork Fri Apr 30 20:24:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429836 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp769290jao; Fri, 30 Apr 2021 13:27:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxhHD+Ps6F8XGmy9mC4+OicmjLPrRjQ+dKTM3jLNC+TAMvHZixJzGtRkC405wcIOKuxAUJm X-Received: by 2002:a05:6602:15c8:: with SMTP id f8mr5137767iow.53.1619814436531; Fri, 30 Apr 2021 13:27:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814436; cv=none; d=google.com; s=arc-20160816; b=q+CvbXnz3uuPts0TGEA92ap/pcVR4246I+hIEJfE/hTxtTLWGDibP52DLPM1ZaSaEF hZ6YtzQo/yI8OAPuhJ6DSCt/yEeJqhpo9RefXbn3Nvw+9VHmrzx0BOuc8Au1R8Xw1CLn 7/WwJUp2SiLJCeEmGL8fKWMfXmn9ygtj2eELVetQ8Xp1oIkCwiVWcFiasIIT1rafg11E fBzBBIXOi8Dr826koFrASHkTg2hRDgoNwklH/npC7veCagm6SKDu5rD8/OJ3wNZauA6A 9b8FPH8gRLIspNQZf5T0NU0tnC+Xta/sd5XtmLkJVZlaLnHAqG+w3wx5g4qY38AGldXV A0yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=HWIZ19OFL/CLOVWMpB31DmzTWZ1yV71b+9MaJe884aZfPxRROXPiBuTeWjEKrI9Y32 wQz9zzhIYltajFkQZ7qboTraO5bN1uvpECdqItgIZZ/MqhABBdSfyqSoiRtu7R0znuNB SPrCFyWjtBOHcd04OROyBgLao+V4MMsRP+VnAZ82fpC3DeMPrIxyuZz43mDKFAi3QLXO LfJvxzT97Y4INuQF0bvjNUGEanfeLPD4qLoQxxQqO2/odCn2sOSgKlIuCHFk+NESqeFn fZjRcyVCoZguKQDwLKAgXXFhALE/9kUFZeEbyBUnPAchH5xNiGrnc4HJMdntHBCLsKNH +w4g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qClG0qVS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x8si3190633ilh.7.2021.04.30.13.27.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:27:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qClG0qVS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54382 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZjH-0000Fd-TQ for patch@linaro.org; Fri, 30 Apr 2021 16:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51400) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiM-00086V-Rf for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:18 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:35838) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiI-0005s5-9a for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:18 -0400 Received: by mail-pl1-x635.google.com with SMTP id t21so6128900plo.2 for ; Fri, 30 Apr 2021 13:26:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=qClG0qVS9uY9kIKPeYLlLKbsC/uBNgXpwMK25iC5eaSR7a/ptjyE3LF4Nw/4xH47VE wNE4TDA/msd7qosWjiek4rixkgDM8tpG3S0vAcxtxPpyMvWwv/8uLG5kv5yrSSDW4V8C 0plSGoch41jv9aheM/RBnbpO6hPxSxiQ6SvC6k+5CVl6p4DmUGTXFukYLotmd3N45BLa pGLxpGe9hS04nusld+8IN+GOeWxbBkYVcvdyTwYxQbqvl6gbN3S7BMAj9XMi8hneWTqj msMtZOkwjmd2ipV8oTssdbnuNcLZj5ldy+ChyLluCn6PpxnxcBMT3WzMgavFh7gnmdh9 J8+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CQ6txFH7Dtop6I+Akap+iOzHMzqCTj0Z29F/hci/3rc=; b=Yab4j28OP/XuqyWKw07Bfa7wVx8zmQQECE3I59SelqRyR3qhcaZjYO4zGchhFKLKPc p9wO5cmqtSPDH1x5LQyLzwWGDHmtSLiz8LYXzsRlDe4EyAUUorIMSlORq2LvtPbsa5J9 xmkczLxP3+ky43hXa2m4EKmzsuIYeWu7Z4G0LVKh+YjwpARbnVA3XOSSxzjTQdd0Ivwm TCf9ElJ1HI82l0fQB1HmA9LBU38Rspi0tjTU/pAA1KO2uqvyyDUSi2p+fvyCzSx1IWyG IWlVgKlm9S2+rW3iu3kWyvuyvu6Rb1jMrYP1pspHxgL97GTVlFqQJ08+IzuIch3ofyuk GxHA== X-Gm-Message-State: AOAM5326xIyl+Bm25v0G7v+Wa4vuEJFuhzLICimURi+35C8rY0/Gfjpw lPnt6zSFprfqnZVhQtLOuBOqm500M2l4jg== X-Received: by 2002:a17:902:9349:b029:ed:2b9:5a02 with SMTP id g9-20020a1709029349b02900ed02b95a02mr7246854plp.54.1619814373032; Fri, 30 Apr 2021 13:26:13 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 03/82] target/arm: Implement SVE2 integer pairwise add and accumulate long Date: Fri, 30 Apr 2021 13:24:51 -0700 Message-Id: <20210430202610.1136687-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 44 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 39 +++++++++++++++++++++++++++++++++ 4 files changed, 102 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index e4cadd2a65..b2a274b40b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -158,6 +158,20 @@ DEF_HELPER_FLAGS_5(sve_umulh_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_umulh_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 557706cacb..0524c01fcf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1100,3 +1100,8 @@ MUL_zzz 00000100 .. 1 ..... 0110 00 ..... ..... @rd_rn_rm SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 + +### SVE2 Integer - Predicated + +SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn +UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c068dfa0d5..42fe315485 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -517,6 +517,50 @@ DO_ZPZZ_D(sve_asr_zpzz_d, int64_t, DO_ASR) DO_ZPZZ_D(sve_lsr_zpzz_d, uint64_t, DO_LSR) DO_ZPZZ_D(sve_lsl_zpzz_d, uint64_t, DO_LSL) +static inline uint16_t do_sadalp_h(uint16_t n, uint16_t m) +{ + int8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_sadalp_s(uint32_t n, uint32_t m) +{ + int16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) +{ + int32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) + +static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) +{ + uint8_t n1 = n, n2 = n >> 8; + return m + n1 + n2; +} + +static inline uint32_t do_uadalp_s(uint32_t n, uint32_t m) +{ + uint16_t n1 = n, n2 = n >> 16; + return m + n1 + n2; +} + +static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) +{ + uint32_t n1 = n, n2 = n >> 32; + return m + n1 + n2; +} + +DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f82d7d96f6..208d9ea7e0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5845,3 +5845,42 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) { return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } + +/* + * SVE2 Integer - Predicated + */ + +static bool do_sve2_zpzz_ool(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_ool(s, a, fn); +} + +static bool trans_SADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_sadalp_zpzz_h, + gen_helper_sve2_sadalp_zpzz_s, + gen_helper_sve2_sadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} + +static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[3] = { + gen_helper_sve2_uadalp_zpzz_h, + gen_helper_sve2_uadalp_zpzz_s, + gen_helper_sve2_uadalp_zpzz_d, + }; + if (a->esz == 0) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); +} From patchwork Fri Apr 30 20:24:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429839 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp772122jao; Fri, 30 Apr 2021 13:31:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJySzv+osL/yTvYV3GuM+L7jFbvsw6xeTjuSu8r5S72aWIO2dMYdu3fUA20jZ83cA7Hg1aoB X-Received: by 2002:a9d:2787:: with SMTP id c7mr5135480otb.105.1619814685543; Fri, 30 Apr 2021 13:31:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814685; cv=none; d=google.com; s=arc-20160816; b=tAO7j6SDSedWUqUntwN1oS7GDhEcwmGrKayeYPj7c/gSawHYOlwSNQRVR8KgZaKstg kEVez5odVR0bnmwSo4Pag03W9zvi7wlp4A09cMQ+4nZyuOLkzSCPZ2Fmio9jDi86aOwu 2AOH8i60YbcR8SM71a5Dq8rTqt9YpLWtTMpmLzyPX4KGK99Q5YDJhjdrqUvW4wm+QM5G BR6ZVUXldf0lZQUEz7IBwXVIcPoouNaju9mIsVSXexarKo2g304qFY+a1Wtf1bcNO7Yo AIuSqN3jblMwZjFeVspHsyW5FIMSs/4iHHpAKOQHUogNBzxNwfU2gELbvlLLFSufYY1T aaQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=Wk9Q2BzONwOx4L8iTCS15Mo8ULc1mnoW4bD4XTnECt8B7gEGOm0VYe7O8ZJVIglqnE u5rWx6aq7sTNi2cVX1qZtJfEWmrXPR235lDLS+hKpcUHWaNHHujWTH917zoxB+XXpXgh fKgZymokO9zPMBxY1koL/4y7ULvLAbXGd6KIbKC2QShRgXb+rivQRK/3z+qorRMzUoqM RzX3Ct1TbJYT9pAPysYlm993ad+hwClStyjAN4EC5/sZ6iKs5ywuQPC2YJcI4TlvWqmd YV5wZfBE7iqNU1RAxiq7aLJ+Gznd7Z6JAZX+cDzh4EKJeq/lusPV4TT+YkGeiel9tCmz DJkw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bNiBy4Q1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p67si5257711oig.196.2021.04.30.13.31.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:31:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bNiBy4Q1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33966 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZnI-0003ZJ-So for patch@linaro.org; Fri, 30 Apr 2021 16:31:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51474) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiP-0008Bk-2m for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:21 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:45576) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiI-0005sq-Rg for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:20 -0400 Received: by mail-pj1-x1029.google.com with SMTP id gc22-20020a17090b3116b02901558435aec1so2370629pjb.4 for ; Fri, 30 Apr 2021 13:26:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=bNiBy4Q1iP117nwQo7mpSz0y0zgrYFLwWrxar+avVYXVO5kbg5EuYEBg0SSxlP6dUb D7K5vcAtnYnf4K9QpN80wCZEu2eSzpVrgVSmg7Z9xGGBA9yljWlLdXae7odGUEzY5HiM xbhk6ZFP+Q4IfFLerruIfqxG/vACqTt+p7vWPI15H8AB79aOua8H1WvPUOujvtvMpjYR Jgh7Kf+AMMpWAAYxQ+zEYJyMZbHqN70woT+RpzUFO7eA5EsHNG6AYfqobjR/acbQBMxV K4ELt7zACkGltUb7y4qlh14yAtKwugy5eXkhU+nYTkvkt2rRXqYm9fFeJzMdFmkH58av 1yOw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FXd4wOc5+bYMVSLk2jKG60M24P25fU16HFUkpUXY4d4=; b=hMOZT1pqyzAjMhg4r9cMSY3IL7+csWhkSb4jZtghMbOB5fbn9dHed2s+TrX8OK5kmK LMNRCAz5EYjwj5B5h2FokkCZGnCZl2qMrPvSmh3pe3tAgXrkDuIF0ngC2/KLDvtGxylm thX4215X+QI/kL3sCR3Dw3iJH7OCgHEJAmJnJUn5B6IGUxrJ8nK92f+VDwrX3HdVk+qt JBrNl+9p0b2mfVNb/oU9I7WbnEnnLs83EXs130Ceoo10g0QpUtQOD+ab8nqU3NEgj4cC wlRvwtn9nqt/vUr0cX4noV00soLQwl9YvUUW/hqaNXPSlQPkfB6L9ntUA5NMk3nYzZAO IuDg== X-Gm-Message-State: AOAM5303OlT8Ps2oN6jPoI2AHdar9E/rB0XsiZY89i2JwQUOZnHTfpu0 vYbcmHzMEFjozrCoGcw12A8lHRHUEu/HKg== X-Received: by 2002:a17:902:9a87:b029:ed:ea18:b2f3 with SMTP id w7-20020a1709029a87b02900edea18b2f3mr6722488plp.69.1619814373604; Fri, 30 Apr 2021 13:26:13 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:13 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 04/82] target/arm: Implement SVE2 integer unary operations (predicated) Date: Fri, 30 Apr 2021 13:24:52 -0700 Message-Id: <20210430202610.1136687-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sqabs, sqneg (laurent desnogues) --- target/arm/helper-sve.h | 13 +++++++++++ target/arm/sve.decode | 7 ++++++ target/arm/sve_helper.c | 29 +++++++++++++++++++---- target/arm/translate-sve.c | 47 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+), 4 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b2a274b40b..9992e93e2b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -502,6 +502,19 @@ DEF_HELPER_FLAGS_4(sve_rbit_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_rbit_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqabs_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqneg_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqneg_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urecpe_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ursqrte_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_splice, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_cmpeq_ppzz_b, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0524c01fcf..5ba542969b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1105,3 +1105,10 @@ PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn UADALP_zpzz 01000100 .. 000 101 101 ... ..... ..... @rdm_pg_rn + +### SVE2 integer unary operations (predicated) + +URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn +URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn +SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn +SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 42fe315485..bbab84e81d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -535,8 +535,8 @@ static inline uint64_t do_sadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_sadalp_zpzz_h, int16_t, H1_2, do_sadalp_h) -DO_ZPZZ(sve2_sadalp_zpzz_s, int32_t, H1_4, do_sadalp_s) +DO_ZPZZ(sve2_sadalp_zpzz_h, uint16_t, H1_2, do_sadalp_h) +DO_ZPZZ(sve2_sadalp_zpzz_s, uint32_t, H1_4, do_sadalp_s) DO_ZPZZ_D(sve2_sadalp_zpzz_d, uint64_t, do_sadalp_d) static inline uint16_t do_uadalp_h(uint16_t n, uint16_t m) @@ -557,8 +557,8 @@ static inline uint64_t do_uadalp_d(uint64_t n, uint64_t m) return m + n1 + n2; } -DO_ZPZZ(sve2_uadalp_zpzz_h, int16_t, H1_2, do_uadalp_h) -DO_ZPZZ(sve2_uadalp_zpzz_s, int32_t, H1_4, do_uadalp_s) +DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) +DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) #undef DO_ZPZZ @@ -728,6 +728,27 @@ DO_ZPZ(sve_rbit_h, uint16_t, H1_2, revbit16) DO_ZPZ(sve_rbit_s, uint32_t, H1_4, revbit32) DO_ZPZ_D(sve_rbit_d, uint64_t, revbit64) +#define DO_SQABS(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ >= 0 ? x_ : x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqabs_b, int8_t, H1, DO_SQABS) +DO_ZPZ(sve2_sqabs_h, int16_t, H1_2, DO_SQABS) +DO_ZPZ(sve2_sqabs_s, int32_t, H1_4, DO_SQABS) +DO_ZPZ_D(sve2_sqabs_d, int64_t, DO_SQABS) + +#define DO_SQNEG(X) \ + ({ __typeof(X) x_ = (X), min_ = 1ull << (sizeof(X) * 8 - 1); \ + x_ == min_ ? -min_ - 1 : -x_; }) + +DO_ZPZ(sve2_sqneg_b, uint8_t, H1, DO_SQNEG) +DO_ZPZ(sve2_sqneg_h, uint16_t, H1_2, DO_SQNEG) +DO_ZPZ(sve2_sqneg_s, uint32_t, H1_4, DO_SQNEG) +DO_ZPZ_D(sve2_sqneg_d, uint64_t, DO_SQNEG) + +DO_ZPZ(sve2_urecpe_s, uint32_t, H1_4, helper_recpe_u32) +DO_ZPZ(sve2_ursqrte_s, uint32_t, H1_4, helper_rsqrte_u32) + /* Three-operand expander, unpredicated, in which the third operand is "wide". */ #define DO_ZZW(NAME, TYPE, TYPEW, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 208d9ea7e0..c30b3c476e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5884,3 +5884,50 @@ static bool trans_UADALP_zpzz(DisasContext *s, arg_rprr_esz *a) } return do_sve2_zpzz_ool(s, a, fns[a->esz - 1]); } + +/* + * SVE2 integer unary operations (predicated) + */ + +static bool do_sve2_zpz_ool(DisasContext *s, arg_rpr_esz *a, + gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ool(s, a, fn); +} + +static bool trans_URECPE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_urecpe_s); +} + +static bool trans_URSQRTE(DisasContext *s, arg_rpr_esz *a) +{ + if (a->esz != 2) { + return false; + } + return do_sve2_zpz_ool(s, a, gen_helper_sve2_ursqrte_s); +} + +static bool trans_SQABS(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqabs_b, gen_helper_sve2_sqabs_h, + gen_helper_sve2_sqabs_s, gen_helper_sve2_sqabs_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqneg_b, gen_helper_sve2_sqneg_h, + gen_helper_sve2_sqneg_s, gen_helper_sve2_sqneg_d, + }; + return do_sve2_zpz_ool(s, a, fns[a->esz]); +} From patchwork Fri Apr 30 20:24:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429838 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp771974jao; Fri, 30 Apr 2021 13:31:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxmVbtLLZMFG53n3u8yF9A+vqJo7NGoubmRl1CQM+Pe7f3zf0IxS9MLvkQ1Oryh1RYM09Qr X-Received: by 2002:a05:6830:2418:: with SMTP id j24mr5155244ots.87.1619814675867; Fri, 30 Apr 2021 13:31:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814675; cv=none; d=google.com; s=arc-20160816; b=LaYYMoZN/3VrTu6YJcXrFkFj66mEXmTVINNTeTYc5w0NHo4FeXhN/wYW53rz+xBKER R06RMm5dcrVVRcwO+5Ds9GPG9MBdK5t0hU8hsKqbM6F5rawYqkI7XQc4g5EyhmPDwGh0 TQxkVUhc9JjYJ1FeRbZg+egIctGV1pHCopP/in4mwzO6xDLL4jA31hI/inP8M8SAH/e0 N6ap11rwu70YvsJ0HBT/wV+17sz8sj6Kp2ImG8xcTadeqgldszIBDYKeWE4n544Vif5J aqpGFzhmrf2GKVV+UP72Dbyt2ArxJcPrGiyz7FYh/o1RcUeHz537IGnSTRzVpQKg4Ulh lOdg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=kwwqa35yamMRvsokvnvUajtWPCkSCA30YoHVU4glh24ypmV9z7lDC+vKhT+t/J3BpF e7BSSaWBX9vAqwXug+3XOryL5K6oTRo1Ix1dksPQDJMNNMPhXoYaHd8OlF+Iz9KrffF1 hH/L2TSh7koFLWwhv30igFhaKv3qnHm1YylcyOmie30MTmkjLeKKyGg5eCbjk0lIIZ2W 74NsmVe5LhtadQKGpfgveuJ0JTvU493JC2vD6QSHDE04kh4P6DojFGbZdIGZrFFA56ip qSJG6X3vr9owEFkFOMIyBOKb+3CHkrUQT1GMIrzp8pQVVrnJPTUVJvp/ZHoMAvWYCEAZ i1TQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bGJNuJLD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h137si928688oib.18.2021.04.30.13.31.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:31:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bGJNuJLD; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36018 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZn9-0004VS-01 for patch@linaro.org; Fri, 30 Apr 2021 16:31:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51472) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiP-0008Bg-2D for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:21 -0400 Received: from mail-pj1-x102f.google.com ([2607:f8b0:4864:20::102f]:33550) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiJ-0005tb-Ug for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:20 -0400 Received: by mail-pj1-x102f.google.com with SMTP id b14-20020a17090a6e0eb0290155c7f6a356so6162189pjk.0 for ; Fri, 30 Apr 2021 13:26:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=bGJNuJLDShsKhKsZ33kNbQJOvga/BQ/QFEa0dw/iZCsSEnSgnUVHCfUAzKRt6enBuJ Ucvmy5G4CIT5/knMQbAEUSaHGnCCkOWf+YfKcXfZBzPz0gMQ7Y3OnwD6+B+5XZGTt7rK W6R8XrqBBNdEvaFTQpkjF6g0RTTQS6JdXCMsfTHFJcByOHJxxps/MGsK3sfC2XiM9Dam wQUlh/fKdOo5YBBsWDodCViPP4ldjRpHGBN0zpwG6sLR+UELwZRgRFbc1M00zt8VZlQ5 Dv8r/ps/hdOfcnHIckFkIAlrs7q+KNy+k2BY8S82Vom99RxPWfGxzcJO0ghKA5KRrvfp oXCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DS4qOR8a0wrDRRaz52YdBuOsYQMMqkUfZ7NchqJV82A=; b=tutV2HWOfTgY2rhfY7ApgproTHOrUpv3qSCXwWsvOkLPQTGJCbhNYRk05eIJcw4Izc 9tko404SAfVrcmUaw2Vh5+4hmBRNj4+wPq2nTg4RGr5dq+kl8ogkSdgbDnRSCeFIS9Dp KqEOYcTUIiCkmqXkfpIOfHZo4MF9YI7Q3vYkY6/6o4ZU5F7LGwFJQnGoys9TE01CMtnc /lP//G30bRnorm8DOQZLAK/RbxUwkmaeBz1GneDGlL8cgf4W2W6BCCkd1/V/2QzUXcBN td9gjzSlU5H1ymzQ8mDuxQEZU70aR2NShAAOgHK6uu45cUBI5B4DfQdp1DuHgfrd+gOi P/yg== X-Gm-Message-State: AOAM531FJRe0mGhBNEsXbfQqd+lBJq1By3oSL/g5DShWuPbrJuEJYt8U ixHC4d1gP6nO9tzZgS1WLFObuX/DlUeRZw== X-Received: by 2002:a17:902:d70f:b029:ec:b679:f122 with SMTP id w15-20020a170902d70fb02900ecb679f122mr7251124ply.38.1619814374336; Fri, 30 Apr 2021 13:26:14 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:14 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 05/82] target/arm: Split out saturating/rounding shifts from neon Date: Fri, 30 Apr 2021 13:24:53 -0700 Message-Id: <20210430202610.1136687-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102f; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Split these operations out into a header that can be shared between neon and sve. The "sat" pointer acts both as a boolean for control of saturating behavior and controls the difference in behavior between neon and sve -- QC bit or no QC bit. Widen the shift operand in the new helpers, as the SVE2 insns treat the whole input element as significant. For the neon uses, truncate the shift to int8_t while passing the parameter. Implement right-shift rounding as tmp = src >> (shift - 1); dst = (tmp >> 1) + (tmp & 1); This is the same number of instructions as the current tmp = 1 << (shift - 1); dst = (src + tmp) >> shift; without any possibility of intermediate overflow. Signed-off-by: Richard Henderson --- v2: Widen the shift operand (laurent desnouges) --- target/arm/vec_internal.h | 138 +++++++++++ target/arm/neon_helper.c | 507 +++++++------------------------------- 2 files changed, 221 insertions(+), 424 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index e3eb3e7a6b..0102547a10 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -30,4 +30,142 @@ static inline void clear_tail(void *vd, uintptr_t opr_sz, uintptr_t max_sz) } } +static inline int32_t do_sqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -bits) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 31; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + int32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + int32_t extval = sextract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return (1u << (bits - 1)) - (src >= 0); +} + +static inline uint32_t do_uqrshl_bhs(uint32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (shift <= -(bits + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < bits) { + uint32_t val = src << shift; + if (bits == 32) { + if (!sat || val >> shift == src) { + return val; + } + } else { + uint32_t extval = extract32(val, 0, bits); + if (!sat || val == extval) { + return extval; + } + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return MAKE_64BIT_MASK(0, bits); +} + +static inline int32_t do_suqrshl_bhs(int32_t src, int32_t shift, int bits, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_bhs(src, shift, bits, round, sat); +} + +static inline int64_t do_sqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -64) { + /* Rounding the sign bit always produces 0. */ + if (round) { + return 0; + } + return src >> 63; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + int64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return src < 0 ? INT64_MIN : INT64_MAX; +} + +static inline uint64_t do_uqrshl_d(uint64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (shift <= -(64 + round)) { + return 0; + } else if (shift < 0) { + if (round) { + src >>= -shift - 1; + return (src >> 1) + (src & 1); + } + return src >> -shift; + } else if (shift < 64) { + uint64_t val = src << shift; + if (!sat || val >> shift == src) { + return val; + } + } else if (!sat || src == 0) { + return 0; + } + + *sat = 1; + return UINT64_MAX; +} + +static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, + bool round, uint32_t *sat) +{ + if (src < 0) { + *sat = 1; + return 0; + } + return do_uqrshl_d(src, shift, round, sat); +} + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/neon_helper.c b/target/arm/neon_helper.c index b637265691..338b9189d5 100644 --- a/target/arm/neon_helper.c +++ b/target/arm/neon_helper.c @@ -11,6 +11,7 @@ #include "cpu.h" #include "exec/helper-proto.h" #include "fpu/softfloat.h" +#include "vec_internal.h" #define SIGNBIT (uint32_t)0x80000000 #define SIGNBIT64 ((uint64_t)1 << 63) @@ -576,496 +577,154 @@ NEON_POP(pmax_s16, neon_s16, 2) NEON_POP(pmax_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_u16, neon_u16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, NULL)) NEON_VOP(shl_s16, neon_s16, 2) #undef NEON_FN -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if ((tmp >= (ssize_t)sizeof(src1) * 8) \ - || (tmp <= -(ssize_t)sizeof(src1) * 8)) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_s32)(uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_rshl_s32)(uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if ((shift >= 32) || (shift <= -32)) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_s64)(uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_rshl_s64)(uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - int64_t val = valop; - if ((shift >= 64) || (shift <= -64)) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000LL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8 || \ - tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (-tmp - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, NULL)) NEON_VOP(rshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, NULL)) NEON_VOP(rshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_rshl_u32)(uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32 || shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, NULL); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_rshl_u64)(uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - if (shift >= 64 || shift < -64) { - val = 0; - } else if (shift == -64) { - /* Rounding a 1-bit result just preserves that bit. */ - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - val <<= shift; - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, NULL); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_u16, neon_u16, 2) -NEON_VOP_ENV(qshl_u32, neon_u32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint32_t HELPER(neon_qshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - val = ~(uint64_t)0; - SET_QC(); - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= -shift; - } else { - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~(uint64_t)0; - } - } - return val; + return do_uqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = src1; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> 31; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +uint64_t HELPER(neon_qshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_uqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) NEON_VOP_ENV(qshl_s16, neon_s16, 2) -NEON_VOP_ENV(qshl_s32, neon_s32, 1) #undef NEON_FN -uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint32_t HELPER(neon_qshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val >>= 63; - } else if (shift < 0) { - val >>= -shift; - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - if (src1 & (1 << (sizeof(src1) * 8 - 1))) { \ - SET_QC(); \ - dest = 0; \ - } else { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = src1 >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - } \ - }} while (0) -NEON_VOP_ENV(qshlu_s8, neon_u8, 4) -NEON_VOP_ENV(qshlu_s16, neon_u16, 2) +uint64_t HELPER(neon_qshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) +{ + return do_sqrshl_d(val, (int8_t)shift, false, env->vfp.qc); +} + +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 8, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s8, neon_s8, 4) #undef NEON_FN -uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +#define NEON_FN(dest, src1, src2) \ + (dest = do_suqrshl_bhs(src1, (int8_t)src2, 16, false, env->vfp.qc)) +NEON_VOP_ENV(qshlu_s16, neon_s16, 2) +#undef NEON_FN + +uint32_t HELPER(neon_qshlu_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - if ((int32_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u32(env, valop, shiftop); + return do_suqrshl_bhs(val, (int8_t)shift, 32, false, env->vfp.qc); } -uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qshlu_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - if ((int64_t)valop < 0) { - SET_QC(); - return 0; - } - return helper_neon_qshl_u64(env, valop, shiftop); + return do_suqrshl_d(val, (int8_t)shift, false, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = ~0; \ - } else { \ - dest = 0; \ - } \ - } else if (tmp < -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp == -(ssize_t)sizeof(src1) * 8) { \ - dest = src1 >> (sizeof(src1) * 8 - 1); \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = ~0; \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u8, neon_u8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_uqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_u16, neon_u16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_u32)(CPUARMState *env, uint32_t val, uint32_t shift) { - uint32_t dest; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = ~0; - } else { - dest = 0; - } - } else if (shift < -32) { - dest = 0; - } else if (shift == -32) { - dest = val >> 31; - } else if (shift < 0) { - uint64_t big_dest = ((uint64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = ~0; - } - } - return dest; + return do_uqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_u64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (int8_t)shiftop; - if (shift >= 64) { - if (val) { - SET_QC(); - val = ~0; - } - } else if (shift < -64) { - val = 0; - } else if (shift == -64) { - val >>= 63; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == UINT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x8000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { \ - uint64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = ~0; - } - } - return val; + return do_uqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } -#define NEON_FN(dest, src1, src2) do { \ - int8_t tmp; \ - tmp = (int8_t)src2; \ - if (tmp >= (ssize_t)sizeof(src1) * 8) { \ - if (src1) { \ - SET_QC(); \ - dest = (typeof(dest))(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } else { \ - dest = 0; \ - } \ - } else if (tmp <= -(ssize_t)sizeof(src1) * 8) { \ - dest = 0; \ - } else if (tmp < 0) { \ - dest = (src1 + (1 << (-1 - tmp))) >> -tmp; \ - } else { \ - dest = src1 << tmp; \ - if ((dest >> tmp) != src1) { \ - SET_QC(); \ - dest = (uint32_t)(1 << (sizeof(src1) * 8 - 1)); \ - if (src1 > 0) { \ - dest--; \ - } \ - } \ - }} while (0) +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 8, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s8, neon_s8, 4) +#undef NEON_FN + +#define NEON_FN(dest, src1, src2) \ + (dest = do_sqrshl_bhs(src1, (int8_t)src2, 16, true, env->vfp.qc)) NEON_VOP_ENV(qrshl_s16, neon_s16, 2) #undef NEON_FN -/* The addition of the rounding constant may overflow, so we use an - * intermediate 64 bit accumulator. */ -uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t valop, uint32_t shiftop) +uint32_t HELPER(neon_qrshl_s32)(CPUARMState *env, uint32_t val, uint32_t shift) { - int32_t dest; - int32_t val = (int32_t)valop; - int8_t shift = (int8_t)shiftop; - if (shift >= 32) { - if (val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } else { - dest = 0; - } - } else if (shift <= -32) { - dest = 0; - } else if (shift < 0) { - int64_t big_dest = ((int64_t)val + (1 << (-1 - shift))); - dest = big_dest >> -shift; - } else { - dest = val << shift; - if ((dest >> shift) != val) { - SET_QC(); - dest = (val >> 31) ^ ~SIGNBIT; - } - } - return dest; + return do_sqrshl_bhs(val, (int8_t)shift, 32, true, env->vfp.qc); } -/* Handling addition overflow with 64 bit input values is more - * tricky than with 32 bit values. */ -uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t valop, uint64_t shiftop) +uint64_t HELPER(neon_qrshl_s64)(CPUARMState *env, uint64_t val, uint64_t shift) { - int8_t shift = (uint8_t)shiftop; - int64_t val = valop; - - if (shift >= 64) { - if (val) { - SET_QC(); - val = (val >> 63) ^ ~SIGNBIT64; - } - } else if (shift <= -64) { - val = 0; - } else if (shift < 0) { - val >>= (-shift - 1); - if (val == INT64_MAX) { - /* In this case, it means that the rounding constant is 1, - * and the addition would overflow. Return the actual - * result directly. */ - val = 0x4000000000000000ULL; - } else { - val++; - val >>= 1; - } - } else { - int64_t tmp = val; - val <<= shift; - if ((val >> shift) != tmp) { - SET_QC(); - val = (tmp >> 63) ^ ~SIGNBIT64; - } - } - return val; + return do_sqrshl_d(val, (int8_t)shift, true, env->vfp.qc); } uint32_t HELPER(neon_add_u8)(uint32_t a, uint32_t b) From patchwork Fri Apr 30 20:24:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429842 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp774368jao; Fri, 30 Apr 2021 13:34:35 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzgSA0TLv+sfRh2tleiuF0LIY0bkdkjIycDxQ0B3w1eWV482kQZ7zr9Ef+0c2VJ1IAKNq/J X-Received: by 2002:a92:c0d0:: with SMTP id t16mr5483359ilf.257.1619814875280; Fri, 30 Apr 2021 13:34:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814875; cv=none; d=google.com; s=arc-20160816; b=C2oT9XjFI4Mk120qGaNgO3Y3TniWOIenPl+ebJuaslBC0ZUz/00HQu5YEGvVl/VTmw 9CfDNpOgixWXaSOmfl2B83K8oRdCHYDoeREghmZE6pJPTigLJ6n4rRH2QTuPx+SrVKPg yPDG7Ojb0IIv6vcOtcLR6d5Ofknox0mALhWU+3sn2vujFVNWqXCxe28fJf3LEdAe+9tB BLsB25vL+HlL98VsIJmYAxCri4N637YvEAivzrVd2F1t0B8ErRQuk7rwP++dcO2vQ2uC JFvOSN2BXaFksTuBgRbb+TEZHumoJVHRXSA8dPxyqvUX8CI6fSvUkACiTa2TR8NiAWfl 4RGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=b9uf/suYl0RdaZWTeaV40OfqeSaxDJubnGAORJOqmKwws8mdLMqQ6l+4/uDYpAnFt/ rG1sSvUFy3jKHeR+xl0zuUgztN0Isi/9bWJRS/odLdMtfK0Dhic2fSK4bV6lOhR2XQCm 8J4Ao0kzPZJHcF8HxldxhmwnPhvtuSpFeO98gIbFiojtgIfNtOIrqEFojVOr/sj+VPpX YyaClKx2MIlPcGsCnk0YyOdR9IjvVMH2jqAoQV4Sjxkly9nxhygrLIo7R6Iz+U85FQA+ JTQB6Z9OmACTWGbdEexuBvoDGTrhV4oVCOSAfk9CXvYAgAArObUWkhYrg08IG5IzIF6r GBHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B8Bax8aS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k5si3902406jad.107.2021.04.30.13.34.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:34:35 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=B8Bax8aS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZqM-0007o6-Le for patch@linaro.org; Fri, 30 Apr 2021 16:34:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51492) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiP-0008D2-Ir for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:21 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:38609) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiK-0005u5-Ey for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:21 -0400 Received: by mail-pf1-x432.google.com with SMTP id j6so9884341pfh.5 for ; Fri, 30 Apr 2021 13:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=B8Bax8aSctJN8iA1afU++eZrRQFfbdo6Emh7+jxj31ODaW3vpE2QPnvp5JM8wNCh3A UpSS5Q8pDTS8HZ/XFxHiZAr7QiVjXwU0hIJna+oyKCj4JSOveqbcmNRr/3UaOkv5XRna e7GP/ArGhnnZnegqAH/0zYYWnmn0vztm8KvusF0+4DMWejZMmXqHlFy5LNGfr/yDyxwG BDeIZwo1hK0TDQMzenwxfMRrG3q8yQ+8gb9BGRPoMvvM3KQPLBYd6A+RsaUUscHWvWf2 ZxIb57WcJRftHDMes5a6jZ21CBUxPpjhf7ovOZ8ogRs+LDOtOeXU3uj80ZkTjxZKAEPq 7WNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VTuzzLDpebdoMraQ+WOsKsSEBj+Qw3Yc0aepQpiGbJo=; b=lLM3pJhcsJ+fgdl1H+j/SDKUNhGjPBPn2sHZP3+mAog7+yUw1MEHg/HpXNiDPQS9ox hhdBN2338VTPeHjM/3RJDtoQcNw2k0WrhTFPp3KslMbyB9QRcJdT+v11dTKKC8bH1dv9 5fLyxeIZiVYY+04HEu3h2cZHKrWU0TBKi7zIWP2yiKgxbjPvJV6d92u6udMmToy0z6/B 5V3c5fEArZn+eIzXMIFNsyWZ8QR2rg+a+yCRHG1LfmhnHw14zg/taP18/faf5CmqaFlI NEDke4Jxj5RzsuVql94Ynqcyl86atvkc6mx9K3huQHlHRqIhjRTgX/DjPv1CCRmWUNsR gYWw== X-Gm-Message-State: AOAM533Y21cYRgodZebPtqSN6h5fZV1HyyaLBihVRYHy2/Z+JR1NAmdD A0VyYuiqBdPADULcv5VAgguhmos51mI64Q== X-Received: by 2002:aa7:8512:0:b029:25a:b502:e1ad with SMTP id v18-20020aa785120000b029025ab502e1admr6678836pfn.22.1619814375111; Fri, 30 Apr 2021 13:26:15 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:14 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 06/82] target/arm: Implement SVE2 saturating/rounding bitwise shift left (predicated) Date: Fri, 30 Apr 2021 13:24:54 -0700 Message-Id: <20210430202610.1136687-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Shift values are always signed (laurent desnogues). --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++ target/arm/sve.decode | 17 +++++++++ target/arm/sve_helper.c | 78 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 18 +++++++++ 4 files changed, 167 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 9992e93e2b..62106c74be 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -172,6 +172,60 @@ DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uadalp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5ba542969b..93f2479693 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1112,3 +1112,20 @@ URECPE 01000100 .. 000 000 101 ... ..... ..... @rd_pg_rn URSQRTE 01000100 .. 000 001 101 ... ..... ..... @rd_pg_rn SQABS 01000100 .. 001 000 101 ... ..... ..... @rd_pg_rn SQNEG 01000100 .. 001 001 101 ... ..... ..... @rd_pg_rn + +### SVE2 saturating/rounding bitwise shift left (predicated) + +SRSHL 01000100 .. 000 010 100 ... ..... ..... @rdn_pg_rm +URSHL 01000100 .. 000 011 100 ... ..... ..... @rdn_pg_rm +SRSHL 01000100 .. 000 110 100 ... ..... ..... @rdm_pg_rn # SRSHLR +URSHL 01000100 .. 000 111 100 ... ..... ..... @rdm_pg_rn # URSHLR + +SQSHL 01000100 .. 001 000 100 ... ..... ..... @rdn_pg_rm +UQSHL 01000100 .. 001 001 100 ... ..... ..... @rdn_pg_rm +SQSHL 01000100 .. 001 100 100 ... ..... ..... @rdm_pg_rn # SQSHLR +UQSHL 01000100 .. 001 101 100 ... ..... ..... @rdm_pg_rn # UQSHLR + +SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm +UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm +SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR +UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index bbab84e81d..7eff204c3b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -26,6 +26,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "tcg/tcg.h" +#include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -561,6 +562,83 @@ DO_ZPZZ(sve2_uadalp_zpzz_h, uint16_t, H1_2, do_uadalp_h) DO_ZPZZ(sve2_uadalp_zpzz_s, uint32_t, H1_4, do_uadalp_s) DO_ZPZZ_D(sve2_uadalp_zpzz_d, uint64_t, do_uadalp_d) +#define do_srshl_b(n, m) do_sqrshl_bhs(n, m, 8, true, NULL) +#define do_srshl_h(n, m) do_sqrshl_bhs(n, m, 16, true, NULL) +#define do_srshl_s(n, m) do_sqrshl_bhs(n, m, 32, true, NULL) +#define do_srshl_d(n, m) do_sqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_srshl_zpzz_b, int8_t, H1_2, do_srshl_b) +DO_ZPZZ(sve2_srshl_zpzz_h, int16_t, H1_2, do_srshl_h) +DO_ZPZZ(sve2_srshl_zpzz_s, int32_t, H1_4, do_srshl_s) +DO_ZPZZ_D(sve2_srshl_zpzz_d, int64_t, do_srshl_d) + +#define do_urshl_b(n, m) do_uqrshl_bhs(n, (int8_t)m, 8, true, NULL) +#define do_urshl_h(n, m) do_uqrshl_bhs(n, (int16_t)m, 16, true, NULL) +#define do_urshl_s(n, m) do_uqrshl_bhs(n, m, 32, true, NULL) +#define do_urshl_d(n, m) do_uqrshl_d(n, m, true, NULL) + +DO_ZPZZ(sve2_urshl_zpzz_b, uint8_t, H1_2, do_urshl_b) +DO_ZPZZ(sve2_urshl_zpzz_h, uint16_t, H1_2, do_urshl_h) +DO_ZPZZ(sve2_urshl_zpzz_s, uint32_t, H1_4, do_urshl_s) +DO_ZPZZ_D(sve2_urshl_zpzz_d, uint64_t, do_urshl_d) + +/* Unlike the NEON and AdvSIMD versions, there is no QC bit to set. */ +#define do_sqshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, false, &discard); }) +#define do_sqshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, false, &discard); }) +#define do_sqshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, false, &discard); }) +#define do_sqshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_sqshl_zpzz_b, int8_t, H1_2, do_sqshl_b) +DO_ZPZZ(sve2_sqshl_zpzz_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZZ(sve2_sqshl_zpzz_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZZ_D(sve2_sqshl_zpzz_d, int64_t, do_sqshl_d) + +#define do_uqshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_uqshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_uqshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, false, &discard); }) +#define do_uqshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, false, &discard); }) + +DO_ZPZZ(sve2_uqshl_zpzz_b, uint8_t, H1_2, do_uqshl_b) +DO_ZPZZ(sve2_uqshl_zpzz_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZZ(sve2_uqshl_zpzz_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZZ_D(sve2_uqshl_zpzz_d, uint64_t, do_uqshl_d) + +#define do_sqrshl_b(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 8, true, &discard); }) +#define do_sqrshl_h(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 16, true, &discard); }) +#define do_sqrshl_s(n, m) \ + ({ uint32_t discard; do_sqrshl_bhs(n, m, 32, true, &discard); }) +#define do_sqrshl_d(n, m) \ + ({ uint32_t discard; do_sqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_sqrshl_zpzz_b, int8_t, H1_2, do_sqrshl_b) +DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) +DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) +DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) + +#define do_uqrshl_b(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) +#define do_uqrshl_h(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, (int16_t)m, 16, true, &discard); }) +#define do_uqrshl_s(n, m) \ + ({ uint32_t discard; do_uqrshl_bhs(n, m, 32, true, &discard); }) +#define do_uqrshl_d(n, m) \ + ({ uint32_t discard; do_uqrshl_d(n, m, true, &discard); }) + +DO_ZPZZ(sve2_uqrshl_zpzz_b, uint8_t, H1_2, do_uqrshl_b) +DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) +DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) +DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c30b3c476e..6c1561d897 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5931,3 +5931,21 @@ static bool trans_SQNEG(DisasContext *s, arg_rpr_esz *a) }; return do_sve2_zpz_ool(s, a, fns[a->esz]); } + +#define DO_SVE2_ZPZZ(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4 * const fns[4] = { \ + gen_helper_sve2_##name##_zpzz_b, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d, \ + }; \ + return do_sve2_zpzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ(SQSHL, sqshl) +DO_SVE2_ZPZZ(SQRSHL, sqrshl) +DO_SVE2_ZPZZ(SRSHL, srshl) + +DO_SVE2_ZPZZ(UQSHL, uqshl) +DO_SVE2_ZPZZ(UQRSHL, uqrshl) +DO_SVE2_ZPZZ(URSHL, urshl) From patchwork Fri Apr 30 20:24:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429843 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp774729jao; Fri, 30 Apr 2021 13:35:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyIMnvH/Tm7MWJ0+yG5wvwrfW1pV/txR34RDhZVemD5SK5sl6Lk+iif9gZ4l8s494ipeE0H X-Received: by 2002:a5e:930d:: with SMTP id k13mr5288902iom.61.1619814913416; Fri, 30 Apr 2021 13:35:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814913; cv=none; d=google.com; s=arc-20160816; b=bLnJIz8zzxzffSbJxtoFo/zY7sflsAfTGoJarT6G0uLU68LfGcYS95Zm+1hopuYiBO iYl8ZXWi5rl+s+sefH+UpVoDBBkA1aCW+38J6+01qBlUlhqAMgzYZ3FycJurPUSuNeAO T1Iit7JVeNaT/VAGp/Ct1aKxuz8+hn8KO8dmXbtLtNJ4KI22bIlxaaMyK77W/NpbpsOM LXVWUGgDnKqCrQk1dvku0kdGCAF91Z8D1gqy9yA9NUFFqhHn7h2nVhCI3gMVqbtz90oz R5T9cpedyF/WMVTsqjIl/KAcMl5wMQZOE45jmTwQmA7+MuAOb6VRjsXoJjFxTpdnRBOW CAjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=Uzv90vRqVGPa19XR1D+u1fS7FuIg4DFEQsRtPJzteZHPe6BgET7jiWjkPaLM2vxLWJ IaP99C/6Ef3X27YRpsjQy7/26xTTxUYAiEXBAW0VA3Rofr5A3uOh66Dokn9wjuX06cwc clI/kom57Lkc/j3kbG2EfnqXzDTW0KZhMPR8ZYEw7LStZjdYOfTIrDDZtcjlW7ZAHAbe BhcPTehWdxrAObxr3K9Nlv6u161JN5NqQXRyHfYmLo2qw/WMp4T7eHCfK2yQRDZVUcHa MQe/saql+hBB20bqKqHHAyz3Bi/GkkEx8kWVyAY7q59F4nLWQ8IRH6cE/zxn6XG6IxQf ZNgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AkAo7J7V; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o24si3551803jah.59.2021.04.30.13.35.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:35:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AkAo7J7V; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46734 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZqy-0000nJ-GK for patch@linaro.org; Fri, 30 Apr 2021 16:35:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51546) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiQ-0008G5-Qx for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:22 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:36499) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiL-0005wD-0k for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:22 -0400 Received: by mail-pj1-x1029.google.com with SMTP id f2-20020a17090a4a82b02900c67bf8dc69so2363597pjh.1 for ; Fri, 30 Apr 2021 13:26:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=AkAo7J7VLDa9cbE2lZulVNBdrMvYaz1gCtFQkqoYhIEryX/Pqrx9jzquXJ5e+QRp5L p7B/GSeBRDkiYtnLxOSkSRB7GhOGcH0kuKaqG5gL58VOExui3FE7fyG0Z3Dz2inkukGU sB06QJQ2UH0ccZCDHXw+GvM7+UFZDLUYdbgEbugeUR+wG+I01WNTkTP8NuQdcWLEu6A1 x7DyxyRZ7spR4c5k2rRpte2kaCDUe0fHJybutzyTD0xHwU+yKdYo2y4rC+goaWQH/N+9 xZq9l6bbV3dA3n0WkkEvamL2J87Rd/58G3HTBA20B+dknb25OudwQsUN/ZuEtU51cmQx Xaxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9fGS2Fo4Z7uDA+2XxvFopAIhWZ1dSlmsnRxCQI2Tn/s=; b=XOUw+vdKUW27dXmOrF0VslrCKq6vM14kYPwb4Ed2bTHzeRHIBnwI0qD+xSNoDukIpn gBwRox/mjuBcvJvwL+4jTzRXlc7XMRwbTBEXwMvTy4F0wibKOEJnyj5Aqm2h1Kbb8ODp JjAx95Eou8OYpJDaNFzMxlY71u9xt2xpQugvXnCo7wVU2vM/eXcb18jlUIacBiJ1sjEA 4MYi+sVyCZwOyoR087qN352LsZ0e6pi5D6bJB9OdMtcJtN4bZ7jtSRM/5ruC6/+5XdEy 0zrAVziGdI5hGVi3HqabyZMF0I7pFUOSxsskqKMd1xKMnsKjkoeUlkP0L1e1AfMzfozL MGWw== X-Gm-Message-State: AOAM53289nV4UEkqp+xL9FCHwzYcv04yFvGXc6/wQrGa4LcvfOHvBdNO 1rqwpT5wg3e88Y7lHmI1zwTR8wARN1gyMg== X-Received: by 2002:a17:90b:294:: with SMTP id az20mr17142766pjb.229.1619814375598; Fri, 30 Apr 2021 13:26:15 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:15 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 07/82] target/arm: Implement SVE2 integer halving add/subtract (predicated) Date: Fri, 30 Apr 2021 13:24:55 -0700 Message-Id: <20210430202610.1136687-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 11 ++++++++ target/arm/sve_helper.c | 39 +++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++ 4 files changed, 112 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 62106c74be..5fdc0d223a 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -226,6 +226,60 @@ DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uqrshl_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_srhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_urhadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_shsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uhsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_sdiv_zpzz_d, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 93f2479693..58c3f7ede4 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1129,3 +1129,14 @@ SQRSHL 01000100 .. 001 010 100 ... ..... ..... @rdn_pg_rm UQRSHL 01000100 .. 001 011 100 ... ..... ..... @rdn_pg_rm SQRSHL 01000100 .. 001 110 100 ... ..... ..... @rdm_pg_rn # SQRSHLR UQRSHL 01000100 .. 001 111 100 ... ..... ..... @rdm_pg_rn # UQRSHLR + +### SVE2 integer halving add/subtract (predicated) + +SHADD 01000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm +UHADD 01000100 .. 010 001 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 010 100 ... ..... ..... @rdn_pg_rm +UHSUB 01000100 .. 010 011 100 ... ..... ..... @rdn_pg_rm +SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm +URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm +SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR +UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7eff204c3b..3703b96eb4 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -639,6 +639,45 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) +#define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) + +DO_ZPZZ(sve2_shadd_zpzz_b, int8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_h, int16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_shadd_zpzz_s, int32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_shadd_zpzz_d, int64_t, DO_HADD_D) + +DO_ZPZZ(sve2_uhadd_zpzz_b, uint8_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_h, uint16_t, H1_2, DO_HADD_BHS) +DO_ZPZZ(sve2_uhadd_zpzz_s, uint32_t, H1_4, DO_HADD_BHS) +DO_ZPZZ_D(sve2_uhadd_zpzz_d, uint64_t, DO_HADD_D) + +#define DO_RHADD_BHS(n, m) (((int64_t)n + m + 1) >> 1) +#define DO_RHADD_D(n, m) ((n >> 1) + (m >> 1) + ((n | m) & 1)) + +DO_ZPZZ(sve2_srhadd_zpzz_b, int8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_h, int16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_srhadd_zpzz_s, int32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_srhadd_zpzz_d, int64_t, DO_RHADD_D) + +DO_ZPZZ(sve2_urhadd_zpzz_b, uint8_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_h, uint16_t, H1_2, DO_RHADD_BHS) +DO_ZPZZ(sve2_urhadd_zpzz_s, uint32_t, H1_4, DO_RHADD_BHS) +DO_ZPZZ_D(sve2_urhadd_zpzz_d, uint64_t, DO_RHADD_D) + +#define DO_HSUB_BHS(n, m) (((int64_t)n - m) >> 1) +#define DO_HSUB_D(n, m) ((n >> 1) - (m >> 1) - (~n & m & 1)) + +DO_ZPZZ(sve2_shsub_zpzz_b, int8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_h, int16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_shsub_zpzz_s, int32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_shsub_zpzz_d, int64_t, DO_HSUB_D) + +DO_ZPZZ(sve2_uhsub_zpzz_b, uint8_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) +DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) +DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) + #undef DO_ZPZZ #undef DO_ZPZZ_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6c1561d897..43690999ab 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5949,3 +5949,11 @@ DO_SVE2_ZPZZ(SRSHL, srshl) DO_SVE2_ZPZZ(UQSHL, uqshl) DO_SVE2_ZPZZ(UQRSHL, uqrshl) DO_SVE2_ZPZZ(URSHL, urshl) + +DO_SVE2_ZPZZ(SHADD, shadd) +DO_SVE2_ZPZZ(SRHADD, srhadd) +DO_SVE2_ZPZZ(SHSUB, shsub) + +DO_SVE2_ZPZZ(UHADD, uhadd) +DO_SVE2_ZPZZ(URHADD, urhadd) +DO_SVE2_ZPZZ(UHSUB, uhsub) From patchwork Fri Apr 30 20:24:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429840 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp773006jao; Fri, 30 Apr 2021 13:32:40 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw03OdGhJKa0HSIRVsnjADrT4BGfHKDyVGOCt3MTxCc19Zh8WyN9A7hKvGvSZXbvxEGhpGo X-Received: by 2002:a9d:1d6a:: with SMTP id m97mr5054369otm.288.1619814760017; Fri, 30 Apr 2021 13:32:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814760; cv=none; d=google.com; s=arc-20160816; b=xfhTlKdZ7tYwMCKdsZXaSlPlPh9yHUSClhgTq8I5sV1TI/kAMOK8KSWcQvTuWtQFTS WWpiTLMApDDhf20gt6fdpF1Wmwp2wo+Z7MNypGxKiRvyLQfEm1zFf7MYfVuql+T1Ft5W N2cbrS9xc7i2E+H3sM4T7a/MCGCRKA+EgyuymkId8RD9N8UKnoYnCfQ40SMsXcbu/bAh YACK5P0wNLFw8D6/25gUWQZq3KfwD/uqb+RPT5ILDfFpGuoyoS93bCIsTl66ap5Re8xO 2OPXFNwX9lMOE3hqk3qi0qKQOI1VLWW4esQforB2betBstYEw8RnpzmPuDCbQrkBUiON DRuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=KQz5u5inkuUu115SjS8UxiE0X2icNgNf3fLMBWZ/qH0DDTot6zq8WLK3oiZeQX+fMP krnbdaDv4OhyzDbxe84heQBqceaenbqtplvHI7uUMZRQH9J408Hmt9PHRSef5uohI2+7 87D78TNax5ohv28SrW9OlywEcIJmAQ33wcW2U9NfXfoK/aLVu1KCgKs7KowK7boV7Tcy Mnb2ofnu2qJzvbf3uSs7Sx/bClwOy29/I52N275hxnLn2+mxMY6WYlz7aV+9jEvrktGQ iMCJnhUuoKHmavcAXa/w3fMftQABheKymncfH09/UsXFSnyZTyjKr9Jepw7wlO80AUuA wc5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kAOKpz7I; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d26si4378982otq.305.2021.04.30.13.32.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:32:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kAOKpz7I; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36576 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZoV-0004s8-B4 for patch@linaro.org; Fri, 30 Apr 2021 16:32:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51558) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiR-0008H0-82 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:23 -0400 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]:54024) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiL-0005xV-Ie for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:22 -0400 Received: by mail-pj1-x102d.google.com with SMTP id p17so9146257pjz.3 for ; Fri, 30 Apr 2021 13:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=kAOKpz7IqZYgLuYBNmsAKQv0O34OmDTL40kRlsMmQ7Fa1geIWJwKkDO68udOFD7VW4 +G0H7h8xAHzfM2Uw8gZMx9VnGFxqbvFSWMFYbz8cnPniiOoHrP8ZStdm4mu/vDipBdPr 7wqHgxREuoSJPMb3sxvdQg5DFDnznrXX9x/OHnYrqtx0ot3SmXH8h2fL3nBP8x5k3Qp4 eGJAzhnaFLslJfG4a2kb1gXA9qbhA9bPLhHrxFbhc65jQpJYd/uK512kyINWTugY7qQL ms7obOxyMhNKrwSlzF8Wssw5AUygQ+rJyTQK1AdQTPPkTWK6gXiOO3DgwRtbqOrxIWbV TqDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=/Nra4WlEPuo4eXKLzm+7jMvvLTFMwTtBD/ofu/WPZv8=; b=Xype+ZN6P14/7JZ1MY5BjQA8yNPukKQIYDYUQdU9+IQ9O/RSaGp6J79xUs17mXutRl kA5ybUAVqXWLkcgXti4zwFIsOeI/BpvYowb3nQSZ/2Om3Gk4MHDsEdRmWhNGpPAGHmnk ds90s4x5wMQLyFO5IKIw/MzjNahWWN5CkbYDNvLb8byt70azp8tsKPKcHtQ6AXtREh5l FYYGia29XbJ9LfJ4+HiFqr6hXckYNBctl001jR1eBCCj/AvF8HRhwTM0cgKgtpelAVZt +UO73YS8aRUSFynkOEewdQ0XqCLsPZDaZ65R6WRvH8+OEo/N8AXRC6BQS5ZRyzYuXswU Y45A== X-Gm-Message-State: AOAM530vARIGo2E9moogoQWfe2GihSTjbg2TxSvqFAC0TtATuWrCBbGN OHHET52Nu5yDJk9CKN4S1+gmSMTrf8zJGA== X-Received: by 2002:a17:90b:1bcd:: with SMTP id oa13mr15773368pjb.22.1619814376242; Fri, 30 Apr 2021 13:26:16 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:16 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 08/82] target/arm: Implement SVE2 integer pairwise arithmetic Date: Fri, 30 Apr 2021 13:24:56 -0700 Message-Id: <20210430202610.1136687-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 45 ++++++++++++++++++++++ target/arm/sve.decode | 8 ++++ target/arm/sve_helper.c | 76 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 6 +++ 4 files changed, 135 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5fdc0d223a..09bc067dd4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -326,6 +326,51 @@ DEF_HELPER_FLAGS_5(sve_sel_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve_sel_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_addp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 58c3f7ede4..61a3321325 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1140,3 +1140,11 @@ SRHADD 01000100 .. 010 100 100 ... ..... ..... @rdn_pg_rm URHADD 01000100 .. 010 101 100 ... ..... ..... @rdn_pg_rm SHSUB 01000100 .. 010 110 100 ... ..... ..... @rdm_pg_rn # SHSUBR UHSUB 01000100 .. 010 111 100 ... ..... ..... @rdm_pg_rn # UHSUBR + +### SVE2 integer pairwise arithmetic + +ADDP 01000100 .. 010 001 101 ... ..... ..... @rdn_pg_rm +SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm +UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm +SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm +UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3703b96eb4..7cc559d950 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -681,6 +681,82 @@ DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) #undef DO_ZPZZ #undef DO_ZPZZ_D +/* + * Three operand expander, operating on element pairs. + * If the slot I is even, the elements from from VN {I, I+1}. + * If the slot I is odd, the elements from from VM {I-1, I}. + * Load all of the input elements in each pair before overwriting output. + */ +#define DO_ZPZZ_PAIR(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +/* Similarly, specialized for 64-bit operands. */ +#define DO_ZPZZ_PAIR_D(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; \ + TYPE *d = vd, *n = vn, *m = vm; \ + uint8_t *pg = vg; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE n0 = n[i], n1 = n[i + 1]; \ + TYPE m0 = m[i], m1 = m[i + 1]; \ + if (pg[H1(i)] & 1) { \ + d[i] = OP(n0, n1); \ + } \ + if (pg[H1(i + 1)] & 1) { \ + d[i + 1] = OP(m0, m1); \ + } \ + } \ +} + +DO_ZPZZ_PAIR(sve2_addp_zpzz_b, uint8_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_h, uint16_t, H1_2, DO_ADD) +DO_ZPZZ_PAIR(sve2_addp_zpzz_s, uint32_t, H1_4, DO_ADD) +DO_ZPZZ_PAIR_D(sve2_addp_zpzz_d, uint64_t, DO_ADD) + +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_b, uint8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_h, uint16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_umaxp_zpzz_s, uint32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_umaxp_zpzz_d, uint64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_uminp_zpzz_b, uint8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_h, uint16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_uminp_zpzz_s, uint32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_uminp_zpzz_d, uint64_t, DO_MIN) + +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_b, int8_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_h, int16_t, H1_2, DO_MAX) +DO_ZPZZ_PAIR(sve2_smaxp_zpzz_s, int32_t, H1_4, DO_MAX) +DO_ZPZZ_PAIR_D(sve2_smaxp_zpzz_d, int64_t, DO_MAX) + +DO_ZPZZ_PAIR(sve2_sminp_zpzz_b, int8_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_h, int16_t, H1_2, DO_MIN) +DO_ZPZZ_PAIR(sve2_sminp_zpzz_s, int32_t, H1_4, DO_MIN) +DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) + +#undef DO_ZPZZ_PAIR +#undef DO_ZPZZ_PAIR_D + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 43690999ab..2d449c9b57 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5957,3 +5957,9 @@ DO_SVE2_ZPZZ(SHSUB, shsub) DO_SVE2_ZPZZ(UHADD, uhadd) DO_SVE2_ZPZZ(URHADD, urhadd) DO_SVE2_ZPZZ(UHSUB, uhsub) + +DO_SVE2_ZPZZ(ADDP, addp) +DO_SVE2_ZPZZ(SMAXP, smaxp) +DO_SVE2_ZPZZ(UMAXP, umaxp) +DO_SVE2_ZPZZ(SMINP, sminp) +DO_SVE2_ZPZZ(UMINP, uminp) From patchwork Fri Apr 30 20:24:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429848 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp776817jao; Fri, 30 Apr 2021 13:38:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwlMVtdZYjl+DIJeto7mmSTGbST9cByoGbwdPpie90HeVoDwvHyXMAR41wpBMeduqZZWIIe X-Received: by 2002:a92:c548:: with SMTP id a8mr5410542ilj.137.1619815101758; Fri, 30 Apr 2021 13:38:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815101; cv=none; d=google.com; s=arc-20160816; b=juNgU7VzSxnvpyX7ORmaRaJcToNfGcavWvgYXNd4NhTwqcolDIc9DqVOwVV3UbaA6t h/tM3hW66G7cSG8ysVrD38heMSZXXmBBUHie9A6iXBrTRn6cYqkTEQofvw8lY0j9p8Rv UAOQ4NbBoW6Fq2cJTK4ZMItlXjeaczzgor8LSEvtif4lfugczCExEvErIM0Mjw6XvHz2 9GmTMHKWx1MMjQghAw9Jdp2o2yo4+1ezQEKXwNjM77kVG/boe6rgGzZLaXJnPJf7hGP8 le34AJ1cK+PN/0tqaAfMFJVaj2y8Kre2aYLTy4v8H/XZ1TLFR2gA+E9kJmewqHU21SCZ cixw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=tr/9A9+gaKPW2tZzd2jvYQhW+eEVnk864f0PDxi7pUxkTj/4ZokKjMhs0uikBunmD9 ASC5wf472Hc6CFBxKLVkcZVtA90efTMXqcCO6M2zWSTNlVX1O0GLP6QGfldZiVNEQv7k YyGTpzcOI+gZ9PS2cVwoOLa+ukCUyeKV5GX0qKKlDGouWRhals4+2b0iSjQFPXg2I6Al TVnP5afJ/zrJPDOjCFX/OY9OeMkDU0xJSY60QIaPa97gyqoms1Qfl7DRo10Udc+1oniR a35TgQkRbvpjMgQ0w1Km+fgjGSVlrOXcohRIM1lAUHGxYRLhaBGaXgy+ps0+0UEwUuMD JtgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vMO3T6RQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a5si3633482ilv.74.2021.04.30.13.38.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:38:21 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vMO3T6RQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55462 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZu1-0004Rm-3I for patch@linaro.org; Fri, 30 Apr 2021 16:38:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiT-0008LD-10 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:25 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:37835) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiM-0005y6-EN for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:24 -0400 Received: by mail-pf1-x42d.google.com with SMTP id b15so5008655pfl.4 for ; Fri, 30 Apr 2021 13:26:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=vMO3T6RQFmb3d6g+dW0eXD4J+ZePZ6nM7eSwe5P6CuRKdoNYeL72k4mRGp+Uirol8O ecgdXqWvSOQ3RjltSCAUkAxAF9KvFXEVeahYLGk/oLwZcwcey2pU54RIBpdN3S+Q6Xt1 0AeJsWN4p6d+QZcBT8tzY5swx2btEmE3UV5tWk+5GFjlYhtm9MossS3y+LJrPyL3z4V5 0wTPLM5hzzs28gJlidLTx1B8NMLqZ15xyB9qGLOYnrjHEvoxoU44T3ZTgs+KXSoa+qzO BjBdZaEWbzC0s1K6cPctqmS2xeZAQyehqY7eZtP+SUw/fwPgEkktvdEkYKYuWKkNVSSn UsuQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nkSSxMkXeoKQp4t8tpIye6M1xfm+ZviLl0PQ26Xf9wU=; b=B7sA6EQDkkX+fwCwqhCL+Er6HVR3mQZbEvj53hlgYVVlM3qSWYUN/76ac74C19DS1d oeiZ+XDTJtXRM7t+LlYRhnYaVQ4JsjOgbmTjPk50tYP/Ygm5PoCbNrzlq4+W8R5N8QMq NMYGVfepgqacyOM02UXSCq7hgP8r3/LilPDmVU5NXgQ/7VA7jeTSxJna5OtQpstxMCbK qAIoyZAMdhTiRbkzwKivDgXickgAk0Li+cDLfU9vLxzNLOiCIxKPM1bu5UeyoQ9y5pw4 E+GWMdQDuhknm9pteYclpEQffxppqpb7tn7nihwbeWAFofZpB/O0B1TIM57EilFx7PyN T9XQ== X-Gm-Message-State: AOAM532NepHHdq6sSPSqseGU8cnZ01HBgePYeSdt8tegxHaEqOiYo/EG CY2Em5yzTRynZXBLn5OhZ0qnInicINLAqQ== X-Received: by 2002:a63:1f53:: with SMTP id q19mr6073020pgm.421.1619814376803; Fri, 30 Apr 2021 13:26:16 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:16 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 09/82] target/arm: Implement SVE2 saturating add/subtract (predicated) Date: Fri, 30 Apr 2021 13:24:57 -0700 Message-Id: <20210430202610.1136687-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 54 +++++++++++ target/arm/sve.decode | 11 +++ target/arm/sve_helper.c | 194 ++++++++++++++++++++++++++----------- target/arm/translate-sve.c | 7 ++ 4 files changed, 210 insertions(+), 56 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 09bc067dd4..37461c9927 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -371,6 +371,60 @@ DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_uminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uqsub_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_suqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_usqadd_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve_asr_zpzw_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_asr_zpzw_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 61a3321325..cd4f73265f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1148,3 +1148,14 @@ SMAXP 01000100 .. 010 100 101 ... ..... ..... @rdn_pg_rm UMAXP 01000100 .. 010 101 101 ... ..... ..... @rdn_pg_rm SMINP 01000100 .. 010 110 101 ... ..... ..... @rdn_pg_rm UMINP 01000100 .. 010 111 101 ... ..... ..... @rdn_pg_rm + +### SVE2 saturating add/subtract (predicated) + +SQADD_zpzz 01000100 .. 011 000 100 ... ..... ..... @rdn_pg_rm +UQADD_zpzz 01000100 .. 011 001 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 010 100 ... ..... ..... @rdn_pg_rm +UQSUB_zpzz 01000100 .. 011 011 100 ... ..... ..... @rdn_pg_rm +SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm +USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm +SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR +UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7cc559d950..12a2078edb 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -678,6 +678,135 @@ DO_ZPZZ(sve2_uhsub_zpzz_h, uint16_t, H1_2, DO_HSUB_BHS) DO_ZPZZ(sve2_uhsub_zpzz_s, uint32_t, H1_4, DO_HSUB_BHS) DO_ZPZZ_D(sve2_uhsub_zpzz_d, uint64_t, DO_HSUB_D) +static inline int32_t do_sat_bhs(int64_t val, int64_t min, int64_t max) +{ + return val >= max ? max : val <= min ? min : val; +} + +#define DO_SQADD_B(n, m) do_sat_bhs((int64_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SQADD_H(n, m) do_sat_bhs((int64_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SQADD_S(n, m) do_sat_bhs((int64_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqadd_d(int64_t n, int64_t m) +{ + int64_t r = n + m; + if (((r ^ n) & ~(n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqadd_zpzz_b, int8_t, H1_2, DO_SQADD_B) +DO_ZPZZ(sve2_sqadd_zpzz_h, int16_t, H1_2, DO_SQADD_H) +DO_ZPZZ(sve2_sqadd_zpzz_s, int32_t, H1_4, DO_SQADD_S) +DO_ZPZZ_D(sve2_sqadd_zpzz_d, int64_t, do_sqadd_d) + +#define DO_UQADD_B(n, m) do_sat_bhs((int64_t)n + m, 0, UINT8_MAX) +#define DO_UQADD_H(n, m) do_sat_bhs((int64_t)n + m, 0, UINT16_MAX) +#define DO_UQADD_S(n, m) do_sat_bhs((int64_t)n + m, 0, UINT32_MAX) + +static inline uint64_t do_uqadd_d(uint64_t n, uint64_t m) +{ + uint64_t r = n + m; + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_uqadd_zpzz_b, uint8_t, H1_2, DO_UQADD_B) +DO_ZPZZ(sve2_uqadd_zpzz_h, uint16_t, H1_2, DO_UQADD_H) +DO_ZPZZ(sve2_uqadd_zpzz_s, uint32_t, H1_4, DO_UQADD_S) +DO_ZPZZ_D(sve2_uqadd_zpzz_d, uint64_t, do_uqadd_d) + +#define DO_SQSUB_B(n, m) do_sat_bhs((int64_t)n - m, INT8_MIN, INT8_MAX) +#define DO_SQSUB_H(n, m) do_sat_bhs((int64_t)n - m, INT16_MIN, INT16_MAX) +#define DO_SQSUB_S(n, m) do_sat_bhs((int64_t)n - m, INT32_MIN, INT32_MAX) + +static inline int64_t do_sqsub_d(int64_t n, int64_t m) +{ + int64_t r = n - m; + if (((r ^ n) & (n ^ m)) < 0) { + /* Signed overflow. */ + return r < 0 ? INT64_MAX : INT64_MIN; + } + return r; +} + +DO_ZPZZ(sve2_sqsub_zpzz_b, int8_t, H1_2, DO_SQSUB_B) +DO_ZPZZ(sve2_sqsub_zpzz_h, int16_t, H1_2, DO_SQSUB_H) +DO_ZPZZ(sve2_sqsub_zpzz_s, int32_t, H1_4, DO_SQSUB_S) +DO_ZPZZ_D(sve2_sqsub_zpzz_d, int64_t, do_sqsub_d) + +#define DO_UQSUB_B(n, m) do_sat_bhs((int64_t)n - m, 0, UINT8_MAX) +#define DO_UQSUB_H(n, m) do_sat_bhs((int64_t)n - m, 0, UINT16_MAX) +#define DO_UQSUB_S(n, m) do_sat_bhs((int64_t)n - m, 0, UINT32_MAX) + +static inline uint64_t do_uqsub_d(uint64_t n, uint64_t m) +{ + return n > m ? n - m : 0; +} + +DO_ZPZZ(sve2_uqsub_zpzz_b, uint8_t, H1_2, DO_UQSUB_B) +DO_ZPZZ(sve2_uqsub_zpzz_h, uint16_t, H1_2, DO_UQSUB_H) +DO_ZPZZ(sve2_uqsub_zpzz_s, uint32_t, H1_4, DO_UQSUB_S) +DO_ZPZZ_D(sve2_uqsub_zpzz_d, uint64_t, do_uqsub_d) + +#define DO_SUQADD_B(n, m) \ + do_sat_bhs((int64_t)(int8_t)n + m, INT8_MIN, INT8_MAX) +#define DO_SUQADD_H(n, m) \ + do_sat_bhs((int64_t)(int16_t)n + m, INT16_MIN, INT16_MAX) +#define DO_SUQADD_S(n, m) \ + do_sat_bhs((int64_t)(int32_t)n + m, INT32_MIN, INT32_MAX) + +static inline int64_t do_suqadd_d(int64_t n, uint64_t m) +{ + uint64_t r = n + m; + + if (n < 0) { + /* Note that m - abs(n) cannot underflow. */ + if (r > INT64_MAX) { + /* Result is either very large positive or negative. */ + if (m > -n) { + /* m > abs(n), so r is a very large positive. */ + return INT64_MAX; + } + /* Result is negative. */ + } + } else { + /* Both inputs are positive: check for overflow. */ + if (r < m || r > INT64_MAX) { + return INT64_MAX; + } + } + return r; +} + +DO_ZPZZ(sve2_suqadd_zpzz_b, uint8_t, H1_2, DO_SUQADD_B) +DO_ZPZZ(sve2_suqadd_zpzz_h, uint16_t, H1_2, DO_SUQADD_H) +DO_ZPZZ(sve2_suqadd_zpzz_s, uint32_t, H1_4, DO_SUQADD_S) +DO_ZPZZ_D(sve2_suqadd_zpzz_d, uint64_t, do_suqadd_d) + +#define DO_USQADD_B(n, m) \ + do_sat_bhs((int64_t)n + (int8_t)m, 0, UINT8_MAX) +#define DO_USQADD_H(n, m) \ + do_sat_bhs((int64_t)n + (int16_t)m, 0, UINT16_MAX) +#define DO_USQADD_S(n, m) \ + do_sat_bhs((int64_t)n + (int32_t)m, 0, UINT32_MAX) + +static inline uint64_t do_usqadd_d(uint64_t n, int64_t m) +{ + uint64_t r = n + m; + + if (m < 0) { + return n < -m ? 0 : r; + } + return r < n ? UINT64_MAX : r; +} + +DO_ZPZZ(sve2_usqadd_zpzz_b, uint8_t, H1_2, DO_USQADD_B) +DO_ZPZZ(sve2_usqadd_zpzz_h, uint16_t, H1_2, DO_USQADD_H) +DO_ZPZZ(sve2_usqadd_zpzz_s, uint32_t, H1_4, DO_USQADD_S) +DO_ZPZZ_D(sve2_usqadd_zpzz_d, uint64_t, do_usqadd_d) + #undef DO_ZPZZ #undef DO_ZPZZ_D @@ -1614,13 +1743,7 @@ void HELPER(sve_sqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int8_t)) { - int r = *(int8_t *)(a + i) + b; - if (r > INT8_MAX) { - r = INT8_MAX; - } else if (r < INT8_MIN) { - r = INT8_MIN; - } - *(int8_t *)(d + i) = r; + *(int8_t *)(d + i) = DO_SQADD_B(b, *(int8_t *)(a + i)); } } @@ -1629,13 +1752,7 @@ void HELPER(sve_sqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int16_t)) { - int r = *(int16_t *)(a + i) + b; - if (r > INT16_MAX) { - r = INT16_MAX; - } else if (r < INT16_MIN) { - r = INT16_MIN; - } - *(int16_t *)(d + i) = r; + *(int16_t *)(d + i) = DO_SQADD_H(b, *(int16_t *)(a + i)); } } @@ -1644,13 +1761,7 @@ void HELPER(sve_sqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int32_t)) { - int64_t r = *(int32_t *)(a + i) + b; - if (r > INT32_MAX) { - r = INT32_MAX; - } else if (r < INT32_MIN) { - r = INT32_MIN; - } - *(int32_t *)(d + i) = r; + *(int32_t *)(d + i) = DO_SQADD_S(b, *(int32_t *)(a + i)); } } @@ -1659,13 +1770,7 @@ void HELPER(sve_sqaddi_d)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(int64_t)) { - int64_t ai = *(int64_t *)(a + i); - int64_t r = ai + b; - if (((r ^ ai) & ~(ai ^ b)) < 0) { - /* Signed overflow. */ - r = (r < 0 ? INT64_MAX : INT64_MIN); - } - *(int64_t *)(d + i) = r; + *(int64_t *)(d + i) = do_sqadd_d(b, *(int64_t *)(a + i)); } } @@ -1678,13 +1783,7 @@ void HELPER(sve_uqaddi_b)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint8_t)) { - int r = *(uint8_t *)(a + i) + b; - if (r > UINT8_MAX) { - r = UINT8_MAX; - } else if (r < 0) { - r = 0; - } - *(uint8_t *)(d + i) = r; + *(uint8_t *)(d + i) = DO_UQADD_B(b, *(uint8_t *)(a + i)); } } @@ -1693,13 +1792,7 @@ void HELPER(sve_uqaddi_h)(void *d, void *a, int32_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint16_t)) { - int r = *(uint16_t *)(a + i) + b; - if (r > UINT16_MAX) { - r = UINT16_MAX; - } else if (r < 0) { - r = 0; - } - *(uint16_t *)(d + i) = r; + *(uint16_t *)(d + i) = DO_UQADD_H(b, *(uint16_t *)(a + i)); } } @@ -1708,13 +1801,7 @@ void HELPER(sve_uqaddi_s)(void *d, void *a, int64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint32_t)) { - int64_t r = *(uint32_t *)(a + i) + b; - if (r > UINT32_MAX) { - r = UINT32_MAX; - } else if (r < 0) { - r = 0; - } - *(uint32_t *)(d + i) = r; + *(uint32_t *)(d + i) = DO_UQADD_S(b, *(uint32_t *)(a + i)); } } @@ -1723,11 +1810,7 @@ void HELPER(sve_uqaddi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t r = *(uint64_t *)(a + i) + b; - if (r < b) { - r = UINT64_MAX; - } - *(uint64_t *)(d + i) = r; + *(uint64_t *)(d + i) = do_uqadd_d(b, *(uint64_t *)(a + i)); } } @@ -1736,8 +1819,7 @@ void HELPER(sve_uqsubi_d)(void *d, void *a, uint64_t b, uint32_t desc) intptr_t i, oprsz = simd_oprsz(desc); for (i = 0; i < oprsz; i += sizeof(uint64_t)) { - uint64_t ai = *(uint64_t *)(a + i); - *(uint64_t *)(d + i) = (ai < b ? 0 : ai - b); + *(uint64_t *)(d + i) = do_uqsub_d(*(uint64_t *)(a + i), b); } } diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 2d449c9b57..609d5ae7b7 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5963,3 +5963,10 @@ DO_SVE2_ZPZZ(SMAXP, smaxp) DO_SVE2_ZPZZ(UMAXP, umaxp) DO_SVE2_ZPZZ(SMINP, sminp) DO_SVE2_ZPZZ(UMINP, uminp) + +DO_SVE2_ZPZZ(SQADD_zpzz, sqadd) +DO_SVE2_ZPZZ(UQADD_zpzz, uqadd) +DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) +DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) +DO_SVE2_ZPZZ(SUQADD, suqadd) +DO_SVE2_ZPZZ(USQADD, usqadd) From patchwork Fri Apr 30 20:24:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429841 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp774278jao; Fri, 30 Apr 2021 13:34:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxqJHWdZHAYsCWIfZ9tcOccsjEnirMjroBFBI60CQqxJEiHwwmobNApUgPrkw4Ld5hM/gTT X-Received: by 2002:a05:6e02:d41:: with SMTP id h1mr5426102ilj.0.1619814868977; Fri, 30 Apr 2021 13:34:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814868; cv=none; d=google.com; s=arc-20160816; b=PZOjUHPonUnoDWIi7qMNQhOs3YQzhq7b1FiP13YvFlL2eHXWHmHgsVeDR7CzVfMjxY YP6UyYq/3HuXPM2k8HzEnGQ6aQ8LzVRzCPkMYNaUbKfa8B3efpaLEYLPUY83cQz1/jTD gwULI7gmZuLb3bCOcnc9iPTjhhaqblid1pwxfmpQiczGXtCmkYHrXA1XMcrAMGRb0+f/ suc+JAzu8LRhXjdPOEZCVQ3K+9sBuOWmAn26N71OmohERZyxwkDvGhuPvd4Gm5/ITUkh Ahfl6+wbG531WVBm2Csd7YteaAWY1UgjLb8/rzFsu/fZOn+NJUvdfDoapc1bwJbVet2d 1Beg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=gyEul5y9FBkG5zFQhwcB0DHZkXlTL2MAvhVSAXPkiuHNTI5McIE+yJbP06xYBE96Fd gkNa4zrXT8LpxkesfV5S30ZmSYpnb1LXhNOexcha6uwljVy5kxMgeigkM+WF0OuAvr8S HAc8oPl/VWvt7HDz4LTJ5+wSKDbuWTaIIjPoxBiriGglmE2dFjygbZjAcAQFwxWajQpo yrNoDkVcTV1NSHroaOGHmSmio1XCQTHGdhzK/H08QuMV5XtmnE+g5vrtsH5hudGZ0Pi8 4t4o1tNkU8t+WLMX7uvqZ17jo5EMS4+1aQsRFWj/valM2QXjDIzJ4NYi9nlF1adDQ+F6 Mj5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i7ulQzC2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j28si1953323jaq.115.2021.04.30.13.34.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:34:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i7ulQzC2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:42592 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZqG-0007YC-AZ for patch@linaro.org; Fri, 30 Apr 2021 16:34:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51682) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiU-0008MO-OR for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:26 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:55821) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiN-0005yz-CA for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:26 -0400 Received: by mail-pj1-x1035.google.com with SMTP id gj14so5155481pjb.5 for ; Fri, 30 Apr 2021 13:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=i7ulQzC2Ed5uEEmIMBfrvejeMsaKCdbi/+MWxXLKyUUvEUga5n0/U9Ck6DSn+cny8x 6JjYoXfUx2r4trEc/C5Jkg7MgCKPDrSIU+Xg0GfnM5tNAdSYYIg8y0272+rz8qGAXEd1 IXfboBfL7xHIWRjrSgg8vjCDLDRqWdsy9Vz5fgid5sujcMkZgv8V5/VI4TaLoljVl1xo GdhybAk7FWXkTGtx3ikDhJjBPaHvGeDY5GZz9K2Xu7GWkxfdmNS1nPLXBt3AIA6nDsYc uWaSopAdPUzHQnztZWBs2+Wv8nF5DJKi0zNbR5uUuivSSejTwvvJDltyjX0S1JWNMJe6 28YA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Lt6mYuE55+/GmyazdMW6oFsb1RA74ewEaDqJsdcvUwc=; b=n/MnMZHX7fgCRR2FyCQq+mEWOmveHB83LpT9J2Ws96md4Yp3YT9p1CjTjjx+gI3c3J SjOF6JLeCOhlbxzn20xXL+N3saUSDU2HUu8W3zkCv3Odp+319vqkcqB5yM/2rEh0nsri KgD6oLqyXhi4iqZXISyItbw2kojoDTApAKLb4wAg6S6K4vKwcB8CA413PXVJ77oH2h3z SGXsYJfAhA51YOXkHFENncZI9dqETj/gb+QAIZErESenzSqdPDiGIOQVIJfgOKJeMOj7 S6rwZoidcXeV9axoz4NRWtgGKVMhKeSOzvV1EAzpUt2jlOkmy39EoRl+jBSTLUm9h5bX PZMQ== X-Gm-Message-State: AOAM530pTNIZOIk5N9yXDog3HsB4VOC37Gxw219aVjAO5wC2QM7krAlT Xhu8YpDL3K/SQcUGmdKDEpJhqTJ/WwqhJw== X-Received: by 2002:a17:90a:b105:: with SMTP id z5mr16433279pjq.187.1619814377493; Fri, 30 Apr 2021 13:26:17 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 10/82] target/arm: Implement SVE2 integer add/subtract long Date: Fri, 30 Apr 2021 13:24:58 -0700 Message-Id: <20210430202610.1136687-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 24 ++++++++++++++++++++ target/arm/sve.decode | 19 ++++++++++++++++ target/arm/sve_helper.c | 43 +++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 46 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 132 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 37461c9927..a81297b387 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1367,6 +1367,30 @@ DEF_HELPER_FLAGS_5(sve_ftmad_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve_ftmad_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cd4f73265f..fbfd57b23a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1159,3 +1159,22 @@ SUQADD 01000100 .. 011 100 100 ... ..... ..... @rdn_pg_rm USQADD 01000100 .. 011 101 100 ... ..... ..... @rdn_pg_rm SQSUB_zpzz 01000100 .. 011 110 100 ... ..... ..... @rdm_pg_rn # SQSUBR UQSUB_zpzz 01000100 .. 011 111 100 ... ..... ..... @rdm_pg_rn # UQSUBR + +#### SVE2 Widening Integer Arithmetic + +## SVE2 integer add/subtract long + +SADDLB 01000101 .. 0 ..... 00 0000 ..... ..... @rd_rn_rm +SADDLT 01000101 .. 0 ..... 00 0001 ..... ..... @rd_rn_rm +UADDLB 01000101 .. 0 ..... 00 0010 ..... ..... @rd_rn_rm +UADDLT 01000101 .. 0 ..... 00 0011 ..... ..... @rd_rn_rm + +SSUBLB 01000101 .. 0 ..... 00 0100 ..... ..... @rd_rn_rm +SSUBLT 01000101 .. 0 ..... 00 0101 ..... ..... @rd_rn_rm +USUBLB 01000101 .. 0 ..... 00 0110 ..... ..... @rd_rn_rm +USUBLT 01000101 .. 0 ..... 00 0111 ..... ..... @rd_rn_rm + +SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm +SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm +UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm +UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 12a2078edb..3d0ee76411 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1113,6 +1113,49 @@ DO_ZZW(sve_lsl_zzw_s, uint32_t, uint64_t, H1_4, DO_LSL) #undef DO_ZPZ #undef DO_ZPZ_D +/* + * Three-operand expander, unpredicated, in which the two inputs are + * selected from the top or bottom half of the wide column. + */ +#define DO_ZZZ_TB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_TB(sve2_saddl_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_saddl_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_saddl_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_ssubl_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_ssubl_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_sabdl_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_sabdl_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZ_TB(sve2_uaddl_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_TB(sve2_uaddl_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_TB(sve2_usubl_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_TB(sve2_usubl_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_TB(sve2_usubl_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZ_TB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 609d5ae7b7..22983b3b85 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5970,3 +5970,49 @@ DO_SVE2_ZPZZ(SQSUB_zpzz, sqsub) DO_SVE2_ZPZZ(UQSUB_zpzz, uqsub) DO_SVE2_ZPZZ(SUQADD, suqadd) DO_SVE2_ZPZZ(USQADD, usqadd) + +/* + * SVE2 Widening Integer Arithmetic + */ + +static bool do_sve2_zzw_ool(DisasContext *s, arg_rrr_esz *a, + gen_helper_gvec_3 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, data, fn); + } + return true; +} + +#define DO_SVE2_ZZZ_TB(NAME, name, SEL1, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], (SEL2 << 1) | SEL1); \ +} + +DO_SVE2_ZZZ_TB(SADDLB, saddl, false, false) +DO_SVE2_ZZZ_TB(SSUBLB, ssubl, false, false) +DO_SVE2_ZZZ_TB(SABDLB, sabdl, false, false) + +DO_SVE2_ZZZ_TB(UADDLB, uaddl, false, false) +DO_SVE2_ZZZ_TB(USUBLB, usubl, false, false) +DO_SVE2_ZZZ_TB(UABDLB, uabdl, false, false) + +DO_SVE2_ZZZ_TB(SADDLT, saddl, true, true) +DO_SVE2_ZZZ_TB(SSUBLT, ssubl, true, true) +DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) + +DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) +DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) +DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) From patchwork Fri Apr 30 20:24:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429846 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp776652jao; Fri, 30 Apr 2021 13:38:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyGBjn+LS6fiLrdCqsZ02HeNLzQgy9n32/XUfBwCzK5t8eqQL7vNKW9PwC51xWpRZmHz09r X-Received: by 2002:a02:a381:: with SMTP id y1mr6652433jak.1.1619815087780; Fri, 30 Apr 2021 13:38:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815087; cv=none; d=google.com; s=arc-20160816; b=TOIvAndk0Qex/1+2rHUyCCyBebT8QF0VtRiDq38A10+AnC4cbBVY/+8lpYAH2CUuqZ BD/OclEXTW0RcWKiBGfA/X6XFiUrRgf3Ip5qEnTrczZGfisAp2Q+LNE2vTeVNwqq8Zjp 4qdIffShnpQTe0uEG7BeuOawpNc4c1wDKLSccW1ynzEtwCxbAoQcOZB3igJq1tnkp6/w auqeZB6RxKxImE3jC7+eW7LA49JMTwFuIJrKqMP1BYQhHf4uksChFPvWOtQrwYjdEGr9 ORV0vAAJ2EAca68EuP16LaARr2R7cuX/rrDWzilYPhpTaguEE9Z4Ju/AjE+ZDf+Rm3T7 CxEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=VHcXq6c7pIOzUNX0kDijgYW2LGuFGyzeKsKItBjsDXc1Ny1jFujUb+CzNFBTobAbNi gR5ztV3PSHJveKyO4dkhug6eMwlhipezdB5/314fo7vfEr+r8V+QMqakmccAjRFKrVld Mal6QePvTcA1r+1knuKxOd22PHtiXE1+nZroaUkFAVGB/kK1k+hpQ3Y7uhaAocWkNhAj rGPYDXcpk05hsr6Fv06PM0ercZ5ZIK+RZQJgA5CHCRc1dfxi1deLlrTXp92aaKhvuorb Hjn1k0V5slQTTNAs1EqwF08LxsvA8N/UDzS6Huls/WV0V6sUSFrwKVY2J/MJi6Ox6oBH 0oAA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uE+DNl4Z; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q19si3213848jaj.123.2021.04.30.13.38.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:38:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uE+DNl4Z; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54358 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZtn-0003yB-6e for patch@linaro.org; Fri, 30 Apr 2021 16:38:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51760) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiY-0008QO-Oq for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:31 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]:37838) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiN-0005zb-Lz for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:30 -0400 Received: by mail-pf1-x430.google.com with SMTP id b15so5008688pfl.4 for ; Fri, 30 Apr 2021 13:26:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=uE+DNl4Z32104nNSkPou4iulwrQVxJiXMsCKSSq4XhAvmsGYzvq0Scl+AKe++orTDp 6BXlweHk7ZimFAwvBhmEC7Lu4AqbdGVjU9vnHqY721gYUyRntXpDJF1E0nF1J7XVd/kl LlVhchQOS8NUHENB+/aJUYMzQnSiBK9tipGtBO1uQ1at27nonvJse7S+ioLp64m3co8d dLQkGxQ2KPdtRViJb4WfsnyyxwEmmpq+Rj38dzN0s+4vS1KYkzZSTAd0uq0w/b/q8bAR srBMBszlz8BfyeD7AmjqeM0SHXgZ99/ngL5ZkhlhD1CHWCaa2JTMLePkbEAbT8ioOpu/ DkWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=br78Q7AmWJ+XfOQwE2rDFXsz+3JbpInND1mUnQKHlAY=; b=IVmxLKFJ1gI0VThdiy48bWDnSEP8KR9t4hXyRjnzO5FlGjhatTUz4/PqQX5ReZQPlm /5SUV4SScNDlY6iAHC+w2wEOlcQ84JgJM7Kyk20j/6ySE7m3ckek6IEPoHBbk8477Qd7 lpeDTMv+OeVvUfbI6xspbpTLJooEZG4+ToJQ3/13n1JILIHKYMtee2hbtEFhg5YY/+oI QqYQRPjtUE4eCXjFVgCw2zzHf+ffukC5emXw+SrR3e//tT9CbahDxSmDyp1OD0hFR1WE i61+tVU7Tm6HxVOIR0/nZgEnBY/Vx/DKte3Eu8ikSrN5pjBIfv16nKqqkGCGy1BfEO2q mVog== X-Gm-Message-State: AOAM531HibwTnhyg3DDzIKqc5gqlfXNS/25kWO3qtelBl7bP4j9bZeqp rPCf8HzNYz1sPWTZ3Mi+SkyrvY5iXoKNPQ== X-Received: by 2002:a65:6a50:: with SMTP id o16mr6335989pgu.6.1619814378075; Fri, 30 Apr 2021 13:26:18 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:17 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 11/82] target/arm: Implement SVE2 integer add/subtract interleaved long Date: Fri, 30 Apr 2021 13:24:59 -0700 Message-Id: <20210430202610.1136687-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 4 ++++ 2 files changed, 10 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fbfd57b23a..12be0584a8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1178,3 +1178,9 @@ SABDLB 01000101 .. 0 ..... 00 1100 ..... ..... @rd_rn_rm SABDLT 01000101 .. 0 ..... 00 1101 ..... ..... @rd_rn_rm UABDLB 01000101 .. 0 ..... 00 1110 ..... ..... @rd_rn_rm UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract interleaved long + +SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm +SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm +SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 22983b3b85..ae8323adb7 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6016,3 +6016,7 @@ DO_SVE2_ZZZ_TB(SABDLT, sabdl, true, true) DO_SVE2_ZZZ_TB(UADDLT, uaddl, true, true) DO_SVE2_ZZZ_TB(USUBLT, usubl, true, true) DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) + +DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) +DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) +DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) From patchwork Fri Apr 30 20:25:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429845 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp775575jao; Fri, 30 Apr 2021 13:36:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwtXqW32eBIgzvM/+5Gb8pyzO7If38dTnL9e1jQuM5SzfXAECzlNxp8FpXEuYgf2lZRisl1 X-Received: by 2002:a5d:960d:: with SMTP id w13mr5219675iol.126.1619814991258; Fri, 30 Apr 2021 13:36:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814991; cv=none; d=google.com; s=arc-20160816; b=MgI5mlKCY8Ck9Y8tpKCdvI8ljiNxyqK0hRPFnrFmiLtj+NSzKjc5AfGK0cbKQl/u8w PkTfm9jt+UV01C4cep/S6FZsqUhaBEod+N/qV6Tcd8w/2aTahm8HZFHOFRMKF4OhkYZR 6bVPafQdlOalon8X0XvOvm+Jg9auKFBisOLHPa9wTAQg+UatjZe2U/8/pSK1LCBfXJmZ nMqOHHIXog7mlIzfnX6htgEiun2iH8BHcwoiWSlVRK/y9Fx1FXyhbX1cz/mhU3HCVj9v 5jocGJnNjgBoToVSs7ZPUl/fOSlYR8OE8Y4sJqH+XQcbyAsFWmOgSa/9P4BqpYqgHllD FsSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=WyoGNq5KS/LKZvI3Xd+vQ/INvLZSnU8ochBIf0QUfi14+EOrGKC1He3wLnbfAJeBsY SCQvi7OE785teQHuxVx+rtOeBxb9wtQQhPEyTJ2FOEUykHvjVdXKeahVCfp4VGhj36tQ HXhaiTJiGk3ZZmgQhQsZSsPk1g6GnvVJdbVqmuu0HoGZrAh4fgMKYA+DX4aXFUjiZQx4 h3WAcGKg716UljphJAXy7GzEW1PTKJtRMWA/DKfYs1HQ1j2DH4fLq8RAPsVAj5PqABKX eRMp6i0FYJx6C0rAP0nKJ/Osu4BI4p4TAYcowujfgDRLv0kyt9Ke2u5dKU0etNE1t+Pl CKcA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ldJ7Q+V0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h5si5007009iol.44.2021.04.30.13.36.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:36:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ldJ7Q+V0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52524 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZsE-0003Bk-Kj for patch@linaro.org; Fri, 30 Apr 2021 16:36:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51758) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiY-0008QA-EA for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:31 -0400 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]:51965) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiO-00060K-0a for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:30 -0400 Received: by mail-pj1-x102c.google.com with SMTP id lp8so6987725pjb.1 for ; Fri, 30 Apr 2021 13:26:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=ldJ7Q+V0ypVZOubAKKrDEP7QeCQKfzxfN0cjC6XRiajaLEkyPqP6vyTRaurJuPIKEj Gv7P7oXXzW2gBq1Vs9mwAtMdsIquzY/j5AdGOramEqe2rlleJqBWtFjRvY+tgaJu8On0 Gy09RiAn/kBgzVQKc9I7pJEUygC9yhNJeEoOzyZs5k/0dQR2vayltTCqfL63M+Lw5s3x Zq/xwTtE+jzJb3nbPiouBJK4mfCSkI1jZQn4wCXW5K80XW6Yy/6on+nUYwab6VvtWp+c aZrmSoMP29gQQ53vhLCKOqBdCZO4QIytsdNnXD49VjnNXPQSBbchhyTDrJL5cpjalPI1 wV9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZS2f7fEpsE5PZphfky/IIrJKblfMHybmN+wC38aYIj8=; b=cTrM8X5PlQvF6Le/hFV3LLYNgl67uNQp/6RhCe7oHkYxN6/PTdhFiR5SqUVwxyA/P3 xZb9an8077ysvdHdUS43DORmiwNVN0VZFFjDGBaaQsa6l72VWTMguYoMJB1PtWrKGpcV hA8AXuaUUL2/VClOcC9uc905sGVMXrVYJlaNAW9cH3xRxWiosrYtcGnaGhR6QBELtu6b JKmbvhES+KRR8QlcDAAlNuG2prvNmFwm9+532DVUWlk7zTT5LPAwpEC838G3zqhtwwgm nEVD1y/qNtpc/E0vLSMHuiaFDRk2jfFuSb8lRFLK2RbXw/dbm2xV46nUpwH3t+YYLXE6 t37w== X-Gm-Message-State: AOAM533YoIRap6ffle8cng8+4JHJwE5O+fk6P0SuABjluZTbF55bjS9q RLnwUWIcik5IPWIqBd+aMnMrlYTGniIFjA== X-Received: by 2002:a17:90a:f2ce:: with SMTP id gt14mr7297754pjb.177.1619814378620; Fri, 30 Apr 2021 13:26:18 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 12/82] target/arm: Implement SVE2 integer add/subtract wide Date: Fri, 30 Apr 2021 13:25:00 -0700 Message-Id: <20210430202610.1136687-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsets (laurent desnogues). --- target/arm/helper-sve.h | 16 ++++++++++++++++ target/arm/sve.decode | 12 ++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 20 ++++++++++++++++++++ 4 files changed, 78 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a81297b387..3286a9c205 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -1391,6 +1391,22 @@ DEF_HELPER_FLAGS_4(sve2_uabdl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_uabdl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_saddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_ssubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_ssubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uaddw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uaddw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_usubw_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_usubw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve_ld1bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld2bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) DEF_HELPER_FLAGS_4(sve_ld3bb_r, TCG_CALL_NO_WG, void, env, ptr, tl, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 12be0584a8..f6f21426ef 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1184,3 +1184,15 @@ UABDLT 01000101 .. 0 ..... 00 1111 ..... ..... @rd_rn_rm SADDLBT 01000101 .. 0 ..... 1000 00 ..... ..... @rd_rn_rm SSUBLBT 01000101 .. 0 ..... 1000 10 ..... ..... @rd_rn_rm SSUBLTB 01000101 .. 0 ..... 1000 11 ..... ..... @rd_rn_rm + +## SVE2 integer add/subtract wide + +SADDWB 01000101 .. 0 ..... 010 000 ..... ..... @rd_rn_rm +SADDWT 01000101 .. 0 ..... 010 001 ..... ..... @rd_rn_rm +UADDWB 01000101 .. 0 ..... 010 010 ..... ..... @rd_rn_rm +UADDWT 01000101 .. 0 ..... 010 011 ..... ..... @rd_rn_rm + +SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm +SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm +USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm +USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3d0ee76411..cf0dbb3987 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1156,6 +1156,36 @@ DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZ_TB +#define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_WTB(sve2_saddw_h, int16_t, int8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_s, int32_t, int16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_saddw_d, int64_t, int32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_ssubw_h, int16_t, int8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_s, int32_t, int16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_ssubw_d, int64_t, int32_t, , H1_4, DO_SUB) + +DO_ZZZ_WTB(sve2_uaddw_h, uint16_t, uint8_t, H1_2, H1, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_s, uint32_t, uint16_t, H1_4, H1_2, DO_ADD) +DO_ZZZ_WTB(sve2_uaddw_d, uint64_t, uint32_t, , H1_4, DO_ADD) + +DO_ZZZ_WTB(sve2_usubw_h, uint16_t, uint8_t, H1_2, H1, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_s, uint32_t, uint16_t, H1_4, H1_2, DO_SUB) +DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) + +#undef DO_ZZZ_WTB + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ae8323adb7..70900c122f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6020,3 +6020,23 @@ DO_SVE2_ZZZ_TB(UABDLT, uabdl, true, true) DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) + +#define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzw_ool(s, a, fns[a->esz], SEL2); \ +} + +DO_SVE2_ZZZ_WTB(SADDWB, saddw, false) +DO_SVE2_ZZZ_WTB(SADDWT, saddw, true) +DO_SVE2_ZZZ_WTB(SSUBWB, ssubw, false) +DO_SVE2_ZZZ_WTB(SSUBWT, ssubw, true) + +DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) +DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) +DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) +DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) From patchwork Fri Apr 30 20:25:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429856 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp781345jao; Fri, 30 Apr 2021 13:45:47 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzd+iE9f+n6N5JPFzv4ldLxX8Hq9WGJRa6EoDQ+JptLhe1QSjHNQ0VOdNHl8U5FU5nkqrRz X-Received: by 2002:a5d:9149:: with SMTP id y9mr5177175ioq.159.1619815547515; Fri, 30 Apr 2021 13:45:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815547; cv=none; d=google.com; s=arc-20160816; b=Vq0i8J7dFvHpjLw0PXfdfQowN54uAm/Fjni62NGwpX6mvkmWwriEmWo4C6IknR1XDT bYHXBw65TizLOV4dpDEjW1IMxebAJkLRHr5MovE/AVtWB3M4IrR6smlkkw9gb7nUJlP8 QtatR7W/UOINJhWdfHJqqGy2VmGqSI1zHaXka9WKEwkPG/5YMcbB23M4ob92XS7voYn4 uEaIgnb+Af4x4zn1sNlXtHr9tN7b4jw7cu0u5vSJAcop6ExUQNvs7Jk3VF5xefre5qiT ukmZIB0UqDnvL4yTP8URbgy9ZvUOMEwov+DGG9EiYNktU1Wb38i1Yv0iXEL5pH4yKXSk zXPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=lSjH1gSwU0ewa0wG9FJpDpNrVyLjLf5vMJfKYsKNRrR21/1KBIxV6EK47dWOxXmJUK 31oLgPaKidYL4wjgiXmx04XXyfKtaW57RizDrQ1KdamjoSa7LqCqy6E+1H0ck39Kfr5L fux+hZug8tiNUNhYRzcjtMspzjXboLHpam1z1OCSQJgzXNh6ZX6WhGLOnrEy4zy5FIxk X1FI1RtxzYtvXHvGhij4SMbw0WSDjg5jiAzKCZ4Q8IMMVCEcFcLSZEM9l6oRB/Q/hbGp 3CTUEykqVqUMuseb2QT0khdLDUGtel6GO8UEVjF0AvV2qex0IMR9mQUdwXcuivHJIprY ad6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bTQ4Ck02; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m12si5155123ioj.96.2021.04.30.13.45.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:45:47 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bTQ4Ck02; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47666 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca1C-0004de-Sf for patch@linaro.org; Fri, 30 Apr 2021 16:45:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51952) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZif-00005U-Ti for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:37 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:40629) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiO-00060l-Fw for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:37 -0400 Received: by mail-pg1-x52e.google.com with SMTP id y30so5010000pgl.7 for ; Fri, 30 Apr 2021 13:26:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=bTQ4Ck02Aw/OMBO5R8SyMjU/NpjP6dsW7h9DcK/IzMVSegdn0V7u0h30oa4IuRuYF1 WDyxUOhVBjrLK1ynXU2dZDS3LzEaRcF5tJPK/xReR5cFrBmk0+quTWVGIy4UdwOqSwkd pUBVpFrVHsxtB3z+GgPW1IDJJuE3wIheB5TD5xI1sUDCUjMCnDZhRW1gOC2Tn5w1mZYp kbbojrsAVGdWuf7di76a6EpoW237xqu972Yd6QoT7dOQ2PHqqR4TjnJJ08TGXhMFikrP aktyDM49hHSHof8d+g5j12GCDcXzq83Kr+MqJIm9NlmP/eQY4zpakScgLkLFlKSCzMGZ K15w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9m49cTBF+uBd2parMcxDIMwzzkCg+x47RKDDIpkpN1g=; b=Wvh+k+xF1MWYAlpkoeAO6lGm8J8fMSfzapWyv/7HxpESAeqoO399UInpLzF9vdzDzp zo1LYWSQwkIkFw/pQ333LcwI+FxaIDTFJVHv0OizQuVFQJi5y3yKJQuMXCc/aeK4MB6J QI2qovq1UGK6BvHO2hUQMK6fgThu5PYc5ufVUaxOaT4Ljj82gFHgPReR3Lv0nLEp+G3I zsY8umSj6f23pGhIfEhmRDcWMpa34FJRBZPCIVTAFF4IsPcqztk/h4KVk7sTBTmUQTon ToQM7LwB5B0F0S79zkLfXlyFDJX6hquZT0DHT8Uf50VRyvTS82/p/uKfyRCwMT7vdNFv iqvg== X-Gm-Message-State: AOAM533woDq5DCe/wwPlNvs6QdbLmlFTcjZAq1rjtbMumCc7igjOB8IX QMxaiJhVetmlZll5QTwGV6nLwSs056Nu7g== X-Received: by 2002:a65:4985:: with SMTP id r5mr6205596pgs.65.1619814379125; Fri, 30 Apr 2021 13:26:19 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:18 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 13/82] target/arm: Implement SVE2 integer multiply long Date: Fri, 30 Apr 2021 13:25:01 -0700 Message-Id: <20210430202610.1136687-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Exclude PMULL from this category for the moment. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 15 +++++++++++++++ target/arm/sve.decode | 9 +++++++++ target/arm/sve_helper.c | 31 +++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 9 +++++++++ 4 files changed, 64 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3286a9c205..ad8121eec6 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2347,4 +2347,19 @@ DEF_HELPER_FLAGS_6(sve_stdd_le_zd_mte, TCG_CALL_NO_WG, DEF_HELPER_FLAGS_6(sve_stdd_be_zd_mte, TCG_CALL_NO_WG, void, env, ptr, ptr, ptr, tl, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_zzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_smull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_smull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_umull_zzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f6f21426ef..d9a72b7661 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1196,3 +1196,12 @@ SSUBWB 01000101 .. 0 ..... 010 100 ..... ..... @rd_rn_rm SSUBWT 01000101 .. 0 ..... 010 101 ..... ..... @rd_rn_rm USUBWB 01000101 .. 0 ..... 010 110 ..... ..... @rd_rn_rm USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm + +## SVE2 integer multiply long + +SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm +SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm +SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm +UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm +UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index cf0dbb3987..f30f3722af 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1154,6 +1154,37 @@ DO_ZZZ_TB(sve2_uabdl_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZ_TB(sve2_uabdl_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZ_TB(sve2_uabdl_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZ_TB(sve2_smull_zzz_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_smull_zzz_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZ_TB(sve2_umull_zzz_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZ_TB(sve2_umull_zzz_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +/* Note that the multiply cannot overflow, but the doubling can. */ +static inline int16_t do_sqdmull_h(int16_t n, int16_t m) +{ + int16_t val = n * m; + return DO_SQADD_H(val, val); +} + +static inline int32_t do_sqdmull_s(int32_t n, int32_t m) +{ + int32_t val = n * m; + return DO_SQADD_S(val, val); +} + +static inline int64_t do_sqdmull_d(int64_t n, int64_t m) +{ + int64_t val = n * m; + return do_sqadd_d(val, val); +} + +DO_ZZZ_TB(sve2_sqdmull_zzz_h, int16_t, int8_t, H1_2, H1, do_sqdmull_h) +DO_ZZZ_TB(sve2_sqdmull_zzz_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZZ_TB(sve2_sqdmull_zzz_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + #undef DO_ZZZ_TB #define DO_ZZZ_WTB(NAME, TYPEW, TYPEN, HW, HN, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 70900c122f..19a1f289d8 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6021,6 +6021,15 @@ DO_SVE2_ZZZ_TB(SADDLBT, saddl, false, true) DO_SVE2_ZZZ_TB(SSUBLBT, ssubl, false, true) DO_SVE2_ZZZ_TB(SSUBLTB, ssubl, true, false) +DO_SVE2_ZZZ_TB(SQDMULLB_zzz, sqdmull_zzz, false, false) +DO_SVE2_ZZZ_TB(SQDMULLT_zzz, sqdmull_zzz, true, true) + +DO_SVE2_ZZZ_TB(SMULLB_zzz, smull_zzz, false, false) +DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) + +DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) +DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ From patchwork Fri Apr 30 20:25:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429850 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp778959jao; Fri, 30 Apr 2021 13:41:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwVDXEfPZkxZtQOw1capWFCKF8u31c12G4wU0pSZuuezOIwLYwOJ+AZd9p+mRlUwGwB/6vA X-Received: by 2002:a9d:6c54:: with SMTP id g20mr5194394otq.116.1619815304519; Fri, 30 Apr 2021 13:41:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815304; cv=none; d=google.com; s=arc-20160816; b=lX+GMyKt/J/zqcrdlTI7wQJx9k9coPL2j0WhCOgvNrH//FyGdH+oPJrL3/GnauXj7a utEQu8BDX6V7zCUBHvGBmmEe/7za61RZ4YrNWIom6CXA7cUMLxB7j8ZbZ1161p4uRmWe dHj5dRCWfoytDvvpylhPo2j9WqZU9b8kx3gTkX6lBauXgjUGF283OlNC8uj8phEr1k/M q7nN0xshiv5fLQvLL5DvBWSG4vNC/8J2TXbatEyrC+qHsPVxExktp4WKGdGX6HYJ0XVv YCh4CQgO/fPl+th/afb3UXtEfJ3YRDnpMKZbpfoAY7loV7LeJTdMidiJMcTfczzgfqK6 tZ1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=m7pBVh/c5AJy1bfjiUtGsaKpsFfSNoR5ZeoaFlHjYWiOUBkOHit1206yJIixJH5Wu5 hW1OPKi13r9ppdOlCZMoZ5z7l6q6SgQoJiXeMXw+eK1spOcWPkTB8kIpSwnYnN5ncykF yrE6m5oZQkR0Ho9U9Tyd1VVQkq9RhBvTZOdY8EqlPbGFL9ZbyqBT93u2wSaY0L+LSohn QjGQ1qPMXUrTRZHAkfqQVkXu+9Nf8hKO6Tk8CiewSwEdYA5Px2wnKBVITCWkVby6C+u1 K06yrP9MllNKrlLwXC15k2+k4Scfa1nBDHZXx/17ocGIj6zSrwUXYJq1C5CLWaVVQBC8 ow/Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=trRNhIVg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n18si4145630otf.19.2021.04.30.13.41.44 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:41:44 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=trRNhIVg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZxH-0007XQ-SO for patch@linaro.org; Fri, 30 Apr 2021 16:41:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51824) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZia-0008Rl-N6 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:32 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:53056) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiP-000614-4a for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:32 -0400 Received: by mail-pj1-x1034.google.com with SMTP id ge1so244033pjb.2 for ; Fri, 30 Apr 2021 13:26:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=trRNhIVgi69kJghI+oxB01zCsaQSUlulS9UUZnDOOpjN4x4dJ3brJIVXezJicq3X1f A1mQGoXPzGBIviVgnxAEAOTQnbjTWtaBkGDbpP7a8Ev9U4/BiMWRzNmiaShe89q3WDRd vHwRVNbD30dEoRxunYSsi0H4floc/efje7xR2xWDYUMeqTzEUKnH3+Y/hJxAI5LfH9Pc +9ZyMIwWm/wN1I0w4Gai5RHc0zvrepaGfwDqoiecoWsFaai0ZW7A+K8GSiotTQuarVuH Wf/hWY3JgDNhkPhjOPDCcnSCuPRo1fpDkqELL8DMNztmiBAuLTJnCdDEqvSrzjEVf9X3 6qmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TFy/ivv1KJ7f/BHQOtAUnbXQPt7AVG5AmrGqJKBkDA4=; b=tJmjonifFxydD1qnROj8bC88NyQ5UQ/dcghmKP68SKwryhnmQJ2Nr1Q89GkDeTGo6f FD3RI5JVFCJmx69GUUxUuvfUkQhOqE6Ghk8W8qNay9uU5Yol8OGvZLAfW2wiLKmpqJiF 4MdXgzTyqDiHUREFvEKe/UymManwwDRWX04sX1BU/4dIh7hk09UJ3fiwuOFc+8Dg9U7A SHrOJimEo5h8pKmT0yjw98oWin26wkv1R8HGe2O9W6h1bxtVEvkVvsFJNKO+pnQc0Hkk cmIWpAjD+/8/NjJ+RAlCVw4og+FcLQZQBJL0vBdurOK5UlA8GbSu25ZQ/T1KWETfD9Gl Sxcg== X-Gm-Message-State: AOAM530azrNVixR05ZWy0z2KC5izlUJC2U6x1WtQTd5NC3Tem02LHXIy 63O18qdl+9eHTu5EGIt1BpUXwd9PpwnfRg== X-Received: by 2002:a17:90a:488a:: with SMTP id b10mr7086548pjh.2.1619814379613; Fri, 30 Apr 2021 13:26:19 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 14/82] target/arm: Implement PMULLB and PMULLT Date: Fri, 30 Apr 2021 13:25:02 -0700 Message-Id: <20210430202610.1136687-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++++++ target/arm/helper-sve.h | 1 + target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 22 ++++++++++++++++++++++ target/arm/vec_helper.c | 24 ++++++++++++++++++++++++ 5 files changed, 59 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index a6e1fa6333..902579d24b 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4231,6 +4231,16 @@ static inline bool isar_feature_aa64_sve2(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SVEVER) != 0; } +static inline bool isar_feature_aa64_sve2_aes(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) != 0; +} + +static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ad8121eec6..bf3e533eb4 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2363,3 +2363,4 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d9a72b7661..016c15ebb6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1201,6 +1201,8 @@ USUBWT 01000101 .. 0 ..... 010 111 ..... ..... @rd_rn_rm SQDMULLB_zzz 01000101 .. 0 ..... 011 000 ..... ..... @rd_rn_rm SQDMULLT_zzz 01000101 .. 0 ..... 011 001 ..... ..... @rd_rn_rm +PMULLB 01000101 .. 0 ..... 011 010 ..... ..... @rd_rn_rm +PMULLT 01000101 .. 0 ..... 011 011 ..... ..... @rd_rn_rm SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 19a1f289d8..fbdccc1c68 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6030,6 +6030,28 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_gvec_pmull_q, gen_helper_sve2_pmull_h, + NULL, gen_helper_sve2_pmull_d, + }; + if (a->esz == 0 && !dc_isar_feature(aa64_sve2_pmull128, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], sel); +} + +static bool trans_PMULLB(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, false); +} + +static bool trans_PMULLT(DisasContext *s, arg_rrr_esz *a) +{ + return do_trans_pmull(s, a, true); +} + #define DO_SVE2_ZZZ_WTB(NAME, name, SEL2) \ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ { \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 40b92100bf..b0ce597060 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -1750,6 +1750,30 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = pmull_h(nn, mm); } } + +static uint64_t pmull_d(uint64_t op1, uint64_t op2) +{ + uint64_t result = 0; + int i; + + for (i = 0; i < 32; ++i) { + uint64_t mask = -((op1 >> i) & 1); + result ^= (op2 << i) & mask; + } + return result; +} + +void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t sel = H4(simd_data(desc)); + intptr_t i, opr_sz = simd_oprsz(desc); + uint32_t *n = vn, *m = vm; + uint64_t *d = vd; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + } +} #endif #define DO_CMP0(NAME, TYPE, OP) \ From patchwork Fri Apr 30 20:25:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429849 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp778245jao; Fri, 30 Apr 2021 13:40:38 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzKsT6gUXjcoVTJkCLxAB4F/QfpyUHcxaP/bXuhosis+WdUkTpjPUEt+dyz2jQoevh2vUn2 X-Received: by 2002:aca:f4cd:: with SMTP id s196mr12481379oih.165.1619815238292; Fri, 30 Apr 2021 13:40:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815238; cv=none; d=google.com; s=arc-20160816; b=dMFObtaTRHP4DJ4rBDkauTmLW8HikNXSewyTRkI4yHCS0p6s7pAVMebTW7ybMlj9+z eedtYV4FG8C3v/Rx3SdeJH/vF0GDK7GHuP2xyIYgrYR3OUgBHLceoaD8BtdrCUtZ9whX D1HFEXBH0KvrS0kTEZ5X65eRQj33O9Bzu/N0+pDF1XeGf7AHd+Prx35CPgl5BJ8l/xDn OD4A0JS+0249DqYBmoWUqPNrMpPw9qphuftoh7M1CZGMsfVEFDCFyvyfBbqADcxMSqO2 5sxNwNNNOaMZ3StCkJuF3qSa7DFRzyzdqkwRRBnjv+PAJgq6r6o04/GKfDK3HIIaGLJp hl6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=JNFy4ZucDu4Z4+3kN4KqnVojsnilxZDH4JLxjhWb7hCWBFLg6zruNOGKhgH9cuzmfy 4G10qPnYqG1kYGkMiZqVEfJZ55wzWD935IgSeQjjxCGfn+86592ypflzDSyNyme8u6yl dLk7STow00fX9xa/FnqH7dZ9/8mm3XoX+oFV1Ic5rzUd/VshdbfL6pgCzdBH2KDzBbuT 30o3unVAaKsiaxt2ZzfQX7Czig2MhtzIzuLDb0Tgk34846CZ2f5S6L0h17z/H7eKLerS xnmelImXRFf1tWfi7Ys3b3ML6QZe9y0jna89PWMlHPVtq5fcrbqp7OosOqRzKK5508XK Ritg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GiWaayaO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m84si1516940oig.173.2021.04.30.13.40.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:40:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GiWaayaO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35278 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZwD-0007qF-J5 for patch@linaro.org; Fri, 30 Apr 2021 16:40:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51944) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZif-0008WD-2l for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:37 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:34428) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiP-00061n-Qv for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:36 -0400 Received: by mail-pj1-x1031.google.com with SMTP id t2-20020a17090a0242b0290155433387beso7264692pje.1 for ; Fri, 30 Apr 2021 13:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=GiWaayaOsQogNvVqHaXQKp6rWVCq07j5ughaLxycEHbN8C9axuqvgrtqgaTIp9rrPt 19TnRoBJTE3IvEwHxUTTSmGDiC06nDXxVl4kfK9lmYlNC/cdhEylexROtKYDkItElzDY xYZvOwyen8oCYSF1LLdNqc13JajrgriaM2ZYr47C3g3o/Jm1u+5ne7ygRR4NRqCY1fmC gf6hMiAwCMOnjYkz6gx+X7dLSsnPNyu+CxJyDYxxwFXlTFk2cVQbQkZC1UouS1VBgWZK HtVIxW1kBbnlvS992BgMOG2NqzRQ0yNXFAoK/TnsCFaG4qdka0r5maMPr5vPhOKC7RFP fCrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RWwVG9bQnvXe28WlqK7SF172m5+Drw7KJ7BBkuXVIG8=; b=a0Bkv2LPWjhYfxJPh56md3qOpt42mEOkXonK6ZACR+MKY7EjnL0OeZUAAKxMfEY2nX fG31NYsgXwmJV016BHulyprqVFSH1CfKHPHJ9K6Es3y4AaeJ3FgiXDmJSKw9Yye5PmHj 2hnascMtfi6yQHVGnGXCW3tnTBRV5lu9aAOhGwWhlIBKYPxxnwtq13pXc3mgzSr8npX8 bNC8pr2PzywDR6WlTD5x+D69ndnIRnqFHtYUy65tctcRwDpNJemzTJHi0Mz1zpJhP2sR oG4Hvb6FZEyKPh191gE12a3CXBdkcHzBfUzMZVgXRKzximYgSTYM2/AfvsUaLDNDbX6h UFeQ== X-Gm-Message-State: AOAM5302ql2BG9M7vxoMjmH4a7Mbt8RoKWnnnntrlSJJeEspZCVq3oEm JZHAbv5es2rIHIyEOsILBYBZyEPkDzum9A== X-Received: by 2002:a17:90a:788d:: with SMTP id x13mr14953211pjk.140.1619814380241; Fri, 30 Apr 2021 13:26:20 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:19 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 15/82] target/arm: Implement SVE2 bitwise shift left long Date: Fri, 30 Apr 2021 13:25:03 -0700 Message-Id: <20210430202610.1136687-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 26 ++++++ target/arm/translate-sve.c | 159 +++++++++++++++++++++++++++++++++++++ 4 files changed, 201 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index bf3e533eb4..740939e7a8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2364,3 +2364,11 @@ DEF_HELPER_FLAGS_4(sve2_umull_zzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_pmull_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sshll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 016c15ebb6..a3191eba7b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1207,3 +1207,11 @@ SMULLB_zzz 01000101 .. 0 ..... 011 100 ..... ..... @rd_rn_rm SMULLT_zzz 01000101 .. 0 ..... 011 101 ..... ..... @rd_rn_rm UMULLB_zzz 01000101 .. 0 ..... 011 110 ..... ..... @rd_rn_rm UMULLT_zzz 01000101 .. 0 ..... 011 111 ..... ..... @rd_rn_rm + +## SVE2 bitwise shift left long + +# Note bit23 == 0 is handled by esz > 0 in do_sve2_shll_tb. +SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl +SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl +USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl +USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index f30f3722af..73aa670a77 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -625,6 +625,8 @@ DO_ZPZZ(sve2_sqrshl_zpzz_h, int16_t, H1_2, do_sqrshl_h) DO_ZPZZ(sve2_sqrshl_zpzz_s, int32_t, H1_4, do_sqrshl_s) DO_ZPZZ_D(sve2_sqrshl_zpzz_d, int64_t, do_sqrshl_d) +#undef do_sqrshl_d + #define do_uqrshl_b(n, m) \ ({ uint32_t discard; do_uqrshl_bhs(n, (int8_t)m, 8, true, &discard); }) #define do_uqrshl_h(n, m) \ @@ -639,6 +641,8 @@ DO_ZPZZ(sve2_uqrshl_zpzz_h, uint16_t, H1_2, do_uqrshl_h) DO_ZPZZ(sve2_uqrshl_zpzz_s, uint32_t, H1_4, do_uqrshl_s) DO_ZPZZ_D(sve2_uqrshl_zpzz_d, uint64_t, do_uqrshl_d) +#undef do_uqrshl_d + #define DO_HADD_BHS(n, m) (((int64_t)n + m) >> 1) #define DO_HADD_D(n, m) ((n >> 1) + (m >> 1) + (n & m & 1)) @@ -1217,6 +1221,28 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel = (simd_data(desc) & 1) * sizeof(TYPEN); \ + int shift = simd_data(desc) >> 1; \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel)); \ + *(TYPEW *)(vd + HW(i)) = nn << shift; \ + } \ +} + +DO_ZZI_SHLL(sve2_sshll_h, int16_t, int8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_sshll_s, int32_t, int16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_sshll_d, int64_t, int32_t, , H1_4) + +DO_ZZI_SHLL(sve2_ushll_h, uint16_t, uint8_t, H1_2, H1) +DO_ZZI_SHLL(sve2_ushll_s, uint32_t, uint16_t, H1_4, H1_2) +DO_ZZI_SHLL(sve2_ushll_d, uint64_t, uint32_t, , H1_4) + +#undef DO_ZZI_SHLL + /* Two-operand reduction expander, controlled by a predicate. * The difference between TYPERED and TYPERET has to do with * sign-extension. E.g. for SMAX, TYPERED must be signed, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fbdccc1c68..afd208212b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6071,3 +6071,162 @@ DO_SVE2_ZZZ_WTB(UADDWB, uaddw, false) DO_SVE2_ZZZ_WTB(UADDWT, uaddw, true) DO_SVE2_ZZZ_WTB(USUBWB, usubw, false) DO_SVE2_ZZZ_WTB(USUBWT, usubw, true) + +static void gen_sshll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int top = imm & 1; + int shl = imm >> 1; + int halfbits = 4 << vece; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_sari_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_sari_vec(vece, d, d, halfbits - shl); + } +} + +static void gen_ushll_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = (imm >> 1); + int shift; + uint64_t mask; + + mask = MAKE_64BIT_MASK(0, halfbits); + mask <<= shl; + mask = dup_const(vece, mask); + + shift = shl - top * halfbits; + if (shift < 0) { + tcg_gen_shri_i64(d, n, -shift); + } else { + tcg_gen_shri_i64(d, n, shift); + } + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_ushll16_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_16, d, n, imm); +} + +static void gen_ushll32_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_32, d, n, imm); +} + +static void gen_ushll64_i64(TCGv_i64 d, TCGv_i64 n, int64_t imm) +{ + gen_ushll_i64(MO_64, d, n, imm); +} + +static void gen_ushll_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t imm) +{ + int halfbits = 4 << vece; + int top = imm & 1; + int shl = imm >> 1; + + if (top) { + if (shl == halfbits) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(halfbits, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shri_vec(vece, d, n, halfbits); + tcg_gen_shli_vec(vece, d, d, shl); + } + } else { + if (shl == 0) { + TCGv_vec t = tcg_temp_new_vec_matching(d); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); + } else { + tcg_gen_shli_vec(vece, d, n, halfbits); + tcg_gen_shri_vec(vece, d, d, halfbits - shl); + } + } +} + +static bool do_sve2_shll_tb(DisasContext *s, arg_rri_esz *a, + bool sel, bool uns) +{ + static const TCGOpcode sshll_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, 0 + }; + static const TCGOpcode ushll_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, 0 + }; + static const GVecGen2i ops[2][3] = { + { { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_h, + .vece = MO_16 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_s, + .vece = MO_32 }, + { .fniv = gen_sshll_vec, + .opt_opc = sshll_list, + .fno = gen_helper_sve2_sshll_d, + .vece = MO_64 } }, + { { .fni8 = gen_ushll16_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_h, + .vece = MO_16 }, + { .fni8 = gen_ushll32_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_s, + .vece = MO_32 }, + { .fni8 = gen_ushll64_i64, + .fniv = gen_ushll_vec, + .opt_opc = ushll_list, + .fno = gen_helper_sve2_ushll_d, + .vece = MO_64 } }, + }; + + if (a->esz < 0 || a->esz > 2 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, (a->imm << 1) | sel, + &ops[uns][a->esz]); + } + return true; +} + +static bool trans_SSHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, false); +} + +static bool trans_SSHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, false); +} + +static bool trans_USHLLB(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, false, true); +} + +static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_shll_tb(s, a, true, true); +} From patchwork Fri Apr 30 20:25:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429863 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp784829jao; Fri, 30 Apr 2021 13:51:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzG9fHK5C0302fRK1W2rW/Ezs3Auu+jeUxZ4i1tv2aLJHtnM7SMg+fVsSroIl8RZU3jVm/f X-Received: by 2002:a05:6e02:1243:: with SMTP id j3mr5585664ilq.192.1619815885014; Fri, 30 Apr 2021 13:51:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815885; cv=none; d=google.com; s=arc-20160816; b=YqQ25PTLr3WD9hhrnVdQYwbdnE5nflZEOYyKCn0UezVhbXpnTXLajktzB5F1hxd4Na VO4+1Bh99DqgWHrjIkPPbg3qi8mNA6IA+SqyfDfFCiuidYdv2zdDK1Z9+uisuWVWEADW jF0FcMqXW3AhcZ05UED9cjzcUl4lqfv6DJk5eWTFEHx8ybDbpQqA/76Q3+WYxQkbt67+ mS+3fMPON3bPd+G/MCoyuYIn/tOOk7t0SQYy9QvG0Fd6J6FNLHOIEUlqCkCYBtDnOqE4 f0VPR0M7ITfQtfqGNILCmp2OA5xh/SWUlRt4cN+MFCAHaUXqu0jLidzuYcXzT9mzNvhD grAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=RdnKetDLgGnbap8vkJ+yS/srbPVfT/yBFyBlbEmHf1WjFVZbit84Uo70TYoFSCHeKw q0KsEoKtCPw3gPx4uoNmCH+G/pUjt4AE60fXLfNdG+gHX2mdXVgI+v0RckMUeGiAEa1w lYf+GllaNJ9K907w0xAdOiFxjoIMTc7xu+VWq8x/WDscsg9O0lYjGj3SD1uai4BQvWwM ce4ROdvP0hyiUGUkc/gRpN8lugFtdRNxXAGQUlWSoE3AT3FkKR16d3cBIo7y3UZLnmy/ g4Ccj/bVuZAmiz5JOi4Zz4FfNsHo6D7iuEtIHWPVwpy8e4plhi0eIo7J0RJYWXfSjfRE Nd9A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TMjG6b4s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f1si3670773jat.67.2021.04.30.13.51.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:51:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TMjG6b4s; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca6e-0003QU-C5 for patch@linaro.org; Fri, 30 Apr 2021 16:51:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51990) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZih-00006h-BH for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:39 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:54028) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiQ-00063A-3Y for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:39 -0400 Received: by mail-pj1-x1031.google.com with SMTP id p17so9146376pjz.3 for ; Fri, 30 Apr 2021 13:26:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=TMjG6b4sQK6W7mQoCXgkWZbQsvZwbuGGWibP5ExEzwKih3U7+RbqNXyTRIApAqkBcR hNdFSSZntGDPdGfcHGtTCYtaQdtG58dQMwvYaMT9xg9YVhjNjV0jeSS2lGIbiSNcwi9D CSDAV8N1WpK4+oGGL1RNzhN0Rjw/gBXu7EFjdW5/6dBSeW2waryCpScpugXHEl8XIXbh LaBmmir2ftr/cIbTr2O1v0A/pS/GhaTbQwIe6IV1w2qHArLkq9cKU0ZC1Bz9zSvBR6qA o6OHtfwunDeaP4AH1D5jYopPDYcao+XbYaCtLEXeFERRlpaQH28lGfWqqdQpOaxQN2UA 48ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=iA5YdMPlGGiyhjKxfBQbY6+WZzBPtERwoPFsnw8+g+g=; b=ouLcSpGiRZ22zGQexmlMi49xXXhrNhPYOSdsRiMlSmAX+NM3IIRuZJQCWz6R01Q109 0eNQn6TNCOgZKlAleSvem5oQzSSqOtOhgRb8B6BE6/yig9tOKwXBxWtedWvVa6bfhxxO QMqvvyr9Eoza6P5YxDHYHe3omXBBXZ8RsWyCPmkC8hcTGRwIKVwxa8aVf4ad8/5ymdzC 2hGQAcOmqFZUvTKQ4qtUSWnHQx2mzv1qvwwg6KB47OjMDp+d4JKb0TsO/qTNW/EcMU8L JoiZHsWYmXGOJmpkfyZTtFkUpnNDmC1L4TbfPw+xw35Qi5zj0LD5Lne7hD6dWzAWwkSG z0vg== X-Gm-Message-State: AOAM533qO1InB/DuA3NVGBcN8XxLPCDEjeRgf84RNKKT0lHb0nEfj1Rv V2iwAT6vjjdxmI9r24Xqnc2/YYjFOH6Dkw== X-Received: by 2002:a17:903:4104:b029:ed:7bda:412c with SMTP id r4-20020a1709034104b02900ed7bda412cmr7239846pld.31.1619814380780; Fri, 30 Apr 2021 13:26:20 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:20 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 16/82] target/arm: Implement SVE2 bitwise exclusive-or interleaved Date: Fri, 30 Apr 2021 13:25:04 -0700 Message-Id: <20210430202610.1136687-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 19 +++++++++++++++++++ 4 files changed, 49 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 740939e7a8..f65818da05 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2372,3 +2372,8 @@ DEF_HELPER_FLAGS_3(sve2_sshll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_ushll_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a3191eba7b..0922a44829 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1215,3 +1215,8 @@ SSHLLB 01000101 .. 0 ..... 1010 00 ..... ..... @rd_rn_tszimm_shl SSHLLT 01000101 .. 0 ..... 1010 01 ..... ..... @rd_rn_tszimm_shl USHLLB 01000101 .. 0 ..... 1010 10 ..... ..... @rd_rn_tszimm_shl USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl + +## SVE2 bitwise exclusive-or interleaved + +EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm +EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 73aa670a77..1de0a9bdc3 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1221,6 +1221,26 @@ DO_ZZZ_WTB(sve2_usubw_d, uint64_t, uint32_t, , H1_4, DO_SUB) #undef DO_ZZZ_WTB +#define DO_ZZZ_NTB(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPE); \ + intptr_t sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPE); \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + H(i + sel1)); \ + TYPE mm = *(TYPE *)(vm + H(i + sel2)); \ + *(TYPE *)(vd + H(i + sel1)) = OP(nn, mm); \ + } \ +} + +DO_ZZZ_NTB(sve2_eoril_b, uint8_t, H1, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_h, uint16_t, H1_2, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_s, uint32_t, H1_4, DO_EOR) +DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) + +#undef DO_ZZZ_NTB + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index afd208212b..509b3bc68c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6030,6 +6030,25 @@ DO_SVE2_ZZZ_TB(SMULLT_zzz, smull_zzz, true, true) DO_SVE2_ZZZ_TB(UMULLB_zzz, umull_zzz, false, false) DO_SVE2_ZZZ_TB(UMULLT_zzz, umull_zzz, true, true) +static bool do_eor_tb(DisasContext *s, arg_rrr_esz *a, bool sel1) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_eoril_b, gen_helper_sve2_eoril_h, + gen_helper_sve2_eoril_s, gen_helper_sve2_eoril_d, + }; + return do_sve2_zzw_ool(s, a, fns[a->esz], (!sel1 << 1) | sel1); +} + +static bool trans_EORBT(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, false); +} + +static bool trans_EORTB(DisasContext *s, arg_rrr_esz *a) +{ + return do_eor_tb(s, a, true); +} + static bool do_trans_pmull(DisasContext *s, arg_rrr_esz *a, bool sel) { static gen_helper_gvec_3 * const fns[4] = { From patchwork Fri Apr 30 20:25:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429851 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp779107jao; Fri, 30 Apr 2021 13:42:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxnDDRa9hFih7vINhdgNtf0qrUuhGrp/vbWKQ0UZNJwOwrU1mnavwcbJAlTMZ2YSzukVj0M X-Received: by 2002:a02:6c01:: with SMTP id w1mr6642731jab.107.1619815320685; Fri, 30 Apr 2021 13:42:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815320; cv=none; d=google.com; s=arc-20160816; b=luh3m2n4YMv0qjHIFFfneD5dKjxRLTBoHg7rnzjZYlKXASD96zNMwputzjtlWM6ZiS yrvF8pehlx5rtdWejgTaeNNw2lk1a/IXsJPckVo1tX91YEpcQa8LIC2mLs9s97sq/Qr5 s1pUWv3eihwigh88ciTUWHheqdpA7P3POTQrHxx37G09vNHAGwon1EBai4XNq+D8vPmR ZRHtvYdGhphu+Nyxw1OfrXsIMdXvN/T1GgAuGoCBCVz+SBZYnMLeY0fOKJN55xt1R5dB LqZGrsODZwuF9oZja8ZvynpifpsTcGsyWkU6KcGqkstydKIFKFr+73/SG/pKZGu2glX2 L2pQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=iD/UqmwOhVSxfmf6Gw0aE2GLJjyBrpQx4BhaOFIdVfxYTR7Fr7USLkcldbAb9NYCDy 8iyjbv2Nu+0Mfnk77ST0aDzUQoKn/H7lKX1npQ+LJaKC+R15VXq0JFZjOx4Iz5F3CvZU oTd3V1flL4AszwtsAmHHfApxKJzJu9kBU2Aufwc3rlrC6HGzB3RCfYz2mFoS7ynSgzXK mRxFptSVctIvRrWecuJxyiO2uOt8RyttEb6lkAzzehFYyQOx8UOjcsXZaq/LwZ1e+UsN 71R2Uu0CB3OSeBoG9W3fJDGM3MUbgx2OaPAAnavN5hzgWozJbryr4PlpRINqhL85AHOH q7zg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ctmbQ+sS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a17si4991011iol.19.2021.04.30.13.42.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:42:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ctmbQ+sS; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38614 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZxY-0000tX-0J for patch@linaro.org; Fri, 30 Apr 2021 16:42:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZif-00005b-VF for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:38 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]:45637) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiQ-000645-Sf for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:37 -0400 Received: by mail-pf1-x42e.google.com with SMTP id i190so9677357pfc.12 for ; Fri, 30 Apr 2021 13:26:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=ctmbQ+sSkl910W6FVbVWBDeyucXx6d3Giv+Tvmdsx61I5gbVX6vBXeCFUX0X4EJCDo zVYlv2/lWpG74YIW1km02mQjgGtGaikwDGIBqT7/Pp0iGNuycL/r2+ngiOl8ofdtDBFz 6CDglC6ugYWlXgDPqDRKNRpek0ZBM+pK8YjzLIGeXsgsSE7XxU1/8cDJtDBcEhiN/t6o CdDfr2EJKXJk6ShSabyzY6dDVavIlC6uSDjQFGFuUdDryG0RTj/cWI6pLwbUtaGcnGY0 6HvNiY7oMqDvQOTw0mZEjL6kEHPD0aPgvT2apBZ/ZIn3uWBJEL2/Kmb8I2IpUexbg14N XOJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BNoITf8uwrxbb0GnHYmFR2vb1nVzSKvKz1i47CoRknE=; b=jDG389Q9dSp8ancYDlExi/GQvMdkaf7dyQpx1vZ6ioIqaVBF+kb+cx2Ybcz6zogYW3 /cWk5RnrWH1jbYIbPS1ba+lh1afFMhv78y9SamZCFRtamABKvS5Jlg+z/slEL/uK01Xt 4koV7HhctY2jkZEJWBL/hVp3nL//4ZaFFb3vhkyEUjvXksdauKNZYqyPuA0im2ALkMTX 0TxNkuKJZpZHtS95dY4hGnvsBrHQlli5T193+4DjbsJ5N/W5oxI0Pey4J00JBtX93FO3 76jGVNWyT5jD3chFwblcCWBPRLakcNcqOUADLbZzgTryB3w6fbx0RXTnhA3+zqQJ4tX4 sBqQ== X-Gm-Message-State: AOAM530BNHNOyz8+f+67jS49AFDnso8pm4d46zZBJVcZqZtYnFQY+6GR cV+chs/Kxf/aNJXCU9ZxouAbkWhmfS6Agw== X-Received: by 2002:a62:2a14:0:b029:263:20c5:6d8c with SMTP id q20-20020a622a140000b029026320c56d8cmr6569177pfq.23.1619814381375; Fri, 30 Apr 2021 13:26:21 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 17/82] target/arm: Implement SVE2 bitwise permute Date: Fri, 30 Apr 2021 13:25:05 -0700 Message-Id: <20210430202610.1136687-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper-sve.h | 15 ++++++++ target/arm/sve.decode | 6 ++++ target/arm/sve_helper.c | 73 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 36 +++++++++++++++++++ 5 files changed, 135 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 902579d24b..ae787fac8a 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4241,6 +4241,11 @@ static inline bool isar_feature_aa64_sve2_pmull128(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, AES) >= 2; } +static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index f65818da05..4861481fe0 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2377,3 +2377,18 @@ DEF_HELPER_FLAGS_4(sve2_eoril_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_eoril_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bext_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bext_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bdep_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bdep_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0922a44829..7cb89a0d47 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1220,3 +1220,9 @@ USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm + +## SVE2 bitwise permute + +BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm +BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm +BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 1de0a9bdc3..d692d2fe3d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,79 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_BITPERM(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + TYPE mm = *(TYPE *)(vm + i); \ + *(TYPE *)(vd + i) = OP(nn, mm, sizeof(TYPE) * 8); \ + } \ +} + +static uint64_t bitextract(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int db, rb = 0; + + for (db = 0; db < n; ++db) { + if ((mask >> db) & 1) { + res |= ((data >> db) & 1) << rb; + ++rb; + } + } + return res; +} + +DO_BITPERM(sve2_bext_b, uint8_t, bitextract) +DO_BITPERM(sve2_bext_h, uint16_t, bitextract) +DO_BITPERM(sve2_bext_s, uint32_t, bitextract) +DO_BITPERM(sve2_bext_d, uint64_t, bitextract) + +static uint64_t bitdeposit(uint64_t data, uint64_t mask, int n) +{ + uint64_t res = 0; + int rb, db = 0; + + for (rb = 0; rb < n; ++rb) { + if ((mask >> rb) & 1) { + res |= ((data >> db) & 1) << rb; + ++db; + } + } + return res; +} + +DO_BITPERM(sve2_bdep_b, uint8_t, bitdeposit) +DO_BITPERM(sve2_bdep_h, uint16_t, bitdeposit) +DO_BITPERM(sve2_bdep_s, uint32_t, bitdeposit) +DO_BITPERM(sve2_bdep_d, uint64_t, bitdeposit) + +static uint64_t bitgroup(uint64_t data, uint64_t mask, int n) +{ + uint64_t resm = 0, resu = 0; + int db, rbm = 0, rbu = 0; + + for (db = 0; db < n; ++db) { + uint64_t val = (data >> db) & 1; + if ((mask >> db) & 1) { + resm |= val << rbm++; + } else { + resu |= val << rbu++; + } + } + + return resm | (resu << rbm); +} + +DO_BITPERM(sve2_bgrp_b, uint8_t, bitgroup) +DO_BITPERM(sve2_bgrp_h, uint16_t, bitgroup) +DO_BITPERM(sve2_bgrp_s, uint32_t, bitgroup) +DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) + +#undef DO_BITPERM + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 509b3bc68c..da2f67fb67 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6249,3 +6249,39 @@ static bool trans_USHLLT(DisasContext *s, arg_rri_esz *a) { return do_sve2_shll_tb(s, a, true, true); } + +static bool trans_BEXT(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bext_b, gen_helper_sve2_bext_h, + gen_helper_sve2_bext_s, gen_helper_sve2_bext_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BDEP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bdep_b, gen_helper_sve2_bdep_h, + gen_helper_sve2_bdep_s, gen_helper_sve2_bdep_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} + +static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_bgrp_b, gen_helper_sve2_bgrp_h, + gen_helper_sve2_bgrp_s, gen_helper_sve2_bgrp_d, + }; + if (!dc_isar_feature(aa64_sve2_bitperm, s)) { + return false; + } + return do_sve2_zzw_ool(s, a, fns[a->esz], 0); +} From patchwork Fri Apr 30 20:25:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429857 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp782686jao; Fri, 30 Apr 2021 13:47:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyLLIg78fH5uoxLg4WNqQAaF6oj2UurV1svAoRFOtxCfwta9hbG4SqfKkHklPcd3IbXFs5T X-Received: by 2002:a5d:8c89:: with SMTP id g9mr5365235ion.27.1619815672363; Fri, 30 Apr 2021 13:47:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815672; cv=none; d=google.com; s=arc-20160816; b=NxH0KOH9aFKxHPnuRrB7oXVk0nyGOKpBAvVtEEE2W3He3bjez8VPKMrbcgjbbPsSNx p2zDRUBcZK8SEYSED7Q1fmAD2scQc+CfpnOjzMj9yhJv/fpGwXvdfBDg3Qr8g0FJ72mg ARLwyvWLAkuDnSDaXc8n30aQiM6DvgijHUig5ztgtE52AJ2CBjLUcRr/UZ3ki038DEep iWfuZRtnZVdWL/jZP7YJ1fKVa6xugKirvRuzbUf6SkHXDB45p/djHQ0Q9SESxFHLseB9 vN4/YmpfMftcw2cT96O1EUpl/kab93md8dCS3Y/86ZGhDmL6ZrlhuYOx1mcxTw57AzPn W+UQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=WTL/vHzY4yERtp37FuFizR3OWzxspZnRYY7Ua6fJdhFSH9+6ZznW/mIlhjxSvQ0Kga SPX3x4qMJpGglqobeeeN/c3i1lr+DNoygNsZELduMsKmtj3NBEKFXK+/v9Tgtosk14rl Rkz46O9BnbcPqQBFDnP7Oes2LTaDMr7A0mLlrR2NrWe0e+iUM1Xe3WmKXA6qGN8jMuDQ l30yPAWlbhlVOrqZvICf9dTNa8mAJqiumyj4bwqQwDcnL0Gpba+5Uop1Dct94mqUKmgo CigBwkMEZoOqg9KnWwZKRc3FXW2Bu9d5FQwtSPGs+3vGqj5B6KmTqNQDELdPwYIQ4syb S00w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i2fK8Tjw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r1si3373574ilb.86.2021.04.30.13.47.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:47:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i2fK8Tjw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56444 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca3D-0008JZ-Ml for patch@linaro.org; Fri, 30 Apr 2021 16:47:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51956) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZig-00006C-9w for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:39 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:36519) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiR-00064o-Ly for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:38 -0400 Received: by mail-pl1-x62c.google.com with SMTP id a11so6346605plh.3 for ; Fri, 30 Apr 2021 13:26:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=i2fK8TjwP6uYbhNmwc5I6y/pgBRv9UunHbJ/uw6M59HLSl8TA4ueHyB133fvFF8rOl pNm1tgr6FOhLg8tgPDXUn3v4I5JgkJS2fXgxgWFo1MWgBY+XFWIwMDMeha7SfndOs1w9 YaAbXjjvD5v8qANRGwKHMyV5EH5IqFQeec4ez4j0ljGti2ZAlLnvf8tk32g8py4wc6nF ybNwR2BZqjkzOA/yLhUEVnTbPjHLJkhC4HsSjhtACmsDtgcg2kwP63+z/FqOJoW0qERm wICM0qtsbMg3XY6VOyMuLI1lz01tIwjShxNx0ZZkCNdrwTT1KxEAd1hfbDmMONvy3znw ix8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=xiy3OFu7y9n0lQLi1Ms0ijwf8zujYC8QTzkyT7iwC1M=; b=aenn3oSLkj5WOQSrzqj7Vo7GTDAkdB9NpuEZJgMt5jJrZNcTIqralRdeL+bdOvtNuS v+KyzIlTC96+6SBAT5msdz/7kffG0jDO2w7YyvGnn4XWjpC4muGj/D2eBCoBQVWLnKPa t6ANIbYcWZ3ROk3tzg8axqVROdrkeEZm+kuxIhqXv59cTLaQ0bnmFDK1myIegEosY97z J8yLJGP2cMh1mBcAMefvJhllAi4Neyo4OIPNyYOZS/ZgC8qDB7hSBez++DcEsDGpuwnS vezkCQ/e6ql6UTdMcDWJ65sMhB4pzSvfRcD8Z++jTiV9eR+Td6wt7ZXcK2DiSEPdUDHN Nf/g== X-Gm-Message-State: AOAM531Wq8WBTbkni0mXsfvhUIeWqULTsyBIe/mq292P7UO4OgXgmmAw mk5pxfcLWZQCDuLUg+Kg/rpIZo4e/AJtwQ== X-Received: by 2002:a17:902:ed52:b029:ed:2f0e:eac with SMTP id y18-20020a170902ed52b02900ed2f0e0eacmr7107370plb.47.1619814382059; Fri, 30 Apr 2021 13:26:22 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 18/82] target/arm: Implement SVE2 complex integer add Date: Fri, 30 Apr 2021 13:25:06 -0700 Message-Id: <20210430202610.1136687-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix subtraction ordering (laurent desnogues). --- target/arm/helper-sve.h | 10 +++++++++ target/arm/sve.decode | 9 ++++++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 31 ++++++++++++++++++++++++++++ 4 files changed, 92 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4861481fe0..c2155cc544 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2392,3 +2392,13 @@ DEF_HELPER_FLAGS_4(sve2_bgrp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_bgrp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_cadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_cadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7cb89a0d47..7508b901d0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1226,3 +1226,12 @@ EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm BDEP 01000101 .. 0 ..... 1011 01 ..... ..... @rd_rn_rm BGRP 01000101 .. 0 ..... 1011 10 ..... ..... @rd_rn_rm + +#### SVE2 Accumulate + +## SVE2 complex integer add + +CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm +CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm +SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm +SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d692d2fe3d..2e09c3e55b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1314,6 +1314,48 @@ DO_BITPERM(sve2_bgrp_d, uint64_t, bitgroup) #undef DO_BITPERM +#define DO_CADD(NAME, TYPE, H, ADD_OP, SUB_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sub_r = simd_data(desc); \ + if (sub_r) { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = ADD_OP(acc_r, el2_i); \ + acc_i = SUB_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } else { \ + for (i = 0; i < opr_sz; i += 2 * sizeof(TYPE)) { \ + TYPE acc_r = *(TYPE *)(vn + H(i)); \ + TYPE acc_i = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE el2_r = *(TYPE *)(vm + H(i)); \ + TYPE el2_i = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + acc_r = SUB_OP(acc_r, el2_i); \ + acc_i = ADD_OP(acc_i, el2_r); \ + *(TYPE *)(vd + H(i)) = acc_r; \ + *(TYPE *)(vd + H(i + sizeof(TYPE))) = acc_i; \ + } \ + } \ +} + +DO_CADD(sve2_cadd_b, int8_t, H1, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_h, int16_t, H1_2, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_s, int32_t, H1_4, DO_ADD, DO_SUB) +DO_CADD(sve2_cadd_d, int64_t, , DO_ADD, DO_SUB) + +DO_CADD(sve2_sqcadd_b, int8_t, H1, DO_SQADD_B, DO_SQSUB_B) +DO_CADD(sve2_sqcadd_h, int16_t, H1_2, DO_SQADD_H, DO_SQSUB_H) +DO_CADD(sve2_sqcadd_s, int32_t, H1_4, DO_SQADD_S, DO_SQSUB_S) +DO_CADD(sve2_sqcadd_d, int64_t, , do_sqadd_d, do_sqsub_d) + +#undef DO_CADD + #define DO_ZZI_SHLL(NAME, TYPEW, TYPEN, HW, HN) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index da2f67fb67..c594c59954 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6285,3 +6285,34 @@ static bool trans_BGRP(DisasContext *s, arg_rrr_esz *a) } return do_sve2_zzw_ool(s, a, fns[a->esz], 0); } + +static bool do_cadd(DisasContext *s, arg_rrr_esz *a, bool sq, bool rot) +{ + static gen_helper_gvec_3 * const fns[2][4] = { + { gen_helper_sve2_cadd_b, gen_helper_sve2_cadd_h, + gen_helper_sve2_cadd_s, gen_helper_sve2_cadd_d }, + { gen_helper_sve2_sqcadd_b, gen_helper_sve2_sqcadd_h, + gen_helper_sve2_sqcadd_s, gen_helper_sve2_sqcadd_d }, + }; + return do_sve2_zzw_ool(s, a, fns[sq][a->esz], rot); +} + +static bool trans_CADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, false); +} + +static bool trans_CADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, false, true); +} + +static bool trans_SQCADD_rot90(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, false); +} + +static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) +{ + return do_cadd(s, a, true, true); +} From patchwork Fri Apr 30 20:25:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429853 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp780491jao; Fri, 30 Apr 2021 13:44:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwNYEoDlzoVNA3k7+n4fyFs/6CwzFn4gnwsJrabUH91/VrG30WaTHyGFKYYTzyXID7OoIeP X-Received: by 2002:a05:6638:2643:: with SMTP id n3mr2350884jat.104.1619815460264; Fri, 30 Apr 2021 13:44:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815460; cv=none; d=google.com; s=arc-20160816; b=Lj0nvcWSizIuL36Yna2ghAN16yqIvS3Z/5jBV/qEDW4vR2dG4WMH3IyoWwr1j2mGaH 51FiFyAjBfkNEFgbsChVXPSH7D+k1i9fdvrZW2zZsCCgC999W3Kt0qGP+/LAR5ruNPcu +ZsrxdGNpfIEd+QMxWJ+H1MBKD57tvVi7S9i5iNwko104DF2/AddG3A3/TwXVhWxNy0k y/JtQ2Pd8jhEDC7wqOb12YDvVZX8JgrV1HvZaYwY5LCdaC3c9TfWfsAGciiP+EZbYMSM Z0+YPHV9UGTweNRvFh3jPI+FM+8Wxiq2wXnbTuL233+5diQrN7522ynNonLu0J0Kv/3E EyEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=fD+WeGOx1ZnG+h7VQTi9xlJhr000RnU2F5nOmZrqJ3pA/raaGQfCdd7WKckz2T3SVM YIFvhycQ985zFukkioMDjCj63PZYxxENOcdEvGoZxKdic+5OWDe4BpdkW+NUdV+q04Yr TTX2c0AQdW4nX4mqJP8Sj76FhS4r2+PrAFaY9yEMYN5dAMbxSUrTgyjDYcCwAeEbIBZA BuohlvTCiqEdyj1y812jROmRNTXoV3pv+BL4N9swmeGTluVY6hNDpYWE6KmFWa1t2p5s CXjXjgupA5zuITgOEuY8VbJN07fy95DrTV0ydCKMrJpNd740OQkbRIgZIYQVyhoTpiJW kbWw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SMmdK9fO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t10si4087534ilu.91.2021.04.30.13.44.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:44:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SMmdK9fO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZzn-0003yG-L7 for patch@linaro.org; Fri, 30 Apr 2021 16:44:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52058) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZil-0000CK-8i for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:43 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:51117) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiS-00065G-3R for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:42 -0400 Received: by mail-pj1-x1031.google.com with SMTP id md17so9636735pjb.0 for ; Fri, 30 Apr 2021 13:26:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=SMmdK9fOkuJImFrSh7pN0lLnuPymMMYaIPqlGkj0QxHw6FPjHGgclriG8WvhiYo079 SgTmmFDdNt7GSEhV3rS2WMBKguDf64Vvp9lHgXbkYinjjmtWQWoVnD6BLwEmmWWhRpAP MWl5+0saB8kYu2eVMR790IXvLam6ETqXnOOHMsBfv+WSkUv3kkfL96G3cw0tiogzmFWt Cal/QbKifdk7yco2fHhzYYiv4D6+tEzGlwIWJp8jfEDg4gu2++h7tB1KE2uNJamtPN4E 3HVw/cL2dO0644lrFG+zy4IgbxBiwkAY1LaJf9HF2dTXkPvh7nnvYNZJCy86fubFGVhv LXLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=85QbQRD/kjbvj7H0EZGYuJDct9yXvao73Ho7HNCSiAg=; b=pFGIZJS74+hMfzATM02tP/Jilrfxy49Qi0olKtzNDYkxzltlUCZeKolghwYhKmmUWw qgMbCf7ili/GaYWJaXvGvHVSLfcpJql8uaGlsQIjUGMmCkDPE9Kc8CkySUv+CUk7V2xZ vmwbeVgHg0gVXl7Z3zvvdHWBr+mIOWZurWAf64LFRKy4GNoMDRFn7F1iqdKP97YkCYzC 6erXJYs2gPlcNk850fVVQuModyixCeCNmZ9gJavC2nXBTQnYPfFF2qdZvioyG4ce9ALC Y3R+7g5liYhJOIigT0+iZA6OnWsukyL1xQf1vJLP0uRYlcMAUKVM+pcJ+s94L3NUgZVi vAVQ== X-Gm-Message-State: AOAM533EVNy0FN5zC1I5Q0U0GQ+XEzhKZSSySU8VwZFxUHx+omiym9Yt rB/dsZQx/YAQYmHm5p9NtAu6Xo40Eysvrw== X-Received: by 2002:a17:90b:3884:: with SMTP id mu4mr4600727pjb.150.1619814382642; Fri, 30 Apr 2021 13:26:22 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 19/82] target/arm: Implement SVE2 integer absolute difference and accumulate long Date: Fri, 30 Apr 2021 13:25:07 -0700 Message-Id: <20210430202610.1136687-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix select offsetting and argument order (laurent desnogues). --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 12 +++++++++ target/arm/sve_helper.c | 23 ++++++++++++++++ target/arm/translate-sve.c | 55 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 104 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index c2155cc544..229fb396b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2402,3 +2402,17 @@ DEF_HELPER_FLAGS_4(sve2_sqcadd_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqcadd_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_uabal_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7508b901d0..56b7353bfa 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -70,6 +70,7 @@ &rpr_s rd pg rn s &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz +&rrrr_esz rd ra rn rm esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -119,6 +120,10 @@ @rdn_i8s ........ esz:2 ...... ... imm:s8 rd:5 \ &rri_esz rn=%reg_movprfx +# Four operand, vector element size +@rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ + &rrrr_esz ra=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -1235,3 +1240,10 @@ CADD_rot90 01000101 .. 00000 0 11011 0 ..... ..... @rdn_rm CADD_rot270 01000101 .. 00000 0 11011 1 ..... ..... @rdn_rm SQCADD_rot90 01000101 .. 00000 1 11011 0 ..... ..... @rdn_rm SQCADD_rot270 01000101 .. 00000 1 11011 1 ..... ..... @rdn_rm + +## SVE2 integer absolute difference and accumulate long + +SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm +SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm +UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm +UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2e09c3e55b..4871e90d9b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1241,6 +1241,29 @@ DO_ZZZ_NTB(sve2_eoril_d, uint64_t, , DO_EOR) #undef DO_ZZZ_NTB +#define DO_ZZZW_ACC(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + intptr_t sel1 = simd_data(desc) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel1)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, mm) + aa; \ + } \ +} + +DO_ZZZW_ACC(sve2_sabal_h, int16_t, int8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_s, int32_t, int16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_sabal_d, int64_t, int32_t, , H1_4, DO_ABD) + +DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) +DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) + +#undef DO_ZZZW_ACC + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c594c59954..6ac50fd61f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -163,6 +163,18 @@ static void gen_gvec_ool_zzz(DisasContext *s, gen_helper_gvec_3 *fn, vsz, vsz, data, fn); } +/* Invoke an out-of-line helper on 4 Zregs. */ +static void gen_gvec_ool_zzzz(DisasContext *s, gen_helper_gvec_4 *fn, + int rd, int rn, int rm, int ra, int data) +{ + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + vsz, vsz, data, fn); +} + /* Invoke an out-of-line helper on 2 Zregs and a predicate. */ static void gen_gvec_ool_zzp(DisasContext *s, gen_helper_gvec_3 *fn, int rd, int rn, int pg, int data) @@ -6316,3 +6328,46 @@ static bool trans_SQCADD_rot270(DisasContext *s, arg_rrr_esz *a) { return do_cadd(s, a, true, true); } + +static bool do_sve2_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, + gen_helper_gvec_4 *fn, int data) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + } + return true; +} + +static bool do_abal(DisasContext *s, arg_rrrr_esz *a, bool uns, bool sel) +{ + static gen_helper_gvec_4 * const fns[2][4] = { + { NULL, gen_helper_sve2_sabal_h, + gen_helper_sve2_sabal_s, gen_helper_sve2_sabal_d }, + { NULL, gen_helper_sve2_uabal_h, + gen_helper_sve2_uabal_s, gen_helper_sve2_uabal_d }, + }; + return do_sve2_zzzz_ool(s, a, fns[uns][a->esz], sel); +} + +static bool trans_SABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, false); +} + +static bool trans_SABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, false, true); +} + +static bool trans_UABALB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, false); +} + +static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_abal(s, a, true, true); +} From patchwork Fri Apr 30 20:25:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429844 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp774975jao; Fri, 30 Apr 2021 13:35:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzeOba1nOpVtB+o9uT4NrbopCGe9Saw8Kc/TEWBHNn+XyCGoSvbxxxlKAZmXd98KfQT1rdn X-Received: by 2002:a05:6638:134c:: with SMTP id u12mr3278851jad.5.1619814937496; Fri, 30 Apr 2021 13:35:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619814937; cv=none; d=google.com; s=arc-20160816; b=x0ptRFeYVHCrj4gDOvjhJak7imXGdyUwgr3y94ArpBb5rSmP9L2xTJF7b9rb3Wkom2 a+6A0OBs+YwtRqb5gUFtoUnjAjs2SmeUsFbrMWmBq8EM0+Zq0JDfS5ZCaOZXT1uYgSfh sYhQo9WY9rGvnZvpcX8CadhSAYVyBi6sI6JJ+8gish0TZd6wvh2qrFjcggOXSnKCQkqb DuChtmu2/yC7Oaoeoz68ZRXotIMhL4hTOmlz8hT8XPwP3XBZJjeyqV0bvDL9CdgJnosK o3H6TJhdbarTEOyWXtJ7VlX6O1Mq49IcLRS7XGjewuFOiIdhNPcA2Rb5hFBXL23X8W75 o7bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=cCdhmGkpiu6BdX+8BjTp8NqC8VumQkDROiUFvOlhcoxgbWQDZv118Xp/zlR7i2UcL+ j8sNOa2xx9bqKp4XbwUVh4j0Iz6IkLzgvxRNfmZfYysx3lVDaxybZUb46csFdJzUdBTW qxIm42VKnuPtHS8wVbsZm8VPkBKpVMtzBDYleBBcqJ+K+g6GeAH0uctmBwHmbJ/ZMmMQ yIrDrGWKsoTi8iqPNAqOuGjzeHYe+Y0OBf0jm0qBfx/+jFn0I5Ms68yKle1PI/FTLXRy U9EmouLF32HLvde9JCNFCpADDHSglD2UwQKwYMrN0rM0do6fgPTCVd1KkwnPaZFfApb2 bdCg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S59PrQZQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f14si3135084jav.125.2021.04.30.13.35.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:35:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S59PrQZQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZrM-00018N-O8 for patch@linaro.org; Fri, 30 Apr 2021 16:35:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52002) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZii-00008m-2V for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:40 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]:37843) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiS-00065q-Vw for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:39 -0400 Received: by mail-pf1-x435.google.com with SMTP id b15so5008792pfl.4 for ; Fri, 30 Apr 2021 13:26:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=S59PrQZQSC77IZgLEIpErMEf0MkIrdaF3uE0u9P5yqJgKeFariqFO+1+AcfE/WVxkQ EsrkpDNw8J0XtT/a4u5FcVxGydOt/fuoFG0lWo+kyeBbA4awACs0tZfqRpsAQIg7CAkm zMoIn/9skax9d+rRXTG3KVuOtvVHnpm9ZWdHtacz57iKnnIKytWPVqreQeoU/1QlTtXI bMO4Z0SQ5vn9rK/lVR6nwfLoL6EqNyEtLGlsU71OhzNb03lr+LjBBz/d31JmSpwe7nAS blHZ9C6Tcz98Ae9SnD9cs3i6XyMb40HVf/eO7hM1Wga7auIJEbzvsNpfq+aoVyEy2/7b Ho5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1T/TF/O8cuPuJru/dQDvvvNN7AjjLNborpcatweuNPQ=; b=Xxw8qy6y+SmeFM3x/5QXLd1n0Uq4q3GPUSX76eh1+Ii717bEGu3VCnFuSAbqH9mDJ7 9XaZzjpGAc6/6g3fWrgQZ8wL8IofMKvcbUktxlJ0sDS2uPcDRkkfGl/JrGeDjFeWOeZH KJaxqkneYDttEtJ3ZUeoWSnDG84Zy3tZtXb1EKPw+XSNUJXaKTrvL4H4iqPL2JH3rCSl 8cW1y2dZd/OXOjOYu1PWgLnaUNfejvqtUfGYCMEXSHniO/cafWrwFhYwje8Cb6SjT7k/ HwIznwWdFPL4W9a8ikeLUYSov31yFFjEBxDCrHAvLgANJaVL1EEPDdBBvkT5y6H8pc40 whLQ== X-Gm-Message-State: AOAM530xLMDCQUxoS6F+cw5Os6PBy9LIzuKPGUBFOiBTYYhXjn0yDY8k 6CHKQrwE0+LtHcZzc9YT/yyiWqRdjVF5RA== X-Received: by 2002:a05:6a00:a95:b029:25c:eb77:723a with SMTP id b21-20020a056a000a95b029025ceb77723amr6410804pfl.23.1619814383527; Fri, 30 Apr 2021 13:26:23 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:23 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 20/82] target/arm: Implement SVE2 integer add/subtract long with carry Date: Fri, 30 Apr 2021 13:25:08 -0700 Message-Id: <20210430202610.1136687-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix sel indexing and argument order (laurent desnogues). --- target/arm/helper-sve.h | 3 +++ target/arm/sve.decode | 6 ++++++ target/arm/sve_helper.c | 34 ++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 4 files changed, 66 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 229fb396b2..4a62012850 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2416,3 +2416,6 @@ DEF_HELPER_FLAGS_5(sve2_uabal_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 56b7353bfa..79046d81e3 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1247,3 +1247,9 @@ SABALB 01000101 .. 0 ..... 1100 00 ..... ..... @rda_rn_rm SABALT 01000101 .. 0 ..... 1100 01 ..... ..... @rda_rn_rm UABALB 01000101 .. 0 ..... 1100 10 ..... ..... @rda_rn_rm UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm + +## SVE2 integer add/subtract long with carry + +# ADC and SBC decoded via size in helper dispatch. +ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm +ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 4871e90d9b..0049ad861f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,40 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = H4(extract32(desc, SIMD_DATA_SHIFT, 1)); + uint32_t inv = -extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint32_t *a = va, *n = vn; + uint64_t *d = vd, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + uint32_t e1 = a[2 * i + H4(0)]; + uint32_t e2 = n[2 * i + sel] ^ inv; + uint64_t c = extract64(m[i], 32, 1); + /* Compute and store the entire 33-bit result at once. */ + d[i] = c + e1 + e2; + } +} + +void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int sel = extract32(desc, SIMD_DATA_SHIFT, 1); + uint64_t inv = -(uint64_t)extract32(desc, SIMD_DATA_SHIFT + 1, 1); + uint64_t *d = vd, *a = va, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; i += 2) { + Int128 e1 = int128_make64(a[i]); + Int128 e2 = int128_make64(n[i + sel] ^ inv); + Int128 c = int128_make64(m[i + 1] & 1); + Int128 r = int128_add(int128_add(e1, e2), c); + d[i + 0] = int128_getlo(r); + d[i + 1] = int128_gethi(r); + } +} + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6ac50fd61f..6f5e39b741 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6371,3 +6371,26 @@ static bool trans_UABALT(DisasContext *s, arg_rrrr_esz *a) { return do_abal(s, a, true, true); } + +static bool do_adcl(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_adcl_s, + gen_helper_sve2_adcl_d, + }; + /* + * Note that in this case the ESZ field encodes both size and sign. + * Split out 'subtract' into bit 1 of the data field for the helper. + */ + return do_sve2_zzzz_ool(s, a, fns[a->esz & 1], (a->esz & 2) | sel); +} + +static bool trans_ADCLB(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, false); +} + +static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_adcl(s, a, true); +} From patchwork Fri Apr 30 20:25:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429862 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp784682jao; Fri, 30 Apr 2021 13:51:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqRY/OTcI2Ra0x4jY7iobsJu8Wb+HWE1UIXpJqffqGgI1o9cHUTHxSf8iyTCsQA9HiI4i9 X-Received: by 2002:a02:ac05:: with SMTP id a5mr6825320jao.89.1619815866422; Fri, 30 Apr 2021 13:51:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815866; cv=none; d=google.com; s=arc-20160816; b=ahTaoXMBprUwpaGpfBriZhXtepHXW3y6rinGi1hHP+wywWbs93n01pPVUi0Z4aKp3k ZQ4smvFbgGZrlGYgrb5aRJcrS+tVBxWolg7hMOIkA1blHNJB4OFZSplJC79rz0c0H+VS SMHM2DEsC/2pbSjzURQUA6GAHox4U/IMGGEE1SoZA4vW5qWczzJAfTUxUTUwCIGSzkad +FhBVWH83XeO9Bxd95vPiTXWC1dTITviMv5anZs8Mx7c33UO5iDHx7nBeIVOn3MKq207 TfpKNVolBds+Jl2puUuc0P4DsXQHF8UnC2xRiCEYm7OexZZkGefScJm6lc/jUFKSz/sK p0hA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=S1xBK6IsRZFDAmktGKF6mcZVmo/0Rf0QAlx0E8xBelieQPH5SnsyjmQKF6TTOIfxB0 YHKtNGFm2WkgRkpONUbrOCUVQlp9qLIGr975K8TxTN0sfRVFydGkFB1BdHsQ4yx8ZPym jz0FcWoCipt5XIfo7f5ltfsFzOX6gceZEreHQveU6C924PdB5pMZp6/ykUFePJUU+oR0 yiQ2/XztvHM/Lqk3fAz4nNZCp2TsvVW5GdOD9r88wANh8YD5I4GIGaQ9ArJbN0AF1YrO 4swIDzjrtD6jPvbsZac5AmzZ4U46uSHyOqUr3W9XTPaMsQEAisMITFnSgNopgux/6W5G 6myg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ej9WJAx7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w2si3174347jaq.45.2021.04.30.13.51.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:51:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ej9WJAx7; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37608 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca6L-0003nn-Q9 for patch@linaro.org; Fri, 30 Apr 2021 16:51:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52092) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZin-0000IP-Fm for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:45 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]:34690) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiU-00066J-0l for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:45 -0400 Received: by mail-pl1-x636.google.com with SMTP id h7so8664656plt.1 for ; Fri, 30 Apr 2021 13:26:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=ej9WJAx7rkgoiWLLxSA7csW1oED2yhDxmuSX3Ns9eo/Z8USq5srHkRxOsSZt2CXfdT F58rbEI2uh42SgmF0l4O2pC0VWl6H284qCFhUvXYxgIHebOkQw5/Y5dKHsmRfHzoVZ1C 51zM9Ywc1Sl4JHpOM5FkCkEag1TWC0xBpScNy9uPo6AuP/fSaybbhNeCJ3jPUkdqTYP6 dC+f4DC9lQAvw5ybgJXkh4y4B9a08F3l6WQhaky4ac/ME7dB8VGOCtiuU3cLdiVVs2hZ KAA1cGROn9byUpHZ+/VdMzp5iFuaY1Sp3lQFSoCAcFXhaYD39WEsKo6bqKHKepbAU2Kl C+/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=KjnryLyEOHY0qpMOVIlDWjkao3wWlEhJ3Rqkl3sc8kg=; b=aY0EZ4PHDnb75OP//SsuxhrpR1+TLY02F0LybwpPP8Fva+CPXptQqdsSJ2PhxCx8El 5rNlIb0pTmeRB8hT+WG6C2hS0TpzANh8DWSjm4llGHgQqNG4aHIajlZvQXMKo9+8NoxA I15XsBQ+oDfg7rFW77KUWXdufcAa5jMxE5Ix5QzWc3U3t8sweeL9rgD9y0kz/EmWZ7Nl LbFW/znhD4QyhMCVLrY7pe9IuXkYyBJYCdqDCk61QDOfVAl9YqBXUWPah4q8bwfRpHKb ye3E+qH5dDOpRcNtd5pPFBu67Pt6aehLJ5WUMQBtiqWzw6cxpESQANwIbx1wH7tobOWU We9g== X-Gm-Message-State: AOAM5310LtldxoljX5e3vb31SfCf+leqLpwSTjBwfGhdRpbtfJP7kDbf 6V4pYDbVaE+I3D0lMYiOmo6INYcyHFUXNA== X-Received: by 2002:a17:902:7444:b029:ed:5334:40b6 with SMTP id e4-20020a1709027444b02900ed533440b6mr6873726plt.35.1619814384223; Fri, 30 Apr 2021 13:26:24 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 21/82] target/arm: Implement SVE2 bitwise shift right and accumulate Date: Fri, 30 Apr 2021 13:25:09 -0700 Message-Id: <20210430202610.1136687-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++++++++++++++++++ 2 files changed, 42 insertions(+) -- 2.25.1 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 79046d81e3..d3c4ec6dd1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1253,3 +1253,11 @@ UABALT 01000101 .. 0 ..... 1100 11 ..... ..... @rda_rn_rm # ADC and SBC decoded via size in helper dispatch. ADCLB 01000101 .. 0 ..... 11010 0 ..... ..... @rda_rn_rm ADCLT 01000101 .. 0 ..... 11010 1 ..... ..... @rda_rn_rm + +## SVE2 bitwise shift right and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr +USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr +SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr +URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6f5e39b741..c11074cccc 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6394,3 +6394,37 @@ static bool trans_ADCLT(DisasContext *s, arg_rrrr_esz *a) { return do_adcl(s, a, true); } + +static bool do_sve2_fn2i(DisasContext *s, arg_rri_esz *a, GVecGen2iFn *fn) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned rd_ofs = vec_full_reg_offset(s, a->rd); + unsigned rn_ofs = vec_full_reg_offset(s, a->rn); + fn(a->esz, rd_ofs, rn_ofs, a->imm, vsz, vsz); + } + return true; +} + +static bool trans_SSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ssra); +} + +static bool trans_USRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_usra); +} + +static bool trans_SRSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_srsra); +} + +static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_ursra); +} From patchwork Fri Apr 30 20:25:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429854 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp780597jao; Fri, 30 Apr 2021 13:44:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzbgEUxmO2rBMQFTlZVvbcpnSbl1VxU3zlzlh6BZWH0uGOzMTYB3O94lSm+gMuAQlKEowL/ X-Received: by 2002:a02:918d:: with SMTP id p13mr6807246jag.51.1619815474176; Fri, 30 Apr 2021 13:44:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815474; cv=none; d=google.com; s=arc-20160816; b=M15HFFTiFdPgaTAIqaJ0VftvJ04gHXvgewQtK7kyqHGX4K725GASLLq7mk9rSJ/375 y+Rc3v4snkYKIrZfa37CBlpze6mq/cV735HcciJ51NKTBUQdVYrUqz33SVKaBxGphgWh eq3Ci9HLk1ZSFVAZIgXyEf1cHe5XwVjIT394wmH1hCkwERbVoFDReRhkfcHMW3ApdAbx Gfh0vq5Q45qtbVNHRQG1ijLKpoP+Mwyky+hUH1goTKNzuImVRN8Fz0G077j+FggscMkO UmpGWFGpZJR3yi9K2vNkE6553HLg8F5/Fg98N+sz6RZ7PtPo1KjHtJuNiG8ZRw+7OE1l +fAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=wQAtYFbssErrSUn6itbM2O5ZV7hufMCAkUEESOTI2VEtqwKZNXDqz1mzmLVQH2WFLP jKIcuoVuvlw9eE1AM43sBJGZSRCJ0KxKOS8hZqaeglPK2egdDPfEhBrI12J5Mv4oK0XF ETy0HwQbDi5kTVN2Ii7mUHKDwAa9RT2ePPixK8LQlO2Tk+i5pJ4CADXtwyoiKvi7YE8g yXFKfFSIWxHzwJWvqZ1w0AKck3m9cjQqA6NdST37s14nJaYrKEQW19MPI9CtTIDcZW+S pdjBXO3PS8oWvXp6pXKuQHdGbP7ISRl/DJRT7ggPC6iRBDbLhb498B1Qxvt3cnQRS4+V fF2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wklgb60D; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y9si3596167jar.97.2021.04.30.13.44.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:44:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Wklgb60D; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca01-0004ZE-Id for patch@linaro.org; Fri, 30 Apr 2021 16:44:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZij-0000Aw-EP for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:41 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]:38800) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiU-00066r-2y for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:41 -0400 Received: by mail-pl1-x635.google.com with SMTP id y2so2972885plr.5 for ; Fri, 30 Apr 2021 13:26:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=Wklgb60DNZIETuk5vMn+bFI6sm6bBUF3KJrs/PpOZd7A5d12r5jrtVrJ8dldNtUq5r w+NS53iGNXsm+phAde55zRYBegkBa2JbRWz12jFvH3Z89F82fS0Z9dXJmLQhpvNyc8te Im1j7j1Vpq26b7cxtdh99QywTODFkVMkhXSVveWRYpi4h/+U2yrBJ0mCnc5gbnkKMdmr 0EuRFxG+kdSn7zNxdAy8rA3iX/L9JhI1pgMxF+zpsfofSk6wYUF89hGWM5QInm8Tyg9g X1hGfJyePzosT3Mwh1p7edTWzDuU13W+xWOlYN/TVzAWYcMI8PPxaJLF/eE6s5WoIdvj Tr7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=XWdhkzqHe8iJijTRujXmdCXfYmUKYPpZlGIVkZOqBWQ=; b=dN+vnc66VAEFekAOvRWYjsnwvZ0Ve8PsIfWzGM9XWLxe5K5rvQlyPgZfL56Y32RVHJ nRc2LswNACrDEeMcogHXylBI9TjKD8eAi+cIFSmEeP+jZ1zig8FYdT2dX1dL9IfZOFkF ImmLLd+V8HfrcYTsxKvHIHb3XSBOZtZlWm13og2mckuFhVFHOu8XL/pOSvJlVNShrpPu 4d9COdMfjjuY7HRWVR7f6oeb/YOQGQg+J2i/ZYl89wq/ruFFCSRAC3SiY3ucfMxtQ5sa DlcwUsI3u3MhyXPdMjOrlZe/0Cqz7t581CSCtOKgt/yJi21L/qAFPKTfo5OslT8k/VqX nLNg== X-Gm-Message-State: AOAM5310QbwGYfT6dOBEEYJbzhVaVhIfrDaT49UGbqvi5HLtgwWEA8NK 9kVBn6nGraNbw41iIQ62yGjPvMBWuq2RNA== X-Received: by 2002:a17:90a:d184:: with SMTP id fu4mr16422647pjb.79.1619814384729; Fri, 30 Apr 2021 13:26:24 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 22/82] target/arm: Implement SVE2 bitwise shift and insert Date: Fri, 30 Apr 2021 13:25:10 -0700 Message-Id: <20210430202610.1136687-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 5 +++++ target/arm/translate-sve.c | 10 ++++++++++ 2 files changed, 15 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index d3c4ec6dd1..695a16551e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1261,3 +1261,8 @@ SSRA 01000101 .. 0 ..... 1110 00 ..... ..... @rd_rn_tszimm_shr USRA 01000101 .. 0 ..... 1110 01 ..... ..... @rd_rn_tszimm_shr SRSRA 01000101 .. 0 ..... 1110 10 ..... ..... @rd_rn_tszimm_shr URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr + +## SVE2 bitwise shift and insert + +SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr +SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c11074cccc..d74a15d8b8 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6428,3 +6428,13 @@ static bool trans_URSRA(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_ursra); } + +static bool trans_SRI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sri); +} + +static bool trans_SLI(DisasContext *s, arg_rri_esz *a) +{ + return do_sve2_fn2i(s, a, gen_gvec_sli); +} From patchwork Fri Apr 30 20:25:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429859 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp783052jao; Fri, 30 Apr 2021 13:48:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz3UXbMtsJG2RqhCQZE+matyEVau93p2gRN18psSYJoTutAASt8jf/mygC4m7PVnh+e52kM X-Received: by 2002:a92:c0c2:: with SMTP id t2mr5509182ilf.210.1619815707442; Fri, 30 Apr 2021 13:48:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815707; cv=none; d=google.com; s=arc-20160816; b=rPi5DmvhorYIYXxQ0xWqssKnZA6WLhWjXz6msKbSKRSUP8DQ0T76ytgX13QNPLAT6Y yOlLmLuKUxQBKBkPAlGqiokLhrnLUoksa1vQFjHc3XhZT5suUMvm+D5+vWC0UTlelduO rNn21Y6jlfa5T/hRkay6tqS6iM6DjmOYUXekjpqBzEzTDeukw8NSRWNvjJb0vCy1wS2X QvR2CzTlv+e0MLEph2UGI5uScRosasyc00fbK7tCql0kTcQJ3NY68ZEtPz6NFFrw3/Yf soj4WD/JwgmW6EqmYfxX+JlB4BycEypGu14ITVAaz0VthkjMf74QgI5KCNnv1azE5JGp sPjQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=otip/xE3tbXZQ5uWasjGcmzZxSSPnJRGIbV1kj9iHBDYtD3PTFqzHoEa1q8+4z5sVM kybHR+YVM+TX2ysl+ZbVOHnW4rlQiYwiHlRPBKJBE4iGOjaL3UkgvXDm7yYDM37AVOeV 3cOyuvTOqwQFoVeQ+2ZRKm637B0kMBrxf3iE9fP1vfofyhU7KTB32t36isc1Ihj5v2KX /glXQw59wZxq85ieWUXPInai8Mf5xySfNXCF9l3JPeAntmOERGqHqAPIeqYHaQD8L3oN 1/MypD3wO9E8F+dooTtzn+ZlcRWNL4h3pTV9lGEmABvD9ivAWiRGHh0ToRg2xzoVnakP QcNQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zT714f3+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h21si3161749jav.99.2021.04.30.13.48.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:48:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zT714f3+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56178 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca3m-0008BY-B9 for patch@linaro.org; Fri, 30 Apr 2021 16:48:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52060) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZil-0000Dt-Vc for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:44 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:33736) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiV-00067f-1L for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:43 -0400 Received: by mail-pl1-x629.google.com with SMTP id b21so8728419plz.0 for ; Fri, 30 Apr 2021 13:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=zT714f3+ug62paiM7iVNnjZHxdRNi+A16Zufeks93HeX70ekC/5lG4TWNzLIYAxd8I G+Gk7yZdIvIvF5ouxWmK2womzLYyGZzqv1Dxr5l8AM8Koo0aVnqIKJdnNOqwP9HPANeA f9q84dEUkS2SMMlK2tiJECAqtsIAqQCDKQeLlVpS7LyuGR6XhEDxZcEd+8FQEH42fHVF Jo9MXcD18oTw/dUlAPUIDcPvQtG7ghrgoALGx19J1sdGqm9CB4f24NVhIGxWN03MkdC/ AjcPTfUqC/6mWAOJtHQQvXNkyu8KCcYqMfQquJVM7CAhZAgGTVpUX5hHL5MyuL1CA/3h /lLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QaR3WAAAd20RRoi+x5aKUmW3qXGunVbXS8iCJuTuVr8=; b=EKltUuybuNZoeIlZrDyjnzsINQbkBEnwB48nOPv6gqrzZJw+v5fBFJ6CHd0BQAiaNK yNKofQr/bIsYCA+b3SwKFtnvp/aeK9j39YH/r1zXuHS+kXGE2EukM2XbP+OdqY1t8U9I ufoGtpFGcVvzzxO3EOqE7e3el6h14Eo65JGEBByJ5f5YxFzC+5MDsNdo5pb8q1K5Efko WXr6OeBGPjLae7znDBXdlZY9wt2IBQRLZINi/m+0cPxddDNV95YZBFUBXjheXp0Wfs5I mWddsxXqle6uKkRBHj3t6GJL3jevKZeaLMpa9vZLmI0IJPgwtKdi6H0iIjd1iHkmDIJn ZoVg== X-Gm-Message-State: AOAM532AUaxRdVVtIxUjuWV4KJV5+yOSmx40TRRWUwksniS/6L9RQ5/v ilcz6Csld4xr5MSI+pOYXhkaVjTAhk6GLA== X-Received: by 2002:a17:903:20ca:b029:ed:eb89:b441 with SMTP id i10-20020a17090320cab02900edeb89b441mr6996280plb.64.1619814385232; Fri, 30 Apr 2021 13:26:25 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 23/82] target/arm: Implement SVE2 integer absolute difference and accumulate Date: Fri, 30 Apr 2021 13:25:11 -0700 Message-Id: <20210430202610.1136687-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 695a16551e..32b15e4192 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1266,3 +1266,9 @@ URSRA 01000101 .. 0 ..... 1110 11 ..... ..... @rd_rn_tszimm_shr SRI 01000101 .. 0 ..... 11110 0 ..... ..... @rd_rn_tszimm_shr SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl + +## SVE2 integer absolute difference and accumulate + +# TODO: Use @rda and %reg_movprfx here. +SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm +UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d74a15d8b8..ba1953118b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6438,3 +6438,24 @@ static bool trans_SLI(DisasContext *s, arg_rri_esz *a) { return do_sve2_fn2i(s, a, gen_gvec_sli); } + +static bool do_sve2_fn_zzz(DisasContext *s, arg_rrr_esz *a, GVecGen3Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, fn, a->esz, a->rd, a->rn, a->rm); + } + return true; +} + +static bool trans_SABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_saba); +} + +static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) +{ + return do_sve2_fn_zzz(s, a, gen_gvec_uaba); +} From patchwork Fri Apr 30 20:25:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429852 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp779342jao; Fri, 30 Apr 2021 13:42:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwTPf8zEiDnC5h7wMC6vraFFvnV+XHSKHmfC29DAS4UZmN22vDY1gi6SoWC/4rT7HW5983L X-Received: by 2002:a02:ac05:: with SMTP id a5mr6794679jao.89.1619815343673; Fri, 30 Apr 2021 13:42:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815343; cv=none; d=google.com; s=arc-20160816; b=hV74jf+z95/lR49Orqftfjj7PLJnmhHBSRz3mmSMXZjLQOPmrSe3qlifcYAZLaElhe CyZmx4nCDQnC559kiDviPHh1qA6vpnw943nzBHm2NRrr2sxysHFHbxgJfDZHeWxlyZma aIBZORcWT+eMJaVHirA8z9B6ygmg6NyhNQkV/o71Yqw0LokZnj2dRoywwUvq+3AFVdFJ FqIa4bAPZ1xBZO96gvX+/HgwtDOlewbb5jWE5sAPjfcjnHwdOm2yB5SVnSAJ3ENEkP7Q 6y4vXRTeAuTIVXWrOmtEzdj2lrzWn5C92w/UINn+l0m1ml+CSBFJzffS1IQRg70VDqLQ 49Qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=OXF36fYrMZv/9zg+/GSZvCv3gSxC/TICzxniozZbF3sy5ypszNm8/LZmbbjP4Jp6bT NSIU3PW1DjSWjBaycYYzz7i/zrgDd0Z00hv7+yZluSYp+3A63wqUHZN8Kiay3bbx3gYd BN/oNlK2W+aErZHsw34hKrcf+Vv4CiWd/8doSlbcYzlwLvHGDgX6GF+yS9cUG7vHzdT/ BT4LGcRe8FkG9FxMjG6nUEwBRrekh2xo97weZfoMGbZr/8bnAoTOnk+bcRCLKa3EZYR8 Mku6pYfqv5bn45z05Nxya66oH6GbnBslDQWMwmA3Yzi6J/5NEtTPMiTGyHvMLOyDAoh0 z7ZA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qZz5qo7q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m6si3957975jav.15.2021.04.30.13.42.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:42:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qZz5qo7q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZxv-0000ih-0a for patch@linaro.org; Fri, 30 Apr 2021 16:42:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52100) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZio-0000K9-3u for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:46 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:44733) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiV-00067m-7Z for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:45 -0400 Received: by mail-pg1-x52b.google.com with SMTP id y32so50168143pga.11 for ; Fri, 30 Apr 2021 13:26:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=qZz5qo7qmWrUMnAtAQUpy/o1d6cjMTo3dfNxSEwkfLGN4/7XIh/W+7A63AbLwAJYRQ bKgqIgif7LZYqR4L88mi5RlILMHjtBAKpN4lHmqCAeZiiB18nSq9f4boGIFDdzWK1+7R 3RYfOoqMGwmpi2e4CVDhgyurzq4M2PsJiqlJUD+ed/RoDjA2SEq3vRqfuCXpsGy1liFf uwX1WMcbaisETbs2//WycEZg8O6HmpOPX4PnNIZ5lei5aA1mzZcHgH0PTjnWcoM8kGge iIv/0swPptgJcPVwtCGTRokRTVFH09NEF0WdlESvH5TwypBmzvZK2HayzGCqT0wNvK+x TGpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uNqnL76ppa/JptnkZAzi8U8bmGMNrJo5oaqSMO/4t5I=; b=StaGetddmj1YZsp4M/bJ74yQvhYJF/N0xCFFUSrVcETjUGUkBGxeBC0oPK32EPTmBZ TFccGs0JXCiZLE8UiiOnNX59d/86YVt8UMTojuHEWn71b4Sf/vq8C8bsyvE6xbEOwgVy z8WHGORlGqrjRcJk+ssDbvyGuwkN4dSI0JUKSqsWDXU9w1ITAzsy/mMkRDyowtSIeaBT CzfNdJGGC5i8Le6Z8nqOo7bpQTl3uCNu+kAkCWeDb43GfBSs9jZifP8mSsAoUvSnB7DZ XxeTJrh5A1T+AIlwwLMd/5x77EcUnug0Os6jfa2sJTdEU4rLicVMp4PCPsIsUeTiAI4C PASw== X-Gm-Message-State: AOAM530jGE2Z4MgpIH7HFTf7KBBDiyw8Wg1PVQFFkxgQTqjx0wTgvq7a C0zBlJR73u6GwfUlK+dUDrRbh1e+40u8Mw== X-Received: by 2002:a62:18d7:0:b029:28c:fdfa:f95b with SMTP id 206-20020a6218d70000b029028cfdfaf95bmr3758760pfy.57.1619814385820; Fri, 30 Apr 2021 13:26:25 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 24/82] target/arm: Implement SVE2 saturating extract narrow Date: Fri, 30 Apr 2021 13:25:12 -0700 Message-Id: <20210430202610.1136687-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 24 ++++ target/arm/sve.decode | 12 ++ target/arm/sve_helper.c | 56 +++++++++ target/arm/translate-sve.c | 238 +++++++++++++++++++++++++++++++++++++ 4 files changed, 330 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 4a62012850..b302203ce8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2419,3 +2419,27 @@ DEF_HELPER_FLAGS_5(sve2_uabal_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_adcl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_adcl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqxtnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 32b15e4192..19866ec4c6 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1272,3 +1272,15 @@ SLI 01000101 .. 0 ..... 11110 1 ..... ..... @rd_rn_tszimm_shl # TODO: Use @rda and %reg_movprfx here. SABA 01000101 .. 0 ..... 11111 0 ..... ..... @rd_rn_rm UABA 01000101 .. 0 ..... 11111 1 ..... ..... @rd_rn_rm + +#### SVE2 Narrowing + +## SVE2 saturating extract narrow + +# Bits 23, 18-16 are zero, limited in the translator via esz < 3 & imm == 0. +SQXTNB 01000101 .. 1 ..... 010 000 ..... ..... @rd_rn_tszimm_shl +SQXTNT 01000101 .. 1 ..... 010 001 ..... ..... @rd_rn_tszimm_shl +UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl +UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl +SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl +SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 0049ad861f..7dca67785a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1264,6 +1264,62 @@ DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) #undef DO_ZZZW_ACC +#define DO_XTNB(NAME, TYPE, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + nn = OP(nn) & MAKE_64BIT_MASK(0, sizeof(TYPE) * 4); \ + *(TYPE *)(vd + i) = nn; \ + } \ +} + +#define DO_XTNT(NAME, TYPE, TYPEN, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc), odd = H(sizeof(TYPEN)); \ + for (i = 0; i < opr_sz; i += sizeof(TYPE)) { \ + TYPE nn = *(TYPE *)(vn + i); \ + *(TYPEN *)(vd + i + odd) = OP(nn); \ + } \ +} + +#define DO_SQXTN_H(n) do_sat_bhs(n, INT8_MIN, INT8_MAX) +#define DO_SQXTN_S(n) do_sat_bhs(n, INT16_MIN, INT16_MAX) +#define DO_SQXTN_D(n) do_sat_bhs(n, INT32_MIN, INT32_MAX) + +DO_XTNB(sve2_sqxtnb_h, int16_t, DO_SQXTN_H) +DO_XTNB(sve2_sqxtnb_s, int32_t, DO_SQXTN_S) +DO_XTNB(sve2_sqxtnb_d, int64_t, DO_SQXTN_D) + +DO_XTNT(sve2_sqxtnt_h, int16_t, int8_t, H1, DO_SQXTN_H) +DO_XTNT(sve2_sqxtnt_s, int32_t, int16_t, H1_2, DO_SQXTN_S) +DO_XTNT(sve2_sqxtnt_d, int64_t, int32_t, H1_4, DO_SQXTN_D) + +#define DO_UQXTN_H(n) do_sat_bhs(n, 0, UINT8_MAX) +#define DO_UQXTN_S(n) do_sat_bhs(n, 0, UINT16_MAX) +#define DO_UQXTN_D(n) do_sat_bhs(n, 0, UINT32_MAX) + +DO_XTNB(sve2_uqxtnb_h, uint16_t, DO_UQXTN_H) +DO_XTNB(sve2_uqxtnb_s, uint32_t, DO_UQXTN_S) +DO_XTNB(sve2_uqxtnb_d, uint64_t, DO_UQXTN_D) + +DO_XTNT(sve2_uqxtnt_h, uint16_t, uint8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_uqxtnt_s, uint32_t, uint16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_uqxtnt_d, uint64_t, uint32_t, H1_4, DO_UQXTN_D) + +DO_XTNB(sve2_sqxtunb_h, int16_t, DO_UQXTN_H) +DO_XTNB(sve2_sqxtunb_s, int32_t, DO_UQXTN_S) +DO_XTNB(sve2_sqxtunb_d, int64_t, DO_UQXTN_D) + +DO_XTNT(sve2_sqxtunt_h, int16_t, int8_t, H1, DO_UQXTN_H) +DO_XTNT(sve2_sqxtunt_s, int32_t, int16_t, H1_2, DO_UQXTN_S) +DO_XTNT(sve2_sqxtunt_d, int64_t, int32_t, H1_4, DO_UQXTN_D) + +#undef DO_XTNB +#undef DO_XTNT + void HELPER(sve2_adcl_s)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ba1953118b..b8ced82e4e 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6459,3 +6459,241 @@ static bool trans_UABA(DisasContext *s, arg_rrr_esz *a) { return do_sve2_fn_zzz(s, a, gen_gvec_uaba); } + +static bool do_sve2_narrow_extract(DisasContext *s, arg_rri_esz *a, + const GVecGen2 ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || a->imm != 0 || + !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, &ops[a->esz]); + } + return true; +} + +static const TCGOpcode sqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_smin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, d, d, t); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnb_vec, + .opt_opc = sqxtn_list, + .fno = gen_helper_sve2_sqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t mask = (1ull << halfbits) - 1; + int64_t min = -1ull << (halfbits - 1); + int64_t max = -min - 1; + + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtnt_vec, + .opt_opc = sqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode uqxtn_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, 0 +}; + +static void gen_uqxtnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnb_vec, + .opt_opc = uqxtn_list, + .fno = gen_helper_sve2_uqxtnb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_uqxtnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQXTNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_h, + .vece = MO_16 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_s, + .vece = MO_32 }, + { .fniv = gen_uqxtnt_vec, + .opt_opc = uqxtn_list, + .load_dest = true, + .fno = gen_helper_sve2_uqxtnt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static const TCGOpcode sqxtun_list[] = { + INDEX_op_shli_vec, INDEX_op_umin_vec, INDEX_op_smax_vec, 0 +}; + +static void gen_sqxtunb_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, d, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, d, d, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunb_vec, + .opt_opc = sqxtun_list, + .fno = gen_helper_sve2_sqxtunb_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} + +static void gen_sqxtunt_vec(unsigned vece, TCGv_vec d, TCGv_vec n) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = (1ull << halfbits) - 1; + + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2 ops[3] = { + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_h, + .vece = MO_16 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_s, + .vece = MO_32 }, + { .fniv = gen_sqxtunt_vec, + .opt_opc = sqxtun_list, + .load_dest = true, + .fno = gen_helper_sve2_sqxtunt_d, + .vece = MO_64 }, + }; + return do_sve2_narrow_extract(s, a, ops); +} From patchwork Fri Apr 30 20:25:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429847 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp776756jao; Fri, 30 Apr 2021 13:38:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwfRprme8dAKEAKryLYf060ApMMSdXt69QvN+Wv6WlrVH5nWx2IyPo51vBmEOtPLxIj4v+H X-Received: by 2002:a02:c773:: with SMTP id k19mr6695809jao.6.1619815097258; Fri, 30 Apr 2021 13:38:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815097; cv=none; d=google.com; s=arc-20160816; b=TNOfK5pRSop+DnL8DCey4F2Bod1mTKgY6zzgotKCMiUTl6mbFgvCO56/6EaSscj+B8 mMDcbRbKzqu28PKRwZ+/iRXe9fu7v9mgqb5eP/s5EgU6gV62wgelBBuEL3dN1/Gsbuq9 uOvpfA47pq89L+Q38gK1IQ7aXl4UKdWBky018mUFzqLWAJlyoLQ4leptRZrfhfqbc0Jw Ehxc1tjejn94n5Fo72BFFBoWyPlvhZNnEMU1MrtGySs9OAvqFF6KBsm8KzUHyBR9J9Vx sfRrenViLFgir2Zc55z3sldbchFN4YHd/hADgL0JrfST32Cbx15cx62ENHCYUQDNZpv6 FS7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=xQsYLpPVDkUuGY13be1OyXB5ri5UrMV4fx5JeuHtH9BCXC93NxE3B2wTyfB3TiyBiP z5SyT8sR7E1Y7iebc+RtI6IcBaexPjsizXgtr5ZdVWMBctYPjqCgaEkbxcrPG0qWhOaw H+cdVC6rFhLrdQjraW/b1zkay/XkwKYkh2l4t19bBY8Jb4PPop5nnnfklCm2qT3kTOv/ +03fiapPAhXQUFjizgl2AOpCV3nMD6a2I4ifeSAcJxOsNDdy2FavYdjoCLwYaOR298Kk FARpuobCVvLZ2sy+94jcmVyG8t8tRgyklx4TVFTZLtV4cQYTiLhSRpEvvyda5MFPLcdF WJZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y6tNz0hh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x16si2887204ilh.134.2021.04.30.13.38.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:38:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y6tNz0hh; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57714 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZtw-0005Lv-IP for patch@linaro.org; Fri, 30 Apr 2021 16:38:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52094) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZin-0000JA-Qm for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:45 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]:33739) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiW-000687-A5 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:45 -0400 Received: by mail-pl1-x62c.google.com with SMTP id b21so8728458plz.0 for ; Fri, 30 Apr 2021 13:26:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=Y6tNz0hhb+Zns2iMX7899pgCoPTB3fF7nMdXYqhpcVRN+EUHk0SeKhrNgDzg5RVrje 5/f2mN+HAlUfa7d54OPgZbRQ5Jj8TOMTYhq2m/auS0uMAgAf6gggzqh5XZas6VXEObve o4sJveLG6/+zuD7hR7Rw/gNbBJf2s2dzZY5PXmdR3mJywryp8lx2MXS2PVcNEgYIidCD Sfl5QOi583WMwh+aw7isNDQu5Z/he/N/D4CgzxpDQiA5EsDEHaY47d4Fsw6vbar24wQC bksRbsIaU9qFi0Wq/JT+ifG88LsLGnupiPNan2zwCy3IlNzcfouP1jzEQ9y7BieUFXf7 6H6A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=7dY9xjD1cw8e6w0AiGk+0Ct4FwwuYvpBMTDUl5kU++c=; b=A9+3gNud5+uNI1VFJ720flagtQTs6sOD1SgtAuWFg70Uni+yk6rNJHMbw4FBJ5NCIr rcs9jo1Cq3KNMPMhxkexM8G1k69DnccJ8aEk44xRuS+OlGZZ7PFFVJJ7n03hzpNaVjdv XdDqPsvbR9SsGvqAsGjYUSFklG9TPAsVtwrc/Be0j1+nlXbyTDWO+34CS3h9Mij8/X2b Kn3u/WRkrOb9VcAFsc3plWWHYzC6NUCOv9w4GP33NjRnWNDXDvNKO3Fgtc6+K4oThIGa AAU/m3CVb6bt5dZMJtTXcn2oqxXuMHIupqHh8JG30bPaQFdDcnVEaY2nDen/s2RRf4OB /jYw== X-Gm-Message-State: AOAM530VmHtp1uaJWrqN1mvHPN1DBrR23iBttDX28kjWKHKn2ESd37kN EW6n08p8rbqznTMsPQ+flgr4eygPm6M4Kw== X-Received: by 2002:a17:90b:4a4e:: with SMTP id lb14mr16170090pjb.115.1619814386527; Fri, 30 Apr 2021 13:26:26 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 25/82] target/arm: Implement SVE2 floating-point pairwise Date: Fri, 30 Apr 2021 13:25:13 -0700 Message-Id: <20210430202610.1136687-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Richard Henderson --- v2: Load all inputs before writing any output (laurent desnogues) --- target/arm/helper-sve.h | 35 +++++++++++++++++++++++++++++ target/arm/sve.decode | 8 +++++++ target/arm/sve_helper.c | 46 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 25 +++++++++++++++++++++ 4 files changed, 114 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index b302203ce8..a033b5f6b2 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2443,3 +2443,38 @@ DEF_HELPER_FLAGS_3(sve2_uqxtnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminnmp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmaxp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 19866ec4c6..9c75ac94c0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1284,3 +1284,11 @@ UQXTNB 01000101 .. 1 ..... 010 010 ..... ..... @rd_rn_tszimm_shl UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl + +## SVE2 floating-point pairwise operations + +FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm +FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm +FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm +FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm +FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 7dca67785a..11f228144c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -890,6 +890,52 @@ DO_ZPZZ_PAIR_D(sve2_sminp_zpzz_d, int64_t, DO_MIN) #undef DO_ZPZZ_PAIR #undef DO_ZPZZ_PAIR_D +#define DO_ZPZZ_PAIR_FP(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, \ + void *status, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; ) { \ + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)); \ + do { \ + TYPE n0 = *(TYPE *)(vn + H(i)); \ + TYPE m0 = *(TYPE *)(vm + H(i)); \ + TYPE n1 = *(TYPE *)(vn + H(i + sizeof(TYPE))); \ + TYPE m1 = *(TYPE *)(vm + H(i + sizeof(TYPE))); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(n0, n1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + if (pg & 1) { \ + *(TYPE *)(vd + H(i)) = OP(m0, m1, status); \ + } \ + i += sizeof(TYPE), pg >>= sizeof(TYPE); \ + } while (i & 15); \ + } \ +} + +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_h, float16, H1_2, float16_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_s, float32, H1_4, float32_add) +DO_ZPZZ_PAIR_FP(sve2_faddp_zpzz_d, float64, , float64_add) + +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_h, float16, H1_2, float16_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_s, float32, H1_4, float32_maxnum) +DO_ZPZZ_PAIR_FP(sve2_fmaxnmp_zpzz_d, float64, , float64_maxnum) + +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_h, float16, H1_2, float16_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_s, float32, H1_4, float32_minnum) +DO_ZPZZ_PAIR_FP(sve2_fminnmp_zpzz_d, float64, , float64_minnum) + +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_h, float16, H1_2, float16_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_s, float32, H1_4, float32_max) +DO_ZPZZ_PAIR_FP(sve2_fmaxp_zpzz_d, float64, , float64_max) + +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_h, float16, H1_2, float16_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_s, float32, H1_4, float32_min) +DO_ZPZZ_PAIR_FP(sve2_fminp_zpzz_d, float64, , float64_min) + +#undef DO_ZPZZ_PAIR_FP + /* Three-operand expander, controlled by a predicate, in which the * third operand is "wide". That is, for D = N op M, the same 64-bit * value of M is used with all of the narrower values of N. diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index b8ced82e4e..1670bcadb1 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6697,3 +6697,28 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) }; return do_sve2_narrow_extract(s, a, ops); } + +static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_4_ptr *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzz_fp(s, a, fn); +} + +#define DO_SVE2_ZPZZ_FP(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_4_ptr * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_zpzz_h, \ + gen_helper_sve2_##name##_zpzz_s, gen_helper_sve2_##name##_zpzz_d \ + }; \ + return do_sve2_zpzz_fp(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZPZZ_FP(FADDP, faddp) +DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) +DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) +DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) +DO_SVE2_ZPZZ_FP(FMINP, fminp) From patchwork Fri Apr 30 20:25:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429858 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp782900jao; Fri, 30 Apr 2021 13:48:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwSs4R6hf/3pWQqqwA2YquhVtelOV7pBNdCWQjC0hqiQ/4BlDqAVfXqhwXMpH7KsgmN6R+H X-Received: by 2002:a92:6011:: with SMTP id u17mr5610661ilb.274.1619815692936; Fri, 30 Apr 2021 13:48:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815692; cv=none; d=google.com; s=arc-20160816; b=CALAmCcm/UFdkHhaDlsjbXjPHBSGzDEw7ex9Lt6No7D2hPdMjw7yhpStq1gN70mcE9 rnH20PT6Y9un+gYacWfKJTnAcaM5C4dHQ6syG635o1xEQlxtSkxuKWg/bqxWHOGyvUHt N2b4k2pnwiYH1uVmvuFvrF6jS8Vt2iXaoyaMFxNMcerBYynncYxtvgrnX1bv86q9L27A q/lzncJODbB2YiF7DjEe4EFBKAt7H4PeELXDrOSVuIijvph4NF/bfAwlQg53WyMhAtyH LhHaXHAH76c/OyTQIJYq+pfIO9WSw/qZsO8PW2PnxlkXYqSu9DniWhvzv8unu1/MXJXe pvRw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=iG18Kuk7eWihjH3ZsI3x4QkvhBX08VH5yAo2jMRMuNKS12s0cim4BSrykVIGN1/9yz nooEW1Ur69BGLr3TmqoZgRJLTyjcCvdAUz2QUrqtnxi0A4BSeZ0onAoDoBPb2fWsVc1f nEzJ+Na2Qm63ZcVr2qD2s1iMU7n2armZHWODY80ySBC6eWUash9Tp4EiN7NDzXhZY1Qp FGz0gazHBsp80NM9/7ussllB5ad/l4FJveHy1+oywJt05PfoiHUyI3ZQBOXscRPsdaec 1yGPkyu13t/Dc38ZSrJel2R/LSzBeocjiVBoLOOgQh++hMPP9z23+hCiHvSWUfYGbZWJ 3blg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="g/PymC7p"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d2si4348867ios.97.2021.04.30.13.48.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:48:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="g/PymC7p"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55238 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca3Y-0007oS-5T for patch@linaro.org; Fri, 30 Apr 2021 16:48:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52152) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiq-0000RY-RK for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:48 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]:36521) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiX-00068P-SZ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:48 -0400 Received: by mail-pl1-x62e.google.com with SMTP id a11so6346741plh.3 for ; Fri, 30 Apr 2021 13:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=g/PymC7pwOkW2psfLqHADns983Q2YIOhWWcQpIFJcALV98DDIlapiF7bREjHW+MduZ I0zobRubXD0XjjnUZdKvtecnn8Xzgn2Z2C7RN+8UQCh8wnWTF38VtECp7toj4VPSBBgR RyUB7xgciFwpk7EM9M2q/wuC5mhOYcSNvhQUUPzaAUCRTxbv1eiu4KWxILL2wi59rMYh j4CFhjmsNVuYm/dX5tdHmgdEXKJxY/DcNSMqH87GfK41phKpR9W7ZM2iMv+1t+jyXEtB q2Qc1qUUCzPFoj+vU2hEh/9YJF9TFFv8BOOY8W6Cnu9CKtv4Qe4NDekmshJSDDfaSdjb dHJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WwOjRgyqW+QscwVMflkQnp8uVzX1HAgfjZDST1BGuos=; b=uUbvBTOhRIg21lefxi4TRzWDVO+U9EqmEDne6NIyIP+JspH1BFwl3JO3wFJmDRXq11 yF75fBEF2GQoxwsU13owEQ3C+I+SYOLfBoYaJ4CPJ9wbE4c/QuQhYm8WtZhPe7VQS9Hk 1q8YL1i9NFgM13IdxiksP6Q9R7RqRmEDU8gHGkblWT0ZYMy5rKX+mSik1HYzxNcA6Txf ahysh7hXYRcYU4xtO3QWCehf3FcLgz+BTKA6UyYmfQ8SpP3+4qk4g8XfEeLICOlqK6yj Q1cZ6s+fMtzKtwJouQtLvxe4vwQWqXdlu0ZqG7+7J793nPcYVznbbpNUQcp6pFbktsdF H8vA== X-Gm-Message-State: AOAM531hxf2ACbRplUQaCJ2ZUVUZEYrCKSABxTdxhK+5qrEjeIF6zwh+ WZGoy6CcyMtd5nCXLgTX5kghDcmcIE6AgQ== X-Received: by 2002:a17:903:248e:b029:ec:b399:7d75 with SMTP id p14-20020a170903248eb02900ecb3997d75mr7019669plw.35.1619814387184; Fri, 30 Apr 2021 13:26:27 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 26/82] target/arm: Implement SVE2 SHRN, RSHRN Date: Fri, 30 Apr 2021 13:25:14 -0700 Message-Id: <20210430202610.1136687-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix typo in gen_shrnb_vec (laurent desnogues) v3: Replace DO_RSHR with an inline function --- target/arm/helper-sve.h | 16 ++++ target/arm/sve.decode | 8 ++ target/arm/sve_helper.c | 54 ++++++++++++- target/arm/translate-sve.c | 160 +++++++++++++++++++++++++++++++++++++ 4 files changed, 236 insertions(+), 2 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a033b5f6b2..2b2ebea631 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2444,6 +2444,22 @@ DEF_HELPER_FLAGS_3(sve2_sqxtunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqxtunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_shrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_shrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9c75ac94c0..169486ecb2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1285,6 +1285,14 @@ UQXTNT 01000101 .. 1 ..... 010 011 ..... ..... @rd_rn_tszimm_shl SQXTUNB 01000101 .. 1 ..... 010 100 ..... ..... @rd_rn_tszimm_shl SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl +## SVE2 bitwise shift right narrow + +# Bit 23 == 0 is handled by esz > 0 in the translator. +SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr +SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr +RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr +RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 11f228144c..3f864da3ab 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1863,6 +1863,17 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, uint32_t desc) \ when N is negative, add 2**M-1. */ #define DO_ASRD(N, M) ((N + (N < 0 ? ((__typeof(N))1 << M) - 1 : 0)) >> M) +static inline uint64_t do_urshr(uint64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else if (sh == 64) { + return x >> 63; + } else { + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1883,12 +1894,51 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) -#undef DO_SHR -#undef DO_SHL #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D +#define DO_SHRNB(NAME, TYPEW, TYPEN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, shift); \ + } \ +} + +#define DO_SHRNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int shift = simd_data(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, shift); \ + } \ +} + +DO_SHRNB(sve2_shrnb_h, uint16_t, uint8_t, DO_SHR) +DO_SHRNB(sve2_shrnb_s, uint32_t, uint16_t, DO_SHR) +DO_SHRNB(sve2_shrnb_d, uint64_t, uint32_t, DO_SHR) + +DO_SHRNT(sve2_shrnt_h, uint16_t, uint8_t, H1_2, H1, DO_SHR) +DO_SHRNT(sve2_shrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_SHR) +DO_SHRNT(sve2_shrnt_d, uint64_t, uint32_t, , H1_4, DO_SHR) + +DO_SHRNB(sve2_rshrnb_h, uint16_t, uint8_t, do_urshr) +DO_SHRNB(sve2_rshrnb_s, uint32_t, uint16_t, do_urshr) +DO_SHRNB(sve2_rshrnb_d, uint64_t, uint32_t, do_urshr) + +DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) +DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) +DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) + +#undef DO_SHRNB +#undef DO_SHRNT + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1670bcadb1..c1a081acaa 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6698,6 +6698,166 @@ static bool trans_SQXTUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_narrow_extract(s, a, ops); } +static bool do_sve2_shr_narrow(DisasContext *s, arg_rri_esz *a, + const GVecGen2i ops[3]) +{ + if (a->esz < 0 || a->esz > MO_32 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + assert(a->imm > 0 && a->imm <= (8 << a->esz)); + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_2i(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vsz, vsz, a->imm, &ops[a->esz]); + } + return true; +} + +static void gen_shrnb_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shri_i64(d, n, shr); + tcg_gen_andi_i64(d, d, mask); +} + +static void gen_shrnb16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_16, d, n, shr); +} + +static void gen_shrnb32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_32, d, n, shr); +} + +static void gen_shrnb64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnb_i64(MO_64, d, n, shr); +} + +static void gen_shrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shri_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnb16_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_h, + .vece = MO_16 }, + { .fni8 = gen_shrnb32_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_s, + .vece = MO_32 }, + { .fni8 = gen_shrnb64_i64, + .fniv = gen_shrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_shrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_shrnt_i64(unsigned vece, TCGv_i64 d, TCGv_i64 n, int shr) +{ + int halfbits = 4 << vece; + uint64_t mask = dup_const(vece, MAKE_64BIT_MASK(0, halfbits)); + + tcg_gen_shli_i64(n, n, halfbits - shr); + tcg_gen_andi_i64(n, n, ~mask); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_or_i64(d, d, n); +} + +static void gen_shrnt16_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_16, d, n, shr); +} + +static void gen_shrnt32_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + gen_shrnt_i64(MO_32, d, n, shr); +} + +static void gen_shrnt64_i64(TCGv_i64 d, TCGv_i64 n, int64_t shr) +{ + tcg_gen_shri_i64(n, n, shr); + tcg_gen_deposit_i64(d, d, n, 32, 32); +} + +static void gen_shrnt_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + uint64_t mask = MAKE_64BIT_MASK(0, halfbits); + + tcg_gen_shli_vec(vece, n, n, halfbits - shr); + tcg_gen_dupi_vec(vece, t, mask); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { INDEX_op_shli_vec, 0 }; + static const GVecGen2i ops[3] = { + { .fni8 = gen_shrnt16_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_h, + .vece = MO_16 }, + { .fni8 = gen_shrnt32_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_s, + .vece = MO_32 }, + { .fni8 = gen_shrnt64_i64, + .fniv = gen_shrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_shrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnb_h }, + { .fno = gen_helper_sve2_rshrnb_s }, + { .fno = gen_helper_sve2_rshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_rshrnt_h }, + { .fno = gen_helper_sve2_rshrnt_s }, + { .fno = gen_helper_sve2_rshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 30 20:25:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429867 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp786667jao; Fri, 30 Apr 2021 13:54:50 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwwfNyHe89VANsFCdvGzyAAafN8gVnuoInElQHmacJYBlU2CThfYWj3VQibjZXxV+R8NDpH X-Received: by 2002:a05:6e02:1243:: with SMTP id j3mr5595509ilq.192.1619816090488; Fri, 30 Apr 2021 13:54:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816090; cv=none; d=google.com; s=arc-20160816; b=PYf3xcCA3hE72PwOof/MlI5vl4cD7fSAulGL5O0O8VGC9p1b2NP9Efq4aktZWXwM+Z Azr4CbDPcxulFYWVPW2QHcfHK6epwrFtUCwPUV2oO6x117GJzyDtTLHU53Df3QUakjtl NJTBVtjN14XbpmsgLhkQW4qx/IKpJCIz1bunz4KFIji9/qUbn4DAHDnPD9IdmIQTAdxE VWQLAcpfR3BjZkgX6vlSoGjeMQ9Z9vZbyDxPO5IlvloRFbtpEZ8wVKmO66WtG69mIv1W KxiMWTDJDfmkBRjZeWhhyBanjdkQhMCaiQjZIxOFbx5D0wb0KdUbjBSD8gSAkrnuQ1an ULHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=QWLN0oRZQvUbXZVnhYRh4P4OXvBIPiBTi4LB9YD2qqMhfivuil2qIVcjAWhsDTULGp +yUz2PCr/nV2F9HAiBhIuOvx2gKdK6Sc+DlHknKUqvfvpzQPIcmZlhybR5RA86GpMlV5 Iw9+k2T+g8WYJVhT/Px/FuNcB0pGVd4Txebg2F7ITTxdlIylWjLmM1cC8XJaDD3nzBin L/AhGyVvkY2bQbaRxxRx8LiEN/J26v47a9oYU6w/MsvVuKEimgoQIsFYtbsBBFp3Hlfi 0ieNzRhGC/21ncfBxIo53JCLonlazUKibScM/q4Un0BDCrF1lfWOui1NI/Nv1L61BpmQ /q6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tAssThl8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h16si3276125jao.90.2021.04.30.13.54.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:54:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tAssThl8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50800 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca9x-0000ll-TF for patch@linaro.org; Fri, 30 Apr 2021 16:54:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52262) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiw-0000g5-HV for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:54 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]:44836) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiY-00069D-4p for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:54 -0400 Received: by mail-pj1-x1030.google.com with SMTP id m6-20020a17090a8586b02901507e1acf0fso2371077pjn.3 for ; Fri, 30 Apr 2021 13:26:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=tAssThl8s8HVqrnAnsi4xoqIfR1vLTEPfy+sWv1Lgk+08oH//sMqZOSVzv8Pfh1Ztg ToVonW8OD1Y0DDKYlTtDG2bN0aYWi1AkZilLpgeOsxYK8BnuXzE2zDxa5hO3PWkyUAJ1 wPeYPLQAGVEbxPvqmx6e6j4mhYDM3SxWnHoc3MEZ8tkJ7keYvxCyL9AnSf9/z6bIQeyL FYfRk2ueLvFpNS2j4PwhO6Em7uNg3K1RI/TtHhyu1COX4aj1zc1fW3PsnnESrhT79zkl r95/UncLpAloMhOaMMZnNIqeGbd1at3B9bZNQt5iaGsY3jBH2EQFxDZ4AvP19/7ahxQV WemQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=3st5GDCoYANtSqf9tcWs4eqMv4/zg0MwHali4f+3cK4=; b=QdJWtPfXjSm86ZoZGIaPblqhNpFplzN9QoiVoaYlfZLTFpCC0NdX2KI1IYDrlQIGa4 6a/9lz4dzDKwr/MgNSSZ0JHojdj1tDsj28eRt4r3F+hvnyRKb5uV6wfPj94Vw2cnbto0 zAaJ5okCTzKvJYlw25DgOzEt/Y3ebffbqKRcfz/np7yibPZmGYI2p69b6BkOgAg6CxwV een86BsyFZAWcbDGc7iZkSzw2TNbEbV5Nt0htI4YkFSfEXbAw4hAzIcDEgFnmCGUm9zF tIfg97eCOXVv47E7SH8qGCLx/1KzcV2IzZT92bKJy77JDTAGzNYHyAMhnF4XoTflAsjN IS7Q== X-Gm-Message-State: AOAM530CAhEdNSyTLlQ87QunaaBIKWnC+91sH2mZxbBMZiVgzTamfUVA rReGW8IxDdZ7+Gvo6SVEp6FJGaH652xRKg== X-Received: by 2002:a17:902:d491:b029:ee:5aa3:7d57 with SMTP id c17-20020a170902d491b02900ee5aa37d57mr6992770plg.26.1619814388082; Fri, 30 Apr 2021 13:26:28 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 27/82] target/arm: Implement SVE2 SQSHRUN, SQRSHRUN Date: Fri, 30 Apr 2021 13:25:15 -0700 Message-Id: <20210430202610.1136687-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 35 ++++++++++++++ target/arm/translate-sve.c | 98 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 153 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2b2ebea631..2e80d9d27b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2460,6 +2460,22 @@ DEF_HELPER_FLAGS_3(sve2_rshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_rshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 169486ecb2..18faa900ca 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1288,6 +1288,10 @@ SQXTUNT 01000101 .. 1 ..... 010 101 ..... ..... @rd_rn_tszimm_shl ## SVE2 bitwise shift right narrow # Bit 23 == 0 is handled by esz > 0 in the translator. +SQSHRUNB 01000101 .. 1 ..... 00 0000 ..... ..... @rd_rn_tszimm_shr +SQSHRUNT 01000101 .. 1 ..... 00 0001 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNB 01000101 .. 1 ..... 00 0010 ..... ..... @rd_rn_tszimm_shr +SQRSHRUNT 01000101 .. 1 ..... 00 0011 ..... ..... @rd_rn_tszimm_shr SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 3f864da3ab..d6b6293ab0 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1874,6 +1874,16 @@ static inline uint64_t do_urshr(uint64_t x, unsigned sh) } } +static inline int64_t do_srshr(int64_t x, unsigned sh) +{ + if (likely(sh < 64)) { + return (x >> sh) + ((x >> (sh - 1)) & 1); + } else { + /* Rounding the sign bit always produces 0. */ + return 0; + } +} + DO_ZPZI(sve_asr_zpzi_b, int8_t, H1, DO_SHR) DO_ZPZI(sve_asr_zpzi_h, int16_t, H1_2, DO_SHR) DO_ZPZI(sve_asr_zpzi_s, int32_t, H1_4, DO_SHR) @@ -1936,6 +1946,31 @@ DO_SHRNT(sve2_rshrnt_h, uint16_t, uint8_t, H1_2, H1, do_urshr) DO_SHRNT(sve2_rshrnt_s, uint32_t, uint16_t, H1_4, H1_2, do_urshr) DO_SHRNT(sve2_rshrnt_d, uint64_t, uint32_t, , H1_4, do_urshr) +#define DO_SQSHRUN_H(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT8_MAX) +#define DO_SQSHRUN_S(x, sh) do_sat_bhs((int64_t)(x) >> sh, 0, UINT16_MAX) +#define DO_SQSHRUN_D(x, sh) \ + do_sat_bhs((int64_t)(x) >> (sh < 64 ? sh : 63), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqshrunb_h, int16_t, uint8_t, DO_SQSHRUN_H) +DO_SHRNB(sve2_sqshrunb_s, int32_t, uint16_t, DO_SQSHRUN_S) +DO_SHRNB(sve2_sqshrunb_d, int64_t, uint32_t, DO_SQSHRUN_D) + +DO_SHRNT(sve2_sqshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRUN_H) +DO_SHRNT(sve2_sqshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRUN_S) +DO_SHRNT(sve2_sqshrunt_d, int64_t, uint32_t, , H1_4, DO_SQSHRUN_D) + +#define DO_SQRSHRUN_H(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT8_MAX) +#define DO_SQRSHRUN_S(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT16_MAX) +#define DO_SQRSHRUN_D(x, sh) do_sat_bhs(do_srshr(x, sh), 0, UINT32_MAX) + +DO_SHRNB(sve2_sqrshrunb_h, int16_t, uint8_t, DO_SQRSHRUN_H) +DO_SHRNB(sve2_sqrshrunb_s, int32_t, uint16_t, DO_SQRSHRUN_S) +DO_SHRNB(sve2_sqrshrunb_d, int64_t, uint32_t, DO_SQRSHRUN_D) + +DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) +DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) +DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c1a081acaa..5ff6b8ffb6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6858,6 +6858,104 @@ static bool trans_RSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrunb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrunb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrunt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, 0); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrunt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrunt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunb_h }, + { .fno = gen_helper_sve2_sqrshrunb_s }, + { .fno = gen_helper_sve2_sqrshrunb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrunt_h }, + { .fno = gen_helper_sve2_sqrshrunt_s }, + { .fno = gen_helper_sve2_sqrshrunt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 30 20:25:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429855 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp780839jao; Fri, 30 Apr 2021 13:44:57 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzZNRPOXAH/TEYjqbIn2r37Od1OWfMA+RJQDXpkDOOjWozANVaqWoYB5SNsT7f/ZEyy/EcX X-Received: by 2002:a05:6638:3395:: with SMTP id h21mr2602337jav.44.1619815497370; Fri, 30 Apr 2021 13:44:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815497; cv=none; d=google.com; s=arc-20160816; b=y4jzRoCf15SuK0OvNrGwaw7pq4sbDFIkNKLlvkh0lhm93VQ3r1jIQeQUu7E3q3xSEW Fh8vvOZ8dTfo+Yi+gc95w/qYTCvKM46JcDJxQaaPVO14V4VXuwEV7XbbKdu2Bz9rmfRG r5QiaXLgFl4WaSG61YHVl1bXgDVIcIGDKP8HrsqeYNxURFDfW4sJKAhXIVlaVnAvougW OWvnykDQ45wE33AoEoedCoZmVJvdPejMWHVXMKMrjddoD1MWrQ4NqJXBOpz0ERXCTWV6 6oPgY8vGp1piZTnBsAkoZe3sBlRkLPBtJDbhS+yjB4PGSOMUDHOKcp+f8E7Pa7aUG3cj iw9w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=T3/Q5wSlmyPsd7HY37ZDu02vgGsiA0YeqsM2p4Dj5kvyXsyVmCQTd8pVVrNbw2ZEWa krA8VrCE3F1kAtzSrF0YqL/rj5hJ+QnZsMPRtvB5TEfOsXdfdBqNByIRc0lFvu5NLH4b ZFDklHD+BzA9qyM1mF04w24NtgtrKe9ATts9v5EWqVgpn1uL7MiSrYMoiHvebiluellf 3rLmlBnmlao7pRgLW52BhNbxJrZdXPXwlOHugPWhM+PsOW8oMyhHIBLusp3+dZh12ofN VchGAfx9Ymdi1m/3s2/fjl8uew/9P/0cqcVCcwQ5j+l5rV/bS/wAyeNEu0kxSCzaWY54 tQ+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=w66AxZrC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g19si3998183jat.91.2021.04.30.13.44.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:44:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=w66AxZrC; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47970 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca0O-0004kn-Kv for patch@linaro.org; Fri, 30 Apr 2021 16:44:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52172) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZis-0000Uf-3p for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:50 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:35649) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiY-00069M-5E for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:49 -0400 Received: by mail-pg1-x52b.google.com with SMTP id q10so50171117pgj.2 for ; Fri, 30 Apr 2021 13:26:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=w66AxZrCkG3ssonbIn+ourMdwQ8GP8AmG5XfGx0+gBRJxAqcyXmcclb/TFJ0EEdugw 0o128aWbqENseLAZwv4PjjH+r+t2SB2TE32d1BzzFT8GmOrQEdXZ4I3K1JHw0F/Cm6x0 hFyZN8Ek9H3loD0Wzq/D+LpxFdqCBPLYxZUwkbzQ3NcklOQ4rY0Kx0o4km1eMbpoCswU wyYt4F98CjZoZg6XMOmpKsD1ley9qcVIwZu9KXqpPDVWArE+pt5aCN9Lu++kNwAHhu2g obmxIiOFv48lLLcVilAVmCKQbBems1kpevQCHgKXEi6dngnCyqxF0KZsYomEvnalYoLg lHSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=TYIH46Yz5nuJnlEKWiLaTEG5vm1zqOPy6lL/fTv4hvo=; b=aUj7zIU/EgnAIk+yFGSyYE3EbuPnbErEAjF+j1uVWULifKA76oqKqWBTH7g9j2cKyi c+IhfGX18KYEpM7XhaQ58tgoBZKWrT8BpLYBold1o61Lu+0b7IuY5pThW2SqvB5p6JoP +zAwU8QRepfG9H4449D0B2zfzSSOH9wF+Dlwq22mGbI6COoxou9Jy9wxTp4K+RjqGYZ5 uYAwAi/hBK+2LI3hEFP7abZDkSlU4x0MD5cIjv6MIL+488NZ7p9OFs5m+9YimUZqSlQf RPgz3cF9qVdKBQ5gvixMcd+C3NVFPBxA1PYrZhZa2JysFDufeCIJQCApOqjRkQK50pqB Vfig== X-Gm-Message-State: AOAM530K28el6nPqK/deR7jBmB5YeInB2JbmWjeDGGxJVL64LLsh3mmi +iy+aBFclq7W/1lllSYU1F3BD6DddHWZkA== X-Received: by 2002:a05:6a00:2389:b029:28c:ede4:8a60 with SMTP id f9-20020a056a002389b029028cede48a60mr3873406pfc.54.1619814388755; Fri, 30 Apr 2021 13:26:28 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 28/82] target/arm: Implement SVE2 UQSHRN, UQRSHRN Date: Fri, 30 Apr 2021 13:25:16 -0700 Message-Id: <20210430202610.1136687-29-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 +++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 ++++++++++ target/arm/translate-sve.c | 93 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 137 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 2e80d9d27b..ba6a24fc8b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 18faa900ca..13b5da0856 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr +UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr +UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr +UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ## SVE2 floating-point pairwise operations diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d6b6293ab0..175f3de08f 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) +#define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) +#define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) + +DO_SHRNB(sve2_uqshrnb_h, uint16_t, uint8_t, DO_UQSHRN_H) +DO_SHRNB(sve2_uqshrnb_s, uint32_t, uint16_t, DO_UQSHRN_S) +DO_SHRNB(sve2_uqshrnb_d, uint64_t, uint32_t, DO_UQSHRN_D) + +DO_SHRNT(sve2_uqshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQSHRN_H) +DO_SHRNT(sve2_uqshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQSHRN_S) +DO_SHRNT(sve2_uqshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQSHRN_D) + +#define DO_UQRSHRN_H(x, sh) MIN(do_urshr(x, sh), UINT8_MAX) +#define DO_UQRSHRN_S(x, sh) MIN(do_urshr(x, sh), UINT16_MAX) +#define DO_UQRSHRN_D(x, sh) MIN(do_urshr(x, sh), UINT32_MAX) + +DO_SHRNB(sve2_uqrshrnb_h, uint16_t, uint8_t, DO_UQRSHRN_H) +DO_SHRNB(sve2_uqrshrnb_s, uint32_t, uint16_t, DO_UQRSHRN_S) +DO_SHRNB(sve2_uqrshrnb_d, uint64_t, uint32_t, DO_UQRSHRN_D) + +DO_SHRNT(sve2_uqrshrnt_h, uint16_t, uint8_t, H1_2, H1, DO_UQRSHRN_H) +DO_SHRNT(sve2_uqrshrnt_s, uint32_t, uint16_t, H1_4, H1_2, DO_UQRSHRN_S) +DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) + #undef DO_SHRNB #undef DO_SHRNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5ff6b8ffb6..12733d9b4f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6956,6 +6956,99 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_uqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_uqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + + tcg_gen_shri_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_umin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_UQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_shri_vec, INDEX_op_umin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_uqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_uqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnb_h }, + { .fno = gen_helper_sve2_uqrshrnb_s }, + { .fno = gen_helper_sve2_uqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_uqrshrnt_h }, + { .fno = gen_helper_sve2_uqrshrnt_s }, + { .fno = gen_helper_sve2_uqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 30 20:25:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429860 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp783112jao; Fri, 30 Apr 2021 13:48:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwmL3nwM4bRD5hEc5xM5PFENnJYLZWFiPAyiFQ0unisAsGgdfjCsfKBub7XcwKGFyNQJ0vQ X-Received: by 2002:a02:6d53:: with SMTP id e19mr6664839jaf.128.1619815713267; Fri, 30 Apr 2021 13:48:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815713; cv=none; d=google.com; s=arc-20160816; b=XVW8dPOGkkQSL0GThqBEcVp1yw78pCr9cy2i4gkgEaGyULtD921bgRv5SyI8b3AyMx 1o7VNsdh2rnaTSNl0560ciNAIXeoOF7WHNGkbX4Iw/EMOeRjnItDgyA335fr3AcKBIXp rPuTlTQtbGEpwYPnCZLnxFcoY4wb9AFc4dpQaa93sC5i0rivhdbPa9SE7A7zenr1Pcdn AydPIAxpa80YdTs5WTZ8a2ElAyOnzXfRWtI59Caq99VmkYXJVeKPZYOXr2cCXHJ7gB2F S6iAvGRkrvdezPjToGumv0+rLyD3KjUADgHCP1YeYcb6DsBOl2e9Wko/FjAmupUpR03y oDlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=hRxVkF3hgSOitnkhrtgvp0V7qXPWg/K7+ZyYtmGDCWRMRzoDkZ1kKd2C8cBl9ziYat VlI38JWU7H1BueT+dmPbRzieDZedED/r2CA5q0F3nTw9M7LSqFsdUXGFnCjYsLPWB4kQ dkvCAt7c/dHW6IZSedc9NbNE5Q8zp9DwPuqwrFjbVJkON8otaa2Ys7au5QP33UsvZhQd VO5dv3SKk7dFWqvl9r6Jl7MiLqkzQJ020HhUXeltKZV1AETbZ9S3KvBkDWoWYqE8kfJE WF3mgq7k6gKycmicSaHP1f5zvF2kxZ6rHdxToullKs1SvKxbI+SVlWaSgQvKfNBQ1Ph9 ny2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vCWnKAls; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s19si3341197jap.52.2021.04.30.13.48.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:48:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vCWnKAls; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca3s-0008Px-Hf for patch@linaro.org; Fri, 30 Apr 2021 16:48:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52186) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZis-0000WW-O5 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:50 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]:38603) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiY-00069Y-Mh for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:50 -0400 Received: by mail-pf1-x42a.google.com with SMTP id j6so9884680pfh.5 for ; Fri, 30 Apr 2021 13:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=vCWnKAls7b/W4bfCsC8sjhELg8r8ooj75pioe37ycTv6j5sjHZGeuUI9tAODxCSqDJ pjr14SX/dBpdp6UdW7p+xcAOF4bkKszq2oRP7a1LzwnFZYmX6x5z8Uc9H9q2j1Xr2mcv pzQCQIwpD8cAOL72PkowvSW883uOFr00Cgz+tUaVc38bQrEvqIe4dg1dqi224r6w8kbG EP97Jj0ddJPQ0RuhA5zMM/omHKg9DAkeLw9iLSE4M4zzxo/UayeO3GZo8OyqEjmv2del sL10LE7jWLCvbtrNwtTg4t5mz4KoxX9hBnLVV2YK9H2IEQgrgVUdHZn5Rumn1OhKjZ0x +/Dw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=feb7AjyBXXqqzvqsPRxefar5HpBH8rI+phXQIUCl9Ao=; b=brth5GdiTI81Vmuo3EoPVmHzU5VujxHtKwRcvntpBip93xkjsG0iveMquhMBfKkPqX dJnphHb/ECmw8QVgR3A+tva+KaciqCe14lGFOIRJPz8MdEyqE0R4cM9AcNGLwLGUqFVa NCtDeTWJIYtfqL9NMgAcln/6LGolRccfTeva/gFwayz9kUj3dLrzJIBolF6Go5WcJENv 86gpb/KTeDS8h3hXEiKYKJYyEXlhi6GlpteRa0oKv7nXCOE4WPU6xbjwXRKqzhzGymtA cImxzBJ4xQe7iOL2JcoxTdT+NeVrDaesYVqJ3CwL9BBE67Ni6qktbL7t0Wi/s1K9phNm /b0A== X-Gm-Message-State: AOAM5337kRsxgXwmKN2k2re/g5ydV8hnrDVLzvs7OmTDIRo/cfrZrA2/ EVF/VA113lJAW411DJldj9fWMuXx7zk1Ag== X-Received: by 2002:aa7:8198:0:b029:274:8a92:51b5 with SMTP id g24-20020aa781980000b02902748a9251b5mr6470090pfi.5.1619814389377; Fri, 30 Apr 2021 13:26:29 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 29/82] target/arm: Implement SVE2 SQSHRN, SQRSHRN Date: Fri, 30 Apr 2021 13:25:17 -0700 Message-Id: <20210430202610.1136687-30-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This completes the section "SVE2 bitwise shift right narrow". Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 16 ++++++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 24 +++++++++ target/arm/translate-sve.c | 105 +++++++++++++++++++++++++++++++++++++ 4 files changed, 149 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index ba6a24fc8b..1c7fe8e417 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2476,6 +2476,22 @@ DEF_HELPER_FLAGS_3(sve2_sqrshrunt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_sqrshrunt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_3(sve2_sqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve2_uqshrnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqshrnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 13b5da0856..0674464695 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1296,6 +1296,10 @@ SHRNB 01000101 .. 1 ..... 00 0100 ..... ..... @rd_rn_tszimm_shr SHRNT 01000101 .. 1 ..... 00 0101 ..... ..... @rd_rn_tszimm_shr RSHRNB 01000101 .. 1 ..... 00 0110 ..... ..... @rd_rn_tszimm_shr RSHRNT 01000101 .. 1 ..... 00 0111 ..... ..... @rd_rn_tszimm_shr +SQSHRNB 01000101 .. 1 ..... 00 1000 ..... ..... @rd_rn_tszimm_shr +SQSHRNT 01000101 .. 1 ..... 00 1001 ..... ..... @rd_rn_tszimm_shr +SQRSHRNB 01000101 .. 1 ..... 00 1010 ..... ..... @rd_rn_tszimm_shr +SQRSHRNT 01000101 .. 1 ..... 00 1011 ..... ..... @rd_rn_tszimm_shr UQSHRNB 01000101 .. 1 ..... 00 1100 ..... ..... @rd_rn_tszimm_shr UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 175f3de08f..02e87c535d 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1971,6 +1971,30 @@ DO_SHRNT(sve2_sqrshrunt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRUN_H) DO_SHRNT(sve2_sqrshrunt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRUN_S) DO_SHRNT(sve2_sqrshrunt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRUN_D) +#define DO_SQSHRN_H(x, sh) do_sat_bhs(x >> sh, INT8_MIN, INT8_MAX) +#define DO_SQSHRN_S(x, sh) do_sat_bhs(x >> sh, INT16_MIN, INT16_MAX) +#define DO_SQSHRN_D(x, sh) do_sat_bhs(x >> sh, INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqshrnb_h, int16_t, uint8_t, DO_SQSHRN_H) +DO_SHRNB(sve2_sqshrnb_s, int32_t, uint16_t, DO_SQSHRN_S) +DO_SHRNB(sve2_sqshrnb_d, int64_t, uint32_t, DO_SQSHRN_D) + +DO_SHRNT(sve2_sqshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQSHRN_H) +DO_SHRNT(sve2_sqshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQSHRN_S) +DO_SHRNT(sve2_sqshrnt_d, int64_t, uint32_t, , H1_4, DO_SQSHRN_D) + +#define DO_SQRSHRN_H(x, sh) do_sat_bhs(do_srshr(x, sh), INT8_MIN, INT8_MAX) +#define DO_SQRSHRN_S(x, sh) do_sat_bhs(do_srshr(x, sh), INT16_MIN, INT16_MAX) +#define DO_SQRSHRN_D(x, sh) do_sat_bhs(do_srshr(x, sh), INT32_MIN, INT32_MAX) + +DO_SHRNB(sve2_sqrshrnb_h, int16_t, uint8_t, DO_SQRSHRN_H) +DO_SHRNB(sve2_sqrshrnb_s, int32_t, uint16_t, DO_SQRSHRN_S) +DO_SHRNB(sve2_sqrshrnb_d, int64_t, uint32_t, DO_SQRSHRN_D) + +DO_SHRNT(sve2_sqrshrnt_h, int16_t, uint8_t, H1_2, H1, DO_SQRSHRN_H) +DO_SHRNT(sve2_sqrshrnt_s, int32_t, uint16_t, H1_4, H1_2, DO_SQRSHRN_S) +DO_SHRNT(sve2_sqrshrnt_d, int64_t, uint32_t, , H1_4, DO_SQRSHRN_D) + #define DO_UQSHRN_H(x, sh) MIN(x >> sh, UINT8_MAX) #define DO_UQSHRN_S(x, sh) MIN(x >> sh, UINT16_MAX) #define DO_UQSHRN_D(x, sh) MIN(x >> sh, UINT32_MAX) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 12733d9b4f..218f1ca5ce 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6956,6 +6956,111 @@ static bool trans_SQRSHRUNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static void gen_sqshrnb_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_and_vec(vece, d, n, t); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_sari_vec, INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnb_vec, + .opt_opc = vec_list, + .fno = gen_helper_sve2_sqshrnb_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static void gen_sqshrnt_vec(unsigned vece, TCGv_vec d, + TCGv_vec n, int64_t shr) +{ + TCGv_vec t = tcg_temp_new_vec_matching(d); + int halfbits = 4 << vece; + int64_t max = MAKE_64BIT_MASK(0, halfbits - 1); + int64_t min = -max - 1; + + tcg_gen_sari_vec(vece, n, n, shr); + tcg_gen_dupi_vec(vece, t, min); + tcg_gen_smax_vec(vece, n, n, t); + tcg_gen_dupi_vec(vece, t, max); + tcg_gen_smin_vec(vece, n, n, t); + tcg_gen_shli_vec(vece, n, n, halfbits); + tcg_gen_dupi_vec(vece, t, MAKE_64BIT_MASK(0, halfbits)); + tcg_gen_bitsel_vec(vece, d, t, d, n); + tcg_temp_free_vec(t); +} + +static bool trans_SQSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const TCGOpcode vec_list[] = { + INDEX_op_shli_vec, INDEX_op_sari_vec, + INDEX_op_smax_vec, INDEX_op_smin_vec, 0 + }; + static const GVecGen2i ops[3] = { + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_h, + .vece = MO_16 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_s, + .vece = MO_32 }, + { .fniv = gen_sqshrnt_vec, + .opt_opc = vec_list, + .load_dest = true, + .fno = gen_helper_sve2_sqshrnt_d, + .vece = MO_64 }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNB(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnb_h }, + { .fno = gen_helper_sve2_sqrshrnb_s }, + { .fno = gen_helper_sve2_sqrshrnb_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + +static bool trans_SQRSHRNT(DisasContext *s, arg_rri_esz *a) +{ + static const GVecGen2i ops[3] = { + { .fno = gen_helper_sve2_sqrshrnt_h }, + { .fno = gen_helper_sve2_sqrshrnt_s }, + { .fno = gen_helper_sve2_sqrshrnt_d }, + }; + return do_sve2_shr_narrow(s, a, ops); +} + static void gen_uqshrnb_vec(unsigned vece, TCGv_vec d, TCGv_vec n, int64_t shr) { From patchwork Fri Apr 30 20:25:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429865 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp786283jao; Fri, 30 Apr 2021 13:54:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwMhn9G0YTWiSSorWRAxNV/5k/McipeOrA5Hn0hunOxiGMlaKiLULASjDWOjc58jLVtu2zv X-Received: by 2002:a92:6908:: with SMTP id e8mr5598789ilc.207.1619816043705; Fri, 30 Apr 2021 13:54:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816043; cv=none; d=google.com; s=arc-20160816; b=Kb2HwoauMNQBpRFCeJynRnYSonJYRNp25nbxx1Cz6uLvus/f8beFzExsBPeob8cTBa MzJBpbozNiysMr4K2ifVgpqeVwatCGGuC2ATK21cNziv9TILeloiDHslIMI6lwixloTN u+DzQduDZdWhq8Mu9a25UN2/iO7Gio+DO1L7jFPoLfBir6C/Mg0olLOhqrCPV8/DPVrp 19V3EWfjE7Nfg9v6qNZOYKD9jAQuTEKiIegRMM+72BYy+BjC0qVVbeVivdZSuanybviK 4LgENkjJ7cPNy+Ymwqk3TyvG7g2dPmZcJ7ut7H/fkV8YDyH5jbZFEOvqT8wPkN+ym1uJ dDIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=yp2+FwV9Iy0XjYM0NO7VMHMNIuodEXGfHOOlSYT2vr0xQEYsCe8HRnjhoovJpBrAnM m7RyBm0pC8AbHlEZp9QXtTMwOB6z52gSv4F2KboQ2XocOW00Mqae11TrONIk8lXj8lo4 cfsmBo61rLLic0A3VgHomTRSoUhGewEwHZnAMHNuKbxnT1YQggvN5Rp9Ao4yqKXGaOM5 ppC5EZb6OYUjCVIjf8kVvgaYTRmEvsTb+Gj00Rr2hydFMtdDJDTDIbIP1G8XAJvRiRxO B9Vj4n8bY8Cvjv+tQA8VWhknEfpCq0r3k8dV0r1wXVbUsqCCh08jkgRhihw3hfhG9Eb1 3XLg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VsuTDF0P; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h16si3691772jaj.74.2021.04.30.13.54.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:54:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VsuTDF0P; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47820 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca9D-0007xn-3A for patch@linaro.org; Fri, 30 Apr 2021 16:54:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52222) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiu-0000ar-Gm for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:52 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]:46671) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiZ-00069i-MQ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:52 -0400 Received: by mail-pl1-x633.google.com with SMTP id s20so21390947plr.13 for ; Fri, 30 Apr 2021 13:26:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=VsuTDF0PpQSXF8U96HzkD0x9Bo4WVe34DB8Q76IqOjv9vqKMAq3y45DddPJkY9ux4m AnLhhPgJ3yCvVvjZDikebA06XKyjcUGT6sawiGBLvahElzeD6cdfkv8gH8Y9JIKlEUrk nsNJsPUAX2HK3i9YH2IML5sfiFL4TKS7A1Kg6ciJ+IOD2AKo6p6sNPhmb3HX5m2tAbu4 FDnhitwwGNX+naQnQJbqU60Y8AEjnWBs1goXbKblIcxf8n19p0OvvAJLpCuVPeO/qkhv kdiC7rBBVkXQwDBmhiaPRzD+j1Uu0fyewK2vFfhRypkyJj0vtifL/bv6lEf1xTjznDv8 Gp3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4NnADjclEJcrO+T+qGPsu3IX/25Q5Wz2cI8VcHKA8p0=; b=TDheKXlOntDz5jyFyXasC74hTsgzhCew4q5wgXwG3VsWvTd+tLWTmTA1jCyq5SOR4u EjVq0dkDaPfHKX09kh9JI5mbuWqC4O9yg73dxEZXqzicBufIu6AkLZeBQC02merkOwLh QXba70VApQfqLOBi2lIU1cESyDtGaJoU0ooC0zm1iDhETqud90LpsUfUygs9gf3OB14i e0AhG+nD1EIqSaUDDhNkqJAd+kRNt5UNWl4d/bumOfp/kxNi/BFfPYVEPqh+lzqbNOZJ 9OSvzUrpeslmZ5HdVFvyGatM8wUJEjwIyyzG6XTsHPikeVQ/MUKiMiATnh/TVcO/yNIE K2AA== X-Gm-Message-State: AOAM531XS/gilAFaTEe/O8V8VOffoC1eYf9paMqp1HXEeZLbV240TmP7 7drPVbmbo1GVqnH7AlnpKVWZKSyVKLMecA== X-Received: by 2002:a17:902:6b43:b029:e6:3d73:e9fb with SMTP id g3-20020a1709026b43b02900e63d73e9fbmr7224334plt.37.1619814390032; Fri, 30 Apr 2021 13:26:30 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 30/82] target/arm: Implement SVE2 WHILEGT, WHILEGE, WHILEHI, WHILEHS Date: Fri, 30 Apr 2021 13:25:18 -0700 Message-Id: <20210430202610.1136687-31-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Rename the existing sve_while (less-than) helper to sve_whilel to make room for a new sve_whileg helper for greater-than. Signed-off-by: Richard Henderson --- v2: Use a new helper function to implement this. v4: Update for PREDDESC. --- target/arm/helper-sve.h | 3 +- target/arm/sve.decode | 2 +- target/arm/sve_helper.c | 38 +++++++++++++++++++++++++- target/arm/translate-sve.c | 56 ++++++++++++++++++++++++++++---------- 4 files changed, 82 insertions(+), 17 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 1c7fe8e417..5bf9fdc7a3 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -913,7 +913,8 @@ DEF_HELPER_FLAGS_4(sve_brkns, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_cntp, TCG_CALL_NO_RWG, i64, ptr, ptr, i32) -DEF_HELPER_FLAGS_3(sve_while, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whilel, TCG_CALL_NO_RWG, i32, ptr, i32, i32) +DEF_HELPER_FLAGS_3(sve_whileg, TCG_CALL_NO_RWG, i32, ptr, i32, i32) DEF_HELPER_FLAGS_4(sve_subri_b, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) DEF_HELPER_FLAGS_4(sve_subri_h, TCG_CALL_NO_RWG, void, ptr, ptr, i64, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0674464695..ae853d21f2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -700,7 +700,7 @@ SINCDECP_z 00100101 .. 1010 d:1 u:1 10000 00 .... ..... @incdec2_pred CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit -WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 1 rn:5 eq:1 rd:4 +WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 ### SVE Integer Wide Immediate - Unpredicated Group diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 02e87c535d..fb38f2c57e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3745,7 +3745,7 @@ uint64_t HELPER(sve_cntp)(void *vn, void *vg, uint32_t pred_desc) return sum; } -uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) +uint32_t HELPER(sve_whilel)(void *vd, uint32_t count, uint32_t pred_desc) { intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); @@ -3771,6 +3771,42 @@ uint32_t HELPER(sve_while)(void *vd, uint32_t count, uint32_t pred_desc) return predtest_ones(d, oprsz, esz_mask); } +uint32_t HELPER(sve_whileg)(void *vd, uint32_t count, uint32_t pred_desc) +{ + intptr_t oprsz = FIELD_EX32(pred_desc, PREDDESC, OPRSZ); + intptr_t esz = FIELD_EX32(pred_desc, PREDDESC, ESZ); + uint64_t esz_mask = pred_esz_masks[esz]; + ARMPredicateReg *d = vd; + intptr_t i, invcount, oprbits; + uint64_t bits; + + if (count == 0) { + return do_zero(d, oprsz); + } + + oprbits = oprsz * 8; + tcg_debug_assert(count <= oprbits); + + bits = esz_mask; + if (oprbits & 63) { + bits &= MAKE_64BIT_MASK(0, oprbits & 63); + } + + invcount = oprbits - count; + for (i = (oprsz - 1) / 8; i > invcount / 64; --i) { + d->p[i] = bits; + bits = esz_mask; + } + + d->p[i] = bits & MAKE_64BIT_MASK(invcount & 63, 64); + + while (--i >= 0) { + d->p[i] = 0; + } + + return predtest_ones(d, oprsz, esz_mask); +} + /* Recursive reduction on a function; * C.f. the ARM ARM function ReducePredicated. * diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 218f1ca5ce..aff85b0220 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3112,7 +3112,14 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) unsigned vsz = vec_full_reg_size(s); unsigned desc = 0; TCGCond cond; + uint64_t maxval; + /* Note that GE/HS has a->eq == 0 and GT/HI has a->eq == 1. */ + bool eq = a->eq == a->lt; + /* The greater-than conditions are all SVE2. */ + if (!a->lt && !dc_isar_feature(aa64_sve2, s)) { + return false; + } if (!sve_access_check(s)) { return true; } @@ -3135,22 +3142,42 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) */ t0 = tcg_temp_new_i64(); t1 = tcg_temp_new_i64(); - tcg_gen_sub_i64(t0, op1, op0); + + if (a->lt) { + tcg_gen_sub_i64(t0, op1, op0); + if (a->u) { + maxval = a->sf ? UINT64_MAX : UINT32_MAX; + cond = eq ? TCG_COND_LEU : TCG_COND_LTU; + } else { + maxval = a->sf ? INT64_MAX : INT32_MAX; + cond = eq ? TCG_COND_LE : TCG_COND_LT; + } + } else { + tcg_gen_sub_i64(t0, op0, op1); + if (a->u) { + maxval = 0; + cond = eq ? TCG_COND_GEU : TCG_COND_GTU; + } else { + maxval = a->sf ? INT64_MIN : INT32_MIN; + cond = eq ? TCG_COND_GE : TCG_COND_GT; + } + } tmax = tcg_const_i64(vsz >> a->esz); - if (a->eq) { + if (eq) { /* Equality means one more iteration. */ tcg_gen_addi_i64(t0, t0, 1); - /* If op1 is max (un)signed integer (and the only time the addition - * above could overflow), then we produce an all-true predicate by - * setting the count to the vector length. This is because the - * pseudocode is described as an increment + compare loop, and the - * max integer would always compare true. + /* + * For the less-than while, if op1 is maxval (and the only time + * the addition above could overflow), then we produce an all-true + * predicate by setting the count to the vector length. This is + * because the pseudocode is described as an increment + compare + * loop, and the maximum integer would always compare true. + * Similarly, the greater-than while has the same issue with the + * minimum integer due to the decrement + compare loop. */ - tcg_gen_movi_i64(t1, (a->sf - ? (a->u ? UINT64_MAX : INT64_MAX) - : (a->u ? UINT32_MAX : INT32_MAX))); + tcg_gen_movi_i64(t1, maxval); tcg_gen_movcond_i64(TCG_COND_EQ, t0, op1, t1, tmax, t0); } @@ -3159,9 +3186,6 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) tcg_temp_free_i64(tmax); /* Set the count to zero if the condition is false. */ - cond = (a->u - ? (a->eq ? TCG_COND_LEU : TCG_COND_LTU) - : (a->eq ? TCG_COND_LE : TCG_COND_LT)); tcg_gen_movi_i64(t1, 0); tcg_gen_movcond_i64(cond, t0, op0, op1, t0, t1); tcg_temp_free_i64(t1); @@ -3181,7 +3205,11 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) ptr = tcg_temp_new_ptr(); tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); - gen_helper_sve_while(t2, ptr, t2, t3); + if (a->lt) { + gen_helper_sve_whilel(t2, ptr, t2, t3); + } else { + gen_helper_sve_whileg(t2, ptr, t2, t3); + } do_pred_flags(t2); tcg_temp_free_ptr(ptr); From patchwork Fri Apr 30 20:25:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429861 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp784496jao; Fri, 30 Apr 2021 13:50:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxu+OqceePh8ojNutaVFLEvIG74nKVn7oCbfhtSVF+xp1sb2FMYNM/ydwjJLgWXebd4c/h9 X-Received: by 2002:a02:cce7:: with SMTP id l7mr6620138jaq.85.1619815845889; Fri, 30 Apr 2021 13:50:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815845; cv=none; d=google.com; s=arc-20160816; b=fm4zc8fMOWGOVz7bKtVGe9na+ic+TaiPEq0dNuv1eC1tIYiduI3hRaj5hs7JEkeZIZ 789I58VHUCM93HCp0jzBFPkJ3DosDYOnZQFEO87taZWTfXfX4ORTOb/m5tyEuNgn37YV wV7ZA0KyoQPZX/LOBwz6hOi/A+2NWIrwmGDKMQdCiYIx+ATJIoanpXNdtnXFCEiKPQIS HjLRwUTeHFmVj2cAm9R9zuWKXH9yQTV/HwNljCIF2Y9zeshLYaSZNbx007bWRMLFhYSY mz3Dm/rM80G7GsYUD9osI7uPJDRuIC7UBbtcK1AijpCGpV4hEc8wh20xvENo2NXMZ9Jw EnuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=JMuYvAumsh7INdsd1PudJOmehSwEhlCEKm4yXO4isd0TAIXaSzDGh2wvHIbQJEngsb DsnDDBRMWqwUfhBcw9fb/kVgQKCqgq8DHHgBIeSSClYi4w5ugDvJflBOFb+T4gIBM3Ey 7PacZYagGZu1bUZ0K0aEaBckQ411td+d0BjWJ3LZfa0+OdcjfJ7zKOaCE8rUl6dEO3a9 yMATYU+trlCYonBtBFboPvd39/bJ1PjLn2PxjHQeWxMtJUGzPdiGaoiHTdemgQ8KPB+c 8wQyJr+g70xGbjxwnpJ6P0+kqO8Vq/O5V3E9sPgnT9sX7z9a+7MqxwGzK6ePRAaivmJs yGaw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EgDCjTgz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q19si3247610jaj.123.2021.04.30.13.50.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:50:45 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EgDCjTgz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35724 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca61-0002x7-7M for patch@linaro.org; Fri, 30 Apr 2021 16:50:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52232) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiv-0000cO-3Q for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:53 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:40942) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZiZ-0006A8-Rn for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:52 -0400 Received: by mail-pf1-x42d.google.com with SMTP id a12so6946803pfc.7 for ; Fri, 30 Apr 2021 13:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=EgDCjTgzHGY9EnajlSErEzUfelodz7q43NajoGuw3DpVGCF5NrfLnb31tKsZiM9LZL vibQs3QG6noSxu0k3aA4dwcyUkgICn93cS4GR64qk+ijsvB5HzHQB7pXpVkbGiw8LnNx esk8Vmgtv2PtwekTaWsWl1ud6bQa2pr6NmtzCGv47eFTBMJFWbqDypFDvHn9l5epJ9fK LZ9dwJKlSrp1dkeLGwK49bja5tGexG7TIt24OW7JPtgRH2Se9Enr0OYTxPQr4F8A4CkP BfONbB3EQ0th3rqHW1FudFdOVdEeooMGoprl1O4QTgbKNgtnIVh6ulTxt3QcQ/NXrRNr MpVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Zvb5+uXwo3ITZTOH3/3hYKLwYLR4lwPy53f9vTvelhk=; b=eQRXmTu9dyBB4PqgfTI8QfHWFLUGAnC7b3rjtbATvx9vIt7tknxd5SatusWu+aFTk4 ilQX5Il5LL7P3IOTZ+Uy5Jy9sAMCRh02Suaf7qniR8iFxLYA5PHumUHRx5f2oR16oZaD yfG+HQBtKtvXopbP6KGSTSPbo0OLZ7BH6mujh5cGuoj2P1qaYqFHeHdGQxEE+FTcnmG7 bsKSzNtHJMPzY7wol7wiNYycadOdW6yIlVqtui47Y6TvByS/PjtA5zOj6LyOu6VtmeTE HYTwFazcn9KUkJeI+e8RokXzhv8GmPmixp+DIBsFw+KG5e3fkGf3lWqllEc2Ldj3MPj9 76Pw== X-Gm-Message-State: AOAM5324nVPR1RzLIud4bCQI7RmoyHAJqsk2Wu6GxRZ8vhtGgmyEfuPR 7kmDadnrp0F6Puj+X5xDBr96LmRXGhRFqg== X-Received: by 2002:aa7:9190:0:b029:22d:6789:cc83 with SMTP id x16-20020aa791900000b029022d6789cc83mr6865402pfa.9.1619814390583; Fri, 30 Apr 2021 13:26:30 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 31/82] target/arm: Implement SVE2 WHILERW, WHILEWR Date: Fri, 30 Apr 2021 13:25:19 -0700 Message-Id: <20210430202610.1136687-32-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix decodetree typo v3: Fix iteration counts (zhiwei). v4: Update for PREDDESC. --- target/arm/sve.decode | 3 ++ target/arm/translate-sve.c | 67 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index ae853d21f2..f365907518 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -702,6 +702,9 @@ CTERM 00100101 1 sf:1 1 rm:5 001000 rn:5 ne:1 0000 # SVE integer compare scalar count and limit WHILE 00100101 esz:2 1 rm:5 000 sf:1 u:1 lt:1 rn:5 eq:1 rd:4 +# SVE2 pointer conflict compare +WHILE_ptr 00100101 esz:2 1 rm:5 001 100 rn:5 rw:1 rd:4 + ### SVE Integer Wide Immediate - Unpredicated Group # SVE broadcast floating-point immediate (unpredicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index aff85b0220..97e113ceec 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3218,6 +3218,73 @@ static bool trans_WHILE(DisasContext *s, arg_WHILE *a) return true; } +static bool trans_WHILE_ptr(DisasContext *s, arg_WHILE_ptr *a) +{ + TCGv_i64 op0, op1, diff, t1, tmax; + TCGv_i32 t2, t3; + TCGv_ptr ptr; + unsigned vsz = vec_full_reg_size(s); + unsigned desc = 0; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (!sve_access_check(s)) { + return true; + } + + op0 = read_cpu_reg(s, a->rn, 1); + op1 = read_cpu_reg(s, a->rm, 1); + + tmax = tcg_const_i64(vsz); + diff = tcg_temp_new_i64(); + + if (a->rw) { + /* WHILERW */ + /* diff = abs(op1 - op0), noting that op0/1 are unsigned. */ + t1 = tcg_temp_new_i64(); + tcg_gen_sub_i64(diff, op0, op1); + tcg_gen_sub_i64(t1, op1, op0); + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, diff, t1); + tcg_temp_free_i64(t1); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op1 == op0, diff == 0, and the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_EQ, diff, op0, op1, tmax, diff); + } else { + /* WHILEWR */ + tcg_gen_sub_i64(diff, op1, op0); + /* Round down to a multiple of ESIZE. */ + tcg_gen_andi_i64(diff, diff, -1 << a->esz); + /* If op0 >= op1, diff <= 0, the condition is always true. */ + tcg_gen_movcond_i64(TCG_COND_GEU, diff, op0, op1, tmax, diff); + } + + /* Bound to the maximum. */ + tcg_gen_umin_i64(diff, diff, tmax); + tcg_temp_free_i64(tmax); + + /* Since we're bounded, pass as a 32-bit type. */ + t2 = tcg_temp_new_i32(); + tcg_gen_extrl_i64_i32(t2, diff); + tcg_temp_free_i64(diff); + + desc = FIELD_DP32(desc, PREDDESC, OPRSZ, vsz / 8); + desc = FIELD_DP32(desc, PREDDESC, ESZ, a->esz); + t3 = tcg_const_i32(desc); + + ptr = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(ptr, cpu_env, pred_full_reg_offset(s, a->rd)); + + gen_helper_sve_whilel(t2, ptr, t2, t3); + do_pred_flags(t2); + + tcg_temp_free_ptr(ptr); + tcg_temp_free_i32(t2); + tcg_temp_free_i32(t3); + return true; +} + /* *** SVE Integer Wide Immediate - Unpredicated Group */ From patchwork Fri Apr 30 20:25:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429864 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp785344jao; Fri, 30 Apr 2021 13:52:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxMXlvb905zGKnnsskiHd1EMk+PwiueY6lMGU6w+qc0DL6u20YIEwd6zb1+47+ekkgNgE4c X-Received: by 2002:a6b:3118:: with SMTP id j24mr5213212ioa.205.1619815940140; Fri, 30 Apr 2021 13:52:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619815940; cv=none; d=google.com; s=arc-20160816; b=oDq+qMaxlIwsTnN1ttGVnWVgBEV9e1OgVo2lG+OBkgAF/O9eVMJxymEsxwTzlT2XGq DfUAZy/O3UI18DK8nkX7qEHGs/u7jhLY2VHWVOGqQ9QiNlue+pB9g6gU/JY7Ecl2W/9x nY5sjNxlosAMvUxUoPsQ4QoX+XGGaF3XCMhRdQLcTS7W62LOgRtaKu43yCaW6cKlUdb2 /mHWoF4vgflxtf4XZpufPWj9lgeAR40hVkoHxeY9NhBnVzXlqEqUSg7ZTcjUXTNn1vJA fynl53Dp4mGJ58tzvKMRv1gF044pzG4FAuc6c6gPClzDsOQW1gFQfmLD0UvSZc/y6fOg a9Pg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=rwuht7QXF4m8PX7CxMLVUSoMwNaEOoJ5mJR7AkvVUgmLCsXT/MfqD6+jOX+8DPdpp7 DSCd9E8dczjoZeEgY+PeBR2MJaHgAQSTESb4l+oYywWR8ob6NAq7ZT7p/PT14z5N1+yH qXR0HclOzyrwpAMMMsuO/ZUNL/5uWa8LUx8NHFX5fbNpg90yn9ga9W5YQ9pd2qHmNbGv kgC9XohvVgqCs845iFFgmVzbnsiE/CvFfqZE94Rz43zV60NXa4pgOpNlsl3qU6aLVPJP ZAVq82gMCEfqDDNwzqBSh/GaVBfdMAazlm6sb64x4s5GQG1AQ5VblzZdt2ygMYgV8S2t ui7w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="TLw2/hJz"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l9si3954420jad.48.2021.04.30.13.52.20 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:52:20 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="TLw2/hJz"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41740 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca7X-0005W5-I8 for patch@linaro.org; Fri, 30 Apr 2021 16:52:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52250) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiv-0000eX-VN for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:54 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]:42735) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZia-0006Ae-EV for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:53 -0400 Received: by mail-pj1-x1030.google.com with SMTP id j6-20020a17090adc86b02900cbfe6f2c96so2378546pjv.1 for ; Fri, 30 Apr 2021 13:26:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=TLw2/hJzAKInLsFW/JlSZEDz1/BQBRU04a/l9ITnivG1YlLfz9z7CrIfvig7ELseBr 8c1+fQBbEI/4EqTUjbx+ho6g0Xjvp9d6IAXnbGlrExfK3IvsqdnLd1AyGe9mxvIcsK7J V+VPBPVdtAYy4ANku7TD9cI4gmqgci5YVo7LCdPZwKzSitagagJdlk2+DK1Z6dqnl2fO jEW33VvlZTM+WPSWjAuS0SWVUIbfvJ0hRZHDS5Fe36TMyeLz3M6vamrjDUIgWBvNqONm iBAowAds5sPMB3Bcfrnpdi/Qy8YM0sJEEgyb0jI+Ij603OsOBwto/2I6uNjcg+XPCX8S EBJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VnUeaFstfauSS4JTjM/wMyZDfm9bVsWwY847CCsEpAM=; b=YnSCUcieeHfNH/9v3GhElAz9dOoMEFoQd6+iVgm9niRB9xebGjNAavvIHf4I428fBX OBV4QzMDNGxnag3SILh5cPW0onhjam9LaQyCR8hEEE2pkJZ10Ks1D1TToMvab3s7p3un MG8jrJ937BAUmLe5qyZqiME2Z2sP0DooNyVJ7bvmDgfBKvI59mh4vm/4NkhnBj1nqLUe 76dab4dSVauEpj2eM0PgGPiZDSPJIbqSfWIfelo37qHOK5Og++LTySPd6qFHm5LordzO kIb7qUfkF+ohxh28RSNF+1+c69iQOjMvmc21VfPZU8IGBe/L0f0XEUWmKNDvwA55D74h SSgg== X-Gm-Message-State: AOAM531Tf/NaeQRBPo7tpw4kHGIWti5pGfspIU1tXnUN+aNrw7fcXLH0 4owq2bPBBtLIzz+bfm/Vw7FLEH6ll56RUg== X-Received: by 2002:a17:902:10b:b029:ed:2b3e:beb4 with SMTP id 11-20020a170902010bb02900ed2b3ebeb4mr7134184plb.64.1619814391119; Fri, 30 Apr 2021 13:26:31 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 32/82] target/arm: Implement SVE2 bitwise ternary operations Date: Fri, 30 Apr 2021 13:25:20 -0700 Message-Id: <20210430202610.1136687-33-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 6 ++ target/arm/sve.decode | 12 +++ target/arm/sve_helper.c | 50 +++++++++ target/arm/translate-sve.c | 213 +++++++++++++++++++++++++++++++++++++ 4 files changed, 281 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 5bf9fdc7a3..df617e3351 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2543,3 +2543,9 @@ DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_fminp_zpzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_eor3, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index f365907518..bf673e2f16 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -124,6 +124,10 @@ @rda_rn_rm ........ esz:2 . rm:5 ... ... rn:5 rd:5 \ &rrrr_esz ra=%reg_movprfx +# Four operand with unused vector element size +@rdn_ra_rm_e0 ........ ... rm:5 ... ... ra:5 rd:5 \ + &rrrr_esz esz=0 rn=%reg_movprfx + # Three operand with "memory" size, aka immediate left shift @rd_rn_msz_rm ........ ... rm:5 .... imm:2 rn:5 rd:5 &rrri @@ -379,6 +383,14 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +# SVE2 bitwise ternary operations +EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BCAX 00000100 01 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 +BSL1N 00000100 01 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +BSL2N 00000100 10 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 +NBSL 00000100 11 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 + ### SVE Index Generation Group # SVE index generation (immediate start, immediate increment) diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fb38f2c57e..b0598f9097 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6792,3 +6792,53 @@ DO_ST1_ZPZ_D(dd_be, zd, MO_64) #undef DO_ST1_ZPZ_S #undef DO_ST1_ZPZ_D + +void HELPER(sve2_eor3)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ m[i] ^ k[i]; + } +} + +void HELPER(sve2_bcax)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = n[i] ^ (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl1n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (~n[i] & k[i]) | (m[i] & ~k[i]); + } +} + +void HELPER(sve2_bsl2n)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = (n[i] & k[i]) | (~m[i] & ~k[i]); + } +} + +void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + uint64_t *d = vd, *n = vn, *m = vm, *k = vk; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 97e113ceec..ab290b9025 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -217,6 +217,17 @@ static void gen_gvec_fn_zzz(DisasContext *s, GVecGen3Fn *gvec_fn, vec_full_reg_offset(s, rm), vsz, vsz); } +/* Invoke a vector expander on four Zregs. */ +static void gen_gvec_fn_zzzz(DisasContext *s, GVecGen4Fn *gvec_fn, + int esz, int rd, int rn, int rm, int ra) +{ + unsigned vsz = vec_full_reg_size(s); + gvec_fn(esz, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), vsz, vsz); +} + /* Invoke a vector move on two Zregs. */ static bool do_mov_z(DisasContext *s, int rd, int rn) { @@ -329,6 +340,208 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzzz(s, fn, a->esz, a->rd, a->rn, a->rm, a->ra); + } + return true; +} + +static void gen_eor3_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_xor_i64(d, d, k); +} + +static void gen_eor3_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_xor_vec(vece, d, d, k); +} + +static void gen_eor3(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_eor3_i64, + .fniv = gen_eor3_vec, + .fno = gen_helper_sve2_eor3, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_EOR3(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_eor3); +} + +static void gen_bcax_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(d, m, k); + tcg_gen_xor_i64(d, d, n); +} + +static void gen_bcax_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_andc_vec(vece, d, m, k); + tcg_gen_xor_vec(vece, d, d, n); +} + +static void gen_bcax(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bcax_i64, + .fniv = gen_bcax_vec, + .fno = gen_helper_sve2_bcax, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BCAX(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bcax); +} + +static void gen_bsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + /* BSL differs from the generic bitsel in argument ordering. */ + tcg_gen_gvec_bitsel(vece, d, a, n, m, oprsz, maxsz); +} + +static bool trans_BSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl); +} + +static void gen_bsl1n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_andc_i64(n, k, n); + tcg_gen_andc_i64(m, m, k); + tcg_gen_or_i64(d, n, m); +} + +static void gen_bsl1n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, n, n); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_andc_vec(vece, n, k, n); + tcg_gen_andc_vec(vece, m, m, k); + tcg_gen_or_vec(vece, d, n, m); + } +} + +static void gen_bsl1n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl1n_i64, + .fniv = gen_bsl1n_vec, + .fno = gen_helper_sve2_bsl1n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL1N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl1n); +} + +static void gen_bsl2n_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + /* + * Z[dn] = (n & k) | (~m & ~k) + * = | ~(m | k) + */ + tcg_gen_and_i64(n, n, k); + if (TCG_TARGET_HAS_orc_i64) { + tcg_gen_or_i64(m, m, k); + tcg_gen_orc_i64(d, n, m); + } else { + tcg_gen_nor_i64(m, m, k); + tcg_gen_or_i64(d, n, m); + } +} + +static void gen_bsl2n_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + if (TCG_TARGET_HAS_bitsel_vec) { + tcg_gen_not_vec(vece, m, m); + tcg_gen_bitsel_vec(vece, d, k, n, m); + } else { + tcg_gen_and_vec(vece, n, n, k); + tcg_gen_or_vec(vece, m, m, k); + tcg_gen_orc_vec(vece, d, n, m); + } +} + +static void gen_bsl2n(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_bsl2n_i64, + .fniv = gen_bsl2n_vec, + .fno = gen_helper_sve2_bsl2n, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_BSL2N(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_bsl2n); +} + +static void gen_nbsl_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, TCGv_i64 k) +{ + tcg_gen_and_i64(n, n, k); + tcg_gen_andc_i64(m, m, k); + tcg_gen_nor_i64(d, n, m); +} + +static void gen_nbsl_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, TCGv_vec k) +{ + tcg_gen_bitsel_vec(vece, d, k, n, m); + tcg_gen_not_vec(vece, d, d); +} + +static void gen_nbsl(unsigned vece, uint32_t d, uint32_t n, uint32_t m, + uint32_t a, uint32_t oprsz, uint32_t maxsz) +{ + static const GVecGen4 op = { + .fni8 = gen_nbsl_i64, + .fniv = gen_nbsl_vec, + .fno = gen_helper_sve2_nbsl, + .vece = MO_64, + .prefer_i64 = TCG_TARGET_REG_BITS == 64, + }; + tcg_gen_gvec_4(d, n, m, a, oprsz, maxsz, &op); +} + +static bool trans_NBSL(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sve2_zzzz_fn(s, a, gen_nbsl); +} + /* *** SVE Integer Arithmetic - Unpredicated Group */ From patchwork Fri Apr 30 20:25:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429868 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp786962jao; Fri, 30 Apr 2021 13:55:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw6Ki6MAvDOLbjzLufzdjUxWLrEJzF3MOeQqodPbBA9wHEMrLKUq8hVupPDhjLtezEOozE7 X-Received: by 2002:a92:360f:: with SMTP id d15mr5986762ila.31.1619816118401; Fri, 30 Apr 2021 13:55:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816118; cv=none; d=google.com; s=arc-20160816; b=eYWzg6PhQiD2EYRom5LgHy8yBDQMAom2f2f/IzNOMiOvFnTevzv75fVmDZopao2G1i 6YKVZBTwi1OcHbk0xW/ZVQqLvjjYLUQrKVgBme90DM/HcmgbytdMZOWCXM4FLZ+l3/K8 eAOgg5q17SbVbOh7OiuOxHwHyd+uVr1MN9bib8N/f2RSnQtlLK/KYgQCtxTIYqV747H2 AiReLJo341F0Hy98b1KJdyiwSuxVvYFYnsyZXfN/EBTsrGkgoWInfXZM0pcEtsq7dQKY CJA+RNNJuoqytfTEzHcdYzQeGNHQLFm5hcwM/i5VLcOTCN1Gi4BIq/wP7bInlwR0G1wZ b3fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=xCzcJ7Ei82m4SWEySBjaMLiv6qKrBBRtBwWEvo2EgDClcLJYek+Xz26Nw8IxG1W4oM flBM1s+n0fP9Rlb1TpuOUJRLhJobQNZdUC8r0Kd3skQMd1fPBr2/hUQwVVJBegSf9Yc9 /R48er+PfAFptRfYvUy5m2qPcQz6kF7uzJBd3+u7CvJ4sYBRS3Zb2cJWbcGJysAbG3oU 07+2lkDuMTAWfqpdd88md6Lc5vS89oy0PlgKlbECOXpONva1PYx/EMt0k10DAfxoGQEx mb0ZieECi7Ce+BOwTsnHAygWrTnk5Pqrq5vhj+zP9ISLCWUaVPUmpnbZU/7hueHjl6xJ GM0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="vz/MBqng"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y9si4728586iot.80.2021.04.30.13.55.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:55:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="vz/MBqng"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50562 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaAP-0000g4-Pg for patch@linaro.org; Fri, 30 Apr 2021 16:55:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52274) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZix-0000i1-7v for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:55 -0400 Received: from mail-pg1-x52e.google.com ([2607:f8b0:4864:20::52e]:46940) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZib-0006BB-16 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:54 -0400 Received: by mail-pg1-x52e.google.com with SMTP id j189so3163479pgd.13 for ; Fri, 30 Apr 2021 13:26:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=vz/MBqngeS8DKNpQJYxYcQ1dyplNGg9d1OPzfZf/01UpQjE9b9A9dPBBVT7kUBlLSr Kdo3Ok1Ar8hCMMfXNga1y4PWucC9oqkcb0JIIi0R4yR3Lt4tSYKrUT+RGilw095vfNix BMBZkEqOBBzZ/qwCS/EtMDb4rGcD3fklCQuf1TdwaIfVCNXqLtAb3OntjisvoLZwODrp kWAK3q3egJZbFM4g3lPblp0waUr+97sO5Wz1ntUc2clb2Arv2XwA4G8GvTMrDqFPIub8 9cumqsX+3ZGxUu319CwkP/BMVQtRwcjJE8jjmYcAsEfOvIhXPQvkr++f58eBonejKBA+ 3O7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=tfgjxN5v8ydM/Pn6KFDCWVXjiic+OopJ/aa+kcSpENo=; b=CejE61dCVm60T6kROr2P1YWq1XPr/9yZ61ADoNlnLrx2IpTWzdMQBaSnsmRfl/LfWK 9ccoojasqEuZ5XtgA3hn2xBNfiLxz7xr/Sknbpd7L5KuLD7tL+qnRPqzOxnh4ZMJY/ke +xuA4s2PhSTwPyUzaRE7Qn6pjqirsefJtWm52DDBooTS3yqGjTtvLux68gMA6JD2V7bZ 05At/ju+k0l4q0JDLTDi6IeGMz1u4pWYRXSwkLKxEpjw/RI2PF6duCfebu561PJBg4eI XoHmsuv2GKxAVKwPPxYQP/4Iin7I4xSL4DZPCXbpAclOZOX/44saQVd3vlviIQUbzsAy Egwg== X-Gm-Message-State: AOAM532T9d0YUB1ZSBQtduvA64CAWK9NZVXwTxbw7blNHpn1gzHXsyff IUYlluWprws+cwTBbRrkPEGTe8CkR4EnSQ== X-Received: by 2002:aa7:9910:0:b029:27e:8025:72bd with SMTP id z16-20020aa799100000b029027e802572bdmr6718776pff.24.1619814391829; Fri, 30 Apr 2021 13:26:31 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:31 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 33/82] target/arm: Implement SVE2 MATCH, NMATCH Date: Fri, 30 Apr 2021 13:25:21 -0700 Message-Id: <20210430202610.1136687-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52e; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Reviewed-by: Richard Henderson Signed-off-by: Stephen Long Message-Id: <20200415145915.2859-1-steplong@quicinc.com> [rth: Expanded comment for do_match2] Signed-off-by: Richard Henderson --- v2: Apply esz_mask to input pg to fix output flags. --- target/arm/helper-sve.h | 10 ++++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 64 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 22 +++++++++++++ 4 files changed, 101 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index df617e3351..11dc6870de 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,16 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, + i32, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index bf673e2f16..47fca5e12d 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +### SVE2 Character Match + +MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm +NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index b0598f9097..010d8b260a 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -6842,3 +6842,67 @@ void HELPER(sve2_nbsl)(void *vd, void *vn, void *vm, void *vk, uint32_t desc) d[i] = ~((n[i] & k[i]) | (m[i] & ~k[i])); } } + +/* + * Returns true if m0 or m1 contains the low uint8_t/uint16_t in n. + * See hasless(v,1) from + * https://graphics.stanford.edu/~seander/bithacks.html#ZeroInWord + */ +static inline bool do_match2(uint64_t n, uint64_t m0, uint64_t m1, int esz) +{ + int bits = 8 << esz; + uint64_t ones = dup_const(esz, 1); + uint64_t signs = ones << (bits - 1); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(esz, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + cmp0 = (cmp0 - ones) & ~cmp0; + cmp1 = (cmp1 - ones) & ~cmp1; + return (cmp0 | cmp1) & signs; +} + +static inline uint32_t do_match(void *vd, void *vn, void *vm, void *vg, + uint32_t desc, int esz, bool nmatch) +{ + uint16_t esz_mask = pred_esz_masks[esz]; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t flags = PREDTEST_INIT; + intptr_t i, j, k; + + for (i = 0; i < opr_sz; i += 16) { + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint16_t pg = *(uint16_t *)(vg + H1_2(i >> 3)) & esz_mask; + uint16_t out = 0; + + for (j = 0; j < 16; j += 8) { + uint64_t n = *(uint64_t *)(vn + i + j); + + for (k = 0; k < 8; k += 1 << esz) { + if (pg & (1 << (j + k))) { + bool o = do_match2(n >> (k * 8), m0, m1, esz); + out |= (o ^ nmatch) << (j + k); + } + } + } + *(uint16_t *)(vd + H1_2(i >> 3)) = out; + flags = iter_predtest_fwd(out, pg, flags); + } + return flags; +} + +#define DO_PPZZ_MATCH(NAME, ESZ, INV) \ +uint32_t HELPER(NAME)(void *vd, void *vn, void *vm, void *vg, uint32_t desc) \ +{ \ + return do_match(vd, vn, vm, vg, desc, ESZ, INV); \ +} + +DO_PPZZ_MATCH(sve2_match_ppzz_b, MO_8, false) +DO_PPZZ_MATCH(sve2_match_ppzz_h, MO_16, false) + +DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) +DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) + +#undef DO_PPZZ_MATCH diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ab290b9025..bdf1da8424 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7462,6 +7462,28 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, + gen_helper_gvec_flags_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_ppzz_flags(s, a, fn); +} + +#define DO_SVE2_PPZZ_MATCH(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ +{ \ + static gen_helper_gvec_flags_4 * const fns[4] = { \ + gen_helper_sve2_##name##_ppzz_b, gen_helper_sve2_##name##_ppzz_h, \ + NULL, NULL \ + }; \ + return do_sve2_ppzz_flags(s, a, fns[a->esz]); \ +} + +DO_SVE2_PPZZ_MATCH(MATCH, match) +DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 30 20:25:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429871 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp788228jao; Fri, 30 Apr 2021 13:57:34 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy3SO01HEuxaV3jXSKGkDo7hl12ZpD0LhEDmTHy72d8H+GIefkWnNQTURalUSu0CM6vvsCp X-Received: by 2002:a92:c56a:: with SMTP id b10mr5655273ilj.64.1619816254369; Fri, 30 Apr 2021 13:57:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816254; cv=none; d=google.com; s=arc-20160816; b=X/JUkW2x1FbrUUnD1RDBxnWxrk3Y/wAltqfqMZ8QQveHOp9zJxydlTZRVkKSpg3qdi TnQHA0JdmpT1RCjIJH5eXuqbORN+8p69lJGX5qaU8Zbb0roNN8+VRF+dl5hby6/S9NPL T1J3hHs3GqwDg+iOxhU8onsugQX28y/uJDhqRDpRphcpFpwFB/tZWCQPWWAxMuN6CAHy f0nlg6ehp45Twhyn/SFQJ199Dh23FEXL6JgkebEWYT89TvKia+7UyvjIJDXKD+xeDue3 7ETbeIEFtwHFpvweXKz7Sf7TRwGuNWzkVCqFRNya4QnjtFJxCYoWHwDNOKOgnCLIqog3 273g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=dYkzfCSlBN8ylqrh3j6KNzpOWfxOI/4xFE+FkqL6AOaGlhCDNeeh1jFb0wJOA3hDS7 UrvFjajGBFp5lB3ibjXy1qxQGPRMdso5Ey394kKOjp6XWgFNrXjbOyTSwX1d+fbFkuJN +BtWdGpdvHZl0wcbSFcljWOTgFCCjkZHLGomrKzGEqVxSzBAG1VDWxrT28H+1GIAhBph RnTSmtytSIjy8K3wMp2Bh1ExVjY7Stk0zx4PxJrSjNVjg36INYOWqZRMRDoKjWr94aJr 3myvqwsRTeyzwp0rTXEmexioxWReu1lrqYguNKk21LTsb+chzjM6sPREAIImRXY1bMMK fQGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ab+xHsrm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q14si3295737ilg.14.2021.04.30.13.57.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:57:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Ab+xHsrm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33482 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaCb-0005J8-KS for patch@linaro.org; Fri, 30 Apr 2021 16:57:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52332) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZiz-0000mb-Qy for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:57 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]:40768) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZib-0006Bv-Mv for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:26:57 -0400 Received: by mail-pl1-x62f.google.com with SMTP id 20so33283906pll.7 for ; Fri, 30 Apr 2021 13:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=Ab+xHsrmulAkSF7LMLGh+7zTQ6hCYhSFU5yfCr/3mk6RDEypqH36Jhi+Xvme+bO14q Br9ghWJSGfCe1H6IinP2Zz0DHXc2TA+5c6uP54rWB2Pt79wufUmZSD3+AmmxwQm4H+ZM PP/BOz9tZmJi4VkSdQTohYmb/F+bVrAyBxjTszM27JvlQaVYMXrf41oivzPOEt0NQAiP 9HnxmzFPMQQ2Z+Uv/o3uz6QAIDeaq1gnfnnTzKZe5sA4LYgAtj2bng69Grklho/MGWSy zeMWN/LpI8IBtGA1kbRFq9bs+EJC0bvn+CiFl0+Jvoq/yxSmVLq5IkjQB5mWou/HR5Gv p+4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wuE0clopIn5KrUkuJnJm6l7c50sb3LTx2vbdW9i3iBQ=; b=tkZzMiA1+IQ6GBlDLRqDngqtfqElWJTUoXnXIaQJM9l/YrgmIOaXI3UxaTWjIF5CfU 5UuaJkPlOJ1P4XZzOXZBZ6gxtmPzhacPQkJboDpM+Yb2jfg0fnOon0bmWqo87kHguV7Z 3k0pjVoye9MR5WOND+V4j4IF5ZsMoezbFs4mmyddgPLCIat3aKNRvnEDr2EJZJMEgqbU XzYKZgc0nEBoNf5h4dWA54vmGMqXEGe0ajEKEV7GaN+xG8Y4sxplWmQ1P1vuHF2ReyS6 A81oqp2ybSL3bmCPR7i0JLW6Jlui+peKaXU2TIOjlRFI8SfG+5EMYBvrjmB0AbinCnTU +kHg== X-Gm-Message-State: AOAM532CR8V99VLqXIPM46fKKvE0nOD+SGWPBpW95rBmstAS+wE6MUv6 jd6W53cYTHEo/dod6+Ry/+HNoIXqXWoFvw== X-Received: by 2002:a17:902:f548:b029:ee:8f40:ecc2 with SMTP id h8-20020a170902f548b02900ee8f40ecc2mr7067210plf.6.1619814392333; Fri, 30 Apr 2021 13:26:32 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 34/82] target/arm: Implement SVE2 saturating multiply-add long Date: Fri, 30 Apr 2021 13:25:22 -0700 Message-Id: <20210430202610.1136687-35-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 ++++++++++ target/arm/sve.decode | 14 ++++++++++ target/arm/sve_helper.c | 30 +++++++++++++++++++++ target/arm/translate-sve.c | 54 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 112 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 11dc6870de..d8f390617c 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2559,3 +2559,17 @@ DEF_HELPER_FLAGS_5(sve2_bcax, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl1n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_bsl2n, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_nbsl, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 47fca5e12d..52f615b39e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1332,3 +1332,17 @@ FMAXNMP 01100100 .. 010 10 0 100 ... ..... ..... @rdn_pg_rm FMINNMP 01100100 .. 010 10 1 100 ... ..... ..... @rdn_pg_rm FMAXP 01100100 .. 010 11 0 100 ... ..... ..... @rdn_pg_rm FMINP 01100100 .. 010 11 1 100 ... ..... ..... @rdn_pg_rm + +#### SVE Integer Multiply-Add (unpredicated) + +## SVE2 saturating multiply-add long + +SQDMLALB_zzzw 01000100 .. 0 ..... 0110 00 ..... ..... @rda_rn_rm +SQDMLALT_zzzw 01000100 .. 0 ..... 0110 01 ..... ..... @rda_rn_rm +SQDMLSLB_zzzw 01000100 .. 0 ..... 0110 10 ..... ..... @rda_rn_rm +SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add interleaved long + +SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm +SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 010d8b260a..859091b7cf 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1400,6 +1400,36 @@ void HELPER(sve2_adcl_d)(void *vd, void *vn, void *vm, void *va, uint32_t desc) } } +#define DO_SQDMLAL(NAME, TYPEW, TYPEN, HW, HN, DMUL_OP, SUM_OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + int sel1 = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + int sel2 = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(TYPEN); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + sel1)); \ + TYPEW mm = *(TYPEN *)(vm + HN(i + sel2)); \ + TYPEW aa = *(TYPEW *)(va + HW(i)); \ + *(TYPEW *)(vd + HW(i)) = SUM_OP(aa, DMUL_OP(nn, mm)); \ + } \ +} + +DO_SQDMLAL(sve2_sqdmlal_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQADD_H) +DO_SQDMLAL(sve2_sqdmlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQADD_S) +DO_SQDMLAL(sve2_sqdmlal_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqadd_d) + +DO_SQDMLAL(sve2_sqdmlsl_zzzw_h, int16_t, int8_t, H1_2, H1, + do_sqdmull_h, DO_SQSUB_H) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, + do_sqdmull_s, DO_SQSUB_S) +DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, + do_sqdmull_d, do_sqsub_d) + +#undef DO_SQDMLAL + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index bdf1da8424..27f9cdb891 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7508,3 +7508,57 @@ DO_SVE2_ZPZZ_FP(FMAXNMP, fmaxnmp) DO_SVE2_ZPZZ_FP(FMINNMP, fminnmp) DO_SVE2_ZPZZ_FP(FMAXP, fmaxp) DO_SVE2_ZPZZ_FP(FMINP, fminp) + +/* + * SVE Integer Multiply-Add (unpredicated) + */ + +static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlal_zzzw_h, + gen_helper_sve2_sqdmlal_zzzw_s, gen_helper_sve2_sqdmlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool do_sqdmlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, + bool sel1, bool sel2) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_sqdmlsl_zzzw_h, + gen_helper_sve2_sqdmlsl_zzzw_s, gen_helper_sve2_sqdmlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], (sel2 << 1) | sel1); +} + +static bool trans_SQDMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, false); +} + +static bool trans_SQDMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, true, true); +} + +static bool trans_SQDMLALBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlal_zzzw(s, a, false, true); +} + +static bool trans_SQDMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, false); +} + +static bool trans_SQDMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, true, true); +} + +static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) +{ + return do_sqdmlsl_zzzw(s, a, false, true); +} From patchwork Fri Apr 30 20:25:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429866 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp786294jao; Fri, 30 Apr 2021 13:54:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwT8ZCB2BGHzeMNeG8PvAM34+JhA5DCaAo3RJ35ws0mgeTzlYYIy0ySPiswmJCo8Klj3Pgu X-Received: by 2002:a05:6e02:1d1b:: with SMTP id i27mr5682086ila.128.1619816045090; Fri, 30 Apr 2021 13:54:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816045; cv=none; d=google.com; s=arc-20160816; b=fK+QY1SLKW16KXWB3bq2tVPG0g1HjPHyR9e9vrzfxovoWBzEibJBpFlKwdtJXgAAOS dMXvpM0SzBYmXpIfNlh4cmNuJcePR5OzjUKgiR0hloIU01SelMCvVigk1dsMCC35WRGu LQQ+vRzvk9fCvSgdU0TTbz6FHIefLgacf4iZVhxhHUKiGzVFmivA1PIhRX6gJr4X6NUA UHOOA5h7go5+ByF3L5i11/ZqrFbj099wN2ph4LZp+iZ84O0TiytdheyrjEzKIQX3SRdG y+KJKW28K8T2EGgaLGvRfnNzIO4x+r0/wq286cJi3KkPMackXF3hT0vYsYI9mRdMNQtY /CDw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=evBh8r5rRwFU+TREl6dgG4MyiO2Zmda+GbppbNBAI/HgyoCGx7DNTRSxWu6JcN4d5c E/a0XIkRI9S8n7Rl1l4Kpf2+0cDl2IzYT50fAcP51Sjh2OuPYx8Zqpk2qa/O2Rag89Xy Mmmd8N5wvBTInOdQxtWP9pKygZi2veaoWJrBC/tv/2lIWwc499a5A5TRCnnHEk6cPDOc XjYugHPoJgg+J+4CBJZ35JFkiVT7G6WQsgcTlUgxAafBGO30gXx+YA2WaIOYszWBT773 blHMBgDDHvNQaVNcRFZEEfQfyDkum53XPzb1aS8FjqyFfDVEd5SgkYb6+umSHwl7TfY9 LymA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zY7B4xdT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x19si3787169jat.51.2021.04.30.13.54.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:54:05 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=zY7B4xdT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:48002 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lca9E-00081o-HC for patch@linaro.org; Fri, 30 Apr 2021 16:54:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52386) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZj6-0000ol-2E for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:05 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]:34647) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZid-0006CU-MJ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:00 -0400 Received: by mail-pf1-x434.google.com with SMTP id 10so6721106pfl.1 for ; Fri, 30 Apr 2021 13:26:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=zY7B4xdTh9Oj79G21DkfvZ7IBYttJk+rt0foO2XtJElSZAQDUqo2SbfJEnKDvaocya ib8lnoet04uYzM9FoAbIoiVyHqVHcFr5Hey20PQZJGjk3gOubvg1yCQGJsbuWMEtJ0Qx JvZf/ywN6TX0o5NH2EMUKWZ6DwV09+we2urpWkecAdAll26Z4/hccZnL02byWUi++Os+ 7jrs9hxjfXCiXTXoE5tGuIBa9wbeM61PpWmeq+AQEuNELytEM1opMtlEG9glGOSRO/NJ nRSryGSVllOS6G6rWsQN2HIlfv9SKwqeIJJ9qUyGxKmHVSlHntmvbXeEH3CdpOstMMAo B0ZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=HrscdamrL4xnD14xZwWaM1FKNhEcxGneRJZ//b9WX/E=; b=HBfeIW6y2hPbztKzXbDNN0v4RsMgCY2F0VYYCzvDUBa4w48YIq0p4oQvJyUGHbSUKf aYE3mf5A93ezIWne9qv/rMncuAR+EekRBTI5WSchDLVkPBk9uTqkObYjyam9GWIdCVhX TIwn0uwrtWXZm0pG21MCda1T9lYIC5do8IlZRWSTD7R5evkJQQMZhSk5/HPoy0eAtrMg ZwZJKdWBn+/Wpy1rpk14A9nYvyLxgu3i0lyHjM0HSbVwGcGSIL7OA2DsxkK7so9w61mw MNK6OWc4JYFwf58Z8KHycbZCSHPEzoRaB9Pt/6w1xDeszRH8MyYV1WQBR+p7HyCtedRE 0Yhg== X-Gm-Message-State: AOAM5309f+18IS4DZW0L0Lt1Ges87lrWrreSbdxRN+Ir3duJ/PfPXQ4D RjBna1jGhPmMy/DdCSHRtuLthLCKwyde9w== X-Received: by 2002:a62:2543:0:b029:278:dfa6:8a4 with SMTP id l64-20020a6225430000b0290278dfa608a4mr6767802pfl.42.1619814392857; Fri, 30 Apr 2021 13:26:32 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 35/82] target/arm: Implement SVE2 saturating multiply-add high Date: Fri, 30 Apr 2021 13:25:23 -0700 Message-Id: <20210430202610.1136687-36-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" SVE2 has two additional sizes of the operation and unlike NEON, there is no saturation flag. Create new entry points for SVE2 that do not set QC. Signed-off-by: Richard Henderson --- target/arm/helper.h | 17 ++++ target/arm/sve.decode | 5 ++ target/arm/translate-sve.c | 18 +++++ target/arm/vec_helper.c | 161 +++++++++++++++++++++++++++++++++++-- 4 files changed, 195 insertions(+), 6 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index 2c412ffd3b..6bb0b0ddc0 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -591,6 +591,23 @@ DEF_HELPER_FLAGS_5(gvec_qrdmlah_s32, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_qrdmlsh_s32, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 52f615b39e..8308c9238a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1346,3 +1346,8 @@ SQDMLSLT_zzzw 01000100 .. 0 ..... 0110 11 ..... ..... @rda_rn_rm SQDMLALBT 01000100 .. 0 ..... 00001 0 ..... ..... @rda_rn_rm SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm + +## SVE2 saturating multiply-add high + +SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm +SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 27f9cdb891..4326b597e6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7562,3 +7562,21 @@ static bool trans_SQDMLSLBT(DisasContext *s, arg_rrrr_esz *a) { return do_sqdmlsl_zzzw(s, a, false, true); } + +static bool trans_SQRDMLAH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlah_b, gen_helper_sve2_sqrdmlah_h, + gen_helper_sve2_sqrdmlah_s, gen_helper_sve2_sqrdmlah_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} + +static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdmlsh_b, gen_helper_sve2_sqrdmlsh_h, + gen_helper_sve2_sqrdmlsh_s, gen_helper_sve2_sqrdmlsh_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b0ce597060..c56337e724 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -22,6 +22,7 @@ #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" +#include "qemu/int128.h" #include "vec_internal.h" /* Note that vector data is stored in host-endian 64-bit chunks, @@ -36,15 +37,55 @@ #define H4(x) (x) #endif +/* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ +static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) +{ + /* + * Simplify: + * = ((a3 << 8) + ((e1 * e2) << 1) + (round << 7)) >> 8 + * = ((a3 << 7) + (e1 * e2) + (round << 6)) >> 7 + */ + int32_t ret = (int32_t)src1 * src2; + if (neg) { + ret = -ret; + } + ret += ((int32_t)src3 << 7) + (round << 6); + ret >>= 7; + + if (ret != (int8_t)ret) { + ret = (ret < 0 ? INT8_MIN : INT8_MAX); + } + return ret; +} + +void HELPER(sve2_sqrdmlah_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], a[i], true, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) { - /* - * Simplify: - * = ((a3 << 16) + ((e1 * e2) << 1) + (1 << 15)) >> 16 - * = ((a3 << 15) + (e1 * e2) + (1 << 14)) >> 15 - */ + /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; if (neg) { ret = -ret; @@ -133,11 +174,35 @@ void HELPER(neon_sqrdmulh_h)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], a[i], true, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) { - /* Simplify similarly to int_qrdmlah_s16 above. */ + /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; if (neg) { ret = -ret; @@ -220,6 +285,90 @@ void HELPER(neon_sqrdmulh_s)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(sve2_sqrdmlah_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], false, true, &discard); + } +} + +void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm, *a = va; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], a[i], true, true, &discard); + } +} + +/* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ +static int64_t do_sat128_d(Int128 r) +{ + int64_t ls = int128_getlo(r); + int64_t hs = int128_gethi(r); + + if (unlikely(hs != (ls >> 63))) { + return hs < 0 ? INT64_MIN : INT64_MAX; + } + return ls; +} + +static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, + bool neg, bool round) +{ + uint64_t l, h; + Int128 r, t; + + /* As in do_sqrdmlah_b, but with 128-bit arithmetic. */ + muls64(&l, &h, m, n); + r = int128_make128(l, h); + if (neg) { + r = int128_neg(r); + } + if (a) { + t = int128_exts64(a); + t = int128_lshift(t, 63); + r = int128_add(r, t); + } + if (round) { + t = int128_exts64(1ll << 62); + r = int128_add(r, t); + } + r = int128_rshift(r, 63); + + return do_sat128_d(r); +} + +void HELPER(sve2_sqrdmlah_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], false, true); + } +} + +void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm, *a = va; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], a[i], true, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 30 20:25:24 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429869 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp787817jao; Fri, 30 Apr 2021 13:56:55 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwOOgT4dA/isJKYrQeFl+htqkVPsFHLzqKCKjjw27zUF+6i5Lfb3dQ4ioigBUp3v8llA0bk X-Received: by 2002:a02:8788:: with SMTP id t8mr6701643jai.135.1619816215885; Fri, 30 Apr 2021 13:56:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816215; cv=none; d=google.com; s=arc-20160816; b=i1HWb7tj3I5tscPcNacGxDAEJJWINrK5QM83GiBbqk7T0Tgpa8hRlwKEG/Vyf+v0/f 7NvAMi6php9jiPa4VLnGmx8A2x1s0P8d7uwtx6kRHydhS3C2daB3RLa951tsKcXVt/1k a/n9J7W/vGjbt8ZhDCFkpiRD3zpY9VNXLm4oCGUp9pmAocQpSE3pwJMgtm5QriZ1j4jV 6JZ4to6w/SkEUgB+X2I63F5TgcWwy2s1487QYtIsk/Ls8ZrcsqHbafahxavKqRaGKSBx k7wzNZ8go5YUutl5UyyCiwg4ArTVsUgZpJipPKVGW6c1nBUj2slcU+WXntgWXNYpMjOD /zgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=0fTsA/r1iN+/sC/SbN6YQ7FkGAAADx9CRengFxJzEZmhp2pRTE7KuNqREgh2zU3I7n K0y5s+pW2Lb2zHkmrcNz6L5hhYWnQe6W4SlY0RlcqT7Nq53phxgehQzgD6yVUi0RdT7w 1VWUkgymSb087MnvEi7g4kct932mMFzkTYA2NKkExva+ziiKFCkjMwHx0HlqEmUWBUd4 MeW5kCr2/iOVODnYev9b/EPj4kvdmLlaQOyXoRJ1g55OH5TIMZnlPqNw0D8snrTTUMes E4jranR7zxouCsxkAjHi+NxLGnbwaJwuTzOemFMKkQ6G6RpMGWZTpBzRO6/d5Yg+FBf9 H4ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Hi1ISgHx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w36si3083433jal.84.2021.04.30.13.56.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:56:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Hi1ISgHx; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56624 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaBz-00039W-8Q for patch@linaro.org; Fri, 30 Apr 2021 16:56:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52388) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZj7-0000pZ-VQ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:05 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]:36525) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZid-0006Ce-MZ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:02 -0400 Received: by mail-pl1-x632.google.com with SMTP id a11so6346862plh.3 for ; Fri, 30 Apr 2021 13:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=Hi1ISgHxIpIls2WC21PNtPV9r5HgjagDikNVbLVwZ2X3OjlRvBGfgi6E5OxXoQLrro Mdc5SkGTVp/KRRdzobyd24EgrI8uGl3hLO6rH1brrmgAB5OMtfFoEyrk1gMkO3+1UvUG tNChIN2ZLOuW2IAQyFdcqsyx2Q1z64nnqi9qE/etqtdEEZEZoL//9iW5bXNqj9MB9xVL 219TLn59L1v1937kvPJH9eTlLaQ4Q5II1oKPZZWlPT/vrK04an1nA7X9ot5SyshoME9o ImX+O6XPglDdfd73qdsb7Xpi0Z8/d1hULPtXKMErpgx+xovR641CqnDgPg8C6OCsDTAq qLvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nJ0tUraTJVmXKzlkMts/F1yRCYj8V5rL50FUHOT1wnQ=; b=pMLqY/s+0BAAsmuDicpAPsmqUK7mhzdO/R7TLQf3+dh2MB1P6ZG2ueF/Jk3EWpOJhE lWBFDPg96onqDOraOZ7CG42nsuRUB6Mjk/kOp0j9qjZLDpsNQcNTG7xSrYUv5xXBZgvc UMwLTXF21eZZln1zY7YnHR0wavUp03XE9X37hh2jXGeEw0iuNzD06Ixj+IWRaJiEQ6sy uOYpBSwpcbdGsnT47UN9H1uYQZr5jHkDRRw3wyfRtVFNXf++MG2MLbaJN5EamVqt93ce J8Nzmtt/nz+M07cZXPRZMm869oqA7WPJ5rQSUmeLr4++VBux6IJs2u9T2TE8VCB34rUL Ltmg== X-Gm-Message-State: AOAM533zq3oKFsExllmAjTGewIYoP20x1XprI/1X9YQ1Wu5pa3NjOl6X 5Unbo5ZkxNDzHy0UBYUQ09CZa5mjsYWTUQ== X-Received: by 2002:a17:90a:b292:: with SMTP id c18mr17544325pjr.179.1619814393505; Fri, 30 Apr 2021 13:26:33 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 36/82] target/arm: Implement SVE2 integer multiply-add long Date: Fri, 30 Apr 2021 13:25:24 -0700 Message-Id: <20210430202610.1136687-37-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 28 ++++++++++++++ target/arm/sve.decode | 11 ++++++ target/arm/sve_helper.c | 18 +++++++++ target/arm/translate-sve.c | 76 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d8f390617c..457a421455 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2573,3 +2573,31 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlal_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_smlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8308c9238a..b28b50e05c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1351,3 +1351,14 @@ SQDMLSLBT 01000100 .. 0 ..... 00001 1 ..... ..... @rda_rn_rm SQRDMLAH_zzzz 01000100 .. 0 ..... 01110 0 ..... ..... @rda_rn_rm SQRDMLSH_zzzz 01000100 .. 0 ..... 01110 1 ..... ..... @rda_rn_rm + +## SVE2 integer multiply-add long + +SMLALB_zzzw 01000100 .. 0 ..... 010 000 ..... ..... @rda_rn_rm +SMLALT_zzzw 01000100 .. 0 ..... 010 001 ..... ..... @rda_rn_rm +UMLALB_zzzw 01000100 .. 0 ..... 010 010 ..... ..... @rda_rn_rm +UMLALT_zzzw 01000100 .. 0 ..... 010 011 ..... ..... @rda_rn_rm +SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm +SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm +UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm +UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 859091b7cf..8b86e7ecd6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1308,6 +1308,24 @@ DO_ZZZW_ACC(sve2_uabal_h, uint16_t, uint8_t, H1_2, H1, DO_ABD) DO_ZZZW_ACC(sve2_uabal_s, uint32_t, uint16_t, H1_4, H1_2, DO_ABD) DO_ZZZW_ACC(sve2_uabal_d, uint64_t, uint32_t, , H1_4, DO_ABD) +DO_ZZZW_ACC(sve2_smlal_zzzw_h, int16_t, int8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_smlal_zzzw_d, int64_t, int32_t, , H1_4, DO_MUL) + +DO_ZZZW_ACC(sve2_umlal_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_MUL) +DO_ZZZW_ACC(sve2_umlal_zzzw_d, uint64_t, uint32_t, , H1_4, DO_MUL) + +#define DO_NMUL(N, M) -(N * M) + +DO_ZZZW_ACC(sve2_smlsl_zzzw_h, int16_t, int8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_s, int32_t, int16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_smlsl_zzzw_d, int64_t, int32_t, , H1_4, DO_NMUL) + +DO_ZZZW_ACC(sve2_umlsl_zzzw_h, uint16_t, uint8_t, H1_2, H1, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_s, uint32_t, uint16_t, H1_4, H1_2, DO_NMUL) +DO_ZZZW_ACC(sve2_umlsl_zzzw_d, uint64_t, uint32_t, , H1_4, DO_NMUL) + #undef DO_ZZZW_ACC #define DO_XTNB(NAME, TYPE, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4326b597e6..0fdfd1e9e0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7580,3 +7580,79 @@ static bool trans_SQRDMLSH_zzzz(DisasContext *s, arg_rrrr_esz *a) }; return do_sve2_zzzz_ool(s, a, fns[a->esz], 0); } + +static bool do_smlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlal_zzzw_h, + gen_helper_sve2_smlal_zzzw_s, gen_helper_sve2_smlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, false); +} + +static bool trans_SMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlal_zzzw(s, a, true); +} + +static bool do_umlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlal_zzzw_h, + gen_helper_sve2_umlal_zzzw_s, gen_helper_sve2_umlal_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, false); +} + +static bool trans_UMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlal_zzzw(s, a, true); +} + +static bool do_smlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_smlsl_zzzw_h, + gen_helper_sve2_smlsl_zzzw_s, gen_helper_sve2_smlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_SMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, false); +} + +static bool trans_SMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_smlsl_zzzw(s, a, true); +} + +static bool do_umlsl_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel) +{ + static gen_helper_gvec_4 * const fns[] = { + NULL, gen_helper_sve2_umlsl_zzzw_h, + gen_helper_sve2_umlsl_zzzw_s, gen_helper_sve2_umlsl_zzzw_d, + }; + return do_sve2_zzzz_ool(s, a, fns[a->esz], sel); +} + +static bool trans_UMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, false); +} + +static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_umlsl_zzzw(s, a, true); +} From patchwork Fri Apr 30 20:25:25 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429880 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp794463jao; Fri, 30 Apr 2021 14:05:29 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz7IxXDqBjhSGVWls22t1XBxagxpIgFpuWw1fUDV5Hi1NhAt+pfi/YYX00fl9SwlxN2qJuB X-Received: by 2002:a6b:b542:: with SMTP id e63mr5400077iof.144.1619816729374; Fri, 30 Apr 2021 14:05:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816729; cv=none; d=google.com; s=arc-20160816; b=xr32ohVU2636iLgGFrf1wQ/79J3K+KHUEG0cVeBvJOOFlRG9D8aCJ2WOCYtbduGhHD wW1gqyyay+p0WfCiMx3vsjTLHFxr+Kig9sU7oa0X+BHC60NWRiFPwkRNzxtN10OChrbD UWJR2YaOlu2XppY6Fg24wGeHeg9Kqg6LC92VzkWLnmy01z5dnVABVpgMilR1/yv8oqBd IuG8gg6d20LK+u20Mkskbp8IPoGoT+41gzuYCf6bX09XMpshppeoCe+Z5TCBtL39pBH4 7sSD4LpNTgXtl5U57mM/hxlct47i/QYBOxBAERUHezq0r+Ujup/JpEaMzumv6MwGYgd4 pVcQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=l8JHfjA/6XIbJqtudy85ci1cMeEhpjxsZRpTASI5M5nQ7ZHSp7zGifLAX+smLlpVQN dW92LwtAKLUxVBo2HnJVBWpNxi/ZWF0JOj8F5Lu8dcNHhI1JTo1ri0UUnJTPbq1gvoKA VW39daI6XLDZDw2renSZ/0BHRMFvuRQDWdCgoTqDbjXy/K7oFGtg+h8QXyrvAR2pND3f g0bFAXWBK/iVWnWN8TG+GzbHj5OqJSL/ZDWoPJocAGupSGQwTzvDxBBB1FlfjaVKbNqw DsaajVlkh7dpBgIao6qJzBBoeEgZgOz8/rC65jLOz7Bt08U5DVSmpoqDFdS+QnqEWiqp Xg2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EuLYShNq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h31si3768735jav.49.2021.04.30.14.05.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:05:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EuLYShNq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53212 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaKG-000544-HM for patch@linaro.org; Fri, 30 Apr 2021 17:05:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52412) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZjD-0000sB-Ns for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:13 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:44743) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZid-0006D8-Ls for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:11 -0400 Received: by mail-pg1-x535.google.com with SMTP id y32so50168327pga.11 for ; Fri, 30 Apr 2021 13:26:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=EuLYShNqUmxSrjnvpVsqSKIdC74QaOOZ/NbgbkNOrCG6GspsUzlf2FvFTAQZ81T6gV HaGvUs9B244pY2aMr5HUEDVtaRLV4lM3NEnri9P+7+egH+EErS+2VJXbX1hcLUKQJN/M DHq3lmFdkReVH/4KQpDc0gR+ULOw3Ri61Wt2zWymECPtrbm2dO5KzR1fjFR0MN1UetYC fqS9mwPhU5uJH637+ijlQTXIfFaHh6X8low6XTymBPA62lcCu3mY9yGM6jcRW82dcmpA CeufPN1Pp7JorE5FUd3Et2sf35AZKcZMLWw6y4qb2Pnb1Fn0UPqAI3w2luTfzpmv2DcS hniQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=W45GPMZZVBfCUkRw+wO8T3FyCeab3ADQB7NYg/wbXMs=; b=FhNirYHJVTYH4W1UuAANuuj0QTFnbXHaaVvn8Poeo568Hq+aQ1WluFJQFx9baLPiQM q4CU8YcBqY7x41EWCVER+lWeCWPoG/8CZuw+MT1XbIf30RNBHp4+aB/NRxUcJkamQY9s DD/NiE3AmSofbTMxBOXevz4n7iFweqylKI/2j2A/UFYGrmKYg3mcFeEIOqircl7QOJ+7 1lSIhaf3F8r9+Bn7QH1sMMkhkddFme72/T4jghT93fxwKvDLnReflnp3rB/nzFaOLBYQ JMmU/RR9nBVU0ZhX9u6H7xvO6PR7A/snWQt8SICBfvFUCOT5YmAXMpv6ecxoeDhpw0dP otTQ== X-Gm-Message-State: AOAM533AGZmVVVSO27HWxYEJT+XHfmsGv8lQ2Xl1EZRiqubHzXfyWpEJ rY16jmD0pF37KojlUsUI1R+FXJXtMTRWdw== X-Received: by 2002:aa7:84d0:0:b029:27d:fff:40c9 with SMTP id x16-20020aa784d00000b029027d0fff40c9mr6427276pfn.7.1619814394100; Fri, 30 Apr 2021 13:26:34 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 37/82] target/arm: Implement SVE2 complex integer multiply-add Date: Fri, 30 Apr 2021 13:25:25 -0700 Message-Id: <20210430202610.1136687-38-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- v2: Fix do_sqrdmlah_d (laurent desnogues) --- target/arm/helper-sve.h | 18 ++++++++++++++++ target/arm/vec_internal.h | 5 +++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 42 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 +++++++++++++++++++++++++++++ target/arm/vec_helper.c | 15 +++++++------- 6 files changed, 109 insertions(+), 8 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 457a421455..d154218452 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2601,3 +2601,21 @@ DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_umlsl_zzzw_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_cmla_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/vec_internal.h b/target/arm/vec_internal.h index 0102547a10..200fb55909 100644 --- a/target/arm/vec_internal.h +++ b/target/arm/vec_internal.h @@ -168,4 +168,9 @@ static inline int64_t do_suqrshl_d(int64_t src, int64_t shift, return do_uqrshl_d(src, shift, round, sat); } +int8_t do_sqrdmlah_b(int8_t, int8_t, int8_t, bool, bool); +int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); +int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); +int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); + #endif /* TARGET_ARM_VEC_INTERNALS_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index b28b50e05c..936977eacb 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1362,3 +1362,8 @@ SMLSLB_zzzw 01000100 .. 0 ..... 010 100 ..... ..... @rda_rn_rm SMLSLT_zzzw 01000100 .. 0 ..... 010 101 ..... ..... @rda_rn_rm UMLSLB_zzzw 01000100 .. 0 ..... 010 110 ..... ..... @rda_rn_rm UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm + +## SVE2 complex integer multiply-add + +CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx +SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8b86e7ecd6..572d41a26c 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1448,6 +1448,48 @@ DO_SQDMLAL(sve2_sqdmlsl_zzzw_d, int64_t, int32_t, , H1_4, #undef DO_SQDMLAL +#define DO_CMLA(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc) / sizeof(TYPE); \ + int rot = simd_data(desc); \ + int sel_a = rot & 1, sel_b = sel_a ^ 1; \ + bool sub_r = rot == 1 || rot == 2; \ + bool sub_i = rot >= 2; \ + TYPE *d = vd, *n = vn, *m = vm, *a = va; \ + for (i = 0; i < opr_sz; i += 2) { \ + TYPE elt1_a = n[H(i + sel_a)]; \ + TYPE elt2_a = m[H(i + sel_a)]; \ + TYPE elt2_b = m[H(i + sel_b)]; \ + d[H(i)] = OP(elt1_a, elt2_a, a[H(i)], sub_r); \ + d[H(i + 1)] = OP(elt1_a, elt2_b, a[H(i + 1)], sub_i); \ + } \ +} + +#define do_cmla(N, M, A, S) (A + (N * M) * (S ? -1 : 1)) + +DO_CMLA(sve2_cmla_zzzz_b, uint8_t, H1, do_cmla) +DO_CMLA(sve2_cmla_zzzz_h, uint16_t, H2, do_cmla) +DO_CMLA(sve2_cmla_zzzz_s, uint32_t, H4, do_cmla) +DO_CMLA(sve2_cmla_zzzz_d, uint64_t, , do_cmla) + +#define DO_SQRDMLAH_B(N, M, A, S) \ + do_sqrdmlah_b(N, M, A, S, true) +#define DO_SQRDMLAH_H(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A, S) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, S, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A, S) \ + do_sqrdmlah_d(N, M, A, S, true) + +DO_CMLA(sve2_sqrdcmlah_zzzz_b, int8_t, H1, DO_SQRDMLAH_B) +DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) +DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) +DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) + +#undef do_cmla +#undef DO_CMLA + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0fdfd1e9e0..6e92abbd8f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7656,3 +7656,35 @@ static bool trans_UMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) { return do_umlsl_zzzw(s, a, true); } + +static bool trans_CMLA_zzzz(DisasContext *s, arg_CMLA_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_cmla_zzzz_b, gen_helper_sve2_cmla_zzzz_h, + gen_helper_sve2_cmla_zzzz_s, gen_helper_sve2_cmla_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} + +static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) +{ + static gen_helper_gvec_4 * const fns[] = { + gen_helper_sve2_sqrdcmlah_zzzz_b, gen_helper_sve2_sqrdcmlah_zzzz_h, + gen_helper_sve2_sqrdcmlah_zzzz_s, gen_helper_sve2_sqrdcmlah_zzzz_d, + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, a->rm, a->ra, a->rot); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index c56337e724..19006f50f7 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -38,8 +38,8 @@ #endif /* Signed saturating rounding doubling multiply-accumulate high half, 8-bit */ -static int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, - bool neg, bool round) +int8_t do_sqrdmlah_b(int8_t src1, int8_t src2, int8_t src3, + bool neg, bool round) { /* * Simplify: @@ -82,8 +82,8 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ -static int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, - bool neg, bool round, uint32_t *sat) +int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int32_t ret = (int32_t)src1 * src2; @@ -199,8 +199,8 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ -static int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, - bool neg, bool round, uint32_t *sat) +int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, + bool neg, bool round, uint32_t *sat) { /* Simplify similarly to do_sqrdmlah_b above. */ int64_t ret = (int64_t)src1 * src2; @@ -321,8 +321,7 @@ static int64_t do_sat128_d(Int128 r) return ls; } -static int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, - bool neg, bool round) +int64_t do_sqrdmlah_d(int64_t n, int64_t m, int64_t a, bool neg, bool round) { uint64_t l, h; Int128 r, t; From patchwork Fri Apr 30 20:25:26 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429875 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp791057jao; Fri, 30 Apr 2021 14:01:33 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiZuGIqt5IPlXmLoWfkpxvIUEaijRjA7BUgzzGg0L97Hg+FI7w0cq+aQw4u94ywDPMBkyV X-Received: by 2002:ac2:5149:: with SMTP id q9mr4455345lfd.361.1619816493819; Fri, 30 Apr 2021 14:01:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816493; cv=none; d=google.com; s=arc-20160816; b=k527SxEXCl9ZIOlZgYQoAfdRBVaWNRQHa4Y/DSlhzqEKhZoIssRAPhCjm5a9J/zfM5 YBVJzAWSVJNAJY15XFAzACJmZCvnFExi3zwxFoDdKX3MXreEpBayYNigiCn6wjFfMntl 45xrZ+d5LS5kbWy/2ro/jBZ+VeP2BJUuQTzS3tlOk4pgvUNqI/znLIKFu3vkJFCOhINe 1scArc7iArfRNHw87ZU9Vq/9RI8J5k2gHzqHI0hecaiKRGmyNU1MOTNOszGWM1sXoZ6t tFVjTvgB8QHLESi4fPEtL328fpvElhbjYdysXJbr9x6I8UZQ/7dao5SwGv2d4FAEAsuz rCbQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=JUUofv2mSz1B8UeQe7VO1FQVNdshmr0/ec9u72Y4kBPijsxvs4hb3Jum63Wx+AVyYE /+BpouEyuimDrL+izWW7OXVsV4HgobM1N8qUVtZvZrdM7SKuuMbW1BggO+Fj8u6vm0vV /uIXbNdxf2oCLs0VBk0qH07Ga6qEcqzrHZc9HLPQUo2K2CDGOxqwI+ZrEcNOR7iSJ7rg +6StQ9unIuvD3Y3Mv3BIuyWZwdldPKIrRbnDCZGfTKi1lPsRxTp7AJT87N+Q6hCoHQPR AxGDFmEdtmLv67BKlWy49Y/Cx1WKNrwL9wQ5mdf5Dnpb8Bn1RmtHwBYTiaQW+3kh/bsQ 9QtQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qBTulA+a; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p6si5161427lfa.194.2021.04.30.14.01.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:01:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qBTulA+a; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44528 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaGS-0001Nc-9B for patch@linaro.org; Fri, 30 Apr 2021 17:01:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52408) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZjD-0000s5-M4 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:11 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:40532) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZie-0006DW-71 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:10 -0400 Received: by mail-pj1-x1029.google.com with SMTP id l10-20020a17090a850ab0290155b06f6267so2339344pjn.5 for ; Fri, 30 Apr 2021 13:26:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=qBTulA+aRJKW7hzHeYeYSBh0+YjgZQB9xA+UgGuMtOJJqFQKvRgNLyexhtH0xYE++3 8imcXTUrOtckRHsWyG0Oz5pDa1I+lWX8UxOZQCWjrGzyilHjTz2PEs5MmLG/8TMrR+ky KuhyC0NyCL0HlUPJ8xir2kVx9AjuNt2FwkOswmzpfT88guyT+vWjmJzV8z2TlypqiEp6 7X1NavU8UZw6/rNgfP+1oUI/Q8kbJn1R2UqxUdPSHDfS7eryjnqjqf6Yrq8jc6e3AUvb GO1f+7uRQ67qvshWJwAjUyUlydQeWFqoPk8iVE8DRx3DU08ED1a1I/jPLsh3MXLSAfyT tGIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jc6bISAZEf9j9mwCzGsWa+M8kmkHcY8KolYL9qX1dMs=; b=cc6rj9HL7HZFg1sLNQHT7gYnmzzdYze/WWaQdC2GNJkzHcU+3u11MAjUksYt6MFfpT M5jePXUx1Z46VivGCWreTWzhILGDWK3Joqo249Rvt0yr0bqeb5cI2et5BiMpDDUI0foD zS93WMnVsSuxBN3nuQilh1B7d8Ug8K7leCqMnaR7/zdJKPlUmK0NwLZBLwuH9M2A6YNm wKp//Vx2bL03m0sD7CsujmdXTXX8vbd7+l7RlGUM/TuRzJV+FMODtJQ5Rds8CbkCZUgD /T/VdCnhNAByVgId1XR5wSR9NQpSOp4J2keMCWKg5rhHROIa4hZhS4+ec5V64LXIWLUH L8Kg== X-Gm-Message-State: AOAM531D0TxHdcC/fxeoG+wa/5IpLmoLkLRg2owbCborZPf6tFKUMom9 537kjiGSBq+fJ0KSFJg2KiuJQq41aWAr2w== X-Received: by 2002:a17:902:e8c1:b029:e9:3c98:2dd2 with SMTP id v1-20020a170902e8c1b02900e93c982dd2mr6926732plg.17.1619814394676; Fri, 30 Apr 2021 13:26:34 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 38/82] target/arm: Implement SVE2 ADDHNB, ADDHNT Date: Fri, 30 Apr 2021 13:25:26 -0700 Message-Id: <20210430202610.1136687-39-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 5 +++++ target/arm/sve_helper.c | 36 ++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 13 +++++++++++++ 4 files changed, 62 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d154218452..a369fd2391 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2509,6 +2509,14 @@ DEF_HELPER_FLAGS_3(sve2_uqrshrnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve2_uqrshrnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 936977eacb..72dd36a5c8 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1320,6 +1320,11 @@ UQSHRNT 01000101 .. 1 ..... 00 1101 ..... ..... @rd_rn_tszimm_shr UQRSHRNB 01000101 .. 1 ..... 00 1110 ..... ..... @rd_rn_tszimm_shr UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr +## SVE2 integer add/subtract narrow high part + +ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm +ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm + ### SVE2 Character Match MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 572d41a26c..2dead1f056 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2112,6 +2112,42 @@ DO_SHRNT(sve2_uqrshrnt_d, uint64_t, uint32_t, , H1_4, DO_UQRSHRN_D) #undef DO_SHRNB #undef DO_SHRNT +#define DO_BINOPNB(NAME, TYPEW, TYPEN, SHIFT, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + i); \ + TYPEW mm = *(TYPEW *)(vm + i); \ + *(TYPEW *)(vd + i) = (TYPEN)OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_BINOPNT(NAME, TYPEW, TYPEN, SHIFT, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, opr_sz = simd_oprsz(desc); \ + for (i = 0; i < opr_sz; i += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + TYPEW mm = *(TYPEW *)(vm + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, mm, SHIFT); \ + } \ +} + +#define DO_ADDHN(N, M, SH) ((N + M) >> SH) + +DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) +DO_BINOPNB(sve2_addhnb_d, uint64_t, uint32_t, 32, DO_ADDHN) + +DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) +DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) + +#undef DO_ADDHN + +#undef DO_BINOPNB + /* Fully general four-operand expander, controlled by a predicate. */ #define DO_ZPZZZ(NAME, TYPE, H, OP) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 6e92abbd8f..86f8a24b5b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7462,6 +7462,19 @@ static bool trans_UQRSHRNT(DisasContext *s, arg_rri_esz *a) return do_sve2_shr_narrow(s, a, ops); } +#define DO_SVE2_ZZZ_NARROW(NAME, name) \ +static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ +{ \ + static gen_helper_gvec_3 * const fns[4] = { \ + NULL, gen_helper_sve2_##name##_h, \ + gen_helper_sve2_##name##_s, gen_helper_sve2_##name##_d, \ + }; \ + return do_sve2_zzz_ool(s, a, fns[a->esz]); \ +} + +DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) +DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Fri Apr 30 20:25:27 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429873 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp788870jao; Fri, 30 Apr 2021 13:58:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxZ1RG5aM5r989wFrr5eh0ydMIsobia4ESnjKRjOJG82vK5GE3LZx/fZjpVnHXrFypqI3RD X-Received: by 2002:a5e:8713:: with SMTP id y19mr5144968ioj.62.1619816321998; Fri, 30 Apr 2021 13:58:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816321; cv=none; d=google.com; s=arc-20160816; b=ytrvCodbJmaYSoH225PhkJvYKwhy6PoWM3DFneyZoprYSFIF62C0CGr38tdb9I5rL3 3HJX2hHYWLqr2veJqR9ULOv6ytEFZcNTkWeFZzKvv3HWk4ZX97gSoRWV+6aQcrkkr+YQ 1NqwgPcdXp9O7/VbEpTBkbUIOYBuyLj6aLRh7KQXZpiWrvcfpHn6hfyfT9kdTgo1lS3O UZbh+a5vfmURMv+dYv4uCag0PI0oj7q1F1DT17CSAQutpc/X1rdN4l75mUwj1NtWVly5 l802EmS0p7YHRJgOmJIJGghvJfUHfxz4Rahk3dQ5hcGxsgyiR3rmUPr4zU+xQ401EYfO lZJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=qzn5m1EcuSNicFyH3V9atGUQ39rpXh9f77VpGgQPKGf/+++DckpEu8PNZrhmiSv6HB CmXMZYQUsWu31xFEihFHzWZyRZmP0QnoqByFYRi2Oukezq0BsO3/W78JwrbtETXf3EIB 9VOhJu752gXJguNUld0MD9VKZGxciwMzVg+TApHKBvp2x/GzYKmaKX8LfrhaClztNuFk VXNVqpvmurhiuSKxXXLQl1EYeEXopg78E26qETcqQ7nuXJdy8H9O9PciYvOYj4Q0JSam A7IcoGzzhGshAyfytnhid7OYx5ocGPEugwBs94geJCUEmnHLJ/zxtjK8mgdlWpe25nv6 mkgw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YeG9Jlsd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z21si4380911ioz.31.2021.04.30.13.58.41 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:58:41 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YeG9Jlsd; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaDh-0006ng-Az for patch@linaro.org; Fri, 30 Apr 2021 16:58:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52398) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZjB-0000rJ-Ne for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:10 -0400 Received: from mail-pg1-x533.google.com ([2607:f8b0:4864:20::533]:34362) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZie-0006De-LC for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:27:07 -0400 Received: by mail-pg1-x533.google.com with SMTP id z16so9255386pga.1 for ; Fri, 30 Apr 2021 13:26:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=YeG9JlsdzFIzsLM6jc7MewVLnPOg/KnT9WgfR1or9lDQCqx+K2fkBoxkexmVp5LZbz M7zQnGk/VAtcjYmO2VCn9foFOYbjMPndsiLOra6TL6i1FrSIVWNh6u4ib20401DwU89C dfi+ucDe3Xpn8kVagJ4hrfNeOvUfjGx12kD7nKzH57LHRh9VAought6GwflKmWEqJ1ec m20Xc4OXJWwAaMdz3VRKO7WOKhart/HKPY+i28nza6euGs70cp81pgS7tvcRW9pkMcS/ H4URQ8ghPkEO7x/KQVp2Kbx6RPY0gSIhGKkR8ORt4iOIw50Orhrrhwjk4ebC40mFTEBB wQew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=JohW9Zep0eTrt4XneckLJzEv6J8bwztIac2YTHSBK88=; b=rpu1qXtaTYaCBDHMFP7C0i0uJAC/pYw5wcFett8ljIJefPeR/rfntkcC4X7nUh4hdk 9g9C9/9gmBmtO503zq2vTOc5MaOuk5/UA575VN6l0l0piGD2+dv4KPac57rqL5dUTvkG 8NJPHrRogUy+yYX4ws9fz/NTrCPxLG6OHKy6dOZGdQRvIv017Tp29eEdauQZl1eT4kgp +EocIn8S++gJSVjlYqTIfVZ3P+eLFa2j7wDRSs/jHVxRYvtDMHbUoaOkCJpJtPLOlN38 npOwrRn0XWOTt0oAl4r87K2i1bKqD46PukARQxZxqAdgwy2ZMePNeEVi03yMiMqvUqQB X9Dg== X-Gm-Message-State: AOAM533jQt+uIcEBY0OwhoNBdbXaWrOvWzlX9wT9ZqSCSggLl/Yku+2v XAKaN2z1umQBRwtt1HgrM99YFWwcjOQRfg== X-Received: by 2002:a63:dc56:: with SMTP id f22mr6275720pgj.287.1619814395359; Fri, 30 Apr 2021 13:26:35 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id e6sm2905257pfd.219.2021.04.30.13.26.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:26:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 39/82] target/arm: Implement SVE2 RADDHNB, RADDHNT Date: Fri, 30 Apr 2021 13:25:27 -0700 Message-Id: <20210430202610.1136687-40-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::533; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x533.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index a369fd2391..8d95c87694 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2517,6 +2517,14 @@ DEF_HELPER_FLAGS_4(sve2_addhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_addhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 72dd36a5c8..dfcfab4bc0 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1324,6 +1324,8 @@ UQRSHRNT 01000101 .. 1 ..... 00 1111 ..... ..... @rd_rn_tszimm_shr ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm +RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm +RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2dead1f056..e6f6e3d5fa 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2135,6 +2135,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ } #define DO_ADDHN(N, M, SH) ((N + M) >> SH) +#define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2144,6 +2145,15 @@ DO_BINOPNT(sve2_addhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_ADDHN) DO_BINOPNT(sve2_addhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_ADDHN) DO_BINOPNT(sve2_addhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_ADDHN) +DO_BINOPNB(sve2_raddhnb_h, uint16_t, uint8_t, 8, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_s, uint32_t, uint16_t, 16, DO_RADDHN) +DO_BINOPNB(sve2_raddhnb_d, uint64_t, uint32_t, 32, DO_RADDHN) + +DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) +DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) + +#undef DO_RADDHN #undef DO_ADDHN #undef DO_BINOPNB diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 86f8a24b5b..af0d0ab279 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7474,6 +7474,8 @@ static bool trans_##NAME(DisasContext *s, arg_rrr_esz *a) \ DO_SVE2_ZZZ_NARROW(ADDHNB, addhnb) DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) +DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) +DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Fri Apr 30 20:25:28 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429876 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp791416jao; Fri, 30 Apr 2021 14:02:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzwfNYU7eg5Wv+JZl/Q9BU3bUoD1BcC/0rIHqXoTKmohQoMMWxOzacQ6rXIrP+tuUMbAMQJ X-Received: by 2002:a19:6b0a:: with SMTP id d10mr4838214lfa.310.1619816519943; Fri, 30 Apr 2021 14:01:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816519; cv=none; d=google.com; s=arc-20160816; b=mQ4cjbfaj63uYSUa+Cgcjx+6sZBi0ZokvoLkU2jbprK4XFqBo2aXGhhmE7aLiQ7ukx lc63cip1GRpHEcT8LCGJAcQhi8765uCDZKoOgKlNoUH2CJ+DPrZSkw0Rcvjpr9uDq9S3 1JQt/tO5RyqKu7EZXj3qv1E0afPUwUNz9mGFz4tRjhDezTeAye0OJR8wksfQSA6b4S6z kRZKTHOeVJHjUZehJ3PYrukVADgDfb4t+dhvWZNezH/ekXkA819OZo7ACYe8bS6ygy6Z MRiysnzu/WUC2KevMY69wBLjiXZ2rJV9fdA9fUzp4cMmIGM6hv2lJJpI9ZBl3grF+Wp7 Y0Ow== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=XK2vwR+tVPf2qfmuxx8XpzCCbqGxyBUNMEj7r4DMSeFSVnB/rVtD2+6aNltTbpwBDH j6WS0ILV8knpprKD7/T85fzZ2a360cl2mSyw/RSOMbpFvHfmroaSyN2vYQVnKBNBGQPe eX16zhXEf+j4NSP0eoKP/ozBPdMVRMjo5/jMAtMqhqfDOO2br8KbDXoDowc19Gc4dMaE EBNdcW1Up6HjLJkQiUIoi7baawG2C4xdcj9w9uZPrNCb39E8psWb0QTOA24kOCIvCijT ysRhLid3qtB5YmYtFEFLJGMZk5gVL1i42bLIAMBKsmfAmcO+sKTHl/+5Ftg6n7gQgCV3 CNMw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lnyJEec3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w4si3502243ljm.244.2021.04.30.14.01.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:01:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lnyJEec3; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46022 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaGs-00024m-Er for patch@linaro.org; Fri, 30 Apr 2021 17:01:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52880) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZlm-00049V-Cr for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:29:50 -0400 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]:55079) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZla-0007fe-0p for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:29:48 -0400 Received: by mail-pj1-x1030.google.com with SMTP id t13so10987103pji.4 for ; Fri, 30 Apr 2021 13:29:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=lnyJEec3yYsHoCQMnWYIIDb3FEvXZHLqxZ48470RL/NJwwW0IBod0PIIOqev7Gm6Jb DaYFwwTJEeRA92i1avMseC7FefX4DSg8SBOcDClou3xFzHNbH9VB0QCM4f5oUpCUBMtA 5Y/2oQ5PAsswrMZFv/6IqUerJE/5PRen/E9449k6J34yDoJRoA7O6wU/P1IsR7a2U9hS pvO05xLbEuf3pcrq94nTM6TVzSWrlEvjLG3yScnZL43VQ4qyl+RJkmPSfqXdStXxmytH J7CAdsO9SE3H9vldMArUZrrSGhdF+96J0/bFq/anjifnRPQQ2jHcfVuGxMX+qEq40CgD Sgtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ieCAX8JfYanxGz+tnZF3xTMiEcjdmHvb76wqgRW4ePI=; b=og6fo6UJ4ZNHaP+Ho6T9X2xhjH29tBSsN3rUlqzUerykQ0InFtIFaSA+6Ds49oPGZ9 QimcH1yvqJvPkaU1wUAdOXRhpQ2xyWdZkfcs0mAPfmvbsX17TWCqYDB6WiFKYhIBdW/Q XsuwACVAC2ASBr/uBRfVhEZRHL1mWM+2Q96d/1CHbwZYWZZJud/fFlqZswjfPfDlSg8A DQiJhN23ORTKphsF0rkMVpao8QrIDfI/+JrXbAfsSxzA8gb+ZG2IOSfmSxm2vTbtRRfW v0pjuOHGzo4+2BGD8WdN6Eu/qqvPA5Piy007i1qrEUSMEIyGeMVWT1hu5ZJZYPAeOL8h AEWA== X-Gm-Message-State: AOAM533ITr5VohTPyW+RBF+MbQ7PngSVGcV32pFdb+ExyC93bIDdA3wg Qs9uRXt6MWtxGimJo2UEXe8E9g8k+V76bg== X-Received: by 2002:a17:90a:4108:: with SMTP id u8mr7151941pjf.126.1619814576358; Fri, 30 Apr 2021 13:29:36 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 40/82] target/arm: Implement SVE2 SUBHNB, SUBHNT Date: Fri, 30 Apr 2021 13:25:28 -0700 Message-Id: <20210430202610.1136687-41-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1030; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1030.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-4-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 3 +++ 4 files changed, 23 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 8d95c87694..3642e7c820 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2525,6 +2525,14 @@ DEF_HELPER_FLAGS_4(sve2_raddhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_raddhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index dfcfab4bc0..c68bfcf6ed 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1326,6 +1326,8 @@ ADDHNB 01000101 .. 1 ..... 011 000 ..... ..... @rd_rn_rm ADDHNT 01000101 .. 1 ..... 011 001 ..... ..... @rd_rn_rm RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm +SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm +SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index e6f6e3d5fa..0df70effe3 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2136,6 +2136,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) +#define DO_SUBHN(N, M, SH) ((N - M) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2153,6 +2154,15 @@ DO_BINOPNT(sve2_raddhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RADDHN) DO_BINOPNT(sve2_raddhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RADDHN) +DO_BINOPNB(sve2_subhnb_h, uint16_t, uint8_t, 8, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_s, uint32_t, uint16_t, 16, DO_SUBHN) +DO_BINOPNB(sve2_subhnb_d, uint64_t, uint32_t, 32, DO_SUBHN) + +DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) +DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) + +#undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index af0d0ab279..55303ba41d 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7477,6 +7477,9 @@ DO_SVE2_ZZZ_NARROW(ADDHNT, addhnt) DO_SVE2_ZZZ_NARROW(RADDHNB, raddhnb) DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) +DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) +DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) + static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) { From patchwork Fri Apr 30 20:25:29 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429883 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp796508jao; Fri, 30 Apr 2021 14:08:07 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwUjSYm0lUtI4a1C5I1YhrHQnGE2rGggy69RpHfUdEqj+WJpwSqT0auVoKiNeR+Hs842R1J X-Received: by 2002:aca:3c09:: with SMTP id j9mr12561779oia.28.1619816887849; Fri, 30 Apr 2021 14:08:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816887; cv=none; d=google.com; s=arc-20160816; b=Wtg3aRI0uXZvuzSxXxiuc157UBuAbq/rt77XS+yrzHVJcnmzkiQSXWqiWBjKw3U1Um 8Josx2ZmWI+U6oIc2bJWdCYEZmORm5Scgyud/PdcnevlcmfsVwcfX73gGaZNBvo3sswU HfbFPN1hbfBFW2VhTBltYFH2bn1StuXAEFgNtccbODVr/Uby18K8rWfxLbXi/KUWRle2 OqeNmtIzVxrjmH/8ZttJZXbKXrH6cFIXVBiBYwvOfaIGMqhuGrU+85Zkna9BOxqbuKIY 77BzBtsOhhQS5lEavSoRABYqypvsDFc70FAWug7I/hsG1d9EinkHsZrS5n9iXXQv7qxd +6Qg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=ZxJBSMCxdxd1LRpNA+yrRGLRkOgWpsaWP7c09b2pltE8mHxXcFBElC2Mq6htUlsN4E r7R+sjvTQG3vRRzHKe9HQWis4KPEbNSCWNy25IkvgN1FL92TYmS3W4nClNGa0wMugXny ncamjBbCqWojeT4Ph9IzPqL+JvYhJ3/uK4F2SlJuZk0lXmEMgfon6olYw7acYpEOKbeS OEBmOMgR1Ae870vKK8GXQPbnviEUjNNRf79r8AB1benwQzoFke/NImNQsZ/V/Bgp43lI gxgjA6paZNJh1BUfMCFbFBg4sdnYzcKMxdK3k4mXrMpNBm1G9Boe/8D1f8mrqYC6vbAy 4G+Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vnoBMCqe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e2si5056102oiw.99.2021.04.30.14.08.07 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:08:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vnoBMCqe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaMp-0001Ez-23 for patch@linaro.org; Fri, 30 Apr 2021 17:08:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0004Bs-4r for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:01 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:38742) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZla-0007fj-LM for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:29:56 -0400 Received: by mail-pj1-x1035.google.com with SMTP id f11-20020a17090a638bb02901524d3a3d48so2349635pjj.3 for ; Fri, 30 Apr 2021 13:29:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=vnoBMCqefZNZ9VpYg2h5lrMtMFDTBEPrXU2HYwLeYcv/AdItpb+tk2nM1LwFQSeZAs 6vMGgt0Gf6Gq+XbigpQawro75fI+76FmpP1h2Fcj85SgyYJ4l26tiKrYgzc0FGG7ztHm +xAkb8ELirzPsTCcLJ+LnmDWZ1wEOiX/mVrWruOcVKNC/S3kXDx+5kGvipcBt62cQPrQ 8FYvfNnUgeG6SZXu7X8neWu+KIb5rEzhwoiFRKTrSNmjnOezA6l9GHqggPADJcQKccrB 4frkNT1JI8EAgQloZnJIC7RXsO/lqIO3+u2TPb9oRFugBEKYx99PNgT8wt1jOe2woRBn ElRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=L9CT6IY88WuROz45EJLGs6Vb8eBbIgK6UvA8l1eIoaI=; b=jma3oI4dqJIR6eP9BxL+2PUDiNBh8xE+qvzJF6mux8Se7p72tLQSHKVGQ+AnVClxK5 MYu/P74llb7aC3Ie7JRBbtTq4ax1dbjJaRAKJou4QWbU5TGOuypWVgpJM8W3lnMGVjWD JAA93nddZawlsdWdzIJy3q9k+V2cedWsDhNksL6lV+m3ofleynVbzQr7WiAXJWM5GLZt wcsjh/7fPLFZgi0Qv7PRjYs2c18Ujcj0k8d0/PE01O13xqiT/9zh5wjEO68WfYS2yoUQ fo+2UGaY/ehQyQCfmaIQ9+OvmTEOdrNeroVGG61lvjy/0ozkD3rHDe0nYnKN6bHcndJh dc5g== X-Gm-Message-State: AOAM531DGdoXUGKdxzsxYHtIf0MGcLPmy8bzRIi/QiUPfUzi4ECvGKii vfcUvDy5HNcFheQZNpmX9Zr8SjYg9UtGQw== X-Received: by 2002:a17:90a:d184:: with SMTP id fu4mr16434352pjb.79.1619814576946; Fri, 30 Apr 2021 13:29:36 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 41/82] target/arm: Implement SVE2 RSUBHNB, RSUBHNT Date: Fri, 30 Apr 2021 13:25:29 -0700 Message-Id: <20210430202610.1136687-42-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long This completes the section 'SVE2 integer add/subtract narrow high part' Signed-off-by: Stephen Long Message-Id: <20200417162231.10374-5-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix round bit type (laurent desnogues) --- target/arm/helper-sve.h | 8 ++++++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 10 ++++++++++ target/arm/translate-sve.c | 2 ++ 4 files changed, 22 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 3642e7c820..98e6b57e38 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2533,6 +2533,14 @@ DEF_HELPER_FLAGS_4(sve2_subhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_subhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_rsubhnt_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_rsubhnt_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_match_ppzz_b, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_match_ppzz_h, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c68bfcf6ed..388bf92acf 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1328,6 +1328,8 @@ RADDHNB 01000101 .. 1 ..... 011 010 ..... ..... @rd_rn_rm RADDHNT 01000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm SUBHNB 01000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm SUBHNT 01000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +RSUBHNB 01000101 .. 1 ..... 011 110 ..... ..... @rd_rn_rm +RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm ### SVE2 Character Match diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 0df70effe3..12acc4fb0b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2137,6 +2137,7 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ #define DO_ADDHN(N, M, SH) ((N + M) >> SH) #define DO_RADDHN(N, M, SH) ((N + M + ((__typeof(N))1 << (SH - 1))) >> SH) #define DO_SUBHN(N, M, SH) ((N - M) >> SH) +#define DO_RSUBHN(N, M, SH) ((N - M + ((__typeof(N))1 << (SH - 1))) >> SH) DO_BINOPNB(sve2_addhnb_h, uint16_t, uint8_t, 8, DO_ADDHN) DO_BINOPNB(sve2_addhnb_s, uint32_t, uint16_t, 16, DO_ADDHN) @@ -2162,6 +2163,15 @@ DO_BINOPNT(sve2_subhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_SUBHN) DO_BINOPNT(sve2_subhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_SUBHN) DO_BINOPNT(sve2_subhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_SUBHN) +DO_BINOPNB(sve2_rsubhnb_h, uint16_t, uint8_t, 8, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_s, uint32_t, uint16_t, 16, DO_RSUBHN) +DO_BINOPNB(sve2_rsubhnb_d, uint64_t, uint32_t, 32, DO_RSUBHN) + +DO_BINOPNT(sve2_rsubhnt_h, uint16_t, uint8_t, 8, H1_2, H1, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_s, uint32_t, uint16_t, 16, H1_4, H1_2, DO_RSUBHN) +DO_BINOPNT(sve2_rsubhnt_d, uint64_t, uint32_t, 32, , H1_4, DO_RSUBHN) + +#undef DO_RSUBHN #undef DO_SUBHN #undef DO_RADDHN #undef DO_ADDHN diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 55303ba41d..49d7a45a50 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7479,6 +7479,8 @@ DO_SVE2_ZZZ_NARROW(RADDHNT, raddhnt) DO_SVE2_ZZZ_NARROW(SUBHNB, subhnb) DO_SVE2_ZZZ_NARROW(SUBHNT, subhnt) +DO_SVE2_ZZZ_NARROW(RSUBHNB, rsubhnb) +DO_SVE2_ZZZ_NARROW(RSUBHNT, rsubhnt) static bool do_sve2_ppzz_flags(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_flags_4 *fn) From patchwork Fri Apr 30 20:25:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429882 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp795049jao; Fri, 30 Apr 2021 14:06:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzX8j4WVInQhssFg3RVenlnHyw6HqrmR4vVZgmogJtbHbwxFuVwUsuvZxxkk47BhjglVGsf X-Received: by 2002:a05:6e02:eeb:: with SMTP id j11mr5484103ilk.23.1619816776494; Fri, 30 Apr 2021 14:06:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816776; cv=none; d=google.com; s=arc-20160816; b=wXJ7ONzR4JoQQNgtYS97i6XaVVmYUexyfOd7j2euSaN5ctPaoROgkLq0AeJP60h1+B jGYz+ueJDycXBmptdkGeHyaxTofmUDlcbFkp65r9QMF7rKWXHHHADO6D3z5qWEDQXEpI 0zMTI3axLb0fjAsfQ3WkJWAqehE3slGNBHdIYFgO1ekjSk0Wi7zc/HGlhJj6TF/vkdfE uFCge062QjGzACLzkSMialH8TikuGpepphnlv38U5cTDzw7ddcnoAizm/PG9tLSjZcc1 ybQIew9xlw6BfXEWaIf2f17IaUnU4XLssy4VnMSI1BrH7+yG8LpwprsCrZZmtvUOgtIx Q/Sw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=zZAGk+6Ei7T+8NVqnWeh6O2FmxBlBWU6e+jsdHTM7/vUBrN08VzWJoOYg2jGADoapL Gqbym54VG+lI23FbwQ+xoL0KOhPDej8IUnmsX9++nJ3JvGkDploEuhYV37lqqOXroV5T 90EjYSkkEWC4yMG4xsJQtHkl8C5+p8birP/A7lHyKugPSy1A8e3lR6MfRAAwVLls8Vv1 isuWy67ZLSdXCW3/O6esI2N7zjSMVAV0mpMKw9Z46Keh269o0xp2gXQc0+wagDG999wR BRMDUOQ246jvEteFlW0kXCTcgRFqAZKOjow70HfxbJ7RAp+FXN9VhRSwgi/q0fJ65+/p 86vA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jt7cUdL4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v11si4199882ilu.142.2021.04.30.14.06.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:06:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jt7cUdL4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55794 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaL1-00066T-S2 for patch@linaro.org; Fri, 30 Apr 2021 17:06:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:52954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0004AB-Iq for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:29:50 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:55080) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZla-0007ft-Om for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:29:50 -0400 Received: by mail-pj1-x1031.google.com with SMTP id t13so10987129pji.4 for ; Fri, 30 Apr 2021 13:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=jt7cUdL4WR0KuW//BJJrW5ZDUfraoxz574yhMMFq6gCdc+TmQTle4oyeJHrVqddzVW Uf+M3L37zm8amccQkU/zPgpSZtO/L0J7cjMjTPNEITRJSN2Lbyzol5h7KOJ9cOiMowE+ qWFyl232y7TMtpl7zIC47xC47kMzI10BwAX7tEhriHSWZalkoKFPd7Ke8TfqODQdH78c 63ZYezOELWSJ12gcR6InZsSfUFaxnqdUpca8dkYllMN4BN4KxdCPo++hZ5RbAEVi4zgf OCsAJTMl2l/GWTA2b/cFLhTWnKGvNKUo5rOjJEx7wYPAl39yie3amDjW9CxPSILX6iVo Oseg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=SGOOa0zT0FE1S29kkqgncHYFmVqbSwpz80PVFtDGE1o=; b=lul9E5wbwcOYGTjU9NBdlcr6bUxQKt+e4tVBX49/4SEE4swUZ4/lJJqouLUgMolT1Q wcrOlRqXH6KDdwYkOkcfraWqZvCfbry73bajkkiSe5bmD8/Psd5UHnApN+YdeXsDBUJ/ FClRY5aNbgGB4LAdndwh4fhXRlFID0gGGE4XvDPiJxlgU0ET96j986Cgjhdw7mCpHeic W6hg7CkZXnAdCm6KmVB/zV4/2UCykp6WyNvcC0xNBvkWci2RaL8/3nlMuyh0b8TL6qwO CopJWbiw1LBtSyph3R/LkcFfcVCLTt0pSuNeZ6kq4ieX8X1tkUyoJbjWhWihdBhB+3jb rPcQ== X-Gm-Message-State: AOAM532pZXzJz/aQdcBbHSgwQtTzldF1NEOeUm1C4c00ArPNKEFuE/M4 sdB0us3l4/eWXnBTdiENF2wvfgUSq1NeAA== X-Received: by 2002:a17:90a:950c:: with SMTP id t12mr16527013pjo.135.1619814577479; Fri, 30 Apr 2021 13:29:37 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 42/82] target/arm: Implement SVE2 HISTCNT, HISTSEG Date: Fri, 30 Apr 2021 13:25:30 -0700 Message-Id: <20210430202610.1136687-43-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200416173109.8856-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fix overlap between output and input vectors. v4: Fix histseg counting (zhiwei). --- target/arm/helper-sve.h | 7 ++ target/arm/sve.decode | 6 ++ target/arm/sve_helper.c | 131 +++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 19 ++++++ 4 files changed, 163 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 98e6b57e38..507a2fea8e 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2551,6 +2551,13 @@ DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_b, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_nmatch_ppzz_h, TCG_CALL_NO_RWG, i32, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 388bf92acf..8f501a083c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -146,6 +146,7 @@ &rprrr_esz rn=%reg_movprfx @rdn_pg_rm_ra ........ esz:2 . ra:5 ... pg:3 rm:5 rd:5 \ &rprrr_esz rn=%reg_movprfx +@rd_pg_rn_rm ........ esz:2 . rm:5 ... pg:3 rn:5 rd:5 &rprr_esz # One register operand, with governing predicate, vector element size @rd_pg_rn ........ esz:2 ... ... ... pg:3 rn:5 rd:5 &rpr_esz @@ -1336,6 +1337,11 @@ RSUBHNT 01000101 .. 1 ..... 011 111 ..... ..... @rd_rn_rm MATCH 01000101 .. 1 ..... 100 ... ..... 0 .... @pd_pg_rn_rm NMATCH 01000101 .. 1 ..... 100 ... ..... 1 .... @pd_pg_rn_rm +### SVE2 Histogram Computation + +HISTCNT 01000101 .. 1 ..... 110 ... ..... ..... @rd_pg_rn_rm +HISTSEG 01000101 .. 1 ..... 101 000 ..... ..... @rd_rn_rm + ## SVE2 floating-point pairwise operations FADDP 01100100 .. 010 00 0 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 12acc4fb0b..8d002fdb65 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7062,3 +7062,134 @@ DO_PPZZ_MATCH(sve2_nmatch_ppzz_b, MO_8, true) DO_PPZZ_MATCH(sve2_nmatch_ppzz_h, MO_16, true) #undef DO_PPZZ_MATCH + +void HELPER(sve2_histcnt_s)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint32_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz; i += 4) { + uint64_t count = 0; + uint8_t pred; + + pred = pg[H1(i >> 3)] >> (i & 7); + if (pred & 1) { + uint32_t nn = n[H4(i >> 2)]; + + for (j = 0; j <= i; j += 4) { + pred = pg[H1(j >> 3)] >> (j & 7); + if ((pred & 1) && nn == m[H4(j >> 2)]) { + ++count; + } + } + } + d[H4(i >> 2)] = count; + } +} + +void HELPER(sve2_histcnt_d)(void *vd, void *vn, void *vm, void *vg, + uint32_t desc) +{ + ARMVectorReg scratch; + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + uint64_t *d = vd, *n = vn, *m = vm; + uint8_t *pg = vg; + + if (d == n) { + n = memcpy(&scratch, n, opr_sz); + if (d == m) { + m = n; + } + } else if (d == m) { + m = memcpy(&scratch, m, opr_sz); + } + + for (i = 0; i < opr_sz / 8; ++i) { + uint64_t count = 0; + if (pg[H1(i)] & 1) { + uint64_t nn = n[i]; + for (j = 0; j <= i; ++j) { + if ((pg[H1(j)] & 1) && nn == m[j]) { + ++count; + } + } + } + d[i] = count; + } +} + +/* + * Returns the number of bytes in m0 and m1 that match n. + * Unlike do_match2 we don't just need true/false, we need an exact count. + * This requires two extra logical operations. + */ +static inline uint64_t do_histseg_cnt(uint8_t n, uint64_t m0, uint64_t m1) +{ + const uint64_t mask = dup_const(MO_8, 0x7f); + uint64_t cmp0, cmp1; + + cmp1 = dup_const(MO_8, n); + cmp0 = cmp1 ^ m0; + cmp1 = cmp1 ^ m1; + + /* + * 1: clear msb of each byte to avoid carry to next byte (& mask) + * 2: carry in to msb if byte != 0 (+ mask) + * 3: set msb if cmp has msb set (| cmp) + * 4: set ~msb to ignore them (| mask) + * We now have 0xff for byte != 0 or 0x7f for byte == 0. + * 5: invert, resulting in 0x80 if and only if byte == 0. + */ + cmp0 = ~(((cmp0 & mask) + mask) | cmp0 | mask); + cmp1 = ~(((cmp1 & mask) + mask) | cmp1 | mask); + + /* + * Combine the two compares in a way that the bits do + * not overlap, and so preserves the count of set bits. + * If the host has an efficient instruction for ctpop, + * then ctpop(x) + ctpop(y) has the same number of + * operations as ctpop(x | (y >> 1)). If the host does + * not have an efficient ctpop, then we only want to + * use it once. + */ + return ctpop64(cmp0 | (cmp1 >> 1)); +} + +void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j; + intptr_t opr_sz = simd_oprsz(desc); + + for (i = 0; i < opr_sz; i += 16) { + uint64_t n0 = *(uint64_t *)(vn + i); + uint64_t m0 = *(uint64_t *)(vm + i); + uint64_t n1 = *(uint64_t *)(vn + i + 8); + uint64_t m1 = *(uint64_t *)(vm + i + 8); + uint64_t out0 = 0; + uint64_t out1 = 0; + + for (j = 0; j < 64; j += 8) { + uint64_t cnt0 = do_histseg_cnt(n0 >> j, m0, m1); + uint64_t cnt1 = do_histseg_cnt(n1 >> j, m0, m1); + out0 |= cnt0 << j; + out1 |= cnt1 << j; + } + + *(uint64_t *)(vd + i) = out0; + *(uint64_t *)(vd + i + 8) = out1; + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 49d7a45a50..0cbca8d715 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7504,6 +7504,25 @@ static bool trans_##NAME(DisasContext *s, arg_rprr_esz *a) \ DO_SVE2_PPZZ_MATCH(MATCH, match) DO_SVE2_PPZZ_MATCH(NMATCH, nmatch) +static bool trans_HISTCNT(DisasContext *s, arg_rprr_esz *a) +{ + static gen_helper_gvec_4 * const fns[2] = { + gen_helper_sve2_histcnt_s, gen_helper_sve2_histcnt_d + }; + if (a->esz < 2) { + return false; + } + return do_sve2_zpzz_ool(s, a, fns[a->esz - 2]); +} + +static bool trans_HISTSEG(DisasContext *s, arg_rrr_esz *a) +{ + if (a->esz != 0) { + return false; + } + return do_sve2_zzz_ool(s, a, gen_helper_sve2_histseg); +} + static bool do_sve2_zpzz_fp(DisasContext *s, arg_rprr_esz *a, gen_helper_gvec_4_ptr *fn) { From patchwork Fri Apr 30 20:25:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429870 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp788063jao; Fri, 30 Apr 2021 13:57:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyoIdY1HqVAtVKvMsgXLYCm6Wy15HKYYuVvUZRYYG3CvDnHrPr+uknPhTS1F6Mj3ddAWC8D X-Received: by 2002:a92:3212:: with SMTP id z18mr5522671ile.171.1619816238658; Fri, 30 Apr 2021 13:57:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816238; cv=none; d=google.com; s=arc-20160816; b=sX6kkKLxNf+NdIMdhXR8ReyFf0kwcXD38V9Hy8COlmgXqFqfvDZcSk73QqWvypfxoU TO59Cy3YX0Grh7mcWcY6VQ+wXxIoY/QhvUF90XUWrUMMXxMhSZgwFKSpyu6Zq4CJ0XrH c4OkxK0Jpso7z/2Br6ln40j7jhHDpXSXZlTgJ+IcbSszsGO2Kl9ozmbTX3pCn2A3L7N4 HKtJ6nuBc4NfCJx+OFOxDoonfRw89ENVs+urOGdQrg6zjr1BFCDMNti9rXZYmM6oCuI4 nTInk/49GBIOHeXBEslVMhiTjMWl/TLNba2vV6GWk7/sdz9HJdYA8judoKheZA4AiwZy HdNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9wtSJ1TR4daGwiyeVd5xHZ/xhp0+r1Sm5he1wDTEIDQ=; b=grYutE5JW+81gFvJVw8roE7rcaI1xgl/D66S0/1fXyUzqTZJxw5zr4qK2yg9ounLCn KQI6sMSmHjOWqkGUJJTkNvJq5+AnHmLNt/cUCdsT3yk4Zcf6rPADmhA+4ipVugiz0oWw CmpVNxkxZw9N3EwVz0davCUmINEkpBSCXrXVLAr4eih226NHA7ELzpeg+yd1NX+ktZBg Uj0PTzQBxRGGgtazN6jLQd6Ufr6vZiHoK3IXp07sXCJxHe29nEI9Klzm+okAlgEbjhvB wSTWBTJKgM/qI1VmgMu7ZVUcRBeLdMVjZ+aaL0VYf+eSgb5medOaqeC5Bp0ZeMV/1pAa 0Xeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JOTOOwNz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r13si3734726ilg.23.2021.04.30.13.57.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:57:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JOTOOwNz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:57610 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaCL-0003YZ-WC for patch@linaro.org; Fri, 30 Apr 2021 16:57:18 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53092) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0004Bz-6a for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:01 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:37431) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlb-0007gM-Cm for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:00 -0400 Received: by mail-pj1-x1033.google.com with SMTP id k3-20020a17090ad083b0290155b934a295so2366606pju.2 for ; Fri, 30 Apr 2021 13:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9wtSJ1TR4daGwiyeVd5xHZ/xhp0+r1Sm5he1wDTEIDQ=; b=JOTOOwNzVdoVhcSw6cmD0R20zlbbcPXiWp+epHoDTQExobI+4wMcsIIcddjbsYo5HD UzU1Z9NAjIP7nwoCMhDk3hDALpo4+ep1OvfBc+vP84/5R//3ZcY8P3Jpo8Q3b40X7ug/ NLz5KoLbwSCzSMaOUpeRKQQxYXUiEBje7y0ACmx77DaAEdW//DlWKeZwJcE1Ll4QT2OK QxR0OYBCuoFPT0EQzbAiX02KwfCpxaSoVbOBvsKsx3CiNyIB1AZYALpBVe5GM1cYgrxh 5uo5ZLdPxMvAw3W+uoEviQuaA8BpaApBlKtVxtguT7+IKOlpY9s+uw9e8Lpv6eM+qH0A iYtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9wtSJ1TR4daGwiyeVd5xHZ/xhp0+r1Sm5he1wDTEIDQ=; b=rQvsH8s/RrGkMKT4b0XIX2aiBzf1mMydjpRkVkLmw5qi9tpnVsV42e6SPOx4t4GH+K 6C1osPz/OCyMO/TYqX2UMjXTByW6MXyAY98YQ/IV58a76SmB36WalRk22bXy5SJEvPBZ orM1bY1/f/KegRRcPba5RNPHLalJrCVyopeKUXkZ2FhicmyueIozVOXygDPwkZyGerQs Jsrc+0bOnCzuumYJnkXo3xXJiXCXEPN9O4d4eeFXGe2zZrxxt6QfeuSKnqURZKEkTTS1 Kx64NfjtFy7tG7inU6uJ35zNW8oCuTLPactUR9rR8LFh2GvpEdwhD3eDLnPE02/qFsES RO0w== X-Gm-Message-State: AOAM533vaz+FWfTtPAq77DV5VVYEmrnO1UrVXy0s70vypn6ptWrJ2z6n YJ98SOLPosacgSKEd86/1ukXyTbJHFKP9g== X-Received: by 2002:a17:90b:17c7:: with SMTP id me7mr13273510pjb.96.1619814578171; Fri, 30 Apr 2021 13:29:38 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 43/82] target/arm: Implement SVE2 XAR Date: Fri, 30 Apr 2021 13:25:31 -0700 Message-Id: <20210430202610.1136687-44-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" In addition, use the same vector generator interface for AdvSIMD. This fixes a bug in which the AdvSIMD insn failed to clear the high bits of the SVE register. Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 4 ++ target/arm/helper.h | 2 + target/arm/translate-a64.h | 3 ++ target/arm/sve.decode | 4 ++ target/arm/sve_helper.c | 39 ++++++++++++++ target/arm/translate-a64.c | 25 ++------- target/arm/translate-sve.c | 104 +++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 12 +++++ 8 files changed, 172 insertions(+), 21 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 507a2fea8e..28b8f00201 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2558,6 +2558,10 @@ DEF_HELPER_FLAGS_5(sve2_histcnt_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_histseg, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_xar_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(sve2_faddp_zpzz_s, TCG_CALL_NO_RWG, diff --git a/target/arm/helper.h b/target/arm/helper.h index 6bb0b0ddc0..23a7ec5638 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -953,6 +953,8 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index 89437276e7..58f50abca4 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -120,5 +120,8 @@ bool disas_sve(DisasContext *, uint32_t); void gen_gvec_rax1(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, uint32_t rm_ofs, uint32_t opr_sz, uint32_t max_sz); +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz); #endif /* TARGET_ARM_TRANSLATE_A64_H */ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8f501a083c..7645587469 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -65,6 +65,7 @@ &rr_dbm rd rn dbm &rrri rd rn rm imm &rri_esz rd rn imm esz +&rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s @@ -384,6 +385,9 @@ ORR_zzz 00000100 01 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 EOR_zzz 00000100 10 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 BIC_zzz 00000100 11 1 ..... 001 100 ..... ..... @rd_rn_rm_e0 +XAR 00000100 .. 1 ..... 001 101 rm:5 rd:5 &rrri_esz \ + rn=%reg_movprfx esz=%tszimm16_esz imm=%tszimm16_shr + # SVE2 bitwise ternary operations EOR3 00000100 00 1 ..... 001 110 ..... ..... @rdn_ra_rm_e0 BSL 00000100 00 1 ..... 001 111 ..... ..... @rdn_ra_rm_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8d002fdb65..c77003217e 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7193,3 +7193,42 @@ void HELPER(sve2_histseg)(void *vd, void *vn, void *vm, uint32_t desc) *(uint64_t *)(vd + i + 8) = out1; } } + +void HELPER(sve2_xar_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 8 - shr; + uint64_t mask = dup_const(MO_8, 0xff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + int shl = 16 - shr; + uint64_t mask = dup_const(MO_16, 0xffff >> shr); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + uint64_t t = n[i] ^ m[i]; + d[i] = ((t >> shr) & mask) | ((t << shl) & ~mask); + } +} + +void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 4; + int shr = simd_data(desc); + uint32_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror32(n[i] ^ m[i], shr); + } +} diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 0c80d0b505..bdd47208b1 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -14349,8 +14349,6 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) int imm6 = extract32(insn, 10, 6); int rn = extract32(insn, 5, 5); int rd = extract32(insn, 0, 5); - TCGv_i64 tcg_op1, tcg_op2, tcg_res[2]; - int pass; if (!dc_isar_feature(aa64_sha3, s)) { unallocated_encoding(s); @@ -14361,25 +14359,10 @@ static void disas_crypto_xar(DisasContext *s, uint32_t insn) return; } - tcg_op1 = tcg_temp_new_i64(); - tcg_op2 = tcg_temp_new_i64(); - tcg_res[0] = tcg_temp_new_i64(); - tcg_res[1] = tcg_temp_new_i64(); - - for (pass = 0; pass < 2; pass++) { - read_vec_element(s, tcg_op1, rn, pass, MO_64); - read_vec_element(s, tcg_op2, rm, pass, MO_64); - - tcg_gen_xor_i64(tcg_res[pass], tcg_op1, tcg_op2); - tcg_gen_rotri_i64(tcg_res[pass], tcg_res[pass], imm6); - } - write_vec_element(s, tcg_res[0], rd, 0, MO_64); - write_vec_element(s, tcg_res[1], rd, 1, MO_64); - - tcg_temp_free_i64(tcg_op1); - tcg_temp_free_i64(tcg_op2); - tcg_temp_free_i64(tcg_res[0]); - tcg_temp_free_i64(tcg_res[1]); + gen_gvec_xar(MO_64, vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), imm6, 16, + vec_full_reg_size(s)); } /* Crypto three-reg imm2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0cbca8d715..eea8b6f1d0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -340,6 +340,110 @@ static bool trans_BIC_zzz(DisasContext *s, arg_rrr_esz *a) return do_zzz_fn(s, a, tcg_gen_gvec_andc); } +static void gen_xar8_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_8, 0xff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 8 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar16_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + TCGv_i64 t = tcg_temp_new_i64(); + uint64_t mask = dup_const(MO_16, 0xffff >> sh); + + tcg_gen_xor_i64(t, n, m); + tcg_gen_shri_i64(d, t, sh); + tcg_gen_shli_i64(t, t, 16 - sh); + tcg_gen_andi_i64(d, d, mask); + tcg_gen_andi_i64(t, t, ~mask); + tcg_gen_or_i64(d, d, t); + tcg_temp_free_i64(t); +} + +static void gen_xar_i32(TCGv_i32 d, TCGv_i32 n, TCGv_i32 m, int32_t sh) +{ + tcg_gen_xor_i32(d, n, m); + tcg_gen_rotri_i32(d, d, sh); +} + +static void gen_xar_i64(TCGv_i64 d, TCGv_i64 n, TCGv_i64 m, int64_t sh) +{ + tcg_gen_xor_i64(d, n, m); + tcg_gen_rotri_i64(d, d, sh); +} + +static void gen_xar_vec(unsigned vece, TCGv_vec d, TCGv_vec n, + TCGv_vec m, int64_t sh) +{ + tcg_gen_xor_vec(vece, d, n, m); + tcg_gen_rotri_vec(vece, d, d, sh); +} + +void gen_gvec_xar(unsigned vece, uint32_t rd_ofs, uint32_t rn_ofs, + uint32_t rm_ofs, int64_t shift, + uint32_t opr_sz, uint32_t max_sz) +{ + static const TCGOpcode vecop[] = { INDEX_op_rotli_vec, 0 }; + static const GVecGen3i ops[4] = { + { .fni8 = gen_xar8_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_b, + .opt_opc = vecop, + .vece = MO_8 }, + { .fni8 = gen_xar16_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_h, + .opt_opc = vecop, + .vece = MO_16 }, + { .fni4 = gen_xar_i32, + .fniv = gen_xar_vec, + .fno = gen_helper_sve2_xar_s, + .opt_opc = vecop, + .vece = MO_32 }, + { .fni8 = gen_xar_i64, + .fniv = gen_xar_vec, + .fno = gen_helper_gvec_xar_d, + .opt_opc = vecop, + .vece = MO_64 } + }; + int esize = 8 << vece; + + /* The SVE2 range is 1 .. esize; the AdvSIMD range is 0 .. esize-1. */ + tcg_debug_assert(shift >= 0); + tcg_debug_assert(shift <= esize); + shift &= esize - 1; + + if (shift == 0) { + /* xar with no rotate devolves to xor. */ + tcg_gen_gvec_xor(vece, rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz); + } else { + tcg_gen_gvec_3i(rd_ofs, rn_ofs, rm_ofs, opr_sz, max_sz, + shift, &ops[vece]); + } +} + +static bool trans_XAR(DisasContext *s, arg_rrri_esz *a) +{ + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + gen_gvec_xar(a->esz, vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), a->imm, vsz, vsz); + } + return true; +} + static bool do_sve2_zzzz_fn(DisasContext *s, arg_rrrr_esz *a, GVecGen4Fn *fn) { if (!dc_isar_feature(aa64_sve2, s)) { diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 19006f50f7..a3d80ecad0 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -2253,3 +2253,15 @@ void HELPER(gvec_umulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } clear_tail(d, opr_sz, simd_maxsz(desc)); } + +void HELPER(gvec_xar_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc) / 8; + int shr = simd_data(desc); + uint64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = ror64(n[i] ^ m[i], shr); + } + clear_tail(d, opr_sz * 8, simd_maxsz(desc)); +} From patchwork Fri Apr 30 20:25:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429874 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp789168jao; Fri, 30 Apr 2021 13:59:10 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyfOuzYlVZL3o04nYp8LtzVVDDg3J6H1xJfg0xJ+1i63N5ZuTDCAjJfbVa49Rboi/VR4Y1D X-Received: by 2002:a6b:6013:: with SMTP id r19mr5393845iog.208.1619816350434; Fri, 30 Apr 2021 13:59:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816350; cv=none; d=google.com; s=arc-20160816; b=K8lz7ryp4Dpx/lRYEz4ACO9BBvQga95GxhV8AqN0GqJ90Vjdg1cVaGdPVl/Op1/MBX SSnyDCnjiOM3ceYgXOnY+lO33961/C+BRP/ggUZ9K+h+3KEH+ifTo5F1vgqYgzvM9yA0 uyZGRUIsm+pAbPsfyktLyHUxnkGsoxdz1AbdHebCNErm9RuemF6aP83wn4ZNkBcY251b Av7y+T1mVxch/PWA1yvUtS93YOmp0RpqCsfVEjVrdfEOH+c1xtZOI/ArXOklVv8BLCFR /sgJlW1kDr62AxzrVCy6DraCnQgzpMFEyb4C+DINt3Wyg9PqXq77uxitbeVQV8fvzWTA 1mfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=vbzo+7meB5EIAr5/3tVpgCQZH+LlGehsL7Kde6sY+2jvPrs6GO0Lkppebj4iA+6RWP XfnV1inZy+IXGCn9fVMA61orx7eKK2uUX6k0UTWJXzWpeJmThUfAiZiIsfvO6HaWfoWh bjW7k2cOxWHWe+BQwk5NGsWEQDEpidBQyfirZihufw7sLi2xeBMt75TKHspdxeU9QZI7 keDEbj+8fddg8joQMoAcEmTmbUD9UH+PGy1NjZzENASTAzH18vp0UnrYePHlF6Nw0xOX YgaUAAg1t9xLqugxQZRbeJm3SlhMMsajQcXbLfAOJSOdFsCoBUGlDRPQcaXAvp9Uzmbx Kt6Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vGvjRHTV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d6si4016543ilv.73.2021.04.30.13.59.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:59:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vGvjRHTV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38156 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaE9-0007CK-QF for patch@linaro.org; Fri, 30 Apr 2021 16:59:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53218) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZm4-0004Ej-FB for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:10 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]:42740) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007h5-2D for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:08 -0400 Received: by mail-pf1-x429.google.com with SMTP id b27so905342pfp.9 for ; Fri, 30 Apr 2021 13:29:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=vGvjRHTVa3JK48nDwrfipfb4ZCugRkn+EOX/G4k2gLqNpPm02UxGUhC4DFsQcjr58q TUZyYcdonisdsMTO+gU59VZYka025arMZPlgh8Mkq6bIGByvhPbAZc/3Kh7AlG0HfVrq HJttUddHR9dAZ88xOAuwwwSCbv36hoaBvkCKhINitT/0TIQkWxER+1Dgy0xW7cVGeUJi xW/t0lXDyBVya4YDhL97c9RvFDlagT6mZP1xwjQFMCRX63qCf3icgObJSkvtEg5VtY/Z PJ6T6SldJOVniv/IWARuufoCtfu1rqa1Oyacaz2fPjsdHee/RMH8QcraIOTtvio9TLM2 8TBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bYBvAAL6DC7Hi2Mjo18sNLpM8Gq57yT/LKmwPXamUwE=; b=l+qlEioxUh6Dz3qLmoorb0o9UUKEy/vbGRZ9Lwnf/1fQhaocmCXRiGjOJgMJQ+i22y /ZYs/F6jqyoaqyw7yBLWjWbV65v7x3pq6sulahCzdxk0PDy04VUiaHnVNlBSu1iegtkk tfOdzlhHoHmvrA/Nt9mXG+iqALU7Lln2VpRvDcscW9RR5TmLYohco4mISiU2pA9/H1Xm B4qfnqKYJHLXDOESVT/kNE7SRFASvse2vH/Me5AMdqZfLAe4ylqgzVedUu0dA5D/y5c+ kgDLiccPlyIMjbq0hdMVu0//s/4nB+cXhjcPGEfHNWdo/+dDzdFnf+gIFub5Khg4PzmV 3PNw== X-Gm-Message-State: AOAM533wm8Jjw2vytBOCtGs+UbBg8GcnLQfNF+nPKpaKweZ/3QF2TZcq 0fh4NuMIWYyp9GfJPWh8AM+rkVEMEvfZTQ== X-Received: by 2002:aa7:80c1:0:b029:262:ab5f:adc5 with SMTP id a1-20020aa780c10000b0290262ab5fadc5mr6747237pfn.60.1619814578788; Fri, 30 Apr 2021 13:29:38 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 44/82] target/arm: Implement SVE2 scatter store insns Date: Fri, 30 Apr 2021 13:25:32 -0700 Message-Id: <20210430202610.1136687-45-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit scatter non-temporal store insns. 64-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) * STNT1D (vector plus scalar) 32-bit * STNT1B (vector plus scalar) * STNT1H (vector plus scalar) * STNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422141553.8037-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 10 ++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 18 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7645587469..5cfe6df0d2 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1388,3 +1388,13 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx + +### SVE2 Memory Store Group + +# SVE2 64-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=2 esz=3 scale=0 + +# SVE2 32-bit scatter non-temporal store (vector plus scalar) +STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ + @rprr_scatter_store xs=0 esz=2 scale=0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index eea8b6f1d0..0356b6a124 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6167,6 +6167,14 @@ static bool trans_ST1_zpiz(DisasContext *s, arg_ST1_zpiz *a) return true; } +static bool trans_STNT1_zprz(DisasContext *s, arg_ST1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_ST1_zprz(s, a); +} + /* * Prefetches */ From patchwork Fri Apr 30 20:25:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429890 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp800925jao; Fri, 30 Apr 2021 14:14:59 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxYZLhwMq75Cl4XCkPYzrltvXqQz1+XVggZAerHVZXuktQcU07WYI6A42pZaGQ+Ei0ZwiL6 X-Received: by 2002:a92:c548:: with SMTP id a8mr5522510ilj.137.1619817299122; Fri, 30 Apr 2021 14:14:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817299; cv=none; d=google.com; s=arc-20160816; b=bpVdnHpB+LavoUlBPX7zhhRr1a9NpkuNCZfpoagkCqYHvSkHyaA5c51WlseN8RxcMv zpKx8sqMdh37ojTSEZ1jFB17dI/W/w44lYvZK0sQY/1V+afmqrHCnakaprCj5rK+FFs+ 8X2TA3dc0JHQpRZPDlEa6xIYT/QneSX6JoYo+x7k5FLxZFUSZ+8tvRHDkeThAz7SHnbU 8I+QLD2V3S75wWXpUErYAmGcAlUTSwg1gZ+co4IRUBn//4CQ2TfG5J/rpzwSL8XvFOeh zAuUev6kCx6J2NAXVe8rO8l72KKfbOP8JKcUxl+bSzckkn5O1M8sxYNXM7tVxNiMmDGj frtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=cH0wZfTPPyo3Y2WWlAmiPCWmvvFrGVEqvITM9U10Fz5+GECaUQLIVAp/Szy/MhTquK RXttK/HPxZFpZuIF3HKVD1wSkZV9+b+aw0gKuMEFNDrxsPvQIocUyiwl1aRIwL11tYYM wTtDtx+kVSolQH4BXzQDhdXmt5rohWV320Xh/QrL1chc/exYUDUzT/CHQLdrWLeWCGNu 39JLn1P9PO5sQ1WUOernydbMhNIH4O40QYp9eu/Xu7VBa0Y+CmzSsPOZMMRdL2LYhVgU puO8L2ASiQvQnlTMfUtW4pGnx5roQ847uVkeALGhvcQB8dJ8k2oBE9+8suzIcZPT5qjr 5SbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Li5SAExO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m3si4571717iol.17.2021.04.30.14.14.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:14:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Li5SAExO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51648 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaTS-0007iu-D0 for patch@linaro.org; Fri, 30 Apr 2021 17:14:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53410) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmK-0004VQ-Eu for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:24 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]:37520) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007hU-AD for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:24 -0400 Received: by mail-pl1-x62b.google.com with SMTP id h20so37235677plr.4 for ; Fri, 30 Apr 2021 13:29:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=Li5SAExOXC1Tq8T0hs1GZHOMpUjWqgmuU/z1uDiOM+DwNbiiEShCeUxO2YuQRH3P0q UN/Yz9Mc4uG836Nd361Zo4/uwpALV6VaBqEhhlxBti/Txbtoax2osSx+AAJuRN9mTz9v SI5ZXCLa3OcXHfJv+0oBZIeFD9l/SzHjFF5jlwrRh2SxkA5Y6E1MfBsGO8gijOMM2AQx DIrOU6SCNaSxKawGttnVlwnyqsYw+q91MX35u8DOVovQcOfEVz37ko4xKeq+WubFsLdp nvRlG7hhwK2ezNw89YtUm307AGpp/MyPcpzfCZD55XkZyFJOlwTDsVIqaaoLPgBTNlrF I/rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bn7hUPo4Fn4gj8R8dHIFe5YgNO7lh3cxv0iB6D6VY7Q=; b=A1JEi2U2Vugg594xRWzbS0G1UUs89RI78WL15fkRdqwbSAsEPnEAbtEGxQ4EfPvc5d xnJFCfoebBh1hKPeNfnRX5QiOQGbTgH9zQIVTjejigcJHeJN5qumlEYlp6sH6nqfeYXJ lQOe3Zi32c/2pLtUOxQX2eOeaiZOfls/f3FXOVyQIDG1SqVHuHxXDdi8W3kBYaOnuEg9 cZaT4WEaQtzJLdIkN9z5g24tOTz28MwJRXbwv4kdSrg8dEZjp+9mqsKCZ5TJPGks625E L48/tvhNsl1+hZ74kjzl6bG3Vum60pIOschz4J6gEfCzqTfUglwPm+hnWOri2Ip1SFk6 jyng== X-Gm-Message-State: AOAM530BmJwV3xgh7Hn1XSfpcWQ7ZdVkNpLq6SimaJykv97xBjopgnuc p3M7vI6I/U3b8KZYCLMk16RynX3LR/hoYQ== X-Received: by 2002:a17:90a:510d:: with SMTP id t13mr6759819pjh.97.1619814579416; Fri, 30 Apr 2021 13:29:39 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 45/82] target/arm: Implement SVE2 gather load insns Date: Fri, 30 Apr 2021 13:25:33 -0700 Message-Id: <20210430202610.1136687-46-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Add decoding logic for SVE2 64-bit/32-bit gather non-temporal load insns. 64-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1SW * LDNT1W (vector plus scalar) * LDNT1D (vector plus scalar) 32-bit * LDNT1SB * LDNT1B (vector plus scalar) * LDNT1SH * LDNT1H (vector plus scalar) * LDNT1W (vector plus scalar) Signed-off-by: Stephen Long Message-Id: <20200422152343.12493-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++++ target/arm/translate-sve.c | 8 ++++++++ 2 files changed, 19 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 5cfe6df0d2..c3958bed6a 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,17 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 Memory Gather Load Group + +# SVE2 64-bit gather non-temporal load +# (scalar plus unpacked 32-bit unscaled offsets) +LDNT1_zprz 1100010 msz:2 00 rm:5 1 u:1 0 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=3 scale=0 ff=0 + +# SVE2 32-bit gather non-temporal load (scalar plus 32-bit unscaled offsets) +LDNT1_zprz 1000010 msz:2 00 rm:5 10 u:1 pg:3 rn:5 rd:5 \ + &rprr_gather_load xs=0 esz=2 scale=0 ff=0 + ### SVE2 Memory Store Group # SVE2 64-bit scatter non-temporal store (vector plus scalar) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0356b6a124..a74c15b23f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6015,6 +6015,14 @@ static bool trans_LD1_zpiz(DisasContext *s, arg_LD1_zpiz *a) return true; } +static bool trans_LDNT1_zprz(DisasContext *s, arg_LD1_zprz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return trans_LD1_zprz(s, a); +} + /* Indexed by [mte][be][xs][msz]. */ static gen_helper_gvec_mem_scatter * const scatter_store_fn32[2][2][2][3] = { { /* MTE Inactive */ From patchwork Fri Apr 30 20:25:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429879 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp794260jao; Fri, 30 Apr 2021 14:05:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwZLxsOBXIWJmoJ5R4oaCsuhncK+C3jf/JG46Blc22YQK16bjuNk5LHPz4OLt993oXRDGP8 X-Received: by 2002:a6b:8ec9:: with SMTP id q192mr5652341iod.12.1619816714911; Fri, 30 Apr 2021 14:05:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816714; cv=none; d=google.com; s=arc-20160816; b=FJ4bGI/NmsRzoGEGV5Av5pryRmp9p2/pggZwbuiYq5VRxiLBDnUE1l24tbEvGYYqO0 +oQFQgRxcUBrfAYuAaod/iprREcb4AuJoy+QDuPd1oxSTBoW7WccaSbfm0KWCpt7OpFi Y61zeoibNs4OZna2d8xW+oY+6SXo4HMtIjIz8c/YTMgUczr0Dov5onxsc08MWUW6dDkn 8AydI7iQ7nZ3HyemMppa/kUN0TqcNz+uOKpWKfR6cVa0yvvJOTogpTw/5CjaR4fFHn9G oeJkfBrFbw6cZPEGUdKufhjM5HGeTOXhlSXpIE2zoGBaHAu76NBfA2UCD5C4nOrRyMZq Vc6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Z0a+MVS2fnmX5CZ/umrWtOeUVCLklS9tHWa1Y1frp2c=; b=Ywa1fBSuZYkksyNqlBVy/K8/pdfuhmQS5yrzFVBqj9boin7SB/nFeiHOg9Xg6iuKxX zK8XuRyHTnrInrzy1KuAA52c4rpUrESpSEn8o5i4nGQ7HtfMw999FUzVWm2tKXFoWebb EcHjedF0zakosWCZz6Rrb/WKzZ9Q/cRPYd7mAuYbZSYoYuSIIq/nfbjXYEWDepyCdzBd v22SQRhBZICxKWXdJRINIbdkw8YcmZ7vjoBzKfgsvp0jhXsBzcfQS24ubBPJbcU0A/WY FvQ0oTBewAvcHQQwiGrBihdB/cnwF0E/MY74JgYXL6K9epbpC9l/3O7mItDiZOF8eywO x8+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UiCLPi+/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o21si3816869jat.114.2021.04.30.14.05.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:05:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="UiCLPi+/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55190 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaK2-0005qy-Bi for patch@linaro.org; Fri, 30 Apr 2021 17:05:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53402) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmJ-0004U9-Uf for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:23 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:37433) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007he-7o for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:23 -0400 Received: by mail-pj1-x1035.google.com with SMTP id k3-20020a17090ad083b0290155b934a295so2366644pju.2 for ; Fri, 30 Apr 2021 13:29:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Z0a+MVS2fnmX5CZ/umrWtOeUVCLklS9tHWa1Y1frp2c=; b=UiCLPi+/ahr6OG0i/KsJZ2DckvMT5uwkmj46WDG26Vt31u/E2LQhw/JgW75Uz+2SNg VnxgKFIGt4PWLnRvjgJzvOVByzf618uOxJTwI9gI9g/hw4JB52bDfl4AIxLUupVHeWoL M0sOZVjDH2HsxAihm2m2UFGXMn4CldpNUYXkte1Dsk1tYCcuWhrd30c1eF4QiM1bos/B vRxdQZzjBRhz9cJEnXA3rjv4yXZ3lUQjQo1H/y+JioUGDLvA6MBn9F+ZEH9LsHdb8E1w 9XdkT4ScT9kJ5Ok6E6BP7jdqigoBEUNCB6dYISTMPZs+CkS7BvdOeohTmNiLhdsaHdew pTwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Z0a+MVS2fnmX5CZ/umrWtOeUVCLklS9tHWa1Y1frp2c=; b=KIzuVSl1v3JD5iri1ekxcFINbTURPaFp5OamF+brMB+JN1JCjd0wSmdi1uUjMvTJyU eqWOCD4K9g6KFdjDkPrcLzmK3iP2BzCJITwlXQXWNoTtCmQbXOgevoUUOzdAWlaS/Lt6 vk0wcWbzbMr1SF+82fHsCa0oRnqBGc8SlZ97rnfQU3ux20LGorh0T+KvE074xAGCN4iL 4rYx4I7CvyD2jK9aEbRad1yk4h4/JJPxjciFxiTY1C0NOUkgo9PCAb7MvXs52+PAFJTb lgE6lNYkZAiio6Ad2hM8Y5j3KuYhY1IbDCA5Tpj0AyJDRp4wuSOIiTkqbRrZyLsZoCd0 qRAg== X-Gm-Message-State: AOAM53085p+K0gnzTgBKHaD1NydS8mV8CwTxX3a+UEnolWld/L5jXx7u iMy1dum5cRcqnvrq/ubPGojc8rHmyG6vwA== X-Received: by 2002:a17:90b:f82:: with SMTP id ft2mr16560847pjb.0.1619814580088; Fri, 30 Apr 2021 13:29:40 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 46/82] target/arm: Implement SVE2 FMMLA Date: Fri, 30 Apr 2021 13:25:34 -0700 Message-Id: <20210430202610.1136687-47-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200422165503.13511-1-steplong@quicinc.com> [rth: Fix indexing in helpers, expand macro to straight functions.] Signed-off-by: Richard Henderson --- target/arm/cpu.h | 10 ++++++ target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 4 +++ target/arm/sve_helper.c | 74 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 34 ++++++++++++++++++ 5 files changed, 125 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index ae787fac8a..595bc6349d 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,16 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve_f32mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; +} + +static inline bool isar_feature_aa64_sve_f64mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F64MM) != 0; +} + /* * Feature tests for "does this exist in either 32-bit or 64-bit?" */ diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 28b8f00201..7e99dcd119 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2662,3 +2662,6 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index c3958bed6a..cb2ee86228 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1389,6 +1389,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +### SVE2 floating point matrix multiply accumulate + +FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm + ### SVE2 Memory Gather Load Group # SVE2 64-bit gather non-temporal load diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c77003217e..f285c90b70 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7232,3 +7232,77 @@ void HELPER(sve2_xar_s)(void *vd, void *vn, void *vm, uint32_t desc) d[i] = ror32(n[i] ^ m[i], shr); } } + +void HELPER(fmmla_s)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float32) * 4); + + for (s = 0; s < opr_sz; ++s) { + float32 *n = vn + s * sizeof(float32) * 4; + float32 *m = vm + s * sizeof(float32) * 4; + float32 *a = va + s * sizeof(float32) * 4; + float32 *d = vd + s * sizeof(float32) * 4; + float32 n00 = n[H4(0)], n01 = n[H4(1)]; + float32 n10 = n[H4(2)], n11 = n[H4(3)]; + float32 m00 = m[H4(0)], m01 = m[H4(1)]; + float32 m10 = m[H4(2)], m11 = m[H4(3)]; + float32 p0, p1; + + /* i = 0, j = 0 */ + p0 = float32_mul(n00, m00, status); + p1 = float32_mul(n01, m01, status); + d[H4(0)] = float32_add(a[H4(0)], float32_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float32_mul(n00, m10, status); + p1 = float32_mul(n01, m11, status); + d[H4(1)] = float32_add(a[H4(1)], float32_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float32_mul(n10, m00, status); + p1 = float32_mul(n11, m01, status); + d[H4(2)] = float32_add(a[H4(2)], float32_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float32_mul(n10, m10, status); + p1 = float32_mul(n11, m11, status); + d[H4(3)] = float32_add(a[H4(3)], float32_add(p0, p1, status), status); + } +} + +void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, + void *status, uint32_t desc) +{ + intptr_t s, opr_sz = simd_oprsz(desc) / (sizeof(float64) * 4); + + for (s = 0; s < opr_sz; ++s) { + float64 *n = vn + s * sizeof(float64) * 4; + float64 *m = vm + s * sizeof(float64) * 4; + float64 *a = va + s * sizeof(float64) * 4; + float64 *d = vd + s * sizeof(float64) * 4; + float64 n00 = n[0], n01 = n[1], n10 = n[2], n11 = n[3]; + float64 m00 = m[0], m01 = m[1], m10 = m[2], m11 = m[3]; + float64 p0, p1; + + /* i = 0, j = 0 */ + p0 = float64_mul(n00, m00, status); + p1 = float64_mul(n01, m01, status); + d[0] = float64_add(a[0], float64_add(p0, p1, status), status); + + /* i = 0, j = 1 */ + p0 = float64_mul(n00, m10, status); + p1 = float64_mul(n01, m11, status); + d[1] = float64_add(a[1], float64_add(p0, p1, status), status); + + /* i = 1, j = 0 */ + p0 = float64_mul(n10, m00, status); + p1 = float64_mul(n11, m01, status); + d[2] = float64_add(a[2], float64_add(p0, p1, status), status); + + /* i = 1, j = 1 */ + p0 = float64_mul(n10, m10, status); + p1 = float64_mul(n11, m11, status); + d[3] = float64_add(a[3], float64_add(p0, p1, status), status); + } +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a74c15b23f..5d455c4db6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -7672,6 +7672,40 @@ DO_SVE2_ZPZZ_FP(FMINP, fminp) * SVE Integer Multiply-Add (unpredicated) */ +static bool trans_FMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + gen_helper_gvec_4_ptr *fn; + + switch (a->esz) { + case MO_32: + if (!dc_isar_feature(aa64_sve_f32mm, s)) { + return false; + } + fn = gen_helper_fmmla_s; + break; + case MO_64: + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + fn = gen_helper_fmmla_d; + break; + default: + return false; + } + + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + TCGv_ptr status = fpstatus_ptr(FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + status, vsz, vsz, 0, fn); + tcg_temp_free_ptr(status); + } + return true; +} + static bool do_sqdmlal_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sel1, bool sel2) { From patchwork Fri Apr 30 20:25:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429891 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp801253jao; Fri, 30 Apr 2021 14:15:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxTc1ci/cYPriVr6hEzRbsmCsSiQxy+N1p6j+U+CieEVeY3RCeL4o0t4We0feuZBi/LC7BR X-Received: by 2002:a05:6638:3708:: with SMTP id k8mr5285419jav.24.1619817327731; Fri, 30 Apr 2021 14:15:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817327; cv=none; d=google.com; s=arc-20160816; b=GgAY1mwHmEjxEHUHXY7GtMNV3I97ZBX6QF2X1gu9f1dI0HisnFM7XfbitnZ/9zXYri Hw4vi233gYkdD1cJITiHUS1Xvsn/HDrReRUoZ9E7jh5X6lp7ryVDoFhmtrhIJEOspaeJ XBfhhgiJAUPHPxl1QXJ2Zn5DoIpFbG+A9bukyQGAzUCHNhMBtGEyIsUvpHBxgAcd5Yib wBmuTaLEIoGXh2FmSvvl9BdZW11IgY0AWnF4xNSheoOX3mAadIsfb8Hm/8lMamTpSgww Sp/jU4c7X4f9CWDS4g6ActDK3gGmvcJ/Em49ZAkPKNoa7fPGmDzXTx8s7JXrAiBtb/9r G0cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=biS3JWPtxwuQ9jQo1R+HoActsqUh9fYZHGmqJRkFZnM=; b=0eF8JF6qiDkoW/3k7EMFamw45pkV5AvNgT+9RRqUCnFgJX5gjmGGU7ToNk2WL26pzq H/rvgvFEptpDEtJn4eFYK6bvhGsVpMu7UoBEMyh81kkhkKgxv8gUDEbIyCGIuRKSvPSc W3QHzBXxMF9EkcdZjqdIMB6DVGpb0X3DDHFkGSiPM05b6JQhudDpf9lk7KHM+nRmWD5P MG3t4BrxuQK/kUc4Cf+BWSCcBYi4Cz5TTia4VHtGmDCOlO+RC+LGsW7YDEWg4YiBHaHI jeAxTSnV/Bt69O5x1vuVsyOv/+3VFPmFgI7eSgadkGrxbhTk8eEQ9XtxpLEzSTIX3Ubz zOlg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FNVF2WGg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b5si4035526ile.66.2021.04.30.14.15.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:15:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=FNVF2WGg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54674 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaTu-0000Y7-Uv for patch@linaro.org; Fri, 30 Apr 2021 17:15:26 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53344) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmH-0004NY-35 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:21 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:43753) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007hl-3s for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:20 -0400 Received: by mail-pj1-x1029.google.com with SMTP id f6-20020a17090a6546b029015088cf4a1eso2378808pjs.2 for ; Fri, 30 Apr 2021 13:29:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=biS3JWPtxwuQ9jQo1R+HoActsqUh9fYZHGmqJRkFZnM=; b=FNVF2WGgEyLNxNc0c7TCUOPS6FRGucXIxnyUwCctUxmVhJewwCvMPDNVtyMcWA4diA 5sLTf/28P5dvmWLykTO5xVnYaO5Md4+ds/bHmDJMAM2w1s5FwGilDymywptJPVCavpan 5E9QqYvaPRQtLuWdv6UvRMBGEYAVdxp91uDx4hxuIoikh8vqFpbOcqyubYtSBrWHGBv4 sQ16ZhH2QB6GQPLQ6w/DVy/QvH43QLepIQ/YqnKwuRXSxlg1qDE9GU8fTnUqXoGimRY8 BRWeHbfAMf18q5NdqbvPN0hFA446n/WkRaZbpYt0Vz30wxv818qBtYRyIh05dk9RMsNl /N2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=biS3JWPtxwuQ9jQo1R+HoActsqUh9fYZHGmqJRkFZnM=; b=e9skJepunT3WN5PoV1ki9kxdFRdeSYvNzeUUj1mj2nqOG3hEn8d7g4XQeNCLWcSf0C fzu1JuCom/INrx6AeX/KPcgnUTiezO7frfs03TIT8Wf0NX5+Web61HKWwaGPvRo8vkqX g5vrNFiefHXohac0frq4SZ6Rr77FuJYV/T1oNWDY1MvQbfhifbwDUnElg9sgLyKKdBvt x38bmwX5/zI4QLUI3CV0Wf+DkFWMPtVdEa1yJlfzD021HZfCKZ7luJDz192Uv4ECxVa0 GRrbjTfJWPPVodxUQYO29OMVp7zsw5lQIMRKy2eOQydtIofFjJDhX0rOL2E+Igyo/UFF RwHw== X-Gm-Message-State: AOAM53117VeTf17n95AdGfTAPbwZC9fQhnTHzs422MTuFLNA4un8nxQZ l6izOmXlMBbB83hbO9LIzFLPa783ym7gXQ== X-Received: by 2002:a17:90a:c903:: with SMTP id v3mr16899148pjt.13.1619814580670; Fri, 30 Apr 2021 13:29:40 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 47/82] target/arm: Implement SVE2 SPLICE, EXT Date: Fri, 30 Apr 2021 13:25:35 -0700 Message-Id: <20210430202610.1136687-48-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200423180347.9403-1-steplong@quicinc.com> [rth: Rename the trans_* functions to *_sve2.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 11 +++++++++-- target/arm/translate-sve.c | 35 ++++++++++++++++++++++++++++++----- 2 files changed, 39 insertions(+), 7 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cb2ee86228..67b6466a1e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -494,10 +494,14 @@ CPY_z_i 00000101 .. 01 .... 00 . ........ ..... @rdn_pg4 imm=%sh8_i8s ### SVE Permute - Extract Group -# SVE extract vector (immediate offset) +# SVE extract vector (destructive) EXT 00000101 001 ..... 000 ... rm:5 rd:5 \ &rrri rn=%reg_movprfx imm=%imm8_16_10 +# SVE2 extract vector (constructive) +EXT_sve2 00000101 011 ..... 000 ... rn:5 rd:5 \ + &rri imm=%imm8_16_10 + ### SVE Permute - Unpredicated Group # SVE broadcast general register @@ -588,9 +592,12 @@ REVH 00000101 .. 1001 01 100 ... ..... ..... @rd_pg_rn REVW 00000101 .. 1001 10 100 ... ..... ..... @rd_pg_rn RBIT 00000101 .. 1001 11 100 ... ..... ..... @rd_pg_rn -# SVE vector splice (predicated) +# SVE vector splice (predicated, destructive) SPLICE 00000101 .. 101 100 100 ... ..... ..... @rdn_pg_rm +# SVE2 vector splice (predicated, constructive) +SPLICE_sve2 00000101 .. 101 101 100 ... ..... ..... @rd_pg_rn + ### SVE Select Vectors Group # SVE select vector elements (predicated) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5d455c4db6..3de14e3af9 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2266,18 +2266,18 @@ static bool trans_CPY_z_i(DisasContext *s, arg_CPY_z_i *a) *** SVE Permute Extract Group */ -static bool trans_EXT(DisasContext *s, arg_EXT *a) +static bool do_EXT(DisasContext *s, int rd, int rn, int rm, int imm) { if (!sve_access_check(s)) { return true; } unsigned vsz = vec_full_reg_size(s); - unsigned n_ofs = a->imm >= vsz ? 0 : a->imm; + unsigned n_ofs = imm >= vsz ? 0 : imm; unsigned n_siz = vsz - n_ofs; - unsigned d = vec_full_reg_offset(s, a->rd); - unsigned n = vec_full_reg_offset(s, a->rn); - unsigned m = vec_full_reg_offset(s, a->rm); + unsigned d = vec_full_reg_offset(s, rd); + unsigned n = vec_full_reg_offset(s, rn); + unsigned m = vec_full_reg_offset(s, rm); /* Use host vector move insns if we have appropriate sizes * and no unfortunate overlap. @@ -2296,6 +2296,19 @@ static bool trans_EXT(DisasContext *s, arg_EXT *a) return true; } +static bool trans_EXT(DisasContext *s, arg_EXT *a) +{ + return do_EXT(s, a->rd, a->rn, a->rm, a->imm); +} + +static bool trans_EXT_sve2(DisasContext *s, arg_rri *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_EXT(s, a->rd, a->rn, (a->rn + 1) % 32, a->imm); +} + /* *** SVE Permute - Unpredicated Group */ @@ -3013,6 +3026,18 @@ static bool trans_SPLICE(DisasContext *s, arg_rprr_esz *a) return true; } +static bool trans_SPLICE_sve2(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzp(s, gen_helper_sve_splice, + a->rd, a->rn, (a->rn + 1) % 32, a->pg, a->esz); + } + return true; +} + /* *** SVE Integer Compare - Vectors Group */ From patchwork Fri Apr 30 20:25:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429894 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp803161jao; Fri, 30 Apr 2021 14:18:37 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzT3rUjzx6wTiJIF+o8no5z728DWenv74CbC+pQ1Ls1xkfRw60q8yXqM84UjWUTgyXkT6dl X-Received: by 2002:a5d:818b:: with SMTP id u11mr2317459ion.36.1619817516868; Fri, 30 Apr 2021 14:18:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817516; cv=none; d=google.com; s=arc-20160816; b=vmt7oTR1QUG7kbW1LGWA4vrDHYiaa3fWf6s55W9V5WfRbhgRJdPyzWKY4GOamC8xGT WNq5gmw4STdK2emiidb/b9ALtu55MMB6xpI1p1fOmXutzvGcxP6PvV2guOusU5eBHG2x /S1uHRpFbb306mlhbVJndSBmjmtCT8wX4iN27Ed2V9/YqKEhJpbtGHY8G4sH1qus6vJr 7wKAGlVI3QKdXb9CNGzMV/JWAAIphSMWQOZ+WytE/rXb2+CUsmxj/0kM/Nci8XGgXwfz AZoizQPDhtp3aPN9aJM0QhYbt1MeIx1Y+lpUrepwTGI1DYZorYs3oGpftUd3pjRCIBED o1sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=bCBGp2GdHMYoIw7eJTRwaDF9qwXNx38EngT3N/EUqtQ=; b=RgaGlxB5AU71umA7H4KVaMSN21ogM8nobtJEAsPYL936OVyKDW0QFR74JATGJoWxRc PoqiXKecoZIhaRAsCZnIlMfu/bpHIExO+ZFrY0aUd4lRBEbSRBnFLuOjMFWNsvyXbYDi lUv4L7s1WFqKv2N+kw8hqLmsZ4Xmgvqi4SRjoEcq/8AR0zbCZG3cD//EUVB+XmC9I51R jnDTsNEmTSJsdr9gXyWkimTux7Bq9qh/CuMUa13PdigacPvWLr0/uCYmjdCzp+AewlIP rTQXhvB9KOjsDze1JbvmbVY5pLkfbSUIYU6qh2P5U0umkPAcuNNT6RvK/JJ08HIluFxq jjug== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SLqF3fGJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j11si3057137ilu.123.2021.04.30.14.18.36 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:18:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=SLqF3fGJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33090 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaWy-0003SC-8m for patch@linaro.org; Fri, 30 Apr 2021 17:18:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53510) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmU-0004iG-PS for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:34 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]:42844) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007hw-E0 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:34 -0400 Received: by mail-pl1-x630.google.com with SMTP id v13so23640550ple.9 for ; Fri, 30 Apr 2021 13:29:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=bCBGp2GdHMYoIw7eJTRwaDF9qwXNx38EngT3N/EUqtQ=; b=SLqF3fGJre/N/BUKO90ay3R92mbdIUxHm2zlnTc9h+ssNMZbGkWmRGH7uOy71vY1C4 vgkZst77SQgBixiIX+eM+PVQ6/y8kv2FlaV4sn/PW1CDcHtK3dlXiwMbBCe+oX+IblH+ jDoD+TWkOGyazPTncI+vZdY/sgStJmVQPEAHA0zjFcSvDHnJNpVRjclzb3F8uOafJmih 2twM57F0r0qjNtzwT3O2II7w57dND2mB/YLhps8xVrJlzl705GveslKfc+jpyAoLgW8x hiNVwa5/p7rWKs1g0qJF9apnVVpgXnEhomKuO4InHkPBix9n2/FkKolCHIH0h7gt17TA 5CSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=bCBGp2GdHMYoIw7eJTRwaDF9qwXNx38EngT3N/EUqtQ=; b=Aihk8N/dWZ2QPIhXyd8NpYGCHIRVZgRYs2h4PuHsZLSD0JrfOKHLAiK5BhvjFoy1Qz AtlcaLZKBueIXdAZATgG/kffMqR9rOTYr7Gz6yHeVRMu1qHxCEmVA1M+Lba7ujuLBQeD wgrYTiClxulkHXun1nSv3Jl36nOlVw5ZCniQnEh8EUXmkfbxMwFfjCMUoBjus7t5kuvt vDh0/yBpgUKp1wZKHTVKji5K/+TN6RJuYiywAMrwtQiuAkbh9z0HGlouAHRE0O2+cBcX FgDgiz2ddlY5sq/JAEhVqOR9H4mN0utUdJKq3tjd0kWrmLy8JjgafBWCvy/EMqi2eYLf adfg== X-Gm-Message-State: AOAM532BEralO9VLbayP0Dic56E5l5sUxRbwYpDB7CsZPPOngNVTBj6/ VX+hY57SgU3nNPEIIWg81hqOGyOTgW1WNw== X-Received: by 2002:a17:90a:af95:: with SMTP id w21mr16717053pjq.178.1619814581298; Fri, 30 Apr 2021 13:29:41 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 48/82] target/arm: Pass separate addend to {U, S}DOT helpers Date: Fri, 30 Apr 2021 13:25:36 -0700 Message-Id: <20210430202610.1136687-49-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- v4: Fix double addition (zhiwei). --- target/arm/helper.h | 20 +++--- target/arm/sve.decode | 7 ++- target/arm/translate-a64.c | 15 ++++- target/arm/translate-neon.c | 10 +-- target/arm/translate-sve.c | 13 ++-- target/arm/vec_helper.c | 120 ++++++++++++++++++++---------------- 6 files changed, 109 insertions(+), 76 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index 23a7ec5638..f4b092ee1c 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -608,15 +608,19 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlah_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_4(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 67b6466a1e..04ef38f148 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -756,12 +756,13 @@ UMIN_zzi 00100101 .. 101 011 110 ........ ..... @rdn_i8u MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s # SVE integer dot product (unpredicated) -DOT_zzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 ra=%reg_movprfx +DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ + ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzx 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ sz=0 ra=%reg_movprfx -DOT_zzx 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ +DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ sz=1 ra=%reg_movprfx # SVE floating-point complex add (predicated) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index bdd47208b1..61c5fa9656 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -683,6 +683,17 @@ static void gen_gvec_op3_qc(DisasContext *s, bool is_q, int rd, int rn, tcg_temp_free_ptr(qc_ptr); } +/* Expand a 4-operand operation using an out-of-line helper. */ +static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, int data, gen_helper_gvec_4 *fn) +{ + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12183,7 +12194,7 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) return; case 0x2: /* SDOT / UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, 0, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b); return; @@ -13442,7 +13453,7 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) switch (16 * u + opcode) { case 0x0e: /* SDOT */ case 0x1e: /* UDOT */ - gen_gvec_op3_ool(s, is_q, rd, rn, rm, index, + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, index, u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b); return; diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index 658bd275da..fa67605fdc 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -230,7 +230,7 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) static bool trans_VDOT(DisasContext *s, arg_VDOT *a) { int opr_sz; - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; if (!dc_isar_feature(aa32_dp, s)) { return false; @@ -252,9 +252,10 @@ static bool trans_VDOT(DisasContext *s, arg_VDOT *a) opr_sz = (1 + a->q) * 8; fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, 0, fn_gvec); return true; } @@ -332,7 +333,7 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { - gen_helper_gvec_3 *fn_gvec; + gen_helper_gvec_4 *fn_gvec; int opr_sz; TCGv_ptr fpst; @@ -357,9 +358,10 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(FPST_STD); - tcg_gen_gvec_3_ool(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->rm), + vfp_reg_offset(1, a->vd), opr_sz, opr_sz, a->index, fn_gvec); tcg_temp_free_ptr(fpst); return true; diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 3de14e3af9..fab5d1c077 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3800,28 +3800,29 @@ DO_ZZI(UMIN, umin) #undef DO_ZZI -static bool trans_DOT_zzz(DisasContext *s, arg_DOT_zzz *a) +static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_b, gen_helper_gvec_sdot_h }, { gen_helper_gvec_udot_b, gen_helper_gvec_udot_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, 0); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->ra, 0); } return true; } -static bool trans_DOT_zzx(DisasContext *s, arg_DOT_zzx *a) +static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) { - static gen_helper_gvec_3 * const fns[2][2] = { + static gen_helper_gvec_4 * const fns[2][2] = { { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } }; if (sve_access_check(s)) { - gen_gvec_ool_zzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, a->index); + gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, + a->ra, a->index); } return true; } diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index a3d80ecad0..f88e572132 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -375,71 +375,76 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, * All elements are treated equally, no matter where they are. */ -void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 4; ++i) { - d[i] += n[i * 4 + 0] * m[i * 4 + 0] - + n[i * 4 + 1] * m[i * 4 + 1] - + n[i * 4 + 2] * m[i * 4 + 2] - + n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (int64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (int64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (int64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (int64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (int64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - d[i] += (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] - + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] - + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] - + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]; + d[i] = (a[i] + + (uint64_t)n[i * 4 + 0] * m[i * 4 + 0] + + (uint64_t)n[i * 4 + 1] * m[i * 4 + 1] + + (uint64_t)n[i * 4 + 2] * m[i * 4 + 2] + + (uint64_t)n[i * 4 + 3] * m[i * 4 + 3]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + int32_t *d = vd, *a = va; int8_t *n = vn; int8_t *m_indexed = (int8_t *)vm + H4(index) * 4; @@ -455,10 +460,11 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) int8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -466,11 +472,12 @@ void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; intptr_t index = simd_data(desc); - uint32_t *d = vd; + uint32_t *d = vd, *a = va; uint8_t *n = vn; uint8_t *m_indexed = (uint8_t *)vm + H4(index) * 4; @@ -486,10 +493,11 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) uint8_t m3 = m_indexed[i * 4 + 3]; do { - d[i] += n[i * 4 + 0] * m0 - + n[i * 4 + 1] * m1 - + n[i * 4 + 2] * m2 - + n[i * 4 + 3] * m3; + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); } while (++i < segend); segend = i + 4; } while (i < opr_sz_4); @@ -497,11 +505,12 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + int64_t *d = vd, *a = va; int16_t *n = vn; int16_t *m_indexed = (int16_t *)vm + index * 4; @@ -509,30 +518,33 @@ void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { - uint64_t d0, d1; + for (i = 0; i < opr_sz_8; i += 2) { + int64_t d0, d1; - d0 = n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (int64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (int64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (int64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (int64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (int64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (int64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (int64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (int64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; intptr_t index = simd_data(desc); - uint64_t *d = vd; + uint64_t *d = vd, *a = va; uint16_t *n = vn; uint16_t *m_indexed = (uint16_t *)vm + index * 4; @@ -540,22 +552,24 @@ void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) * Process the entire segment all at once, writing back the results * only after we've consumed all of the inputs. */ - for (i = 0; i < opr_sz_8 ; i += 2) { + for (i = 0; i < opr_sz_8; i += 2) { uint64_t d0, d1; - d0 = n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; + d0 = a[i + 0]; + d0 += n[i * 4 + 0] * (uint64_t)m_indexed[i * 4 + 0]; d0 += n[i * 4 + 1] * (uint64_t)m_indexed[i * 4 + 1]; d0 += n[i * 4 + 2] * (uint64_t)m_indexed[i * 4 + 2]; d0 += n[i * 4 + 3] * (uint64_t)m_indexed[i * 4 + 3]; - d1 = n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; + + d1 = a[i + 1]; + d1 += n[i * 4 + 4] * (uint64_t)m_indexed[i * 4 + 0]; d1 += n[i * 4 + 5] * (uint64_t)m_indexed[i * 4 + 1]; d1 += n[i * 4 + 6] * (uint64_t)m_indexed[i * 4 + 2]; d1 += n[i * 4 + 7] * (uint64_t)m_indexed[i * 4 + 3]; - d[i + 0] += d0; - d[i + 1] += d1; + d[i + 0] = d0; + d[i + 1] = d1; } - clear_tail(d, opr_sz, simd_maxsz(desc)); } From patchwork Fri Apr 30 20:25:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429881 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp795025jao; Fri, 30 Apr 2021 14:06:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzsYAh6CofGvQHPLjsUbG1w/vZUtAl64vY3f58Rs35ONuF39wbaImm0hac18l6/OxfG5LVP X-Received: by 2002:a02:9109:: with SMTP id a9mr6874760jag.93.1619816774997; Fri, 30 Apr 2021 14:06:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816774; cv=none; d=google.com; s=arc-20160816; b=lYVQLbVa9vTG5MxQYKvdfkbRDlh1nJHeYdZHCTVPedmEAvxFY6tZgsg4KUu5GUdJVV eOL0zBI8o79RHdVzRjzWvR9Jol4WWwGotpMCauNu9+bj2pMJy7Y/x/labytN8H8eJZLM ZaxnNd8RPA8TUN0TBxrPn+faJRczA/dq/LZN2q+EAsb+9IvID38tvhDjyrFM/BvMsygj NwyOCUD7do2ug5ZRRE09rCpJrYgtP6ADRBjGUdlkUnDdSLLiAyvBw+BLoGYT8BF0KQ9O ltu9pPjRjtQ+HK+9JOierFilwcrf0Ilt7zsu+jhtSIk5n3ZHVOJpUEhVXR7ffcwouXiQ DndA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=9nv/plSGKJoDrX4wUb0IosGgt/lOajdP1wQI4nOYPDs=; b=fWvKaM7ec5Ql7adGl/wMFVAqYepNibE5/5zM97ByJeUW1M51EHpbxekzcSKht27PhN d79v/t5DTusEb0x21pdMDuS1wRxz52qSDD30G2VgoimvABTRx3sr4M1v+x3hQGt5WVwS /ev7MEIGk9cGRU2YXTVvAPfRwUd/tZakCAGvjD2Kn3slKmDgy2QEWtq7IvgmzMDbOZ+3 1vvUOyw+WtJ+MA1cLSwfxBTZ63a6BOD8GaqvCE4uSyr1PCtO2QmitoM3oKNuTnT5WyJ1 oGBIQhAe45y6HLJcsCq1drfbCrkko+3FvzJo715d+tFCqIlYAcalYs64kT5A3qRW5Ytr b2JQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Ip8/LWDP"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q4si3343733jas.111.2021.04.30.14.06.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:06:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="Ip8/LWDP"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55694 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaL0-00063U-Am for patch@linaro.org; Fri, 30 Apr 2021 17:06:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53254) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZm8-0004Fe-Cd for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:17 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]:40773) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlk-0007i4-4i for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:10 -0400 Received: by mail-pl1-x62b.google.com with SMTP id 20so33287663pll.7 for ; Fri, 30 Apr 2021 13:29:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=9nv/plSGKJoDrX4wUb0IosGgt/lOajdP1wQI4nOYPDs=; b=Ip8/LWDPVVhLTtHatZHTwi0YxE0ppy3h02FdvC693gftdGKoNNdry6LoHkhNlwzlYh HDFb1fdu/eZfrmvjGJD9tYwqPeI6wfv8WXS6Ufl0vyvlORucBH0R9KbGdfWZHNiWJfY3 5Q9UnD3xBAgvi+XI7+o9rgKYU23YKbg5LycfFNzIVsiF9mUBLT7NZfJDj2AK10RUBMPo AbqllkrT5vR0hkmtF6PT6zkdTa45y/xZPl6wo/Ao440yNREtcsg5j3djXnLsP/OfDuke lA2rK6fEMGkTLe0Wj1v8ooop8wjpS8Ot0vZtlhGA14q1JVRs/LCRHNEulURmSshCPzSa 7wyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=9nv/plSGKJoDrX4wUb0IosGgt/lOajdP1wQI4nOYPDs=; b=L4OX05VazoAcpnMalx2FlW54KE6pIBwpRExzx9QOKu0TWM7GhlqOn1X5yY6iQGtOmN /CwtEGIabe5Kd8VPe1hssfIMOOjJDc8gRyparDrgXf+QEXfjC7048h+5uVKQfBZYPcQX JDPj8bpef8+QDVS9MRUJzRSMqRYrg5+J0l/Wj4Tj4C9FkA+PBOArEgTLF6rajSqjY03s oP/8LyilDuESQJUeS2jmJEodOSI2H4Iz7p1T6kTnuS6zKiDK+WE9l2cAhc7omguJ6CIy 3Uy5Hlv7mrXQO3Ex9AbChmkJ8zdquC3SzyBeqQyIw3xTfvrcpME3EchgADluPODxc3W1 0QNA== X-Gm-Message-State: AOAM531hIHXEdZJG+TqLYwnNBlTTznRY62R/qDEH5zp5WXF7XAEUSChT DGoTI0CH0ttIsr+nt0KA2Kt1ymCtrv7cZg== X-Received: by 2002:a17:90a:e645:: with SMTP id ep5mr16868687pjb.143.1619814581913; Fri, 30 Apr 2021 13:29:41 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 49/82] target/arm: Pass separate addend to FCMLA helpers Date: Fri, 30 Apr 2021 13:25:37 -0700 Message-Id: <20210430202610.1136687-50-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" For SVE, we potentially have a 4th argument coming from the movprfx instruction. Currently we do not optimize movprfx, so the problem is not visible. Signed-off-by: Richard Henderson --- target/arm/helper.h | 20 +++++++-------- target/arm/translate-a64.c | 28 +++++++++++++++++---- target/arm/translate-neon.c | 10 +++++--- target/arm/translate-sve.c | 5 ++-- target/arm/vec_helper.c | 50 +++++++++++++++---------------------- 5 files changed, 62 insertions(+), 51 deletions(-) -- 2.25.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index f4b092ee1c..72c5bf6aca 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -629,16 +629,16 @@ DEF_HELPER_FLAGS_5(gvec_fcadds, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(gvec_fcaddd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlah_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlas_idx, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) -DEF_HELPER_FLAGS_5(gvec_fcmlad, TCG_CALL_NO_RWG, - void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlah_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlas_idx, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(gvec_fcmlad, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_paddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(neon_pmaxh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 61c5fa9656..a8edd2d281 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -694,6 +694,23 @@ static void gen_gvec_op4_ool(DisasContext *s, bool is_q, int rd, int rn, is_q ? 16 : 8, vec_full_reg_size(s), data, fn); } +/* + * Expand a 4-operand + fpstatus pointer + simd data value operation using + * an out-of-line helper. + */ +static void gen_gvec_op4_fpst(DisasContext *s, bool is_q, int rd, int rn, + int rm, int ra, bool is_fp16, int data, + gen_helper_gvec_4_ptr *fn) +{ + TCGv_ptr fpst = fpstatus_ptr(is_fp16 ? FPST_FPCR_F16 : FPST_FPCR); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), + vec_full_reg_offset(s, rn), + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, ra), fpst, + is_q ? 16 : 8, vec_full_reg_size(s), data, fn); + tcg_temp_free_ptr(fpst); +} + /* Set ZF and NF based on a 64 bit result. This is alas fiddlier * than the 32 bit equivalent. */ @@ -12205,15 +12222,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) rot = extract32(opcode, 0, 2); switch (size) { case 1: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, true, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, true, rot, gen_helper_gvec_fcmlah); break; case 2: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlas); break; case 3: - gen_gvec_op3_fpst(s, is_q, rd, rn, rm, false, rot, + gen_gvec_op4_fpst(s, is_q, rd, rn, rm, rd, false, rot, gen_helper_gvec_fcmlad); break; default: @@ -13464,9 +13481,10 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) { int rot = extract32(insn, 13, 2); int data = (index << 2) | rot; - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, rd), vec_full_reg_offset(s, rn), - vec_full_reg_offset(s, rm), fpst, + vec_full_reg_offset(s, rm), + vec_full_reg_offset(s, rd), fpst, is_q ? 16 : 8, vec_full_reg_size(s), data, size == MO_64 ? gen_helper_gvec_fcmlas_idx diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index fa67605fdc..45fa5166f3 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -155,7 +155,7 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) { int opr_sz; TCGv_ptr fpst; - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; if (!dc_isar_feature(aa32_vcma, s) || (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s))) { @@ -180,9 +180,10 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); fn_gvec_ptr = (a->size == MO_16) ? gen_helper_gvec_fcmlah : gen_helper_gvec_fcmlas; - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); @@ -293,7 +294,7 @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) { - gen_helper_gvec_3_ptr *fn_gvec_ptr; + gen_helper_gvec_4_ptr *fn_gvec_ptr; int opr_sz; TCGv_ptr fpst; @@ -322,9 +323,10 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) gen_helper_gvec_fcmlah_idx : gen_helper_gvec_fcmlas_idx; opr_sz = (1 + a->q) * 8; fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - tcg_gen_gvec_3_ptr(vfp_reg_offset(1, a->vd), + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->vm), + vfp_reg_offset(1, a->vd), fpst, opr_sz, opr_sz, (a->index << 2) | a->rot, fn_gvec_ptr); tcg_temp_free_ptr(fpst); diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fab5d1c077..17789ce173 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4383,7 +4383,7 @@ static bool trans_FCMLA_zpzzz(DisasContext *s, arg_FCMLA_zpzzz *a) static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) { - static gen_helper_gvec_3_ptr * const fns[2] = { + static gen_helper_gvec_4_ptr * const fns[2] = { gen_helper_gvec_fcmlah_idx, gen_helper_gvec_fcmlas_idx, }; @@ -4393,9 +4393,10 @@ static bool trans_FCMLA_zzxz(DisasContext *s, arg_FCMLA_zzxz *a) if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_ptr status = fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); - tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), status, vsz, vsz, a->index * 4 + a->rot, fns[a->esz - 1]); diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index f88e572132..b19877e0d3 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -657,13 +657,11 @@ void HELPER(gvec_fcaddd)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -680,19 +678,17 @@ void HELPER(gvec_fcmlah)(void *vd, void *vn, void *vm, float16 e4 = e2; float16 e3 = m[H2(i + 1 - flip)] ^ neg_imag; - d[H2(i)] = float16_muladd(e2, e1, d[H2(i)], 0, fpst); - d[H2(i + 1)] = float16_muladd(e4, e3, d[H2(i + 1)], 0, fpst); + d[H2(i)] = float16_muladd(e2, e1, a[H2(i)], 0, fpst); + d[H2(i + 1)] = float16_muladd(e4, e3, a[H2(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float16 *d = vd; - float16 *n = vn; - float16 *m = vm; + float16 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -716,20 +712,18 @@ void HELPER(gvec_fcmlah_idx)(void *vd, void *vn, void *vm, float16 e2 = n[H2(j + flip)]; float16 e4 = e2; - d[H2(j)] = float16_muladd(e2, e1, d[H2(j)], 0, fpst); - d[H2(j + 1)] = float16_muladd(e4, e3, d[H2(j + 1)], 0, fpst); + d[H2(j)] = float16_muladd(e2, e1, a[H2(j)], 0, fpst); + d[H2(j + 1)] = float16_muladd(e4, e3, a[H2(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -746,19 +740,17 @@ void HELPER(gvec_fcmlas)(void *vd, void *vn, void *vm, float32 e4 = e2; float32 e3 = m[H4(i + 1 - flip)] ^ neg_imag; - d[H4(i)] = float32_muladd(e2, e1, d[H4(i)], 0, fpst); - d[H4(i + 1)] = float32_muladd(e4, e3, d[H4(i + 1)], 0, fpst); + d[H4(i)] = float32_muladd(e2, e1, a[H4(i)], 0, fpst); + d[H4(i + 1)] = float32_muladd(e4, e3, a[H4(i + 1)], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float32 *d = vd; - float32 *n = vn; - float32 *m = vm; + float32 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint32_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -782,20 +774,18 @@ void HELPER(gvec_fcmlas_idx)(void *vd, void *vn, void *vm, float32 e2 = n[H4(j + flip)]; float32 e4 = e2; - d[H4(j)] = float32_muladd(e2, e1, d[H4(j)], 0, fpst); - d[H4(j + 1)] = float32_muladd(e4, e3, d[H4(j + 1)], 0, fpst); + d[H4(j)] = float32_muladd(e2, e1, a[H4(j)], 0, fpst); + d[H4(j + 1)] = float32_muladd(e4, e3, a[H4(j + 1)], 0, fpst); } } clear_tail(d, opr_sz, simd_maxsz(desc)); } -void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, +void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, void *va, void *vfpst, uint32_t desc) { uintptr_t opr_sz = simd_oprsz(desc); - float64 *d = vd; - float64 *n = vn; - float64 *m = vm; + float64 *d = vd, *n = vn, *m = vm, *a = va; float_status *fpst = vfpst; intptr_t flip = extract32(desc, SIMD_DATA_SHIFT, 1); uint64_t neg_imag = extract32(desc, SIMD_DATA_SHIFT + 1, 1); @@ -812,8 +802,8 @@ void HELPER(gvec_fcmlad)(void *vd, void *vn, void *vm, float64 e4 = e2; float64 e3 = m[i + 1 - flip] ^ neg_imag; - d[i] = float64_muladd(e2, e1, d[i], 0, fpst); - d[i + 1] = float64_muladd(e4, e3, d[i + 1], 0, fpst); + d[i] = float64_muladd(e2, e1, a[i], 0, fpst); + d[i + 1] = float64_muladd(e4, e3, a[i + 1], 0, fpst); } clear_tail(d, opr_sz, simd_maxsz(desc)); } From patchwork Fri Apr 30 20:25:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429885 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp797315jao; Fri, 30 Apr 2021 14:09:15 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyBtMSfw1bjB6M0j6fVVIRhBIFgnrRCeG7KemCthfYUmT+mHncafqY+mOOCP57zvTEf7hvp X-Received: by 2002:a05:6e02:1d0e:: with SMTP id i14mr5191777ila.230.1619816954881; Fri, 30 Apr 2021 14:09:14 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816954; cv=none; d=google.com; s=arc-20160816; b=UN0p6jfeFz1AJ2ueJlzLnzljBsNJOKmVXhssfea1xVGdtImf/eCvQs2qjmmQccGycO xIkhAhm2Hvi23w790pnBJeVYTLrv7UYns/QnW1yWmD7jPm6KK/iLAzY+X5hcoONpUqsD iHRyZ1W+LjnXSkryQhELYxQ4ZV154kDqg2twZARwS1hh/F7RxlVDzeaSUA2LHg55GHKf OnWGEAmqhVsxtagnvGh/td3q3GYCKw66/OzCQLBClNzeX5OI3AuOL660B1zlHHxx0dfZ ZplxzbBke1/ykwteBdn1pmBNUT/+dhrEnHR91fDbCWsUf7yX1gYRN1KQtexprDPYgcUR etww== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=I4EJT/ZWFQ6IB/dGHyJmDlUDSt3e92FwFyTDZa4sVYkzEW59m3B/DJnA19MxCGskl4 MXv/o1bQWXn7khKLLryopGfUG5PTTkNhb3FKyCuNXffkdvpq2V1C8GtczxCqR0njazbp QYPVtfg7ZWihjKODhLBe2QjeHUS3cKsVFzpubWrc3PpZf3ssn8RADbeIaboHVPBiAZGs kYRXITwZLjyVjaQjxCKg2w/qOc8lhdcmrjxByaPzK3txUD4+68fz1YB7zIei5LKiRPH7 g0vhU20lFpZtHB2jwLm3EMSmUFaSFzHzwQixSZjttrliWX1xg7VVessrPU1D3s+rqi3v Fn6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nw+BwVlz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v21si4160098jas.29.2021.04.30.14.09.14 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:09:14 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Nw+BwVlz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:35570 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaNu-0000z3-AZ for patch@linaro.org; Fri, 30 Apr 2021 17:09:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53468) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmO-0004aH-FO for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:28 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]:33749) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007iB-DS for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:28 -0400 Received: by mail-pl1-x62a.google.com with SMTP id b21so8732226plz.0 for ; Fri, 30 Apr 2021 13:29:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=Nw+BwVlzPjvbq6hwS+9+9IBuU9JKmSAgEjPrbj2E9Sx/Ot7I2Tllh16XPMX0jdmJbh o4PQkG8wkPnqps0HEGYSG/6FwZFV5sARcZ2QzpbNLjXk7SXP7d0VUEmtfPXGSYATCOdm 5EMYKr1DcAPc/GclzYYYUbvbWQW5YwjIyWaw7WrkVZjXM66pjUlzQJMLnYh9hz428gdJ ROlkWLpMC0Iv0hXkUfyrIqphL6lnqCAs5Xr8bnTSn+HZ7JEEXjqLRUx+9b6eVg0c4Ow5 A6tRTSiOxiGeiAb7QevViWH4hTM4aIam7lWzgZnQmex6+rMni5+lwoOdDDtuOODF0Sm0 KxIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RfpadBU7KlY8uj8mgdV4cd4GjWhl0iserVzzK3wcZFQ=; b=sgL3aP6HG4Q6t6Cu/f3Bx2wC0E2BCnTRsnFgXaeyTCLIL8PuAu5MbTG6hWo4p55A99 XCqY5sl1DnEPUuZO6glJRU+tAj/8AqqYmipdhPo4KOg5d4dKmjzGQwI/RgQBw/w6b4Fh PVMCrmOc2JHK4xEpb8DaASNGuHbu0wGESmf0bnfTp2RXkBNBzBedJC5h6zP+qVwVpzVa Qp3H6XO6PhDly/6VPvRxrei/IXgqyKdmm4ZrSmfZh5hJ6cjHYF0ZpfeA/RKAE/BlQDdS uBMomZ+xIiMfhCFT++xHkY8zy0vyAeKlQvbym6roTt9yz3oj4zgHw1fg/cOeQBlhCrvy 5fDQ== X-Gm-Message-State: AOAM5329bQgBimjHW0qf1ckp3pthnO/NJ0CvlHax5IvDgql/BQ2oGX14 i/XA+2jtUYF/AbpC26CJqOArjRjYQu9oUg== X-Received: by 2002:a17:90a:fd92:: with SMTP id cx18mr6940822pjb.209.1619814582527; Fri, 30 Apr 2021 13:29:42 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 50/82] target/arm: Split out formats for 2 vectors + 1 index Date: Fri, 30 Apr 2021 13:25:38 -0700 Message-Id: <20210430202610.1136687-51-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Currently only used by FMUL, but will shortly be used more. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 04ef38f148..a504b55dad 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -67,6 +67,7 @@ &rri_esz rd rn imm esz &rrri_esz rd rn rm imm esz &rrr_esz rd rn rm esz +&rrx_esz rd rn rm index esz &rpr_esz rd pg rn esz &rpr_s rd pg rn s &rprr_s rd pg rn rm s @@ -245,6 +246,12 @@ @rpri_scatter_store ....... msz:2 .. imm:5 ... pg:3 rn:5 rd:5 \ &rpri_scatter_store +# Two registers and a scalar by N-bit index +@rrx_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_22_19 +@rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz +@rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -792,10 +799,9 @@ FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ ### SVE FP Multiply Indexed Group # SVE floating-point multiply (indexed) -FMUL_zzx 01100100 0.1 .. rm:3 001000 rn:5 rd:5 \ - index=%index3_22_19 esz=1 -FMUL_zzx 01100100 101 index:2 rm:3 001000 rn:5 rd:5 esz=2 -FMUL_zzx 01100100 111 index:1 rm:4 001000 rn:5 rd:5 esz=3 +FMUL_zzx 01100100 0. 1 ..... 001000 ..... ..... @rrx_3 esz=1 +FMUL_zzx 01100100 10 1 ..... 001000 ..... ..... @rrx_2 esz=2 +FMUL_zzx 01100100 11 1 ..... 001000 ..... ..... @rrx_1 esz=3 ### SVE FP Fast Reduction Group From patchwork Fri Apr 30 20:25:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429886 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp798984jao; Fri, 30 Apr 2021 14:12:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxwK+COPPzdRr2StPrWgbw6F6hz+Xo4lDLx1vecWbdXb+Qfs0TGVMO0St2YcKOkb3LIlKxD X-Received: by 2002:a05:6e02:160a:: with SMTP id t10mr6057429ilu.261.1619817125425; Fri, 30 Apr 2021 14:12:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817125; cv=none; d=google.com; s=arc-20160816; b=aApogG/cpJrg6zGRCbBuUeKjtm66UGrKvkTS8AzPXU3doUB65Un0ruB5ylkEeLagfT sjc5qqf4ditB+im1TZLt0aH0PmnWcvy7O/SZrCc4t7CuCDm6zB2Q9HBgP5X3bhh4FbOO 0JlemcwiqxUr4q5tLcPNjURQfYYrmvADPYIualkZfTPDMcfirz501ZNKwLFz+g7y18id T+WDpQ8SwgapMVymFkkBNY6OFy5ZEGLd8LzfBqmcNK9a/SV0l31/3SPMMtVoyWH6loyE PfD9SwIi1StLG0QxBdk6dmFDuAMFlQUxJksQOL+zODBQLKRwjoySL88OIAU80yEc5Tlt 7UNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=8LKuIB6wJ5LVWgX/IoMOV5QKzsiXXzsJsAVqZyabZRk=; b=ZHzciF9lrQWs1+ohPuegAdDgLA+f0alNrolFDyKmSSY9cTe2ixOJdeMZtlDOG0O3U9 FJJhScwBLOlmfFkmZy9s8mpS+BOSRdbFJZb9gZ7D14uKVS4Z/QIPyJiQLP+Q+7JJ/FrR LpaRp2L2EcHUfETBNzFBKZKBsZaap1GBn5tH0HqOL7T2xtzucfb4TXvuYF175oBoCo1J pMHd7Nb9Fh85t65RGhixvh4wWLlAwae+J8YMQwi37yPgsMYnJu+7ljJ8Ec20R4BPv58g 3C4Z9LW+clfHcpglTjs33YsP+ZVVoXODkh5jwn1zha6Pj9hrfLX39H2ejfbYkHmyEaar 2Sag== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TcR0mnWQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p15si5641588iov.79.2021.04.30.14.12.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:12:05 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TcR0mnWQ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43078 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaQe-0004D9-Rl for patch@linaro.org; Fri, 30 Apr 2021 17:12:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53434) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmL-0004XW-QD for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:27 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:35671) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007iM-B2 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:25 -0400 Received: by mail-pf1-x42f.google.com with SMTP id i13so2283940pfu.2 for ; Fri, 30 Apr 2021 13:29:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=8LKuIB6wJ5LVWgX/IoMOV5QKzsiXXzsJsAVqZyabZRk=; b=TcR0mnWQoBm2Dz+HCRCM0f6shUpETxDfLqhNVM7TLKFha8TrvA6vcFdNwTSKZ2denq Za1zNiQnhNLJPHyIqgIBmTzSjVvevrbcbSuaS3LSJuzx1da8rieNlk762lsFilSPedmk 0jTWn9b8eHp/CnpUyBK/4TVOLxnaR1TfeD5mfgt/TCEnIQWBdvgaUvB4B4m2G2p0Wi+S loF48jPPHDwspatpkNZI4tVk2uEAB+DllvHykxfFyl89N9+NeeBX0D7njKQA3CwFAbNX DRdITkQlsnh11uPeLtq/OZ9hOV6zD3VngEVHzwH48C8WLSivH77EgUAJEcUuWs+b1Lz6 Ppxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8LKuIB6wJ5LVWgX/IoMOV5QKzsiXXzsJsAVqZyabZRk=; b=GrlIfTakKnObvqZ278m6ZjDm57/7jCsNfcNlFfN9rQd2GOeL4blW6VMwTr06O66Crm kHFDRvDT38M2DOqrcPRBapdTZMQfCKQxJaUe08iGH1wr5qpAPXKqwvmliToJm3xYCvyj 1VvD4YR5BHH59a+66v6BLaBnlEZMK0SFIjQMz2G6U98pHdR/sfAVopENV12Lxhh4P/VH w87RScBd5djoHp4hKovF6mTh8xFJgIOZovTeLm/y/OeSzp7xi6/MCvIPb8Vuk2bHuTMH 2Sva6vjPejrj5rUydrlMQgE8HfcwH364qIE5T9DxsR1qqrO3C+D1fxAV1yqMdRjtROS6 otRQ== X-Gm-Message-State: AOAM530NIjfXN3EU0/odN0jmdjGGeA5/wodBm4WXqMGMmtd4lqH04VW7 YhRnCGzZwzWgRHZ98oTIK1uN031rz3KCcg== X-Received: by 2002:a05:6a00:1502:b029:275:f676:5eef with SMTP id q2-20020a056a001502b0290275f6765eefmr6760555pfu.30.1619814583078; Fri, 30 Apr 2021 13:29:43 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 51/82] target/arm: Split out formats for 3 vectors + 1 index Date: Fri, 30 Apr 2021 13:25:39 -0700 Message-Id: <20210430202610.1136687-52-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Used by FMLA and DOT, but will shortly be used more. Split FMLA from FMLS to avoid an extra sub field; similarly for SDOT from UDOT. Signed-off-by: Richard Henderson --- target/arm/sve.decode | 29 +++++++++++++++++++---------- target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++---------- 2 files changed, 47 insertions(+), 20 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a504b55dad..74ac72bdbd 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -73,6 +73,7 @@ &rprr_s rd pg rn rm s &rprr_esz rd pg rn rm esz &rrrr_esz rd ra rn rm esz +&rrxr_esz rd rn rm ra index esz &rprrr_esz rd pg rn rm ra esz &rpri_esz rd pg rn imm esz &ptrue rd esz pat s @@ -252,6 +253,14 @@ @rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz @rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz +# Three registers and a scalar by N-bit index +@rrxr_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_22_19 +@rrxr_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx +@rrxr_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -767,10 +776,10 @@ DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx # SVE integer dot product (indexed) -DOT_zzxw 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ - sz=0 ra=%reg_movprfx -DOT_zzxw 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ - sz=1 ra=%reg_movprfx +SDOT_zzxw_s 01000100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 +SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 +UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 +UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ @@ -789,12 +798,12 @@ FCMLA_zzxz 01100100 11 1 index:1 rm:4 0001 rot:2 rn:5 rd:5 \ ### SVE FP Multiply-Add Indexed Group # SVE floating-point multiply-add (indexed) -FMLA_zzxz 01100100 0.1 .. rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx index=%index3_22_19 esz=1 -FMLA_zzxz 01100100 101 index:2 rm:3 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=2 -FMLA_zzxz 01100100 111 index:1 rm:4 00000 sub:1 rn:5 rd:5 \ - ra=%reg_movprfx esz=3 +FMLA_zzxz 01100100 0. 1 ..... 000000 ..... ..... @rrxr_3 esz=1 +FMLA_zzxz 01100100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 +FMLA_zzxz 01100100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 +FMLS_zzxz 01100100 0. 1 ..... 000001 ..... ..... @rrxr_3 esz=1 +FMLS_zzxz 01100100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 +FMLS_zzxz 01100100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 ### SVE FP Multiply Indexed Group diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 17789ce173..bb0cffa3d9 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,26 +3813,34 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } -static bool trans_DOT_zzxw(DisasContext *s, arg_DOT_zzxw *a) +static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) { - static gen_helper_gvec_4 * const fns[2][2] = { - { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, - { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } - }; - + if (fn == NULL) { + return false; + } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fns[a->u][a->sz], a->rd, a->rn, a->rm, - a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); } return true; } +#define DO_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_zzxz_ool(s, a, FUNC); } + +DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) +DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) +DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) +DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) + +#undef DO_RRXR /* *** SVE Floating Point Multiply-Add Indexed Group */ -static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +static bool do_FMLA_zzxz(DisasContext *s, arg_rrxr_esz *a, bool sub) { static gen_helper_gvec_4_ptr * const fns[3] = { gen_helper_gvec_fmla_idx_h, @@ -3847,13 +3855,23 @@ static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), vec_full_reg_offset(s, a->ra), - status, vsz, vsz, (a->index << 1) | a->sub, + status, vsz, vsz, (a->index << 1) | sub, fns[a->esz - 1]); tcg_temp_free_ptr(status); } return true; } +static bool trans_FMLA_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, false); +} + +static bool trans_FMLS_zzxz(DisasContext *s, arg_FMLA_zzxz *a) +{ + return do_FMLA_zzxz(s, a, true); +} + /* *** SVE Floating Point Multiply Indexed Group */ From patchwork Fri Apr 30 20:25:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429872 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp788577jao; Fri, 30 Apr 2021 13:58:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy3ENNeAnQ9qpLtpAVoqLt2Ar8rTkmcHYacx3vAa/zGLqV51bhRnyDZxbKD6k64Ednlu8sm X-Received: by 2002:a05:6602:22c9:: with SMTP id e9mr5383567ioe.73.1619816293719; Fri, 30 Apr 2021 13:58:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816293; cv=none; d=google.com; s=arc-20160816; b=KWLCb83Qt+l8isY7zylVSauy4Ny6EdRmp0H9JEiVdZKxzy01qywjh3nYTTXEoS6VE0 Ptn+ZlO3cTf56KXxARjL59wsprzP/IHaeU/gWD5QeBxzGDHdYH3ASHbR7tmn3/vTnXt+ DkFYY8JrBjiStN//NORBeNgmRVXcjJFzCITztE6qqB6gUkaY4xAtSSAThNT9F/2+ZiwP STa7Gh+V5f0lbMWW5wviL2Q5AatXiXFH0huYSqAKyx8DR1lxRbogKMD/1wArpx/tuq+P orA6YMbdfWY5n3ZRhEfYvYDEyvUmRWy+lahxu7fZveHSEQC5FXo9QmNj4q0BSfykdEpT aumw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=E5JoMyVcVDEYFTAKWLhzjLu8b1RSgcPEa5JNeY4Iwkw=; b=b55kr3q23BuKN6uEZBSKfwrar8it4z0BnF802C+wLjP+/1ughjP13LHbIrhLLg8ZJF EpkvaxAqc8ByV4lQeaJM4gZ3ckUWjxLXp2K8XDqKYPsa6O7LpxCQ/otmxVWI6ngeVRQ7 FKubcYCQqWCUZeeCzBzFBkG6cxv6kuTOH2GNfNHgm+usfddCOhfZwVjbGN4M8N/F5SRa cXhv08KWwXJrDaTATHxFeFWtyaKK7iOLiNEg26OKCZgVevvgYGZnLbZ7nSYyRRiG1Ka4 9qZ1l3ZAEAM9MnGNjNygWmHoqvp4XAG0/EvAvDtvzPrrXkehXrnpwqQzXcUg0eXNGdRw HjSw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kfyBX1uO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w26si3329167jao.65.2021.04.30.13.58.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 13:58:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kfyBX1uO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34336 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaDF-0005dq-56 for patch@linaro.org; Fri, 30 Apr 2021 16:58:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53288) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmE-0004JT-Th for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:18 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]:45009) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007iW-33 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:18 -0400 Received: by mail-pf1-x42f.google.com with SMTP id m11so6099163pfc.11 for ; Fri, 30 Apr 2021 13:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=E5JoMyVcVDEYFTAKWLhzjLu8b1RSgcPEa5JNeY4Iwkw=; b=kfyBX1uOWwVOdyVtgsgJwX+p8x/BZLEK83wSYkt5spuNmJo1HuA4L6umlglGGrjWnD HaNQS70zZCtw/04LDunI/3LM9ncGGwS69O3ScfDEC5e5JE17NaOD2QlXi90tAEFgZKTd hYk8sNyuVZF/WDMi+zf7x5cl94E6Ys/ULrGpcIi35Ah1SJm0y9ZmzEPqDM80bmvE2WRi lkwavPGYdxo+2WHk08r/lukMxdN8ePw7kdVITGV7wqMKF+Yon+9gddI70HLboVmO9Cx8 BrlM3Gxd8vHSODS5ce2xXVGEBiY2LMxmUXmkb3AOjh1p3+kNtE42MMPaRbQF8T5B2kUe X81w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=E5JoMyVcVDEYFTAKWLhzjLu8b1RSgcPEa5JNeY4Iwkw=; b=jyvXHvfvz5eRTj5YJ734cvJxeQmxWFJPXfZZ40OCgC03jjF3RovkjMnGjYHhLDetnH yXpIeRpLkRTDUeYCVhYm5fms11XjOVLG8d+dK2FKVvggwuvJesiq9woZr7e4pJEGcINO 2uChQwHCDsBEUmPXr/RXF0kDkfoTV1jO7IcajpLMcvx6IPD/GGmal9LIp9cBUe6P7xft zadagTlrta0bvf77aQ1zYTNrsOjK8WOkNpNQn6a27ykwH6/mVQ2FomGgXT2yEF7dWFBw U6hCVAiCfAGTJkCas2Pu13evwmWaL3keEK9j6O6XmHnMqbPCGF0RANAgwAS8USyKywiY gpoA== X-Gm-Message-State: AOAM530CoOxb4kju2WokvWefDTQxddQHImQ1MBz+KmQxkZWFdzJkG6lm IIxP+FDKsZYXH3VetTeHH48huE+kYM3jFw== X-Received: by 2002:a65:4986:: with SMTP id r6mr6398649pgs.392.1619814583762; Fri, 30 Apr 2021 13:29:43 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 52/82] target/arm: Implement SVE2 integer multiply (indexed) Date: Fri, 30 Apr 2021 13:25:40 -0700 Message-Id: <20210430202610.1136687-53-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 30 ++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 74ac72bdbd..65cb0a2206 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -775,12 +775,19 @@ MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s DOT_zzzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 \ ra=%reg_movprfx +#### SVE Multiply - Indexed + # SVE integer dot product (indexed) SDOT_zzxw_s 01000100 10 1 ..... 000000 ..... ..... @rrxr_2 esz=2 SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 +# SVE2 integer multiply (indexed) +MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 +MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 +MUL_zzx_d 01000100 11 1 ..... 111110 ..... ..... @rrx_1 esz=3 + # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \ rn=%reg_movprfx diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index bb0cffa3d9..3f54636e11 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3813,6 +3813,10 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) return true; } +/* + * SVE Multiply - Indexed + */ + static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn) { @@ -3836,6 +3840,32 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR +static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn) +{ + if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, a->index, fn); + } + return true; +} + +#define DO_SVE2_RRX(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_ool(s, a, FUNC); } + +DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) +DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) +DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) + +#undef DO_SVE2_RRX + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 30 20:25:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429887 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp799064jao; Fri, 30 Apr 2021 14:12:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx+OPlTQDYtB9PqNFR+n4X5BIq7PiPNRhS+LuP9+ISVg9+scG9EvmVYF+9bk19w70Hpkwmv X-Received: by 2002:a5d:8b01:: with SMTP id k1mr5171181ion.127.1619817133240; Fri, 30 Apr 2021 14:12:13 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817133; cv=none; d=google.com; s=arc-20160816; b=03fhLrhgD4cVA+pPaAm+sHSKy0iPDn6kIUPtylYyEKbvFkeMGUU8DoM5CmGl9Fp+BV 1bWIMMQPHBY2DkZxasBacb3w7JadjrWuHJejZ7pkYZrAA1cnMCJTiRl0SycGzbyNaRbD hgNZ89XDO6o/b+hKgaqkwE6qivdgMQuGpMz/TQ2uSBBWfgG3xiUvIW8kzP70Rg+dJhNi C6SeAq0Yq4sO/UyeypkAASVM/BBLQ3fheBd9r1M5CDv26blmWW+sVYWk/pYM829opVX9 WelDTXtmmJD0A2fhznT7skX1uNwXf5M7AkHPPJu6yDgNLaJF/bHNRlp3MPrCFUULGuUl DU+A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4LOZdC1ItbQynZDOtyUtTS7T/flVKohWME+t/j4leKo=; b=AtKhsxi+XnJskqzXkAF3UzinwYq2lldHpKoYqV+KVN/v2lYEUN7H71ss+MRDNb8YOS nAog26FYdrSXnYp9skppMnzVKUZ6jnWbRIWAXgwxrfcowo+XXcBKBGHBN3kTuFJ6YxDh Gret6kGYY4BpKFW3PvrZBaeHW0sOLchy2D5Ghq68IIksionP4CjCjAhhxuGSHqEQPqQM 8BKN2uUXgP+P1HvzpXS/mAQgIi6Y9hFYP7keRkMrPcAxAt0xISKpq3s4LrfFwtsC2YIM SCHsGCNzW7aIzBMduc27sjVNqGd5XeZtjm49G9y4EJ2W7xVhRVaQiXSTjI22vVJvefna 4gvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OqouuTZe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k9si5488934iow.63.2021.04.30.14.12.13 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:12:13 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=OqouuTZe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaQk-0005OG-Qi for patch@linaro.org; Fri, 30 Apr 2021 17:12:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53326) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmG-0004M7-8p for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:20 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:39544) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007is-5i for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:19 -0400 Received: by mail-pj1-x1031.google.com with SMTP id z6-20020a17090a1706b0290155e8a752d8so2349207pjd.4 for ; Fri, 30 Apr 2021 13:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4LOZdC1ItbQynZDOtyUtTS7T/flVKohWME+t/j4leKo=; b=OqouuTZeYzkZmv5DeNtksNqG4mz7iKqSX6Ko8pGmHfkOk0jEZ8Z5HdQQv/F8lixylw H6u79bTGyH1mEuew9lHwdfeiJfloFEwcHIeC8BC5mg7x8UMC+PiIIkQ3Pri652Fc4GH2 lb2XfnuMPnh0n5BE6A18t1JwHNsVRs1mFAhEeOiDlOlhhuv9N6Eo6vkk7QoT506zfYOO 21s+rcr8oz8HUNpcg7KNZdlQs6lssLO1Hs0j4YxEIlS4rVTMA+AR97UkmC2KClyzZ1mH 27SJ5BShemlJCE2XRe2Ratql3rnKaJkFj9aeU6Ts9ZulkbwSM1WjZdcX/0UBDZ6h+Cqm hnOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4LOZdC1ItbQynZDOtyUtTS7T/flVKohWME+t/j4leKo=; b=DW7cg1Tsk2jfyZ3AlN3sEK/YEhKOLRCd9dNdMfXwKXXK3MhPz+YOXmFZtQmp30MV53 aYVPoPP//wx+w2Kp3u8C9w+e75kJH1jcKUOcAoZmq0hMccb7mXdtY8kkh1oQBa81Fr74 JPfg6PnngsuWHiU23nOwiseqcyIqAKVydwZlnLW10E2c8lW666FZZycWEJkiUKpdJ2AG HfsC6HAp05L7Di0nPd5L5ogaMtB0WDi8dJGTwlWN4/lnmQ+85ywKbUqdLKcLW31ruMxa +1bk6K6uT+Zw99goEyhzGVmVnJI16u65dw1nRfQQoJty2tuoj3PweCFdy8ZhtG4EIVjZ ilRA== X-Gm-Message-State: AOAM5316RzQ38qrTQSOFcrYYuWWWU9ceIsRBZdE1DjHDDi4XJIHs29KV 3GbQKtJt8AzJ4GFKTghM0ywC6VN9GUAaVA== X-Received: by 2002:a17:90a:f491:: with SMTP id bx17mr15463073pjb.176.1619814584204; Fri, 30 Apr 2021 13:29:44 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 53/82] target/arm: Implement SVE2 integer multiply-add (indexed) Date: Fri, 30 Apr 2021 13:25:41 -0700 Message-Id: <20210430202610.1136687-54-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 8 ++++++++ target/arm/translate-sve.c | 23 +++++++++++++++++++++++ 2 files changed, 31 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 65cb0a2206..9bfaf737b7 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -783,6 +783,14 @@ SDOT_zzxw_d 01000100 11 1 ..... 000000 ..... ..... @rrxr_1 esz=3 UDOT_zzxw_s 01000100 10 1 ..... 000001 ..... ..... @rrxr_2 esz=2 UDOT_zzxw_d 01000100 11 1 ..... 000001 ..... ..... @rrxr_1 esz=3 +# SVE2 integer multiply-add (indexed) +MLA_zzxz_h 01000100 0. 1 ..... 000010 ..... ..... @rrxr_3 esz=1 +MLA_zzxz_s 01000100 10 1 ..... 000010 ..... ..... @rrxr_2 esz=2 +MLA_zzxz_d 01000100 11 1 ..... 000010 ..... ..... @rrxr_1 esz=3 +MLS_zzxz_h 01000100 0. 1 ..... 000011 ..... ..... @rrxr_3 esz=1 +MLS_zzxz_s 01000100 10 1 ..... 000011 ..... ..... @rrxr_2 esz=2 +MLS_zzxz_d 01000100 11 1 ..... 000011 ..... ..... @rrxr_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 3f54636e11..4fdec08f46 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3866,6 +3866,29 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zzxz_ool(s, a, fn); +} + +#define DO_SVE2_RRXR(NAME, FUNC) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_ool(s, a, FUNC); } + +DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) +DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) +DO_SVE2_RRXR(trans_MLA_zzxz_d, gen_helper_gvec_mla_idx_d) + +DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) +DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) +DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) + +#undef DO_SVE2_RRXR + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 30 20:25:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429904 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp809083jao; Fri, 30 Apr 2021 14:28:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwiz6BP22WhjfgP8q5ZNiSlZiuGWPQ+xYiz7juIMjlqkFgvX7E38tILvzvPeArj6H2lfuzR X-Received: by 2002:a05:6e02:12c9:: with SMTP id i9mr5870881ilm.276.1619818103463; Fri, 30 Apr 2021 14:28:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818103; cv=none; d=google.com; s=arc-20160816; b=NQNj8t4Wz88ryvk9RpYZaDASACWnQTC6wDWXi0JpU04ppTa0lGw8tTjC17vWgYLtze sjVxQ593mSj0Xa5tFx/RJPjDoFsnVIZZX4zb5Jlsi+f6f69pUFeG/Q3RXn1g5Gugd70c stabHVeziw5hKFXPJF6A2p3dEy3ff//+FZsFlRwzwwZuntPY1JY74XpFM9beLKO9OnPP Mt02ZvriHTb38HnZfqLJWy1KJiLRQ+pMI9FUMhcXa845KLO5a9Va0KzAF1akjOLwN5mk RYiT1++WFDwiRJmF0PzkHW0onrU/L01ySJZgoJc/IDU2C17PaRsOY8UhYT0ALYEWbagI pWCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1m4KD6Jjgt61SLAO+2TZcBYwYJOymz2qy3ZggTsf0vg=; b=Gk5gK2GqbsrJigtrUZ8uKJ8Il2d/Gkf830hH68brzxcSgKi0n8EqJuoANGXU6LvElP zdW/8n80PmpRqaURsOHtsUHfkZxQKR/bniU+k8cK1Qz+GH4UjMZhhwzNn9nQKlWi3wpz ZfTbXJKExv9++T+XDqC5+fLh3wU6dKdDqoGpM1TTltt+wyQdBtZZq/9h7HL5BDC7nTBY Rp/YpLqT/ko/dXeHL+k2Xxz9PdENnZBHA5A/QZjNqipHeKIVQwdYHBPWiKugnStfOxMr PXTA0jDyD7E0teJgoPGpdVbSC63aXVaYDDkH+sSCzyGuc7YKpVsCYa9nFvFwdSdISlZh udwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="xPYZFV/t"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k7si3352445ili.130.2021.04.30.14.28.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:28:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="xPYZFV/t"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:34176 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcagQ-000761-SL for patch@linaro.org; Fri, 30 Apr 2021 17:28:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmV-0004jY-8k for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:35 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:35674) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007jN-EO for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:34 -0400 Received: by mail-pg1-x535.google.com with SMTP id q10so50175856pgj.2 for ; Fri, 30 Apr 2021 13:29:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1m4KD6Jjgt61SLAO+2TZcBYwYJOymz2qy3ZggTsf0vg=; b=xPYZFV/tybMZlLV13BVsuTxoxUc2SR6uUZ2OHvY+2NBKo6/sgrI2+5NJXHfWau/JE8 cM6vtI9cRnsuEnXPayE1ph9zL+bziSTV4y30lkikeZEtXVRvoPCeDiVnM+w4TILFvmW5 QxODNp4KOI9EYDWP2I4GB7PdVGwV8T3lyKkcNb/gidtuexlLNSPtobZWnG0/TP22JlAc VTJTvXn2/hjz5VSABqGUMdzckwLwcTZvIqCOzZN4Kpd66RrH6wFYULMmrIFWdRoP9jT/ LNqeyXbX5oSpVr/PV8pf4VdKqeSWGfJWmVhyFFPbH0NUT9pYoTrZh3xBBlBQUrRFlYFF ejPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1m4KD6Jjgt61SLAO+2TZcBYwYJOymz2qy3ZggTsf0vg=; b=Bzv1NsX3Pxrx45xfkmOLpu15vmrvmimesUMen4m4wZUU+K2G1KFxFVh+kmeWoNXhKP GvkNjB1ZdDu9Cq8WGimiG1pRnuwqixEfBgxe2jqi2q6BasxDVrryGjofH0MRCAi5y/dO iS0Dv3Hc17oUhupDMFXDstHvLUm3we211eXMoaPvcBvH3iciJpCRd6/QR8iJeIWNxO9n HFOzmlo80e27fmuaj/ZQreYtSsyp6062OzGsYSt3ZO2JnS+ANNG+9y2EnlDFF9viaIjN DXkUZeB+GDvjXm+83rDz5WP3ojyEABgKWk9onAvaIarrWl6aFYDVp05fUTJVR1iKsw/e 5fzw== X-Gm-Message-State: AOAM530WQ2LFDjb7TnyYdbVJPLYlE4oj/bXaaklzYr8lPAdjyZchqTtK GXFtCeVvqDRx9s6zCtd2tRLq+W205GEh3A== X-Received: by 2002:a62:1d8a:0:b029:24c:4aa1:ad01 with SMTP id d132-20020a621d8a0000b029024c4aa1ad01mr6385504pfd.27.1619814584739; Fri, 30 Apr 2021 13:29:44 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 54/82] target/arm: Implement SVE2 saturating multiply-add high (indexed) Date: Fri, 30 Apr 2021 13:25:42 -0700 Message-Id: <20210430202610.1136687-55-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 14 +++++++++++++ target/arm/sve.decode | 8 ++++++++ target/arm/sve_helper.c | 40 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 8 ++++++++ 4 files changed, 70 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 7e99dcd119..fe67574741 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2665,3 +2665,17 @@ DEF_HELPER_FLAGS_5(sve2_sqrdcmlah_zzzz_d, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_6(fmmla_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_6(fmmla_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlah_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 9bfaf737b7..1956d96ad5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -791,6 +791,14 @@ MLS_zzxz_h 01000100 0. 1 ..... 000011 ..... ..... @rrxr_3 esz=1 MLS_zzxz_s 01000100 10 1 ..... 000011 ..... ..... @rrxr_2 esz=2 MLS_zzxz_d 01000100 11 1 ..... 000011 ..... ..... @rrxr_1 esz=3 +# SVE2 saturating multiply-add high (indexed) +SQRDMLAH_zzxz_h 01000100 0. 1 ..... 000100 ..... ..... @rrxr_3 esz=1 +SQRDMLAH_zzxz_s 01000100 10 1 ..... 000100 ..... ..... @rrxr_2 esz=2 +SQRDMLAH_zzxz_d 01000100 11 1 ..... 000100 ..... ..... @rrxr_1 esz=3 +SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 +SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 +SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index f285c90b70..fc4a943029 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1487,9 +1487,49 @@ DO_CMLA(sve2_sqrdcmlah_zzzz_h, int16_t, H2, DO_SQRDMLAH_H) DO_CMLA(sve2_sqrdcmlah_zzzz_s, int32_t, H4, DO_SQRDMLAH_S) DO_CMLA(sve2_sqrdcmlah_zzzz_d, int64_t, , DO_SQRDMLAH_D) +#undef DO_SQRDMLAH_B +#undef DO_SQRDMLAH_H +#undef DO_SQRDMLAH_S +#undef DO_SQRDMLAH_D #undef do_cmla #undef DO_CMLA +#define DO_ZZXZ(NAME, TYPE, H, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t oprsz = simd_oprsz(desc), segment = 16 / sizeof(TYPE); \ + intptr_t i, j, idx = simd_data(desc); \ + TYPE *d = vd, *a = va, *n = vn, *m = (TYPE *)vm + H(idx); \ + for (i = 0; i < oprsz / sizeof(TYPE); i += segment) { \ + TYPE mm = m[i]; \ + for (j = 0; j < segment; j++) { \ + d[i + j] = OP(n[i + j], mm, a[i + j]); \ + } \ + } \ +} + +#define DO_SQRDMLAH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, false, true, &discard); }) +#define DO_SQRDMLAH_D(N, M, A) do_sqrdmlah_d(N, M, A, false, true) + +DO_ZZXZ(sve2_sqrdmlah_idx_h, int16_t, H2, DO_SQRDMLAH_H) +DO_ZZXZ(sve2_sqrdmlah_idx_s, int32_t, H4, DO_SQRDMLAH_S) +DO_ZZXZ(sve2_sqrdmlah_idx_d, int64_t, , DO_SQRDMLAH_D) + +#define DO_SQRDMLSH_H(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_h(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_S(N, M, A) \ + ({ uint32_t discard; do_sqrdmlah_s(N, M, A, true, true, &discard); }) +#define DO_SQRDMLSH_D(N, M, A) do_sqrdmlah_d(N, M, A, true, true) + +DO_ZZXZ(sve2_sqrdmlsh_idx_h, int16_t, H2, DO_SQRDMLSH_H) +DO_ZZXZ(sve2_sqrdmlsh_idx_s, int32_t, H4, DO_SQRDMLSH_S) +DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) + +#undef DO_ZZXZ + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 4fdec08f46..d9d444d3ca 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3887,6 +3887,14 @@ DO_SVE2_RRXR(trans_MLS_zzxz_h, gen_helper_gvec_mls_idx_h) DO_SVE2_RRXR(trans_MLS_zzxz_s, gen_helper_gvec_mls_idx_s) DO_SVE2_RRXR(trans_MLS_zzxz_d, gen_helper_gvec_mls_idx_d) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_h, gen_helper_sve2_sqrdmlah_idx_h) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_s, gen_helper_sve2_sqrdmlah_idx_s) +DO_SVE2_RRXR(trans_SQRDMLAH_zzxz_d, gen_helper_sve2_sqrdmlah_idx_d) + +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_h, gen_helper_sve2_sqrdmlsh_idx_h) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_s, gen_helper_sve2_sqrdmlsh_idx_s) +DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) + #undef DO_SVE2_RRXR /* From patchwork Fri Apr 30 20:25:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429901 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp806989jao; Fri, 30 Apr 2021 14:24:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqQyz+VVcxTEfdRSTKkBERo1CR/DuS2CNBFcUWYOPXBmhDDxcFUqZruIGx9zWWnE+vEJWM X-Received: by 2002:a05:6e02:1a61:: with SMTP id w1mr6360505ilv.304.1619817888209; Fri, 30 Apr 2021 14:24:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817888; cv=none; d=google.com; s=arc-20160816; b=H23Bw1n6lF/X8afVMu9BU62qzEN2msGyeQ0HZGOs9yNbKr4EvFP7zY7dd9kL91g19F fVuWHQyv6nH4Ng4hAr0bEHYSgsRjmSVm/FOubEavT6DsOR2Nr55hFWnbtd8ZGyDXVQ2G fYWUUvwUEi2faIXRKd35nc1PV14bRdooujm//muK7gv592Yb+3CwYbmf41tFP0/Qbqxm amo+X0TotHfBgmXFa9ShvbwjL5AtGRnFkB5un4IeaJFYe6+N4B05ZPqUlGBnoAN3Ns+e NCChbwX3pWKYdcv4Kd3SbYbtQuZOPJn7YAeelEpFFmccfGw+YGmH4jqTpvb0HBqwkV8P rb/Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FrURjPokXJn/LkVchLyJhBRsfW1xgk5qHq2OmKbnw0s=; b=KeoPILxKMAWprsPwuufKPGsrVVdDGTlyrO9xWr6oItn3UfZwFb8GHL4zMVBM3c+lEN o9rK9Q/OJnYY20Ya1w88utEvFFpeBoSzX+F4TEUGxXm22bk5ocKp/mqhopNRGgd6yeD9 yEStEoD3CLC+VhQ9s6gZh+l1U+4DSO7FmInyMgkZNqPNtWbSkwtUGd2sBEFhF57xZAXo zj8JzS1ucVcuwAezIm/nrzXiovGoCjePW1t0z9nBo9+mmoJlNGySFPU5bxKv7cEYFSmh XamF+JLwiinBQ2krcv8eKH5TlrKXOPU0kV8r3ebUYYOvDkPYQq+8gVJGXrVM9/sXYQqO N1Iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="f9Kf0r/w"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d25si4401490ioh.30.2021.04.30.14.24.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:24:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="f9Kf0r/w"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:53526 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcacw-0003Qs-Ub for patch@linaro.org; Fri, 30 Apr 2021 17:24:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53470) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmP-0004c2-1K for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:32 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]:42739) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007jY-D2 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:28 -0400 Received: by mail-pj1-x102a.google.com with SMTP id j6-20020a17090adc86b02900cbfe6f2c96so2382780pjv.1 for ; Fri, 30 Apr 2021 13:29:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FrURjPokXJn/LkVchLyJhBRsfW1xgk5qHq2OmKbnw0s=; b=f9Kf0r/wjJqlxw7JoVwHti6r8rOrzc/Kq43XNSKoqA5CK3AYc9hsB5kowpHVtrrVfQ uVYThXnZfV1DY97K2kaAiW9MENeyyNA7jLVRscVypq+YtaYyuBZj1NeYkKZS4Rld6XnV RI5jxrNoiWKr9nAQr7MmHLW+OgFsbN/7MSaXoYyL1GskQDNnkMyVDaypiwdaxMo2y/IZ 6UtUPtJP+TCzVeAjHHi5aFpFdD3mPa8E5qTKS/aF2WV9xzg5taj1+pUCH2lxtF4cHnep VDm+Lh+yjfOwVXwoNGNY163KhZtLeoGcsP2NtkFU8HCSKPviVNpYF2BW12+kkDBGTwIf +0eQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FrURjPokXJn/LkVchLyJhBRsfW1xgk5qHq2OmKbnw0s=; b=fb0W8KH+sUZIUN1wK91sSCCWppqNLaXcie//qwKwgRoEDj1ix6rxK+3rF+i2QZhkRg +72dLmmHR95GXxRbwYmjpN5rP8QnukQJHxtNSVbZHHTI8p69hjOQOuYpvA2YmW5KiIrb zw9stYYAx5EMPDrzexA65rF2YEhoi6Kfs2M+7TH5Abv6DBjWv00n+qTuMXjgQXUObnLv O7YkLaks9A56gpPRgoLL+YPPXloMh2rl3wv4SnePbwu20mOdW8j73dm9U4DLK+cCS6l7 nhXB4/6Aed1hEwq9d3Tj52mEHXwanQmMdehquedYLLlOJ89EXwhb1A9fpamy5g/23wjE 5qjg== X-Gm-Message-State: AOAM533SA0n9/JNus2mhLpQJ8q4DMHiYxxMc6ybVB7jqEdaOU44/23Ld wQ3sNgdP2GK+5b2oaA6+ya59KzxlzKEEuQ== X-Received: by 2002:a17:90a:8410:: with SMTP id j16mr7044250pjn.120.1619814585348; Fri, 30 Apr 2021 13:29:45 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 55/82] target/arm: Implement SVE2 saturating multiply-add (indexed) Date: Fri, 30 Apr 2021 13:25:43 -0700 Message-Id: <20210430202610.1136687-56-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 9 +++++++++ target/arm/sve.decode | 18 ++++++++++++++++++ target/arm/sve_helper.c | 30 ++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 32 ++++++++++++++++++++++++-------- 4 files changed, 81 insertions(+), 8 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index fe67574741..08398800bd 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2679,3 +2679,12 @@ DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqrdmlsh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlal_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 1956d96ad5..8d2709d3cc 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -30,6 +30,8 @@ %size_23 23:2 %dtype_23_13 23:2 13:2 %index3_22_19 22:1 19:2 +%index3_19_11 19:2 11:1 +%index2_20_11 20:1 11:1 # A combination of tsz:imm3 -- extract esize. %tszimm_esz 22:2 5:5 !function=tszimm_esz @@ -261,6 +263,12 @@ @rrxr_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx +# Three registers and a scalar by N-bit index, alternate +@rrxr_3a ........ .. ... rm:3 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index3_19_11 +@rrxr_2a ........ .. .. rm:4 ...... rn:5 rd:5 \ + &rrxr_esz ra=%reg_movprfx index=%index2_20_11 + ########################################################################### # Instruction patterns. Grouped according to the SVE encodingindex.xhtml. @@ -799,6 +807,16 @@ SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 +# SVE2 saturating multiply-add (indexed) +SQDMLALB_zzxw_s 01000100 10 1 ..... 0010.0 ..... ..... @rrxr_3a esz=2 +SQDMLALB_zzxw_d 01000100 11 1 ..... 0010.0 ..... ..... @rrxr_2a esz=3 +SQDMLALT_zzxw_s 01000100 10 1 ..... 0010.1 ..... ..... @rrxr_3a esz=2 +SQDMLALT_zzxw_d 01000100 11 1 ..... 0010.1 ..... ..... @rrxr_2a esz=3 +SQDMLSLB_zzxw_s 01000100 10 1 ..... 0011.0 ..... ..... @rrxr_3a esz=2 +SQDMLSLB_zzxw_d 01000100 11 1 ..... 0011.0 ..... ..... @rrxr_2a esz=3 +SQDMLSLT_zzxw_s 01000100 10 1 ..... 0011.1 ..... ..... @rrxr_3a esz=2 +SQDMLSLT_zzxw_d 01000100 11 1 ..... 0011.1 ..... ..... @rrxr_2a esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fc4a943029..c43c38044b 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1530,6 +1530,36 @@ DO_ZZXZ(sve2_sqrdmlsh_idx_d, int64_t, , DO_SQRDMLSH_D) #undef DO_ZZXZ +#define DO_ZZXW(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + TYPEW aa = *(TYPEW *)(va + HW(i + j)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm, aa); \ + } \ + } \ +} + +#define DO_SQDMLAL_S(N, M, A) DO_SQADD_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLAL_D(N, M, A) do_sqadd_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlal_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLAL_S) +DO_ZZXW(sve2_sqdmlal_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLAL_D) + +#define DO_SQDMLSL_S(N, M, A) DO_SQSUB_S(A, do_sqdmull_s(N, M)) +#define DO_SQDMLSL_D(N, M, A) do_sqsub_d(A, do_sqdmull_d(N, M)) + +DO_ZZXW(sve2_sqdmlsl_idx_s, int32_t, int16_t, H1_4, H1_2, DO_SQDMLSL_S) +DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) + +#undef DO_ZZXW + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index d9d444d3ca..1915f68af6 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3817,21 +3817,21 @@ static bool trans_DOT_zzzz(DisasContext *s, arg_DOT_zzzz *a) * SVE Multiply - Indexed */ -static bool do_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (fn == NULL) { return false; } if (sve_access_check(s)) { - gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, a->index); + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); } return true; } #define DO_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_zzxz_ool(s, a, FUNC); } + { return do_zzxz_data(s, a, FUNC, a->index); } DO_RRXR(trans_SDOT_zzxw_s, gen_helper_gvec_sdot_idx_b) DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) @@ -3866,18 +3866,18 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX -static bool do_sve2_zzxz_ool(DisasContext *s, arg_rrxr_esz *a, - gen_helper_gvec_4 *fn) +static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, + gen_helper_gvec_4 *fn, int data) { if (!dc_isar_feature(aa64_sve2, s)) { return false; } - return do_zzxz_ool(s, a, fn); + return do_zzxz_data(s, a, fn, data); } #define DO_SVE2_RRXR(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ - { return do_sve2_zzxz_ool(s, a, FUNC); } + { return do_sve2_zzxz_data(s, a, FUNC, a->index); } DO_SVE2_RRXR(trans_MLA_zzxz_h, gen_helper_gvec_mla_idx_h) DO_SVE2_RRXR(trans_MLA_zzxz_s, gen_helper_gvec_mla_idx_s) @@ -3897,6 +3897,22 @@ DO_SVE2_RRXR(trans_SQRDMLSH_zzxz_d, gen_helper_sve2_sqrdmlsh_idx_d) #undef DO_SVE2_RRXR +#define DO_SVE2_RRXR_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrxr_esz *a) \ + { return do_sve2_zzxz_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLALB_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_s, gen_helper_sve2_sqdmlal_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLALT_zzxw_d, gen_helper_sve2_sqdmlal_idx_d, true) + +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLB_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, false) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_s, gen_helper_sve2_sqdmlsl_idx_s, true) +DO_SVE2_RRXR_TB(trans_SQDMLSLT_zzxw_d, gen_helper_sve2_sqdmlsl_idx_d, true) + +#undef DO_SVE2_RRXR_TB + /* *** SVE Floating Point Multiply-Add Indexed Group */ From patchwork Fri Apr 30 20:25:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429899 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp805460jao; Fri, 30 Apr 2021 14:22:16 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaSIOARZ8Xij6e1XhfzL3l5LTwcRyTDPwXnpr/cFPiujzTrzHxXyJBdxfG5j3KrXiPKCL0 X-Received: by 2002:a6b:6013:: with SMTP id r19mr5466879iog.208.1619817736598; Fri, 30 Apr 2021 14:22:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817736; cv=none; d=google.com; s=arc-20160816; b=L6cBApCk8pU0Bk/Pt7yGp8sPrJauswtghznqtwjb4sHeSghJEzYBTeyqaKF/SYLJtq 00RtF8AGMJbKSprThMDiWfqHCtjNDkXunoUL/cTGLVnCK+YLqwM/1DYEtMcBDKALXbRy MdP587pDRXjtUTM18jT7m2M3/paLV5vCVn87W9syMMo1tzNPkZvaRL0LSTx9wh1iqazI +xFN/x4t7VGNu49/xq2DnpPNFc6gNXIVd1Fqkq8+CMCqhnnuzG3ctxUESzNnN/7+f8w6 R2kLc/Bfhg3dXBYdtK9QSz2hnUqrx3zRTyWs7/mfeHRGOvAcmNBIaoRcuJqUo3IZSxv5 Elxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=1gr+w+D07DwVhmgy58KbkyZwXkW20B50hh9NUpReRmk=; b=Xm/ThnlOrMggvyturhu7v8AbD53Zp2eA9EQbjNevCmZifn5mSZ6xsu3gr4rbcYGBsI CkAELQbG7iw81ZHgPY60IITM7v0AMmGGtSD77EOg6TDMJxTSoS14gn/wnjpaqpRtKTRT lLeWGMLo6LcysUtFdIg/z0UmCu+F1Jlzn/+xvtWFZLMnW3DyzpaXvjm9bVQpT/yA8Y0L TINg8PL3UVXkBJTLmxyLSUHTCaruo2UjM4SDgUVw2APH7iEMedhKQZl4SQGuW64yKo6N meo4LUFCVRD+CbN0pCdOgzFkFnRnpZwklyw22nBtJ0rBiVFs8N9P7w6TRa3xb3YqChQI VE1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="AFO/XDJ/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v11si4252173ilu.142.2021.04.30.14.22.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:22:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="AFO/XDJ/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44912 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaaV-0008Nj-MA for patch@linaro.org; Fri, 30 Apr 2021 17:22:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53458) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmN-0004YM-Dk for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:27 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]:40777) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007jm-Al for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:27 -0400 Received: by mail-pl1-x62f.google.com with SMTP id 20so33287782pll.7 for ; Fri, 30 Apr 2021 13:29:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1gr+w+D07DwVhmgy58KbkyZwXkW20B50hh9NUpReRmk=; b=AFO/XDJ/e7gTveRun3YiV10LF5JCucqDYRTyWNqdAJgyHMGFp2g6HKrkfWcKA9iUA8 oU6SHUpFsrRoeWd7gCOUPpXpv3Qxcb0wqkOju7F8KTJWbYCCy/AAwSCnc1XUCriIWnhB UA+3fwe9JnjCvrnkIWQyvHt/HBITnHWhFVJNQZjONHEFcN2vBnUoxf/rNiIUBpCJBm5c /i6fIYj9XcHB8GLNxbFYeiyXivFt7II0+acugPN+wF58MUnjK6v6TsOAnXuuPNPqyvNr xUFIYesrHFvEeWUmZttw/pmv6fKqQOa/4oek+4RgXNaehqad1eoHVuJmkywn5UeypVNI Uq2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1gr+w+D07DwVhmgy58KbkyZwXkW20B50hh9NUpReRmk=; b=llnNqmWzHEYRMxuLg1ztwP+zu8+wkvHXUPKtEjggNM2A/mUaut2Fu9KFRgpiN+81vT aPimC2o/hMc4D6/dAK/P7hKpDPMQnxpDnqeJ9vXVt7NhsX51PLl6FEwSh+5QX93E2SmP dyb9K4XxutlhWjv8fMlVEb6zlRXV5cVUgJSs/HcTyLY+TE6OTkP0/fvDUnjaksUNnDdj kDI9x4oR/WFhIF2Y+Q9msfXC2P8x1fOjNQUUbD5dnht1eHdJuEUXH7UyYREV7lGXHKYB w/fcR5dGl4vPPVT3ymAViQMCgVwmWSYTcb1B8uWn0kuzkp4vNvV9rnv8uH9eAtmYUo74 Zp/g== X-Gm-Message-State: AOAM531TW5wofVVe17ylYrAB2nc3CBrVDe740nAEKXHAn1rX0Nb618LV oRHCx+dIMFztjkBCoBpbQd0fkNNytqzpQQ== X-Received: by 2002:a17:90a:4748:: with SMTP id y8mr6903832pjg.31.1619814585962; Fri, 30 Apr 2021 13:29:45 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 56/82] target/arm: Implement SVE2 saturating multiply (indexed) Date: Fri, 30 Apr 2021 13:25:44 -0700 Message-Id: <20210430202610.1136687-57-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 12 ++++++++++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 19 +++++++++++++++---- 4 files changed, 52 insertions(+), 4 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 08398800bd..0be0d90bee 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2688,3 +2688,8 @@ DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_sqdmlsl_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 8d2709d3cc..a3b9fb95f9 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -255,6 +255,12 @@ @rrx_2 ........ .. . index:2 rm:3 ...... rn:5 rd:5 &rrx_esz @rrx_1 ........ .. . index:1 rm:4 ...... rn:5 rd:5 &rrx_esz +# Two registers and a scalar by N-bit index, alternate +@rrx_3a ........ .. . .. rm:3 ...... rn:5 rd:5 \ + &rrx_esz index=%index3_19_11 +@rrx_2a ........ .. . . rm:4 ...... rn:5 rd:5 \ + &rrx_esz index=%index2_20_11 + # Three registers and a scalar by N-bit index @rrxr_3 ........ .. . .. rm:3 ...... rn:5 rd:5 \ &rrxr_esz ra=%reg_movprfx index=%index3_22_19 @@ -817,6 +823,12 @@ SQDMLSLB_zzxw_d 01000100 11 1 ..... 0011.0 ..... ..... @rrxr_2a esz=3 SQDMLSLT_zzxw_s 01000100 10 1 ..... 0011.1 ..... ..... @rrxr_3a esz=2 SQDMLSLT_zzxw_d 01000100 11 1 ..... 0011.1 ..... ..... @rrxr_2a esz=3 +# SVE2 saturating multiply (indexed) +SQDMULLB_zzx_s 01000100 10 1 ..... 1110.0 ..... ..... @rrx_3a esz=2 +SQDMULLB_zzx_d 01000100 11 1 ..... 1110.0 ..... ..... @rrx_2a esz=3 +SQDMULLT_zzx_s 01000100 10 1 ..... 1110.1 ..... ..... @rrx_3a esz=2 +SQDMULLT_zzx_d 01000100 11 1 ..... 1110.1 ..... ..... @rrx_2a esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index c43c38044b..e8a8425522 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -1560,6 +1560,26 @@ DO_ZZXW(sve2_sqdmlsl_idx_d, int64_t, int32_t, , H1_4, DO_SQDMLSL_D) #undef DO_ZZXW +#define DO_ZZX(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + intptr_t i, j, oprsz = simd_oprsz(desc); \ + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT, 1) * sizeof(TYPEN); \ + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 1, 3) * sizeof(TYPEN); \ + for (i = 0; i < oprsz; i += 16) { \ + TYPEW mm = *(TYPEN *)(vm + i + idx); \ + for (j = 0; j < 16; j += sizeof(TYPEW)) { \ + TYPEW nn = *(TYPEN *)(vn + HN(i + j + sel)); \ + *(TYPEW *)(vd + HW(i + j)) = OP(nn, mm); \ + } \ + } \ +} + +DO_ZZX(sve2_sqdmull_idx_s, int32_t, int16_t, H1_4, H1_2, do_sqdmull_s) +DO_ZZX(sve2_sqdmull_idx_d, int64_t, int32_t, , H1_4, do_sqdmull_d) + +#undef DO_ZZX + #define DO_BITPERM(NAME, TYPE, OP) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1915f68af6..acf16c4c43 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3840,8 +3840,8 @@ DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) #undef DO_RRXR -static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, - gen_helper_gvec_3 *fn) +static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, + gen_helper_gvec_3 *fn, int data) { if (fn == NULL || !dc_isar_feature(aa64_sve2, s)) { return false; @@ -3851,14 +3851,14 @@ static bool do_sve2_zzx_ool(DisasContext *s, arg_rrx_esz *a, tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), vec_full_reg_offset(s, a->rm), - vsz, vsz, a->index, fn); + vsz, vsz, data, fn); } return true; } #define DO_SVE2_RRX(NAME, FUNC) \ static bool NAME(DisasContext *s, arg_rrx_esz *a) \ - { return do_sve2_zzx_ool(s, a, FUNC); } + { return do_sve2_zzx_data(s, a, FUNC, a->index); } DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) @@ -3866,6 +3866,17 @@ DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) #undef DO_SVE2_RRX +#define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ + static bool NAME(DisasContext *s, arg_rrx_esz *a) \ + { return do_sve2_zzx_data(s, a, FUNC, (a->index << 1) | TOP); } + +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_s, gen_helper_sve2_sqdmull_idx_s, false) +DO_SVE2_RRX_TB(trans_SQDMULLB_zzx_d, gen_helper_sve2_sqdmull_idx_d, false) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_s, gen_helper_sve2_sqdmull_idx_s, true) +DO_SVE2_RRX_TB(trans_SQDMULLT_zzx_d, gen_helper_sve2_sqdmull_idx_d, true) + +#undef DO_SVE2_RRX_TB + static bool do_sve2_zzxz_data(DisasContext *s, arg_rrxr_esz *a, gen_helper_gvec_4 *fn, int data) { From patchwork Fri Apr 30 20:25:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429895 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp803891jao; Fri, 30 Apr 2021 14:20:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwWXmPXKo8dxnZyli5lT198b2I2AN6IkOa6/yCqmHA59QcKs78AcqT5LQUJe/w4pgLLAF6M X-Received: by 2002:a05:6638:3048:: with SMTP id u8mr6712534jak.91.1619817600765; Fri, 30 Apr 2021 14:20:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817600; cv=none; d=google.com; s=arc-20160816; b=leIyb3T4sUQNxuo3GT5fAjMv3IJrXH60A9IFygfcfrEXhRR/U7EJPPIgr0wHTQ3Awx X18B2M15rHyM1df0z6PS9T+EJZS+kS1nlRl6WSbVowO0BJ7kXgzsuYIggsucNMVUc/FX hp7YHBEDVftJZ+WPz6Wy4q4Arx683GpERakUKMul7omWOYMliGKRd1Ch4Efw2CN10sN6 GZ/uQ7Uz+ihqAWDCQS2iZgc9M7+fePJVE9OV0chuaHUR9+Qa/p3aHV2xWkaZfICA6/k8 v5sABzuz7mOGqRLA1ND9vGBYRmEMs4JS10Mw9+U3ZQnzrFE/WwRPiHo3VcWG2L3FwqpI 3pvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=I4fHMYBO4ihQ+/Yw2g+7CZYltNBdKISEE3KJi6AN1fs=; b=NI+U8sVrJTNqaM5OGj3R5nBAaZqdbroVxZgDixrvFPCww99jo4qu1ecglQIWYOT/Ee Jq56/juZc2BqySpF8gSUOsw0P73fah7vP2M09Uf+DaP10cEJaC088vr3tPMs0tbBFadG X2AUiGW9cPrvJmw4NZAJNaDbRcGz+P82atilzM7bzf9DBrHWypLSrHXDeKs4YQPTvptf PImcHNBYs9kzOETlTl+pQKJZ2uM/1+m6nqVLuQWF57Wp7+lzSGmJpK0CIYoqaqWC6vzX G6U302+Rtmc684+6jXM4hsIcWZJnvBjyrntJ+87Zd5W15GrlJ7oNvrg5nVu0dMOUDD9l HZCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="aYR+Bf/4"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t10si4200814ilu.91.2021.04.30.14.20.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:20:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="aYR+Bf/4"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36230 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaYK-0004j0-3J for patch@linaro.org; Fri, 30 Apr 2021 17:20:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53380) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmI-0004RH-Vw for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:23 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]:40645) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007jz-7k for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:22 -0400 Received: by mail-pg1-x52a.google.com with SMTP id y30so5015145pgl.7 for ; Fri, 30 Apr 2021 13:29:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I4fHMYBO4ihQ+/Yw2g+7CZYltNBdKISEE3KJi6AN1fs=; b=aYR+Bf/4Xy2a4VMMkTtg/PQWDEK+5BWGPx1BJOD3mMCH7wTnddeXUD7g3gbggVyXFR nQEIxIdivC0NPJwUOeAiamjKXbJT3l903LMr3DRtEVi83YIOl3crCXAdqY4uGqA5uIP3 t44Xcz7RVL+sH9CNtnaf9FFbl2ss4d7XayoPllTTJFerb2r3X5SY384XUbEeZYyR6Vdd 83GlJQp2+ylvkHEfDIPjecC8EGYm7H5Pz86ce3VPyiuqwRTH7LwA5nemjO25JI+ocoTM u3AgkiYHR1eB91BJ9kuOZMoyzZoFukfPd1q8id//vllKfAbAWFLLE3zLQBBT2rUTo2HD fFAA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I4fHMYBO4ihQ+/Yw2g+7CZYltNBdKISEE3KJi6AN1fs=; b=LTKeQq89tKfOuZ6w0soMoXATnhdKlbc0az4qLAijG+//KCbRmcOWT+ExCxlU8RMRYI ZhK0Uglou1mRdeMjDDQ4QGBn5zkTl7MmtADV5dvfARPr8Ng94P/WPboJHCiFVTECZt7l 98q8hdiAq0aqXSSHP6/43f4oD/t8LNf9rc1m0pNuAW9Q7oW4AMnUC5oab2Q5KaxJ3Q7B z4xSs+D52d/TWMLS9Rpm1SE3QnRgsfKFXDQLPk1t2VUrZPXkLVwjPJvsX377z0e5xe8i Si8OoJcKegNGu4r8vyZdhE0CBi4ZkFyv9lCNPoOyMCClewk+YhmTZfG6uIOc02ABVr0o Dlng== X-Gm-Message-State: AOAM530JwZy71vz23EIob/8EazxbZ17o09GHWASnp5pK+P5zCakGOZqZ sT6L/V91x7Q+SHMMCiAcZdOBenZTHcZKJg== X-Received: by 2002:aa7:9a8d:0:b029:27c:a7c8:b7b0 with SMTP id w13-20020aa79a8d0000b029027ca7c8b7b0mr6425186pfi.17.1619814587087; Fri, 30 Apr 2021 13:29:47 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 57/82] target/arm: Implement SVE2 signed saturating doubling multiply high Date: Fri, 30 Apr 2021 13:25:45 -0700 Message-Id: <20210430202610.1136687-58-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 10 +++++ target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 18 ++++++++ target/arm/vec_helper.c | 84 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 116 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index 72c5bf6aca..eb94b6b1e6 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -957,6 +957,16 @@ DEF_HELPER_FLAGS_5(neon_sqrdmulh_h, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_5(neon_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index a3b9fb95f9..407d3019d1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1202,6 +1202,10 @@ SMULH_zzz 00000100 .. 1 ..... 0110 10 ..... ..... @rd_rn_rm UMULH_zzz 00000100 .. 1 ..... 0110 11 ..... ..... @rd_rn_rm PMUL_zzz 00000100 00 1 ..... 0110 01 ..... ..... @rd_rn_rm_e0 +# SVE2 signed saturating doubling multiply high (unpredicated) +SQDMULH_zzz 00000100 .. 1 ..... 0111 00 ..... ..... @rd_rn_rm +SQRDMULH_zzz 00000100 .. 1 ..... 0111 01 ..... ..... @rd_rn_rm + ### SVE2 Integer - Predicated SADALP_zpzz 01000100 .. 000 100 101 ... ..... ..... @rdm_pg_rn diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index acf16c4c43..e32094dcdb 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -6419,6 +6419,24 @@ static bool trans_PMUL_zzz(DisasContext *s, arg_rrr_esz *a) return do_sve2_zzz_ool(s, a, gen_helper_gvec_pmul_b); } +static bool trans_SQDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqdmulh_b, gen_helper_sve2_sqdmulh_h, + gen_helper_sve2_sqdmulh_s, gen_helper_sve2_sqdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + +static bool trans_SQRDMULH_zzz(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqrdmulh_b, gen_helper_sve2_sqrdmulh_h, + gen_helper_sve2_sqrdmulh_s, gen_helper_sve2_sqrdmulh_d, + }; + return do_sve2_zzz_ool(s, a, fns[a->esz]); +} + /* * SVE2 Integer - Predicated */ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index b19877e0d3..25061c15e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -81,6 +81,26 @@ void HELPER(sve2_sqrdmlsh_b)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int8_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz; ++i) { + d[i] = do_sqrdmlah_b(n[i], m[i], 0, false, true); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 16-bit */ int16_t do_sqrdmlah_h(int16_t src1, int16_t src2, int16_t src3, bool neg, bool round, uint32_t *sat) @@ -198,6 +218,28 @@ void HELPER(sve2_sqrdmlsh_h)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int16_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 2; ++i) { + d[i] = do_sqrdmlah_h(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -309,6 +351,28 @@ void HELPER(sve2_sqrdmlsh_s)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, false, &discard); + } +} + +void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *n = vn, *m = vm; + uint32_t discard; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = do_sqrdmlah_s(n[i], m[i], 0, false, true, &discard); + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -368,6 +432,26 @@ void HELPER(sve2_sqrdmlsh_d)(void *vd, void *vn, void *vm, } } +void HELPER(sve2_sqdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, false); + } +} + +void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int64_t *d = vd, *n = vn, *m = vm; + + for (i = 0; i < opr_sz / 8; ++i) { + d[i] = do_sqrdmlah_d(n[i], m[i], 0, false, true); + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 30 20:25:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429884 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp797215jao; Fri, 30 Apr 2021 14:09:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzjs9TPLQ3Dz3YnAm/BFxgr6VOpwnD6a9Q6GjqEdz3Gb//CBOu1gg9+je8LDoNTlONL2i6i X-Received: by 2002:a05:6638:dc2:: with SMTP id m2mr6820682jaj.123.1619816943461; Fri, 30 Apr 2021 14:09:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816943; cv=none; d=google.com; s=arc-20160816; b=hGlMTGAGpSQlWbB1jNmwELZHxjPTjRMegqp2kCwy8zqHYK4JjXTgic73On6M1dazKE ijzlgCa1nVvCdgi+rjzAeqxvhDJ+I92caft1QmOBIGsPldD3dAL8af2Lhjl0vPNTBame GLx2AdQiHYtQV8wU51EAVpdtdR5Jqdv08rQM8+geya7n09bwgXnyhqkmviJaZTxVId0g UhC6qGb0Xuajno1Z2CzwZ+V28GQFtX95v84VcPJAjJ6cFn4Zc6ci/B6W77PIX+7EwJZ0 pqfo/E/ajp09j2F6idPleLOy5W5XiVWKCRte/w0+pOL3TEj7eHk2NnunYtxz8u/mbWL/ 3l3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=76u00K3o67SzMGIaSyMgoPwtbQjy3qcI0996M+o1XzQ=; b=Y69KsFwHy2fRDGu4XUMbKzTyV5TSmIrVOlbJPP0sPSgIKkzkoUWcyhXhH3/eKlBN2J 312dcyXI+jYGmJPcs6uR//gmEqTu6Z4Vondmni0w9mylPGaSPi/riINS5B2QJwiPsSXb 00OZLvvgqeP4XlswneYEne58/VPaeXdnaw0noKCeqjKnwCAczi4Gb6v1TR0rU8YPs0Ge 4VoGLW7uh7oBY8S50jRjar7CEW8JrI/ebn5YNh9Z28oV9/f36i6DMVQpOip7C0A6RztI 3ugH/ylaDkXNWFeOtGZvKx4H9f1bnHb0SAhImdYYfUCRNive/ceT50K/NulkQrxKt1KY lXVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tByRmDz+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i10si3482242jak.88.2021.04.30.14.09.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:09:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tByRmDz+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:37344 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaNi-0001iL-Rm for patch@linaro.org; Fri, 30 Apr 2021 17:09:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53276) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmE-0004Ib-Iu for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:18 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]:37860) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007k9-2Y for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:18 -0400 Received: by mail-pf1-x435.google.com with SMTP id b15so5014176pfl.4 for ; Fri, 30 Apr 2021 13:29:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=76u00K3o67SzMGIaSyMgoPwtbQjy3qcI0996M+o1XzQ=; b=tByRmDz+5Bu0YH2BsAYc/OtZL/mwa1fr1q/sZ2V0ZaIZ5Hb2bNbSRdFNnrJTtgRY8P RweS9E46umLV9tzVdCqxoH7g8H7yhOdQnUXU/pfc98u00IAzlNafys5q6+isBHpo5yuW Hh+NA7qJbNA9YgkrzoaJix0vToKk0PpuI0c7IGxJ8fcgyOq8cfABF5VfyFLCJyRWNPaE AfLK9vJxlW7S0hDbY3nUiYENjQRSSQdxivLKbGo4SbUWaHrv8yp1+jHGnLIVmPcmR8bT S2zymvLMx7sUchPog36aAV2V9vga+Utj3lJtvdJCCG1QDoGmRnb2XzZXP8e8s8VBd9aj bW2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=76u00K3o67SzMGIaSyMgoPwtbQjy3qcI0996M+o1XzQ=; b=ZbaOeTT9Dkic4vhhavA47fVFEtYIag+SKijqPSrpib+KznOVUwLRHo2tHFZh4M5hXF gHQY8KhyLfWslnx1g1fBW251kMkjY3D8eWv0Y9FES4Vwu4lkUjcbu4YZqmY1CeYxBRPe 4z4Z62Hkgp6KlLrXqXQYcI0wEKwIi7DxDXLbaeqtTTZs0BMLn4fFuR3IkRrLU3fvrVhW v9YzYdNs0lenKyKHbmVBX9mrgn6L7DRsc8DtcdAweXoOGSL7kRqX19p2CP/R/XziOgD/ tbFN7K2K8Gs7GdYhch18+n1Oi+/q/E1Sorn0bHWqMX4bT11GcgIYatXrFwc6MmdD7FnN +JMQ== X-Gm-Message-State: AOAM530Sm/FK1aFV1Walc2U+Hwvd6qFVP0zfsydkIXkCHR3lt4Gzd7h7 S2icV/0nwddLHW8n+MALMa2c70JzXQSbKw== X-Received: by 2002:a05:6a00:138f:b029:272:931d:23bd with SMTP id t15-20020a056a00138fb0290272931d23bdmr6649284pfg.40.1619814587619; Fri, 30 Apr 2021 13:29:47 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 58/82] target/arm: Implement SVE2 saturating multiply high (indexed) Date: Fri, 30 Apr 2021 13:25:46 -0700 Message-Id: <20210430202610.1136687-59-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 14 ++++++ target/arm/sve.decode | 8 ++++ target/arm/translate-sve.c | 8 ++++ target/arm/vec_helper.c | 88 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 118 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index eb94b6b1e6..e7c463fff5 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -967,6 +967,20 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqrdmulh_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 407d3019d1..35010d755f 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -829,6 +829,14 @@ SQDMULLB_zzx_d 01000100 11 1 ..... 1110.0 ..... ..... @rrx_2a esz=3 SQDMULLT_zzx_s 01000100 10 1 ..... 1110.1 ..... ..... @rrx_3a esz=2 SQDMULLT_zzx_d 01000100 11 1 ..... 1110.1 ..... ..... @rrx_2a esz=3 +# SVE2 saturating multiply high (indexed) +SQDMULH_zzx_h 01000100 0. 1 ..... 111100 ..... ..... @rrx_3 esz=1 +SQDMULH_zzx_s 01000100 10 1 ..... 111100 ..... ..... @rrx_2 esz=2 +SQDMULH_zzx_d 01000100 11 1 ..... 111100 ..... ..... @rrx_1 esz=3 +SQRDMULH_zzx_h 01000100 0. 1 ..... 111101 ..... ..... @rrx_3 esz=1 +SQRDMULH_zzx_s 01000100 10 1 ..... 111101 ..... ..... @rrx_2 esz=2 +SQRDMULH_zzx_d 01000100 11 1 ..... 111101 ..... ..... @rrx_1 esz=3 + # SVE2 integer multiply (indexed) MUL_zzx_h 01000100 0. 1 ..... 111110 ..... ..... @rrx_3 esz=1 MUL_zzx_s 01000100 10 1 ..... 111110 ..... ..... @rrx_2 esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index e32094dcdb..c72d1e4bf0 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3864,6 +3864,14 @@ DO_SVE2_RRX(trans_MUL_zzx_h, gen_helper_gvec_mul_idx_h) DO_SVE2_RRX(trans_MUL_zzx_s, gen_helper_gvec_mul_idx_s) DO_SVE2_RRX(trans_MUL_zzx_d, gen_helper_gvec_mul_idx_d) +DO_SVE2_RRX(trans_SQDMULH_zzx_h, gen_helper_sve2_sqdmulh_idx_h) +DO_SVE2_RRX(trans_SQDMULH_zzx_s, gen_helper_sve2_sqdmulh_idx_s) +DO_SVE2_RRX(trans_SQDMULH_zzx_d, gen_helper_sve2_sqdmulh_idx_d) + +DO_SVE2_RRX(trans_SQRDMULH_zzx_h, gen_helper_sve2_sqrdmulh_idx_h) +DO_SVE2_RRX(trans_SQRDMULH_zzx_s, gen_helper_sve2_sqrdmulh_idx_s) +DO_SVE2_RRX(trans_SQRDMULH_zzx_d, gen_helper_sve2_sqrdmulh_idx_d) + #undef DO_SVE2_RRX #define DO_SVE2_RRX_TB(NAME, FUNC, TOP) \ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 25061c15e1..8b7269d8e1 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -240,6 +240,36 @@ void HELPER(sve2_sqrdmulh_h)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int16_t *d = vd, *n = vn, *m = (int16_t *)vm + H2(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 2; i += 16 / 2) { + int16_t mm = m[i]; + for (j = 0; j < 16 / 2; ++j) { + d[i + j] = do_sqrdmlah_h(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 32-bit */ int32_t do_sqrdmlah_s(int32_t src1, int32_t src2, int32_t src3, bool neg, bool round, uint32_t *sat) @@ -373,6 +403,36 @@ void HELPER(sve2_sqrdmulh_s)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, false, &discard); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_s)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int32_t *d = vd, *n = vn, *m = (int32_t *)vm + H4(idx); + uint32_t discard; + + for (i = 0; i < opr_sz / 4; i += 16 / 4) { + int32_t mm = m[i]; + for (j = 0; j < 16 / 4; ++j) { + d[i + j] = do_sqrdmlah_s(n[i + j], mm, 0, false, true, &discard); + } + } +} + /* Signed saturating rounding doubling multiply-accumulate high half, 64-bit */ static int64_t do_sat128_d(Int128 r) { @@ -452,6 +512,34 @@ void HELPER(sve2_sqrdmulh_d)(void *vd, void *vn, void *vm, uint32_t desc) } } +void HELPER(sve2_sqdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, false); + } + } +} + +void HELPER(sve2_sqrdmulh_idx_d)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc); + int idx = simd_data(desc); + int64_t *d = vd, *n = vn, *m = (int64_t *)vm + idx; + + for (i = 0; i < opr_sz / 8; i += 16 / 8) { + int64_t mm = m[i]; + for (j = 0; j < 16 / 8; ++j) { + d[i + j] = do_sqrdmlah_d(n[i + j], mm, 0, false, true); + } + } +} + /* Integer 8 and 16-bit dot-product. * * Note that for the loops herein, host endianness does not matter From patchwork Fri Apr 30 20:25:47 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429878 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp792599jao; Fri, 30 Apr 2021 14:03:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy4fOgbtCy18Hlcgm4DGp0Oh56MjebU2b7704aEBYl6En7km4ANj51r6Z2uVhngYzns4M9m X-Received: by 2002:a1c:cc18:: with SMTP id h24mr18119591wmb.23.1619816601717; Fri, 30 Apr 2021 14:03:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816601; cv=none; d=google.com; s=arc-20160816; b=zsIyGAf/d9DVMZvYZicy1BvrB0fAP7cmlR/Q8/WE9cgL43Nn41P0ME4UVkN+8MfMUQ Ir1V1ThvbAFJvF6SKQv4yPsrew0fW89XodctOjMqQ1oShJ78PGSKqMkiL1kFwrqPf9tU eSTYeDmdNosnrelp771l21DePK5oFwmLO673KZKfsaNvwxkYoBgUXaT1JSc50raA56Vw 4yLNLtYFjzaQ/Yc3QsQ3x5u4r2f9XKg51wzUk1OwsJA8bSmJn0YuKzw2toPUdGGIf6eR ZLXVZI43rIcYaKwfB77uleTRDpqwVAPXyomxTCuGKxLHDjwI0kqB3VTmIEeLz/5Ow/k/ Ii3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=RdFmu/25x020HctPhXmoL5iGzR1SRnNRa4SVWqSe98Q=; b=azvtLefH2jmnMwdkKboDcXMJAg06JM+QsLQs3DW9qhOAGiHS+hsXmbw1Kz1B1Zbn+E sOco/KBR5f7+K4HgCS+gKAgtfwFRol9NhDEJzuSa/4WnxELZxFTa77FEJMTuMYqG/Nm3 BsjamOBY1Dyt7mDNjlmEKwa/4SxLjQBCiqsntWK/NRMTQmcHDLE7pVj8lseh45gKOpRb V/e2agtchIJVDtXcXup2RiFIrUa5NC/9YGKWJ3qTaM4ix9wVE/KOoitUQ1ISVDaanKyU HbeTGtcdto/NvfP3bYIc+7YHRzL+6CCU6Mfi0HS2NT8uRwY1O0jB1d0ATRD4votfr+Ei rbSg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HML6FltO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m23si13828804wmi.103.2021.04.30.14.03.21 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:03:21 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HML6FltO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46932 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaIC-0002R3-2j for patch@linaro.org; Fri, 30 Apr 2021 17:03:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53256) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZm8-0004Ff-EK for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:17 -0400 Received: from mail-pg1-x52f.google.com ([2607:f8b0:4864:20::52f]:36791) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007kV-2Q for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:11 -0400 Received: by mail-pg1-x52f.google.com with SMTP id j7so40766656pgi.3 for ; Fri, 30 Apr 2021 13:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=RdFmu/25x020HctPhXmoL5iGzR1SRnNRa4SVWqSe98Q=; b=HML6FltOKshsUTbQIW9shJRkhrw9/e2zz7yEjDYjDzUG/GUphtwP/x85abNJePtcAf h/q1T//gYhyMyRjPQ/hPRWuSvraciDORyP66etuTaTNCp0DQ+V0ncAoZPyYp7TjnYm0E /9I6pBvbVgrjo7LkPECSLFs6YzyQFlrFyx9v71TtwWZ7rz8Kr1CMMnXSuKMoAb4/bplU BGGYoQZ9eND7wrNsWz/Pv5BKd+mTES8ICpoQRuObZRDK42KQ+Ryssx0Mb1TTG7PfJtEF ljoV6Y8zMWSnaJQB8f4EhjxvjdijqBl6PdNQ2jYYMcQbH4kgcbwty8yzVAVJUuKMQ3/S 2Rtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=RdFmu/25x020HctPhXmoL5iGzR1SRnNRa4SVWqSe98Q=; b=fnRSCRm6pG2y2+eueTdQ4cGqdAAb4iadjttVhazjJip0KfF4s2sAnBiTu6+YB328bb kHnJnOIqYbq+wZHSNDY3WRGzaoclVoIlaN389Yim3Wc4gpBM+Is9EygNjsdyEI8iZlc+ hewUNTMFG2Ejgdx2g3fAqrJ7J7UOcAXteTSmWFUxf7DojvmIN11gBsFP8LLrIaQMDVA/ Erp59gEWEWHWYN4q38BK5h6NKDCOc1MLe4CIiP2CFH6z7NbPeu46EmafIQgFtoo2RDdg JupTRX/mRu8hq9BNyLwy5/nGQiyLXfdehHhZuBt7lU+eh8lg8d/VsIjlcKp4uTlah2oB RYgw== X-Gm-Message-State: AOAM533c6iKI0NNT0evCvNWb9K9+yVnBrN4qR1rZE6Sl0LyhiNP5etfh iGWC+6E/fXFRCEY9iUKP6bAUG0d/TuSPoA== X-Received: by 2002:aa7:9910:0:b029:27e:8025:72bd with SMTP id z16-20020aa799100000b029027e802572bdmr6729213pff.24.1619814588233; Fri, 30 Apr 2021 13:29:48 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 59/82] target/arm: Implement SVE mixed sign dot product (indexed) Date: Fri, 30 Apr 2021 13:25:47 -0700 Message-Id: <20210430202610.1136687-60-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52f; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++ target/arm/helper.h | 4 +++ target/arm/sve.decode | 4 +++ target/arm/translate-sve.c | 16 +++++++++ target/arm/vec_helper.c | 68 ++++++++++++++++++++++++++++++++++++++ 5 files changed, 97 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 595bc6349d..0a41142d35 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; +} + static inline bool isar_feature_aa64_sve_f32mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, F32MM) != 0; diff --git a/target/arm/helper.h b/target/arm/helper.h index e7c463fff5..e4c6458f98 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -621,6 +621,10 @@ DEF_HELPER_FLAGS_5(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_sudot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_idx_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 35010d755f..05360e2608 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -813,6 +813,10 @@ SQRDMLSH_zzxz_h 01000100 0. 1 ..... 000101 ..... ..... @rrxr_3 esz=1 SQRDMLSH_zzxz_s 01000100 10 1 ..... 000101 ..... ..... @rrxr_2 esz=2 SQRDMLSH_zzxz_d 01000100 11 1 ..... 000101 ..... ..... @rrxr_1 esz=3 +# SVE mixed sign dot product (indexed) +USDOT_zzxw_s 01000100 10 1 ..... 000110 ..... ..... @rrxr_2 esz=2 +SUDOT_zzxw_s 01000100 10 1 ..... 000111 ..... ..... @rrxr_2 esz=2 + # SVE2 saturating multiply-add (indexed) SQDMLALB_zzxw_s 01000100 10 1 ..... 0010.0 ..... ..... @rrxr_3a esz=2 SQDMLALB_zzxw_d 01000100 11 1 ..... 0010.0 ..... ..... @rrxr_2a esz=3 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c72d1e4bf0..c988d0125a 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3838,6 +3838,22 @@ DO_RRXR(trans_SDOT_zzxw_d, gen_helper_gvec_sdot_idx_h) DO_RRXR(trans_UDOT_zzxw_s, gen_helper_gvec_udot_idx_b) DO_RRXR(trans_UDOT_zzxw_d, gen_helper_gvec_udot_idx_h) +static bool trans_SUDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a, gen_helper_gvec_sudot_idx_b, a->index); +} + +static bool trans_USDOT_zzxw_s(DisasContext *s, arg_rrxr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + return do_zzxz_data(s, a, gen_helper_gvec_usdot_idx_b, a->index); +} + #undef DO_RRXR static bool do_sve2_zzx_data(DisasContext *s, arg_rrx_esz *a, diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 8b7269d8e1..98b707f4f5 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -677,6 +677,74 @@ void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_sudot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + int32_t *d = vd, *a = va; + int8_t *n = vn; + uint8_t *m_indexed = (uint8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + uint8_t m0 = m_indexed[i * 4 + 0]; + uint8_t m1 = m_indexed[i * 4 + 1]; + uint8_t m2 = m_indexed[i * 4 + 2]; + uint8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_usdot_idx_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, segend, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + uint32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m_indexed = (int8_t *)vm + index * 4; + + /* + * Notice the special case of opr_sz == 8, from aa64/aa32 advsimd. + * Otherwise opr_sz is a multiple of 16. + */ + segend = MIN(4, opr_sz_4); + i = 0; + do { + int8_t m0 = m_indexed[i * 4 + 0]; + int8_t m1 = m_indexed[i * 4 + 1]; + int8_t m2 = m_indexed[i * 4 + 2]; + int8_t m3 = m_indexed[i * 4 + 3]; + + do { + d[i] = (a[i] + + n[i * 4 + 0] * m0 + + n[i * 4 + 1] * m1 + + n[i * 4 + 2] * m2 + + n[i * 4 + 3] * m3); + } while (++i < segend); + segend = i + 4; + } while (i < opr_sz_4); + + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { From patchwork Fri Apr 30 20:25:48 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429877 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp791551jao; Fri, 30 Apr 2021 14:02:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyg3oyZDI1J3qZHfW1FPyz99Uftyj7/o+10Y49ACdhe7etMIIKREhiVEKgJigLAoxCieLLk X-Received: by 2002:adf:e783:: with SMTP id n3mr9731584wrm.154.1619816529086; Fri, 30 Apr 2021 14:02:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619816529; cv=none; d=google.com; s=arc-20160816; b=qjvmbQtI6pjfGcp5K8Aqtl+5ne28U4vB6cFCWGwo3mipAt/tpKFqpONjxWEqZPIVDs vxWyWBArQVWgW+51PZnrBKoBwnYvWJmOu7N2Zne+xX7CzfYTx1fyKxoMbuXMX+bKSu/2 L8+wZC7n2/xypqecEbLloLz6OxR6n4/5tTYbUPRD2ULb6of8TS59f3Q1+sgSUAgTSdhq dd4pUWOA7VAEy9yXN1H0xdGPoikA6jWHJLLPhayt2ICQL5aIDeazcqHPMAR/kkn3ndo1 yQJBFLHDOdgPBDlpPkFHg/MnjmfqqSMqSiYWYfvttepZwfjtobYl1oQgFwocHDsKUnh5 hskQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=vUNPTNdfvzy60eNisdwBmT1BzT2snFcTBFFJdjdBzuE=; b=mZXAQRlmlMEvAzZ2OaPVnn9+Oi2yA/O2Vqpa8jca4eCl9G8RdJpV66H8DjVYnEH5sN CvcstyiW6Tf6GPk+UxHglxpUVt50zdTe8vEz9ZIw5Jo0aiCW2CYoJyIwHwsWfMoQNb1u 95iAGP9P4singWyTbSI8lanbz9SVZeQNvWy5IXPNuCSZhbQMS2+uEJjRiAsRFpu2M/Lz 0wkKi9ivZvfTBV5bve5h36MnMOMuTfmGCIDIx7/nq2rhrFekTdRuXgHR5Rz1b79lXbvF YbYbGPmsQJ+9KRLw7IlF3M+UlLYJO0yW1O5crfdxXZBiEKtsmqQ5I7GoQ8S/xAA2a7+x t0jw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DWRfRI0B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a20si3196757wrc.65.2021.04.30.14.02.08 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:02:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=DWRfRI0B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44154 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaH1-0001Do-Dl for patch@linaro.org; Fri, 30 Apr 2021 17:02:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53328) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmG-0004ME-9Z for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:20 -0400 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]:33616) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007kd-7P for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:19 -0400 Received: by mail-pg1-x52b.google.com with SMTP id t22so14930016pgu.0 for ; Fri, 30 Apr 2021 13:29:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vUNPTNdfvzy60eNisdwBmT1BzT2snFcTBFFJdjdBzuE=; b=DWRfRI0BB7QZZuEceoxeR1/ZAZZUh6tpSZDss6r1XAnGUzpDy4HrpR680862rsW7Vb 2X2yejhu3Y+t7DtqKr+/SEvbSQawqeEFTxlG1xqh/geSIE6YIg7KDnTpI3YzWlJWmfqy rAtnxDcT2YaUBM1sVy3TmMZm5+4tfZ/ByBBnHM9Q9nhEowK6eIOfCF9jAs78xPF1FFOe x5e05SYSg2JiBd2/eauNZ9whFu1WTCBOFPn3YMMoat/UGizJMxpzATabdqIJHVkIGZGE 5pCgjnCtwUUX28XGWRPewBzZRU76u7KkY2pdXneKsI4cdaANGSry++HKB4RFvfZSNPdx sNoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vUNPTNdfvzy60eNisdwBmT1BzT2snFcTBFFJdjdBzuE=; b=nuDGz46d6EH8nxqS3hZaO4UiKR57dXmiKKZ9EzsQOsF00yzswKRdK/uHUazeyV08LR yILjmw3lqkkCUQRv3AdM0OWTz8eIt1oWHgJc/0XFqAjKZA/J2j9Tl5Hll1qm8d7vVEv1 7tCMn/xSR2exPttSLZRG5rjWGiNXAyJSBMWmnTqxCYjlTnww2LqG4hsxlGZaynk9CMzM SBsSHhTrUaKqxWHAIHQNvF0gguFClXFfAPIxEKv3Qw/3q8MvonlN9Dej/BJAKrS9QOjy NEEFnAJhoEpi88pPeR3FRJZrJH3tDaI9uq5VKENLAezXhEYC10dmFHCZduljFjC5Dr/V B74Q== X-Gm-Message-State: AOAM531TTWk745gaoxRsO+VQEHpINgT8vSNy5AP1kR6nFSU7tXbeiZm4 M76MY+AmqG6wEZZXH6Oy2aqFH/ca2FUWaw== X-Received: by 2002:a63:1f25:: with SMTP id f37mr6372080pgf.142.1619814588855; Fri, 30 Apr 2021 13:29:48 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 60/82] target/arm: Implement SVE mixed sign dot product Date: Fri, 30 Apr 2021 13:25:48 -0700 Message-Id: <20210430202610.1136687-61-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52b; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 2 ++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ target/arm/vec_helper.c | 18 ++++++++++++++++++ 4 files changed, 40 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index e4c6458f98..86f938c938 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -612,6 +612,8 @@ DEF_HELPER_FLAGS_5(gvec_sdot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usdot_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 05360e2608..73f1348313 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1481,6 +1481,10 @@ UMLSLT_zzzw 01000100 .. 0 ..... 010 111 ..... ..... @rda_rn_rm CMLA_zzzz 01000100 esz:2 0 rm:5 0010 rot:2 rn:5 rd:5 ra=%reg_movprfx SQRDCMLAH_zzzz 01000100 esz:2 0 rm:5 0011 rot:2 rn:5 rd:5 ra=%reg_movprfx +## SVE mixed sign dot product + +USDOT_zzzz 01000100 .. 0 ..... 011 110 ..... ..... @rda_rn_rm + ### SVE2 floating point matrix multiply accumulate FMMLA 01100100 .. 1 ..... 111001 ..... ..... @rda_rn_rm diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index c988d0125a..8952990453 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8056,3 +8056,19 @@ static bool trans_SQRDCMLAH_zzzz(DisasContext *s, arg_SQRDCMLAH_zzzz *a) } return true; } + +static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) +{ + if (a->esz != 2 || !dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + vsz, vsz, 0, gen_helper_gvec_usdot_b); + } + return true; +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 98b707f4f5..9b2a4d5b7e 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -579,6 +579,24 @@ void HELPER(gvec_udot_b)(void *vd, void *vn, void *vm, void *va, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_usdot_b)(void *vd, void *vn, void *vm, + void *va, uint32_t desc) +{ + intptr_t i, opr_sz = simd_oprsz(desc); + int32_t *d = vd, *a = va; + uint8_t *n = vn; + int8_t *m = vm; + + for (i = 0; i < opr_sz / 4; ++i) { + d[i] = (a[i] + + n[i * 4 + 0] * m[i * 4 + 0] + + n[i * 4 + 1] * m[i * 4 + 1] + + n[i * 4 + 2] * m[i * 4 + 2] + + n[i * 4 + 3] * m[i * 4 + 3]); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_sdot_h)(void *vd, void *vn, void *vm, void *va, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Fri Apr 30 20:25:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429889 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp800444jao; Fri, 30 Apr 2021 14:14:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxPHmWxrufxZfz5Dz6cFHNt5LDZUJJKepOp2ZyfTnszGTr0nc7LHalTxkyPJIk6F8o0xl6/ X-Received: by 2002:a92:d684:: with SMTP id p4mr5908773iln.150.1619817257464; Fri, 30 Apr 2021 14:14:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817257; cv=none; d=google.com; s=arc-20160816; b=TiPAgywQenCSThy068ustuS54TbxLactG02pyQIQ8W8zNDmF9LTX8ePLuyj9BtbRtN VlrETeMc85QXQ0yjVUI6TYb//b9ERwMizjh8DctmHJcnnEFxwh2KDzasVjKrk09H6uA3 J89qpcQ6ObECvOcFd9X5QRkwuXXWqZc8P+zsQ36+0TTGKbKVnmAe+amuc8DKKOIR4k0e TvIHb+l7t6oNMlpEF1+MI5o8nTg8uMGBRcUHxPsIvZwhfayuFAt9Muz1+YI/XuHLq3Dl Gs9tloWGNenpLWNai7NVl7dDe2M5W/IDClGjUdQMQWGs8phMhuAz599oFlRvc0T5OKM/ vCxw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=grtE+IKARzmMb4HRIktrtzr0mKc9uwbmVIePGd0MZ+U=; b=Vp6S2lTSqS6jFrT4DP7qfLLCkoLIun1cNzvDEQodxnOPTfSPBWDcuZS//jcG5VeCqu eAXzuQlWAUvO+oZzMtUiag7rp33lvFWOOSDw+3VrvXlkM8DkmXnt5BuEKUdjn7pBEPir ELPJAS/JL+GeQ6kioswuhsu5eN0hv5gltSri9tJTY9don5EO8y+OX1vQSyv5xvqpxkId wdTnxj2neKYlbv6NpmZiZQNdRv51cEsLPnrSb8J9OjdhdKMEE/Dn3tY1chr/ynsmq3L+ aBBvNlpiQVuNu4cyyXfENYpOXzNp3XrxyReZHSKpqNNwXNIPmwfZIhMMDQ/zy1g33Eqg ZtUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h2Dz3C9q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q2si4736214iow.34.2021.04.30.14.14.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:14:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=h2Dz3C9q; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51724 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaSm-0007kq-Sg for patch@linaro.org; Fri, 30 Apr 2021 17:14:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53544) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmW-0004ml-Ds for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:36 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]:39772) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlm-0007ks-Nq for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:35 -0400 Received: by mail-pf1-x436.google.com with SMTP id c17so14648246pfn.6 for ; Fri, 30 Apr 2021 13:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=grtE+IKARzmMb4HRIktrtzr0mKc9uwbmVIePGd0MZ+U=; b=h2Dz3C9qhefPzZY5xiyJB5ygseEXFK65KoIKk7zh4pac66Pg8mpo18aNCB39QNzefz Ny/BD4gCl5ZG0pTCt5IvisSW+UIOVA8L7muSZBnz+i2CbGCMaqUEg+RSJ+DtmScMJMza DRxjsLU6l5sX74Hn8pBz4kYRz+d5rvLiqjW55iTRTiS6PqyHwbVrPlW14EoY/FILyucc sdFs2E7J2EWE+1J1DRdA+ASDDVyxr/qeO+Av9835G5ssuTxpsYDWobT98jIF+WKp6657 cwzD7DXSosivzUZqPXwLGKPt63GGOP58W0t8ZK3IdlRcWxbrLWFBuATaKBK3xQgatuu6 gfJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=grtE+IKARzmMb4HRIktrtzr0mKc9uwbmVIePGd0MZ+U=; b=DEwkgzv4bqr0zlhZ9MLmAP379fdnx47nTFqPhiVieQFJfwZxVmonX63iM+pOKumc0x FyYAuuUAY48AAJF7jPC8SpzMlazUDAi162h6JFv2K7Ocl/H5V2zem4M1MeVdWd6Izn7h w4XAFKTm0/dmc3GenH8eFADYTYzkCRplWrLHi+ERXiNr2H6F4hYZFvEZPluKhX6CaJDD hhGOqGCTLVvTDbL2Eo5zwDvQKpu5kBafwCxwfCDFrlgAhVe8k3eDbzZVjWMblNqd/XRt MZoyrAg/R8NDHTbutxJom4+1z8uxAVbCJkzSTu01OrkEJQVgupn5SKhRsHpzCa0RoY01 iHzg== X-Gm-Message-State: AOAM5320arHTnFhQWUhf1hl03I4pRiVST1UxM4+L7mymckfxnAVkwhHO OJ36juprPtXGTtKZqsiwiswc2WkuWyJS4A== X-Received: by 2002:a62:3486:0:b029:24c:34c0:3c7a with SMTP id b128-20020a6234860000b029024c34c03c7amr6318292pfa.36.1619814589422; Fri, 30 Apr 2021 13:29:49 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 61/82] target/arm: Implement SVE2 crypto unary operations Date: Fri, 30 Apr 2021 13:25:49 -0700 Message-Id: <20210430202610.1136687-62-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 6 ++++++ target/arm/translate-sve.c | 11 +++++++++++ 2 files changed, 17 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 73f1348313..6ab13b2f78 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1509,3 +1509,9 @@ STNT1_zprz 1110010 .. 00 ..... 001 ... ..... ..... \ # SVE2 32-bit scatter non-temporal store (vector plus scalar) STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ @rprr_scatter_store xs=0 esz=2 scale=0 + +### SVE2 Crypto Extensions + +# SVE2 crypto unary operations +# AESMC and AESIMC +AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 8952990453..5219b93abd 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8072,3 +8072,14 @@ static bool trans_USDOT_zzzz(DisasContext *s, arg_USDOT_zzzz *a) } return true; } + +static bool trans_AESMC(DisasContext *s, arg_AESMC *a) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zz(s, gen_helper_crypto_aesmc, a->rd, a->rd, a->decrypt); + } + return true; +} From patchwork Fri Apr 30 20:25:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429902 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp807859jao; Fri, 30 Apr 2021 14:26:18 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy+sb1WpuZKNZROaUpvtT61AepQB8CY2Wj9WVon7mgKT6gPU/tKrEvLG5kW8MvDbylek0eQ X-Received: by 2002:a02:7348:: with SMTP id a8mr5720775jae.116.1619817978162; Fri, 30 Apr 2021 14:26:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817978; cv=none; d=google.com; s=arc-20160816; b=WALgtmKgStmBJFPm20mqAvpqRQTfaYILQdZqAv6GMUKxdrlCXlGs6mz973EVn4Sknf oUcNmIuDjjk2sjmG3wnGtwkypdsc536S6+zcZcvQjAF5cqWyGTzudqy5O2w+KeHreSg/ lRRinKGTJ/XCrd68UUxIoocyspJU11ChkoCtWqNhF2yun4+O877PW70mBnoyjBOspcSP X5DNAaWnKWP3hPYuzI2An8Gl8NSNSSUFPVcMVkMQlqKPWIbtzz1mgFBW0UuAcsfGyvrv 6kpZ0LHoa3ze7dcHCq4efkBi2JTR7MGET7ARo754Ar7T1QmydhL9Q/nOdNyjaxZr+1N4 9cAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0UnzIflfhV5UCyvLXEO7jKi/iiKGyKifImpfUPcU4RQ=; b=kAKaFEVZIjf5nCwolTrE4W3zrY6yADBFfKWU2PYNBgQ9G6oJm187N1Cf4yvXKW9XRO 6+MfR6fADrFgwjcrENsiMfcwhGIfYcOokPABzmVgV6aXWRs9fw1VP06JS+wGjcENBtsF lTCeRibliJDLFGlfZ/byf5PIPYcemd8ai3bef2VG27gibhycO167oI8lODp94AbiX6xm 6IfnYA5pra5qox/vbZE/R8fwHuMCcqr+Ixatm+Nt6PR1i1tFQyeIbovH7z0o0tDivY8R dT/5c/HM2mWkbl58WAzekXsCNVNwbxlhguPVlOTd9Oa7HKFvapdlOEma+ImX5N76bMs0 jx0A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lprJ4Mxa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t3si5157092iov.10.2021.04.30.14.26.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:26:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lprJ4Mxa; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:54104 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaeP-0003eW-L3 for patch@linaro.org; Fri, 30 Apr 2021 17:26:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53740) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZml-00052O-OA for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:51 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]:46958) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007lC-3a for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:51 -0400 Received: by mail-pg1-x530.google.com with SMTP id j189so3168366pgd.13 for ; Fri, 30 Apr 2021 13:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0UnzIflfhV5UCyvLXEO7jKi/iiKGyKifImpfUPcU4RQ=; b=lprJ4MxayrxOS8rkpbcJCGkOpgHj6lkqyLTESyBf/sQGH6UG3EI1DT8IQBlMls81Gp EwCDOkg1V6E5NkJdndOIKyqCXF1TPG0G95reOVqstxEHQa9QU5yhetzR+L3JRtk55A0S Cl3yQG7WqffMPxmIBaqJ6NSFR47bj2fqzVt5VzSC/ibmshz0QSTL4FWn6dSMumv20sN0 KnIIpiMNcfpygb09/NRWTlctAwoVtjSHeHfKMJwg6SHMGFvG9Od8Bi1E8I6xlKwG6d72 MAyLU/kx9EIVyOLRzTDEoJ0UuIyc2oO38tU0xgUE/n8hYxchCQsFspJ/5hJjmlGibQm7 UnYQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0UnzIflfhV5UCyvLXEO7jKi/iiKGyKifImpfUPcU4RQ=; b=fDv62Dr4RXaDTsrp1W/6R8p+F/p049bfR4dV4fwHcaM6YUwaTd+rj6IV0gWDgH67nv +KXtkSybdMpV2AlJAToM5huohHQwd/aNgvlIIPvmwN/yghANI3tvKLR3Rl9UQvU6O/wV IlrX7oZGwGiA6AS7AP1cqIRQ2EyI1e6IkusjDHIlI7h9ztfF0toXb5x8a3bg0ro1iYX8 MZO1552zsEBeORkRLeG+WnP5nq5pTbmvvf+cVXYqNJfYiXA5HB1959Jo0PaX9OT+f0F6 kfx5CW81r1EMiIrK/U1r+A+Irc2aE8I9fjb8SU2ww/p4EfGdFVjGt7Vb8U8gdrMXO14j joHw== X-Gm-Message-State: AOAM530NTyEmTCpMVb/D9yD4ELaoA/giHAzjSY45nAVrFQ41RB5FSjUT nfPRPI+id7h1bs1awxM6FL0Ex3/mRoOYbw== X-Received: by 2002:a62:2c46:0:b029:245:6391:b631 with SMTP id s67-20020a622c460000b02902456391b631mr6353204pfs.67.1619814589929; Fri, 30 Apr 2021 13:29:49 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 62/82] target/arm: Implement SVE2 crypto destructive binary operations Date: Fri, 30 Apr 2021 13:25:50 -0700 Message-Id: <20210430202610.1136687-63-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 7 +++++++ target/arm/translate-sve.c | 38 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 50 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 0a41142d35..384c92eebb 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; +} + static inline bool isar_feature_aa64_sve_i8mm(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, I8MM) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 6ab13b2f78..fb4d32691e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -118,6 +118,8 @@ @pd_pn_pm ........ esz:2 .. rm:4 ....... rn:4 . rd:4 &rrr_esz @rdn_rm ........ esz:2 ...... ...... rm:5 rd:5 \ &rrr_esz rn=%reg_movprfx +@rdn_rm_e0 ........ .. ...... ...... rm:5 rd:5 \ + &rrr_esz rn=%reg_movprfx esz=0 @rdn_sh_i8u ........ esz:2 ...... ...... ..... rd:5 \ &rri_esz rn=%reg_movprfx imm=%sh8_i8u @rdn_i8u ........ esz:2 ...... ... imm:8 rd:5 \ @@ -1515,3 +1517,8 @@ STNT1_zprz 1110010 .. 10 ..... 001 ... ..... ..... \ # SVE2 crypto unary operations # AESMC and AESIMC AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 + +# SVE2 crypto destructive binary operations +AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 +AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 +SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5219b93abd..fd4ec4a66b 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8083,3 +8083,41 @@ static bool trans_AESMC(DisasContext *s, arg_AESMC *a) } return true; } + +static bool do_aese(DisasContext *s, arg_rrr_esz *a, bool decrypt) +{ + if (!dc_isar_feature(aa64_sve2_aes, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, gen_helper_crypto_aese, + a->rd, a->rn, a->rm, decrypt); + } + return true; +} + +static bool trans_AESE(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, false); +} + +static bool trans_AESD(DisasContext *s, arg_rrr_esz *a) +{ + return do_aese(s, a, true); +} + +static bool do_sm4(DisasContext *s, arg_rrr_esz *a, gen_helper_gvec_3 *fn) +{ + if (!dc_isar_feature(aa64_sve2_sm4, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fn, a->rd, a->rn, a->rm, 0); + } + return true; +} + +static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4e); +} From patchwork Fri Apr 30 20:25:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429905 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp809363jao; Fri, 30 Apr 2021 14:28:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6kYyuckzeevUc3tjpWc81LoDGAnV798SwGOoc9da4xruWDsd+Nn1cImVFYFNyO6gJImZZ X-Received: by 2002:a05:6638:2054:: with SMTP id t20mr7067054jaj.75.1619818129569; Fri, 30 Apr 2021 14:28:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818129; cv=none; d=google.com; s=arc-20160816; b=yJzRoMqSD2iGhN6jwwF2dHE8QAz0HdQqnHWHAAE2LsOAf1T79Zvmpksg+6hfFpOiPS xgNZgahN4ktf5FmQWJRTHK0ZHjwZvV3hRsCY0sSxPgwmm4QxgV4cuAJq2g3F2yCMfZSj /5yF9ziWoO8jyje2ebnEK8Q8NEpuV6TB1+5VOFN3eTdKxpv2cAB60P/mxeBBnv4UCMnL y/H4zWEdezlTWD3M8bDQhQ1ADLTqtZ4VU7vmf+2IwoLjpWb4hSxYFxaG6SnkA4xtbIKr lmGCsY+gVi2cqKNUKfBn887JdvHOlC7O7/uHKYFpCUwTWr8jcogufjPmxAs/7F0Pf/TR OvlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=CXYlVo4thPOg47NozYtH3Ja8wzwpSJTG3c2FPfnTAhM=; b=jCPHhkIya09zPOj1+UP9zV3jza5l/zAqS0tKGhKbSfCKzAdwWRl+EqTR2un5QqoXfZ xZqb5uXW24o6OKLmy1WpKE058k3WvYWv3auqVsHy2RuVNEDoYztSGxsRDTy+O+l2rnUk Tp7dX0L8pfT8fByel0DZ4P5ylLP5TjXqn0r//2zYy708NlcU2InTkW49Iq5qyrk1IPkD Kdwlc10xJfl65k7KKyL28RbC71yOkrFAqBXOoruZRSW20Q0kYlhpN9abDzHMu3sqZypp 3iFUeCcgshAgelCMWYE784+akWAe8roZ8OvKjzeM0uY5pTZs+mBIWgPbupk19CD63QBu C7nw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y3g0lfHV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q14si3367628ilg.14.2021.04.30.14.28.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:28:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Y3g0lfHV; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33222 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcagq-0006iz-R5 for patch@linaro.org; Fri, 30 Apr 2021 17:28:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53776) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmq-00054X-Nm for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:57 -0400 Received: from mail-pf1-x435.google.com ([2607:f8b0:4864:20::435]:45663) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007lr-9g for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:52 -0400 Received: by mail-pf1-x435.google.com with SMTP id i190so9682884pfc.12 for ; Fri, 30 Apr 2021 13:29:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CXYlVo4thPOg47NozYtH3Ja8wzwpSJTG3c2FPfnTAhM=; b=Y3g0lfHVJqE7tAI0bXBuq28F6X/2rDpc+YYK3cLRFXAz0UYEKyp9q3cDt8ia3p21XP na1HC2IGnqfZ1lg1Q4/7qy/VJ9AMsX7CDNiPWHxnDMR/pYYkmnFSZXJgCSuH0oZGijUg 4aScBZ0VjpW407+afjhMOCT4v8oCXjKFgd/tyss1Hf7AzoOqU+VKijfF5y/Z1yBd/xkZ 97dXJ9PY/naVvQ43KOZ1v83g5VZfSWrnnAa9Yf1tkR//kZs9ixlPRmK8ioeZ1FpclkBX HehguYjH39dda/raZuvA+qao1gMvmFjbG0oNxKCe0j+TMYU5r6UAYGs9j66ePerrC5Ic 1xLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CXYlVo4thPOg47NozYtH3Ja8wzwpSJTG3c2FPfnTAhM=; b=A2iZ5B5Zn/JSbWqDtbDYqJdUWg1k07oTqa+y4BnKPD18VgJDsuifnR7NmMRGNU6n2E Z1fUhgzPEMbt1OCb1SreDGeWD/b/g78ldH4eMMKcI1oisfTDUvSx1/Dzwj3VZX41AzRm PDQi1CNyr+zqHVWqIE10Ceu+fJ5S4L/JxiLlZXTMjs23gkUmSfLE0iAx7TnRbk4/h8GM EbuRpD8iDKBAkLuRxHpqb6BxP9XQ1tBlEiyTA+0AnLOq4DM4S5ABJEoQqlYnhKRNnBd4 RYuyCCEQkH2z+zuMFWQdOQTl5ljMY6v3zu2aroCq4nC1L7t8b6W9i65pyNzdYnOukwXh FOaA== X-Gm-Message-State: AOAM530eMQ+Et/2v1V6ujMVbMOlkWaskgT1TvNbtD9k/m+ETSaunQiXI SgMPn52c+149HLRcnNYtxeDVeIqugwiuNA== X-Received: by 2002:a62:d409:0:b029:27d:338:1cca with SMTP id a9-20020a62d4090000b029027d03381ccamr6739543pfh.25.1619814590459; Fri, 30 Apr 2021 13:29:50 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 63/82] target/arm: Implement SVE2 crypto constructive binary operations Date: Fri, 30 Apr 2021 13:25:51 -0700 Message-Id: <20210430202610.1136687-64-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::435; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x435.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 3 files changed, 25 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index 384c92eebb..c75601b221 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4246,6 +4246,11 @@ static inline bool isar_feature_aa64_sve2_bitperm(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, BITPERM) != 0; } +static inline bool isar_feature_aa64_sve2_sha3(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SHA3) != 0; +} + static inline bool isar_feature_aa64_sve2_sm4(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64zfr0, ID_AA64ZFR0, SM4) != 0; diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fb4d32691e..7a2770cb0c 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1522,3 +1522,7 @@ AESMC 01000101 00 10000011100 decrypt:1 00000 rd:5 AESE 01000101 00 10001 0 11100 0 ..... ..... @rdn_rm_e0 AESD 01000101 00 10001 0 11100 1 ..... ..... @rdn_rm_e0 SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 + +# SVE2 crypto constructive binary operations +SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 +RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index fd4ec4a66b..5f2a2b1eb2 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8121,3 +8121,19 @@ static bool trans_SM4E(DisasContext *s, arg_rrr_esz *a) { return do_sm4(s, a, gen_helper_crypto_sm4e); } + +static bool trans_SM4EKEY(DisasContext *s, arg_rrr_esz *a) +{ + return do_sm4(s, a, gen_helper_crypto_sm4ekey); +} + +static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2_sha3, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_fn_zzz(s, gen_gvec_rax1, MO_64, a->rd, a->rn, a->rm); + } + return true; +} From patchwork Fri Apr 30 20:25:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429900 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp806904jao; Fri, 30 Apr 2021 14:24:41 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyDBW2526dIREDnzXRbY9fsfPFnrLdisUuc8KFz9DIIhxgdbZD5Yfso1/g4nPZpcNXK1la3 X-Received: by 2002:a02:37c9:: with SMTP id r192mr6763760jar.88.1619817881002; Fri, 30 Apr 2021 14:24:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817880; cv=none; d=google.com; s=arc-20160816; b=0Vdsj9efdPZuNbb6G7H01NVFe131RaZrVqSW7Uk9qiO6VXPspGqR98rPxqONS9yAdC mdFO1kNGA4oS1toMLnp4a7v4mSRNYWAj7PL5tJ93NepNId1YsNxJuJ4t5Z39OiJ16ypL VByOC6cQSIyGEmNnQWbKToEFEQbrOudmc3vSXqK17yjjedTnI4tz0D5jPgefyi5ByAdt QOz5wc+wHBKQHNnc0jghzW3+s7e9YpCJx6BPF46bf9YynnPkshiaKLaFDEdkGh8LOhqO cj/AfRrj+Jf1CDyA43agq/513mE2GrwrchVRhTVsaQXt18BpOMThcZk7+SLDTDSTCX+A xQ8w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=ZZIhiZEVQ2AxfjzmXXYKf6LRw34rLcHgzjMNXP5Mdl4=; b=ucFRIuGKc9qb94jBfkWsq93DOTwZsTvsCPFYXPRmG4dl+7F+pZAfsRK+xezLDSsU7f IybzhCX/BkLfQivWBX+u1pmWE2rEogzT1QWF8OZgB5Qdxe1IL/zCtEbbky/xjNzgfBk4 OYB2joriIHqk4KzbK2fvcJCrgqhhYE/LKu97Ra6NuGixw9WIsvseHSY8KQKzctebjG4v 2k+/5oxDjZDgbQ3W+KIXUYyHoXb0V+K/L2nMLRTI1Q8UickP8LyLAv8f+4JVyPnE3bkc gN20O6ixHKO+BRuTIxA7jiNFjBnuN2BkzXYJpP6IFbzVXO/XsU9nFV7woKNZ48Beun/J sXwA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lNsU4iIY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m18si3137635ilj.155.2021.04.30.14.24.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:24:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lNsU4iIY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50602 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcacq-0002FC-E8 for patch@linaro.org; Fri, 30 Apr 2021 17:24:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53678) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmg-0004yt-9P for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:46 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]:42751) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlw-0007mo-NP for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:45 -0400 Received: by mail-pf1-x434.google.com with SMTP id b27so905658pfp.9 for ; Fri, 30 Apr 2021 13:29:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ZZIhiZEVQ2AxfjzmXXYKf6LRw34rLcHgzjMNXP5Mdl4=; b=lNsU4iIY2mWlrpwlei/lP3DCaLEg/i3TnvWIQK840Dsx2L0tI+oum6QzQi2MgLTDEx 1G1fiFFsO+LnV3SLZGem9UXehPdSNIqFrSBHROvtlKkfXv9Zz5FpKNAG+3Irhorxlzqa O5DK8rFsMQcAbFlJk/xUaMjwBSmOjuQPTyKlM73xFOtOaLChYntCpW1dTtKXDfZ8xXQT whP814DDbIosDjEKmeph3DblajCsXy4v2Htz6TCuK0dPjvXotDWUyqDUP80IaDC8y7Xz /tzGgt3YJt8gt3tDeMJndI2GmBxJ9tFrZah1G1MCtCBXIpu20HGJkpye4h7sGgaBPfda vgHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ZZIhiZEVQ2AxfjzmXXYKf6LRw34rLcHgzjMNXP5Mdl4=; b=bPoTNjGLif1V4yvL6OfzeEsZaxsp6azfhTMvtFbLotWx8Vf7eW8fcsbe2VQp91UHTx HTmqSsRwI7dgkP+ttk8IbQTsEpkwn2xp51A2x+fvu5pPs0Lk8boaXlIwL0EKWRSiM5MW BNSWiPH2ZVBw9Ov4BhKli9/guRgDFRviWG8E6rsoTHLX0Bgn6nHhqGJJPFgtkfLvSgV+ eoXxAoMqQJ2anlMprYQUhxcfUGBWEzNMOWrYeGMTGB6g42KYafiOPhsYIPE+YR2LCE8D GvbnyL70YR/UznxQPvnPummnaMZs17i8kIbeES4GXoV7Eo1u38xHULwy7otqNLeqyozy pLyA== X-Gm-Message-State: AOAM533QKNuHGPgGxO4fwhBHNnJoeAx1iwFdWmy4Ry7USO72f0xYjXeE 6z8lyAUhLzWgIl01rtUVGl4fMI0yWg7QtQ== X-Received: by 2002:a63:1624:: with SMTP id w36mr6094220pgl.240.1619814591093; Fri, 30 Apr 2021 13:29:51 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:50 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 64/82] target/arm: Implement SVE2 TBL, TBX Date: Fri, 30 Apr 2021 13:25:52 -0700 Message-Id: <20210430202610.1136687-65-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428144352.9275-1-steplong@quicinc.com> [rth: rearrange the macros a little and rebase] Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 10 +++++ target/arm/sve.decode | 5 +++ target/arm/sve_helper.c | 90 ++++++++++++++++++++++++++++++-------- target/arm/translate-sve.c | 33 ++++++++++++++ 4 files changed, 119 insertions(+), 19 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 0be0d90bee..41c08a963b 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -661,6 +661,16 @@ DEF_HELPER_FLAGS_4(sve_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_tbl_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_tbx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_tbx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_3(sve_sunpk_h, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_s, TCG_CALL_NO_RWG, void, ptr, ptr, i32) DEF_HELPER_FLAGS_3(sve_sunpk_d, TCG_CALL_NO_RWG, void, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 7a2770cb0c..38aaf1b37e 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -558,6 +558,11 @@ TBL 00000101 .. 1 ..... 001100 ..... ..... @rd_rn_rm # SVE unpack vector elements UNPK 00000101 esz:2 1100 u:1 h:1 001110 rn:5 rd:5 +# SVE2 Table Lookup (three sources) + +TBL_sve2 00000101 .. 1 ..... 001010 ..... ..... @rd_rn_rm +TBX 00000101 .. 1 ..... 001011 ..... ..... @rd_rn_rm + ### SVE Permute - Predicates Group # SVE permute predicate elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index e8a8425522..8dc04441aa 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2915,28 +2915,80 @@ void HELPER(sve_rev_d)(void *vd, void *vn, uint32_t desc) } } -#define DO_TBL(NAME, TYPE, H) \ -void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ -{ \ - intptr_t i, opr_sz = simd_oprsz(desc); \ - uintptr_t elem = opr_sz / sizeof(TYPE); \ - TYPE *d = vd, *n = vn, *m = vm; \ - ARMVectorReg tmp; \ - if (unlikely(vd == vn)) { \ - n = memcpy(&tmp, vn, opr_sz); \ - } \ - for (i = 0; i < elem; i++) { \ - TYPE j = m[H(i)]; \ - d[H(i)] = j < elem ? n[H(j)] : 0; \ - } \ +typedef void tb_impl_fn(void *, void *, void *, void *, uintptr_t, bool); + +static inline void do_tbl1(void *vd, void *vn, void *vm, uint32_t desc, + bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); + + if (unlikely(vd == vn)) { + vn = memcpy(&scratch, vn, oprsz); + } + + fn(vd, vn, NULL, vm, oprsz, is_tbx); } -DO_TBL(sve_tbl_b, uint8_t, H1) -DO_TBL(sve_tbl_h, uint16_t, H2) -DO_TBL(sve_tbl_s, uint32_t, H4) -DO_TBL(sve_tbl_d, uint64_t, ) +static inline void do_tbl2(void *vd, void *vn0, void *vn1, void *vm, + uint32_t desc, bool is_tbx, tb_impl_fn *fn) +{ + ARMVectorReg scratch; + uintptr_t oprsz = simd_oprsz(desc); -#undef TBL + if (unlikely(vd == vn0)) { + vn0 = memcpy(&scratch, vn0, oprsz); + if (vd == vn1) { + vn1 = vn0; + } + } else if (unlikely(vd == vn1)) { + vn1 = memcpy(&scratch, vn1, oprsz); + } + + fn(vd, vn0, vn1, vm, oprsz, is_tbx); +} + +#define DO_TB(SUFF, TYPE, H) \ +static inline void do_tb_##SUFF(void *vd, void *vt0, void *vt1, \ + void *vm, uintptr_t oprsz, bool is_tbx) \ +{ \ + TYPE *d = vd, *tbl0 = vt0, *tbl1 = vt1, *indexes = vm; \ + uintptr_t i, nelem = oprsz / sizeof(TYPE); \ + for (i = 0; i < nelem; ++i) { \ + TYPE index = indexes[H1(i)], val = 0; \ + if (index < nelem) { \ + val = tbl0[H(index)]; \ + } else { \ + index -= nelem; \ + if (tbl1 && index < nelem) { \ + val = tbl1[H(index)]; \ + } else if (is_tbx) { \ + continue; \ + } \ + } \ + d[H(i)] = val; \ + } \ +} \ +void HELPER(sve_tbl_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbl_##SUFF)(void *vd, void *vn0, void *vn1, \ + void *vm, uint32_t desc) \ +{ \ + do_tbl2(vd, vn0, vn1, vm, desc, false, do_tb_##SUFF); \ +} \ +void HELPER(sve2_tbx_##SUFF)(void *vd, void *vn, void *vm, uint32_t desc) \ +{ \ + do_tbl1(vd, vn, vm, desc, true, do_tb_##SUFF); \ +} + +DO_TB(b, uint8_t, H1) +DO_TB(h, uint16_t, H2) +DO_TB(s, uint32_t, H4) +DO_TB(d, uint64_t, ) + +#undef DO_TB #define DO_UNPK(NAME, TYPED, TYPES, HD, HS) \ void HELPER(NAME)(void *vd, void *vn, uint32_t desc) \ diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 5f2a2b1eb2..7bf8326fab 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2417,6 +2417,39 @@ static bool trans_TBL(DisasContext *s, arg_rrr_esz *a) return true; } +static bool trans_TBL_sve2(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_4 * const fns[4] = { + gen_helper_sve2_tbl_b, gen_helper_sve2_tbl_h, + gen_helper_sve2_tbl_s, gen_helper_sve2_tbl_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fns[a->esz], a->rd, a->rn, + (a->rn + 1) % 32, a->rm, 0); + } + return true; +} + +static bool trans_TBX(DisasContext *s, arg_rrr_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_tbx_b, gen_helper_sve2_tbx_h, + gen_helper_sve2_tbx_s, gen_helper_sve2_tbx_d + }; + + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzz(s, fns[a->esz], a->rd, a->rn, a->rm, 0); + } + return true; +} + static bool trans_UNPK(DisasContext *s, arg_UNPK *a) { static gen_helper_gvec_2 * const fns[4][2] = { From patchwork Fri Apr 30 20:25:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429893 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp802939jao; Fri, 30 Apr 2021 14:18:12 -0700 (PDT) X-Google-Smtp-Source: ABdhPJySPwk9CUtY1ki2TmZ0C+buUS/Ayu0GMY9CG7tHCuQqoQG+LhyhjxzX8fbn97PkKfWfnrtl X-Received: by 2002:a92:c525:: with SMTP id m5mr5484119ili.298.1619817492826; Fri, 30 Apr 2021 14:18:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817492; cv=none; d=google.com; s=arc-20160816; b=Iww63csEG2+XXD1+XcW/Q5Vt0yyB3fj7Zphgv7xgBmKa3eB/fAxxzj5WvAoi1MykAE wLR+q7xOUjN9xXOQh0yh6QeIrWzrtIGVCF8Ib/J1I6Kriv+3O4CCK95zNMeu7LSihlqx brR6gUHcDX5G6PsjO6uBTUYARYoHhZyWcaMjoL2/XZxQ3HyeziyYpzhWjTJEjKGLgbDY L704PVPieGOuVIkMVj/tgEgI+pki755IsAPbZaDM9ZUlaXGt3Ygy4zqMKxJqI1QmRoPi RJQCxz2oP9T9BlnwychhUi6fWYn6z1MpyPbqUcNE/igweaaDgtY7ZxQrROyoSlz0lzxO YsYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=lV1FkQTTe9ttlxJb25lFiBi91cAwlSKcX3Bog2LiPP4=; b=rtTFj09D2FNFGOBzxtiOwfquqEwc+OsvwkNkT2PTjlsEk8U4FgycZgm/xcdynQRfj6 L06/TcpkqiJQViesMV58WAv4XxjmNRGH3LEsaucloax7rMkKygHpva/qzrgWKYJ6be+c sMuNxI3Uu4S9bPeGNhO1OcIt19LDytlB4joUl5bbcqQUfQ19lRp36tfbjARRfyzW+3qF ucGUpw11W7treedQi+RrKxGstHheKbs5qAeaV90rdSZH5W0O8vFuRnz74Mvc91v4WHNu WTgCUEauEuxyOhj5/Q3wPbDk2wlYxN0pCIrayNESSM13W8qV2uZj3uTirTDhfvV+Lg2y 453A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WIwX628d; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d6si4076777ilv.73.2021.04.30.14.18.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:18:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=WIwX628d; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33358 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaWa-0003YR-7i for patch@linaro.org; Fri, 30 Apr 2021 17:18:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53610) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmd-0004tL-KK for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:43 -0400 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]:37495) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlw-0007n2-BT for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:43 -0400 Received: by mail-pg1-x535.google.com with SMTP id p2so34732311pgh.4 for ; Fri, 30 Apr 2021 13:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=lV1FkQTTe9ttlxJb25lFiBi91cAwlSKcX3Bog2LiPP4=; b=WIwX628dsQXHurf5DqqtFiEQ9BYxLPm1oWyTJNbATSDUk6RaWTCWbZaztgiYaig8ej 3Pp6tmdmsarCspbSVrOG6vJFdDOkr43PQ3ZBxnp6t8ifACZz5xy9AFDR7D2AKqjABVpR f9QPQqp0cKjQrhApqwn+6B+FFmOL/iypARdoKflcFjW0R2yuE8Hlwj8LApoMLXniMpLN intXhbWcgDEik8GpG1y4ZBG4oD8T3imq5tQqF1RqN4pQjnq5NqmYFvRhW9nLhx8yu1NA b5w8qsPkurauIquJlc8iB9w9DDeb1j35ozeYk581wyQtMXAsUU4D7KmXYbV3t7OOSQdu Du+Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=lV1FkQTTe9ttlxJb25lFiBi91cAwlSKcX3Bog2LiPP4=; b=I2E87VOZ0b7RDoR5e2gzqz5X3LZOJoSji3FKCt1NlfCN8+lXsPcTreUq8ApVSk/zq9 vqIipn8Umdo5sWIWNGBk35aSDLi4UiqBizzqOxcQdxezxhXxtsWck+sqbCAwelLcHVHj 1vnyBOL6PC1qUDPkKGcLBNmB6HBCjR79FTynOoXDX8tKUyI60dkTnYvs8xNoRuVzVGsX mTPjIIg7evCqU6YpBXjKBmSUlec4ag0BM1XtwjV5dGUleh7SoqLE5RofKNwVjWpGtOUc ZropBM4MFTEF5dtvPCSZ5ZO77/T5CVW1/CJMzRAg88wDTjUGl0g+77qUhMH9nRrmhtya b6Ag== X-Gm-Message-State: AOAM530FcFRj95fIVOv/Yg1lE4JTWWkcWeILi7IjuGLI3M9yn9/wba48 qkzdtv1G0X6Uz6Wmcu6W2sn8mGVyTkgzig== X-Received: by 2002:a05:6a00:1409:b029:27f:fb6a:24b5 with SMTP id l9-20020a056a001409b029027ffb6a24b5mr6716813pfu.18.1619814591686; Fri, 30 Apr 2021 13:29:51 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:51 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 65/82] target/arm: Implement SVE2 FCVTNT Date: Fri, 30 Apr 2021 13:25:53 -0700 Message-Id: <20210430202610.1136687-66-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::535; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x535.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-2-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 4 ++++ target/arm/sve_helper.c | 20 ++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 45 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 41c08a963b..d6b064bdc9 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2703,3 +2703,8 @@ DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 38aaf1b37e..afc53639ac 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1531,3 +1531,7 @@ SM4E 01000101 00 10001 1 11100 0 ..... ..... @rdn_rm_e0 # SVE2 crypto constructive binary operations SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 + +### SVE2 floating-point convert precision odd elements +FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 8dc04441aa..6164ae17cc 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7448,3 +7448,23 @@ void HELPER(fmmla_d)(void *vd, void *vn, void *vm, void *va, d[3] = float64_add(a[3], float64_add(p0, p1, status), status); } } + +#define DO_FCVTNT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEW nn = *(TYPEW *)(vn + HW(i)); \ + *(TYPEN *)(vd + HN(i + sizeof(TYPEN))) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) +DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 7bf8326fab..537449cbfa 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8170,3 +8170,19 @@ static bool trans_RAX1(DisasContext *s, arg_rrr_esz *a) } return true; } + +static bool trans_FCVTNT_sh(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_sh); +} + +static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); +} From patchwork Fri Apr 30 20:25:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429913 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp816583jao; Fri, 30 Apr 2021 14:39:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzMz0tcMw6TVZksPK9LicoLMnwUA4e0s0jUUFAh2eakg1zGmVeOWnfByaqpvEWHf+85ip68 X-Received: by 2002:a6b:5918:: with SMTP id n24mr5426553iob.93.1619818788100; Fri, 30 Apr 2021 14:39:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818788; cv=none; d=google.com; s=arc-20160816; b=UnYvp/h32bxs8T94j2SpjaHbtb91wabBmAQ0aVQK9qG9ezkPzfNYcxJZvssLZnWtSZ 1zk20eLUp1JZ7rw6uK1eB/DPGk9PdQWaAmZ7QQj/kBouRfhRlBb/F5ZJbA5gj2R6yPbu IMRYZkLYnr72T/SUir2Mz7pwuMPTCOTEiU9URY+eq89hBSYvDhq41neWlNBr1Bvc/6IR YWMbngTntl/Q/DT5td8/CnLX54rff3cLZl1Cyqf6Tlr8Ewwlbipi0965HSW2vS6FJg4B ZGQU5+RrKsEjzoN4B1GPLaB1d5l+dEc121OAUbROtyejsOKjYyiNfZmyg79uWbU9NeOm jmEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=Ux7AjIofl30XpVNkB5THnEw/ARFu5PvhGsc0ueXu0yU=; b=PHRKmBXHS+/L6wT6OrcWfXOaWNLsVR+KJz17sC91v1Vi+L7IBFipygt59YkLPV0GUd hoMKMmI+L8krsMV8iINrLgCZ7gz9zr7tESIQy6k5Bt82wQ3Yr70GJOXvqd+988h35toM 7+8kGcBShAmAKcnCbYWoMCV0geaIg3Eq9SEybyyE5qdCAPkbZCYBGbU9nHbDDHj3pyjo 96wUh1msnyx3P2GbUEtXb+dN1e+A1Rd5afgn8425CC+umdMkvCWXcqeJpY3LOvMz92FV PnEeiqE/YN/Q+HpsL/VUOHLvteybHvcRQakCqbLiMqydbjbcAL+R05PVAzumUeCA9vR6 euKQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="einjk/rK"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m6si4139310jav.15.2021.04.30.14.39.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:39:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="einjk/rK"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:36980 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcarT-0002vp-Gg for patch@linaro.org; Fri, 30 Apr 2021 17:39:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53856) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmy-00058S-0t for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:04 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]:46013) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007nB-I0 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:01 -0400 Received: by mail-pl1-x634.google.com with SMTP id p17so11327221plf.12 for ; Fri, 30 Apr 2021 13:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=Ux7AjIofl30XpVNkB5THnEw/ARFu5PvhGsc0ueXu0yU=; b=einjk/rKQ//C6rXCR9DuqbowPFXRvBt/PkA9pqn85xTQOe3BG1ohM9ltLqw0gGRotM Wksw5+kTJFM4y+tca6vgpSymeX6gyZwSGu2qaUE2HoGTht8UGLqGecCoCbGejzlpIcTp 5Eb9pG008ZR6D8DafXssfYzQQKs2g23rzSeUkVuvYaLJuSgHRlXtImIn8m+xC/4BIH4V VKEn965apV1/YrfaCN8vg3D68WZPzvNZkN34c8Zd/AR4GSPhGKqbJfribeNz5mpbZCNw xCCvsUf1juuDtJmjfbubd4gu0VoM7mxwuO2EwHlAKQP32W9eyJK8uore53RGlWKUj8mA N5mA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=Ux7AjIofl30XpVNkB5THnEw/ARFu5PvhGsc0ueXu0yU=; b=bqDQq7wwxT1Zn0g5pvlVcdSV/1mmNCZYAGaCfV1cilf2OvG7fL1lDbNHIiiPcRYxeX RnChyDDrlYg78ER6P0CWi4/0wISQM+NGxG2SWzizHzMGAPy2EJchynPTWwwnkLFR71+X JmXB5RaC095L/TlaCJe77DVNUHfpk3v1HefnAVbCqPK5VCsjyeTD81g4mygSDQIMl8il 2+iF1XFr5yHjfYV47/bC7sj0Bsavw9volPrv8+0CXlyR4AG10QZVaJbl6+awjmmK+WSQ dT+MICOHPx/+8SXxlzn2RTiocwcLPN01X4JEXqui/UuN+twValgwqEh7L7QcT6H+jt7n 5Xcg== X-Gm-Message-State: AOAM5334jDyouKydpHl/ujiCjfJZrIPx+HSregsB+AFD59PT4ajfdyZ4 42/hGWKA+OYKsgIy+SkkWw/MWoxiohiOoA== X-Received: by 2002:a17:902:82c3:b029:ed:64b9:6119 with SMTP id u3-20020a17090282c3b02900ed64b96119mr7104892plz.20.1619814592305; Fri, 30 Apr 2021 13:29:52 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 66/82] target/arm: Implement SVE2 FCVTLT Date: Fri, 30 Apr 2021 13:25:54 -0700 Message-Id: <20210430202610.1136687-67-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-3-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 5 +++++ target/arm/sve.decode | 2 ++ target/arm/sve_helper.c | 23 +++++++++++++++++++++++ target/arm/translate-sve.c | 16 ++++++++++++++++ 4 files changed, 46 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index d6b064bdc9..30b6dc49c8 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2708,3 +2708,8 @@ DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index afc53639ac..fb998f5f34 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1534,4 +1534,6 @@ RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 6164ae17cc..2684f40a62 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -7468,3 +7468,26 @@ void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ DO_FCVTNT(sve2_fcvtnt_sh, uint32_t, uint16_t, H1_4, H1_2, sve_f32_to_f16) DO_FCVTNT(sve2_fcvtnt_ds, uint64_t, uint32_t, H1_4, H1_2, float64_to_float32) + +#define DO_FCVTLT(NAME, TYPEW, TYPEN, HW, HN, OP) \ +void HELPER(NAME)(void *vd, void *vn, void *vg, void *status, uint32_t desc) \ +{ \ + intptr_t i = simd_oprsz(desc); \ + uint64_t *g = vg; \ + do { \ + uint64_t pg = g[(i - 1) >> 6]; \ + do { \ + i -= sizeof(TYPEW); \ + if (likely((pg >> (i & 63)) & 1)) { \ + TYPEN nn = *(TYPEN *)(vn + HN(i + sizeof(TYPEN))); \ + *(TYPEW *)(vd + HW(i)) = OP(nn, status); \ + } \ + } while (i & 63); \ + } while (i != 0); \ +} + +DO_FCVTLT(sve2_fcvtlt_hs, uint32_t, uint16_t, H1_4, H1_2, sve_f16_to_f32) +DO_FCVTLT(sve2_fcvtlt_sd, uint64_t, uint32_t, H1_4, H1_2, float32_to_float64) + +#undef DO_FCVTLT +#undef DO_FCVTNT diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 537449cbfa..f5bf122204 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8186,3 +8186,19 @@ static bool trans_FCVTNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FCVTLT_hs(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_hs); +} + +static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); +} From patchwork Fri Apr 30 20:25:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429898 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp805191jao; Fri, 30 Apr 2021 14:21:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzmgAq6FmMBogmY/vwMrPtdM4J9qnRNiZR1Qq+TNu2mXbDY1J0XdMU6PPCnZZK4tzhkogqi X-Received: by 2002:a05:6e02:de3:: with SMTP id m3mr5777848ilj.93.1619817712763; Fri, 30 Apr 2021 14:21:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817712; cv=none; d=google.com; s=arc-20160816; b=u5dLLSqnWX99dxk4k0gCyacyCy32cvTPVf1HwYxk7/sTjkwf9vIYBqhRF2HtsHHYrs MZXRZLy91733dQXisNEj/1mCDsqOCrNW1kMc+HjeZTmnZjvjno5Ult0rsxffMaCpWTxb XMtd8DucBYGJRJv/FAO4CG5JRckh38IYwhi+zmEOejERBafWY7DAkkUWB6Bp1P9z5z9A anAW6Of3dyNGLbXZ5S5QQejfhw+GrhNiUYMdVD2QGlb/PyBVUmVREHtesJrSCDuWvlT1 pRc4dh/Utzymj7rEXUDcWbFioGuzMQ13Z5LhquasExHDc/3hgYbwR9+jf/C1VWB/+1ZU ftuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=B/7aU0EzUxhV/kKb8WHkerqUukEIGKD6u5NHmPGNPZk=; b=Otia4LVlsokYKBTV1m8yIxPDWkL5duIs8YT7xi3FQYFkduO7VoTIEGsYltdMSwVaFG nw1aczpPYgv9laoIV+QUsXV+H9T389NU/H2pKGI6RO3089RhUpvtE0Dlcx1YTkmn05Y7 xcMe7vNRE0jpzoKJ/H3l8dHlxncgvuWCTjcF6nGC5msmXkUfJlTvHOYdEqCmQIiAocl/ 899Ru8KgLJ+AxfBiRNILB6sYSmZ9VNYAaCo8P00eoONZS9wXGMfX6F0lxCCB6BVjbIKe 2T5nj8/ujpUmXsdm+3lb+Z3sadH0jcin69i+53uqUmG2xlcKvllgnLhkE9t2y19JjPb4 kaUg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xHiPV3FA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n1si3086559ili.3.2021.04.30.14.21.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:21:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xHiPV3FA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43194 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaa8-0007gp-5v for patch@linaro.org; Fri, 30 Apr 2021 17:21:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53918) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZn5-0005DX-Fz for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:11 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]:34690) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007nI-L3 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:11 -0400 Received: by mail-pl1-x62b.google.com with SMTP id h7so8668715plt.1 for ; Fri, 30 Apr 2021 13:29:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=B/7aU0EzUxhV/kKb8WHkerqUukEIGKD6u5NHmPGNPZk=; b=xHiPV3FAQFXx3WWVf2xNqEQAoO8qxfkoqt4t+6zsWr6wGJIzWKO2oyCdxrOE6tUd8E gAvcmpoGm2cbP7b0uP6Q+A94808Hx5dNrKMq0rZ7pyTgsbIW+fQmO+jJUJT1aNfaEUdX 68PLfmemfMVlq5RtC7zC1zoKY23NI6Y4bFSaZhWGveMjcE7NE/Oh5/UX+6JEVYG8RpJb VZupCASnIyoWbTDGIeIYogg9yRS9sjSBCNl13iB2hPiVHAXA+9JSUuDzeySpltR508za 14xYIv88lXHgJWQ41eg/86xr7bd/0Ig4nrQt8CixG+sY4R+r9AhPXbwZohEESSxePG+s +pfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=B/7aU0EzUxhV/kKb8WHkerqUukEIGKD6u5NHmPGNPZk=; b=JJsLcuMf94dt/S2pi77BDX4AGPLMe7Jl0CaIYK3ENNvUTmaQbQ7BSCulhVTGCGLxDo tWY4eCGX/m63Y4YUg9S/y69mSMSaCqsATaURwyWJuOhv67nEQwbGbHF5aFURa8JDXWT+ a2gIyMwAS9k2UP6IGb5YJ2idO1KMBAUIJjnWu3icriWON5NwzLt1XF2tfPZ/oQ8/USaF vCL9+DhoxmDLfCvUVrsW+MWegaS3LBThZfSl3ujkSEhmMyx+B6k+p3dzaqr440rKI2aH wG6tD0O81PLb4k2OK/G2F2KksP7fS1/IXenApls3E1tuELExk8LZ8HcrV7wCiTpHcj2F aDag== X-Gm-Message-State: AOAM531LnR94XIf3cKlrK4RcGXr7D4akWS+7WR8w1OhW5U9Hhe+ho8qd 2nakaeJBzxZvy/BozHQxwUkYPRG4myk1Qg== X-Received: by 2002:a17:902:8604:b029:e6:60ad:6921 with SMTP id f4-20020a1709028604b02900e660ad6921mr7276222plo.15.1619814592834; Fri, 30 Apr 2021 13:29:52 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 67/82] target/arm: Implement SVE2 FCVTXNT, FCVTX Date: Fri, 30 Apr 2021 13:25:55 -0700 Message-Id: <20210430202610.1136687-68-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200428174332.17162-4-steplong@quicinc.com> [rth: Use do_frint_mode, which avoids a specific runtime helper.] Signed-off-by: Richard Henderson --- target/arm/sve.decode | 2 ++ target/arm/translate-sve.c | 49 ++++++++++++++++++++++++++++++-------- 2 files changed, 41 insertions(+), 10 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index fb998f5f34..46153d6a84 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1533,6 +1533,8 @@ SM4EKEY 01000101 00 1 ..... 11110 0 ..... ..... @rd_rn_rm_e0 RAX1 01000101 00 1 ..... 11110 1 ..... ..... @rd_rn_rm_e0 ### SVE2 floating-point convert precision odd elements +FCVTXNT_ds 01100100 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 +FCVTX_ds 01100101 00 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index f5bf122204..87e5c8ac63 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4715,11 +4715,9 @@ static bool trans_FRINTX(DisasContext *s, arg_rpr_esz *a) return do_zpz_ptr(s, a->rd, a->rn, a->pg, a->esz == MO_16, fns[a->esz - 1]); } -static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) +static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, + int mode, gen_helper_gvec_3_ptr *fn) { - if (a->esz == 0) { - return false; - } if (sve_access_check(s)) { unsigned vsz = vec_full_reg_size(s); TCGv_i32 tmode = tcg_const_i32(mode); @@ -4730,7 +4728,7 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), vec_full_reg_offset(s, a->rn), pred_full_reg_offset(s, a->pg), - status, vsz, vsz, 0, frint_fns[a->esz - 1]); + status, vsz, vsz, 0, fn); gen_helper_set_rmode(tmode, tmode, status); tcg_temp_free_i32(tmode); @@ -4741,27 +4739,42 @@ static bool do_frint_mode(DisasContext *s, arg_rpr_esz *a, int mode) static bool trans_FRINTN(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_nearest_even); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_nearest_even, frint_fns[a->esz - 1]); } static bool trans_FRINTP(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_up); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_up, frint_fns[a->esz - 1]); } static bool trans_FRINTM(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_down); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_down, frint_fns[a->esz - 1]); } static bool trans_FRINTZ(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_to_zero); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_to_zero, frint_fns[a->esz - 1]); } static bool trans_FRINTA(DisasContext *s, arg_rpr_esz *a) { - return do_frint_mode(s, a, float_round_ties_away); + if (a->esz == 0) { + return false; + } + return do_frint_mode(s, a, float_round_ties_away, frint_fns[a->esz - 1]); } static bool trans_FRECPX(DisasContext *s, arg_rpr_esz *a) @@ -8202,3 +8215,19 @@ static bool trans_FCVTLT_sd(DisasContext *s, arg_rpr_esz *a) } return do_zpz_ptr(s, a->rd, a->rn, a->pg, false, gen_helper_sve2_fcvtlt_sd); } + +static bool trans_FCVTX_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve_fcvt_ds); +} + +static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); +} From patchwork Fri Apr 30 20:25:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429908 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp811970jao; Fri, 30 Apr 2021 14:32:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy9Oc0qkY2eMwDodOWn2G7BD0LSBn7zX7fmrrMtm8TtvKj9bXQhdaRw9c4dSprJqWTN/I7O X-Received: by 2002:a05:6830:1017:: with SMTP id a23mr5135201otp.270.1619818373512; Fri, 30 Apr 2021 14:32:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818373; cv=none; d=google.com; s=arc-20160816; b=HooD+f/Hed48ku9Ai5sRvK1ydCgAUeR9z9hHp7Xfw0Q0KYXVGH+zJeu4ikTg3ZOr5s hs4ShDTy1wB9cGbGkhHWNVyQPTPJJ6KBz7rgLfwK0+/MYQc99/gKs0+BCo+y+cU3r3CW Bvr4yvgKAx6QmDCLMVQRcjBAlKlTwWYg02m7k+4ILNIxeogXzcGDmcgYrvmWyHtvFoQ0 MMMw3LvsVeRF8x+6XgjXN8ZgUjhD8addZqJFpuIBslzyCBnRn4LNcaWjN5fbDNuPepxa pBq2lUognkeS2mCMgmLQ2OQlPtN7amcRgYo2T8rQQnetEBSQvIXnXv13hnplSM2cLm/6 cnoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=fjikBWdGLG8hsmxHGbr3YnxshOOyIRVIIXUXcEjndK4=; b=ta0sz75QHIox4nHnVLeAcz8mWpeTx6ckC3PtuuPI8swKVj0cOMyoAnvgQDHmfPlorG YqRdyyECOAAP1i2KClAuMBr2miNQQmKrdWu+bIbarqR83hg6DGmF88HmnmC9ZPwRWjQj Pn1KBX9JM5nv+DkCOuxrJNTjn9oaPN6XMnfBhR+n2qy+PIJgFTpfoNz3JSvHVoD5vrWa +apdOcjGJ/keCEifzhdRDWWakIgjLMN9ioygyztW8uVn58G9C+sS0xlxk/SlshnUFCFm K/jCI5uXhaBePAUoVW3XV97c7rg8PtsxzTgcJJ7F1lZCjnrFwMdRuEwtv5oToOcU3PtZ qnhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xG6iIK1E; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j104si5802776otj.77.2021.04.30.14.32.53 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:32:53 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xG6iIK1E; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47822 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcakm-0004Gp-RI for patch@linaro.org; Fri, 30 Apr 2021 17:32:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53786) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZms-00056B-I1 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:00 -0400 Received: from mail-pg1-x52a.google.com ([2607:f8b0:4864:20::52a]:36787) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007nW-Fg for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:57 -0400 Received: by mail-pg1-x52a.google.com with SMTP id j7so40766775pgi.3 for ; Fri, 30 Apr 2021 13:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fjikBWdGLG8hsmxHGbr3YnxshOOyIRVIIXUXcEjndK4=; b=xG6iIK1ECcXW1UYf81DzPjjQMOxnsDVbXpWAYc6rfdozU6oZ0fOaB5CPlr3YtX53FH 0Pa5z11okzwxjlaaocBqhBWpdjFSMfdawCEWMxityWbRhNTCe1G6l2Ei61L4tbdtSLG1 teKPfybUAHD3waPEX2fLumAfFe8RaoW3ZnzO1JzWiowqCQW04WyhJxHgx+MCByeBqvHQ CGPNj0xJnSLcsEUD20gdm8L8CkExm0A9tQ3JUifb1NOARAKZlUgSLVeVzb9VDBx1uyar WJl92pihlm1DU5vq7N8nlh9LBkGBmT1d86Vm+kyLpDrDOkqhq0es3qRirVA3F6FJDG4L RstQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fjikBWdGLG8hsmxHGbr3YnxshOOyIRVIIXUXcEjndK4=; b=pdkcpekhmzyrk8lTODNPxwp02ZVbAPaM2Rsokr4KILy7npkdRvhdq7xN7K0bWstAou QbM9sCLt3uoJsold4S37nnGfvqa94s1uy8kdsZKpo69O6uYQe4+NAfrwINlZ1jKRcjV9 iFYaoHJ9AMLhfcSD68R6BZiYUEO6C4OTYGUM9l0/VG4MLuDkoT1VYMszvNvYGxPsxySG uK1l3yjLbk6Gc6mCINkl9zdIayrHwKaoJB8Io2vlHEau774kXcEIfZnq9V2YNk1lcARf Vb8DTQ9PONtp2kbblJFZXPGwsCQfEc8mA5lV72oKhvj5dK1oKeZu3FxmISboUY2VFaVc WxQQ== X-Gm-Message-State: AOAM533J0hkmNKxCVisW9dv9IAaGKRsvC/2LExjM3DxKWB5HlrCXCWKA Q4YiOBFWrIM9RzgdhagoUawgojxkVXOMJg== X-Received: by 2002:a62:4d86:0:b029:252:c889:2dd8 with SMTP id a128-20020a624d860000b0290252c8892dd8mr6621103pfb.41.1619814593369; Fri, 30 Apr 2021 13:29:53 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 68/82] target/arm: Implement SVE2 FLOGB Date: Fri, 30 Apr 2021 13:25:56 -0700 Message-Id: <20210430202610.1136687-69-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52a; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Signed-off-by: Stephen Long Message-Id: <20200430191405.21641-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- v2: Fixed esz index and c++ comments v3: Fixed denormal arithmetic and raise invalid. --- target/arm/helper-sve.h | 4 +++ target/arm/sve.decode | 3 +++ target/arm/sve_helper.c | 52 ++++++++++++++++++++++++++++++++++++++ target/arm/translate-sve.c | 24 ++++++++++++++++++ 4 files changed, 83 insertions(+) -- 2.25.1 diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 30b6dc49c8..96bd200e73 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2713,3 +2713,7 @@ DEF_HELPER_FLAGS_5(sve2_fcvtlt_hs, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtlt_sd, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_5(flogb_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(flogb_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 46153d6a84..17adb393ff 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1539,3 +1539,6 @@ FCVTNT_sh 01100100 10 0010 00 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_hs 01100100 10 0010 01 101 ... ..... ..... @rd_pg_rn_e0 FCVTNT_ds 01100100 11 0010 10 101 ... ..... ..... @rd_pg_rn_e0 FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 + +### SVE2 floating-point convert to integer +FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 2684f40a62..754301a3a6 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4575,6 +4575,58 @@ DO_ZPZ_FP(sve_ucvt_dh, uint64_t, , uint64_to_float16) DO_ZPZ_FP(sve_ucvt_ds, uint64_t, , uint64_to_float32) DO_ZPZ_FP(sve_ucvt_dd, uint64_t, , uint64_to_float64) +static int16_t do_float16_logb_as_int(float16 a, float_status *s) +{ + if (float16_is_normal(a)) { + return extract16(a, 10, 5) - 15; + } else if (float16_is_infinity(a)) { + return INT16_MAX; + } else if (float16_is_any_nan(a) || float16_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT16_MIN; + } else { + /* + * denormal: bias - fractional_zeros + * = bias + masked_zeros - uint32_zeros + */ + return -15 + 22 - clz32(extract16(a, 0, 10)); + } +} + +static int32_t do_float32_logb_as_int(float32 a, float_status *s) +{ + if (float32_is_normal(a)) { + return extract32(a, 23, 8) - 127; + } else if (float32_is_infinity(a)) { + return INT32_MAX; + } else if (float32_is_any_nan(a) || float32_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT32_MIN; + } else { + /* denormal (see above) */ + return -127 + 9 - clz32(extract32(a, 0, 23)); + } +} + +static int64_t do_float64_logb_as_int(float64 a, float_status *s) +{ + if (float64_is_normal(a)) { + return extract64(a, 52, 11) - 1023; + } else if (float64_is_infinity(a)) { + return INT64_MAX; + } else if (float64_is_any_nan(a) || float64_is_zero(a)) { + float_raise(float_flag_invalid, s); + return INT64_MIN; + } else { + /* denormal (see above) */ + return -1023 + 12 - clz64(extract64(a, 0, 52)); + } +} + +DO_ZPZ_FP(flogb_h, float16, H1_2, do_float16_logb_as_int) +DO_ZPZ_FP(flogb_s, float32, H1_4, do_float32_logb_as_int) +DO_ZPZ_FP(flogb_d, float64, , do_float64_logb_as_int) + #undef DO_ZPZ_FP static void do_fmla_zpzzz_h(void *vd, void *vn, void *vm, void *va, void *vg, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 87e5c8ac63..a949f53f4a 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8231,3 +8231,27 @@ static bool trans_FCVTXNT_ds(DisasContext *s, arg_rpr_esz *a) } return do_frint_mode(s, a, float_round_to_odd, gen_helper_sve2_fcvtnt_ds); } + +static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) +{ + static gen_helper_gvec_3_ptr * const fns[] = { + NULL, gen_helper_flogb_h, + gen_helper_flogb_s, gen_helper_flogb_d + }; + + if (!dc_isar_feature(aa64_sve2, s) || fns[a->esz] == NULL) { + return false; + } + if (sve_access_check(s)) { + TCGv_ptr status = + fpstatus_ptr(a->esz == MO_16 ? FPST_FPCR_F16 : FPST_FPCR); + unsigned vsz = vec_full_reg_size(s); + + tcg_gen_gvec_3_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + pred_full_reg_offset(s, a->pg), + status, vsz, vsz, 0, fns[a->esz]); + tcg_temp_free_ptr(status); + } + return true; +} From patchwork Fri Apr 30 20:25:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429909 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp812687jao; Fri, 30 Apr 2021 14:33:52 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyMlA60qU7kyAfQPE4kfImc1EvPRCGl9qNbyCuwYn1OX6t4g7T++E9TmNA9GY5CpfwhH+vn X-Received: by 2002:aca:1e16:: with SMTP id m22mr5504794oic.153.1619818432306; Fri, 30 Apr 2021 14:33:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818432; cv=none; d=google.com; s=arc-20160816; b=XXgUNtNDOVKxdINsbOeUrSaCJphKgA6KKmhibtSDNe5UyyJHprQw+nKBI2dJtxUW7b 6pNAeqY3DABc5WgJ1u84Uu+PTuplUlXMSGw6ldE9a9jHrUck0yvYNtn9QBPdsZS2Avoi 7WjR2x0ZyjRXdgnoeDOykFlGBA8Ff3hU0IByR9Y1E0/mMU0DTjPZDAZoBHdzwJH+Bkzt NaMH2NQcfVroP9xT3wkw0rdoO39pfazRdgZ2mc17y3WWYYUGkZz5zEZobvs+eVsmCxrl /2OWisRHQX+7tp8072rh3xnm2dUDKgRcBomi/asoZGH609na4I78ouFfJXf+YY82ClD9 ID7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=4gmAbhTw2CPFzXNTLSUglW0T0gkZnp8rIYwg9raCB64=; b=HxoON9tQBxtW7tPXLbq4fc1VFuS/mp7eSW8vs97oCa64r9Qc4drao32W5QtEDvMQ5+ ZMcR66fXShNfAxDFj+gADY82UvAznJl7PTqQj8vzY6kJhi+w147Io+QEWbCWRPaFKoSY M8gcqpnJi3ye6E/7LghZOWi1Bhp5khtHzLYyUU3gm65eEpWpV2fVjNNXMLezVWRYbpkG 49o0cHnQ+duxKqKD1rCrQoff3kY9zDj+mGkpPB71+7/rzOMCrkTsHP+YX7Wo1YOFcDOp u23O1c4yW6cf0YSx3o5K3o9PpVidcypJhbLOhlwU7KVT/KDIxHVdolFnxVtr2/EdytEe KlGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N0AMgebr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e3si5815949oti.274.2021.04.30.14.33.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:33:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=N0AMgebr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51108 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcalj-0005ZW-H1 for patch@linaro.org; Fri, 30 Apr 2021 17:33:51 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53962) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZn7-0005IJ-Jy for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:13 -0400 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]:38743) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007nb-RD for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:13 -0400 Received: by mail-pj1-x1035.google.com with SMTP id f11-20020a17090a638bb02901524d3a3d48so2349988pjj.3 for ; Fri, 30 Apr 2021 13:29:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=4gmAbhTw2CPFzXNTLSUglW0T0gkZnp8rIYwg9raCB64=; b=N0AMgebrFAToV899lzi2/3ia4zIMPfYdDEfc0SUQbVNgbiz6uabwwSzis6HoRK0+ik cwPBKZKusu9+ieb6gV9jiQlYn1qQufGewlRjYcWUQwgnRZgr/uno6UzGxtOe+bsBM+LT eQ9Uw4DYCEJ9dpARBl9ta1shkjzatrM4s6q+oyi/PzOVJLC96I0uwtUJ7EsPHkrXu45v E2QZu2nzb1027Px0alhLDYl6F3FHo/tz91tvzA4BIt+/hx2867TYvxwIyIqBkpciOtIf 4oPiJRwlOn0dUw507e0mgfzJXLsKDuE+b0nHzRFK/i24OU1gRWewYiksu5GPbd69WNIt Oc3A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=4gmAbhTw2CPFzXNTLSUglW0T0gkZnp8rIYwg9raCB64=; b=JKqbTu/XUDz40EhBWt5W5QXT3W3m0OJku5MVudHpG6/YNiTHk1AYKBYyMSS3FokqaM o4hkc/BN/EjCFtJr/uixUCOKiQxTiKqwHSjfAJpvRFIBn74dge/fm69uj2DG2JmL7I05 Kl29PoLBaKEn1XJ+7u2XVUjTQQNnOvHs2SqVTF09ACZTzsh31aF22G3zwCqcj014TuXV RcJohKnygVMAsFlpIBEk7jt+9WZRGMlZPwFnRxebLS9qsj8BMFFU74p3qEOLB7EWTdQ4 DQAC+lDXknkm9vShpr+W8EXTe+P+psUxsI1DwNBjRQP1oNy/ScStkMJ8OwR72i8snkR1 fb6A== X-Gm-Message-State: AOAM530NVTTdBn/Tt7BsnCtyEOkXmhwGqyBe8HL3ILInFyEr7Hlkuwtx 18u0Abr4MBVCG0JrewXbq+Wx4sqLWNxBIQ== X-Received: by 2002:a17:902:7c0f:b029:ec:f5e2:4442 with SMTP id x15-20020a1709027c0fb02900ecf5e24442mr6838798pll.81.1619814593997; Fri, 30 Apr 2021 13:29:53 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 69/82] target/arm: Share table of sve load functions Date: Fri, 30 Apr 2021 13:25:57 -0700 Message-Id: <20210430202610.1136687-70-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1035; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1035.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The table used by do_ldrq is a subset of the table used by do_ld_zpa; we can share them by passing dtype instead of msz to do_ldrq. Signed-off-by: Richard Henderson --- target/arm/translate-sve.c | 254 ++++++++++++++++++------------------- 1 file changed, 126 insertions(+), 128 deletions(-) -- 2.25.1 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index a949f53f4a..ca393164bc 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5153,128 +5153,130 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, tcg_temp_free_i32(t_desc); } +/* Indexed by [mte][be][dtype][nreg] */ +static gen_helper_gvec_mem * const ldr_fns[2][2][16][4] = { + { /* mte inactive, little-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, + gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, + { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, + gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, + { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, + gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, + + /* mte inactive, big-endian */ + { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, + gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, + { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, + gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, + { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, + gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, + { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, + gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, + + { /* mte active, little-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_le_r_mte, + gen_helper_sve_ld2hh_le_r_mte, + gen_helper_sve_ld3hh_le_r_mte, + gen_helper_sve_ld4hh_le_r_mte }, + { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_le_r_mte, + gen_helper_sve_ld2ss_le_r_mte, + gen_helper_sve_ld3ss_le_r_mte, + gen_helper_sve_ld4ss_le_r_mte }, + { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_le_r_mte, + gen_helper_sve_ld2dd_le_r_mte, + gen_helper_sve_ld3dd_le_r_mte, + gen_helper_sve_ld4dd_le_r_mte } }, + + /* mte active, big-endian */ + { { gen_helper_sve_ld1bb_r_mte, + gen_helper_sve_ld2bb_r_mte, + gen_helper_sve_ld3bb_r_mte, + gen_helper_sve_ld4bb_r_mte }, + { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hh_be_r_mte, + gen_helper_sve_ld2hh_be_r_mte, + gen_helper_sve_ld3hh_be_r_mte, + gen_helper_sve_ld4hh_be_r_mte }, + { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1ss_be_r_mte, + gen_helper_sve_ld2ss_be_r_mte, + gen_helper_sve_ld3ss_be_r_mte, + gen_helper_sve_ld4ss_be_r_mte }, + { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, + + { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, + { gen_helper_sve_ld1dd_be_r_mte, + gen_helper_sve_ld2dd_be_r_mte, + gen_helper_sve_ld3dd_be_r_mte, + gen_helper_sve_ld4dd_be_r_mte } } }, +}; + static void do_ld_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype, int nreg) { - static gen_helper_gvec_mem * const fns[2][2][16][4] = { - { /* mte inactive, little-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r, gen_helper_sve_ld2hh_le_r, - gen_helper_sve_ld3hh_le_r, gen_helper_sve_ld4hh_le_r }, - { gen_helper_sve_ld1hsu_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld2ss_le_r, - gen_helper_sve_ld3ss_le_r, gen_helper_sve_ld4ss_le_r }, - { gen_helper_sve_ld1sdu_le_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r, gen_helper_sve_ld2dd_le_r, - gen_helper_sve_ld3dd_le_r, gen_helper_sve_ld4dd_le_r } }, - - /* mte inactive, big-endian */ - { { gen_helper_sve_ld1bb_r, gen_helper_sve_ld2bb_r, - gen_helper_sve_ld3bb_r, gen_helper_sve_ld4bb_r }, - { gen_helper_sve_ld1bhu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r, gen_helper_sve_ld2hh_be_r, - gen_helper_sve_ld3hh_be_r, gen_helper_sve_ld4hh_be_r }, - { gen_helper_sve_ld1hsu_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld2ss_be_r, - gen_helper_sve_ld3ss_be_r, gen_helper_sve_ld4ss_be_r }, - { gen_helper_sve_ld1sdu_be_r, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r, gen_helper_sve_ld2dd_be_r, - gen_helper_sve_ld3dd_be_r, gen_helper_sve_ld4dd_be_r } } }, - - { /* mte active, little-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_le_r_mte, - gen_helper_sve_ld2hh_le_r_mte, - gen_helper_sve_ld3hh_le_r_mte, - gen_helper_sve_ld4hh_le_r_mte }, - { gen_helper_sve_ld1hsu_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_le_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_le_r_mte, - gen_helper_sve_ld2ss_le_r_mte, - gen_helper_sve_ld3ss_le_r_mte, - gen_helper_sve_ld4ss_le_r_mte }, - { gen_helper_sve_ld1sdu_le_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_le_r_mte, - gen_helper_sve_ld2dd_le_r_mte, - gen_helper_sve_ld3dd_le_r_mte, - gen_helper_sve_ld4dd_le_r_mte } }, - - /* mte active, big-endian */ - { { gen_helper_sve_ld1bb_r_mte, - gen_helper_sve_ld2bb_r_mte, - gen_helper_sve_ld3bb_r_mte, - gen_helper_sve_ld4bb_r_mte }, - { gen_helper_sve_ld1bhu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bsu_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bdu_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1sds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hh_be_r_mte, - gen_helper_sve_ld2hh_be_r_mte, - gen_helper_sve_ld3hh_be_r_mte, - gen_helper_sve_ld4hh_be_r_mte }, - { gen_helper_sve_ld1hsu_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1hds_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1hss_be_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1ss_be_r_mte, - gen_helper_sve_ld2ss_be_r_mte, - gen_helper_sve_ld3ss_be_r_mte, - gen_helper_sve_ld4ss_be_r_mte }, - { gen_helper_sve_ld1sdu_be_r_mte, NULL, NULL, NULL }, - - { gen_helper_sve_ld1bds_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bss_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1bhs_r_mte, NULL, NULL, NULL }, - { gen_helper_sve_ld1dd_be_r_mte, - gen_helper_sve_ld2dd_be_r_mte, - gen_helper_sve_ld3dd_be_r_mte, - gen_helper_sve_ld4dd_be_r_mte } } }, - }; gen_helper_gvec_mem *fn - = fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][nreg]; /* * While there are holes in the table, they are not @@ -5512,14 +5514,8 @@ static bool trans_LDNF1_zpri(DisasContext *s, arg_rpri_load *a) return true; } -static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) +static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) { - static gen_helper_gvec_mem * const fns[2][4] = { - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_le_r, - gen_helper_sve_ld1ss_le_r, gen_helper_sve_ld1dd_le_r }, - { gen_helper_sve_ld1bb_r, gen_helper_sve_ld1hh_be_r, - gen_helper_sve_ld1ss_be_r, gen_helper_sve_ld1dd_be_r }, - }; unsigned vsz = vec_full_reg_size(s); TCGv_ptr t_pg; TCGv_i32 t_desc; @@ -5551,7 +5547,9 @@ static void do_ldrq(DisasContext *s, int zt, int pg, TCGv_i64 addr, int msz) t_pg = tcg_temp_new_ptr(); tcg_gen_addi_ptr(t_pg, cpu_env, poff); - fns[s->be_data == MO_BE][msz](cpu_env, t_pg, addr, t_desc); + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); tcg_temp_free_ptr(t_pg); tcg_temp_free_i32(t_desc); @@ -5573,7 +5571,7 @@ static bool trans_LD1RQ_zprr(DisasContext *s, arg_rprr_load *a) TCGv_i64 addr = new_tmp_a64(s); tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), msz); tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); - do_ldrq(s, a->rd, a->pg, addr, msz); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } @@ -5583,7 +5581,7 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) if (sve_access_check(s)) { TCGv_i64 addr = new_tmp_a64(s); tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 16); - do_ldrq(s, a->rd, a->pg, addr, dtype_msz(a->dtype)); + do_ldrq(s, a->rd, a->pg, addr, a->dtype); } return true; } From patchwork Fri Apr 30 20:25:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429906 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp810246jao; Fri, 30 Apr 2021 14:30:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxiZt4AssXo7hGz5QR9RNpIMefUxRJY8X5KKPToyYC+rg0nv1vp+scnL2RRCT9JcnSqopz4 X-Received: by 2002:a05:6808:7cc:: with SMTP id f12mr2958656oij.131.1619818216947; Fri, 30 Apr 2021 14:30:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818216; cv=none; d=google.com; s=arc-20160816; b=thRWTaqD/pCBKUrupyh8GJ4YWn0MbDR3BOy+FJ+BV3Z3i7pjeHZ/H1m4HTseD8djVr 4gKdQP9Txwh9XjK0gyxCjXC3evLMLbgupV2KNqNDacsvDN7dj8QhjTAOm1YKf5ltKKuc fuLOE27g3A4AuARVTE+g2+/Qkwl3/sJVwzieKwxD45OJB76KX2rwx1GrvBr172ZkOQJd idcvdt20x/22/ThY43xvyytIhF10IRyiCMB/4q2S6oB/g03cFA8OD1nhJoNwvXY+i3iN Ll0nc5pD0043yH1jsMwpMaCvvS/mPRRQHoTpd32Qke+hQRq842+VnDfOoJerKS3Qk5eN u5Cg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=I+TBcwieytkPS2OYIsYtt2Bc7+CxcGyhXME8l9tBoxY=; b=XYyNpXVJ2MchoZYUSAYEarelrhoSTgpJ1ESnFoVewdxRVKnsI58omghuCznHbWQ8QW 5cjjgdUbie3Q5pFT9c4RtwwRk6X9ktgq6+M6QnzFxKPLgs1FIMou/OHF/Doxlgaz97/e 8FQti4damLixc3gWxiJyq5+jRtMYeVToCCStE34BUIrBqXf7WrS8HI8B7yoEfM9/8CBq dnEup7dtFMKpndTdh8tk1ZGMmQrNN3dRlSak4TlDVFjagLm8RFw7SFMBDHKz2MnoQOQd TFJEzD6xRbsWTZwB3OvRd3X9xUhwIZYw6tr+tXXuMGLOjIIdLVqJEzFdSxzdFPzWBMhC a1cw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=turO+97E; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t14si4522117otk.319.2021.04.30.14.30.16 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:30:16 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=turO+97E; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:38014 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaiG-0000Bl-3i for patch@linaro.org; Fri, 30 Apr 2021 17:30:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53738) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZml-00051y-HF for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:51 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:45596) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007nj-2y for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:51 -0400 Received: by mail-pj1-x1033.google.com with SMTP id gc22-20020a17090b3116b02901558435aec1so2375243pjb.4 for ; Fri, 30 Apr 2021 13:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=I+TBcwieytkPS2OYIsYtt2Bc7+CxcGyhXME8l9tBoxY=; b=turO+97EA0Uxp80NQc82rF2bja7TdY8v5e1KWIEUYF6qbzkt+a9vCsEDl5FrfqRPbp ILT883tYcTn6+1mCqE2rG6LAgF9vMHUX/eaby+Pgfd97g/ZGlZtuDqJ9dnfUTpJrGJHL NBcXTzdNJuHN7M1u+dHfVip4xpGXYPcH+SXV3WPAF2YO1KSgxjgS4rS4NvcFnkoq+py4 4AueR3Gz2kWKnaxYmuhp0nGMjdy8G9a3WFxfBUNLIduKwaY2mGOC16mYhKf96/SV+daB JjZbTsXpJ/VTukLr15jjuLl9R9h1jjKzJn3OgVuwH/kVJKE21c9DCtLyolNWNAK1X/lD kCow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=I+TBcwieytkPS2OYIsYtt2Bc7+CxcGyhXME8l9tBoxY=; b=ctZxolPRidE4+ym1FWY284uzC8M7bIJFQX0YpZPVfoNx2Bt7HYZhw/YABkjVJmm9Qo YdQQgL7p8yrSmydwv4GpDizmtJ9r8xTgrqewCPFWauWiRVwazBQDLjSDh35s9deVO3cs 9t0Ph5pyYLkgZ8Q5oVkhtyU9FyjlN/C4voGdq2uVGudNn3dBqDIl5vFG9tC5YfdaDG0e Eq9WAZVTXwZc6JA5swX344k/6Bz/R8FxlyfsRJDV7tflHbwryyCFQP5pwdS+xOa9yrmW e/MmEPca1ltFcK6nlNeWY+KRMszkTCC5vhgtQgWdkn0yOmi7GCObi7ZMM/MdSzp9SANy BvGQ== X-Gm-Message-State: AOAM531fz5ENh2k+R1+vtgOgG7EeUyVJmrK/qeE6BitAhfs/EMzSWBD5 r7WkNGMs4SfJSnjWXgL7OVOIOY/Mw07+Xg== X-Received: by 2002:a17:903:3106:b029:e9:15e8:250e with SMTP id w6-20020a1709033106b02900e915e8250emr6933921plc.33.1619814594579; Fri, 30 Apr 2021 13:29:54 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 70/82] target/arm: Implement SVE2 LD1RO Date: Fri, 30 Apr 2021 13:25:58 -0700 Message-Id: <20210430202610.1136687-71-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/sve.decode | 4 ++ target/arm/translate-sve.c | 97 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 101 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 17adb393ff..df870ce23b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1077,11 +1077,15 @@ LD_zpri 1010010 .. nreg:2 0.... 111 ... ..... ..... @rpri_load_msz # SVE load and broadcast quadword (scalar plus scalar) LD1RQ_zprr 1010010 .. 00 ..... 000 ... ..... ..... \ @rprr_load_msz nreg=0 +LD1RO_zprr 1010010 .. 01 ..... 000 ... ..... ..... \ + @rprr_load_msz nreg=0 # SVE load and broadcast quadword (scalar plus immediate) # LD1RQB, LD1RQH, LD1RQS, LD1RQD LD1RQ_zpri 1010010 .. 00 0.... 001 ... ..... ..... \ @rpri_load_msz nreg=0 +LD1RO_zpri 1010010 .. 01 0.... 001 ... ..... ..... \ + @rpri_load_msz nreg=0 # SVE 32-bit gather prefetch (scalar plus 32-bit scaled offsets) PRF 1000010 00 -1 ----- 0-- --- ----- 0 ---- diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index ca393164bc..8a4eb8542f 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -5586,6 +5586,103 @@ static bool trans_LD1RQ_zpri(DisasContext *s, arg_rpri_load *a) return true; } +static void do_ldro(DisasContext *s, int zt, int pg, TCGv_i64 addr, int dtype) +{ + unsigned vsz = vec_full_reg_size(s); + unsigned vsz_r32; + TCGv_ptr t_pg; + TCGv_i32 t_desc; + int desc, poff, doff; + + if (vsz < 32) { + /* + * Note that this UNDEFINED check comes after CheckSVEEnabled() + * in the ARM pseudocode, which is the sve_access_check() done + * in our caller. We should not now return false from the caller. + */ + unallocated_encoding(s); + return; + } + + /* Load the first octaword using the normal predicated load helpers. */ + + poff = pred_full_reg_offset(s, pg); + if (vsz > 32) { + /* + * Zero-extend the first 32 bits of the predicate into a temporary. + * This avoids triggering an assert making sure we don't have bits + * set within a predicate beyond VQ, but we have lowered VQ to 2 + * for this load operation. + */ + TCGv_i64 tmp = tcg_temp_new_i64(); +#ifdef HOST_WORDS_BIGENDIAN + poff += 4; +#endif + tcg_gen_ld32u_i64(tmp, cpu_env, poff); + + poff = offsetof(CPUARMState, vfp.preg_tmp); + tcg_gen_st_i64(tmp, cpu_env, poff); + tcg_temp_free_i64(tmp); + } + + t_pg = tcg_temp_new_ptr(); + tcg_gen_addi_ptr(t_pg, cpu_env, poff); + + desc = simd_desc(32, 32, zt); + t_desc = tcg_const_i32(desc); + + gen_helper_gvec_mem *fn + = ldr_fns[s->mte_active[0]][s->be_data == MO_BE][dtype][0]; + fn(cpu_env, t_pg, addr, t_desc); + + tcg_temp_free_ptr(t_pg); + tcg_temp_free_i32(t_desc); + + /* + * Replicate that first octaword. + * The replication happens in units of 32; if the full vector size + * is not a multiple of 32, the final bits are zeroed. + */ + doff = vec_full_reg_offset(s, zt); + vsz_r32 = QEMU_ALIGN_DOWN(vsz, 32); + if (vsz >= 64) { + tcg_gen_gvec_dup_mem(5, doff + 32, doff, vsz_r32 - 32, vsz - 32); + } else if (vsz > vsz_r32) { + /* Nop move, with side effect of clearing the tail. */ + tcg_gen_gvec_mov(MO_64, doff, doff, vsz_r32, vsz); + } +} + +static bool trans_LD1RO_zprr(DisasContext *s, arg_rprr_load *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + if (a->rm == 31) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_shli_i64(addr, cpu_reg(s, a->rm), dtype_msz(a->dtype)); + tcg_gen_add_i64(addr, addr, cpu_reg_sp(s, a->rn)); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + +static bool trans_LD1RO_zpri(DisasContext *s, arg_rpri_load *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + TCGv_i64 addr = new_tmp_a64(s); + tcg_gen_addi_i64(addr, cpu_reg_sp(s, a->rn), a->imm * 32); + do_ldro(s, a->rd, a->pg, addr, a->dtype); + } + return true; +} + /* Load and broadcast element. */ static bool trans_LD1R_zpri(DisasContext *s, arg_rpri_load *a) { From patchwork Fri Apr 30 20:25:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429897 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp805168jao; Fri, 30 Apr 2021 14:21:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxtI9KzuSG+/wJtbAbh6xj38GpKDew3PM3+YbdbtqLorI1PgTLHHngrrele3QEjzmnmMiFZ X-Received: by 2002:a02:818b:: with SMTP id n11mr6977603jag.62.1619817711457; Fri, 30 Apr 2021 14:21:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817711; cv=none; d=google.com; s=arc-20160816; b=IoeOVgYlsrfKRyNKZsnZrLVDVFDP9jZV8KxYdu12KMI66mqBntF6r+ZmTlJhDhMbtS 7W81xnuHU11wuK2BgIC7kTpzx7ORebVCO1mevTo+ojcQ+oDb2gmiG2JigoqMYk8aG03r IGG8NuTuQH72oRkvzrdIpWqWBZNOMWWcmH7OZUAIiU6dJMLxf6LcqeQ/E3vZAVYBxWrW 9x5QHDIxJxaYmNTJtx/tAgmV+6cSdTKRu6Gs4FcoOtys2+xu1ymxIZAYfvma/wK803WY fqZUXCGz9mGcPNROX3ofhBXldnouCtIq8TINC1cWdS55fy4UUUYyn0iBnr3qWpHR1ZYW XV3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=FKg75aAFmN25FAqgzIOqHHzocUDdmnMr2DmC+4pZfoE=; b=ZMs7dfxDf7yU/Y064xNV1wO8xPxFe55rKXTtw92a/Ee8um2Xco43tQ6miYPZJat48i G1bB9OfR7PPdNEJs+PqTRjx0V1XnIOB1U625R2MHNj4vBP+sIszj25hUuITDmXFpkGrc L6bc0R80XpHDG1yIBXrVbLt0OZNG54WaCe/WbXYeznyZrMUu9aHcFmFe/zVjK5+m4mOz Z/CbJoTe4UifqoncKHMnV6dxu6xnQ1yDPYykZwXF8m2xFCMQnI3/1Br+n3nqGforq9vi j4oUcrCjWJTyQSwaTfTDZ8v0adOmAJG407ORscteB75E1wvpi8KYNz1uJ0xQZ3Z0ASy4 TlIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZuYn00yA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x4si4753257iow.8.2021.04.30.14.21.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:21:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZuYn00yA; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:43056 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaa6-0007dj-Tm for patch@linaro.org; Fri, 30 Apr 2021 17:21:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53620) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZme-0004tu-08 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:44 -0400 Received: from mail-pj1-x102e.google.com ([2607:f8b0:4864:20::102e]:33561) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlw-0007nu-QB for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:43 -0400 Received: by mail-pj1-x102e.google.com with SMTP id b14-20020a17090a6e0eb0290155c7f6a356so6166566pjk.0 for ; Fri, 30 Apr 2021 13:29:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=FKg75aAFmN25FAqgzIOqHHzocUDdmnMr2DmC+4pZfoE=; b=ZuYn00yAKRYSGvlpEJ/WH0g1rFI3fec/4AnGo1GDWMCFVmMKiFkM4MuN9u9j9sr8xo IGqBDbugYpYIGCwSRoNQpGcTfDzFqdSdqiqu++n0MBRpZam6S9k2Q7jT0jeN3Aaf1pho /4P2hw8E/50iU9YTQ8oq0rMs2y2LB8NdSJrJFkuu9WJ35/6vvniwpj3tool8fECCbhvk pomhVCcacLlt1Vye75YTUHFooorEo9BZ+8rFn/2tBmA4A+8WIrkVf+hO25GFvBDC3e9B Zua9GcvcyJT0mc/J/UCrMEE0bOe1b91FNoeCgW0s2AzNds76YtXEKmuSDHcu8OoWVYnN qdxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=FKg75aAFmN25FAqgzIOqHHzocUDdmnMr2DmC+4pZfoE=; b=Z6YXzOeRT7ghbIA6vNn4UivZU42EAoCeT/fq02rF0UARv5+HF3m5a8yDhH592N9ide aDVjAYLL5C0qafV2Rsnh8mnMKzBv0o3L1+houkKlcvNzUt9eWL2Qom3ORHtdB3jEXRmb JufzcCAhWVKLvcCCIDH8cYIgokvHmDHFxuOdfL80qQE5aKiE8wCA4q8XEC/vPVL+LoNt FWsp2ZJ837EM6DfkSlaoBeRnm0F7eEpggZ1YQrMjUCxZGOY7oMsh9USkrHmjI4aeL1UP 5NxHt7mDeokJS7NaRbEf1jFSILYRJ3Am4O22fJEJMIEoGYyxrAZQUyUJw1rS0HTgy8an qGTQ== X-Gm-Message-State: AOAM531h7B2ONLvg8TSUda3tZyomKuNxoPgRAvPJJmSWcQSnnZQStp2q 8fORKnAkT6BEE6H41ehABhwnHqd7vzMTSw== X-Received: by 2002:a17:902:e353:b029:ed:866b:7624 with SMTP id p19-20020a170902e353b02900ed866b7624mr7315170plc.25.1619814595232; Fri, 30 Apr 2021 13:29:55 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 71/82] target/arm: Implement 128-bit ZIP, UZP, TRN Date: Fri, 30 Apr 2021 13:25:59 -0700 Message-Id: <20210430202610.1136687-72-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102e; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 3 ++ target/arm/sve.decode | 8 ++++++ target/arm/sve_helper.c | 29 +++++++++++++------ target/arm/translate-sve.c | 58 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 8 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 96bd200e73..6e9479800d 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -689,16 +689,19 @@ DEF_HELPER_FLAGS_4(sve_zip_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_zip_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_zip_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_uzp_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uzp_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_trn_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_trn_q, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(sve_compact_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) diff --git a/target/arm/sve.decode b/target/arm/sve.decode index df870ce23b..32e11301a5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -590,6 +590,14 @@ UZP2_z 00000101 .. 1 ..... 011 011 ..... ..... @rd_rn_rm TRN1_z 00000101 .. 1 ..... 011 100 ..... ..... @rd_rn_rm TRN2_z 00000101 .. 1 ..... 011 101 ..... ..... @rd_rn_rm +# SVE2 permute vector segments +ZIP1_q 00000101 10 1 ..... 000 000 ..... ..... @rd_rn_rm_e0 +ZIP2_q 00000101 10 1 ..... 000 001 ..... ..... @rd_rn_rm_e0 +UZP1_q 00000101 10 1 ..... 000 010 ..... ..... @rd_rn_rm_e0 +UZP2_q 00000101 10 1 ..... 000 011 ..... ..... @rd_rn_rm_e0 +TRN1_q 00000101 10 1 ..... 000 110 ..... ..... @rd_rn_rm_e0 +TRN2_q 00000101 10 1 ..... 000 111 ..... ..... @rd_rn_rm_e0 + ### SVE Permute - Predicated Group # SVE compress active elements diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index 754301a3a6..d5701cb4e8 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -3338,36 +3338,45 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(2 * i + 0)) = *(TYPE *)(vn + H(i)); \ *(TYPE *)(vd + H(2 * i + sizeof(TYPE))) = *(TYPE *)(vm + H(i)); \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_ZIP(sve_zip_b, uint8_t, H1) DO_ZIP(sve_zip_h, uint16_t, H1_2) DO_ZIP(sve_zip_s, uint32_t, H1_4) DO_ZIP(sve_zip_d, uint64_t, ) +DO_ZIP(sve2_zip_q, Int128, ) #define DO_UZP(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ { \ intptr_t oprsz = simd_oprsz(desc); \ - intptr_t oprsz_2 = oprsz / 2; \ intptr_t odd_ofs = simd_data(desc); \ - intptr_t i; \ + intptr_t i, p; \ ARMVectorReg tmp_m; \ if (unlikely((vm - vd) < (uintptr_t)oprsz)) { \ vm = memcpy(&tmp_m, vm, oprsz); \ } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(2 * i + odd_ofs)); \ - } \ - for (i = 0; i < oprsz_2; i += sizeof(TYPE)) { \ - *(TYPE *)(vd + H(oprsz_2 + i)) = *(TYPE *)(vm + H(2 * i + odd_ofs)); \ - } \ + i = 0, p = odd_ofs; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vn + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + p -= oprsz; \ + do { \ + *(TYPE *)(vd + H(i)) = *(TYPE *)(vm + H(p)); \ + i += sizeof(TYPE), p += 2 * sizeof(TYPE); \ + } while (p < oprsz); \ + tcg_debug_assert(i == oprsz); \ } DO_UZP(sve_uzp_b, uint8_t, H1) DO_UZP(sve_uzp_h, uint16_t, H1_2) DO_UZP(sve_uzp_s, uint32_t, H1_4) DO_UZP(sve_uzp_d, uint64_t, ) +DO_UZP(sve2_uzp_q, Int128, ) #define DO_TRN(NAME, TYPE, H) \ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ @@ -3381,12 +3390,16 @@ void HELPER(NAME)(void *vd, void *vn, void *vm, uint32_t desc) \ *(TYPE *)(vd + H(i + 0)) = ae; \ *(TYPE *)(vd + H(i + sizeof(TYPE))) = be; \ } \ + if (sizeof(TYPE) == 16 && unlikely(oprsz & 16)) { \ + memset(vd + oprsz - 16, 0, 16); \ + } \ } DO_TRN(sve_trn_b, uint8_t, H1) DO_TRN(sve_trn_h, uint16_t, H1_2) DO_TRN(sve_trn_s, uint32_t, H1_4) DO_TRN(sve_trn_d, uint64_t, ) +DO_TRN(sve2_trn_q, Int128, ) #undef DO_ZIP #undef DO_UZP diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 8a4eb8542f..cb62ad36f4 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -2624,6 +2624,32 @@ static bool trans_ZIP2_z(DisasContext *s, arg_rrr_esz *a) return do_zip(s, a, true); } +static bool do_zip_q(DisasContext *s, arg_rrr_esz *a, bool high) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + unsigned high_ofs = high ? QEMU_ALIGN_DOWN(vsz, 32) / 2 : 0; + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn) + high_ofs, + vec_full_reg_offset(s, a->rm) + high_ofs, + vsz, vsz, 0, gen_helper_sve2_zip_q); + } + return true; +} + +static bool trans_ZIP1_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, false); +} + +static bool trans_ZIP2_q(DisasContext *s, arg_rrr_esz *a) +{ + return do_zip_q(s, a, true); +} + static gen_helper_gvec_3 * const uzp_fns[4] = { gen_helper_sve_uzp_b, gen_helper_sve_uzp_h, gen_helper_sve_uzp_s, gen_helper_sve_uzp_d, @@ -2639,6 +2665,22 @@ static bool trans_UZP2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, uzp_fns[a->esz]); } +static bool trans_UZP1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_uzp_q); +} + +static bool trans_UZP2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_uzp_q); +} + static gen_helper_gvec_3 * const trn_fns[4] = { gen_helper_sve_trn_b, gen_helper_sve_trn_h, gen_helper_sve_trn_s, gen_helper_sve_trn_d, @@ -2654,6 +2696,22 @@ static bool trans_TRN2_z(DisasContext *s, arg_rrr_esz *a) return do_zzz_data_ool(s, a, 1 << a->esz, trn_fns[a->esz]); } +static bool trans_TRN1_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 0, gen_helper_sve2_trn_q); +} + +static bool trans_TRN2_q(DisasContext *s, arg_rrr_esz *a) +{ + if (!dc_isar_feature(aa64_sve_f64mm, s)) { + return false; + } + return do_zzz_data_ool(s, a, 16, gen_helper_sve2_trn_q); +} + /* *** SVE Permute Vector - Predicated Group */ From patchwork Fri Apr 30 20:26:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429911 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp814532jao; Fri, 30 Apr 2021 14:36:51 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaoQltI39/gL53In3OCSVq24TMQuxYTI7Ky1ZEDeg1iQRQmppWHCaEIvpinI9fMYwiKBwS X-Received: by 2002:a05:6830:23a5:: with SMTP id m5mr5176541ots.127.1619818611334; Fri, 30 Apr 2021 14:36:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818611; cv=none; d=google.com; s=arc-20160816; b=zY+nQRQ+LhF9dixRifRJj4q9OnhEz6xNrY8oIHafoQyR6Q0OH7K1R7ZIDP6iqwlYdU 2tc3PMd5imYo6znJ4hPTJ625FIHXn9cWKLoYBcBWogHPsr96Sp3nUeZhHhoSk0vB2M+F 858quDngn4xRHWuJCmr/V/LRcq8nCSf9WP8sK6Z6HUJMfViGT4IXMRIbRBxnt4J4rzUc LFcEEt80O/PH8er/TgP0thSc5b2DRy80+3eL8ZUrOlGHrZMPJ7Vv+9exOZlFZS9LRjFY oBd3ETyjtcMGNRsFhUIT95nABIEfgeLSHW11LcLq0T9PCPOtB/qjgLP6mAERAhZHcHLe 59jg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=sLQPCzmyPNeWM67SqnAwrsV4gkne6+ynibb1m47AV8M=; b=Yo7F0kQ6b56lROrU8qrZTomvPbIOCHPMMUGNqIlI93/NCRYnEGyVuYYlToBGagU99r 5vPvcDBBSfnuOCjmXbELEbjbwIbxC0+Lnl1Cdivz0JU1OSxgmFXaxf4pH3BXYwTP3Z6I PajuPa2b66nFV2o8f+D6yqTlKnldEKsJXLiX8EWOMGtvQKWUAW+JaA1azOfc61Tv9gyI ymw4MDni1SwMjaZNNSSXM5ilnCygFgZvJv4y8EgE93BbeGk73VhVGfrvx9R0hyTJH7Q1 JgwvkWhHzn2aO85aNipizo7eqf/qsLSF+R8XQUTGdSl77Ub0K0KJBM9P83VGdgVXNt8e njZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L6V4lu/S"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l42si5102108otl.233.2021.04.30.14.36.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:36:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="L6V4lu/S"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:56494 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaoc-0007iw-MA for patch@linaro.org; Fri, 30 Apr 2021 17:36:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53784) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZms-00056A-IO for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:00 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]:33749) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007ny-33 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:58 -0400 Received: by mail-pl1-x629.google.com with SMTP id b21so8732467plz.0 for ; Fri, 30 Apr 2021 13:29:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=sLQPCzmyPNeWM67SqnAwrsV4gkne6+ynibb1m47AV8M=; b=L6V4lu/SILWRj2bwsgTb+1dpG+tADM2y/y7dfuqR89x551+14YpzBKhkARVwLiBMzn JwCer4/ErE6pPMdxIcQ0s2Z286B20cbDXllGorfKugOQPJ9Z1iUY1iKe0k84vq314tW9 Zdfk7d8ZTZz6l+UX/2hkDX4z+yGG5NbXvL68iur6VFaaxrSw8QQ6HskxJDnPaqFRuofK 4sd7mnqWn713hkY4+V95XCFhamWyZNKrTrsAj4XFW88qrCRLxbquC3NIJiG2Dp4Vfc+G kfKXZ17BaMDPo/jqDSr1dN9jyGvQMpOSehv3au1rsm1nrZfqdG5umMrBLLRx2zAgkmL/ QUGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=sLQPCzmyPNeWM67SqnAwrsV4gkne6+ynibb1m47AV8M=; b=VpF/lQyRQZDByJexcDxfVQ2PrmXcSBkLw8yFdmuI0hykv+iDesol1hltli/90WHCK+ m27v2Y1U6LT+MXoh02oqTXueCYTcCgag5igqZnekMOVz5hOGH9G4UOLYSiim6rs0q3jN tthPTRpAozAgovJiC1FBWNuUrJCPtT+e1Et+IS6OSVHcZHzC3YiWBgJWCHsQAXqm9QkU rTp4RU6wv2mYtPEn/lj7710LQuYsC5E4uCl7qNDHZlsn5+BCmCB0LiiGH5TYK+jWDuGG 0CoMCN1OVF3GSkS4O0CYbjL/FwSqai+gR549eUuA8vSAFZrEw6L6CUtNo27rmySs6eXv EV6Q== X-Gm-Message-State: AOAM530Y0lDVWZMTAVvEpXngcE+21j2IpiQCsbxBhnOQ9vC3IXz/vWk0 mEkbIRG0nzjubk48pf3iSguzF+4FW7mFdA== X-Received: by 2002:a17:902:c943:b029:ee:8f40:6225 with SMTP id i3-20020a170902c943b02900ee8f406225mr6910980pla.52.1619814595788; Fri, 30 Apr 2021 13:29:55 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 72/82] target/arm: Implement SVE2 bitwise shift immediate Date: Fri, 30 Apr 2021 13:26:00 -0700 Message-Id: <20210430202610.1136687-73-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements SQSHL/UQSHL, SRSHR/URSHR, and SQSHLU Signed-off-by: Stephen Long Message-Id: <20200430194159.24064-1-steplong@quicinc.com> Signed-off-by: Richard Henderson --- target/arm/helper-sve.h | 33 +++++++++++++++++++++ target/arm/sve.decode | 5 ++++ target/arm/sve_helper.c | 35 ++++++++++++++++++++++ target/arm/translate-sve.c | 60 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 133 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper-sve.h b/target/arm/helper-sve.h index 6e9479800d..fa7418e706 100644 --- a/target/arm/helper-sve.h +++ b/target/arm/helper-sve.h @@ -2707,6 +2707,39 @@ DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_sqdmull_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_h, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_uqshl_zpzi_d, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_srshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_srshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_urshr_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_urshr_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + +DEF_HELPER_FLAGS_4(sve2_sqshlu_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_s, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(sve2_sqshlu_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(sve2_fcvtnt_sh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(sve2_fcvtnt_ds, TCG_CALL_NO_RWG, diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 32e11301a5..cfdee8955b 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -340,6 +340,11 @@ ASR_zpzi 00000100 .. 000 000 100 ... .. ... ..... @rdn_pg_tszimm_shr LSR_zpzi 00000100 .. 000 001 100 ... .. ... ..... @rdn_pg_tszimm_shr LSL_zpzi 00000100 .. 000 011 100 ... .. ... ..... @rdn_pg_tszimm_shl ASRD 00000100 .. 000 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHL_zpzi 00000100 .. 000 110 100 ... .. ... ..... @rdn_pg_tszimm_shl +UQSHL_zpzi 00000100 .. 000 111 100 ... .. ... ..... @rdn_pg_tszimm_shl +SRSHR 00000100 .. 001 100 100 ... .. ... ..... @rdn_pg_tszimm_shr +URSHR 00000100 .. 001 101 100 ... .. ... ..... @rdn_pg_tszimm_shr +SQSHLU 00000100 .. 001 111 100 ... .. ... ..... @rdn_pg_tszimm_shl # SVE bitwise shift by vector (predicated) ASR_zpzz 00000100 .. 010 000 100 ... ..... ..... @rdn_pg_rm diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index d5701cb4e8..c5c3017745 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -2084,6 +2084,41 @@ DO_ZPZI(sve_asrd_h, int16_t, H1_2, DO_ASRD) DO_ZPZI(sve_asrd_s, int32_t, H1_4, DO_ASRD) DO_ZPZI_D(sve_asrd_d, int64_t, DO_ASRD) +/* SVE2 bitwise shift by immediate */ +DO_ZPZI(sve2_sqshl_zpzi_b, int8_t, H1, do_sqshl_b) +DO_ZPZI(sve2_sqshl_zpzi_h, int16_t, H1_2, do_sqshl_h) +DO_ZPZI(sve2_sqshl_zpzi_s, int32_t, H1_4, do_sqshl_s) +DO_ZPZI_D(sve2_sqshl_zpzi_d, int64_t, do_sqshl_d) + +DO_ZPZI(sve2_uqshl_zpzi_b, uint8_t, H1, do_uqshl_b) +DO_ZPZI(sve2_uqshl_zpzi_h, uint16_t, H1_2, do_uqshl_h) +DO_ZPZI(sve2_uqshl_zpzi_s, uint32_t, H1_4, do_uqshl_s) +DO_ZPZI_D(sve2_uqshl_zpzi_d, uint64_t, do_uqshl_d) + +DO_ZPZI(sve2_srshr_b, int8_t, H1, do_srshr) +DO_ZPZI(sve2_srshr_h, int16_t, H1_2, do_srshr) +DO_ZPZI(sve2_srshr_s, int32_t, H1_4, do_srshr) +DO_ZPZI_D(sve2_srshr_d, int64_t, do_srshr) + +DO_ZPZI(sve2_urshr_b, uint8_t, H1, do_urshr) +DO_ZPZI(sve2_urshr_h, uint16_t, H1_2, do_urshr) +DO_ZPZI(sve2_urshr_s, uint32_t, H1_4, do_urshr) +DO_ZPZI_D(sve2_urshr_d, uint64_t, do_urshr) + +#define do_suqrshl_b(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int8_t)m, 8, false, &discard); }) +#define do_suqrshl_h(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, (int16_t)m, 16, false, &discard); }) +#define do_suqrshl_s(n, m) \ + ({ uint32_t discard; do_suqrshl_bhs(n, m, 32, false, &discard); }) +#define do_suqrshl_d(n, m) \ + ({ uint32_t discard; do_suqrshl_d(n, m, false, &discard); }) + +DO_ZPZI(sve2_sqshlu_b, int8_t, H1, do_suqrshl_b) +DO_ZPZI(sve2_sqshlu_h, int16_t, H1_2, do_suqrshl_h) +DO_ZPZI(sve2_sqshlu_s, int32_t, H1_4, do_suqrshl_s) +DO_ZPZI_D(sve2_sqshlu_d, int64_t, do_suqrshl_d) + #undef DO_ASRD #undef DO_ZPZI #undef DO_ZPZI_D diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index cb62ad36f4..1421e892d4 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -1044,6 +1044,66 @@ static bool trans_ASRD(DisasContext *s, arg_rpri_esz *a) } } +static bool trans_SQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshl_zpzi_b, gen_helper_sve2_sqshl_zpzi_h, + gen_helper_sve2_sqshl_zpzi_s, gen_helper_sve2_sqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_UQSHL_zpzi(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_uqshl_zpzi_b, gen_helper_sve2_uqshl_zpzi_h, + gen_helper_sve2_uqshl_zpzi_s, gen_helper_sve2_uqshl_zpzi_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SRSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_srshr_b, gen_helper_sve2_srshr_h, + gen_helper_sve2_srshr_s, gen_helper_sve2_srshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_URSHR(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_urshr_b, gen_helper_sve2_urshr_h, + gen_helper_sve2_urshr_s, gen_helper_sve2_urshr_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + +static bool trans_SQSHLU(DisasContext *s, arg_rpri_esz *a) +{ + static gen_helper_gvec_3 * const fns[4] = { + gen_helper_sve2_sqshlu_b, gen_helper_sve2_sqshlu_h, + gen_helper_sve2_sqshlu_s, gen_helper_sve2_sqshlu_d, + }; + if (a->esz < 0 || !dc_isar_feature(aa64_sve2, s)) { + return false; + } + return do_zpzi_ool(s, a, fns[a->esz]); +} + /* *** SVE Bitwise Shift - Predicated Group */ From patchwork Fri Apr 30 20:26:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429892 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp802491jao; Fri, 30 Apr 2021 14:17:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwdaF4UaiQj+davj19+mjeBnaaYmaoEnopHK5RvPWp9JWwYt+fMGB7mXDI9ZSgHoXvhrZUk X-Received: by 2002:a92:d308:: with SMTP id x8mr5585274ila.301.1619817445160; Fri, 30 Apr 2021 14:17:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817445; cv=none; d=google.com; s=arc-20160816; b=avsvHGpJ9bzLzMViLDZUNsWTjN62oq8TomQzv1RVmTqiYv1RMrPepWYQbPmbtq52Iw YO2qBNJaXfGjVygzwx0qSV5ce2a+bABFbEjyvrOkfQ49tZeu2z/XmVn/rHeqcjisWVzz wSq4WQAiBY+UtofEwCO1+p6XVAmwTjWgipH3Rwcww/8Y9QYiWTDVyloMmAKAvPLg+upo 7MWPd2OcEewankpB7DOLzyJD04QkX0E2qfNC9B/UhfhsCN5Fze2ArON9f6P/9Q8DHjyE lqySidwF5K9e1ssd105IV+kaCORS9zApLvWEqq095uczCLrzBx/FKJNY7XpoT5zFYyNH wDWw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+CI3rO2/fAJr6W/cC9qBjDDasv8sl2L3ez2hfNlA2z8=; b=scPTGjH3aXPAHpLy1ji6g8HMWTxU0dXlWO4ZAV0VBRZC70kU0Me602H98/Yd1LZbzm TdK+uTdmO3MQd45oHOda+mLLCmWXRwXSe8bWLwwoF6suxCyrDx+Q1ezzlqzLq5JqRA5X eZSVwUQ/iga6nRf8OlzNoJT58mrMPqHQ3Q0ZbwnCcTC34U817twT94Uf4M7fRwUcRdp+ pIuht0k+96ap/1vRtlegmRJ1v8BUkE4JBJBbm6g0fdOpsM2BwMW0Tj4mHtkfIoz3RzK8 94OEssvDZCIquuD032YADTiEu7M6dUBcarQcHVl/RzFefBuGrM/lLQ1kFz+Rmd0yZTpQ yhZQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ejxOchp0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z30si4112429iow.85.2021.04.30.14.17.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:17:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ejxOchp0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:59412 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaVo-0002eA-GT for patch@linaro.org; Fri, 30 Apr 2021 17:17:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53788) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZms-00056F-JI for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:00 -0400 Received: from mail-pj1-x1034.google.com ([2607:f8b0:4864:20::1034]:51131) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007o9-3L for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:57 -0400 Received: by mail-pj1-x1034.google.com with SMTP id md17so9641461pjb.0 for ; Fri, 30 Apr 2021 13:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+CI3rO2/fAJr6W/cC9qBjDDasv8sl2L3ez2hfNlA2z8=; b=ejxOchp0ei3LtUgMBNG3+Ii3fWt7NG4uw9uqApZG2ezFScQffUGwQqbzk9i9v0NU8u DVylHcA9wmsWP2XNkRfo6bZLlVqnYKsL4ffEgvEhsA9Ch19LBQM1mQ6lZHtAwqINSyhA Dv2WA7vWpv8HbrGDQkWgn52tTmpl3CgprLDyY5p+amzd37tYv8Ye9k5mxZStypwvJdXl i0lysohQhf9uW+RYuiWGjA9ZFXN51W4kPu0/t8LX6QmcETsflQcqdpnqPO7RMSLDwrrs tJ12GX5ES/BFYd5SDnWLu06SlMCrff8kmKrudmPvOY0PDxkptFjHX3Olkv3LAIsYpQPv +f5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+CI3rO2/fAJr6W/cC9qBjDDasv8sl2L3ez2hfNlA2z8=; b=OHVp6KkL/hczg9aq3yxja7nnPoMoEiH3Lk9oPFJcLp+vXXFDbioUyQr2gB6ISxcW3j up4QaInF9R+HT+KAF96UBhMwVRgfox00ZdMdZmZJoL1qrb53Hyl0dX5wlUhUktMdRVvo V30OHWKxh0ZmLGsBbvbMsRP84aAiM+3AtIF9lTCXAyRjAlE9S3P6C+y9GG6afPNhMJ4G 7sJE4rv+ynQHkaTeFSmWSXc5T55zJFNFPrlPejtRHYyGKArrEyG5OQa2YDfgTkaw6WZk U1mcPTSs14/r6pKw3q7T9Dgda6ytLXw31MmxMAw6bCwdw9PGhnnjWxMwKxgjh7NgZD5W tHQg== X-Gm-Message-State: AOAM533cwUp13mrKihpF50Wf1wlj32S2W1b1vTZbz8xkbrS5oitRaUqs M7Tw/PbvdyVRTERadCRx0K+9sZuMoKMv6A== X-Received: by 2002:a17:90a:650c:: with SMTP id i12mr17031182pjj.204.1619814596450; Fri, 30 Apr 2021 13:29:56 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 73/82] target/arm: Implement SVE2 fp multiply-add long Date: Fri, 30 Apr 2021 13:26:01 -0700 Message-Id: <20210430202610.1136687-74-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1034; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1034.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Stephen Long Implements both vectored and indexed FMLALB, FMLALT, FMLSLB, FMLSLT Signed-off-by: Stephen Long Message-Id: <20200504171240.11220-1-steplong@quicinc.com> [rth: Rearrange to use float16_to_float32_by_bits.] Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 +++ target/arm/sve.decode | 14 +++++++ target/arm/translate-sve.c | 75 ++++++++++++++++++++++++++++++++++++++ target/arm/vec_helper.c | 51 ++++++++++++++++++++++++++ 4 files changed, 145 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index 86f938c938..e8b16a401f 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -987,6 +987,11 @@ DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(sve2_sqrdmulh_idx_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzzw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) #ifdef TARGET_AARCH64 diff --git a/target/arm/sve.decode b/target/arm/sve.decode index cfdee8955b..63870b7539 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -132,6 +132,8 @@ &rrrr_esz ra=%reg_movprfx # Four operand with unused vector element size +@rda_rn_rm_e0 ........ ... rm:5 ... ... rn:5 rd:5 \ + &rrrr_esz esz=0 ra=%reg_movprfx @rdn_ra_rm_e0 ........ ... rm:5 ... ... ra:5 rd:5 \ &rrrr_esz esz=0 rn=%reg_movprfx @@ -1559,3 +1561,15 @@ FCVTLT_sd 01100100 11 0010 11 101 ... ..... ..... @rd_pg_rn_e0 ### SVE2 floating-point convert to integer FLOGB 01100101 00 011 esz:2 0101 pg:3 rn:5 rd:5 &rpr_esz + +### SVE2 floating-point multiply-add long (vectors) +FMLALB_zzzw 01100100 10 1 ..... 10 0 00 0 ..... ..... @rda_rn_rm_e0 +FMLALT_zzzw 01100100 10 1 ..... 10 0 00 1 ..... ..... @rda_rn_rm_e0 +FMLSLB_zzzw 01100100 10 1 ..... 10 1 00 0 ..... ..... @rda_rn_rm_e0 +FMLSLT_zzzw 01100100 10 1 ..... 10 1 00 1 ..... ..... @rda_rn_rm_e0 + +### SVE2 floating-point multiply-add long (indexed) +FMLALB_zzxw 01100100 10 1 ..... 0100.0 ..... ..... @rrxr_3a esz=2 +FMLALT_zzxw 01100100 10 1 ..... 0100.1 ..... ..... @rrxr_3a esz=2 +FMLSLB_zzxw 01100100 10 1 ..... 0110.0 ..... ..... @rrxr_3a esz=2 +FMLSLT_zzxw 01100100 10 1 ..... 0110.1 ..... ..... @rrxr_3a esz=2 diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 1421e892d4..acb8603418 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8468,3 +8468,78 @@ static bool trans_FLOGB(DisasContext *s, arg_rpr_esz *a) } return true; } + +static bool do_FMLAL_zzzw(DisasContext *s, arg_rrrr_esz *a, bool sub, bool sel) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, (sel << 1) | sub, + gen_helper_sve2_fmlal_zzzw_s); + } + return true; +} + +static bool trans_FMLALB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, false); +} + +static bool trans_FMLALT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, false, true); +} + +static bool trans_FMLSLB_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, false); +} + +static bool trans_FMLSLT_zzzw(DisasContext *s, arg_rrrr_esz *a) +{ + return do_FMLAL_zzzw(s, a, true, true); +} + +static bool do_FMLAL_zzxw(DisasContext *s, arg_rrxr_esz *a, bool sub, bool sel) +{ + if (!dc_isar_feature(aa64_sve2, s)) { + return false; + } + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_4_ptr(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vec_full_reg_offset(s, a->ra), + cpu_env, vsz, vsz, + (a->index << 2) | (sel << 1) | sub, + gen_helper_sve2_fmlal_zzxw_s); + } + return true; +} + +static bool trans_FMLALB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, false); +} + +static bool trans_FMLALT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, false, true); +} + +static bool trans_FMLSLB_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, false); +} + +static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) +{ + return do_FMLAL_zzxw(s, a, true, true); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index 9b2a4d5b7e..ea08b15c55 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -29,10 +29,14 @@ so addressing units smaller than that needs a host-endian fixup. */ #ifdef HOST_WORDS_BIGENDIAN #define H1(x) ((x) ^ 7) +#define H1_2(x) ((x) ^ 6) +#define H1_4(x) ((x) ^ 4) #define H2(x) ((x) ^ 3) #define H4(x) ((x) ^ 1) #else #define H1(x) (x) +#define H1_2(x) (x) +#define H1_4(x) (x) #define H2(x) (x) #define H4(x) (x) #endif @@ -1907,6 +1911,27 @@ void HELPER(gvec_fmlal_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzzw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + sel)) ^ negn; + float16 mm_16 = *(float16 *)(vm + H1_2(i + sel)); + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i)); + + *(float32 *)(vd + H1_4(i)) = float32_muladd(nn, mm, aa, 0, status); + } +} + static void do_fmlal_idx(float32 *d, void *vn, void *vm, float_status *fpst, uint32_t desc, bool fz16) { @@ -1951,6 +1976,32 @@ void HELPER(gvec_fmlal_idx_a64)(void *vd, void *vn, void *vm, get_flush_inputs_to_zero(&env->vfp.fp_status_f16)); } +void HELPER(sve2_fmlal_zzxw_s)(void *vd, void *vn, void *vm, void *va, + void *venv, uint32_t desc) +{ + intptr_t i, j, oprsz = simd_oprsz(desc); + uint16_t negn = extract32(desc, SIMD_DATA_SHIFT, 1) << 15; + intptr_t sel = extract32(desc, SIMD_DATA_SHIFT + 1, 1) * sizeof(float16); + intptr_t idx = extract32(desc, SIMD_DATA_SHIFT + 2, 3) * sizeof(float16); + CPUARMState *env = venv; + float_status *status = &env->vfp.fp_status; + bool fz16 = get_flush_inputs_to_zero(&env->vfp.fp_status_f16); + + for (i = 0; i < oprsz; i += 16) { + float16 mm_16 = *(float16 *)(vm + i + idx); + float32 mm = float16_to_float32_by_bits(mm_16, fz16); + + for (j = 0; j < 16; j += sizeof(float32)) { + float16 nn_16 = *(float16 *)(vn + H1_2(i + j + sel)) ^ negn; + float32 nn = float16_to_float32_by_bits(nn_16, fz16); + float32 aa = *(float32 *)(va + H1_4(i + j)); + + *(float32 *)(vd + H1_4(i + j)) = + float32_muladd(nn, mm, aa, 0, status); + } + } +} + void HELPER(gvec_sshl_b)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t i, opr_sz = simd_oprsz(desc); From patchwork Fri Apr 30 20:26:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429888 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp799342jao; Fri, 30 Apr 2021 14:12:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJy6noWAXU2RTdeO+2GpcOcxa+NxiGrfsSwnNa5v+i+o/OIlCx1sgIls6sE3y7qESDrN2BaA X-Received: by 2002:a02:c6d3:: with SMTP id r19mr6727073jan.92.1619817165095; Fri, 30 Apr 2021 14:12:45 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817165; cv=none; d=google.com; s=arc-20160816; b=XtQvcZ4TkntNeCYLAznI34cfFP1SJM7DsVH4q4O5jPTuUX2nWTdXn38+trVjQU3AT9 hf3z6Y7f6AaxO8k7XgR+MBOyh41+auIUsGz1cIpLVuB1SOujXLpj7RNyG4vtXWzx3nHu xd90lDVvJjAaj09tu3031CieAkQcX3d2lQawwYmdEN3okYhfe7ZfHzZZnb0vSBeExhfg u08RTmf/ctJ45P84I6T39u/qZrccylMOZmrXfuq0XsPFZPVuOZNquwbHCHJvcuFL5HFA GoCxeisH1ePRHojk869qjV/AoTJ84AE1lgOrBDWpD3fold6TBq56M0hVJgRpz/MSBTFR 2JCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=cRF8Mv+M0NzHkJxcMg0tP1cErf1whBJVFSultKETQRM=; b=FsPEQjTH9A6NuqjxXMf0BqzDyNVOxUoDKTIkpHhDDWr3Szhagl5XUA8KYf3jkvoe08 l5iUpvzTRu302CGgGPBdrx0dsbz4XBnV1MYT89vY2RXlEOcWZo0XWlpqnsMvBmZ1QFRF uRoYV3nsS0VIB8bz3PAUUq92+/NHS3IUHmGFvxmuSyOjwXI63246TatlXreQCY8QlMZS jxlLDN5LDTmYePxjzn0t1xwO9DvNNnen7EYkpf63ktvEGdLszro9B7cAW+Gd6tsll32n it88BR0rp1uJyxQrOPIM0ZMMaLvHgvShNuxOjvy4TLlZXh6jx44PcXZ97SQfD8lEanvB PLVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JItORHXN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a21si3999878jat.63.2021.04.30.14.12.45 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:12:45 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JItORHXN; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:47722 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaRI-00066x-FR for patch@linaro.org; Fri, 30 Apr 2021 17:12:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53684) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmg-0004zs-Pg for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:46 -0400 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]:43577) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007oH-04 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:46 -0400 Received: by mail-pg1-x534.google.com with SMTP id p12so50171972pgj.10 for ; Fri, 30 Apr 2021 13:29:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cRF8Mv+M0NzHkJxcMg0tP1cErf1whBJVFSultKETQRM=; b=JItORHXNVtCDA74Dgt76waqUAe8+Mmip0wKIrv1Uxa5ZlmUPJf0XICf8pXb6jrE7y4 tOvG99ah2QCcUdQhmBhK/f9gXRRsDOuGnVXujooYf2ZCHRZ34ZNUsNLVFogiPZ6/5Nnx yLJVqROjceF8+8I798IwnjQoqoiIx3boW7rPP/OyJCV7qpVcPOqksYFReyP+1/ZlB44e Q3dfmKPfJQhTCjXTqn3WkfkxsO8/TqmHkB5SHN0wSJDubD4QETRFctGH3AT/twa4huoW i+FTgBAsmlybuUc8PAGzXUz+93+ERDylTU2Uoc4yaj/3HQDEE2hX1AFGyess8LX4Eukd 2D4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cRF8Mv+M0NzHkJxcMg0tP1cErf1whBJVFSultKETQRM=; b=gQeUuEMTfJDUvUe4THLulUt093lWtnJ2vSIPlyYGq1Xe6MZ+WNyC0dshOBp1XZtteZ DkFH1/OVrKT0rTCbHPeoibbEV61wPpF5kD0gBwP/RLWLg0QnzpOU4UNOMcZeFERvstqc iSoc2UCf7bs2Bj1hpgHoCGOQWvkOTbhbWYVtokyZEC87v06IB4pkwdrcvTsJSb36C6Ty viZQyNzWBE89svAkq/4xkCh6Ik9GCzC1/8mr9t4pLwzbKrVmVXBBJpV55qNZxGv9/QSi cTzXaYsNhkx24W5dhJYkLVh464B/2QyO+DUj9vhSUIXfidM55Q/BpsqwEkeudfRu6HZ4 Y1Cg== X-Gm-Message-State: AOAM5322U/tWrYMsboXWxFzDxcptqxSnFNjccJR41Jd1BsAfYge0lgRA TvB9hd3YxL2IOvCbgO32ZTdsTotlLhyTtQ== X-Received: by 2002:aa7:8f15:0:b029:27a:4eb9:7c9a with SMTP id x21-20020aa78f150000b029027a4eb97c9amr6617043pfr.62.1619814597009; Fri, 30 Apr 2021 13:29:57 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 74/82] target/arm: Implement aarch64 SUDOT, USDOT Date: Fri, 30 Apr 2021 13:26:02 -0700 Message-Id: <20210430202610.1136687-75-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::534; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x534.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/translate-a64.c | 25 +++++++++++++++++++++++++ 2 files changed, 30 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index c75601b221..b2b684df55 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -4206,6 +4206,11 @@ static inline bool isar_feature_aa64_rcpc_8_4(const ARMISARegisters *id) return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, LRCPC) >= 2; } +static inline bool isar_feature_aa64_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX64(id->id_aa64isar1, ID_AA64ISAR1, I8MM) != 0; +} + static inline bool isar_feature_aa64_ccidx(const ARMISARegisters *id) { return FIELD_EX64(id->id_aa64mmfr2, ID_AA64MMFR2, CCIDX) != 0; diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index a8edd2d281..c875481784 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -12175,6 +12175,13 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) } feature = dc_isar_feature(aa64_dp, s); break; + case 0x03: /* USDOT */ + if (size != MO_32) { + unallocated_encoding(s); + return; + } + feature = dc_isar_feature(aa64_i8mm, s); + break; case 0x18: /* FCMLA, #0 */ case 0x19: /* FCMLA, #90 */ case 0x1a: /* FCMLA, #180 */ @@ -12215,6 +12222,10 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b); return; + case 0x3: /* USDOT */ + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, gen_helper_gvec_usdot_b); + return; + case 0x8: /* FCMLA, #0 */ case 0x9: /* FCMLA, #90 */ case 0xa: /* FCMLA, #180 */ @@ -13360,6 +13371,13 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) return; } break; + case 0x0f: /* SUDOT, USDOT */ + if (is_scalar || (size & 1) || !dc_isar_feature(aa64_i8mm, s)) { + unallocated_encoding(s); + return; + } + size = MO_32; + break; case 0x11: /* FCMLA #0 */ case 0x13: /* FCMLA #90 */ case 0x15: /* FCMLA #180 */ @@ -13474,6 +13492,13 @@ static void disas_simd_indexed(DisasContext *s, uint32_t insn) u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b); return; + case 0x0f: /* SUDOT, USDOT */ + gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, index, + extract32(insn, 23, 1) + ? gen_helper_gvec_usdot_idx_b + : gen_helper_gvec_sudot_idx_b); + return; + case 0x11: /* FCMLA #0 */ case 0x13: /* FCMLA #90 */ case 0x15: /* FCMLA #180 */ From patchwork Fri Apr 30 20:26:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429910 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp812870jao; Fri, 30 Apr 2021 14:34:06 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyQtcy5mhZT6V8S0fFmB04D1fR+zruf5T810uha6u3gFvz9TD23Yuq09ZqpMzInMR2j01g2 X-Received: by 2002:a02:a10d:: with SMTP id f13mr6950941jag.72.1619818446485; Fri, 30 Apr 2021 14:34:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818446; cv=none; d=google.com; s=arc-20160816; b=SE5T130rGAGkpn9N7nYwpAREqlKDiyGhqW/MI8+4vYHfHsw7eIgMTJD9T8avrs6wjA A7X9v3NhR293taQsJv5rKc8F5ki4ToEvLxtTtzg4rLp+Qs34AOU5wJQnANEOZ26lXPLP ykYbYc0uNhF0vfNXWHbFuXg8u88unKXCSx21ZEJFdsNDrFP/MI5+jmjmsKBW4K5A2ZhC dRFZFNwjIjkjIQfbNkg3ee9+RwsRuzOx/7uaNyw6lF2QhSKdZ8ItMPXlCquAXdxCvbaN vcj5YmQiiFQ5rFKhb1m1euim9plthJtWGsHyIRIi4wL5wfR3irZOhCph+5giy26X/+Nb l1dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=o9jWx7r6+bkvJPhlg56uHSLWaUr8Yv1wsvob/iql19s=; b=V1CaVFR2LICqgat0POJdMlvSoDYkue8KeHku4wlFdzzHY9r7QfJJy/4nrVlgxHr6qO seClU+QalnmKIuZjdtreB+n7/cJOI4xl/qYXnGgzYfSwtffMVzsM/f6o1kLtyWgEnVCy N+GUMGI5KX6wp1TB8wFnq0cuBfamgW2SMWMLMIsVnj1vYFfm9Q9bJEDwmzloWPCPilmx 3sLBAYtYClZPj4VfbVqBdmEw4kcZkUY25q0q61rkuG6M21NmD7oKFkkNeK+IfMePFy+S SOl89kjVKFH4WYvPeJdA/srppOO6zQAqkU8ZwHZ9K/YGGblbpA2aSG77jqR8WCiR9Exq s8bw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VjfrlyZG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b14si3289029ilh.27.2021.04.30.14.34.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:34:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=VjfrlyZG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:49708 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcalx-00051j-Ry for patch@linaro.org; Fri, 30 Apr 2021 17:34:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53902) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZn0-0005BN-JT for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:10 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]:36538) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007oQ-K5 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:05 -0400 Received: by mail-pl1-x634.google.com with SMTP id a11so6351029plh.3 for ; Fri, 30 Apr 2021 13:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=o9jWx7r6+bkvJPhlg56uHSLWaUr8Yv1wsvob/iql19s=; b=VjfrlyZG9AZqpL0Jb0zSGf45KwKVUS8W6RVY/MnK8TknIDaTZcAAKIsXCWqNG8cFuC m5m4cwaX/3E2T8iF7ge2iMYjJrNOK9BLpGX6FwU7cvluggXAq74k+5Khxu1BZJ+QWAnN QOc9TTVjtHS3JQwWoJ9ArbCyKN31xKOOcSKwM4mtGiWbhRtEBu1XYXam773M1DU2z5Qm lBbZPAbXXX4gydXEXokPpjAGrjDHPgq/foZkxQMmRkrEh16rnZnMmnyvESOYrIw5DL1w Bn0o5+NwSsEDhjXjqf2DgJ7u2hvH82vAVQt7sRsfvFLQTvvdm2w25t7u6MjE1A9uaBrg Xo5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=o9jWx7r6+bkvJPhlg56uHSLWaUr8Yv1wsvob/iql19s=; b=qLRIbRpr0dJk0J4Z6+munOjwyycEdyo1k1nQ9DT76jh6ozw/LBEN/+cH+L6QxloNiX POoe/9tluCZQv4/pwJRc+U9VmJdB+RBwG5IzpmJ3TfeE7IEcRiYbkuHaeonsU1WuIARn sPcb6oy6qtJBYMkmzOGMlxe0Kcb2iQatyPVYPd1pes6puxr4xSUyXzlzEBReF0gLKPE/ zi90elwSqbaQBQBHk+Kj2WpfDVRMxQZ+QWeuAEpwA/hagL2VM1cmw2166P06xy6aRzwI +iHlnxmm+h5LylW5gdzWyslOtSdpfF9sOsdpSn57EoiMu3waDIyiZlW1dav0WXrHrdgr yeBA== X-Gm-Message-State: AOAM531SdWtaKVOWK+hGdvXKlWGKAVWpMTQGNi6eHgqLzHaxKH3HQetj /0K6kDKExHuKHHxLIbj0Ja1yrqu/vF1sHA== X-Received: by 2002:a17:90a:e54e:: with SMTP id ei14mr7523285pjb.142.1619814597644; Fri, 30 Apr 2021 13:29:57 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 75/82] target/arm: Split out do_neon_ddda_fpst Date: Fri, 30 Apr 2021 13:26:03 -0700 Message-Id: <20210430202610.1136687-76-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Split out a helper that can handle the 4-register format for helpers shared with SVE. Signed-off-by: Richard Henderson --- target/arm/translate-neon.c | 98 ++++++++++++++++--------------------- 1 file changed, 43 insertions(+), 55 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index 45fa5166f3..41a7b2208e 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -151,24 +151,21 @@ static void neon_store_element64(int reg, int ele, MemOp size, TCGv_i64 var) } } -static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) +static bool do_neon_ddda_fpst(DisasContext *s, int q, int vd, int vn, int vm, + int data, ARMFPStatusFlavour fp_flavor, + gen_helper_gvec_4_ptr *fn_gvec_ptr) { - int opr_sz; - TCGv_ptr fpst; - gen_helper_gvec_4_ptr *fn_gvec_ptr; - - if (!dc_isar_feature(aa32_vcma, s) - || (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s))) { - return false; - } - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { + if (((vd | vn | vm) & 0x10) && !dc_isar_feature(aa32_simd_r32, s)) { return false; } - if ((a->vn | a->vm | a->vd) & a->q) { + /* + * UNDEF accesses to odd registers for each bit of Q. + * Q will be 0b111 for all Q-reg instructions, otherwise + * when we have mixed Q- and D-reg inputs. + */ + if (((vd & 1) * 4 | (vn & 1) * 2 | (vm & 1)) & q) { return false; } @@ -176,20 +173,34 @@ static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) return true; } - opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - fn_gvec_ptr = (a->size == MO_16) ? - gen_helper_gvec_fcmlah : gen_helper_gvec_fcmlas; - tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - fpst, opr_sz, opr_sz, a->rot, - fn_gvec_ptr); + int opr_sz = q ? 16 : 8; + TCGv_ptr fpst = fpstatus_ptr(fp_flavor); + + tcg_gen_gvec_4_ptr(vfp_reg_offset(1, vd), + vfp_reg_offset(1, vn), + vfp_reg_offset(1, vm), + vfp_reg_offset(1, vd), + fpst, opr_sz, opr_sz, data, fn_gvec_ptr); tcg_temp_free_ptr(fpst); return true; } +static bool trans_VCMLA(DisasContext *s, arg_VCMLA *a) +{ + if (!dc_isar_feature(aa32_vcma, s)) { + return false; + } + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_neon_ddda_fpst(s, a->q * 7, a->vd, a->vn, a->vm, a->rot, + FPST_STD_F16, gen_helper_gvec_fcmlah); + } + return do_neon_ddda_fpst(s, a->q * 7, a->vd, a->vn, a->vm, a->rot, + FPST_STD, gen_helper_gvec_fcmlas); +} + static bool trans_VCADD(DisasContext *s, arg_VCADD *a) { int opr_sz; @@ -294,43 +305,20 @@ static bool trans_VFML(DisasContext *s, arg_VFML *a) static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) { - gen_helper_gvec_4_ptr *fn_gvec_ptr; - int opr_sz; - TCGv_ptr fpst; + int data = (a->index << 2) | a->rot; if (!dc_isar_feature(aa32_vcma, s)) { return false; } - if (a->size == MO_16 && !dc_isar_feature(aa32_fp16_arith, s)) { - return false; + if (a->size == MO_16) { + if (!dc_isar_feature(aa32_fp16_arith, s)) { + return false; + } + return do_neon_ddda_fpst(s, a->q * 6, a->vd, a->vn, a->vm, data, + FPST_STD_F16, gen_helper_gvec_fcmlah_idx); } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vd | a->vn) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fn_gvec_ptr = (a->size == MO_16) ? - gen_helper_gvec_fcmlah_idx : gen_helper_gvec_fcmlas_idx; - opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(a->size == MO_16 ? FPST_STD_F16 : FPST_STD); - tcg_gen_gvec_4_ptr(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - fpst, opr_sz, opr_sz, - (a->index << 2) | a->rot, fn_gvec_ptr); - tcg_temp_free_ptr(fpst); - return true; + return do_neon_ddda_fpst(s, a->q * 6, a->vd, a->vn, a->vm, data, + FPST_STD, gen_helper_gvec_fcmlas_idx); } static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) From patchwork Fri Apr 30 20:26:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429903 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp808804jao; Fri, 30 Apr 2021 14:28:00 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwvygKKwNFna2bPsaagLoI07SNUY8d+KEgPoMlRhoN9yNwBVvm2sPrLqOmckyN+upCx0QGU X-Received: by 2002:a05:6e02:214d:: with SMTP id d13mr5561575ilv.65.1619818080744; Fri, 30 Apr 2021 14:28:00 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818080; cv=none; d=google.com; s=arc-20160816; b=sxgI/SbuKwBPJn+mJzQWXLsVj92p0SYa1bJKHRKKN+1RC3JBgmOMd1RSol2lXXu7vz Bbl/ibqJZLpmhP2PXz9H2wP5wm09IylApIUgRFgc1/NO584jsbmygWtKKmC4KAsJVxiw gvVD7DFvW6tAvZKXwof7AsZrUKOvrJBt0pB3dJuYgA8PfAcFMDtjHZVCpjC9BDOQcA2U Z5sx/kzkFZkPlccP4dwqol2PTEisf8K1mVBwXpm+QqNsBVvvJPAniJgENBtYD65Qs+Se Z+H1FBw1ZbmvKDX/D6NQNEJPbE+LNSZedvHliH5eSJIUuaU8LIFBKRKy1GEfuug96PzV vRpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=wpJcy1e6R8SkvPfghmeWun8lKvG5ixv1LSGzenTGPsI=; b=H42LM83sENV/LMym+QS33QdcL0TPvi1pVwjrJFz01EJHfs+5rv5SqIZp+Wpwf8KE3O x6aEwvL/c82a+uGrxj9UQO/IzNG2+cA2n8YU9LSd9qtsSMg6GDT4ztGdjf/AL05UMGJ5 bT8L3pMqL82crDOyDGVsXqCPZPHcG0wBwW/NSO6Tie+kDzY0X69gzBjmXtkTn8Hr4JOJ hGPn/Xk+Si1bSr6Ofho9asTlMWkNdUx5bavOi0DYFwBEGji28G1xkAUo4R8FBprsZ1Sp I1tM/l8t9sG/ZWEHeBxHXRbsJY2i7IcQkyXARIgaAHq2M6OKM2LFPcG6SGsGMOsyhVbG LZbQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fualBbly; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p15si4208173ilo.83.2021.04.30.14.28.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:28:00 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fualBbly; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:58644 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcag4-0005bT-6u for patch@linaro.org; Fri, 30 Apr 2021 17:28:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53686) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmh-00050R-0j for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:47 -0400 Received: from mail-pj1-x1033.google.com ([2607:f8b0:4864:20::1033]:44852) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007oX-0N for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:46 -0400 Received: by mail-pj1-x1033.google.com with SMTP id m6-20020a17090a8586b02901507e1acf0fso2375534pjn.3 for ; Fri, 30 Apr 2021 13:29:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=wpJcy1e6R8SkvPfghmeWun8lKvG5ixv1LSGzenTGPsI=; b=fualBbly2B4omzNxg9shtHYSa0jqFzslZYTr0PmXTB9xiQBo72tmu2FvxLVAt3NmGn CeCk/iJeN4RB/y5tX2nKcZGH8Cc8lLZIOJaRY7A4fWCA40hoz3Qdju4d8dmMSHjVdkYO YSp9upAfd2WU2s4dpbj0UUgaVT8Ym4OVpC37xdKYj+7pxdhWboKOWzBLc33XV7fpXdWV lPIBKhAhurr89xNTC3ldqgfG0+obsJjQ6L9RiDWrJTFyC2ebaHn0IhYaupMBca1RuYPs OzZMAyKSxAwOjwNYteCJfSSO2uG6RCwOEJAU0OpuL/7F4phzV96bW5ugCbVPqjCXZstf OerQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=wpJcy1e6R8SkvPfghmeWun8lKvG5ixv1LSGzenTGPsI=; b=tRwixV2DH5b1rU6sBaXt41C6HvDndL35DiXi0fNAmCf1/wGR+wtBOTPJ6XUgILuWb8 JVJen0N2RVD92Gt2ZDkvFgYTPg54BucqZ4fXXxpcnddOW875p0iXL+7Nsb1mGU9PRyK7 fJ9XIXhIoFWKpVTrGElrpQceUqjJB3xSR/LwL0tM6OfF/0dLnZ+aXZq4q61c+D8/9n74 KxBvarYoav0nibBGq8bAOLzhV/85fT/FoargX1ZmQV/lSpKjjmbQkl53YWD+ZEedqjMo euk/TGMcT8EN8BSd+316o0iyEKggR/VoJgasViEMr1doh1a+SwubqAQEYvgYdA8z26JK 4K4Q== X-Gm-Message-State: AOAM532I/O515ajJyrBidsb72k0/VrQWDXAK8ozPsbO3fupbNmZxuQKG +SljTv/TKVbPDV6cgmHKqN1fsYd0bvV6kQ== X-Received: by 2002:a17:90a:c984:: with SMTP id w4mr16569921pjt.110.1619814598243; Fri, 30 Apr 2021 13:29:58 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 76/82] target/arm: Remove unused fpst from VDOT_scalar Date: Fri, 30 Apr 2021 13:26:04 -0700 Message-Id: <20210430202610.1136687-77-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1033; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1033.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Cut and paste error from another pattern. Signed-off-by: Richard Henderson --- target/arm/translate-neon.c | 3 --- 1 file changed, 3 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index 41a7b2208e..a0e267694b 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -325,7 +325,6 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { gen_helper_gvec_4 *fn_gvec; int opr_sz; - TCGv_ptr fpst; if (!dc_isar_feature(aa32_dp, s)) { return false; @@ -347,13 +346,11 @@ static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; opr_sz = (1 + a->q) * 8; - fpst = fpstatus_ptr(FPST_STD); tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), vfp_reg_offset(1, a->vn), vfp_reg_offset(1, a->rm), vfp_reg_offset(1, a->vd), opr_sz, opr_sz, a->index, fn_gvec); - tcg_temp_free_ptr(fpst); return true; } From patchwork Fri Apr 30 20:26:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429896 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp804789jao; Fri, 30 Apr 2021 14:21:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzcj9JUX6lJRl9OAA2EGTy57+9DGstvOpBg9urnQlpjXP88v02dyQ1ppD+/rlte+rPhLWSj X-Received: by 2002:a9d:30b:: with SMTP id 11mr4994749otv.298.1619817683845; Fri, 30 Apr 2021 14:21:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619817683; cv=none; d=google.com; s=arc-20160816; b=E6XIekBore9UJzvHywtzYwvcHmGFN5+IalGKeMMeyqyICTz3EfSjsIrx+LaUHMfjun i80/VHDSma7BjpqhO2WOI15zO2jV2OswAFTccfYpt4/cPC9IK/oZ8Wo2g5TsVK7lIR3k OPckaqwAe7LJHGeF62e95tV8jJ4Dhy8JA8HTzMwqmb8te45bdo1M7ttMd723OcE8Oqbj KbRyoY1NbY5WdygJMwXEapDhTOLVtfAotr/QFhaB9OUsbh3Lx+sB2Sv5h410NM11BsCH 6rwmYDQTCZfJiwOq1ZKtpls/7NU3YpqH5x1UQiESxkyKdaGWxON79u9mY4ioENtjFqLo E/0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=kpdZDVpYUbdNl8Imm7Lczl+8zIvDSosqNuzK4QWcI1g=; b=b8u9lYawAA65iLDQJ9sVdblu30RRLqj4MXn/HDNJBx4r5X4OnVcsPdFZ6wPpO0RfRg WpGOtF40qOTXNwW5JzamyQlXRQXmTX/KAulLd/LRVR8G1RYQqSICxToau5lK/flaha2h tM3mYUrEG6j5wQH74LDVcAEf/hJ0aDBY5HtQL9Nwb3WnuiPUQGe/45TNvoURCtcFYjnI t8N+B5J1u3hpcXIvE46wjkINuP2z5zLjxp1aYCJdaCtcgOytO+etv1uR0sQmfIArJ3b3 aUMTFwqZQGu1OSdKyMMmbblq2YpOeVPWh5a/I2EIycn5GOQWpVq+qTtipu96HGWAybXV H5tw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XYkNSFN9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l199si4825426oib.275.2021.04.30.14.21.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:21:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XYkNSFN9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41988 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcaZf-000772-6h for patch@linaro.org; Fri, 30 Apr 2021 17:21:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53646) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmf-0004xE-Md for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:45 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]:36780) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlw-0007oi-Vn for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:30:45 -0400 Received: by mail-pf1-x42e.google.com with SMTP id p4so1772908pfo.3 for ; Fri, 30 Apr 2021 13:29:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=kpdZDVpYUbdNl8Imm7Lczl+8zIvDSosqNuzK4QWcI1g=; b=XYkNSFN9n4cMNNKaT3b9EIefAaRzAvv5I3veEoRh388TteTKFy+89BfDEdl8/mENM6 RIUuDk1I0xly0pXO/ZzJninYSYOawoEz7yqzJ4KwGEotaxzlBGs+embmjGFY151J7NV2 b/QkcTShBQcgreKJ6aSd4ZG6m4GdkYAyueOuq0zCBMP7EAE6w8kQDXcnrajMWdsrOyjh qPs73VFdXQvPuFRCMpDAp9FzqpP0XJ7jwqVTs/C/1Et9mjrPwLYSdaXHYrqrAEIcil8q kmxK+/ODyQIS8kmPxnBSR327KHc3ExfJJlHKR5vdk6xO39YpwP6gc/5nxfaAK/snSkR5 PiBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=kpdZDVpYUbdNl8Imm7Lczl+8zIvDSosqNuzK4QWcI1g=; b=XSbGQ5yt/PFkNWcs8k+yIYR5Qu8S0KqG6R24fN8UNlV4OVGWX9g9l+38xrBvuctBt8 +YK7K0KbMI19ogzVKh416zkAf55WKiqNhm6IRcQ7d1OYAvYs0G4FlevIwM8zTRyAWWpN 6sf97I4f3S3Escxbwmn793zIS8rqc39UN34lhrd1hCx7tUyf+R4HhZ0F8I2fdINMDFeh 1jJmeULFR2pDOTQplt2qACfjd4SzOqevcvmFIzrbZokiUsfkJ29kXqsywzFFx5tVmDhc RCJ5ICHuJwrzRYlGFHSJRvza78mEqBeQ8Pibm52gzh3m7wxjMpx+p1y9hWzSNsis9I0W FDnw== X-Gm-Message-State: AOAM53352qVN7FToo0U/QBSFXmFcDX9Tkchgj/Es+w09SsQrnJ6YfQNU dZ4t+lsdM1hACRvmDcODvQRxoNx0cJGwzg== X-Received: by 2002:aa7:8103:0:b029:247:74a8:e54d with SMTP id b3-20020aa781030000b029024774a8e54dmr6540683pfi.60.1619814598857; Fri, 30 Apr 2021 13:29:58 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 77/82] target/arm: Fix decode for VDOT (indexed) Date: Fri, 30 Apr 2021 13:26:05 -0700 Message-Id: <20210430202610.1136687-78-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We were extracting the M register twice, once incorrectly as M:vm and once correctly as rm. Remove the incorrect name and remove the incorrect decode. Signed-off-by: Richard Henderson --- target/arm/neon-shared.decode | 4 +- target/arm/translate-neon.c | 90 +++++++++++++++-------------------- 2 files changed, 40 insertions(+), 54 deletions(-) -- 2.25.1 diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index ca0c699072..facb621450 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -61,8 +61,8 @@ VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=2 index=0 -VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 rm:4 \ - vm=%vm_dp vn=%vn_dp vd=%vd_dp +VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 vm:4 \ + vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 %vfml_scalar_q1_index 5:1 3:1 diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index a0e267694b..52b75ff76f 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -151,6 +151,36 @@ static void neon_store_element64(int reg, int ele, MemOp size, TCGv_i64 var) } } +static bool do_neon_ddda(DisasContext *s, int q, int vd, int vn, int vm, + int data, gen_helper_gvec_4 *fn_gvec) +{ + /* UNDEF accesses to D16-D31 if they don't exist. */ + if (((vd | vn | vm) & 0x10) && !dc_isar_feature(aa32_simd_r32, s)) { + return false; + } + + /* + * UNDEF accesses to odd registers for each bit of Q. + * Q will be 0b111 for all Q-reg instructions, otherwise + * when we have mixed Q- and D-reg inputs. + */ + if (((vd & 1) * 4 | (vn & 1) * 2 | (vm & 1)) & q) { + return false; + } + + if (!vfp_access_check(s)) { + return true; + } + + int opr_sz = q ? 16 : 8; + tcg_gen_gvec_4_ool(vfp_reg_offset(1, vd), + vfp_reg_offset(1, vn), + vfp_reg_offset(1, vm), + vfp_reg_offset(1, vd), + opr_sz, opr_sz, data, fn_gvec); + return true; +} + static bool do_neon_ddda_fpst(DisasContext *s, int q, int vd, int vn, int vm, int data, ARMFPStatusFlavour fp_flavor, gen_helper_gvec_4_ptr *fn_gvec_ptr) @@ -241,35 +271,13 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) static bool trans_VDOT(DisasContext *s, arg_VDOT *a) { - int opr_sz; - gen_helper_gvec_4 *fn_gvec; - if (!dc_isar_feature(aa32_dp, s)) { return false; } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn | a->vm) & 0x10)) { - return false; - } - - if ((a->vn | a->vm | a->vd) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - opr_sz = (1 + a->q) * 8; - fn_gvec = a->u ? gen_helper_gvec_udot_b : gen_helper_gvec_sdot_b; - tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->vm), - vfp_reg_offset(1, a->vd), - opr_sz, opr_sz, 0, fn_gvec); - return true; + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + a->u + ? gen_helper_gvec_udot_b + : gen_helper_gvec_sdot_b); } static bool trans_VFML(DisasContext *s, arg_VFML *a) @@ -323,35 +331,13 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) { - gen_helper_gvec_4 *fn_gvec; - int opr_sz; - if (!dc_isar_feature(aa32_dp, s)) { return false; } - - /* UNDEF accesses to D16-D31 if they don't exist. */ - if (!dc_isar_feature(aa32_simd_r32, s) && - ((a->vd | a->vn) & 0x10)) { - return false; - } - - if ((a->vd | a->vn) & a->q) { - return false; - } - - if (!vfp_access_check(s)) { - return true; - } - - fn_gvec = a->u ? gen_helper_gvec_udot_idx_b : gen_helper_gvec_sdot_idx_b; - opr_sz = (1 + a->q) * 8; - tcg_gen_gvec_4_ool(vfp_reg_offset(1, a->vd), - vfp_reg_offset(1, a->vn), - vfp_reg_offset(1, a->rm), - vfp_reg_offset(1, a->vd), - opr_sz, opr_sz, a->index, fn_gvec); - return true; + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + a->u + ? gen_helper_gvec_udot_idx_b + : gen_helper_gvec_sdot_idx_b); } static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) From patchwork Fri Apr 30 20:26:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429907 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp811273jao; Fri, 30 Apr 2021 14:31:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJymxoz3EOGuJB59IVjm2P/ohkheYuz3+WKQbzlz7fRyGlMxAjH+WnxBZMjs1HBGppi9aVi5 X-Received: by 2002:a05:6830:1016:: with SMTP id a22mr2491765otp.98.1619818307901; Fri, 30 Apr 2021 14:31:47 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818307; cv=none; d=google.com; s=arc-20160816; b=AFJHpc86iQh8g75Uqw7/9vrad4a/VhdLtYV2V3BHrr97/WH7uYadyQNjqaoOTMm3Pz btgva0Il31BcErlyq9I5XLNSm/yQKDmSNlhjqtnrSBju7WGMoA7sXzVP33Kj3iC9ebJP v1+vM9b5PlGLWTB26t+1/BblVny3Ne2FXIbktmgB9quGgSj3LpCmTv0HPB0mkKDJ/gnE xhvPRy87HVFE5fTv/6JVTUAGeAWTpylk1MBEWSlq2NATgw2db3EQ/knr+tT8L4JIs19X v0xVzZHzGyCkkzcoir5UfFCyCbv+vyjsaO565mb/leViZ/hiY3tfcLMuoUGmxb/N1o9/ UkwA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=5QZBm0ZbmeHWM8+kmZROLRH67189ZahrRUX3tXQtOLs=; b=nqnkIAWuFhuJK9cSvBHPuDLt+dSsAjhyzM+TjoAfwDEPPw//aWaTLHCPZnHC+Nr6Pg A6mQXN14zwh359Adf8jGhBf8ELBKrDrpmVqW1cgo+R8uQjrpzZuemwDh/H3Jtd9K5+Ll 7OLF6jNHIrjeWwjN7lDkjgfdQSQFRV6o368FZMblu9ToL9ayzu+tVpJxr1OdN4DMzFWa JCNwjJEat6FIweqlb89ZUJ7lSQQc/sjS8GSwM/Vd2r0/nsr3ajroVLgrCAl8P8bUIJMp UT6ovuZvzIYOTeX074oV57eRS6SR5IBiAxmDQ3owEzbH+yf5mmvZwKoDGTEoygSNSw+h ukwQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L6EBd6sv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l13si4131232oou.34.2021.04.30.14.31.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:31:47 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=L6EBd6sv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:41226 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcajj-0001Vg-7M for patch@linaro.org; Fri, 30 Apr 2021 17:31:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53832) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZmw-00057j-6G for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:02 -0400 Received: from mail-pj1-x1029.google.com ([2607:f8b0:4864:20::1029]:37423) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007ov-H6 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:01 -0400 Received: by mail-pj1-x1029.google.com with SMTP id k3-20020a17090ad083b0290155b934a295so2367029pju.2 for ; Fri, 30 Apr 2021 13:30:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=5QZBm0ZbmeHWM8+kmZROLRH67189ZahrRUX3tXQtOLs=; b=L6EBd6svZU7a1AGWporDcn5T9zOKueKfe1gG6ZJR7JfzLeV9hQc7SQ8cdchctd9ItP KgnPxbeLBYPBaN8Jgnh0ixDQPwWyprqDGwCZMIR8YqCD5CNdHFzeWuNOlx+wlrhqBerR HxnJGJoVNz1zvt2lvmPpsKjCboKpdVSKHY9qy/Upin3p41/iSxtvOAMfDaumclqKwJkH oG4b7MDn9FEaAtXWs8B1TTQ+aY/mfUDgjZ/DJoEFSUyv90W0eV2i43mPxfHryn+457RL 4Ch8H5KFfKNGcRhYT42WH7npS+4TlKU/r0AwRWEeN4PatL4d4FWxrS5m+JwC/V2KBqFq 4Nyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=5QZBm0ZbmeHWM8+kmZROLRH67189ZahrRUX3tXQtOLs=; b=LFBuZ0OnFCLJYMX3Cqj9jkBB9TUHmonoz15CSKt9KzL0rFo42MO3o2is7ymlblLIgc S3ML1TRxd7ZvB5+WsrS1eyDofphk/w3XSUxTbsKVXPe7jjx6Zw2IGv03UnbNiY8Hj6U6 UmrL8VoFX4hN54GmjSl5itj42m7RbUTc542ENQQUTGjL/9HLP9YcVnzZOTSxnaW1jkoq RwhWlzwFQnEM5GTbYfGJep50oBYu6FN+hTT6rK8vR77fZBXB2HUgOjqXvS1ncycrgmPF LcbUBY8VsqqNASAK6Z4U47HZhsDokNKAiHsrVzn4AZJVZvhh+urTgEREB93jAP09P2LK AC+A== X-Gm-Message-State: AOAM532zbhsEqwp+RJ19E1vGHJI+gnArwf5wE/ddynzWuzL54MWWHP+R I5cDtEizVy7M1hT7pIh6ybeqnsoPEwt/4A== X-Received: by 2002:a17:90a:e649:: with SMTP id ep9mr15423267pjb.8.1619814599705; Fri, 30 Apr 2021 13:29:59 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:29:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 78/82] target/arm: Split decode of VSDOT and VUDOT Date: Fri, 30 Apr 2021 13:26:06 -0700 Message-Id: <20210430202610.1136687-79-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1029; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1029.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now that we have a common helper, sharing decode does not save much. Also, this will solve an upcoming naming problem. Signed-off-by: Richard Henderson --- target/arm/neon-shared.decode | 9 ++++++--- target/arm/translate-neon.c | 30 ++++++++++++++++++++++-------- 2 files changed, 28 insertions(+), 11 deletions(-) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index facb621450..2d94369750 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -46,8 +46,9 @@ VCMLA 1111 110 rot:2 . 1 . .... .... 1000 . q:1 . 0 .... \ VCADD 1111 110 rot:1 1 . 0 . .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=%vcadd_size -# VUDOT and VSDOT -VDOT 1111 110 00 . 10 .... .... 1101 . q:1 . u:1 .... \ +VSDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp # VFM[AS]L @@ -61,7 +62,9 @@ VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp size=2 index=0 -VDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 u:1 vm:4 \ +VSDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 0 vm:4 \ + vn=%vn_dp vd=%vd_dp +VUDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 1 vm:4 \ vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index 52b75ff76f..194e329b6c 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -269,15 +269,22 @@ static bool trans_VCADD(DisasContext *s, arg_VCADD *a) return true; } -static bool trans_VDOT(DisasContext *s, arg_VDOT *a) +static bool trans_VSDOT(DisasContext *s, arg_VSDOT *a) { if (!dc_isar_feature(aa32_dp, s)) { return false; } return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, - a->u - ? gen_helper_gvec_udot_b - : gen_helper_gvec_sdot_b); + gen_helper_gvec_sdot_b); +} + +static bool trans_VUDOT(DisasContext *s, arg_VUDOT *a) +{ + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_udot_b); } static bool trans_VFML(DisasContext *s, arg_VFML *a) @@ -329,15 +336,22 @@ static bool trans_VCMLA_scalar(DisasContext *s, arg_VCMLA_scalar *a) FPST_STD, gen_helper_gvec_fcmlas_idx); } -static bool trans_VDOT_scalar(DisasContext *s, arg_VDOT_scalar *a) +static bool trans_VSDOT_scalar(DisasContext *s, arg_VSDOT_scalar *a) { if (!dc_isar_feature(aa32_dp, s)) { return false; } return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, - a->u - ? gen_helper_gvec_udot_idx_b - : gen_helper_gvec_sdot_idx_b); + gen_helper_gvec_sdot_idx_b); +} + +static bool trans_VUDOT_scalar(DisasContext *s, arg_VUDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_dp, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_udot_idx_b); } static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) From patchwork Fri Apr 30 20:26:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429912 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp815252jao; Fri, 30 Apr 2021 14:37:48 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwbC8TaXmM1Y3RaluUeN0i0Aqkneh++Izdixy0Ns/++2dEVrAQ5GhQxV1azmuiPikU3tzpa X-Received: by 2002:aca:bc42:: with SMTP id m63mr12244664oif.96.1619818668475; Fri, 30 Apr 2021 14:37:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818668; cv=none; d=google.com; s=arc-20160816; b=KjThlEGF5YV0pSN7CmrtrOr+m/jFbMzLrQC0SJrW6NkzrBXuryfLj+biuY2gDmRSQ6 oKVzwWGHlX3btdLZp6TTyhZaivsQV3kTXgNQimwfNXFlOjkNynjElvx/T32l21us2SZe Ccser8KMuzEJSFOK86hPhAFNSt+P1pM9d/PFzwC2Eo3lD51u/aPf278wyRzcIRbUXJCP a1oYZ88QkAeRQmWp/78QjFUjn3gN8C4qHmLZ0lLWYdIkOvYKftEy/7/uPcU2Itrc5Ggx D4WyiVuvKhtPXzxAVAyvDSCfUD8FqjWuHRHly6ybkR5vwSLwIxahM7xq5QB9ecifdFLR TgFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=QgPHonaDrMGTIgpcdL1HmrXmjritKf6UcNKnh0LjfQg=; b=TbdTrS36OjrPWIcsbQxYTQvplu8kwLgx50GETjHRlBd/4Lk8aUtuEu7aMX4StKTvLZ m+i5QlNfQfePAA/q1vUPuBee5Zm40soZeLAiFAgkEOXbuvpHRGiCf5HFCVr2+uiyHt3k nWjIrD+PEbCWLCo1a+Grc6/bJ8A16WVzG3gA8nc6L6hbrrB5SBmYYfjk1epMo+JFcNoN 8b8F87gLk2VSN2KS91OvcWEnszsF325hpk4hWgVXDTGHjV+x+kfGM732km5i/kO0s7/b DBL1tPr9DkpKExOozABv5numrSWogQ2i+AjA/uKzn88jg/z35hVMa7Px4Sybk5a+boQ/ um4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GKXTcRAt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id e25si3726692otk.89.2021.04.30.14.37.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:37:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GKXTcRAt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:60568 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcapX-00012T-ON for patch@linaro.org; Fri, 30 Apr 2021 17:37:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZn5-0005EL-Up for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:11 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]:39439) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZlx-0007p5-LM for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:31:11 -0400 Received: by mail-pl1-x62d.google.com with SMTP id s15so8821211plg.6 for ; Fri, 30 Apr 2021 13:30:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=QgPHonaDrMGTIgpcdL1HmrXmjritKf6UcNKnh0LjfQg=; b=GKXTcRAt7h3AxVyN2yG3IqOzKCt9aI8vedY+F4Dh23B3s9Grk6YPbJYTJEDZrgef2H 8As9kOvWZSH7DwZiCLcu7aKE+ZCCCzXSRERpbd/Tf6D4+pJ2haUXrj6V+h/mMy3xHoLq n9AEa6MLbpICeyiZkWFB4RiYyTPgfecEk8WQMJh6uOHe6xAk1rzqFaa7tyHbjVU3J3mU OyNxKth9NLwcv9BunjLj+YYGOEDQsD18ahiSbrAZF8RXC57GK/iW2l3UXiG8p6grdlLp +bgluQ1+tWKjcw5/o4s2yEcFESafX8mM39BX9QzmHVhXipm28qkgml8rf35K5lqMX6n2 gRBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=QgPHonaDrMGTIgpcdL1HmrXmjritKf6UcNKnh0LjfQg=; b=f5bDue1PIWVEVBdsE2JrV8Y1YCJyBil2xJKpSevuNbNapubHXIcKjuFzTxtfWwjy3z 1D5UPWkStA0PG6e/8HdgYKQ0cVFg1ARXb0FoQOgn10UBqGdXD/w1MS7WiNQh7dJS38u0 Ng+qWls3siZxCi6diEUnXtxUMx+73mS9g0LTRBMOqt8VqW/aSjrAjXIKBfMLPZKPObxp /5MwWQW54kv742PRfzpFxWniEuTO3DlwlPFH9+9P/W7tjIUAF23P56d5JK4mo+GKYtKW Vw34YiVtwXQGOxKEwBIft9EV5tooaaIYpysRLu7GdHmtIclizjJeBMsvVOkaIsQP/+Ef KNTw== X-Gm-Message-State: AOAM533xFRRvIVvGqINfT0uKFyr/TdnXB93H0eZbcbcno1OnmTeLChd8 HoD2hY6QuuhTchtDALFORTdx2OrOo9m2nw== X-Received: by 2002:a17:90a:5806:: with SMTP id h6mr7100545pji.14.1619814600318; Fri, 30 Apr 2021 13:30:00 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3788781pgt.42.2021.04.30.13.29.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:30:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 79/82] target/arm: Implement aarch32 VSUDOT, VUSDOT Date: Fri, 30 Apr 2021 13:26:07 -0700 Message-Id: <20210430202610.1136687-80-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.h | 5 +++++ target/arm/neon-shared.decode | 6 ++++++ target/arm/translate-neon.c | 27 +++++++++++++++++++++++++++ 3 files changed, 38 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/cpu.h b/target/arm/cpu.h index b2b684df55..5850db90f3 100644 --- a/target/arm/cpu.h +++ b/target/arm/cpu.h @@ -3783,6 +3783,11 @@ static inline bool isar_feature_aa32_predinv(const ARMISARegisters *id) return FIELD_EX32(id->id_isar6, ID_ISAR6, SPECRES) != 0; } +static inline bool isar_feature_aa32_i8mm(const ARMISARegisters *id) +{ + return FIELD_EX32(id->id_isar6, ID_ISAR6, I8MM) != 0; +} + static inline bool isar_feature_aa32_ras(const ARMISARegisters *id) { return FIELD_EX32(id->id_pfr0, ID_PFR0, RAS) != 0; diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index 2d94369750..5befaec87b 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -50,6 +50,8 @@ VSDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 0 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp VUDOT 1111 110 00 . 10 .... .... 1101 . q:1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUSDOT 1111 110 01 . 10 .... .... 1101 . q:1 . 0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp # VFM[AS]L VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ @@ -66,6 +68,10 @@ VSDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 0 vm:4 \ vn=%vn_dp vd=%vd_dp VUDOT_scalar 1111 1110 0 . 10 .... .... 1101 . q:1 index:1 1 vm:4 \ vn=%vn_dp vd=%vd_dp +VUSDOT_scalar 1111 1110 1 . 00 .... .... 1101 . q:1 index:1 0 vm:4 \ + vn=%vn_dp vd=%vd_dp +VSUDOT_scalar 1111 1110 1 . 00 .... .... 1101 . q:1 index:1 1 vm:4 \ + vn=%vn_dp vd=%vd_dp %vfml_scalar_q0_rm 0:3 5:1 %vfml_scalar_q1_index 5:1 3:1 diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index 194e329b6c..b7774caf9e 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -287,6 +287,15 @@ static bool trans_VUDOT(DisasContext *s, arg_VUDOT *a) gen_helper_gvec_udot_b); } +static bool trans_VUSDOT(DisasContext *s, arg_VUSDOT *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_usdot_b); +} + static bool trans_VFML(DisasContext *s, arg_VFML *a) { int opr_sz; @@ -354,6 +363,24 @@ static bool trans_VUDOT_scalar(DisasContext *s, arg_VUDOT_scalar *a) gen_helper_gvec_udot_idx_b); } +static bool trans_VUSDOT_scalar(DisasContext *s, arg_VUSDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_usdot_idx_b); +} + +static bool trans_VSUDOT_scalar(DisasContext *s, arg_VSUDOT_scalar *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, a->q * 6, a->vd, a->vn, a->vm, a->index, + gen_helper_gvec_sudot_idx_b); +} + static bool trans_VFML_scalar(DisasContext *s, arg_VFML_scalar *a) { int opr_sz; From patchwork Fri Apr 30 20:26:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429914 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp817954jao; Fri, 30 Apr 2021 14:42:11 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzjrvRmsBGFfVeJazSmHBHOQdy14hVcUipwERCC5bVOSuMsLr/x5iYZxTcmc/ZpFaHUdBb1 X-Received: by 2002:aca:3385:: with SMTP id z127mr5389965oiz.23.1619818930910; Fri, 30 Apr 2021 14:42:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619818930; cv=none; d=google.com; s=arc-20160816; b=cOAXXjv67guuhOESuNYuMKSqokV57cKmnaMKhD/hL7vwKFaAer5moG0cmOgsG314dn 5A4SMJDNdAK/LBHDZCD4WypG4xIMUQ0zWVh0nX4pBknOHC6T9lYtySiGe0Wm4JPxuZqN BCRbDU+fru7aj3PVRMFOSl7fFbpdfbSbyOKaVl1hCXwTOCp/PJPTOzTGus+h9a8u4jw/ cHuV+MpaC8sh6ygJ8ii/EuyPTgTa/r4eNqNIcNIvLbsB1Dpp99Ld1BGxILOu87ap5Jax tqJ4OZiqObZcXxducaFGBBDc4NJK7xxDu5/hH5Tfd0r5qSGxAtMimRfjiNdpgfKgPihl zIhQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BAqr4W6B826glKmooOGK9KlVf8wkCriM0YiYwy5pluA=; b=oNr6fn4w1AuvIBIyrvP8kuCMlZj8w7Cyug9qLANf3tR2W2UZWkFvUBAjon2g5FNx+j xD5ZeS96sy6GdQS0dvvqharfSs2eEtfFe/Ruz84dxOY++3H6NyN6pWN5mC8ANIiDyZVe 0amBCYqN22nAgD+233n1/8BUe87IqQJ0kJFySj56/oD0o+R2iOXYiaf8/62uamIZd5iA BqIRLM1gYHqcXVagGu+KYjTIxfxn6KoS3N4/vOLs5iNBw7jNZ9GZ+mXjV/mZMoZU3Agq vpLko5z/41BEj76qpB7sEAlhFgUVKHI4CuhZRi4p7BtSslvFLP/QT2TKFzmrnZzeKLN5 lbJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UTnAeyK8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a16si4905302oiw.238.2021.04.30.14.42.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:42:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=UTnAeyK8; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46754 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcatm-000780-9e for patch@linaro.org; Fri, 30 Apr 2021 17:42:10 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54328) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZoy-0006kq-AB for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:08 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]:34703) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZot-0000rq-M2 for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:06 -0400 Received: by mail-pl1-x62e.google.com with SMTP id h7so8672581plt.1 for ; Fri, 30 Apr 2021 13:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=BAqr4W6B826glKmooOGK9KlVf8wkCriM0YiYwy5pluA=; b=UTnAeyK8a7JJOdheRaaSDnKifPiR1/6PeejNk//7QyAc0uJFVIK3WEgsx/VrWSj3lp 7N0kyUC96Upe7MNYyFcCkkD+k7FJ1HhvVeyy1/4mBOK9wLkzr9WjRZN+TZmopghdYaeL UnNStcnm41VrcegIUWTGl8xh1s1SqxLRwa+MgzRijeEYdIdpFniW7mebhCeuB5oe/EoU g31KZhcFoLwrFP/ywzAsR5HV4kYK08eda/pSb0Cq0854XQB+wXX+ijLoIQ62jYwBc7BY 3gJLK+Y5ObEg8ZAQJcnvOAv9Yxa/Xvtbth8Los7FhQ/qAnIuiFH/b8HGrOruXs6DbthS 6vmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=BAqr4W6B826glKmooOGK9KlVf8wkCriM0YiYwy5pluA=; b=hs7agh+baCWJsJXsHl5NR+jrVF5+POB74Uci6txNhzI92I5Qt2iovlGqc7VijOoM0X e5aplb7qxeYlRnelBigrtXY5Pb/C/Z+odHGYK1cZSVgrEU/31hc3EGGI/Hn2Lzj43ljf vczyZriD36ME1MnieyLf0R908nNAG1Uc6BKTRJkb4xMQu9o+eHcBcQsAdrj3GyybYDSV ysb/jny+pDDeqXgv5aayTtusrt3j0+WFUOWkkSt9rFWl8tJph7tfcC18w1wDamlbJo+k 4g2Fb9SMck6UIj8zPURXUaShoEM5zVqC9h8Xa6c8r8zk/P8rUxeUJupp8Iz5NxZglgp7 c5pg== X-Gm-Message-State: AOAM5311VmA5GEitzZ4ngKUDW9Rkzw+U37OLuLs0dq6VRoa3yzCLqRz0 +8MX4NWhJTx+v8lTYLaWzKcSkGBj30CN9A== X-Received: by 2002:a17:90a:4108:: with SMTP id u8mr7165208pjf.126.1619814781364; Fri, 30 Apr 2021 13:33:01 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3792511pgt.42.2021.04.30.13.33.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:33:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 80/82] target/arm: Implement integer matrix multiply accumulate Date: Fri, 30 Apr 2021 13:26:08 -0700 Message-Id: <20210430202610.1136687-81-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is {S,U,US}MMLA for both AArch64 AdvSIMD and SVE, and V{S,U,US}MMLA.S8 for AArch32 NEON. Signed-off-by: Richard Henderson --- target/arm/helper.h | 7 ++++ target/arm/neon-shared.decode | 7 ++++ target/arm/sve.decode | 6 +++ target/arm/translate-a64.c | 18 ++++++++ target/arm/translate-neon.c | 27 ++++++++++++ target/arm/translate-sve.c | 27 ++++++++++++ target/arm/vec_helper.c | 77 +++++++++++++++++++++++++++++++++++ 7 files changed, 169 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/target/arm/helper.h b/target/arm/helper.h index e8b16a401f..33df62f44d 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -994,6 +994,13 @@ DEF_HELPER_FLAGS_6(sve2_fmlal_zzxw_s, TCG_CALL_NO_RWG, DEF_HELPER_FLAGS_4(gvec_xar_d, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_smmla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_ummla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_5(gvec_usmmla_b, TCG_CALL_NO_RWG, + void, ptr, ptr, ptr, ptr, i32) + #ifdef TARGET_AARCH64 #include "helper-a64.h" #include "helper-sve.h" diff --git a/target/arm/neon-shared.decode b/target/arm/neon-shared.decode index 5befaec87b..cc9f4cdd85 100644 --- a/target/arm/neon-shared.decode +++ b/target/arm/neon-shared.decode @@ -59,6 +59,13 @@ VFML 1111 110 0 s:1 . 10 .... .... 1000 . 0 . 1 .... \ VFML 1111 110 0 s:1 . 10 .... .... 1000 . 1 . 1 .... \ vm=%vm_dp vn=%vn_dp vd=%vd_dp q=1 +VSMMLA 1111 1100 0.10 .... .... 1100 .1.0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUMMLA 1111 1100 0.10 .... .... 1100 .1.1 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp +VUSMMLA 1111 1100 1.10 .... .... 1100 .1.0 .... \ + vm=%vm_dp vn=%vn_dp vd=%vd_dp + VCMLA_scalar 1111 1110 0 . rot:2 .... .... 1000 . q:1 index:1 0 vm:4 \ vn=%vn_dp vd=%vd_dp size=1 VCMLA_scalar 1111 1110 1 . rot:2 .... .... 1000 . q:1 . 0 .... \ diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 63870b7539..3d7c4fa6e5 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -1364,6 +1364,12 @@ USHLLT 01000101 .. 0 ..... 1010 11 ..... ..... @rd_rn_tszimm_shl EORBT 01000101 .. 0 ..... 10010 0 ..... ..... @rd_rn_rm EORTB 01000101 .. 0 ..... 10010 1 ..... ..... @rd_rn_rm +## SVE integer matrix multiply accumulate + +SMMLA 01000101 00 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 +USMMLA 01000101 10 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 +UMMLA 01000101 11 0 ..... 10011 0 ..... ..... @rda_rn_rm_e0 + ## SVE2 bitwise permute BEXT 01000101 .. 0 ..... 1011 00 ..... ..... @rd_rn_rm diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index c875481784..ceac0ee2bd 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -12182,6 +12182,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) } feature = dc_isar_feature(aa64_i8mm, s); break; + case 0x04: /* SMMLA */ + case 0x14: /* UMMLA */ + case 0x05: /* USMMLA */ + if (!is_q || size != MO_32) { + unallocated_encoding(s); + return; + } + feature = dc_isar_feature(aa64_i8mm, s); + break; case 0x18: /* FCMLA, #0 */ case 0x19: /* FCMLA, #90 */ case 0x1a: /* FCMLA, #180 */ @@ -12226,6 +12235,15 @@ static void disas_simd_three_reg_same_extra(DisasContext *s, uint32_t insn) gen_gvec_op4_ool(s, is_q, rd, rn, rm, rd, 0, gen_helper_gvec_usdot_b); return; + case 0x04: /* SMMLA, UMMLA */ + gen_gvec_op4_ool(s, 1, rd, rn, rm, rd, 0, + u ? gen_helper_gvec_ummla_b + : gen_helper_gvec_smmla_b); + return; + case 0x05: /* USMMLA */ + gen_gvec_op4_ool(s, 1, rd, rn, rm, rd, 0, gen_helper_gvec_usmmla_b); + return; + case 0x8: /* FCMLA, #0 */ case 0x9: /* FCMLA, #90 */ case 0xa: /* FCMLA, #180 */ diff --git a/target/arm/translate-neon.c b/target/arm/translate-neon.c index b7774caf9e..7232685106 100644 --- a/target/arm/translate-neon.c +++ b/target/arm/translate-neon.c @@ -4036,3 +4036,30 @@ static bool trans_VTRN(DisasContext *s, arg_2misc *a) tcg_temp_free_i32(tmp2); return true; } + +static bool trans_VSMMLA(DisasContext *s, arg_VSMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_smmla_b); +} + +static bool trans_VUMMLA(DisasContext *s, arg_VUMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_ummla_b); +} + +static bool trans_VUSMMLA(DisasContext *s, arg_VUSMMLA *a) +{ + if (!dc_isar_feature(aa32_i8mm, s)) { + return false; + } + return do_neon_ddda(s, 7, a->vd, a->vn, a->vm, 0, + gen_helper_gvec_usmmla_b); +} diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index acb8603418..5037e6b7eb 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -8543,3 +8543,30 @@ static bool trans_FMLSLT_zzxw(DisasContext *s, arg_rrxr_esz *a) { return do_FMLAL_zzxw(s, a, true, true); } + +static bool do_i8mm_zzzz_ool(DisasContext *s, arg_rrrr_esz *a, + gen_helper_gvec_4 *fn, int data) +{ + if (!dc_isar_feature(aa64_sve_i8mm, s)) { + return false; + } + if (sve_access_check(s)) { + gen_gvec_ool_zzzz(s, fn, a->rd, a->rn, a->rm, a->ra, data); + } + return true; +} + +static bool trans_SMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_smmla_b, 0); +} + +static bool trans_USMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_usmmla_b, 0); +} + +static bool trans_UMMLA(DisasContext *s, arg_rrrr_esz *a) +{ + return do_i8mm_zzzz_ool(s, a, gen_helper_gvec_ummla_b, 0); +} diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index ea08b15c55..6c9f1e5146 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -2578,3 +2578,80 @@ void HELPER(gvec_xar_d)(void *vd, void *vn, void *vm, uint32_t desc) } clear_tail(d, opr_sz * 8, simd_maxsz(desc)); } + +/* + * Integer matrix-multiply accumulate + */ + +static uint32_t do_smmla_b(uint32_t sum, void *vn, void *vm) +{ + int8_t *n = vn, *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static uint32_t do_ummla_b(uint32_t sum, void *vn, void *vm) +{ + uint8_t *n = vn, *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static uint32_t do_usmmla_b(uint32_t sum, void *vn, void *vm) +{ + uint8_t *n = vn; + int8_t *m = vm; + + for (intptr_t k = 0; k < 8; ++k) { + sum += n[H1(k)] * m[H1(k)]; + } + return sum; +} + +static void do_mmla_b(void *vd, void *vn, void *vm, void *va, uint32_t desc, + uint32_t (*inner_loop)(uint32_t, void *, void *)) +{ + intptr_t seg, opr_sz = simd_oprsz(desc); + + for (seg = 0; seg < opr_sz; seg += 16) { + uint32_t *d = vd + seg; + uint32_t *a = va + seg; + uint32_t sum0, sum1, sum2, sum3; + + /* + * Process the entire segment at once, writing back the + * results only after we've consumed all of the inputs. + * + * Key to indicies by column: + * i j i j + */ + sum0 = a[H4(0 + 0)]; + sum0 = inner_loop(sum0, vn + seg + 0, vm + seg + 0); + sum1 = a[H4(0 + 1)]; + sum1 = inner_loop(sum1, vn + seg + 0, vm + seg + 8); + sum2 = a[H4(2 + 0)]; + sum2 = inner_loop(sum2, vn + seg + 8, vm + seg + 0); + sum3 = a[H4(2 + 1)]; + sum3 = inner_loop(sum3, vn + seg + 8, vm + seg + 8); + + d[H4(0)] = sum0; + d[H4(1)] = sum1; + d[H4(2)] = sum2; + d[H4(3)] = sum3; + } + clear_tail(vd, opr_sz, simd_maxsz(desc)); +} + +#define DO_MMLA_B(NAME, INNER) \ + void HELPER(NAME)(void *vd, void *vn, void *vm, void *va, uint32_t desc) \ + { do_mmla_b(vd, vn, vm, va, desc, INNER); } + +DO_MMLA_B(gvec_smmla_b, do_smmla_b) +DO_MMLA_B(gvec_ummla_b, do_ummla_b) +DO_MMLA_B(gvec_usmmla_b, do_usmmla_b) From patchwork Fri Apr 30 20:26:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429915 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp819305jao; Fri, 30 Apr 2021 14:44:31 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzUYiMspYgaO1YXRFODP1AakbprJkLN3TlYx7gCBXQ2VJ6/RQcpSICwtDdoxZz7T+C9x7xw X-Received: by 2002:a92:cc4e:: with SMTP id t14mr5409648ilq.38.1619819071675; Fri, 30 Apr 2021 14:44:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619819071; cv=none; d=google.com; s=arc-20160816; b=U+hIFOv6tfviNSZmZrjc3eJys8JkYy3LgMtPzQdE1OkHHHtF/ngyTMev0DnJi8wgjT NYa7ekr7Bvnu3i40ZNMU0FyPGvOQash+qYWVw4pd0peesdwqvcg4QZSyfAjCuKKEeaKF HCbzHafT6uv2OVWHXe1GM1LDRVbsL0Ilblt9YkaAUNHk/dai0Ap/zKhe8nQHi4slXI4V Njh3lmH9JfuQ7Do3MckhmGguJhjnxJR18WimZ56ecSQqtKgo8LsPuB/KBGIcZZn3G5ie zI5gqUILdOAqfXKrZz7haF81AQgLVBRKfG7I8UPezw7Y4l73/WDeAcUQ1CdI3vkYzX7Z 3VAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=PS5/3Bgg+wTxPQ6ZWYhKexmWIte/yItcs5KmNlLkiRY=; b=LTY3BGraZk8ioe36wg5afC6GGzog0q1FSIr5+3I+eR0bJlQDVLM/dQX4fbmY1ca+5U bVo+Obw5bibVxYCxCWTW3y7Crr4seFBv1QHtQEs3Re0i6k9vHRm3coh8CPaO6DMWQBBy Impb9gHN3zbk82uWKq197zoP+3jims0ZFO3NAXFaB6uQBuvu8pjk0rVY6SDsVoJZhTTJ ZbeRM7NPZPIuHnt28VI+TOew75e/zcBwjzeRUQIWYU6yHYUuqYhAvv1senrxJtDEAt5B QdFSHQZOk7RzDZ2UCkY9Bcq5oU6qIPh5NsqaRZVRE0pXeAKmaeiHZmAdaGtKQF89j4lt DOEA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SRfl8eTf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f11si4018944ila.138.2021.04.30.14.44.31 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:44:31 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SRfl8eTf; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:55064 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcavx-00027Z-Pu for patch@linaro.org; Fri, 30 Apr 2021 17:44:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54348) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZp8-0006mU-Kc for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:20 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]:37870) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZot-0000rt-MG for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:13 -0400 Received: by mail-pf1-x42d.google.com with SMTP id b15so5019551pfl.4 for ; Fri, 30 Apr 2021 13:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=PS5/3Bgg+wTxPQ6ZWYhKexmWIte/yItcs5KmNlLkiRY=; b=SRfl8eTfYpam230n/syaxqtDBTUBQX21+jpNyMtnieQq8aM+PShwWvX2NLIZ9yf9Rk ydp+45FKAx0TCloU8RRCgGwgyiA57DLBn6MicEpyZYoY7tqLl3rCrBCJKR1EP7TQPVv9 SJMmi6C2rVTL0tLA+00A33qsQpGdPj0P8m/JdIo8b6ahEAAHeQAc4bmRuCwneC0hmky1 xIL2Z4x0zIrIEP0BWatO8Q2P4FfKNfHpL0uLdX+MUdB8hVL5aYKC5KfJRET2DGqPsSbJ 2FG2ocEp7Gukmrz55K7UVDMnCOhkj2hUusRlYBUNtO8EdTyiDwiZyi3undLJTul8iy2U RQmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=PS5/3Bgg+wTxPQ6ZWYhKexmWIte/yItcs5KmNlLkiRY=; b=oMUE8J4DLxEBu5AZb2IKxSKqlNLAbp39DyQlD0r7H8FIuvrIHZnuiOUWvjX1k+6YJG QYqt1dnYPT6QnRy8skoVqjX4XIPlyRD+0JQdW9ENrJMArEXakQ7HW0W/k02IiHC28V4c qtr3j+IDc76CrIkatGRUh5Tw7cE1XTfjsBMBPCdYMjk4OM8YxPOhb/SwDvjoggxyFOHt kOG2Zp9Q84a52+qZ6VPE7Q7We/Y5avpRcaxBMjW++T9Am1Qnr1qQARQzSrrXX3tCgoH7 uy/ZNi52hLbi40i2HVjWKXJIWq1NlXTs7cZaYp5xo/he1IDKmLN5z1eKAN7JBi6LLebq UcFQ== X-Gm-Message-State: AOAM533x5Ebt/L/AHr+qwEcprPcl4p1yPsBEVqyLhPEodzNqCbYDUr7n RTno3a9DhGmaWXyV4SC6TgIIZE+pdfbYpg== X-Received: by 2002:a65:60d8:: with SMTP id r24mr6364448pgv.226.1619814781951; Fri, 30 Apr 2021 13:33:01 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3792511pgt.42.2021.04.30.13.33.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:33:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 81/82] linux-user/aarch64: Enable hwcap bits for sve2 and related extensions Date: Fri, 30 Apr 2021 13:26:09 -0700 Message-Id: <20210430202610.1136687-82-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- linux-user/elfload.c | 10 ++++++++++ 1 file changed, 10 insertions(+) -- 2.25.1 Reviewed-by: Peter Maydell diff --git a/linux-user/elfload.c b/linux-user/elfload.c index fc9c4f12be..299116d450 100644 --- a/linux-user/elfload.c +++ b/linux-user/elfload.c @@ -648,8 +648,18 @@ static uint32_t get_elf_hwcap2(void) uint32_t hwcaps = 0; GET_FEATURE_ID(aa64_dcpodp, ARM_HWCAP2_A64_DCPODP); + GET_FEATURE_ID(aa64_sve2, ARM_HWCAP2_A64_SVE2); + GET_FEATURE_ID(aa64_sve2_aes, ARM_HWCAP2_A64_SVEAES); + GET_FEATURE_ID(aa64_sve2_pmull128, ARM_HWCAP2_A64_SVEPMULL); + GET_FEATURE_ID(aa64_sve2_bitperm, ARM_HWCAP2_A64_SVEBITPERM); + GET_FEATURE_ID(aa64_sve2_sha3, ARM_HWCAP2_A64_SVESHA3); + GET_FEATURE_ID(aa64_sve2_sm4, ARM_HWCAP2_A64_SVESM4); GET_FEATURE_ID(aa64_condm_5, ARM_HWCAP2_A64_FLAGM2); GET_FEATURE_ID(aa64_frint, ARM_HWCAP2_A64_FRINT); + GET_FEATURE_ID(aa64_sve_i8mm, ARM_HWCAP2_A64_SVEI8MM); + GET_FEATURE_ID(aa64_sve_f32mm, ARM_HWCAP2_A64_SVEF32MM); + GET_FEATURE_ID(aa64_sve_f64mm, ARM_HWCAP2_A64_SVEF64MM); + GET_FEATURE_ID(aa64_i8mm, ARM_HWCAP2_A64_I8MM); GET_FEATURE_ID(aa64_rndr, ARM_HWCAP2_A64_RNG); GET_FEATURE_ID(aa64_bti, ARM_HWCAP2_A64_BTI); GET_FEATURE_ID(aa64_mte, ARM_HWCAP2_A64_MTE); From patchwork Fri Apr 30 20:26:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 429916 Delivered-To: patch@linaro.org Received: by 2002:a02:c901:0:0:0:0:0 with SMTP id t1csp821548jao; Fri, 30 Apr 2021 14:48:23 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxCVUW5WOruL3fbsovM7GuzuCSCKjZAUMn0CYq4LootHx4n8mbTrRReqfY8aDxYjHgfJZCm X-Received: by 2002:a05:6808:14c8:: with SMTP id f8mr13045848oiw.55.1619819303781; Fri, 30 Apr 2021 14:48:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1619819303; cv=none; d=google.com; s=arc-20160816; b=bKdBvShzUFPILRU0SN84gNdIEu1ITMhloMC3LVq2StVtCA53BM5OJsm3lR3dsNfy28 UNwJjZRIlozx1vvJTXdox9pDF/8RKFoqfTticFv5oT00eRursWk1GLL+ybLrGkqyAwm6 wX3t2IFybRbKvF0yGceojBkz7bYy+URc769s+6ELoZ6RnejbgStsQlVOLjfT41aJfpGW XdBW0B9vsCV6Vb+PVI43fUvtOnfEG+fG1GTabTqQZlyi7Fi0EnqkPASWBWTBdL0y5DYe 0dw+RjxWe+Vg+uYxLN/u0VIy1tAa1RH0df89v4u5DcinVlLbqag7qWSLVQpTBXpHnVtl qtBw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=WF7jakKDCABHLnWSe8tP3c6xYoRSHPgVojWB7PcKR0Nlud0ApqdqPDRqg+yOir8lhM RocEL1MwyZDde+C8SdUE8jD1+si1cVp2hOYdcucSAsQo4+2svl276YfOjjIgSWvaTDtk BmG9ExBog/zNk9pCKrj7xNevLpRjt9CGXvss/CaLLtsrG2tyoltm6oScJqGQxhH05USA pdzqeI5gRyPYEqABxHNjupGr9/v4rWemKGJ1Mj2U6KdMwWbDI8CYho+7FFChWt0U1vVQ EFkABSJkDMW+Ke9h2axRfJ8p5AGapnUsjW24/rXb0neatmJzSMQeu8tVjeBLIgzHjpN4 xtow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F+SFglPe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s4si4594076otq.39.2021.04.30.14.48.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 30 Apr 2021 14:48:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=F+SFglPe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:33074 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcazn-000536-7L for patch@linaro.org; Fri, 30 Apr 2021 17:48:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54358) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lcZpA-0006n7-1P for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:20 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:35783) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lcZot-0000s6-NZ for qemu-devel@nongnu.org; Fri, 30 Apr 2021 16:33:17 -0400 Received: by mail-pj1-x1031.google.com with SMTP id h14-20020a17090aea8eb02901553e1cc649so2392343pjz.0 for ; Fri, 30 Apr 2021 13:33:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=F+SFglPeAJACCHKXIlweuKO6pSv5yG7lT733zNeN4pdwHcJj5R7V860HrPZTh42ZBB pSfX+5LyPhGStoSrV75UOTBRoIUjosZwOlEPnaitPaAVKVQ/zDdfgoSli/OD32gkhEKl J4tS5paOK/yvcoftJ3pJkhplCI5M07gbvXNoqdVfcnTuTZhvg41WqPo8mwL29mZJjeYm jKbS+xTEijDvz2dtk9zTr5erHhJ6mMhQ+dahC9IqutIaqsz9wXWcHajJCh9fwETx+5L3 Bn88bk4kVTHxAxphHdcBS+XlUlN9n1l4mQtVmM86W7/SdGAIHn93p82FvmsayBItqsyG oJng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0MbRdnRJn0x9uVTH+qro7/oyKU7eLBQcZeSxN6NVvH4=; b=R+VQBWzX8SAlzvqnzv4ltQrmAfTjmoFqy7h//04DHsSOPSU92KT3KyYJdUfzpQcPyg PFkNQ15bbmFy9DUJo0kwJVL7kwDpLRp3unyaHbk/uz9PyZbZk3VbUm0h9tyyQRtQCxha P1s9nfMy1jgLqjfJ9P5j3AoEIetHU82SHF/GKNTIK16QUFM7NBIpN9ihcdxERSq2y8Kk vULGW92NVsGT7eWCC0l5umBINyvGs0pNv+T9ofBORncE2Etyy7NQnVn4A6lAtKi6r4S/ MkXn+NpA/J76jGbE4hjABK/xNzGsTKfIxRbjdXbcVBpF9Gs2mCml4YtMxH1PiAgZnAHb 6mmQ== X-Gm-Message-State: AOAM530wzRMucqZ6C3QGdPv3cOBRlBr7QIYhhy0eDfA0ZX1/qnm+VrfW 53ipsk/yJTipieEPzrCoA8t2ngwjAzhBBg== X-Received: by 2002:a17:90a:d184:: with SMTP id fu4mr16447281pjb.79.1619814782517; Fri, 30 Apr 2021 13:33:02 -0700 (PDT) Received: from localhost.localdomain ([71.212.144.24]) by smtp.gmail.com with ESMTPSA id q23sm3792511pgt.42.2021.04.30.13.33.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 30 Apr 2021 13:33:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v6 82/82] target/arm: Enable SVE2 and related extensions Date: Fri, 30 Apr 2021 13:26:10 -0700 Message-Id: <20210430202610.1136687-83-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210430202610.1136687-1-richard.henderson@linaro.org> References: <20210430202610.1136687-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/cpu.c | 1 + target/arm/cpu64.c | 13 +++++++++++++ 2 files changed, 14 insertions(+) -- 2.25.1 diff --git a/target/arm/cpu.c b/target/arm/cpu.c index 0dd623e590..30fd5d5ff7 100644 --- a/target/arm/cpu.c +++ b/target/arm/cpu.c @@ -1464,6 +1464,7 @@ static void arm_cpu_realizefn(DeviceState *dev, Error **errp) u = cpu->isar.id_isar6; u = FIELD_DP32(u, ID_ISAR6, JSCVT, 0); + u = FIELD_DP32(u, ID_ISAR6, I8MM, 0); cpu->isar.id_isar6 = u; u = cpu->isar.mvfr0; diff --git a/target/arm/cpu64.c b/target/arm/cpu64.c index f0a9e968c9..379f90fab8 100644 --- a/target/arm/cpu64.c +++ b/target/arm/cpu64.c @@ -662,6 +662,7 @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64ISAR1, SPECRES, 1); t = FIELD_DP64(t, ID_AA64ISAR1, FRINTTS, 1); t = FIELD_DP64(t, ID_AA64ISAR1, LRCPC, 2); /* ARMv8.4-RCPC */ + t = FIELD_DP64(t, ID_AA64ISAR1, I8MM, 1); cpu->isar.id_aa64isar1 = t; t = cpu->isar.id_aa64pfr0; @@ -702,6 +703,17 @@ static void aarch64_max_initfn(Object *obj) t = FIELD_DP64(t, ID_AA64MMFR2, ST, 1); /* TTST */ cpu->isar.id_aa64mmfr2 = t; + t = cpu->isar.id_aa64zfr0; + t = FIELD_DP64(t, ID_AA64ZFR0, SVEVER, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, AES, 2); /* PMULL */ + t = FIELD_DP64(t, ID_AA64ZFR0, BITPERM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SHA3, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, SM4, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, I8MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F32MM, 1); + t = FIELD_DP64(t, ID_AA64ZFR0, F64MM, 1); + cpu->isar.id_aa64zfr0 = t; + /* Replicate the same data to the 32-bit id registers. */ u = cpu->isar.id_isar5; u = FIELD_DP32(u, ID_ISAR5, AES, 2); /* AES + PMULL */ @@ -718,6 +730,7 @@ static void aarch64_max_initfn(Object *obj) u = FIELD_DP32(u, ID_ISAR6, FHM, 1); u = FIELD_DP32(u, ID_ISAR6, SB, 1); u = FIELD_DP32(u, ID_ISAR6, SPECRES, 1); + u = FIELD_DP32(u, ID_ISAR6, I8MM, 1); cpu->isar.id_isar6 = u; u = cpu->isar.id_pfr0;