From patchwork Thu Jun 21 01:53:57 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 139434 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp1490000lji; Wed, 20 Jun 2018 19:25:22 -0700 (PDT) X-Google-Smtp-Source: ADUXVKLKB0bALr7OPggYuBmWbu60giJ4hNUfSAwC7QdfEmch6ow8Uyyd+jDmT3xDIlit3OjqzWV8 X-Received: by 2002:a37:2a8e:: with SMTP id q14-v6mr20619868qkq.380.1529547922311; Wed, 20 Jun 2018 19:25:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1529547922; cv=none; d=google.com; s=arc-20160816; b=Giv9Ph8PdkOaUb5QFgzri8/eroUXMIILaYi9JvCjWaVPgsrm8+11TKRFWKbaq4Xma/ n5DE60jPw5aHOfDAhqQ8mkMLDu1tkut90M8s/jO7Jn1pVyG7jMiLLxQPPjECOmhKmAx4 YBqb/y/nvw6+g17GXDVgkBegQY8OdUYTLnVOOgzyU87xMM4KGl2N7Fz2iOVYitgijywZ 92zbtPj9aGTKeAMjjqLF4Eu1JZ3cF9EJ0SxtMJFHKDCAv9TZtUWf+DjQTC/xHZlGJR0J mGmcEHMjN9a13SHVRkq6+zGjgBr2MXcmaicdserjkcd1Irh7HSrhhW/zmsri/2RnIabH 11/A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:references:in-reply-to :message-id:date:to:from:dkim-signature:arc-authentication-results; bh=VTLYFUk8noLkXNDXuc9Ff1/aKekYRtej65w/CxbcRBo=; b=bkpQXABSdM4Wm8FcBX0X6nCXBalDGVfKJT65tUr5QZEdoQa5YEY4VtLBHjoNZOpEvb Kf3zDYrL/q/65/Pd8LPo1JJIwRRFEg4xdwN6cdJVqZULIvX/MD4PiY2Krtyn+pKze4AI 5kCuBMZzslEiVGz3/QKTJR4TeH61skD2bvhpj2cG8dJt44zITwXmAgNpBBN0tbNCHnus 9LTWqI5t2q8cA0humH3HOkEhisRVYEmd9aaoJqFx41Hr+JUJNi0Pvc6JWZPrQuZMuhCJ 1+1gKhO+wG/1I1wQtRmbSQXSoyTFFmZ6Fkc9BpDDEHHAiF0w/8PuzD/K91Oy9ohzknhB cU2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=aOZuopV2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id o133-v6si1181431qke.209.2018.06.20.19.25.22 for (version=TLS1 cipher=AES128-SHA bits=128/128); Wed, 20 Jun 2018 19:25:22 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=aOZuopV2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:52647 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVpHl-0003ja-RR for patch@linaro.org; Wed, 20 Jun 2018 22:25:21 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:39590) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1fVooZ-00050I-BD for qemu-devel@nongnu.org; Wed, 20 Jun 2018 21:55:13 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1fVooX-0003Ya-P6 for qemu-devel@nongnu.org; Wed, 20 Jun 2018 21:55:11 -0400 Received: from mail-pl0-x22f.google.com ([2607:f8b0:400e:c01::22f]:38073) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1fVooX-0003Xw-GG for qemu-devel@nongnu.org; Wed, 20 Jun 2018 21:55:09 -0400 Received: by mail-pl0-x22f.google.com with SMTP id d10-v6so758077plo.5 for ; Wed, 20 Jun 2018 18:55:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=VTLYFUk8noLkXNDXuc9Ff1/aKekYRtej65w/CxbcRBo=; b=aOZuopV2Xits/v/nAXedaE3FJGRzUWFxnvI87kFStOR1xpGE3QD7HnYVh1fEwifXy4 ccJzOG4YXOjfaAMqIi4s3pOIWvXjxtqtRiAg56V+QIEy9VpNw/kKOazL8n1R3te5+497 DZxM9fUXtZBH1MFKCGCCMKJDADZfpqyEq3DHg= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=VTLYFUk8noLkXNDXuc9Ff1/aKekYRtej65w/CxbcRBo=; b=C9DPXN0itOBrSVWPeF+E2fah96auQiREf18B8ISnoAmFN9uiB+eJCte0WvmxJrZiP4 59TUvyZK5Hzu2DzXaEl9ca0WL76aXeuCKtrPVQK6YuAsnYWo++hcy2ACuOMNYU3xpO+K vfBcjS6AKCxcunJpj6Zv3SPdEIc+dZ+UAe1OWqbQs13Cpm1xb6P7diQdaUagLdytPpkK UW8ba/ZS2b9mJQmW0ZaikAaYLlMCoKmDL5Gcv5hECVS+RzaWs+iPwEaGTeApOBc3EMuv bwWiobnSCdmtHIhOmFbj+FZcwCWctE54GyGPm+cY0F81o8xvGwIjsHYXmFf1zlFuG1YB 0R5g== X-Gm-Message-State: APt69E3gKc9szqq7zYnhqM5b9iQRMbnRAoMvt9PWtp+f/Gk1TXpbAkD9 5hmvJfqVwkaFCewKRhc0iHBKQ2TGSZg= X-Received: by 2002:a17:902:22e:: with SMTP id 43-v6mr9206749plc.82.1529546108212; Wed, 20 Jun 2018 18:55:08 -0700 (PDT) Received: from cloudburst.twiddle.net (mta-98-147-121-51.hawaii.rr.com. [98.147.121.51]) by smtp.gmail.com with ESMTPSA id a27-v6sm6187946pfc.18.2018.06.20.18.55.06 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 20 Jun 2018 18:55:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Date: Wed, 20 Jun 2018 15:53:57 -1000 Message-Id: <20180621015359.12018-34-richard.henderson@linaro.org> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180621015359.12018-1-richard.henderson@linaro.org> References: <20180621015359.12018-1-richard.henderson@linaro.org> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:400e:c01::22f Subject: [Qemu-devel] [PATCH v5 33/35] target/arm: Implement SVE dot product (indexed) X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.maydell@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Richard Henderson --- target/arm/helper.h | 5 ++ target/arm/translate-sve.c | 18 +++++++ target/arm/vec_helper.c | 96 ++++++++++++++++++++++++++++++++++++++ target/arm/sve.decode | 8 +++- 4 files changed, 126 insertions(+), 1 deletion(-) -- 2.17.1 diff --git a/target/arm/helper.h b/target/arm/helper.h index e23ce7ff19..59e8c3bd1b 100644 --- a/target/arm/helper.h +++ b/target/arm/helper.h @@ -588,6 +588,11 @@ DEF_HELPER_FLAGS_4(gvec_udot_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_sdot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_4(gvec_udot_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sdot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_udot_idx_b, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_sdot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) +DEF_HELPER_FLAGS_4(gvec_udot_idx_h, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, i32) + DEF_HELPER_FLAGS_5(gvec_fcaddh, TCG_CALL_NO_RWG, void, ptr, ptr, ptr, ptr, i32) DEF_HELPER_FLAGS_5(gvec_fcadds, TCG_CALL_NO_RWG, diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index aa109208e5..af2958be10 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -3440,6 +3440,24 @@ static bool trans_DOT_zzz(DisasContext *s, arg_DOT_zzz *a, uint32_t insn) return true; } +static bool trans_DOT_zzx(DisasContext *s, arg_DOT_zzx *a, uint32_t insn) +{ + static gen_helper_gvec_3 * const fns[2][2] = { + { gen_helper_gvec_sdot_idx_b, gen_helper_gvec_sdot_idx_h }, + { gen_helper_gvec_udot_idx_b, gen_helper_gvec_udot_idx_h } + }; + + if (sve_access_check(s)) { + unsigned vsz = vec_full_reg_size(s); + tcg_gen_gvec_3_ool(vec_full_reg_offset(s, a->rd), + vec_full_reg_offset(s, a->rn), + vec_full_reg_offset(s, a->rm), + vsz, vsz, a->index, fns[a->u][a->sz]); + } + return true; +} + + /* *** SVE Floating Point Multiply-Add Indexed Group */ diff --git a/target/arm/vec_helper.c b/target/arm/vec_helper.c index c16a30c3b5..3117ee39cd 100644 --- a/target/arm/vec_helper.c +++ b/target/arm/vec_helper.c @@ -261,6 +261,102 @@ void HELPER(gvec_udot_h)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } +void HELPER(gvec_sdot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + uint32_t *d = vd; + int8_t *n = vn, *m = vm; + + for (i = 0; i < opr_sz_4; i = j) { + int8_t m0 = m[(i + index) * 4 + 0]; + int8_t m1 = m[(i + index) * 4 + 1]; + int8_t m2 = m[(i + index) * 4 + 2]; + int8_t m3 = m[(i + index) * 4 + 3]; + + j = i; + do { + d[j] += n[j * 4 + 0] * m0 + + n[j * 4 + 1] * m1 + + n[j * 4 + 2] * m2 + + n[j * 4 + 3] * m3; + } while (++j < MIN(i + 4, opr_sz_4)); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_udot_idx_b)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc), opr_sz_4 = opr_sz / 4; + intptr_t index = simd_data(desc); + uint32_t *d = vd; + uint8_t *n = vn, *m = vm; + + for (i = 0; i < opr_sz_4; i = j) { + uint8_t m0 = m[(i + index) * 4 + 0]; + uint8_t m1 = m[(i + index) * 4 + 1]; + uint8_t m2 = m[(i + index) * 4 + 2]; + uint8_t m3 = m[(i + index) * 4 + 3]; + + j = i; + do { + d[j] += n[j * 4 + 0] * m0 + + n[j * 4 + 1] * m1 + + n[j * 4 + 2] * m2 + + n[j * 4 + 3] * m3; + } while (++j < MIN(i + 4, opr_sz_4)); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_sdot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; + intptr_t index = simd_data(desc); + uint64_t *d = vd; + int16_t *n = vn, *m = vm; + + for (i = 0; i < opr_sz_8; i = j) { + int64_t m0 = m[(i + index) * 4 + 0]; + int64_t m1 = m[(i + index) * 4 + 1]; + int64_t m2 = m[(i + index) * 4 + 2]; + int64_t m3 = m[(i + index) * 4 + 3]; + + j = i; + do { + d[j] += n[j * 4 + 0] * m0 + + n[j * 4 + 1] * m1 + + n[j * 4 + 2] * m2 + + n[j * 4 + 3] * m3; + } while (++j < MIN(i + 2, opr_sz_8)); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + +void HELPER(gvec_udot_idx_h)(void *vd, void *vn, void *vm, uint32_t desc) +{ + intptr_t i, j, opr_sz = simd_oprsz(desc), opr_sz_8 = opr_sz / 8; + intptr_t index = simd_data(desc); + uint64_t *d = vd; + uint16_t *n = vn, *m = vm; + + for (i = 0; i < opr_sz_8; i = j) { + uint64_t m0 = m[(i + index) * 4 + 0]; + uint64_t m1 = m[(i + index) * 4 + 1]; + uint64_t m2 = m[(i + index) * 4 + 2]; + uint64_t m3 = m[(i + index) * 4 + 3]; + + j = i; + do { + d[j] += n[j * 4 + 0] * m0 + + n[j * 4 + 1] * m1 + + n[j * 4 + 2] * m2 + + n[j * 4 + 3] * m3; + } while (++j < MIN(i + 2, opr_sz_8)); + } + clear_tail(d, opr_sz, simd_maxsz(desc)); +} + void HELPER(gvec_fcaddh)(void *vd, void *vn, void *vm, void *vfpst, uint32_t desc) { diff --git a/target/arm/sve.decode b/target/arm/sve.decode index 0b29da9f3a..f69ff285a1 100644 --- a/target/arm/sve.decode +++ b/target/arm/sve.decode @@ -722,7 +722,13 @@ UMIN_zzi 00100101 .. 101 011 110 ........ ..... @rdn_i8u MUL_zzi 00100101 .. 110 000 110 ........ ..... @rdn_i8s # SVE integer dot product (unpredicated) -DOT_zzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 +DOT_zzz 01000100 1 sz:1 0 rm:5 00000 u:1 rn:5 rd:5 ra=%reg_movprfx + +# SVE integer dot product (indexed) +DOT_zzx 01000100 101 index:2 rm:3 00000 u:1 rn:5 rd:5 \ + sz=0 ra=%reg_movprfx +DOT_zzx 01000100 111 index:1 rm:4 00000 u:1 rn:5 rd:5 \ + sz=1 ra=%reg_movprfx # SVE floating-point complex add (predicated) FCADD 01100100 esz:2 00000 rot:1 100 pg:3 rm:5 rd:5 \