From patchwork Thu Feb 8 13:54:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Sandiford X-Patchwork-Id: 127669 Delivered-To: patch@linaro.org Received: by 10.46.124.24 with SMTP id x24csp1508334ljc; Thu, 8 Feb 2018 05:54:41 -0800 (PST) X-Google-Smtp-Source: AH8x227QKn6P/L8TdW3W7VEppSZ34ptNuGzZ/gu2WaRVsXd5nCn5Yy/1vakUUW8OZOT0I/b5/OBJ X-Received: by 2002:a17:902:66e6:: with SMTP id e93-v6mr700475plk.173.1518098081000; Thu, 08 Feb 2018 05:54:41 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518098080; cv=none; d=google.com; s=arc-20160816; b=S5/s4rRiY2Yd1397FUnouBJUMR5WHymA4/2Lf3Lu4Sh9BCl4NK3YcapUrWz4tYWzkc jApyqLJcIp3xSeG93PtLg22CA+HXhEKzBuCvjHke/o/isUazb4pEyZLX5S7vc6YGzJCp gWBqFZakxCEpQ3E5L77Vh6aguJZExxV7msjclBBE/VU/3+eaPWc77Fsr0enaUk1j14YI ApwVMEtqneXzJgcPIZxcrFZj8RerWfS6aAoJP8iLSyB5nuELGT+hBX8HH4juGicqT80F Qw21tWU652rs29MUByLtyl7q+S70Bgk0w9uHqWKGKNmN37UFY1zh1veG56Yw5Hqj20j8 a2ZQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:user-agent:message-id:date:subject:cc:mail-followup-to :to:from:delivered-to:sender:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature:arc-authentication-results; bh=GX9UJZ9uJTMMf8jo1EvbaJ2NzycX4YnEcnBdqUJEOR8=; b=G7wzKQPUaP7QXxxZ6zgQSgJWgZce2qx6WsEZ9bFtL3EVqtNP7Dk3Hra00lls0wjfuJ BVy00lwKoll+A++yK7xhEZAeMNaQeKzPa1I4lCy7dnvCegSg0vggUPIYwKID61FocsUm XY6ar4cxReYwj6WQYd4TnAh/7TS3st9nyw0PtXi5XMNeg9lPJYnQDjEJUuuJGoEVHQFi t7RoXQxFk8LxJyCTjAyo4K2DV4xLRRsW5in+A05nHYikZfOa0pu1NWlhlVjKcN2EHHAU unhkK/svK2Z/2L33JCccslVoAymw4WD8nesZMZNlwKAQF/lb2g74ahsyCWgAVZLYYAkG PWDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=vOLwvjcN; spf=pass (google.com: domain of gcc-patches-return-472852-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-472852-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id g8si2420266pgc.644.2018.02.08.05.54.40 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 08 Feb 2018 05:54:40 -0800 (PST) Received-SPF: pass (google.com: domain of gcc-patches-return-472852-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@gcc.gnu.org header.s=default header.b=vOLwvjcN; spf=pass (google.com: domain of gcc-patches-return-472852-patch=linaro.org@gcc.gnu.org designates 209.132.180.131 as permitted sender) smtp.mailfrom=gcc-patches-return-472852-patch=linaro.org@gcc.gnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:mime-version:content-type; q=dns; s=default; b=AyeujdNbJYdpVaek574y3jB0qErMAb35V+VBiPhDQIsBMlYvHK bNlI0E5dmiMgdfQnMRemuD7IydWqgi3tQiFBKfKlkHoYbMd5bt1Tm8fA1s7dcsKT OVG5up+ZI9KaZK+DIZcWiykA2rS7J1p8zYZ7SaRprZwO//RjrNChLIfdI= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=gcc.gnu.org; h=list-id :list-unsubscribe:list-archive:list-post:list-help:sender:from :to:cc:subject:date:message-id:mime-version:content-type; s= default; bh=RaS3GOfE8bM05opaj/DVEfm907c=; b=vOLwvjcNPe66oGT0UjFO iX0Qe7R6Ejrg7KvzleazXL/Kzp9RJCl9He1BoGa/xXmcRsxKxZmMG+ucSXTxUxXg BtBa1WtLhkASPhFjapa35z/fXl6EWVyJ9Wm3oLXSjSsxC10LKrlmIIFaaR7BvPuz 9O8cblmKQrm0e9tVngtJAtE= Received: (qmail 7460 invoked by alias); 8 Feb 2018 13:54:29 -0000 Mailing-List: contact gcc-patches-help@gcc.gnu.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Archive: List-Post: List-Help: Sender: gcc-patches-owner@gcc.gnu.org Delivered-To: mailing list gcc-patches@gcc.gnu.org Received: (qmail 7449 invoked by uid 89); 8 Feb 2018 13:54:28 -0000 Authentication-Results: sourceware.org; auth=none X-Virus-Found: No X-Spam-SWARE-Status: No, score=-11.0 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy= X-HELO: mail-wm0-f54.google.com Received: from mail-wm0-f54.google.com (HELO mail-wm0-f54.google.com) (74.125.82.54) by sourceware.org (qpsmtpd/0.93/v0.84-503-g423c35a) with ESMTP; Thu, 08 Feb 2018 13:54:26 +0000 Received: by mail-wm0-f54.google.com with SMTP id f3so10120953wmc.1 for ; Thu, 08 Feb 2018 05:54:26 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:mail-followup-to:cc:subject:date :message-id:user-agent:mime-version; bh=GX9UJZ9uJTMMf8jo1EvbaJ2NzycX4YnEcnBdqUJEOR8=; b=XyVCZaV+/CIT45UtFUTaHzwfT2PPqb4SqsP66dLpe1XS4TAqAuTA9Natm6IfVhGyjF JxjjqIZsYaSPl6Swr9TP3by1F5Y5PtPJPFc+79UALTK4WAX/Vw8CYgpGyLnFBPkJMsDO cy0xGmmkNuhe61zIRuIOUpnCVKT4xZXFmnXsJsgDSisysxFbadh+xPacqoDr3Ha0Rxj3 jWy9oMjyd6AHhSeWiyFwj4hQGk+4uhKB+h+KaIeXCAH4S5M4FP7zudwuP4bjL0n8J4Dj Mhccc3O5WUfKq8f1U/DGChMaP4mXwbZkbDZHKJMyXoIqPxUsuOQpOEYnCR6ziDsnXDAN /V+g== X-Gm-Message-State: APf1xPCyy5AekWwiBmI+gN69OeEww02BIrqRxxTd7Lsfr9xxHE+rsLYz 3QHhnoEnD9PrY3Gl5wHQkP0v+A== X-Received: by 10.28.169.200 with SMTP id s191mr1172273wme.9.1518098064658; Thu, 08 Feb 2018 05:54:24 -0800 (PST) Received: from localhost (188.29.164.130.threembb.co.uk. [188.29.164.130]) by smtp.gmail.com with ESMTPSA id h13sm13423wrc.89.2018.02.08.05.54.23 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 08 Feb 2018 05:54:23 -0800 (PST) From: Richard Sandiford To: gcc-patches@gcc.gnu.org Mail-Followup-To: gcc-patches@gcc.gnu.org, richard.earnshaw@arm.com, james.greenhalgh@arm.com, marcus.shawcroft@arm.com, richard.sandiford@linaro.org Cc: richard.earnshaw@arm.com, james.greenhalgh@arm.com, marcus.shawcroft@arm.com Subject: [AArch64] Add SVE mul_highpart patterns Date: Thu, 08 Feb 2018 13:54:22 +0000 Message-ID: <874lmry4c1.fsf@linaro.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.3 (gnu/linux) MIME-Version: 1.0 One advantage of the new permute handling compared to the old way is that we can now easily take advantage of the vectoriser's divmod patterns for SVE. I realise we're in stage 4, but this is entirely SVE-specific. Tested on aarch64-linux-gnu and aarch64_be-elf. OK to install? Richard 2018-02-08 Richard Sandiford gcc/ * config/aarch64/iterators.md (UNSPEC_SMUL_HIGHPART) (UNSPEC_UMUL_HIGHPART): New constants. (MUL_HIGHPART): New int iteraor. (su): Handle UNSPEC_SMUL_HIGHPART and UNSPEC_UMUL_HIGHPART. * config/aarch64/aarch64-sve.md (mul3_highpart): New define_expand. (*mul3_highpart): New define_insn. gcc/testsuite/ * gcc.target/aarch64/sve/mul_highpart_1.c: New test. * gcc.target/aarch64/sve/mul_highpart_1_run.c: Likewise. Index: gcc/config/aarch64/iterators.md =================================================================== --- gcc/config/aarch64/iterators.md 2018-01-26 15:14:35.386171048 +0000 +++ gcc/config/aarch64/iterators.md 2018-02-08 13:51:56.252511923 +0000 @@ -438,6 +438,8 @@ (define_c_enum "unspec" UNSPEC_ANDF ; Used in aarch64-sve.md. UNSPEC_IORF ; Used in aarch64-sve.md. UNSPEC_XORF ; Used in aarch64-sve.md. + UNSPEC_SMUL_HIGHPART ; Used in aarch64-sve.md. + UNSPEC_UMUL_HIGHPART ; Used in aarch64-sve.md. UNSPEC_COND_ADD ; Used in aarch64-sve.md. UNSPEC_COND_SUB ; Used in aarch64-sve.md. UNSPEC_COND_SMAX ; Used in aarch64-sve.md. @@ -1467,6 +1469,8 @@ (define_int_iterator UNPACK [UNSPEC_UNPA (define_int_iterator UNPACK_UNSIGNED [UNSPEC_UNPACKULO UNSPEC_UNPACKUHI]) +(define_int_iterator MUL_HIGHPART [UNSPEC_SMUL_HIGHPART UNSPEC_UMUL_HIGHPART]) + (define_int_iterator SVE_COND_INT_OP [UNSPEC_COND_ADD UNSPEC_COND_SUB UNSPEC_COND_SMAX UNSPEC_COND_UMAX UNSPEC_COND_SMIN UNSPEC_COND_UMIN @@ -1558,7 +1562,9 @@ (define_int_attr logicalf_op [(UNSPEC_AN (define_int_attr su [(UNSPEC_UNPACKSHI "s") (UNSPEC_UNPACKUHI "u") (UNSPEC_UNPACKSLO "s") - (UNSPEC_UNPACKULO "u")]) + (UNSPEC_UNPACKULO "u") + (UNSPEC_SMUL_HIGHPART "s") + (UNSPEC_UMUL_HIGHPART "u")]) (define_int_attr sur [(UNSPEC_SHADD "s") (UNSPEC_UHADD "u") (UNSPEC_SRHADD "sr") (UNSPEC_URHADD "ur") Index: gcc/config/aarch64/aarch64-sve.md =================================================================== --- gcc/config/aarch64/aarch64-sve.md 2018-02-01 11:04:16.723192040 +0000 +++ gcc/config/aarch64/aarch64-sve.md 2018-02-08 13:51:56.252511923 +0000 @@ -980,6 +980,34 @@ (define_insn "*msub3" mls\t%0., %1/m, %2., %3." ) +;; Unpredicated highpart multiplication. +(define_expand "mul3_highpart" + [(set (match_operand:SVE_I 0 "register_operand") + (unspec:SVE_I + [(match_dup 3) + (unspec:SVE_I [(match_operand:SVE_I 1 "register_operand") + (match_operand:SVE_I 2 "register_operand")] + MUL_HIGHPART)] + UNSPEC_MERGE_PTRUE))] + "TARGET_SVE" + { + operands[3] = force_reg (mode, CONSTM1_RTX (mode)); + } +) + +;; Predicated highpart multiplication. +(define_insn "*mul3_highpart" + [(set (match_operand:SVE_I 0 "register_operand" "=w") + (unspec:SVE_I + [(match_operand: 1 "register_operand" "Upl") + (unspec:SVE_I [(match_operand:SVE_I 2 "register_operand" "%0") + (match_operand:SVE_I 3 "register_operand" "w")] + MUL_HIGHPART)] + UNSPEC_MERGE_PTRUE))] + "TARGET_SVE" + "mulh\t%0., %1/m, %0., %3." +) + ;; Unpredicated NEG, NOT and POPCOUNT. (define_expand "2" [(set (match_operand:SVE_I 0 "register_operand") Index: gcc/testsuite/gcc.target/aarch64/sve/mul_highpart_1.c =================================================================== --- /dev/null 2018-02-08 11:17:10.862716283 +0000 +++ gcc/testsuite/gcc.target/aarch64/sve/mul_highpart_1.c 2018-02-08 13:51:56.252511923 +0000 @@ -0,0 +1,25 @@ +/* { dg-do assemble { target aarch64_asm_sve_ok } } */ +/* { dg-options "-O2 -ftree-vectorize -fno-vect-cost-model --save-temps" } */ + +#include + +#define DEF_LOOP(TYPE) \ +void __attribute__ ((noipa)) \ +mod_##TYPE (TYPE *dst, TYPE *src, int count) \ +{ \ + for (int i = 0; i < count; ++i) \ + dst[i] = src[i] % 17; \ +} + +#define TEST_ALL(T) \ + T (int32_t) \ + T (uint32_t) \ + T (int64_t) \ + T (uint64_t) + +TEST_ALL (DEF_LOOP) + +/* { dg-final { scan-assembler-times {\tsmulh\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s, z[0-9]+\.s\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tumulh\tz[0-9]+\.s, p[0-7]/m, z[0-9]+\.s, z[0-9]+\.s\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tsmulh\tz[0-9]+\.d, p[0-7]/m, z[0-9]+\.d, z[0-9]+\.d\n} 1 } } */ +/* { dg-final { scan-assembler-times {\tumulh\tz[0-9]+\.d, p[0-7]/m, z[0-9]+\.d, z[0-9]+\.d\n} 1 } } */ Index: gcc/testsuite/gcc.target/aarch64/sve/mul_highpart_1_run.c =================================================================== --- /dev/null 2018-02-08 11:17:10.862716283 +0000 +++ gcc/testsuite/gcc.target/aarch64/sve/mul_highpart_1_run.c 2018-02-08 13:51:56.253511883 +0000 @@ -0,0 +1,29 @@ +/* { dg-do run } */ +/* { dg-options "-O2 -ftree-vectorize -fno-vect-cost-model --save-temps" } */ + +#include "mul_highpart_1.c" + +#define N 79 + +#define TEST_LOOP(TYPE) \ + { \ + TYPE dst[N], src[N]; \ + for (int i = 0; i < N; ++i) \ + { \ + src[i] = i * 7 + i % 3; \ + if (i % 11 > 7) \ + src[i] = -src[i]; \ + asm volatile ("" ::: "memory"); \ + } \ + mod_##TYPE (dst, src, N); \ + for (int i = 0; i < N; ++i) \ + if (dst[i] != src[i] % 17) \ + __builtin_abort (); \ + } + +int +main (void) +{ + TEST_ALL (TEST_LOOP); + return 0; +}