From patchwork Fri Sep 15 16:42:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723144 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488803wro; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEAQW2ygf2BST8bvqtmM2DqUAzA6Ey33F6E/2n9+MgIooAqpjLVyBEJNLLRt2vvL9KAe2XA X-Received: by 2002:ac8:5a47:0:b0:40f:f058:1478 with SMTP id o7-20020ac85a47000000b0040ff0581478mr2740866qta.30.1694796279262; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796279; cv=none; d=google.com; s=arc-20160816; b=wbNQYLLrrqH8lD+OeCWK25FHdx/kDeDs+NcQ6YcPlLMPJ5f2Xg1vmqOi/0NGS1cB9x 5yurByyYSg7ZQtQw/8EtvCStscImRp7kncFNJ3pTPIIweaAz4Hxi1SsQ01j3vcXUt3Gv RZSKUxV5nlONK5sbRacjkDD9U2GRKd7FpdHhr6PEPjVC16m5Z6VxadHIAYYW6OecB0+e UCfl4iDtvhLLbLx0zSiSSVYs6MFigIoxB3sKr1gb9dGii0neQEh3KIcT7UwPgs2ohxdS 0ToRfUSso0smmAcy2ENTJDcng1IG6EZ2HpO0KN1kXiEgUx5bvH6Hbc0BM6Z6QI/YMGn1 tD3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=b1NLv1NBVG2fZ70ohDbUJX8NkURlg/p6IknSOTaveXw=; fh=9wguxkwuuSBbOibCpq3+Qud4pl++l1lkTpkAOiV7WAk=; b=mK0IBgqxlPQK1wBSs4n43w2xdwuW8cBI+WQE0xNqLJ2zaCBLLvNU4hM/wxGXglDtXO 2COaci+38hWAqqbPqNaV34ag3RfEZ+u1DwSMo7qbQGpRqfidZGvVeKeE8uqsdXXpU07D h1H0M5AYZzDMFAWb8y+gJ+Z7gYYwNS9N0r82bcmPR0FYkCgaI+MV/12CCUYthURkZ8f1 UOgOextRkPIUi2lOTm6/P2LTvTcwQqRzqE8dgj4FhyfXFIu+JbGiZQZ3VkZKBulZ5SfK sD/UBXmsFPHhvhjs0KGaGGMjyTqk9fPKZrmmtW008oC97U0ZNOPF8zIWPLZnrO23QrwP l5MQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T6oMtT9W; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b4-20020ac85bc4000000b0040378f30d44si2643118qtb.172.2023.09.15.09.44.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T6oMtT9W; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBtu-0004QY-6y; Fri, 15 Sep 2023 12:42:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBts-0004Kf-OG for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:36 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtr-0004sq-4E for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:36 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fe39555a0so2117534b3a.3 for ; Fri, 15 Sep 2023 09:42:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796153; x=1695400953; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b1NLv1NBVG2fZ70ohDbUJX8NkURlg/p6IknSOTaveXw=; b=T6oMtT9WVYjCicF1ZWc7TNMa6Yn3ik7rnBAsqvlZmPoG7EQKquju2V1NjQdkPvHLws y7wgx5tWN/4J67jjrjYrdDwCXMR03jLNMtzQDQsKlyNz/E5c41ZhdWdrY8uEhEVUr1Iz QZUqJQ+LdEQ3R0BeiNVz0bj9p4Cu/8Voq5WOfskeyZvtJKU4QAJxchTEZH7r6H5PIhbR rCmWfVxWpabXbR+mAzqPq3d/IPK14lPwfWaX9Y9YmQNtedS0HmKM4+OKwvhT2rT10Sva LhpN+hMp3mCl9/+nfeG8RzCPvmPNCwztNN4Bko39Li6K5nW9wN7ozM8AYkPYkaFMQHcT yV0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796153; x=1695400953; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b1NLv1NBVG2fZ70ohDbUJX8NkURlg/p6IknSOTaveXw=; b=YfUgnAoK3SZgBk1H6kg3GRkB5LnuoRjykoZbb6nMpnuub2KPzGC9KmSB++uV1Z6udA uaOBQ//jhYsrf/d+yggLPBbYQ9GAixZG5GAB7O6fut1l7ERlccsreZyIWmbgg7sJML4D oHTrpvglXfHY1dI1qAheYlh35oAch9ES1k1ghFPxcWm/G0Sqeo6GfW0zO9DtNTsYwGi5 bMQG0w/9YfC+0yACPLyT/uwG5Dqx2zFApJRuh+oYDgdEDqa0yb8LqfgglwG2Vvw+twO/ dqI15boNiuYyq4i3Q7t4qi00BYuA4s4BxJFCMMDqePAWX3kLdcuLPchW29eO9PDTt+fr GJbA== X-Gm-Message-State: AOJu0YzYB8Dk1q5NXfaFVBHZC7kZpwzEMikayCUuJbXzyERxwro5YwIQ QQdk04x9e2z6IrYqku/dsFI9O54UhYFPLJVhEzI= X-Received: by 2002:a05:6a00:1996:b0:68f:dfea:9100 with SMTP id d22-20020a056a00199600b0068fdfea9100mr2451745pfl.21.1694796153619; Fri, 15 Sep 2023 09:42:33 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel Subject: [PULL 01/19] crypto: Add generic 8-bit carry-less multiply routines Date: Fri, 15 Sep 2023 09:42:13 -0700 Message-Id: <20230915164231.123580-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Ard Biesheuvel Signed-off-by: Richard Henderson --- include/crypto/clmul.h | 41 +++++++++++++++++++++++++++++ crypto/clmul.c | 60 ++++++++++++++++++++++++++++++++++++++++++ crypto/meson.build | 9 ++++--- 3 files changed, 107 insertions(+), 3 deletions(-) create mode 100644 include/crypto/clmul.h create mode 100644 crypto/clmul.c diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h new file mode 100644 index 0000000000..153b5e3057 --- /dev/null +++ b/include/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef CRYPTO_CLMUL_H +#define CRYPTO_CLMUL_H + +/** + * clmul_8x8_low: + * + * Perform eight 8x8->8 carry-less multiplies. + */ +uint64_t clmul_8x8_low(uint64_t, uint64_t); + +/** + * clmul_8x4_even: + * + * Perform four 8x8->16 carry-less multiplies. + * The odd bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_even(uint64_t, uint64_t); + +/** + * clmul_8x4_odd: + * + * Perform four 8x8->16 carry-less multiplies. + * The even bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_odd(uint64_t, uint64_t); + +/** + * clmul_8x4_packed: + * + * Perform four 8x8->16 carry-less multiplies. + */ +uint64_t clmul_8x4_packed(uint32_t, uint32_t); + +#endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c new file mode 100644 index 0000000000..82d873fee5 --- /dev/null +++ b/crypto/clmul.c @@ -0,0 +1,60 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#include "qemu/osdep.h" +#include "crypto/clmul.h" + +uint64_t clmul_8x8_low(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0101010101010101ull) * 0xff; + r ^= m & mask; + m = (m << 1) & 0xfefefefefefefefeull; + n >>= 1; + } + return r; +} + +static uint64_t clmul_8x4_even_int(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0001000100010001ull) * 0xffff; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_8x4_even(uint64_t n, uint64_t m) +{ + n &= 0x00ff00ff00ff00ffull; + m &= 0x00ff00ff00ff00ffull; + return clmul_8x4_even_int(n, m); +} + +uint64_t clmul_8x4_odd(uint64_t n, uint64_t m) +{ + return clmul_8x4_even(n >> 8, m >> 8); +} + +static uint64_t unpack_8_to_16(uint64_t x) +{ + return (x & 0x000000ff) + | ((x & 0x0000ff00) << 8) + | ((x & 0x00ff0000) << 16) + | ((x & 0xff000000) << 24); +} + +uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) +{ + return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); +} diff --git a/crypto/meson.build b/crypto/meson.build index 5f03a30d34..9ac1a89802 100644 --- a/crypto/meson.build +++ b/crypto/meson.build @@ -48,9 +48,12 @@ if have_afalg endif crypto_ss.add(when: gnutls, if_true: files('tls-cipher-suites.c')) -util_ss.add(files('sm4.c')) -util_ss.add(files('aes.c')) -util_ss.add(files('init.c')) +util_ss.add(files( + 'aes.c', + 'clmul.c', + 'init.c', + 'sm4.c', +)) if gnutls.found() util_ss.add(gnutls) endif From patchwork Fri Sep 15 16:42:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723147 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488941wro; Fri, 15 Sep 2023 09:45:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFhqrvd4GM0lBzownGFq6jwI5/coWGfgMNt6aRJNLr1SDkVkxaGORcANOazTlZkpTAXgdTA X-Received: by 2002:a0c:aa07:0:b0:641:8ba9:68ab with SMTP id d7-20020a0caa07000000b006418ba968abmr2116926qvb.42.1694796301132; Fri, 15 Sep 2023 09:45:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796301; cv=none; d=google.com; s=arc-20160816; b=katBPb7jjvMohCY2q1kdddKtLly2YuAw4nSDu1Jcp6RhudsMDZcuzVtNYNSvV7QMtX UL7AJ2FT6tHYz61ZMJpZfI6MEcinKcyvmGJDRqd4VeyYtMbvDoS/R9EMaND06NOp4pbO EkrXKEl+7sLJERblIutCdOWV4UuuhnkYxGwFzFF6BLNELvatQhGKo3Hzi/eIc95UIinC 2PkpPIwD1V3X+8k+q28kMNKCX5BMf9X5sw0iHNCJe35z40Y4EwwAxHfvq3QJroNQasDY o6M0HyXgQnuddxXjdoC2J9WJgw7grob+G64ZzGcwEjnfNIQLR6x1fsoE5gC3nZ7IA+hV enxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=r8TnoO1tEi6BKTTer6dzQslM6M5l0y17tGjLbzRdHTVXni1Lnme/Cs0agc5792YChc LtzpDHaiCaFJU7CDmcw1HXOrVTicmklKSBKp4ULwRodx94cRVcV0I0s/GtrLese+06g9 fYOlKojt3Sy0bOeqjOw4FZB5pFHsGlKglVSc7AeFGx84tdgK0TBaPjgHSoCsp5QbVsdO mLhdr/XsjJpXV/O9GgFW81b5JoXy4kmG4lWsQMqtgHB7Z9ORi796Q0ytQnybzw3nj4Ms AgZQSfXWacJAVvCoWv2ueZNoU6sJ/wXguGYgPd/jDYAWvhybuDsTcQ61SupZuky9D+XG NZNw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nznHy4yv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id tn24-20020a05620a3c1800b0076c93ca09b6si2462828qkn.189.2023.09.15.09.45.00 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:45:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nznHy4yv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBtu-0004Qh-Os; Fri, 15 Sep 2023 12:42:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtt-0004QN-NE for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:37 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtr-0004sy-OT for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:37 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-68c576d35feso2118524b3a.2 for ; Fri, 15 Sep 2023 09:42:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796154; x=1695400954; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; b=nznHy4yvS/+/CVlcHBCQyb5mr6F0ku8JnasA+UUo98x21HSqg/W9jsZCh7MEE7PHp8 NyN9Izgff7ny0iE3gJGTSgGR68+IAjZqSkZfImol8nGFwn/GjxixUiMMBwtU5/8jNfmp rTbmS1wYDc60FjPpztvM7wVlqa1z++3sOOGHIBWn0DrernO2u184UImwPCL8EuygJ0zC t8PipW7pI4g6zGb+EV376hDXM+sfPrcBS/9o+9zc2o0pKtX1q8azMEa0aBqAjkmS5YwZ 2KGSZcqunA77sYH7QcK31e4xaFzmZBcO4MgBEaqDbKV6Bbg1TiLSitkgRist93GIHr9/ pPIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796154; x=1695400954; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; b=EMYzSdUBlojTW+ku7h+SX69qpSpQP+gE95BIeRYW8YhY+oAR/dkgL5RrGyUeR1BU04 simEcJvdbvD+CQFsiCfdGxiV4V5bGCAsyNcBwT7naURx0jhspyiwPXfYSXin3lS2xqOq wwodlatR5p67sBSDyZypiVVxnrj2OSimbuZoam2UVBbtS7aV+5qGj9NUIW6QI8RvAtbi fzYjwoXanett+tYHRq0y9flZu6ZD23Qbk7/JHVf6mpA0lMKdkZM01crGIwNFshnqOFjg niuNgRGd6/loyXd8vPsTmwhzG/MWP9/2RnNL2/pR5rvz3coKlTwUDUjkueXWP9UKD8uM IXdA== X-Gm-Message-State: AOJu0YxbN3H5axlxfXDCcRHZQ19eInBZ///XyWVtVT+HQiGiG269z6lN u4hIBK1AtdgFI9DfJJkDwfwky2+F7rX6KxSK1l0= X-Received: by 2002:a05:6a21:2720:b0:157:609f:6012 with SMTP id rm32-20020a056a21272000b00157609f6012mr2153268pzb.61.1694796154461; Fri, 15 Sep 2023 09:42:34 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 02/19] target/arm: Use clmul_8* routines Date: Fri, 15 Sep 2023 09:42:14 -0700 Message-Id: <20230915164231.123580-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of pmull_h. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_internal.h | 5 ---- target/arm/tcg/mve_helper.c | 8 ++---- target/arm/tcg/vec_helper.c | 53 ++++------------------------------- 3 files changed, 9 insertions(+), 57 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index 1f4ed80ff7..c4afba6d9f 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,11 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 8 x 8 -> 16 vector polynomial multiply where the inputs are - * in the low 8 bits of each 16-bit element -*/ -uint64_t pmull_h(uint64_t op1, uint64_t op2); /* * 16 x 16 -> 32 vector polynomial multiply where the inputs are * in the low 16 bits of each 32-bit element diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 403b345ea3..96ddfb4b3a 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -26,6 +26,7 @@ #include "exec/exec-all.h" #include "tcg/tcg.h" #include "fpu/softfloat.h" +#include "crypto/clmul.h" static uint16_t mve_eci_mask(CPUARMState *env) { @@ -984,15 +985,12 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPH_MASK 0x00ff00ff00ff00ffULL #define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBH(N, M) pmull_h((N) & VMULLPH_MASK, (M) & VMULLPH_MASK) -#define DO_VMULLPTH(N, M) DO_VMULLPBH((N) >> 8, (M) >> 8) #define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) #define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) -DO_2OP(vmullpbh, 8, uint64_t, DO_VMULLPBH) -DO_2OP(vmullpth, 8, uint64_t, DO_VMULLPTH) +DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) +DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 6712a2c790..cd630ff905 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -23,6 +23,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "qemu/int128.h" +#include "crypto/clmul.h" #include "vec_internal.h" /* @@ -1986,21 +1987,11 @@ void HELPER(gvec_ushl_h)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = n[i]; - uint64_t mm = m[i]; - uint64_t rr = 0; - - for (j = 0; j < 8; ++j) { - uint64_t mask = (nn & 0x0101010101010101ull) * 0xff; - rr ^= mm & mask; - mm = (mm << 1) & 0xfefefefefefefefeull; - nn >>= 1; - } - d[i] = rr; + d[i] = clmul_8x8_low(n[i], m[i]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } @@ -2038,22 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -/* - * 8x8->16 polynomial multiply. - * - * The byte inputs are expanded to (or extracted from) half-words. - * Note that neon and sve2 get the inputs from different positions. - * This allows 4 bytes to be processed in parallel with uint64_t. - */ - -static uint64_t expand_byte_to_half(uint64_t x) -{ - return (x & 0x000000ff) - | ((x & 0x0000ff00) << 8) - | ((x & 0x00ff0000) << 16) - | ((x & 0xff000000) << 24); -} - uint64_t pmull_w(uint64_t op1, uint64_t op2) { uint64_t result = 0; @@ -2067,29 +2042,16 @@ uint64_t pmull_w(uint64_t op1, uint64_t op2) return result; } -uint64_t pmull_h(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 8; ++i) { - uint64_t mask = (op1 & 0x0001000100010001ull) * 0xffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; uint64_t nn = n[hi], mm = m[hi]; - d[0] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[0] = clmul_8x4_packed(nn, mm); nn >>= 32; mm >>= 32; - d[1] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[1] = clmul_8x4_packed(nn, mm); clear_tail(d, 16, simd_maxsz(desc)); } @@ -2102,10 +2064,7 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = (n[i] >> shift) & 0x00ff00ff00ff00ffull; - uint64_t mm = (m[i] >> shift) & 0x00ff00ff00ff00ffull; - - d[i] = pmull_h(nn, mm); + d[i] = clmul_8x4_even(n[i] >> shift, m[i] >> shift); } } From patchwork Fri Sep 15 16:42:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723148 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp489010wro; Fri, 15 Sep 2023 09:45:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFuH/ESyOSu5ygswa4zbL4UgYd5yzGU+0jUNFTFi6YIMdxKmiapZF2i/NFrwl28zAeyp+GP X-Received: by 2002:a05:6102:3a55:b0:44d:3f58:c42f with SMTP id c21-20020a0561023a5500b0044d3f58c42fmr2224585vsu.11.1694796309936; Fri, 15 Sep 2023 09:45:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796309; cv=none; d=google.com; s=arc-20160816; b=jPRcj6NnylM0DADihPZWjdV2CmkxxfUNIP7gS/KXhi9X58cHhVvXZCXFhBUFcz0Jnl a7roUpbUCC8e8k8MftzMPYKgJtoUmGYaX/PwA7jJDvvq49QgC7X/rhxureI0lh8tW5F7 vVuD2QYBs3m3ohel8tOQFCBBPbJ6Q05y5eo4waImC5/P5LKG7l0HeMJoUhS9q+77wYnA p8UnEYYeadZdDtreW8frwFHTMxRcnSLt4+l09rISygJPTu2B8fPF9M/r/iespvXiuPsT fg9Ix+KmXTn8fTM+tUggH+M8hQ5I8w1xDusDBCTjy8pRdBXDM7zZPMDHt7L+o5gHaBdY Gthg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=i6KDlYnvpFknUoBFdY4l90oaLjQJFIMI6wu5rJzg6HQ8AQsd3VmWxIN5QC1jiT9YOS w63hojQaHYm39xzjRLOqDKBvfJgRunlZqqp2OGlLJgeJi7bQqMxxmhnBXHOQP3Ght0E8 EpoLhf0sV4PbrLeiZFYq3VN+dwOkEzM8mVLswBJVpNrG2X5yzp49MPxGCu6crwa5Gbql n7RFyr1Q/bLBxJY259+GzXs2dfLcydY4F3igKSdssZ19spp+4/keSOpGCUoRSBQESgPL t9gqnJReIkD7PZwxCTScbaXZGJOaeihd3SUYzjzXduSeEyWfy/x6ejSrR8qvIUwS2dgi IIgg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZRkTKckm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ri12-20020a05620a8f8c00b00770f3ea254fsi2494123qkn.171.2023.09.15.09.45.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:45:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ZRkTKckm; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBtv-0004Rb-Va; Fri, 15 Sep 2023 12:42:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtu-0004QZ-6p for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:38 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBts-0004t4-KD for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:37 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fb2e9ebcdso1944656b3a.2 for ; Fri, 15 Sep 2023 09:42:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796155; x=1695400955; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; b=ZRkTKckmQP4GfMt7g1ytDalJALVslhW/D0RWtSEo92CRpLuuL1T0ginyFTDaeDm7AT GTetWbC6pWl0p7xFPvCnnfTia73g9DNvXzwkfsXtlLU+bOxbJk60dAMY8Aozl0b9+KuI AJ0gJegSjZPnHpzV28HGHxWcELqSJwBqW3q6+eNVQsLDw3ro7ev0oZ6ZcW7OHimVa4d7 GnceHVWYn+FRji96PrbjdF7JS6iivlZ21dmHzQ5cM2+jDryfV9y6oKGqXE41ffAdZJI1 W2UJqrV/3PTpODVFFd6Sx1R7zHkiQFrAKTFzuQmbvt4yh5/FMF32n+vVJ/IAjejwrI1q 1b2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796155; x=1695400955; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; b=THwjpuVFhCu7ILURIDXUv1Jr5sRGU7q8TxdqPKX7yqREFwcS6533o1WhtX5IkohRg0 xn0IqahycIku76PBmkknc0WiTOSo9U8PUmirOmZWVhnnPNdfBK2YaoRHeYpEICYvOFNV 5uxP+MZiGioTr3BLLEbEBEDfid6Cai107/cskwU9VGN0rNENOJQbhaha98lQ6T0gCsqq B7cfmkxRb3SE6PM6aTyIrdkKFjNwDT7pHC0ybgJoanI4Ejmm25lZXqhLrBDKKhpxZACr jN0dQFhVKxVbEx1KioKDUBQZe+U/zKKlGrSsekaLgcjlHTl6FZ2iccl/hrwgr1TMMaDI X8eQ== X-Gm-Message-State: AOJu0Yw2dyS0CamPz8MxiBEABMRgSA82SYETMfc5I99kgTx1/UAeasZj l7Ekt5k6RmEIxtsDgzpbY736JGNnxdzomKh9vmE= X-Received: by 2002:a05:6a20:12c4:b0:159:b45e:1c7d with SMTP id v4-20020a056a2012c400b00159b45e1c7dmr2651096pzg.15.1694796155282; Fri, 15 Sep 2023 09:42:35 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 03/19] target/s390x: Use clmul_8* routines Date: Fri, 15 Sep 2023 09:42:15 -0700 Message-Id: <20230915164231.123580-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of galois_multiply8. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 32 ++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 53ab5c5eb3..edff4d6b2b 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -14,6 +14,7 @@ #include "vec.h" #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" +#include "crypto/clmul.h" static bool s390_vec_is_zero(const S390Vector *v) { @@ -179,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(8, 16) DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) @@ -203,6 +203,34 @@ static S390Vector galois_multiply64(uint64_t a, uint64_t b) return res; } +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +static inline uint64_t do_gfma8(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_8x4_even(n, m) ^ clmul_8x4_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm8)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma8(q2[0], q3[0], 0); + q1[1] = do_gfma8(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t desc) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma8(q2[0], q3[0], q4[0]); + q1[1] = do_gfma8(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -220,7 +248,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(8, 16) DEF_VGFM(16, 32) DEF_VGFM(32, 64) @@ -257,7 +284,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(8, 16) DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) From patchwork Fri Sep 15 16:42:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723143 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488805wro; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFsgFYnR4r0XM4JhzM0l+SMVwTcaG6sZsrXu1609LF8oFo8yTqEeH7Tqfxbppo4pQQEsIFy X-Received: by 2002:ac8:5a13:0:b0:403:72fa:630b with SMTP id n19-20020ac85a13000000b0040372fa630bmr3065128qta.58.1694796279479; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796279; cv=none; d=google.com; s=arc-20160816; b=IsBN5Uw/ykIy0CrVdD+2W9SadUQ9vFIGX2vCL8+zCpbuDXn5xOyJslj9qBIQ02QXiI jCF+b4pdYAZL0jMBrO+Azq29oi26gBp5BTNnCfdu/fifq8fH6WLvJk75dknKdo4hLaH+ 3QKTCDaOvTIkzphkWAy+Tj6U487wA3b38/dHVlz16DBC9bpcRTSQT1Yho3bYmot2gEUS yO3YaBurvP/cq2NBaazFtiZy3TQcNl8a9eRW9+bS+ONmwBED4rrPc90Nrfx0UlwdZ9ii Z0EIPhfSzndzH8wlrXhrPL8YuT2G5T1IkPzbw0Oip2o8Jt5GEWMbHihPMNRg2rBMl99H 0bYg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=lrofiEdqgOl3RcwS/CbqZ7l9hqJEkeBbwbYNYeWuQXE=; fh=PnYt+qEB9tAfMKoqBm2xjKOFpYyFFGPudh5cVIoieJM=; b=lD571LS3p1yVAwvzDSxja0rFN03aq0a26Pb4Q8bh650xbcXlCFGIyIWwZpUijVSiNo 3ErZ7vTGjSUXzedo0HNVdUoFuSQpditQGMri9vikyijrPEy8qVmxk/HVY0oKw93P3Lxv OZgkPpNeVZmifOYBrT4d5H99mtOntX6AtOIQm8bVQHAeDxDZufKdbRGaDcKDvHZoJ08L bN/Ti6VyXR8kwl0PwXCobq6ybM1Vbg8tzV2dDZb164hN/OY4yDgytrQi3YO+cnlZIxWN v2OS6mMY21czdd3TmQy+8DUH+KVkq1ZZtMo2sx+vWM0JWbCSAJq7JYFt3tL2EqSmWF62 LDgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lGpy69nP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w6-20020ac87e86000000b00410a99cfc4dsi2618341qtj.341.2023.09.15.09.44.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lGpy69nP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBtw-0004Rn-HF; Fri, 15 Sep 2023 12:42:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtv-0004Qq-1t for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:39 -0400 Received: from mail-pf1-x431.google.com ([2607:f8b0:4864:20::431]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtt-0004tN-Dc for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:38 -0400 Received: by mail-pf1-x431.google.com with SMTP id d2e1a72fcca58-68fdd6011f2so1946257b3a.3 for ; Fri, 15 Sep 2023 09:42:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796156; x=1695400956; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=lrofiEdqgOl3RcwS/CbqZ7l9hqJEkeBbwbYNYeWuQXE=; b=lGpy69nPnY5EoWwgYozfK5+UWIz9Uv4PFm4haZPNsMMa4Gcv5/igRGOTSHZ20hEnnT NgjL/JRyc9n7HRZvVDq8++Tj0hupX1WK2OF81JIeaEr1J1wR5mwvzIlLyCewQI5glnlZ oPZlZo6dBpxvPf9ax/OlWy4W5HlpkfMEW4GgkUBRuWp9OgIffGtOoARyAuOEDpeFV3Bl 94hiAqBFI0gaEpD14CmEyqTvVwF0OqeFfqYM2kDm9asOXWV3NGtwBGXOVhR1UQm754ki MfCH1zIWohV52woEHKw2vWzV5kNn0Q5Atcnp4DfNFpaPJ2+A8iK75s5Vt4tmdYuXAQyv h5Kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796156; x=1695400956; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lrofiEdqgOl3RcwS/CbqZ7l9hqJEkeBbwbYNYeWuQXE=; b=Fk+XyPMl1OylBQivnQ/CKJTAEAjB/VldYv8QTmS/K/6QYQDjG7HlXRzIoAOLg8h4hq IuOVBbkxa9tUb5F+bjBv7CZStJ/e+7QNjBCvndcybWa6p38XZcdBCm4PN5r7ZcH9Yv99 c7Tv8X9b5vsDOFUiy1yR1aRJKKOKCpWmwz4IJO0IGFgTW7gkcoLEpKZXZeCjU8mFeIHc XrJ5/mrKmP86rBqlySzfWeM9rVs3EsoCzzKBgkEtUEP26VJaf/rpoY0T7fSdPuKaEFHK P89W1tvWNdGwIAIRp5t0u8KTAx0tb29cggtfLjkV5KddOCCzREuw/eviVmvqKLYGeiim 2NbQ== X-Gm-Message-State: AOJu0Ywi7lxGrZKiE8noefU7DCVMnJmg/TN807soy83x+Sog1Sa3/De+ Sr/i4wRwiUQHEGlLtO6lzXXL4tQ00ERXmLJGVrI= X-Received: by 2002:a05:6a00:23c4:b0:68e:29a6:e247 with SMTP id g4-20020a056a0023c400b0068e29a6e247mr2702662pfc.10.1694796156214; Fri, 15 Sep 2023 09:42:36 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 04/19] target/ppc: Use clmul_8* routines Date: Fri, 15 Sep 2023 09:42:16 -0700 Message-Id: <20230915164231.123580-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::431; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x431.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 96cdb3c7e3..f45b24d321 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -25,6 +25,7 @@ #include "exec/helper-proto.h" #include "crypto/aes.h" #include "crypto/aes-round.h" +#include "crypto/clmul.h" #include "fpu/softfloat.h" #include "qapi/error.h" #include "qemu/guest-random.h" @@ -1424,6 +1425,18 @@ void helper_vbpermq(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) #undef VBPERMQ_INDEX #undef VBPERMQ_DW +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_8x4_even(aa, bb) ^ clmul_8x4_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1444,7 +1457,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumb, u8, u16, uint16_t) PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) From patchwork Fri Sep 15 16:42:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723149 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp489029wro; Fri, 15 Sep 2023 09:45:12 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHYd1t+M1zA5fa8Dhya0IPV65Ap/OJzU8DlHgw4uAyha1NmBPNPvy6s3PTYY9pwANyPhBip X-Received: by 2002:a05:622a:120b:b0:416:5e11:f7ec with SMTP id y11-20020a05622a120b00b004165e11f7ecmr2818088qtx.52.1694796312637; Fri, 15 Sep 2023 09:45:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796312; cv=none; d=google.com; s=arc-20160816; b=hvCOGg2dSsPoTGraLGXy64WrxYNFVxZ/DldnKpG+/Zz+aWnvU5XpefzGbTPOZv2Nut x5vbqBEooaj2r9FWjnfpFDfBkh4fDWB7JLoY2mzmuMoW6690dpx0OdJbSmB5+lMIysOn jRmWDrEo1Phxp03ARtB5b7CS7dOFttH7GQZs6PJIkXvOGE8EJlEwqvgYS7cKu0OHJYpR Pes3sae02c14yrx9jY/unVxMVqjF+c2fgDWYb3ScbRVD44ytORQFD4t8NqQRQaGgrMpK 7yLjBl1fz3S9ET+31HJZuC9qMFPsVDwBeMnhtqItmRbEy6M4zmEgD9wMhQ8VYUUAaHic K5mA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=E2sTDZYIaL3GhBPyCh9S4Dw9cyt088qCUZyaRlZTVTE=; fh=9wguxkwuuSBbOibCpq3+Qud4pl++l1lkTpkAOiV7WAk=; b=KDfiUzmnePdPQVscRfAsdyykiKEaiEBBqzLyVU0s7nRvvmWQKjDD8f6zubuxBBdqVY Dz2ymxL0txPQlue/ahpOiNGt5c+Jty5WOnXhp6tjzi/7SrH8mZsXS2NtVjwpUHPVXKFV AV52cxJJBJbP2hs1bsLv2KN0uN34vGJHskLNq+O/fsnB0yjiAwcLOWmJy/6HfOqQks9K UGoZScJUYkPxz5ilOgKKFHeL3R8EMH2Qn+uCLhGtspqUxigZOWP4b4b/vESW+YV01H1k PLkwP7CVtI4wCtPap58Ifpr9XiR2Ech6XPfhpH5zTie5zTvnIV7UkkipvgQWqe35jaeY 9Ykw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wVFzv+ax; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s14-20020a05622a1a8e00b00403f3f821d2si2486635qtc.426.2023.09.15.09.45.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:45:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wVFzv+ax; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBtx-0004SA-E7; Fri, 15 Sep 2023 12:42:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtv-0004Rc-VE for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:39 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtu-0004tY-FB for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:39 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-68fb2e9ebbfso1947658b3a.2 for ; Fri, 15 Sep 2023 09:42:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796157; x=1695400957; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E2sTDZYIaL3GhBPyCh9S4Dw9cyt088qCUZyaRlZTVTE=; b=wVFzv+ax3s0/d6WnXWsYafhnibMfwMFeWc9A2JqVf5eZhr7osijrOSuch2a2Du/uMt QU9adusygMlJMmMVtItEfbRCyLA8/CsWb7EhUiU0kvezLn/Kqtjw88Lx1wuOcipQ3lGr PeyevTDicKHNTr9z2Rqxe0e/pe6S5KXPuMZmV9ZlHqlu2iTH9n0ur4klP4WKDmgFwYOn OX5V3QAr8ypabKLfoHNy18BQptHpmv7hamxhpw6SjEyhxHvmbyyM/2UXFmMz+7e5ddVC R+s5JpBVUrLBqt6s/FM1ANPy+AKmBbIwetQcFA7RCwBeBmDM6ltsO+FoCPQu3jgIyyFs opKQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796157; x=1695400957; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E2sTDZYIaL3GhBPyCh9S4Dw9cyt088qCUZyaRlZTVTE=; b=TPpDVP4pjUxMKMsOFvUSv8amnBRyO5qAdA1/JEGPgfs5giJJUIkVIzFHujDvDiJMQ8 VlZrgCd2nUnVOq7cZ51ODlf2q6tge73jw+e9JlN+WJi7cNEMZynMkAJhRXJFheVu2que cxz+/jwItEaoDg8NevljoCzSk+sCVCNzQQ09UfJqMXQipArQTDQZX7g1LXrnAjbr6wCm 9ytoNaQLBOEtBdtIRAkfdZP16yGvyun+sxF1rr8+TjuPU3VXJRBbG1IzbL9+mDCkti+w xvzUXqagbNInEj9alQ5XPB11ij9GFtLH3Dp/i/D1lQnA9WhXZhEBzEy1ebAcJ7x0Feq5 yJgA== X-Gm-Message-State: AOJu0YyvvLZGFCshYLEB/LGuWC/Gfrk7Sugk8ASSSIs0Fv0XqrPNFgdj KBKxtCtkBg2TybjqsxfSoe4ozI+fg5EOM1Yvk2o= X-Received: by 2002:a05:6a00:1acd:b0:68a:604f:420d with SMTP id f13-20020a056a001acd00b0068a604f420dmr2407297pfv.3.1694796157114; Fri, 15 Sep 2023 09:42:37 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel Subject: [PULL 05/19] crypto: Add generic 16-bit carry-less multiply routines Date: Fri, 15 Sep 2023 09:42:17 -0700 Message-Id: <20230915164231.123580-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Ard Biesheuvel Signed-off-by: Richard Henderson --- include/crypto/clmul.h | 16 ++++++++++++++++ crypto/clmul.c | 21 +++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 153b5e3057..72672b237c 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -38,4 +38,20 @@ uint64_t clmul_8x4_odd(uint64_t, uint64_t); */ uint64_t clmul_8x4_packed(uint32_t, uint32_t); +/** + * clmul_16x2_even: + * + * Perform two 16x16->32 carry-less multiplies. + * The odd words of the inputs are ignored. + */ +uint64_t clmul_16x2_even(uint64_t, uint64_t); + +/** + * clmul_16x2_odd: + * + * Perform two 16x16->32 carry-less multiplies. + * The even words of the inputs are ignored. + */ +uint64_t clmul_16x2_odd(uint64_t, uint64_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 82d873fee5..2c87cfbf8a 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -58,3 +58,24 @@ uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) { return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); } + +uint64_t clmul_16x2_even(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + n &= 0x0000ffff0000ffffull; + m &= 0x0000ffff0000ffffull; + + for (int i = 0; i < 16; ++i) { + uint64_t mask = (n & 0x0000000100000001ull) * 0xffffffffull; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) +{ + return clmul_16x2_even(n >> 16, m >> 16); +} From patchwork Fri Sep 15 16:42:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723135 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488448wro; Fri, 15 Sep 2023 09:43:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHf14Y5fhQnTrSy4O6iV15Qgrui77SMxRMREjoKzdMxyiAmjDBHA9LLr5PXfNNMpGoGP9k8 X-Received: by 2002:a05:620a:1a0d:b0:76f:ac3:806f with SMTP id bk13-20020a05620a1a0d00b0076f0ac3806fmr2934697qkb.13.1694796226541; Fri, 15 Sep 2023 09:43:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796226; cv=none; d=google.com; s=arc-20160816; b=fSMUbP7HYAZMb05WY6lWKT12zlEQA+kI9LTBg7HrZ6K3UBpi4J5OkE7w+TbsXCYyw4 rOWAqHPEllMMHQPOGm506mD/ooIJXPN6u0WxxVh2EmouqS8TX4hHYTELw1KR5PTOkv9e GBXhGaD+jRFRMZ2lay0UHBLpGC7i58LNg2zZOWTNRK1tq9Cwu59FjlDUaoz6GUPtpFJm HysDVz2WJzZWEG6tsrUrBQScgov8R3ao/gVk8YklmNROeY6jCU0s1M+1TBEeVN9HG3k7 81+Yzbx4o0HQ3hNrETxS9qrZAox5SpNDgG7h3LOPPYuSAYS5OBxhee+6kc56ui1aFi3S AL+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=Cy0Vn6d1oiJzEpEN5baSD0Ul251EohWTTv7P+MlFEQQj+pdvhIyrO753CzWKUdqrRt N6tJdcab2vG0g3tu4vgqoEWgirwzywvrK1iYDJOn/zoyA9zkzoy3j4Xr4W2Nl83qbDX/ 5iyOWiOV84qSMLCNDL54x1ex6s7h9fh9B+MtBd2ou1JvuRvJSyG9mNMBtQjxcfHqWFzh KbPzepTNW3i0PETRy9DYfOM6o8NhSxOkStN7Tp26sVPkHHVkUImxTNX4cdiLzs35LW4f U/ZpX18TcaMCCWKCoLG0XLdodTtOlzYDHTaFbqiBqn8Ac4p4BWsV1S4K+j3pMw/JsvsZ 1G7Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uszINTY9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ra19-20020a05620a8c9300b00772dd17e0a7si2471623qkn.187.2023.09.15.09.43.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uszINTY9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBty-0004Sa-LZ; Fri, 15 Sep 2023 12:42:42 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtx-0004S2-5G for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:41 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtv-0004tu-GM for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:40 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-68fc292de9dso1741831b3a.0 for ; Fri, 15 Sep 2023 09:42:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796158; x=1695400958; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; b=uszINTY93z64c5xS2WK0WrEVGE6UGrafjRE03mus10CKe7mu7aE2FtW8qKKwJ8yjhO q+TvB/t9QUEC+DNRZbzW75fgRrtGyQeZCgoi+rn/ebq60HqfEuhz2ty7mNLinxx1Owpy Yr9eSV0LFtazxNtdybn/CzpPuandN51xVres2yVRO16HO6bXmfHEdKosVY20VukuL9+s GA+CUjYLfVU6vn3b4TO5v5sbZNCOcVplDuJaN/M0+/eSPIGixRz3taMekBoTb3DAxUIo lbosHRC1LI5O30G/E6gnrVirAiEBt494ub6MroAoro6hQ5sFX5t05oTW52GW5i1CjgJV fz3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796158; x=1695400958; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; b=W2UtM/gjQV7CRNj4/frA98EDp2hEBtGBR08j+R7FusbMOoyLp2YR9KIkK16Hs2byU9 ZKrWrltdiYVePg/JUetTfEzIdsGGBWW6BvA2ZkL6hz+Mc1Iw0pN1kj+CfMHbnuQpc7Wm nAEj7j6+j7yNpljBgIaYGPT1GFNhdPUVpCp87LQF/UlSoZPCBK+7J0D/Br2DL2kaqQjQ n4tTOZXRwDXKU9Wy7Qj13Ejcwu5zFPXWREIG2+XQ8XijK3vK/5kG1hkRGriC6VVvux8Y A5Jk30Bomlav8Ha7VhP84m29BajZzivSWyKeyfDxIW8gGhkS0fl23ZdR5E/MTeXskSgB w5yg== X-Gm-Message-State: AOJu0YwO7P8U1ApgDsTIHGnX7PszDdMrlp7MfkI6cbjiB55URP7Spc/m YDoiv8+xDDOn3nFZhmW4CN6n/N6ykV+xUXcgz5o= X-Received: by 2002:a05:6a00:1f97:b0:68f:cbd3:5b01 with SMTP id bg23-20020a056a001f9700b0068fcbd35b01mr5936723pfb.13.1694796158156; Fri, 15 Sep 2023 09:42:38 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 06/19] target/arm: Use clmul_16* routines Date: Fri, 15 Sep 2023 09:42:18 -0700 Message-Id: <20230915164231.123580-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of pmull_w. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_internal.h | 6 ------ target/arm/tcg/mve_helper.c | 8 ++------ target/arm/tcg/vec_helper.c | 13 ------------- 3 files changed, 2 insertions(+), 25 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index c4afba6d9f..3ca1b94ccf 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,12 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 16 x 16 -> 32 vector polynomial multiply where the inputs are - * in the low 16 bits of each 32-bit element - */ -uint64_t pmull_w(uint64_t op1, uint64_t op2); - /** * bfdotadd: * @sum: addend diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 96ddfb4b3a..c666a96ba1 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -985,14 +985,10 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) -#define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) - DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) -DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) -DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) +DO_2OP(vmullpbw, 8, uint64_t, clmul_16x2_even) +DO_2OP(vmullptw, 8, uint64_t, clmul_16x2_odd) /* * Because the computation type is at least twice as large as required, diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index cd630ff905..5def86b573 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2029,19 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -uint64_t pmull_w(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 16; ++i) { - uint64_t mask = (op1 & 0x0000000100000001ull) * 0xffffffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); From patchwork Fri Sep 15 16:42:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723146 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488861wro; Fri, 15 Sep 2023 09:44:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHxxfd7QiAtwqWQdCWjf3CY5NdnTdWV8JRKZNisNpKxiZjJFCo1WXV/fuYW8b46RrHSnXnL X-Received: by 2002:ac8:7d96:0:b0:410:37b2:3c98 with SMTP id c22-20020ac87d96000000b0041037b23c98mr2607444qtd.38.1694796288193; Fri, 15 Sep 2023 09:44:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796288; cv=none; d=google.com; s=arc-20160816; b=BYtdKjk1Bzjfm/0WIEIJMkNAuQRiX4RVF7Gau26nMAVUzwc+Fwi6+oKavZC7CnMm2a +UUjUIj3wUmkZ7enOZ5tNDKU+edISzaiEtYhZ7oFHVuuoYpaYFlNJq5KgcmoPx/yJztS ADhGy1bqctbhCpc2UY8v9AcVpkqeBtC8Cbhk2JtD6rY4AHTm7RvgCdhKoHTnqH/FAL+4 DvhytyP0tcFiTYxe75LMdnlwdeLygHnsWxtkAWzdDPOgjROTdM7MicRgmX25tc3NYTVW x6TlnFU9d79fqJWWRUzKzcX3oLJH95oV/3oUEzGJMLniFHVByaEvB7sL7Xu2ed/BCohE TaPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=qA36dECbcvmEYe8Uq4b9zIixRLRPiisiy0gacf2I3q33obShSfm4vEscrlKb7igsdh Ntbg3DmL4AuquUh9GKW483g36GdPIxgmEuYjhN69X0UtT4iAbTiZqIsrMpKGdipX5AgO ER4t1Nl2scngbQdsOCdEuYVyLSJDyMgBMlCeO5KX55fOKLjVQgzDA/r6f+kImZnEHIIS A5n6aVXFOriCCeiWIuppJ7hG/EgqqkDOiGESiCwogOEwLkjQZqBix7liBRBUstTjJdwN E/oojD6jzhSXuC7e7ullxGD7/hdjZ7NhB9BkUSxeOVKqU9ejv//Dg523b+zVjWXekZQo lKFw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HxRuXjve; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k2-20020a05620a414200b0076d8ffbd9cbsi2716372qko.414.2023.09.15.09.44.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HxRuXjve; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu0-0004T4-0s; Fri, 15 Sep 2023 12:42:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBty-0004SS-60 for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:42 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtw-0004uI-I4 for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:41 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-68fc9e0e22eso1947060b3a.1 for ; Fri, 15 Sep 2023 09:42:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796159; x=1695400959; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; b=HxRuXjvebGnRljb1E0EEDqebZiVoB76kgKh+dr9ErEL7LP1NabrrgxdkVERZESMjJ1 AnSXj5Xe+OY4PcjUdicca3Di1CKR84D/Vp4WgVu3dWkQI7L1b21GPe7WNPqUaosT+J4I TaiE5tduR1AEFE3u4z437PtqBUdpY5MV5B9hbx2Sj9vtevzgNp093GY7HEOtumOjWeZA b3XdagyFxxQTfiF/rU0qWLTxzGO9PbnBPJFkiMTNUVMj4DyE5j/1gm0sjrzC5q3c1ygN rt5FQcPPYVVxWJalDJPYs5lZQWu89jt886g/ZID7Kk6MWbL+z6YEyOIaFULInBCDo5Yq inug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796159; x=1695400959; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; b=hPMJ5DVJrzjXDx7VnoGX9IyApUKnE8r9igkmjt4Y2Ew2oxDpH6TqwJe52uHCb0/7+H oSP067DEEmpTlvDMtiM0JHTXBXoxr/03bUdseJKiJQyokzS6YF9Kn0zEXetAWwLPGQul gHgw+z+8/sWEgsz7wcld30cL0328ZSgqAsRK3E3xWbDqJTyx+LCYop21nbR1UN0G8MZl mGP9fhWxZeCJ6WhK5DZ5JIwoX8DOS1eXPaX4I6MzRXP/JEbcUhHWod39deoTWtIYKvEE vrMkYPC2U29CperXNMhWGkdBM9q1FzUcarZDdr9ohAEpZOrtu+dxm+cX7TnyYSswIIMI D3qA== X-Gm-Message-State: AOJu0YzbvjHbxdikhgxriBhsRdVKPxTCHjpFZWmP3PaJo4PFfYGlIHst c4hFX+A5/9Vzkxv43JFXFr1JYOFZt/l3hBC12rA= X-Received: by 2002:aa7:88d6:0:b0:68e:4653:2a2b with SMTP id k22-20020aa788d6000000b0068e46532a2bmr2610222pff.24.1694796159188; Fri, 15 Sep 2023 09:42:39 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 07/19] target/s390x: Use clmul_16* routines Date: Fri, 15 Sep 2023 09:42:19 -0700 Message-Id: <20230915164231.123580-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of galois_multiply16. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index edff4d6b2b..11477556e5 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -180,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) @@ -231,6 +230,30 @@ void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma8(q2[1], q3[1], q4[1]); } +static inline uint64_t do_gfma16(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_16x2_even(n, m) ^ clmul_16x2_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm16)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma16(q2[0], q3[0], 0); + q1[1] = do_gfma16(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma16(q2[0], q3[0], q4[0]); + q1[1] = do_gfma16(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -248,7 +271,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(16, 32) DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, @@ -284,7 +306,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, From patchwork Fri Sep 15 16:42:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723137 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488484wro; Fri, 15 Sep 2023 09:43:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGmRbjJKm0YFl4jdLvoasuxp0Wp3ZHzUFtzUlD16YzlBusza9emNYj75Brc+ZOP1scptzfb X-Received: by 2002:a05:620a:1daf:b0:76f:122b:aa3 with SMTP id pj47-20020a05620a1daf00b0076f122b0aa3mr1930256qkn.44.1694796231233; Fri, 15 Sep 2023 09:43:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796231; cv=none; d=google.com; s=arc-20160816; b=lfVhXpsebcyHRr88CfASeATEnuq7TEsFWedwQiNLSbUynOaqqJRLZqHo2SvD9RGCBP wQUh4TxNkgAjT0ilupgqHj5sjgeoEQvBaU4HEosQ95oumusWLxuVuedwxO0WICsGAkrZ Sbf2RnVb2+Fs18ZkHo6p/5L0YlIu152vJvYD3b2CEfYr7CkILOn6LXNECL6Opsn/PUb7 oEurfOt5sR/IefCqBvqAqVHaRin0+H9PB24ZDpvW4f3cfarSI4KWoi6b2wfQHOokwrCH EUqaBHh8Z3MVtk6nxSgQ2cAi/R7EFEV2BGvpbobPWonFU0WAlLOS2u8xIqlJ3VLXthCP T1CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=BL7EbX1EMhValKoX7e6yR5u3D5ZsID7pdC/xAbPtSgQ=; fh=PnYt+qEB9tAfMKoqBm2xjKOFpYyFFGPudh5cVIoieJM=; b=RM48KX7apNp79cGzPr3qyhNqs4rjO5HOIRRGNpYADBhnWzPtQJCp7uPF7w54TGdMQq rlmI7MWZ6IrBXm63dtyBlzqvBn9XCF7XeqSCyq5ej0nppxrw3eCXvUkrdH4GMZ932KOa pMSOKu365cZtX5A3cci61d/fOvY95Ilc3kU0WDUAtHO6mJzeTymek8AN541w4NR25zQ1 9cuvTI5GygR/qOq4lFt+XmSM5gd67zyNrr8plpSqocDxKfbOWTdV0AdG1BohTQ5eEVRs PNm7GP3Zw/SIJOs+cPKW3mN1Y4KRKaftILVo6Zn9SWHw6I8DravNPuoTw28O2ba1r3bV NtOw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ppPnd8PG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ro22-20020a05620a399600b0076d1c784892si2520721qkn.478.2023.09.15.09.43.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ppPnd8PG; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu1-0004Th-5l; Fri, 15 Sep 2023 12:42:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBtz-0004Sp-4y for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:43 -0400 Received: from mail-pf1-x42a.google.com ([2607:f8b0:4864:20::42a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBtx-0004uU-HV for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:42 -0400 Received: by mail-pf1-x42a.google.com with SMTP id d2e1a72fcca58-68fb98745c1so1949948b3a.1 for ; Fri, 15 Sep 2023 09:42:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796160; x=1695400960; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=BL7EbX1EMhValKoX7e6yR5u3D5ZsID7pdC/xAbPtSgQ=; b=ppPnd8PGjQIdN0/Is0+THxhjil2HH8c6Luk9SIlO6X+eo/q9hoiMhRQsyqbYIhSXnr 6tgmm93JxpdwGpztudvJyGWe+Di24IeveBDGgQMU3j8h03XTS6NhOXCUBTXOgVMb3Shp AEkDTMp68v3z/76SXoXkJ7sxTADFJJJkArrU4LigHLxvG4Ts3Y9GigcjEnYfP+LS+v+f Z5RJzQ1RMKRd7Ov3qp/tH+DlLcFg3Ui75UoTsNVxuz0Wa/Pa1qGGZmRNKETITKxYcf11 5F6HeKrplrXOxdH35LdDYeveFKIU4b6OZOPm6FuxMbXGE1puaVZ6jkkcLsWgrb4CdetC 5yjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796160; x=1695400960; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BL7EbX1EMhValKoX7e6yR5u3D5ZsID7pdC/xAbPtSgQ=; b=P1j3qXuPxCIoEOCkhJfWwYiwvITzK3Zw7hoDOfz00rohLrjHKx7Fv2JPXHQ/fRrqoz ihQByZ3GBQNjO2CKf6arYQDYISBYxleDbcUrZE8zXBUsV98EtU7CN1YpAtxQqRfmhNSg tmOybYEmIA0rRK6Z9O/rqNTQWZe+wLI8GwkJydZJfX/S9E7Xty49jbTlC/bASXwPJF6i zaOfK+H0pKjI2Vum7LPxHamL1pP/e6BS3rLT7JJ1Xg8OwmAS+KzIWPGHRqO4RiIwXu57 nLPXw5R5ITqk3BLxvzOk2jd7qsvh/jXhKavJsG9xVXNRXhtHknIC4xAfMJ2fq8L2K3p8 6wfw== X-Gm-Message-State: AOJu0YzAoIg0ZmKD3DrDaVsuD1TuSlSNGOCVhNMYZzIsFOr3qbV6AkZl 7GMkrNEFu5eGNt53NLMvq48MDbaT/HKBxXmjrl0= X-Received: by 2002:a05:6a00:1409:b0:68f:cd32:c52d with SMTP id l9-20020a056a00140900b0068fcd32c52dmr2328733pfu.14.1694796160177; Fri, 15 Sep 2023 09:42:40 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.39 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 08/19] target/ppc: Use clmul_16* routines Date: Fri, 15 Sep 2023 09:42:20 -0700 Message-Id: <20230915164231.123580-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42a; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index f45b24d321..ebb2957fe7 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1437,6 +1437,14 @@ void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } +void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_16x2_even(aa, bb) ^ clmul_16x2_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1457,7 +1465,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) From patchwork Fri Sep 15 16:42:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723132 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488285wro; Fri, 15 Sep 2023 09:43:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE5zz8Eo7Va4MaeoD82azRSBSQ5iBGBQjaBIBCmvAK+ItQcyimVKyTPMuxA3nYtjABkGvFz X-Received: by 2002:a05:6214:e4a:b0:651:6604:bee6 with SMTP id o10-20020a0562140e4a00b006516604bee6mr5583514qvc.30.1694796208810; Fri, 15 Sep 2023 09:43:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796208; cv=none; d=google.com; s=arc-20160816; b=KeWM4aU7+wQNXUT0h9BRZVhgABQVjvzuurLh7RRvM9g9HC/KWkYKco/zMLf9uy2WO8 74iuXfo9MmR62IyuGEr7M+d78deQWsl4sK4Zi4poEFisF9jffDiNd8zpDeILAN0bj8mc Z46QcehDA642SbsPQqK+xLiWsOQD5k+Pc/AJEJj5kmokIyB6XhJn2VEf7RqGzR+Ikxys HWujBjfnxaGh013Vp/1QVU2ARodSxXuu6bhzHSKPIuLidPIDXByVkD0ugei+HCxrATo7 Sg1yFKyNRpez848QtaZ/IMEM/39YVy46XZB9P+c8rD8ehSyFKLHtK4WLwNORSU/9zUtW doiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=5E1DttN1y2odVDPPAQFBLbvH/8uGREEL7i929ZC+pCE=; fh=9wguxkwuuSBbOibCpq3+Qud4pl++l1lkTpkAOiV7WAk=; b=SaxTeiTW8oeuxguxuKbJNMhQQvirQZmaajgHXbk4lYsS/0yiKTfKdHxEr0d+5EjlPP hczLtdIKKqnfg0/1Cb8NaxrVFFhHNQOVPLFqKQqHxAb3KLUJM2T0lx9mWUHQUVU/l92b cESc44K9tWXzaFBYXcCaSnJBPoF/lJnYAUNgY/LDn8a305wrSOWKUlKPL1NI1c6C5KZi Auc7HmDLG+VuY3DrVwR2v6G6giIHQo5rLziXbS7nH9hHs5yMY6KvTIxexIL15X7XZOku weYmTyctDZW32ja0iN1uDgMNdz35ycVzZ+TbO93nE93uY2OVJ2qFr+I98cty/uNc9QSM 0g6w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="N9G/t3xK"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v16-20020a05620a0a9000b0077075391d6dsi2519098qkg.156.2023.09.15.09.43.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:28 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="N9G/t3xK"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu2-0004U0-CD; Fri, 15 Sep 2023 12:42:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu0-0004TK-FR for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:44 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBty-0004ui-Su for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:44 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-68fc292de9dso1741872b3a.0 for ; Fri, 15 Sep 2023 09:42:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796161; x=1695400961; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=5E1DttN1y2odVDPPAQFBLbvH/8uGREEL7i929ZC+pCE=; b=N9G/t3xKBEped5pkrYdIqSasnkwf5WmyIjE3E3Old1WaCB10uswQmDRGnMACvEMfbw zeYIYs4TjRpCFngTRrkZfpwbiG39/WuPyzGBRdSBdnrpKKnbP8dbTu1zfeNtKVHEm8dv i20op74KqiXebGko1xNgd4RXdIEM2/1exmeYKsCijeVxn3KCh66KdjRLhIEZ5XhhZfI6 ipj7GDddwQlqbJTtnKuX7BJmqfkYw2/kriA0MfvoI4+CTYZytj7N/spPIJ8XA4csE+4Y NZjCPk0MXkECF6BrvTjTj5wXg0wUx6d3gGAOJbC8dGkxCcDeAI+xrJeb/emzvlp50JzU gKZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796161; x=1695400961; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5E1DttN1y2odVDPPAQFBLbvH/8uGREEL7i929ZC+pCE=; b=sn3DgC8Mg1HUUZJ0RLROnV2Hz/xe99mPeNls4K7dCwmjK8UNH7MFHyDuf+0QJXJH+P 4RgJlWwSHaMKFwjCJKfchLfTJoLTkkl4Qm4bb3/j1nPDFfqju5dr4gGgQ4BpjQ6UXyIJ bYJ2qpwmsm2DJ9zuPu1hsMakRxfCr8xLdXOYJV8PgsmbESRMgaCOjiTzXOJvKeaomXaI Z3Jl6rAiNcVtLNINhxlQgkCjzPhfPpbcbJ3ZssV5V8I6D62WPf9yBeNK4t3D63276p0H KvcUA/Fh5wlAKPxdJGs85UrLuEauzGFNTN3SXJWfgk8cKxT8LGVR+Fdm2wlAqH1FW4u8 IN/g== X-Gm-Message-State: AOJu0YxRKI5x/VUy99irEuYnG5FMICu72jiQsmPVeC0G756mUpBEM1hR QBYAwbhGYMFYHqBfdkIKlwNPzRjtDOV1c0x/lTg= X-Received: by 2002:a05:6a20:3d26:b0:140:d536:d434 with SMTP id y38-20020a056a203d2600b00140d536d434mr3029428pzi.10.1694796161016; Fri, 15 Sep 2023 09:42:41 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel Subject: [PULL 09/19] crypto: Add generic 32-bit carry-less multiply routines Date: Fri, 15 Sep 2023 09:42:21 -0700 Message-Id: <20230915164231.123580-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Ard Biesheuvel Signed-off-by: Richard Henderson --- include/crypto/clmul.h | 7 +++++++ crypto/clmul.c | 13 +++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 72672b237c..80de516464 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -54,4 +54,11 @@ uint64_t clmul_16x2_even(uint64_t, uint64_t); */ uint64_t clmul_16x2_odd(uint64_t, uint64_t); +/** + * clmul_32: + * + * Perform a 32x32->64 carry-less multiply. + */ +uint64_t clmul_32(uint32_t, uint32_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 2c87cfbf8a..36ada1be9d 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -79,3 +79,16 @@ uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) { return clmul_16x2_even(n >> 16, m >> 16); } + +uint64_t clmul_32(uint32_t n, uint32_t m32) +{ + uint64_t r = 0; + uint64_t m = m32; + + for (int i = 0; i < 32; ++i) { + r ^= n & 1 ? m : 0; + n >>= 1; + m <<= 1; + } + return r; +} From patchwork Fri Sep 15 16:42:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723140 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488728wro; Fri, 15 Sep 2023 09:44:31 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHoIaV2Nhj/SwXzvXkO+EzzNA2RNOVbCXorezeJJVRuGHfLdU757s9J57sVA001FQA3b6vJ X-Received: by 2002:a05:6871:827:b0:1d6:3b76:aae1 with SMTP id q39-20020a056871082700b001d63b76aae1mr2603359oap.39.1694796270725; Fri, 15 Sep 2023 09:44:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796270; cv=none; d=google.com; s=arc-20160816; b=kEerzCp0eBscZWCDj0w0ctJcdsOKXWr1pmdveZs3q/gnOaQkCMfF/gRp2jFINCjaJq qzL/bUmIAc+qSt0MnW+eiCrXn8HoN8ZkoGFkeN2A4gaw4LS2wKREIQNfLU8sK4Q+fQXG EvsJfNERIHCDjWD3dsjzLgy3/GsN3UsSFu5FrYw+i75RdN920IuPrhgojzzi+08TWHB2 kyFLWeMtkmgZC8+bOAa+668aCKhRNvpCdR4nRCYIic10c84olQALrzqEB3A36UGZgFIV GSRB0MGg7gZnhzq9G1Fm22jTilNU9VmZ7UsUVes84HYzYsdLqngxhdewY/WSvBYp8gN+ juAA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=Cccs7oyVEShFQdZBQOxywWhqz0xwK4vcC9KGpgd+um/EoaJvqMdNwLr0I+5mFLb2l7 JaQCRiLS+IVrbZuhWTx2OR1gWg0fCN/CIIOgSv1L0j0NUQdNhZpcu8OwSvElWFwx66Da Xh4YViu9TN4WqduUrtYJVMHHXQoKQRaLygUN9Mr9p7Hjg09Bj8IX9tXyyXEiTI9qZtI2 5cUySE4WGwHOuAeV+DLs++fwA9Oz5QJ0P6slCUVWseIt04212+3t+fftgCTpTGG2MvCz Q11lEeCuqKuLal5NrQrIakQRaFYHFjub5K+SmKRD8tkifMldhV5zD8u1AHVr59Wyqap0 WN1w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dtxMgAGu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u18-20020a05622a199200b00417985652bdsi1186818qtc.772.2023.09.15.09.44.30 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:30 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dtxMgAGu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu3-0004Um-Ui; Fri, 15 Sep 2023 12:42:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu0-0004TU-M3 for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:44 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBty-0004v1-UJ for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:44 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-68fbb10dea4so1940248b3a.3 for ; Fri, 15 Sep 2023 09:42:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796162; x=1695400962; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; b=dtxMgAGuXal5R1KvPFIk35pt80vYUhOzF/P7w1dB/KsbfD2v0PZZ6UfVNA1TB5BlR0 odmBokJNaF/6Cf1Nl0tpfQtj9eo9mSKdCLmYcvHH7YfqYTz87h1bMCG/blvSea16rMXh 6acP4cNcBeibPyPfbmegSKyEzFkqqvdDa0uHN5UtgLhCJkEwouEJS7mU9+vzZmSV6gII QiqWIeoASTAev9vmakUtFwyyLRZepUcaOTu2kJwomUOeFOY+LMsxe0EX8c7Dm9D4oZcE o9YqtCKMgcMrems9pFumfv2eWUmHLaDdkRaTN7eUhJAvRL2naE/X4TOgyiKJ+fHDpqMH +nrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796162; x=1695400962; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; b=pd09KhZF7/xIPh2cVpKHm9yT1hUOhKTbrPsh1XF0Fb58VoyPbyAhYUlZwBGE3k6Fyz lBCH/+crL5uVLHqO+gdy1qFgseDYGQ2IgALkRAs94aJIVQ+c5cmjxBVvLsrn8+bE9cBF MCHx3u3fsOfsCLFd1XRrcfykvp1yqZKjAzUTccHYT2ZcQHMJgm/qUKfcNNltT9MIVZIT QeddR+vo3VrOKXZjisMP1jSr+7GL4VPUQ7OVp0o5n5zZaE13ezLaFe3IebLvAyZRetFF bWgiWW0MnW+UlVj7KB/CRjxOnwNIr3JFNCNzdhFIFNUwL8ofmC7l0BfPDyJiJLHs5hyA 98LQ== X-Gm-Message-State: AOJu0Yz2/KRXnYXqesKLxweH3IslocDO7dGMWJ8EivB9RtcD1l9TFdIP krNqE+hQqtuIulQzrBw3841sX4yx8Xrknd8WmnY= X-Received: by 2002:a05:6a20:5489:b0:14d:9938:735f with SMTP id i9-20020a056a20548900b0014d9938735fmr2726889pzk.17.1694796161776; Fri, 15 Sep 2023 09:42:41 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 10/19] target/arm: Use clmul_32* routines Date: Fri, 15 Sep 2023 09:42:22 -0700 Message-Id: <20230915164231.123580-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of pmull_d. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_helper.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 5def86b573..ffb4b44ce4 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2055,18 +2055,6 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) } } -static uint64_t pmull_d(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - - for (i = 0; i < 32; ++i) { - uint64_t mask = -((op1 >> i) & 1); - result ^= (op2 << i) & mask; - } - return result; -} - void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t sel = H4(simd_data(desc)); @@ -2075,7 +2063,7 @@ void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd; for (i = 0; i < opr_sz / 8; ++i) { - d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + d[i] = clmul_32(n[2 * i + sel], m[2 * i + sel]); } } #endif From patchwork Fri Sep 15 16:42:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723141 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488763wro; Fri, 15 Sep 2023 09:44:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGzUefNH6GmH0kZXjRb0ZnYYr1in4BetAV3uZO3xRYhh5lK0QxwmDAsZjm98l2FMEEDh3LY X-Received: by 2002:a0c:ac01:0:b0:656:1958:aba0 with SMTP id l1-20020a0cac01000000b006561958aba0mr1953377qvb.13.1694796276075; Fri, 15 Sep 2023 09:44:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796276; cv=none; d=google.com; s=arc-20160816; b=1BFZ0NM1oDVPlU3UF6iIHmuxp0n9rCKqswBpD4QF68llUxziZTSJTbHEQlSFqrnbJf RB//m+dqL1Is17HNwOsrsEgEVk/8swrTVnMwfc5JcJnBcbW9sWWzoPBJqij/PI6nTALk qrpSTAyaD79I10+UAE8wm5fT5iz7n6SCnb6Mrx4SnXj8ELoNZqSiniyHbLx+CYftwfTf Rds0SBJTONC6eQB1vfrVnh7M5ZD6bySGyis26ITbz24Jh9cO6Zn0LQA3UlIeoiXUhwMC WHf1ff6o507foi6skqz+tuLMHhenuY2lZOeAsPBTBCfaOGfaFPtaHKDLNDVok/lttcw5 n9MQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=uTeUMk89bbc4KOV/EI9+qfDgIX7yjaPDjzfDSiSqRSBD0+UJZJ6DC00molgIKu1YWu cbLe/CpJpnwomP/1SlALeh1ySIyLfpE/XOa3MVb1nF/3u5ZgYIfIqwwF77nfRNxCtKE8 SZYprjepdmT42Y06PYU37W0jeQQzJ4lWGfeeDWeliyr19TWaER9kaHfOyKREgHGS+/Zw l69Cu4df7wRTMplGObUrf2hPEah1NrqdQOD6TFfz4Uzd8JeBQGauibjUUZEHVatbc7h1 EMC5Y8In4vRl/UKNESek/NKcsCLhG0k6kU8TgLwmcMLIkd+QI9beKUhSvZZwSapRHqrV x72w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PMWdIkfw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s9-20020a0ca609000000b0063217340105si2439195qva.536.2023.09.15.09.44.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=PMWdIkfw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu5-0004VW-67; Fri, 15 Sep 2023 12:42:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu2-0004UV-Me for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:46 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu1-0004w9-1e for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:46 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-68fb79ef55eso2146752b3a.0 for ; Fri, 15 Sep 2023 09:42:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796162; x=1695400962; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; b=PMWdIkfwC61+8PGZrE/P5bWc4p7fH3392/XU18lhRK4xO4P6ZJ2MGYxL6NhdMGKapn /n5KapAX8lF1ZXj9pUDtalmYPELYF/uTSe9P9A9Ju03IU3kWPBZRCEq3LmyFjZxNzASj 33BhVhpXVqtEy1qv/0uVyN6pr5Futzm+Btv7FdLk0OlVFAnUwoCwGYDyvQfo5SyfWXZY C+9N/AD6Rl/qvA00vEqjec4oWEX9DTvWl5YWQYf6Zuv/kPumBy3rJMJL7nenmxu/2EWw ZsmyVSpC/8VKZyQRZ43VSRp1gM8ltvOLseEMu1rrZVJbd/jr4iI9rSfSUGY0BNMuWxnE 3TOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796162; x=1695400962; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; b=FD9lNuHUPe1AD1JyDQTOQuxTUNFLKkxSUmSQr1hwuLDiSSdTwldoroCbEWLqcDH48l AOymYdDUdqyn3qYLwEVjOgG/JYuCeUS0zNtzEDhrIqRZegj8jjjsABLBi/36RnCrExxh whlhUwb2kT2BvKGCeA5iFo2IQ9ZD7yOgDWQlQ95YOhQsIMwdRk/833QG0316f6L0+Pgp wg7yCsvhzvSudyDlij0A9w1NlH+ljrkj/wDS27R1apY+LLG3lFvtcm3AzI+8Kmt+KPKi 5vVK9JGrtwgXg9GZAa8w6/vlPwQvFQSGYMybTXq9GN2h0eOMGkdyfiyLEJOeej1X0UBJ bX9Q== X-Gm-Message-State: AOJu0YxCpDuI/BpY5eDukG0pgR7qIKu+fE+GtJirFhqUuHwD/wSu9uFr hHJxG7cxtq3RPqBxc0ZTwpyYqSSCY5tNwAI1nB4= X-Received: by 2002:a05:6a20:8f01:b0:136:faec:a7dc with SMTP id b1-20020a056a208f0100b00136faeca7dcmr2989402pzk.11.1694796162576; Fri, 15 Sep 2023 09:42:42 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 11/19] target/s390x: Use clmul_32* routines Date: Fri, 15 Sep 2023 09:42:23 -0700 Message-Id: <20230915164231.123580-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of galois_multiply32. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 75 +++++++++---------------------- 1 file changed, 22 insertions(+), 53 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 11477556e5..ba284b5379 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -165,22 +165,6 @@ DEF_VCTZ(8) DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -#define DEF_GALOIS_MULTIPLY(BITS, TBITS) \ -static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ - uint##TBITS##_t b) \ -{ \ - uint##TBITS##_t res = 0; \ - \ - while (b) { \ - if (b & 0x1) { \ - res = res ^ a; \ - } \ - a = a << 1; \ - b = b >> 1; \ - } \ - return res; \ -} -DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) { @@ -254,24 +238,29 @@ void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma16(q2[1], q3[1], q4[1]); } -#define DEF_VGFM(BITS, TBITS) \ -void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ - uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ +static inline uint64_t do_gfma32(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_32(n, m) ^ clmul_32(n >> 32, m >> 32) ^ a; +} + +void HELPER(gvec_vgfm32)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma32(q2[0], q3[0], 0); + q1[1] = do_gfma32(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma32(q2[0], q3[0], q4[0]); + q1[1] = do_gfma32(q2[1], q3[1], q4[1]); } -DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) @@ -288,26 +277,6 @@ void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, s390_vec_xor(v1, &tmp1, &tmp2); } -#define DEF_VGFMA(BITS, TBITS) \ -void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ - const void *v4, uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - d = d ^ s390_vec_read_element##TBITS(v4, i); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ -} -DEF_VGFMA(32, 64) - void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { From patchwork Fri Sep 15 16:42:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723142 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488800wro; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH6iwwMp3dRu4VWJu/ybSHpA8WYKzVYsG8wI8KdP/znuTr7XmTL+lvok3PBmWEaDJA4pSXC X-Received: by 2002:a05:6102:34da:b0:440:a8c8:f34 with SMTP id a26-20020a05610234da00b00440a8c80f34mr2565481vst.3.1694796279089; Fri, 15 Sep 2023 09:44:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796279; cv=none; d=google.com; s=arc-20160816; b=t+w6jLnGG9UGWlmvsWk30DSI/qzkpDneDIgzTP9Onp2hVtDpc1b17iSuyKHF4Qc1Lq p6eo+bH6dsf+uo71yAT6drN59iq7LwtVIdM8hJ6RmMduApMcRxXnmR//7fK63n9zjYNB 4s6ThJIJO50gYo0eleZ5rJz6zlJrTGDvt6X7JkILa3evrC2fCAvPIA0cYaZiP7hm7HjH lws43jH1FwJ3UpZuEg+H8yXHH8CZdF7B/G1OLNRi2pHeQjN4VKa5qIwtzrof30hyB5Jm Do4qrUne6QBET5AQol5y9IIFA1Jo6RFvu4vTFmQTm29U5reJzHxf7/X2LjDnlx3w5NZd RVNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=/GHGqee03AmoquehQcXqyFpwhRv/6DB5C+2O1I5qXiA=; fh=PnYt+qEB9tAfMKoqBm2xjKOFpYyFFGPudh5cVIoieJM=; b=B3dMQgHVSR+7eB23sqSG/ndcqIONjnSb6S56GL5cK/anjh4ENO/B219uiLeEhfO5rL eCIJoZvceCPOGST2KD3BE1fk8oZl5OwvjhzuQpiZVgxCezFHirZy0vZfd+rc+5pERST+ ECPr/qxaAFpEfcolWUf3MJFFHL/ecewD0I8q5qLuLdzhBjLO4kGSjsG1bFS2Elat+8xH 6AEzTJCl91+R70IJIbpuThrPMsHmZj9vYxKsxPXiQJy92T50vLDUIg3cyuTelaHJemHv pKFG8BWPm1oqg9EURCV/lwuHAWyzCoib3U28jzRF6ngtmcDcB6iC/yQ5w7Gdt+cmkSJm LEwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hZa36b7c; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n17-20020a0cdc91000000b0064f387ba538si2457203qvk.291.2023.09.15.09.44.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hZa36b7c; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu4-0004V3-KV; Fri, 15 Sep 2023 12:42:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu2-0004UE-GI for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:46 -0400 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu0-0004w4-UF for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:46 -0400 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-68fb5bd8f02so2279738b3a.0 for ; Fri, 15 Sep 2023 09:42:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796163; x=1695400963; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=/GHGqee03AmoquehQcXqyFpwhRv/6DB5C+2O1I5qXiA=; b=hZa36b7cKvfSMEJRbyOnG2C2ZF6k6cSpi8xtFRm53IP+jJoEqNOyjWKfa1YCxB9Iz/ zWJXQ1eyJS0b8OGfOKy+CPsMNf8o3muegyvcG4pfuLzesDrSUfYj/2vNA8ylSaJ1o0Yl CE5wjDbRowYDdBL6jEdW5rbxn2caesLdvXKLM92OXFP5d26H/hrEDYnl6aI2eBAxlHlJ yFj2ddu5sr4v8zqQxprF9VO8ZZr3XaP2ffNqyDI7hKnkSV+X83vwXYFme4l5kWOa+TcO lul8w4fI9/AfHPuzavSGwjw+CKGmBvO6dsqiQ+jl/NRFfCYvH7+noDIVzU56bl73la0r XkZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796163; x=1695400963; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/GHGqee03AmoquehQcXqyFpwhRv/6DB5C+2O1I5qXiA=; b=wx4BJFuA9jb6C2asenurRqDBRJe4qeTcglVDsc+bWdb+0UIi43jzUDaCyYlnuoxM21 bml+Em5UQmR8zHzMPLzNSvLVkUbaTUwnhicM5XrH/sZrLII5U2boW7ouLo0/23JnvlW0 B+hxP0XOH4L0re5n86YXlB5SWW5G7YG1bnzfimd+qrhFqe1g74ztzp2cqMgLzp0VE4v1 aW/84BZiTK4+bkr40ANq2uQezM+tWDFTANcST95M7BATeXh4m3xAHiuL54Fs1mHta3kB sR79D+RxHgItjOFn4luaUFT+5E+Ld29xr6jNBlYGLYYDnDGLtYdnbwGKpw8lYNmVQIju EFQw== X-Gm-Message-State: AOJu0Ywk4I+eucu/jDZparxCCyeQVOO0qlLfFHYo7DrCbQ9BzmPPn6Y+ ZEKNfLDUvTbiMu0F1BYHBVcG1SOSnq/363RWIUI= X-Received: by 2002:a05:6a00:24d2:b0:68f:d35d:2168 with SMTP id d18-20020a056a0024d200b0068fd35d2168mr2393689pfv.0.1694796163459; Fri, 15 Sep 2023 09:42:43 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.42 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PULL 12/19] target/ppc: Use clmul_32* routines Date: Fri, 15 Sep 2023 09:42:24 -0700 Message-Id: <20230915164231.123580-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::429; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x429.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index ebb2957fe7..1ea42b4ede 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1445,28 +1445,14 @@ void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } -#define PMSUM(name, srcfld, trgfld, trgtyp) \ -void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ -{ \ - int i, j; \ - trgtyp prod[sizeof(ppc_avr_t) / sizeof(a->srcfld[0])]; \ - \ - VECTOR_FOR_INORDER_I(i, srcfld) { \ - prod[i] = 0; \ - for (j = 0; j < sizeof(a->srcfld[0]) * 8; j++) { \ - if (a->srcfld[i] & (1ull << j)) { \ - prod[i] ^= ((trgtyp)b->srcfld[i] << j); \ - } \ - } \ - } \ - \ - VECTOR_FOR_INORDER_I(i, trgfld) { \ - r->trgfld[i] = prod[2 * i] ^ prod[2 * i + 1]; \ - } \ +void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_32(aa, bb) ^ clmul_32(aa >> 32, bb >> 32); + } } -PMSUM(vpmsumw, u32, u64, uint64_t) - void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { int i, j; From patchwork Fri Sep 15 16:42:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723133 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488326wro; Fri, 15 Sep 2023 09:43:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFJHI72LIKBw+yHVd2am9Oat1/o74zlBPbIFB53o5nBxMQ5VYawkOyEd02MB35V/BtrUrqL X-Received: by 2002:a05:6358:63a2:b0:140:f00a:ee2d with SMTP id k34-20020a05635863a200b00140f00aee2dmr2975105rwh.23.1694796213140; Fri, 15 Sep 2023 09:43:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796213; cv=none; d=google.com; s=arc-20160816; b=qQ0Y1Ycuu4+hGTSeZXJEdpICcRSEZCIY4NXRj6UdZP4AmClfmHwTCj3TG/4KEGD+zW JmGjZxYUZ30YAOTmjVb5LzxMIsIntCKnRFraDkID2xPF4PlUGm5xkqhORXPS28v/KoxH ahLoaQJ+5l6OxUbEWAjCjRTlZUjnQdxkJNEoEIj4RJrr6D++52jBfJ96Y+W5ayEOeyu+ dA2UsIpFLrbk6dpJPfUc2aMwOmbu9qSTZE7LPLpQa8OjDEWH+oAKtaA++hTHOT9KwXih hRHocxIhLxBbKALOpRdgMYisaI6xNlOFypgG5ED96rJX2Dukym4bkUoUCqvhcEgiR3el V8lw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=SHZnlFNBtvPPOH9mS0fn2N0/KTEtZ6bjCbR6h+ACJf0=; fh=9wguxkwuuSBbOibCpq3+Qud4pl++l1lkTpkAOiV7WAk=; b=n5WmJ+evoxAqAAMM0e15nqsEtgJQJujyDPJb+MePqOBZbz/ubkPcw+O2ThWEt7oGqQ waADzSkyR1g6ypA8WYjXm6aQxtheY6hwUdCV4JqCPoAO/dNEPbYTXy+7bcchTRoxrQDy fublkWbKnFLAKSYTjFq//vpIfq9lge0CjZ55yPDFRKRjMNyMXc0MV0aUG2x3f9eEsZ/F 0hDxL8X42qLHPtfETMx3j2e4BO4cC0ZF/V3C8wJR7bQurQWKPZTUOdg0l41yG0bHbAsz XSsKWXVNs1f3Ewyvu1+8/AlnqriIO9s1d20Pc23qF77hgs89Wsc56U7ljjenItvJ0fxK P55w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jUAadngI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p6-20020a0ccb86000000b00655e4826736si2438221qvk.69.2023.09.15.09.43.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=jUAadngI; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu9-0004Xn-Oa; Fri, 15 Sep 2023 12:42:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu6-0004W6-VG for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu1-0004wP-PB for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-68fb6fd2836so2137923b3a.0 for ; Fri, 15 Sep 2023 09:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796164; x=1695400964; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SHZnlFNBtvPPOH9mS0fn2N0/KTEtZ6bjCbR6h+ACJf0=; b=jUAadngIND8Ll50rBMZnQfoekaFHNdGIWd02imDcMtbMBtlZInK5HpvUk0QXFdOWRF NEiHvpzmnzCSHk9W/TLELATsK/y4Fpimi/6VdqU+HaVd0p+NZd0a0hD7iGOtD+tIVxVw P9oQWSewjQJ3nfXqBK4N6K/fiQtexkvKwwsQhrzJ2IlwcTOqO4VAg1W/zFSADeFgYCjo /oUiBa5x9I67QFMV9hCt/5ExnJfCDW7IkhC/yNqqT7WLAe15ecKRCHM4fPKzNpgLNZy/ lQCCNBKKvnFc7uivirmu3qEcMe44m7gbwVvGPFNG5AJM3myr9jDRlCfPUtNJdox2pimI UTXw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796164; x=1695400964; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SHZnlFNBtvPPOH9mS0fn2N0/KTEtZ6bjCbR6h+ACJf0=; b=onZwj4RgxjwCE2pqrcuJV+LDFhuMj0N3Q/lHfKCb1UrLGG7u98NfLJl8/+//PK7DrV Ktej4pDAosNz8K4qE0aRWHVLgt91sLl5EVJEJpv8i+T2iw74/i4zmjQuHLFNFSljPG+2 6ZtjK+LK0UIiXYYmTFjgZxKZ/MyUpFJmASSXUMdaqEXNrSdKPWIKPRYeaO6ZuvOvgc/Q wngCaoJpMosxkfcQE+acnPnL1FnMch0MEgsUt72toxWBcwgHmtOz8Xoh2hHjZgXF+UPN uAM/v/a2qG39OTG9Ea52z4tHHAKCy9T18NEmcuuIez1MGhaKdMfC4EQcX9mhyc7lrUP9 qQcA== X-Gm-Message-State: AOJu0YwICeFiNuIAyikwGB1FT6ddicMAJGz03BI9SdHWydKm3hnlN8x5 nvdE19N8htETyC7AG4KX5+w9sh+TAQTd7cLERiI= X-Received: by 2002:a05:6a20:938a:b0:142:952d:3e61 with SMTP id x10-20020a056a20938a00b00142952d3e61mr2657794pzh.7.1694796164263; Fri, 15 Sep 2023 09:42:44 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:43 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel Subject: [PULL 13/19] crypto: Add generic 64-bit carry-less multiply routine Date: Fri, 15 Sep 2023 09:42:25 -0700 Message-Id: <20230915164231.123580-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Reviewed-by: Ard Biesheuvel Signed-off-by: Richard Henderson --- host/include/generic/host/crypto/clmul.h | 15 +++++++++++++++ include/crypto/clmul.h | 19 +++++++++++++++++++ crypto/clmul.c | 17 +++++++++++++++++ 3 files changed, 51 insertions(+) create mode 100644 host/include/generic/host/crypto/clmul.h diff --git a/host/include/generic/host/crypto/clmul.h b/host/include/generic/host/crypto/clmul.h new file mode 100644 index 0000000000..915bfb88d3 --- /dev/null +++ b/host/include/generic/host/crypto/clmul.h @@ -0,0 +1,15 @@ +/* + * No host specific carry-less multiply acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef GENERIC_HOST_CRYPTO_CLMUL_H +#define GENERIC_HOST_CRYPTO_CLMUL_H + +#define HAVE_CLMUL_ACCEL false +#define ATTR_CLMUL_ACCEL + +Int128 clmul_64_accel(uint64_t, uint64_t) + QEMU_ERROR("unsupported accel"); + +#endif /* GENERIC_HOST_CRYPTO_CLMUL_H */ diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 80de516464..446931fe05 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -8,6 +8,9 @@ #ifndef CRYPTO_CLMUL_H #define CRYPTO_CLMUL_H +#include "qemu/int128.h" +#include "host/crypto/clmul.h" + /** * clmul_8x8_low: * @@ -61,4 +64,20 @@ uint64_t clmul_16x2_odd(uint64_t, uint64_t); */ uint64_t clmul_32(uint32_t, uint32_t); +/** + * clmul_64: + * + * Perform a 64x64->128 carry-less multiply. + */ +Int128 clmul_64_gen(uint64_t, uint64_t); + +static inline Int128 clmul_64(uint64_t a, uint64_t b) +{ + if (HAVE_CLMUL_ACCEL) { + return clmul_64_accel(a, b); + } else { + return clmul_64_gen(a, b); + } +} + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 36ada1be9d..9e3e61a77d 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -92,3 +92,20 @@ uint64_t clmul_32(uint32_t n, uint32_t m32) } return r; } + +Int128 clmul_64_gen(uint64_t n, uint64_t m) +{ + uint64_t rl = 0, rh = 0; + + /* Bit 0 can only influence the low 64-bit result. */ + if (n & 1) { + rl = m; + } + + for (int i = 1; i < 64; ++i) { + uint64_t mask = -((n >> i) & 1); + rl ^= (m << i) & mask; + rh ^= (m >> (64 - i)) & mask; + } + return int128_make128(rl, rh); +} From patchwork Fri Sep 15 16:42:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723130 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488190wro; Fri, 15 Sep 2023 09:43:10 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGDlfMfroT1oKhbJVur8ATl8woLs/LQMJpMsGUrME4pypJ5s9xf/Mn8/GO2lQpbeU07yxsX X-Received: by 2002:ac8:58cb:0:b0:417:98a1:3403 with SMTP id u11-20020ac858cb000000b0041798a13403mr2229947qta.25.1694796190645; Fri, 15 Sep 2023 09:43:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796190; cv=none; d=google.com; s=arc-20160816; b=xLgQ6vdV149u3mHPfuLzRE9Uj1qmfVOKtMc2KttYBq21zfkJT2HTtZ+3oNrPIQ0DGy UwK/QUQSiVEro8IsaAR8uJ33Tx4SrVXunvI0GCwXfWtxDbsOXbxmwP+0mWWL164PdTbj RoH0C1XfT+x5i/R+GzmJt0cWE8laDL9lhHaw9rwuduAtkvZ/+lc5xjdYaKij4g8Gb9Lt rexRD47LKaU19bPC4640kiXS3C5TVG/m4JR1daowpvj7xtM6xChq24r1PuCFRSNI9SXL uAIwMHJAn5SlpzdhKm1hv/FzfMRFAMjw+SlGq+BR2G6n4RzJ4qknBT9Sfa9pWyaX7f34 gYSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=Ck7thSQikr5eHgEGKSryXMSp6WTWO/7rpF45l+xSJfKr243IgCiJzkN/RlgHD3DHoK feKDcIBsrhXpoh4Jb5XmjiMYLM4BliWWQ4HDbTrQEhtSkYA+fw5/rYOr4HhJmF6567L7 wX9B4TZ7bg75j339Ep4xQIp3CuzF5zafl6pP62VS/YWGlimb5VmFNm7pJmFzdcLYWpst P7/coYKSsKfGyCw9VjfgTlH4irFYWxRtiLYhist1+Hzm2uHX2sVhbgc+qj/gp415Rtgl mqFnmtb+OWcU4z5tuxGEpFkSqgWvOvF7tm+8CL7U3wii1FK4pwPeEw8An4JhaxsITkf1 3NDA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MLzuqBHT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i19-20020ac85c13000000b0040556bb3b65si2649675qti.674.2023.09.15.09.43.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=MLzuqBHT; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBuB-0004YY-CX; Fri, 15 Sep 2023 12:42:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu7-0004W8-0c for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:52 -0400 Received: from mail-pf1-x42c.google.com ([2607:f8b0:4864:20::42c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu2-0004wW-6b for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-68fcb4dc8a9so2247964b3a.2 for ; Fri, 15 Sep 2023 09:42:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796165; x=1695400965; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; b=MLzuqBHT1waQVa1NmOP9k3UJtevIvMy1oTuSwiUvfGcDmEeoZ9Jb3+fvztG9ofYGsq UAxlC9Ff02l0EfDVaEMMq91hzSnWiRSl8PTwAgzn5StD1egatIJxGD5Mj0ex9m9LidX+ fBHett9ZGt6UtHUeTy4nzZW9r9Km9aQFpr0e12yZGXtyYTE8LNPuevRrDJBupeixOQft sjG21otw/lldKsRfH4Ek+/ZKH+n0gl0qpAtHJf4L4YjtaBPzYpji4IERipln9z1G8yzQ il9Dh46xVgsOeRK9HBh1pwDqjj7KsKsVGMwOQpHk3fCki+XaBRdQ3reKnUzE8prMvZ4I YqQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796165; x=1695400965; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; b=Y6jMYPGnlj+6+o8G9CCvO+0ckE8nqrVZIv130nglztMOKhQOWIr/Nj9XMwLhPl6dVb HVbIVQRwXxnJbL0+MgdothUDkFUEEmUnjihT4uuzDANBAumzNe72MXZ3RwhyL76/OlXP EHUgQndeJ6bdcLKas1nSc+DCiHeJl+YdXJmCwmiyA+qo0wtxDLBZoT0h9UQx+mcRPGKL OLZ2qa3l+Si0eJSStAP6j6DqDyHQVjgEVZRIgf6tZhbxdnm2Srt8zAc/8b6WZMiAlleO 5Ln+YWsDUMw74UB2NfWq1S7cEW8eBsMsEWw1fJ7dqb2G6ECwHqOzN00ub6K2nuYD3/sc tfMg== X-Gm-Message-State: AOJu0YxhosdMNcgHQPiEQHtB8/cqutLg1nopQ/FrHhXSsNA5Q6i80VWU jsMEFkhKWzyy6jWHEhZ56NNAxiWFd75JeS9KT3U= X-Received: by 2002:a05:6a20:bb02:b0:157:7568:6796 with SMTP id fc2-20020a056a20bb0200b0015775686796mr1890837pzb.60.1694796164956; Fri, 15 Sep 2023 09:42:44 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:44 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 14/19] target/arm: Use clmul_64 Date: Fri, 15 Sep 2023 09:42:26 -0700 Message-Id: <20230915164231.123580-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42c; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, T_SPF_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_helper.c | 22 ++++------------------ 1 file changed, 4 insertions(+), 18 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index ffb4b44ce4..1f93510b85 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2003,28 +2003,14 @@ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); intptr_t hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; i += 2) { - uint64_t nn = n[i + hi]; - uint64_t mm = m[i + hi]; - uint64_t rhi = 0; - uint64_t rlo = 0; - - /* Bit 0 can only influence the low 64-bit result. */ - if (nn & 1) { - rlo = mm; - } - - for (j = 1; j < 64; ++j) { - uint64_t mask = -((nn >> j) & 1); - rlo ^= (mm << j) & mask; - rhi ^= (mm >> (64 - j)) & mask; - } - d[i] = rlo; - d[i + 1] = rhi; + Int128 r = clmul_64(n[i + hi], m[i + hi]); + d[i] = int128_getlo(r); + d[i + 1] = int128_gethi(r); } clear_tail(d, opr_sz, simd_maxsz(desc)); } From patchwork Fri Sep 15 16:42:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723138 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488604wro; Fri, 15 Sep 2023 09:44:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH1ZxBUDTWhuGF46550uNI8eZR30SvMMHkKpM51loSF192y83sWBb1q6bwyIPRgm6DbmIFD X-Received: by 2002:a05:622a:51:b0:416:e3ae:8cd0 with SMTP id y17-20020a05622a005100b00416e3ae8cd0mr7493533qtw.14.1694796246820; Fri, 15 Sep 2023 09:44:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796246; cv=none; d=google.com; s=arc-20160816; b=D2+IU5ZYI52LLf2JYR9Mt+LL9+eAjEPbACXByoZJN1y4KUKhpcUg0zrLfdxQs0akgj vN8+mMQH1vCnWyn1KMBZ0Yw5qpVukH2Ypy4kCCb5Ocdz+kWUcFwSxjxNG9T/yNUpdZU6 9QgE/f5rHzqVONDEOJdsQSGWnbvHnpL5zVG26PaueW2WVIFzzOlu2PBBcEpbwnzms8eF x0WUnmppIGPAytjdiUEaDydeHUFBeOVFhcG/+EGRlKnfmcjNQH4ckrCvaMpL8oMVeysm OGGEFeEfGfiApHaasRl+N4tAbg6/pGPmSJzUV3EAv0CMTdCPhuIVPjOCP+84WlR9jxK2 ogxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=b+3/8IiHtNpBs5dwlAlcrSMhCtecqGkkP2HhC4y759E=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=FvUJZThm/sYyA26fnqpXKvtIS4kzuQ2/qJnRdEX+LzkcV6LMsQ0lb65cFLecmaeeBS 1ULqUsnisrMIChgLuq5YzNtMzdswt9wT6DpPS7hJEzipEDjJcZ75/ANJYmqpejvHSMiP NQwCY8ZyrsNiguwSLrPxKjt0Njb8L2f73SGIp4wA/IsyzDreoPTtEnlMCriV2f0rONgz rmIGHsi6XQ0NW+8aukrVn5WrD+Bwz3xFDSPTVeVIGEGkZJjq783VH1s1NMQPIAPaquJz citN8zTbTiCho6nI/7Mu+A+zPsNdAIIe+FG++Lg0QOjdzIZsBWKSH/rUhu01OFJQdukR Lg9w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TrLqvRrk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m20-20020ac85b14000000b004108c0630a5si2716413qtw.194.2023.09.15.09.44.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TrLqvRrk; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu6-0004Vj-Fs; Fri, 15 Sep 2023 12:42:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu4-0004VR-VH for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:48 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu3-0004wm-Cg for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:48 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-68fdd5c1bbbso1722775b3a.1 for ; Fri, 15 Sep 2023 09:42:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796166; x=1695400966; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b+3/8IiHtNpBs5dwlAlcrSMhCtecqGkkP2HhC4y759E=; b=TrLqvRrkTYTzuerEopt6OwLgWG+K1+eZXhWlX7gLqT0ERI1lu9nCQOaj/YXeueIbg/ Ndru3nJK8XqtfVgGmoB47DhwhkF6uHcet2JBGtYUhKdtIYBw8Xe7jktUCAFhFzyEEk8Y 8zrzbw6Z0XLZ4hl57B80j+CrYOc/hd/tS5NAyj2roRG2vw+k/c59FwiDadY4pqARujgl VYzwET2qadZlj08pjLDrxZ+rrhF4sU75z3ZMlS9g6n/3BLLrxz9XsImGwuE6suNd1xWx eINCtF1qg6XW/8lx+swIAS1PMdZ+eLGl8G4DONrh6p8b+9MwzVKyi4VG0Lg1cDhkOtVm 1xPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796166; x=1695400966; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b+3/8IiHtNpBs5dwlAlcrSMhCtecqGkkP2HhC4y759E=; b=Ld3tWyey6gp/7jfLZ2cJpnPszToXdUtw3K811gjJiEJHbpo+L+9RclE4RbkfHagTp6 Zjetx3+ICloxh7+nURrIH+h45zEOwqNYwiqp/gync8QvYjePltppANtQLeOfs3fhb+rr smrz5wBrMWK8EufhYs3cGRhBHJtin0V63c7nM4J20bEmKM34rM/BpTTtz7IYw960zMQj dnCKZRU+DUgx+TQINNLDXylm/wIupjUddtvjG+SPSEQInXO75NY/VdSJ3domrVqGp5tk JGArRnx/DRnWAI1vVSzakCW4uxCQc1LiXY8a801nyR5oqEcH9qTS1L2go7XW3y9FLwsP itqg== X-Gm-Message-State: AOJu0YydBY5AlytmRIv3D3ZmbidpwhMv/3F7ywhL+nFPqqAxUIJ7eJy/ DTFd2J4ZX7W0WI36Kll4O+ipLh2quEq1u1OyviA= X-Received: by 2002:a05:6a20:430d:b0:159:d061:58e9 with SMTP id h13-20020a056a20430d00b00159d06158e9mr7314745pzk.26.1694796165954; Fri, 15 Sep 2023 09:42:45 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:45 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 15/19] target/i386: Use clmul_64 Date: Fri, 15 Sep 2023 09:42:27 -0700 Message-Id: <20230915164231.123580-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/i386/ops_sse.h | 40 +++++++++------------------------------- 1 file changed, 9 insertions(+), 31 deletions(-) diff --git a/target/i386/ops_sse.h b/target/i386/ops_sse.h index a0e425733f..33908c0691 100644 --- a/target/i386/ops_sse.h +++ b/target/i386/ops_sse.h @@ -20,6 +20,7 @@ #include "crypto/aes.h" #include "crypto/aes-round.h" +#include "crypto/clmul.h" #if SHIFT == 0 #define Reg MMXReg @@ -2122,41 +2123,18 @@ target_ulong helper_crc32(uint32_t crc1, target_ulong msg, uint32_t len) #endif -#if SHIFT == 1 -static void clmulq(uint64_t *dest_l, uint64_t *dest_h, - uint64_t a, uint64_t b) -{ - uint64_t al, ah, resh, resl; - - ah = 0; - al = a; - resh = resl = 0; - - while (b) { - if (b & 1) { - resl ^= al; - resh ^= ah; - } - ah = (ah << 1) | (al >> 63); - al <<= 1; - b >>= 1; - } - - *dest_l = resl; - *dest_h = resh; -} -#endif - void glue(helper_pclmulqdq, SUFFIX)(CPUX86State *env, Reg *d, Reg *v, Reg *s, uint32_t ctrl) { - uint64_t a, b; - int i; + int a_idx = (ctrl & 1) != 0; + int b_idx = (ctrl & 16) != 0; - for (i = 0; i < 1 << SHIFT; i += 2) { - a = v->Q(((ctrl & 1) != 0) + i); - b = s->Q(((ctrl & 16) != 0) + i); - clmulq(&d->Q(i), &d->Q(i + 1), a, b); + for (int i = 0; i < SHIFT; i++) { + uint64_t a = v->Q(2 * i + a_idx); + uint64_t b = s->Q(2 * i + b_idx); + Int128 *r = (Int128 *)&d->ZMM_X(i); + + *r = clmul_64(a, b); } } From patchwork Fri Sep 15 16:42:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723131 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488195wro; Fri, 15 Sep 2023 09:43:11 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHBTzXN0UzCgkGSUNzLJ7ORlwNzZzPcgy9KDabsT43A8WSCG0/aItt0jVotR7xecegwuisI X-Received: by 2002:a1f:df03:0:b0:495:febd:9187 with SMTP id w3-20020a1fdf03000000b00495febd9187mr2588248vkg.0.1694796191038; Fri, 15 Sep 2023 09:43:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796191; cv=none; d=google.com; s=arc-20160816; b=LDaBpcpunmuqFe4cYV5zaiahc7mI4lDF0hZ3YA9vJ0qf7cujw5MIemPE3NPxfR5EHl GgOyg6gkehSnoJZukle9MHmb6Y2bLS/OoCEKZHHdOketHH8lIsOtO9WMshjZTcyYm47o 5eQGuABYLg/vRCHkAguuJY/UpF8NUlPXcKuegWWvvSh2/A25nk7WYeTOu2sHwdSafa6m 7mJF7VTuVRKO/k6/e/6es4Yb/SK2TlgfHEONDW7lV5nYyFEZhnEUMfEYb6/oFMZu50xU YXppSIt2E8ViSZbNgpdVCP6aKMyvLcwwNvLfVhSa/SH5oj1z81B/vYYya/Iyu7Xy6/Oi R9Bw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=FtAcyeGzgl3itMEed21QzOZU/TN8YrJ8FEjkKrDeS66CLUjStlYS5qPRlzC0030gcc wLAMGm8nt+IYPhIOi5vN8USBk9KMs83ZSy19BE0cJfV+9jws9DY147RvBeJ+mvAueE1v UMYpC1/OL07DDwASR3OXsAHwqRQH7qRdc6MLnTBJU7iDcGQu3JHaqRMlec+K6BytqdIS A9d3wR82TSSzHKlGE9DbhXI+GybXpkkKwoymmsfd4h83tbXbfTDA+b9SGAHHGpLbKu/T bV/DtbHa+QkA5sce/5Vtt84UgoJ6Qu3WSLpF+35NBglEtZH1x4kUZ8gDb/sKr7zmfd77 auxg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q2jbA0or; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y19-20020a05622a121300b00417a07685e9si178899qtx.147.2023.09.15.09.43.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=q2jbA0or; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu8-0004Wv-Am; Fri, 15 Sep 2023 12:42:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu6-0004Ve-9A for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu4-0004x0-6M for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:49 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-68fb7074348so2162535b3a.2 for ; Fri, 15 Sep 2023 09:42:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796167; x=1695400967; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; b=q2jbA0or9bLiKmnvehCglhM+gRZjrwGLNVEhIUfbgtz/gDNGPoEzYzgsqBeEI/B/SS MvWeJ8qjW0on6pyr5Wj43r75WwhbHkbjiibg6zaf3fHS1gho7BCZwVLty4x9RJCbgyw+ 4DlZvq/gHk3lr70QBCsjvgkWFwfb+LZmyKeLzoQEOPSayBpbF1SDojrzgNivuXFbPmMx hLqxFs1t2j2JyT1R6XfzdZP1oTswgI1pj1r2NfYv5aRpl6lRn06vdby5OhSol3uMY/nA nl0K6HO0LCmQbs3IVxDgJXsO6IyXydE2ZmXqpGfeYpcplqlS/kIBPp+tJb8O+sxk/uip C2xA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796167; x=1695400967; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; b=eNuCAtjRM1UJLnxQruMn0Fgfsk6WeFwli8Dy60My1cKNa1lcOVVmttrC5NldA0up7a XMveBmo+rRbHWObiGOo5kmMjJgm22LmmCICw+5gUuaCfmD9Oo96jTFMWtWh9i5fNESmp m+Hl7Q7znfiNJmktTCTXBVATWnJQLp4Zmbyql63BPkuogtleXUxXRHJjjH3zpEdWsyuj n4kZdxFBi6DZhuzmfBeX3zudgmUbeiZaQASOXU73HaZ+/iJKdjhlGha7cYTuqkgpN0oq 2ldfkIRLetsIG7fjKFUaW+gTquSLck/3Zeosd5a67QxvO2cYSUNnW+AZP2wDWtXOE8pk fxLA== X-Gm-Message-State: AOJu0YzzKc8pxbk6YwzfypGkAlg87aHvoLZVhw04xfdDWp+X8M/aME5r E4bsiKDZYtV8b6bGx69ble+Fssd/2geFt9qxV8E= X-Received: by 2002:a05:6a20:1442:b0:14c:a53c:498e with SMTP id a2-20020a056a20144200b0014ca53c498emr2841900pzi.42.1694796166943; Fri, 15 Sep 2023 09:42:46 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:46 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 16/19] target/s390x: Use clmul_64 Date: Fri, 15 Sep 2023 09:42:28 -0700 Message-Id: <20230915164231.123580-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the generic routine for 64-bit carry-less multiply. Remove our local version of galois_multiply64. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 58 +++++++------------------------ 1 file changed, 12 insertions(+), 46 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index ba284b5379..b18d8a6d16 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -21,13 +21,6 @@ static bool s390_vec_is_zero(const S390Vector *v) return !v->doubleword[0] && !v->doubleword[1]; } -static void s390_vec_xor(S390Vector *res, const S390Vector *a, - const S390Vector *b) -{ - res->doubleword[0] = a->doubleword[0] ^ b->doubleword[0]; - res->doubleword[1] = a->doubleword[1] ^ b->doubleword[1]; -} - static void s390_vec_and(S390Vector *res, const S390Vector *a, const S390Vector *b) { @@ -166,26 +159,6 @@ DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -static S390Vector galois_multiply64(uint64_t a, uint64_t b) -{ - S390Vector res = {}; - S390Vector va = { - .doubleword[1] = a, - }; - S390Vector vb = { - .doubleword[1] = b, - }; - - while (!s390_vec_is_zero(&vb)) { - if (vb.doubleword[1] & 0x1) { - s390_vec_xor(&res, &res, &va); - } - s390_vec_shl(&va, &va, 1); - s390_vec_shr(&vb, &vb, 1); - } - return res; -} - /* * There is no carry across the two doublewords, so their order does * not matter. Nor is there partial overlap between registers. @@ -265,32 +238,25 @@ void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(v1, &tmp1, &tmp2); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = int128_gethi(r); + q1[1] = int128_getlo(r); } void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(&tmp1, &tmp1, &tmp2); - s390_vec_xor(v1, &tmp1, v4); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = q4[0] ^ int128_gethi(r); + q1[1] = q4[1] ^ int128_getlo(r); } #define DEF_VMAL(BITS) \ From patchwork Fri Sep 15 16:42:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723139 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488690wro; Fri, 15 Sep 2023 09:44:22 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEFs4ZgBjNm+XyzKvjrHwkJbpw75abHuJI8z1OXwnQYavIdI0zhRAPUJBiHykrbHMircxVg X-Received: by 2002:a05:620a:2988:b0:76f:1aaa:1bee with SMTP id r8-20020a05620a298800b0076f1aaa1beemr3131997qkp.62.1694796262138; Fri, 15 Sep 2023 09:44:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796262; cv=none; d=google.com; s=arc-20160816; b=ejbe/hiprPFLfjppHIKXYPkHRQWZ8VPwAQyM9goJKE19qobBrv57YYVNLM/XdwX5bf CxCQGmTClaYkpz+CKn8uElG0SUv3NsGLT/ob9M+ikawIiuo/LFkhmI6tNgcjWD4i7re/ a2E0F/zS4fIxiIrBn8bu3kFA613zz+otbvc8nKiyOyoCBRfz4wzJvysFt9VRCOGrT42g GPFpflnAcma3sHBIRstYX17mhhb8ifWQ110u8D/yGpr+zPVRsDGCEtOSmg3KWZO2Bnhu HWk6MV6clJeEK1UZeGOflmw91oXGA09x0iHKtPmrmJ/RmHkYDv6renf6398/hqMwN/iK qR7A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=2Rw61LzYCVJHqZF5cECHlyAs/184pzzLTdpSydfaqRc=; fh=kQadAhxV42GJ3cd2VUw1RyURv/zJQGvmUdF7iLEVTtc=; b=YK5uhCtNMtSuauaVCCRSxInImF4UZq4EllyEKw4YQT7jHwDc/0zmjmyHphHpevbN+v I5/CqKwE9219sznFAAgHRSzVrVrJdX+MCQ+ZK7Gqt3f/draCEXY3UYeRfjQ7N2s3gHn5 w3k4GPTHe2yG25A9+kljlYF+uTXIEuVnxMLlIvtfOuB7buEpGNVjkOrl+V1kZPcOJt0X x3ppnaoIcxpu82fQ7cSGs6+2rQHfDQfn/vxnZT35Trq100E+GYfXW3KYRuvqlBP4bRXi BizQBDu8knNDsWXEqhVv1xhmGI5Q9Cdh7M0NlbJhqDqy68UjfjmH5CtPwNuBH+hPZG0q Iv2A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LgM9q1o1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id op4-20020a05620a534400b0076dae6bce18si2562590qkn.352.2023.09.15.09.44.22 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:44:22 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LgM9q1o1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu9-0004XR-4z; Fri, 15 Sep 2023 12:42:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu6-0004W5-Tc for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu5-0004x6-GG for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:50 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-69042d398b1so989433b3a.0 for ; Fri, 15 Sep 2023 09:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796168; x=1695400968; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2Rw61LzYCVJHqZF5cECHlyAs/184pzzLTdpSydfaqRc=; b=LgM9q1o1ACtTVPeJnffKhNDMZzPVeM1QtgytIyhEsDkoJYB7208bUVxucr9Swpr3hT 1YHNHoiPxLh9ygVjzwzTPxE9J1UTYz6CkUqDlo3zedrSGazRwLcsn7I1xHBpbG/zFY4F Th/Mql75CYayuyQohw5ixcySO2OFoF3zBHtTGMEsDtjOHq/V1SgH9lJVzkymL5bXqwyK FpPbL1z0tlorxnkgHA0oAsDvscWfhtFck7U5CLaRrcv3/tik/HUz5/5XAX0W4VYNr3KT E1d/cgcWbFUCSbh7G1eo8OPku67RQTqlf0r7utDLm4ePX1k4kuGZ6x2PUI8DGKtdgPhp BuBw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796168; x=1695400968; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Rw61LzYCVJHqZF5cECHlyAs/184pzzLTdpSydfaqRc=; b=VnyzUTFItrfU5qqzIcKlQuGP5l/Ebb7os2VI6kNiybJVf+HRRMdnJXrPgSm85BKJ+D mAKAVSD3eBG5Na0e99dSUPG2ax9o5eefsc+xzyX+fcPx3ClusqjLDMV8MApgZbf87Lda l38rNlMWiutGIIqSaxQKhCkHnyPOzZcrEa7001EFuJuqHs3LbbmtKfljYLRl520Mq2tL noImBWR3JaMVoqvkMJTdWp5S6jXatdkp2+hvNC3Z3cfAPirMpR4LXFAUWxfAaSriRQsd nGKeyf50BRP/gvTnOge3pUCO1AwPYYdOkGd2rUhLQCrkNEkyCBAGiGLXTr8DnzQ83s7O kjYg== X-Gm-Message-State: AOJu0YxAaqZQCr+Rg6/iBzXtIIH823UaAvK6kKvEfZKeseJSGm2kjG8g BoJ01UzKa76JqHcheFvsRs11oA/t+1ZhdjTfa94= X-Received: by 2002:a05:6a21:47cb:b0:153:24c2:c93b with SMTP id as11-20020a056a2147cb00b0015324c2c93bmr2212368pzc.1.1694796167987; Fri, 15 Sep 2023 09:42:47 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:47 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 17/19] target/ppc: Use clmul_64 Date: Fri, 15 Sep 2023 09:42:29 -0700 Message-Id: <20230915164231.123580-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 1ea42b4ede..6fd00684a5 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1455,20 +1455,9 @@ void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { - int i, j; - Int128 tmp, prod[2] = {int128_zero(), int128_zero()}; - - for (j = 0; j < 64; j++) { - for (i = 0; i < ARRAY_SIZE(r->u64); i++) { - if (a->VsrD(i) & (1ull << j)) { - tmp = int128_make64(b->VsrD(i)); - tmp = int128_lshift(tmp, j); - prod[i] = int128_xor(prod[i], tmp); - } - } - } - - r->s128 = int128_xor(prod[0], prod[1]); + Int128 e = clmul_64(a->u64[0], b->u64[0]); + Int128 o = clmul_64(a->u64[1], b->u64[1]); + r->s128 = int128_xor(e, o); } #if HOST_BIG_ENDIAN From patchwork Fri Sep 15 16:42:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723136 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488449wro; Fri, 15 Sep 2023 09:43:46 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGW1omn5xoHWqa4LuoECTIkJ8r+rwDltPhrAdG/Wr8wffMcTNXvuhbbUb+LsQJYb2EYvbDD X-Received: by 2002:ad4:4c48:0:b0:651:9fa8:9f62 with SMTP id cs8-20020ad44c48000000b006519fa89f62mr2465128qvb.48.1694796226725; Fri, 15 Sep 2023 09:43:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796226; cv=none; d=google.com; s=arc-20160816; b=Vpnqj/FgZNz4pF81pVkY8ql4ehn3LPxLxL4S09PszrDG6Y0GK8+5SXTvS7uTpa2A28 acD4z+DhpttVSVJa3YoqZnprFObB5N86XgSEKMr8eiEkqKhi9XwpUbxGSXXTTFr6Guoa 8IOhx3leWDnG9Sunc+CBIoz4Vc297A2XEg6Hzs4kg9W/MQ1EPmNUFn1bihDWY3Io5d3Y D3AKQQ40gU8BNxr5pgJekErwtTeQDtZNh5yMDdNbeIcVBm9Xiijhaxmu8jzEGZB0GAdY MN0COs6TzZCuIastUqH8LhAzxudcwY0LTzUUfEmj2BVfXYe0rPVKr3ag6M+ETWNXd1Ry gbfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=+M2eLP1MxR8ITUHAyqevQXr9QFlyFahQKSge9OJBONo=; fh=9wguxkwuuSBbOibCpq3+Qud4pl++l1lkTpkAOiV7WAk=; b=WndYeitoeYdA2sVfnTpQlby6lMQILtiX7osLl7/jXbeK03Z5YQ4yPQLlqeWnYnEgdc LIZ2tEnN3y+WDBEkY6TrSsQse6MGw14bJ5n/AvGY7W1uvOJ7LAoebDzRP9nnBNu5U2il GXTwMmbkUKnQEiwX4LCnj+HqiD+MHu+Km21oXIWx5udqxT5sKgQVoXrrb2mfU0th8aMb DnFIl4hbCzOZAkLL+yx272LhGoc5LJ0O6GoZ33oYiMtLWNGS/xkWWxe6MeQmyCmfFost zsOMR8AEfFrHNpGBWDUGyU2ziHG0Q7TTJWw7Y7YZ5C98mWoR9xrcqfVKAgqtQ6fK6vzI DZ4Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vZjifz+t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id d17-20020a0cdb11000000b00655c625e66csi2510086qvk.341.2023.09.15.09.43.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vZjifz+t; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBu9-0004Xk-NM; Fri, 15 Sep 2023 12:42:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu7-0004WU-Pk for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:51 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu6-0004xJ-67 for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:51 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-68fc1bbc94eso2018006b3a.3 for ; Fri, 15 Sep 2023 09:42:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796169; x=1695400969; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+M2eLP1MxR8ITUHAyqevQXr9QFlyFahQKSge9OJBONo=; b=vZjifz+tfhcOdUjJoAeQUXBlEao50fxlO8brSd24jT9+yf80gl59kdpx1PDzd+Pfpo 2PfDPepM9h0WgINr2mpUYc5w61QUV8WkkFuafsXKZtnf7OJck28bKa2K5vZnps9R8H55 tOMgk8u28gHWrMVb7SnbfD81ooIRNVXWoxIm2PaVeEhte48z025PaO3V+4jPjNKM15nr rE6Yv9TdIQ9+AcjMdLUfAlQTzM4h8KybtmZgX9dzxh55UPo+fs3mqJinTlVeLaTSR+gs UD9UxPFx92PBhql/X9dafNuGopCVmI0+slCjBq2SEiCHLxwr4l5r1FgrLyeEuafa4ivJ mq7A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796169; x=1695400969; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+M2eLP1MxR8ITUHAyqevQXr9QFlyFahQKSge9OJBONo=; b=uO0PfmSEYPkdb4/N83+OfqLR4nzmcD0Sr2cHA3b8Fkp8am0/yMEp2arZZT1jRHm3Ck a6NfwGO0naLZuZF2t9YxZPp1EiRLJ63EdYUXihs4d8Gb5850dlTFiibjBoILPLBIJuzZ jn+ZiiF14Q1tncrhAynHY4uKTZ+6tYhYCo76W1gbsxBF/2xoHQoAw8Z4uSic5m0M/DzH WURfyysnMS0LhLucOgxWoAtN3oU6MLHzIvZzxBgzV6BQ1LhCN8opf0SDSvD0TsGk0kxl KLNreHNw7psJYj8Vf/LKK6YhjjwvH1H0dwHm4NLXGcuAyxC32TbNfDnh+Xphcsox8WsQ 1yyQ== X-Gm-Message-State: AOJu0YynMDf+Xhf2Xqw2q2kMYwz0dAbluHz3uaEnJWf2XX0ZZ9oK9/ev kBBkB6gWaHqjuAves7WauOENhlkQjaUMWw3aOMs= X-Received: by 2002:a05:6a20:12ca:b0:131:a21:9f96 with SMTP id v10-20020a056a2012ca00b001310a219f96mr2914813pzg.6.1694796168907; Fri, 15 Sep 2023 09:42:48 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:48 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel Subject: [PULL 18/19] host/include/i386: Implement clmul.h Date: Fri, 15 Sep 2023 09:42:30 -0700 Message-Id: <20230915164231.123580-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PCLMUL in cpuinfo; implement the accel hook. Reviewed-by: Ard Biesheuvel Signed-off-by: Richard Henderson --- host/include/i386/host/cpuinfo.h | 1 + host/include/i386/host/crypto/clmul.h | 29 +++++++++++++++++++++++++ host/include/x86_64/host/crypto/clmul.h | 1 + include/qemu/cpuid.h | 3 +++ util/cpuinfo-i386.c | 1 + 5 files changed, 35 insertions(+) create mode 100644 host/include/i386/host/crypto/clmul.h create mode 100644 host/include/x86_64/host/crypto/clmul.h diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h index 073d0a426f..7ae21568f7 100644 --- a/host/include/i386/host/cpuinfo.h +++ b/host/include/i386/host/cpuinfo.h @@ -27,6 +27,7 @@ #define CPUINFO_ATOMIC_VMOVDQA (1u << 16) #define CPUINFO_ATOMIC_VMOVDQU (1u << 17) #define CPUINFO_AES (1u << 18) +#define CPUINFO_PCLMUL (1u << 19) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/i386/host/crypto/clmul.h b/host/include/i386/host/crypto/clmul.h new file mode 100644 index 0000000000..dc3c814797 --- /dev/null +++ b/host/include/i386/host/crypto/clmul.h @@ -0,0 +1,29 @@ +/* + * x86 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef X86_HOST_CRYPTO_CLMUL_H +#define X86_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +#if defined(__PCLMUL__) +# define HAVE_CLMUL_ACCEL true +# define ATTR_CLMUL_ACCEL +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PCLMUL) +# define ATTR_CLMUL_ACCEL __attribute__((target("pclmul"))) +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { __m128i v; Int128 s; } u; + + u.v = _mm_clmulepi64_si128(_mm_set_epi64x(0, n), _mm_set_epi64x(0, m), 0); + return u.s; +} + +#endif /* X86_HOST_CRYPTO_CLMUL_H */ diff --git a/host/include/x86_64/host/crypto/clmul.h b/host/include/x86_64/host/crypto/clmul.h new file mode 100644 index 0000000000..f25eced416 --- /dev/null +++ b/host/include/x86_64/host/crypto/clmul.h @@ -0,0 +1 @@ +#include "host/include/i386/host/crypto/clmul.h" diff --git a/include/qemu/cpuid.h b/include/qemu/cpuid.h index 35325f1995..b11161555b 100644 --- a/include/qemu/cpuid.h +++ b/include/qemu/cpuid.h @@ -25,6 +25,9 @@ #endif /* Leaf 1, %ecx */ +#ifndef bit_PCLMUL +#define bit_PCLMUL (1 << 1) +#endif #ifndef bit_SSE4_1 #define bit_SSE4_1 (1 << 19) #endif diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c index b2ed65bb10..9fddb18303 100644 --- a/util/cpuinfo-i386.c +++ b/util/cpuinfo-i386.c @@ -39,6 +39,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) info |= (c & bit_SSE4_1 ? CPUINFO_SSE4 : 0); info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0); info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0); + info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0); /* Our AES support requires PSHUFB as well. */ info |= ((c & bit_AES) && (c & bit_SSSE3) ? CPUINFO_AES : 0); From patchwork Fri Sep 15 16:42:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 723134 Delivered-To: patch@linaro.org Received: by 2002:adf:f0d1:0:b0:31d:da82:a3b4 with SMTP id x17csp488343wro; Fri, 15 Sep 2023 09:43:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHPxCUAehZLcMwcrMxFvksQHgTI27tCag7Ovyc7/x2IcUiDCT7lHO26o/Wuc1k4zaMtT1Ke X-Received: by 2002:a0c:ab1c:0:b0:63c:f605:48a with SMTP id h28-20020a0cab1c000000b0063cf605048amr2471897qvb.56.1694796216111; Fri, 15 Sep 2023 09:43:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1694796216; cv=none; d=google.com; s=arc-20160816; b=bvFkPQCxYJU+q42rXdGa0sH3sxY+IcwGbN5eXDAuG5UAbHzCPp6Et6cklXMi5ORVoC 6jNu9+FdRPFWg/jcVyPvYcdVHwDV6G/H6JqTxNqW3X5W6aqf1Gzt2uQyqAw9YsDYwisC 2RtxMg04rqcKinjIfT/7dYjN2Ng2ZERcQaKelzMhC/wjHGiDiBpdwr6/UeELMHsk6HI5 cV0DwjA1pxhjrUNzZTQtyFf3L9qaXq56lE7iIbYJBNoapelCNR2Bxye18hwbGLj85Cut u69aKZAFaCw/xszvEqMRVRnCGgnyxuN6gcaI1NcKGLiZ1Bk0hI3XPyPo0378ZiAc3Vok ZjCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=26aP/OX6idfw5HUk5RP35KGUcj8QnPWK4lUzZokYGcA=; fh=Te/NSOMwRTSPdvjpWoYaRFEelRwTFYBsxaiZirbKEXg=; b=FGNyOKoVY2yT37NFR7c1oorQlGDChb+Fm9W+6g1cIuRU0HbEcCFFmFzScMDSURgiLM hant89/OJj5mVOH+VP6oFhY+YoTLDDYXYR1Bos4uOqwwM3Tj3AFF+64yNwo79NA7eNns JGulohSVLva4ouHou01+J7qjafqjVYPnmLBKL3hM8WvF5AoPJBKalGKDTmQ2+ZwTUIIo DeLyieKGnNFx73kf7bfxk6r9Lab/ronvZoeVf0cHXfUspn8oJLe6npb8AvxHayhezNSI 713QehAFiju3+2K2nJfJfnGhYil4emZOwONXewnJKckSC4iaGkv5KmwSYTrE/ktKhqRs jCeg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=X9mwh62I; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m16-20020ad44d50000000b0064f93526acdsi2377884qvm.542.2023.09.15.09.43.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Sep 2023 09:43:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=X9mwh62I; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qhBuB-0004Yc-Rj; Fri, 15 Sep 2023 12:42:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qhBu9-0004Xj-Jh for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:53 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qhBu8-0004xU-0S for qemu-devel@nongnu.org; Fri, 15 Sep 2023 12:42:53 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-68fb898ab3bso1947820b3a.3 for ; Fri, 15 Sep 2023 09:42:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1694796170; x=1695400970; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=26aP/OX6idfw5HUk5RP35KGUcj8QnPWK4lUzZokYGcA=; b=X9mwh62IZxuK6sgKwMAEMThT2Q2HOdpw6zWO1obfjuriIklwrf6U15lGjRr64hVxzr r0ZeOYuRnt3DN7qpM018Bdoul87RPhTijX6l4KJaWpVjWXQYTMRLyL1UGOSxlSGlD7CV 9tvBoc2UbbBbrbe+SZZE+6d7xn3tZNGtLBJ6+jKfZRJDcBPpRrrnLSM1ICVb9bY67dPx UCJwTIqQE2M7WMD8LGjQEzFFWVTrAi3O/z2IQOwfsgVnyQL6epgptX7ZV+Aer2rQiOp2 eBusDAeRuFNlIU7O8yS6ae1VMUa7hnAnUEsQs7wUWJxMeXr3z7TE/31lSEokKFtZUhXF JxTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694796170; x=1695400970; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=26aP/OX6idfw5HUk5RP35KGUcj8QnPWK4lUzZokYGcA=; b=pgQuNIaKNzaE9pACGRnMuEZih0kRSrV4xm5kdNRx+Oi7nzSX6d7gzs2eNgzPsA0Jly 97IcS3dP5DtUkGs2cn02jJatxHqbmW3H8K2MIR/RGOZNuzWmlxZZ7Pq+sjlsV20etAZY qZx1LocpT2uAQBrFG+AaRezQc6laUYdgS06tgeD1d36D+vTHwvj9WF6XjBKmEgOgJP7q 6075J2pbcAaoVCN5b334FXrEw3WHSMh9LaexWJhzeDEzRovtRTTpFVcIEKz6XzzUS6Hd 1DMJqDW7pBFNB7l8qTneCele3KrH9wQIERujRGls0O1yFLvSW0O5RD42oH6/o4OdgzzE YjCQ== X-Gm-Message-State: AOJu0Yx0jCNEfZWsUOCM4TRWpoz7zLJd4yn68LHP2P4hfL1j7OUXiqem mehcfIAaaV+YfBU+zLCvnyjNCu2A6gLNuBEVo+k= X-Received: by 2002:a05:6a20:1581:b0:148:f16f:113f with SMTP id h1-20020a056a20158100b00148f16f113fmr2735613pzj.12.1694796169829; Fri, 15 Sep 2023 09:42:49 -0700 (PDT) Received: from stoup.. ([71.212.131.115]) by smtp.gmail.com with ESMTPSA id s187-20020a6377c4000000b00577f55e4a4esm3010087pgc.20.2023.09.15.09.42.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 15 Sep 2023 09:42:49 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: Ard Biesheuvel , =?utf-8?q?Philippe_Mathieu-Daud=C3=A9?= Subject: [PULL 19/19] host/include/aarch64: Implement clmul.h Date: Fri, 15 Sep 2023 09:42:31 -0700 Message-Id: <20230915164231.123580-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230915164231.123580-1-richard.henderson@linaro.org> References: <20230915164231.123580-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PMULL in cpuinfo; implement the accel hook. Acked-by: Ard Biesheuvel Tested-by: Ard Biesheuvel Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- host/include/aarch64/host/cpuinfo.h | 1 + host/include/aarch64/host/crypto/clmul.h | 41 ++++++++++++++++++++++++ util/cpuinfo-aarch64.c | 4 ++- 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 host/include/aarch64/host/crypto/clmul.h diff --git a/host/include/aarch64/host/cpuinfo.h b/host/include/aarch64/host/cpuinfo.h index 769626b098..fe8c3b3fd1 100644 --- a/host/include/aarch64/host/cpuinfo.h +++ b/host/include/aarch64/host/cpuinfo.h @@ -10,6 +10,7 @@ #define CPUINFO_LSE (1u << 1) #define CPUINFO_LSE2 (1u << 2) #define CPUINFO_AES (1u << 3) +#define CPUINFO_PMULL (1u << 4) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/aarch64/host/crypto/clmul.h b/host/include/aarch64/host/crypto/clmul.h new file mode 100644 index 0000000000..bb516d8b2f --- /dev/null +++ b/host/include/aarch64/host/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * AArch64 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef AARCH64_HOST_CRYPTO_CLMUL_H +#define AARCH64_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +/* + * 64x64->128 pmull is available with FEAT_PMULL. + * Both FEAT_AES and FEAT_PMULL are covered under the same macro. + */ +#ifdef __ARM_FEATURE_AES +# define HAVE_CLMUL_ACCEL true +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PMULL) +#endif +#if !defined(__ARM_FEATURE_AES) && defined(CONFIG_ARM_AES_BUILTIN) +# define ATTR_CLMUL_ACCEL __attribute__((target("+crypto"))) +#else +# define ATTR_CLMUL_ACCEL +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { poly128_t v; Int128 s; } u; + +#ifdef CONFIG_ARM_AES_BUILTIN + u.v = vmull_p64((poly64_t)n, (poly64_t)m); +#else + asm(".arch_extension aes\n\t" + "pmull %0.1q, %1.1d, %2.1d" : "=w"(u.v) : "w"(n), "w"(m)); +#endif + return u.s; +} + +#endif /* AARCH64_HOST_CRYPTO_CLMUL_H */ diff --git a/util/cpuinfo-aarch64.c b/util/cpuinfo-aarch64.c index 7d39f47e3b..e0e1fe6071 100644 --- a/util/cpuinfo-aarch64.c +++ b/util/cpuinfo-aarch64.c @@ -56,12 +56,14 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) unsigned long hwcap = qemu_getauxval(AT_HWCAP); info |= (hwcap & HWCAP_ATOMICS ? CPUINFO_LSE : 0); info |= (hwcap & HWCAP_USCAT ? CPUINFO_LSE2 : 0); - info |= (hwcap & HWCAP_AES ? CPUINFO_AES: 0); + info |= (hwcap & HWCAP_AES ? CPUINFO_AES : 0); + info |= (hwcap & HWCAP_PMULL ? CPUINFO_PMULL : 0); #endif #ifdef CONFIG_DARWIN info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE") * CPUINFO_LSE; info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE2") * CPUINFO_LSE2; info |= sysctl_for_bool("hw.optional.arm.FEAT_AES") * CPUINFO_AES; + info |= sysctl_for_bool("hw.optional.arm.FEAT_PMULL") * CPUINFO_PMULL; #endif cpuinfo = info;