From patchwork Mon Aug 21 16:18:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715379 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924829wrs; Mon, 21 Aug 2023 09:19:47 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGs4DEWAvUbdzXhEjJwnKkeTAW96EB7mCo7kX2sD+GN1X4lz2AkEt2tQEBh1UuOKYLmSLUm X-Received: by 2002:a05:620a:b06:b0:767:26d:1142 with SMTP id t6-20020a05620a0b0600b00767026d1142mr8195232qkg.21.1692634786811; Mon, 21 Aug 2023 09:19:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634786; cv=none; d=google.com; s=arc-20160816; b=fHTu54t2Mog9j3yUXwU5nz6hGX7mLVn0uIgRch7HSIpV4hV66jBjmpxQv+MdwUOr1d oM8ZKC7nRKguuc4ZM2wT46xC5wl2lrM5vS2r65cp8YeIV1NN4w7OYXo33WI6O4icBTke QGlH22R8kNzOa1XVXqiEGKtZimgiwHhsn7jca8XIovDwgU+R3PDnGDBLQ9kDmRVvlntF TAuD13Y2Tog6GoiZK1lDRpYRP0a/9P8fDFHkMlS2PxN08DdK956VNE+EofgAWt00p1IM 4FFdSJKnZkjt264aMxu7+Il9qog1bVxC3UAQ9GEhUAijvKeTj8ef6J+tWpqhD8DNZwfd y+Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=i78r5+dhWO2ItlOwsQIOsAXAZIfTre5X4OBHCSbCo/erxhxsaFcrKH6sApcBaKWdJ2 bw0hYMv3/OBlmax6rj0vMWxThHndVy/OQXWoStUme2kD2og8xg4LmTbL2it+u+CWMrTl JStcLeQauZpvaTM3HBGxADZetdeQUMVAn2uaaqNjGJ2JbqV4JM8QaZ24JrUX1CyWevtS WtUVb2NcN6pEfqrjZB1btYmeuXY9UNoU7LDXz+XXiPJvc/ra6f0NsiQ6npCsG8higP5V cjy5kRoKSAumygWzHXSoNjox0h30nLW535gvqfxbNTHgfw4oLdfe0BE19QAz3mG3LBFd UfHg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xVCAyVBy; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s9-20020a05620a29c900b0076cb6c9e0a6si57297qkp.336.2023.08.21.09.19.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xVCAyVBy; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cM-0001T2-HQ; Mon, 21 Aug 2023 12:19:02 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cK-0001SY-57 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:00 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cH-0005b7-PN for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:18:59 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bdaeb0f29aso21174715ad.2 for ; Mon, 21 Aug 2023 09:18:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634736; x=1693239536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; b=xVCAyVBy5QIyd/LkOZ/Y1ilE3FRVG1Z8sQjCZLOomDgsqF+X/dciH7n/NI2ufoHk6V uK0Nn8ZPd3FRFxZEZ0WcNpGDgd12YlgIfQVQDAz7udcm/gu4sGhqunvV3VPl5fQhTp+o 4NHldpb3mYSZ0D3jJDPJnpxMZvZaOjgW0/jMbOKlZb2CJVnjxpLwqKIHyym2cPLSgLnB VuLUtMX931T2DPv4KFQ2EcOBdjBDPQrsROHzZAT81teQfo48DVl7dJz/nsvGnKn0k3fu pMT3sIzwHkLrxL4ETzJ9wMRXaI1fky5PctH2TNi1TwjDy+BDcpRSf9goeIl+NAk/e0Y5 Kn0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634736; x=1693239536; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; b=YzCMIXPpmnce/LFWDNw7kAa8As+WAoZr5wz47DJ3R4yZTIYRp20QpZqxQaocWU8xS7 rGkONUi2yC+dLvePDAEUw1pd+ye4SbdkIU/htEBh4A39pV2ZtHga3xclFnfkaZzDENgv UsIQzRhVbjd4YJwFTQiWVj8mNkjreU4Df6MpJOMIiDRCamz1hCAT+UkIvLA7t/F5v9rv hmzBkqGvvHccJFY2l8rVdLEYGmEKttz2O42S/a1G5SNYM0ZNBhb+ju5iDHjSsy0zprj2 YaYATyZcgvbyqrkXtdM40iGr7gpavcff6D5XrQmCrihFK+hzUzdbhX8qsx84U0gdqj7h g6Pg== X-Gm-Message-State: AOJu0Yy2R8ELegcxmkLa7H7cno6o0h1zqP1UuImxGwiud3z0Wd4oyey6 Ugpy1KkwgJ58jbD6SjswZHyNcYWFr+SnYNBzYHk= X-Received: by 2002:a17:902:ce81:b0:1bd:d92d:6b2 with SMTP id f1-20020a170902ce8100b001bdd92d06b2mr5570741plg.10.1692634736293; Mon, 21 Aug 2023 09:18:56 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.18.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:18:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 01/19] crypto: Add generic 8-bit carry-less multiply routines Date: Mon, 21 Aug 2023 09:18:35 -0700 Message-Id: <20230821161854.419893-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 41 +++++++++++++++++++++++++++++ crypto/clmul.c | 60 ++++++++++++++++++++++++++++++++++++++++++ crypto/meson.build | 9 ++++--- 3 files changed, 107 insertions(+), 3 deletions(-) create mode 100644 include/crypto/clmul.h create mode 100644 crypto/clmul.c diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h new file mode 100644 index 0000000000..153b5e3057 --- /dev/null +++ b/include/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef CRYPTO_CLMUL_H +#define CRYPTO_CLMUL_H + +/** + * clmul_8x8_low: + * + * Perform eight 8x8->8 carry-less multiplies. + */ +uint64_t clmul_8x8_low(uint64_t, uint64_t); + +/** + * clmul_8x4_even: + * + * Perform four 8x8->16 carry-less multiplies. + * The odd bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_even(uint64_t, uint64_t); + +/** + * clmul_8x4_odd: + * + * Perform four 8x8->16 carry-less multiplies. + * The even bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_odd(uint64_t, uint64_t); + +/** + * clmul_8x4_packed: + * + * Perform four 8x8->16 carry-less multiplies. + */ +uint64_t clmul_8x4_packed(uint32_t, uint32_t); + +#endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c new file mode 100644 index 0000000000..82d873fee5 --- /dev/null +++ b/crypto/clmul.c @@ -0,0 +1,60 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#include "qemu/osdep.h" +#include "crypto/clmul.h" + +uint64_t clmul_8x8_low(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0101010101010101ull) * 0xff; + r ^= m & mask; + m = (m << 1) & 0xfefefefefefefefeull; + n >>= 1; + } + return r; +} + +static uint64_t clmul_8x4_even_int(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0001000100010001ull) * 0xffff; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_8x4_even(uint64_t n, uint64_t m) +{ + n &= 0x00ff00ff00ff00ffull; + m &= 0x00ff00ff00ff00ffull; + return clmul_8x4_even_int(n, m); +} + +uint64_t clmul_8x4_odd(uint64_t n, uint64_t m) +{ + return clmul_8x4_even(n >> 8, m >> 8); +} + +static uint64_t unpack_8_to_16(uint64_t x) +{ + return (x & 0x000000ff) + | ((x & 0x0000ff00) << 8) + | ((x & 0x00ff0000) << 16) + | ((x & 0xff000000) << 24); +} + +uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) +{ + return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); +} diff --git a/crypto/meson.build b/crypto/meson.build index 5f03a30d34..9ac1a89802 100644 --- a/crypto/meson.build +++ b/crypto/meson.build @@ -48,9 +48,12 @@ if have_afalg endif crypto_ss.add(when: gnutls, if_true: files('tls-cipher-suites.c')) -util_ss.add(files('sm4.c')) -util_ss.add(files('aes.c')) -util_ss.add(files('init.c')) +util_ss.add(files( + 'aes.c', + 'clmul.c', + 'init.c', + 'sm4.c', +)) if gnutls.found() util_ss.add(gnutls) endif From patchwork Mon Aug 21 16:18:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715384 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924985wrs; Mon, 21 Aug 2023 09:20:01 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFZalhp32nfEzKNixQ9ihozauw2Dvjmf9OomITEPWG/mFbbz3EekpBEGz1FMd6VxVhDKyeg X-Received: by 2002:a05:6358:441e:b0:13a:1e90:4813 with SMTP id z30-20020a056358441e00b0013a1e904813mr5839977rwc.29.1692634801431; Mon, 21 Aug 2023 09:20:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634801; cv=none; d=google.com; s=arc-20160816; b=A6h99f3L9hWzWqMVffxgCfcYSK65hMQL7SjhacrJZqKJbsd1c1i0YIXmEkQXus5dNj OzRiyr2pfKDRfIXFY1jjEF/DdsCDBFdvxLVAkFp6Nq0olim6YOoZkorGZopAHrLNZ8/S KZqsixWFkOw/a4hKbpjWsQsUd/OevftImh8mYdj0BuR79ZQrE6W1fdm1Rj8wleq1u3t+ LeGAJWCSSAZ46QjKQxeKFFBLqGOn6liAF58d5GAwnrtmGId5ZNNSE6aNIKhQANHNKWxo BrOZfdDkIi9aEz2dRumAJRbAhfdonJFKlJcwTRUHHVTfvkklejQfriJpb0aK7YQ/nkCh p/JA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=ptcGjiwATB4D9yRB5apI8G71Y/WmsV3YvaOijZo8nguKd7jeAmDoyWSZKzrJ9XrWCo mBUq58vDKZfEQcKxNoeDT2ulKVIfRVg0psZkUcrTbgOulZKCYqmHmkl2uyQhGs7d2e4+ HK8PsWg0UHMrh1BgzxqKPggPkGaTZSe+7kMkCuzsZRlfGTgTyGp8Mh+DRiSGDpXFhGO/ 1cjGKpZJkAdtaCHaVgPcudyEkJPp08yrntRXAYWF2VFpQQ6tZdq6WKCJ8qiUEjbyTrvz 6fB+0kQujVMDlO6rcTMyDbv2DhX5HfpJ8U6ySYL3ZG2aBYLmjIe/k8zeU/7b3yABBmB+ 78KQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u2j93TqE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g18-20020a05620a13d200b00767b7c5897fsi4826283qkl.412.2023.08.21.09.20.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:20:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=u2j93TqE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cQ-0001UO-MH; Mon, 21 Aug 2023 12:19:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cL-0001So-RK for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:02 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cI-0005bF-FQ for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:01 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bbc87ded50so21387465ad.1 for ; Mon, 21 Aug 2023 09:18:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634737; x=1693239537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; b=u2j93TqEDoLP0fnfpWt+2pGpkYwiCd90KnGLCOidQ+lAr4kgqaigxX26WD7Y5CFEP7 3rvCkGB8OIBenXNUowB4qQJeNCaUz9EEIn88cpQ5WmbbCaqN/Qpu72xnULRhfUI05pRi 43Fuy2XhIfttra66UX72Hoo13lP5lnjU/ebutEgmQNYNsML3PjhbUMovhDxW1hIGsW44 Wbt9kX7PA7+wHPT6SOGpUbFJMr/u+T/Ro4iN2L26fI2b2Adna4yE96lCfZDtin6TG9Ts lw/Y6bR1sAvGX2yZfI6Y/thP7coMBttevS21b7+1Tigi0hhC2NDCmIAVrcH2Xb1GDZ7U OlIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634737; x=1693239537; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0e/zp7VtlxeASFRSSj/wbgTxg8eFq50bfESp96aa+Vg=; b=ZJ8zr0t50Bx/jOFOaq8ZUhkHTAbytJBcxlXoJwd2ee2Mn6BXkQIBFnu2ItQWaWH8f4 QD/FkErwJ5mdqim4yLbgCqGeK+MSgTPwsJt5QMTMtjd+AoxxuHDt2yJhsR3SOX9xVvKB RCs9Mc4BpyppJo0vvOXZksPkefKPQc4MdvYpy/9n47zW3JLViGuZVJ8uu8v+RYJD3jqO Thp84f3EJJSAjSmJRzSfEVYcv+TI2wggFF47ptBxj13a0R8Q2DQN71oc9PZ3RojaNbZU hLy++ijXKg7u3qyJqGrF+hLj6oY53rv+oEvTZA7HltaiCagInfoIHsIbwBLR6bSj5cV5 W7EQ== X-Gm-Message-State: AOJu0YyfGLK2pvZxAS12q4Uk7GH7/ZYIgkDwlYqkReh6S5UDS01A0IWz 3TWI2a+tdiRw+ORyI3EqCpFnPMFcSyUdqNGJs60= X-Received: by 2002:a17:903:2286:b0:1b3:b3c5:1d1f with SMTP id b6-20020a170903228600b001b3b3c51d1fmr6553186plh.8.1692634737275; Mon, 21 Aug 2023 09:18:57 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.18.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:18:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 02/19] target/arm: Use clmul_8* routines Date: Mon, 21 Aug 2023 09:18:36 -0700 Message-Id: <20230821161854.419893-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of pmull_h. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_internal.h | 5 ---- target/arm/tcg/mve_helper.c | 8 ++---- target/arm/tcg/vec_helper.c | 53 ++++------------------------------- 3 files changed, 9 insertions(+), 57 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index 1f4ed80ff7..c4afba6d9f 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,11 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 8 x 8 -> 16 vector polynomial multiply where the inputs are - * in the low 8 bits of each 16-bit element -*/ -uint64_t pmull_h(uint64_t op1, uint64_t op2); /* * 16 x 16 -> 32 vector polynomial multiply where the inputs are * in the low 16 bits of each 32-bit element diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 403b345ea3..96ddfb4b3a 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -26,6 +26,7 @@ #include "exec/exec-all.h" #include "tcg/tcg.h" #include "fpu/softfloat.h" +#include "crypto/clmul.h" static uint16_t mve_eci_mask(CPUARMState *env) { @@ -984,15 +985,12 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPH_MASK 0x00ff00ff00ff00ffULL #define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBH(N, M) pmull_h((N) & VMULLPH_MASK, (M) & VMULLPH_MASK) -#define DO_VMULLPTH(N, M) DO_VMULLPBH((N) >> 8, (M) >> 8) #define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) #define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) -DO_2OP(vmullpbh, 8, uint64_t, DO_VMULLPBH) -DO_2OP(vmullpth, 8, uint64_t, DO_VMULLPTH) +DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) +DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 6712a2c790..cd630ff905 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -23,6 +23,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "qemu/int128.h" +#include "crypto/clmul.h" #include "vec_internal.h" /* @@ -1986,21 +1987,11 @@ void HELPER(gvec_ushl_h)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = n[i]; - uint64_t mm = m[i]; - uint64_t rr = 0; - - for (j = 0; j < 8; ++j) { - uint64_t mask = (nn & 0x0101010101010101ull) * 0xff; - rr ^= mm & mask; - mm = (mm << 1) & 0xfefefefefefefefeull; - nn >>= 1; - } - d[i] = rr; + d[i] = clmul_8x8_low(n[i], m[i]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } @@ -2038,22 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -/* - * 8x8->16 polynomial multiply. - * - * The byte inputs are expanded to (or extracted from) half-words. - * Note that neon and sve2 get the inputs from different positions. - * This allows 4 bytes to be processed in parallel with uint64_t. - */ - -static uint64_t expand_byte_to_half(uint64_t x) -{ - return (x & 0x000000ff) - | ((x & 0x0000ff00) << 8) - | ((x & 0x00ff0000) << 16) - | ((x & 0xff000000) << 24); -} - uint64_t pmull_w(uint64_t op1, uint64_t op2) { uint64_t result = 0; @@ -2067,29 +2042,16 @@ uint64_t pmull_w(uint64_t op1, uint64_t op2) return result; } -uint64_t pmull_h(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 8; ++i) { - uint64_t mask = (op1 & 0x0001000100010001ull) * 0xffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; uint64_t nn = n[hi], mm = m[hi]; - d[0] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[0] = clmul_8x4_packed(nn, mm); nn >>= 32; mm >>= 32; - d[1] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[1] = clmul_8x4_packed(nn, mm); clear_tail(d, 16, simd_maxsz(desc)); } @@ -2102,10 +2064,7 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = (n[i] >> shift) & 0x00ff00ff00ff00ffull; - uint64_t mm = (m[i] >> shift) & 0x00ff00ff00ff00ffull; - - d[i] = pmull_h(nn, mm); + d[i] = clmul_8x4_even(n[i] >> shift, m[i] >> shift); } } From patchwork Mon Aug 21 16:18:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715378 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924743wrs; Mon, 21 Aug 2023 09:19:36 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFLISA85HvMk3AYVtvk+fhZyLiz9Cu2hqXU/MYNVKgQKtjQbZDl5pIy/jjOh9k5O4ZikUNg X-Received: by 2002:a05:622a:11:b0:410:165:9e5b with SMTP id x17-20020a05622a001100b0041001659e5bmr7380167qtw.65.1692634776094; Mon, 21 Aug 2023 09:19:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634776; cv=none; d=google.com; s=arc-20160816; b=EuQpDO1HVABxHdk7qtbym6wBnnhW9bhELdLsYZV602ghV6AyT4XXpycxGaIW11yff7 2xytwY1kN6NYl9SdW/L4KncKoUGU2s96xyPNS64mglMd10oLePVYgLfIlFCKVP84d1Km kLitOAznkcjs93nPlau0iRRfA81IkFRQE6QhSEqs1p0eDxUh2Zn1wCdAMUr1dHsnAinp JEoPq8gEt7qn1yZLjtSSGcNcjctVjezfPkIfNJs0VmDRHVsjHvd9CAHrrQpfMwfThg+7 V5o3SP7NdKnKkh0p1UyIUe3WLZH4wCJNjvuPkNqjJmxuogudDnoMXREmwRTDIWG9QW/p 2zdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=srblMvZdnz6xTtnRGixxnreIZlQMmwvjoW3Ku8YrPEcaoJAqq+Li/ynKY9SQlm+Fih aQxA2Hg+Ird4kMS9LYRod/b26vbMqg0EBMU2/xfrQ6VZa+czpB6aycz8H4uZNhwWvBkb Unp4utZUp8SE6FSl3E3lD5akRck2H5Kgpji+xJSXmB4m2cQTOAZWU87xTW4c8dfpXRRa KXH5TK3WmLVsSCKY0H6YBuSIev4HvC6LohHrb6RxNNPMKyilc3kZOFzt63gIwKW4UT2j ZhKeHN7ImGAuoc1CpvZlH0G1fysGRRPSa82ZqH63x4Ljm0O8zFjNRWyxY8v+iLWBc1Eo g/Ig== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NQupEBIY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id pz6-20020a05620a640600b0076619dfa345si4934887qkn.736.2023.08.21.09.19.35 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:36 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NQupEBIY; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cN-0001Tc-8X; Mon, 21 Aug 2023 12:19:03 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cM-0001Ss-0D for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:02 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cJ-0005bL-Hp for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:01 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bf3a2f4528so25022475ad.2 for ; Mon, 21 Aug 2023 09:18:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634738; x=1693239538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; b=NQupEBIYMzUZ0PkCl7sXjsjYb8gouT7klDTzuOxUIPBe3wEXlGfXXvQbU2StyL3gQE vGyX6Nay8o4t7WwslhI/YCR6+YOSJajq7ajCd5sp5t0boMKixASiljPTfDePsoPHRm6h 6HbDchJRsSHt2LWuao3XRDu3ZDuJx1Xv9MP8JQau+DrnEyS7ADQ6jicv2PDOiClbii94 O3GBRISV8wQYCsL/1rMjojF0/hFMY1hHjd912gVdofQNOyJvO33viuJMsnH1ehyIv8SI zY2Ac1GI071hrNkkLcRt9pMVTyJavvjkrSUhKVl1TeMhSg+wyfi/hDY2KCBn20LNIqVg m1Lw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634738; x=1693239538; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=q1HqpqG1SCyo6N98nxssbdjX05tzqj9OxhfNx1abB4M=; b=hymrmUZUFwQ9vmrwlHaHTb66Fj59yJ/fmciFI8tpn5npUsPM/wCZfTE4aLg6mhrV9q 77fNLPLn6CeMinEz0LMOZkzYNBBeBvuMyEkthAqatMbv3Fcf1kmWtJSBRjMU/KoZInbU 3xkcz0risj0WBpoGJRYL0Kq+ORALXtbCTXCDmBRH5xCXpBKWhjjvNIhBA7K69wtybTra PI7QSFUQtdsN6xzyxwS1VeRKQtzHv0B/O92pv3y7N9hefO0yl8ieekb/eKdBz/g1aIr7 WFaJAnP26C6b2vQ0YhHozu1XhIsx/uxS4/vSzj4pD3eODkSW3wnh9Pgu1z6fkLiKWSj1 Fkew== X-Gm-Message-State: AOJu0YzGMIl7qh3zgJgy/fqOAZz3sugYddMQujeNpOSf5tAI+GPBSOsc IOBCVFB/VCakL7COrWd8zsy/0/CVJ3AzoN3PWvY= X-Received: by 2002:a17:902:f681:b0:1bb:9c45:130f with SMTP id l1-20020a170902f68100b001bb9c45130fmr8156841plg.69.1692634738112; Mon, 21 Aug 2023 09:18:58 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.18.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:18:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 03/19] target/s390x: Use clmul_8* routines Date: Mon, 21 Aug 2023 09:18:37 -0700 Message-Id: <20230821161854.419893-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of galois_multiply8. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 32 ++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 53ab5c5eb3..edff4d6b2b 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -14,6 +14,7 @@ #include "vec.h" #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" +#include "crypto/clmul.h" static bool s390_vec_is_zero(const S390Vector *v) { @@ -179,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(8, 16) DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) @@ -203,6 +203,34 @@ static S390Vector galois_multiply64(uint64_t a, uint64_t b) return res; } +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +static inline uint64_t do_gfma8(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_8x4_even(n, m) ^ clmul_8x4_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm8)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma8(q2[0], q3[0], 0); + q1[1] = do_gfma8(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t desc) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma8(q2[0], q3[0], q4[0]); + q1[1] = do_gfma8(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -220,7 +248,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(8, 16) DEF_VGFM(16, 32) DEF_VGFM(32, 64) @@ -257,7 +284,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(8, 16) DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) From patchwork Mon Aug 21 16:18:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715380 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924850wrs; Mon, 21 Aug 2023 09:19:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG4A2ErQOeeH+E2YTpkxGxOswSS96KlE8++fZCNEjMaiM3bqK5ZOhxYaXjlvHxPQgqvqZyp X-Received: by 2002:a0c:e391:0:b0:631:f9ad:1d46 with SMTP id a17-20020a0ce391000000b00631f9ad1d46mr7810638qvl.47.1692634788609; Mon, 21 Aug 2023 09:19:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634788; cv=none; d=google.com; s=arc-20160816; b=KamuHSLbC0fyMzLIttSNUCUlPtfoJj3P7I2hMe6RacrX0ZTq//gPIvNhalNgZ7H/+F B22ZxhubD0DgQgtfwwf8l27ERJGPzevqLMCcFNIlCrtpxcG/cZiVHfpu3uk6HU+q/6fL /k1uf6n6boYmCzBWToUbuEMOXZNAcDvjtab/yEyo7bIeQRvHhl2SccpdlzQwzNIUb/2G A42x6Nz0fKA2bTcajit3fmNA1JNKlQbBJZ3GdjZyPnzDtoV2KsouarjA/VisFu4tjz4E XUznQK7DUR46Ngux/5k4oncLmIqrO1YN4Qlwe5KZJbKkcj72bOlZmXubo6GsKGUN5hn9 PfSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=i4FMn0L6oLRdyT3+P8mIUUkaD0nrXr+no7Mh5NVF8S/i6IJDSQok/IzzKsQJm2yaGd dkueFRUYCxv+Xfb8omzYoDRxmyTLsm9scOWckcnyWA7MtwTSVRW+/SeRHWebV9JcVuiQ aD7oO37FuaeDRegQsC5GdoslHA5PK5E9NSEWUjio623qDlQpYayuMEVZoTCkJ/Ycfd3D OwPHTvOjC3kUiauewmfgFG7lLbFP24frh2y6Q3LWsqa4crIL8HDYU97b74Ahnw4g0Hro LATeU3WMh1AZGWKjcwx12uxJhUWc4yOJwnh0W1YqI9rGW5S8EaYLax9KJhTRS4KsQL+5 VD7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Uqd3pp3n; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z13-20020a0cf00d000000b0063d39088a13si30365qvk.101.2023.08.21.09.19.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Uqd3pp3n; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cm-0001cQ-UD; Mon, 21 Aug 2023 12:19:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cM-0001TR-PT for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:02 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cK-0005bZ-K9 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:02 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bda9207132so27361495ad.0 for ; Mon, 21 Aug 2023 09:19:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634739; x=1693239539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; b=Uqd3pp3n4hUiSS5XL89jyxcWHBHr0ICdGqNFkeNQuu+Pz6RFanOeIg88XphotCcQc4 8Rr25+UQAxqcNGhSYP0TxYQ/jivnI5itbbd8gLr5eboTJYa+XlIFpvOi++eWgEx9ZsMl Eds2VHnjqKNTlQfjw/kcJvbTdkXQX6BqE3Y3wGPJ47GNGe6wZRmYw4BtyDHhDdxID3hj IVM3ZkOqK10sZduxeDXWYkfBKwCAJspdup+fPbd4pnH1Il5j6fqYhm0kezQ42xGtWSl8 FPTYEzTYptAw3L5VpaqF1lC3p90rFa1UpoOUlwZrh8Q9GUVsHuCcQhDojg8pHDNMk1NE Rigg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634739; x=1693239539; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; b=YyNAnvlxM0owG82/2bszfEnI9ryZwwm9YDYEAo03+OyGn94tTKcOSgbvkS2KjeXqiX MOsIwFTGwPV60tKy1fYUIsmxB0n+R3Cvh10nSLjSXyuhqGZ/NZAJYl+u7Tl9fCKDmVqk cYFk9EEg9JcMtEPXZf97IMIzlsU/kIEnqTW5xJz3BqezVBz0o+vVn8d4OwF6b4V4/WV9 idGAfMD3EMLzC6tct6WBJ/HCxYQOBbfxGmPi/LkWD01kq3va2ROtYRykRkr2VSssOwgQ 0Eohf9+YIIZYOuNjJix/hlNK0DbilJjzzxkwYaR7EJKAqNo1Wr5wFFjQ8AaX2BQvBHGj iL6w== X-Gm-Message-State: AOJu0YzUBueam2R9S+PyctJQ2GnTmss1sY5dCA0FzIFzmRhoJw+CABto mhB9WZ823HgOx5uVvFUlEFxiHQ+wNetxw6mzIJ0= X-Received: by 2002:a17:902:f684:b0:1bf:205e:fe5d with SMTP id l4-20020a170902f68400b001bf205efe5dmr9682581plg.7.1692634739368; Mon, 21 Aug 2023 09:18:59 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.18.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:18:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 04/19] target/ppc: Use clmul_8* routines Date: Mon, 21 Aug 2023 09:18:38 -0700 Message-Id: <20230821161854.419893-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 834da80fe3..343874863a 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -26,6 +26,7 @@ #include "exec/helper-proto.h" #include "crypto/aes.h" #include "crypto/aes-round.h" +#include "crypto/clmul.h" #include "fpu/softfloat.h" #include "qapi/error.h" #include "qemu/guest-random.h" @@ -1425,6 +1426,18 @@ void helper_vbpermq(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) #undef VBPERMQ_INDEX #undef VBPERMQ_DW +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_8x4_even(aa, bb) ^ clmul_8x4_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1445,7 +1458,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumb, u8, u16, uint16_t) PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) From patchwork Mon Aug 21 16:18:39 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715393 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925953wrs; Mon, 21 Aug 2023 09:21:55 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGANLpgJYa2hEp8aOMhh+Ir2KY/bGDUho7OHC3nL+rSOGdZpHssJD3fwi6QvRqOWo6rYw0m X-Received: by 2002:ac8:7dcc:0:b0:400:797e:d694 with SMTP id c12-20020ac87dcc000000b00400797ed694mr10120968qte.11.1692634915171; Mon, 21 Aug 2023 09:21:55 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634915; cv=none; d=google.com; s=arc-20160816; b=EjumCNmjuHuA0EpqBBGUxhnEpYtFAHt17/cYNL4hPqEQKVFfFt2fhZ8IGNCrek514Z nQOuVjpk/nU0gpf4ic+MRTGaq1L0/ENHcBBDkr8djZLZmLu3Mawtch5vyScQw8DWc9pD eEw1svJG4b7hQ02yLVJKaoyEKu+TnaA1Ztw0WqZ7ZmzULOxmuoOUiut25iS03V6WHeYi RBtj4qh9mWkhRBCW005wakl2aP37Y4OT9YkPMOzBMr/buINGpUWu1pR/QlfEGdHER2FE O4dSTaudDOiaazXajZquI/yG2uauO58lIaQDvq9oQdtuzGTSuZifYQI2sQMVnCG2IRfJ DofA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=a5j7c2yLNNNI4JXLYpMR3lXWk/ubDO2wGYJjwp6C8ThPntNolx1XeOi/qQaZtZCgW9 L1dS+/Qv/WFcmbvKAEQal6fxD5fGWAey/DgQO861tLrYBuHXs1LfDMnxhhhHpOCA9aUo 39KpERCxNI3Q4I9uNC1082I2+o7iDE9V5lGdu4smCHxoVp/tx8NEsFryutikiZpQWhz9 R7DTMZ6OFSZAS3iNYj0AfyA3ElJASSHwPjUAISg2FuX8/SlkAtu1xM+LJuiPG87AW9Jw BQNqP/A+j5Ne30CrSqcXIBPYnkctc/3Bt3g5+Kq1G2wELz/K/ETIPWDsr2ivBgN3XjPy Z6cA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dkEsU7mr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r1-20020ac85c81000000b00403b5a14298si4965349qta.650.2023.08.21.09.21.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:55 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=dkEsU7mr; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cn-0001gi-OU; Mon, 21 Aug 2023 12:19:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cO-0001Tk-0Y for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:05 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cL-0005bi-No for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:03 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bd9b4f8e0eso21234125ad.1 for ; Mon, 21 Aug 2023 09:19:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634740; x=1693239540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; b=dkEsU7mrh/Aaf5+tRoyHr9crd4Tul2iahjevYFk1TwWQ7ZR8LCGoH4SCWkGWGTjnGH jmMLggHxSjg+EToygCn1SaP+FCpehfGneGD8NzcHRMZOAypOPBvTJELX3Tvolgw0hAAP 7nCYMfPfrJwFhrA1I2mKxOKyJXNpIVjen9Ixj2p2oa3Uk79H4UkTjstLz7vmj1BNCARH Eo9Tst97QGOVipf5f8S02p7RzBDLKsT4gtcfrEGDFCwWDLwIKxoJ2yQFlNmpZhqgOvJX 5rzoQu1Mms8tExFRzuZmjLuJBAakHV6P8ecOuT/6nytoj1I65UdQGnpUT5ELDHRApK4w 3M9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634740; x=1693239540; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; b=aLMmC5KFgky51w23e/l82lZFcp5gP0MarsBKztVdCX5JVjgVx6zyrNlW/OLDXdM5Xy HV1/Du1TC4KJi7gvSA2suhZMIouaKz1jrendonXtGrOEwGIt/vDsxd57krgkcbozkVo2 FKT12tg7jFuKwLNOu8Z0yUR85acCEGiN4ibhADE7OkCsMYvLvFdJ0tXk3oe/+ESL758T wJncWFBBfiPHpK62kIdVgzUNcpmW6CUyB5I8uOBLIT2hDi1VOS2O68aKRfYo/+TzV2cj NInZT14gENYfN9IoNB5Z+Jrm8L2SWVgCoqpPDXkIrdhBzFD2FFy/cIvtWB323wvhIRYJ 1mgg== X-Gm-Message-State: AOJu0YzAzmbsThkeUgw5jModwjRa336VKZ2qg2k/m2O4vv2rHc4BqZJZ 44oahZToSI9RSPcOV3dRTdMqpoKConT/Sje2Ub0= X-Received: by 2002:a17:902:d705:b0:1bb:1523:b311 with SMTP id w5-20020a170902d70500b001bb1523b311mr4298283ply.41.1692634740262; Mon, 21 Aug 2023 09:19:00 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.18.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:18:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 05/19] crypto: Add generic 16-bit carry-less multiply routines Date: Mon, 21 Aug 2023 09:18:39 -0700 Message-Id: <20230821161854.419893-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 16 ++++++++++++++++ crypto/clmul.c | 21 +++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 153b5e3057..c7ad28aa85 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -38,4 +38,20 @@ uint64_t clmul_8x4_odd(uint64_t, uint64_t); */ uint64_t clmul_8x4_packed(uint32_t, uint32_t); +/** + * clmul_16x2_even: + * + * Perform two 16x16->32 carry-less multiplies. + * The odd words of the inputs are ignored. + */ +uint64_t clmul_16x2_even(uint64_t, uint64_t); + +/** + * clmul_16x2_odd: + * + * Perform two 16x16->32 carry-less multiplies. + * The even bytes of the inputs are ignored. + */ +uint64_t clmul_16x2_odd(uint64_t, uint64_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 82d873fee5..2c87cfbf8a 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -58,3 +58,24 @@ uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) { return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); } + +uint64_t clmul_16x2_even(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + n &= 0x0000ffff0000ffffull; + m &= 0x0000ffff0000ffffull; + + for (int i = 0; i < 16; ++i) { + uint64_t mask = (n & 0x0000000100000001ull) * 0xffffffffull; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) +{ + return clmul_16x2_even(n >> 16, m >> 16); +} From patchwork Mon Aug 21 16:18:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715383 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924961wrs; Mon, 21 Aug 2023 09:19:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFNsebZzcweUsVmWXB4iYmjkC6IUn9UkGBTQMvMxTm/rVjdpBrCgF+4qkk73SZHmV/UzKfK X-Received: by 2002:ac8:7d84:0:b0:402:2e84:f06e with SMTP id c4-20020ac87d84000000b004022e84f06emr8114981qtd.27.1692634799416; Mon, 21 Aug 2023 09:19:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634799; cv=none; d=google.com; s=arc-20160816; b=t1VR9K5JsYfhakbHz/FUOiq+wSUUugn1iZm31b2sh/QbUbKX6COuWYvDEMmZkXs7EU cvI+VQYzV+td1c2h5se5mRgyEKbppPQhiBqF9MHB3rGDzJBUQdN3TUSatMsrPOS4wU7x hze899kKzjJbul9tHPKT0aPSbG2q9Wi1RFPmTftKZ+FlLFy7sskT+p1bOd0oj2GOUde/ AaiYVWKujWuUN0UUOh8BKMntfiuSnCmVuieeQfV+fAbriRfKiJ69iBjOud3pQgSs0DmB jzSzmN4cWX+RNWm/cDOxanwHK+BgxJJXdhAAL1D1NZV/b9qYtdUPYrbBCGXVuUvtZoTn x32Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=BviDyc27jMn9O0eiohzCnNX1lnooOa0e7qMrn4ol2nMiEo4lhn71K9zvqACTKfEIAi V4vvvtTU8tg9vbKHOddN/TJRCNZVC3iXQLKLBpgOk5mCr6VcPtCTqFHr/aB8rTOm+SnK Kdg1GxNpeL4HuY41xlFbItf5hUhEDxqCEAIa7KNHSJ2m33HLwwgMNnB+vHSPOw4CIxAQ 079kp3Vty1Rz2Ocs7rUJcZZMJ3qR3PmlZ5EYfqyJvIn/qy8B6FtS6fioUoxJfDI2T02K o44QmMadkLL2IU6cY4m1twyvkbfXbh4PaPH/Ai0d3qqStuypudXQ54mvlKRlp2PanQCd Spdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i9Pul3Ab; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id l18-20020a05622a175200b0041095900432si1950300qtk.776.2023.08.21.09.19.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=i9Pul3Ab; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cm-0001Wl-OC; Mon, 21 Aug 2023 12:19:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cP-0001Tm-6n for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:05 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cM-0005bw-JM for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:04 -0400 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1bdf4752c3cso18697195ad.2 for ; Mon, 21 Aug 2023 09:19:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634741; x=1693239541; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; b=i9Pul3Abu2Ih4oaJAslQeER23SCU1VmvZM7Y5D30Y6DYKiqjKAPycL898FqD6Yrswk WtTxnlqy/CQZnrUQvp+7ToVsCOBVPBUmBRayPEVrpxRkRPoXwSSM7ff4CNsp3L9Ylbh4 IErifGiIs4tfi1euqT+9QWZ622G5PWEIwsl0NcCn4mkpRD1bbxtL+UtfrwmH3fYL3xAZ EYDWmfQ0BzCz/pSoYzhWr8kgXp9xBuAbD8R+B07gR5UkV/L+xbulczPKWHSDeMFSUJFm GjwYJBzQ0RmVwYYArS3/liRh0H3I6CV2Jhwd+W0uCw/Cn2r3nUpZraZ2d9bvJ5ZNHx6l mQ8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634741; x=1693239541; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iPO8TcwjCQpKboJkMUgpUg7QTAvab+8lUqTEXa4XMVA=; b=CqEiOareqFntWce0MDqCF5MphP8y2qdnfTKoRUqiIeELsxc21JvUaI5Vf6HK4YxAzu KFIYkZCq3xUgtlpVpKhPiSk7bjD9gnx8lP1nJ8o3AxNeq22kn9+wZi8gvylOvQOkyu1S Tr8oTQyrVpQfFJ5+fnBh2j8xBv4FATBq2LI3NLbhawDU+FZTSFyeUjVlupfjcnQF5ga8 5vNln62EiJlnxA0kc0+TwHPlXfRoNXDQgeqQxMn+ehFp8P58QeaC9+ihLX5FGWtI/vpg 2rljaJ2YwBRij6kWnTNUWQxaBWxyCD+JyjjWhyA1I5vsFzsX1gZD+/eXTSaPAPxVMnSg nkQA== X-Gm-Message-State: AOJu0Yw4cG/0Q2w0vqsvsosrTZLvqbAgPj1Sc2sQFwR2rDyrzR4M0qvG tbldTyk/ZmDthuqUhniB1zrF6u77hkFeuA3pUAU= X-Received: by 2002:a17:903:11cc:b0:1b9:da8b:4eac with SMTP id q12-20020a17090311cc00b001b9da8b4eacmr4743575plh.56.1692634741170; Mon, 21 Aug 2023 09:19:01 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 06/19] target/arm: Use clmul_16* routines Date: Mon, 21 Aug 2023 09:18:40 -0700 Message-Id: <20230821161854.419893-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of pmull_w. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_internal.h | 6 ------ target/arm/tcg/mve_helper.c | 8 ++------ target/arm/tcg/vec_helper.c | 13 ------------- 3 files changed, 2 insertions(+), 25 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index c4afba6d9f..3ca1b94ccf 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,12 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 16 x 16 -> 32 vector polynomial multiply where the inputs are - * in the low 16 bits of each 32-bit element - */ -uint64_t pmull_w(uint64_t op1, uint64_t op2); - /** * bfdotadd: * @sum: addend diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 96ddfb4b3a..c666a96ba1 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -985,14 +985,10 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) -#define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) - DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) -DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) -DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) +DO_2OP(vmullpbw, 8, uint64_t, clmul_16x2_even) +DO_2OP(vmullptw, 8, uint64_t, clmul_16x2_odd) /* * Because the computation type is at least twice as large as required, diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index cd630ff905..5def86b573 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2029,19 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -uint64_t pmull_w(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 16; ++i) { - uint64_t mask = (op1 & 0x0000000100000001ull) * 0xffffffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); From patchwork Mon Aug 21 16:18:41 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715389 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925780wrs; Mon, 21 Aug 2023 09:21:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEmlXhJqBiF1UQqfWYs0sH9u/SGxrKxiXzKd6xSkXG6QhjXc5suYO00d/XIAnhdJkknkxgS X-Received: by 2002:a0c:df89:0:b0:64a:6ae6:46f6 with SMTP id w9-20020a0cdf89000000b0064a6ae646f6mr7549959qvl.2.1692634894607; Mon, 21 Aug 2023 09:21:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634894; cv=none; d=google.com; s=arc-20160816; b=bGQWml/i4pi7ppJDiApj/UNDuvpUiMg62X3oR1MaCgyPcs+MCEn38Sl9l90UPCh+Mw bnYUhFWHKuysbdfPAA/XeiBYsvjjypZp0QAAsK4W9/1BG4VVSipTnRB61FoEBTYr/3+H Fs6U8/eOdjUl9aXhOVgc+BA2EAN0BLh0u2ZyqRjc+nr+idZ7MPsed6v3c2H68r7ZGy0i 33HLKun20qhd6i1EhWDqe4wZIqk6DiuaTHcB5Yh6DS3LBYfnkoMUdjCw2E27C/NVeiz7 qT78j8exebqg0KzJovBbgUea5XBajV32LxQuL1/Y52n7FhKHOCqRA3paz1JHjZsPtIaB fnzg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=dbSTojQMp70UIjYG+2obz3ykmb1poYGzWECyjsWPNMlZil1oACI5n6lqWHBpQfJUcH zTQ2GJmSXHZoNzGp17gEmsNvUVDKozc4YxAv+1+pIaGnS+1LyMKNS8D9HZb4IfdjN4JP u9LISAwRJqWFvQ0H1MN1SB2XAUI8PcfvXy1fyOuu/sEqUXAcJ6HqezvOSjfpBJ8tG0LK rWf+ivKanWwWWM2KfU+05mLR02Ps+LDvU5fZM92cK55CkBEgZrvenNOmQyS4/8pSRLWv z2Ed9Pm+a3yT2mD5ubHos1cVGgs6dOfpjwMoQRrGcwj/q3NNQqKENdVpkkhutEP3HHO6 /snA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CesaKTvz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f7-20020a0ccc87000000b0064718f836f6si59372qvl.552.2023.08.21.09.21.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=CesaKTvz; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cp-0001jf-0q; Mon, 21 Aug 2023 12:19:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cP-0001UE-Sn for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:05 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cN-0005cD-I9 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:05 -0400 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1bd9b4f8e0eso21234505ad.1 for ; Mon, 21 Aug 2023 09:19:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634742; x=1693239542; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; b=CesaKTvzUC/8eEwCu7nznCGFFyf6iaJqRk9V4r0aj+u3K4iXLHf/6ElY1y+HSkmauw sc68m2ZEBzg/bflvu14xEDyyn+kEW5ufb3l+xb4uv0ajwgnr7zoaQ6qZhyNQfi6I7X4b jOy4D8/EzpWK71LXLfNXg8RfXu4mjPusFYvtMimHMPdLYgN+Oxq3QMhBQY7glX6T45+e 81VEgkhCGpZ7KtHdqE+cr/RLIgLQaCLhm/eA96G/1nL3ut7f2AjnD0oKNSQhNpdcamGQ rfNUvCwqScoj18MuoZJrQanItTgLAeOu3K4LXwQlsuJ+jAXEPOmiHHqV0TScipe91MX1 7RpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634742; x=1693239542; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rjJiUhlPai6mmk7xoBYCGw+/59480gYFwt2JNZw/Qk4=; b=cZ+rABt12FDHAlEJcE9fM8b1MsEZA3jVpPGTBCAgeH134SzZEdUz1Xr/RysCjwB6RC NidgbDbRCjIOnNZ/6jmIfYkqlOwjmULg4M64dLr5jKWrZTP3VL3WlMrpl+KxR2Aqkq6t GNNs8MnlLqfbV5p1bIXOaDv8nqPOoI70aXiWB65GGH+TARolQ+5DKE6kOLPPQSQD/dQi lLNcrOxKozvqU3GLcTvBc/lrxVkTMLmuvnOtF8qy7j9/58geguYmmxtU3FzpntMM0d+Y cKrwoyEnaoBVtRCS7XrOEId7Z1cRucAl+eBsGp3L2SKOWdpBkWznDJcKtP7jlb5G76yJ bXkQ== X-Gm-Message-State: AOJu0YxAFGDlBDCWrlThPZGxE4epg0ypyLw4KQfeQ9aLgpvUK1Bm2X/7 /RJkSVmDCJoekfS3JAM5zfZyk+x4GOYUPAsonbI= X-Received: by 2002:a17:902:dac2:b0:1bb:99d3:6a53 with SMTP id q2-20020a170902dac200b001bb99d36a53mr5721262plx.20.1692634742149; Mon, 21 Aug 2023 09:19:02 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 07/19] target/s390x: Use clmul_16* routines Date: Mon, 21 Aug 2023 09:18:41 -0700 Message-Id: <20230821161854.419893-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of galois_multiply16. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index edff4d6b2b..11477556e5 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -180,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) @@ -231,6 +230,30 @@ void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma8(q2[1], q3[1], q4[1]); } +static inline uint64_t do_gfma16(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_16x2_even(n, m) ^ clmul_16x2_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm16)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma16(q2[0], q3[0], 0); + q1[1] = do_gfma16(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma16(q2[0], q3[0], q4[0]); + q1[1] = do_gfma16(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -248,7 +271,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(16, 32) DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, @@ -284,7 +306,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, From patchwork Mon Aug 21 16:18:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715385 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924992wrs; Mon, 21 Aug 2023 09:20:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFaDjvzQeeVlsVlFdgjoau00FmVMFpAu3j/2y4vyih2w4OJi81ycG0iO1yDMqf/wOj9E/dl X-Received: by 2002:ac8:5e4f:0:b0:410:8bcc:fbac with SMTP id i15-20020ac85e4f000000b004108bccfbacmr9329032qtx.7.1692634803452; Mon, 21 Aug 2023 09:20:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634803; cv=none; d=google.com; s=arc-20160816; b=e/OOdXTGGjRTCCNa4+MVcssmfwbiNQt9b0wHgQ+R0UNnwdDpLSzeJIZKCsdeQpK+4z 3yPbQSvT3xVcIZCjfdxezJ0sI6ckntE0Px4kHFyylNMWP2subiN8VlKssGVvnuplUfEP MtjBfc08rC4ImGVaTMziF5jEOC9dCQk0Q5F53bpoil1Wp2suqYvbHvU4H6T2X692us6P k7lRQUrIZskRWzPDp9juwOrm4Nk1LG2qOj5eMHzKARxcAAZXrMJi4/ckvI/2/oYW6q1v MyahqGzB5BGO/JX1cCcNDNM5E6TxoLZNzgLpOkjc3J/vPcM5z2hndqoqv+jUI4wrQwe5 UPbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=nzTF7MYMD0Pada5eVtLxZ2WzlZg2ZgP74dMNYp90YyF6qT74fNtZGCSUY9717T0BeW aJl/60t0fSe6efwmvgwFa+jDTUjC4t+LFJaQ6BZbsPkTA4xjjkT+zI2OYHmX3WKI2z5w giZSdDAqdtZOQqs+7cWAfeWjXfzy6v0P8OXGouOjVSaQg9ea/8f9REeTafgDfCMFAFXO fiBldZBEdPIVzqG/WE4B15WtC+d9TzZZJz3mo0btViTylhvKMGKi8zHry+17jPWv1Et9 iBRPwizyukzLz/54QWXSAzALEayfYBBnTnBJTilENRWQEdZpbMHYj9QJSXusn84xe1Zn baeQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ehAokeZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o19-20020ac87c53000000b0040fd591f651si693034qtv.523.2023.08.21.09.20.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:20:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ehAokeZt; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7co-0001jD-UR; Mon, 21 Aug 2023 12:19:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cQ-0001UY-W5 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:07 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cO-0005cP-HP for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:06 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1bdc243d62bso20707575ad.3 for ; Mon, 21 Aug 2023 09:19:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634743; x=1693239543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; b=ehAokeZtPiM/VMZkTTjopbJALWIlTO1Pt+Dh8O0CIM8jfuFAZ+04Rz/tViQkx8AeaI 7mIB3hQ/78SKdfz9r5gwZQU8fFS4UELHCGjEzViQVO/tdqYO5PIU28SMWn2VbeQoU+Cb X8B/NLvCEnOuEvmiy7CjYv1hlzgiBlbPPKie2Rl38fD3Sb81uhA/KlLigoTAW6db8lNI H0HhXRUNmLzxOBoSx5riqaF96uMQLeERpaBJUu3GQ7Y8Qdyn+rlwjItYOIQtlqqowl+R DgXZYIJBRqVvUm3zaPR0ehZxoLQmW9kzXC0G4HaIETAqbBosfLDKSyZzUj5GzjcTU7E3 q91w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634743; x=1693239543; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; b=ikV8/yOhf+g11TUPMuNIEwrjXsgqSfZN+1+iDWJD2A45FW+BAarEZ2eqsNQOlRdhlS adg/SxGAUC8uNNOJxY8i/Di7ANsdCNWDtz/ZlcSWiP8ZDeXC8F6xICovbE8SUzAbU1NX H4IvI6Z6jH7W4MEqKXA2ssm/H7jSUiO18KHa+o04PylOMFOWgkv8fNNXBr65HGIkte1j CrWWLZG9M0GN1PhUMTzcZC60avCuAAIKf6QRrpeXrKwABGlhX2c2fp/Vsw6EBUWgcNii E7TSQbkGiJGUeqdA0id4k5B7z+Ba9l/H/z8C2p9OmeufnBOEN9je3SMolRC1wm0AeXgM +qyw== X-Gm-Message-State: AOJu0YxeIGF0By0yp6zA6uMuqpAbRKDPOyidfiLawnEwycIklHRAzrhu r3btuq74HzF07jB0Mc55FIQJQhd7tbBSTDtgH5M= X-Received: by 2002:a17:903:491:b0:1bd:ccee:8f26 with SMTP id jj17-20020a170903049100b001bdccee8f26mr4099677plb.15.1692634743161; Mon, 21 Aug 2023 09:19:03 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 08/19] target/ppc: Use clmul_16* routines Date: Mon, 21 Aug 2023 09:18:42 -0700 Message-Id: <20230821161854.419893-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 343874863a..10e19d8c9b 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1438,6 +1438,14 @@ void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } +void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_16x2_even(aa, bb) ^ clmul_16x2_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1458,7 +1466,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) From patchwork Mon Aug 21 16:18:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715386 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925224wrs; Mon, 21 Aug 2023 09:20:33 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE6stSkfRzjvY5acJYA6R1UcaJuY+QimpliAlSWQfj82uTReieeyFArfXxlAYZklZs88isE X-Received: by 2002:a81:4e4e:0:b0:589:e742:8cd8 with SMTP id c75-20020a814e4e000000b00589e7428cd8mr8703803ywb.40.1692634832875; Mon, 21 Aug 2023 09:20:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634832; cv=none; d=google.com; s=arc-20160816; b=QDzuxgzBZEuY7UCZZPa9wKqjPg69/Y7XgZoQrwsrcJHiuoV78udv6X2CzW0iyY+s3r KnYUb1rSprDSg9IdtyOuewL17DPEw77kEh48VJIsD0/WCq2TCPbHx0Gh3lP3HOEAeMMq Q2DMnnD1iMC7DpUwwmUI6yzg56ukq+qIs6m5mY7n3uuMn5dA4g4Vh6e6b2pcVkOmsH+5 1J9cqANIxptl3Yva6gWdiQGwLtUTeojTIk1ySKnj176ubqWVlBUFBe24/Z9gfcxO3Ysc 5mwIQSggC/iKJlxU425vibYZl1zSlFdZ9M1bnAP1Q0lVwl8zClCEFiIkjuLiTqmrgDp9 aVUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=z50jJjJ6WSDnrBdIqwjY8jrYhzjeLeB8KcmZsMaKcTnNkgw05btAfHQYehgUWHreSU S3wCmKlKiLqzz7NCtiP2Q3vaT0RIRSwnY0P+gL00RrWGvHA+hIuxxaY0mwvZIydS97kw hJDdZivBQkhLLhm7CkjmY5on3M3ESMF1+OQ3j8cuwPgxqkEU6uXUW746/lgF96QrLIch Vjo+RWi4BovIbLASSFk+xlibhnPMz9EBYzpk/5EP7wAzVgNO+vtqtfYht4P1YLivlF7l oKYE1AswcBLOW31RNl4heKbRh10vqjWakXzTgXihHwIfCalqHFWnUBEPxuXZ12hgHee8 VNbg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d1omPxXO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 6-20020ac84e86000000b0040349cff069si5121868qtp.748.2023.08.21.09.20.32 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:20:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=d1omPxXO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cn-0001hG-Ur; Mon, 21 Aug 2023 12:19:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cR-0001Uo-7N for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:07 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cP-0005cf-4U for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:06 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1bf3a2f4528so25023195ad.2 for ; Mon, 21 Aug 2023 09:19:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634744; x=1693239544; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; b=d1omPxXOeye7RcxrOdZL+G8JdUouonOh3dnmZqblNK337xdPX9R2FMrMh+PlLAkpxB KKmfGJ8D7cHVZyawOJhy1Y7JLufT3JmFHoFYVpiyy7ziGbMcA4+GJ+5ES2onv1pdTr5Z 2kpbtQ2/7UynNhjuLgZYPLh3XoeiGgyAhhFtB8cUMvba/AhkMe9ml3gk/KNiVSO+hQih dkwHTQYkldkRIoVpRrdc33H2SdGX1z+5qt5jS87CjDl6/GpG7dgWw3frrmAZCxGZnHQt B6s9OaUJjDyJq/ePIBkLaat4pCZtl4cypW1Uo/BZlHDKK8C7I6wqsNoCzt88U7JoBypx JcaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634744; x=1693239544; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; b=NJ/EmyStH1QBrSEed6t+r+M2EgiiZPFxQmlphXFRgSuumIFIfLhCwg6tqZkV/jASUO j2S1D4W4UTNKYopjnoAcNmyyURuzRYnp0/C95Sfbsxa6tF1HtiX34cOD0AhnlmZRF/5+ ZwAXMgz5Jba2cj3xlTp1eRlfvGOL1EAQ2g0J3FspR0JYvgLxAqOhkoIHpggQxW31Fasc wFQC+I2q26Ym60C76T0N0gcbfpdJtYLZj2O3RNZ6T7k6/vwejr8FW/Rzew2ya2+YZUlM KOUUOctS7MdkWSaAVaKegfiGAvQNjZQ1I8jZyxiSeRUP9h+1dOu5Nvc2aPi0FIQ2jo8J o1JQ== X-Gm-Message-State: AOJu0Yz8xl9oYacfncMsZ7OQkRvozXEugeY32CHqt4XEbWHl8GAk6hmU f9wey/f5cl0S4T8vdLrXM4Ol3ukWie5N2QwUm3w= X-Received: by 2002:a17:902:fe83:b0:1b8:76ce:9d91 with SMTP id x3-20020a170902fe8300b001b876ce9d91mr3236110plm.1.1692634743951; Mon, 21 Aug 2023 09:19:03 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 09/19] crypto: Add generic 32-bit carry-less multiply routines Date: Mon, 21 Aug 2023 09:18:43 -0700 Message-Id: <20230821161854.419893-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 7 +++++++ crypto/clmul.c | 13 +++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index c7ad28aa85..0ea25a252c 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -54,4 +54,11 @@ uint64_t clmul_16x2_even(uint64_t, uint64_t); */ uint64_t clmul_16x2_odd(uint64_t, uint64_t); +/** + * clmul_32: + * + * Perform a 32x32->64 carry-less multiply. + */ +uint64_t clmul_32(uint32_t, uint32_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 2c87cfbf8a..36ada1be9d 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -79,3 +79,16 @@ uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) { return clmul_16x2_even(n >> 16, m >> 16); } + +uint64_t clmul_32(uint32_t n, uint32_t m32) +{ + uint64_t r = 0; + uint64_t m = m32; + + for (int i = 0; i < 32; ++i) { + r ^= n & 1 ? m : 0; + n >>= 1; + m <<= 1; + } + return r; +} From patchwork Mon Aug 21 16:18:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715395 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1926859wrs; Mon, 21 Aug 2023 09:23:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEukW1Dzo9xKw3geBpDx8MZhej1ivmeBH6km/1Eyf+A94n1qT2TWiMMyUoiq04tV58NWgAb X-Received: by 2002:a0c:db83:0:b0:647:4060:1074 with SMTP id m3-20020a0cdb83000000b0064740601074mr9749202qvk.22.1692635038551; Mon, 21 Aug 2023 09:23:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692635038; cv=none; d=google.com; s=arc-20160816; b=Jzm2NOL+4zKJc/uP9mH3V0V+QhOZ6iu4NUh2oJVAwz0um7vuldQBAwJEkGt2/HiybC BUDjcoo1XuLZ2Q75UgH1gn69zufoVNJ7nW8NybmPhjJFlv73GrKfaIi33Pj0G8PPpt23 g2FOqdoxcMMf17Bx7++0eJc+tUgcOpkiDM+F6UxvtghFGalhceedcyXHGaZ8dnDItDvq P3E5N2jCYapj8FiEZFHVRpiOuYtgvuuM4rvWxcay07r1T3c0zhEZeOGnSeewjIzOUuX7 CUSI1+RY4yAB5ZnB0kSBVZZ7vhD2JzWle02+QlwN2bhPXA/n6C5ii/ACDxS/wU1q+PYY N+XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=TCQhpXw7gV268pM1Ff5dkR2fhUtnuoyESLLuw6pqQ/p92r3NkaFQAS1KwN/0LQUsXa /Y37RCpjHy67pwwyogjWWw3dZxjCtxClQPbZTZ4CMkw514Di6bEuqrrpgDYJ9iphX9TU UF/4GGS0LW2aOKjWZDOow/X2dSxdS0cp0mioSWcfqEJATLFwO9CT8BOKgF6ysd+zPmPu rz2Psk73EQkkhpselzppXYKiIFAdk6TP8XVE1ObyenyzhceHP2H85B0YUayxu16ebRsW MD7++iIEKf3TkwH7H8YJSUATfWullql8syrwa8w2A40ToR+pgKAl9I8GMrJJPnn6QL9Y Kynw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JPwA7U1l; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z2-20020a0cf002000000b0064727571e8esi5463698qvk.172.2023.08.21.09.23.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:23:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=JPwA7U1l; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cq-0001mt-1Y; Mon, 21 Aug 2023 12:19:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cS-0001VG-QD for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cQ-0005cq-9L for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:08 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bc83a96067so18879675ad.0 for ; Mon, 21 Aug 2023 09:19:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634745; x=1693239545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; b=JPwA7U1lnq87ZJz4GLDZpKAqSmYvQKmBOxmGJfYb41Ex6ML8pS7y2f2qi17zKQH6zL UjopOsqj7tHnjM7yJICIO7FYxxGiuO52iYVuti+sRki0btgcfP9dcZx1pxm6pqy7pWkp yYXOvL0rEvU2+2WI0gV1uEQqviOfrTAp5S1+f4ttHYiqy28IYwckk5Qh0gKpGlh5wHu8 2tJHpVlDAPzSBm+5MFm/qzX3AZPdlNTS8fZytXrn5orMww8kDdOBrPmHMm9PbbQgnlEX D1+aSXQeP024mlEEIwojO6vDo7mJ8XFdzEzQCAzA/epb/9XeCiJs4kflp/AEnFctL/ye ISdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634745; x=1693239545; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cZrc3DuTynbE6ZobmlduknjscYzm5QXC9S5Fbkh1+j8=; b=kMxZEsJcFuM2IzCDaPwnwUTubuUXER2gp4PFJDa/hqCpNctE0OoroW+CVWCU3JpdxI YE7uR1zBwIDOS/YdXECbQmHCyYYjUtwnld/O1pv59DAD2kQFZ65TLcONYbip02s8mC4T 9RPNCiaWWUR3oSOjUa0l/jXQgpt5wdlfHtdpQpQ/C+W1DzcsIRuc8JD7nHZsoANjkBcT 7GSnahye5s/WTyvyYGoer8QRkvZDuXmWxE99NRUyg37fmnRXyeSU7OJiR8CXH9y9kNWF SAw8sV/Uj9VP/u3tWYfWQuf4Zujc1gR7s/8xqidNVc7sMXhHcZlOghLle2ZSJzNBPJL3 0rmg== X-Gm-Message-State: AOJu0Yzbg31cIeg8EUmrAR5hDvaaeweYZLNza3sfBH5SI0lpVpnUN/gN lX4gPUIvrMu9Hcc1K1d/X5+g7jyQVzkkyfZxK7Q= X-Received: by 2002:a17:902:ce8e:b0:1b8:a389:4410 with SMTP id f14-20020a170902ce8e00b001b8a3894410mr6265584plg.0.1692634744826; Mon, 21 Aug 2023 09:19:04 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 10/19] target/arm: Use clmul_32* routines Date: Mon, 21 Aug 2023 09:18:44 -0700 Message-Id: <20230821161854.419893-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of pmull_d. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_helper.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 5def86b573..ffb4b44ce4 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2055,18 +2055,6 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) } } -static uint64_t pmull_d(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - - for (i = 0; i < 32; ++i) { - uint64_t mask = -((op1 >> i) & 1); - result ^= (op2 << i) & mask; - } - return result; -} - void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t sel = H4(simd_data(desc)); @@ -2075,7 +2063,7 @@ void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd; for (i = 0; i < opr_sz / 8; ++i) { - d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + d[i] = clmul_32(n[2 * i + sel], m[2 * i + sel]); } } #endif From patchwork Mon Aug 21 16:18:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715387 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925233wrs; Mon, 21 Aug 2023 09:20:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHmYdo4i9lXc4rWiK3JWA7KqffLVsPj+znKsWIL+faVOXdw0NBC7hahoCRdj5ELHtkmqqsP X-Received: by 2002:a0c:e084:0:b0:64f:3e27:1e4a with SMTP id l4-20020a0ce084000000b0064f3e271e4amr4000384qvk.19.1692634833857; Mon, 21 Aug 2023 09:20:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634833; cv=none; d=google.com; s=arc-20160816; b=i09ot+BmoQcI0vET09IK6PxhRtT5seopBa5UeTaH/uUAB6AEb28h1aD8/qy9h7a1RV sGx1D04fiIRdeDJQhs+z4Ksuld2bh3rbhjJOmu4ew0Tjl0XlfxESHRg2RracMtH4LEQb LtTkEl/viqsRU+02kuZPhsn2/cxe07VSIiBVis+vaZh13WbiIjngBB3qhehhYmqxsMsT BZSSNO1Epynp/Ft1OuhqzxI/YsahFlBtjwbM1H4GY5QQCZNgPlEM8YPSlDVYZgQzf22W H+e8lmPURl/IYvWN6T+j0m5c53QXwbSCore+wHkfIo4JeWisXx6dF/uyKkSJ9kpGRiUb KXPQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=h7q5Sh7UvOuvGVbkfEmLCCine5lONOrG+RivxSq2+JSUv4LjRP/UBlfabYL7inY0Dp 0BegLWgXqV9XECMjb31LS1pdZzShtXzaLu1Ov+/KWXczA7b7o9ja3Zh9WaS6HmzQBZq3 lkxb0pCwhAuSQBeONhtYj3tVJMJtDIllpM1+hVL4f4QrAMv3mjN8TK+QugchOU2a19sa p0y1meqVVDxZh7S8p8Xy/cSQ/IOwaH1ewtDVDFkLfxyLgknJayo89FMgfCgkdsk2iGnw RtOaLNjEs1tUCfL4Lci6PGpK7lS2rbqNqdDLyjKgNQc6XJN779j0URqwPmitYDaLfcLy zGHA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QaUHdhM5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id c12-20020a0ce14c000000b0064f3bee93b8si2026312qvl.248.2023.08.21.09.20.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:20:33 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=QaUHdhM5; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cn-0001h2-Ru; Mon, 21 Aug 2023 12:19:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cT-0001VH-FL for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cR-0005dA-4f for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:09 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bf6ea270b2so8275155ad.0 for ; Mon, 21 Aug 2023 09:19:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634746; x=1693239546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; b=QaUHdhM5BX0BweXF2swyVuViQLyHVeetSLreVhr/8erup89KfAGq70Vl+ItSFCYI4M VYjOe5oVwsbCoQIsaIOrY2u0kx0g5bsvompIcAsGGN1w77GWOXvxghX+lhDWsqvlQTQp QwO1hZvlqp5ZUCDADxzukS8/2JrxG6xpuSdaVQLyY5ds1pvMBz2WLUBsJEwCzKQW4ul5 zBLb8GsX9jTFgKAfW54fHqulOao8atMk0mNbxtoyueSYGTsJq2OUBoq1Xl6rfm75s0Yw 9b0bNboeNuxhrB/MyaSd1cl+ssyc8aCd2uLY/1fMDKHasx+vMxtnH0tetY/0q0KC360M qAUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634746; x=1693239546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W1FQxefxQxkmUZ1SixwcnXZvZP/IO9ZUpCwPt+2NITw=; b=GO9jZF96Eu2TncxL6MB/8XW949Qs9ViEmnvW76k1JdfGlyQL3h9MMhBQ+zJEanskWT 5sSbuLl8R32NbEHP+7ZzBFDEkUzEQyS+fGltZQwWy7jppz5oBg7EhrkGpWzFvkl/XDx3 eOeR/HyrWOeV/Rw0SozKAQdMytlqJu9Z9SnJyBt7t2XZpqpljajXYQpZpbiBpmXw8Zgd 80bVgBrzK1izUYNjvsknGp0ro2u+UI6odmvg2EXZXHfzBoPGUrXIDcBVERvqBGcoQwa3 AIeiYvNq12qFgkNRy3xTOw0dJvi1mzoA+8G4HRkHQfgYtmSWwxKT/F/OCxwXnG6TKOir 3SnQ== X-Gm-Message-State: AOJu0Yw+Rko806mBHI7LSey5wwjFNd3YQaycxxSkWDS0lP0ACpVZpUuU IJXHhlDjqSSMnVtgt9ONwSeN9cDB16cWQkW1EMY= X-Received: by 2002:a17:902:c101:b0:1bf:6cbc:6ead with SMTP id 1-20020a170902c10100b001bf6cbc6eadmr3504244pli.22.1692634745828; Mon, 21 Aug 2023 09:19:05 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 11/19] target/s390x: Use clmul_32* routines Date: Mon, 21 Aug 2023 09:18:45 -0700 Message-Id: <20230821161854.419893-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of galois_multiply32. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 75 +++++++++---------------------- 1 file changed, 22 insertions(+), 53 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 11477556e5..ba284b5379 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -165,22 +165,6 @@ DEF_VCTZ(8) DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -#define DEF_GALOIS_MULTIPLY(BITS, TBITS) \ -static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ - uint##TBITS##_t b) \ -{ \ - uint##TBITS##_t res = 0; \ - \ - while (b) { \ - if (b & 0x1) { \ - res = res ^ a; \ - } \ - a = a << 1; \ - b = b >> 1; \ - } \ - return res; \ -} -DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) { @@ -254,24 +238,29 @@ void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma16(q2[1], q3[1], q4[1]); } -#define DEF_VGFM(BITS, TBITS) \ -void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ - uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ +static inline uint64_t do_gfma32(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_32(n, m) ^ clmul_32(n >> 32, m >> 32) ^ a; +} + +void HELPER(gvec_vgfm32)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma32(q2[0], q3[0], 0); + q1[1] = do_gfma32(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma32(q2[0], q3[0], q4[0]); + q1[1] = do_gfma32(q2[1], q3[1], q4[1]); } -DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) @@ -288,26 +277,6 @@ void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, s390_vec_xor(v1, &tmp1, &tmp2); } -#define DEF_VGFMA(BITS, TBITS) \ -void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ - const void *v4, uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - d = d ^ s390_vec_read_element##TBITS(v4, i); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ -} -DEF_VGFMA(32, 64) - void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { From patchwork Mon Aug 21 16:18:46 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715381 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924939wrs; Mon, 21 Aug 2023 09:19:57 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFP2AJ1neqrIhu+Z0fMm8CXp3MisH80Sj1/Og2d6jxSBGkR1L8jdsOuHQALt3jN/3UiIZgP X-Received: by 2002:a0c:eb4c:0:b0:641:8df1:79e2 with SMTP id c12-20020a0ceb4c000000b006418df179e2mr6833729qvq.30.1692634797301; Mon, 21 Aug 2023 09:19:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634797; cv=none; d=google.com; s=arc-20160816; b=sqShE3oru4NXAfMBrgRnIe6FqDp1orq9FBnZampNKdk5RYDU4G7t5Xscekk1f0a0kw AwKjqi8B15Om56t/BUbWDmL/WzwyndMlkZdSiMj/lib0X61o5LU5qFHciHeCWRWe9QTs jj0lla4xcm2ikQ0JFVhZsRXHIsnuovhkQY+d/FCueQzl7cU/gQ2C5fIirj3jHyCuq2Xh BFK/jDURFLd9Y91cevOa2UOChorwHgcei4ucywB9YRp0o+ZUIMpZF5GlYIN1fvpjayRW vkdPiDzCOefaQovU2DF6KBkZloLPcXTQNCYzFA4Q+VTuHv1aVj3KUDrofPSUX0xBePF4 t5ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=Vy8EyaCTIOonIwxHP+2BSW5TmpYtPaDNTMhdxwUovSkiYey7FxW3Yuj6u1kjbGCNDm qDB3UnZ8rVzaSTqAqIJPV7O0BVRFG0pJ0+iIql03PZMki2QxSRUXGhSnQow62czJUf6S 4oeYuhnB3OyZx5eR0AwEZWdBE8KZHEwB0oYbgEaTrEmMrMDH0k8wFpyhsze2dlSMq6Rs W2oD8x68+VTH2f5Pk2xSShv0Jf+uUScQodY6iA+uJvFtD+FtXQYonfZwvyZJVf545UmP 1z1dC6JI7JsLHrS+5qIIwtg7Rdn/Awmd/z9I1/njQXGOXino8fJ880kxZZ7YAIbXiHYQ DKxQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nIeynuDE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k19-20020a05621414f300b006430c810209si5013039qvw.108.2023.08.21.09.19.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nIeynuDE; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cs-0001vb-So; Mon, 21 Aug 2023 12:19:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cZ-0001Va-5J for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cS-0005dV-A1 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:10 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-68a42d06d02so771010b3a.0 for ; Mon, 21 Aug 2023 09:19:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634746; x=1693239546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; b=nIeynuDEs0kpNs/miJbGhjMavgYqA1EgL+mcu36Sph9T0LJ1tODu7QwEHribXt06Wg ArEkEegk8s3VW3bas8d3962RTcT+T+7kJmxdxG/E2DEtc8yXG+p3VPKP86ePtZnT/FxK DEtZEBZhfsEqnnqZv2742ldxGazUQfeGYZHI7UpUkp1D4nU+iXQsZKEe4MZIIAGWxl5R ujDWwazdtJ54ci460Q89zf1pqu8JEm0dRt86LPf0/nYX71kUXZa09V9HzVklfnfVYhrt TLqzVMIQmIPtQiKRnZ+kblPbM7oVwmFgFtRjLloNPfXYBu+ZsGmJykT3O/sw1Ov+DdRg 8kDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634746; x=1693239546; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; b=VhbSNrrlRVCShl2pocCjPZaDyg5oidMvDhg0GjYsGKLdgzQk5Ep4EXiB0ghPu8e9xR cGeFqvwP54JwVl3YrWCJb9ercuZcCW2L85SEfBMatQdCtgw5VCtOEDXNT02G+E8K8Bph OcaMUqI7QiLmdDW0w3/DxT5dO30X0RozP2Bv2ZOf6ufLe7O06dRedH7UgStmzFZdIw3s da7LIFcDCXS3U9CZ3LqJkbyyOdbCwuG4HdthMXpIxQ1k1EjBlXJMgNhwDHSwepRpcd/R qQ8S3DCO6FTQEx1cm21GcT969GXHzESQBLKxucP6GWDOwBfPFZOjHM0u7DtFE2osxlCx OiEQ== X-Gm-Message-State: AOJu0YyNHrgkzN20rfwYPWbu9uYD0k6fYl02hLg/T3igDtXZXGogwYkc eqYN5sCXdBR7iXMQ/A8AmWgL04O6QKS68A9571E= X-Received: by 2002:a05:6a20:dd9f:b0:137:9622:17b7 with SMTP id kw31-20020a056a20dd9f00b00137962217b7mr5003877pzb.25.1692634746644; Mon, 21 Aug 2023 09:19:06 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 12/19] target/ppc: Use clmul_32* routines Date: Mon, 21 Aug 2023 09:18:46 -0700 Message-Id: <20230821161854.419893-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 10e19d8c9b..ce793cf163 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1446,28 +1446,14 @@ void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } -#define PMSUM(name, srcfld, trgfld, trgtyp) \ -void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ -{ \ - int i, j; \ - trgtyp prod[sizeof(ppc_avr_t) / sizeof(a->srcfld[0])]; \ - \ - VECTOR_FOR_INORDER_I(i, srcfld) { \ - prod[i] = 0; \ - for (j = 0; j < sizeof(a->srcfld[0]) * 8; j++) { \ - if (a->srcfld[i] & (1ull << j)) { \ - prod[i] ^= ((trgtyp)b->srcfld[i] << j); \ - } \ - } \ - } \ - \ - VECTOR_FOR_INORDER_I(i, trgfld) { \ - r->trgfld[i] = prod[2 * i] ^ prod[2 * i + 1]; \ - } \ +void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_32(aa, bb) ^ clmul_32(aa >> 32, bb >> 32); + } } -PMSUM(vpmsumw, u32, u64, uint64_t) - void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { int i, j; From patchwork Mon Aug 21 16:18:47 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715392 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925932wrs; Mon, 21 Aug 2023 09:21:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IF2jg9KnVEU6IMVFjAa4/bz8p5pIG/hYXoZC9FCyKo4uAtNvN+gcve9So6bQb0PgTycuiRU X-Received: by 2002:a0c:b699:0:b0:646:1c12:a7a2 with SMTP id u25-20020a0cb699000000b006461c12a7a2mr8600938qvd.33.1692634911878; Mon, 21 Aug 2023 09:21:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634911; cv=none; d=google.com; s=arc-20160816; b=kZv5oRcqVX45Fsnd4MbRLgfs54G0n1PNflrzysFBSShtIIgezcbVNz95InsaW+HNcW OICJcN3ghmh8zyoKLU8OepKdx6DoRytpDYO0Oj2lpsRXH4PoJk+iK16cZN4UruDiypjB VeQmbbDgVnwD6VwyeygLwqoLxDPbYYkp94MPcGRURxIIfzOPW8pEK1qNDKHp1GrjvEZg PNMSZfWiKlJSY/71bREyD0a/5Ns9IEC815MJ9F+3JDFQq4OVNpb0p2ntRFYms9bgwzE2 3s2f5SGD+n+kOCDAqqngnJQlKTjaYAUWgspw+wbVfh5oGj0B2T2c93RRFr8GePqbYIzc dkyg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=Zucdk4qb84qisgGhPnIxse/xZlGoWPexNEtMrM77q7jyqlMJEsrLa2x92nqj191/jU MfpK+Tkr568e1RynPiqK4/B18Db26E3AV2CJLVxqhsVLH/1cGSYInK3/Ll8sIQDK3JHc 4hYUwv5ODTfiy4FxCVTXuCzmuxNU8vAdmHyUyg6MIjKdZVXjqDX/tCaEg4xvD5oXiabs 9Ir+5AB7fMxHW3wDusTZJ9EIYxA6kkKcWqwFUEre/C2jemVgF+hEEjw6Jtj9P3A94QH9 Fhoo3vkSAFHvxnEQ1TxIAacoN/5IyGs+xvmvP2P0/PnUISZ6YRAJCJr80OAm27jk7TBO ABYw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YkRmhrz4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x15-20020a0ce0cf000000b006418e9adf3fsi5020533qvk.202.2023.08.21.09.21.51 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:51 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=YkRmhrz4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cq-0001oP-T1; Mon, 21 Aug 2023 12:19:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cZ-0001Vb-A1 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cS-0005dZ-RB for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:10 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1bba48b0bd2so21246505ad.3 for ; Mon, 21 Aug 2023 09:19:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634747; x=1693239547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; b=YkRmhrz4bFm+nEFxqDe/RyAxNveMWtH8J4e6a8ZZPGW4EgcCGkImG8BzpuuZM2xdVk wJ5230HdpgWIlQvp1Y2+6eqXvyJ93OkGhnBxlxVfEt4jui6I55yUw1Yh1KiDVKgZZsJH ATceOlujXKy1QPvSCTcaySY9UIaFBv748Iu8VPICPekPgr0WPtEHiSbD7I/FuvXEmnz5 tkyl+IhgEREKTo3YAVlF0PrUFm48E6vgkQISZg5ySnP/WmvecqMeuUC/LP1qLmYnbfM9 XCwOF5n326/Bozoz5/TjQJQf9L3fnGDseYKlvbnZ6zGW5VuXoNe75bJ1xA5rpt11T3p3 XjgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634747; x=1693239547; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; b=ErW7ghAIJOG/CBpvN8OPMtIH1YruPXKcpbL1PeMWL9fVhb07NnPVQ+CE09+6vJbtO4 ukM9hjgEVPa0LdvwtT5J1AQNJfZrWDAftlaokEcoTqVkZ/NFFn12FCmAfTvtMzo0Zfse xouFWzC5slukQCwQ7HfIMDBvMdGGjigtGx/Kz0uwiDmTIXaCocwZFW8u1GhwUQEXbmNU R5L5WtDTcCEiWShjlbzw7dBJzKRdiIc59ZsV5+Fa6waxEvRoiG3uS2IBhZy3CA+wTglj n1fXWaSgT+oao+aqCu8wu6vGEwsiKWWLgTKDU5bP/XMFsv4hXFZvzDjhS/d3nxnWursc JmGw== X-Gm-Message-State: AOJu0YzFUdEKdmCkNmDg1nQXRuzDTAOniKKReKLFw630lJZUOZtN5AJJ LOMRcVAyHVIlYhz0TGwluSMtjqsWzmzz+rT2eSg= X-Received: by 2002:a17:903:278f:b0:1bc:e6a:205f with SMTP id jw15-20020a170903278f00b001bc0e6a205fmr4731120plb.20.1692634747440; Mon, 21 Aug 2023 09:19:07 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 13/19] crypto: Add generic 64-bit carry-less multiply routine Date: Mon, 21 Aug 2023 09:18:47 -0700 Message-Id: <20230821161854.419893-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- host/include/generic/host/crypto/clmul.h | 15 +++++++++++++++ include/crypto/clmul.h | 19 +++++++++++++++++++ crypto/clmul.c | 18 ++++++++++++++++++ 3 files changed, 52 insertions(+) create mode 100644 host/include/generic/host/crypto/clmul.h diff --git a/host/include/generic/host/crypto/clmul.h b/host/include/generic/host/crypto/clmul.h new file mode 100644 index 0000000000..915bfb88d3 --- /dev/null +++ b/host/include/generic/host/crypto/clmul.h @@ -0,0 +1,15 @@ +/* + * No host specific carry-less multiply acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef GENERIC_HOST_CRYPTO_CLMUL_H +#define GENERIC_HOST_CRYPTO_CLMUL_H + +#define HAVE_CLMUL_ACCEL false +#define ATTR_CLMUL_ACCEL + +Int128 clmul_64_accel(uint64_t, uint64_t) + QEMU_ERROR("unsupported accel"); + +#endif /* GENERIC_HOST_CRYPTO_CLMUL_H */ diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 0ea25a252c..c82d2d7559 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -8,6 +8,9 @@ #ifndef CRYPTO_CLMUL_H #define CRYPTO_CLMUL_H +#include "qemu/int128.h" +#include "host/crypto/clmul.h" + /** * clmul_8x8_low: * @@ -61,4 +64,20 @@ uint64_t clmul_16x2_odd(uint64_t, uint64_t); */ uint64_t clmul_32(uint32_t, uint32_t); +/** + * clmul_64: + * + * Perform a 64x64->128 carry-less multiply. + */ +Int128 clmul_64_gen(uint64_t, uint64_t); + +static inline Int128 clmul_64(uint64_t a, uint64_t b) +{ + if (HAVE_CLMUL_ACCEL) { + return clmul_64_accel(a, b); + } else { + return clmul_64_gen(a, b); + } +} + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 36ada1be9d..abf79cc49a 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -92,3 +92,21 @@ uint64_t clmul_32(uint32_t n, uint32_t m32) } return r; } + +Int128 clmul_64_gen(uint64_t n, uint64_t m) +{ + uint64_t rl = 0, rh = 0; + + /* Bit 0 can only influence the low 64-bit result. */ + if (n & 1) { + rl = m; + } + + for (int i = 1; i < 64; ++i) { + uint64_t mask = -(n & 1); + rl ^= (m << i) & mask; + rh ^= (m >> (64 - i)) & mask; + n >>= 1; + } + return int128_make128(rl, rh); +} From patchwork Mon Aug 21 16:18:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715391 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925920wrs; Mon, 21 Aug 2023 09:21:51 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHrlaxCjOeexLHjEsyGGTuN6i/zkl0uhErP6CGAkv02YjeRPLRHAS6dhPcnf3oTvzrmptvP X-Received: by 2002:a0c:ab05:0:b0:64f:3f26:2236 with SMTP id h5-20020a0cab05000000b0064f3f262236mr2720430qvb.16.1692634910754; Mon, 21 Aug 2023 09:21:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634910; cv=none; d=google.com; s=arc-20160816; b=WalXZWuFp/KxQ2vsjBvseuE5Xg93HkpPb0kPEs+UGJppKpWKh2AJV3EvQMeIbOrOUn IeH/Bt26lJuukEXg3nsI3wgDoEs9GAbsb2hiKButfPb7e1g1dJy99Aw+SFuMJe9pBMnF ucdboqojg/j6HZ56BiC5f/Ep3tBXczxVsFnwxltOM0tSLOFYuuZKgzXWXtFlaEdgKv61 LStk2nCZVvZNNOIheA88cQLZFr6RCUGA/y9tczOmPGOfqojsA5Cm46yuujWA4/+HeVNR 1AvDBdyfq8nqcFDyIpHFZP9UsZph8Z6zlPfRMkb/j1KyIfgalfaIv1/ez8nnPAOsF2OB 5o9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=VNG3N8WNTdR97ypAgweii32ozK0a5IgKKpJ3f0VgmLoPGMBaQITdMm6dS5InoqCMIu 00srSvkP3A21OcMpgfloZvdOioz0NcjsTOLHAYs7jLJ5zVG0APJxcyRNZTdioLoHR2ne 8V084pq/LzY364lbN0Aqm1KuR344p6D0dgBdVVynb6jbrrglGgjIh/vw8736jpWf+UFk sCTWaEF9EZ/0PDhwZDe8WINrjjy2i5qGokPawHp/TqQMWU15y+36BujSQlXSRQWi54Ib mjYbVkHo6M84i+/PboSNpToScQOiRQrXgqNT537bxeVKr8ozVmuJ42R6p3ftqTb5T5Em LImA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AWHYctkq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p16-20020a0cf550000000b006471113a986si55264qvm.323.2023.08.21.09.21.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AWHYctkq; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cr-0001qd-Rb; Mon, 21 Aug 2023 12:19:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cb-0001Vi-8Z for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:20 -0400 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cW-0005dn-70 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:15 -0400 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-56b2e689828so999635a12.1 for ; Mon, 21 Aug 2023 09:19:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634748; x=1693239548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; b=AWHYctkqAfQRB+kLjK7K2oY+ra01tdCFC9xxKekA+mPYcQE76r5bW6IlzQOmijDxJC ju+0pZliqjWDKqZ7UcpU5P0y9cei/aVzXFrsmqiBuOOGHM3ySSjwZRXTq8CMFx7LfBwj EwexEoh7qCFsPu8eEIleqjzGU6mtN9IoOkjRt0ey5Zrl31Jd5TJTYHWDE3BQgnou47Wz oefVoXl2qlWumUKem46DhcDtQfIjy4HYvy2Q2D/ebb7ONzUpVEodXRVAiJUbvIDL5j4d ypnMvbGwp/m/XXlbW4e7SaOHIqHH4NpDOY03RXDg638ogQbl/vWd5UZzP/T267aUbBYx 02+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634748; x=1693239548; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LeRwmoZj3gayyKtebRoC0JW2YuHlKTaNGyIqgqME+tw=; b=fpoIUX1UvgACRQ3sruSlTg3KxiPiztd2M8hCMgHm2sZuavA5uF4ZeaEOusvDXkqC8s G8+oe4y4OYJLxRYfdCvky/wkxRRZkCr6PlomIZ2AZrXmDm4/ONkbHi+c9sYzDAIzv7sO gX1F6GAjp3guVv6Xob5OVypKaCB8WHpJRzQZudJFnD+KNinqc6wDR9g2EmJT2Mula+nn 1cG2cOnLvsQlIwpesmeIdRHKqe9eM6yuop3qNZz/NUPLPDE9ZmYFU2D64VHHsgrIzfNo PW81Y9IUmN2oiDkvnQAl7P6A7WiBjqa0Kc0RZ3Lk3m+PISXZzYzJjJ0oSf7kG3eu9M2E Ja9w== X-Gm-Message-State: AOJu0YzkktU3RoyL3/mZjnxzix+uzEnDReugotyM/IPtL4ZTQ1+ZGFdo k4XdFqsLu/Y73jW2q3cKLeX2QTlnLP5tkW5opv0= X-Received: by 2002:a05:6a20:3945:b0:140:3554:3f44 with SMTP id r5-20020a056a20394500b0014035543f44mr10294183pzg.22.1692634748425; Mon, 21 Aug 2023 09:19:08 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 14/19] target/arm: Use clmul_64 Date: Mon, 21 Aug 2023 09:18:48 -0700 Message-Id: <20230821161854.419893-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::530; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x530.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/arm/tcg/vec_helper.c | 22 ++++------------------ 1 file changed, 4 insertions(+), 18 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index ffb4b44ce4..1f93510b85 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2003,28 +2003,14 @@ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); intptr_t hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; i += 2) { - uint64_t nn = n[i + hi]; - uint64_t mm = m[i + hi]; - uint64_t rhi = 0; - uint64_t rlo = 0; - - /* Bit 0 can only influence the low 64-bit result. */ - if (nn & 1) { - rlo = mm; - } - - for (j = 1; j < 64; ++j) { - uint64_t mask = -((nn >> j) & 1); - rlo ^= (mm << j) & mask; - rhi ^= (mm >> (64 - j)) & mask; - } - d[i] = rlo; - d[i + 1] = rhi; + Int128 r = clmul_64(n[i + hi], m[i + hi]); + d[i] = int128_getlo(r); + d[i + 1] = int128_gethi(r); } clear_tail(d, opr_sz, simd_maxsz(desc)); } From patchwork Mon Aug 21 16:18:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715388 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925744wrs; Mon, 21 Aug 2023 09:21:30 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEYxEbTzdLnugyGq7B9fwWkw0vyBo8/XRKTisOq6lVFFeBEhe9swN/TNETghyBSX+x/S/TP X-Received: by 2002:a05:622a:41c7:b0:403:b382:613f with SMTP id ce7-20020a05622a41c700b00403b382613fmr7243440qtb.44.1692634889965; Mon, 21 Aug 2023 09:21:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634889; cv=none; d=google.com; s=arc-20160816; b=tL+fgFcR4JvmL/W04zc983HOyhURq6zs5Xd5wLB8W4AP+xbC/YAPSay3lTNj354WIG b8WypbnVTwxuCf6yVvt0q6bwh1NPcIr4RtGrCenN7PRdnxFv9xelT20HmR5q6C0Z9ToH AKuxWrxdr+mBO3MXbQVa5ym+3oojWcuGAWJt0VOJTrzZxISMu8Db8fsFzQqOQnoV0qTH pouV7SOxGsFNV3ozU0dpSLDGk4iS7hEc3Br4iG12Qp3zxTyHrX6Y5cTICVR1K8g4+hpC 30FzQ89BVZ5xRJZROQ/HmeuXEspCxYz4GUFwKuCFIrB6U58D1bHe8l1lVjuehONW0T59 UrCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=vNIfl8Zno324RGQZHPKZSQ9h8zmbuXQm4lCNKlapd7s=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=WTjuQl+UEvMRxC/XwFKc7mMr9/YW2g8/5SQJK7HVoJxFyQTNBkmigjAzgl3jNj4OMA qC+nKLPWTfqfQ3jjk+o6etpiyorUZ6vcV/ixk/EWjvYbi7h9vJV0JxSx/32kBKSEDovh W1269Wnyvt2sIluVnMd1KxBLMntdbGqD8xNeOrzhN2SrERrh3n7F9gIho1ZHi6vxsSN5 ZGsUHXqzmFyJ7emQacmpyscGzv76cXe8jwePQTMdG7zD373ie2jzhG9eBb5TNEut/T7g NPVE6plToiw6M7w3G2mdgynQwpVgVYoWdnxqSPZWXnhHWlKbgg8jObT2Lbzg2qEHbj2T uR/g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XrOtuhcZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id g4-20020ac87f44000000b0040fd6e080dbsi53487qtk.170.2023.08.21.09.21.29 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:29 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XrOtuhcZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cp-0001mk-Vy; Mon, 21 Aug 2023 12:19:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cd-0001Xp-8J for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:28 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cW-0005dv-9Y for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1bdc243d62bso20708645ad.3 for ; Mon, 21 Aug 2023 09:19:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634749; x=1693239549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vNIfl8Zno324RGQZHPKZSQ9h8zmbuXQm4lCNKlapd7s=; b=XrOtuhcZL3T0R9NEmXel42Xf0nTYSPeNY0og+JTZFPgk+Lf6RENIxtcxrEmmXsCVrb FD8g4FYT2Ug07vboCuMszQY69bFwatag75fWq4qyt85ML/Ccb+6aZrCoKbeUK14oXs6M ly/EwYIjt9cJVz7sshp9kWBBXVa07fr0qC/NClo4g0Hw1laFHIZ6REKz4hYJC8i7JPOe +RBdEBTJk1rl/6ML3q7aHZ2JB7qFF0oIwiGnA/D63W7tY7eUXRyMgD3ddZQBaS+RfkRe kxauQFNWkcjz9IlQDubKtMIazeQFweqka+Ng1Jzvtyq7SFYLiEj/tRHc+IihrqZZTzq4 XZDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634749; x=1693239549; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vNIfl8Zno324RGQZHPKZSQ9h8zmbuXQm4lCNKlapd7s=; b=Ouz8SMR+ObSwessFFGX/RrLfHKooVWVb8hDEOZcTPlDPpoZPofyczKoLSX2gqELy9v +jsZBl7esJqE7dNl1iU1ibBP6YZTJCZ/uTjopUkie0zNxJ/rytGEdXTGCS51nLl69Z7G EeMsl9Qk2kcFT3/Bv9v5coID7FCBgWV2gJrVhO4Yfl9W3bPE12wG+7WqsJFYn8+gRKL4 qmmnNmC/11wHrENAKmz1ZYJFx43EvxCpWkDalJDzAaKoZh2+wUJxpplFJ9yGh9H7Uiga gVfYqAn6Axo8CZL+FwCFJAxCRthTio0n0VHKm4NW33v0WiyrLacsYKP6p/9P7IgLAddQ wuqw== X-Gm-Message-State: AOJu0Yw8+CThU+ILprZ7yVLUVTc6ZKZRvOxFSMRRwFsQ9OlOvOHZgTaJ i4stZvd6PmaR9BckGQ8qIj9lB6i8SmwGoE8LlfM= X-Received: by 2002:a17:902:ab54:b0:1bc:2bd:8523 with SMTP id ij20-20020a170902ab5400b001bc02bd8523mr4466019plb.24.1692634749288; Mon, 21 Aug 2023 09:19:09 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 15/19] target/i386: Use clmul_64 Date: Mon, 21 Aug 2023 09:18:49 -0700 Message-Id: <20230821161854.419893-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/i386/ops_sse.h | 40 +++++++++------------------------------- 1 file changed, 9 insertions(+), 31 deletions(-) diff --git a/target/i386/ops_sse.h b/target/i386/ops_sse.h index a0e425733f..33908c0691 100644 --- a/target/i386/ops_sse.h +++ b/target/i386/ops_sse.h @@ -20,6 +20,7 @@ #include "crypto/aes.h" #include "crypto/aes-round.h" +#include "crypto/clmul.h" #if SHIFT == 0 #define Reg MMXReg @@ -2122,41 +2123,18 @@ target_ulong helper_crc32(uint32_t crc1, target_ulong msg, uint32_t len) #endif -#if SHIFT == 1 -static void clmulq(uint64_t *dest_l, uint64_t *dest_h, - uint64_t a, uint64_t b) -{ - uint64_t al, ah, resh, resl; - - ah = 0; - al = a; - resh = resl = 0; - - while (b) { - if (b & 1) { - resl ^= al; - resh ^= ah; - } - ah = (ah << 1) | (al >> 63); - al <<= 1; - b >>= 1; - } - - *dest_l = resl; - *dest_h = resh; -} -#endif - void glue(helper_pclmulqdq, SUFFIX)(CPUX86State *env, Reg *d, Reg *v, Reg *s, uint32_t ctrl) { - uint64_t a, b; - int i; + int a_idx = (ctrl & 1) != 0; + int b_idx = (ctrl & 16) != 0; - for (i = 0; i < 1 << SHIFT; i += 2) { - a = v->Q(((ctrl & 1) != 0) + i); - b = s->Q(((ctrl & 16) != 0) + i); - clmulq(&d->Q(i), &d->Q(i + 1), a, b); + for (int i = 0; i < SHIFT; i++) { + uint64_t a = v->Q(2 * i + a_idx); + uint64_t b = s->Q(2 * i + b_idx); + Int128 *r = (Int128 *)&d->ZMM_X(i); + + *r = clmul_64(a, b); } } From patchwork Mon Aug 21 16:18:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715382 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1924952wrs; Mon, 21 Aug 2023 09:19:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFTLIZOqqYt9WX4PE6d/2QEr8vP7E4lYsNq7W/+837ucr2L+yr3NGKt0e1lBc9Jl6UKEzKX X-Received: by 2002:a05:6359:5c1a:b0:134:d4b6:1c47 with SMTP id pu26-20020a0563595c1a00b00134d4b61c47mr5861054rwb.22.1692634798983; Mon, 21 Aug 2023 09:19:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634798; cv=none; d=google.com; s=arc-20160816; b=BCKiRyuwIHW9NrHUWVS8uovXLBQ1ErQlMoPcfIoAKg9kkX+9C518blqt3XLCXcrhhI XgKOZuszpc44CVap3c/OzRyMeEJd6AanCMOVxYR2hQEDP90wnYKBEeb7zQ0LFJeqvda2 gZalO1efOycVxgqpnQWdbQvyCDxCJR930pv3As08e/O5HoxZRYTQUtWU0pX2jR/ftOrR JvpTey/HiFjYtDVXG4HNiCEI0L3s8Ad7DMd0C8U4QnJY2/p2caL3Pqy4FCr1Gkuk8cwL wACx+Rb0t08XIc2Ul/g6pyXjoJgziupklR+OCG95V/8KiVxIjU14FVSmpa4KtajBW+WG pgtQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=ZSG3M86mJQVuVgrw5eeUdmAA8IHqWjyoFWBdmt46E194U4dkp3igw3mnYm4cRAmwEe 8mB6xctIm38+Z/4qkEyOC42DUAWJazdM56tv4xkfVNj0V3F+De6dvBQiSWRsy3e0Tefo WQO9Gdqu3AEhOk4go7kYcxgzzfetdZ4+NJIC0/lJeTGcHpGqoce+WyMT1vX/t5KSm4PT 4zjJRpPHyXYxgon2s8pJzZ39Kw93hqL6Zy+gFzDMpRmsgq1PhOr4S6zrjyKH7XmtQaTc 6CRc/r8UAO/wbdOPZyAe8kpJxLwMwkbmtANUQHON5RVltgky3RhXO9rVkQozB7jFohgH s/IA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e5mmFuf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id dw13-20020a05620a600d00b0076daa47a1d3si1022135qkb.150.2023.08.21.09.19.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:19:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=e5mmFuf+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7co-0001jK-V6; Mon, 21 Aug 2023 12:19:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cd-0001Xl-8Z for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:28 -0400 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cW-0005e3-VR for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bc83a96067so18880465ad.0 for ; Mon, 21 Aug 2023 09:19:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634750; x=1693239550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; b=e5mmFuf+NIEuPC6VsBhFAkPw+4IGqOGBEoCYZj/CXsT8OBN/RTnqr/x0NroMLMpFPN 3RS+mTLypgM9W9ifTfNCWGrxe78jXtnmujwjOvODYkhs81kmmfgHlh9rHD9D+XoqjHKo dCBZ1ta+gE5k3gljySPUXMLy5GC10e4E4E7/JzhoCNatgKfT9uXiPZe07U+bKqTZmAPt KTbvfl/OTWUI5gwsulnYZ+3v3g3b+3EM/8MfO2kxUizPPo+H5md3enYEemPZY0zmD63f ZR32IBUeKL3vbFAveTKgvgL/ljTj6QMlUGiCxmtwXVRqyxZKhiIFavBx06kgsbE5jp63 1Wow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634750; x=1693239550; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mA3cEFgfatbmP6xNVJY+hnrvCkua8Jz5pM6YfngoFhc=; b=c7Hagr2bZi7VgLdGXLUTeYaJw3WdNXJaDUhjKHk+cJb+/PML7y0UTBaLAT0WJGhEy7 fRdioOoOvJVklg3YLZzug5yKwBBnHWDDAvn87f6DsFrLuGox0jNxUPUxcYF7cKtuDhrF eZsP042GniX3sRVJPW07el1Z4SuxbDqOAuJnFWhrqsarmvSPN3JXfS8hsgLJqv20FTee WmJjGNAd4eIWs7LbiFktq/f19oru0aSm/LMhcLYZjyGUKaFl23o3GirffvYUmKR2V5nF jiaAY/+M+geuirKdKJ8DbQO650jgwVoC2PsRQXDOr5mODJGmjhSGzj2oKKeHGRW4MEh+ +/jQ== X-Gm-Message-State: AOJu0YxBvsDKYdNxbpqKYU/bgh4t5y68on0aiFRG+lEhfqLiFH2AMR/8 ov8ojVt/cmUYxUEigbihZh64YQMFBr20G4grIF8= X-Received: by 2002:a17:903:124b:b0:1bb:a367:a70 with SMTP id u11-20020a170903124b00b001bba3670a70mr5715990plh.17.1692634750133; Mon, 21 Aug 2023 09:19:10 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 16/19] target/s390x: Use clmul_64 Date: Mon, 21 Aug 2023 09:18:50 -0700 Message-Id: <20230821161854.419893-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62e; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the generic routine for 64-bit carry-less multiply. Remove our local version of galois_multiply64. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/s390x/tcg/vec_int_helper.c | 58 +++++++------------------------ 1 file changed, 12 insertions(+), 46 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index ba284b5379..b18d8a6d16 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -21,13 +21,6 @@ static bool s390_vec_is_zero(const S390Vector *v) return !v->doubleword[0] && !v->doubleword[1]; } -static void s390_vec_xor(S390Vector *res, const S390Vector *a, - const S390Vector *b) -{ - res->doubleword[0] = a->doubleword[0] ^ b->doubleword[0]; - res->doubleword[1] = a->doubleword[1] ^ b->doubleword[1]; -} - static void s390_vec_and(S390Vector *res, const S390Vector *a, const S390Vector *b) { @@ -166,26 +159,6 @@ DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -static S390Vector galois_multiply64(uint64_t a, uint64_t b) -{ - S390Vector res = {}; - S390Vector va = { - .doubleword[1] = a, - }; - S390Vector vb = { - .doubleword[1] = b, - }; - - while (!s390_vec_is_zero(&vb)) { - if (vb.doubleword[1] & 0x1) { - s390_vec_xor(&res, &res, &va); - } - s390_vec_shl(&va, &va, 1); - s390_vec_shr(&vb, &vb, 1); - } - return res; -} - /* * There is no carry across the two doublewords, so their order does * not matter. Nor is there partial overlap between registers. @@ -265,32 +238,25 @@ void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(v1, &tmp1, &tmp2); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = int128_gethi(r); + q1[1] = int128_getlo(r); } void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(&tmp1, &tmp1, &tmp2); - s390_vec_xor(v1, &tmp1, v4); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = q4[0] ^ int128_gethi(r); + q1[1] = q4[1] ^ int128_getlo(r); } #define DEF_VMAL(BITS) \ From patchwork Mon Aug 21 16:18:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715394 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1926467wrs; Mon, 21 Aug 2023 09:23:02 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGDl3Z4OS+ofvIkCr4iQo3bSPmWa+ENVASJ+4MdhOKrPHcTf2eudIqQk2H2UjrXvABH6t0e X-Received: by 2002:a05:620a:4146:b0:76d:a569:ccb4 with SMTP id k6-20020a05620a414600b0076da569ccb4mr4456381qko.0.1692634981906; Mon, 21 Aug 2023 09:23:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634981; cv=none; d=google.com; s=arc-20160816; b=foJ7tOaH8CT723eamd13gwHlkseU+TiJpS2vlssJp/NXb0sviofOkOhyvIsDDWcFC7 iZcElxk8JRT6KSjdKEClIyVlUBogIRYGHGKh+su3sVZIVKqe9p1LcniHB1dl4lEuFMq6 O33n1zuD+1AEF9LIQMCZhnS5c19iaPfhvvodxYHsGeeoeJJUA7WO9UbdZNUbMBrb5xbD Kyr8b0cvTFwfZ+n2bdYMLyV+5XLcBsvCoSg6bwe8O3s9m5dO4IHJUEEjpl6RrRPRxz+u eueoLGHkgp3MWoieGXEIOcDIdNzMpLWKA4tfqRgfUG4+N843+Fg1mtG1usDU2NS/THQG N+Xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=RNSYabeHfWwXe7wDrGrGc9BDLh4vte4K5oHOOd2gwYw=; fh=ihrF4BApiA3Z+XvqRPGwUXuGbDlKBmHEuYVfGNAjZIQ=; b=PkMHcGw71I6gTO6zAq7KGbFgyue/mDY5jvJ7DvUPMoZ3GZbz+jGKRxdcciq/jJhrzG E5gmd9DIE75EWK/NPU+0Ok/F3rXU6FN4YCyBE220v1Wqng/BnrD33hM3SFArRKNTQ9am W6Ep8F4sXEX1zWCPnbsqVZYilkbuk/iAOgShJgEGAOS+9peleTAWYljCgXdE0RZUvlm4 1RrNt+NsD3IQx5IOxEh76v22aymCFe//7zCUENiCMGR/ToPG4C3lq35sxuy4qbWRhhPV FjUvdIa7fqAexkOn3YnPzFj3D0BV1g41WQG/QmJM2FhZIPaK2c98tO0imFnwhI1DCon2 pdPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TEZPas9n; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id h6-20020a05620a400600b0076d8fb61eb7si4491409qko.715.2023.08.21.09.23.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:23:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TEZPas9n; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cu-00023d-DF; Mon, 21 Aug 2023 12:19:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cd-0001Xj-8S for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:28 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cX-0005e7-H3 for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:17 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-68a440a8a20so1129645b3a.3 for ; Mon, 21 Aug 2023 09:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634751; x=1693239551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RNSYabeHfWwXe7wDrGrGc9BDLh4vte4K5oHOOd2gwYw=; b=TEZPas9nnKBxQpT77XYGEAjcDfHu/DuT7Ewb29LfCiWL6H9UIK6DmVMsEXa3Dsf2fH mkok2z/PH5jbYeGlhY75a9EqKP1cT4yRD1uVN5yEnDjojUItBkE1sk75HY9XfRZJ1KaO 445Lluw//xyyZBPdLXeXWQ3VYAgUpt6ITBuw1wJav02N3RK1crluBmquT14gD//ik6Kk ft8M/ma1FAajzwt6aW8DSpQq0rs8VzoTD8fCRKstTyjcUssR6pdXFlTBjV3gzBgRA0O9 0lWPnGBTc4+z3MHfoFybiR97WYH5GeAD2ZtVU6ZyHPJWe18gJVcpBVpM1kDAmP3FqT4m Hv8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634751; x=1693239551; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RNSYabeHfWwXe7wDrGrGc9BDLh4vte4K5oHOOd2gwYw=; b=DA0KZaGrksrxRviY1obfNv1PsQvgh2qJ0Q5qrKG4dq8U92sJ7Zz8BVwrak2BbllvxG hPR7/1j9WGfbT2QYBOkQklx2YySXex/oIExalsL3EH4reiiNjVbBXdYXogWavsYRhSj/ dSSNN+K9p/QlN8cIUALh5nOxCrrAzZ1Ar34Z7KmCztz55OuF2l7FB5Jw51cX5K/otFWT AZAmRWFYa+tR/tu3xTxOgUmGMdhUUIBTiqDTBKgPjU/x0L87uwsvBfeuqjJiPFcW19YB xnXZjVz3Dd5hr/XLLQ6aFtprtcHv9Zg3Z7PxsJHSgxS0vRJJR/YRDqcTBpJt0cpgOg6M z/kw== X-Gm-Message-State: AOJu0Yxxm69NMpzLnl5vU0QpdIDJHSDvOjOR3ArJKldiH9guaRS34+bl IErLxeJb5jAhEq4KUi3+TLRVQbQT9Mt4zCpDkhA= X-Received: by 2002:a05:6a20:d7:b0:140:bc76:961c with SMTP id 23-20020a056a2000d700b00140bc76961cmr6490917pzh.60.1692634751275; Mon, 21 Aug 2023 09:19:11 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:10 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org, =?utf-8?q?Philippe_Mathieu-Daud?= =?utf-8?q?=C3=A9?= Subject: [PATCH v3 17/19] target/ppc: Use clmul_64 Date: Mon, 21 Aug 2023 09:18:51 -0700 Message-Id: <20230821161854.419893-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index ce793cf163..432834c7d5 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1456,20 +1456,9 @@ void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { - int i, j; - Int128 tmp, prod[2] = {int128_zero(), int128_zero()}; - - for (j = 0; j < 64; j++) { - for (i = 0; i < ARRAY_SIZE(r->u64); i++) { - if (a->VsrD(i) & (1ull << j)) { - tmp = int128_make64(b->VsrD(i)); - tmp = int128_lshift(tmp, j); - prod[i] = int128_xor(prod[i], tmp); - } - } - } - - r->s128 = int128_xor(prod[0], prod[1]); + Int128 e = clmul_64(a->u64[0], b->u64[0]); + Int128 o = clmul_64(a->u64[1], b->u64[1]); + r->s128 = int128_xor(e, o); } #if HOST_BIG_ENDIAN From patchwork Mon Aug 21 16:18:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715396 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1926864wrs; Mon, 21 Aug 2023 09:23:59 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEzVRMwbe9rqwjhLFbu0jnNCOkxoEFYUr9S+DVDvPopLPg+KWu2sdOXgVks0xLofh1ajLxZ X-Received: by 2002:a05:620a:f10:b0:767:7843:973f with SMTP id v16-20020a05620a0f1000b007677843973fmr8580968qkl.35.1692635039402; Mon, 21 Aug 2023 09:23:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692635039; cv=none; d=google.com; s=arc-20160816; b=y6x1ThYHYywPB/k7SMWEmJYFz4MMm+8kq6ui/RQgNPJw5WJLys99VFxbb//iOIqsD9 yGEi9tVXcfw9Eiwx9QFfid3KXh8tfz5G/7XpBDjBeLa2ryMsuU+9jb8B3/B0Paoui+uL ZWJk7CWsjU2hOpowj6BQkFcQ96RY5PuovpTNNmSZb7OkYAj13N0ZeBzmK+15lboBma94 N8BB7oVv5TDtPTq682Svxdjfzrrz8QE4M8M9RtkE4Y+vSik2PcpSqpz+6YTxGBlSDUf6 qFp4rzXNe/BRIn9wn6fOpUS6h//Bi9ZxOSX8vhS5oSW7fkIsCeO8PHNKf0zvZORq+SfS Kl5g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=O/ujeMQ9EKwgbYCWdLIfXdfu+WJ4It/hbCRBVAgAFRAofzEdIZqwj8w3z57+kOovCf unifsFdpNLgPM1yKnaKc6716JNv2hO/X0nPffDalvI3FJfSefu6I0wWu6DMH19NBrAx2 IMQkbeU04RXCHG8Ai91Zd1ltAVC6GoExsFQNmmVH7DM01ZqchVFZgzV6oL9s2tdyJBps geK8gDV0oQ5jMP9gmPeUfnjIr8QlDLJeD/CQNxYDmk4vW8S4MaKxgKNMOVl7vQ5d/D8H 5Wr8eLogBz7PO5ia980CGXpS0tS+L0SW4MduMZCcfK4BzzGrbTMJL5OH0Ui4BWFSYU5t t3Wg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xSyaOTcl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a12-20020a05620a438c00b0076ccac03bf4si4813696qkp.54.2023.08.21.09.23.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:23:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=xSyaOTcl; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cv-00026F-9f; Mon, 21 Aug 2023 12:19:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cf-0001Yu-5i for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:28 -0400 Received: from mail-pf1-x42f.google.com ([2607:f8b0:4864:20::42f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cZ-0005eG-3q for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:19 -0400 Received: by mail-pf1-x42f.google.com with SMTP id d2e1a72fcca58-68a3236a414so1561142b3a.0 for ; Mon, 21 Aug 2023 09:19:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634752; x=1693239552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; b=xSyaOTclCysSBD/Cx6LQ6/SDXyFJDFEgBPdPxyv5DIV19z0fHqKklwjyXZzyUu3006 C6R2qiRp0hsMdx3RGF6PwkplF5NWUm+psan0ics80ES1AsH+vbM150nQHIAy15uxxaIH GkPLMIFgbAKG1ielww7PNr2QVRqZ4lyRyRnGLThfM3afCS5H0L1HezaE2xv7BzA7JPB6 8c2lyaqMwlaALBgGSx3X68QbVizsumluMoWcddGGdEPBPTaV9QML2Yu4kRLbvj6gdXEB satf7UWqDAU9MLa3GbGwIy7jAJMgrPN4AwEmQO/IJw+KBuK83wShqLBkow7n6poFPDdH eqJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634752; x=1693239552; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; b=Lv5NLEsaOhoEQ1CTiKdyIcT2CFA6Egv451ZndlBHUFTbR8owuejOhSp0QU2ke1yS1k 4n0BurL3E5IPr1HMiU65511el195j7IuGtM67x+voBdxTTk9PLcy7feYoh/xahaG0zz0 SA3Z4Uzu/D5u4lDtCsGeDWometYLRCEiJYl/2W1lMGYBJ49go4//Xyvr0OY3Rc2/0t6H OETFHylECu2pKDHSBkjy/MhQ6X2rY+oyBFPdSoWGIISruWOHaRSxKKRld7QRoCTOH6TN 3cFYPrSVLtFcPHXTMi8A30UjyFrTy/St/0cKIqr70OORUYkTQ3sJnMOtXbM+0K0mFnNm CWPg== X-Gm-Message-State: AOJu0YyQKTwLOKHVaNqRh9VpB9Ztu/u+mqg9g3kcbFKRXp3v80cWmJDn V2yWlKOloWviwNAHP/YtCSzDJUWVQ6ssC2O4a4w= X-Received: by 2002:a05:6a20:26a5:b0:142:952d:3e61 with SMTP id h37-20020a056a2026a500b00142952d3e61mr6203714pze.7.1692634752157; Mon, 21 Aug 2023 09:19:12 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:11 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 18/19] host/include/i386: Implement clmul.h Date: Mon, 21 Aug 2023 09:18:52 -0700 Message-Id: <20230821161854.419893-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42f; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PCLMUL in cpuinfo; implement the accel hook. Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- host/include/i386/host/cpuinfo.h | 1 + host/include/i386/host/crypto/clmul.h | 29 +++++++++++++++++++++++++ host/include/x86_64/host/crypto/clmul.h | 1 + include/qemu/cpuid.h | 3 +++ util/cpuinfo-i386.c | 1 + 5 files changed, 35 insertions(+) create mode 100644 host/include/i386/host/crypto/clmul.h create mode 100644 host/include/x86_64/host/crypto/clmul.h diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h index 073d0a426f..7ae21568f7 100644 --- a/host/include/i386/host/cpuinfo.h +++ b/host/include/i386/host/cpuinfo.h @@ -27,6 +27,7 @@ #define CPUINFO_ATOMIC_VMOVDQA (1u << 16) #define CPUINFO_ATOMIC_VMOVDQU (1u << 17) #define CPUINFO_AES (1u << 18) +#define CPUINFO_PCLMUL (1u << 19) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/i386/host/crypto/clmul.h b/host/include/i386/host/crypto/clmul.h new file mode 100644 index 0000000000..dc3c814797 --- /dev/null +++ b/host/include/i386/host/crypto/clmul.h @@ -0,0 +1,29 @@ +/* + * x86 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef X86_HOST_CRYPTO_CLMUL_H +#define X86_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +#if defined(__PCLMUL__) +# define HAVE_CLMUL_ACCEL true +# define ATTR_CLMUL_ACCEL +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PCLMUL) +# define ATTR_CLMUL_ACCEL __attribute__((target("pclmul"))) +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { __m128i v; Int128 s; } u; + + u.v = _mm_clmulepi64_si128(_mm_set_epi64x(0, n), _mm_set_epi64x(0, m), 0); + return u.s; +} + +#endif /* X86_HOST_CRYPTO_CLMUL_H */ diff --git a/host/include/x86_64/host/crypto/clmul.h b/host/include/x86_64/host/crypto/clmul.h new file mode 100644 index 0000000000..f25eced416 --- /dev/null +++ b/host/include/x86_64/host/crypto/clmul.h @@ -0,0 +1 @@ +#include "host/include/i386/host/crypto/clmul.h" diff --git a/include/qemu/cpuid.h b/include/qemu/cpuid.h index 35325f1995..b11161555b 100644 --- a/include/qemu/cpuid.h +++ b/include/qemu/cpuid.h @@ -25,6 +25,9 @@ #endif /* Leaf 1, %ecx */ +#ifndef bit_PCLMUL +#define bit_PCLMUL (1 << 1) +#endif #ifndef bit_SSE4_1 #define bit_SSE4_1 (1 << 19) #endif diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c index 3a7b7e0ad1..36783fd199 100644 --- a/util/cpuinfo-i386.c +++ b/util/cpuinfo-i386.c @@ -39,6 +39,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) info |= (c & bit_SSE4_1 ? CPUINFO_SSE4 : 0); info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0); info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0); + info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0); /* Our AES support requires PSHUFB as well. */ info |= ((c & bit_AES) && (c & bit_SSSE3) ? CPUINFO_AES : 0); From patchwork Mon Aug 21 16:18:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715390 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp1925778wrs; Mon, 21 Aug 2023 09:21:34 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEVFZEjwXL0Xie4Uq7Ij4H/UYJeAjSvOeM3KRyV1h/fjPgVQ6Y0zb36QlcG9wSntUu1Yttm X-Received: by 2002:a05:622a:1492:b0:410:a018:c7f3 with SMTP id t18-20020a05622a149200b00410a018c7f3mr1878622qtx.9.1692634894466; Mon, 21 Aug 2023 09:21:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692634894; cv=none; d=google.com; s=arc-20160816; b=FH7zBiIC59aEWqqbbPbBji5dZAMoiKlNya4AFMtzP60vOZxaMG83LDXhvWOV4Z5SRz RuPNW3MqHbhQst9OALoRwI5eCUaBJ91TJRCqbx8tCCGHBWKgchV/rIFprA2/6TpNWU5M W8Bn5c9zw2NY7t7tnmGwqyhA2bBj/6qAJPkYbs/Lx0qPqpnFuiwnPnIaNfLXRIkna/t1 1DE1VHDvCml++NBJIueHu3wHcEgwhB3tnOEclDws9yO3U7p1FTVyh8QQjnm9zTKpNkD2 TmU1L8j4WXWcbgWTLUvNeaC1BKcj0zleDK2J8zUxkVWznNc8m2diAmwZAtfoMb3XPcmx 9rpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=mo0KPz9GEcKcR0eyjs/idf0YCm3xF3yPcNW7LGw1nra95dx9cd1utNxqGJTzGmwaP5 lJwLpNWcwaHxjIkbPD5SH0H6ORO5YVrNbQE9EBIWsF/OUrADxZW60O39Fl1IF9C2ZjVQ 8B2ZWuez7MPrRT5HTJWyvCF6Wxa1UNAUnTzwJKgIvdURBZl9c6QwTEwJY63RIdKOqddS E9XUQI9sTZnQNX6KjAsU6+LGHQYPZLcD8eDnYszeMr4MsYTubdY5LMwmwcCIoBRGuMOB 5s567Q4T1Gcd++RHA11n5UJjfEmCoDhYukoLfzCBjUR6L0pYe1smDh2SroFZrW53gMeq JpCw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=day2z50H; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x15-20020a05622a000f00b00403b84be504si5084369qtw.580.2023.08.21.09.21.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 21 Aug 2023 09:21:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=day2z50H; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qY7cq-0001mv-2s; Mon, 21 Aug 2023 12:19:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qY7cf-0001Yv-5h for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:28 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qY7cZ-0005eR-3m for qemu-devel@nongnu.org; Mon, 21 Aug 2023 12:19:19 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bdc19b782aso21393845ad.0 for ; Mon, 21 Aug 2023 09:19:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692634753; x=1693239553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; b=day2z50HRJjW9OCksML6OR951DxvW4xBvBgF+gHx4csdOQlfduokMerJXepsXyQZM5 jWMHYoSyDspxtisqAf3/OIIRrXuhFQOhKaWtmrc6HDKj5xTDYySWSey1eOY8dvopiQHl y1g9of+/bJRjR9E8+LEPqEMmwG4e/YqDjafhCO+PsbMmvmkXtQUMYUsa01DNlKmwkdAD hPzKLUM1Y8n5MiYuh02a8XtzfeNHP+MiVMy7DlhII4rbubjBa+qSRcCEoa7vXypl8ESK Zrh6uJXaLsUQO/OyODkz4ATBEBgDD36wxKvxDNrpplI7ZPSJDf7oAsFo8Jvre6CRO799 e5SA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692634753; x=1693239553; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; b=VDYquRenrxDLahPxbSS9K83Ihg73XeJd/CZ/VGoZHRC+/yat+/hD1gzxKtBbR4JlOc 9vZnQbkxCrF+N1hfN45wu/pRgSm3VWieEmN0jt80Q77X+JAqgFrLCOnOOdBfuRY27hsn z8xe2mehQ4mxM/+Ooy8P5jE1zqtuFEYIS5nMsKFARB22HM4megTsK/+pF7vf1PefW0Pp xh6zjSEXT5pJuiUeciceloPlcTOZDivQFwNvCA534zdFp0AWOekFGVB4da7eF3B7sfxn zCjmDlmyPUcHr35T/wVHWnW97X9Zs43CJc3oc0kDrkNoK23QEigWAMiF0Y6lY0J009Le m0hw== X-Gm-Message-State: AOJu0YwgZLtv0u13R0yRWsVUsaCJdMvTDX6SaLGxzYTGgpbCISse/KiK tWEG2UIOQzQkcQHwiVEC2wP3gLiZaxzwLNsCGF0= X-Received: by 2002:a17:902:eccf:b0:1bb:fffd:63d8 with SMTP id a15-20020a170902eccf00b001bbfffd63d8mr6036367plh.36.1692634752983; Mon, 21 Aug 2023 09:19:12 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:2c08:e710:4459:46f1]) by smtp.gmail.com with ESMTPSA id e16-20020a17090301d000b001ab2b4105ddsm7234549plh.60.2023.08.21.09.19.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 21 Aug 2023 09:19:12 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v3 19/19] host/include/aarch64: Implement clmul.h Date: Mon, 21 Aug 2023 09:18:53 -0700 Message-Id: <20230821161854.419893-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230821161854.419893-1-richard.henderson@linaro.org> References: <20230821161854.419893-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PMULL in cpuinfo; implement the accel hook. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- host/include/aarch64/host/cpuinfo.h | 1 + host/include/aarch64/host/crypto/clmul.h | 41 ++++++++++++++++++++++++ util/cpuinfo-aarch64.c | 4 ++- 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 host/include/aarch64/host/crypto/clmul.h diff --git a/host/include/aarch64/host/cpuinfo.h b/host/include/aarch64/host/cpuinfo.h index 769626b098..fe8c3b3fd1 100644 --- a/host/include/aarch64/host/cpuinfo.h +++ b/host/include/aarch64/host/cpuinfo.h @@ -10,6 +10,7 @@ #define CPUINFO_LSE (1u << 1) #define CPUINFO_LSE2 (1u << 2) #define CPUINFO_AES (1u << 3) +#define CPUINFO_PMULL (1u << 4) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/aarch64/host/crypto/clmul.h b/host/include/aarch64/host/crypto/clmul.h new file mode 100644 index 0000000000..bb516d8b2f --- /dev/null +++ b/host/include/aarch64/host/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * AArch64 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef AARCH64_HOST_CRYPTO_CLMUL_H +#define AARCH64_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +/* + * 64x64->128 pmull is available with FEAT_PMULL. + * Both FEAT_AES and FEAT_PMULL are covered under the same macro. + */ +#ifdef __ARM_FEATURE_AES +# define HAVE_CLMUL_ACCEL true +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PMULL) +#endif +#if !defined(__ARM_FEATURE_AES) && defined(CONFIG_ARM_AES_BUILTIN) +# define ATTR_CLMUL_ACCEL __attribute__((target("+crypto"))) +#else +# define ATTR_CLMUL_ACCEL +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { poly128_t v; Int128 s; } u; + +#ifdef CONFIG_ARM_AES_BUILTIN + u.v = vmull_p64((poly64_t)n, (poly64_t)m); +#else + asm(".arch_extension aes\n\t" + "pmull %0.1q, %1.1d, %2.1d" : "=w"(u.v) : "w"(n), "w"(m)); +#endif + return u.s; +} + +#endif /* AARCH64_HOST_CRYPTO_CLMUL_H */ diff --git a/util/cpuinfo-aarch64.c b/util/cpuinfo-aarch64.c index ababc39550..1d565b8420 100644 --- a/util/cpuinfo-aarch64.c +++ b/util/cpuinfo-aarch64.c @@ -56,12 +56,14 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) unsigned long hwcap = qemu_getauxval(AT_HWCAP); info |= (hwcap & HWCAP_ATOMICS ? CPUINFO_LSE : 0); info |= (hwcap & HWCAP_USCAT ? CPUINFO_LSE2 : 0); - info |= (hwcap & HWCAP_AES ? CPUINFO_AES: 0); + info |= (hwcap & HWCAP_AES ? CPUINFO_AES : 0); + info |= (hwcap & HWCAP_PMULL ? CPUINFO_PMULL : 0); #endif #ifdef CONFIG_DARWIN info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE") * CPUINFO_LSE; info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE2") * CPUINFO_LSE2; info |= sysctl_for_bool("hw.optional.arm.FEAT_AES") * CPUINFO_AES; + info |= sysctl_for_bool("hw.optional.arm.FEAT_PMULL") * CPUINFO_PMULL; #endif cpuinfo = info;