From patchwork Sat Aug 19 01:02:01 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715005 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp800970wrs; Fri, 18 Aug 2023 18:04:07 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEHUlj+1err0l9OiFUsYiTOu8DXp/DcqYzO4fNGlx7Wt0W9/g+pRr5cvw9ISwi646/+ltmG X-Received: by 2002:a05:622a:11c3:b0:404:979d:c45a with SMTP id n3-20020a05622a11c300b00404979dc45amr1113480qtk.28.1692407046806; Fri, 18 Aug 2023 18:04:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407046; cv=none; d=google.com; s=arc-20160816; b=CGT0jexWYFPr3wWsuVgLKlrOnEzRMTfmfLHRZjEzQCa8mxSu6ss0caqxhhAgnfkmj4 VBkbTamIwvdgv+vFBHG0SY16CFk46LsE4ayNdtZY7wxOAhmdO/bsCRarhhQ80jp1CgRy JKWpa1ZI85/w4Jg3Xz38JqdKqMXjQFEM4FMw7TSMhpO6LYv7fF6ADd5SMI9SmRpssbNI DQ/BmyemH51g1tCayOMIgPx1NT8ZPBzaHkT75ExFSqpaXoBUP4chtJAEFEIldknJ8O9T sa7eFmU+H9OGO6usTkz1VR/fKTp4od80OUMPhK+Z8bHRSuAFBuHCy/B1OZVPj63renKM NBlQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=GA+BBv/5ghEL8ru+ax4XZW6Y/DypVAtXl4ib7qhG71DPXEoKm9OQaCqUuWLDZy5oFz i+A/M/2DHgQAUmYGiHW84+VBBSs1hQ7kwxXNMPB0ElMzpjrUMzXWvV4WPb6kR9oE9r6L jCUQwCurFzA6u+ACIjVmnBGnoyx0f91e3LpN1afaqxqb17IvWE2bgOaHl+ce5Kv5xoAs QPY+Au+wzIBzBpRZyzfhlm07hbJtV8trVnu4Um7SAnmsiSSMdk/k4Kr9zOcX5BWZoKgb foeaoBu6oYTI3h2WIMhfZAi+///2pfyeuHRyrhxGdKR/fx2HATCIsIJx+5Ea5oBjeIbO scCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Dvc7qTBv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id m14-20020a05622a118e00b00403ab0d6550si1960708qtk.213.2023.08.18.18.04.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Dvc7qTBv; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMG-0008Q9-Mf; Fri, 18 Aug 2023 21:02:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAME-0008PU-QI for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:26 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMC-0007Kh-Am for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:26 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bc3d94d40fso12489335ad.3 for ; Fri, 18 Aug 2023 18:02:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406941; x=1693011741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; b=Dvc7qTBv4QzAtYROGST9D2CmvA+u5/pauRnyedkdFuaYG0M4LePTS30ayluwjGTNFB lxjij/nbWXA+YZ5Wr7uzqqv3SDEdegCDNscxDPiX5cQpkZjnchNQKg5fHDQUUMWycytM 6ANpBIewuUrl0j5OM4EYqCQxavGgSr0Rhl+bJ4BYZwHFZg88++n52XlnaoVK5wwTE9Ka I0dmoMbBh5qLtIUejrhyI0QxSlWicMup7lhAcSaZ5EDII/6PV/9N/Qv96oluo37As19u 0HeF+1w9orckytIV8uTQBRYBMUayQP/NfgpD3KNfCa90hVm0Q5dGeWfBUB/mg7xqivOo j7hQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406941; x=1693011741; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZGGgJ/u6kYIIhaOOh+kZR6FBn3VxoSdiYPAwK2Kqk9k=; b=SncrRIjg17ElS7zcYbYDJRKhR71h2RGAzd9HQpZ1KOqXu2NAXknCnYmX0H39vS5uXx /M7pAATXbUFg5ByCseNU9MM3qNttSuwJSyjuu27PgWK/ftddB0XXyOsVrqc6/QkrgVO4 TLgGur2uY08vOFtvkCrJlOX/+j0Y7xO1UHtpZPVjQSVvCWbhBdMNdeQ4QfDpzCn+mT3b p5O/QZ4KVxjeSGJ1dFPcsSNEZW5glbkyHZzXqqV1HBqUZSPeXdXZwHmWUX5IZieRYgYQ aZ0kncASAbY4i22iNdlWvY3l/4uD+sVrlhkojmLHKwhy9KIdLh775uiDmVhgSJDhbpSq i1ew== X-Gm-Message-State: AOJu0YwwjWannVxds8OxSYkz9sqLnppqAOm3ZLgvxgfY+gCz4jHq8Xwg zYMmai38mfkS1OCP7laj7G2fQkxMHbi9D7lYpzI= X-Received: by 2002:a17:902:ec8f:b0:1bb:ce4a:5893 with SMTP id x15-20020a170902ec8f00b001bbce4a5893mr1054184plg.30.1692406941410; Fri, 18 Aug 2023 18:02:21 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:21 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 01/18] crypto: Add generic 8-bit carry-less multiply routines Date: Fri, 18 Aug 2023 18:02:01 -0700 Message-Id: <20230819010218.192706-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 41 +++++++++++++++++++++++++++++ crypto/clmul.c | 60 ++++++++++++++++++++++++++++++++++++++++++ crypto/meson.build | 9 ++++--- 3 files changed, 107 insertions(+), 3 deletions(-) create mode 100644 include/crypto/clmul.h create mode 100644 crypto/clmul.c diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h new file mode 100644 index 0000000000..153b5e3057 --- /dev/null +++ b/include/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#ifndef CRYPTO_CLMUL_H +#define CRYPTO_CLMUL_H + +/** + * clmul_8x8_low: + * + * Perform eight 8x8->8 carry-less multiplies. + */ +uint64_t clmul_8x8_low(uint64_t, uint64_t); + +/** + * clmul_8x4_even: + * + * Perform four 8x8->16 carry-less multiplies. + * The odd bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_even(uint64_t, uint64_t); + +/** + * clmul_8x4_odd: + * + * Perform four 8x8->16 carry-less multiplies. + * The even bytes of the inputs are ignored. + */ +uint64_t clmul_8x4_odd(uint64_t, uint64_t); + +/** + * clmul_8x4_packed: + * + * Perform four 8x8->16 carry-less multiplies. + */ +uint64_t clmul_8x4_packed(uint32_t, uint32_t); + +#endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c new file mode 100644 index 0000000000..82d873fee5 --- /dev/null +++ b/crypto/clmul.c @@ -0,0 +1,60 @@ +/* + * Carry-less multiply operations. + * SPDX-License-Identifier: GPL-2.0-or-later + * + * Copyright (C) 2023 Linaro, Ltd. + */ + +#include "qemu/osdep.h" +#include "crypto/clmul.h" + +uint64_t clmul_8x8_low(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0101010101010101ull) * 0xff; + r ^= m & mask; + m = (m << 1) & 0xfefefefefefefefeull; + n >>= 1; + } + return r; +} + +static uint64_t clmul_8x4_even_int(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + for (int i = 0; i < 8; ++i) { + uint64_t mask = (n & 0x0001000100010001ull) * 0xffff; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_8x4_even(uint64_t n, uint64_t m) +{ + n &= 0x00ff00ff00ff00ffull; + m &= 0x00ff00ff00ff00ffull; + return clmul_8x4_even_int(n, m); +} + +uint64_t clmul_8x4_odd(uint64_t n, uint64_t m) +{ + return clmul_8x4_even(n >> 8, m >> 8); +} + +static uint64_t unpack_8_to_16(uint64_t x) +{ + return (x & 0x000000ff) + | ((x & 0x0000ff00) << 8) + | ((x & 0x00ff0000) << 16) + | ((x & 0xff000000) << 24); +} + +uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) +{ + return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); +} diff --git a/crypto/meson.build b/crypto/meson.build index 5f03a30d34..9ac1a89802 100644 --- a/crypto/meson.build +++ b/crypto/meson.build @@ -48,9 +48,12 @@ if have_afalg endif crypto_ss.add(when: gnutls, if_true: files('tls-cipher-suites.c')) -util_ss.add(files('sm4.c')) -util_ss.add(files('aes.c')) -util_ss.add(files('init.c')) +util_ss.add(files( + 'aes.c', + 'clmul.c', + 'init.c', + 'sm4.c', +)) if gnutls.found() util_ss.add(gnutls) endif From patchwork Sat Aug 19 01:02:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715006 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801076wrs; Fri, 18 Aug 2023 18:04:24 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGf7FgvepIyGuePSP7BaSwHrtLuzC+VFIKO/Tov3ISwNjb9vCbpPojX0XUm9TDmXL2jLcaq X-Received: by 2002:ac8:5e0b:0:b0:40f:a7be:97fd with SMTP id h11-20020ac85e0b000000b0040fa7be97fdmr844415qtx.42.1692407064456; Fri, 18 Aug 2023 18:04:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407064; cv=none; d=google.com; s=arc-20160816; b=MfuIVm4AgphEGxXJq4uGTW7m7HstsRT+FCJpmNKWuR1Df6sB3xgSoJi8Q+UjqitA5i NEJwsxFxVh4Rl+xkp3dBwicUti90PxOXBiBq7eZVyZYyZSmTX66uLE3LUJo5kfUeX6md gNFATFenXn1HLSv/VFkn0zPo6rvtylbIgvlCZ68nzN+0uaq34dCH9DBrEY8y8hCpzndU LQoPVyKHoDSnLPtZZYeV9XMb6UlFXQe4e6HcfGgkGjKT9aKuAngAV+hpFq9ZY/NAB2va kBp/fj8bTCY6r33JDco5hG+Il54NMWQg8gnz1xX84rKYSLv/rlSBYKlolSPNydxIvR1R xWdQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=y9w0S/pc005a7b816dzSReKj9icWPRL/k+/UuSXXFEo=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=Bvsof7qcLTm/5OEz4OMr1YEY7k/jeAPSh3CMP8ZfMM1QvAUi7EzR0Sx8V6Sl0a18c2 +lvZVkcYim+GCzWiGfRYgDq74T+C3Y9WBFltQ1h3PmLi3oUoOjfSEkZHyGs+S6X/uhy2 +3qkZWNhRKnsfFPRo6ByDL+JDB4ge2bdiErE2EyT2ATzt+FkamWz5OwEqRyijLHoD+lj HTSD7IggVGIGrt2Z6BKTHTWbK61bH8S9Qgcr7a7cerwfSt194jceZurSCwF+JsxvjeMN aNRxLioVZXC7uverfv1B8PaerPpdEtYV3k458r5ZEgG9PjvdEFjhlbqxfkHElALfibj1 WYVw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LJ9EY7Uw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t26-20020a05622a181a00b00403a6629564si2000806qtc.538.2023.08.18.18.04.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=LJ9EY7Uw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008QS-Ak; Fri, 18 Aug 2023 21:02:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMF-0008Pk-NB for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:28 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMC-0007Kt-O2 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:27 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bf55a81eeaso2263395ad.0 for ; Fri, 18 Aug 2023 18:02:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406942; x=1693011742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y9w0S/pc005a7b816dzSReKj9icWPRL/k+/UuSXXFEo=; b=LJ9EY7UwyLgzVBBTfKdQGQxWSptb5GEpGbzqyTk4LuvspyCR9Jt2vC5XwXmyALnlWO nS+fF4Asmg9cYML3xZSQ/tqICsXZ9PJ0YGgV/CBwN6AoDkEf7wX7USEUvx/OS3lmQZgf mxpBrHP8DOl1WYzmwqJVRzYjnIoRBNRc7HA0dpv17ieo5bJpj+tyFDV8vSZavHpiG2yq JjMb34qdERaC7ClQSidxEvsWpK2pfqCGd7ROVmQaBorMwakKy5xkbzb5sT9H60T0poKN 6xML3kjTZDSHygHE/ZKjZBB4MH1tJbBKB9HPuwmWaOkVmpDFho359KOIxD9cLJptVBA0 JCHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406942; x=1693011742; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y9w0S/pc005a7b816dzSReKj9icWPRL/k+/UuSXXFEo=; b=ILzAWkn4FMPSVbqT8KF7A5wQNAmfd2fbLMNBJy+/YpHJhPIiCgg2GU986ilcYGRula ciFiVeUyKUvo3DhznRR28ATTrsidjEdDhDtU84t2kO/j7KbQp0Zwkqrs3yqScnlmsqSk PlTLuJSttn/Pddpii7iYLiHxzZAPLBvbewpIt6029xr9TUwlXEmYJ09w/Gjc3De1wxSm 2xC8jpoABjeb10ZKP6mQwe/BuDRqUKdH5LpT6ycqDA5Qx7/5VKAXJ0NdlVW4wd1BVpUv HDkPmWublsWdlN0qvDyeM6wv+zN3XGhnZkIHEH/dNmm9OCaoOmYYHCSSiAnxvKI/MifL X9EA== X-Gm-Message-State: AOJu0YxOGiIvTJOFksKOKD1mqUHnwbcBgsGm4YDIhrTmxsPJEmOs4o+e Lvp5jUtrvpRHSbefb3ClMS0RLVrnbZkP7p89KhE= X-Received: by 2002:a17:903:124e:b0:1bd:b8c0:b57e with SMTP id u14-20020a170903124e00b001bdb8c0b57emr855050plh.40.1692406942375; Fri, 18 Aug 2023 18:02:22 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 02/18] target/arm: Use clmul_8* routines Date: Fri, 18 Aug 2023 18:02:02 -0700 Message-Id: <20230819010218.192706-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of pmull_h. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/arm/tcg/vec_internal.h | 5 ---- target/arm/tcg/mve_helper.c | 8 ++---- target/arm/tcg/vec_helper.c | 53 ++++------------------------------- 3 files changed, 9 insertions(+), 57 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index 1f4ed80ff7..c4afba6d9f 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,11 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 8 x 8 -> 16 vector polynomial multiply where the inputs are - * in the low 8 bits of each 16-bit element -*/ -uint64_t pmull_h(uint64_t op1, uint64_t op2); /* * 16 x 16 -> 32 vector polynomial multiply where the inputs are * in the low 16 bits of each 32-bit element diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 403b345ea3..96ddfb4b3a 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -26,6 +26,7 @@ #include "exec/exec-all.h" #include "tcg/tcg.h" #include "fpu/softfloat.h" +#include "crypto/clmul.h" static uint16_t mve_eci_mask(CPUARMState *env) { @@ -984,15 +985,12 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPH_MASK 0x00ff00ff00ff00ffULL #define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBH(N, M) pmull_h((N) & VMULLPH_MASK, (M) & VMULLPH_MASK) -#define DO_VMULLPTH(N, M) DO_VMULLPBH((N) >> 8, (M) >> 8) #define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) #define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) -DO_2OP(vmullpbh, 8, uint64_t, DO_VMULLPBH) -DO_2OP(vmullpth, 8, uint64_t, DO_VMULLPTH) +DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) +DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 6712a2c790..cd630ff905 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -23,6 +23,7 @@ #include "tcg/tcg-gvec-desc.h" #include "fpu/softfloat.h" #include "qemu/int128.h" +#include "crypto/clmul.h" #include "vec_internal.h" /* @@ -1986,21 +1987,11 @@ void HELPER(gvec_ushl_h)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = n[i]; - uint64_t mm = m[i]; - uint64_t rr = 0; - - for (j = 0; j < 8; ++j) { - uint64_t mask = (nn & 0x0101010101010101ull) * 0xff; - rr ^= mm & mask; - mm = (mm << 1) & 0xfefefefefefefefeull; - nn >>= 1; - } - d[i] = rr; + d[i] = clmul_8x8_low(n[i], m[i]); } clear_tail(d, opr_sz, simd_maxsz(desc)); } @@ -2038,22 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -/* - * 8x8->16 polynomial multiply. - * - * The byte inputs are expanded to (or extracted from) half-words. - * Note that neon and sve2 get the inputs from different positions. - * This allows 4 bytes to be processed in parallel with uint64_t. - */ - -static uint64_t expand_byte_to_half(uint64_t x) -{ - return (x & 0x000000ff) - | ((x & 0x0000ff00) << 8) - | ((x & 0x00ff0000) << 16) - | ((x & 0xff000000) << 24); -} - uint64_t pmull_w(uint64_t op1, uint64_t op2) { uint64_t result = 0; @@ -2067,29 +2042,16 @@ uint64_t pmull_w(uint64_t op1, uint64_t op2) return result; } -uint64_t pmull_h(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 8; ++i) { - uint64_t mask = (op1 & 0x0001000100010001ull) * 0xffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; uint64_t nn = n[hi], mm = m[hi]; - d[0] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[0] = clmul_8x4_packed(nn, mm); nn >>= 32; mm >>= 32; - d[1] = pmull_h(expand_byte_to_half(nn), expand_byte_to_half(mm)); + d[1] = clmul_8x4_packed(nn, mm); clear_tail(d, 16, simd_maxsz(desc)); } @@ -2102,10 +2064,7 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; ++i) { - uint64_t nn = (n[i] >> shift) & 0x00ff00ff00ff00ffull; - uint64_t mm = (m[i] >> shift) & 0x00ff00ff00ff00ffull; - - d[i] = pmull_h(nn, mm); + d[i] = clmul_8x4_even(n[i] >> shift, m[i] >> shift); } } From patchwork Sat Aug 19 01:02:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715001 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp800787wrs; Fri, 18 Aug 2023 18:03:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFgzLgqn3c7zD/2ugXaQIjJHhL5B1hrAinlQ6zxGmWdhdDIqtj35XAgLZMOjEm/O3x5DmeQ X-Received: by 2002:a05:620a:394c:b0:767:a1ab:b3db with SMTP id qs12-20020a05620a394c00b00767a1abb3dbmr942509qkn.16.1692407019810; Fri, 18 Aug 2023 18:03:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407019; cv=none; d=google.com; s=arc-20160816; b=p5DpKKnf1dq1CDUT5ZNeynFWUtKvilDSrcLK0I1A/KKsJINRdBVz8tqYFK3Mhjk5GV lH1MOO6Uddi85MnoHlYPkFN4vrdqpz4ZF+h4JpOw0n2LX0bdvAuPrjjjsAYXpYmMOsIA ri0N8zit8STJcfdJruX2IHPuR1fiSUeUlR7B05Q0LwfbEwSiauu0UVV3RkHbzeSfW1rU 0t7/q/UN4qI/hRMh2WUaBUbxTjlz5KbYu2wecnhgti7Cx2w6Lyc4K6a7u2Cd/WEWIlqY gduGU3Zy/4NEyN3iZQi9mb3/kAiKM+HVPNh3foUJMJxJdTdGqO+am1PwpXruSZ2QCnv+ cgHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=7MlO/4lz23eNLFrlyrvCLQF2KGu4x8cNZ2TLPxKRIDg=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=dq4yjXtX+A+e8nY4ockb/hC4/lPWYEEXq24f5bCHTG2QuG4UtCwmqn9jyXqUKlhqyI TPfXjEMrlejbWiaN3G0pU+ra9FDK+1TfrqL9dnnxG3Hm3JlxZj+zdWTrJQs+Cbzgpd1S GbtTY/DgtAz1EwuG4emJDa8fLl0U3uT48H3h6JRFZtXL1DJv/uqMfDQ2D+tg3Kq9V7uz 5ZY96fYiIiUp2c0L57Zb80ZVr2wF1ChV9uJSxWzYjwiDUinoD4nry9G0ld77Hy5lziuj 0RR0A9N0T6fFjnaOccY+OS2uqbnRdyxj1uKmsoWO93E+ODWfUxFyn13wN3/NidiRTMXB 8EPQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H70jcr5B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s11-20020ac85ccb000000b00403ad99b849si2017075qta.789.2023.08.18.18.03.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:03:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=H70jcr5B; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMH-0008QQ-FB; Fri, 18 Aug 2023 21:02:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMF-0008Pd-FH for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:27 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMC-0007Ky-M0 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:27 -0400 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1bee82fab5aso12288785ad.3 for ; Fri, 18 Aug 2023 18:02:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406943; x=1693011743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7MlO/4lz23eNLFrlyrvCLQF2KGu4x8cNZ2TLPxKRIDg=; b=H70jcr5BfrlqBN0+pLjDPgqW+cWguEilfHAWtClZWlwrUtNDHgD3uVTPWaVR4qRTyY Va4MmaWdThnG94WnKFg9+VVPMwH55eQZzBubMSnUCInWcqoC5uLX7TeRDZm3PvUjdPvp SPN2DokvV7Duef9esDrH+4AWoo1eqM0g9CBMK4IAi/oeU0Rp+XsN28mTbp6rjrVPOFDJ 5okLb05kYHbkMqWZCQX0yo2sD0c3aWc/TPDXa6q3kpZy2VVO8IieM0B0KzfEi882r5tG FXbruWVc3SXnlt7OE7mhNaKzwBJAvP1zarWoJBjwHTD2QCYOmNm/13ZSLnzIR+GccvLh Vgnw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406943; x=1693011743; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7MlO/4lz23eNLFrlyrvCLQF2KGu4x8cNZ2TLPxKRIDg=; b=TL8njucepis+tHkT5f7eaQSrww4JYjXslHUvVGO7wnTBimta+l6w9JyCHN+sX5KH0R qDweF7zRNYFgJEGGo+Lz0z+e3PRMoXmCQURm6mNphTh4fv7bXFhVmIUkG93bYX1aiVjx wEy5JRfn2uk1jkrpJ+XJfA7WE0tW+RyXf7Emszqs82OAjVzeDgMZGvz/yO0hb2uMf+zR JMroVDQP5pn1DsZSdEl260jflGK1WLyQv7pKM7Oo6CGL6ZYKFxR3LDFkI5Oo21qHp5hj Rc/2GD5cCa7/Me2eqiehTn43n5qTaK1UVP4XOjTTVgk0tB0mGtuQLtdBYDx92RYRCiU3 RGgA== X-Gm-Message-State: AOJu0YyxXwoblws6q6+PknbrBrW5Z/3vmHpSfMdQcj1JO2wEV2WL8lG9 538npG355g5AxOrmnd8e/MSkL+0phJ9CAbUujHw= X-Received: by 2002:a17:903:2291:b0:1bc:6c8:cde3 with SMTP id b17-20020a170903229100b001bc06c8cde3mr1000380plh.57.1692406943358; Fri, 18 Aug 2023 18:02:23 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:22 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 03/18] target/s390x: Use clmul_8* routines Date: Fri, 18 Aug 2023 18:02:03 -0700 Message-Id: <20230819010218.192706-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Remove our local version of galois_multiply8. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/s390x/tcg/vec_int_helper.c | 32 ++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 53ab5c5eb3..edff4d6b2b 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -14,6 +14,7 @@ #include "vec.h" #include "exec/helper-proto.h" #include "tcg/tcg-gvec-desc.h" +#include "crypto/clmul.h" static bool s390_vec_is_zero(const S390Vector *v) { @@ -179,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(8, 16) DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) @@ -203,6 +203,34 @@ static S390Vector galois_multiply64(uint64_t a, uint64_t b) return res; } +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +static inline uint64_t do_gfma8(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_8x4_even(n, m) ^ clmul_8x4_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm8)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma8(q2[0], q3[0], 0); + q1[1] = do_gfma8(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t desc) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma8(q2[0], q3[0], q4[0]); + q1[1] = do_gfma8(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -220,7 +248,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(8, 16) DEF_VGFM(16, 32) DEF_VGFM(32, 64) @@ -257,7 +284,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(8, 16) DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) From patchwork Sat Aug 19 01:02:04 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715002 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp800789wrs; Fri, 18 Aug 2023 18:03:40 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGGg7Ux254TcnJuG1kmXdfirLcL+lcThPqpotQ3yELsbRFmXgomabbbiS0jq1Iw6uaslYje X-Received: by 2002:a25:bccb:0:b0:d50:60a4:a1b6 with SMTP id l11-20020a25bccb000000b00d5060a4a1b6mr795256ybm.63.1692407019813; Fri, 18 Aug 2023 18:03:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407019; cv=none; d=google.com; s=arc-20160816; b=F5LHDirnzweKfM0NWEr5+dQdtGNEhdhj9HKcINz/Z6dwH5jd8tE9UGX4C1E1NLKAGD 4HHR0uhOXGsEYCOhDqgL/ynvUITMt97oTkulnBOnHxATGSKniHBbJ7VxYUI/UCisOhwq QknNBLIp+EUi4ceGKc1DvG9WRarfzfc1P+ZxedSLcoNyA1xbYmwk48OUaZPCputQPMWb KRLvGEIJz7lJL4PfFkFqhfs5iBHQ0iUpxcEolQv/b3JbvrFOkCXnol1tvo6y9JUYeaB+ FaWIL9KeJgGfYE7fYeaiutJ/usxnNsFGEzDdo44eShTqoPPaipD+4cjfjIL4b+eiDi95 IuFQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=NYXP0KtEdlq5N7AzdI34nZHHl1ivIDv804PLJER56Mlc4ZSV+qbWa2cTkOSfIVlD4+ 0YZLRpB+xREYO1+saMJvSC0onMFO2Qdl5fhJhEhM8N/jdm6aej1Sb2quctehlHTV5rW6 6Hymgbwc4+3xY3Bc+MUD5rreZ1uJ+qu8DHJTahnxFxyca+VkhqhpCVQ78PuqG1Hlqega MCqXeAuTMt0Z77BhV/d9V8y7yPwOamD0js935AxUBFjVJOiHS4tPXEB+M7eCQGQ3E86A nBgxjQS02QCXXO2s5cTCDT0630w3MCl++eCTBXuOPB9WJ5f8u6Yd2ws0H9/P/50aS9OG XcLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ebjYenVL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u17-20020a0cdd11000000b00635d97cbcc6si2088681qvk.162.2023.08.18.18.03.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:03:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ebjYenVL; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008QR-6A; Fri, 18 Aug 2023 21:02:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMG-0008Py-HR for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:28 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAME-0007LA-3i for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:28 -0400 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1bf092a16c9so12716825ad.0 for ; Fri, 18 Aug 2023 18:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406944; x=1693011744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; b=ebjYenVL0k7sB74DGZWzH/tol1yehlK3e+XLJ+Kr4Ax2TE5Cu5QpgjJ9ygo5ZHp6Vy azAPtQ0vNSVUPk1/i+wHRADaN+e6/D5nTWJ62K5oDwzdC1hxk+nw9dSXwVhfbxhAIlXY zRJJPBtX2J02LJwHlV1necNQCZdAgdYpQp3YrGK6qOeZFAhdQXk1KJ4tJX/JUeJ5VtzD iiFY3XddkrnS8LF4TCZCTnv8auqRP6UxEHew23SjfEtKs1KFNEBRAGJQYBhHTNOe6Kkj VIUH9PWU1MCM+b7PNfgg7YomawTYxOnL74qJMNsY3/8BKSy3WWgkI+5BAuAgKuJkTbi5 4Mrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406944; x=1693011744; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/XAMC2SBnpNGK/nIW2dhBAzm2obzzeFfdL7+o9evNL4=; b=Gs7OHK+tfZH3F5JshyiPh2nI90ru2NVR+u0MOBcIg6NQVlw4cL9oe6lnR6KQLXefkJ bAYpN2weIY4P9kJyhnfwXu49CLy1JCrsr7BHKFuBj+hjNwQh3XSgSkZ5yUKaXCVjQr+G hhhk5yrPzA7XjBSvwrQLArvikrKWQeQsVcdz7vPUcajChlkj9Nb8kHzpYlrJzqa6YGDf IGG9tP9ll/rh1RoDHekDtyBAh6pzhCr7KvQqe2L04zKKX2+PLU81xIBpIgJEOHFPP5wb wzMjpQupgA7CJ6sUg5VOAczuljlDUS9hHGZLrB+ivhC/+DrABmeILtA6tDT4VGCoDDTK /ecg== X-Gm-Message-State: AOJu0YyacT2HmGV9ownmCfoeGr3uHneg8Ag03bH8BAQ0Rbf1smhUhGpp 54xoRhPmeZJrgMwp4GcinC48Xnx0TEKRBJ38U/U= X-Received: by 2002:a17:903:2290:b0:1b8:83a3:7db6 with SMTP id b16-20020a170903229000b001b883a37db6mr980824plh.44.1692406944099; Fri, 18 Aug 2023 18:02:24 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:23 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 04/18] target/ppc: Use clmul_8* routines Date: Fri, 18 Aug 2023 18:02:04 -0700 Message-Id: <20230819010218.192706-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 8-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 834da80fe3..343874863a 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -26,6 +26,7 @@ #include "exec/helper-proto.h" #include "crypto/aes.h" #include "crypto/aes-round.h" +#include "crypto/clmul.h" #include "fpu/softfloat.h" #include "qapi/error.h" #include "qemu/guest-random.h" @@ -1425,6 +1426,18 @@ void helper_vbpermq(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) #undef VBPERMQ_INDEX #undef VBPERMQ_DW +/* + * There is no carry across the two doublewords, so their order does + * not matter. Nor is there partial overlap between registers. + */ +void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_8x4_even(aa, bb) ^ clmul_8x4_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1445,7 +1458,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumb, u8, u16, uint16_t) PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) From patchwork Sat Aug 19 01:02:05 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715009 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801234wrs; Fri, 18 Aug 2023 18:04:48 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE4ZI3aXe38devLw3m6u3COEgMqmyJtrVEwGkrgJnCOrAHzq5BPX3IojFjQRXkpQNiiypTj X-Received: by 2002:a05:622a:588:b0:405:5a1b:2767 with SMTP id c8-20020a05622a058800b004055a1b2767mr764892qtb.6.1692407088735; Fri, 18 Aug 2023 18:04:48 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407088; cv=none; d=google.com; s=arc-20160816; b=atF6NCr85Xi3Jq76JWk+R2/NqfYRplTe25sWndAGsZLx9u6G8T20wA9PuKVgrRtfwA ovc9JdnyMLxfSAKVRTdwHxtPOSO3g2WSlxGSjGNa8dJNV3f4yDqP3TlqHtkjU8w/DPxv ieflGh9jWRklz6PGDfpOcMjdArT+vfXK4VijZjMzDkxfsaunhnRwMEOTQnKbJ1jyUtB8 G3yEmujpObNcaqTs8zUw8OyH+nWARsHk5nHhOXUML3RC61NuPkhl/OogC78IfN/tYf2W FuVnKWzZbHLZhDaEh2hjCowp4rZWSw/WusvMeGeDF36QtsRCIehf4mJuTP26Lgo808Cp bMXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=bxF8lNTq8tt6BcZ7x5yhV73vwjz9AYCbg/+iu63XKGrcYfSW6GFrU8ccTvWmqnZ1kX EDP0frdYKOe3/Eo0O25qIEtFDcojBS8m+iIBjM3t4rwQi5pejZWTTkc6zENui7FNViAc RwNnBMkfYddUE34nDcfWUlPKLKHlEKHVdX+6dOTMx9CoZb8M6SRCJz52yOORAPpUxi/V IddTCS0mtL9n7l2mcepwTDzGmMSjDCWUDuRfsB4RxuXFIsdacnQ/PJHlLlRaVmcSqTZ7 YhyjCMl/zHThrpUyX2WtgkxpjmCetGl4skXmP8ya5ioh3AERFVmajkfS2oVfhy8RrfEZ 5CNA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kql0IMv2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id t1-20020ac85881000000b004035739ed02si2049856qta.198.2023.08.18.18.04.48 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:48 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=kql0IMv2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008Qz-Un; Fri, 18 Aug 2023 21:02:30 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMG-0008QG-T2 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:28 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAME-0007LJ-Nm for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:28 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bf3a2f44ffso9956005ad.1 for ; Fri, 18 Aug 2023 18:02:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406945; x=1693011745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; b=kql0IMv2Yow40XiBFqkHQpHh0ydvj/vs9ypV8Z/oQY4Ivg9SgyQvoVxYNN4BpJnGn3 1VZo/s8wB3fBVFZXAm8zXZaI17twWZ4cs2oHwDrwoSih59Gj3TK27SzLsk8/Q4rKBHCD rrtCzFSYL4XgFk/VUHSoKuu0MleW8/CAiMms0dI2n5M3Zj9rSnCZDqm9GgSLckzqW2J5 nWrfiuRTuZGY3vGg0Nl8WGhpCt41OL+zLFTH6B13ZcCoCKAeaus5ixWwv+38wGYrtatp esXydpEWV6yRwuSGTNYDpLIlYrGcP8UXNC9OenivTpsbfIRpUZWfeb5DE3GLY+UknZYS fwsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406945; x=1693011745; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=u46/76awb7qaVCGpS1Wp9lBTQhA5OFbmzjWjo/GEWhA=; b=ZuntxOqI4rxRMZyxmhd/vIFzycMnoEm4m+JSi5BiX1IRipj1bWHe/vZ6Bx6n4XaphH AaT7n5/DUehb1ti7fwdFO1mrD5aZLZXeqro5DLPnwESrC8lqs9U26bJ1TQik+7yufT4M v0WxFZuo/EZvKs/HI0S3eTKETBVlIdRT4CE9j5Py7xJi0l8LrKnTjsJgyX+/R8pT5lAU sfRw43ZJkivwmZ+CD/ay/dUJ1/g802e70CErmDXPJJOkF61zcyPEyZCqAhLx2MGbQ0dX 0Lw9pbOwTEgqdcqbKd/XrVtTGUp7bvvjatwvJQpuJY6+scuda3SfRMy1vVT3D+AP/VHX ULVA== X-Gm-Message-State: AOJu0YwCi1l8S4Si+N5v83IZKXfqQBTWPwAGZU6vXF4vaj3UZusiNqaL R2DYnhkcuxK9nPPzfzr5jbcp92slR5w15622lPk= X-Received: by 2002:a17:903:22cd:b0:1bd:a50a:e2d0 with SMTP id y13-20020a17090322cd00b001bda50ae2d0mr946722plg.66.1692406944900; Fri, 18 Aug 2023 18:02:24 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:24 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 05/18] crypto: Add generic 16-bit carry-less multiply routines Date: Fri, 18 Aug 2023 18:02:05 -0700 Message-Id: <20230819010218.192706-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 16 ++++++++++++++++ crypto/clmul.c | 21 +++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 153b5e3057..c7ad28aa85 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -38,4 +38,20 @@ uint64_t clmul_8x4_odd(uint64_t, uint64_t); */ uint64_t clmul_8x4_packed(uint32_t, uint32_t); +/** + * clmul_16x2_even: + * + * Perform two 16x16->32 carry-less multiplies. + * The odd words of the inputs are ignored. + */ +uint64_t clmul_16x2_even(uint64_t, uint64_t); + +/** + * clmul_16x2_odd: + * + * Perform two 16x16->32 carry-less multiplies. + * The even bytes of the inputs are ignored. + */ +uint64_t clmul_16x2_odd(uint64_t, uint64_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 82d873fee5..2c87cfbf8a 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -58,3 +58,24 @@ uint64_t clmul_8x4_packed(uint32_t n, uint32_t m) { return clmul_8x4_even_int(unpack_8_to_16(n), unpack_8_to_16(m)); } + +uint64_t clmul_16x2_even(uint64_t n, uint64_t m) +{ + uint64_t r = 0; + + n &= 0x0000ffff0000ffffull; + m &= 0x0000ffff0000ffffull; + + for (int i = 0; i < 16; ++i) { + uint64_t mask = (n & 0x0000000100000001ull) * 0xffffffffull; + r ^= m & mask; + n >>= 1; + m <<= 1; + } + return r; +} + +uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) +{ + return clmul_16x2_even(n >> 16, m >> 16); +} From patchwork Sat Aug 19 01:02:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715004 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp800947wrs; Fri, 18 Aug 2023 18:04:03 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHID2YlUYiXPxJx/w6cW0YE8ojOnnkgsSXRlcvSCY4NBocnTE6fvr51am21yYheW3sXMlgG X-Received: by 2002:a05:620a:4720:b0:766:f94d:b1a5 with SMTP id bs32-20020a05620a472000b00766f94db1a5mr1233912qkb.7.1692407042860; Fri, 18 Aug 2023 18:04:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407042; cv=none; d=google.com; s=arc-20160816; b=k+82L5ORixIjnJOxe21UxbhBMyetVs3fAf7Wqd/cDr4CJtd/LUe6VrkWY0wq7U1Ab0 H9N/iId3MreyXQab7L2ni9pfYlOM+IElgCRGe6RZQwZqMwxvLmpBn5y8qvuXZZZ9IPuZ +XzWUDc4n+5ep+tzQFu2iV5+H+dLccc7D4ZCeszB9EU6YHKwe/iy/fYZxVLevhDKU3XJ GHbE/haoo9OsfYcLmyqSRBF1v5uh2SKcBYEVCf0Ku4/zwl5kScPiaYHH18jR+B3AMt/1 6Uq4TZQnUKzEnHBHNainEQCCC6C6L7TpKM5mimKwwnkpKeyv1mDWmlUjMAURLQ098i1i 2Frw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=BZh7KEWKFGt6A4dyVK13WHinjbZLYVQb3GflKbX48PQ=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=jWdSYsw/ty5xDv8lT70iUGqllt+QkImNvdxVd8lpeTWoPwe36Noo/nVNkl+EoZJHkR xcZ4+t+J5OFxngBF5Lifv8bFSBUG8d59+O1rzZEzXE0V0+jUPcT5a6nsj5IvZEX4lgBU 5W3sRfVRVOQpoAxeLY/AGnIydVnwhAwnZhTcxNQjZ0vxL2GKAg8rMjKNOxvHw+CEDDUa aC2Bf4M/zG6SWV4/gUf0nRthm8iM1MpJ3ydXv/S3ssocyRM94qY6/BbBi/2mJzTZgmEq W/2+H5hTsKGmxNXm8hhdnIeYWtMN316fklJg87KNqkCGQ7GfaTqrXal1sbdPSbQHRUdM SOJQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nHSf3lhu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id bn31-20020a05620a2adf00b0076d312ae8c0si203473qkb.75.2023.08.18.18.04.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:02 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=nHSf3lhu; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMJ-0008Rh-QI; Fri, 18 Aug 2023 21:02:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008QV-F8 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMG-0007LS-5K for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1bddac1b7bfso10763355ad.0 for ; Fri, 18 Aug 2023 18:02:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406946; x=1693011746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BZh7KEWKFGt6A4dyVK13WHinjbZLYVQb3GflKbX48PQ=; b=nHSf3lhu49Z5FSGNDBnw6e94UBjSAzSjIS5SOkXLdo9wknKm4VR+2GUcMsCUmj99z7 FXHp2/FN7gzeM3cZ3Nybkj8vcBRktmL6knVPrLujnfgUtmLjvkeTfj053FfaY2ussB1a 0iSW1ydYc5Uv/iWrQIOMEknhrTJZBph5YV5k7cDf4GQyIqzKgzaaQbkdWIk49mt3xx4Z uWAWdswlGicEQLSPdPJiXgb6SEMrTnOkvsYlT83Qmgas/G80cTcfYZodjQGQa9fZFBgF xugCMhQ/Ox2SYXD3GdxvWBn315sZxc1w5YtTssgEHiKlzfXO1KUV+RrcmLRKfpLRRWIO lUNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406946; x=1693011746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BZh7KEWKFGt6A4dyVK13WHinjbZLYVQb3GflKbX48PQ=; b=Hv9VEYrBAf9jNvPBHNTQ//WKl5G+bJWuD8Zv+QrNIQt0ayZ0ktzhThjLKQNw4FdVg5 hwwJyQ8ZJtq6oLnNp9J3du8qxkV11nTcDas/7GfAOLkmAK9KGJJBYWsuozZNz9BURPmP 759Juo9Fma2cQ6o6O1Te7DyDHyDuUThAMIIc59yrGjGRaDB0g4dzg6jZ7TrGFQuMkDYg OPMrh0hX4VqcU7fs9N61FoGCp4uHfl/So8VgOn1xEt4Glr5fF8Rpfce7dFmxdZaxsf6V wehJcyR2HSHzttzQIP8SXvqEsasQQ0xheJGSUx6u0opytyc//kC6cMFtGbO8BynRfZCL dNFg== X-Gm-Message-State: AOJu0YwL0tYFQo6rA/mAsvOmMpHNNqzVm3kNviy1gVBToZB5edStul8E 49Xm3TahYF+VGOCmBhkCXbgx3n3/SI4g4NxqJnY= X-Received: by 2002:a17:903:44e:b0:1bf:3fe5:5e71 with SMTP id iw14-20020a170903044e00b001bf3fe55e71mr1040104plb.6.1692406945784; Fri, 18 Aug 2023 18:02:25 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:25 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 06/18] target/arm: Use clmul_16* routines Date: Fri, 18 Aug 2023 18:02:06 -0700 Message-Id: <20230819010218.192706-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62f; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62f.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of pmull_w. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/arm/tcg/vec_internal.h | 6 ------ target/arm/tcg/mve_helper.c | 8 ++------ target/arm/tcg/vec_helper.c | 13 ------------- 3 files changed, 2 insertions(+), 25 deletions(-) diff --git a/target/arm/tcg/vec_internal.h b/target/arm/tcg/vec_internal.h index c4afba6d9f..3ca1b94ccf 100644 --- a/target/arm/tcg/vec_internal.h +++ b/target/arm/tcg/vec_internal.h @@ -219,12 +219,6 @@ int16_t do_sqrdmlah_h(int16_t, int16_t, int16_t, bool, bool, uint32_t *); int32_t do_sqrdmlah_s(int32_t, int32_t, int32_t, bool, bool, uint32_t *); int64_t do_sqrdmlah_d(int64_t, int64_t, int64_t, bool, bool); -/* - * 16 x 16 -> 32 vector polynomial multiply where the inputs are - * in the low 16 bits of each 32-bit element - */ -uint64_t pmull_w(uint64_t op1, uint64_t op2); - /** * bfdotadd: * @sum: addend diff --git a/target/arm/tcg/mve_helper.c b/target/arm/tcg/mve_helper.c index 96ddfb4b3a..c666a96ba1 100644 --- a/target/arm/tcg/mve_helper.c +++ b/target/arm/tcg/mve_helper.c @@ -985,14 +985,10 @@ DO_2OP_L(vmulltuw, 1, 4, uint32_t, 8, uint64_t, DO_MUL) * Polynomial multiply. We can always do this generating 64 bits * of the result at a time, so we don't need to use DO_2OP_L. */ -#define VMULLPW_MASK 0x0000ffff0000ffffULL -#define DO_VMULLPBW(N, M) pmull_w((N) & VMULLPW_MASK, (M) & VMULLPW_MASK) -#define DO_VMULLPTW(N, M) DO_VMULLPBW((N) >> 16, (M) >> 16) - DO_2OP(vmullpbh, 8, uint64_t, clmul_8x4_even) DO_2OP(vmullpth, 8, uint64_t, clmul_8x4_odd) -DO_2OP(vmullpbw, 8, uint64_t, DO_VMULLPBW) -DO_2OP(vmullptw, 8, uint64_t, DO_VMULLPTW) +DO_2OP(vmullpbw, 8, uint64_t, clmul_16x2_even) +DO_2OP(vmullptw, 8, uint64_t, clmul_16x2_odd) /* * Because the computation type is at least twice as large as required, diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index cd630ff905..5def86b573 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2029,19 +2029,6 @@ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) clear_tail(d, opr_sz, simd_maxsz(desc)); } -uint64_t pmull_w(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - for (i = 0; i < 16; ++i) { - uint64_t mask = (op1 & 0x0000000100000001ull) * 0xffffffff; - result ^= op2 & mask; - op1 >>= 1; - op2 <<= 1; - } - return result; -} - void HELPER(neon_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) { int hi = simd_data(desc); From patchwork Sat Aug 19 01:02:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715014 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801340wrs; Fri, 18 Aug 2023 18:05:06 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGqJegyx/XNjrE/g175mWRw88xYYSTD07f01niXBpl3yXDvu/f4qb/aFMzSQFxB0iNS3Oza X-Received: by 2002:a05:620a:3183:b0:76c:d05f:99f5 with SMTP id bi3-20020a05620a318300b0076cd05f99f5mr977111qkb.52.1692407106757; Fri, 18 Aug 2023 18:05:06 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407106; cv=none; d=google.com; s=arc-20160816; b=yNrhOxGvsXpNIEmCobZXEXjnx4w5NGXpRJrJtPIFzmuLsFzH3gZkJvkb4wjUH5ohZn mgHMXgrm0fEq9z6yy/Rig4FESg0f67EZ6fdJ6KVjCP3kHsEe+s18Js5kwnIH+Jsk8UBT O/yisP1JdAjJ7yfyGCUSHJv44bG6bDjpQ0tXDFYIZaFsPPu8FcHP2Unzzitk7y8Gj00j 17BaRfWqqN2ZYpMoBa3z7zgA6QE45wDRgehT0vDbZYpGg1AZ8aNowWdBEdYdoKrKyTZw uT59FJYy8ZCxtj50cNGlfbGqv5L+oFtZFxbZzp9g5H8xVFG47DLyOPUPfRjjM11tRiAW 8WCQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ZRvobRxifTOwS5r7R7zXbKsRuZh6imu3x1ylIR15QTo=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=xXgd445ZkDBAhH+iXikDMHBtMbeW1fEhjo96+v5fLwpWQrc23QCIg7QJq9YC6//lcj BcsPUK01Auzinf3q4blpkHKRw60w29Un3qze8dQGuZfHterGMTFiXwNPKeEFtASpm60v dlV4bRcPKmGHjUOH5JGEX8jgcg8EN02J4PoK25ocfRJZFoHZDkNFFNyOGFrivsZdkzky 6hGsVT32Q883nTcKb+GD7DfcZXjAD5R38vpoUdd0/a/OYgYbrmgaF9xYMMdd/S8qYCnv sGFmeq0eyqJ+3hzoiEQtz7ClNflVrSv4dsYqihiGrEhciMdaFjmDKkPOUex7EofJINOB KJvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vSko8l00; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id i20-20020a05620a249400b0076d8644a47csi1931561qkn.433.2023.08.18.18.05.06 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:05:06 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=vSko8l00; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAML-0008Sj-LN; Fri, 18 Aug 2023 21:02:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008QT-Az for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMG-0007Lf-1a for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6889078ee66so1257382b3a.0 for ; Fri, 18 Aug 2023 18:02:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406946; x=1693011746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZRvobRxifTOwS5r7R7zXbKsRuZh6imu3x1ylIR15QTo=; b=vSko8l00lJ/TbMwRXhOCOhzuMltviL06MrhXmbaZ+WA6VbtgxNFy4qNtBdn5AcXqPj zY06qybaC4EC3+HvW8OM8cA9TFgcQfE7HTxbKu+s1tV0/9etrIbXocr2MUHMBitX0yuB XMD6D/DXPDDzsSwYFB4k5WPzkIJdki4I7/lSGFyW2EYemGgctdPIM/sbeX0K6IaXhlSe adR5pOXzBUJObFqd86n2dWDMreKNyLBtSPXM9kqEEkzGtGN0DA3NP294kDr8mQH5Q3yI 8/ZxlHiPxltspoAwp0Pl78TPJgi79ocr/dY4m6bxxOctdxIW6peLObg1z3ZMXWctvgmy Ijug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406946; x=1693011746; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZRvobRxifTOwS5r7R7zXbKsRuZh6imu3x1ylIR15QTo=; b=gOVdQv0FBLZQOFzDESr3SOBGcDY9ua8P05n/I6J+ARkWmAWfGj+a/NizsERQXyneAx gZG/7688w/FV0vi72oUVWUtS2geIYxznzR43MS5jpBajHsndVtFf7gTnXlv4F5lOsqEs E/YundQMy1t+Km7eLQuqMcBt8ANLlmunicddT9c+lpuIbA2uAi5bM2VYhey1ouzON+DT n2YpIHGeRsmiuZrdyQqCa5LjXIWMM/VyF8i9bGRoXvr/yltRnAeY49FW3f0pzz9d1wVb tafmcBFnRC0dXZgLuAsYYSjf3QaaSkeQM2Pba5TWgkRwhD+Y7INF02AZ1PjOsEHFHSWD 7S5Q== X-Gm-Message-State: AOJu0YzakWh4vXmm7UulJLolXmHW0vdf7IIEy6uQDQPDPc+s1MnOXW3H 7DzsnNlTuPz9PbDChls0uscFmFpwFT4HB2R2sc0= X-Received: by 2002:a05:6a20:8f09:b0:137:2281:944f with SMTP id b9-20020a056a208f0900b001372281944fmr916565pzk.40.1692406946554; Fri, 18 Aug 2023 18:02:26 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:26 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 07/18] target/s390x: Use clmul_16* routines Date: Fri, 18 Aug 2023 18:02:07 -0700 Message-Id: <20230819010218.192706-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Remove our local version of galois_multiply16. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/s390x/tcg/vec_int_helper.c | 27 ++++++++++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index edff4d6b2b..11477556e5 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -180,7 +180,6 @@ static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ } \ return res; \ } -DEF_GALOIS_MULTIPLY(16, 32) DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) @@ -231,6 +230,30 @@ void HELPER(gvec_vgfma8)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma8(q2[1], q3[1], q4[1]); } +static inline uint64_t do_gfma16(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_16x2_even(n, m) ^ clmul_16x2_odd(n, m) ^ a; +} + +void HELPER(gvec_vgfm16)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma16(q2[0], q3[0], 0); + q1[1] = do_gfma16(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma16(q2[0], q3[0], q4[0]); + q1[1] = do_gfma16(q2[1], q3[1], q4[1]); +} + #define DEF_VGFM(BITS, TBITS) \ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ uint32_t desc) \ @@ -248,7 +271,6 @@ void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFM(16, 32) DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, @@ -284,7 +306,6 @@ void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ s390_vec_write_element##TBITS(v1, i, d); \ } \ } -DEF_VGFMA(16, 32) DEF_VGFMA(32, 64) void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, From patchwork Sat Aug 19 01:02:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715011 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801254wrs; Fri, 18 Aug 2023 18:04:52 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEub/Qj4KbzSiDPYzIlamOE0eUf8bdwusrYk1DFICNtU58L6fHVDh5P2SAGstd6tPH2yXPP X-Received: by 2002:a05:622a:293:b0:403:abf5:6865 with SMTP id z19-20020a05622a029300b00403abf56865mr1258741qtw.18.1692407092336; Fri, 18 Aug 2023 18:04:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407092; cv=none; d=google.com; s=arc-20160816; b=B9zBCQArAuMpCr0fzx9mkk49bQC9q8VpBb0fUeiumU3i9wyF/CTbt2mowfvIYuAwak TtngjckSASwzQK/ntA+TIpvlUBfnvjhkYu1zVQ+1jFMAi4iGVpnmPMf52tYU5RMsVSGC ucMxmkbcLUArRdqyZ9sTe7UeOUdgzUBKoXnNvMf9wHJBlsTbShxCzS9DZniDkXCl0ixO 4NxW2tLmEu2irPQPpyEzjjOpvU4X/Q1lxwygJTKiXQlo3AIWTmh1KCpVv6vfpf1m7fkP SyYUJ0itjfuFClczz0/3kwfwpMWd8+kIsbt+H53jacMmKVpwQM+kTYiFLuQxBBkevy8H iYCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=L/6MN9as8lzqPSDjQk1xR47J92g7S+YZpuyD5tzU7zAU9rPnAYfqSzbEiRIsx+U/Ai d7KmwrzU9erghb0aknJnnnQ4rV7ziAYt6/3W9A9sNq5GfFutiIur87K+qDWQV4gOmylN u1Bb+sujqwpJWx9aFE+X1/CIlja3uieb9WuqniGLDXnBuoOw4Y4ggymLHzbggEzYgc8q dyZtFhac2xmhJZsqSFbUQn6EPxWaEME/PiLJTx6tGqL4abfmR3Op2KXl3HoGrW3KjF/9 xkzUgc9OVWAcSczUwIrSfVheYhYRYAHZIazTfmosOC0aFzEzMp+hCHfyPHfJK6+TPuAM +a1Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bNuLoAkM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w21-20020ac87e95000000b0040357313971si803161qtj.662.2023.08.18.18.04.52 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:52 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=bNuLoAkM; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAML-0008Sg-2W; Fri, 18 Aug 2023 21:02:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMI-0008Qw-Pm for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMG-0007Lz-JY for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:30 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bf1935f6c2so10615435ad.1 for ; Fri, 18 Aug 2023 18:02:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406947; x=1693011747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; b=bNuLoAkMvsnk7IlF5GA67UzN/sLQwe5Mpxe9SIjU1F9mpw46gaCysXmsTmVYXCWXP8 7BHZ9Crmq0DU6u4FTw/XYneQGMirkwXGhorTKEvs03efRPFkR8M6IwCC4jXf0JK9haMN RLOXKXr/comzzR4dp2nnxrONt5FAry34zXlihy7/Mmip2oQKN8IylOSqdadgftI718Sm 0qkxkVZmokAioB6Y5PW9JGuzy2VWxZZLhJS0ksZV8q3ifGLiLFvvJKjGEME1bkR1NimU Bp5lzSivTHkXrDDyVx9xs/BS+RA7Z/p0P2F+eZa1jlxoJ3EhCPtBHZvHRghHULU7nlSJ 9dVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406947; x=1693011747; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m/tLt6CaJsWtmfAucUsswRTzLCX37AmxqBCjKQhyCi4=; b=My/7vNv+ACl3ZCmVAznlWtswDHevJJeUqnONIjoUVxFMmqQ1hy3a4v1g8qUD1BIeiR cuU7JN1Em/fit/EOLS3j3l3RXdRvvGFxhhXFp6Th8RvPqe1ga5w375IXizUkB8OEjlwr c/Q5qKzohf5uja84mif1lsYE+H5FYTw88u94evzLIhHsz2aSRfmmzWbjNMODYTSNR0G7 VbtLcwh1UFCPVwlVKe4w/o9VPIGRilVjnjCj6Ov6J9NC9KxtpJPKEpiTDLhEdSx+fU3/ KgnRfEQrNAvnkohMTlvfqWOIws0HbkF9qtyvitSw+Bk52S7nv3JS22+ij5eZXvPDLJgV cZ0g== X-Gm-Message-State: AOJu0YwKV+TgEBMmFxAWc2Xq86xJ2MIUjuoamPokGZQlEkOtzC7J1Tnz uM2H3QSplUb061AWj+GXnGX0gq0EzYfJDCbgxNo= X-Received: by 2002:a17:902:e847:b0:1bc:7e37:e832 with SMTP id t7-20020a170902e84700b001bc7e37e832mr1257594plg.19.1692406947432; Fri, 18 Aug 2023 18:02:27 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 08/18] target/ppc: Use clmul_16* routines Date: Fri, 18 Aug 2023 18:02:08 -0700 Message-Id: <20230819010218.192706-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 16-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 343874863a..10e19d8c9b 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1438,6 +1438,14 @@ void helper_vpmsumb(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } +void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_16x2_even(aa, bb) ^ clmul_16x2_odd(aa, bb); + } +} + #define PMSUM(name, srcfld, trgfld, trgtyp) \ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ { \ @@ -1458,7 +1466,6 @@ void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ } \ } -PMSUM(vpmsumh, u16, u32, uint32_t) PMSUM(vpmsumw, u32, u64, uint64_t) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) From patchwork Sat Aug 19 01:02:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715013 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801280wrs; Fri, 18 Aug 2023 18:04:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGOrMxALoN36sIsky0AcXB8RWybcDEfxJHj3GiWij0ewGQ2+y1R1srWBa7XT4Ma4wEmhboD X-Received: by 2002:a05:6214:5ecc:b0:641:8ba9:68ab with SMTP id mn12-20020a0562145ecc00b006418ba968abmr815185qvb.42.1692407096480; Fri, 18 Aug 2023 18:04:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407096; cv=none; d=google.com; s=arc-20160816; b=CpBMmAt88VKHrXSyODwz9T8HutVG7HkxkbgSRGGLXtRMOs3NhRsyikg3Wwp0UeetlT MP8Mj6CcHojroxh4zqSsOFug7JdgXHDci2RzFppLmsNyCG0PxF4Pm25he5vmQsoNQxNN jjhPO8GmPm4fOPhEOJQs3lE7x+ZLlWD2LgQwEsW+aLPtxiZZSsShVBPvTTv6FV6PiR0J /O/53MxOLqxK1Ob2HlVCBrxyvoOyStM+e+23M2lpPi/6Jhqu74ZchYGbTVfNdp+DNotI xK7ncbDCCI8jKggVO8iHxVvpvn3D25UBw4avzSb27wPBs55mid2tYLh503BcJZf7PUyG CiYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=YG9y8b2Lx3GV+yq15Q5VxObM4Wt3DkUarvh3MEuvUIE0Xe+cytmO2ZN7lZ5tiILPd1 d/lPwiY36oksb2mj2WMxtEZQ+velyDl33QF7H81PDHCP2BbArpxcUjnSts2XxtNrChME dBz67YX2FVxYhW039IthniRBRctzmuTzT5sOYMQ0yN8yLpxp4ta0roSxmILNSEz83qRD 98e6KSHOmF8f8aej3l68N+NaYQCBndXR6wdoql2gZTHFNoZGghxRHldXRAXCX0rFntAW Vcje0JnQA+IWUWO/jYxJhijgwi11IH+Z+sKuV3xDZHWsCmTQVDNjNDLdOMLv2V+y8gFE rswQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SzIYJkjo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id x6-20020a05620a448600b00767468da8f2si2029135qkp.350.2023.08.18.18.04.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=SzIYJkjo; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAML-0008Si-HK; Fri, 18 Aug 2023 21:02:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMJ-0008Rj-OV for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:31 -0400 Received: from mail-pl1-x62d.google.com ([2607:f8b0:4864:20::62d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMH-0007MD-Hh for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:31 -0400 Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1bc3d94d40fso12489645ad.3 for ; Fri, 18 Aug 2023 18:02:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406948; x=1693011748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; b=SzIYJkjoisy0WLLry0ywCtUCLnDiVrgW1pnm1NuMQSwe1ZDHuk8B7Wsk2XhgiGdpa1 oiBQvm4ipIb/ZtYwXzgKfcWNpAsFxtysnknB0tmJdJbFWxyTe4b0Th1a/dHl3G4De9Ei nLYTCgHKF8UUGoP5kxkStMFdyusB112mRQMVVs9TJTXafS1EZesZptwqNLGleAoaDJT6 benQoqmMXYJh8/R5rjJureg6wSsj66tJ6X8i/BF0FSULcfezgf5vd+COqzmNBLC6p0Rm fzQU/Q6LN68a7rBNvBDmXRdP+XsZOqUVG8BxnjDeMr463nWA5/NYjshj6IVjLCxwF/1w ZqLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406948; x=1693011748; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OchzeURE1BgR8OyI+34sloHIxez963wM6RFcZPAc55w=; b=h27oTEn99FlyR3aft3KuBauaxfGYJsh5duVahDuVg6P16M471JUJJxOxDgvooy9dsM /owCBdfY+KruYNNLXT0yRP0RSKTmSjHPTGlYeYKrHkrwK70m9jHpoUEIDmA0wdc9tait oJuQTqeL9alevtBUGY4WE1huupf4TLvEbW7JE+tvwFuNCkPyXcL5zgSFUpB9HLHJLJZL rbdw0rvBdyNf2fWo6xTkyQn13qA9Yv2YTgnEdJhtd7q665WJI9i91m8SwzeDU7KEyYi7 nclMmBMXh3eogPJV45snYLqXRhY3B/P/T4SQDt2VTUutt7n/va6UlBkEzbsXN1eVW77F WStA== X-Gm-Message-State: AOJu0YwyYf8iY+aTqPmQ6rURp4v6Zl7R7ibot/givHfGhhMQh2+rERLx I0eugrwYYT/5BlfQeGgFQuJZivS/B0tNu8Arnrg= X-Received: by 2002:a17:902:a5c2:b0:1bf:41b6:938e with SMTP id t2-20020a170902a5c200b001bf41b6938emr822326plq.40.1692406948260; Fri, 18 Aug 2023 18:02:28 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:27 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 09/18] crypto: Add generic 32-bit carry-less multiply routines Date: Fri, 18 Aug 2023 18:02:09 -0700 Message-Id: <20230819010218.192706-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62d; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Ard Biesheuvel --- include/crypto/clmul.h | 7 +++++++ crypto/clmul.c | 13 +++++++++++++ 2 files changed, 20 insertions(+) diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index c7ad28aa85..0ea25a252c 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -54,4 +54,11 @@ uint64_t clmul_16x2_even(uint64_t, uint64_t); */ uint64_t clmul_16x2_odd(uint64_t, uint64_t); +/** + * clmul_32: + * + * Perform a 32x32->64 carry-less multiply. + */ +uint64_t clmul_32(uint32_t, uint32_t); + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 2c87cfbf8a..36ada1be9d 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -79,3 +79,16 @@ uint64_t clmul_16x2_odd(uint64_t n, uint64_t m) { return clmul_16x2_even(n >> 16, m >> 16); } + +uint64_t clmul_32(uint32_t n, uint32_t m32) +{ + uint64_t r = 0; + uint64_t m = m32; + + for (int i = 0; i < 32; ++i) { + r ^= n & 1 ? m : 0; + n >>= 1; + m <<= 1; + } + return r; +} From patchwork Sat Aug 19 01:02:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715012 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801275wrs; Fri, 18 Aug 2023 18:04:56 -0700 (PDT) X-Google-Smtp-Source: AGHT+IG3QLlQIR5lIDPvQaepUgYhWWoHzVH1yagdUBbGNG5FFN3S3UgDLqdRkhZTrGsMIPFjgd8u X-Received: by 2002:a05:620a:1338:b0:76c:c1cf:7a35 with SMTP id p24-20020a05620a133800b0076cc1cf7a35mr798129qkj.46.1692407096093; Fri, 18 Aug 2023 18:04:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407096; cv=none; d=google.com; s=arc-20160816; b=Sv9/LVjylkFvUU0lFHCWScjr9arMmD1fYyNZ+EB5b5lnTCSAjLbBS9pTY6K3ZEbLaC KtEpYGqgBTboLzfaLoCEK9DZYFTfYnarQIqiaPN1yTzhnSOcjjei1kh3QpK4VXJUOvtZ NSf4pHyfQTPDzwKsuF0h3o3SdjR92pZ4gODBb/vBQ330HBHU3Hqjmv2pGZCnG04ojRWf hbv7sAz8POyJn4EMi3fjhPppo63ACnM2BN/wAq/dV9SifBwn19Fqr8DdeZLWFzQAteVB CjAUsoMaMXDmxBZicTXgJxnkbjFLgoMPwSkpS6720mnGa42xntiaR2rHqqs91DnGmg2U yrJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=TypTZmAqIsrHDgQCTA3GQ1oQT8s9Tg4Z3ciGBNGUTVA=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=X6Lky9tPg3WZO5T+tvXd4wpxzShNoQEjWfpmF67iwhEA2vRuZ9/bAfJH3hByU9eruY 6EFN9A7ssOqens5lVXN1/ZwGyiK6uCGjzRUdGqffnacp4uxDirnf0bbcuTEKAqHE686j 30Gv53lo6NFvBHN2o9oIV2wavYJZUGKffbvtpkgKfebLJeHYeDRaPipSd1e859GzIFTc iKcheS6MEGcefPNm0pnwCGjthWE8CzD6NYy3YusgqIMFe5S5A46q7FquchKciAiHbYBc lgw+/+hiv8zIUwez2ewR5/sApDr/DKSvbeNrHkDUU3tN5hY05cEJS5Z+NGeqcgmHoQF0 ebJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RTcFweoe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ox19-20020a05620a829300b0076cdc1a072esi1786414qkn.320.2023.08.18.18.04.55 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RTcFweoe; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMM-0008TP-C0; Fri, 18 Aug 2023 21:02:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMK-0008SY-OW for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:32 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMI-0007MO-Ho for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:32 -0400 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1bba48b0bd2so11377355ad.3 for ; Fri, 18 Aug 2023 18:02:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406949; x=1693011749; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TypTZmAqIsrHDgQCTA3GQ1oQT8s9Tg4Z3ciGBNGUTVA=; b=RTcFweoeFcCB3o6g0PECxilxkaXy7/sVahUCD4vC7GmQmRYUd1jpTkELTQv/OajolF /MVK1e6lbHz/TiQZUZ23xiwolYGd39j6rJSdnuf2hcQ0RyJVYbiaWDo2DT6FoeA04RHG N0BZWY4HBDRHhf2clEP5cw4dc9Ev7bDCitw57xSZUgbY8yCpAlCE9b8SU+CDjzvqdqm6 AaDDi4v/nydiy+JFHRssO3APGM8PZDhrJ4A5iHS3HQz21dCFUlpy9daEHYNdW550+AHR KbFzHXKEyyHjk5PJRVBYc4N8m2kRO59GAHj+V4ELxOooVx92RvpAowCtegKWJFCdZBAF wy5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406949; x=1693011749; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TypTZmAqIsrHDgQCTA3GQ1oQT8s9Tg4Z3ciGBNGUTVA=; b=CECRJZC4fyhOfPCQedUiZwUMZz3out1GMQlgVSVgCpWW3NN9vTPdQId27Vil23QYUK FC6VMDLJcoXh6OfYci90kZqPNWBx2YyctFwTGJv0S748+4Dq7SoXOODeGtxLe0pOb3Hr ltLOied4iDzeOR60rnYwosd2Z/my+DRqtOX/Da8cnXsZRNJXvMveQBDYBiE7cAxNWlBN 0rmizuIwLdzkazfC+cPHWb1vyDPG+EIvoL93TNyA2X/8B2N+rDMcwKboofYrfl0G2f+8 D/FG7voTwSvUZax7sJvWwaQdhQKzeK47kLifA4mO2Pit8dwReaCjX9juhcKVRwzidpQq +BYA== X-Gm-Message-State: AOJu0Yx3mfMVWRZH6OdzM7w+uq74djyy9QnaEfDm72bboq9Y9APAVjy/ DwJhaKL2BfepJxGBo/eLMaqcgsiB3ahRSu8eq/0= X-Received: by 2002:a17:902:eccf:b0:1bb:fffd:63d8 with SMTP id a15-20020a170902eccf00b001bbfffd63d8mr897436plh.36.1692406949152; Fri, 18 Aug 2023 18:02:29 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:28 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 10/18] target/arm: Use clmul_32* routines Date: Fri, 18 Aug 2023 18:02:10 -0700 Message-Id: <20230819010218.192706-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of pmull_d. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/arm/tcg/vec_helper.c | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index 5def86b573..ffb4b44ce4 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2055,18 +2055,6 @@ void HELPER(sve2_pmull_h)(void *vd, void *vn, void *vm, uint32_t desc) } } -static uint64_t pmull_d(uint64_t op1, uint64_t op2) -{ - uint64_t result = 0; - int i; - - for (i = 0; i < 32; ++i) { - uint64_t mask = -((op1 >> i) & 1); - result ^= (op2 << i) & mask; - } - return result; -} - void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) { intptr_t sel = H4(simd_data(desc)); @@ -2075,7 +2063,7 @@ void HELPER(sve2_pmull_d)(void *vd, void *vn, void *vm, uint32_t desc) uint64_t *d = vd; for (i = 0; i < opr_sz / 8; ++i) { - d[i] = pmull_d(n[2 * i + sel], m[2 * i + sel]); + d[i] = clmul_32(n[2 * i + sel], m[2 * i + sel]); } } #endif From patchwork Sat Aug 19 01:02:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715018 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801422wrs; Fri, 18 Aug 2023 18:05:18 -0700 (PDT) X-Google-Smtp-Source: AGHT+IH74zkUcKw9aZOJ9ihHaZr2GCXJW9lv+0gKliL38LVDPQK74tfYQxc17+nCYIIcrHOrp70O X-Received: by 2002:a05:620a:2a07:b0:767:df5f:c6b9 with SMTP id o7-20020a05620a2a0700b00767df5fc6b9mr1079594qkp.2.1692407118593; Fri, 18 Aug 2023 18:05:18 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407118; cv=none; d=google.com; s=arc-20160816; b=S9Y6GKQNNAWOBEF8xCyPQebvrTaDQfFgtHG72Jc2NHuUvR6FMA5L11U30RDKLNMmSF h+K8ls5tfw9znM4Xr1MnhdMnboeJVF+qbYRg8yvUkWWq32bLiJZ2hgfYtXy1v9o67xsC FcHLBTY450w5ll6/NW1Q6xdQtCOwsroykwRT8M4601D+BVFBvGI7+3jwPzx9d3hKm+Jl uKHblmA9lRIoP0WPw/kegs7XFNVyDitMa+AnWzjyVRAq58dz5MQfjTVhILNlQKBoid9b BYeO+87GQKR4e1Y7MSZxYgiFFRjFCnwqt+G3h/JDIsaR2DejT61a+1WOb6YWaTu9ef4S CH5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=K0bXa9+pwNK5r/+j4mMdNNnTuQ3CG4xPOoFlK3zKTy0=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=nFDzH7lVXl9NC3RDybxMncsPR2pv353f5p+IQDou508q0Rp8QXFIyohO2+RF64RaQa q6cqvIGr+buSQ72WtklN+1lOTEL/FmtPmOkxg6+B2CiuhaPldrsO66XdgV/oTyZfyR4U hJGn9Yh8H15x2rQesDDosoGsa+P5aD898CFVzPcVVooWsWQsTxfKXGqjX3D2Z8j+9Pih a/pbkBBieDJG/jW7a6h86c6TFRffs8u/+SC/BwTpjkiKg7SK0MGVwsBXNn3dVYmrhdWK t5X3BMdR2T9jHMn0t568EugfPn9nC3d/oUlWonZAzPCzTgLn4uwWZRKPgJV6aDqXg3PJ 2TTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hBu6Xb06; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 9-20020ac85749000000b003ff1fe3cd40si1990917qtx.162.2023.08.18.18.05.18 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:05:18 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hBu6Xb06; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMN-0008VY-Av; Fri, 18 Aug 2023 21:02:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAML-0008Ss-Tu for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:34 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMJ-0007Mn-Hl for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:33 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1bdf98a6086so12558985ad.0 for ; Fri, 18 Aug 2023 18:02:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406950; x=1693011750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K0bXa9+pwNK5r/+j4mMdNNnTuQ3CG4xPOoFlK3zKTy0=; b=hBu6Xb06heDXO+hqL+WwmKwwyhJY59ZwFziIyGTJ4LGcBbAscDgTcbMh4Lup8iSCT8 X1vwT5S9Y8ArHQbCtSYcSeoDmB1i9Lkmi98uWBewYzasY+oZnEHM1zoaFhKZrLQkMQGS JZ81Ju2O5f+Gj5UfmUA9GGuGDjr5gI6WDk0NTu+8PnblTqdqfHHEcih9CIjJBH2rk+02 3kEXJwXscIZ+efiYw9BPvaVgGJBmDevK2Pr9U7nDgEqRI5Uil+PDn3bMU78KMRR5OYAo KZEaNHOUzG6kp8/vNsp1lBexEQmgLwAHLviZO0IutF/0InXqZyOPE833TyNTUY4wt6Vn Gr7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406950; x=1693011750; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K0bXa9+pwNK5r/+j4mMdNNnTuQ3CG4xPOoFlK3zKTy0=; b=P9jH1XhOBSSIkuNgm+5blNfffkIHgEmdXz0qJIlvvVAWKi/OTpDAQ1UebpSDJ+fuFp E+30xHO2bDdOcL5x/xsvL277kAxGU9EIzpAev2PN2VU7jXYJNQf61XpmiXFwe3QjNt+B EsdfDrxNOm+oRULpzR0Oa0gsALnu51oyBSleASCiSc096dzyI4DyFimPyB1hu1WS9ckP 3+gpsriLkge2SYd6hBqRdlqV281gvnWfaWeKTS/xXnCB8D3C9IM2bZkVL8WkCZeBEdXA BsqUCUXoVAD3/UsvcWc8aUBt1tfG16q9j2zMUU2k8u0RMpqPDcOINp2BC3jl5dkE/mfA weXg== X-Gm-Message-State: AOJu0YyZBQuEIYO7ZWwxaZxh2aR2lurPYGO10DXFr1J++y1pJpIYkQcn b92ydSuNC/QsLSywutpamv6zF7ojTodxZ0bl/xw= X-Received: by 2002:a17:902:e984:b0:1bc:14f0:b76c with SMTP id f4-20020a170902e98400b001bc14f0b76cmr868598plb.65.1692406950328; Fri, 18 Aug 2023 18:02:30 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:29 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 11/18] target/s390x: Use clmul_32* routines Date: Fri, 18 Aug 2023 18:02:11 -0700 Message-Id: <20230819010218.192706-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Remove our local version of galois_multiply32. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/s390x/tcg/vec_int_helper.c | 75 +++++++++---------------------- 1 file changed, 22 insertions(+), 53 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index 11477556e5..ba284b5379 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -165,22 +165,6 @@ DEF_VCTZ(8) DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -#define DEF_GALOIS_MULTIPLY(BITS, TBITS) \ -static uint##TBITS##_t galois_multiply##BITS(uint##TBITS##_t a, \ - uint##TBITS##_t b) \ -{ \ - uint##TBITS##_t res = 0; \ - \ - while (b) { \ - if (b & 0x1) { \ - res = res ^ a; \ - } \ - a = a << 1; \ - b = b >> 1; \ - } \ - return res; \ -} -DEF_GALOIS_MULTIPLY(32, 64) static S390Vector galois_multiply64(uint64_t a, uint64_t b) { @@ -254,24 +238,29 @@ void HELPER(gvec_vgfma16)(void *v1, const void *v2, const void *v3, q1[1] = do_gfma16(q2[1], q3[1], q4[1]); } -#define DEF_VGFM(BITS, TBITS) \ -void HELPER(gvec_vgfm##BITS)(void *v1, const void *v2, const void *v3, \ - uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ +static inline uint64_t do_gfma32(uint64_t n, uint64_t m, uint64_t a) +{ + return clmul_32(n, m) ^ clmul_32(n >> 32, m >> 32) ^ a; +} + +void HELPER(gvec_vgfm32)(void *v1, const void *v2, const void *v3, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + + q1[0] = do_gfma32(q2[0], q3[0], 0); + q1[1] = do_gfma32(q2[1], q3[1], 0); +} + +void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, + const void *v4, uint32_t d) +{ + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + + q1[0] = do_gfma32(q2[0], q3[0], q4[0]); + q1[1] = do_gfma32(q2[1], q3[1], q4[1]); } -DEF_VGFM(32, 64) void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) @@ -288,26 +277,6 @@ void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, s390_vec_xor(v1, &tmp1, &tmp2); } -#define DEF_VGFMA(BITS, TBITS) \ -void HELPER(gvec_vgfma##BITS)(void *v1, const void *v2, const void *v3, \ - const void *v4, uint32_t desc) \ -{ \ - int i; \ - \ - for (i = 0; i < (128 / TBITS); i++) { \ - uint##BITS##_t a = s390_vec_read_element##BITS(v2, i * 2); \ - uint##BITS##_t b = s390_vec_read_element##BITS(v3, i * 2); \ - uint##TBITS##_t d = galois_multiply##BITS(a, b); \ - \ - a = s390_vec_read_element##BITS(v2, i * 2 + 1); \ - b = s390_vec_read_element##BITS(v3, i * 2 + 1); \ - d = d ^ galois_multiply32(a, b); \ - d = d ^ s390_vec_read_element##TBITS(v4, i); \ - s390_vec_write_element##TBITS(v1, i, d); \ - } \ -} -DEF_VGFMA(32, 64) - void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { From patchwork Sat Aug 19 01:02:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715007 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801083wrs; Fri, 18 Aug 2023 18:04:25 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHcVSXXDOHuyiEPFcFg96Zucju0pAq3dB8qDc9qHjvWQpgPnJp1ZktzDqA4e8fJ0rsSiOkq X-Received: by 2002:a25:c7d3:0:b0:d07:e56a:7423 with SMTP id w202-20020a25c7d3000000b00d07e56a7423mr938550ybe.17.1692407065552; Fri, 18 Aug 2023 18:04:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407065; cv=none; d=google.com; s=arc-20160816; b=RZxmMCyWdTKT+M7i1ZATGLRzrxV7r0HPpegp1MG00z2hcPsE6RtavWDWtC80BGf/Kg BlOuz1j0DLyQInhPdI6FPsOJ+d1pOsW0di24FoD7+uy1BTv7dv8UtCEptk4PV69ynuzd vQIJ7/8EogjUtZHKnb1Onf349lmnFuF03V+VhP4SPExmILLPTD3nx/KyASDuKSoj97oA BiDk6I87tnE9DVvBAUoaUQIeITqrykHd27LgBzWLMB8oK7PO/8fCFvqY5RgUda00e82Q MzgiYTe6nbvQKMPNl0X8dQiNLwyrlcdkfbqkzJsabZb43qwQWPfFrLdhQxgWQqB9PtVr baDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=rJ1wb4vlwFw889RyC47BqL/x55IYK7X1qi4QU5s3c05EIMr2Iv93jY+p48e/TbzTNy S4F99v6uQGNNNjb9FmDGKja+0t5td3t9PVRPcyxvepAyP8p/86wf3D4TFK49mcnb40RI 1cS/xnEYeUPuHsjZ37b4obtUy0Gx3xuV3gCKZgAncYS3LFg6Rwz4sGVGHDRYeLJIV9T0 wqcCw4wi9WNoyGcpAZLjlG9466MdZO+TtfQLIb/IiEMoOokdwsQcr4jDs6z29O8LJLrQ yqPsFDF9wmazWXjjJhhBVs7ZET8Yus5K7FaI/36ortuGiwvqYEI/L8OBV4YQHRW8uF5x rwsg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HsRBblDc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id fh17-20020a0562141a1100b0063c6f79f586si2098852qvb.520.2023.08.18.18.04.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=HsRBblDc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMO-0008WB-Du; Fri, 18 Aug 2023 21:02:36 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMM-0008Ty-P0 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:34 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMK-0007N4-Fa for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:34 -0400 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1bdcb800594so12031905ad.1 for ; Fri, 18 Aug 2023 18:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406951; x=1693011751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; b=HsRBblDc/DT4SvE7vRSfwkYQa4f/+zjvf6D7JWCOFtStkhNYMpjuEyB0mw+LQN0qjP 0DWfkfIfjJYBOO4zHOtMz/PPdMu6UDS/ws11fhrb49+yqjp+qeycmVyqQiPY2TfwX9fR NHlDJNpQR2CIbxOkTcSuyYWNZ0maXKIcTR0dLk7JgCrK1E1+DuKuIMRXAyK2dqWcb6xj vCYMFyCQySa9izxGjGKRL4GXel2tEPQ3yIMDrU3Kpo6JnzKlqEcNLR2qrmX12AbyyMaJ PjPaVQNoTPH38rluohmYmJa4njKcdny/91HQ9A4EROFBNUDldr+QtAAYY2PrMzeRdL37 p4FA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406951; x=1693011751; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=47nFV1aM9JE1wMIAXCDJba3wZ1tlxMIcg3CYQKdGdcE=; b=bbFzD44VMd3icHWO6Ihg6CoX5YvM5kEHj+yagRRcfTuZK8enV6JFBMdK4mVLIQ939Y SEyqYMZ4wE0WIU24sewBjFEP6u79+c+RXq1G8s6uCXxCctFFNHYDBi0rYur7arZhlc09 XQwzZ/C1LfUd8yMCYH0ORxTvzqzsUxbknC6lqEotZb/vHVLaLcHVzLyB5RH+e/4xQQ+3 q3CmwVzF/M240jeD8I6Y7WvyW9pHvgD5VJ0y9RMSq4gNVczukSqfI1EQFq7iEevwQCKX imMv75zSNhAQDUTbTe7sImKq93dzjY1bP6NnU5P7Gu3nrcY0PteasKpJvf9SG98V99U5 yZZw== X-Gm-Message-State: AOJu0YzVCOG9bT/u//ed9JNSosb8fRfnonYg8quyBxSv7cgnod3lyHiO L54pjURd7h4axQDOkv8Idut+NnuREJ6k/5jrvRc= X-Received: by 2002:a17:902:f684:b0:1bc:25ed:374 with SMTP id l4-20020a170902f68400b001bc25ed0374mr817370plg.49.1692406951158; Fri, 18 Aug 2023 18:02:31 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:30 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 12/18] target/ppc: Use clmul_32* routines Date: Fri, 18 Aug 2023 18:02:12 -0700 Message-Id: <20230819010218.192706-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routines for 32-bit carry-less multiply. Signed-off-by: Richard Henderson --- target/ppc/int_helper.c | 26 ++++++-------------------- 1 file changed, 6 insertions(+), 20 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index 10e19d8c9b..ce793cf163 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1446,28 +1446,14 @@ void helper_vpmsumh(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) } } -#define PMSUM(name, srcfld, trgfld, trgtyp) \ -void helper_##name(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) \ -{ \ - int i, j; \ - trgtyp prod[sizeof(ppc_avr_t) / sizeof(a->srcfld[0])]; \ - \ - VECTOR_FOR_INORDER_I(i, srcfld) { \ - prod[i] = 0; \ - for (j = 0; j < sizeof(a->srcfld[0]) * 8; j++) { \ - if (a->srcfld[i] & (1ull << j)) { \ - prod[i] ^= ((trgtyp)b->srcfld[i] << j); \ - } \ - } \ - } \ - \ - VECTOR_FOR_INORDER_I(i, trgfld) { \ - r->trgfld[i] = prod[2 * i] ^ prod[2 * i + 1]; \ - } \ +void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) +{ + for (int i = 0; i < 2; ++i) { + uint64_t aa = a->u64[i], bb = b->u64[i]; + r->u64[i] = clmul_32(aa, bb) ^ clmul_32(aa >> 32, bb >> 32); + } } -PMSUM(vpmsumw, u32, u64, uint64_t) - void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { int i, j; From patchwork Sat Aug 19 01:02:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715008 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801096wrs; Fri, 18 Aug 2023 18:04:27 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFC5MrMJ+LOu5E+Iy14s5mbiKKX8pL/VNEbws/Z4gj1oooYf+F2Ga0JmLV/1xQ0+svQcqWC X-Received: by 2002:ac8:584a:0:b0:405:42e9:8a8e with SMTP id h10-20020ac8584a000000b0040542e98a8emr636948qth.57.1692407067341; Fri, 18 Aug 2023 18:04:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407067; cv=none; d=google.com; s=arc-20160816; b=r3UjRSroHH3EzTp41ivg7qhnEUOaBeiHYmN+WG5wvdtRBQzQncPwfzW5tViWZGbPDJ CAv7ueiowuApNyZNm/0VmXjXHxIgHDEWWNcUHpjjE9NwIKk4QS2W/hCixO9YWV1Y4bdz qRyjWY3B/8WqFOcTF951ulP30G9vb50GLChLR3R2qYcF3h6RZQUOGopQHga6DaMolem2 LGngR6SU7S731K03ZH1u9sEsaj6WoAC4PgAE4P9Et9j4lyUPEFC9qYMit2uKUB91tfzW wy37pAT6ILzH7kUelS5K0pMPtSCmLpEs4m3yUGi7Bi0R408NDXX2gwF1gdkV9lakb4Gm UyLQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=IxOEDw7RsoN9q8QgKWkH8REcRkgTW7AGyPNgOm6ctSxcouO0gEkJQuTSX3kjSAk4sG u/eDx/S6nBagkIn28GcJXk/YQVpEbgP/nAddDhHrj51CekBYZ/dS22ANEWCCC0csboZ4 37f0MS2aWvfJU9YLwB9VZ/s5ANKX06EwEl3TvU1/npyTVXoIPWytKSLqR/XJibkj2JXb 7yyAnDJvcCjKwIr6m7n90RbvIJ8WUgFJjmvN+7g4s256uxqEgzxkVQ8bK/znk4W8eQwo cKrqn8fZrEq+u/1RVRyiuAalejmHGKY0ENW2DBMmM0ujuNTryhjK6P+fbh1+zs/MsOC8 7maw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GO9tzWnB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u8-20020a05622a14c800b00403ea9d6372si1883583qtx.701.2023.08.18.18.04.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=GO9tzWnB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMP-00004u-UH; Fri, 18 Aug 2023 21:02:37 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMN-0008Va-U5 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:35 -0400 Received: from mail-pf1-x436.google.com ([2607:f8b0:4864:20::436]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAML-0007NB-8h for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:35 -0400 Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-688769faa09so1280745b3a.2 for ; Fri, 18 Aug 2023 18:02:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406952; x=1693011752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; b=GO9tzWnBDyuYpkjZjBP8D51UKzm9b+u8Q2Sh21xFzclplOboankLLX6S/WzS5uuqUN 5wy1Tt4ydPuz/Uu36fCB3UKr1/KrVhb3V/LNTd8LKemECh44RWYBt5zSKggRempzXVJv e4RftS0mH+BMEi61NRam/pb+FWBD7vhiuElobhU3HQwa9FZFJMXVEeFDRGrXFdKTyqph AuzUcPlGRMasFaMEpzTM5ISbCwYm7eSbvR/XNxDFyHFwO+2ygSvw1IvJ30mjwJqlahmq /Jje0ah7XcWCxJrpwAbHCBMSTc4QmpwRe2i3IA9QurHonnH+aDg3telecSizHG0pnfnU OWfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406952; x=1693011752; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=v9aGtnTZpFDMgY3v11X1zR3alCRPd1TwbYXyjZl8p88=; b=THgVLSv40JGP3pVDVgCZ8QPL7eBxYN/lWByzlcFQi26UeJ538drjJ4KLXQVv3KTc/r IEIzesvbSDVgClaqtKnu5nIUGDRy36wr8hPC9Lb8TAyzdtaweKmfVmknm0fuEoqCYroz TjTED2PHMm1YaAqUan1bX6rgBTXKeOkG2COQsEcpm6C/74ma+WqFelbQOijI3mrUgw/C Ro11c3Fhco3VEhRTd+HE/G1mgj/n8/tziBOeXFVRbzssklZw3nE9Mf213Pgr/SNXY765 rD9+A3WOFyaxMy/rq6KLs+x8Ss/V+Y9opuXihKj1HqwqYgLRQa9Lu9eB2hyJiBWo//Zw Z52g== X-Gm-Message-State: AOJu0YyWhwT84tvO3jfFR9eVU7psSWfcuvppvvXk53y4SdEGaUohNsDF AdYBS8ZPXK8AD+ARRR2dUQh4A7J+PtxKwlQTCgM= X-Received: by 2002:a05:6a21:488b:b0:13a:59b1:c88b with SMTP id av11-20020a056a21488b00b0013a59b1c88bmr811957pzc.45.1692406951964; Fri, 18 Aug 2023 18:02:31 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:31 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 13/18] crypto: Add generic 64-bit carry-less multiply routine Date: Fri, 18 Aug 2023 18:02:13 -0700 Message-Id: <20230819010218.192706-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::436; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x436.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson --- host/include/generic/host/crypto/clmul.h | 15 +++++++++++++++ include/crypto/clmul.h | 19 +++++++++++++++++++ crypto/clmul.c | 18 ++++++++++++++++++ 3 files changed, 52 insertions(+) create mode 100644 host/include/generic/host/crypto/clmul.h diff --git a/host/include/generic/host/crypto/clmul.h b/host/include/generic/host/crypto/clmul.h new file mode 100644 index 0000000000..915bfb88d3 --- /dev/null +++ b/host/include/generic/host/crypto/clmul.h @@ -0,0 +1,15 @@ +/* + * No host specific carry-less multiply acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef GENERIC_HOST_CRYPTO_CLMUL_H +#define GENERIC_HOST_CRYPTO_CLMUL_H + +#define HAVE_CLMUL_ACCEL false +#define ATTR_CLMUL_ACCEL + +Int128 clmul_64_accel(uint64_t, uint64_t) + QEMU_ERROR("unsupported accel"); + +#endif /* GENERIC_HOST_CRYPTO_CLMUL_H */ diff --git a/include/crypto/clmul.h b/include/crypto/clmul.h index 0ea25a252c..c82d2d7559 100644 --- a/include/crypto/clmul.h +++ b/include/crypto/clmul.h @@ -8,6 +8,9 @@ #ifndef CRYPTO_CLMUL_H #define CRYPTO_CLMUL_H +#include "qemu/int128.h" +#include "host/crypto/clmul.h" + /** * clmul_8x8_low: * @@ -61,4 +64,20 @@ uint64_t clmul_16x2_odd(uint64_t, uint64_t); */ uint64_t clmul_32(uint32_t, uint32_t); +/** + * clmul_64: + * + * Perform a 64x64->128 carry-less multiply. + */ +Int128 clmul_64_gen(uint64_t, uint64_t); + +static inline Int128 clmul_64(uint64_t a, uint64_t b) +{ + if (HAVE_CLMUL_ACCEL) { + return clmul_64_accel(a, b); + } else { + return clmul_64_gen(a, b); + } +} + #endif /* CRYPTO_CLMUL_H */ diff --git a/crypto/clmul.c b/crypto/clmul.c index 36ada1be9d..abf79cc49a 100644 --- a/crypto/clmul.c +++ b/crypto/clmul.c @@ -92,3 +92,21 @@ uint64_t clmul_32(uint32_t n, uint32_t m32) } return r; } + +Int128 clmul_64_gen(uint64_t n, uint64_t m) +{ + uint64_t rl = 0, rh = 0; + + /* Bit 0 can only influence the low 64-bit result. */ + if (n & 1) { + rl = m; + } + + for (int i = 1; i < 64; ++i) { + uint64_t mask = -(n & 1); + rl ^= (m << i) & mask; + rh ^= (m >> (64 - i)) & mask; + n >>= 1; + } + return int128_make128(rl, rh); +} From patchwork Sat Aug 19 01:02:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715015 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801362wrs; Fri, 18 Aug 2023 18:05:09 -0700 (PDT) X-Google-Smtp-Source: AGHT+IE0arNlWv6cgLcCq1FqesemGgEhysYDcdV7ckTJhOLtHRxkVnPU9Exd1TF+camD/B6g4jYI X-Received: by 2002:a05:620a:4482:b0:76c:9931:cbd with SMTP id x2-20020a05620a448200b0076c99310cbdmr1055772qkp.28.1692407109483; Fri, 18 Aug 2023 18:05:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407109; cv=none; d=google.com; s=arc-20160816; b=yvPDfqiVyMK9dE5zwtCOqb4sdZhdBtZQ+qPUJnDXKechrrj9F7VfIZ41BY6RdBMBgI VZHwfPvqbAGxUZeHV/4Dil7br9ZrD4ZDhXXZSYcDw3Kf/NKNJtIwdq1ENpjvOntU1rN8 oOuwry0clzkeJfr3JS9dbw/uh7Bya+C9964JZ1Z8xdEgXWax2jkOMyPgItVDEkbOgqSx 89qbMogVui341vwvQVBooEDaiSd8UuW1g6dxkbtmKsOOwWaWBEkL4ceiLHVWTE7h/m1p sb0Dww4vlZ8AqVFdHmjLPqAjtChH+ozxKA89UB2ZYB032iAETlP2/i98D2SOhu7TA/Cr uI3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=KOR8Q/KDHweXHhtPJBOF6KbMNecqFI3FxCLD3VnqMyo=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=U0eKzuGqGIqsV0pah+qQWObfPfVcpWyoU8kJ4XJhqVELip209T9oE6pTdcNE3ZaYaA MSkQ8lvmL4h/2KAT6rHchRx0+oz+hG4yksHoUrZmltvgzPK85v9LEfG4Tid66Lv0Kajq PDTVQYxx0x7T5IU/Wl95k/+EkQZqDvQIndtB2G1VLVcxsl9+iEwchO1bRlQjsEqbKtSX XnNIOnEnQ73H9Z/XTrliEce6KiJKeOE/Q9qtN4afGJeksLkdHVe9IWuULtTm3ulJq8qF j1uQ9zgZZKHPrmu8vVtitAECKvNRUovW+tCw6DY5Z7qQdebi2wq7t8O4LKG7KQ8woJ8a doRQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="xBw/Xgej"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ox19-20020a05620a829300b0076cdc1a072esi1786657qkn.320.2023.08.18.18.05.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:05:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="xBw/Xgej"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMQ-00004x-0f; Fri, 18 Aug 2023 21:02:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMO-0008WG-FN for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:36 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMM-0007NZ-5l for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:36 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1bf48546ccfso6980285ad.2 for ; Fri, 18 Aug 2023 18:02:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406953; x=1693011753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KOR8Q/KDHweXHhtPJBOF6KbMNecqFI3FxCLD3VnqMyo=; b=xBw/XgejVnzOPPO3ZgB8tMHJfr8SSTR+y4Ga0aJvwGQxJfb/fqTrYcsNQWJ+wZmNvi f59dpHckMtlZbNxgSVE8DTdA8DmLlLP+gNjLXkglKAUiSF/0356Gp+my5wUYHaM77mMU uG0yQ2gkwIs20ebfhboJoTwwrqIM6A5KoBIzhR40ttMEjaG8z4drF/1QiZsLtZjV3+Kx ONIL1P3XAt3Xeynk6oGxVf5j0SSescsnfzbhwH6Q5hgoOWm4oqt/MDOs5pfMU65utHxw ye3pLwD7IFk0ei6NMbC2oX6IdzmarZQPdBex/A4oDi+BRYS1jlOO74tlHkGbCi1dNi2x h6ew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406953; x=1693011753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KOR8Q/KDHweXHhtPJBOF6KbMNecqFI3FxCLD3VnqMyo=; b=iBNNThCyFfbsZcpBBabryViRLukCdo7lLTZBzUuiJiH6ZnQiKCPE+RD+gxE3Z0TAjI mqPnsQbtlr7ulQaOA6CjyDrm3JxRM9rfcS0OXxA/POfwS1j/JyxoL/UEdLK618YR5jqN HsUe/wXtEL+W2PczxPC6dUgmJ9NLSo6gHoiYmHxh8t3OakMIxKDwBVcfY+G6iC0P+R3w NmUAvtNJtkMJWGDzBkowwm4GPN3mdweySE/BrlppLL6z/JY7zYlrXObbFEfgbSPH2Rf9 a0fNRddTwys6DdjXMp23Py4Y5XPWD3c7iVN17Wim2Lorwb4xBKhdjuVBaJh5sGBHjcYZ auRg== X-Gm-Message-State: AOJu0Yx+DPnYmySoP21i0RWXRmpjn2tj+zIPhpU1FfMXG+7GTZS/0u79 EUsML0BaICsC0m2qzXUY3TvwZ6solhwgd2EZFcw= X-Received: by 2002:a17:902:ea0d:b0:1b8:9552:2249 with SMTP id s13-20020a170902ea0d00b001b895522249mr752814plg.43.1692406952936; Fri, 18 Aug 2023 18:02:32 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:32 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 14/18] target/arm: Use clmul_64 Date: Fri, 18 Aug 2023 18:02:14 -0700 Message-Id: <20230819010218.192706-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/arm/tcg/vec_helper.c | 22 ++++------------------ 1 file changed, 4 insertions(+), 18 deletions(-) diff --git a/target/arm/tcg/vec_helper.c b/target/arm/tcg/vec_helper.c index ffb4b44ce4..1f93510b85 100644 --- a/target/arm/tcg/vec_helper.c +++ b/target/arm/tcg/vec_helper.c @@ -2003,28 +2003,14 @@ void HELPER(gvec_pmul_b)(void *vd, void *vn, void *vm, uint32_t desc) */ void HELPER(gvec_pmull_q)(void *vd, void *vn, void *vm, uint32_t desc) { - intptr_t i, j, opr_sz = simd_oprsz(desc); + intptr_t i, opr_sz = simd_oprsz(desc); intptr_t hi = simd_data(desc); uint64_t *d = vd, *n = vn, *m = vm; for (i = 0; i < opr_sz / 8; i += 2) { - uint64_t nn = n[i + hi]; - uint64_t mm = m[i + hi]; - uint64_t rhi = 0; - uint64_t rlo = 0; - - /* Bit 0 can only influence the low 64-bit result. */ - if (nn & 1) { - rlo = mm; - } - - for (j = 1; j < 64; ++j) { - uint64_t mask = -((nn >> j) & 1); - rlo ^= (mm << j) & mask; - rhi ^= (mm >> (64 - j)) & mask; - } - d[i] = rlo; - d[i + 1] = rhi; + Int128 r = clmul_64(n[i + hi], m[i + hi]); + d[i] = int128_getlo(r); + d[i + 1] = int128_gethi(r); } clear_tail(d, opr_sz, simd_maxsz(desc)); } From patchwork Sat Aug 19 01:02:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715016 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801397wrs; Fri, 18 Aug 2023 18:05:15 -0700 (PDT) X-Google-Smtp-Source: AGHT+IHLtL60dZOzomG2X90EONn8hGDw1Ubg0I0QE8SFc56cSIXKF9bQxzpeRNZcOtH+SC9e5IsL X-Received: by 2002:a05:620a:2805:b0:76c:ba10:4973 with SMTP id f5-20020a05620a280500b0076cba104973mr964027qkp.37.1692407115573; Fri, 18 Aug 2023 18:05:15 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407115; cv=none; d=google.com; s=arc-20160816; b=uBDWtISn/6y/9ULrOgrQYwu1qrM3Ei1wsh0cebNMtw3mHM8+dby8vGnHmipb5d0/nf ut7ltO7vRokAu+ke7KSJ57YbAEndA1vjyuWjocBV9hxSAuSzphW41eIGmztasPosrbfc jqLCh0FCPoIFiUvMSgEYlvX6Sg0HDcm90XgxT6Nl8tKHFjFdAELM141nWjRkLcCWLpAr pph5XIbme8DrnPwJqRe9VnW2SsZZlo+0REohw22PVa4odj54W4ZtxpmWnSSK1PhG96+v SFMsrviLjGgOmfIl6/Q926VsJqJWcmj8sVq7HDm+i34q4nFBQ1ajPPBm6zlzXtubImbV zjCw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Xb/RiA6QDgK0yKGdPpbNiiP/1RVfomObwJ/soV+H/NI=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=LaBYERfXBw3ZVvc0yFqkAb7CN2OR/ynVxuVaypsE7fpGL/D9/re8IT3s/OXzaEGjlV h3Sb+BI2sS9SQCyVbYXXw+iv9CPDeXIcAd5x5G+8swMZvIzTx3l9o5RtBWA3HO67bOEF 3LPQ78+kFwfnwcw1Xn34YYPkdsOQwT7iR8puxb/VX44rjyapJTLvsqNdmTLHwdckInx7 LhPEHJ4UILKzwzcLKfF7ba29txq/XXh9SnayR3LjtporqDrZj7co9a8KAEwC0+U+ROgY DkylrLW3dC14dMMr3b6Lfcl6ZMreq0JR/xZLsS0OfAmqCrrfSN9MsCqJh5X5BV/VG+6W jutA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=U6llev1T; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n18-20020a05620a295200b00767d8ef89fcsi1999029qkp.416.2023.08.18.18.05.15 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:05:15 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=U6llev1T; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMR-00005Y-1i; Fri, 18 Aug 2023 21:02:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMP-0008WQ-JO for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:37 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMN-0007No-A8 for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:37 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-68879c4cadeso1250788b3a.2 for ; Fri, 18 Aug 2023 18:02:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406954; x=1693011754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Xb/RiA6QDgK0yKGdPpbNiiP/1RVfomObwJ/soV+H/NI=; b=U6llev1Tgms8+Ou53A/OT3AE6N5QTcKdGvI4MgGYBOuWxsW0wONp0/Jh6xdqPl55BT 2Paf523cm0zjg6evM0FP+albVYb7KtYsGts2M640vWESfDogMP9CECYtoRCS4WL2P7fB xt8XKO1tvCvGWQkcTJ1D9BTtwSax08Hdj4XLxIHnS/F9K69q3mfryRiX1jzH++H38bPb yObNX5cjrJ4vHTdGcmFCbIuL8u9SNl+8ffCn5GSSZJ3yulnAC5DGr/Ij3E6JdGnWyD0w He/YThTK+ibiO1WI/xOLZVq1lglNOZurdUzhf0H8dFnOmRFHk56UjeVvi6ncWdSLD74A JdgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406954; x=1693011754; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xb/RiA6QDgK0yKGdPpbNiiP/1RVfomObwJ/soV+H/NI=; b=MchNeNQWSJbENRFtwXNe/gJQgmbKM2HJu0sByVcUzZKTmVnje5gomBrprZbNuRhVxh 1REc90WG5ShCksCy576AwG2VbU+UcYsUKKxRVSF4/+To3sKF1hOkNEEOAWJy6sHhj63B TPYqt+JAlEMrXQWIkuBTkzbV9pUwEphVmZS0dsavr7NSh7p65QgE3GnkO1cK+JMLVd4P sUwGqg/KzlR3LRMoEJPmxVRFeC/1i3jG4lMjUXCp7+oxy4zxqvid6hzsWaewmy6HkpnL L9fU0l1kwXe9SeNxJJfU9oWp/Erft5mw6S5MOpvLGRDSmCx+pQWGxU5DQTi5sbS5/K/L oJag== X-Gm-Message-State: AOJu0Yw1rAHaPhsyez3/XvYllDBF7nxsTA+oguk7Dy2pxTz54pJ3dH+1 W9iqqnOHewK7Rj80li1Er79yyy6HG7yBmdnn5tE= X-Received: by 2002:a05:6a21:4987:b0:13f:9b98:c416 with SMTP id ax7-20020a056a21498700b0013f9b98c416mr681706pzc.55.1692406953929; Fri, 18 Aug 2023 18:02:33 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:33 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 15/18] target/s390x: Use clmul_64 Date: Fri, 18 Aug 2023 18:02:15 -0700 Message-Id: <20230819010218.192706-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x433.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the generic routine for 64-bit carry-less multiply. Remove our local version of galois_multiply64. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/s390x/tcg/vec_int_helper.c | 58 +++++++------------------------ 1 file changed, 12 insertions(+), 46 deletions(-) diff --git a/target/s390x/tcg/vec_int_helper.c b/target/s390x/tcg/vec_int_helper.c index ba284b5379..b18d8a6d16 100644 --- a/target/s390x/tcg/vec_int_helper.c +++ b/target/s390x/tcg/vec_int_helper.c @@ -21,13 +21,6 @@ static bool s390_vec_is_zero(const S390Vector *v) return !v->doubleword[0] && !v->doubleword[1]; } -static void s390_vec_xor(S390Vector *res, const S390Vector *a, - const S390Vector *b) -{ - res->doubleword[0] = a->doubleword[0] ^ b->doubleword[0]; - res->doubleword[1] = a->doubleword[1] ^ b->doubleword[1]; -} - static void s390_vec_and(S390Vector *res, const S390Vector *a, const S390Vector *b) { @@ -166,26 +159,6 @@ DEF_VCTZ(16) /* like binary multiplication, but XOR instead of addition */ -static S390Vector galois_multiply64(uint64_t a, uint64_t b) -{ - S390Vector res = {}; - S390Vector va = { - .doubleword[1] = a, - }; - S390Vector vb = { - .doubleword[1] = b, - }; - - while (!s390_vec_is_zero(&vb)) { - if (vb.doubleword[1] & 0x1) { - s390_vec_xor(&res, &res, &va); - } - s390_vec_shl(&va, &va, 1); - s390_vec_shr(&vb, &vb, 1); - } - return res; -} - /* * There is no carry across the two doublewords, so their order does * not matter. Nor is there partial overlap between registers. @@ -265,32 +238,25 @@ void HELPER(gvec_vgfma32)(void *v1, const void *v2, const void *v3, void HELPER(gvec_vgfm64)(void *v1, const void *v2, const void *v3, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(v1, &tmp1, &tmp2); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = int128_gethi(r); + q1[1] = int128_getlo(r); } void HELPER(gvec_vgfma64)(void *v1, const void *v2, const void *v3, const void *v4, uint32_t desc) { - S390Vector tmp1, tmp2; - uint64_t a, b; + uint64_t *q1 = v1; + const uint64_t *q2 = v2, *q3 = v3, *q4 = v4; + Int128 r; - a = s390_vec_read_element64(v2, 0); - b = s390_vec_read_element64(v3, 0); - tmp1 = galois_multiply64(a, b); - a = s390_vec_read_element64(v2, 1); - b = s390_vec_read_element64(v3, 1); - tmp2 = galois_multiply64(a, b); - s390_vec_xor(&tmp1, &tmp1, &tmp2); - s390_vec_xor(v1, &tmp1, v4); + r = int128_xor(clmul_64(q2[0], q3[0]), clmul_64(q2[1], q3[1])); + q1[0] = q4[0] ^ int128_gethi(r); + q1[1] = q4[1] ^ int128_getlo(r); } #define DEF_VMAL(BITS) \ From patchwork Sat Aug 19 01:02:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715017 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801410wrs; Fri, 18 Aug 2023 18:05:17 -0700 (PDT) X-Google-Smtp-Source: AGHT+IFydiCgohE0sSBmOTafSI+Zhmur9Nki5ZbafJ5d8H7jkVDkK1nh40AhiFOEnt1Qe5vHAlU8 X-Received: by 2002:a05:622a:cc:b0:403:8de7:4503 with SMTP id p12-20020a05622a00cc00b004038de74503mr1024626qtw.52.1692407117278; Fri, 18 Aug 2023 18:05:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407117; cv=none; d=google.com; s=arc-20160816; b=ZMlsdQzUMMJ8a+zT+WfYWJ49+bE41l2AZFxgm6tTOMZx/Oa6g02xDNe8vZ8ZLi5Xd8 K0diC8n+6ypexyZ/BICSBHtdS/LM3CjFKgGXNfKaLYq2Bj8+WozXZGwu7mTs8/hPDqIx 3GiDWwHOX5T5kydXyeQnSNA2HhdnWn5BzHNa3XKvMFS7o5ndX1sQhbFy1kjV2VHs69Lz xbTsvOejtOCtSachbx9MYC5fI8Y5BOfe4dPblMTksL4vvKlyaR1novzD1YHSvhcvytME zSrSSbZlMzm/Uq3b+egl0Lg9UBUP9aHfeUVxzLQR2d5XRo/3snNSzQVqcdgvNkqWdvv7 JuGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=EYOvUjl9DKYepdyDRgw1XoKhgsFy5EQ3dFtSTrNTOWE=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=LBo7BuyCmrFasi5lZPiNkAd4JtrsKZPME/xomYciKXjCBu4uMFWOi2j8X15NVyvDTM gOV7spJJgIP5a5nYpxxVr5tHSEpzHFAUk2LBE7B3ypRqk6dyoxvIFw0g/VhdsP8zGAze 2bE05ihoz7YbNpTsqto5EIcRFXlrxF+/n0r0g8X3ECbI346zb7QDZmsFgjslBGz+6Yo+ 5VUIsD+55W2Bsh2a9bBEZVkFwdM8NkL+3/rof2eRy7X9ps6I7zROF6WrLfj/9ll4UEAV F/1s2Asrv0kv+bzoxd6tbq4ONzYth745YN9sYR3eKwzYxU7iI9ohnS/JSijaf91k3os9 lzcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T7YmyQ0h; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b8-20020a05622a020800b00403c944ba12si2025310qtx.775.2023.08.18.18.05.17 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:05:17 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T7YmyQ0h; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMR-00006A-NU; Fri, 18 Aug 2023 21:02:39 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMQ-00005B-4R for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:38 -0400 Received: from mail-pl1-x633.google.com ([2607:f8b0:4864:20::633]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMN-0007O0-Vg for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:37 -0400 Received: by mail-pl1-x633.google.com with SMTP id d9443c01a7336-1bdca7cc28dso12355495ad.1 for ; Fri, 18 Aug 2023 18:02:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406955; x=1693011755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EYOvUjl9DKYepdyDRgw1XoKhgsFy5EQ3dFtSTrNTOWE=; b=T7YmyQ0hTa37BA8zQsPUY0C+QlKcFL2EtaFxnuGn/8GRI+8FmFF2G1d2a4hMAOsFsH 9kNiuYpTgitTw4ouQUkU8zuhSmZm5YRzCu+RAZ4kDNpwCoyHSFfKgLkMwdNtNqYfIpCe Bd4VRdeYE0Fno6yZbceSknSoxct6f6EVY9gO/uWdJzcEomrpkLe7pn6fd33DpXhTWhGa MwP8Ua3Ea/hFj6kkgkGjloZNg+F4uBB2LCkIA/HKO2CmfOPsPh+7OloXeNHmjbSWuUwf F/0RVxMWMsb1qo0a9VbiomN7vZjkAKo85jtuO8fKR7ghVWbvBiXFzZNEpds8llA715hC 81aw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406955; x=1693011755; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EYOvUjl9DKYepdyDRgw1XoKhgsFy5EQ3dFtSTrNTOWE=; b=Edb/dS6MqBtCYNItoN/k6Xai7wuYLezQWXKqrB+Tp7vvLemtKxMvtpJQSpOqjHWxeJ tcrSD1Q+PRj02eAF2qJM31jyf1t5AanLqO3HYn3xerAeDao716tdn/eeemzzDvfDBxlu GP2cp7xRk1gv/6k3S7OdvAx8Uv33muPkBeITBrKSTvwBtmOAv/tIIL+Ufms4Y+LWS8fU 8YwtZJ6975eR6wfJOwSF9CYMw0ikHJ4VJjp4lkduR1ybMDGJ+dCxbq2zbPH3JVtpYBae dfq0Y+DiWioDdBorDFxZZ7Cjcn3Ipo7kWxvKcIYKxpgzhlthIUfPntGI9SaBdPeAU/u1 Z9MA== X-Gm-Message-State: AOJu0Ywb5PuESQcWTwsvZ/smeXiL4V+pEaTcDWbS/87i8o1tNDUG3i9s 2i5TZlAfxbZMOh70L4/rw2ip3rWeIcVnkkpYaGI= X-Received: by 2002:a17:902:ce8e:b0:1b8:8223:8bdd with SMTP id f14-20020a170902ce8e00b001b882238bddmr860340plg.59.1692406954751; Fri, 18 Aug 2023 18:02:34 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:34 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 16/18] target/ppc: Use clmul_64 Date: Fri, 18 Aug 2023 18:02:16 -0700 Message-Id: <20230819010218.192706-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::633; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x633.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use generic routine for 64-bit carry-less multiply. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- target/ppc/int_helper.c | 17 +++-------------- 1 file changed, 3 insertions(+), 14 deletions(-) diff --git a/target/ppc/int_helper.c b/target/ppc/int_helper.c index ce793cf163..432834c7d5 100644 --- a/target/ppc/int_helper.c +++ b/target/ppc/int_helper.c @@ -1456,20 +1456,9 @@ void helper_vpmsumw(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) void helper_VPMSUMD(ppc_avr_t *r, ppc_avr_t *a, ppc_avr_t *b) { - int i, j; - Int128 tmp, prod[2] = {int128_zero(), int128_zero()}; - - for (j = 0; j < 64; j++) { - for (i = 0; i < ARRAY_SIZE(r->u64); i++) { - if (a->VsrD(i) & (1ull << j)) { - tmp = int128_make64(b->VsrD(i)); - tmp = int128_lshift(tmp, j); - prod[i] = int128_xor(prod[i], tmp); - } - } - } - - r->s128 = int128_xor(prod[0], prod[1]); + Int128 e = clmul_64(a->u64[0], b->u64[0]); + Int128 o = clmul_64(a->u64[1], b->u64[1]); + r->s128 = int128_xor(e, o); } #if HOST_BIG_ENDIAN From patchwork Sat Aug 19 01:02:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715003 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp800918wrs; Fri, 18 Aug 2023 18:03:58 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEwE2OnS6s+CmU2u4+d+wDw+VKzR6a/ij7kYDM3jE1CPtbKoGednH92sQYgK/d59MaJ5MUB X-Received: by 2002:a0c:e14f:0:b0:64c:16aa:88c2 with SMTP id c15-20020a0ce14f000000b0064c16aa88c2mr870713qvl.39.1692407038619; Fri, 18 Aug 2023 18:03:58 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407038; cv=none; d=google.com; s=arc-20160816; b=Dsg7NX1gm8rMYxiaiE1/ytZixebPzVQW1wzh4PSJftK/iYyeqsM/Fbw73SMWS2MG/q 5e14pxFKP2Fu7Cm2IvbFIrTt2J9pi1oJJpWtVPjBfUR8kHDs8X9N6DHst4xFbfzMIHzV RCN4UsnAWBKkWxusiU0xXH/Hb7lv4F+mq1k1u+yRNDMsQCvSA6VOxqh9xOPNpQYrRmaR opSYjkCHusQvYwzFLGIKTlF3/5kzOBo6+WSlcb8Ji6dTVK08TSA0Gl4FsmebgPrkKtXR whmmS29jPaF250DEmUtqvVQHV5eVouO6r5mEVEuuMGtRRPs5gr8D84Pbtyl8dM64xIMe QbZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=Z+B0WDoHaiE/+orXkv/LSBlZMJ/5Xr/iMjSAMh4W7WR/k2MCY+W1F7gbrZJ8VWSRHv 06qCJ2KFOpYlHXwNUy7IC5+3TQ7y4jQjmxgaUzQb/h7VBIs2XW2yg/NLMIvWiX/pbb3d emWIXwfpZWcj1YPKv+pEkfYz48mPwTSQsaC2Nxyz1mnw/UDSIDCiuFlV0Zjt5pp8rTzL zyNc0hwa4XQiMxIDqkyVETtdF8WYKajg7pT+GO7EyE3mwgPuYJSxQTw0ebQHBN/4Qi9g XW3DbHMfreu9Zg4NgTmF7k7TzJErdVfrwPJieKFG3/21GJt8YJaT3WmYnvUxwAgfadKp Q8Uw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EaSWqFVB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b3-20020a0ccd03000000b0063cebcc91eesi2061179qvm.529.2023.08.18.18.03.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:03:58 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=EaSWqFVB; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMS-00007D-Mf; Fri, 18 Aug 2023 21:02:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMR-00005n-Ee for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:39 -0400 Received: from mail-pf1-x42b.google.com ([2607:f8b0:4864:20::42b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMP-0007OD-4z for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:39 -0400 Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-689f1e81293so1221922b3a.3 for ; Fri, 18 Aug 2023 18:02:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406956; x=1693011756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; b=EaSWqFVBZp2gT5as7i537FHXEhejJalpf/JDwMki+MUXt1BLyJAXlOh4Z3VgPEuUNC KrAvmBF4jcGdkt6+ZXm0bmvCMDxV3QXCc2jnVroi7GaC9s140fm0lFZ+e8MBsBayM7uH IPz927FhrcLzzGXeRfKp2JL4LsXN+uzfltSEk2KmOCdVJSV3CO7o4IH2sCpU7EYAUVNP SuaWVgPxUZk5UokQ85gwqoROh6H5eUM1dld9AV0Fv5rWojGl3vkakq06rLIzuK62ACco FR1/VXX5ge0dXvUxVD6fdOj+giijbvtfwhK58Z2yWCFGCrcHHXDSCN36WaqMmRv0aqrF aBjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406956; x=1693011756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fd0V1dVewg5WOTiDo3UWfFkJLmgfwo7yGoKXR6xz6U0=; b=ZmzptIUh7k4cQsImH/Y6idmn+QrqBVclUipldbBUVPcI4xdsh11NH0quMO38vFYDih oHxNyZFQwQETUrowp5PLhAo+aEAIer90/ayyFyA6H+fB6VJWzAPedBu0c0pyXSvCGPUl pqCz2pKzY1EiTxQmP+lIu+Ww66X1j3uUa+ysFVDJKFtWD324ffJTC7FNboUzPPvP5iPl h90fbhddBXpEI3w6H2WOPd2Z6ugzfQ8rrHh7s+ipi8oqLrT4F843+CXzuIRw6OmYUaLz a6Wpx7wRHZ1TI676f1H9UKtrk4SQlFmrnRosHPi/0+3z9xj2bk0xrX3vAwlB+0DSaCvW +qFA== X-Gm-Message-State: AOJu0YxDceQK0txTiyFqdp+0uAVND9G3pVoVMa1aoi5mCJYoq0XiU5r0 3L1hs3IcScuaOEuYdLP+7uhbZ9mQ60wDeB4m2yU= X-Received: by 2002:a05:6a21:a103:b0:140:ca4c:7411 with SMTP id aq3-20020a056a21a10300b00140ca4c7411mr767585pzc.27.1692406955751; Fri, 18 Aug 2023 18:02:35 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:35 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 17/18] host/include/i386: Implement clmul.h Date: Fri, 18 Aug 2023 18:02:17 -0700 Message-Id: <20230819010218.192706-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42b; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PCLMUL in cpuinfo; implement the accel hook. Signed-off-by: Richard Henderson --- host/include/i386/host/cpuinfo.h | 1 + host/include/i386/host/crypto/clmul.h | 29 +++++++++++++++++++++++++ host/include/x86_64/host/crypto/clmul.h | 1 + include/qemu/cpuid.h | 3 +++ util/cpuinfo-i386.c | 1 + 5 files changed, 35 insertions(+) create mode 100644 host/include/i386/host/crypto/clmul.h create mode 100644 host/include/x86_64/host/crypto/clmul.h diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h index 073d0a426f..7ae21568f7 100644 --- a/host/include/i386/host/cpuinfo.h +++ b/host/include/i386/host/cpuinfo.h @@ -27,6 +27,7 @@ #define CPUINFO_ATOMIC_VMOVDQA (1u << 16) #define CPUINFO_ATOMIC_VMOVDQU (1u << 17) #define CPUINFO_AES (1u << 18) +#define CPUINFO_PCLMUL (1u << 19) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/i386/host/crypto/clmul.h b/host/include/i386/host/crypto/clmul.h new file mode 100644 index 0000000000..dc3c814797 --- /dev/null +++ b/host/include/i386/host/crypto/clmul.h @@ -0,0 +1,29 @@ +/* + * x86 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef X86_HOST_CRYPTO_CLMUL_H +#define X86_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +#if defined(__PCLMUL__) +# define HAVE_CLMUL_ACCEL true +# define ATTR_CLMUL_ACCEL +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PCLMUL) +# define ATTR_CLMUL_ACCEL __attribute__((target("pclmul"))) +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { __m128i v; Int128 s; } u; + + u.v = _mm_clmulepi64_si128(_mm_set_epi64x(0, n), _mm_set_epi64x(0, m), 0); + return u.s; +} + +#endif /* X86_HOST_CRYPTO_CLMUL_H */ diff --git a/host/include/x86_64/host/crypto/clmul.h b/host/include/x86_64/host/crypto/clmul.h new file mode 100644 index 0000000000..f25eced416 --- /dev/null +++ b/host/include/x86_64/host/crypto/clmul.h @@ -0,0 +1 @@ +#include "host/include/i386/host/crypto/clmul.h" diff --git a/include/qemu/cpuid.h b/include/qemu/cpuid.h index 35325f1995..b11161555b 100644 --- a/include/qemu/cpuid.h +++ b/include/qemu/cpuid.h @@ -25,6 +25,9 @@ #endif /* Leaf 1, %ecx */ +#ifndef bit_PCLMUL +#define bit_PCLMUL (1 << 1) +#endif #ifndef bit_SSE4_1 #define bit_SSE4_1 (1 << 19) #endif diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c index 3a7b7e0ad1..36783fd199 100644 --- a/util/cpuinfo-i386.c +++ b/util/cpuinfo-i386.c @@ -39,6 +39,7 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) info |= (c & bit_SSE4_1 ? CPUINFO_SSE4 : 0); info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0); info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0); + info |= (c & bit_PCLMUL ? CPUINFO_PCLMUL : 0); /* Our AES support requires PSHUFB as well. */ info |= ((c & bit_AES) && (c & bit_SSSE3) ? CPUINFO_AES : 0); From patchwork Sat Aug 19 01:02:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 715010 Delivered-To: patch@linaro.org Received: by 2002:a5d:484e:0:b0:317:ecd7:513f with SMTP id n14csp801238wrs; Fri, 18 Aug 2023 18:04:49 -0700 (PDT) X-Google-Smtp-Source: AGHT+IGA/Bk6GGWKBnRoVrAHqkxlCpXCxAZCdS9ZXpG4uhsNSR+BX3pe3s9BYSCMXGObc8CIRF4y X-Received: by 2002:a05:622a:24a:b0:403:e3ce:565b with SMTP id c10-20020a05622a024a00b00403e3ce565bmr1033754qtx.4.1692407089678; Fri, 18 Aug 2023 18:04:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1692407089; cv=none; d=google.com; s=arc-20160816; b=0KpVbZVc8R4485Qaxd4Lw10GLTypwPMSuXb1itZb+pMrkIWeFEFNBKI3oWzlBOngGZ otLRSfjHpr0IC6P3HgLQsEVY3lbU3H/VXP2hpQv8xTc3OY93mDAAY2z9l5H0GVMCasle fr4lVfFddY/RFEke6CtH3H9x1hXU0z4bfSK1znPugLZplzmevcUznwBgUFE1w1QLgrWo soaQJNXKI0h64kfLPguU7PO3HZn1w5q/eepkymk2YhPiYxlsk98ti9CLWvdZxg/rEhog atc2SRgbndw7VxTrll1aEDeL4mZpA//mK6I6S5iqwPRMDj1GsEjjN2w14236zHr5rIXg 6ZqQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; fh=4xNVIrgzKzsUJFZ/LK5iWzslbipp0obh3zmyxXTMM5Q=; b=kHo4f52wmF5aBK+NLe7IC4dn7hl5bFG5UaNYWG/yNDNgodO/EuNoUHA2dcogBJ/I6e DjinDcQgRfQIP5YTyt33L2T4AMqDa6CM0KZ/TcAqOmRSXbJ0c/udjnVdSN7aM9tjb8A/ 2w8KDAE9Iz/aesCoUPniSkWcsLnC16KPVmZPDyPCVGbdItvWRjpcJEnuZX4B4er907b6 1X4WdaJKuAHF/YXkVRKocV64+Xs6PhdLF8XuRqj7ga9qDe3PToMaOdHAAVUvP3DO7dAi JGAf5Gs92mHpQbph7IrbGAuabeTluclQxc7ecR5AqHDr6m+TLKRAIAJaQA4axgKylFVF FOog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cDEv3xVg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n15-20020ac85b4f000000b0040fdbe81133si1988990qtw.233.2023.08.18.18.04.49 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 18 Aug 2023 18:04:49 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cDEv3xVg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qXAMT-00008V-Rk; Fri, 18 Aug 2023 21:02:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qXAMS-00006l-BL for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:40 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1qXAMQ-0007OW-1B for qemu-devel@nongnu.org; Fri, 18 Aug 2023 21:02:40 -0400 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-688779ffc0aso1250878b3a.3 for ; Fri, 18 Aug 2023 18:02:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1692406957; x=1693011757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; b=cDEv3xVgKJ+kMldXKVSpiuhZZNvEcubmf18ANleRJMRLEl0id+Tut1MFrkJtmSBxEA CGRVOaY8oJe9gi9uE06KAQgVjDxx/lchh9kp1VUIueinrT/0Pc3j52FumcUhre3Xh+Tq fpcDfTcBp8dXWT7QAkYbUmt2hR+Ipsu9ZiQn2O2SM8fhHkkLW/WE8vxW0HUj+Ub93JVf ciJZQYNiCUN1IMzFgcDfjKo/xYBf85ljzLNF3XoQ9AdoNre0lqbbyN94yt0R7mNlymyZ KmnJ5811mgXkQtNfd6lGuCsGLuUL9Z1FSoFD9ftT1IcOg2PYRQ0HN10CaDEpKOXMnsio qoCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1692406957; x=1693011757; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=GXTBIttRUHw7smngsq4Ik28aPo1QFTm9JIek17XN5Es=; b=hKdbyjji50nl008/dQkKzJ4W1b3GRcdC53KHWEnb58YpKqumYQF05ZRLuuyEnrHJ8J s3pNdYIOVRAXgpiLqqIFc4GLwH1rvUcdpdiLmDaIIrt5/ZUwuNgnC2kbjwcgmRE79EzK 0ZdWAs+A6djPgmVQcZF5NXY+w9VBgdSQXwf9Y1Y+uKie2kL6HT/cepMjJI1K0mloEPJw Jojlu4fueoq+Uyqmj872KRyNFM/fXuwJXRhQHe/+Sj1cpX92bgZMrqHuQr00nIQJ2hyn 5X0DouJiDSN5z5wASdCjoZlUt4sdpo8AHScB1aDxWtdEceNGvoAEA56I1ji0WnMulbuf 6tgQ== X-Gm-Message-State: AOJu0Yy4tBa7QQ6ozd4j4Da643nmMozZCUjSqqKsz4RhUfSTnbY1MB5z 09/mSeKilKLeJSCq5IIi+lerKY9f6sgazAKMeGY= X-Received: by 2002:a05:6a20:1006:b0:13f:83a2:df54 with SMTP id gs6-20020a056a20100600b0013f83a2df54mr712551pzc.34.1692406956741; Fri, 18 Aug 2023 18:02:36 -0700 (PDT) Received: from stoup.. ([2602:47:d483:7301:cf24:6daf:2b9e:7972]) by smtp.gmail.com with ESMTPSA id b13-20020a170902d50d00b001b87bedcc6fsm2372019plg.93.2023.08.18.18.02.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Aug 2023 18:02:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: berrange@redhat.com, ardb@kernel.org Subject: [PATCH v2 18/18] host/include/aarch64: Implement clmul.h Date: Fri, 18 Aug 2023 18:02:18 -0700 Message-Id: <20230819010218.192706-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230819010218.192706-1-richard.henderson@linaro.org> References: <20230819010218.192706-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::432; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x432.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Detect PMULL in cpuinfo; implement the accel hook. Signed-off-by: Richard Henderson --- host/include/aarch64/host/cpuinfo.h | 1 + host/include/aarch64/host/crypto/clmul.h | 41 ++++++++++++++++++++++++ util/cpuinfo-aarch64.c | 4 ++- 3 files changed, 45 insertions(+), 1 deletion(-) create mode 100644 host/include/aarch64/host/crypto/clmul.h diff --git a/host/include/aarch64/host/cpuinfo.h b/host/include/aarch64/host/cpuinfo.h index 769626b098..fe8c3b3fd1 100644 --- a/host/include/aarch64/host/cpuinfo.h +++ b/host/include/aarch64/host/cpuinfo.h @@ -10,6 +10,7 @@ #define CPUINFO_LSE (1u << 1) #define CPUINFO_LSE2 (1u << 2) #define CPUINFO_AES (1u << 3) +#define CPUINFO_PMULL (1u << 4) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/host/include/aarch64/host/crypto/clmul.h b/host/include/aarch64/host/crypto/clmul.h new file mode 100644 index 0000000000..bb516d8b2f --- /dev/null +++ b/host/include/aarch64/host/crypto/clmul.h @@ -0,0 +1,41 @@ +/* + * AArch64 specific clmul acceleration. + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef AARCH64_HOST_CRYPTO_CLMUL_H +#define AARCH64_HOST_CRYPTO_CLMUL_H + +#include "host/cpuinfo.h" +#include + +/* + * 64x64->128 pmull is available with FEAT_PMULL. + * Both FEAT_AES and FEAT_PMULL are covered under the same macro. + */ +#ifdef __ARM_FEATURE_AES +# define HAVE_CLMUL_ACCEL true +#else +# define HAVE_CLMUL_ACCEL likely(cpuinfo & CPUINFO_PMULL) +#endif +#if !defined(__ARM_FEATURE_AES) && defined(CONFIG_ARM_AES_BUILTIN) +# define ATTR_CLMUL_ACCEL __attribute__((target("+crypto"))) +#else +# define ATTR_CLMUL_ACCEL +#endif + +static inline Int128 ATTR_CLMUL_ACCEL +clmul_64_accel(uint64_t n, uint64_t m) +{ + union { poly128_t v; Int128 s; } u; + +#ifdef CONFIG_ARM_AES_BUILTIN + u.v = vmull_p64((poly64_t)n, (poly64_t)m); +#else + asm(".arch_extension aes\n\t" + "pmull %0.1q, %1.1d, %2.1d" : "=w"(u.v) : "w"(n), "w"(m)); +#endif + return u.s; +} + +#endif /* AARCH64_HOST_CRYPTO_CLMUL_H */ diff --git a/util/cpuinfo-aarch64.c b/util/cpuinfo-aarch64.c index ababc39550..1d565b8420 100644 --- a/util/cpuinfo-aarch64.c +++ b/util/cpuinfo-aarch64.c @@ -56,12 +56,14 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) unsigned long hwcap = qemu_getauxval(AT_HWCAP); info |= (hwcap & HWCAP_ATOMICS ? CPUINFO_LSE : 0); info |= (hwcap & HWCAP_USCAT ? CPUINFO_LSE2 : 0); - info |= (hwcap & HWCAP_AES ? CPUINFO_AES: 0); + info |= (hwcap & HWCAP_AES ? CPUINFO_AES : 0); + info |= (hwcap & HWCAP_PMULL ? CPUINFO_PMULL : 0); #endif #ifdef CONFIG_DARWIN info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE") * CPUINFO_LSE; info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE2") * CPUINFO_LSE2; info |= sysctl_for_bool("hw.optional.arm.FEAT_AES") * CPUINFO_AES; + info |= sysctl_for_bool("hw.optional.arm.FEAT_PMULL") * CPUINFO_PMULL; #endif cpuinfo = info;