From patchwork Sat May 20 16:26:08 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684319 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448901wrt; Sat, 20 May 2023 09:31:27 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6nKA9mgYzsoz6VWJO4DpGPHaIlVeO+9A4wdJJbTt4tHz2hreCFomAgvY9t9nuDvvY33lqF X-Received: by 2002:ac8:5f8f:0:b0:3f3:9526:42fb with SMTP id j15-20020ac85f8f000000b003f3952642fbmr11337664qta.12.1684600287392; Sat, 20 May 2023 09:31:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600287; cv=none; d=google.com; s=arc-20160816; b=E7h+8KBcGPTEIz3jN59/6G6z/2+OmlILmCY4hC8OmEjQDC5TKEXO7OwLUJ8WVW++6f JAEEWinMv51xemjp6lm0wWFjwUzY9y5SEXVL02XMbdBbC35781WuNf6c5LeY+3zOhtHW E3jFnFbbhpxQ47Zt9H1ae+jigxEY8JkCdRmt3p/RotY/Z2vBHnFrQkUvibgxZr1jw4dB RHwOKrn1D2ht/TTZafqk41LzJTESPxW0kC1nsD/m2XM0fvj0PLdzcaI+lh4iw6s4T8Xi 1ynQx9MX2xzJ1T7UiF1lsX6Z/F7HDGl18VN3t1aIsFMbwbd3AiKoDaL7nav71MucmOjz lYAw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=qoslcpBGrNylzkKTGRsE4PymBBD7m5mdTsORg0M/q+0=; b=FR2lvwu0f30bITH23XoX+PylZ1BIUeZPrAE+Juidhy1tHNEPPMFZwXo1PhiJgbo/PZ D3KU79j+VXfzjQM+loS8MYs3Lhbztli3GTheu2WXy32B/mi4URAE7EgGf0L9jeS+Y15V QEGIpi/svP/ddJslJpUxvEtgz4DI721uWzyIhNf7RJ9EPrqP/cr+wrFtMiKfP//CB+Qq sRdjURXUSwC/6Mu06+b6T570op7gDv6IyOo/xUiLGPguJmwPkQ5mgy5w8/ORyUs8G1Jp G1UxhFVNdUyxV0aqh+t4aS/jHz7UFRZp+00Z1PoehSpsBXOQBMTEldR1DaG/4ZQGhdxL eJTg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T64ArRbU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id r13-20020ac85c8d000000b003e970fedc49si1346706qta.51.2023.05.20.09.31.27 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:27 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=T64ArRbU; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPn-0003Wq-9W; Sat, 20 May 2023 12:26:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPl-0003VA-PK for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:41 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPi-000349-I9 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:41 -0400 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1ae507af2e5so14196295ad.1 for ; Sat, 20 May 2023 09:26:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684599997; x=1687191997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qoslcpBGrNylzkKTGRsE4PymBBD7m5mdTsORg0M/q+0=; b=T64ArRbUugSqIW7icPZKpZ9mqaN+wHRWLTFNW+h7CUqPl8ADdAGpTHdprrVGukBILX HKp4EAvdyZRK6GTkExyn3zJU9f1bfajV/tOTa9JYHgC2iO42w4BkODy8ShVHlIPGvhwL zow1U/aif9AhGdkFlMYiGXPHNuIHpUdBMVjg7neSb9/UP9ZCFmc+LsyA1tB0+NNw+gdm XtyXJ62o7QRIO0btYuaCxr4StKwEHqJWiMl19/Bj1n8VPTMReQv222t1T5hxEayH4/Z+ hoUh7ZTkFceww5lYUJF0AaFnFCAkKAxoUiZkttYzzvgSOtqj4Ce5KxhSyR3wg7PT7weB MtWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684599997; x=1687191997; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qoslcpBGrNylzkKTGRsE4PymBBD7m5mdTsORg0M/q+0=; b=GpNUS/VqMOd+1euOWqD1Dj0XeUDsyJi+2+GyFJokXRPaGG8GvXlAxhcCqXFZFmigZX NGjFhYAA43L5kN4qvxKpBfA0TKyPJSR8PIXPsYdD3FayA9FdlJWB4YtKzzy9KMoThj58 4gKge1WA4b0PbeUV+nUlII+cP91n7ies4r00KEeWKaOiQT+M3A/gWc0eepxnEEwBmpUm PfPyXHaNe9OLWcMbADJOGlP/AJ0Uw4gq3j9UOAnADm+DnY3WHd1KkbRPxH7+wznHdYv6 fgd4YNso6X2tYp/LN8xIRr9B/aCTZLJQtmJRICzyEFSZRmZmJiWUNB/0yB/+XRkhEvPw Ukgg== X-Gm-Message-State: AC+VfDzvhODP1RBVGwz9HsnRz5rJiatEs8i726ONDF0oN7IJ1rwiBJY1 5YUoRMlFrTx7UN4QqN5eztYPUllUzXRKXGKOBk4= X-Received: by 2002:a17:903:2444:b0:1a6:9762:6eed with SMTP id l4-20020a170903244400b001a697626eedmr7579258pls.22.1684599997354; Sat, 20 May 2023 09:26:37 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:36 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Juan Quintela Subject: [PATCH 01/27] util: Introduce host-specific cpuinfo.h Date: Sat, 20 May 2023 09:26:08 -0700 Message-Id: <20230520162634.3991009-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The entire contents of the header is host-specific, but the existence of such a header is not, which could prevent some host specific ifdefs at the top of the file for the include. Add host/include/{arch,generic} to the project arguments. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé Reviewed-by: Alex Bennée --- host/include/generic/host/cpuinfo.h | 4 ++++ meson.build | 8 ++++++++ 2 files changed, 12 insertions(+) create mode 100644 host/include/generic/host/cpuinfo.h diff --git a/host/include/generic/host/cpuinfo.h b/host/include/generic/host/cpuinfo.h new file mode 100644 index 0000000000..eca672064a --- /dev/null +++ b/host/include/generic/host/cpuinfo.h @@ -0,0 +1,4 @@ +/* + * No host specific cpu indentification. + * SPDX-License-Identifier: GPL-2.0-or-later + */ diff --git a/meson.build b/meson.build index 0a5cdefd4d..4ffc0d3e59 100644 --- a/meson.build +++ b/meson.build @@ -512,6 +512,14 @@ add_project_arguments('-iquote', '.', '-iquote', meson.current_source_dir() / 'include', language: all_languages) +host_include = meson.current_source_dir() / 'host/include/' +if fs.is_dir(host_include / host_arch) + add_project_arguments('-iquote', host_include / host_arch, + language: all_languages) +endif +add_project_arguments('-iquote', host_include / 'generic', + language: all_languages) + sparse = find_program('cgcc', required: get_option('sparse')) if sparse.found() run_target('sparse', From patchwork Sat May 20 16:26:09 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684315 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448687wrt; Sat, 20 May 2023 09:30:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ46EeqN/MeM1cnHvut+4JPawkSkKI61u1uXTJVniLGk9EeLius/KVisXAKjHJ0Oii4QrvIX X-Received: by 2002:a05:6214:d0b:b0:616:4e24:ff28 with SMTP id 11-20020a0562140d0b00b006164e24ff28mr9784609qvh.0.1684600257653; Sat, 20 May 2023 09:30:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600257; cv=none; d=google.com; s=arc-20160816; b=yOg65ezzGeqd6p9AbQBzKITM/7w+m/8/olwoh0EDTxB+rUeMdi4V5sHBfZC84Go2YA Tu355GHXbs5lGBM6yMAbMnTT2E+gF6AGtkzlDMhd4HS7CXSpjKWrNf83Oe4B6rWMx8mo OZiH1/xFUpcEoyrbSRf+s0p6XI/jSaPxTImM/tTOa4Vx2c/UK4BNJLdye7vGR4BkNWII 186pRFv5GhvQOif23gcwMc8nNRDvMPg+YcQ1kxL3i8ZoItmA2AG6qI7vrRGA+JBnJtiV gtN9mKxyhYswvUVIhfmReS84T7T67BIXs5tOknU67VZXzAZktdO7klRT1dFSum1Z39xE 2g2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=TYqOsGeHTDLXuN6eBb6Sa53nknSV8fJ9idFTpYYJCX7o7HImXT/3RgwAg6y/OxtANu wPHuGtUKgyClmuxdcDKq+3a39L+ogaf3yywBDi8LvKIkvdahWit8dNe1/nhd144Mwqjy sPnMjLn1z95HSWvZW8QanmH9grwxcV1OLgH7ozEOWm2ted1p0YeaCHFgsitVgwCg0vqq lf9HPV4ySW7sQnLE9txqFc4pGCGK4dUGJYGgeKdFPve0tpFGiaeu2uw2PXTgUrtlE0DN 6W57/gZ//Xa0jvJSoKg1+Q6C0BSLMoWhe/F1msJpWTV1WfFNazxQYOAum92ciIwIVOOQ FcOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AdVsLY18; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z14-20020a05620a100e00b0075aca7d2290si1056327qkj.93.2023.05.20.09.30.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=AdVsLY18; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPp-0003ZM-8N; Sat, 20 May 2023 12:26:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPn-0003XE-Gp for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:43 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPj-00034M-L4 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:43 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1ae3ed1b0d6so31067385ad.3 for ; Sat, 20 May 2023 09:26:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684599998; x=1687191998; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=AdVsLY18YJRkO0YFsCpCabtaphGX1HNg/d16gnu0nM+82yihUeUwpJvb8cIkt9vZDJ jheJS6o49BuXyHfzf8A3qH7eyNITu6irGuUn3lSlXKF4DpxaQHeIAb5SW8RYzygqHuU1 ZAgoIgQlsVykpO/zgDlyo5jwSq5bUvCca4aQBKQ6gnm6zvmf04+0OIPPJx/xpgvrhfNp MTHe+YhU8Vi0dP3+WvyEdqlVYyhru4zgjVxogD5V5F1aUtHREbUfjFwuVcjkW1CxNchd cCZA0LST9B8MNMRyuS0QFUPrBXQ1IAp//BPJF2pZWzhnjW/WsQaYr8DcRqI+33QHxlIO HlNQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684599998; x=1687191998; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wlUqj841x578nzMKn5ImdhcaDhdsmsJg4wjQFvgryxE=; b=dPiElw5i0Trg+5wsW40j+DEx90mlGURweA05+AeO/QdhIZahXyRJryLuLvDMho9FIv jtUp+fN8ScPJJAzjcnNWWBHavExeVDFP8QKgWIZiA0I7f0e2GnBGWcl2c1Y5/F+atljE FCKgtjtkO+wMBKXNOrruviFFpCrmCZHP5ukwH5oTxQOrUkaA3s+ofvPDVWjU+lJ/60wv EdbaQ9PaYzpRXPsUYWPAvR323+wz7gHqtqRsSrMS2RbxKSWAKjFn6Hhn4FKLcZWG7I1f K1PagAT5rIhKhl/KxLGp+zHg4Wjd2KqRK1xsiBLSAesvlkBMQu261JsuwDDMeIqZuUhp Nc4w== X-Gm-Message-State: AC+VfDwoCE/NUguuM4uIEC9sCjSEezK9v4dg50xdQRCedRdD7d9dOYeC F+OpIadSuNeSbvNy7pi2LHNOvz/ZQtwkiZz79Rc= X-Received: by 2002:a17:902:eccc:b0:1ae:8741:d1d9 with SMTP id a12-20020a170902eccc00b001ae8741d1d9mr5515732plh.17.1684599998154; Sat, 20 May 2023 09:26:38 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:37 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Juan Quintela Subject: [PATCH 02/27] util: Add cpuinfo-i386.c Date: Sat, 20 May 2023 09:26:09 -0700 Message-Id: <20230520162634.3991009-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Add cpuinfo.h for i386 and x86_64, and the initialization for that in util/. Populate that with a slightly altered copy of the tcg host probing code. Other uses of cpuid.h will be adjusted one patch at a time. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson Reviewed-by: Alex Bennée --- host/include/i386/host/cpuinfo.h | 38 ++++++++++++ host/include/x86_64/host/cpuinfo.h | 1 + util/cpuinfo-i386.c | 97 ++++++++++++++++++++++++++++++ util/meson.build | 4 ++ 4 files changed, 140 insertions(+) create mode 100644 host/include/i386/host/cpuinfo.h create mode 100644 host/include/x86_64/host/cpuinfo.h create mode 100644 util/cpuinfo-i386.c diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h new file mode 100644 index 0000000000..e6f7461378 --- /dev/null +++ b/host/include/i386/host/cpuinfo.h @@ -0,0 +1,38 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for x86. + */ + +#ifndef HOST_CPUINFO_H +#define HOST_CPUINFO_H + +/* Digested version of */ + +#define CPUINFO_ALWAYS (1u << 0) /* so cpuinfo is nonzero */ +#define CPUINFO_CMOV (1u << 1) +#define CPUINFO_MOVBE (1u << 2) +#define CPUINFO_LZCNT (1u << 3) +#define CPUINFO_POPCNT (1u << 4) +#define CPUINFO_BMI1 (1u << 5) +#define CPUINFO_BMI2 (1u << 6) +#define CPUINFO_SSE2 (1u << 7) +#define CPUINFO_SSE4 (1u << 8) +#define CPUINFO_AVX1 (1u << 9) +#define CPUINFO_AVX2 (1u << 10) +#define CPUINFO_AVX512F (1u << 11) +#define CPUINFO_AVX512VL (1u << 12) +#define CPUINFO_AVX512BW (1u << 13) +#define CPUINFO_AVX512DQ (1u << 14) +#define CPUINFO_AVX512VBMI2 (1u << 15) +#define CPUINFO_ATOMIC_VMOVDQA (1u << 16) + +/* Initialized with a constructor. */ +extern unsigned cpuinfo; + +/* + * We cannot rely on constructor ordering, so other constructors must + * use the function interface rather than the variable above. + */ +unsigned cpuinfo_init(void); + +#endif /* HOST_CPUINFO_H */ diff --git a/host/include/x86_64/host/cpuinfo.h b/host/include/x86_64/host/cpuinfo.h new file mode 100644 index 0000000000..67debab9a0 --- /dev/null +++ b/host/include/x86_64/host/cpuinfo.h @@ -0,0 +1 @@ +#include "host/include/i386/host/cpuinfo.h" diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c new file mode 100644 index 0000000000..434319aa71 --- /dev/null +++ b/util/cpuinfo-i386.c @@ -0,0 +1,97 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for x86. + */ + +#include "qemu/osdep.h" +#include "host/cpuinfo.h" +#ifdef CONFIG_CPUID_H +# include "qemu/cpuid.h" +#endif + +unsigned cpuinfo; + +/* Called both as constructor and (possibly) via other constructors. */ +unsigned __attribute__((constructor)) cpuinfo_init(void) +{ + unsigned info = cpuinfo; + + if (info) { + return info; + } + +#ifdef CONFIG_CPUID_H + unsigned max, a, b, c, d, b7 = 0, c7 = 0; + + max = __get_cpuid_max(0, 0); + + if (max >= 7) { + __cpuid_count(7, 0, a, b7, c7, d); + info |= (b7 & bit_BMI ? CPUINFO_BMI1 : 0); + info |= (b7 & bit_BMI2 ? CPUINFO_BMI2 : 0); + } + + if (max >= 1) { + __cpuid(1, a, b, c, d); + + info |= (d & bit_CMOV ? CPUINFO_CMOV : 0); + info |= (d & bit_SSE2 ? CPUINFO_SSE2 : 0); + info |= (c & bit_SSE4_1 ? CPUINFO_SSE4 : 0); + info |= (c & bit_MOVBE ? CPUINFO_MOVBE : 0); + info |= (c & bit_POPCNT ? CPUINFO_POPCNT : 0); + + /* For AVX features, we must check available and usable. */ + if ((c & bit_AVX) && (c & bit_OSXSAVE)) { + unsigned bv = xgetbv_low(0); + + if ((bv & 6) == 6) { + info |= CPUINFO_AVX1; + info |= (b7 & bit_AVX2 ? CPUINFO_AVX2 : 0); + + if ((bv & 0xe0) == 0xe0) { + info |= (b7 & bit_AVX512F ? CPUINFO_AVX512F : 0); + info |= (b7 & bit_AVX512VL ? CPUINFO_AVX512VL : 0); + info |= (b7 & bit_AVX512BW ? CPUINFO_AVX512BW : 0); + info |= (b7 & bit_AVX512DQ ? CPUINFO_AVX512DQ : 0); + info |= (c7 & bit_AVX512VBMI2 ? CPUINFO_AVX512VBMI2 : 0); + } + + /* + * The Intel SDM has added: + * Processors that enumerate support for Intel® AVX + * (by setting the feature flag CPUID.01H:ECX.AVX[bit 28]) + * guarantee that the 16-byte memory operations performed + * by the following instructions will always be carried + * out atomically: + * - MOVAPD, MOVAPS, and MOVDQA. + * - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128. + * - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded + * with EVEX.128 and k0 (masking disabled). + * Note that these instructions require the linear addresses + * of their memory operands to be 16-byte aligned. + * + * AMD has provided an even stronger guarantee that processors + * with AVX provide 16-byte atomicity for all cachable, + * naturally aligned single loads and stores, e.g. MOVDQU. + * + * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 + */ + __cpuid(0, a, b, c, d); + if (c == signature_INTEL_ecx || c == signature_AMD_ecx) { + info |= CPUINFO_ATOMIC_VMOVDQA; + } + } + } + } + + max = __get_cpuid_max(0x8000000, 0); + if (max >= 1) { + __cpuid(0x80000001, a, b, c, d); + info |= (c & bit_LZCNT ? CPUINFO_LZCNT : 0); + } +#endif + + info |= CPUINFO_ALWAYS; + cpuinfo = info; + return info; +} diff --git a/util/meson.build b/util/meson.build index e1f1c39e10..b3be9fad5d 100644 --- a/util/meson.build +++ b/util/meson.build @@ -108,3 +108,7 @@ if have_block endif util_ss.add(when: 'CONFIG_LINUX', if_true: files('vfio-helpers.c')) endif + +if cpu in ['x86', 'x86_64'] + util_ss.add(files('cpuinfo-i386.c')) +endif From patchwork Sat May 20 16:26:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684296 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447639wrt; Sat, 20 May 2023 09:27:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4xHuQYLBFP4Hxg355X7uXJ2puib5D966jc9S61TS3g6FBYNin+1DjJOL1Da1l+F02kf/iq X-Received: by 2002:ac8:59cd:0:b0:3f5:29b8:7846 with SMTP id f13-20020ac859cd000000b003f529b87846mr9211252qtf.65.1684600077186; Sat, 20 May 2023 09:27:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600077; cv=none; d=google.com; s=arc-20160816; b=xjF7jtMYUBzgRDZN0vPXghclvRODjcIRD5GmAv/6nlLyglvvL8yybQaDgz+TUMcpDa /uHYG4pIg1TKF9yMkuhrbyzgVHKVZ+ba6XLI3RBV70IZ5LKDtdnst9FQKyAN9HukkGU8 ERExYHb8OjqWu1EBL4TuoJgfoJ6K8HOP+kCrBI0tXOkIoHy4+3UVL3Sf8haHX7ybsTop KISrI2MQDe0etUWdcJTtPE769Px+/Lrodb2kGN/WPGZsWiIaVsj1rBVYjrsi04n42aXN b3viN1h1lyuuy6FBjVQAUSVkoeMdJylqfjMr0a94Fu7KGOzUf+0D87GU5YiQNccIS8in pZZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=mk0dDz5gmRRTSAPj0jzWEkv1hQDxm/1QwYfm3/sQAKxoy1cRgmvGqEwQAdPbax0oj8 8/zThtTqbZ33PHBr/o1qQbL9xhxtDVAgSyqZqJhkWDGh1h5LRKCPIVHe4Mz4K38mXctR yxH8w8E0IwS+ZkT+RDJNLw7w3nlgRb/Nf8lav/QWcdvUdxIDiFuiu/ZMOYQYRjrNxkbj SUFRwA1u/Lh1NLi+LzWPJdzIIonn93ayyENs44sy3Do3QnaDiNTxuq9aaD5UmMd8klNu /PEMHAEpKIccAf1ovv1urCizmcAwM904ub/aHpILvK9Bpa3Q8QtPHQVkETt9//SgnjVR 1Qxw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NlifBBE1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id j16-20020a05622a039000b003f391e7d585si1322231qtx.543.2023.05.20.09.27.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:27:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NlifBBE1; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPo-0003Yp-LQ; Sat, 20 May 2023 12:26:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPn-0003Wr-8G for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:43 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPk-00034U-77 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:42 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1ae4c5e1388so43575155ad.1 for ; Sat, 20 May 2023 09:26:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684599999; x=1687191999; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=NlifBBE1Sa6pTUAOl4gfdmRW6vLX2I8IGs1+HiFEjYALEjk2ZZ+t2KUXd6lGZ7hQDZ EBknP+1txz5J7xK7VkWQ584fETgPX0yzmEz2hZFq7GDBepVh+tITghv81Bei7o3PhA9W O+C4DJCrcWg+KDdXZn1FUuS1SPiqo/gykqI4P7/UV+s0ud09xP37yPQggAqAb9jm+lH0 KVMUKydJc05VUgNfN2ieVYHbOVoLAZNx8T7t9Z3w0GzthFLLgoweuRflwUI0fK4BbcvD yGy+TU3SusHeH9DiY+fMz3tX4qTsnknrsxj/JcKIkK2sbSAeHg3LVlHiLvzG1LDsqMfR 3ZIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684599999; x=1687191999; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8wi2bOWjMyVLEFUf4QBGubplY21YXqeLF4j5hewlIPc=; b=fLHQBXyQ+/thsqQSvT+G2Klai/dOKiHOguJnA5wuO7OL533kdpwGI6t66W9X/0yWBF n/Znv0WCsQh7GahmwgOlD9BenSVaHGA6aSuLl58RkhZieNcXBubVzsSBJXBu42dHSu2D OeZjzWsrI0VqpBNQLZJ8iPdo7OHWjvd+AdfIDZ6Nt3EbJg8g9xJBVLllOt9jVM2PfhLx fAZ4yvZOcBymPsiQ6C204X+NR398Y1xI57AjyE7qSql8Si+nS7RI0KUn48kSZblMr0QT 7gpCyOXTmZ4R67A04ITx9AJ8bNHAvdd2zYyqn0OxwdVAAc7/9H2iQcL6MAuN9O5aATIh Og1Q== X-Gm-Message-State: AC+VfDwrIXfWqeniCJRC1iSu8YiKQPivar5T2mM4E8myX/Hf1NWZcj7E o7OcVBo+CaL1xSq4LNFHXhuYlwNldQ7LzWcVehM= X-Received: by 2002:a17:902:ec90:b0:1ad:e633:ee8f with SMTP id x16-20020a170902ec9000b001ade633ee8fmr7592178plg.32.1684599998981; Sat, 20 May 2023 09:26:38 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH 03/27] util: Add i386 CPUINFO_ATOMIC_VMOVDQU Date: Sat, 20 May 2023 09:26:10 -0700 Message-Id: <20230520162634.3991009-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Add a bit to indicate when VMOVDQU is also atomic if aligned. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- host/include/i386/host/cpuinfo.h | 1 + util/cpuinfo-i386.c | 4 +++- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/host/include/i386/host/cpuinfo.h b/host/include/i386/host/cpuinfo.h index e6f7461378..a6537123cf 100644 --- a/host/include/i386/host/cpuinfo.h +++ b/host/include/i386/host/cpuinfo.h @@ -25,6 +25,7 @@ #define CPUINFO_AVX512DQ (1u << 14) #define CPUINFO_AVX512VBMI2 (1u << 15) #define CPUINFO_ATOMIC_VMOVDQA (1u << 16) +#define CPUINFO_ATOMIC_VMOVDQU (1u << 17) /* Initialized with a constructor. */ extern unsigned cpuinfo; diff --git a/util/cpuinfo-i386.c b/util/cpuinfo-i386.c index 434319aa71..ab6143d9e7 100644 --- a/util/cpuinfo-i386.c +++ b/util/cpuinfo-i386.c @@ -77,8 +77,10 @@ unsigned __attribute__((constructor)) cpuinfo_init(void) * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 */ __cpuid(0, a, b, c, d); - if (c == signature_INTEL_ecx || c == signature_AMD_ecx) { + if (c == signature_INTEL_ecx) { info |= CPUINFO_ATOMIC_VMOVDQA; + } else if (c == signature_AMD_ecx) { + info |= CPUINFO_ATOMIC_VMOVDQA | CPUINFO_ATOMIC_VMOVDQU; } } } From patchwork Sat May 20 16:26:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684312 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448352wrt; Sat, 20 May 2023 09:30:09 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4638AsW0cI8Tce+yDdOLGe+mVDYXLZNTt2jCMXaNv/tmjZgpHPiEEGBeyBOCeu1YPM2iCk X-Received: by 2002:ac8:570b:0:b0:3f2:384:9292 with SMTP id 11-20020ac8570b000000b003f203849292mr9719971qtw.34.1684600209210; Sat, 20 May 2023 09:30:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600209; cv=none; d=google.com; s=arc-20160816; b=On00eZO2T9nbhls7Omt9Ju5VyYBiGQNaNke4u9zpF3l1YP5ZgATCzGXxqcS0s9LgOa J4y2bT+G1cj1rxaS1QA8PrLpt3dVKYemLNi2OJFqTTD2nJvPoY8t14Z29bHr2/dnZS4l rDGqJjhsgTT5QZBL0SLRgnr30NBkPUuyBOM4OLDmEGn2xS72P0yUO9+X8V7iUdpSpLRT ilCJ/SFRRntBNGjfoEhN0aiL4LlbYaQl5NejSy3A52Yy9n3wZOogAzpF4q5YF0l4hEgY 7mYt7cDPIDSdPFtqgBOFsyMFuWtkbw5rfOw3iJQbPRn+u4rLUms1rM5v8toR0v6XB+w5 IrUg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=i/gWsJoZAiQf7NAbYtXvnXM9n+B/DceSNMQENU72UVMy/s4/uO9lX1lLqx/2QkzSxe nXr6OCWqbyq9fCkI+IfMZvv5/1TAkYAFMw6Mf7agtJ9QlgaiGl0inMP40nEEtLHb38mC 8mtOdIpeeSTrhMdqckKKxqzk7NQZsm9nnsB6v89KLLdFF5W56H1xNVhDSwET3H2+Eahh 8mVpSS42g2IC6Vs2dye7tuSh0yzUEMWRWQ63PHFV35tshLLv2jqQUWbXqe0HoUUr45+L CIKBbSzO6K9stelR7vNdJqDckmo8y7hqN316/MA6AhJ0nHowWt3NcgmzqlvnC3CaAt4t hPZg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="j0ddd/BW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v4-20020ac85784000000b003f205e65e7dsi1373392qta.234.2023.05.20.09.30.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="j0ddd/BW"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPp-0003ZZ-G6; Sat, 20 May 2023 12:26:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPn-0003Xv-Rw for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:43 -0400 Received: from mail-pg1-x52c.google.com ([2607:f8b0:4864:20::52c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPl-00034w-B0 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:43 -0400 Received: by mail-pg1-x52c.google.com with SMTP id 41be03b00d2f7-53033a0b473so3076172a12.0 for ; Sat, 20 May 2023 09:26:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600000; x=1687192000; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=j0ddd/BWfqAwWCBp9V2e2w0sN/fZcGA7IrEFyVG8TlViwstyE6fti3rUSDbLBXv1sA ZL9n9Aa/qGl9Li5knkAJo40OmALD8cvNfUz8Xj53+TnABgZP/7SDvnC0x4I5vRhOt3WQ cHW57QotbSIEgcOYw4YoI9Y/z1Oi71nMUeFSBrKcOZP7Gnj9nAdBF99L3oispDWCzW0F yj65wTjbo29g4W/C3BjJrJkAc76yLpNqOZ19FViSwdE4lLSXmTaAtE7XF8yiIrwW6Ilz Z4JL84lDFC13wzmLDXWoK7KbW9VUTfNSh9gW+UlroD/klNy58v3ImMdBdKr7+iLc3QAo EOhA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600000; x=1687192000; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=frmGIXxr84Ww41VQFYmyyhkxPgV+QZOjqj6i0A0DQ+A=; b=PJW1cuRNKhg/ijmHMt4t0KyhUjQyooOlpBQsmUnd9X6peO8zajkWpe2e3BO44z/41P kVgRvTxfUjqqNgTx56l9Rn+n/+ueHCUu+GPKpG5cgvE+LhuF+UbSekVHX85tpz8Hw8WU 3xbDXsnwpghZaR5soPnuMsMtcgNA10g9L3nAnsw7wxL6+420paRDQjFOoEenhU8UKVxZ 0ADKpz5ZiKJlUOZl8Kf/01kfA3rsEwAgjWy0twOCoXVM+LKywkJYDjSrOxyYY7Jwskd9 4wT/lIgAhe0zmTt6n8v1o01CIKIFvzEK4WAEySJAScqJtLlNf/vYYje64SndtuSKfU5I Qarg== X-Gm-Message-State: AC+VfDyEikgKtr4xXTsUkJjFCqJ1MOCHJtSo4kORJaJuGt6B1BQl2Jyb ABzHdKSQdHpEv2v3JQLkcCQXCIR9udtwieP4Uok= X-Received: by 2002:a17:902:c944:b0:1a6:46f2:4365 with SMTP id i4-20020a170902c94400b001a646f24365mr7895420pla.30.1684599999734; Sat, 20 May 2023 09:26:39 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH 04/27] tcg/i386: Use host/cpuinfo.h Date: Sat, 20 May 2023 09:26:11 -0700 Message-Id: <20230520162634.3991009-5-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::52c; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x52c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use the CPUINFO_* bits instead of the individual boolean variables that we had been using. Remove all of the init code that was moved over to cpuinfo-i386.c. Note that have_avx512* check both AVX512{F,VL}, as we had previously done during tcg_target_init. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- tcg/i386/tcg-target.h | 28 +++++---- tcg/i386/tcg-target.c.inc | 123 ++------------------------------------ 2 files changed, 22 insertions(+), 129 deletions(-) diff --git a/tcg/i386/tcg-target.h b/tcg/i386/tcg-target.h index 0b5a2c68c5..0106946996 100644 --- a/tcg/i386/tcg-target.h +++ b/tcg/i386/tcg-target.h @@ -25,6 +25,8 @@ #ifndef I386_TCG_TARGET_H #define I386_TCG_TARGET_H +#include "host/cpuinfo.h" + #define TCG_TARGET_INSN_UNIT_SIZE 1 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 31 @@ -111,16 +113,22 @@ typedef enum { # define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_BY_REF #endif -extern bool have_bmi1; -extern bool have_popcnt; -extern bool have_avx1; -extern bool have_avx2; -extern bool have_avx512bw; -extern bool have_avx512dq; -extern bool have_avx512vbmi2; -extern bool have_avx512vl; -extern bool have_movbe; -extern bool have_atomic16; +#define have_bmi1 (cpuinfo & CPUINFO_BMI1) +#define have_popcnt (cpuinfo & CPUINFO_POPCNT) +#define have_avx1 (cpuinfo & CPUINFO_AVX1) +#define have_avx2 (cpuinfo & CPUINFO_AVX2) +#define have_movbe (cpuinfo & CPUINFO_MOVBE) +#define have_atomic16 (cpuinfo & CPUINFO_ATOMIC_VMOVDQA) + +/* + * There are interesting instructions in AVX512, so long as we have AVX512VL, + * which indicates support for EVEX on sizes smaller than 512 bits. + */ +#define have_avx512vl ((cpuinfo & CPUINFO_AVX512VL) && \ + (cpuinfo & CPUINFO_AVX512F)) +#define have_avx512bw ((cpuinfo & CPUINFO_AVX512BW) && have_avx512vl) +#define have_avx512dq ((cpuinfo & CPUINFO_AVX512DQ) && have_avx512vl) +#define have_avx512vbmi2 ((cpuinfo & CPUINFO_AVX512VBMI2) && have_avx512vl) /* optional instructions */ #define TCG_TARGET_HAS_div2_i32 1 diff --git a/tcg/i386/tcg-target.c.inc b/tcg/i386/tcg-target.c.inc index 8b9a5f00e5..bfe9d98b7e 100644 --- a/tcg/i386/tcg-target.c.inc +++ b/tcg/i386/tcg-target.c.inc @@ -158,42 +158,14 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) # define SOFTMMU_RESERVE_REGS 0 #endif -/* The host compiler should supply to enable runtime features - detection, as we're not going to go so far as our own inline assembly. - If not available, default values will be assumed. */ -#if defined(CONFIG_CPUID_H) -#include "qemu/cpuid.h" -#endif - /* For 64-bit, we always know that CMOV is available. */ #if TCG_TARGET_REG_BITS == 64 -# define have_cmov 1 -#elif defined(CONFIG_CPUID_H) -static bool have_cmov; +# define have_cmov true #else -# define have_cmov 0 -#endif - -/* We need these symbols in tcg-target.h, and we can't properly conditionalize - it there. Therefore we always define the variable. */ -bool have_bmi1; -bool have_popcnt; -bool have_avx1; -bool have_avx2; -bool have_avx512bw; -bool have_avx512dq; -bool have_avx512vbmi2; -bool have_avx512vl; -bool have_movbe; -bool have_atomic16; - -#ifdef CONFIG_CPUID_H -static bool have_bmi2; -static bool have_lzcnt; -#else -# define have_bmi2 0 -# define have_lzcnt 0 +# define have_cmov (cpuinfo & CPUINFO_CMOV) #endif +#define have_bmi2 (cpuinfo & CPUINFO_BMI2) +#define have_lzcnt (cpuinfo & CPUINFO_LZCNT) static const tcg_insn_unit *tb_ret_addr; @@ -3961,93 +3933,6 @@ static void tcg_out_nop_fill(tcg_insn_unit *p, int count) static void tcg_target_init(TCGContext *s) { -#ifdef CONFIG_CPUID_H - unsigned a, b, c, d, b7 = 0, c7 = 0; - unsigned max = __get_cpuid_max(0, 0); - - if (max >= 7) { - /* BMI1 is available on AMD Piledriver and Intel Haswell CPUs. */ - __cpuid_count(7, 0, a, b7, c7, d); - have_bmi1 = (b7 & bit_BMI) != 0; - have_bmi2 = (b7 & bit_BMI2) != 0; - } - - if (max >= 1) { - __cpuid(1, a, b, c, d); -#ifndef have_cmov - /* For 32-bit, 99% certainty that we're running on hardware that - supports cmov, but we still need to check. In case cmov is not - available, we'll use a small forward branch. */ - have_cmov = (d & bit_CMOV) != 0; -#endif - - /* MOVBE is only available on Intel Atom and Haswell CPUs, so we - need to probe for it. */ - have_movbe = (c & bit_MOVBE) != 0; - have_popcnt = (c & bit_POPCNT) != 0; - - /* There are a number of things we must check before we can be - sure of not hitting invalid opcode. */ - if (c & bit_OSXSAVE) { - unsigned bv = xgetbv_low(0); - - if ((bv & 6) == 6) { - have_avx1 = (c & bit_AVX) != 0; - have_avx2 = (b7 & bit_AVX2) != 0; - - /* - * There are interesting instructions in AVX512, so long - * as we have AVX512VL, which indicates support for EVEX - * on sizes smaller than 512 bits. We are required to - * check that OPMASK and all extended ZMM state are enabled - * even if we're not using them -- the insns will fault. - */ - if ((bv & 0xe0) == 0xe0 - && (b7 & bit_AVX512F) - && (b7 & bit_AVX512VL)) { - have_avx512vl = true; - have_avx512bw = (b7 & bit_AVX512BW) != 0; - have_avx512dq = (b7 & bit_AVX512DQ) != 0; - have_avx512vbmi2 = (c7 & bit_AVX512VBMI2) != 0; - } - - /* - * The Intel SDM has added: - * Processors that enumerate support for Intel® AVX - * (by setting the feature flag CPUID.01H:ECX.AVX[bit 28]) - * guarantee that the 16-byte memory operations performed - * by the following instructions will always be carried - * out atomically: - * - MOVAPD, MOVAPS, and MOVDQA. - * - VMOVAPD, VMOVAPS, and VMOVDQA when encoded with VEX.128. - * - VMOVAPD, VMOVAPS, VMOVDQA32, and VMOVDQA64 when encoded - * with EVEX.128 and k0 (masking disabled). - * Note that these instructions require the linear addresses - * of their memory operands to be 16-byte aligned. - * - * AMD has provided an even stronger guarantee that processors - * with AVX provide 16-byte atomicity for all cachable, - * naturally aligned single loads and stores, e.g. MOVDQU. - * - * See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 - */ - if (have_avx1) { - __cpuid(0, a, b, c, d); - have_atomic16 = (c == signature_INTEL_ecx || - c == signature_AMD_ecx); - } - } - } - } - - max = __get_cpuid_max(0x8000000, 0); - if (max >= 1) { - __cpuid(0x80000001, a, b, c, d); - /* LZCNT was introduced with AMD Barcelona and Intel Haswell CPUs. */ - have_lzcnt = (c & bit_LZCNT) != 0; - } -#endif /* CONFIG_CPUID_H */ - tcg_target_available_regs[TCG_TYPE_I32] = ALL_GENERAL_REGS; if (TCG_TARGET_REG_BITS == 64) { tcg_target_available_regs[TCG_TYPE_I64] = ALL_GENERAL_REGS; From patchwork Sat May 20 16:26:12 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684305 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448085wrt; Sat, 20 May 2023 09:29:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4xf77DMM3RVwz/4fsze1DMkCpwy1q6jK2P3pFS5Y49tQq1CqWsIxVUkhuh85nHbGMgpf/Y X-Received: by 2002:ac8:5705:0:b0:3f4:e0cc:7389 with SMTP id 5-20020ac85705000000b003f4e0cc7389mr10664609qtw.68.1684600164852; Sat, 20 May 2023 09:29:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600164; cv=none; d=google.com; s=arc-20160816; b=KOwOjJ7Q+7a+TuLtMDXhMWaCKKQ5yDsW8Q9TD8Bn2AQpZ4Hx/5eslY2BcIRZkDZj3O UL8js1pGTYd8VoVMccjDaDkrl+9vj9XBPPKFGiqHJYLcpnU3FEV/inxveacKyp+1q98c h9D8WArfMo51ecvHybpPQTwVxfN8klUiQVe0u++wWXeVQxs3sjv7q+jiAl6se7eOLKIY FFmCKWbh6FECUuNAIL3VdRmgbVy4CYmZmFVxanJZA7pAYQMGZ/++w4kAMuExwdhJJb/+ sGwxT3KdUbZdiBL+AH9SZzJIyCbdJVV7i7GvBEK8MFCXhCmsqDJaweewJzl010zc23f4 TaSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=0s8LM7sAwxI9sHohEbb/6koJ7+Sa7aGJNbr8QVUNjuSBou18PWWcvoK5mVzsVXiSrM NDvealYb1Vw7ZAHEoD9H74BBg6VxPORaMPJE6jIBBRmnQpEKI8RgDls8eJ+BoKjayyWZ Re/vUXIJ11sdXEmsNfNwNwe5aOKqrxUgv3iCgHgHWyJOErCJgkXSsVRvEaVZ9B1UYxJh C//SeszDijqU5bIULzKa2Eo61uS0rL3Iqr6FbWW212aYQcuiLG4WCyDglZJlAiwN1zxr 8y55sPF4EgmT6uI295ReGPAA027nacxBmiw2CIhWXiNgxnG/5iiAzlwaFrVMsbOXM+C3 1xCA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ftns1YCZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o7-20020a05622a008700b003f5279fc4e2si1289230qtw.472.2023.05.20.09.29.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ftns1YCZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPq-0003aC-1k; Sat, 20 May 2023 12:26:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPo-0003Ys-Qt for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:44 -0400 Received: from mail-pl1-x62b.google.com ([2607:f8b0:4864:20::62b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPl-000352-UQ for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:44 -0400 Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1ae3ed1b08eso44072265ad.0 for ; Sat, 20 May 2023 09:26:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600000; x=1687192000; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=ftns1YCZXVXo6UzN2ClFhxIZnc+TKeKlz2IlfyDXsQ4vzrk4S8ylrmBsFrzJZpIKcW 2O/ufe+sZHU+5XeyRtuL/NAbKBIVvMSAFJPMuLcKqgFgbMVwkM+pnMwm8gARKj7Cdo8i VFiR2c8mVbIQlbpXcwSGFAE6s3PsUmxaEANoF88MXDlF+7DfkGHbOCycdKp0Yx7wLLMD cz+aCqFe9m3MYqbPQzR9/TpAhQrxX91sYkGpvosCOPGFE97TlYn/G3/PnAeFJ1nj26ii IDyqbbNOWVIZ3pukwnRzVb9pmtMhrU3jfahxNaq8MMRLlCGm1rlVxp2ml54wJ/wX+hjp SsNg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600000; x=1687192000; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WcxZjIY/T80MjuA6OZ00h+5fCLAwNHjF9M1KXt4rlvU=; b=DehAoLg0/H6SlM2J4W5ezgV8FnpJjr/Kapmtx+eXIPdndLJtBFA7RjXtKYvjBG15sX vwc3g5gz3A1IcP/G6+SaWP4FXun63/NleR8VKKu79O9VgNT63LGzvNx1sINve99am1lF 3RlCKnFOnP8vfOvi2xs/M9ZaUxdi7CbAok+sh0vX/F+X/2sw5QYeWONNdG4a7aTC1SAg ueNiWBIQUP5fz9SK2+YE6SOoqjPJmib8rZV6T5uYW0fURaeXBzoWH4n2kSCwcGVRPnWB +aErlfa3Ij1uzrPSaaC0qx4xafvLqd757xsyOiCmfaqMJ7xokmDhWWRxkyhGQMDhmSCp gL+g== X-Gm-Message-State: AC+VfDyQKO1eO15LKHSeUgnfiOoK+X6b9u0V23hIoZRdjl+UneVHstPl 1PO9zKTQckTe9FsEc19v3lqtdQedQOncl2ZX6IU= X-Received: by 2002:a17:902:bb16:b0:1aa:ee36:40a5 with SMTP id im22-20020a170902bb1600b001aaee3640a5mr5204155plb.34.1684600000535; Sat, 20 May 2023 09:26:40 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 05/27] util/bufferiszero: Use i386 host/cpuinfo.h Date: Sat, 20 May 2023 09:26:12 -0700 Message-Id: <20230520162634.3991009-6-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62b; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62b.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use cpuinfo_init() during init_accel(), and the variable cpuinfo during test_buffer_is_zero_next_accel(). Adjust the logic that cycles through the set of accelerators for testing. Signed-off-by: Richard Henderson --- util/bufferiszero.c | 126 ++++++++++++++++---------------------------- 1 file changed, 45 insertions(+), 81 deletions(-) diff --git a/util/bufferiszero.c b/util/bufferiszero.c index 1886bc5ba4..d3c14320ef 100644 --- a/util/bufferiszero.c +++ b/util/bufferiszero.c @@ -24,6 +24,7 @@ #include "qemu/osdep.h" #include "qemu/cutils.h" #include "qemu/bswap.h" +#include "host/cpuinfo.h" static bool buffer_zero_int(const void *buf, size_t len) @@ -184,111 +185,74 @@ buffer_zero_avx512(const void *buf, size_t len) } #endif /* CONFIG_AVX512F_OPT */ - -/* Note that for test_buffer_is_zero_next_accel, the most preferred - * ISA must have the least significant bit. - */ -#define CACHE_AVX512F 1 -#define CACHE_AVX2 2 -#define CACHE_SSE4 4 -#define CACHE_SSE2 8 - -/* Make sure that these variables are appropriately initialized when +/* + * Make sure that these variables are appropriately initialized when * SSE2 is enabled on the compiler command-line, but the compiler is * too old to support CONFIG_AVX2_OPT. */ #if defined(CONFIG_AVX512F_OPT) || defined(CONFIG_AVX2_OPT) -# define INIT_CACHE 0 -# define INIT_ACCEL buffer_zero_int +# define INIT_USED 0 +# define INIT_LENGTH 0 +# define INIT_ACCEL buffer_zero_int #else # ifndef __SSE2__ # error "ISA selection confusion" # endif -# define INIT_CACHE CACHE_SSE2 -# define INIT_ACCEL buffer_zero_sse2 +# define INIT_USED CPUINFO_SSE2 +# define INIT_LENGTH 64 +# define INIT_ACCEL buffer_zero_sse2 #endif -static unsigned cpuid_cache = INIT_CACHE; +static unsigned used_accel = INIT_USED; +static unsigned length_to_accel = INIT_LENGTH; static bool (*buffer_accel)(const void *, size_t) = INIT_ACCEL; -static int length_to_accel = 64; -static void init_accel(unsigned cache) +static unsigned __attribute__((noinline)) +select_accel_cpuinfo(unsigned info) { - bool (*fn)(const void *, size_t) = buffer_zero_int; - if (cache & CACHE_SSE2) { - fn = buffer_zero_sse2; - length_to_accel = 64; - } -#ifdef CONFIG_AVX2_OPT - if (cache & CACHE_SSE4) { - fn = buffer_zero_sse4; - length_to_accel = 64; - } - if (cache & CACHE_AVX2) { - fn = buffer_zero_avx2; - length_to_accel = 128; - } -#endif + static const struct { + unsigned bit; + unsigned len; + bool (*fn)(const void *, size_t); + } all[] = { #ifdef CONFIG_AVX512F_OPT - if (cache & CACHE_AVX512F) { - fn = buffer_zero_avx512; - length_to_accel = 256; - } + { CPUINFO_AVX512F, 256, buffer_zero_avx512 }, #endif - buffer_accel = fn; +#ifdef CONFIG_AVX2_OPT + { CPUINFO_AVX2, 128, buffer_zero_avx2 }, + { CPUINFO_SSE4, 64, buffer_zero_sse4 }, +#endif + { CPUINFO_SSE2, 64, buffer_zero_sse2 }, + { CPUINFO_ALWAYS, 0, buffer_zero_int }, + }; + + for (unsigned i = 0; i < ARRAY_SIZE(all); ++i) { + if (info & all[i].bit) { + length_to_accel = all[i].len; + buffer_accel = all[i].fn; + return all[i].bit; + } + } + return 0; } #if defined(CONFIG_AVX512F_OPT) || defined(CONFIG_AVX2_OPT) -#include "qemu/cpuid.h" - -static void __attribute__((constructor)) init_cpuid_cache(void) +static void __attribute__((constructor)) init_accel(void) { - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - unsigned cache = 0; - - if (max >= 1) { - __cpuid(1, a, b, c, d); - if (d & bit_SSE2) { - cache |= CACHE_SSE2; - } - if (c & bit_SSE4_1) { - cache |= CACHE_SSE4; - } - - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - unsigned bv = xgetbv_low(0); - __cpuid_count(7, 0, a, b, c, d); - if ((bv & 0x6) == 0x6 && (b & bit_AVX2)) { - cache |= CACHE_AVX2; - } - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512F)) { - cache |= CACHE_AVX512F; - } - } - } - cpuid_cache = cache; - init_accel(cache); + used_accel = select_accel_cpuinfo(cpuinfo_init()); } #endif /* CONFIG_AVX2_OPT */ bool test_buffer_is_zero_next_accel(void) { - /* If no bits set, we just tested buffer_zero_int, and there - are no more acceleration options to test. */ - if (cpuid_cache == 0) { - return false; - } - /* Disable the accelerator we used before and select a new one. */ - cpuid_cache &= cpuid_cache - 1; - init_accel(cpuid_cache); - return true; + /* + * Accumulate the accelerators that we've already tested, and + * remove them from the set to test this round. We'll get back + * a zero from select_accel_cpuinfo when there are no more. + */ + unsigned used = select_accel_cpuinfo(cpuinfo & ~used_accel); + used_accel |= used; + return used; } static bool select_accel_fn(const void *buf, size_t len) From patchwork Sat May 20 16:26:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684314 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448631wrt; Sat, 20 May 2023 09:30:50 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7BwqAgYJAux8nvMesIWMiR9CcRVKgMz4S1khnhIh+XBMIBqvpXvyE61X/8x6OGNuwe9Jti X-Received: by 2002:ad4:5967:0:b0:5e0:ad80:6846 with SMTP id eq7-20020ad45967000000b005e0ad806846mr10243188qvb.0.1684600250382; Sat, 20 May 2023 09:30:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600250; cv=none; d=google.com; s=arc-20160816; b=X9LNLvJj17VxCjRo2PCwFkJcwe+RX6ypiPKWyCtDmvWBIAQPAe5ZGqlcrpaBKmBcWz kYjPTU7OSk71i1v4pTN9Lu9J2ppSR8YUnFUaIsZcaa8J0RWR9GsyjOYT7UG6sEMbUQnP ES7tFNRBYAamW3vEv3w1X8khKbqZnJAnS/RF6PGc8Eg8nYQZctStrjXSiZylhidtpQc5 rvllgrJnBxyxz92JLjs12cVNrx38ZbznxAm5DvktfEXEVX2KKJJyMAjIqOHXb0FBIBad KK70G3N4gLeY5m2tXgayKfq9vxEUj6uQkryRqSmiJexsMebm4iMMbf/ARjlHaQ7AvpHe RIrA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=c7P7Vin8L5oOdklYlQpcFJQuSwSHqEwu5asdVawy7fbFE94hKlStLwcNnlaieHNl1y pFsYvfIhc+h1EPSPvn0GtmeCTK261C+naogvs/xYnelv0v0ncPGN0fw4SfbQU185yqXy BX9nAP9TquDMdtzWZxbEwL50i+gMyn+b6kXl26spF9Ux24+hA90V55k7sii81USE4IKf FbuTKHyFLYVUPvI/7RJ+kwrNeWqf/hZvspZ7FEHgsl0xW7jPb1Q1edBFzdzwUgGTnJyX X697H5NUqbkApe8xVqWubeAWuXEmaJENCs21e4vvzKa2H3G2wJY1xhJ/q6Du1YWsoGNA 1MiQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S3C5W9l2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f22-20020ae9ea16000000b0074e211374f8si984228qkg.711.2023.05.20.09.30.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:50 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=S3C5W9l2; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPq-0003aV-MI; Sat, 20 May 2023 12:26:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPp-0003ZX-Ck for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:45 -0400 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPm-00035d-Tg for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:45 -0400 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1ae557aaf1dso37487775ad.2 for ; Sat, 20 May 2023 09:26:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600001; x=1687192001; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=S3C5W9l29tHbB0CbIR/XyPKetPQ5zqRW++q9amRBFfI0RpOpWpOxN3oriGV5TJUupa nku9WoZdEuvtS/aKLoVKUEIu/AOiYzP+DYyUbPlhFCJITGWhZzYR5exxi3lhUyTBQOLI F/TP6QY9D2o3OejOXMX/j+p2wD2u1DhvmkjY3n8Xo3KnrFA40YQffqJrrcAxx5uss3JL LvpwrwUbFn83J4Q2fG7Hst/eIF/pGoUeSEblNXcbsRNSNeiohbk5rFt/njT4Ay7PeNQf FXG3irN/2rtsWr4S8u1yp1qpS8OamwYsD2ddUb7W+ZDOTpV75qUuGQBFfnuzoUWsdxIw Mrew== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600001; x=1687192001; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TCEmCa6svz6iI89FPgua0fOjkJEldQJYp9iT0pK2PQY=; b=Y3ZNnnSAWZTiMfCkchIgcmQhA07KJ2FDAADo3SsY2FxVO1uKAX2XmTTUanE6r1TwP0 /tJ14YXq5jUhmkSCkaYvRBMOfUzx5FR8iTazm4PH89efZA3R089coMf+BCC+82O/CCZa c51ft84IaXFUAJYbE+hcX7QWFIgZrJEEQZw339y1bgRpsEeaq3+ltaRX+3uVagps49Xb hLcOQbLJjEBqL6wamdbfmYH/ykiAAQFzWfGtDMkwYMJYoEUlOm7ublZXI7Fwz4OoKztm w19AEy9ZdWmqshg1y3iH3WAaSFRkRnUJMSSlEMOC8GQRXqIQp/iFuPaKDrVxYpVeQ/xQ HIiw== X-Gm-Message-State: AC+VfDyDEDFudjhuQUXFbBdhuOIRq/K0WDAz3nrXnUwP6Ltu+R6U+l8P Yrjn8h9R4bE8jA10aokKV40xlCRgczIuoR2sC9k= X-Received: by 2002:a17:903:25ce:b0:1ac:797b:8cf6 with SMTP id jc14-20020a17090325ce00b001ac797b8cf6mr5911958plb.69.1684600001507; Sat, 20 May 2023 09:26:41 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:41 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Juan Quintela Subject: [PATCH 06/27] migration/xbzrle: Shuffle function order Date: Sat, 20 May 2023 09:26:13 -0700 Message-Id: <20230520162634.3991009-7-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::632; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x632.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Place the CONFIG_AVX512BW_OPT block at the top, which will aid function selection in the next patch. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson --- migration/xbzrle.c | 244 ++++++++++++++++++++++----------------------- 1 file changed, 122 insertions(+), 122 deletions(-) diff --git a/migration/xbzrle.c b/migration/xbzrle.c index 258e4959c9..751b5428f7 100644 --- a/migration/xbzrle.c +++ b/migration/xbzrle.c @@ -15,6 +15,128 @@ #include "qemu/host-utils.h" #include "xbzrle.h" +#if defined(CONFIG_AVX512BW_OPT) +#include + +int __attribute__((target("avx512bw"))) +xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, + uint8_t *dst, int dlen) +{ + uint32_t zrun_len = 0, nzrun_len = 0; + int d = 0, i = 0, num = 0; + uint8_t *nzrun_start = NULL; + /* add 1 to include residual part in main loop */ + uint32_t count512s = (slen >> 6) + 1; + /* countResidual is tail of data, i.e., countResidual = slen % 64 */ + uint32_t count_residual = slen & 0b111111; + bool never_same = true; + uint64_t mask_residual = 1; + mask_residual <<= count_residual; + mask_residual -= 1; + __m512i r = _mm512_set1_epi32(0); + + while (count512s) { + int bytes_to_check = 64; + uint64_t mask = 0xffffffffffffffff; + if (count512s == 1) { + bytes_to_check = count_residual; + mask = mask_residual; + } + __m512i old_data = _mm512_mask_loadu_epi8(r, + mask, old_buf + i); + __m512i new_data = _mm512_mask_loadu_epi8(r, + mask, new_buf + i); + uint64_t comp = _mm512_cmpeq_epi8_mask(old_data, new_data); + count512s--; + + bool is_same = (comp & 0x1); + while (bytes_to_check) { + if (d + 2 > dlen) { + return -1; + } + if (is_same) { + if (nzrun_len) { + d += uleb128_encode_small(dst + d, nzrun_len); + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + nzrun_len = 0; + } + /* 64 data at a time for speed */ + if (count512s && (comp == 0xffffffffffffffff)) { + i += 64; + zrun_len += 64; + break; + } + never_same = false; + num = ctz64(~comp); + num = (num < bytes_to_check) ? num : bytes_to_check; + zrun_len += num; + bytes_to_check -= num; + comp >>= num; + i += num; + if (bytes_to_check) { + /* still has different data after same data */ + d += uleb128_encode_small(dst + d, zrun_len); + zrun_len = 0; + } else { + break; + } + } + if (never_same || zrun_len) { + /* + * never_same only acts if + * data begins with diff in first count512s + */ + d += uleb128_encode_small(dst + d, zrun_len); + zrun_len = 0; + never_same = false; + } + /* has diff, 64 data at a time for speed */ + if ((bytes_to_check == 64) && (comp == 0x0)) { + i += 64; + nzrun_len += 64; + break; + } + num = ctz64(comp); + num = (num < bytes_to_check) ? num : bytes_to_check; + nzrun_len += num; + bytes_to_check -= num; + comp >>= num; + i += num; + if (bytes_to_check) { + /* mask like 111000 */ + d += uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + nzrun_len = 0; + is_same = true; + } + } + } + + if (nzrun_len != 0) { + d += uleb128_encode_small(dst + d, nzrun_len); + /* overflow */ + if (d + nzrun_len > dlen) { + return -1; + } + nzrun_start = new_buf + i - nzrun_len; + memcpy(dst + d, nzrun_start, nzrun_len); + d += nzrun_len; + } + return d; +} +#endif + /* page = zrun nzrun | zrun nzrun page @@ -175,125 +297,3 @@ int xbzrle_decode_buffer(uint8_t *src, int slen, uint8_t *dst, int dlen) return d; } - -#if defined(CONFIG_AVX512BW_OPT) -#include - -int __attribute__((target("avx512bw"))) -xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen) -{ - uint32_t zrun_len = 0, nzrun_len = 0; - int d = 0, i = 0, num = 0; - uint8_t *nzrun_start = NULL; - /* add 1 to include residual part in main loop */ - uint32_t count512s = (slen >> 6) + 1; - /* countResidual is tail of data, i.e., countResidual = slen % 64 */ - uint32_t count_residual = slen & 0b111111; - bool never_same = true; - uint64_t mask_residual = 1; - mask_residual <<= count_residual; - mask_residual -= 1; - __m512i r = _mm512_set1_epi32(0); - - while (count512s) { - int bytes_to_check = 64; - uint64_t mask = 0xffffffffffffffff; - if (count512s == 1) { - bytes_to_check = count_residual; - mask = mask_residual; - } - __m512i old_data = _mm512_mask_loadu_epi8(r, - mask, old_buf + i); - __m512i new_data = _mm512_mask_loadu_epi8(r, - mask, new_buf + i); - uint64_t comp = _mm512_cmpeq_epi8_mask(old_data, new_data); - count512s--; - - bool is_same = (comp & 0x1); - while (bytes_to_check) { - if (d + 2 > dlen) { - return -1; - } - if (is_same) { - if (nzrun_len) { - d += uleb128_encode_small(dst + d, nzrun_len); - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - nzrun_len = 0; - } - /* 64 data at a time for speed */ - if (count512s && (comp == 0xffffffffffffffff)) { - i += 64; - zrun_len += 64; - break; - } - never_same = false; - num = ctz64(~comp); - num = (num < bytes_to_check) ? num : bytes_to_check; - zrun_len += num; - bytes_to_check -= num; - comp >>= num; - i += num; - if (bytes_to_check) { - /* still has different data after same data */ - d += uleb128_encode_small(dst + d, zrun_len); - zrun_len = 0; - } else { - break; - } - } - if (never_same || zrun_len) { - /* - * never_same only acts if - * data begins with diff in first count512s - */ - d += uleb128_encode_small(dst + d, zrun_len); - zrun_len = 0; - never_same = false; - } - /* has diff, 64 data at a time for speed */ - if ((bytes_to_check == 64) && (comp == 0x0)) { - i += 64; - nzrun_len += 64; - break; - } - num = ctz64(comp); - num = (num < bytes_to_check) ? num : bytes_to_check; - nzrun_len += num; - bytes_to_check -= num; - comp >>= num; - i += num; - if (bytes_to_check) { - /* mask like 111000 */ - d += uleb128_encode_small(dst + d, nzrun_len); - /* overflow */ - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - nzrun_len = 0; - is_same = true; - } - } - } - - if (nzrun_len != 0) { - d += uleb128_encode_small(dst + d, nzrun_len); - /* overflow */ - if (d + nzrun_len > dlen) { - return -1; - } - nzrun_start = new_buf + i - nzrun_len; - memcpy(dst + d, nzrun_start, nzrun_len); - d += nzrun_len; - } - return d; -} -#endif From patchwork Sat May 20 16:26:14 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684297 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447640wrt; Sat, 20 May 2023 09:27:57 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4LBIsP0W5BXUU1CtUWqjFSgY2YeQwM64m1BDYVDDLxlxKVI50qeRLaZH6xzm/EY051QSiB X-Received: by 2002:ac8:7f41:0:b0:3f2:1708:1372 with SMTP id g1-20020ac87f41000000b003f217081372mr9742134qtk.17.1684600077294; Sat, 20 May 2023 09:27:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600077; cv=none; d=google.com; s=arc-20160816; b=xk2ujeRcHVTAehnMwUVob+wxKYyO/zJos7Ye4ra88+Tt5yzJ7i4nMtfMc/pvBuT/Ce 47rlF1wK6VPRQpQdaOPhj06kB12CdpiufmuRsyLhCuXBgi/v7moJqCl7tvmqTGgwdTBE z6Hw2QjD2Xmeis+zOd9+28PBBnfXhiOmT98+FZvohHCNzdUE9vz/sTADooYdVmiwDD4U nSnvNXdOTm0LpQJLuexjIEb8Kxo8zfav3N+IWoc/X8O7Iz48rKsE+xIz8JujNzWDTpd6 2DFbP/wj3eIhIi4An7mO5BKKFWGFbiZQqti3nqtBJfi5wm/UhmDqbli6rrHCDRJ9CH3V xkRA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=UGkOfurHvoQAbBMjZmdrjImgcN2aCINIU2JfgGlGJi3FT6aF3DrtH73GAa/CNPMca1 R69oFa42s4bdZROutWglrYhfjQhiX7UqGTIc7LSAZQSKOXQU1MNRPDcpQpLIa6jO/zSq W3DDpp/sNejfnBZ8r89rvREeyb1WxF54ptekBXaVDWBG6+UZ8O1st3TqHVks+u5KpJbo EK8RCpfaa5fvzRAVlASKj0A0dWVOfDY7EBv1F7IWP1sOuuvqnWD1RvB4z8eNLLRRlDE0 D9yyMQBWnNFqo6PuzrWH8HOnQjHB2+OifxzxplZTAG6pHxVKfufFJ6nonG82zZq6Hqwb uWdg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="CVGU/baz"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f8-20020ae9ea08000000b0074e4a5d1edfsi1013120qkg.706.2023.05.20.09.27.57 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:27:57 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="CVGU/baz"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PPs-0003bm-Jn; Sat, 20 May 2023 12:26:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PPr-0003b9-84 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:47 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPo-000362-5E for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:47 -0400 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1ae875bf125so10481895ad.1 for ; Sat, 20 May 2023 09:26:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600002; x=1687192002; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=CVGU/baz7BSKbPjXBeOzd+AB7MwuhTN/7B3c9xbIc7VTxAYC2U3HHVbGfHVu8ch7/F gtSW33Z65RaQLG3ov8kd6YYomc/w21oyU8d8vd7YxvRi8do1u3u2xMeczTd53a2VDmUO pib4O2bIq1aaTtXKq59Ok0Wjec/Kezj5jRSSM9VNifC6sKMqIwUEzGlbsWK8lnjo6XtJ y3xKGdTTWHa9ljcJHQALLh+0o1mOVxwnsmQ3L/fYyIvL8wmsujXw++wTO06V9juUykor f87M7F1laQ2aG5uQKq99OPkKJcFMjcpqWBeTVSePytyyluoU6wCXthoQ7yJfOOXHzbi5 uIPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600002; x=1687192002; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ihz9eMnKafBCXMKR5tA0Mrs4r+NsNN8WEkICVzjP6LI=; b=AaPDR2YF40L0WOpuZysha7ihWhauegoiAY+fJN1kRDr8cYKo060hynamSiyuPBMzUp 5+YcNZ1CXCVpG89RN5dkSk70rJTIB40EEol8cQzMnIVFtjtGm6GOjqFZ0JIpF4ycQ+ey jIGOM7uGYCYinfj8WuQA/AHAe/CG8PCAt+5vn8ntcy829EdN9UFqf2WHf5FpeIX/tz2i 0SxGjzyUUHm8jtUB/EQWJYSyCNK669DUM3rXk9pIQdt4kHvneArCGvaTPswFsLmCd2Rv wEFiWXTiTGZLYYiZytmSq33LqgDUyMnggu6zYaBgP7B3ZJromdsrH0q42jxm+j9XfEE/ 9nSQ== X-Gm-Message-State: AC+VfDwx5FbgLyAz8dMUMbnqnMMTnx+4WvYoEuqdlyw2eapCxn7IXQSG vE53/782N2KGHaZa+K1SUDHKvFG0urJFp93euPA= X-Received: by 2002:a17:902:76cb:b0:19f:3797:d8de with SMTP id j11-20020a17090276cb00b0019f3797d8demr5694479plt.9.1684600002505; Sat, 20 May 2023 09:26:42 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:42 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Juan Quintela Subject: [PATCH 07/27] migration/xbzrle: Use i386 host/cpuinfo.h Date: Sat, 20 May 2023 09:26:14 -0700 Message-Id: <20230520162634.3991009-8-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Perform the function selection once, and only if CONFIG_AVX512_OPT is enabled. Centralize the selection to xbzrle.c, instead of spreading the init across 3 files. Remove xbzrle-bench.c. The benefit of being able to benchmark the different implementations is less important than not peeking into the internals of the implementation. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson --- migration/xbzrle.h | 5 +- migration/ram.c | 34 +-- migration/xbzrle.c | 26 +- tests/bench/xbzrle-bench.c | 469 ------------------------------------- tests/unit/test-xbzrle.c | 49 +--- tests/bench/meson.build | 6 - 6 files changed, 39 insertions(+), 550 deletions(-) delete mode 100644 tests/bench/xbzrle-bench.c diff --git a/migration/xbzrle.h b/migration/xbzrle.h index 6feb49160a..39e651b9ec 100644 --- a/migration/xbzrle.h +++ b/migration/xbzrle.h @@ -18,8 +18,5 @@ int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, uint8_t *dst, int dlen); int xbzrle_decode_buffer(uint8_t *src, int slen, uint8_t *dst, int dlen); -#if defined(CONFIG_AVX512BW_OPT) -int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen); -#endif + #endif diff --git a/migration/ram.c b/migration/ram.c index 9fb076fa58..88a6c82e63 100644 --- a/migration/ram.c +++ b/migration/ram.c @@ -90,34 +90,6 @@ #define RAM_SAVE_FLAG_MULTIFD_FLUSH 0x200 /* We can't use any flag that is bigger than 0x200 */ -int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, - uint8_t *, int) = xbzrle_encode_buffer; -#if defined(CONFIG_AVX512BW_OPT) -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - xbzrle_encode_buffer_func = xbzrle_encode_buffer_avx512; - } - } - } -} -#endif - XBZRLECacheStats xbzrle_counters; /* used by the search for pages to send */ @@ -660,9 +632,9 @@ static int save_xbzrle_page(RAMState *rs, PageSearchStatus *pss, memcpy(XBZRLE.current_buf, *current_data, TARGET_PAGE_SIZE); /* XBZRLE encoding (if there is no overflow) */ - encoded_len = xbzrle_encode_buffer_func(prev_cached_page, XBZRLE.current_buf, - TARGET_PAGE_SIZE, XBZRLE.encoded_buf, - TARGET_PAGE_SIZE); + encoded_len = xbzrle_encode_buffer(prev_cached_page, XBZRLE.current_buf, + TARGET_PAGE_SIZE, XBZRLE.encoded_buf, + TARGET_PAGE_SIZE); /* * Update the cache contents, so that it corresponds to the data diff --git a/migration/xbzrle.c b/migration/xbzrle.c index 751b5428f7..3eddcf249b 100644 --- a/migration/xbzrle.c +++ b/migration/xbzrle.c @@ -17,8 +17,9 @@ #if defined(CONFIG_AVX512BW_OPT) #include +#include "host/cpuinfo.h" -int __attribute__((target("avx512bw"))) +static int __attribute__((target("avx512bw"))) xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, uint8_t *dst, int dlen) { @@ -135,6 +136,29 @@ xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, } return d; } + +static int xbzrle_encode_buffer_int(uint8_t *old_buf, uint8_t *new_buf, + int slen, uint8_t *dst, int dlen); + +static int (*accel_func)(uint8_t *, uint8_t *, int, uint8_t *, int); + +static void __attribute__((constructor)) init_accel(void) +{ + unsigned info = cpuinfo_init(); + if (info & CPUINFO_AVX512BW) { + accel_func = xbzrle_encode_buffer_avx512; + } else { + accel_func = xbzrle_encode_buffer_int; + } +} + +int xbzrle_encode_buffer(uint8_t *old_buf, uint8_t *new_buf, int slen, + uint8_t *dst, int dlen) +{ + return accel_func(old_buf, new_buf, slen, dst, dlen); +} + +#define xbzrle_encode_buffer xbzrle_encode_buffer_int #endif /* diff --git a/tests/bench/xbzrle-bench.c b/tests/bench/xbzrle-bench.c deleted file mode 100644 index 8848a3a32d..0000000000 --- a/tests/bench/xbzrle-bench.c +++ /dev/null @@ -1,469 +0,0 @@ -/* - * Xor Based Zero Run Length Encoding unit tests. - * - * Copyright 2013 Red Hat, Inc. and/or its affiliates - * - * Authors: - * Orit Wasserman - * - * This work is licensed under the terms of the GNU GPL, version 2 or later. - * See the COPYING file in the top-level directory. - * - */ -#include "qemu/osdep.h" -#include "qemu/cutils.h" -#include "../migration/xbzrle.h" - -#if defined(CONFIG_AVX512BW_OPT) -#define XBZRLE_PAGE_SIZE 4096 -static bool is_cpu_support_avx512bw; -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - is_cpu_support_avx512bw = false; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - is_cpu_support_avx512bw = true; - } - } - } - return ; -} - -struct ResTime { - float t_raw; - float t_512; -}; - - -/* Function prototypes -int xbzrle_encode_buffer_avx512(uint8_t *old_buf, uint8_t *new_buf, int slen, - uint8_t *dst, int dlen); -*/ -static void encode_decode_zero(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0; - int dlen = 0, dlen512 = 0; - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - buffer[1000 + i] = i; - buffer512[1000 + i] = i; - } - - buffer[1000 + diff_len + 3] = 103; - buffer[1000 + diff_len + 5] = 105; - - buffer512[1000 + diff_len + 3] = 103; - buffer512[1000 + diff_len + 5] = 105; - - /* encode zero page */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(buffer512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(buffer512); - g_free(compressed512); - -} - -static void test_encode_decode_zero_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_zero(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Zero test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_unchanged(struct ResTime *res) -{ - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0; - int dlen = 0, dlen512 = 0; - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - test[1000 + i] = i + 4; - test512[1000 + i] = i + 4; - } - - test[1000 + diff_len + 3] = 107; - test[1000 + diff_len + 5] = 109; - - test512[1000 + diff_len + 3] = 107; - test512[1000 + diff_len + 5] = 109; - - /* test unchanged buffer */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(test); - g_free(compressed); - g_free(test512); - g_free(compressed512); - -} - -static void test_encode_decode_unchanged_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_unchanged(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Unchanged test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_1_byte(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - int dlen = 0, rc = 0, dlen512 = 0, rc512 = 0; - uint8_t buf[2]; - uint8_t buf512[2]; - - test[XBZRLE_PAGE_SIZE - 1] = 1; - test512[XBZRLE_PAGE_SIZE - 1] = 1; - - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(dlen == (uleb128_encode_small(&buf[0], 4095) + 2)); - - rc = xbzrle_decode_buffer(compressed, dlen, buffer, XBZRLE_PAGE_SIZE); - g_assert(rc == XBZRLE_PAGE_SIZE); - g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(dlen512 == (uleb128_encode_small(&buf512[0], 4095) + 2)); - - rc512 = xbzrle_decode_buffer(compressed512, dlen512, buffer512, - XBZRLE_PAGE_SIZE); - g_assert(rc512 == XBZRLE_PAGE_SIZE); - g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_1_byte_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_1_byte(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("1 byte test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_overflow(struct ResTime *res) -{ - uint8_t *compressed = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - - for (i = 0; i < XBZRLE_PAGE_SIZE / 2 - 1; i++) { - test[i * 2] = 1; - test512[i * 2] = 1; - } - - /* encode overflow */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - rc = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - g_assert(rc == -1); - - t_start512 = clock(); - rc512 = xbzrle_encode_buffer_avx512(buffer512, test512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - g_assert(rc512 == -1); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_overflow_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_overflow(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Overflow test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_range_avx512(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - int dlen = 0, dlen512 = 0; - - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1006); - - for (i = diff_len; i > 0; i--) { - buffer[1000 + i] = i; - test[1000 + i] = i + 4; - buffer512[1000 + i] = i; - test512[1000 + i] = i + 4; - } - - buffer[1000 + diff_len + 3] = 103; - test[1000 + diff_len + 3] = 107; - - buffer[1000 + diff_len + 5] = 105; - test[1000 + diff_len + 5] = 109; - - buffer512[1000 + diff_len + 3] = 103; - test512[1000 + diff_len + 3] = 107; - - buffer512[1000 + diff_len + 5] = 105; - test512[1000 + diff_len + 5] = 109; - - /* test encode/decode */ - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); - g_assert(rc < XBZRLE_PAGE_SIZE); - g_assert(memcmp(test, buffer, XBZRLE_PAGE_SIZE) == 0); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - rc512 = xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE_PAGE_SIZE); - g_assert(rc512 < XBZRLE_PAGE_SIZE); - g_assert(memcmp(test512, buffer512, XBZRLE_PAGE_SIZE) == 0); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_range_avx512(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Encode decode test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} - -static void encode_decode_random(struct ResTime *res) -{ - uint8_t *buffer = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *buffer512 = g_malloc0(XBZRLE_PAGE_SIZE); - uint8_t *compressed512 = g_malloc(XBZRLE_PAGE_SIZE); - uint8_t *test512 = g_malloc0(XBZRLE_PAGE_SIZE); - int i = 0, rc = 0, rc512 = 0; - int dlen = 0, dlen512 = 0; - - int diff_len = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); - /* store the index of diff */ - int dirty_index[diff_len]; - for (int j = 0; j < diff_len; j++) { - dirty_index[j] = g_test_rand_int_range(0, XBZRLE_PAGE_SIZE - 1); - } - for (i = diff_len - 1; i >= 0; i--) { - buffer[dirty_index[i]] = i; - test[dirty_index[i]] = i + 4; - buffer512[dirty_index[i]] = i; - test512[dirty_index[i]] = i + 4; - } - - time_t t_start, t_end, t_start512, t_end512; - t_start = clock(); - dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); - t_end = clock(); - float time_val = difftime(t_end, t_start); - rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); - g_assert(rc < XBZRLE_PAGE_SIZE); - - t_start512 = clock(); - dlen512 = xbzrle_encode_buffer_avx512(test512, buffer512, XBZRLE_PAGE_SIZE, - compressed512, XBZRLE_PAGE_SIZE); - t_end512 = clock(); - float time_val512 = difftime(t_end512, t_start512); - rc512 = xbzrle_decode_buffer(compressed512, dlen512, test512, XBZRLE_PAGE_SIZE); - g_assert(rc512 < XBZRLE_PAGE_SIZE); - - res->t_raw = time_val; - res->t_512 = time_val512; - - g_free(buffer); - g_free(compressed); - g_free(test); - g_free(buffer512); - g_free(compressed512); - g_free(test512); - -} - -static void test_encode_decode_random_avx512(void) -{ - int i; - float time_raw = 0.0, time_512 = 0.0; - struct ResTime res; - for (i = 0; i < 10000; i++) { - encode_decode_random(&res); - time_raw += res.t_raw; - time_512 += res.t_512; - } - printf("Random test:\n"); - printf("Raw xbzrle_encode time is %f ms\n", time_raw); - printf("512 xbzrle_encode time is %f ms\n", time_512); -} -#endif - -int main(int argc, char **argv) -{ - g_test_init(&argc, &argv, NULL); - g_test_rand_int(); - #if defined(CONFIG_AVX512BW_OPT) - if (likely(is_cpu_support_avx512bw)) { - g_test_add_func("/xbzrle/encode_decode_zero", test_encode_decode_zero_avx512); - g_test_add_func("/xbzrle/encode_decode_unchanged", - test_encode_decode_unchanged_avx512); - g_test_add_func("/xbzrle/encode_decode_1_byte", test_encode_decode_1_byte_avx512); - g_test_add_func("/xbzrle/encode_decode_overflow", - test_encode_decode_overflow_avx512); - g_test_add_func("/xbzrle/encode_decode", test_encode_decode_avx512); - g_test_add_func("/xbzrle/encode_decode_random", test_encode_decode_random_avx512); - } - #endif - return g_test_run(); -} diff --git a/tests/unit/test-xbzrle.c b/tests/unit/test-xbzrle.c index 547046d093..b6996de69a 100644 --- a/tests/unit/test-xbzrle.c +++ b/tests/unit/test-xbzrle.c @@ -16,35 +16,6 @@ #define XBZRLE_PAGE_SIZE 4096 -int (*xbzrle_encode_buffer_func)(uint8_t *, uint8_t *, int, - uint8_t *, int) = xbzrle_encode_buffer; -#if defined(CONFIG_AVX512BW_OPT) -#include "qemu/cpuid.h" -static void __attribute__((constructor)) init_cpu_flag(void) -{ - unsigned max = __get_cpuid_max(0, NULL); - int a, b, c, d; - if (max >= 1) { - __cpuid(1, a, b, c, d); - /* We must check that AVX is not just available, but usable. */ - if ((c & bit_OSXSAVE) && (c & bit_AVX) && max >= 7) { - int bv; - __asm("xgetbv" : "=a"(bv), "=d"(d) : "c"(0)); - __cpuid_count(7, 0, a, b, c, d); - /* 0xe6: - * XCR0[7:5] = 111b (OPMASK state, upper 256-bit of ZMM0-ZMM15 - * and ZMM16-ZMM31 state are enabled by OS) - * XCR0[2:1] = 11b (XMM state and YMM state are enabled by OS) - */ - if ((bv & 0xe6) == 0xe6 && (b & bit_AVX512BW)) { - xbzrle_encode_buffer_func = xbzrle_encode_buffer_avx512; - } - } - } - return ; -} -#endif - static void test_uleb(void) { uint32_t i, val; @@ -83,8 +54,8 @@ static void test_encode_decode_zero(void) buffer[1000 + diff_len + 5] = 105; /* encode zero page */ - dlen = xbzrle_encode_buffer_func(buffer, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(buffer, buffer, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == 0); g_free(buffer); @@ -107,8 +78,8 @@ static void test_encode_decode_unchanged(void) test[1000 + diff_len + 5] = 109; /* test unchanged buffer */ - dlen = xbzrle_encode_buffer_func(test, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(test, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == 0); g_free(test); @@ -125,8 +96,8 @@ static void test_encode_decode_1_byte(void) test[XBZRLE_PAGE_SIZE - 1] = 1; - dlen = xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(dlen == (uleb128_encode_small(&buf[0], 4095) + 2)); rc = xbzrle_decode_buffer(compressed, dlen, buffer, XBZRLE_PAGE_SIZE); @@ -150,8 +121,8 @@ static void test_encode_decode_overflow(void) } /* encode overflow */ - rc = xbzrle_encode_buffer_func(buffer, test, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + rc = xbzrle_encode_buffer(buffer, test, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); g_assert(rc == -1); g_free(buffer); @@ -181,8 +152,8 @@ static void encode_decode_range(void) test[1000 + diff_len + 5] = 109; /* test encode/decode */ - dlen = xbzrle_encode_buffer_func(test, buffer, XBZRLE_PAGE_SIZE, compressed, - XBZRLE_PAGE_SIZE); + dlen = xbzrle_encode_buffer(test, buffer, XBZRLE_PAGE_SIZE, + compressed, XBZRLE_PAGE_SIZE); rc = xbzrle_decode_buffer(compressed, dlen, test, XBZRLE_PAGE_SIZE); g_assert(rc < XBZRLE_PAGE_SIZE); diff --git a/tests/bench/meson.build b/tests/bench/meson.build index 4e6b469066..3c799dbd98 100644 --- a/tests/bench/meson.build +++ b/tests/bench/meson.build @@ -3,12 +3,6 @@ qht_bench = executable('qht-bench', sources: 'qht-bench.c', dependencies: [qemuutil]) -if have_system -xbzrle_bench = executable('xbzrle-bench', - sources: 'xbzrle-bench.c', - dependencies: [qemuutil,migration]) -endif - qtree_bench = executable('qtree-bench', sources: 'qtree-bench.c', dependencies: [qemuutil]) From patchwork Sat May 20 16:26:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684298 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447715wrt; Sat, 20 May 2023 09:28:10 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6zGUvw54sxrIhrUph6xeanYdFis8BTCVJBHozEQzm2oDPl71V8OMxY/vMBuhtFX5SFCT5n X-Received: by 2002:a05:622a:2cd:b0:3f5:4902:2175 with SMTP id a13-20020a05622a02cd00b003f549022175mr10468581qtx.35.1684600090183; Sat, 20 May 2023 09:28:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600090; cv=none; d=google.com; s=arc-20160816; b=br2W3MhIhiTbjwRWECDsYzG7lH4sGKPCd/4rfbbifj26QU+mHXkEVhLX6ZkY95w3LN 0eM75L4/UvGusiJyVUPZoBFTWpVI5yBPeOY4+9hQnwrRrPaQ0tpw7ty3JXi0T5/U1drs +tNvatmQgiBPrUTDmA2iwLSqdBGOF+bY95Jp4aTBL7VNvqF/UgKb1RBDFvzZZXDR4v7L MX8i5ovJx2Dn9J0W4kvpObUmrtYGcSf15nCY5/+NzcInVS2pUb991bYSB4wd1rrhN/k8 o66EwaVcStWrC7u7ZdWl8VsflCLdUDZNc0G+z98IuT0lYJOqxW76OgFp8Gq03pAPG1s+ h12w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=DPmIawU8I5Q/EGoP++BdMIUFPhxrsmE1o4I152DpCJoxH09OK/QzA6SgOH47AA81+O WTzYITaIOCl+TL/KoCX5blXuDiy584yGWt/saPIhGd4GDikRKv02g8eJ/zRp8PVj/IZt QMkUT9GYzh7zB6TOMnFlKWIBKWgXP8qO3jXZglWob9uP8LSJbs4vJPL7xYUwwjgg9Oda C2cr2e3iQULdaGdgJuoxppWd6ApMUsh8+vQr4FZcZGa/KAybM2tMY9QAN3HKabt4Hm70 oX4gIGECmaz7nnUxYl1XZmBUYkhpurBkbtooHGnFlA7u1u4wUO/17C8GA0WuHahU+kjV xKIg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uqxdT0BK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id u4-20020a05620a120400b0074e1849da39si1043910qkj.48.2023.05.20.09.28.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:28:10 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=uqxdT0BK; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQ8-0003eg-Ht; Sat, 20 May 2023 12:27:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-0003dw-7o for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:59 -0400 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPy-00038c-UA for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:56 -0400 Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-5208be24dcbso3065035a12.1 for ; Sat, 20 May 2023 09:26:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600012; x=1687192012; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=uqxdT0BKeHFvCuoLSxdqKCOjb59ZE4mFfez8SEtlk82WI5VZTZlJzVFA87O4a75Or1 YJ/YCdF8IqcytWVL/yefxXyCG39NCSqdCHZdD0srTslid0vSx9A2MVmWvzO+DsiCpTVl o+DMHgWOp22AbEfUJJytFG317SXiNDdwSw4nP4s+IEqjWrx7+NFTloPiqNzRFPLbP+KI fyM2Vnc0Zef9e/BVFoskpGjo33DyOEYmUjo/nmIguhX91HWIWE1Yv/z3z2Wiu64tZ9QQ Zl+7AeJCOR3XFrOLtLY70KKzZOvBPXoGiyxHd6uCNSLVqcjvlwd2bj35g+UFsy/7MCgQ XLVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600012; x=1687192012; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1XkUZP/KXpryW+1SxKnXlwtBsZlj1WR79DzIRZIAO0I=; b=DzBShK8evrrOcD00rulmyUeNF38U2jZ5NzHDpr6lkaYUAFczP8hwJJHkto5kZYxViw Dc8NZgT8KpDw7OcF6c70e5SEeWksoFW+RUZs8ZshwJSTpY+U+0HW0SXoR5vx0ZfdAzqj ASqB6btvm/4rQ2b2kR8ULPZCmdQl9L+cswUX7CRPWfLXWHT3BveUtR4SQ6r2cL8cUXYw A9lZmgx9O18dpk3FYQMQEE5svar6WeOtm+Fqo5DeaUyt+sX69POt6oqsIHqWOmvlc8+R ivjw5Sl1sd8krQQJvj6yh31Y5lIej7O+o1mOS0M19wdBS+ywXFUZQqTQiCWdHjTBfni3 v8AQ== X-Gm-Message-State: AC+VfDyI5s1YWM7oFg1vVuuEd9lEDuyE6owTTtiLL2l0s1ANxu9XU0e4 Hl3gE2xcLBVPTnHhj9bETlfCYOGloZGBzxpHiII= X-Received: by 2002:a17:902:f807:b0:1ad:e5e6:6c30 with SMTP id ix7-20020a170902f80700b001ade5e66c30mr5851307plb.44.1684600012633; Sat, 20 May 2023 09:26:52 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:52 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Juan Quintela Subject: [PATCH 08/27] migration: Build migration_files once Date: Sat, 20 May 2023 09:26:15 -0700 Message-Id: <20230520162634.3991009-9-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::532; envelope-from=richard.henderson@linaro.org; helo=mail-pg1-x532.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org The items in migration_files are built for libmigration and included info softmmu_ss from there; no need to also include them directly. Reviewed-by: Juan Quintela Signed-off-by: Richard Henderson --- migration/meson.build | 1 - 1 file changed, 1 deletion(-) diff --git a/migration/meson.build b/migration/meson.build index a8e01e70ae..8ba6e420fe 100644 --- a/migration/meson.build +++ b/migration/meson.build @@ -8,7 +8,6 @@ migration_files = files( 'qemu-file.c', 'yank_functions.c', ) -softmmu_ss.add(migration_files) softmmu_ss.add(files( 'block-dirty-bitmap.c', From patchwork Sat May 20 16:26:16 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684306 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448083wrt; Sat, 20 May 2023 09:29:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7ipOnwcgZYFZwifljGLD2rGVZh7I1vJsTnJPrwxMtbwUIMo1kkFmndI8j6W4GELwZOOi3A X-Received: by 2002:a05:6214:5015:b0:61b:7680:1283 with SMTP id jo21-20020a056214501500b0061b76801283mr11562894qvb.16.1684600164634; Sat, 20 May 2023 09:29:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600164; cv=none; d=google.com; s=arc-20160816; b=Zx6K1f11YNATDuYUoRVLlwhanIsf3mg2GFlQoB8ulxcK0FH9fHJnDnVaBGo9xgMVgd CL+1qQKWGWh6Pso8CrC7juSVkBYmCc8v0p/hyFIsH4MLhohWvbSxwz9/FoVRUnOT2TSj SebElJnRcYsEMbWigz6QoT/XWtMp0YW7jPuAL/89eShbnmNbblr8IDb1gQhF4tu3zY+y 1dBLB6o9p+vESlHJtKbiCk7hrGRU9piyWqXs71W/T9/yh6kcxPaG0aiXcCx1/dsxMFZN eVWvTU8q6BhffeUOCu8VENA+YFyHDmPtw45pHgIUfa+Ez7PHrX6W2ov4LPlCyPD+mod3 Uqew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=X30yx0nChpPlQnT8HJnlth+jav9XRP7NjEDJvAoJmzyFCV489m2pA14RrA1TMt3RUU cOcqS5DHSF3ysjPMSEfaiFfCL7vqQxb9kzpqC5+mcUD9vm1Ke3z/+xVBBE+IkmslMNku nzk7NF+IafXzHYZjvGdIX/zwQj0I04FzxJo9c5rTdaariQqNVfclGJbMHCkSisQQ224m SexMu0fKKvpjOqiNgeLymAuF/zHEyOToEMmoEfOmiKk1QQRRaEO7BNuohe7a28aMncE/ KLGlVta+VjrL2e2Xy9zXeSUVwKqfHYjQKCEnX5nhPYbKhaB6X6vWeCwhngcMAj9mmFOy qIkQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="m/UfHBSe"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a7-20020a0562140c2700b0061a07f758b8si1219475qvd.210.2023.05.20.09.29.24 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:24 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="m/UfHBSe"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQG-0003ht-Fu; Sat, 20 May 2023 12:27:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-0003dy-8b for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:59 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPz-00038j-0n for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:57 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-6436dfa15b3so3150831b3a.1 for ; Sat, 20 May 2023 09:26:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600014; x=1687192014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=m/UfHBSe6O3aPYJNbumwyTu4QcddKupZZKm8kqaA2kvl7R58V6KAhfxpTKRZ2V4Qks L4x/Bgby/SvT2fw2ZKK44nvnX5EEoa+4aAhxlwER6JlazE1xGuIPMY18/ve58tj0gDzr EZtpMMEQbsvIYFQ1eG+e9Jm5ByeeyLdqN+LwOFnG/IPLv1OBbETscveCVjiRMUJugo9V WGFkKZNPYp1VQ4EPqlRKhKrqG9OZ3eJ227oauapwjW+m3lBWO//9RFUp6K+pNAbxjBLe ytxJOSFVuA0qfDXZk21KuDuNxkxQc8Qopnd7pAdwihLMMLJv1fnFfifygjDXlWKqYjYl c6Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600014; x=1687192014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LBBBG0WTc1cH3EmTG+zv0uyjIvA9PNjDKH+Kc3n5h9w=; b=G2CfC1RzFMif6oITtPImlflHZVdO6GsOuRcOHxywRmIL0WcSbPsB3nKJdIYNdBe3SW kG7QUZVO4+3qLXFHmdVhzmcIMjX8nV9Rb/U2rjftBGxmt2rm/f4Cddad8dLS/SWpBI+/ eN5PLIdpmvDhHySZv3b1jeAx8A238A4my4hYHzkDKCjQiO4ETwezGakez5pxnE5bzA04 t0H/D4Gl5mDrDzltzoJhyQdNmvfFIazgEpIKYVfBitYcPRzrsYa1frHeMGuZjK8X9aAs k6vjHm8hWLWMj47Vc2hC5RnHQXqQuL13f0kW1NHUonoJOSc9L71i8dv19/KSMiqua/NH rbTA== X-Gm-Message-State: AC+VfDzRuluGXrQJ62xN6RbV7yfFnqu0xozg5BUSR6+oju+KzL/Wb6kc jM9JEViKqOemDgh2WuU/rO96L34t+5jq5RxhNRQ= X-Received: by 2002:a17:903:11c9:b0:1aa:ff41:31a7 with SMTP id q9-20020a17090311c900b001aaff4131a7mr7078964plh.13.1684600013640; Sat, 20 May 2023 09:26:53 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:53 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, Peter Maydell Subject: [PATCH 09/27] util: Add cpuinfo-aarch64.c Date: Sat, 20 May 2023 09:26:16 -0700 Message-Id: <20230520162634.3991009-10-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Move the code from tcg/. The only use of these bits so far is with respect to the atomicity of tcg operations. Reviewed-by: Peter Maydell Signed-off-by: Richard Henderson --- host/include/aarch64/host/cpuinfo.h | 22 ++++++++++ tcg/aarch64/tcg-target.h | 6 ++- util/cpuinfo-aarch64.c | 67 +++++++++++++++++++++++++++++ tcg/aarch64/tcg-target.c.inc | 40 ----------------- util/meson.build | 4 +- 5 files changed, 96 insertions(+), 43 deletions(-) create mode 100644 host/include/aarch64/host/cpuinfo.h create mode 100644 util/cpuinfo-aarch64.c diff --git a/host/include/aarch64/host/cpuinfo.h b/host/include/aarch64/host/cpuinfo.h new file mode 100644 index 0000000000..82227890b4 --- /dev/null +++ b/host/include/aarch64/host/cpuinfo.h @@ -0,0 +1,22 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for AArch64. + */ + +#ifndef HOST_CPUINFO_H +#define HOST_CPUINFO_H + +#define CPUINFO_ALWAYS (1u << 0) /* so cpuinfo is nonzero */ +#define CPUINFO_LSE (1u << 1) +#define CPUINFO_LSE2 (1u << 2) + +/* Initialized with a constructor. */ +extern unsigned cpuinfo; + +/* + * We cannot rely on constructor ordering, so other constructors must + * use the function interface rather than the variable above. + */ +unsigned cpuinfo_init(void); + +#endif /* HOST_CPUINFO_H */ diff --git a/tcg/aarch64/tcg-target.h b/tcg/aarch64/tcg-target.h index 74ee2ed255..d5f7614880 100644 --- a/tcg/aarch64/tcg-target.h +++ b/tcg/aarch64/tcg-target.h @@ -13,6 +13,8 @@ #ifndef AARCH64_TCG_TARGET_H #define AARCH64_TCG_TARGET_H +#include "host/cpuinfo.h" + #define TCG_TARGET_INSN_UNIT_SIZE 4 #define TCG_TARGET_TLB_DISPLACEMENT_BITS 24 #define MAX_CODE_GEN_BUFFER_SIZE ((size_t)-1) @@ -57,8 +59,8 @@ typedef enum { #define TCG_TARGET_CALL_ARG_I128 TCG_CALL_ARG_EVEN #define TCG_TARGET_CALL_RET_I128 TCG_CALL_RET_NORMAL -extern bool have_lse; -extern bool have_lse2; +#define have_lse (cpuinfo & CPUINFO_LSE) +#define have_lse2 (cpuinfo & CPUINFO_LSE2) /* optional instructions */ #define TCG_TARGET_HAS_div_i32 1 diff --git a/util/cpuinfo-aarch64.c b/util/cpuinfo-aarch64.c new file mode 100644 index 0000000000..f99acb7884 --- /dev/null +++ b/util/cpuinfo-aarch64.c @@ -0,0 +1,67 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Host specific cpu indentification for AArch64. + */ + +#include "qemu/osdep.h" +#include "host/cpuinfo.h" + +#ifdef CONFIG_LINUX +# ifdef CONFIG_GETAUXVAL +# include +# else +# include +# include "elf.h" +# endif +#endif +#ifdef CONFIG_DARWIN +# include +#endif + +unsigned cpuinfo; + +#ifdef CONFIG_DARWIN +static bool sysctl_for_bool(const char *name) +{ + int val = 0; + size_t len = sizeof(val); + + if (sysctlbyname(name, &val, &len, NULL, 0) == 0) { + return val != 0; + } + + /* + * We might in the future ask for properties not present in older kernels, + * but we're only asking about static properties, all of which should be + * 'int'. So we shouln't see ENOMEM (val too small), or any of the other + * more exotic errors. + */ + assert(errno == ENOENT); + return false; +} +#endif + +/* Called both as constructor and (possibly) via other constructors. */ +unsigned __attribute__((constructor)) cpuinfo_init(void) +{ + unsigned info = cpuinfo; + + if (info) { + return info; + } + + info = CPUINFO_ALWAYS; + +#ifdef CONFIG_LINUX + unsigned long hwcap = qemu_getauxval(AT_HWCAP); + info |= (hwcap & HWCAP_ATOMICS ? CPUINFO_LSE : 0); + info |= (hwcap & HWCAP_USCAT ? CPUINFO_LSE2 : 0); +#endif +#ifdef CONFIG_DARWIN + info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE") * CPUINFO_LSE; + info |= sysctl_for_bool("hw.optional.arm.FEAT_LSE2") * CPUINFO_LSE2; +#endif + + cpuinfo = info; + return info; +} diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc index bc6b99a1bd..84283665e7 100644 --- a/tcg/aarch64/tcg-target.c.inc +++ b/tcg/aarch64/tcg-target.c.inc @@ -13,12 +13,6 @@ #include "../tcg-ldst.c.inc" #include "../tcg-pool.c.inc" #include "qemu/bitops.h" -#ifdef __linux__ -#include -#endif -#ifdef CONFIG_DARWIN -#include -#endif /* We're going to re-use TCGType in setting of the SF bit, which controls the size of the operation performed. If we know the values match, it @@ -77,9 +71,6 @@ static TCGReg tcg_target_call_oarg_reg(TCGCallReturnKind kind, int slot) return TCG_REG_X0 + slot; } -bool have_lse; -bool have_lse2; - #define TCG_REG_TMP TCG_REG_X30 #define TCG_VEC_TMP TCG_REG_V31 @@ -2878,39 +2869,8 @@ static TCGConstraintSetIndex tcg_target_op_def(TCGOpcode op) } } -#ifdef CONFIG_DARWIN -static bool sysctl_for_bool(const char *name) -{ - int val = 0; - size_t len = sizeof(val); - - if (sysctlbyname(name, &val, &len, NULL, 0) == 0) { - return val != 0; - } - - /* - * We might in the future ask for properties not present in older kernels, - * but we're only asking about static properties, all of which should be - * 'int'. So we shouln't see ENOMEM (val too small), or any of the other - * more exotic errors. - */ - assert(errno == ENOENT); - return false; -} -#endif - static void tcg_target_init(TCGContext *s) { -#ifdef __linux__ - unsigned long hwcap = qemu_getauxval(AT_HWCAP); - have_lse = hwcap & HWCAP_ATOMICS; - have_lse2 = hwcap & HWCAP_USCAT; -#endif -#ifdef CONFIG_DARWIN - have_lse = sysctl_for_bool("hw.optional.arm.FEAT_LSE"); - have_lse2 = sysctl_for_bool("hw.optional.arm.FEAT_LSE2"); -#endif - tcg_target_available_regs[TCG_TYPE_I32] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_I64] = 0xffffffffu; tcg_target_available_regs[TCG_TYPE_V64] = 0xffffffff00000000ull; diff --git a/util/meson.build b/util/meson.build index b3be9fad5d..3a93071d27 100644 --- a/util/meson.build +++ b/util/meson.build @@ -109,6 +109,8 @@ if have_block util_ss.add(when: 'CONFIG_LINUX', if_true: files('vfio-helpers.c')) endif -if cpu in ['x86', 'x86_64'] +if cpu == 'aarch64' + util_ss.add(files('cpuinfo-aarch64.c')) +elif cpu in ['x86', 'x86_64'] util_ss.add(files('cpuinfo-i386.c')) endif From patchwork Sat May 20 16:26:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684302 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447977wrt; Sat, 20 May 2023 09:29:04 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ41utT+yJlzVgEk/kaJBKGBmVd+nqMTcZCoTs8g6ilUA76UzOUACEZ/8RshikXZuCYzYxFa X-Received: by 2002:ac8:5847:0:b0:3f4:f5f4:d5df with SMTP id h7-20020ac85847000000b003f4f5f4d5dfmr10080035qth.59.1684600143907; Sat, 20 May 2023 09:29:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600143; cv=none; d=google.com; s=arc-20160816; b=uHvpGBYM3GXyT3KUNvafK1JxwjuWd/EO7V8s3RrkE+lepLyqR9SnvsTJFbEHVKNLUK 2beNJSceYZzAfSdUXthZiuYGRQw4IypqffkOFuae9Frhkabw+/gozBDDZUmPdmfRbBOl btAPYv6WZ61wISlNQ47z3RdQmnXrNP2X+bJ/jeSJSjD4FTbImqUmWi8hFqFu4dogkYng g9rNlwSyHP52fw1XmHSxzxT2q31wDkcE5bca8+SoobmTjAgp9D+pCrMBto0rtJtu1kri ZfOxHbKsVeL4wSiJkKKdj56ZTWDxFZKrzgZ8RwKVOejyKkAYY83uqQyGi193H2rKIte4 EICg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=eDLGAPMJ+rUVMTpdZgvvo26halub+heLPUT3U1Wrh9A=; b=Nei8+J8tPFYbcvl36Uatu2lzn7OXFBpdb9i+A9DMfJ/MSpMBu6WSCFezdVT3nHPZ8J LF+vJ5Z88P1jTHjniMyAygfK8djrZsvCqVN390Yg4dN0RQ0R6K7UYQa8+HKF16DIha9C jeJCjri8dXAylW6R69jmcSfUDRa1dh8GFm2DDMpBPJAUNjO6LPLXn6jJpPgO+SjvKnZe p5PdrUwzU7/FqiJHj/+oO8Fy602pUzPA0pxIjc1oodxt9pJQ+hBgEJvQTEUv9C0V39SX Tc0PAwvKScn3Lya/djz9oimVvR7LkD3fnmf3Bp5m8959/1isVA1O1bYLyo6MmIALDkIi jgmA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RhvuOrA9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id p15-20020ae9f30f000000b0075954cd91f4si1021011qkg.717.2023.05.20.09.29.03 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=RhvuOrA9; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQ8-0003eo-HW; Sat, 20 May 2023 12:27:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-0003e0-AH for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:59 -0400 Received: from mail-pf1-x42e.google.com ([2607:f8b0:4864:20::42e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PPz-00038q-Ux for qemu-devel@nongnu.org; Sat, 20 May 2023 12:26:58 -0400 Received: by mail-pf1-x42e.google.com with SMTP id d2e1a72fcca58-6436dfa15b3so3150839b3a.1 for ; Sat, 20 May 2023 09:26:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600014; x=1687192014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=eDLGAPMJ+rUVMTpdZgvvo26halub+heLPUT3U1Wrh9A=; b=RhvuOrA9VZC0gJf2P1BbhjiCzc+M4HomovA6Y7JVBqSDAKbPtOf7HXtB3bxlexrZ+l qjU2nskHJC0dvrhgi7C/yJt91qlWFTRfx7jGLdR55ApFFWeTVgQguTGsOEfjN5tTdI7o NYxi2W0KbSZ32MTqlExIMcPdfe9zdGW0qN1Qa2uNmmNGnsS7MOp8+VxxiQ8tNTAosacU DO0xuKBTblOSiq7n7oFK7/C/iayiOxtO0RZzzeDsBNRuXLb1djtqXUND/M+WoKe//YWb w91ywwTHi/4bs/nngm6My3WgnJ4O+7VEnWqvzCITZ2a47bdxYUzfIN5hUejGcnEcHSrU KwQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600014; x=1687192014; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eDLGAPMJ+rUVMTpdZgvvo26halub+heLPUT3U1Wrh9A=; b=Rs9qanW19yddNhtQiFjoXbtvK9a3xzLd9xQv1xjJ9+lC1SiyT5eifblRb1QkwBNsoF Z0rr8LujCeTwz02J/CACfGkc6KkQw8P4ttIdJiDwcgibdRZFESQB0LNRbjcaFmXja8ax dxI6anTnRMp6F05WXUnuhi+1sZu2og3J8ST9FC0EiloN/UIkWsg4rJEb5uYsAyN7i4vU QomNOF/S/wwcLWzeeN3poCmQ2rkvaEmRUxZ/eoqyJ9g28nprf+HEMty4iyH8IneH7QKj Svgk1Hor+RK+ks4b1vK3+b8cT/ob3pF+wxrqixCRaDNsINw0PFpsBEB/8dMwizxPchch 9mng== X-Gm-Message-State: AC+VfDyetnIa7FF0Iexq1ywmOP7VjB7T/eMqrNBD1KCOKIDHX/4UhyWn Aixb8/xgdfiUi1MyvUABjHQoU/1aE6OU6Vf2hN0= X-Received: by 2002:a17:902:bf06:b0:1ab:1bdd:b307 with SMTP id bi6-20020a170902bf0600b001ab1bddb307mr5232400plb.51.1684600014560; Sat, 20 May 2023 09:26:54 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:54 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 10/27] include/host: Split out atomic128-cas.h Date: Sat, 20 May 2023 09:26:17 -0700 Message-Id: <20230520162634.3991009-11-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42e; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42e.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Separates the aarch64-specific portion into its own file. Signed-off-by: Richard Henderson --- host/include/aarch64/host/atomic128-cas.h | 43 ++++++++++++++++++ host/include/generic/host/atomic128-cas.h | 43 ++++++++++++++++++ include/qemu/atomic128.h | 55 +---------------------- 3 files changed, 87 insertions(+), 54 deletions(-) create mode 100644 host/include/aarch64/host/atomic128-cas.h create mode 100644 host/include/generic/host/atomic128-cas.h diff --git a/host/include/aarch64/host/atomic128-cas.h b/host/include/aarch64/host/atomic128-cas.h new file mode 100644 index 0000000000..1247995419 --- /dev/null +++ b/host/include/aarch64/host/atomic128-cas.h @@ -0,0 +1,43 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Compare-and-swap for 128-bit atomic operations, generic version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_CAS_H +#define AARCH64_ATOMIC128_CAS_H + +/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) +#include "host/include/generic/host/atomic128-cas.h" +#else +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + uint64_t cmpl = int128_getlo(cmp), cmph = int128_gethi(cmp); + uint64_t newl = int128_getlo(new), newh = int128_gethi(new); + uint64_t oldl, oldh; + uint32_t tmp; + + asm("0: ldaxp %[oldl], %[oldh], %[mem]\n\t" + "cmp %[oldl], %[cmpl]\n\t" + "ccmp %[oldh], %[cmph], #0, eq\n\t" + "b.ne 1f\n\t" + "stlxp %w[tmp], %[newl], %[newh], %[mem]\n\t" + "cbnz %w[tmp], 0b\n" + "1:" + : [mem] "+m"(*ptr), [tmp] "=&r"(tmp), + [oldl] "=&r"(oldl), [oldh] "=&r"(oldh) + : [cmpl] "r"(cmpl), [cmph] "r"(cmph), + [newl] "r"(newl), [newh] "r"(newh) + : "memory", "cc"); + + return int128_make128(oldl, oldh); +} +# define HAVE_CMPXCHG128 1 +#endif + +#endif /* AARCH64_ATOMIC128_CAS_H */ diff --git a/host/include/generic/host/atomic128-cas.h b/host/include/generic/host/atomic128-cas.h new file mode 100644 index 0000000000..513622fe34 --- /dev/null +++ b/host/include/generic/host/atomic128-cas.h @@ -0,0 +1,43 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Compare-and-swap for 128-bit atomic operations, generic version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef HOST_ATOMIC128_CAS_H +#define HOST_ATOMIC128_CAS_H + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); + return r.s; +} +# define HAVE_CMPXCHG128 1 +#elif defined(CONFIG_CMPXCHG128) +static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +{ + Int128Alias r, c, n; + + c.s = cmp; + n.s = new; + r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); + return r.s; +} +# define HAVE_CMPXCHG128 1 +#else +/* Fallback definition that must be optimized away, or error. */ +Int128 QEMU_ERROR("unsupported atomic") + atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new); +# define HAVE_CMPXCHG128 0 +#endif + +#endif /* HOST_ATOMIC128_CAS_H */ diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index d0ba0b9c65..10a2322c44 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -41,60 +41,7 @@ * Therefore, special case each platform. */ -#if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - Int128Alias r, c, n; - - c.s = cmp; - n.s = new; - r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); - return r.s; -} -# define HAVE_CMPXCHG128 1 -#elif defined(CONFIG_CMPXCHG128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - Int128Alias r, c, n; - - c.s = cmp; - n.s = new; - r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); - return r.s; -} -# define HAVE_CMPXCHG128 1 -#elif defined(__aarch64__) -/* Through gcc 8, aarch64 has no support for 128-bit at all. */ -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) -{ - uint64_t cmpl = int128_getlo(cmp), cmph = int128_gethi(cmp); - uint64_t newl = int128_getlo(new), newh = int128_gethi(new); - uint64_t oldl, oldh; - uint32_t tmp; - - asm("0: ldaxp %[oldl], %[oldh], %[mem]\n\t" - "cmp %[oldl], %[cmpl]\n\t" - "ccmp %[oldh], %[cmph], #0, eq\n\t" - "b.ne 1f\n\t" - "stlxp %w[tmp], %[newl], %[newh], %[mem]\n\t" - "cbnz %w[tmp], 0b\n" - "1:" - : [mem] "+m"(*ptr), [tmp] "=&r"(tmp), - [oldl] "=&r"(oldl), [oldh] "=&r"(oldh) - : [cmpl] "r"(cmpl), [cmph] "r"(cmph), - [newl] "r"(newl), [newh] "r"(newh) - : "memory", "cc"); - - return int128_make128(oldl, oldh); -} -# define HAVE_CMPXCHG128 1 -#else -/* Fallback definition that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") - atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new); -# define HAVE_CMPXCHG128 0 -#endif /* Some definition for HAVE_CMPXCHG128 */ - +#include "host/atomic128-cas.h" #if defined(CONFIG_ATOMIC128) static inline Int128 atomic16_read(Int128 *ptr) From patchwork Sat May 20 16:26:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684322 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp449029wrt; Sat, 20 May 2023 09:31:46 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5mYP6gsmtflp5vi/4PE3a1d93LV0S/B9VXa18uXjP8dRd2L64DbD9g4+RUzzJfVd568oCe X-Received: by 2002:a05:6214:1309:b0:623:9cab:3e18 with SMTP id pn9-20020a056214130900b006239cab3e18mr11455636qvb.26.1684600306757; Sat, 20 May 2023 09:31:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600306; cv=none; d=google.com; s=arc-20160816; b=yIRM7f4UZcll1DK/a+Goa4D+wW5qnrzlr9OpgwRHcE9b4HBdqoO0Gm9l2ynA4EI9jE pIWxAjaZtbRgxcFYtDl1T2vfOoiwzUvdZ4bEKB7zBSSjorxR+g80DM/3YC3SZY99d9o4 Yo+V8hCSI5MSgvq558xq/wPh42py7eRSOmDwU6kdNDlANeUsKDL4kOzeoghlzzT3jVZ5 kSMuL5Zp/8e/NOq7jnhyt2PQ/4kivnCpph0xqCDZ9bD5smlM+nRdYV9Zg/wOpFocxROm cRmN3MDmSt/1NBXDhXqk+LtiOW3dGQNnstyzY6oUGQgvRVIc6fKCWfKn9PnDZg4Ycbli mQNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=SUpVnyemWEPOpO605ffDicbCJQoqq3GtxGHxc5DnjBc=; b=P6LergZ6iqbkOtj87Ar/0U6lNkAuwUZAYnf8rL1ryxIVFi3Sohi80GsB8h6mgj6FsQ 12Zbt5wZ27QIVSEPTy/XOh7yuIlP4KTTTdmHIkrc899iWgiBl6nHvtDPRgmJt9rz0CCs /ZJKe6G4RATehT/DdXD3qHg5RJg3okrjsUhs/L2/ANhAzdkl0ZPUzG/++PgZcNhyDcSK sbJ/zeAQ4oRhPn/RGLrL7BmPnkFVFn/wCus8ZylwI9ezcj8CNagGqDbegu/VpZiHYRux ZJYzU6+0TJ0EyEAQvUTs0Hus5QhpG5tzhcF36gOqj41Lkln9X2E5vojMkFhvjavxA1zG 9wuw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tsLk+47i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id o28-20020a05620a15dc00b0075836fb7ab5si1058392qkm.224.2023.05.20.09.31.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tsLk+47i; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQn-0004P2-19; Sat, 20 May 2023 12:27:45 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ9-0003fM-4g for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:06 -0400 Received: from mail-pf1-x430.google.com ([2607:f8b0:4864:20::430]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-000393-1q for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:03 -0400 Received: by mail-pf1-x430.google.com with SMTP id d2e1a72fcca58-64d2981e3abso1974689b3a.1 for ; Sat, 20 May 2023 09:26:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600015; x=1687192015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SUpVnyemWEPOpO605ffDicbCJQoqq3GtxGHxc5DnjBc=; b=tsLk+47irudQUM9AhQj4lU+z/0Vqob39hWiHRZQyxpzh8u1vnhjsqccVnqTvXmQtRq Fk/cWvoflknTP0XHXU3OXgL/Eh3v47Ku4zHGmUKsfYdnWBzf/9shiPxbvcKpZnDdNIIV cejUNPq7T1tJ/PXUKjc6FxC4maxN6uflFA6yBp2x9rl3fsTR8zKPok0ZxOMhJabaf87n tTMi8cuZORfUPkfzta4LsjNe00C+uFJpfV1JMUmgJyMugNMTCfatibFW723w1K75k3hz a2DKv6vziKgMtcCmvdJ7l93WyWFFU5VwsJYIyR8ByIFgCrcwLTatC9uwdbEQRJfj6mf4 L9HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600015; x=1687192015; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SUpVnyemWEPOpO605ffDicbCJQoqq3GtxGHxc5DnjBc=; b=MxbXUaFicaVxcoGQe+x+bOLKcv17DaUMYKflV0vIEE4w0CEfpPLTDGcyUDSqj7YCUk /I+UN8fGQGuAk6X75ayLGp9PrPe7cQOzuNVack2jluDpxhED/tWYf/kCJOZ6pLMefc6C W+ts53sX/TlvNIJu9TI0/ZW92+ZCIxSiODUZ+ps5SXAV+NsA3bbg2GUsoyLY+CTEah/u vzedlCvrVT45EoetumK/CBPXXAmSeb5lcXi0mfkKduBKLfmtpQ36Mdw3/xh1Tkmyve0N Hjx355EnIqqNRcVPrVOS5Zo144EG9gopdYflsdYTOTxBMUre/6sHPNBu8okfK9tdKJOy 0bXw== X-Gm-Message-State: AC+VfDyk5U5IqsosNkwhLvvCzHyUdsEXurRgoc0wvFIYje8JaVBSlSML 8iAGFWz/uJ0hxkqHQd3poUt240sgQ1wyrNK68xE= X-Received: by 2002:a17:902:f7d6:b0:1aa:fec9:5219 with SMTP id h22-20020a170902f7d600b001aafec95219mr5812680plw.61.1684600015618; Sat, 20 May 2023 09:26:55 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:55 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 11/27] include/host: Split out atomic128-ldst.h Date: Sat, 20 May 2023 09:26:18 -0700 Message-Id: <20230520162634.3991009-12-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::430; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x430.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Separates the aarch64-specific portion into its own file. Signed-off-by: Richard Henderson --- host/include/aarch64/host/atomic128-cas.h | 2 +- host/include/aarch64/host/atomic128-ldst.h | 49 ++++++++++++++ host/include/generic/host/atomic128-ldst.h | 57 +++++++++++++++++ include/qemu/atomic128.h | 74 +--------------------- 4 files changed, 108 insertions(+), 74 deletions(-) create mode 100644 host/include/aarch64/host/atomic128-ldst.h create mode 100644 host/include/generic/host/atomic128-ldst.h diff --git a/host/include/aarch64/host/atomic128-cas.h b/host/include/aarch64/host/atomic128-cas.h index 1247995419..33f365ce67 100644 --- a/host/include/aarch64/host/atomic128-cas.h +++ b/host/include/aarch64/host/atomic128-cas.h @@ -1,6 +1,6 @@ /* * SPDX-License-Identifier: GPL-2.0-or-later - * Compare-and-swap for 128-bit atomic operations, generic version. + * Compare-and-swap for 128-bit atomic operations, aarch64 version. * * Copyright (C) 2018, 2023 Linaro, Ltd. * diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h new file mode 100644 index 0000000000..c2e7b44bc5 --- /dev/null +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -0,0 +1,49 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, aarch64 version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_LDST_H +#define AARCH64_ATOMIC128_LDST_H + +/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ +#if !defined(CONFIG_ATOMIC128) && !defined(CONFIG_USER_ONLY) +/* We can do better than cmpxchg for AArch64. */ +static inline Int128 atomic16_read(Int128 *ptr) +{ + uint64_t l, h; + uint32_t tmp; + + /* The load must be paired with the store to guarantee not tearing. */ + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "stxp %w[tmp], %[l], %[h], %[mem]\n\t" + "cbnz %w[tmp], 0b" + : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + + return int128_make128(l, h); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + uint64_t l = int128_getlo(val), h = int128_gethi(val); + uint64_t t1, t2; + + /* Load into temporaries to acquire the exclusive access lock. */ + asm("0: ldxp %[t1], %[t2], %[mem]\n\t" + "stxp %w[t1], %[l], %[h], %[mem]\n\t" + "cbnz %w[t1], 0b" + : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) + : [l] "r"(l), [h] "r"(h)); +} + +# define HAVE_ATOMIC128 1 +#else +#include "host/include/generic/host/atomic128-ldst.h" +#endif + +#endif /* AARCH64_ATOMIC128_LDST_H */ diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h new file mode 100644 index 0000000000..e7354a9255 --- /dev/null +++ b/host/include/generic/host/atomic128-ldst.h @@ -0,0 +1,57 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, generic version. + * + * Copyright (C) 2018, 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef HOST_ATOMIC128_LDST_H +#define HOST_ATOMIC128_LDST_H + +#if defined(CONFIG_ATOMIC128) +static inline Int128 atomic16_read(Int128 *ptr) +{ + Int128Alias r; + + r.i = qatomic_read__nocheck((__int128_t *)ptr); + return r.s; +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + Int128Alias v; + + v.s = val; + qatomic_set__nocheck((__int128_t *)ptr, v.i); +} + +# define HAVE_ATOMIC128 1 +#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 +static inline Int128 atomic16_read(Int128 *ptr) +{ + /* Maybe replace 0 with 0, returning the old value. */ + Int128 z = int128_make64(0); + return atomic16_cmpxchg(ptr, z, z); +} + +static inline void atomic16_set(Int128 *ptr, Int128 val) +{ + Int128 old = *ptr, cmp; + do { + cmp = old; + old = atomic16_cmpxchg(ptr, cmp, val); + } while (int128_ne(old, cmp)); +} + +# define HAVE_ATOMIC128 1 +#else +/* Fallback definitions that must be optimized away, or error. */ +Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); +void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); +# define HAVE_ATOMIC128 0 +#endif + +#endif /* HOST_ATOMIC128_LDST_H */ diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index 10a2322c44..3a8adb4d47 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -42,78 +42,6 @@ */ #include "host/atomic128-cas.h" - -#if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_read(Int128 *ptr) -{ - Int128Alias r; - - r.i = qatomic_read__nocheck((__int128_t *)ptr); - return r.s; -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - Int128Alias v; - - v.s = val; - qatomic_set__nocheck((__int128_t *)ptr, v.i); -} - -# define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && defined(__aarch64__) -/* We can do better than cmpxchg for AArch64. */ -static inline Int128 atomic16_read(Int128 *ptr) -{ - uint64_t l, h; - uint32_t tmp; - - /* The load must be paired with the store to guarantee not tearing. */ - asm("0: ldxp %[l], %[h], %[mem]\n\t" - "stxp %w[tmp], %[l], %[h], %[mem]\n\t" - "cbnz %w[tmp], 0b" - : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); - - return int128_make128(l, h); -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - uint64_t l = int128_getlo(val), h = int128_gethi(val); - uint64_t t1, t2; - - /* Load into temporaries to acquire the exclusive access lock. */ - asm("0: ldxp %[t1], %[t2], %[mem]\n\t" - "stxp %w[t1], %[l], %[h], %[mem]\n\t" - "cbnz %w[t1], 0b" - : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) - : [l] "r"(l), [h] "r"(h)); -} - -# define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 -static inline Int128 atomic16_read(Int128 *ptr) -{ - /* Maybe replace 0 with 0, returning the old value. */ - Int128 z = int128_make64(0); - return atomic16_cmpxchg(ptr, z, z); -} - -static inline void atomic16_set(Int128 *ptr, Int128 val) -{ - Int128 old = *ptr, cmp; - do { - cmp = old; - old = atomic16_cmpxchg(ptr, cmp, val); - } while (int128_ne(old, cmp)); -} - -# define HAVE_ATOMIC128 1 -#else -/* Fallback definitions that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); -void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); -# define HAVE_ATOMIC128 0 -#endif /* Some definition for HAVE_ATOMIC128 */ +#include "host/atomic128-ldst.h" #endif /* QEMU_ATOMIC128_H */ From patchwork Sat May 20 16:26:19 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684310 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448189wrt; Sat, 20 May 2023 09:29:43 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7uUiFbnXjtPllcDU5oCXeV8MP8JCaIpKULs7Dmt48Tyhb5rToxpqGLuK9dWeCK6DIUXFwL X-Received: by 2002:a05:622a:1493:b0:3f1:a6d:d99c with SMTP id t19-20020a05622a149300b003f10a6dd99cmr10740237qtx.34.1684600183689; Sat, 20 May 2023 09:29:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600183; cv=none; d=google.com; s=arc-20160816; b=PkS3itr0c9yVWyUUHa0SCv0hS45u50ryToxm9g3UdphaqlcvY7KpSQEqLnV7QAbupq a7gDFAy1jUSmf5pVfoF9/OexeoaJCzSxKhAH3kL/17OoSkfQl3CM/QN11RPLg8qDGFYR Ju7SPXc1aS4IEP0njTvSw505imATKfwRomTyc/uHSEOnxCBsATfJZn9CtG5WUiLEzRNg 1qfNx9rr7Moyw3g69pr0UlcYCBtH17wOZcsMgUQA4bu4wl9roVGjhlY+qCYir5QIt5bX EgzS+RTHF89UqPeXXOdM2GGAZNb8NWOurX6tCxv14ulVsPAgqQzBUiv+PrzlcS8DDBKa vmmA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=AHAmgIp57eQq/xk6K8cmnraRLR5HX4TvEutgtaIW/48=; b=qnzaRBOGMmvu0PBoo4I84o4t3LCEeOtUomNIoiWclEtfy1cl/HfXhpocpwNWlDSqrj oL5YMC3XDyuRghNMFn/E6VrqAdJ05Rr5QBsdWqUB8A75JNcek0A1AITI41xitZUcIO29 bVclBDhULA5DkeMJ2t4BZxYaVKnNjOBMysz7IZh58YyEKWAoggrIHsR5Fmb5QaiEhadl p/+MgRu6GZoSnixLIzRPO8B6Cy0mLX/T+uRZh4PFEk6LDigKII+mK+Im7+2QHGQVKtdW IoR0sji84JWI73vYDkxzpH6Dg9f9nYfGPmJkmqBehKq4fENqp8Nwgr8y8snhWPixwNVQ 7bOA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gVu4BAXH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z19-20020ac87f93000000b003f4ef4d694asi1370202qtj.4.2023.05.20.09.29.43 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:43 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=gVu4BAXH; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQH-0003kL-60; Sat, 20 May 2023 12:27:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ9-0003fO-64 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:06 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-00039B-1v for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:03 -0400 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1ae851f2a7dso11661935ad.0 for ; Sat, 20 May 2023 09:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600016; x=1687192016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AHAmgIp57eQq/xk6K8cmnraRLR5HX4TvEutgtaIW/48=; b=gVu4BAXH5NbAJI0BPJ2q1YYcSDfuVbxJV8Jtpze6pFN5Ow8tbjCDYwsd2Hnre/CYWS x0Nq25pzGKhpqA9bacvZy6inhNLlfysgpqUDWKgBQpYn9IRIb8soCNynNGTkC+2uU/Cj uH6JFJg2SfTGpcFQDbcJukVweHHZDiU2/PunLTbIvp18RB+i+whcWyA2+W776LmSMDsE woqwHEGtaulH8/orhywc189NWc1dG5YwChzo9TL8Zy0ZIYI0UcIpQi3Z26KgVyaWKReZ yGCVdEKH/b3T49/aMM/NDkmsLfUzD7AHrEGNx39ue+/84KctH16mbh+qKG69oJM2sLvD DZ/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600016; x=1687192016; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AHAmgIp57eQq/xk6K8cmnraRLR5HX4TvEutgtaIW/48=; b=aokROlIISklo7EEWDhe+3MXcsN8FgvW+Vgqt//8Sfa1aMXHfd6k/o2jKrtGqYW8WzR Ca64DhWBqwTKiyV27udMN7sJu0Hxa/0lxVgUhfJB+eLDZefMUhHxwx16OWJ5QiJ9l8uv VR+1gaM2ouF98hiGmIiP63FxebK204xjCS+ETbLZ8LKYRi3AkB6Zw905t6Gzk1EqLic9 L4t/WPuToyIopdD/BZdnmQSNudmfM+cULb2wxYvSAyUVo/C0xFC9IoUd67zV3akloaW5 w6YAE5WXT/Dbiyrp1OLqqTdvdkJ2wA3f9cjo6pMGovmHSplZNxHQcBty/5MYV/vE/ji0 67CQ== X-Gm-Message-State: AC+VfDxT3R4DRMnv5PcKT79mcsu29fnkTam3pU888sKcj7IpFk4tp1ZJ r75ixIVXesZQRjgcMQwWKHEzVjtGTkxcdHQllDI= X-Received: by 2002:a17:902:ce81:b0:1ac:61ad:d6f2 with SMTP id f1-20020a170902ce8100b001ac61add6f2mr6773788plg.5.1684600016500; Sat, 20 May 2023 09:26:56 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 12/27] meson: Fix detect atomic128 support with optimization Date: Sat, 20 May 2023 09:26:19 -0700 Message-Id: <20230520162634.3991009-13-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Silly typo: sizeof(16) != 16. Fixes: e61f1efeb730 ("meson: Detect atomic128 support with optimization") Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- meson.build | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meson.build b/meson.build index 4ffc0d3e59..5e7fc6345f 100644 --- a/meson.build +++ b/meson.build @@ -2555,7 +2555,7 @@ if has_int128 # __alignof(unsigned __int128) for the host. atomic_test_128 = ''' int main(int ac, char **av) { - unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], sizeof(16)); + unsigned __int128 *p = __builtin_assume_aligned(av[ac - 1], 16); p[1] = __atomic_load_n(&p[0], __ATOMIC_RELAXED); __atomic_store_n(&p[2], p[3], __ATOMIC_RELAXED); __atomic_compare_exchange_n(&p[4], &p[5], p[6], 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED); From patchwork Sat May 20 16:26:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684308 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448164wrt; Sat, 20 May 2023 09:29:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ494im7MxnRWHOAG1Di1Qief+dbVLc/Al/WbZ45GXBxP2oOGoiVc8SGGsf/UJ27QNPE+uOu X-Received: by 2002:a05:6214:c2e:b0:621:8ca6:2ac8 with SMTP id a14-20020a0562140c2e00b006218ca62ac8mr10069474qvd.16.1684600178665; Sat, 20 May 2023 09:29:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600178; cv=none; d=google.com; s=arc-20160816; b=PbASmfoeK5LOo9NhWjPJmpLw0074qeRhbIteBBhSVLlPY9a+IhKqZAjOi4byP6r4c9 AImCEJbhHBQhoJW7BrIA34tFucZfdDsbI9UZt+FmAX/hyUx+TrHfHA1PHfH26jw8bnbg 61mETVcabvu70wgXparSx7iNPj7WVBPnZSaer+ObWzBx0pcj3f6sqsL/EJOV4pOAiJ0S hVk5WYkyqIpCxAARoilsNh8PHffYb40fyipJ/Zhx4e57Qne38n5bD7BQ+hLRF7SYeYMq Rj7vYHhZLOUrkncBdzBLVdoNYRfCidNZQngGP/84Saoj/aLmFghCb8SSA8+zAAvLEYis ImZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=GxQyIFbLZCZxxhrok/J7/MX47kFN/7dHwtmaEz8DzvXulKSAZ1Xh6htHn+U+3hdNYa QkTxd6LZAgV6xY9g2rAckNKcdiegPD8E0m/R6vYyoxR6u+XG1gFbipBWc1rkciQOVQjg iC8j0OBVDWOqQtLhubqXGr46h4kZPRxIhKPkgAubTcRCHGXEd5m98IZKKi8YgvTFhT0e cQfWOwKt3am1Z1+16Hp1vd1vKwOFYH3R1hTNxSFXQZ8idJXxMKVQpTM1ynLyt15G4CJM u0AivBvhTo0HWDlvjMUhOgStO7477KHJQm7lb1stqdpxLhJ5HYthu7TqwupBwKkO7Cdl /J9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lGaHcHYc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id gu6-20020a056214260600b006236d96c8e3si1275177qvb.348.2023.05.20.09.29.38 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=lGaHcHYc; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQJ-0003mZ-5i; Sat, 20 May 2023 12:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQ9-0003fJ-4F for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:06 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ3-00039N-25 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:03 -0400 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1ae6dce19f7so20701315ad.3 for ; Sat, 20 May 2023 09:26:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600017; x=1687192017; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=lGaHcHYcW5sessr2JXeHN3gEF1bQzcAqnz0Z2gpqw7SBttpcTsXKlN6oMp0WuV6JP8 UMLwxrg6zUclXcwkwmbD2vc/+o62K56rf/QeJvAiIl6X8sp3hgWJDs1QEuh5qifTQO57 zmNYgtYq4A8pc7d5d3z3xv3wze05HGzxpcU5GcWc1YLBm21dZc9JjoYkyAETHIPpWmbl sRsPb0hzWfBXmOTxQiL+lrRnH6kSZ4sItIWbx/rsdCjm4jVqJAI5VSRzVHVcRvb/KSAp Yp4Y6naleRuQ2/c1/gpZzN3DTeEJ7rNbGyC24f0+ybBNUJO/N1e83y11KvmSqpHOj5+g 7SzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600017; x=1687192017; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2DqFQEVjlcRQv07fj6FpnhLPU4q7Af597SdqlFdm/+U=; b=hA73ODfY6XQ59xDch3OqJ3B0AbvqP+vm7pA3iF3J3j7KttMLi0DMr2eStHFpkwuUsM qKF79QMUudfdt/d0hWfP5IBezTGEq7nrVMxIPYY1x0YM5CfvGy9tIsSFBs93a6Ls/E+t 7X0A8BEropQKID9MGZXAydPRwz4IAgea032p1dw4UVPpjVPxDvww9sQHkh/itAOt3ffP y1eqRbf3oRbqp70RJeKNvk5UJb3I86BsfXurpzjgtXqQTk1CDKK1hDKQH17nupDzpR3q 3f8Kf9WFg/Q0p42HmUfMmRlroTzncztboqNNMbRV3B/y2b5Rv3Z7kwjefm3tpNeMybsz SfAg== X-Gm-Message-State: AC+VfDyOwmsbUtn73apO8w6b+BSqHhWSv1fMe+KoUxu8c5WP2x+dsFqG b9nOOalyTzsYesd8/M0s6VRxXPRapq30JJ0GDoA= X-Received: by 2002:a17:903:2444:b0:1aa:f203:781c with SMTP id l4-20020a170903244400b001aaf203781cmr7423499pls.44.1684600017246; Sat, 20 May 2023 09:26:57 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:56 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 13/27] include/qemu: Move CONFIG_ATOMIC128_OPT handling to atomic128.h Date: Sat, 20 May 2023 09:26:20 -0700 Message-Id: <20230520162634.3991009-14-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Not only the routines in ldst_atomicity.c.inc need markup, but also the ones in the headers. Signed-off-by: Richard Henderson --- host/include/generic/host/atomic128-cas.h | 12 ++++++++---- host/include/generic/host/atomic128-ldst.h | 18 ++++++++++++------ include/qemu/atomic128.h | 17 +++++++++++++++++ accel/tcg/ldst_atomicity.c.inc | 17 ----------------- 4 files changed, 37 insertions(+), 27 deletions(-) diff --git a/host/include/generic/host/atomic128-cas.h b/host/include/generic/host/atomic128-cas.h index 513622fe34..991d3da082 100644 --- a/host/include/generic/host/atomic128-cas.h +++ b/host/include/generic/host/atomic128-cas.h @@ -12,24 +12,28 @@ #define HOST_ATOMIC128_CAS_H #if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r, c, n; c.s = cmp; n.s = new; - r.i = qatomic_cmpxchg__nocheck((__int128_t *)ptr, c.i, n.i); + r.i = qatomic_cmpxchg__nocheck(ptr_align, c.i, n.i); return r.s; } # define HAVE_CMPXCHG128 1 #elif defined(CONFIG_CMPXCHG128) -static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r, c, n; c.s = cmp; n.s = new; - r.i = __sync_val_compare_and_swap_16((__int128_t *)ptr, c.i, n.i); + r.i = __sync_val_compare_and_swap_16(ptr_align, c.i, n.i); return r.s; } # define HAVE_CMPXCHG128 1 diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index e7354a9255..46911dfb61 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -12,32 +12,38 @@ #define HOST_ATOMIC128_LDST_H #if defined(CONFIG_ATOMIC128) -static inline Int128 atomic16_read(Int128 *ptr) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read(Int128 *ptr) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r; - r.i = qatomic_read__nocheck((__int128_t *)ptr); + r.i = qatomic_read__nocheck(ptr_align); return r.s; } -static inline void atomic16_set(Int128 *ptr, Int128 val) +static inline void ATTRIBUTE_ATOMIC128_OPT +atomic16_set(Int128 *ptr, Int128 val) { + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias v; v.s = val; - qatomic_set__nocheck((__int128_t *)ptr, v.i); + qatomic_set__nocheck(ptr_align, v.i); } # define HAVE_ATOMIC128 1 #elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 -static inline Int128 atomic16_read(Int128 *ptr) +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read(Int128 *ptr) { /* Maybe replace 0 with 0, returning the old value. */ Int128 z = int128_make64(0); return atomic16_cmpxchg(ptr, z, z); } -static inline void atomic16_set(Int128 *ptr, Int128 val) +static inline void ATTRIBUTE_ATOMIC128_OPT +atomic16_set(Int128 *ptr, Int128 val) { Int128 old = *ptr, cmp; do { diff --git a/include/qemu/atomic128.h b/include/qemu/atomic128.h index 3a8adb4d47..34554bf0ac 100644 --- a/include/qemu/atomic128.h +++ b/include/qemu/atomic128.h @@ -15,6 +15,23 @@ #include "qemu/int128.h" +/* + * If __alignof(unsigned __int128) < 16, GCC may refuse to inline atomics + * that are supported by the host, e.g. s390x. We can force the pointer to + * have our known alignment with __builtin_assume_aligned, however prior to + * GCC 13 that was only reliable with optimization enabled. See + * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 + */ +#if defined(CONFIG_ATOMIC128_OPT) +# if !defined(__OPTIMIZE__) +# define ATTRIBUTE_ATOMIC128_OPT __attribute__((optimize("O1"))) +# endif +# define CONFIG_ATOMIC128 +#endif +#ifndef ATTRIBUTE_ATOMIC128_OPT +# define ATTRIBUTE_ATOMIC128_OPT +#endif + /* * GCC is a house divided about supporting large atomic operations. * diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index ba5db7c366..b89631bbef 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -16,23 +16,6 @@ #endif #define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) -/* - * If __alignof(unsigned __int128) < 16, GCC may refuse to inline atomics - * that are supported by the host, e.g. s390x. We can force the pointer to - * have our known alignment with __builtin_assume_aligned, however prior to - * GCC 13 that was only reliable with optimization enabled. See - * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=107389 - */ -#if defined(CONFIG_ATOMIC128_OPT) -# if !defined(__OPTIMIZE__) -# define ATTRIBUTE_ATOMIC128_OPT __attribute__((optimize("O1"))) -# endif -# define CONFIG_ATOMIC128 -#endif -#ifndef ATTRIBUTE_ATOMIC128_OPT -# define ATTRIBUTE_ATOMIC128_OPT -#endif - #if defined(CONFIG_ATOMIC128) # define HAVE_al16_fast true #else From patchwork Sat May 20 16:26:21 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684313 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448447wrt; Sat, 20 May 2023 09:30:23 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7ox5OqjkMfsFPuTTjs1qYEVcE9fRMNb/S1L4AvVhp55Kl0mVyISxHOJ8Ext5nfrpAEWN+T X-Received: by 2002:a05:622a:256:b0:3f5:30fd:a2ca with SMTP id c22-20020a05622a025600b003f530fda2camr9151022qtx.19.1684600223786; Sat, 20 May 2023 09:30:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600223; cv=none; d=google.com; s=arc-20160816; b=eZwY0utYJ1l88UCdXU/5DQYh+wdeaYJkDTwbJ0o5YJHLED3Q5Qly2QJN3F/EMraFLZ CAMJtwKO8oMeWFi9KzXk+B09SmEIObOZfu9LYW8NM5rHf47AsqlGDs05nIuXPpvSpDfJ B9mcWwaZBCKpiWoEtFBRVdyT4fvr5bWTcX3k82t6XrKzNFbL19gxR8f+zeoc/RLWPuIW tfyyZtqmovmxCA2iAmTwBlRgW6YkNoh0wPdXdh0wk4EsGxZwZg/c1aEIulLGfPKfqUfu 4LEOM4F1dqNxHhV0HsAxs1f/dpaQ1lSkrmS3pU2/s/KjmREgW1VKS7pIcgZthdh/C4FD Cveg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=FX68OpTplII0gpJSh+PbQ/Fe08RRRG5aZ959wyDX+duetYnas+Oj7eZKCRCWu1h2U+ qB74EEuARbEn4SPJ2s7jAQpczVqIlOwxR5mlcLcHaz5ZedV6VQbs83ioD5WQdY1kXZQP 3dHTtNbQrXs42tu0jaeUi6wS8/QIM47rYdZ20/4VP88PrusYg4WI6Dsfdw70dNGQXgwB G/QXRWc9ZhPyUQRwwRXw7ZXrU7AOjS5SYSjGDQ0ejFwiPzeaQVMFb8dbNTPzOIgwG2QP qn86IA4Nixe7V3DjUWc1qRRbPw9ztPK3007MYHoRZd7JkNMKRyM3+A9XRHCASlPy/9v5 9jIA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=jdehzckn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id s5-20020a05622a018500b003e392e95990si1339359qtw.251.2023.05.20.09.30.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=jdehzckn; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQJ-0003ni-M9; Sat, 20 May 2023 12:27:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQC-0003hg-SS for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:10 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ5-00039Y-0W for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:07 -0400 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1ae875bf125so10482645ad.1 for ; Sat, 20 May 2023 09:26:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600018; x=1687192018; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=jdehzcknbhahn3YrpWtWGvjVp+1YD5TSwOTtUpns1ZN888QVuPolQfP6mxrxJi6VeR wONd70ntXWkW0l3xzzPD1KAJjM+e8xhZLBuHHBNl8EjziMWqVlgI8eeZDVRsPkzd3mEA 5fm8J/w0qxduqS+Tbn27laID0m9lEPmsrIgMFCXV/4H6cGiwy6q5D/Iaoo8gmMIMKJdF D1D2nLvwAET9HJWCqiONrLsyGSig9jcuad2jm2NMahqoi7+LtCMy9OPm3oAsDw+vrF8Q 42YKisUPxW0058wu02OzJxSgxIjLFFj9HNyoQmIUaPUcaQWlS/fQ6/Qp+d9mJii686GK ynKA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600018; x=1687192018; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LDPGtChIiM90g/ZXe1/z5yfO/noMZETMdebtBSg+Cq0=; b=DYRspKZM0uXLmDNL6VRJiVSyMJegNPxjZvTUHB0Sy29E8weFcKdG264UGtTuMT5G4X w6LMFtLeUH9rIIMIJe3KGKbT3kMLNtsc/jn/DGLh4by+J3jp2kiiay3Q8mucrWNhTQZo H1jLQewW9oPc0R92MQUTEDejwop01C7I3HEOiTONzUcx67x9JnO16r4lKsAvMv6OYbE2 A3yO1yy4LX34y+oaNnXWd2/kWcFmTCFQS+qwoLkrxB5gOCurzuPdNY9oYBB90GcgeFYT TrlqWP9lhvvq4z5dAmdQMaAU70Nc0hlJ57S3p7J4qo8wF7911Wv6gpRR/54LPJZqWuMk QSOg== X-Gm-Message-State: AC+VfDyloicQQAknWrpKT3zARwPdB25HlZcRqXxNrN99itriIg+brdRC XnFrq/W5vMyCkLKmw1xHWbPBn8iPY8XyCBQhrQo= X-Received: by 2002:a17:902:c20c:b0:1ab:d2c:a1a6 with SMTP id 12-20020a170902c20c00b001ab0d2ca1a6mr5793978pll.69.1684600018265; Sat, 20 May 2023 09:26:58 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:57 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-ppc@nongnu.org, Daniel Henrique Barboza , =?utf-8?q?C=C3=A9dric_Le_G?= =?utf-8?q?oater?= , David Gibson , Greg Kurz Subject: [PATCH 14/27] target/ppc: Use tcg_gen_qemu_{ld, st}_i128 for LQARX, LQ, STQ Date: Sat, 20 May 2023 09:26:21 -0700 Message-Id: <20230520162634.3991009-15-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No need to roll our own, as this is now provided by tcg. This was the last use of retxl, so remove that too. Signed-off-by: Richard Henderson --- Cc: qemu-ppc@nongnu.org Cc: Daniel Henrique Barboza Cc: "Cédric Le Goater" Cc: David Gibson Cc: Greg Kurz --- target/ppc/cpu.h | 1 - target/ppc/helper.h | 9 ---- target/ppc/mem_helper.c | 48 -------------------- target/ppc/translate.c | 34 ++------------- target/ppc/translate/fixedpoint-impl.c.inc | 51 +++------------------- 5 files changed, 11 insertions(+), 132 deletions(-) diff --git a/target/ppc/cpu.h b/target/ppc/cpu.h index 1c02596d9f..0f9f2e1a0c 100644 --- a/target/ppc/cpu.h +++ b/target/ppc/cpu.h @@ -1124,7 +1124,6 @@ struct CPUArchState { /* used to speed-up TLB assist handlers */ target_ulong nip; /* next instruction pointer */ - uint64_t retxh; /* high part of 128-bit helper return */ /* when a memory exception occurs, the access type is stored here */ int access_type; diff --git a/target/ppc/helper.h b/target/ppc/helper.h index 0beaca5c7a..38efbc351c 100644 --- a/target/ppc/helper.h +++ b/target/ppc/helper.h @@ -810,12 +810,3 @@ DEF_HELPER_4(DSCLIQ, void, env, fprp, fprp, i32) DEF_HELPER_1(tbegin, void, env) DEF_HELPER_FLAGS_1(fixup_thrm, TCG_CALL_NO_RWG, void, env) - -#ifdef TARGET_PPC64 -DEF_HELPER_FLAGS_3(lq_le_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) -DEF_HELPER_FLAGS_3(lq_be_parallel, TCG_CALL_NO_WG, i64, env, tl, i32) -DEF_HELPER_FLAGS_5(stq_le_parallel, TCG_CALL_NO_WG, - void, env, tl, i64, i64, i32) -DEF_HELPER_FLAGS_5(stq_be_parallel, TCG_CALL_NO_WG, - void, env, tl, i64, i64, i32) -#endif diff --git a/target/ppc/mem_helper.c b/target/ppc/mem_helper.c index 1578887a8f..46eae65819 100644 --- a/target/ppc/mem_helper.c +++ b/target/ppc/mem_helper.c @@ -367,54 +367,6 @@ target_ulong helper_lscbx(CPUPPCState *env, target_ulong addr, uint32_t reg, return i; } -#ifdef TARGET_PPC64 -uint64_t helper_lq_le_parallel(CPUPPCState *env, target_ulong addr, - uint32_t opidx) -{ - Int128 ret; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - ret = cpu_atomic_ldo_le_mmu(env, addr, opidx, GETPC()); - env->retxh = int128_gethi(ret); - return int128_getlo(ret); -} - -uint64_t helper_lq_be_parallel(CPUPPCState *env, target_ulong addr, - uint32_t opidx) -{ - Int128 ret; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - ret = cpu_atomic_ldo_be_mmu(env, addr, opidx, GETPC()); - env->retxh = int128_gethi(ret); - return int128_getlo(ret); -} - -void helper_stq_le_parallel(CPUPPCState *env, target_ulong addr, - uint64_t lo, uint64_t hi, uint32_t opidx) -{ - Int128 val; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - val = int128_make128(lo, hi); - cpu_atomic_sto_le_mmu(env, addr, val, opidx, GETPC()); -} - -void helper_stq_be_parallel(CPUPPCState *env, target_ulong addr, - uint64_t lo, uint64_t hi, uint32_t opidx) -{ - Int128 val; - - /* We will have raised EXCP_ATOMIC from the translator. */ - assert(HAVE_ATOMIC128); - val = int128_make128(lo, hi); - cpu_atomic_sto_be_mmu(env, addr, val, opidx, GETPC()); -} -#endif - /*****************************************************************************/ /* Altivec extension helpers */ #if HOST_BIG_ENDIAN diff --git a/target/ppc/translate.c b/target/ppc/translate.c index f603f1a939..1720570b9b 100644 --- a/target/ppc/translate.c +++ b/target/ppc/translate.c @@ -3757,6 +3757,7 @@ static void gen_lqarx(DisasContext *ctx) { int rd = rD(ctx->opcode); TCGv EA, hi, lo; + TCGv_i128 t16; if (unlikely((rd & 1) || (rd == rA(ctx->opcode)) || (rd == rB(ctx->opcode)))) { @@ -3772,36 +3773,9 @@ static void gen_lqarx(DisasContext *ctx) lo = cpu_gpr[rd + 1]; hi = cpu_gpr[rd]; - if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { - if (HAVE_ATOMIC128) { - TCGv_i32 oi = tcg_temp_new_i32(); - if (ctx->le_mode) { - tcg_gen_movi_i32(oi, make_memop_idx(MO_LE | MO_128 | MO_ALIGN, - ctx->mem_idx)); - gen_helper_lq_le_parallel(lo, cpu_env, EA, oi); - } else { - tcg_gen_movi_i32(oi, make_memop_idx(MO_BE | MO_128 | MO_ALIGN, - ctx->mem_idx)); - gen_helper_lq_be_parallel(lo, cpu_env, EA, oi); - } - tcg_gen_ld_i64(hi, cpu_env, offsetof(CPUPPCState, retxh)); - } else { - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; - return; - } - } else if (ctx->le_mode) { - tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_LEUQ | MO_ALIGN_16); - tcg_gen_mov_tl(cpu_reserve, EA); - gen_addr_add(ctx, EA, EA, 8); - tcg_gen_qemu_ld_i64(hi, EA, ctx->mem_idx, MO_LEUQ); - } else { - tcg_gen_qemu_ld_i64(hi, EA, ctx->mem_idx, MO_BEUQ | MO_ALIGN_16); - tcg_gen_mov_tl(cpu_reserve, EA); - gen_addr_add(ctx, EA, EA, 8); - tcg_gen_qemu_ld_i64(lo, EA, ctx->mem_idx, MO_BEUQ); - } + t16 = tcg_temp_new_i128(); + tcg_gen_qemu_ld_i128(t16, EA, ctx->mem_idx, DEF_MEMOP(MO_128 | MO_ALIGN)); + tcg_gen_extr_i128_i64(lo, hi, t16); tcg_gen_st_tl(hi, cpu_env, offsetof(CPUPPCState, reserve_val)); tcg_gen_st_tl(lo, cpu_env, offsetof(CPUPPCState, reserve_val2)); diff --git a/target/ppc/translate/fixedpoint-impl.c.inc b/target/ppc/translate/fixedpoint-impl.c.inc index 02d86b77a8..f47f1a50e8 100644 --- a/target/ppc/translate/fixedpoint-impl.c.inc +++ b/target/ppc/translate/fixedpoint-impl.c.inc @@ -72,7 +72,7 @@ static bool do_ldst_quad(DisasContext *ctx, arg_D *a, bool store, bool prefixed) #if defined(TARGET_PPC64) TCGv ea; TCGv_i64 low_addr_gpr, high_addr_gpr; - MemOp mop; + TCGv_i128 t16; REQUIRE_INSNS_FLAGS(ctx, 64BX); @@ -101,51 +101,14 @@ static bool do_ldst_quad(DisasContext *ctx, arg_D *a, bool store, bool prefixed) low_addr_gpr = cpu_gpr[a->rt + 1]; high_addr_gpr = cpu_gpr[a->rt]; } + t16 = tcg_temp_new_i128(); - if (tb_cflags(ctx->base.tb) & CF_PARALLEL) { - if (HAVE_ATOMIC128) { - mop = DEF_MEMOP(MO_128); - TCGv_i32 oi = tcg_constant_i32(make_memop_idx(mop, ctx->mem_idx)); - if (store) { - if (ctx->le_mode) { - gen_helper_stq_le_parallel(cpu_env, ea, low_addr_gpr, - high_addr_gpr, oi); - } else { - gen_helper_stq_be_parallel(cpu_env, ea, high_addr_gpr, - low_addr_gpr, oi); - - } - } else { - if (ctx->le_mode) { - gen_helper_lq_le_parallel(low_addr_gpr, cpu_env, ea, oi); - tcg_gen_ld_i64(high_addr_gpr, cpu_env, - offsetof(CPUPPCState, retxh)); - } else { - gen_helper_lq_be_parallel(high_addr_gpr, cpu_env, ea, oi); - tcg_gen_ld_i64(low_addr_gpr, cpu_env, - offsetof(CPUPPCState, retxh)); - } - } - } else { - /* Restart with exclusive lock. */ - gen_helper_exit_atomic(cpu_env); - ctx->base.is_jmp = DISAS_NORETURN; - } + if (store) { + tcg_gen_concat_i64_i128(t16, low_addr_gpr, high_addr_gpr); + tcg_gen_qemu_st_i128(t16, ea, ctx->mem_idx, DEF_MEMOP(MO_128)); } else { - mop = DEF_MEMOP(MO_UQ); - if (store) { - tcg_gen_qemu_st_i64(low_addr_gpr, ea, ctx->mem_idx, mop); - } else { - tcg_gen_qemu_ld_i64(low_addr_gpr, ea, ctx->mem_idx, mop); - } - - gen_addr_add(ctx, ea, ea, 8); - - if (store) { - tcg_gen_qemu_st_i64(high_addr_gpr, ea, ctx->mem_idx, mop); - } else { - tcg_gen_qemu_ld_i64(high_addr_gpr, ea, ctx->mem_idx, mop); - } + tcg_gen_qemu_ld_i128(t16, ea, ctx->mem_idx, DEF_MEMOP(MO_128)); + tcg_gen_extr_i128_i64(low_addr_gpr, high_addr_gpr, t16); } #else qemu_build_not_reached(); From patchwork Sat May 20 16:26:22 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684316 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448705wrt; Sat, 20 May 2023 09:30:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7emnPQnLZDFYwBo9H1//PkRIkYJeQDYnafI5hdZ//05mFE4458hHa1BXX/8tfA84TVGsvq X-Received: by 2002:a05:6214:528f:b0:5ef:6841:492 with SMTP id kj15-20020a056214528f00b005ef68410492mr8904823qvb.46.1684600259412; Sat, 20 May 2023 09:30:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600259; cv=none; d=google.com; s=arc-20160816; b=du/98Hxixotmf8kP0FkggA0sYlre83JspaeET/8M0zZqQyY/f8jvelOzyrVNh55sZt pjk46u6tYxhKFrvdnDBAqxf1SDASmnTEzv1luwynGmszRWgKPhhBnhE0uVx08uHuRhKg 94fSLojxCh+b55p0XnzcBLAgSlbu3wkgMzHfVS5RmPRqMIVh2CtN9ogYI93w6Ou6xdMU 2NIz7HCpZ+1umJ6ojv5JqxN/Czji9ZLXRArYANFx5bpxQJbip5l3xeeybrtDHd19Tggl gKtlRCZ1xnqu6Grxt+zHWXl09AwN868s9g6fRz34R8aYox2CVDUpWWUhnKtdZDYWqwFj i4EA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=k38SnU1AW5+4YbZuns9NLZl/SY89QwwaRWKo3oo0nIPvKRYf+fp4xhYSaj0Q+o/3BH d1eatNYlSvIAx8CZWHMam/GHOyXMFO+mCZ5n88G8OG+3NIb65ujiHTj+H8ugYZ5Bh3cG JkF7cwyVGjpG5i1mF+05U7gtQD8I+t2w++WQqtEKBXG2mdVRb8pqrLEVSkv5R1PK1AJM QxCaBQR74CdO6rSvU9NkNN4tfOn1PVJtmCfbbms+TDiISgLdtF1l3vp33e43mFfY+HoO Mu9STQirT0HTbr4k5ky4vlNSwTcOCfEDFcLXEQYUyqa4maqVLW+p0ttWDuVE7gKEtkD4 NGjA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WrCgOZf0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id fw14-20020a056214238e00b006215293def4si1197322qvb.252.2023.05.20.09.30.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=WrCgOZf0; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQI-0003lC-7S; Sat, 20 May 2023 12:27:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQA-0003fW-Gy for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:06 -0400 Received: from mail-pj1-x102a.google.com ([2607:f8b0:4864:20::102a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ5-0003A3-0c for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:06 -0400 Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-25372604818so1708213a91.2 for ; Sat, 20 May 2023 09:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600019; x=1687192019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=WrCgOZf0ZCp6j36nn+TV8MLwdJxCa6t06XeLNlxv9AFquxR+mz95mTk39SCA8sarwG eCyGVl9K4d+qj4lDJu21piQ5UjaDOe6Zq6RD9esdfoLwaz5QXeeGsGVZC+bKqhhZuwHm mOOspHcn/WMG3mUdTIVzozxIw9VQYd1airni9ba6NVA7/lTH0Op7/6BR4nIHIu1sxgvE aEfPGFp4qJK3vJZYjGT6uMazfCi9fRLhddje0SPUc3VWcU5A95WK8h38NPftIdZn0bs/ SJdo5HC/pWYIsMrSPiuEeHcmn/iiC4tnZxoooDfkOv1JAgO3LoEfsf+Bi/2K1nwRMfX5 seLQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600019; x=1687192019; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xmt+ITqZqh7I7gFJaXBYqdm/Oa0NR4029Dhdaw3mjBo=; b=N44b4VLN1wJZqYrmsduRzGBUZVseJVPKHF/I1LwuIEBg4Rm93UKk5hrTAYcIlM0wIB Rbr72uHjUDzOMdqhHPKvXrkCZbJgxvAyZ/SK+gKDS9PqiZzIBP4lt6+qMiX1FKe8t9LM 0m3j3T5agSddiM5K5CuGEPslj9zWplnEmxVwn31SQdWPWXUirSiSdxUFhKWhI5/wbSmH sP31zljYGmA2yi+6p6yYFvjyh18Xpp9yg02DAgntwYXcDu+asta5rMo/rrjwFhxbNWbM sBYzD8h2mqR35QsFEhZ4jM/OBtDf4c7cuJICnRU3Nq/Gku2JDyIIz+o6ciKZbGUs6g0S u0ig== X-Gm-Message-State: AC+VfDwcoBS2ce6mMVu2XKJNz37EMGKiyCyTzX3ZyfXZTAxe8zwQSV81 f2+UPVhA4SMWSw1v2D8ue/465uY1YLDTwa6w52g= X-Received: by 2002:a17:90a:d98f:b0:253:62c2:4e1b with SMTP id d15-20020a17090ad98f00b0025362c24e1bmr4953024pjv.48.1684600019387; Sat, 20 May 2023 09:26:59 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:58 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org, David Hildenbrand , Ilya Leoshkevich Subject: [PATCH 15/27] target/s390x: Use tcg_gen_qemu_{ld, st}_i128 for LPQ, STPQ Date: Sat, 20 May 2023 09:26:22 -0700 Message-Id: <20230520162634.3991009-16-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102a; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org No need to roll our own, as this is now provided by tcg. This was the last use of retxl, so remove that too. Signed-off-by: Richard Henderson --- Cc: qemu-s390x@nongnu.org Cc: David Hildenbrand Cc: Ilya Leoshkevich --- target/s390x/cpu.h | 3 -- target/s390x/helper.h | 4 --- target/s390x/tcg/mem_helper.c | 61 -------------------------------- target/s390x/tcg/translate.c | 30 +++++----------- target/s390x/tcg/insn-data.h.inc | 2 +- 5 files changed, 9 insertions(+), 91 deletions(-) diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h index c47e7adcb1..f130c29f83 100644 --- a/target/s390x/cpu.h +++ b/target/s390x/cpu.h @@ -76,9 +76,6 @@ struct CPUArchState { float_status fpu_status; /* passed to softfloat lib */ - /* The low part of a 128-bit return, or remainder of a divide. */ - uint64_t retxl; - PSW psw; S390CrashReason crash_reason; diff --git a/target/s390x/helper.h b/target/s390x/helper.h index 341bc51ec2..7529e725f2 100644 --- a/target/s390x/helper.h +++ b/target/s390x/helper.h @@ -108,10 +108,6 @@ DEF_HELPER_FLAGS_2(sfas, TCG_CALL_NO_WG, void, env, i64) DEF_HELPER_FLAGS_2(srnm, TCG_CALL_NO_WG, void, env, i64) DEF_HELPER_FLAGS_1(popcnt, TCG_CALL_NO_RWG_SE, i64, i64) DEF_HELPER_2(stfle, i32, env, i64) -DEF_HELPER_FLAGS_2(lpq, TCG_CALL_NO_WG, i64, env, i64) -DEF_HELPER_FLAGS_2(lpq_parallel, TCG_CALL_NO_WG, i64, env, i64) -DEF_HELPER_FLAGS_4(stpq, TCG_CALL_NO_WG, void, env, i64, i64, i64) -DEF_HELPER_FLAGS_4(stpq_parallel, TCG_CALL_NO_WG, void, env, i64, i64, i64) DEF_HELPER_4(mvcos, i32, env, i64, i64, i64) DEF_HELPER_4(cu12, i32, env, i32, i32, i32) DEF_HELPER_4(cu14, i32, env, i32, i32, i32) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 8b58b8d88d..0e0d66b3b6 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -2398,67 +2398,6 @@ uint64_t HELPER(lra)(CPUS390XState *env, uint64_t addr) } #endif -/* load pair from quadword */ -uint64_t HELPER(lpq)(CPUS390XState *env, uint64_t addr) -{ - uintptr_t ra = GETPC(); - uint64_t hi, lo; - - check_alignment(env, addr, 16, ra); - hi = cpu_ldq_data_ra(env, addr + 0, ra); - lo = cpu_ldq_data_ra(env, addr + 8, ra); - - env->retxl = lo; - return hi; -} - -uint64_t HELPER(lpq_parallel)(CPUS390XState *env, uint64_t addr) -{ - uintptr_t ra = GETPC(); - uint64_t hi, lo; - int mem_idx; - MemOpIdx oi; - Int128 v; - - assert(HAVE_ATOMIC128); - - mem_idx = cpu_mmu_index(env, false); - oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - v = cpu_atomic_ldo_be_mmu(env, addr, oi, ra); - hi = int128_gethi(v); - lo = int128_getlo(v); - - env->retxl = lo; - return hi; -} - -/* store pair to quadword */ -void HELPER(stpq)(CPUS390XState *env, uint64_t addr, - uint64_t low, uint64_t high) -{ - uintptr_t ra = GETPC(); - - check_alignment(env, addr, 16, ra); - cpu_stq_data_ra(env, addr + 0, high, ra); - cpu_stq_data_ra(env, addr + 8, low, ra); -} - -void HELPER(stpq_parallel)(CPUS390XState *env, uint64_t addr, - uint64_t low, uint64_t high) -{ - uintptr_t ra = GETPC(); - int mem_idx; - MemOpIdx oi; - Int128 v; - - assert(HAVE_ATOMIC128); - - mem_idx = cpu_mmu_index(env, false); - oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - v = int128_make128(low, high); - cpu_atomic_sto_be_mmu(env, addr, v, oi, ra); -} - /* Execute instruction. This instruction executes an insn modified with the contents of r1. It does not change the executed instruction in memory; it does not change the program counter. diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c index d6670e6a87..3eb3708d55 100644 --- a/target/s390x/tcg/translate.c +++ b/target/s390x/tcg/translate.c @@ -335,11 +335,6 @@ static void store_freg32_i64(int reg, TCGv_i64 v) tcg_gen_st32_i64(v, cpu_env, freg32_offset(reg)); } -static void return_low128(TCGv_i64 dest) -{ - tcg_gen_ld_i64(dest, cpu_env, offsetof(CPUS390XState, retxl)); -} - static void update_psw_addr(DisasContext *s) { /* psw.addr */ @@ -3130,15 +3125,9 @@ static DisasJumpType op_lpd(DisasContext *s, DisasOps *o) static DisasJumpType op_lpq(DisasContext *s, DisasOps *o) { - if (!(tb_cflags(s->base.tb) & CF_PARALLEL)) { - gen_helper_lpq(o->out, cpu_env, o->in2); - } else if (HAVE_ATOMIC128) { - gen_helper_lpq_parallel(o->out, cpu_env, o->in2); - } else { - gen_helper_exit_atomic(cpu_env); - return DISAS_NORETURN; - } - return_low128(o->out2); + o->out_128 = tcg_temp_new_i128(); + tcg_gen_qemu_ld_i128(o->out_128, o->in2, get_mem_index(s), + MO_TE | MO_128 | MO_ALIGN); return DISAS_NEXT; } @@ -4533,14 +4522,11 @@ static DisasJumpType op_stmh(DisasContext *s, DisasOps *o) static DisasJumpType op_stpq(DisasContext *s, DisasOps *o) { - if (!(tb_cflags(s->base.tb) & CF_PARALLEL)) { - gen_helper_stpq(cpu_env, o->in2, o->out2, o->out); - } else if (HAVE_ATOMIC128) { - gen_helper_stpq_parallel(cpu_env, o->in2, o->out2, o->out); - } else { - gen_helper_exit_atomic(cpu_env); - return DISAS_NORETURN; - } + TCGv_i128 t16 = tcg_temp_new_i128(); + + tcg_gen_concat_i64_i128(t16, o->out2, o->out); + tcg_gen_qemu_st_i128(t16, o->in2, get_mem_index(s), + MO_TE | MO_128 | MO_ALIGN); return DISAS_NEXT; } diff --git a/target/s390x/tcg/insn-data.h.inc b/target/s390x/tcg/insn-data.h.inc index 1f1ac742a9..bcc70d99ba 100644 --- a/target/s390x/tcg/insn-data.h.inc +++ b/target/s390x/tcg/insn-data.h.inc @@ -570,7 +570,7 @@ D(0xc804, LPD, SSF, ILA, 0, 0, new_P, r3_P32, lpd, 0, MO_TEUL) D(0xc805, LPDG, SSF, ILA, 0, 0, new_P, r3_P64, lpd, 0, MO_TEUQ) /* LOAD PAIR FROM QUADWORD */ - C(0xe38f, LPQ, RXY_a, Z, 0, a2, r1_P, 0, lpq, 0) + C(0xe38f, LPQ, RXY_a, Z, 0, a2, 0, r1_D64, lpq, 0) /* LOAD POSITIVE */ C(0x1000, LPR, RR_a, Z, 0, r2_32s, new, r1_32, abs, abs32) C(0xb900, LPGR, RRE, Z, 0, r2, r1, 0, abs, abs64) From patchwork Sat May 20 16:26:23 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684317 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448704wrt; Sat, 20 May 2023 09:30:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6yfUd/uqE6WFfnLNVXCVM5jQlTDAK0yc8XhSTrUff+LIGaz89DN9OpgOTgb30/0B9k/8aS X-Received: by 2002:ad4:5bcc:0:b0:621:8b5:229a with SMTP id t12-20020ad45bcc000000b0062108b5229amr9359490qvt.2.1684600259046; Sat, 20 May 2023 09:30:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600259; cv=none; d=google.com; s=arc-20160816; b=dFYSRnlA4vFYTl3yNVrbmrlHreH7OV8bhU00NvtexKnlaND2wqGdyQbuYr6jKnxXgH GHV4LpVABK9qSYRJ14o4VJIZxX1xHrSuSowbE5ccWDoRkIcTIWr30xucYUFiNRa57deM qz59X0aA+r71FKnYy2qf1gCTkMz44oY2+PWjnzY3TibcxdgxELob4h06axZN9JHCokVY hw2wS3R23cVHxoO5/jRgA6Nr4tj/nKZcQUHpfvBBgmUHHuTjXftR9FLDdnm5O0uNk+YA OW7MZZzUkRDXZ1oaqZNXG5pLMAX6TiP2Zyimkk6knFDRQfq6o8z1jl+XAkrfpSwFbp3i v7wA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=ol9f0HLEAIGA8FowB6n0ZN2fTDIRFVUsiNSJUMVLzsQ=; b=C2P8QZahDVMFV+l+qPLQOVnSY3GsRH7AFh7sKM/MY3L6G6TeiGpJR18qxokzDGYK91 NwyCn2chkLAJKAFkgQYvE9zcTHfK5FgHNOISZa16HpayAv5XmLSP4NDm+jp/QpOB0eqC yrdJ9x7cdUTRP1Gng3pynTSNubm57F4aE8+NqZo2np0fcwHVJ325uRIDyeBUdhqbZyRF enxpqSuAgDJxAjpa98GSlaUNwItuzyZ/eF/px79EO5pZjnZAzXvDgk96PqxEj5eFIx7a nGcNLsNtov79wUKrHYa4fLa4QDhnDxjSWsEBMXxf/2d+3L2oAI2mn1UnwLQTHnznJMUB N82g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sJMTFE3Y; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id k7-20020ad45be7000000b00621b361f1eesi1212310qvc.552.2023.05.20.09.30.58 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:30:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=sJMTFE3Y; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQa-000412-4k; Sat, 20 May 2023 12:27:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQG-0003je-S2 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: from mail-pf1-x42d.google.com ([2607:f8b0:4864:20::42d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ8-0003AF-On for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-64d2467d640so3202291b3a.1 for ; Sat, 20 May 2023 09:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600020; x=1687192020; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ol9f0HLEAIGA8FowB6n0ZN2fTDIRFVUsiNSJUMVLzsQ=; b=sJMTFE3YKF6ehj2VDFrYCYffF1mmvmh44k/t2htIoR8bmDQHHD3bTMLLbg1v7PP2ce LV2BxW2xTX8dPzsVbScDYJ+6dVAwFlfGDLWO2pmQOxZMm64NoYc7xLqJaH7JPAeykubN hNeaxg5/JmFB04AGglbksViWp3uhyGNgL3D706zRCdUuTCjMX/eAwIT5IML+a3WuiSUN Ud0dSsP9e4evmcoxoUnpAGl8H9SRo5Ml5AdaWodKx3BUkeDSlpk6GPSJPpyWRS63ZQNT 2V1CsMkAiWSX/eO2G/snYnAax09RHGb0mxU30NgeSCAgce95K8b9nApAfagAkeKJCqsk QT8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600020; x=1687192020; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ol9f0HLEAIGA8FowB6n0ZN2fTDIRFVUsiNSJUMVLzsQ=; b=eUdTYehocqS8YigUemJb2+eNvI5ltS5i0f9zdGhM4sXVuuwSImy/G1fLc+Nal42pDg pa/z6d54qOTdK5ZkTO2d664PDSN3J6XpXxC2APPZu0UlCFzEQFE/X4Mtw7TIYtfMDOgC JyPdqSwSnjOLJHzqcItl0QAZ9CCc3cNt5IKbBxmmAtPsbhcTM+f7IUE7yJDyHKrjrqG9 VNfCx8oKYsiMiIwqnzsA6BMbr/9NFJL6VyME0nnzS/PmGBaHshwI7yvl0xO4gS229wco ezjnh4acPVRnIRpIXzRzPo3ibdBrHj1DqZebAbF+YdDkEYbiP/YInsFZ+hqL6ZqIfuws DZ9w== X-Gm-Message-State: AC+VfDw2kS0hbuDPUFBDtOGzjncqgHVK4aIN5I0ZBOSXwLaodECV4vUF skd1qr51rGrXkL+qwSBu3ZhuC+zneZ+efGdZn7A= X-Received: by 2002:a17:903:32c7:b0:1ac:43ea:7882 with SMTP id i7-20020a17090332c700b001ac43ea7882mr7700810plr.29.1684600020328; Sat, 20 May 2023 09:27:00 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.26.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:26:59 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 16/27] accel/tcg: Unify cpu_{ld,st}*_{be,le}_mmu Date: Sat, 20 May 2023 09:26:23 -0700 Message-Id: <20230520162634.3991009-17-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::42d; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x42d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With the current structure of cputlb.c, there is no difference between the little-endian and big-endian entry points, aside from the assert. Unify the pairs of functions. The only use of the functions with explicit endianness was in target/sparc64, and that was only to satisfy the assert. Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- include/exec/cpu_ldst.h | 58 ++----- accel/tcg/cputlb.c | 122 +++----------- accel/tcg/user-exec.c | 322 ++++++++++-------------------------- target/arm/tcg/m_helper.c | 4 +- target/sparc/ldst_helper.c | 18 +- accel/tcg/ldst_common.c.inc | 24 +-- 6 files changed, 137 insertions(+), 411 deletions(-) diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index 7c867c94c3..fc1d3d9301 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -207,43 +207,21 @@ void cpu_stq_le_mmuidx_ra(CPUArchState *env, abi_ptr ptr, uint64_t val, int mmu_idx, uintptr_t ra); uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr ptr, - MemOpIdx oi, uintptr_t ra); - -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra); -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra); +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr ptr, MemOpIdx oi, uintptr_t ra); +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra); void cpu_stb_mmu(CPUArchState *env, abi_ptr ptr, uint8_t val, MemOpIdx oi, uintptr_t ra); -void cpu_stw_be_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stl_be_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stq_be_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stw_le_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stl_le_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, - MemOpIdx oi, uintptr_t ra); -void cpu_stq_le_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, - MemOpIdx oi, uintptr_t ra); - -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra); -void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOpIdx oi, uintptr_t ra); +void cpu_stw_mmu(CPUArchState *env, abi_ptr ptr, uint16_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_stl_mmu(CPUArchState *env, abi_ptr ptr, uint32_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_stq_mmu(CPUArchState *env, abi_ptr ptr, uint64_t val, + MemOpIdx oi, uintptr_t ra); +void cpu_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val, + MemOpIdx oi, uintptr_t ra); uint32_t cpu_atomic_cmpxchgb_mmu(CPUArchState *env, target_ulong addr, uint32_t cmpv, uint32_t newv, @@ -416,9 +394,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_ldsw_mmuidx_ra cpu_ldsw_be_mmuidx_ra # define cpu_ldl_mmuidx_ra cpu_ldl_be_mmuidx_ra # define cpu_ldq_mmuidx_ra cpu_ldq_be_mmuidx_ra -# define cpu_ldw_mmu cpu_ldw_be_mmu -# define cpu_ldl_mmu cpu_ldl_be_mmu -# define cpu_ldq_mmu cpu_ldq_be_mmu # define cpu_stw_data cpu_stw_be_data # define cpu_stl_data cpu_stl_be_data # define cpu_stq_data cpu_stq_be_data @@ -428,9 +403,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_stw_mmuidx_ra cpu_stw_be_mmuidx_ra # define cpu_stl_mmuidx_ra cpu_stl_be_mmuidx_ra # define cpu_stq_mmuidx_ra cpu_stq_be_mmuidx_ra -# define cpu_stw_mmu cpu_stw_be_mmu -# define cpu_stl_mmu cpu_stl_be_mmu -# define cpu_stq_mmu cpu_stq_be_mmu #else # define cpu_lduw_data cpu_lduw_le_data # define cpu_ldsw_data cpu_ldsw_le_data @@ -444,9 +416,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_ldsw_mmuidx_ra cpu_ldsw_le_mmuidx_ra # define cpu_ldl_mmuidx_ra cpu_ldl_le_mmuidx_ra # define cpu_ldq_mmuidx_ra cpu_ldq_le_mmuidx_ra -# define cpu_ldw_mmu cpu_ldw_le_mmu -# define cpu_ldl_mmu cpu_ldl_le_mmu -# define cpu_ldq_mmu cpu_ldq_le_mmu # define cpu_stw_data cpu_stw_le_data # define cpu_stl_data cpu_stl_le_data # define cpu_stq_data cpu_stq_le_data @@ -456,9 +425,6 @@ static inline CPUTLBEntry *tlb_entry(CPUArchState *env, uintptr_t mmu_idx, # define cpu_stw_mmuidx_ra cpu_stw_le_mmuidx_ra # define cpu_stl_mmuidx_ra cpu_stl_le_mmuidx_ra # define cpu_stq_mmuidx_ra cpu_stq_le_mmuidx_ra -# define cpu_stw_mmu cpu_stw_le_mmu -# define cpu_stl_mmu cpu_stl_le_mmu -# define cpu_stq_mmu cpu_stq_le_mmu #endif uint8_t cpu_ldb_code_mmu(CPUArchState *env, abi_ptr addr, diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index ae0fbcdee2..b1e13d165c 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2575,89 +2575,45 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, MemOpIdx oi, uintptr_t ra) return ret; } -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint16_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint32_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { uint64_t ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); plugin_load_cb(env, addr, oi); return ret; } -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint16_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); - ret = do_ld2_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint32_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); - ret = do_ld4_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - uint64_t ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); - ret = do_ld8_mmu(env, addr, oi, ra, MMU_DATA_LOAD); - plugin_load_cb(env, addr, oi); - return ret; -} - -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { Int128 ret; - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); - ret = do_ld16_mmu(env, addr, oi, ra); - plugin_load_cb(env, addr, oi); - return ret; -} - -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - Int128 ret; - - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); ret = do_ld16_mmu(env, addr, oi, ra); plugin_load_cb(env, addr, oi); return ret; @@ -3045,66 +3001,34 @@ void cpu_stb_mmu(CPUArchState *env, target_ulong addr, uint8_t val, plugin_store_cb(env, addr, oi); } -void cpu_stw_be_mmu(CPUArchState *env, target_ulong addr, uint16_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_stw_mmu(CPUArchState *env, target_ulong addr, uint16_t val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUW); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_16); do_st2_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stl_be_mmu(CPUArchState *env, target_ulong addr, uint32_t val, +void cpu_stl_mmu(CPUArchState *env, target_ulong addr, uint32_t val, MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUL); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_32); do_st4_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stq_be_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_stq_mmu(CPUArchState *env, target_ulong addr, uint64_t val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_BEUQ); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_64); do_st8_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } -void cpu_stw_le_mmu(CPUArchState *env, target_ulong addr, uint16_t val, - MemOpIdx oi, uintptr_t retaddr) +void cpu_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, + MemOpIdx oi, uintptr_t retaddr) { - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUW); - do_st2_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_stl_le_mmu(CPUArchState *env, target_ulong addr, uint32_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUL); - do_st4_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_stq_le_mmu(CPUArchState *env, target_ulong addr, uint64_t val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP | MO_SIZE)) == MO_LEUQ); - do_st8_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_st16_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_BE|MO_128)); - do_st16_mmu(env, addr, val, oi, retaddr); - plugin_store_cb(env, addr, oi); -} - -void cpu_st16_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr) -{ - tcg_debug_assert((get_memop(oi) & (MO_BSWAP|MO_SIZE)) == (MO_LE|MO_128)); + tcg_debug_assert((get_memop(oi) & MO_SIZE) == MO_128); do_st16_mmu(env, addr, val, oi, retaddr); plugin_store_cb(env, addr, oi); } diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 36ad8284a5..19c2849c21 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -940,8 +940,8 @@ uint8_t cpu_ldb_mmu(CPUArchState *env, abi_ptr addr, return ret; } -static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint16_t do_ld2_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint16_t ret; @@ -950,59 +950,35 @@ static uint16_t do_ld2_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_2(env, ra, haddr, mop); clear_helper_retaddr(); + + if (mop & MO_BSWAP) { + ret = bswap16(ret); + } return ret; } tcg_target_ulong helper_lduw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint16_t ret = do_ld2_he_mmu(env, addr, mop, ra); - - if (mop & MO_BSWAP) { - ret = bswap16(ret); - } - return ret; + return do_ld2_mmu(env, addr, get_memop(oi), ra); } tcg_target_ulong helper_ldsw_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int16_t ret = do_ld2_he_mmu(env, addr, mop, ra); + return (int16_t)do_ld2_mmu(env, addr, get_memop(oi), ra); +} - if (mop & MO_BSWAP) { - ret = bswap16(ret); - } +uint16_t cpu_ldw_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + uint16_t ret = do_ld2_mmu(env, addr, get_memop(oi), ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return ret; } -uint16_t cpu_ldw_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint16_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld2_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be16(ret); -} - -uint16_t cpu_ldw_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint16_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld2_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le16(ret); -} - -static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint32_t do_ld4_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint32_t ret; @@ -1011,59 +987,35 @@ static uint32_t do_ld4_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_4(env, ra, haddr, mop); clear_helper_retaddr(); + + if (mop & MO_BSWAP) { + ret = bswap32(ret); + } return ret; } tcg_target_ulong helper_ldul_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint32_t ret = do_ld4_he_mmu(env, addr, mop, ra); - - if (mop & MO_BSWAP) { - ret = bswap32(ret); - } - return ret; + return do_ld4_mmu(env, addr, get_memop(oi), ra); } tcg_target_ulong helper_ldsl_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - int32_t ret = do_ld4_he_mmu(env, addr, mop, ra); + return (int32_t)do_ld4_mmu(env, addr, get_memop(oi), ra); +} - if (mop & MO_BSWAP) { - ret = bswap32(ret); - } +uint32_t cpu_ldl_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) +{ + uint32_t ret = do_ld4_mmu(env, addr, get_memop(oi), ra); + qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); return ret; } -uint32_t cpu_ldl_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint32_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld4_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be32(ret); -} - -uint32_t cpu_ldl_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint32_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld4_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le32(ret); -} - -static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static uint64_t do_ld8_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; uint64_t ret; @@ -1072,14 +1024,6 @@ static uint64_t do_ld8_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_8(env, ra, haddr, mop); clear_helper_retaddr(); - return ret; -} - -uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - uint64_t ret = do_ld8_he_mmu(env, addr, mop, ra); if (mop & MO_BSWAP) { ret = bswap64(ret); @@ -1087,32 +1031,22 @@ uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, return ret; } -uint64_t cpu_ldq_be_mmu(CPUArchState *env, abi_ptr addr, +uint64_t helper_ldq_mmu(CPUArchState *env, uint64_t addr, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint64_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld8_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_be64(ret); + return do_ld8_mmu(env, addr, get_memop(oi), ra); } -uint64_t cpu_ldq_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +uint64_t cpu_ldq_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - uint64_t ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld8_he_mmu(env, addr, mop, ra); + uint64_t ret = do_ld8_mmu(env, addr, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - return cpu_to_le64(ret); + return ret; } -static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, - MemOp mop, uintptr_t ra) +static Int128 do_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOp mop, uintptr_t ra) { void *haddr; Int128 ret; @@ -1121,14 +1055,6 @@ static Int128 do_ld16_he_mmu(CPUArchState *env, abi_ptr addr, haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_LOAD); ret = load_atom_16(env, ra, haddr, mop); clear_helper_retaddr(); - return ret; -} - -Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - Int128 ret = do_ld16_he_mmu(env, addr, mop, ra); if (mop & MO_BSWAP) { ret = bswap128(ret); @@ -1136,38 +1062,22 @@ Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, return ret; } +Int128 helper_ld16_mmu(CPUArchState *env, uint64_t addr, + MemOpIdx oi, uintptr_t ra) +{ + return do_ld16_mmu(env, addr, get_memop(oi), ra); +} + Int128 helper_ld_i128(CPUArchState *env, uint64_t addr, MemOpIdx oi) { return helper_ld16_mmu(env, addr, oi, GETPC()); } -Int128 cpu_ld16_be_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) +Int128 cpu_ld16_mmu(CPUArchState *env, abi_ptr addr, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - Int128 ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - ret = do_ld16_he_mmu(env, addr, mop, ra); + Int128 ret = do_ld16_mmu(env, addr, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - if (!HOST_BIG_ENDIAN) { - ret = bswap128(ret); - } - return ret; -} - -Int128 cpu_ld16_le_mmu(CPUArchState *env, abi_ptr addr, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - Int128 ret; - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - ret = do_ld16_he_mmu(env, addr, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); - if (HOST_BIG_ENDIAN) { - ret = bswap128(ret); - } return ret; } @@ -1195,13 +1105,17 @@ void cpu_stb_mmu(CPUArchState *env, abi_ptr addr, uint8_t val, qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, - MemOp mop, uintptr_t ra) +static void do_st2_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_16); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap16(val); + } store_atom_2(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1209,41 +1123,27 @@ static void do_st2_he_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, void helper_stw_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap16(val); - } - do_st2_he_mmu(env, addr, val, mop, ra); + do_st2_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stw_be_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, +void cpu_stw_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st2_he_mmu(env, addr, be16_to_cpu(val), mop, ra); + do_st2_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stw_le_mmu(CPUArchState *env, abi_ptr addr, uint16_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st2_he_mmu(env, addr, le16_to_cpu(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOp mop, uintptr_t ra) +static void do_st4_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_32); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap32(val); + } store_atom_4(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1251,41 +1151,27 @@ static void do_st4_he_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, void helper_stl_mmu(CPUArchState *env, uint64_t addr, uint32_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap32(val); - } - do_st4_he_mmu(env, addr, val, mop, ra); + do_st4_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stl_be_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) +void cpu_stl_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, + MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st4_he_mmu(env, addr, be32_to_cpu(val), mop, ra); + do_st4_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stl_le_mmu(CPUArchState *env, abi_ptr addr, uint32_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st4_he_mmu(env, addr, le32_to_cpu(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, - MemOp mop, uintptr_t ra) +static void do_st8_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_64); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap64(val); + } store_atom_8(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1293,41 +1179,27 @@ static void do_st8_he_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, void helper_stq_mmu(CPUArchState *env, uint64_t addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap64(val); - } - do_st8_he_mmu(env, addr, val, mop, ra); + do_st8_mmu(env, addr, val, get_memop(oi), ra); } -void cpu_stq_be_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, +void cpu_stq_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - do_st8_he_mmu(env, addr, cpu_to_be64(val), mop, ra); + do_st8_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } -void cpu_stq_le_mmu(CPUArchState *env, abi_ptr addr, uint64_t val, - MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - do_st8_he_mmu(env, addr, cpu_to_le64(val), mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, - MemOp mop, uintptr_t ra) +static void do_st16_mmu(CPUArchState *env, abi_ptr addr, Int128 val, + MemOp mop, uintptr_t ra) { void *haddr; tcg_debug_assert((mop & MO_SIZE) == MO_128); haddr = cpu_mmu_lookup(env, addr, mop, ra, MMU_DATA_STORE); + + if (mop & MO_BSWAP) { + val = bswap128(val); + } store_atom_16(env, ra, haddr, mop, val); clear_helper_retaddr(); } @@ -1335,12 +1207,7 @@ static void do_st16_he_mmu(CPUArchState *env, abi_ptr addr, Int128 val, void helper_st16_mmu(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - if (mop & MO_BSWAP) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); + do_st16_mmu(env, addr, val, get_memop(oi), ra); } void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) @@ -1348,29 +1215,10 @@ void helper_st_i128(CPUArchState *env, uint64_t addr, Int128 val, MemOpIdx oi) helper_st16_mmu(env, addr, val, oi, GETPC()); } -void cpu_st16_be_mmu(CPUArchState *env, abi_ptr addr, - Int128 val, MemOpIdx oi, uintptr_t ra) +void cpu_st16_mmu(CPUArchState *env, abi_ptr addr, + Int128 val, MemOpIdx oi, uintptr_t ra) { - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_BE); - if (!HOST_BIG_ENDIAN) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} - -void cpu_st16_le_mmu(CPUArchState *env, abi_ptr addr, - Int128 val, MemOpIdx oi, uintptr_t ra) -{ - MemOp mop = get_memop(oi); - - tcg_debug_assert((mop & MO_BSWAP) == MO_LE); - if (HOST_BIG_ENDIAN) { - val = bswap128(val); - } - do_st16_he_mmu(env, addr, val, mop, ra); + do_st16_mmu(env, addr, val, get_memop(oi), ra); qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); } diff --git a/target/arm/tcg/m_helper.c b/target/arm/tcg/m_helper.c index 9758f225d6..9cef70e5c9 100644 --- a/target/arm/tcg/m_helper.c +++ b/target/arm/tcg/m_helper.c @@ -1937,8 +1937,8 @@ static bool do_v7m_function_return(ARMCPU *cpu) */ mmu_idx = arm_v7m_mmu_idx_for_secstate(env, true); oi = make_memop_idx(MO_LEUL, arm_to_core_mmu_idx(mmu_idx)); - newpc = cpu_ldl_le_mmu(env, frameptr, oi, 0); - newpsr = cpu_ldl_le_mmu(env, frameptr + 4, oi, 0); + newpc = cpu_ldl_mmu(env, frameptr, oi, 0); + newpsr = cpu_ldl_mmu(env, frameptr + 4, oi, 0); /* Consistency checks on new IPSR */ newpsr_exc = newpsr & XPSR_EXCP; diff --git a/target/sparc/ldst_helper.c b/target/sparc/ldst_helper.c index 7972d56a72..981a47d8bb 100644 --- a/target/sparc/ldst_helper.c +++ b/target/sparc/ldst_helper.c @@ -1334,25 +1334,13 @@ uint64_t helper_ld_asi(CPUSPARCState *env, target_ulong addr, ret = cpu_ldb_mmu(env, addr, oi, GETPC()); break; case 2: - if (asi & 8) { - ret = cpu_ldw_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldw_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldw_mmu(env, addr, oi, GETPC()); break; case 4: - if (asi & 8) { - ret = cpu_ldl_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldl_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldl_mmu(env, addr, oi, GETPC()); break; case 8: - if (asi & 8) { - ret = cpu_ldq_le_mmu(env, addr, oi, GETPC()); - } else { - ret = cpu_ldq_be_mmu(env, addr, oi, GETPC()); - } + ret = cpu_ldq_mmu(env, addr, oi, GETPC()); break; default: g_assert_not_reached(); diff --git a/accel/tcg/ldst_common.c.inc b/accel/tcg/ldst_common.c.inc index 6ac8d871a3..5f8144b33a 100644 --- a/accel/tcg/ldst_common.c.inc +++ b/accel/tcg/ldst_common.c.inc @@ -26,7 +26,7 @@ uint32_t cpu_lduw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx); - return cpu_ldw_be_mmu(env, addr, oi, ra); + return cpu_ldw_mmu(env, addr, oi, ra); } int cpu_ldsw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, @@ -39,21 +39,21 @@ uint32_t cpu_ldl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx); - return cpu_ldl_be_mmu(env, addr, oi, ra); + return cpu_ldl_mmu(env, addr, oi, ra); } uint64_t cpu_ldq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx); - return cpu_ldq_be_mmu(env, addr, oi, ra); + return cpu_ldq_mmu(env, addr, oi, ra); } uint32_t cpu_lduw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx); - return cpu_ldw_le_mmu(env, addr, oi, ra); + return cpu_ldw_mmu(env, addr, oi, ra); } int cpu_ldsw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, @@ -66,14 +66,14 @@ uint32_t cpu_ldl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx); - return cpu_ldl_le_mmu(env, addr, oi, ra); + return cpu_ldl_mmu(env, addr, oi, ra); } uint64_t cpu_ldq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx); - return cpu_ldq_le_mmu(env, addr, oi, ra); + return cpu_ldq_mmu(env, addr, oi, ra); } void cpu_stb_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, @@ -87,42 +87,42 @@ void cpu_stw_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUW | MO_UNALN, mmu_idx); - cpu_stw_be_mmu(env, addr, val, oi, ra); + cpu_stw_mmu(env, addr, val, oi, ra); } void cpu_stl_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUL | MO_UNALN, mmu_idx); - cpu_stl_be_mmu(env, addr, val, oi, ra); + cpu_stl_mmu(env, addr, val, oi, ra); } void cpu_stq_be_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_BEUQ | MO_UNALN, mmu_idx); - cpu_stq_be_mmu(env, addr, val, oi, ra); + cpu_stq_mmu(env, addr, val, oi, ra); } void cpu_stw_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUW | MO_UNALN, mmu_idx); - cpu_stw_le_mmu(env, addr, val, oi, ra); + cpu_stw_mmu(env, addr, val, oi, ra); } void cpu_stl_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint32_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUL | MO_UNALN, mmu_idx); - cpu_stl_le_mmu(env, addr, val, oi, ra); + cpu_stl_mmu(env, addr, val, oi, ra); } void cpu_stq_le_mmuidx_ra(CPUArchState *env, abi_ptr addr, uint64_t val, int mmu_idx, uintptr_t ra) { MemOpIdx oi = make_memop_idx(MO_LEUQ | MO_UNALN, mmu_idx); - cpu_stq_le_mmu(env, addr, val, oi, ra); + cpu_stq_mmu(env, addr, val, oi, ra); } /*--------------------------*/ From patchwork Sat May 20 16:26:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684307 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448141wrt; Sat, 20 May 2023 09:29:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7XO5vjXTDFcvcWpRbYXkR//dp1mHcCAYkj2uBTif4xPIymOohjMlIA2vLmmccA4OuQY8MU X-Received: by 2002:ac8:7f8f:0:b0:3f5:16af:17db with SMTP id z15-20020ac87f8f000000b003f516af17dbmr10463776qtj.0.1684600174117; Sat, 20 May 2023 09:29:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600174; cv=none; d=google.com; s=arc-20160816; b=01D07HJ8ZzRxK+bhC9AukBHsIMXptB+JqHkzgeOeBTwYJnd1OClAZXpdnv1upUFpY7 g8uq45iqIYz+pkUKenehmrAAlEBnSvLvpgWqDL0HgZ6sx7puKb3UCS+hQ1KSYysNFkNA XKEdTri2M0JMLUoe//plPqgjdpojZAd3K40p5x2+/rblIYe7+Xm21R0IdZsNj7A0nMbj fNSIz7Yx41Pluj/3oQWKy2KFoLh2e6iX3ZGb+IcDGWWLG+EWFoD+KC+56vCBi1s2NTIH U0n1uqYwS0U36DFfU1rP7ZrSv9oEfcfT4Ph6Hxm1+6Mac7vZ8QzFgulSF9bbgU49G0TE V8IA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=vQccedObj4utMlf35wyWC0cbgMCPr9JJZ7oY1hv6HgQ=; b=p8KG7Wn7s5ewj26lnLiWlS/eyV6HhovwfKsVlDl0ZD6+3WbAQEHFdWrKMMxEXnrNur 4IafIoOqBGsk+2jihZNSjW7NkceIaXrPrwKozzHYAGd02RhcbDq7arHKr3wK9k4Dz4jK zNo8Vpg8AClqQCespUtrN0zJ7oawE2vOlpF51pDY6LXNKLxuxYLdxnCUBmJa0cFBKR0p 2wOHeJTwepCTTkwqUgKQPdaiAfjHw2+Sd1g86itEQOivcLX2EHUMX3CTSKHgV7zUbmsH 6zflXGHpdWFf86a8vvoAH8txv4p7czEf/jgduJjOgWYJqsiOpgo6wK9DAdv3/6EE8+Fp HXVQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XKI5yfv+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f10-20020ac87f0a000000b003f4232ea13fsi1301359qtk.614.2023.05.20.09.29.33 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=XKI5yfv+; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQI-0003lI-At; Sat, 20 May 2023 12:27:14 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQC-0003hj-TU for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:10 -0400 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ8-000349-O2 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:08 -0400 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1ae507af2e5so14197815ad.1 for ; Sat, 20 May 2023 09:27:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600021; x=1687192021; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vQccedObj4utMlf35wyWC0cbgMCPr9JJZ7oY1hv6HgQ=; b=XKI5yfv+SgDnQucNQwDJ6OSB1LfdiwE6VehPGMn8vcaoaKm3jqBPQmG/nT+Jj7ordb hXRlbDAlb3TMNyAYGpd118W0AALLzrXNBzJw+UCZ0LeSHLU0wdsLJT5wZXkGr2e3inqF YbJ+J3yAEapIc76wAvwxmxxiwVNRiI3jo9tCTfTaxamp3AuU9mYY/X3pHZdnRbdXYLV2 TNPVHvdtB2i5t9vpyxp543xRLqc+Lfr9Vo2SmiGklEluHDQ9dTOBDGVK6rQvKIy1ePmq sos+IZ9F7znLsq0Ss53PAHEXm6L6wlkBTzydKFCT+EdZj9853fLPTuOYS7cvrinpp4x4 8oag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600021; x=1687192021; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vQccedObj4utMlf35wyWC0cbgMCPr9JJZ7oY1hv6HgQ=; b=dYevnj9//HzhQO0HHmDE7vR38KTFE8b8qGvKnV89TrAwhscUl5SD9vw+hhSgdv5z2i zDRJv+c3wQJNbBGHKFVU1KstzaM0EdWbary70DCLgiur4vPnOoPF6REfzAlcC1mIKXfF 8RY5xNo+zRkgu1pzAxhAcgSE/a4uukxyGaYPlevuI+H4JJlxSNZe1HvA9i1CGAJGn7kh rILy2WgQbNfnBKERcOzGwUXaNy3/4OxtKw2MmZq621F04hw6Aw5VyDENY7aMPjXtGOR2 NKtcVC0jI43wOghdL3P973WWhQkyL1uohKwd8S2eCjjzmyCOr16hs5yME/chEJFGVcTw Tr3w== X-Gm-Message-State: AC+VfDzVzenSIz2Mx4Ccu+kCKiJqe2b2UMBaKF3vp9dlPAul4FGSoAxC SAOA+Rp1Q+K1qhXmLsCA2POzrrX16WM4H+kH6JE= X-Received: by 2002:a17:903:32cd:b0:1a9:581b:fbb1 with SMTP id i13-20020a17090332cd00b001a9581bfbb1mr7576495plr.32.1684600021189; Sat, 20 May 2023 09:27:01 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:00 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org, David Hildenbrand , Ilya Leoshkevich Subject: [PATCH 17/27] target/s390x: Use cpu_{ld,st}*_mmu in do_csst Date: Sat, 20 May 2023 09:26:24 -0700 Message-Id: <20230520162634.3991009-18-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62a; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62a.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use cpu_ld16_mmu and cpu_st16_mmu to eliminate the special case, and change all of the *_data_ra functions to match. Signed-off-by: Richard Henderson --- Cc: qemu-s390x@nongnu.org Cc: David Hildenbrand Cc: Ilya Leoshkevich --- target/s390x/tcg/mem_helper.c | 65 ++++++++++++++--------------------- 1 file changed, 26 insertions(+), 39 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index 0e0d66b3b6..b6cf24403c 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1737,6 +1737,9 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, uint64_t a2, bool parallel) { uint32_t mem_idx = cpu_mmu_index(env, false); + MemOpIdx oi16 = make_memop_idx(MO_TE | MO_128, mem_idx); + MemOpIdx oi8 = make_memop_idx(MO_TE | MO_64, mem_idx); + MemOpIdx oi4 = make_memop_idx(MO_TE | MO_32, mem_idx); uintptr_t ra = GETPC(); uint32_t fc = extract32(env->regs[0], 0, 8); uint32_t sc = extract32(env->regs[0], 8, 8); @@ -1780,15 +1783,17 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, } } - /* All loads happen before all stores. For simplicity, load the entire - store value area from the parameter list. */ - svh = cpu_ldq_data_ra(env, pl + 16, ra); - svl = cpu_ldq_data_ra(env, pl + 24, ra); + /* + * All loads happen before all stores. For simplicity, load the entire + * store value area from the parameter list. + */ + svh = cpu_ldq_mmu(env, pl + 16, oi8, ra); + svl = cpu_ldq_mmu(env, pl + 24, oi8, ra); switch (fc) { case 0: { - uint32_t nv = cpu_ldl_data_ra(env, pl, ra); + uint32_t nv = cpu_ldl_mmu(env, pl, oi4, ra); uint32_t cv = env->regs[r3]; uint32_t ov; @@ -1801,8 +1806,8 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra); #endif } else { - ov = cpu_ldl_data_ra(env, a1, ra); - cpu_stl_data_ra(env, a1, (ov == cv ? nv : ov), ra); + ov = cpu_ldl_mmu(env, a1, oi4, ra); + cpu_stl_mmu(env, a1, (ov == cv ? nv : ov), oi4, ra); } cc = (ov != cv); env->regs[r3] = deposit64(env->regs[r3], 32, 32, ov); @@ -1811,21 +1816,20 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, case 1: { - uint64_t nv = cpu_ldq_data_ra(env, pl, ra); + uint64_t nv = cpu_ldq_mmu(env, pl, oi8, ra); uint64_t cv = env->regs[r3]; uint64_t ov; if (parallel) { #ifdef CONFIG_ATOMIC64 - MemOpIdx oi = make_memop_idx(MO_TEUQ | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi, ra); + ov = cpu_atomic_cmpxchgq_be_mmu(env, a1, cv, nv, oi8, ra); #else /* Note that we asserted !parallel above. */ g_assert_not_reached(); #endif } else { - ov = cpu_ldq_data_ra(env, a1, ra); - cpu_stq_data_ra(env, a1, (ov == cv ? nv : ov), ra); + ov = cpu_ldq_mmu(env, a1, oi8, ra); + cpu_stq_mmu(env, a1, (ov == cv ? nv : ov), oi8, ra); } cc = (ov != cv); env->regs[r3] = ov; @@ -1834,27 +1838,19 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, case 2: { - uint64_t nvh = cpu_ldq_data_ra(env, pl, ra); - uint64_t nvl = cpu_ldq_data_ra(env, pl + 8, ra); - Int128 nv = int128_make128(nvl, nvh); + Int128 nv = cpu_ld16_mmu(env, pl, oi16, ra); Int128 cv = int128_make128(env->regs[r3 + 1], env->regs[r3]); Int128 ov; if (!parallel) { - uint64_t oh = cpu_ldq_data_ra(env, a1 + 0, ra); - uint64_t ol = cpu_ldq_data_ra(env, a1 + 8, ra); - - ov = int128_make128(ol, oh); + ov = cpu_ld16_mmu(env, a1, oi16, ra); cc = !int128_eq(ov, cv); if (cc) { nv = ov; } - - cpu_stq_data_ra(env, a1 + 0, int128_gethi(nv), ra); - cpu_stq_data_ra(env, a1 + 8, int128_getlo(nv), ra); + cpu_st16_mmu(env, a1, nv, oi16, ra); } else if (HAVE_CMPXCHG128) { - MemOpIdx oi = make_memop_idx(MO_TE | MO_128 | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi, ra); + ov = cpu_atomic_cmpxchgo_be_mmu(env, a1, cv, nv, oi16, ra); cc = !int128_eq(ov, cv); } else { /* Note that we asserted !parallel above. */ @@ -1876,29 +1872,20 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, if (cc == 0) { switch (sc) { case 0: - cpu_stb_data_ra(env, a2, svh >> 56, ra); + cpu_stb_mmu(env, a2, svh >> 56, make_memop_idx(MO_8, mem_idx), ra); break; case 1: - cpu_stw_data_ra(env, a2, svh >> 48, ra); + cpu_stw_mmu(env, a2, svh >> 48, + make_memop_idx(MO_TE | MO_16, mem_idx), ra); break; case 2: - cpu_stl_data_ra(env, a2, svh >> 32, ra); + cpu_stl_mmu(env, a2, svh >> 32, oi4, ra); break; case 3: - cpu_stq_data_ra(env, a2, svh, ra); + cpu_stq_mmu(env, a2, svh, oi8, ra); break; case 4: - if (!parallel) { - cpu_stq_data_ra(env, a2 + 0, svh, ra); - cpu_stq_data_ra(env, a2 + 8, svl, ra); - } else if (HAVE_ATOMIC128) { - MemOpIdx oi = make_memop_idx(MO_TEUQ | MO_ALIGN_16, mem_idx); - Int128 sv = int128_make128(svl, svh); - cpu_atomic_sto_be_mmu(env, a2, sv, oi, ra); - } else { - /* Note that we asserted !parallel above. */ - g_assert_not_reached(); - } + cpu_st16_mmu(env, a2, int128_make128(svl, svh), oi16, ra); break; default: g_assert_not_reached(); From patchwork Sat May 20 16:26:25 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684304 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448076wrt; Sat, 20 May 2023 09:29:24 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ79IP5F0dXg4MlQ7WF8GS0dVzRXTEITDC+6IWIKggJsaGuCTCNrDpaWzMEHoY5NJsJQThmr X-Received: by 2002:ac8:5acd:0:b0:3f3:9013:5839 with SMTP id d13-20020ac85acd000000b003f390135839mr10738901qtd.23.1684600163986; Sat, 20 May 2023 09:29:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600163; cv=none; d=google.com; s=arc-20160816; b=e3v6QhmEXkT26dyEuJXWZ+K2vcjh8nY9fdpsPMoEt6uxN0fYp147QAOAJlvypHNyrW +ysrh3hKJ3XjHmbUhLnDHlI63OPwvJG73oQ+IrrQTxDGhXf+uHYi3F+74kSkWakIIqb8 bXmRXoIcN3Morju5hiuKxmjl4AXYdEmdVmrrKUR6FJPNV5+q5L9Xn7eQDutfmKrPzskI bNAco0hvq/Lm3hwuGzTtvS9INKiiSpRPAywXvI2D+TwkNTbhjWz+hw+oczY5T3dFIl4D jeYNisDeQmXpBhCj8BJLHqO8IOmYCQ4sPpCJOhYtpOdzdl4PyXQehVqAYPuj1mkXoNvj fk5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=qaZpDDjPwlY2m7x6v9ar9hpPELeL8Q6hzaJ5d16jETI=; b=PG3wWx7/dHUnrbhpUXEozvHFpvRW3YXwOUfKNAUDhxqAp/lV+z02ABIPFQufmoQid6 Q43ZErKljGzazS9HjyCm0S9JEmHKuz7usiR69twfDZ3bfUX56HmMRF/+Hf2pXKPxlcdd O8ysTgXFdos2v/0T+R9W6RugaWtb1ZZJVSoqMx7/YF8pulDa6hnUTWPEiV4B432w2pKs TuK865pNF4qIN1UWwARxUHoo7KwgFf/l7Uow3ghvJ7uvJRU8krZgOBo6EtH2txe3uVhA AYVRTYQEW9zsFZseZeWAmJTfNfautC4Kdx1IwfJ74o2CNJKws6kGFrcjHsXnak53gDa4 uQjQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="U2/mcl5/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id b1-20020ac87fc1000000b003f0abf74b20si1348834qtk.443.2023.05.20.09.29.23 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:23 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="U2/mcl5/"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQK-0003ot-8b; Sat, 20 May 2023 12:27:16 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQE-0003i4-U0 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ8-0003Aq-Q6 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:10 -0400 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1ae851f2a7dso11662285ad.0 for ; Sat, 20 May 2023 09:27:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600022; x=1687192022; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qaZpDDjPwlY2m7x6v9ar9hpPELeL8Q6hzaJ5d16jETI=; b=U2/mcl5/P+7RETzri4waiMyUilkBv7sfeUJDAQF/mdwr6jDCvTX1jwvGvtYEg+KctE DYUMOlvF0E9a+ifbbgo5CfvJvkRFDGhnnTE5lQZq0IDWDgSMhLrNyOI/FA/kcyDsW016 f9YDA63+/UuJHRxMvYvUF2URHuEGGXQmSxJi//cvfrhDVKYbFNh6yAEWXj6IyQT5BuSH McEQnIWDL14nNfVPpoBA/JJKhrNcdMvNy8Gx53akC7aIhFFNJ13BlEwyTAOkWsPyLRMZ 9ftWFpO2D227iLt56McjV/fdqbDUtjUCGYKwM8LEkMjsq+HiG4CANQSzFcwAB80yehlZ uG7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600022; x=1687192022; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qaZpDDjPwlY2m7x6v9ar9hpPELeL8Q6hzaJ5d16jETI=; b=baX87BB2XMo8xZ4TO6wvVWpyGjuZ/xXOPJQB8ot98ya2iIHbTOsvToYL/o9a7hqfwf JTtbd03fZXMZ+Q8b/G2u0eeQ+my9/4RJZE9eYV9UAv08cpwe2XM++90mViqpdFhkfusi sMV4YZ43qaxCTnKcELJPI4qu6t0fz9L+jzkC2IoGs4IB7xqYNovFn7RBUgeVrXBtzKRD 0Ocw020ARF6hpU7ovsloJN/8j/xxZpVRn2jVjgVGAXijpCsdaMmoqWkfIXo7Qy/z6FXV WlhRCJelUbQzuPbWYWQGHV48zMT6sR8ClGNbweujILgtt42EniRbnG8BK2iOzg5uGLAn OlDA== X-Gm-Message-State: AC+VfDwzufV6RJydVbsWNMKWceW6aIWe0UA5bibMRWRelKs1e9PtQYsI C4VHNCy3VfB7GUVpnRqtkLq4BMttNqXcHXESZz0= X-Received: by 2002:a17:902:e74e:b0:1a9:a408:a52f with SMTP id p14-20020a170902e74e00b001a9a408a52fmr10584163plf.24.1684600021975; Sat, 20 May 2023 09:27:01 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:01 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org, David Hildenbrand , Ilya Leoshkevich Subject: [PATCH 18/27] target/s390x: Always use cpu_atomic_cmpxchgl_be_mmu in do_csst Date: Sat, 20 May 2023 09:26:25 -0700 Message-Id: <20230520162634.3991009-19-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::635; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x635.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Eliminate the CONFIG_USER_ONLY specialization. Signed-off-by: Richard Henderson Reviewed-by: David Hildenbrand --- Cc: qemu-s390x@nongnu.org Cc: David Hildenbrand Cc: Ilya Leoshkevich --- target/s390x/tcg/mem_helper.c | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index b6cf24403c..bad789a742 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1798,13 +1798,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, uint32_t ov; if (parallel) { -#ifdef CONFIG_USER_ONLY - uint32_t *haddr = g2h(env_cpu(env), a1); - ov = qatomic_cmpxchg__nocheck(haddr, cv, nv); -#else - MemOpIdx oi = make_memop_idx(MO_TEUL | MO_ALIGN, mem_idx); - ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi, ra); -#endif + ov = cpu_atomic_cmpxchgl_be_mmu(env, a1, cv, nv, oi4, ra); } else { ov = cpu_ldl_mmu(env, a1, oi4, ra); cpu_stl_mmu(env, a1, (ov == cv ? nv : ov), oi4, ra); From patchwork Sat May 20 16:26:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684299 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447838wrt; Sat, 20 May 2023 09:28:34 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4vqbCk0dWxnCTV2vRDa+2EIRRuD3rZ6S9Zp6Ttg09u2gDfC7UgcemTHl/e5LBzzOZqVbjW X-Received: by 2002:ad4:4eef:0:b0:624:fa24:3d50 with SMTP id dv15-20020ad44eef000000b00624fa243d50mr2165919qvb.30.1684600114228; Sat, 20 May 2023 09:28:34 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600114; cv=none; d=google.com; s=arc-20160816; b=1KgmR/maGyCW1fYCrXmi7YmeiFDEYAYPDZzoVN41rkTlNGGKliPFzIBdVHs15wsbvv rENYjbPO9ia6xc3FSgZVgN1L3dKCRU22JXzry0QtDk83Xn5un987UpmxsJqXJqd05Ie0 cThDGFoncdFTnS5Zc2RMZW4eK8+DQZos8kFw2RnuzmjW6YCprtFclBLVwjFf7a4486+Q dbM2sQ+b6RNrvU9zs29rWq5TFToNQfWgWu9h/Mi3RpIznDTvNERj7iMeBYOFwMhOz8YF 6Mrka1HFsnlEOlm6nn4ualIULe5OtCQE+Lg5ICT4XeuFIGbux4IFBb6sOwiRPHafhwPJ T8tw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=E5TTA0bq/ugY+eZz7k6us4XdKRdL7V+m6msEaP3WMZetBFHCSc7A+ZymFxIsVJT2KC ZzeUrlSzvRB1KH3ye6Z3VHKztUJx58RTn5TNO50yUnJKa096UQOfcYp77Z1rG1o03aKx xTokVX7bjezuzcHNwWidSUi+gl2wKz/yG4pCeOvBtappOBaU3+rvuX16rsWF3OjejZLX W3PH8BcygrAsct9jQTnDxDJ/U6MUxrTiTvn9y6x4x5plttNUrUQG2uad53D1JjHE1UbR Vqay1fIMszKaf4WwFV+fhqf+4v21X633PcBQsm2Tq3XX0Jjg7YoQ6vlb77flU2fZbJsG Talw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NiebfK18; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id f15-20020a0562141d2f00b006192eaa74cdsi1219820qvd.470.2023.05.20.09.28.34 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:28:34 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=NiebfK18; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQS-0003xV-OH; Sat, 20 May 2023 12:27:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQE-0003i2-Ra for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:11 -0400 Received: from mail-pl1-x634.google.com ([2607:f8b0:4864:20::634]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ8-00034U-Os for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:09 -0400 Received: by mail-pl1-x634.google.com with SMTP id d9443c01a7336-1ae4c5e1388so43577015ad.1 for ; Sat, 20 May 2023 09:27:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600023; x=1687192023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=NiebfK18huzPkhni+eWrEvgLUoAeCbNUHXgL1m8ZmVsaZ/tXbk3W1BYq4xbNX5Hara mvCjhPbiucajpzL7c25ARj1GHEY3t25etV0sKwlp+T7iklwI5T3QCq3+bXyQR15bn8eu qXRBAZyvo1BglRPL2/xBJNkT2wvh5q6WnDJMDb36VOVUjyghU0BJF/+wHnR2DiBtK091 druZTr95/vDtHcq5qXzxzfxcmCNM5jpje+kyYHjYZK+WgHxKUcraOOZHM60xcoAyCXmt itrU93lE7vqPSUaB0BqfRj0Xzj7CR8RO5EQr7pvW8WeM+cDFWRgMuV7X+vu/Y+cMxlBh +uUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600023; x=1687192023; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MPL5Chc7uKNtPykCbIf1bx9tBI7z2B/RniQ4iaiA6MY=; b=UAdhXAweMW0UvXuSjc0VrmeHD+l2QgKELHpu5JSiuzJyuptHP1Il5UPOJ5ygOT+QbO l6BAKb/0WVBo5lU8nv2LGL3q/ExwRMw0CTgG57sqCd+ftagi/2rnWMsGoxeYhUVN895Y SF42EG3R/awhB+eb4OZr86eYuqjc3hIAW4oWap9wWYIZmieq1Mw9iRgNI5y4GRACaowR VmtTpwMVBfWXgzMxyn9YeRpGJErA+Gt+eEJMgW+54DCPDOCk2j8W+6jLezq9mkBJ1pUl inUVSX9MFctmjitPSCTxC/DQGaA7iE+oMla70MGsYaCGtvMT4MquMWO98ccrfqm78v3K 7v1w== X-Gm-Message-State: AC+VfDwNdJnPWo6U60yeDFyCPayw5H3LxK3Da+saJQxDbNhvxyFDEI5n fB3HBSedba6jvrtHrCoccSFPZ5O/1LFeETTZXrc= X-Received: by 2002:a17:902:e811:b0:1af:981b:eeff with SMTP id u17-20020a170902e81100b001af981beeffmr2243569plg.64.1684600022784; Sat, 20 May 2023 09:27:02 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:02 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 19/27] accel/tcg: Remove cpu_atomic_{ld,st}o_*_mmu Date: Sat, 20 May 2023 09:26:26 -0700 Message-Id: <20230520162634.3991009-20-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::634; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x634.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Atomic load/store of 128-byte quantities is now handled by cpu_{ld,st}16_mmu. Signed-off-by: Richard Henderson --- accel/tcg/atomic_template.h | 61 +++-------------------------------- include/exec/cpu_ldst.h | 9 ------ accel/tcg/atomic_common.c.inc | 14 -------- 3 files changed, 4 insertions(+), 80 deletions(-) diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index 404a530f7c..30eee9d066 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -87,33 +87,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, return ret; } -#if DATA_SIZE >= 16 -#if HAVE_ATOMIC128 -ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ, retaddr); - DATA_TYPE val; - - val = atomic16_read(haddr); - ATOMIC_MMU_CLEANUP; - atomic_trace_ld_post(env, addr, oi); - return val; -} - -void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_WRITE, retaddr); - - atomic16_set(haddr, val); - ATOMIC_MMU_CLEANUP; - atomic_trace_st_post(env, addr, oi); -} -#endif -#else +#if DATA_SIZE < 16 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { @@ -188,7 +162,7 @@ GEN_ATOMIC_HELPER_FN(smax_fetch, MAX, SDATA_TYPE, new) GEN_ATOMIC_HELPER_FN(umax_fetch, MAX, DATA_TYPE, new) #undef GEN_ATOMIC_HELPER_FN -#endif /* DATA SIZE >= 16 */ +#endif /* DATA SIZE < 16 */ #undef END @@ -220,34 +194,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, return BSWAP(ret); } -#if DATA_SIZE >= 16 -#if HAVE_ATOMIC128 -ABI_TYPE ATOMIC_NAME(ld)(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ, retaddr); - DATA_TYPE val; - - val = atomic16_read(haddr); - ATOMIC_MMU_CLEANUP; - atomic_trace_ld_post(env, addr, oi); - return BSWAP(val); -} - -void ATOMIC_NAME(st)(CPUArchState *env, target_ulong addr, ABI_TYPE val, - MemOpIdx oi, uintptr_t retaddr) -{ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_WRITE, retaddr); - - val = BSWAP(val); - atomic16_set(haddr, val); - ATOMIC_MMU_CLEANUP; - atomic_trace_st_post(env, addr, oi); -} -#endif -#else +#if DATA_SIZE < 16 ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { @@ -326,7 +273,7 @@ GEN_ATOMIC_HELPER_FN(add_fetch, ADD, DATA_TYPE, new) #undef ADD #undef GEN_ATOMIC_HELPER_FN -#endif /* DATA_SIZE >= 16 */ +#endif /* DATA_SIZE < 16 */ #undef END #endif /* DATA_SIZE > 1 */ diff --git a/include/exec/cpu_ldst.h b/include/exec/cpu_ldst.h index fc1d3d9301..5939688f69 100644 --- a/include/exec/cpu_ldst.h +++ b/include/exec/cpu_ldst.h @@ -300,15 +300,6 @@ Int128 cpu_atomic_cmpxchgo_be_mmu(CPUArchState *env, target_ulong addr, Int128 cmpv, Int128 newv, MemOpIdx oi, uintptr_t retaddr); -Int128 cpu_atomic_ldo_le_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -Int128 cpu_atomic_ldo_be_mmu(CPUArchState *env, target_ulong addr, - MemOpIdx oi, uintptr_t retaddr); -void cpu_atomic_sto_le_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr); -void cpu_atomic_sto_be_mmu(CPUArchState *env, target_ulong addr, Int128 val, - MemOpIdx oi, uintptr_t retaddr); - #if defined(CONFIG_USER_ONLY) extern __thread uintptr_t helper_retaddr; diff --git a/accel/tcg/atomic_common.c.inc b/accel/tcg/atomic_common.c.inc index fe0eea018f..f255c9e215 100644 --- a/accel/tcg/atomic_common.c.inc +++ b/accel/tcg/atomic_common.c.inc @@ -19,20 +19,6 @@ static void atomic_trace_rmw_post(CPUArchState *env, uint64_t addr, qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_RW); } -#if HAVE_ATOMIC128 -static void atomic_trace_ld_post(CPUArchState *env, uint64_t addr, - MemOpIdx oi) -{ - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_R); -} - -static void atomic_trace_st_post(CPUArchState *env, uint64_t addr, - MemOpIdx oi) -{ - qemu_plugin_vcpu_mem_cb(env_cpu(env), addr, oi, QEMU_PLUGIN_MEM_W); -} -#endif - /* * Atomic helpers callable from TCG. * These have a common interface and all defer to cpu_atomic_* From patchwork Sat May 20 16:26:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684321 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448979wrt; Sat, 20 May 2023 09:31:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ7Ai7cGgo0wYl5WJmL4Bs0pKhbE56N0ZxN9Nz+l/yqdamlcDlTC9niY7/L9qJOJxH/+Ovdp X-Received: by 2002:ac8:5747:0:b0:3f5:3991:97a5 with SMTP id 7-20020ac85747000000b003f5399197a5mr11094046qtx.54.1684600300715; Sat, 20 May 2023 09:31:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600300; cv=none; d=google.com; s=arc-20160816; b=UQ17Ow4hlbg0ky9lKAs4CgTjFpLhV5m6wkPQKXPmHGYtVB7fuPhhhIrHniuO2zZfUz Q7kQB9ZCmLgHw8aX5fY64K1XsviDviQHALFSW0FsvUhmNA5c9qcRLoADBMHD1byDXabX zdtcQhIZ9wO9W/kKmirp58MyC2s2LkopdMj1GRR3RW8S+PW1xQH8p3GmyGH05eVcP59D kUAeHolhiheiKngeprwW4EO36o/rZfW6L/1fQRwIb/9GDo8BEBjq1Hi7iZjR3SMKG3B0 q1wdiHnSiS+Zgty7XYiom4z8P1pklq/EufBzwjhNfWLSdo6FOTqaa0GjCUEanakfurrI dgfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=Vivv1DEDp/zgZ1RofUWbSm5C4MPgbeui7u7ZeA/i0A6oPBPt28PV7raqdSnHGE4gwr QAZ4zUOYt0UUDC3Gcaknp7jHvfpKDeP0sPZX1NPr6D8PB81e1vPwqh9bSUsI/SNqH0WG AO2Ryzgq5YaGiAIOuspiEfw9wQQoUjsevBkjKlyuOLzp6OJ56+u0sfSAh//H3g1r5JKD WO3s3cbhUTPH9m/tIko3vYotFrymjCGTmFF4Obbl8KIS2v2uWjR3YG9spr/zX8e+KxA6 5LWn50RW9zBijKhoAIIOFXpLGczomljEjY8W1BMoOsl0m8i7ywk1amY3xG2yNR1qEFPX 3c8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hOev8R87; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id 5-20020ac85745000000b003f6a309c439si677611qtx.451.2023.05.20.09.31.40 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:40 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=hOev8R87; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQO-0003tv-8d; Sat, 20 May 2023 12:27:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQG-0003jd-SW for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQ8-000340-P2 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:11 -0400 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1ae50da739dso29710025ad.1 for ; Sat, 20 May 2023 09:27:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600024; x=1687192024; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=hOev8R87druHwFEKlgebrKd4480KKxlTBTo+c9cvTcJj4nL3vbUhYBaS5MhLolttI/ 1ZzDQMrST1ZegTPkHgLsr2eTsc5FrbK6v655V4nlUQ0MQVNPtNzpvH2ikLFDJJINC4G1 HydPzwWf+GiMDYYUTEirU6XJktpJ3DUGQk/V3POJOAbqpKD6I0EWC/o1RygzyoDhuo9H WKn3TfCvzx2t5ab42MYUH5ZOJfPYdA2e0CMjY91rjGoGR+KlUZTRRpaYeYHuFIavxUX9 E/fXcis/mP2S1RDOk+bBHVV5hNfjy6J7aC8a2/iKyJXpZaHG9k2oZALemRO7mkrkXo7S ORjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600024; x=1687192024; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b4QbT7Qp39kM9Wky+d26o1epuHAtvA7Ubq3UboPhQe0=; b=kfjOHSpdjK2TYzYP+2q1G/+1eZ9ciyzVoGMKGgcYBUJkFwaJYJNNzQ9WsuY/QnYuAQ pJqwb4jYLl0JYAShNbwuuL/vLh5CqVrYBrcBOw5HZCm5yQ7NBk1QIJ43Oat2yLbviz2Q bmhreRO9vIYlzho755VHq/4EiZrXskmBZY4hUBYlsvD6ALjBIF5oNifogFazuweS+of7 P+DdGtwX3WqcdPiB+GrgO9Npa5J+EdMR1Tvd3XeCkA0+yl+oKuVVNGtOHqurSdiEeBGJ fG41rTGDvIleFaxOVUA/II2FQt0VWV2b7xArsvPMb6AvzUKP9rZ45MGNhR5jlOCPNsmm g7Zg== X-Gm-Message-State: AC+VfDzWN3oKsvDxGnylmqpVYUOArudaTPBUfZHJFUyJyJMfKPtWTaWK zw+yhvpsozt1uuAgGirBSaPTGon3ZGj/W4KGysk= X-Received: by 2002:a17:902:c944:b0:1a6:46f2:4365 with SMTP id i4-20020a170902c94400b001a646f24365mr7896263pla.30.1684600023796; Sat, 20 May 2023 09:27:03 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:03 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 20/27] accel/tcg: Remove prot argument to atomic_mmu_lookup Date: Sat, 20 May 2023 09:26:27 -0700 Message-Id: <20230520162634.3991009-21-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Now that load/store are gone, we're always passing PAGE_READ | PAGE_WRITE for RMW atomic operations. Signed-off-by: Richard Henderson --- accel/tcg/atomic_template.h | 32 ++++++-------- accel/tcg/cputlb.c | 85 ++++++++++++++----------------------- accel/tcg/user-exec.c | 8 +--- 3 files changed, 45 insertions(+), 80 deletions(-) diff --git a/accel/tcg/atomic_template.h b/accel/tcg/atomic_template.h index 30eee9d066..e312acd16d 100644 --- a/accel/tcg/atomic_template.h +++ b/accel/tcg/atomic_template.h @@ -73,8 +73,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; #if DATA_SIZE == 16 @@ -91,8 +90,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; ret = qatomic_xchg__nocheck(haddr, val); @@ -105,9 +103,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \ { \ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - DATA_TYPE ret; \ + DATA_TYPE *haddr, ret; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ ret = qatomic_##X(haddr, val); \ ATOMIC_MMU_CLEANUP; \ atomic_trace_rmw_post(env, addr, oi); \ @@ -137,9 +134,8 @@ GEN_ATOMIC_HELPER(xor_fetch) ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \ { \ - XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - XDATA_TYPE cmp, old, new, val = xval; \ + XDATA_TYPE *haddr, cmp, old, new, val = xval; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ smp_mb(); \ cmp = qatomic_read__nocheck(haddr); \ do { \ @@ -180,8 +176,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE cmpv, ABI_TYPE newv, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); DATA_TYPE ret; #if DATA_SIZE == 16 @@ -198,8 +193,7 @@ ABI_TYPE ATOMIC_NAME(cmpxchg)(CPUArchState *env, target_ulong addr, ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) { - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, - PAGE_READ | PAGE_WRITE, retaddr); + DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); ABI_TYPE ret; ret = qatomic_xchg__nocheck(haddr, BSWAP(val)); @@ -212,9 +206,8 @@ ABI_TYPE ATOMIC_NAME(xchg)(CPUArchState *env, target_ulong addr, ABI_TYPE val, ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE val, MemOpIdx oi, uintptr_t retaddr) \ { \ - DATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - DATA_TYPE ret; \ + DATA_TYPE *haddr, ret; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ ret = qatomic_##X(haddr, BSWAP(val)); \ ATOMIC_MMU_CLEANUP; \ atomic_trace_rmw_post(env, addr, oi); \ @@ -241,9 +234,8 @@ GEN_ATOMIC_HELPER(xor_fetch) ABI_TYPE ATOMIC_NAME(X)(CPUArchState *env, target_ulong addr, \ ABI_TYPE xval, MemOpIdx oi, uintptr_t retaddr) \ { \ - XDATA_TYPE *haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, \ - PAGE_READ | PAGE_WRITE, retaddr); \ - XDATA_TYPE ldo, ldn, old, new, val = xval; \ + XDATA_TYPE *haddr, ldo, ldn, old, new, val = xval; \ + haddr = atomic_mmu_lookup(env, addr, oi, DATA_SIZE, retaddr); \ smp_mb(); \ ldn = qatomic_read__nocheck(haddr); \ do { \ diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index b1e13d165c..9cb0b697d1 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -1896,12 +1896,9 @@ static bool mmu_lookup(CPUArchState *env, target_ulong addr, MemOpIdx oi, /* * Probe for an atomic operation. Do not allow unaligned operations, * or io operations to proceed. Return the host address. - * - * @prot may be PAGE_READ, PAGE_WRITE, or PAGE_READ|PAGE_WRITE. */ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, - MemOpIdx oi, int size, int prot, - uintptr_t retaddr) + MemOpIdx oi, int size, uintptr_t retaddr) { uintptr_t mmu_idx = get_mmuidx(oi); MemOp mop = get_memop(oi); @@ -1937,54 +1934,37 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, tlbe = tlb_entry(env, mmu_idx, addr); /* Check TLB entry and enforce page permissions. */ - if (prot & PAGE_WRITE) { - tlb_addr = tlb_addr_write(tlbe); - if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_STORE, - addr & TARGET_PAGE_MASK)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_STORE, mmu_idx, retaddr); - index = tlb_index(env, mmu_idx, addr); - tlbe = tlb_entry(env, mmu_idx, addr); - } - tlb_addr = tlb_addr_write(tlbe) & ~TLB_INVALID_MASK; - } - - if (prot & PAGE_READ) { - /* - * Let the guest notice RMW on a write-only page. - * We have just verified that the page is writable. - * Subpage lookups may have left TLB_INVALID_MASK set, - * but addr_read will only be -1 if PAGE_READ was unset. - */ - if (unlikely(tlbe->addr_read == -1)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_LOAD, mmu_idx, retaddr); - /* - * Since we don't support reads and writes to different - * addresses, and we do have the proper page loaded for - * write, this shouldn't ever return. But just in case, - * handle via stop-the-world. - */ - goto stop_the_world; - } - /* Collect TLB_WATCHPOINT for read. */ - tlb_addr |= tlbe->addr_read; - } - } else /* if (prot & PAGE_READ) */ { - tlb_addr = tlbe->addr_read; - if (!tlb_hit(tlb_addr, addr)) { - if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_LOAD, - addr & TARGET_PAGE_MASK)) { - tlb_fill(env_cpu(env), addr, size, - MMU_DATA_LOAD, mmu_idx, retaddr); - index = tlb_index(env, mmu_idx, addr); - tlbe = tlb_entry(env, mmu_idx, addr); - } - tlb_addr = tlbe->addr_read & ~TLB_INVALID_MASK; + tlb_addr = tlb_addr_write(tlbe); + if (!tlb_hit(tlb_addr, addr)) { + if (!victim_tlb_hit(env, mmu_idx, index, MMU_DATA_STORE, + addr & TARGET_PAGE_MASK)) { + tlb_fill(env_cpu(env), addr, size, + MMU_DATA_STORE, mmu_idx, retaddr); + index = tlb_index(env, mmu_idx, addr); + tlbe = tlb_entry(env, mmu_idx, addr); } + tlb_addr = tlb_addr_write(tlbe) & ~TLB_INVALID_MASK; } + /* + * Let the guest notice RMW on a write-only page. + * We have just verified that the page is writable. + * Subpage lookups may have left TLB_INVALID_MASK set, + * but addr_read will only be -1 if PAGE_READ was unset. + */ + if (unlikely(tlbe->addr_read == -1)) { + tlb_fill(env_cpu(env), addr, size, MMU_DATA_LOAD, mmu_idx, retaddr); + /* + * Since we don't support reads and writes to different + * addresses, and we do have the proper page loaded for + * write, this shouldn't ever return. But just in case, + * handle via stop-the-world. + */ + goto stop_the_world; + } + /* Collect TLB_WATCHPOINT for read. */ + tlb_addr |= tlbe->addr_read; + /* Notice an IO access or a needs-MMU-lookup access */ if (unlikely(tlb_addr & (TLB_MMIO | TLB_DISCARD_WRITE))) { /* There's really nothing that can be done to @@ -2000,11 +1980,8 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, } if (unlikely(tlb_addr & TLB_WATCHPOINT)) { - QEMU_BUILD_BUG_ON(PAGE_READ != BP_MEM_READ); - QEMU_BUILD_BUG_ON(PAGE_WRITE != BP_MEM_WRITE); - /* therefore prot == watchpoint bits */ - cpu_check_watchpoint(env_cpu(env), addr, size, - full->attrs, prot, retaddr); + cpu_check_watchpoint(env_cpu(env), addr, size, full->attrs, + BP_MEM_READ | BP_MEM_WRITE, retaddr); } return hostaddr; diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 19c2849c21..1e085b1210 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1323,12 +1323,9 @@ uint64_t cpu_ldq_code_mmu(CPUArchState *env, abi_ptr addr, /* * Do not allow unaligned operations to proceed. Return the host address. - * - * @prot may be PAGE_READ, PAGE_WRITE, or PAGE_READ|PAGE_WRITE. */ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, - MemOpIdx oi, int size, int prot, - uintptr_t retaddr) + MemOpIdx oi, int size, uintptr_t retaddr) { MemOp mop = get_memop(oi); int a_bits = get_alignment_bits(mop); @@ -1336,8 +1333,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, /* Enforce guest required alignment. */ if (unlikely(addr & ((1 << a_bits) - 1))) { - MMUAccessType t = prot == PAGE_READ ? MMU_DATA_LOAD : MMU_DATA_STORE; - cpu_loop_exit_sigbus(env_cpu(env), addr, t, retaddr); + cpu_loop_exit_sigbus(env_cpu(env), addr, MMU_DATA_STORE, retaddr); } /* Enforce qemu required alignment. */ From patchwork Sat May 20 16:26:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684303 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448025wrt; Sat, 20 May 2023 09:29:12 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5eAxpzCd9jexdKjqAKcwJbEmOAHwJEbIRourSiAnpXNrYwkSSQYnwDC4oXfNDmtwb+voLn X-Received: by 2002:a05:6214:c46:b0:5f1:606b:a9ca with SMTP id r6-20020a0562140c4600b005f1606ba9camr9696990qvj.37.1684600152768; Sat, 20 May 2023 09:29:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600152; cv=none; d=google.com; s=arc-20160816; b=cd61cvWtBX0vRKheQNL0b7aZJXw7xOt3KLtn/omOfam3IsMLxE7/REMlF78/aijS4g MWlu+XTlpyUx+HgckTf2TtCDX8XkCd66Nr+KqhZFaoxCELoZ72orXJq4JesfhyleH7Rd ham/54n/uNBb9t6nN12QGkib0RaLJsSDDIPHEvNPO3IkSf/OBWFDaTCIdcIFtAZ8/G63 BHZrfHS7ePa5jqFAtcTgmjLDdze+cXCcomU7uhQ+vqLFvqjkEjHV/0s6YA0xj/BMxeGN g7S/Udf49tqj9pMX6pCYBcpvtz3xKqRjcDRkENIUhn0iRla/ZhLPUTBw0S4O3zAFH9Fz 2hfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=oSXjZgIGjllrMA9XNuenvDLTD+dbMABgvhmoFNTKWzQ=; b=C1gnyPz15urBq9y1B5bDaOFpqt2vv/qOHBK++5uq4JWWffpM0+0F/h0abauJbtv1pQ jDZLZ/F7CKpteEMV85plymAwaOpIsRm6L8IpNqa/jUS8l0SPsQysilvYHtgvoOwC89g9 TQNmwhbns6xjEY0BbsCvlVXdvS232WmJ6S2ZMjvak7pljR6L6t3r6GvnC9pEhKZwJmBu 4pmcbM+LrSaWIwjQ9xUMFzkgV6AXsPrjXauCrXMBF+p8wWRKhq+67+v5vdKzPaQ6JAtn ZIWKmZ6ai6eMiiRCsXSXCt0XyhswVcDmpTXUP7WRDrTLL6wUfCbSrbKIa632Eekwm+gC zh/w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cR0s5s+l; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id eq15-20020ad4596f000000b0061b637a64e9si1195607qvb.336.2023.05.20.09.29.12 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=cR0s5s+l; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQS-0003wK-2Q; Sat, 20 May 2023 12:27:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQG-0003ja-So for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQA-0003CN-0p for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:12 -0400 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1ae5dc9eac4so26547465ad.1 for ; Sat, 20 May 2023 09:27:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600025; x=1687192025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oSXjZgIGjllrMA9XNuenvDLTD+dbMABgvhmoFNTKWzQ=; b=cR0s5s+lF8pjYqJZrBb0EyoeLMRd5TCsLMYTNLhJXgLPlnvZPSuDrrkQj5/hIIHAva S0idrUSUl5f8iQ6O/DLbYcfJiNloEKS/oLNITSHc5Lca+3HXraAlAS2M65D3l+iWPA2e m9rc5Z8hnRX5dvbtEBAM1RftiwwwFk4bxY1J3rcOBv4JKwgx2tWM72Iakxx/e06rZrUJ GHrjtX7lRul7/eUuThAgxXGUsLSzlFTA78pveZO96VJ5MIkX3HMkLeinEtQszQV8cCNg LL8PYtwqqmho9jUZrQ/f+oWuK0/M8a8rVEySLD3xxWcqSQIAwWKVzfeUNKIIQobExENH SyLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600025; x=1687192025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oSXjZgIGjllrMA9XNuenvDLTD+dbMABgvhmoFNTKWzQ=; b=YxTvbEPDWtaBZzmEQ78KnUC3DdstcQdDVuCHiwfLewUV/Atsl0WkKOz+1++qLvt/X2 vMHBDvce9Y4lTART8Omn95PZSK4TDkcTOTO/3ZBQJF85vICFEUjj9gnDjSrE4UFN2FsR 7EsYV+Qo1LvkjOcXumygnskKffkW9pU9NmxuaR1zNnC43f+iAVN5MlWqgqlc7N2ZF33r fOjFxZLnruXJzi/1ihTFfWl95enJ9LhcvSiRcKlDx+cbwPwFhrPNZCxpDFjbOmYjKcEh x/6xhwJfkON/QDVJVf9rrpl+tFSor4tR6J7Fg2eLSZwvJjQppc/4xUd9EVBmm2GPvP6a dz1w== X-Gm-Message-State: AC+VfDzfgAkx0bV2aR8Y5SUFTEZHk2ngxBg6j3pNrCimy6a0ixsX48tq DHv4RBa9XeUwijRiFF9Fne4/pAav6y9ep3u7GCw= X-Received: by 2002:a17:903:1210:b0:1ac:b363:83a6 with SMTP id l16-20020a170903121000b001acb36383a6mr6381627plh.17.1684600024836; Sat, 20 May 2023 09:27:04 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:04 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 21/27] accel/tcg: Eliminate #if on HAVE_ATOMIC128 and HAVE_CMPXCHG128 Date: Sat, 20 May 2023 09:26:28 -0700 Message-Id: <20230520162634.3991009-22-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org These symbols will shortly become dynamic runtime tests and therefore not appropriate for the preprocessor. Use the matching CONFIG_* symbols for that purpose. Signed-off-by: Richard Henderson --- host/include/aarch64/host/atomic128-cas.h | 2 ++ host/include/generic/host/atomic128-ldst.h | 2 +- accel/tcg/cputlb.c | 2 +- accel/tcg/user-exec.c | 2 +- 4 files changed, 5 insertions(+), 3 deletions(-) diff --git a/host/include/aarch64/host/atomic128-cas.h b/host/include/aarch64/host/atomic128-cas.h index 33f365ce67..ff0451d1aa 100644 --- a/host/include/aarch64/host/atomic128-cas.h +++ b/host/include/aarch64/host/atomic128-cas.h @@ -37,6 +37,8 @@ static inline Int128 atomic16_cmpxchg(Int128 *ptr, Int128 cmp, Int128 new) return int128_make128(oldl, oldh); } + +# define CONFIG_CMPXCHG128 1 # define HAVE_CMPXCHG128 1 #endif diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 46911dfb61..06a62e9dd0 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -33,7 +33,7 @@ atomic16_set(Int128 *ptr, Int128 val) } # define HAVE_ATOMIC128 1 -#elif !defined(CONFIG_USER_ONLY) && HAVE_CMPXCHG128 +#elif defined(CONFIG_CMPXCHG128) && !defined(CONFIG_USER_ONLY) static inline Int128 ATTRIBUTE_ATOMIC128_OPT atomic16_read(Int128 *ptr) { diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 9cb0b697d1..0bd06bf894 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -3038,7 +3038,7 @@ void cpu_st16_mmu(CPUArchState *env, target_ulong addr, Int128 val, #include "atomic_template.h" #endif -#if HAVE_CMPXCHG128 || HAVE_ATOMIC128 +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) #define DATA_SIZE 16 #include "atomic_template.h" #endif diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c index 1e085b1210..dc8d6b5d40 100644 --- a/accel/tcg/user-exec.c +++ b/accel/tcg/user-exec.c @@ -1371,7 +1371,7 @@ static void *atomic_mmu_lookup(CPUArchState *env, target_ulong addr, #include "atomic_template.h" #endif -#if HAVE_ATOMIC128 || HAVE_CMPXCHG128 +#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) #define DATA_SIZE 16 #include "atomic_template.h" #endif From patchwork Sat May 20 16:26:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684323 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp449031wrt; Sat, 20 May 2023 09:31:47 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6Zj4jqTXSnUGw0FM5AZfCa+jM+f7jol2tJ3V7g/cJNrS/Mh7BOYg1s657IF2NxUY7AC/o4 X-Received: by 2002:a05:622a:14b:b0:3f6:a1e9:1c05 with SMTP id v11-20020a05622a014b00b003f6a1e91c05mr2744583qtw.52.1684600306881; Sat, 20 May 2023 09:31:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600306; cv=none; d=google.com; s=arc-20160816; b=Jgu45cDqOg6FmDwQFMukNDd/pWnnNoBOrtZ6otoi68IeHf6gwr9AreZxMk4Om7XGBr 3X56g4jmmnUfIsT+9cTyKloVjPky8H6OiKM+cIDr0ygNusJL+bnjUUr9Ua3j+NODkjRS UEXJQxXi1Jc02r2phsyuReWHTW+kjf0kbTyXSDP5YS55siqAhGnwhDN1pIDzV+2qhuSR M2GEzasJn6AdgtUwMDJP2jmY+yMV3zcxWRkGSTtScHZVEW6fkgo3kA01obdDFkaS3SiU 1chkSJ/cRX2XrMlygqnAlG+B8SjWfBAqiHGfdIc7uO+bzW5W9d+435O6oJckzKbaSPVi WkKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=CGOMI9oWPo3BxTxZ0mc++YW+gNIJuzyjA8l4rsjXfws=; b=bv+JRSQQJpqf3rWHQVZVjxOH/kuXw45mIehXYSafJhwyIPoF/l2xlEh0azBg57O9yI mqoxoLh7AJTqPLmzZLIjX9AKMYgjeXmjCTm7ytEi5do7gGimu7bSnhCtoJBKAVKMjt9S WzcJqLxR+VskNL7g3A/QjWckgRbnbp+4GaeC6rsE7Gd3t2EZFjloojlshxD5LmCvDAMO E+rMUeXbos6iv4KK4yROw3BeSnh4Tlc8fd9YfEFnVj25EEcs52kl+IIYwEynN+coWlnq I5+Zw21iQLq1ji/VrKJ2KnKdaqwqHbqXKDKl/fjV6Qd/GzBwIK4EDG4rhV3MPdjYUBEP 70aQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pkWmyPa4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id a7-20020a05622a064700b003e4e9ddc6f9si1351332qtb.118.2023.05.20.09.31.46 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:46 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=pkWmyPa4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQO-0003tx-8N; Sat, 20 May 2023 12:27:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQI-0003lD-3o for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:14 -0400 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQB-0003Cm-D8 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:13 -0400 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-64d41d8bc63so858921b3a.0 for ; Sat, 20 May 2023 09:27:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600025; x=1687192025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CGOMI9oWPo3BxTxZ0mc++YW+gNIJuzyjA8l4rsjXfws=; b=pkWmyPa4dErujdaLu++fdfX9S8D492g1tUcbNwqC+jeVrA1uzIOc+/3UDO4kU2x5a8 5h0x14O0xBXUVJxvTOFFkYs7EusA4q90/agou2cuRkLvQl+eRUgDOJqKdle7AMEVfWAY 7oToMI7GC2U7GqDBIsNkfpxYc+lqQbAw9Di7vUOYCJV8DEisqyMigLXrtLTFs+NzoRFC CpXX2gyh+mbS6t/HZ7vgbE2RJms5KxLk+ph9SjcH8segsKZxDNHU3j8yF1iZjao6/ZER U9hCdAPQXbdytRKiFU9GjgIVIShA74eU1B0jGr25yBvlhMccpCBfiHSvPEnBVOf6KB80 4PVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600025; x=1687192025; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CGOMI9oWPo3BxTxZ0mc++YW+gNIJuzyjA8l4rsjXfws=; b=VQ6ZoVaOVpTLDcaJkafcVrZ4fNInQBFfQSQzp8VKdd5r6rz+FNOLINV9/ssw4O8R7n tLM1Z/slNI2epiZy6E6sxgQvYQdBWIaISdYuNQbtZifKWvEA+C4bnf+HghFDpN7/EEI9 xgU9e2y8knb0CEYkOw0d1FeFtuSwhVbrdkQkNw0AmnWrsx+qf6mjCFcfl0duCL5FUvOJ 4lk3IkoixMXWf0Cm4DBW1bQR8A+cOCX1FmHMR94xBnoWzN+9W3USkFzWCoZnczs597O8 Volau17STsNIylBujhpRFoeFg2Yrusgqnb/YExmLGNJLbtYcWN62l8z5Dbxd4pOLRfrB EW+A== X-Gm-Message-State: AC+VfDxmyUKeFLSTtmpi3xGT2Swb49c1I6D1ma5CNqhpFFWCLX8y0n0r RfnSsc46ErNYIeIZoB2uz3lFxmLFTae0QB34YzM= X-Received: by 2002:a17:902:d2c2:b0:1ac:b4db:6a62 with SMTP id n2-20020a170902d2c200b001acb4db6a62mr6328290plc.65.1684600025593; Sat, 20 May 2023 09:27:05 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 22/27] qemu/atomic128: Split atomic16_read Date: Sat, 20 May 2023 09:26:29 -0700 Message-Id: <20230520162634.3991009-23-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::434; envelope-from=richard.henderson@linaro.org; helo=mail-pf1-x434.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Create both atomic16_read_ro and atomic16_read_rw. Previously we pretended that we had atomic16_read in system mode, because we "know" that all ram is always writable to the host. Now, expose read-only and read-write versions all of the time. For aarch64, do not fall back to __atomic_read_16 even if supported by the compiler, to work around a clang bug. Signed-off-by: Richard Henderson --- host/include/aarch64/host/atomic128-ldst.h | 21 ++++++++------- host/include/generic/host/atomic128-ldst.h | 31 ++++++++++++++++------ target/s390x/tcg/mem_helper.c | 2 +- 3 files changed, 36 insertions(+), 18 deletions(-) diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h index c2e7b44bc5..6959b2bd8e 100644 --- a/host/include/aarch64/host/atomic128-ldst.h +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -11,10 +11,18 @@ #ifndef AARCH64_ATOMIC128_LDST_H #define AARCH64_ATOMIC128_LDST_H -/* Through gcc 10, aarch64 has no support for 128-bit atomics. */ -#if !defined(CONFIG_ATOMIC128) && !defined(CONFIG_USER_ONLY) -/* We can do better than cmpxchg for AArch64. */ -static inline Int128 atomic16_read(Int128 *ptr) +/* + * Through gcc 10, aarch64 has no support for 128-bit atomics. + * Through clang 16, without -march=armv8.4-a, __atomic_load_16 + * is incorrectly expanded to a read-write operation. + */ + +#define HAVE_ATOMIC128_RO 0 +#define HAVE_ATOMIC128_RW 1 + +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); + +static inline Int128 atomic16_read_rw(Int128 *ptr) { uint64_t l, h; uint32_t tmp; @@ -41,9 +49,4 @@ static inline void atomic16_set(Int128 *ptr, Int128 val) : [l] "r"(l), [h] "r"(h)); } -# define HAVE_ATOMIC128 1 -#else -#include "host/include/generic/host/atomic128-ldst.h" -#endif - #endif /* AARCH64_ATOMIC128_LDST_H */ diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 06a62e9dd0..79d208b7a4 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -12,16 +12,25 @@ #define HOST_ATOMIC128_LDST_H #if defined(CONFIG_ATOMIC128) +# define HAVE_ATOMIC128_RO 1 +# define HAVE_ATOMIC128_RW 1 + static inline Int128 ATTRIBUTE_ATOMIC128_OPT -atomic16_read(Int128 *ptr) +atomic16_read_ro(const Int128 *ptr) { - __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + const __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); Int128Alias r; r.i = qatomic_read__nocheck(ptr_align); return r.s; } +static inline Int128 ATTRIBUTE_ATOMIC128_OPT +atomic16_read_rw(Int128 *ptr) +{ + return atomic16_read_ro(ptr); +} + static inline void ATTRIBUTE_ATOMIC128_OPT atomic16_set(Int128 *ptr, Int128 val) { @@ -32,10 +41,14 @@ atomic16_set(Int128 *ptr, Int128 val) qatomic_set__nocheck(ptr_align, v.i); } -# define HAVE_ATOMIC128 1 -#elif defined(CONFIG_CMPXCHG128) && !defined(CONFIG_USER_ONLY) +#elif defined(CONFIG_CMPXCHG128) +# define HAVE_ATOMIC128_RO 0 +# define HAVE_ATOMIC128_RW 1 + +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); + static inline Int128 ATTRIBUTE_ATOMIC128_OPT -atomic16_read(Int128 *ptr) +atomic16_read_rw(Int128 *ptr) { /* Maybe replace 0 with 0, returning the old value. */ Int128 z = int128_make64(0); @@ -52,12 +65,14 @@ atomic16_set(Int128 *ptr, Int128 val) } while (int128_ne(old, cmp)); } -# define HAVE_ATOMIC128 1 #else +# define HAVE_ATOMIC128_RO 0 +# define HAVE_ATOMIC128_RW 0 + /* Fallback definitions that must be optimized away, or error. */ -Int128 QEMU_ERROR("unsupported atomic") atomic16_read(Int128 *ptr); +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); +Int128 QEMU_ERROR("unsupported atomic") atomic16_read_rw(Int128 *ptr); void QEMU_ERROR("unsupported atomic") atomic16_set(Int128 *ptr, Int128 val); -# define HAVE_ATOMIC128 0 #endif #endif /* HOST_ATOMIC128_LDST_H */ diff --git a/target/s390x/tcg/mem_helper.c b/target/s390x/tcg/mem_helper.c index bad789a742..db22995171 100644 --- a/target/s390x/tcg/mem_helper.c +++ b/target/s390x/tcg/mem_helper.c @@ -1778,7 +1778,7 @@ static uint32_t do_csst(CPUS390XState *env, uint32_t r3, uint64_t a1, max = 3; #endif if ((HAVE_CMPXCHG128 ? 0 : fc + 2 > max) || - (HAVE_ATOMIC128 ? 0 : sc > max)) { + (HAVE_ATOMIC128_RW ? 0 : sc > max)) { cpu_loop_exit_atomic(env_cpu(env), ra); } } From patchwork Sat May 20 16:26:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684320 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448960wrt; Sat, 20 May 2023 09:31:38 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5ARyTExwaHE+4rUgQMXS5VUQcC0GAx5JcFxImCXyzWKq09HYqC5bY1tjH6kVjq3F0dMkl6 X-Received: by 2002:ad4:5def:0:b0:621:3979:9be7 with SMTP id jn15-20020ad45def000000b0062139799be7mr9879003qvb.23.1684600297789; Sat, 20 May 2023 09:31:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600297; cv=none; d=google.com; s=arc-20160816; b=UQ2FraG7IHlGKP01pCbzFs1pAcKv/b7Jl+mHOMNjR4nTQeaXPzFbk8nhVu8sCE335e 50x4p0bHLa/nm02ndfHpo+V2flda0YCeRR18GC3DYMMtVc9mmv1ziH0pHCLTkMq1gnds IAef/CIuQEFC3swWNVXu0tKqLXCmRHD7/y7rUvS1k96AriPess9yz+j3Z9cVRL7NYIpq L6Ta7DBwycw+3cCG7lrblQmQcuEHvUCfTfsk8QnFRz6yrD8sjOuk9EkuQzTkm50pAxNV hnSYefLjkkU8QhAVB86+OMYsNvq9Lk4Z0x15oXjjq+NdM6wcHgZ8qVc4Cp3Qqfk8Bxj6 MHjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=ADUn+M7EqR0YphnOI5NrNnwymRSG8PojJQO9jJjyKPWgLbozcylNTCmfdu4MeZgw8+ 9Ocsj2mSv/YVP6ZVBG23vbxvT7F6lGELkQikrtqtwnfllCWyHH4JVYqSmGXZbTU4DQW+ m0oRPwawgcXdWaeymbo6cl1bo+MOpzo2ygRZP/vFW+LV2uqHJxEGdBQ/eEcX0BI8wSBf zyOfhX4oUBLfaUhLgJIdvhbbTEjym53qakQexdwbJ7sqMvZm90lr+++1SDQlAfp1p6xW AXYEfraaNd45dzw8axwbl5UVjD5sd8QKSg5cGnM0PTY/Nn1xx7/AD39Zo1W3yU+f7f95 dY0w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ti+wWeMg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id q14-20020a056214194e00b005ef676dbf83si1205817qvk.554.2023.05.20.09.31.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:37 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=ti+wWeMg; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQg-000442-6a; Sat, 20 May 2023 12:27:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQI-0003lo-Kw for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:14 -0400 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQB-0003DB-Mk for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:14 -0400 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1ae763f9c0bso19177635ad.2 for ; Sat, 20 May 2023 09:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600026; x=1687192026; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=ti+wWeMggL+updxo7npyLU6E7n0t0wTIC2QaiZuU7+qej1bOnC7wwhRMZNtME0MxAt mn+6XenhT4Z/XwUVTEIYnlOaWC6dCFDQHAq9r/zo1nt4bO+GBb9I2nQ4GEMnEGNMvyiD c/mx2vdg3jan3T+vu/leMFhISwkLXvLxKDBbsJs550dTkcwA+EwUdhuxPJERPVqosC+d 07wnlSsaW2+ZF3ei21FJFWwwFpXJCmpXPtjzepA4ZF3DW5F22u/XJKDuoVIapNchsd3D 3cktPlQV7u6yxvv2jkABYgoHY+4ZXLekHsXVhjwJ17KhdFKSuG5laL9lfknXrXmmhfKm 2usA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600026; x=1687192026; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=HBWDA1+uXlJlZoqOFW38eoLfuSmJeXie3CQojpPMW+I=; b=MOV0dfjL2XlTEhEPzgEblNGpqB+gdeUSBXv5htgjuOQjBzr90BHhczsoL7YO9Q0ugK qrAFfppMuignNXzaYsjKBsdL6r84NCGYEA1/xZFEB/xh38gQpyzgnEuwqUd3JgJyh/+J M4UMU5wzMe8CPdy8T47HT3++Ryp+sNkFOtFCPqakZMiLDz5MGqOeps93UngIf81IIUbo 1TKmlQCI13gal82MoyQwDbZSHK7klV3UGhmNBNkgyUNtUk3j1lsdhxwaNQxRZlRLRbda D4jGRKJRZkwUK2cyILnGC2lPG4G/VE5zxA37xX8UaoQjyXQLhD4NTrxUSbvsmLLuMk1w yZ2g== X-Gm-Message-State: AC+VfDzm7t+Syj2rNhqfF3xeuirtyruvIDqOVRQxpznLfgK1afOsKQxl FadmjfuGZGrcvfzMRyXTBo9u70kzbVvW/6bE3Wg= X-Received: by 2002:a17:902:cacc:b0:1a6:8031:59e7 with SMTP id y12-20020a170902cacc00b001a6803159e7mr5435095pld.46.1684600026320; Sat, 20 May 2023 09:27:06 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:05 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 23/27] accel/tcg: Correctly use atomic128.h in ldst_atomicity.c.inc Date: Sat, 20 May 2023 09:26:30 -0700 Message-Id: <20230520162634.3991009-24-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::630; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x630.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Remove the locally defined load_atomic16 and store_atomic16, along with HAVE_al16 and HAVE_al16_fast in favor of the routines defined in atomic128.h. Signed-off-by: Richard Henderson --- accel/tcg/cputlb.c | 2 +- accel/tcg/ldst_atomicity.c.inc | 118 +++++++-------------------------- 2 files changed, 24 insertions(+), 96 deletions(-) diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c index 0bd06bf894..90c72c9940 100644 --- a/accel/tcg/cputlb.c +++ b/accel/tcg/cputlb.c @@ -2712,7 +2712,7 @@ static uint64_t do_st16_leN(CPUArchState *env, MMULookupPageData *p, case MO_ATOM_WITHIN16_PAIR: /* Since size > 8, this is the half that must be atomic. */ - if (!HAVE_al16) { + if (!HAVE_ATOMIC128_RW) { cpu_loop_exit_atomic(env_cpu(env), ra); } return store_whole_le16(p->haddr, p->size, val_le); diff --git a/accel/tcg/ldst_atomicity.c.inc b/accel/tcg/ldst_atomicity.c.inc index b89631bbef..0f6b3f8ab6 100644 --- a/accel/tcg/ldst_atomicity.c.inc +++ b/accel/tcg/ldst_atomicity.c.inc @@ -16,18 +16,6 @@ #endif #define HAVE_al8_fast (ATOMIC_REG_SIZE >= 8) -#if defined(CONFIG_ATOMIC128) -# define HAVE_al16_fast true -#else -# define HAVE_al16_fast false -#endif -#if defined(CONFIG_ATOMIC128) || defined(CONFIG_CMPXCHG128) -# define HAVE_al16 true -#else -# define HAVE_al16 false -#endif - - /** * required_atomicity: * @@ -146,26 +134,6 @@ static inline uint64_t load_atomic8(void *pv) return qatomic_read__nocheck(p); } -/** - * load_atomic16: - * @pv: host address - * - * Atomically load 16 aligned bytes from @pv. - */ -static inline Int128 ATTRIBUTE_ATOMIC128_OPT -load_atomic16(void *pv) -{ -#ifdef CONFIG_ATOMIC128 - __uint128_t *p = __builtin_assume_aligned(pv, 16); - Int128Alias r; - - r.u = qatomic_read__nocheck(p); - return r.s; -#else - qemu_build_not_reached(); -#endif -} - /** * load_atomic8_or_exit: * @env: cpu context @@ -211,8 +179,8 @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) { Int128 *p = __builtin_assume_aligned(pv, 16); - if (HAVE_al16_fast) { - return load_atomic16(p); + if (HAVE_ATOMIC128_RO) { + return atomic16_read_ro(p); } #ifdef CONFIG_USER_ONLY @@ -232,14 +200,9 @@ static Int128 load_atomic16_or_exit(CPUArchState *env, uintptr_t ra, void *pv) * In system mode all guest pages are writable, and for user-only * we have just checked writability. Try cmpxchg. */ -#if defined(CONFIG_CMPXCHG128) - /* Swap 0 with 0, with the side-effect of returning the old value. */ - { - Int128Alias r; - r.u = __sync_val_compare_and_swap_16((__uint128_t *)p, 0, 0); - return r.s; + if (HAVE_ATOMIC128_RW) { + return atomic16_read_rw(p); } -#endif /* Ultimate fallback: re-execute in serial context. */ cpu_loop_exit_atomic(env_cpu(env), ra); @@ -360,11 +323,10 @@ static uint64_t load_atom_extract_al16_or_exit(CPUArchState *env, uintptr_t ra, static inline uint64_t ATTRIBUTE_ATOMIC128_OPT load_atom_extract_al16_or_al8(void *pv, int s) { -#if defined(CONFIG_ATOMIC128) uintptr_t pi = (uintptr_t)pv; int o = pi & 7; int shr = (HOST_BIG_ENDIAN ? 16 - s - o : o) * 8; - __uint128_t r; + Int128 r; pv = (void *)(pi & ~7); if (pi & 8) { @@ -373,18 +335,14 @@ load_atom_extract_al16_or_al8(void *pv, int s) uint64_t b = qatomic_read__nocheck(p8 + 1); if (HOST_BIG_ENDIAN) { - r = ((__uint128_t)a << 64) | b; + r = int128_make128(b, a); } else { - r = ((__uint128_t)b << 64) | a; + r = int128_make128(a, b); } } else { - __uint128_t *p16 = __builtin_assume_aligned(pv, 16, 0); - r = qatomic_read__nocheck(p16); + r = atomic16_read_ro(pv); } - return r >> shr; -#else - qemu_build_not_reached(); -#endif + return int128_getlo(int128_urshift(r, shr)); } /** @@ -472,7 +430,7 @@ static uint16_t load_atom_2(CPUArchState *env, uintptr_t ra, if (likely((pi & 1) == 0)) { return load_atomic2(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 2); } @@ -511,7 +469,7 @@ static uint32_t load_atom_4(CPUArchState *env, uintptr_t ra, if (likely((pi & 3) == 0)) { return load_atomic4(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 4); } @@ -557,7 +515,7 @@ static uint64_t load_atom_8(CPUArchState *env, uintptr_t ra, if (HAVE_al8 && likely((pi & 7) == 0)) { return load_atomic8(pv); } - if (HAVE_al16_fast) { + if (HAVE_ATOMIC128_RO) { return load_atom_extract_al16_or_al8(pv, 8); } @@ -607,8 +565,8 @@ static Int128 load_atom_16(CPUArchState *env, uintptr_t ra, * If the host does not support 16-byte atomics, wait until we have * examined the atomicity parameters below. */ - if (HAVE_al16_fast && likely((pi & 15) == 0)) { - return load_atomic16(pv); + if (HAVE_ATOMIC128_RO && likely((pi & 15) == 0)) { + return atomic16_read_ro(pv); } atmax = required_atomicity(env, pi, memop); @@ -687,36 +645,6 @@ static inline void store_atomic8(void *pv, uint64_t val) qatomic_set__nocheck(p, val); } -/** - * store_atomic16: - * @pv: host address - * @val: value to store - * - * Atomically store 16 aligned bytes to @pv. - */ -static inline void ATTRIBUTE_ATOMIC128_OPT -store_atomic16(void *pv, Int128Alias val) -{ -#if defined(CONFIG_ATOMIC128) - __uint128_t *pu = __builtin_assume_aligned(pv, 16); - qatomic_set__nocheck(pu, val.u); -#elif defined(CONFIG_CMPXCHG128) - __uint128_t *pu = __builtin_assume_aligned(pv, 16); - __uint128_t o; - - /* - * Without CONFIG_ATOMIC128, __atomic_compare_exchange_n will always - * defer to libatomic, so we must use __sync_*_compare_and_swap_16 - * and accept the sequential consistency that comes with it. - */ - do { - o = *pu; - } while (!__sync_bool_compare_and_swap_16(pu, o, val.u)); -#else - qemu_build_not_reached(); -#endif -} - /** * store_atom_4x2 */ @@ -957,7 +885,7 @@ static uint64_t store_whole_le16(void *pv, int size, Int128 val_le) int sh = o * 8; Int128 m, v; - qemu_build_assert(HAVE_al16); + qemu_build_assert(HAVE_ATOMIC128_RW); /* Like MAKE_64BIT_MASK(0, sz), but larger. */ if (sz <= 64) { @@ -1017,7 +945,7 @@ static void store_atom_2(CPUArchState *env, uintptr_t ra, return; } } else if ((pi & 15) == 7) { - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { Int128 v = int128_lshift(int128_make64(val), 56); Int128 m = int128_lshift(int128_make64(0xffff), 56); store_atom_insert_al16(pv - 7, v, m); @@ -1086,7 +1014,7 @@ static void store_atom_4(CPUArchState *env, uintptr_t ra, return; } } else { - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { store_whole_le16(pv, 4, int128_make64(cpu_to_le32(val))); return; } @@ -1151,7 +1079,7 @@ static void store_atom_8(CPUArchState *env, uintptr_t ra, } break; case MO_64: - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { store_whole_le16(pv, 8, int128_make64(cpu_to_le64(val))); return; } @@ -1177,8 +1105,8 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, uint64_t a, b; int atmax; - if (HAVE_al16_fast && likely((pi & 15) == 0)) { - store_atomic16(pv, val); + if (HAVE_ATOMIC128_RW && likely((pi & 15) == 0)) { + atomic16_set(pv, val); return; } @@ -1206,7 +1134,7 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, } break; case -MO_64: - if (HAVE_al16) { + if (HAVE_ATOMIC128_RW) { uint64_t val_le; int s2 = pi & 15; int s1 = 16 - s2; @@ -1233,8 +1161,8 @@ static void store_atom_16(CPUArchState *env, uintptr_t ra, } break; case MO_128: - if (HAVE_al16) { - store_atomic16(pv, val); + if (HAVE_ATOMIC128_RW) { + atomic16_set(pv, val); return; } break; From patchwork Sat May 20 16:26:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684309 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448166wrt; Sat, 20 May 2023 09:29:39 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5VXOlls2v9SXkaroTYzmFi96e3KZ8GVr0z6h80kHDxpt9h3JGLGGVo7qTt9lWviGNJu/M7 X-Received: by 2002:a05:6214:20ef:b0:623:7108:362d with SMTP id 15-20020a05621420ef00b006237108362dmr12461322qvk.9.1684600179205; Sat, 20 May 2023 09:29:39 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600179; cv=none; d=google.com; s=arc-20160816; b=dABR6JsK5DT3gWoE2w1AtIu0jzgyk2ta9apWqZ9B29oYgxHztUfasCFFWiktaNXABf BG+6dtWDDdMUMgjtf4gHsDCKVKjHl1aeUNr14+HxaEam+QmWcxBzt7Jp5Yu+pxrDmTzJ kEct13V8KYDG8zEcsVanZyE7jw7GQN7kMwRumzDJ9T28Rfq7a9uKqjLfmS7xeUyZJofB R3JZuMU7mthraors3ZQMM8KupKb2D6hgKOAb9GFnG8dlDyLUlnFKy3si8SsD/NHh2hMF fMevrG4l0aF1JUo/g241PXoZobQrnx3L/q3bazf5DQ/TlBJ5uC97DDEM/A/VqdX+CSv3 BiIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=1Plm35dUE66Q3yWg+6SkVizMCgYtBvFECTpZ00ZQ+pM=; b=s22FQpxNY5/JcbRTMRI/7F8Zu0D+aPYhwv2FDN3Ki4+SXrYXKFSqBngtHT3keB2eZT OYqY9iEazV1j6DGGbcf67CCPbjxmnuB0Hi8aLiWlqxlSKTTZU8NRDC0clLHdMRrTmfba nFw0IJ64i44HPsQ8UqppVoTJ5bre9x/yL6jGVK+N0QpU2XRyNAStKxuX/dF33025tjih PZc/VZFwsgb/0rcyO2HOQxJPZzchHOPsRpY1Kp8sPv6sqtsUa3rN4Dk3xnQtv6px5jOl 5vTj1P5yfXWsIBhJka/1OnpIOtLaC+1mBHbbPDeo2syV32nasUkMQGIQPPrFv4wJdET+ fIog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qdVu7HsJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id jm10-20020ad45eca000000b005e8828d61c5si1257163qvb.517.2023.05.20.09.29.39 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:39 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=qdVu7HsJ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQR-0003vN-OP; Sat, 20 May 2023 12:27:23 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQJ-0003mt-9l for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:15 -0400 Received: from mail-pl1-x629.google.com ([2607:f8b0:4864:20::629]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQC-00034A-Ky for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:15 -0400 Received: by mail-pl1-x629.google.com with SMTP id d9443c01a7336-1ae8de081ccso9157615ad.1 for ; Sat, 20 May 2023 09:27:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600027; x=1687192027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1Plm35dUE66Q3yWg+6SkVizMCgYtBvFECTpZ00ZQ+pM=; b=qdVu7HsJyKzcF/SXMi+QAjlBa6PoQzAIuj3X/cwjIZ2bCDDd6Dbt1f6IwDulOKnEG8 HJhBp0T+ARPB4j9nyKtijEzqZc0ZlSFhOXJiOpFElsAbY53arqbsqpoem6l+cnJPwL+t EKTH83FlWt++EpGFK0fUQ3ZXhASPaStvhN5azH4UPRKUVTkU0DEpegiJXA1uLElkBbhd mqOuHXi4XFHlwVIMilgZtxOXkSg8HYEA0GFUPYn9AOhE46hIrz4vAb2CrGomtY37ZlIz Zz0oQrYzltKxtj6wzwDvEdi7UmIxEGfA9cM4J3dvMP73pEBIwcLGwviBz+Mbz6FnE9xA ta/Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600027; x=1687192027; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1Plm35dUE66Q3yWg+6SkVizMCgYtBvFECTpZ00ZQ+pM=; b=ghkL37dn19jN/0lj690XYD5HyReBNI39Slo3U6GfIJA09JOL9vHRHeMh+YstZHrpFl kmYYGB+n1Sy+uZXrZxT5i58YJsqytnuvYQRZxtW5OX0LeZGsvMWiOBbSKKfoakKkgNIq /HusvIpGhlAFjN9iG1M/zemU3+tND5IEdiFPRY12214kJwm8pGHawQh//l1ToKynnFOR TaiM1TluPZsEkQve127mXii0hlwDXh/ukCN1ASQajr2TB6OQC+hhBrKGJgvQSrMV5YT8 zX5hkCVKdlYsLZMN+Pbt9sFLHzfqdh8WuSTeqP1oPm6SLfXNPAkKvmX3Ci/lTBrpBnVs j2aA== X-Gm-Message-State: AC+VfDzxKKDxlPu+efpwEWen9M6g0LprQKQTVktL+7jXO2DZ1AFs1J3q ObtEH+uurVAJeYeWwRPMV4+8GrEpf/cTpZuEJ5k= X-Received: by 2002:a17:903:2287:b0:1ad:ea13:1914 with SMTP id b7-20020a170903228700b001adea131914mr10974248plh.30.1684600027284; Sat, 20 May 2023 09:27:07 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:06 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 24/27] tcg: Split out tcg/debug-assert.h Date: Sat, 20 May 2023 09:26:31 -0700 Message-Id: <20230520162634.3991009-25-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::629; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x629.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Signed-off-by: Richard Henderson Reviewed-by: Philippe Mathieu-Daudé --- include/tcg/debug-assert.h | 17 +++++++++++++++++ include/tcg/tcg.h | 9 +-------- 2 files changed, 18 insertions(+), 8 deletions(-) create mode 100644 include/tcg/debug-assert.h diff --git a/include/tcg/debug-assert.h b/include/tcg/debug-assert.h new file mode 100644 index 0000000000..596765a3d2 --- /dev/null +++ b/include/tcg/debug-assert.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: MIT */ +/* + * Define tcg_debug_assert + * Copyright (c) 2008 Fabrice Bellard + */ + +#ifndef TCG_DEBUG_ASSERT_H +#define TCG_DEBUG_ASSERT_H + +#if defined CONFIG_DEBUG_TCG || defined QEMU_STATIC_ANALYSIS +# define tcg_debug_assert(X) do { assert(X); } while (0) +#else +# define tcg_debug_assert(X) \ + do { if (!(X)) { __builtin_unreachable(); } } while (0) +#endif + +#endif diff --git a/include/tcg/tcg.h b/include/tcg/tcg.h index cd6327b175..072c35f7f5 100644 --- a/include/tcg/tcg.h +++ b/include/tcg/tcg.h @@ -34,6 +34,7 @@ #include "tcg/tcg-mo.h" #include "tcg-target.h" #include "tcg/tcg-cond.h" +#include "tcg/debug-assert.h" /* XXX: make safe guess about sizes */ #define MAX_OP_PER_INSTR 266 @@ -222,14 +223,6 @@ typedef uint64_t tcg_insn_unit; /* The port better have done this. */ #endif - -#if defined CONFIG_DEBUG_TCG || defined QEMU_STATIC_ANALYSIS -# define tcg_debug_assert(X) do { assert(X); } while (0) -#else -# define tcg_debug_assert(X) \ - do { if (!(X)) { __builtin_unreachable(); } } while (0) -#endif - typedef struct TCGRelocation TCGRelocation; struct TCGRelocation { QSIMPLEQ_ENTRY(TCGRelocation) next; From patchwork Sat May 20 16:26:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684311 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448287wrt; Sat, 20 May 2023 09:30:00 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4rKN/JdAIXBd8bAO6I12elDwAgT2XMib7dLBfsSPmhUp2DO2J2Cww+87XcC7T/yGpbFp5d X-Received: by 2002:a05:6214:c6c:b0:621:1808:e93a with SMTP id t12-20020a0562140c6c00b006211808e93amr9573260qvj.42.1684600199920; Sat, 20 May 2023 09:29:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600199; cv=none; d=google.com; s=arc-20160816; b=Q8UdDPuYU/+HNtHosIRM45M6J2ZoL9j1hRnum5YWdgooBW/rqLXdXXiDih6ZE9a3zd 2xG6fWQdLV9svRDWQZKfGi1huLTO6cHwxlWemSf1LzrJcqXUb4WDlA1Lgz1gEdPD2xf5 +Cw3FazcjKDp5kQtGCRFJLrSozgtP8uvpTNb5OWSWpG6LAgXqqALjXymLwFWVgbHzI/R lSRvC8F5qRZgiYi2oYweT0ZAwdpRcx3X1EtgfUnx1l3rNjSDGdJRjSk+/HqH96lwnJ0f ntf8HnMfFMfcF0ERY8GUcwwaGYJTipwVIt1w+saa8tAuhIxe8aoXcIj6UfBDj0zOHi4k VzoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=iDh6pfPbm0aG8/Qwxnm0ntN2QVw6y+1RrtCSwvRdB/wUv4elurxp7RaAUmOYS7nj07 VLXcKbSIdnh6zqBSKXKS5Iuh4TlDzycbNtT+Sw8oeyvVscOtmJSZekUG6dbydyp4MmIJ 2Ym3MvJlP8VLLS35Ng8jWF/KrUogKHky/O5ig2S3qSzYJ05f1QNlBkLZxRe5jXuJvODp gBS60lu/uvDLJCFKL2UUhWmiW45mqDJia/L4u0TE7F6dKThPhHnTwFwliaZxAIjyBAsm xAGauPMNAGpAkL39j0ZGT2D8oM1msBQ/R0CsSb64M434nt3gXWwD/ON/nNWNMo/4c3Hp XqbA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wUixfCHP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id jr7-20020a0562142a8700b005994d62c136si1211966qvb.447.2023.05.20.09.29.59 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:59 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=wUixfCHP; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQm-0004Jw-9w; Sat, 20 May 2023 12:27:44 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQJ-0003nB-DK for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:15 -0400 Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQE-000392-GP for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:15 -0400 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1ae615d5018so26547555ad.1 for ; Sat, 20 May 2023 09:27:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600028; x=1687192028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=wUixfCHPzv4t4m33BQzAwn58Cw6b65/96GrMJuwsDRylO2TW2hHfkgTZ8tQRBQzjLm R9jz4khe9Z7XEeKDn74C0wmfSdws6D0fQI8ndPID73egkLFouW93b0zyR8JznVC31VUU 5eraGvW+1QVoNH+qESq0SamQ+hBHtM0Qzj6CMWnAP7/hInANMJM1l9V7AYCGPa9PyR4w k8LGYAMc+NJ96JJqo1pgCTVan/4718W7NCoXp87u31H/9xCox2C27GkejHfG4pafDEg4 WISn/G16TgYlSHV4ls9FkgcEQ/jgN+35XBqMqURyWdvMHgu+spf7dy0v8ZWQ6ODpZ7GW JHgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600028; x=1687192028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6WXDf+ZM8xKRR3ok9Sko5lUMigoP5WIzPelyBaHT6/A=; b=PaAqZ9J8iRHM0lq+N2/X+xgLqfp7ZFEdAyX72VshZb3qVh0Ne9F7WTlp5AzazpX1BF i7mBY5LexqOSDjUbNTT3R/F3tnmms8K6lCrVpc/IJ14Kq6TzFE9H1abu7QbFCXP/rGDS NxeWaXlH63iegY49KQ6chf+jUr75ggWAsZl3BJw8SZyRKTWYvJKg/F7PuE+rz0x4GgIM wSXLGzuMUa86pDZfmYKclD8tngG8oZgICjdHL2A5FjLzmuQnuV5MHfLCurHarZi8Y/Dj p41scgWkwPIzw2t4yTYL8IjtmFbJjzbjaRW/xCNfBPr8jJZc4YnZ4RV/+LNPYxN+Iybs Nm0g== X-Gm-Message-State: AC+VfDxs0qzdW7+2NfpXo57meKk3knw5oakNtbjNfU16AhBGNrHjVbFa UI5agY9qpmzYpdrbLq3sqhzMT1VAW1uqCt9yxl8= X-Received: by 2002:a17:902:d511:b0:1ab:7c4:eb24 with SMTP id b17-20020a170902d51100b001ab07c4eb24mr8129667plg.22.1684600028280; Sat, 20 May 2023 09:27:08 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:07 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 25/27] qemu/atomic128: Improve cmpxchg fallback for atomic16_set Date: Sat, 20 May 2023 09:26:32 -0700 Message-Id: <20230520162634.3991009-26-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::62c; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x62c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org Use __sync_bool_compare_and_swap_16 to control the loop, rather than a separate comparison. Signed-off-by: Richard Henderson --- host/include/generic/host/atomic128-ldst.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/host/include/generic/host/atomic128-ldst.h b/host/include/generic/host/atomic128-ldst.h index 79d208b7a4..80fff0643a 100644 --- a/host/include/generic/host/atomic128-ldst.h +++ b/host/include/generic/host/atomic128-ldst.h @@ -58,11 +58,14 @@ atomic16_read_rw(Int128 *ptr) static inline void ATTRIBUTE_ATOMIC128_OPT atomic16_set(Int128 *ptr, Int128 val) { - Int128 old = *ptr, cmp; + __int128_t *ptr_align = __builtin_assume_aligned(ptr, 16); + __int128_t old; + Int128Alias new; + + new.s = val; do { - cmp = old; - old = atomic16_cmpxchg(ptr, cmp, val); - } while (int128_ne(old, cmp)); + old = *ptr_align; + } while (!__sync_bool_compare_and_swap_16(ptr_align, old, new.i)); } #else From patchwork Sat May 20 16:26:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684318 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp448883wrt; Sat, 20 May 2023 09:31:25 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6V21mZmVOBd+z/n3nnfWsuom++nedDz1NapAhVwKggvXstRw9x5MtxpugXAMdxFO75v+xR X-Received: by 2002:a05:6214:d4b:b0:616:5c8b:582 with SMTP id 11-20020a0562140d4b00b006165c8b0582mr12175036qvr.37.1684600285250; Sat, 20 May 2023 09:31:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600285; cv=none; d=google.com; s=arc-20160816; b=V1yqQlzPZulCKdXdLQ0UUtE/hKMYvEOHLSpbvq0chGJoLKL7TmrJwJ/6+IdMDIOAED HPC3ynb9tdcqeAIOaXhB+dicbHtbZtOnFfR7iv1WRWZApCuf3y7Jke1HyRnybZdPs2k7 +lo27fdpl9ySo1ea4ukKi+tO8eq3+2SHYd76gBLfXkW329CyTN3TU8GrWk8cSkuGUa61 wa6u/5bgHRJ99hCS67V7cEWVgAYGr8eXYTahVZavQnqUpZnFD4iReh+8MnhSJ9fiQ0Ia b4r4ohfibBy91QBIu1SPKBz5Wv1p/XOCllMrGZ2dasxKGnHi2dnIEJc2v+vuQcCzLBTY AuIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=Ls+MmvQQFANFQAxZzSV9sTIr5Z8/+s6kpLEK2WXn+jw=; b=Zhh7WaUN9Z2VezXQg/3LOtz9vqj55J/l7mHsDnSIklW0G2Pc79Hw3gvUT2NeQNBZsy /W66I98NJ0ohnRx46//dN3rt5N0UQqeqV68aUyvwJHQ2EHTJX+0xRKoq0+9jcWZ6+R/w XEZZrQJGgK7z4P62lMx1DGnxMN/lNZmH09167GfZf3tprO3RK68IAUBPZiHyekcaSAmC jt9LrqS8V6jS6JzLRxm8ZlI+dQgd3HO+abu5cH5IEZCK7DUCbKJQ483Fvejr+y+fm5VR ZUzUGm9+wnU0lFrCmkbFrK1mjSKlx8z/DxYML/wOeCPwr4GM+f98X777/P8xgdSK8poM r46Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TXyXj9II; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id ke10-20020a056214300a00b005f2e72039b2si1166292qvb.565.2023.05.20.09.31.25 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:31:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=TXyXj9II; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQc-00042q-AS; Sat, 20 May 2023 12:27:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQJ-0003oa-W4 for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:16 -0400 Received: from mail-pl1-x636.google.com ([2607:f8b0:4864:20::636]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQE-00039N-Hl for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:15 -0400 Received: by mail-pl1-x636.google.com with SMTP id d9443c01a7336-1ae6dce19f7so20701925ad.3 for ; Sat, 20 May 2023 09:27:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600029; x=1687192029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ls+MmvQQFANFQAxZzSV9sTIr5Z8/+s6kpLEK2WXn+jw=; b=TXyXj9IID37hb3JfhyZe6THdJxh+CuYdwSOs+eZ0SmNTPqvgCK92fM0C9GHuIAgvpS +Z43yHQkErIDFAAhtcp82bYHyGMIwj6OIJpOq4ep0zKoNBP02o4RV8c5Emy/sgVt7BMp xYm9RF5+3wAmhSWlIfmA4CYb7RcSHpKQieVqJmJfzGND/SDuGA9mV+LOZ6Y7oCBaDw5u Mg+fwu6T3UFtc0uL9FiObYNfLhQqZE6l+ifeQ6GW5jXVz/Sff6NShHdLQVDrE7Rc3IMQ yK1B5OaKI1IuBZpvO9fqw44tzKEDSb2Hl5u4JIH6X4Dm8mq/5ptN0bVUFZsjjR47roe5 JeIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600029; x=1687192029; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ls+MmvQQFANFQAxZzSV9sTIr5Z8/+s6kpLEK2WXn+jw=; b=IAyuuMXYUv6XzxltQnrlW2iz1l1VE9mmp4VI/0geSGMq5S63wTAhFr7f/1I/rLM6KL 0/VgAxakoDlrJk55jbr7SBMVIDsNKjl1jKU2EgSniS4Rp4lu6KLV54NzaLv+u2Co1OXl 4dEhP+C+CrQ7+sCtGO6deHFimcGsvuExeczAGHe50HEtxsv/jF239dZ2kSCBqh/nmmMj dypdNui1dFhQNNz5wuLdP+E1iggRH83+e1/SOR/W2S6JjBtXH3g2Rew2E6ibEZorqHD6 SZgmjfop5VdVHR+fmoCIYonZskbRW/5T7VM8wN0YnN2k6jZ6ma11RexTZe3cwsSz8AuJ GY/w== X-Gm-Message-State: AC+VfDysFPMPK7hPafH/ksyc4XrcFygs0a4vv9mOSkdwG1t9FT8Md3Dz AP7b7dNNafVo23OAEzTtTPocRW9ReQP5v/OrTqs= X-Received: by 2002:a17:902:e80b:b0:1ae:62ed:9630 with SMTP id u11-20020a170902e80b00b001ae62ed9630mr7650150plg.15.1684600029290; Sat, 20 May 2023 09:27:09 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:08 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 26/27] qemu/atomic128: Add runtime test for FEAT_LSE2 Date: Sat, 20 May 2023 09:26:33 -0700 Message-Id: <20230520162634.3991009-27-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::636; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x636.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With FEAT_LSE2, load and store of int128 is directly supported. Signed-off-by: Richard Henderson --- host/include/aarch64/host/atomic128-ldst.h | 53 ++++++++++++++++------ 1 file changed, 40 insertions(+), 13 deletions(-) diff --git a/host/include/aarch64/host/atomic128-ldst.h b/host/include/aarch64/host/atomic128-ldst.h index 6959b2bd8e..57455c9b06 100644 --- a/host/include/aarch64/host/atomic128-ldst.h +++ b/host/include/aarch64/host/atomic128-ldst.h @@ -11,27 +11,48 @@ #ifndef AARCH64_ATOMIC128_LDST_H #define AARCH64_ATOMIC128_LDST_H +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + /* * Through gcc 10, aarch64 has no support for 128-bit atomics. * Through clang 16, without -march=armv8.4-a, __atomic_load_16 * is incorrectly expanded to a read-write operation. + * + * Anyway, this method allows runtime detection of FEAT_LSE2. */ -#define HAVE_ATOMIC128_RO 0 +#define HAVE_ATOMIC128_RO (cpuinfo & CPUINFO_LSE2) #define HAVE_ATOMIC128_RW 1 -Int128 QEMU_ERROR("unsupported atomic") atomic16_read_ro(const Int128 *ptr); +static inline Int128 atomic16_read_ro(const Int128 *ptr) +{ + uint64_t l, h; + + tcg_debug_assert(HAVE_ATOMIC128_RO); + /* With FEAT_LSE2, 16-byte aligned LDP is atomic. */ + asm("ldp %[l], %[h], %[mem]" + : [l] "=r"(l), [h] "=r"(h) : [mem] "m"(*ptr)); + + return int128_make128(l, h); +} static inline Int128 atomic16_read_rw(Int128 *ptr) { uint64_t l, h; uint32_t tmp; - /* The load must be paired with the store to guarantee not tearing. */ - asm("0: ldxp %[l], %[h], %[mem]\n\t" - "stxp %w[tmp], %[l], %[h], %[mem]\n\t" - "cbnz %w[tmp], 0b" - : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + if (cpuinfo & CPUINFO_LSE2) { + /* With FEAT_LSE2, 16-byte aligned LDP is atomic. */ + asm("ldp %[l], %[h], %[mem]" + : [l] "=r"(l), [h] "=r"(h) : [mem] "m"(*ptr)); + } else { + /* The load must be paired with the store to guarantee not tearing. */ + asm("0: ldxp %[l], %[h], %[mem]\n\t" + "stxp %w[tmp], %[l], %[h], %[mem]\n\t" + "cbnz %w[tmp], 0b" + : [mem] "+m"(*ptr), [tmp] "=r"(tmp), [l] "=r"(l), [h] "=r"(h)); + } return int128_make128(l, h); } @@ -41,12 +62,18 @@ static inline void atomic16_set(Int128 *ptr, Int128 val) uint64_t l = int128_getlo(val), h = int128_gethi(val); uint64_t t1, t2; - /* Load into temporaries to acquire the exclusive access lock. */ - asm("0: ldxp %[t1], %[t2], %[mem]\n\t" - "stxp %w[t1], %[l], %[h], %[mem]\n\t" - "cbnz %w[t1], 0b" - : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) - : [l] "r"(l), [h] "r"(h)); + if (cpuinfo & CPUINFO_LSE2) { + /* With FEAT_LSE2, 16-byte aligned STP is atomic. */ + asm("stp %[l], %[h], %[mem]" + : [mem] "=m"(*ptr) : [l] "r"(l), [h] "r"(h)); + } else { + /* Load into temporaries to acquire the exclusive access lock. */ + asm("0: ldxp %[t1], %[t2], %[mem]\n\t" + "stxp %w[t1], %[l], %[h], %[mem]\n\t" + "cbnz %w[t1], 0b" + : [mem] "+m"(*ptr), [t1] "=&r"(t1), [t2] "=&r"(t2) + : [l] "r"(l), [h] "r"(h)); + } } #endif /* AARCH64_ATOMIC128_LDST_H */ From patchwork Sat May 20 16:26:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 684300 Delivered-To: patch@linaro.org Received: by 2002:a5d:4e01:0:0:0:0:0 with SMTP id p1csp447968wrt; Sat, 20 May 2023 09:29:03 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ6JY7fiUV+yGHgoIG2K9WiEvDcrTTrnGvQKp3UQTERZ/S1idGwJH8+NAHZrvyT8RGjhHYUN X-Received: by 2002:ac8:5cca:0:b0:3f5:c257:9f17 with SMTP id s10-20020ac85cca000000b003f5c2579f17mr10117283qta.34.1684600143197; Sat, 20 May 2023 09:29:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684600143; cv=none; d=google.com; s=arc-20160816; b=EkcZGV/SlJNHPeCcB/1y3NOXhKXCzWJOhhgg8uk8S+xrbeAM/5vYlH3v/Rl0swcs0O Pgp1htlIDzNNHP+RhrcYSjq61uf9gMzwiLCQQI7hW2qh4B7qFxO0APl7Qj8Z0dHQZ4j9 vKY2cVP6Njl5gYvJDE6eYf7s69L1ghlQVHi7hSu8yfQtmMbCIH0vzQhWRNH+RKZx4Bdz IazTd8ti4ow8jWvAuK++Y6/oiUZQ8FEY/V4OEQ7SLXBhANbJV5jsoax9I3z6K2TtFHCs m+QC9Wbik2F4THnBj6IvFMuUocCNgPZrYinHa07VWsFp4GZeBWlA2Cy8rW//K0ubV4rC ureA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=eyyAeqhdvJDxVNJtszvSVJxM+9U7iQwcioqA1lOhcgjGO04tptvoaVktRjDZ6Iu5d4 S5LTCcb9oW0SSMgqEIm3FAE5KAr1LtW0n10QxYdh5WlHzqZKRLstTPwNO6SVnYJvsZZ6 fsfAS+tFH1t17wxDZStWe8ibkuZaZzM/IZiWhFNbpwWpKvOstm5aKAF+xxULVlWU5Gry hUBYh205eutkV6P9K9/tAlkxg7J3HlE9oeCElX2PiMHMFGLij/ssClNWXiwgZdrjGa/I +2wY8Cou0kYAMNoTDuxYSdr5PSkfCbWGlu9quW6NPrql5TBuoC2NEPEHEPvQO2Wyutvx UbgQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fa3ZDap4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w6-20020a05620a128600b0074ad42d7d2esi1049056qki.255.2023.05.20.09.29.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Sat, 20 May 2023 09:29:03 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=fa3ZDap4; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1q0PQo-0004Xp-3K; Sat, 20 May 2023 12:27:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q0PQK-0003p5-KS for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:16 -0400 Received: from mail-pl1-x631.google.com ([2607:f8b0:4864:20::631]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q0PQG-0003EU-Hu for qemu-devel@nongnu.org; Sat, 20 May 2023 12:27:16 -0400 Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1ae58e4b295so27664005ad.2 for ; Sat, 20 May 2023 09:27:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1684600030; x=1687192030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=fa3ZDap4SA2JE2/yoWHCOSK40WvRXSEpoCEObqBLDos563HTiYYqiHGXjrTP7kAAC0 Oh+YQwnP5n3se3wmgzjjZlWSl4WZaJlYO32M8B/7Uvf1EWmaVN5tVrmKKx0dCilW8uQM P9BD8/80ksebA5CmjtRWA2WhS4IDrXAqFkMo8kgfOVO96ckI85womNRfGn+0QJWKgfqF n6rQDveuB/wIZyxcIwfL9wKQDIVSe8Ixa1te28NICKwroR/sXSbkXVZT2ngrm800xAHd nQLeRLGijWRj0IMQK4cME/fK0i+p+q7OxfgbFlSKmQ5JtP6LV2voM9G/UXzlO235M2EP eHMA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684600030; x=1687192030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FvkUN9jgmDAOLN4anBgxZjSYCqoddaXAUfRdd1Denco=; b=NAM2on8RQoXJnIdOjZWLJQ2uDP9VDL02bD93jT3JpjJW3YYecxHoFzz3nBXDCS0+J7 lY8QZihTj8RJWc//DrO0kIzMoREnObFudw3/WuDfd9Djs+h+caLmmRBKgFdqRHlaRiL9 NqpM8CFVUVvQuld9fuXJMiCGV8l7VDpAkQV0DlXlKuBv4soTQblIJMnrCnN4eUydpEnl L/jtj5/xR+9+6CX1QmL3zDi3o/STc44sOiELz0galL+Ki3HmlVClTP1numZwowNVZxGI Zd7kZ/B92ZQIpnxFnfYIrJ7vxQpyqvPMajQDCxZCbAtRs8cG2RcV8CT7WJD/vncPmSvd DgnQ== X-Gm-Message-State: AC+VfDw0noJxkOEXHe3bNlqW9y/XDEntt36TFapUsfXQtSPF+++9UFqA k21inFEk3PxNxGErIXmP1MtE7KzvKDWdcNKEi0Y= X-Received: by 2002:a17:903:2310:b0:1ad:164:74fc with SMTP id d16-20020a170903231000b001ad016474fcmr8419787plh.20.1684600030124; Sat, 20 May 2023 09:27:10 -0700 (PDT) Received: from stoup.. ([2602:ae:1598:4c01:d078:d94e:cb2b:a055]) by smtp.gmail.com with ESMTPSA id d12-20020a170903230c00b001a5260a6e6csm1697104plh.206.2023.05.20.09.27.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 20 May 2023 09:27:09 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Cc: qemu-arm@nongnu.org Subject: [PATCH 27/27] qemu/atomic128: Add x86_64 atomic128-ldst.h Date: Sat, 20 May 2023 09:26:34 -0700 Message-Id: <20230520162634.3991009-28-richard.henderson@linaro.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230520162634.3991009-1-richard.henderson@linaro.org> References: <20230520162634.3991009-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::631; envelope-from=richard.henderson@linaro.org; helo=mail-pl1-x631.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: qemu-devel-bounces+patch=linaro.org@nongnu.org With CPUINFO_ATOMIC_VMOVDQA, we can perform proper atomic load/store without cmpxchg16b. Signed-off-by: Richard Henderson --- host/include/x86_64/host/atomic128-ldst.h | 54 +++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100644 host/include/x86_64/host/atomic128-ldst.h diff --git a/host/include/x86_64/host/atomic128-ldst.h b/host/include/x86_64/host/atomic128-ldst.h new file mode 100644 index 0000000000..4be9071d3f --- /dev/null +++ b/host/include/x86_64/host/atomic128-ldst.h @@ -0,0 +1,54 @@ +/* + * SPDX-License-Identifier: GPL-2.0-or-later + * Load/store for 128-bit atomic operations, x86_64 version. + * + * Copyright (C) 2023 Linaro, Ltd. + * + * See docs/devel/atomics.rst for discussion about the guarantees each + * atomic primitive is meant to provide. + */ + +#ifndef AARCH64_ATOMIC128_LDST_H +#define AARCH64_ATOMIC128_LDST_H + +#include "host/cpuinfo.h" +#include "tcg/debug-assert.h" + +#define HAVE_ATOMIC128_RO likely(cpuinfo & CPUINFO_ATOMIC_VMOVDQA) +#define HAVE_ATOMIC128_RW 1 + +static inline Int128 atomic16_read_ro(const Int128 *ptr) +{ + Int128Alias r; + + tcg_debug_assert(HAVE_ATOMIC128_RO); + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr)); + + return r.s; +} + +static inline Int128 atomic16_read_rw(Int128 *ptr) +{ + Int128Alias r; + + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=x" (r.i) : "m" (*ptr)); + } else { + r.i = __sync_val_compare_and_swap_16(ptr, 0, 0); + } + return r.s; +} + +static inline void atomic16_set(Int128 *ptr, Int128Alias val) +{ + if (HAVE_ATOMIC128_RO) { + asm("vmovdqa %1, %0" : "=m"(*ptr) : "x" (val.i)); + } else { + Int128Alias old; + do { + old.s = *ptr; + } while (!__sync_bool_compare_and_swap_16(ptr, old.i, val.i)); + } +} + +#endif /* AARCH64_ATOMIC128_LDST_H */