From patchwork Fri Apr 2 16:18:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 414382 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp1536334jai; Fri, 2 Apr 2021 09:19:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzPCJYwsL2D/MNkbWTOJwHAjRPoe1p0JF2xnAewzH1QDw3aTD56wU2YrKyaQ/c0kK3UeEZZ X-Received: by 2002:a05:6602:1406:: with SMTP id t6mr11700260iov.154.1617380382174; Fri, 02 Apr 2021 09:19:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1617380382; cv=none; d=google.com; s=arc-20160816; b=zRBar3LTSbum6Nf47pAJZdAfMdvPRhX8W4j6dx5l5D/JSbw19dddIHEGvy6XRA7lDK LjEgZOkzDXc6htXW6ZoxVgBPf6tMqONTchsDnBER0AJwsLDSH2MqHk6dTdr1ZXy0jM5d IENWw3bbEes8HyZiYcekoHEWKu+BnaPFryg3yS30LjUJVLbTdYAbop4/g+1jODFxbG7N XvJLfMcpxECaAC2RJ0KMKHfHy2KqdaX8pWsKb1y46BxOL9mf0YDftsBECGqh5Hghvck9 ReNW88ts3AvoFws0k1jE7wphnXpbiwGghn+S4d+kXsHIC4gpyIBZ+tVcUJfuWSFIFTiC 3YuA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=fmCgzHtzk5qDGe2eGqNAc4vF9R/bc/UzkhnNYf8s1aw=; b=SZ5HPPgjkDw27Dw2LWHI/P4pGITObBjfJShXSK14U5/q0eI+whJ7Yy+F3who4840OY kPspIHd60f7POj+6utrtYwnhu5ifwSRI/Kyp4Bk3vFapxEyIe7wUE8EneOvvCuydLMVc 5SFaeAduBB2cGEt5Mh72FZqnaP24LFqoBWWk36Os8+V/zygqUwvVGhb7d0faRgm32Oio /kDPEPTBj4ht9eApE3ZUc8S/zKgbHmZ8b/kvau9KvJl39oKpPVG9ftHvNH2jTojKdp2Q IeyLQEmVnH8z0oCDtIKRqh5Z7y7VrMvl/yM+BdPBFMNl92zxKyap5uljw24l9pXRBvtE 3L6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="l9y2N/FQ"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id w4si8153507ill.114.2021.04.02.09.19.42 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Apr 2021 09:19:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b="l9y2N/FQ"; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:45606 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSMWL-0001Ry-Eh for patch@linaro.org; Fri, 02 Apr 2021 12:19:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50954) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSMVQ-0000vC-CY for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:44 -0400 Received: from mail-pj1-x102c.google.com ([2607:f8b0:4864:20::102c]:34348) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lSMVN-00035N-1Q for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:44 -0400 Received: by mail-pj1-x102c.google.com with SMTP id cl21-20020a17090af695b02900c61ac0f0e9so6368396pjb.1 for ; Fri, 02 Apr 2021 09:18:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=fmCgzHtzk5qDGe2eGqNAc4vF9R/bc/UzkhnNYf8s1aw=; b=l9y2N/FQMRgq1j+ah+xdquCl5SSxrHBiED0OUHza/FscU/Lp7cdzoLyMh73W+tzlDx mjD9L/l8cxDVuZiTEDoHux51ozT6oCdK8M4UZZVG8/GfQkteGDmfxDtIGfTh0s//Xj5e sEJBi8VJ1rbtBg3NkAM/yLYb74i2s5m0ZAaT7a/1PU1V/5Oscu1N2bTSHkobcvdUt2GZ VWU+fWQmtAwTXtvWTId2zfXPTJfivQNQsmppzqCAs5o0vKGgxhXtZSKui4AMVVCwVMOb +kSmlnGVLvoD0CWWo52c9ZuPhfV3W5qtlt9Dpkf3kUIN54ZqxZvwHLIHxONubLH5A+Xk tZzg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=fmCgzHtzk5qDGe2eGqNAc4vF9R/bc/UzkhnNYf8s1aw=; b=k4WbTCaEGxo0n/P6R7kFsQXMY+XMwrnt9lTg6qOZEymRctDJVdBWIITh3n9gfli41c PCTcV3zkjKOswrLewMCloAk6HHAl4LoMr5dV+VwaClGEuQauVaDR+2ze+wlnvmdMNSAJ xWy1koCTpBXFlW+BX6TVSCJrH19xZHfy0unVegvvX8z8ABCZtZOdGl8ofm1KvB6gXX6S qZY5bWoML6M36i8K5ePXH1e4cqEjderYX9adHifjaB0G93+PKpTG6wxEXE/EmF7oB0tX kE3faCS6IEvJ264WocflqoGZYDlfGLQURmCQwpMMEoeQGk96BWcRZigiGh8Tr3RLZnQa wNwQ== X-Gm-Message-State: AOAM5305BiUxXP/EYcE8+SebeTfOwIHmQVRUqz0zO+JT1gblZh6A6trq RhoaARxvSbl6o5SMXbl+exwtxhwBZ5Lghg== X-Received: by 2002:a17:90a:5910:: with SMTP id k16mr14965471pji.207.1617380318663; Fri, 02 Apr 2021 09:18:38 -0700 (PDT) Received: from localhost.localdomain (h216-228-167-147.bendor.dedicated.static.tds.net. [216.228.167.147]) by smtp.gmail.com with ESMTPSA id m7sm8821761pjc.54.2021.04.02.09.18.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Apr 2021 09:18:38 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 1/3] target/arm: Check PAGE_WRITE_ORG for MTE writeability Date: Fri, 2 Apr 2021 09:18:33 -0700 Message-Id: <20210402161835.286665-2-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210402161835.286665-1-richard.henderson@linaro.org> References: <20210402161835.286665-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102c; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102c.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We can remove PAGE_WRITE when (internally) marking a page read-only because it contains translated code. This can be triggered by tests/tcg/aarch64/bti-2, after having serviced SIGILL trampolines on the stack. Signed-off-by: Richard Henderson --- target/arm/mte_helper.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.25.1 diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 0bbb9ec346..8be17e1b70 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -83,7 +83,7 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx, uint8_t *tags; uintptr_t index; - if (!(flags & (ptr_access == MMU_DATA_STORE ? PAGE_WRITE : PAGE_READ))) { + if (!(flags & (ptr_access == MMU_DATA_STORE ? PAGE_WRITE_ORG : PAGE_READ))) { /* SIGSEGV */ arm_cpu_tlb_fill(env_cpu(env), ptr, ptr_size, ptr_access, ptr_mmu_idx, false, ra); From patchwork Fri Apr 2 16:18:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 414381 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp1535890jai; Fri, 2 Apr 2021 09:19:05 -0700 (PDT) X-Google-Smtp-Source: ABdhPJz2zWapR0fNQQG+61dAKsHGqFemiEaoIUMenJz9a2F9685y/e1MVjRkDHOjRug8XedkcNXV X-Received: by 2002:a02:6989:: with SMTP id e131mr13655657jac.105.1617380345725; Fri, 02 Apr 2021 09:19:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1617380345; cv=none; d=google.com; s=arc-20160816; b=LJLYKdNa82G8+oPYXSgRihZ7cDKuSggXqC8a+HoTftTB9kiZPQYzXCN1I50vuiY+b2 BkRtKE0AMS/YXLzs+e2hCh85FIwXIDAqpp6Dvnbet5qM1RtSlgJtwGateEdhqVtPpl7d 0Zsu0kZ6lqiRhKzC78P3kNIcZy5yAURbkDPVQV8m5dMdYvaa/PjBbKVipsoaYw8PJM5a 9MbXjQuc9hh7peRlWKOy+Ws8/uBbGfI3LjEauamcf9kB5v02G29tVnB2L0oOWnqzNT9W zVJowj9xd5mWlKOHGsqtiFJtuwmA+JjzD+4u10MC0me2zHiCNgPCbmR+i5MKDklPw6R/ Cj+g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=IKepXQzwJaF4zrXZKA428tFFtkd3HFANGqNk/XwK+Aw=; b=dWY+3yFQ3cpLlv4iFueCtZZ44Yla5zimWOOlIhKQuCBJ9aM4OCeoHG+CbjAld5C2MN 1Fnll7hDJHQ65FlrwiODW71IiVtfPK7YSYs50Hr8T4Z8BrD0DlPrXxj1OJ9/3MxScBtJ rHzbMNkp8q20PwT5ZLYCotZOYiBvbw381xs2hkZtYsNOovB2oBHkeOUzflhatv0ypwzO dIrTZeusPGgfqmJ60IPGXkVycxlzHc6RIQ+WP0cpZN1VqLJgjqEuUs+Dweedj1Xle7y3 po1njyXVl+Vz3Z+itqwKGobrM1w0XnYWdyODRTPyIRAepyyzUhQxHfqN+4D5EFJqkqEe 2f7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tMSgWrHZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id z8si9251662jan.13.2021.04.02.09.19.05 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Apr 2021 09:19:05 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=tMSgWrHZ; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44814 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSMVl-00018P-52 for patch@linaro.org; Fri, 02 Apr 2021 12:19:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:51024) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSMVT-00010f-4j for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:47 -0400 Received: from mail-pj1-x102d.google.com ([2607:f8b0:4864:20::102d]:51107) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lSMVN-00036k-Ew for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:46 -0400 Received: by mail-pj1-x102d.google.com with SMTP id gb6so2933869pjb.0 for ; Fri, 02 Apr 2021 09:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=IKepXQzwJaF4zrXZKA428tFFtkd3HFANGqNk/XwK+Aw=; b=tMSgWrHZefbytCxxstE4w2I7g0E1AC0GriTUcR2ILiOaPDJV2rVgxgof24XxpGCWSE m4fagTJbmAxX1mIXISEYFDtPwhY1eqEBEE5kqJQ+hh+oxW8sbRqx/bpMTREModDl1dTY 5n7mRxfxUj2HsjGaQnaqcOSUBc5QhF0cVgHVNGvCOBrM7tYSi2PDA6yWssqjm26T6M2A vqazKm1AVBYbSkvTbqUZkayiCCQl31K65eilK3XRxBfmOkN926E8+DXkw6Wreu8h5tFt e/HEod7B/dYUpExZvoS9W55pJJoV+i2s2eR7eJqQZO1pW/Ouj26tjFFc4P9Qf4L29PWk HpoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=IKepXQzwJaF4zrXZKA428tFFtkd3HFANGqNk/XwK+Aw=; b=QSSnMHVaDdTl9cpGUot2WA6bNwh/gPE+iCgLRKSGfhNfciY4M7ULQzlN5KGyzxBQn+ FHb8jexqh2iPSBGkdPHruxxoCrbY53q/oHllLf7T/RUWoJw0B/RRsgpnsuK9PN/qjwq6 cN4e9JmoDUp468hAKGqSQzIqcw9QlgdjriHTjtxoqW2Sv1H0YohBBGJIYyOjI56uMnbO VOavGoDBFM+JJcdSCr6sCyzT83nzPeO0EIjhbillfHn3YoW5as2VRSP/1I13YP/yvBAt 8MGMgRKR8S1fOXBTwUcnuEVs59iqFeOpdJq4V2FEwT1WZKveDdOyKe9+x4XYgMwlRZ3X 12BA== X-Gm-Message-State: AOAM530d0BQ4IVrzT1PIup0fpIOvazqR0lXc0U0NPB29XQdGdLmKqKo4 pbahqJZbUxx9pSWd+jWHsEEpSmQWX9WKJA== X-Received: by 2002:a17:90b:4b8c:: with SMTP id lr12mr14843085pjb.124.1617380319787; Fri, 02 Apr 2021 09:18:39 -0700 (PDT) Received: from localhost.localdomain (h216-228-167-147.bendor.dedicated.static.tds.net. [216.228.167.147]) by smtp.gmail.com with ESMTPSA id m7sm8821761pjc.54.2021.04.02.09.18.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Apr 2021 09:18:39 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 2/3] target/arm: Fix unaligned mte checks Date: Fri, 2 Apr 2021 09:18:34 -0700 Message-Id: <20210402161835.286665-3-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210402161835.286665-1-richard.henderson@linaro.org> References: <20210402161835.286665-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::102d; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x102d.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We were incorrectly assuming that only the first byte of an MTE access is checked against the tags. But per the ARM, unaligned accesses are pre-decomposed into single-byte accesses. So by the time we reach the actual MTE check in the ARM pseudocode, all accesses are aligned. Therefore, drop mte_check1, since we cannot know a priori that an access is aligned. Rename mte_checkN to mte_check, which now handles all accesses. Rename mte_probe1 to mte_probe, and use a common helper. Drop the computation of the faulting nth element, since all accesses can be considered to devolve to bytes, and simply compute the faulting address. Buglink: https://bugs.launchpad.net/bugs/1921948 Signed-off-by: Richard Henderson --- target/arm/helper-a64.h | 3 +- target/arm/internals.h | 13 +-- target/arm/translate-a64.h | 2 +- target/arm/mte_helper.c | 169 ++++++++++++------------------ target/arm/sve_helper.c | 96 ++++++----------- target/arm/translate-a64.c | 31 +++--- target/arm/translate-sve.c | 9 +- tests/tcg/aarch64/mte-5.c | 44 ++++++++ tests/tcg/aarch64/Makefile.target | 2 +- 9 files changed, 169 insertions(+), 200 deletions(-) create mode 100644 tests/tcg/aarch64/mte-5.c -- 2.25.1 diff --git a/target/arm/helper-a64.h b/target/arm/helper-a64.h index c139fa81f9..7b706571bb 100644 --- a/target/arm/helper-a64.h +++ b/target/arm/helper-a64.h @@ -104,8 +104,7 @@ DEF_HELPER_FLAGS_3(autdb, TCG_CALL_NO_WG, i64, env, i64, i64) DEF_HELPER_FLAGS_2(xpaci, TCG_CALL_NO_RWG_SE, i64, env, i64) DEF_HELPER_FLAGS_2(xpacd, TCG_CALL_NO_RWG_SE, i64, env, i64) -DEF_HELPER_FLAGS_3(mte_check1, TCG_CALL_NO_WG, i64, env, i32, i64) -DEF_HELPER_FLAGS_3(mte_checkN, TCG_CALL_NO_WG, i64, env, i32, i64) +DEF_HELPER_FLAGS_3(mte_check, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(mte_check_zva, TCG_CALL_NO_WG, i64, env, i32, i64) DEF_HELPER_FLAGS_3(irg, TCG_CALL_NO_RWG, i64, env, i64, i64) DEF_HELPER_FLAGS_4(addsubg, TCG_CALL_NO_RWG_SE, i64, env, i64, s32, i32) diff --git a/target/arm/internals.h b/target/arm/internals.h index f11bd32696..817d3aa51b 100644 --- a/target/arm/internals.h +++ b/target/arm/internals.h @@ -1137,19 +1137,16 @@ FIELD(PREDDESC, DATA, 8, 24) */ #define SVE_MTEDESC_SHIFT 5 -/* Bits within a descriptor passed to the helper_mte_check* functions. */ +/* Bits within a descriptor passed to the helper_mte_check function. */ FIELD(MTEDESC, MIDX, 0, 4) FIELD(MTEDESC, TBI, 4, 2) FIELD(MTEDESC, TCMA, 6, 2) FIELD(MTEDESC, WRITE, 8, 1) -FIELD(MTEDESC, ESIZE, 9, 5) -FIELD(MTEDESC, TSIZE, 14, 10) /* mte_checkN only */ +FIELD(MTEDESC, SIZEM1, 12, 10) /* size - 1 */ -bool mte_probe1(CPUARMState *env, uint32_t desc, uint64_t ptr); -uint64_t mte_check1(CPUARMState *env, uint32_t desc, - uint64_t ptr, uintptr_t ra); -uint64_t mte_checkN(CPUARMState *env, uint32_t desc, - uint64_t ptr, uintptr_t ra); +bool mte_probe(CPUARMState *env, uint32_t desc, uint64_t ptr); +uint64_t mte_check(CPUARMState *env, uint32_t desc, + uint64_t ptr, uintptr_t ra); static inline int allocation_tag_from_addr(uint64_t ptr) { diff --git a/target/arm/translate-a64.h b/target/arm/translate-a64.h index 3668b671dd..6c4bbf9096 100644 --- a/target/arm/translate-a64.h +++ b/target/arm/translate-a64.h @@ -44,7 +44,7 @@ TCGv_i64 clean_data_tbi(DisasContext *s, TCGv_i64 addr); TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, bool tag_checked, int log2_size); TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, - bool tag_checked, int count, int log2_esize); + bool tag_checked, int total_size); /* We should have at some point before trying to access an FP register * done the necessary access check, so assert that diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c index 8be17e1b70..62bea7ad4a 100644 --- a/target/arm/mte_helper.c +++ b/target/arm/mte_helper.c @@ -121,7 +121,7 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx, * exception for inaccessible pages, and resolves the virtual address * into the softmmu tlb. * - * When RA == 0, this is for mte_probe1. The page is expected to be + * When RA == 0, this is for mte_probe. The page is expected to be * valid. Indicate to probe_access_flags no-fault, then assert that * we received a valid page. */ @@ -617,80 +617,6 @@ static void mte_check_fail(CPUARMState *env, uint32_t desc, } } -/* - * Perform an MTE checked access for a single logical or atomic access. - */ -static bool mte_probe1_int(CPUARMState *env, uint32_t desc, uint64_t ptr, - uintptr_t ra, int bit55) -{ - int mem_tag, mmu_idx, ptr_tag, size; - MMUAccessType type; - uint8_t *mem; - - ptr_tag = allocation_tag_from_addr(ptr); - - if (tcma_check(desc, bit55, ptr_tag)) { - return true; - } - - mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); - type = FIELD_EX32(desc, MTEDESC, WRITE) ? MMU_DATA_STORE : MMU_DATA_LOAD; - size = FIELD_EX32(desc, MTEDESC, ESIZE); - - mem = allocation_tag_mem(env, mmu_idx, ptr, type, size, - MMU_DATA_LOAD, 1, ra); - if (!mem) { - return true; - } - - mem_tag = load_tag1(ptr, mem); - return ptr_tag == mem_tag; -} - -/* - * No-fault version of mte_check1, to be used by SVE for MemSingleNF. - * Returns false if the access is Checked and the check failed. This - * is only intended to probe the tag -- the validity of the page must - * be checked beforehand. - */ -bool mte_probe1(CPUARMState *env, uint32_t desc, uint64_t ptr) -{ - int bit55 = extract64(ptr, 55, 1); - - /* If TBI is disabled, the access is unchecked. */ - if (unlikely(!tbi_check(desc, bit55))) { - return true; - } - - return mte_probe1_int(env, desc, ptr, 0, bit55); -} - -uint64_t mte_check1(CPUARMState *env, uint32_t desc, - uint64_t ptr, uintptr_t ra) -{ - int bit55 = extract64(ptr, 55, 1); - - /* If TBI is disabled, the access is unchecked, and ptr is not dirty. */ - if (unlikely(!tbi_check(desc, bit55))) { - return ptr; - } - - if (unlikely(!mte_probe1_int(env, desc, ptr, ra, bit55))) { - mte_check_fail(env, desc, ptr, ra); - } - - return useronly_clean_ptr(ptr); -} - -uint64_t HELPER(mte_check1)(CPUARMState *env, uint32_t desc, uint64_t ptr) -{ - return mte_check1(env, desc, ptr, GETPC()); -} - -/* - * Perform an MTE checked access for multiple logical accesses. - */ - /** * checkN: * @tag: tag memory to test @@ -753,38 +679,49 @@ static int checkN(uint8_t *mem, int odd, int cmp, int count) return n; } -uint64_t mte_checkN(CPUARMState *env, uint32_t desc, - uint64_t ptr, uintptr_t ra) +/* + * mte_check_int: + * @env: CPU environment + * @desc: MTEDESC descriptor + * @ptr: virtual address of the base of the access + * @fault: return virtual address of the first check failure + * + * Internal routine for both mte_probe and mte_check. + * Return zero on failure, filling in *fault. + * Return negative on trivial success for tbi disabled. + * Return positive on success with tbi enabled. + */ +static int mte_check_int(CPUARMState *env, uint32_t desc, + uint64_t ptr, uintptr_t ra, uint64_t *fault) { int mmu_idx, ptr_tag, bit55; - uint64_t ptr_last, ptr_end, prev_page, next_page; + uint64_t ptr_last, prev_page, next_page; uint64_t tag_first, tag_end; uint64_t tag_byte_first, tag_byte_end; - uint32_t esize, total, tag_count, tag_size, n, c; + uint32_t sizem1, tag_count, tag_size, n, c; uint8_t *mem1, *mem2; MMUAccessType type; bit55 = extract64(ptr, 55, 1); + *fault = ptr; /* If TBI is disabled, the access is unchecked, and ptr is not dirty. */ if (unlikely(!tbi_check(desc, bit55))) { - return ptr; + return -1; } ptr_tag = allocation_tag_from_addr(ptr); if (tcma_check(desc, bit55, ptr_tag)) { - goto done; + return 1; } mmu_idx = FIELD_EX32(desc, MTEDESC, MIDX); type = FIELD_EX32(desc, MTEDESC, WRITE) ? MMU_DATA_STORE : MMU_DATA_LOAD; - esize = FIELD_EX32(desc, MTEDESC, ESIZE); - total = FIELD_EX32(desc, MTEDESC, TSIZE); + sizem1 = FIELD_EX32(desc, MTEDESC, SIZEM1); - /* Find the addr of the end of the access, and of the last element. */ - ptr_end = ptr + total; - ptr_last = ptr_end - esize; + /* Find the addr of the last byte of the access. */ + ptr_last = ptr + sizem1; /* Round the bounds to the tag granule, and compute the number of tags. */ tag_first = QEMU_ALIGN_DOWN(ptr, TAG_GRANULE); @@ -802,12 +739,19 @@ uint64_t mte_checkN(CPUARMState *env, uint32_t desc, if (likely(tag_end - prev_page <= TARGET_PAGE_SIZE)) { /* Memory access stays on one page. */ tag_size = (tag_byte_end - tag_byte_first) / (2 * TAG_GRANULE); - mem1 = allocation_tag_mem(env, mmu_idx, ptr, type, total, + mem1 = allocation_tag_mem(env, mmu_idx, ptr, type, sizem1 + 1, MMU_DATA_LOAD, tag_size, ra); if (!mem1) { - goto done; + return 1; + } + + /* + * Perform all of the comparisons, recognizing that most are + * aligned operations that do not cross granule boundaries. + */ + if (likely(tag_count == 1)) { + return ptr_tag == load_tag1(ptr, mem1); } - /* Perform all of the comparisons. */ n = checkN(mem1, ptr & TAG_GRANULE, ptr_tag, tag_count); } else { /* Memory access crosses to next page. */ @@ -817,7 +761,7 @@ uint64_t mte_checkN(CPUARMState *env, uint32_t desc, tag_size = (tag_byte_end - next_page) / (2 * TAG_GRANULE); mem2 = allocation_tag_mem(env, mmu_idx, next_page, type, - ptr_end - next_page, + ptr_last - next_page + 1, MMU_DATA_LOAD, tag_size, ra); /* @@ -831,31 +775,54 @@ uint64_t mte_checkN(CPUARMState *env, uint32_t desc, } if (n == c) { if (!mem2) { - goto done; + return 1; } n += checkN(mem2, 0, ptr_tag, tag_count - c); } } /* - * If we failed, we know which granule. Compute the element that - * is first in that granule, and signal failure on that element. + * If we failed, we know which granule -- signal failure at that address. */ if (unlikely(n < tag_count)) { - uint64_t fail_ofs; - - fail_ofs = tag_first + n * TAG_GRANULE - ptr; - fail_ofs = ROUND_UP(fail_ofs, esize); - mte_check_fail(env, desc, ptr + fail_ofs, ra); + if (n > 0) { + *fault = tag_first + n * TAG_GRANULE; + } + return 0; } - done: + return 1; +} + +/* + * No-fault probe, to be used by SVE for MemSingleNF. + * Returns false if the access is Checked and the check failed. This + * is only intended to probe the tag -- the validity of the page must + * be checked beforehand. + */ +bool mte_probe(CPUARMState *env, uint32_t desc, uint64_t ptr) +{ + uint64_t discard; + return mte_check_int(env, desc, ptr, 0, &discard) != 0; +} + +uint64_t mte_check(CPUARMState *env, uint32_t desc, + uint64_t ptr, uintptr_t ra) +{ + uint64_t fault; + int ret = mte_check_int(env, desc, ptr, ra, &fault); + + if (unlikely(ret == 0)) { + mte_check_fail(env, desc, fault, ra); + } else if (ret < 0) { + return ptr; + } return useronly_clean_ptr(ptr); } -uint64_t HELPER(mte_checkN)(CPUARMState *env, uint32_t desc, uint64_t ptr) +uint64_t HELPER(mte_check)(CPUARMState *env, uint32_t desc, uint64_t ptr) { - return mte_checkN(env, desc, ptr, GETPC()); + return mte_check(env, desc, ptr, GETPC()); } /* diff --git a/target/arm/sve_helper.c b/target/arm/sve_helper.c index fd6c58f96a..9382ece660 100644 --- a/target/arm/sve_helper.c +++ b/target/arm/sve_helper.c @@ -4382,13 +4382,9 @@ static void sve_cont_ldst_watchpoints(SVEContLdSt *info, CPUARMState *env, #endif } -typedef uint64_t mte_check_fn(CPUARMState *, uint32_t, uint64_t, uintptr_t); - -static inline QEMU_ALWAYS_INLINE -void sve_cont_ldst_mte_check_int(SVEContLdSt *info, CPUARMState *env, - uint64_t *vg, target_ulong addr, int esize, - int msize, uint32_t mtedesc, uintptr_t ra, - mte_check_fn *check) +static void sve_cont_ldst_mte_check(SVEContLdSt *info, CPUARMState *env, + uint64_t *vg, target_ulong addr, int esize, + int msize, uint32_t mtedesc, uintptr_t ra) { intptr_t mem_off, reg_off, reg_last; @@ -4405,7 +4401,7 @@ void sve_cont_ldst_mte_check_int(SVEContLdSt *info, CPUARMState *env, uint64_t pg = vg[reg_off >> 6]; do { if ((pg >> (reg_off & 63)) & 1) { - check(env, mtedesc, addr, ra); + mte_check(env, mtedesc, addr, ra); } reg_off += esize; mem_off += msize; @@ -4422,7 +4418,7 @@ void sve_cont_ldst_mte_check_int(SVEContLdSt *info, CPUARMState *env, uint64_t pg = vg[reg_off >> 6]; do { if ((pg >> (reg_off & 63)) & 1) { - check(env, mtedesc, addr, ra); + mte_check(env, mtedesc, addr, ra); } reg_off += esize; mem_off += msize; @@ -4431,30 +4427,6 @@ void sve_cont_ldst_mte_check_int(SVEContLdSt *info, CPUARMState *env, } } -typedef void sve_cont_ldst_mte_check_fn(SVEContLdSt *info, CPUARMState *env, - uint64_t *vg, target_ulong addr, - int esize, int msize, uint32_t mtedesc, - uintptr_t ra); - -static void sve_cont_ldst_mte_check1(SVEContLdSt *info, CPUARMState *env, - uint64_t *vg, target_ulong addr, - int esize, int msize, uint32_t mtedesc, - uintptr_t ra) -{ - sve_cont_ldst_mte_check_int(info, env, vg, addr, esize, msize, - mtedesc, ra, mte_check1); -} - -static void sve_cont_ldst_mte_checkN(SVEContLdSt *info, CPUARMState *env, - uint64_t *vg, target_ulong addr, - int esize, int msize, uint32_t mtedesc, - uintptr_t ra) -{ - sve_cont_ldst_mte_check_int(info, env, vg, addr, esize, msize, - mtedesc, ra, mte_checkN); -} - - /* * Common helper for all contiguous 1,2,3,4-register predicated stores. */ @@ -4463,8 +4435,7 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, const int N, uint32_t mtedesc, sve_ldst1_host_fn *host_fn, - sve_ldst1_tlb_fn *tlb_fn, - sve_cont_ldst_mte_check_fn *mte_check_fn) + sve_ldst1_tlb_fn *tlb_fn) { const unsigned rd = simd_data(desc); const intptr_t reg_max = simd_oprsz(desc); @@ -4493,9 +4464,9 @@ void sve_ldN_r(CPUARMState *env, uint64_t *vg, const target_ulong addr, * Handle mte checks for all active elements. * Since TBI must be set for MTE, !mtedesc => !mte_active. */ - if (mte_check_fn && mtedesc) { - mte_check_fn(&info, env, vg, addr, 1 << esz, N << msz, - mtedesc, retaddr); + if (mtedesc) { + sve_cont_ldst_mte_check(&info, env, vg, addr, 1 << esz, N << msz, + mtedesc, retaddr); } flags = info.page[0].flags | info.page[1].flags; @@ -4621,8 +4592,7 @@ void sve_ldN_r_mte(CPUARMState *env, uint64_t *vg, target_ulong addr, mtedesc = 0; } - sve_ldN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn, - N == 1 ? sve_cont_ldst_mte_check1 : sve_cont_ldst_mte_checkN); + sve_ldN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn); } #define DO_LD1_1(NAME, ESZ) \ @@ -4630,7 +4600,7 @@ void HELPER(sve_##NAME##_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, 1, 0, \ - sve_##NAME##_host, sve_##NAME##_tlb, NULL); \ + sve_##NAME##_host, sve_##NAME##_tlb); \ } \ void HELPER(sve_##NAME##_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -4644,13 +4614,13 @@ void HELPER(sve_##NAME##_le_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, 0, \ - sve_##NAME##_le_host, sve_##NAME##_le_tlb, NULL); \ + sve_##NAME##_le_host, sve_##NAME##_le_tlb); \ } \ void HELPER(sve_##NAME##_be_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, 1, 0, \ - sve_##NAME##_be_host, sve_##NAME##_be_tlb, NULL); \ + sve_##NAME##_be_host, sve_##NAME##_be_tlb); \ } \ void HELPER(sve_##NAME##_le_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -4693,7 +4663,7 @@ void HELPER(sve_ld##N##bb_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), MO_8, MO_8, N, 0, \ - sve_ld1bb_host, sve_ld1bb_tlb, NULL); \ + sve_ld1bb_host, sve_ld1bb_tlb); \ } \ void HELPER(sve_ld##N##bb_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -4707,13 +4677,13 @@ void HELPER(sve_ld##N##SUFF##_le_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, 0, \ - sve_ld1##SUFF##_le_host, sve_ld1##SUFF##_le_tlb, NULL); \ + sve_ld1##SUFF##_le_host, sve_ld1##SUFF##_le_tlb); \ } \ void HELPER(sve_ld##N##SUFF##_be_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_ldN_r(env, vg, addr, desc, GETPC(), ESZ, ESZ, N, 0, \ - sve_ld1##SUFF##_be_host, sve_ld1##SUFF##_be_tlb, NULL); \ + sve_ld1##SUFF##_be_host, sve_ld1##SUFF##_be_tlb); \ } \ void HELPER(sve_ld##N##SUFF##_le_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -4826,7 +4796,7 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, if (fault == FAULT_FIRST) { /* Trapping mte check for the first-fault element. */ if (mtedesc) { - mte_check1(env, mtedesc, addr + mem_off, retaddr); + mte_check(env, mtedesc, addr + mem_off, retaddr); } /* @@ -4869,7 +4839,7 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, /* Watchpoint hit, see below. */ goto do_fault; } - if (mtedesc && !mte_probe1(env, mtedesc, addr + mem_off)) { + if (mtedesc && !mte_probe(env, mtedesc, addr + mem_off)) { goto do_fault; } /* @@ -4919,7 +4889,7 @@ void sve_ldnfff1_r(CPUARMState *env, void *vg, const target_ulong addr, & BP_MEM_READ)) { goto do_fault; } - if (mtedesc && !mte_probe1(env, mtedesc, addr + mem_off)) { + if (mtedesc && !mte_probe(env, mtedesc, addr + mem_off)) { goto do_fault; } host_fn(vd, reg_off, host + mem_off); @@ -5090,8 +5060,7 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, uint32_t desc, const uintptr_t retaddr, const int esz, const int msz, const int N, uint32_t mtedesc, sve_ldst1_host_fn *host_fn, - sve_ldst1_tlb_fn *tlb_fn, - sve_cont_ldst_mte_check_fn *mte_check_fn) + sve_ldst1_tlb_fn *tlb_fn) { const unsigned rd = simd_data(desc); const intptr_t reg_max = simd_oprsz(desc); @@ -5117,9 +5086,9 @@ void sve_stN_r(CPUARMState *env, uint64_t *vg, target_ulong addr, * Handle mte checks for all active elements. * Since TBI must be set for MTE, !mtedesc => !mte_active. */ - if (mte_check_fn && mtedesc) { - mte_check_fn(&info, env, vg, addr, 1 << esz, N << msz, - mtedesc, retaddr); + if (mtedesc) { + sve_cont_ldst_mte_check(&info, env, vg, addr, 1 << esz, N << msz, + mtedesc, retaddr); } flags = info.page[0].flags | info.page[1].flags; @@ -5233,8 +5202,7 @@ void sve_stN_r_mte(CPUARMState *env, uint64_t *vg, target_ulong addr, mtedesc = 0; } - sve_stN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn, - N == 1 ? sve_cont_ldst_mte_check1 : sve_cont_ldst_mte_checkN); + sve_stN_r(env, vg, addr, desc, ra, esz, msz, N, mtedesc, host_fn, tlb_fn); } #define DO_STN_1(N, NAME, ESZ) \ @@ -5242,7 +5210,7 @@ void HELPER(sve_st##N##NAME##_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MO_8, N, 0, \ - sve_st1##NAME##_host, sve_st1##NAME##_tlb, NULL); \ + sve_st1##NAME##_host, sve_st1##NAME##_tlb); \ } \ void HELPER(sve_st##N##NAME##_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -5256,13 +5224,13 @@ void HELPER(sve_st##N##NAME##_le_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, 0, \ - sve_st1##NAME##_le_host, sve_st1##NAME##_le_tlb, NULL); \ + sve_st1##NAME##_le_host, sve_st1##NAME##_le_tlb); \ } \ void HELPER(sve_st##N##NAME##_be_r)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ { \ sve_stN_r(env, vg, addr, desc, GETPC(), ESZ, MSZ, N, 0, \ - sve_st1##NAME##_be_host, sve_st1##NAME##_be_tlb, NULL); \ + sve_st1##NAME##_be_host, sve_st1##NAME##_be_tlb); \ } \ void HELPER(sve_st##N##NAME##_le_r_mte)(CPUARMState *env, void *vg, \ target_ulong addr, uint32_t desc) \ @@ -5373,7 +5341,7 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, info.attrs, BP_MEM_READ, retaddr); } if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) { - mte_check1(env, mtedesc, addr, retaddr); + mte_check(env, mtedesc, addr, retaddr); } host_fn(&scratch, reg_off, info.host); } else { @@ -5386,7 +5354,7 @@ void sve_ld1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, BP_MEM_READ, retaddr); } if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) { - mte_check1(env, mtedesc, addr, retaddr); + mte_check(env, mtedesc, addr, retaddr); } tlb_fn(env, &scratch, reg_off, addr, retaddr); } @@ -5552,7 +5520,7 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, */ addr = base + (off_fn(vm, reg_off) << scale); if (mtedesc) { - mte_check1(env, mtedesc, addr, retaddr); + mte_check(env, mtedesc, addr, retaddr); } tlb_fn(env, vd, reg_off, addr, retaddr); @@ -5588,7 +5556,7 @@ void sve_ldff1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, } if (mtedesc && arm_tlb_mte_tagged(&info.attrs) && - !mte_probe1(env, mtedesc, addr)) { + !mte_probe(env, mtedesc, addr)) { goto fault; } @@ -5773,7 +5741,7 @@ void sve_st1_z(CPUARMState *env, void *vd, uint64_t *vg, void *vm, } if (mtedesc && arm_tlb_mte_tagged(&info.attrs)) { - mte_check1(env, mtedesc, addr, retaddr); + mte_check(env, mtedesc, addr, retaddr); } } i += 1; diff --git a/target/arm/translate-a64.c b/target/arm/translate-a64.c index 0b42e53500..44f32fc0a8 100644 --- a/target/arm/translate-a64.c +++ b/target/arm/translate-a64.c @@ -253,10 +253,7 @@ static void gen_probe_access(DisasContext *s, TCGv_i64 ptr, } /* - * For MTE, check a single logical or atomic access. This probes a single - * address, the exact one specified. The size and alignment of the access - * is not relevant to MTE, per se, but watchpoints do require the size, - * and we want to recognize those before making any other changes to state. + * For MTE, check a single logical or atomic access. */ static TCGv_i64 gen_mte_check1_mmuidx(DisasContext *s, TCGv_i64 addr, bool is_write, bool tag_checked, @@ -272,11 +269,11 @@ static TCGv_i64 gen_mte_check1_mmuidx(DisasContext *s, TCGv_i64 addr, desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); - desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << log2_size); + desc = FIELD_DP32(desc, MTEDESC, SIZEM1, (1 << log2_size) - 1); tcg_desc = tcg_const_i32(desc); ret = new_tmp_a64(s); - gen_helper_mte_check1(ret, cpu_env, tcg_desc, addr); + gen_helper_mte_check(ret, cpu_env, tcg_desc, addr); tcg_temp_free_i32(tcg_desc); return ret; @@ -295,9 +292,9 @@ TCGv_i64 gen_mte_check1(DisasContext *s, TCGv_i64 addr, bool is_write, * For MTE, check multiple logical sequential accesses. */ TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, - bool tag_checked, int log2_esize, int total_size) + bool tag_checked, int total_size) { - if (tag_checked && s->mte_active[0] && total_size != (1 << log2_esize)) { + if (tag_checked && s->mte_active[0]) { TCGv_i32 tcg_desc; TCGv_i64 ret; int desc = 0; @@ -306,17 +303,16 @@ TCGv_i64 gen_mte_checkN(DisasContext *s, TCGv_i64 addr, bool is_write, desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); - desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << log2_esize); - desc = FIELD_DP32(desc, MTEDESC, TSIZE, total_size); + desc = FIELD_DP32(desc, MTEDESC, SIZEM1, total_size - 1); tcg_desc = tcg_const_i32(desc); ret = new_tmp_a64(s); - gen_helper_mte_checkN(ret, cpu_env, tcg_desc, addr); + gen_helper_mte_check(ret, cpu_env, tcg_desc, addr); tcg_temp_free_i32(tcg_desc); return ret; } - return gen_mte_check1(s, addr, is_write, tag_checked, log2_esize); + return clean_data_tbi(s, addr); } typedef struct DisasCompare64 { @@ -2966,8 +2962,7 @@ static void disas_ldst_pair(DisasContext *s, uint32_t insn) } clean_addr = gen_mte_checkN(s, dirty_addr, !is_load, - (wback || rn != 31) && !set_tag, - size, 2 << size); + (wback || rn != 31) && !set_tag, 2 << size); if (is_vector) { if (is_load) { @@ -3713,8 +3708,8 @@ static void disas_ldst_multiple_struct(DisasContext *s, uint32_t insn) * Issue the MTE check vs the logical repeat count, before we * promote consecutive little-endian elements below. */ - clean_addr = gen_mte_checkN(s, tcg_rn, is_store, is_postidx || rn != 31, - size, total); + clean_addr = gen_mte_checkN(s, tcg_rn, is_store, + is_postidx || rn != 31, total); /* * Consecutive little-endian elements from a single register @@ -3866,8 +3861,8 @@ static void disas_ldst_single_struct(DisasContext *s, uint32_t insn) total = selem << scale; tcg_rn = cpu_reg_sp(s, rn); - clean_addr = gen_mte_checkN(s, tcg_rn, !is_load, is_postidx || rn != 31, - scale, total); + clean_addr = gen_mte_checkN(s, tcg_rn, !is_load, + is_postidx || rn != 31, total); tcg_ebytes = tcg_const_i64(1 << scale); for (xs = 0; xs < selem; xs++) { diff --git a/target/arm/translate-sve.c b/target/arm/translate-sve.c index 0eefb61214..584c4d047c 100644 --- a/target/arm/translate-sve.c +++ b/target/arm/translate-sve.c @@ -4264,7 +4264,7 @@ static void do_ldr(DisasContext *s, uint32_t vofs, int len, int rn, int imm) dirty_addr = tcg_temp_new_i64(); tcg_gen_addi_i64(dirty_addr, cpu_reg_sp(s, rn), imm); - clean_addr = gen_mte_checkN(s, dirty_addr, false, rn != 31, len, MO_8); + clean_addr = gen_mte_checkN(s, dirty_addr, false, rn != 31, len); tcg_temp_free_i64(dirty_addr); /* @@ -4352,7 +4352,7 @@ static void do_str(DisasContext *s, uint32_t vofs, int len, int rn, int imm) dirty_addr = tcg_temp_new_i64(); tcg_gen_addi_i64(dirty_addr, cpu_reg_sp(s, rn), imm); - clean_addr = gen_mte_checkN(s, dirty_addr, false, rn != 31, len, MO_8); + clean_addr = gen_mte_checkN(s, dirty_addr, false, rn != 31, len); tcg_temp_free_i64(dirty_addr); /* Note that unpredicated load/store of vector/predicate registers @@ -4509,8 +4509,7 @@ static void do_mem_zpa(DisasContext *s, int zt, int pg, TCGv_i64 addr, desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); - desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << msz); - desc = FIELD_DP32(desc, MTEDESC, TSIZE, mte_n << msz); + desc = FIELD_DP32(desc, MTEDESC, SIZEM1, (mte_n << msz) - 1); desc <<= SVE_MTEDESC_SHIFT; } else { addr = clean_data_tbi(s, addr); @@ -5189,7 +5188,7 @@ static void do_mem_zpz(DisasContext *s, int zt, int pg, int zm, desc = FIELD_DP32(desc, MTEDESC, TBI, s->tbid); desc = FIELD_DP32(desc, MTEDESC, TCMA, s->tcma); desc = FIELD_DP32(desc, MTEDESC, WRITE, is_write); - desc = FIELD_DP32(desc, MTEDESC, ESIZE, 1 << msz); + desc = FIELD_DP32(desc, MTEDESC, SIZEM1, (1 << msz) - 1); desc <<= SVE_MTEDESC_SHIFT; } desc = simd_desc(vsz, vsz, desc | scale); diff --git a/tests/tcg/aarch64/mte-5.c b/tests/tcg/aarch64/mte-5.c new file mode 100644 index 0000000000..6dbd6ab3ea --- /dev/null +++ b/tests/tcg/aarch64/mte-5.c @@ -0,0 +1,44 @@ +/* + * Memory tagging, faulting unaligned access. + * + * Copyright (c) 2021 Linaro Ltd + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "mte.h" + +void pass(int sig, siginfo_t *info, void *uc) +{ + assert(info->si_code == SEGV_MTESERR); + exit(0); +} + +int main(int ac, char **av) +{ + struct sigaction sa; + void *p0, *p1, *p2; + long excl = 1; + + enable_mte(PR_MTE_TCF_SYNC); + p0 = alloc_mte_mem(sizeof(*p0)); + + /* Create two differently tagged pointers. */ + asm("irg %0,%1,%2" : "=r"(p1) : "r"(p0), "r"(excl)); + asm("gmi %0,%1,%0" : "+r"(excl) : "r" (p1)); + assert(excl != 1); + asm("irg %0,%1,%2" : "=r"(p2) : "r"(p0), "r"(excl)); + assert(p1 != p2); + + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = pass; + sa.sa_flags = SA_SIGINFO; + sigaction(SIGSEGV, &sa, NULL); + + /* Store store two different tags in sequential granules. */ + asm("stg %0, [%0]" : : "r"(p1)); + asm("stg %0, [%0]" : : "r"(p2 + 16)); + + /* Perform an unaligned load crossing the granules. */ + asm volatile("ldr %0, [%1]" : "=r"(p0) : "r"(p1 + 12)); + abort(); +} diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index 56e48f4b34..6c95dd8b9e 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -37,7 +37,7 @@ AARCH64_TESTS += bti-2 # MTE Tests ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_MTE),) -AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 +AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 mte-5 mte-%: CFLAGS += -march=armv8.5-a+memtag endif From patchwork Fri Apr 2 16:18:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Richard Henderson X-Patchwork-Id: 414383 Delivered-To: patch@linaro.org Received: by 2002:a02:8562:0:0:0:0:0 with SMTP id g89csp1537976jai; Fri, 2 Apr 2021 09:22:01 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzNfpR7zf4OP3TWPTZyFzL31dI4vPZgqpJAK8TqbXHfpgSAS+Zs5h4HPO6cCwz+c+pVOlI0 X-Received: by 2002:a6b:5002:: with SMTP id e2mr11314567iob.43.1617380521726; Fri, 02 Apr 2021 09:22:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1617380521; cv=none; d=google.com; s=arc-20160816; b=ulm+gsY+INzHedz0QC/tESjzUMiiG1r3RDAsKqDkubBD2c1UIKbotUzX6Te0R5UL6A jjRLfgooE6CVWmtVWB8jW7dmoPZoQZbFsfrycts8pjvAefIMtDa5Vmy6gCJNTRjuwccb mTA6mbIOcjijvtz0ju1bvp7qKXWnQHXbcST9eAYmVhy0zimvrhF+ubhEL/Z+/vj/ilgu iPbQg47/9GHmIDjpxCIsDg9UDB/lNWeEKUtB/JJMm5iffAS3nnnAkibw6gGxxo607SM+ bcaO0kA6maEeDEcVlEpHwmtvtKzy4gx7tnbCWFXrquqrLAgiFpUwuSbYv5JQeVgmAIhk Fxtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=40EtQ0GsJCeH0a6Y7JFQ1124iOFiLbPm35cGCuUgHGs=; b=RCpLBY+bnxuXZgt3rA8ec62YrthhuLk4p4KrUNQNkQvxHBgRRk1lsAvi6YFgH5XyMY Q7xk8winQ+8T/uzYjOWwELJnm0ZYjFGLocJSOkutO2L8s0dUJ/lnr1HBIQex3mQsuEkp pWjyKanpA1kde6sOlW5OaMvAXaEVDAvE5/ybiNnYLiTWvOdTd9MWGFpvX0MrxG41J2I2 khvALLSxo8cUqToCwen1RmR7G6mXsFxWsJwWCcr2uUbEfo2JLdfOLJpLoA9XxEKak0Oe u2BVXss0Ct559ZAQxsuzGjNOJLYuE489vzpzW5L5xsFwoAeI86aBirYDCwdiPHpmK4Za 5JkA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IwtQGeiO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id y5si7717551jai.73.2021.04.02.09.22.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 02 Apr 2021 09:22:01 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=IwtQGeiO; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:51864 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lSMYa-0004E1-MP for patch@linaro.org; Fri, 02 Apr 2021 12:22:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50994) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lSMVR-0000yA-UM for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:45 -0400 Received: from mail-pj1-x1031.google.com ([2607:f8b0:4864:20::1031]:46748) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1lSMVO-00037O-2u for qemu-devel@nongnu.org; Fri, 02 Apr 2021 12:18:45 -0400 Received: by mail-pj1-x1031.google.com with SMTP id q6-20020a17090a4306b02900c42a012202so2766717pjg.5 for ; Fri, 02 Apr 2021 09:18:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=40EtQ0GsJCeH0a6Y7JFQ1124iOFiLbPm35cGCuUgHGs=; b=IwtQGeiOJi/QpBb85ZOaOnyDWtCfLsvyWvPo6PtnLrRpJ/9D9De5lADwWEPDB33W85 R472l6g4KJwXIaC9ojOAKhRSwKocxYnk2m/O11ffMuDX221Ftz87fSY8Y7wZb8Y8xRt8 gl7CMSQP3AJgsGeF3+3UIaDBlq7QpVayg2+Ur8ws2tvzOCql8JIJSUAmu6X0oTet8BYc nEicT2kvLpTQvAZKEb+JE7znfocLzzTqv50nHki1qG62hwjFnZ8mmM4MJH/SKPrn+PpK KenVrOuoDGgVFlV1l7061Zzkabutuajfy8fcQk2LpC7xlOqakNjnwDPssdt2vxyQUDwI LZkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=40EtQ0GsJCeH0a6Y7JFQ1124iOFiLbPm35cGCuUgHGs=; b=TMwQ9QDDedddDEVgNJgyxacR29tXaSjS3BV7G+odLO4IAXZxRESEYoNAFmSpQQvu3W A96XSTO4ZM3itEw/lqeoxrSDLU6RgeNX6aVpQDfNM8YQUmyj4T5H33C6qCYuzW1hqoYl lY1+ydUx6VzHCwPgiqzQ/lJCa8Z7uqUuzofPS5TmqAhN8CpWZCiGvI8ZICreTRc2xNY8 nSLIaPCshrsvKXiiOUCksUb1mEKXJAVXF/SWINZRq9Sws1e/9xtYdcEpCA1wjgyK4MRB 2uR5PL1SzZbo09xVzVmXnZKhKjK0YC/fBYkjjxaIFBTbe8iM/7AMG/ymJRjXn2X7DNxg 489A== X-Gm-Message-State: AOAM531PJdv7ggZZdSCo7+lWtt3t6arC45M6rBWGATg16BYpJIZZiYhT AVkQVn0iuEWc6fDeyLxPRhX5zDaeCrRmJQ== X-Received: by 2002:a17:90a:d3d2:: with SMTP id d18mr14329233pjw.111.1617380320765; Fri, 02 Apr 2021 09:18:40 -0700 (PDT) Received: from localhost.localdomain (h216-228-167-147.bendor.dedicated.static.tds.net. [216.228.167.147]) by smtp.gmail.com with ESMTPSA id m7sm8821761pjc.54.2021.04.02.09.18.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Apr 2021 09:18:40 -0700 (PDT) From: Richard Henderson To: qemu-devel@nongnu.org Subject: [PATCH v2 3/3] accel/tcg: Preserve PAGE_ANON when changing page permissions Date: Fri, 2 Apr 2021 09:18:35 -0700 Message-Id: <20210402161835.286665-4-richard.henderson@linaro.org> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20210402161835.286665-1-richard.henderson@linaro.org> References: <20210402161835.286665-1-richard.henderson@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::1031; envelope-from=richard.henderson@linaro.org; helo=mail-pj1-x1031.google.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: qemu-arm@nongnu.org, Stephen Long Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Using mprotect() to change PROT_* does not change the MAP_ANON previously set with mmap(). Our linux-user version of MTE only works with MAP_ANON pages, so losing PAGE_ANON caused MTE to stop working. Reported-by: Stephen Long Signed-off-by: Richard Henderson --- tests/tcg/aarch64/mte.h | 3 ++- accel/tcg/translate-all.c | 9 +++++-- tests/tcg/aarch64/mte-6.c | 43 +++++++++++++++++++++++++++++++ tests/tcg/aarch64/Makefile.target | 2 +- 4 files changed, 53 insertions(+), 4 deletions(-) create mode 100644 tests/tcg/aarch64/mte-6.c -- 2.25.1 diff --git a/tests/tcg/aarch64/mte.h b/tests/tcg/aarch64/mte.h index 141cef522c..0805676b11 100644 --- a/tests/tcg/aarch64/mte.h +++ b/tests/tcg/aarch64/mte.h @@ -48,7 +48,8 @@ static void enable_mte(int tcf) } } -static void *alloc_mte_mem(size_t size) +static void * alloc_mte_mem(size_t size) __attribute__((unused)); +static void * alloc_mte_mem(size_t size) { void *p = mmap(NULL, size, PROT_READ | PROT_WRITE | PROT_MTE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c index f32df8b240..ba6ab09790 100644 --- a/accel/tcg/translate-all.c +++ b/accel/tcg/translate-all.c @@ -2714,6 +2714,8 @@ void page_set_flags(target_ulong start, target_ulong end, int flags) a missing call to h2g_valid. */ assert(end - 1 <= GUEST_ADDR_MAX); assert(start < end); + /* Only set PAGE_ANON with new mappings. */ + assert(!(flags & PAGE_ANON) || (flags & PAGE_RESET)); assert_memory_lock(); start = start & TARGET_PAGE_MASK; @@ -2737,11 +2739,14 @@ void page_set_flags(target_ulong start, target_ulong end, int flags) p->first_tb) { tb_invalidate_phys_page(addr, 0); } - if (reset_target_data && p->target_data) { + if (reset_target_data) { g_free(p->target_data); p->target_data = NULL; + p->flags = flags; + } else { + /* Using mprotect on a page does not change MAP_ANON. */ + p->flags = (p->flags & PAGE_ANON) | flags; } - p->flags = flags; } } diff --git a/tests/tcg/aarch64/mte-6.c b/tests/tcg/aarch64/mte-6.c new file mode 100644 index 0000000000..60d51d18be --- /dev/null +++ b/tests/tcg/aarch64/mte-6.c @@ -0,0 +1,43 @@ +#include "mte.h" + +void pass(int sig, siginfo_t *info, void *uc) +{ + assert(info->si_code == SEGV_MTESERR); + exit(0); +} + +int main(void) +{ + enable_mte(PR_MTE_TCF_SYNC); + + void *brk = sbrk(16); + if (brk == (void *)-1) { + perror("sbrk"); + return 2; + } + + if (mprotect(brk, 16, PROT_READ | PROT_WRITE | PROT_MTE)) { + perror("mprotect"); + return 2; + } + + int *p1, *p2; + long excl = 1; + + asm("irg %0,%1,%2" : "=r"(p1) : "r"(brk), "r"(excl)); + asm("gmi %0,%1,%0" : "+r"(excl) : "r"(p1)); + asm("irg %0,%1,%2" : "=r"(p2) : "r"(brk), "r"(excl)); + asm("stg %0,[%0]" : : "r"(p1)); + + *p1 = 0; + + struct sigaction sa; + memset(&sa, 0, sizeof(sa)); + sa.sa_sigaction = pass; + sa.sa_flags = SA_SIGINFO; + sigaction(SIGSEGV, &sa, NULL); + + *p2 = 0; + + abort(); +} diff --git a/tests/tcg/aarch64/Makefile.target b/tests/tcg/aarch64/Makefile.target index 6c95dd8b9e..928357b10a 100644 --- a/tests/tcg/aarch64/Makefile.target +++ b/tests/tcg/aarch64/Makefile.target @@ -37,7 +37,7 @@ AARCH64_TESTS += bti-2 # MTE Tests ifneq ($(DOCKER_IMAGE)$(CROSS_CC_HAS_ARMV8_MTE),) -AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 mte-5 +AARCH64_TESTS += mte-1 mte-2 mte-3 mte-4 mte-5 mte-6 mte-%: CFLAGS += -march=armv8.5-a+memtag endif