From patchwork Fri May 29 13:21:43 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Robert Foley X-Patchwork-Id: 187304 Delivered-To: patch@linaro.org Received: by 2002:a92:5b0a:0:0:0:0:0 with SMTP id p10csp581040ilb; Fri, 29 May 2020 06:23:09 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxzx7kWT3MAgDVf6iMWBVtMwf14fDds539gsA1sGxGt1v6pY2qnGUvXdQ0bPoSXiivcPEld X-Received: by 2002:a25:b788:: with SMTP id n8mr14416580ybh.226.1590758589551; Fri, 29 May 2020 06:23:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1590758589; cv=none; d=google.com; s=arc-20160816; b=KY7WGXsTSX3EkCbACP9+5JfuZNpMLuETjJKosdBtVldmRqot35nREPvbo8Xhvv8R2/ dXNoV9wxCtCL2URfYAUEkomObCnKvbiqchvXP4kXpoRl+YEKApCBQ5uWYPzzq/CBg9s4 WlzxhJlDXMzakw3kkqRdWpUHYg+QjAPRBZULH6fsKKxtavAhTnGkmC6CcE41SeSZjSv4 SnF+9Y2fWs7j46vvQGsyRtXIOztjbOyVYpJJ63P2pdJyOZau4c9Wv7o1c8Oxye956QWw 9TowN+Mh/dn6+X+Wgd573S6kfG0luQah3wIoiaPeTyTXMO9IwUF0qdezuz28Owe3fCad WAHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:message-id:date:subject:to:from:dkim-signature; bh=WUZT8MmOayZGLSCYKoc1qodVCAWn7YuXqviDoocPmbE=; b=uI7W8PRWHiZiKNbnWp/aCmBY3BIyYrXAaU4nvflvLYGKyE2PA5JZMwAC1UKnotnXT8 /ZEr3pozNlwOnBd7MV4vhFDl4skk9lqFc+XSnfMWA3tED5u+Sexx3nYZ9Ci4C1jaahRN gNAOyDfy0SsT1byRfSGjj4OqOX01ZdGS4ppOHrOAMdVAd9EVLkGuRm4s1NmjwrVZouVV r0nDIMNJbKLh8xtfuLp7u/FDR6wlBChcM1fD+4PQDvlw/vPuoo24kNcIgLjgxlpw5dP8 zSwXBWoz3UV4J+WbH4iILvo8RErEZoA528FTMqI0xyUXCfQKHSktBSq3KbY+qfqWrZlC KyBA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qk39QB5S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id n8si7641274ybk.265.2020.05.29.06.23.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 29 May 2020 06:23:09 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=qk39QB5S; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:44962 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jeeyb-00037G-0b for patch@linaro.org; Fri, 29 May 2020 09:23:09 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:42418) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jeeyK-00035i-3y for qemu-devel@nongnu.org; Fri, 29 May 2020 09:22:52 -0400 Received: from mail-pl1-x644.google.com ([2607:f8b0:4864:20::644]:36687) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jeeyI-00038d-EH for qemu-devel@nongnu.org; Fri, 29 May 2020 09:22:51 -0400 Received: by mail-pl1-x644.google.com with SMTP id bg4so1163654plb.3 for ; Fri, 29 May 2020 06:22:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=WUZT8MmOayZGLSCYKoc1qodVCAWn7YuXqviDoocPmbE=; b=qk39QB5SZxtjxxmxaGGuXThdjRLQm7aj/9L6/Ud2yo5TeIbBv5WvoTpe3iUYulhF6E jWYvp+8iMY9x3HzTGQ7vRMxVjWocnMFkiEFcoLI+zMUaCW9wSOIuiRsgjA8SNFJ5u1gp KQ5xRfeE5899EeiZGYHQEzf24dou4ulRS6LtbweQqgWfjuZB8F8uz4JeXc685j0B8WeH a5vAaQywnuisX9l5+W81ItDo7zFtj599A/O6+l62qqhVUUrS+DgIs9Xrom813z2CqJre bEc7OmfRZIsARUGba7l4uut7kN9CkZ58l0ARPjtIxNACBkFcCb6hpkVTCyvD290HC9Wh Pw6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=WUZT8MmOayZGLSCYKoc1qodVCAWn7YuXqviDoocPmbE=; b=q0iBiMqvizvrzPkZHD7vOALFbujbOfi0QJT//PvSPZa0S9czvo1Rj9JvYbPobbQVEH LQL2hF3hcqn17vOrF2nUFM8e2guhLHPILxnNtNHH7c7WO6fHBns865VMdywv93Lfewcu DuR1SnR+KL0B8VR1jKyA6pOY/GaBj0uSEcGwwJzlsV3ETn+aZPeUayWYg3LuN9C4QbIn IPLRPGA3CtO9jF/6OVfwgbKss3xMk08W3yMdsBZ52I8aKTLoEFIInTFDdAojihEbSz9+ ri8SopDMvJ1I1F9AFzBSsJ3OchXUaQGhh2AkqRDB7EuIgbRvnloYPCbpqgjarVNGOvj5 T2mA== X-Gm-Message-State: AOAM5304tFSTLEUSTDz+CTdlR24SwEtcJO1D2J4GxQdhg5Bh7pz4oO3Z jalE2pAk8UB/IXPkU38q4jQASpS8OXBphA== X-Received: by 2002:a17:90a:d191:: with SMTP id fu17mr9696225pjb.228.1590758567768; Fri, 29 May 2020 06:22:47 -0700 (PDT) Received: from Rfoley-MA01.hsd1.ma.comcast.net ([2601:199:4480:60c0:75ac:d268:da04:dde4]) by smtp.gmail.com with ESMTPSA id e16sm6946374pgg.8.2020.05.29.06.22.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2020 06:22:46 -0700 (PDT) From: Robert Foley To: qemu-devel@nongnu.org Subject: [PATCH v1 00/12] Add Thread Sanitizer support to QEMU Date: Fri, 29 May 2020 09:21:43 -0400 Message-Id: <20200529132143.702-1-robert.foley@linaro.org> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::644; envelope-from=robert.foley@linaro.org; helo=mail-pl1-x644.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: peter.puhov@linaro.org, cota@braap.org, alex.bennee@linaro.org, robert.foley@linaro.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Changes in v1: - Refocused the patch more on enabling TSan and less on removing warnings. - Because of this we removed several patches from the series that were fixing and/or suppressing warnings. - Removed from the series some patches that had overlap with the per-cpu-lock patch.  We will let the per-cpu-lock patch fix those warnings. - Removed built-in support for blacklist.   The idea here being that all warnings should eventually be fixed and this blacklist is for test/debug. - Enhanced the documentation on TSan support in testing.rst. - Improved the comments in tsan.h. - Changed configure to error out if --enable-tsan provided but TSan not available. This patch series continues the work done by Emilio Cota and others to add Thread Sanitizer (TSan) support to QEMU. The starting point for this work was Emilio's branch here: https://github.com/cota/qemu/commits/tsan specifically this commit: 0be125fc0afd47218b34d2019abdd19b644f3199 The main purpose of this patch is to enable TSan support so that QEMU developers can start using the tool. We found this tool useful and even ran it on our recent changes in the cpu-locks series, which fixes many warnings. Clearly there is work to do here to clean up all the warnings. :) We have also made an effort to introduce enough of the TSan suppression mechanisms, so that others can continue this work. This series adds support for: - configure option for --enable-tsan. - testing.rst has the full details on how to use TSan with or without docker, including all the suppression mechanisms. - We added an Ubuntu 20.04 docker that supports TSan builds. - We added an example blacklist file for files or functions TSan should ignore at compile time. This can now be specified manually. - Added a suppression file for TSan to suppress certain warnings at run time. - Added tsan.h with annotations which also can be used to suppress warnings. Issues: - When running docker-test-quick under TSan there are several tests which hang - The unit tests which seem to hang under TSan: test-char, test-qdev-global-props, and test-qga.  - If we comment out those tests, check-unit finishes, albeit with a couple of warnings. :) Emilio G. Cota (7): cpu: convert queued work to a QSIMPLEQ thread: add qemu_spin_destroy cputlb: destroy CPUTLB with tlb_destroy qht: call qemu_spin_destroy for head buckets tcg: call qemu_spin_destroy for tb->jmp_lock translate-all: call qemu_spin_destroy for PageDesc thread: add tsan annotations to QemuSpin Lingfeng Yang (1): configure: add --enable-tsan flag + fiber annotations for coroutine-ucontext Robert Foley (4): tests/docker: Added docker build support for TSan. include/qemu: Added tsan.h for annotations. util: Added tsan annotate for thread name. docs: Added details on TSan to testing.rst accel/tcg/cputlb.c | 15 +++ accel/tcg/translate-all.c | 19 +++- configure | 41 ++++++++ cpus-common.c | 25 ++--- cpus.c | 14 ++- docs/devel/testing.rst | 113 +++++++++++++++++++++ exec.c | 1 + hw/core/cpu.c | 1 + include/exec/exec-all.h | 8 ++ include/hw/core/cpu.h | 6 +- include/qemu/thread.h | 38 ++++++- include/qemu/tsan.h | 71 +++++++++++++ include/tcg/tcg.h | 3 +- tcg/tcg.c | 19 +++- tests/docker/Makefile.include | 2 + tests/docker/common.rc | 19 ++++ tests/docker/dockerfiles/ubuntu2004.docker | 65 ++++++++++++ tests/tsan/blacklist.tsan | 10 ++ tests/tsan/suppressions.tsan | 14 +++ util/coroutine-ucontext.c | 97 ++++++++++++++++-- util/qemu-thread-posix.c | 2 + util/qht.c | 1 + 22 files changed, 547 insertions(+), 37 deletions(-) create mode 100644 include/qemu/tsan.h create mode 100644 tests/docker/dockerfiles/ubuntu2004.docker create mode 100644 tests/tsan/blacklist.tsan create mode 100644 tests/tsan/suppressions.tsan -- 2.17.1