From patchwork Fri Jun 12 19:02:31 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 187860 Delivered-To: patch@linaro.org Received: by 2002:a92:cf06:0:0:0:0:0 with SMTP id c6csp663883ilo; Fri, 12 Jun 2020 12:05:56 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwpwWJbk+Hws3LFMEBR9IvvgEWiXRyf0oQDp/aZvPhwuwhA7dBXUYOjYMwVz0jPSXfxFwQx X-Received: by 2002:a25:bf8f:: with SMTP id l15mr21941455ybk.453.1591988756783; Fri, 12 Jun 2020 12:05:56 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591988756; cv=none; d=google.com; s=arc-20160816; b=bfbYnN6ug8AIqZZMSp0JQ3+navI5fFjRPIJyIOivRBv/YylMkddwWde5wfuPYuOyyU 5woHRgViLYEmjtuJMoXsWJs7a5Aw+XTkOFj7dGsuVv4meyVIe6phs255YkzZcMN2q52P Az4RaMFlmg7PnFGKivKtALhbXsSoFWpx22M6+r0N6v6RO9OXw/XcZf1yBL6FwWv2giJ+ YR9SlNqwjvnKKsLgCxnNJlYuN/fpanqviZHsyeaKj3sc+Mcem9Unco03b6kVewsO6XVw M3gsNXWMwf8rWSeYHrzdsf4gUVKr2gn42Kgmd552lwK2wuGg544SAu0dq41f1LBZBJfq 5kQg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:cc:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:content-transfer-encoding :mime-version:references:in-reply-to:message-id:date:subject:to:from :dkim-signature; bh=+fhzkIekd9HvnT+zTrzjSgSzaU+uY2T8F/QfwK7CaNs=; b=PIBH52qnbWXSBv78KXYZRkrnoMAlR1fg9nHAtzzaJjHmo9uLL7ffmNGI09JpNCSnLj JhFPcDd6N62u3QO9sd8xBxkbi6IeRhFSe7Thcaf+0vRg8ZlQEY6/8C1B0RKXwy7o3txK HUVQHAb7aoDm5fglTHXQPk5tis3c57Mfm4hY4yaO6Afl06ooQMAS5ZNMs0mjBRkcRNxe pvcYA/38XjTjYG6ii3Lqfd9g08oi+vztBharIn20CWNobmbSOhKIaHWCjE8J+6iWmTGb uKB792+mPCdYq0Btr3qAATGjfC4iOQ/0Df0N53hPlkGYpVa7yNMwfeZcprsM3EKY1L5R ia/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iYv3bZyw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from lists.gnu.org (lists.gnu.org. [209.51.188.17]) by mx.google.com with ESMTPS id v88si5831403ybi.197.2020.06.12.12.05.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Fri, 12 Jun 2020 12:05:56 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org header.s=google header.b=iYv3bZyw; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom="qemu-devel-bounces+patch=linaro.org@nongnu.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:50260 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jjp00-0004hj-6J for patch@linaro.org; Fri, 12 Jun 2020 15:05:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:54924) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jjoxM-0000tZ-KZ for qemu-devel@nongnu.org; Fri, 12 Jun 2020 15:03:14 -0400 Received: from mail-wr1-x442.google.com ([2a00:1450:4864:20::442]:42919) by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jjoxF-0006uE-M6 for qemu-devel@nongnu.org; Fri, 12 Jun 2020 15:03:10 -0400 Received: by mail-wr1-x442.google.com with SMTP id p5so10789184wrw.9 for ; Fri, 12 Jun 2020 12:02:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=+fhzkIekd9HvnT+zTrzjSgSzaU+uY2T8F/QfwK7CaNs=; b=iYv3bZyw/geh4bQygkWEEWtIFxDK1MAoKBbcs8sOzuLIV/3IjPeNPPWjaYpKEiKN+z QLJt1jGexa7UG/nFpb260hJLzs9zE6NeMFFQB7qOjVnPTziWlMASLEp0OmWkq3kUgwFe CiQYWGFaiYH/TddVQ1KhUNbsCG4FDA2ovA+zDmlQ5JNJaHjH51d/RB7pgAMcWZ/yD0RK rbwLIIuByprx5/sXgwYsPONWatRGoxqjGulS1/P2L/r5ZoD3iUzLXipniWpdU2I0XhzX b9XQwJqe6kMRO8BvqmXYXg/VitfB6Fx/lzLpAFyQ7JiXnSFHwc4rp9kCIgd8BW+5ytvL rlvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=+fhzkIekd9HvnT+zTrzjSgSzaU+uY2T8F/QfwK7CaNs=; b=a28pTYSKzON0PEkHtXC56Ep9P1jfH5PUtLC84F8lguraGdRdwiRZhYPGDfOIU6uFNe Y4gpYxkHSvYGXVnSmplsPRyn4Imbeg7yLLXv2B6ui+rNq6qBIsvr6/wue/9xhkJUXc/q 2IYE6yx70esQ8pPRuysEBZL0MtSLUB9H8fbOxzXQc8djfaVMhAREQpFe9GEzG4HxcOk6 nfij+CzyXJ661MpW4uNrvh98bONp6cLrT99qlEOR44dgxpvDak+M4qOCtV5VhMAVu1Ax Jkx1i6pNYAe2bzfSZOJ/RAwiPyON4w2jcI2Je67wOAIlYXQYBru5+qbRMVZjRWYKd9Nq 1jeQ== X-Gm-Message-State: AOAM530yudXFgp95FUR49HnrKY9zgbLmU6pnv1rVmORTqdWpfJytIvTo ZyYoF4j4cmbGeVUL9Ms0BGZ06A== X-Received: by 2002:adf:f812:: with SMTP id s18mr15692418wrp.28.1591988576758; Fri, 12 Jun 2020 12:02:56 -0700 (PDT) Received: from zen.linaroharston ([51.148.130.216]) by smtp.gmail.com with ESMTPSA id 23sm9723140wmg.10.2020.06.12.12.02.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 12 Jun 2020 12:02:50 -0700 (PDT) Received: from zen.lan (localhost [127.0.0.1]) by zen.linaroharston (Postfix) with ESMTP id 16A5E1FF7E; Fri, 12 Jun 2020 20:02:40 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: qemu-devel@nongnu.org Subject: [PATCH v1 12/18] tests/docker: Added docker build support for TSan. Date: Fri, 12 Jun 2020 20:02:31 +0100 Message-Id: <20200612190237.30436-13-alex.bennee@linaro.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20200612190237.30436-1-alex.bennee@linaro.org> References: <20200612190237.30436-1-alex.bennee@linaro.org> MIME-Version: 1.0 Received-SPF: pass client-ip=2a00:1450:4864:20::442; envelope-from=alex.bennee@linaro.org; helo=mail-wr1-x442.google.com X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache. That's all we know. X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: fam@euphon.net, berrange@redhat.com, Robert Foley , =?utf-8?q?Philippe_Mathieu-Da?= =?utf-8?q?ud=C3=A9?= , richard.henderson@linaro.org, f4bug@amsat.org, =?utf-8?q?Alex_Benn?= =?utf-8?b?w6ll?= , cota@braap.org, aurelien@aurel32.net Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" From: Robert Foley Added a new docker for ubuntu 20.04. This docker has support for Thread Sanitizer including one patch we need in one of the header files. https://github.com/llvm/llvm-project/commit/a72dc86cd This command will build with tsan enabled: make docker-test-tsan-ubuntu2004 V=1 Also added the TSAN suppresion file to disable certain cases of TSAN warnings. Cc: Fam Zheng Cc: Philippe Mathieu-Daudé Signed-off-by: Robert Foley Reviewed-by: Alex Bennée Signed-off-by: Alex Bennée Message-Id: <20200609200738.445-10-robert.foley@linaro.org> --- tests/docker/dockerfiles/ubuntu2004.docker | 65 ++++++++++++++++++++++ tests/docker/test-tsan | 44 +++++++++++++++ tests/tsan/blacklist.tsan | 10 ++++ tests/tsan/suppressions.tsan | 14 +++++ 4 files changed, 133 insertions(+) create mode 100644 tests/docker/dockerfiles/ubuntu2004.docker create mode 100755 tests/docker/test-tsan create mode 100644 tests/tsan/blacklist.tsan create mode 100644 tests/tsan/suppressions.tsan -- 2.20.1 diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker new file mode 100644 index 00000000000..6050ce7e8a8 --- /dev/null +++ b/tests/docker/dockerfiles/ubuntu2004.docker @@ -0,0 +1,65 @@ +FROM ubuntu:20.04 +ENV PACKAGES flex bison \ + ccache \ + clang-10\ + gcc \ + gettext \ + git \ + glusterfs-common \ + libaio-dev \ + libattr1-dev \ + libbrlapi-dev \ + libbz2-dev \ + libcacard-dev \ + libcap-ng-dev \ + libcurl4-gnutls-dev \ + libdrm-dev \ + libepoxy-dev \ + libfdt-dev \ + libgbm-dev \ + libgtk-3-dev \ + libibverbs-dev \ + libiscsi-dev \ + libjemalloc-dev \ + libjpeg-turbo8-dev \ + liblzo2-dev \ + libncurses5-dev \ + libncursesw5-dev \ + libnfs-dev \ + libnss3-dev \ + libnuma-dev \ + libpixman-1-dev \ + librados-dev \ + librbd-dev \ + librdmacm-dev \ + libsasl2-dev \ + libsdl2-dev \ + libseccomp-dev \ + libsnappy-dev \ + libspice-protocol-dev \ + libspice-server-dev \ + libssh-dev \ + libusb-1.0-0-dev \ + libusbredirhost-dev \ + libvdeplug-dev \ + libvte-2.91-dev \ + libxen-dev \ + libzstd-dev \ + make \ + python3-yaml \ + python3-sphinx \ + sparse \ + texinfo \ + xfslibs-dev\ + vim +RUN apt-get update && \ + DEBIAN_FRONTEND=noninteractive apt-get -y install $PACKAGES +RUN dpkg -l $PACKAGES | sort > /packages.txt +ENV FEATURES clang tsan pyyaml sdl2 + +# https://bugs.launchpad.net/qemu/+bug/1838763 +ENV QEMU_CONFIGURE_OPTS --disable-libssh + +# Apply patch https://reviews.llvm.org/D75820 +# This is required for TSan in clang-10 to compile with QEMU. +RUN sed -i 's/^const/static const/g' /usr/lib/llvm-10/lib/clang/10.0.0/include/sanitizer/tsan_interface.h diff --git a/tests/docker/test-tsan b/tests/docker/test-tsan new file mode 100755 index 00000000000..eb40ac45b7a --- /dev/null +++ b/tests/docker/test-tsan @@ -0,0 +1,44 @@ +#!/bin/bash -e +# +# This test will use TSan as part of a build and a make check. +# +# Copyright (c) 2020 Linaro +# Copyright (c) 2016 Red Hat Inc. +# +# Authors: +# Robert Foley +# Originally based on test-quick from Fam Zheng +# +# This work is licensed under the terms of the GNU GPL, version 2 +# or (at your option) any later version. See the COPYING file in +# the top-level directory. + +. common.rc + +setup_tsan() +{ + requires clang tsan + tsan_log_dir="/tmp/qemu-test/build/tsan" + mkdir -p $tsan_log_dir > /dev/null || true + EXTRA_CONFIGURE_OPTS="${EXTRA_CONFIGURE_OPTS} --enable-tsan \ + --cc=clang-10 --cxx=clang++-10 \ + --disable-werror --extra-cflags=-O0" + # detect deadlocks is false currently simply because + # TSan crashes immediately with deadlock detector enabled. + # We have maxed out the history size to get the best chance of finding + # warnings during testing. + # Note, to get TSan to fail on warning, use exitcode=66 below. + tsan_opts="suppressions=/tmp/qemu-test/src/tests/tsan/suppressions.tsan\ + detect_deadlocks=false history_size=7\ + halt_on_error=0 exitcode=0 verbose=5\ + log_path=$tsan_log_dir/tsan_warning" + export TSAN_OPTIONS="$tsan_opts" +} + +cd "$BUILD_DIR" + +TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \ +setup_tsan +build_qemu +check_qemu +install_qemu diff --git a/tests/tsan/blacklist.tsan b/tests/tsan/blacklist.tsan new file mode 100644 index 00000000000..75e444f5dc6 --- /dev/null +++ b/tests/tsan/blacklist.tsan @@ -0,0 +1,10 @@ +# This is an example blacklist. +# To enable use of the blacklist add this to configure: +# "--extra-cflags=-fsanitize-blacklist=/tests/tsan/blacklist.tsan" +# The eventual goal would be to fix these warnings. + +# TSan is not happy about setting/getting of dirty bits, +# for example, cpu_physical_memory_set_dirty_range, +# and cpu_physical_memory_get_dirty. +src:bitops.c +src:bitmap.c diff --git a/tests/tsan/suppressions.tsan b/tests/tsan/suppressions.tsan new file mode 100644 index 00000000000..73414b9ebd9 --- /dev/null +++ b/tests/tsan/suppressions.tsan @@ -0,0 +1,14 @@ +# This is the set of runtime suppressions of TSan warnings. +# The goal would be to have here only items we do not +# plan to fix, and to explain why for each item. + +# TSan reports a double lock on RECURSIVE mutexes. +# Since the recursive lock is intentional, we choose to ignore it. +mutex:aio_context_acquire +mutex:pthread_mutex_lock + +# TSan reports a race betwen pthread_mutex_init() and +# pthread_mutex_lock(). Since this is outside of QEMU, +# we choose to ignore it. +race:pthread_mutex_init +race:pthread_mutex_lock