From patchwork Fri May 5 10:38:14 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98608 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp49264qge; Fri, 5 May 2017 03:38:12 -0700 (PDT) X-Received: by 10.55.24.9 with SMTP id j9mr11212606qkh.25.1493980692204; Fri, 05 May 2017 03:38:12 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id z93si4175657qtc.212.2017.05.05.03.38.11 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:38:12 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46164 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acj-0001m4-P1 for patch@linaro.org; Fri, 05 May 2017 06:38:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52557) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acO-0001kR-RU for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acM-0006hc-B4 for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: from mail-wm0-x235.google.com ([2a00:1450:400c:c09::235]:33998) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acM-0006h4-47 for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:46 -0400 Received: by mail-wm0-x235.google.com with SMTP id u65so1973321wmu.1 for ; Fri, 05 May 2017 03:37:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=jLILibzHXJUUa8wxesL+Eva8CzOGXRv70HjSijvpmwc=; b=bKlf/OUNgO+NtQBWowsCMLE3usthcDXyxxMvRkNmDCB+bKFZEnQAuyPQzuUZOXjfHL 4/eix6QtKK5LxRvauORqdRjC0KkzMyCRSj977CKxFwDZ3blQrfQ4eBr3lu+l1tEcjGdZ hFtW9y9/7OFMBAq1Zai+SZZps/5plBANbgTaE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=jLILibzHXJUUa8wxesL+Eva8CzOGXRv70HjSijvpmwc=; b=ZQkm8nNCy0OXGR8OLPmtbkLH4CCzje4adYYetH2J/L51n34eRjmuMAilK584dS85e7 Hap7GAbs7QYPWltDGdx0YkzyPRBJP4YhHuKp66zugEaFpaqljBNziCTe61xR1Dw4e6DA g3ISDDkiO1SUOo7OhZ3B99THLvcgs9G+cm4Nq6vnyQnI2t8pwdL7bB9c3Vw0wAW9Ci6H Izji8U0aVzLio9L8zwmvQUb8ZcZIOEYN7/d4RMDncT4YSgREAXi7+cWY0Wccvh/OWQcU pg4H+43obIATw+/tRuYia+X8xBJBOoh9i9ZyUm8uk4Fvo5SZ6wxEmdB5w6SjbckIumFa XTyg== X-Gm-Message-State: AN3rC/4yMe1ucq7ABtKP+Lu3CUKyIeWlH/U1VoUZf8/GiBEMOdc2tsnz zwZ8CQjjsZ1mqFo4 X-Received: by 10.28.169.198 with SMTP id s189mr4797589wme.91.1493980663450; Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id z14sm1607896wmz.2.2017.05.05.03.37.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:41 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id BA6C93E0198; Fri, 5 May 2017 11:38:22 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:14 +0100 Message-Id: <20170505103822.20641-2-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::235 Subject: [Qemu-devel] [RFC PATCH v1 1/9] target/arm/arm-powertctl: drop BQL assertions X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Peter Maydell , cota@braap.org, =?utf-8?q?Al?= =?utf-8?q?ex_Benn=C3=A9e?= , qemu-devel@nongnu.org, "open list:ARM" Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The powerctl code is run in the context of the vCPU changing power state. It does not need the BQL to protect its changes. Signed-off-by: Alex Bennée --- target/arm/arm-powerctl.c | 8 -------- 1 file changed, 8 deletions(-) -- 2.11.0 diff --git a/target/arm/arm-powerctl.c b/target/arm/arm-powerctl.c index 25207cb850..9661a59dad 100644 --- a/target/arm/arm-powerctl.c +++ b/target/arm/arm-powerctl.c @@ -124,7 +124,6 @@ static void arm_set_cpu_on_async_work(CPUState *target_cpu_state, g_free(info); /* Finally set the power status */ - assert(qemu_mutex_iothread_locked()); target_cpu->power_state = PSCI_ON; } @@ -135,8 +134,6 @@ int arm_set_cpu_on(uint64_t cpuid, uint64_t entry, uint64_t context_id, ARMCPU *target_cpu; struct CpuOnInfo *info; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 " (EL %d, %s) @ 0x%" PRIx64 " with R0 = 0x%" PRIx64 "\n", cpuid, target_el, target_aa64 ? "aarch64" : "aarch32", entry, context_id); @@ -227,7 +224,6 @@ static void arm_set_cpu_off_async_work(CPUState *target_cpu_state, { ARMCPU *target_cpu = ARM_CPU(target_cpu_state); - assert(qemu_mutex_iothread_locked()); target_cpu->power_state = PSCI_OFF; target_cpu_state->halted = 1; target_cpu_state->exception_index = EXCP_HLT; @@ -238,8 +234,6 @@ int arm_set_cpu_off(uint64_t cpuid) CPUState *target_cpu_state; ARMCPU *target_cpu; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 "\n", cpuid); /* change to the cpu we are powering up */ @@ -274,8 +268,6 @@ int arm_reset_cpu(uint64_t cpuid) CPUState *target_cpu_state; ARMCPU *target_cpu; - assert(qemu_mutex_iothread_locked()); - DPRINTF("cpu %" PRId64 "\n", cpuid); /* change to the cpu we are resetting */ From patchwork Fri May 5 10:38:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98614 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp50966qge; Fri, 5 May 2017 03:42:42 -0700 (PDT) X-Received: by 10.200.49.46 with SMTP id g43mr4977867qtb.229.1493980962144; Fri, 05 May 2017 03:42:42 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id g13si4478215qke.63.2017.05.05.03.42.41 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:42:42 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46185 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6ah5-0005lA-J8 for patch@linaro.org; Fri, 05 May 2017 06:42:39 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52552) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acO-0001kO-Ph for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:50 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acM-0006hT-1X for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: from mail-wm0-x22b.google.com ([2a00:1450:400c:c09::22b]:38057) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acL-0006hE-OE for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:45 -0400 Received: by mail-wm0-x22b.google.com with SMTP id 142so2193782wma.1 for ; Fri, 05 May 2017 03:37:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=0Ifc9M3/T3Gpinmp5zHYu47Xs9+QxkkkW2XJKkdvdvg=; b=I2NTTsZJdSVRZHD29TWVZi67AFTB5u7jyqTsEqMpQiUd/vq2EFPrdRr9ocOUMP+ppi rpHyfeiG9Drti7kLdcURFFwq2Zu95h5HClval39VcMDIBsbt2fNZFeN5AUBl57KJzVi9 aYsUEj/68MfS8j3TiIoiajSL0q0bpsEFEdWW4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=0Ifc9M3/T3Gpinmp5zHYu47Xs9+QxkkkW2XJKkdvdvg=; b=cbW1ydOLK+HP9coQ2uVzGy6tlsdIULl6YaPbkVmho7TykbtedY7gR9wkUpSVqyAUpW 0Efh8DHbr1mKLpa/BAAK8l5k6I1m4xNwjjte+WOJCGhqo7C0p5XpILajlg9QbNDKi3LH Rm/mh+iqsoer8NXvVfJUsu6o5uuydnTwUptgG5qSNwqW/vEYpRN2H89m66nCf0Nf+m6B mWl3JtYxHe4SIlKV9VRfLdFdX1CHy4sDYeFddvRQdsEkKBXTynrvlD49vB8Zm/DB7eJq C5oyZGgJ7oFwcBoQ03eFylYF/hPxNiuPHh+4cVomnEy9M8iVr4wFzySLqSC9a4/MejKk wFmA== X-Gm-Message-State: AN3rC/6Wel5OWibvq6PDp0dhVk/lEyUotoR1LoeQ6r6onGFLyjejfciw 5w1I7eBfJsUXL2M6 X-Received: by 10.28.147.198 with SMTP id v189mr4806110wmd.52.1493980664588; Fri, 05 May 2017 03:37:44 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id j133sm1908331wmg.26.2017.05.05.03.37.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id CDDA73E01B2; Fri, 5 May 2017 11:38:22 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:15 +0100 Message-Id: <20170505103822.20641-3-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22b Subject: [Qemu-devel] [RFC PATCH v1 2/9] cpus: push BQL lock to qemu_*_wait_io_event X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Eduardo Habkost , "open list:Overall" , Peter Crosthwaite , qemu-devel@nongnu.org, cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , Richard Henderson Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" We only really need to grab the lock for initial setup (so we don't race with the thread-spawning thread). After that we can drop the lock for the whole main loop and only grab it for waiting for IO events. There is a slight wrinkle for the round-robin TCG thread as we also expire timers which needs to be done under BQL as they are in the main-loop. This is stage one of reducing the lock impact as we can drop the requirement of implicit BQL for async work and only grab the lock when we need to sleep on the cpu->halt_cond. Signed-off-by: Alex Bennée --- cpus.c | 27 ++++++++++++++++++++------- kvm-all.c | 4 ---- target/i386/hax-all.c | 2 -- 3 files changed, 20 insertions(+), 13 deletions(-) -- 2.11.0 diff --git a/cpus.c b/cpus.c index 740b8dc3f8..89ae8cb30a 100644 --- a/cpus.c +++ b/cpus.c @@ -1069,6 +1069,8 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (qemu_tcg_should_sleep(cpu)) { stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); @@ -1077,15 +1079,21 @@ static void qemu_tcg_wait_io_event(CPUState *cpu) start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { + qemu_mutex_lock_iothread(); + while (cpu_thread_is_idle(cpu)) { qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); } qemu_wait_io_event_common(cpu); + + qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg) @@ -1111,6 +1119,8 @@ static void *qemu_kvm_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); + qemu_cond_signal(&qemu_cpu_cond); do { @@ -1153,10 +1163,10 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) /* signal CPU creation */ cpu->created = true; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { - qemu_mutex_unlock_iothread(); do { int sig; r = sigwait(&waitset, &sig); @@ -1167,6 +1177,7 @@ static void *qemu_dummy_cpu_thread_fn(void *arg) } qemu_mutex_lock_iothread(); qemu_wait_io_event_common(cpu); + qemu_mutex_unlock_iothread(); } return NULL; @@ -1255,11 +1266,9 @@ static int tcg_cpu_exec(CPUState *cpu) #ifdef CONFIG_PROFILER ti = profile_getclock(); #endif - qemu_mutex_unlock_iothread(); cpu_exec_start(cpu); ret = cpu_exec(cpu); cpu_exec_end(cpu); - qemu_mutex_lock_iothread(); #ifdef CONFIG_PROFILER tcg_time += profile_getclock() - ti; #endif @@ -1318,6 +1327,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) qemu_wait_io_event_common(cpu); } } + qemu_mutex_unlock_iothread(); start_tcg_kick_timer(); @@ -1327,6 +1337,9 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu->exit_request = 1; while (1) { + + qemu_mutex_lock_iothread(); + /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ qemu_account_warp_timer(); @@ -1335,6 +1348,8 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) */ handle_icount_deadline(); + qemu_mutex_unlock_iothread(); + if (!cpu) { cpu = first_cpu; } @@ -1360,9 +1375,7 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) cpu_handle_guest_debug(cpu); break; } else if (r == EXCP_ATOMIC) { - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); break; } } else if (cpu->stop) { @@ -1403,6 +1416,7 @@ static void *qemu_hax_cpu_thread_fn(void *arg) current_cpu = cpu; hax_init_vcpu(cpu); + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); while (1) { @@ -1452,6 +1466,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) cpu->created = true; cpu->can_do_io = 1; current_cpu = cpu; + qemu_mutex_unlock_iothread(); qemu_cond_signal(&qemu_cpu_cond); /* process any pending work */ @@ -1476,9 +1491,7 @@ static void *qemu_tcg_cpu_thread_fn(void *arg) g_assert(cpu->halted); break; case EXCP_ATOMIC: - qemu_mutex_unlock_iothread(); cpu_exec_step_atomic(cpu); - qemu_mutex_lock_iothread(); default: /* Ignore everything else? */ break; diff --git a/kvm-all.c b/kvm-all.c index 90b8573656..4cb5347205 100644 --- a/kvm-all.c +++ b/kvm-all.c @@ -1963,8 +1963,6 @@ int kvm_cpu_exec(CPUState *cpu) return EXCP_HLT; } - qemu_mutex_unlock_iothread(); - do { MemTxAttrs attrs; @@ -2093,8 +2091,6 @@ int kvm_cpu_exec(CPUState *cpu) } } while (ret == 0); - qemu_mutex_lock_iothread(); - if (ret < 0) { cpu_dump_state(cpu, stderr, fprintf, CPU_DUMP_CODE); vm_stop(RUN_STATE_INTERNAL_ERROR); diff --git a/target/i386/hax-all.c b/target/i386/hax-all.c index ef13015215..71f3c01977 100644 --- a/target/i386/hax-all.c +++ b/target/i386/hax-all.c @@ -513,9 +513,7 @@ static int hax_vcpu_hax_exec(CPUArchState *env) hax_vcpu_interrupt(env); - qemu_mutex_unlock_iothread(); hax_ret = hax_vcpu_run(vcpu); - qemu_mutex_lock_iothread(); current_cpu = cpu; /* Simply continue the vcpu_run if system call interrupted */ From patchwork Fri May 5 10:38:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98610 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp49808qge; Fri, 5 May 2017 03:39:38 -0700 (PDT) X-Received: by 10.237.53.124 with SMTP id b57mr39644266qte.240.1493980778829; Fri, 05 May 2017 03:39:38 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id j12si3857654qta.322.2017.05.05.03.39.38 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:39:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46171 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6ae8-0002p2-CZ for patch@linaro.org; Fri, 05 May 2017 06:39:36 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52547) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acO-0001kK-Oe for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acM-0006ht-PS for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: from mail-wr0-x236.google.com ([2a00:1450:400c:c0c::236]:35806) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acM-0006hP-Ii for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:46 -0400 Received: by mail-wr0-x236.google.com with SMTP id z52so1382916wrc.2 for ; Fri, 05 May 2017 03:37:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=J7pbJPElMpl2w4dxyhLtwY0a9V8tQ9I8PPeYNnuqM4o=; b=fTRnoMWkv2fUcG7c4F9/wxpVxvPbMnzLtdvJ0xeGfliOM+eoUH0fKSknw2M3xGUlNa k9hFq18efmdVTNQ+1QM1n95fO/tUjFaYky8tMgxYDdpR3lzuQxoZGv7482V/F7aoh8Cc 800Cl2WdM6IpIDUi442HIpbyoKeiYYpZkzkyo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=J7pbJPElMpl2w4dxyhLtwY0a9V8tQ9I8PPeYNnuqM4o=; b=W1aA9Vr9l3owLxHEENIeToQRLm9CtS3gKr085zuNUWEU229ysi+dkPIWsLz+qG3x8Z /HqiEyPgtRCCDNQxnvsmqaabziTgzlCCwiOeiOs3QICuMtF5SJtBFCR1u1lMSQXrbgHZ qeIDdem+1mNjLRPQm58+2igJeNG8TdkVXyrE9aH6LGYz7rZ/cW+kcEKaJKvOQYR4DDRL 7sjwMhPKU9CiQzs9uB8csR8NRyJhI0QfrrD+nupGoRRpDJBQNSM3OkmO4Wdn0ATcZ+Sy vY0PC/qX6XGD8qXvEWtMKfHGTHa7SvGgA8SPwJnb7FBlozj+271HBI9E4jVxyKx2Yosf wH7Q== X-Gm-Message-State: AN3rC/5dhr83VCuDp+OQWSGyA5mcyOz4I3LDAkGqxgUOJEaRTLwEQMnq ioBIoohJlr0wHGvd X-Received: by 10.223.169.34 with SMTP id u31mr36586826wrc.46.1493980665456; Fri, 05 May 2017 03:37:45 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 184sm1733953wmr.30.2017.05.05.03.37.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id DFA8F3E01BB; Fri, 5 May 2017 11:38:22 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:16 +0100 Message-Id: <20170505103822.20641-4-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c0c::236 Subject: [Qemu-devel] [RFC PATCH v1 3/9] cpus: only take BQL for sleeping threads X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Richard Henderson , cota@braap.org, =?utf-8?q?Alex_Be?= =?utf-8?b?bm7DqWU=?= , qemu-devel@nongnu.org, Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now the only real need to hold the BQL is for when we sleep on the cpu->halt conditional. The lock is actually dropped while the thread sleeps so the actual window for contention is pretty small. This also means we can remove the special case hack for exclusive work and simply declare that work no longer has an implicit BQL held. This isn't a major problem async work is generally only changing things in the context of its own vCPU. If it needs to work across vCPUs it should be using the exclusive mechanism or possibly taking the lock itself. Signed-off-by: Alex Bennée --- cpus-common.c | 13 +++++-------- cpus.c | 10 ++++------ 2 files changed, 9 insertions(+), 14 deletions(-) -- 2.11.0 diff --git a/cpus-common.c b/cpus-common.c index 59f751ecf9..64661c3193 100644 --- a/cpus-common.c +++ b/cpus-common.c @@ -310,6 +310,11 @@ void async_safe_run_on_cpu(CPUState *cpu, run_on_cpu_func func, queue_work_on_cpu(cpu, wi); } +/* Work items run outside of the BQL. This is essential for avoiding a + * deadlock for exclusive work but also applies to non-exclusive work. + * If the work requires cross-vCPU changes then it should use the + * exclusive mechanism. + */ void process_queued_cpu_work(CPUState *cpu) { struct qemu_work_item *wi; @@ -327,17 +332,9 @@ void process_queued_cpu_work(CPUState *cpu) } qemu_mutex_unlock(&cpu->work_mutex); if (wi->exclusive) { - /* Running work items outside the BQL avoids the following deadlock: - * 1) start_exclusive() is called with the BQL taken while another - * CPU is running; 2) cpu_exec in the other CPU tries to takes the - * BQL, so it goes to sleep; start_exclusive() is sleeping too, so - * neither CPU can proceed. - */ - qemu_mutex_unlock_iothread(); start_exclusive(); wi->func(cpu, wi->data); end_exclusive(); - qemu_mutex_lock_iothread(); } else { wi->func(cpu, wi->data); } diff --git a/cpus.c b/cpus.c index 89ae8cb30a..df279dd320 100644 --- a/cpus.c +++ b/cpus.c @@ -1069,31 +1069,29 @@ static bool qemu_tcg_should_sleep(CPUState *cpu) static void qemu_tcg_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (qemu_tcg_should_sleep(cpu)) { + qemu_mutex_lock_iothread(); stop_tcg_kick_timer(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } start_tcg_kick_timer(); qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void qemu_kvm_wait_io_event(CPUState *cpu) { - qemu_mutex_lock_iothread(); while (cpu_thread_is_idle(cpu)) { + qemu_mutex_lock_iothread(); qemu_cond_wait(cpu->halt_cond, &qemu_global_mutex); + qemu_mutex_unlock_iothread(); } qemu_wait_io_event_common(cpu); - - qemu_mutex_unlock_iothread(); } static void *qemu_kvm_cpu_thread_fn(void *arg) From patchwork Fri May 5 10:38:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98612 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp50152qge; Fri, 5 May 2017 03:40:32 -0700 (PDT) X-Received: by 10.200.52.157 with SMTP id w29mr7177124qtb.161.1493980832497; Fri, 05 May 2017 03:40:32 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id m33si4507408qta.202.2017.05.05.03.40.32 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:40:32 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46180 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6af0-0003wW-5h for patch@linaro.org; Fri, 05 May 2017 06:40:30 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52549) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acO-0001kM-Ow for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:49 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acN-0006i3-Fb for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]:38065) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acN-0006hm-8m for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:47 -0400 Received: by mail-wm0-x22d.google.com with SMTP id 142so2194341wma.1 for ; Fri, 05 May 2017 03:37:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=1WFbFJNSotD2gTXV1zW0vWegCKhbzIF0g3lK6fXNy1Q=; b=aS3KR0Jl2GevSgXBSaKiPCcP5E0N8qbfSp13J/Nt1aho0JoEjrH9rCTqNIbI4UIG97 eyiAGZ/275/0CHHvzHslduvigz7vkTRmnt6lxjHw3Y9c0w0SSUHoxrFpuw9kjDYKJLLx vkLBmSWnhxopnYCoqIo5qTgrFw2maFsbUprnM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=1WFbFJNSotD2gTXV1zW0vWegCKhbzIF0g3lK6fXNy1Q=; b=FEvFVTjTiDu+bLWN90jvEzmQBt36YSlsQV6VUdx6qzcCu6PKkDfyPr5S/pMlY1CugY mwRwP5/0iQKT1y6z2C96vpa0omOPEUPCrSFtUqbeYF2m/9b1xfg8fnijVAZkGsHmchp1 PVH35ZRV11aoN/gt9OsgYBKWfXaxgqS9+S12Efwfsl+cv2iTAhFY7Lop8Cz9IAlScswb 16/fiSVsyEOLXO3lbdNP3H65B3eKiM98dPmDL/v3W1rNuWOmit+qY6GOx95qxwCQVVeG Kr2mU/aJbJtMLOVwf9mQH1fXwlJ5UAhV8l23pVfNG6lppQLE4Mg2G4Fh2MkOh5iie10U SuSA== X-Gm-Message-State: AN3rC/5Hf6DihN1SoHaqZudfAZKxy0Bv2TnKvGj/0RNR9xIDO8hTqIBF 8caYNPvXS9JE95Ho X-Received: by 10.28.16.212 with SMTP id 203mr4479186wmq.26.1493980666192; Fri, 05 May 2017 03:37:46 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id t26sm5136118wra.43.2017.05.05.03.37.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id EFFBE3E01C6; Fri, 5 May 2017 11:38:22 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:17 +0100 Message-Id: <20170505103822.20641-5-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22d Subject: [Qemu-devel] [RFC PATCH v1 4/9] replay/replay-internal.c: track holding of replay_lock X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This is modelled after the iothread mutex lock. We keep a TLS flag to indicate when that thread has acquired the lock and assert we don't double-lock or release when we shouldn't have. Signed-off-by: Alex Bennée --- replay/replay-internal.c | 11 +++++++++++ 1 file changed, 11 insertions(+) -- 2.11.0 diff --git a/replay/replay-internal.c b/replay/replay-internal.c index fca8514012..157c863e25 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -179,13 +179,24 @@ void replay_mutex_destroy(void) qemu_mutex_destroy(&lock); } +static __thread bool replay_locked; + +static bool replay_mutex_locked(void) +{ + return replay_locked; +} + void replay_mutex_lock(void) { + g_assert(!replay_mutex_locked()); qemu_mutex_lock(&lock); + replay_locked = true; } void replay_mutex_unlock(void) { + g_assert(replay_mutex_locked()); + replay_locked = false; qemu_mutex_unlock(&lock); } From patchwork Fri May 5 10:38:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98615 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp51441qge; Fri, 5 May 2017 03:44:07 -0700 (PDT) X-Received: by 10.233.235.72 with SMTP id b69mr12142484qkg.206.1493981047132; Fri, 05 May 2017 03:44:07 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id p123si4372909qkf.234.2017.05.05.03.44.06 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:44:07 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46194 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6aiS-0007BM-L9 for patch@linaro.org; Fri, 05 May 2017 06:44:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52590) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acQ-0001ld-Ko for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:51 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acP-0006kX-K1 for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:50 -0400 Received: from mail-wm0-x230.google.com ([2a00:1450:400c:c09::230]:38077) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acP-0006iW-Ej for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:49 -0400 Received: by mail-wm0-x230.google.com with SMTP id 142so2195248wma.1 for ; Fri, 05 May 2017 03:37:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=UjuCmPJ4ZrsILx8Tqiu2TOS2wJzIiENPvjExPUWXnYE=; b=ahC3aURoMHjDQtol17jLB74i2u9wx8XgXZzI3c8kgn6aSJczbIwnpxcl/phDhdIYFl ZLya8NxipsxbP8z03Tf+hqWNoyGcgJJyqYHRbq1uBZAfSk7QBexK50bwhP9l8czGCdYc n6ZfOUDAt+TCf4c3BERkjzCdd4tzti7lWkl/A= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=UjuCmPJ4ZrsILx8Tqiu2TOS2wJzIiENPvjExPUWXnYE=; b=fVZhJg4mVdg4P50ABaraOjm3GS3jgYBBXpgYzWFWZFunxkvhklzth3VSoaofTl/A1l aJC0xcGSqmA8XnPF4LsxcUIjzlpvlXXe/h1jn/Oyd8O87ftBMcZ5zgAy8xuSCxuz9G1T s+hxBAYBuU0oSWaEeLdcuZDsiGr8UipKC8grALHOoYSo6G9ciADgporkB9DHqCBgBFoS ceI3shu2DlRKJOV4PisAA9sNt2FlcvR0vzgxajcrFjWfmqNeFW5/d+6l26p099AQKPg1 2MCW/onlPAOgf59E7qnMPEmmn2VTmWUA3XgJ4mY/2Ocl8x48EvwhnV3KyxfGpK4s+mAj R/lA== X-Gm-Message-State: AN3rC/4pLvANQn5f7yACyRhkEXizSk+0PA5fdOcX5wq8F06jP8FGt4bp zePl1sWeDkqV0ABz X-Received: by 10.28.69.203 with SMTP id l72mr4841739wmi.75.1493980668320; Fri, 05 May 2017 03:37:48 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 136sm1928463wmx.25.2017.05.05.03.37.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 0FA193E01C7; Fri, 5 May 2017 11:38:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:18 +0100 Message-Id: <20170505103822.20641-6-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::230 Subject: [Qemu-devel] [RFC PATCH v1 5/9] replay: make locking visible outside replay code X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" The replay_mutex_lock/unlock/locked functions are now going to be used for ensuring lock-step behaviour between the two threads. Make them public API functions and also provide stubs for non-QEMU builds on common paths. Signed-off-by: Alex Bennée --- include/sysemu/replay.h | 14 ++++++++++++++ replay/replay-internal.c | 5 ++--- replay/replay-internal.h | 5 ++--- stubs/replay.c | 15 +++++++++++++++ 4 files changed, 33 insertions(+), 6 deletions(-) -- 2.11.0 diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h index f1c0712795..08bffa2dc2 100644 --- a/include/sysemu/replay.h +++ b/include/sysemu/replay.h @@ -46,6 +46,20 @@ extern ReplayMode replay_mode; /* Name of the initial VM snapshot */ extern char *replay_snapshot; +/* Replay locking + * + * The locks are needed to protect the shared structures and log file + * when doing record/replay. They also are the main sync-point between + * the main-loop thread and the vCPU thread. This was a role + * previously filled by the BQL which has been busy trying to reduce + * its impact across the code. This ensures blocks of events stay + * sequential and reproducible. + */ + +void replay_mutex_lock(void); +void replay_mutex_unlock(void); +bool replay_mutex_locked(void); + /* Replay process control functions */ /*! Enables recording or saving event log with specified parameters */ diff --git a/replay/replay-internal.c b/replay/replay-internal.c index 157c863e25..e6b2fdb6c1 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -181,7 +181,7 @@ void replay_mutex_destroy(void) static __thread bool replay_locked; -static bool replay_mutex_locked(void) +bool replay_mutex_locked(void) { return replay_locked; } @@ -204,7 +204,7 @@ void replay_mutex_unlock(void) void replay_save_instructions(void) { if (replay_file && replay_mode == REPLAY_MODE_RECORD) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); int diff = (int)(replay_get_current_step() - replay_state.current_step); /* Time can only go forward */ @@ -215,6 +215,5 @@ void replay_save_instructions(void) replay_put_dword(diff); replay_state.current_step += diff; } - replay_mutex_unlock(); } } diff --git a/replay/replay-internal.h b/replay/replay-internal.h index ed66ed803c..b0cf78f16f 100644 --- a/replay/replay-internal.h +++ b/replay/replay-internal.h @@ -97,12 +97,11 @@ int64_t replay_get_qword(void); void replay_get_array(uint8_t *buf, size_t *size); void replay_get_array_alloc(uint8_t **buf, size_t *size); -/* Mutex functions for protecting replay log file */ +/* Mutex functions for protecting replay log file and ensuring + * synchronisation between vCPU and main-loop threads. */ void replay_mutex_init(void); void replay_mutex_destroy(void); -void replay_mutex_lock(void); -void replay_mutex_unlock(void); /*! Checks error status of the file. */ void replay_check_error(void); diff --git a/stubs/replay.c b/stubs/replay.c index 9c8aa48c9c..39c5746043 100644 --- a/stubs/replay.c +++ b/stubs/replay.c @@ -72,3 +72,18 @@ uint64_t blkreplay_next_id(void) { return 0; } + +void replay_mutex_lock(void) +{ + abort(); +} + +void replay_mutex_unlock(void) +{ + abort(); +} + +bool replay_mutex_locked(void) +{ + return false; +} From patchwork Fri May 5 10:38:19 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98613 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp50947qge; Fri, 5 May 2017 03:42:38 -0700 (PDT) X-Received: by 10.55.129.66 with SMTP id c63mr3452057qkd.183.1493980958516; Fri, 05 May 2017 03:42:38 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id v9si4431734qtc.188.2017.05.05.03.42.37 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:42:38 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46186 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6ah1-0005zG-UB for patch@linaro.org; Fri, 05 May 2017 06:42:35 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52598) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acR-0001lt-8S for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acO-0006k2-UT for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:51 -0400 Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]:38073) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acO-0006iN-K7 for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:48 -0400 Received: by mail-wm0-x22d.google.com with SMTP id 142so2194967wma.1 for ; Fri, 05 May 2017 03:37:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=f+pDGak0AvfQ3GalV2g4j61QVeXI9rn0C+l/CZRDUCA=; b=X1ezSvdAfnlDbU/vBIO0cS5ra/tw44oHvYIlOzpuGZwXeoXUDa9iZ+/3J1s0KAYa3i 0Pg3pxCpV4oWG3r8nQj49ycqfO4MiQowDc/NhPTt9VRXZQJABHPeSMw8wZYotWGkh6To vdFY2Qci27R4G9t55yJovPHGGFNSBlY9B7fVI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=f+pDGak0AvfQ3GalV2g4j61QVeXI9rn0C+l/CZRDUCA=; b=QKgeJO3u6jUvCHR41nUKGsHcb7OWx/6HDjK+omC7XMwNx50Krfp2TuYOBggmbvmUQ5 fXrsCGiIlzF5mixRRZCrGJA8dRm3odu0Yruugiy/MK3pd6KLzb1xfG97gakIRzTMoBdH XeB3Er2ZpuJ1qZ/aZHPYqaiq/kcWGWTTNDej3I4pTYXIZkqCwqIm2BM4rvv0r+UFHaB+ eK5ZXXEzgYEqof9gEc0Fik+zPeDXWrDo5RPfC0v03DXGN/ByOgcL9bHpQDMRS1cL/xfQ IONyrYHaYPx/8PH64Xb7nGN4P14PSuc0J9hXUDL95nmEGuYzvhjEHc4HsNjpTBr3CdvX kSDQ== X-Gm-Message-State: AN3rC/61n3zJFYZlwSYqVE1lxYSMHv2wI8HPpbQ+doEM+kOe1FGbdZzQ gJpqcZ+pvI+xPMXC X-Received: by 10.28.0.200 with SMTP id 191mr5143650wma.12.1493980667416; Fri, 05 May 2017 03:37:47 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 92sm4930962wra.0.2017.05.05.03.37.42 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:43 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 293A43E01F8; Fri, 5 May 2017 11:38:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:19 +0100 Message-Id: <20170505103822.20641-7-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22d Subject: [Qemu-devel] [RFC PATCH v1 6/9] replay: push replay_mutex_lock up the call tree X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Richard Henderson , cota@braap.org, =?utf-8?q?Alex_Be?= =?utf-8?b?bm7DqWU=?= , qemu-devel@nongnu.org, Peter Crosthwaite Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Now instead of using the replay_lock to guard the output of the log we now use it to protect the whole execution section. This replaces what the BQL used to do when it was held during TCG execution. We also introduce some rules for locking order - mainly that you cannot take the replay_mutex while holding the BQL. This leads to some slight sophistry during start-up and extending the replay_mutex_destroy function to unlock the mutex without checking for the BQL condition so it can be cleanly dropped in the non-replay case. Signed-off-by: Alex Bennée --- cpus.c | 32 ++++++++++++++++++++++++++++++++ docs/replay.txt | 19 +++++++++++++++++++ include/sysemu/replay.h | 2 ++ replay/replay-char.c | 21 ++++++++------------- replay/replay-events.c | 18 +++++------------- replay/replay-internal.c | 18 +++++++++++++----- replay/replay-time.c | 10 +++++----- replay/replay.c | 40 ++++++++++++++++++++-------------------- util/main-loop.c | 23 ++++++++++++++++++++--- vl.c | 2 ++ 10 files changed, 126 insertions(+), 59 deletions(-) -- 2.11.0 diff --git a/cpus.c b/cpus.c index df279dd320..7cb6719f40 100644 --- a/cpus.c +++ b/cpus.c @@ -1235,6 +1235,10 @@ static void prepare_icount_for_run(CPUState *cpu) insns_left = MIN(0xffff, cpu->icount_budget); cpu->icount_decr.u16.low = insns_left; cpu->icount_extra = cpu->icount_budget - insns_left; + + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_lock(); + } } } @@ -1250,6 +1254,10 @@ static void process_icount_data(CPUState *cpu) cpu->icount_budget = 0; replay_account_executed_instructions(); + + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_unlock(); + } } } @@ -1336,6 +1344,10 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) while (1) { + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_lock(); + } + qemu_mutex_lock_iothread(); /* Account partial waits to QEMU_CLOCK_VIRTUAL. */ @@ -1348,6 +1360,10 @@ static void *qemu_tcg_rr_cpu_thread_fn(void *arg) qemu_mutex_unlock_iothread(); + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_unlock(); + } + if (!cpu) { cpu = first_cpu; } @@ -1611,12 +1627,28 @@ void pause_all_vcpus(void) cpu_stop_current(); } + /* We need to drop the replay_lock so any vCPU threads woken up + * can finish their replay tasks + */ + if (replay_mode != REPLAY_MODE_NONE) { + g_assert(replay_mutex_locked()); + qemu_mutex_unlock_iothread(); + replay_mutex_unlock(); + qemu_mutex_lock_iothread(); + } + while (!all_vcpus_paused()) { qemu_cond_wait(&qemu_pause_cond, &qemu_global_mutex); CPU_FOREACH(cpu) { qemu_cpu_kick(cpu); } } + + if (replay_mode != REPLAY_MODE_NONE) { + qemu_mutex_unlock_iothread(); + replay_mutex_lock(); + qemu_mutex_lock_iothread(); + } } void cpu_resume(CPUState *cpu) diff --git a/docs/replay.txt b/docs/replay.txt index 486c1e0e9d..e0198785bc 100644 --- a/docs/replay.txt +++ b/docs/replay.txt @@ -49,6 +49,25 @@ Modifications of qemu include: * recording/replaying user input (mouse and keyboard) * adding internal checkpoints for cpu and io synchronization +Locking and thread synchronisation +---------------------------------- + +Previously the synchronisation of the main thread and the vCPU thread +was ensured by the holding of the BQL. However the trend has been to +reduce the time the BQL was held across the system including under TCG +system emulation. As it is important that batches of events are kept +in sequence (e.g. expiring timers and checkpoints in the main thread +while instruction checkpoints are written by the vCPU thread) we need +another lock to keep things in lock-step. This role is now handled by +the replay_mutex_lock. It used to be held only for each event being +written but now it is held for a whole execution period. This results +in a deterministic ping-pong between the two main threads. + +As deadlocks are easy to introduce a new rule is introduced that the +replay_mutex_lock is taken before any BQL locks. Conversely you cannot +release the replay_lock while the BQL is still held. + + Non-deterministic events ------------------------ diff --git a/include/sysemu/replay.h b/include/sysemu/replay.h index 08bffa2dc2..2bf075f8ac 100644 --- a/include/sysemu/replay.h +++ b/include/sysemu/replay.h @@ -62,6 +62,8 @@ bool replay_mutex_locked(void); /* Replay process control functions */ +/*! Enables and take replay locks (even if we don't use it) */ +void replay_init_locks(void); /*! Enables recording or saving event log with specified parameters */ void replay_configure(struct QemuOpts *opts); /*! Initializes timers used for snapshotting and enables events recording */ diff --git a/replay/replay-char.c b/replay/replay-char.c index aa65955942..2911afc8f2 100755 --- a/replay/replay-char.c +++ b/replay/replay-char.c @@ -96,25 +96,24 @@ void *replay_event_char_read_load(void) void replay_char_write_event_save(int res, int offset) { + g_assert(replay_mutex_locked()); + replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_WRITE); replay_put_dword(res); replay_put_dword(offset); - replay_mutex_unlock(); } void replay_char_write_event_load(int *res, int *offset) { + g_assert(replay_mutex_locked()); + replay_account_executed_instructions(); - replay_mutex_lock(); if (replay_next_event_is(EVENT_CHAR_WRITE)) { *res = replay_get_dword(); *offset = replay_get_dword(); replay_finish_event(); - replay_mutex_unlock(); } else { - replay_mutex_unlock(); error_report("Missing character write event in the replay log"); exit(1); } @@ -122,23 +121,21 @@ void replay_char_write_event_load(int *res, int *offset) int replay_char_read_all_load(uint8_t *buf) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + if (replay_next_event_is(EVENT_CHAR_READ_ALL)) { size_t size; int res; replay_get_array(buf, &size); replay_finish_event(); - replay_mutex_unlock(); res = (int)size; assert(res >= 0); return res; } else if (replay_next_event_is(EVENT_CHAR_READ_ALL_ERROR)) { int res = replay_get_dword(); replay_finish_event(); - replay_mutex_unlock(); return res; } else { - replay_mutex_unlock(); error_report("Missing character read all event in the replay log"); exit(1); } @@ -146,19 +143,17 @@ int replay_char_read_all_load(uint8_t *buf) void replay_char_read_all_save_error(int res) { + g_assert(replay_mutex_locked()); assert(res < 0); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_READ_ALL_ERROR); replay_put_dword(res); - replay_mutex_unlock(); } void replay_char_read_all_save_buf(uint8_t *buf, int offset) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_CHAR_READ_ALL); replay_put_array(buf, offset); - replay_mutex_unlock(); } diff --git a/replay/replay-events.c b/replay/replay-events.c index 94a6dcccfc..f96c24cb4b 100644 --- a/replay/replay-events.c +++ b/replay/replay-events.c @@ -77,16 +77,14 @@ bool replay_has_events(void) void replay_flush_events(void) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } - replay_mutex_unlock(); } void replay_disable_events(void) @@ -100,14 +98,14 @@ void replay_disable_events(void) void replay_clear_events(void) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } - replay_mutex_unlock(); } /*! Adds specified async event to the queue */ @@ -134,9 +132,8 @@ void replay_add_event(ReplayAsyncEventKind event_kind, event->opaque2 = opaque2; event->id = id; - replay_mutex_lock(); + g_assert(replay_mutex_locked()); QTAILQ_INSERT_TAIL(&events_list, event, events); - replay_mutex_unlock(); } void replay_bh_schedule_event(QEMUBH *bh) @@ -208,10 +205,7 @@ void replay_save_events(int checkpoint) while (!QTAILQ_EMPTY(&events_list)) { Event *event = QTAILQ_FIRST(&events_list); replay_save_event(event, checkpoint); - - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); QTAILQ_REMOVE(&events_list, event, events); g_free(event); } @@ -295,9 +289,7 @@ void replay_read_events(int checkpoint) if (!event) { break; } - replay_mutex_unlock(); replay_run_event(event); - replay_mutex_lock(); g_free(event); replay_finish_event(); diff --git a/replay/replay-internal.c b/replay/replay-internal.c index e6b2fdb6c1..d036a0259b 100644 --- a/replay/replay-internal.c +++ b/replay/replay-internal.c @@ -174,11 +174,6 @@ void replay_mutex_init(void) qemu_mutex_init(&lock); } -void replay_mutex_destroy(void) -{ - qemu_mutex_destroy(&lock); -} - static __thread bool replay_locked; bool replay_mutex_locked(void) @@ -186,15 +181,28 @@ bool replay_mutex_locked(void) return replay_locked; } +void replay_mutex_destroy(void) +{ + if (replay_mutex_locked()) { + qemu_mutex_unlock(&lock); + } + qemu_mutex_destroy(&lock); +} + + +/* Ordering constraints, replay_lock must be taken before BQL */ void replay_mutex_lock(void) { + g_assert(!qemu_mutex_iothread_locked()); g_assert(!replay_mutex_locked()); qemu_mutex_lock(&lock); replay_locked = true; } +/* BQL can't be held when releasing the replay_lock */ void replay_mutex_unlock(void) { + g_assert(!qemu_mutex_iothread_locked()); g_assert(replay_mutex_locked()); replay_locked = false; qemu_mutex_unlock(&lock); diff --git a/replay/replay-time.c b/replay/replay-time.c index f70382a88f..6a7565ec8d 100644 --- a/replay/replay-time.c +++ b/replay/replay-time.c @@ -17,13 +17,13 @@ int64_t replay_save_clock(ReplayClockKind kind, int64_t clock) { - replay_save_instructions(); if (replay_file) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); + + replay_save_instructions(); replay_put_event(EVENT_CLOCK + kind); replay_put_qword(clock); - replay_mutex_unlock(); } return clock; @@ -46,16 +46,16 @@ void replay_read_next_clock(ReplayClockKind kind) /*! Reads next clock event from the input. */ int64_t replay_read_clock(ReplayClockKind kind) { + g_assert(replay_file && replay_mutex_locked()); + replay_account_executed_instructions(); if (replay_file) { int64_t ret; - replay_mutex_lock(); if (replay_next_event_is(EVENT_CLOCK + kind)) { replay_read_next_clock(kind); } ret = replay_state.cached_clock[kind]; - replay_mutex_unlock(); return ret; } diff --git a/replay/replay.c b/replay/replay.c index f810628cac..b79d4db3ba 100644 --- a/replay/replay.c +++ b/replay/replay.c @@ -79,8 +79,9 @@ int replay_get_instructions(void) void replay_account_executed_instructions(void) { + g_assert(replay_mutex_locked()); + if (replay_mode == REPLAY_MODE_PLAY) { - replay_mutex_lock(); if (replay_state.instructions_count > 0) { int count = (int)(replay_get_current_step() - replay_state.current_step); @@ -99,24 +100,22 @@ void replay_account_executed_instructions(void) qemu_notify_event(); } } - replay_mutex_unlock(); } } bool replay_exception(void) { + if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_EXCEPTION); - replay_mutex_unlock(); return true; } else if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); bool res = replay_has_exception(); if (res) { - replay_mutex_lock(); replay_finish_event(); - replay_mutex_unlock(); } return res; } @@ -128,10 +127,9 @@ bool replay_has_exception(void) { bool res = false; if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); replay_account_executed_instructions(); - replay_mutex_lock(); res = replay_next_event_is(EVENT_EXCEPTION); - replay_mutex_unlock(); } return res; @@ -140,17 +138,15 @@ bool replay_has_exception(void) bool replay_interrupt(void) { if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_save_instructions(); - replay_mutex_lock(); replay_put_event(EVENT_INTERRUPT); - replay_mutex_unlock(); return true; } else if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); bool res = replay_has_interrupt(); if (res) { - replay_mutex_lock(); replay_finish_event(); - replay_mutex_unlock(); } return res; } @@ -162,10 +158,9 @@ bool replay_has_interrupt(void) { bool res = false; if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); replay_account_executed_instructions(); - replay_mutex_lock(); res = replay_next_event_is(EVENT_INTERRUPT); - replay_mutex_unlock(); } return res; } @@ -173,9 +168,8 @@ bool replay_has_interrupt(void) void replay_shutdown_request(void) { if (replay_mode == REPLAY_MODE_RECORD) { - replay_mutex_lock(); + g_assert(replay_mutex_locked()); replay_put_event(EVENT_SHUTDOWN); - replay_mutex_unlock(); } } @@ -189,9 +183,9 @@ bool replay_checkpoint(ReplayCheckpoint checkpoint) return true; } - replay_mutex_lock(); if (replay_mode == REPLAY_MODE_PLAY) { + g_assert(replay_mutex_locked()); if (replay_next_event_is(EVENT_CHECKPOINT + checkpoint)) { replay_finish_event(); } else if (replay_state.data_kind != EVENT_ASYNC) { @@ -204,15 +198,21 @@ bool replay_checkpoint(ReplayCheckpoint checkpoint) checkpoint were processed */ res = replay_state.data_kind != EVENT_ASYNC; } else if (replay_mode == REPLAY_MODE_RECORD) { + g_assert(replay_mutex_locked()); replay_put_event(EVENT_CHECKPOINT + checkpoint); replay_save_events(checkpoint); res = true; } out: - replay_mutex_unlock(); return res; } +void replay_init_locks(void) +{ + replay_mutex_init(); + replay_mutex_lock(); /* Hold while we start-up */ +} + static void replay_enable(const char *fname, int mode) { const char *fmode = NULL; @@ -232,8 +232,6 @@ static void replay_enable(const char *fname, int mode) atexit(replay_finish); - replay_mutex_init(); - replay_file = fopen(fname, fmode); if (replay_file == NULL) { fprintf(stderr, "Replay: open %s: %s\n", fname, strerror(errno)); @@ -273,6 +271,8 @@ void replay_configure(QemuOpts *opts) Location loc; if (!opts) { + /* we no longer need this lock */ + replay_mutex_destroy(); return; } diff --git a/util/main-loop.c b/util/main-loop.c index 19cad6b8b6..ec661d1530 100644 --- a/util/main-loop.c +++ b/util/main-loop.c @@ -29,6 +29,7 @@ #include "qemu/sockets.h" // struct in_addr needed for libslirp.h #include "sysemu/qtest.h" #include "sysemu/cpus.h" +#include "sysemu/replay.h" #include "slirp/libslirp.h" #include "qemu/main-loop.h" #include "block/aio.h" @@ -245,19 +246,26 @@ static int os_host_main_loop_wait(int64_t timeout) timeout = SCALE_MS; } + if (timeout) { spin_counter = 0; - qemu_mutex_unlock_iothread(); } else { spin_counter++; } + qemu_mutex_unlock_iothread(); + + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_unlock(); + } ret = qemu_poll_ns((GPollFD *)gpollfds->data, gpollfds->len, timeout); - if (timeout) { - qemu_mutex_lock_iothread(); + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_lock(); } + qemu_mutex_lock_iothread(); + glib_pollfds_poll(); g_main_context_release(context); @@ -463,8 +471,17 @@ static int os_host_main_loop_wait(int64_t timeout) poll_timeout_ns = qemu_soonest_timeout(poll_timeout_ns, timeout); qemu_mutex_unlock_iothread(); + + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_unlock(); + } + g_poll_ret = qemu_poll_ns(poll_fds, n_poll_fds + w->num, poll_timeout_ns); + if (replay_mode != REPLAY_MODE_NONE) { + replay_mutex_lock(); + } + qemu_mutex_lock_iothread(); if (g_poll_ret > 0) { for (i = 0; i < w->num; i++) { diff --git a/vl.c b/vl.c index f46e070e0d..b00685adfb 100644 --- a/vl.c +++ b/vl.c @@ -2989,6 +2989,8 @@ int main(int argc, char **argv, char **envp) qemu_init_cpu_list(); qemu_init_cpu_loop(); + + replay_init_locks(); qemu_mutex_lock_iothread(); atexit(qemu_run_exit_notifiers); From patchwork Fri May 5 10:38:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98616 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp51687qge; Fri, 5 May 2017 03:44:47 -0700 (PDT) X-Received: by 10.55.78.15 with SMTP id c15mr13811200qkb.62.1493981087810; Fri, 05 May 2017 03:44:47 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id b71si2041061qka.47.2017.05.05.03.44.47 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:44:47 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46199 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6aj7-0007rv-9x for patch@linaro.org; Fri, 05 May 2017 06:44:45 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52617) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acS-0001nR-Du for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acR-0006n9-Hq for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:52 -0400 Received: from mail-wm0-x22d.google.com ([2a00:1450:400c:c09::22d]:36653) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acR-0006lv-C1 for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:51 -0400 Received: by mail-wm0-x22d.google.com with SMTP id u65so20032188wmu.1 for ; Fri, 05 May 2017 03:37:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=WMhlBMa+zXPOCyNJcczyZTic6ek6xBp/a7Sqlf0P5H0=; b=KfquEcEIiwW72E+00cm12jtRooGmPXOoXrxwKRcVrLf6IVACiRJBEk/qu99ZPBtq1+ xdk9/YyEBTNsDzU73i+oxcs5Kio5P0MmYLBCcBkXieH/jeCg0JdaF8UcYbZCuy+P7sQw hpjaR5JMlRd2k10jKN18B6U5I390uk2DXcOpE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=WMhlBMa+zXPOCyNJcczyZTic6ek6xBp/a7Sqlf0P5H0=; b=uVvLUDYNSKehApOiP75UpBIY1OnbBAZwYEiHigW/xrKj2pU3fXf0iY/4CG1JH2+rp1 KEj5AiwxhZI5eShwa3dgLpUIEWy43bEinSnQ1+cIYgyOac7XZyGLjKXaiJRmCj8sw2Qw xcPFzyNmZq7kgphP/TnLZamC9Sx5T+gpXp0w6eCDc7BCTq6d08P7omilfwZn3vHBb8Yc prz9CNJP9BgPvzxnY9zYKm/SU190R1QNe71+8ZvKo0ogloeA1Kffjwv18NCWLcifDBxh rsJUFVeBQT+8RjoJJHpePkISJWVPQ19726QgGRMGr7/5MLSfZOqhD5+rI3E4Kp7WPx1J 4Z4g== X-Gm-Message-State: AN3rC/5g3f5t74eAUycK4VsnnI/382skBfMUO8w8JiN8CYsj1znKk+1Q d79YkJQ1+4jZ4Vsq X-Received: by 10.28.232.72 with SMTP id f69mr4768918wmh.6.1493980670362; Fri, 05 May 2017 03:37:50 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id u200sm1834578wmd.16.2017.05.05.03.37.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:48 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 3A8753E0225; Fri, 5 May 2017 11:38:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:20 +0100 Message-Id: <20170505103822.20641-8-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::22d Subject: [Qemu-devel] [RFC PATCH v1 7/9] scripts/qemu-gdb: add simple tcg lock status helper X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Add a simple helper to dump lock state. Signed-off-by: Alex Bennée --- scripts/qemu-gdb.py | 3 ++- scripts/qemugdb/tcg.py | 46 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 48 insertions(+), 1 deletion(-) create mode 100644 scripts/qemugdb/tcg.py -- 2.11.0 diff --git a/scripts/qemu-gdb.py b/scripts/qemu-gdb.py index b3f8e04f77..d58213e549 100644 --- a/scripts/qemu-gdb.py +++ b/scripts/qemu-gdb.py @@ -26,7 +26,7 @@ import os, sys sys.path.append(os.path.dirname(__file__)) -from qemugdb import aio, mtree, coroutine +from qemugdb import aio, mtree, coroutine, tcg class QemuCommand(gdb.Command): '''Prefix for QEMU debug support commands''' @@ -38,6 +38,7 @@ QemuCommand() coroutine.CoroutineCommand() mtree.MtreeCommand() aio.HandlersCommand() +tcg.TCGLockStatusCommand() coroutine.CoroutineSPFunction() coroutine.CoroutinePCFunction() diff --git a/scripts/qemugdb/tcg.py b/scripts/qemugdb/tcg.py new file mode 100644 index 0000000000..8c7f1d7454 --- /dev/null +++ b/scripts/qemugdb/tcg.py @@ -0,0 +1,46 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- +# +# GDB debugging support, TCG status +# +# Copyright 2016 Linaro Ltd +# +# Authors: +# Alex Bennée +# +# This work is licensed under the terms of the GNU GPL, version 2. See +# the COPYING file in the top-level directory. +# +# Contributions after 2012-01-13 are licensed under the terms of the +# GNU GPL, version 2 or (at your option) any later version. + +# 'qemu tcg-lock-status' -- display the TCG lock status across threads + +import gdb + +class TCGLockStatusCommand(gdb.Command): + '''Display TCG Execution Status''' + def __init__(self): + gdb.Command.__init__(self, 'qemu tcg-lock-status', gdb.COMMAND_DATA, + gdb.COMPLETE_NONE) + + def invoke(self, arg, from_tty): + gdb.write("Thread, BQL (iothread_mutex), Replay, Blocked?\n") + for thread in gdb.inferiors()[0].threads(): + thread.switch() + + iothread = gdb.parse_and_eval("iothread_locked") + replay = gdb.parse_and_eval("replay_locked") + + frame = gdb.selected_frame() + if frame.name() == "__lll_lock_wait": + frame.older().select() + mutex = gdb.parse_and_eval("mutex") + owner = gdb.parse_and_eval("mutex->__data.__owner") + blocked = ("__lll_lock_wait waiting on %s from %d" % + (mutex, owner)) + else: + blocked = "not blocked" + + gdb.write("%d/%d, %s, %s, %s\n" % (thread.num, thread.ptid[1], + iothread, replay, blocked)) From patchwork Fri May 5 10:38:21 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98611 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp50113qge; Fri, 5 May 2017 03:40:25 -0700 (PDT) X-Received: by 10.237.53.124 with SMTP id b57mr39646927qte.240.1493980825327; Fri, 05 May 2017 03:40:25 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id x134si4355229qka.214.2017.05.05.03.40.24 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:40:25 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46175 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6aes-0003m9-Oh for patch@linaro.org; Fri, 05 May 2017 06:40:22 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52611) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acR-0001ml-Uw for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:53 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acQ-0006mC-VG for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:51 -0400 Received: from mail-wm0-x231.google.com ([2a00:1450:400c:c09::231]:38084) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acQ-0006ku-PC for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:50 -0400 Received: by mail-wm0-x231.google.com with SMTP id 142so2195740wma.1 for ; Fri, 05 May 2017 03:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=uEH+K4irOp7LRSuvkX7MT2Qak9ld7gQVKrCmdAfzJyM=; b=LGQtevDTY1BJXnaEsq4Zc+erpD/iT9tdaPMYTaYZiEGGcnQwGzOodZuMz5RdLHyz9L wecuYRtKhBK58V86CgyMb3TBaPzLG+OYOTvkXWWVla4PpeqcagV2qBpa7WeHYn4uxqCR 4BhcTRBofbE5lrVJBnRI8YMYJozwVpSo5oW84= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=uEH+K4irOp7LRSuvkX7MT2Qak9ld7gQVKrCmdAfzJyM=; b=Q2araSteUzjw0tG5Qc3I5IgH/wKQ/NhtHJJ7mS2H4w39KRyEWST1wTGsWWLhDSozr8 ovRjNOATkYDQ0kjsnc4CmAU8J8jVQ9BKMczGOynepPJUjBQEkYHWuP2Z68Y/MFOuXldf S1+UD3cZTb40mB5U0CW0dwTP2Gtqvt/DKNQAjYZUlOyTpc/b3ObuZ10Mmw5rchHtfLNE HeOLtKBccPg8/vSBRVVhUnweTIewZCOw1PmerZ75Clc7snc3msCmoXV2T57xgcbyvRgi nnfioeeh9nrhbLm2klHvZM6fJMs9BIoaJFY+D4xcL0KXVV/4tqCOS0AWPO6oV65cTYZr 7thA== X-Gm-Message-State: AN3rC/6iHrUlDaJI4l9jmoL2JUeRAjV1pL7GqjLANjU3LOqNNFw3+I4b dxFz2bqCn2Bh+Iia X-Received: by 10.28.156.206 with SMTP id f197mr4688736wme.40.1493980669655; Fri, 05 May 2017 03:37:49 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id t2sm1789279wme.16.2017.05.05.03.37.43 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:48 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 4CF773E023D; Fri, 5 May 2017 11:38:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:21 +0100 Message-Id: <20170505103822.20641-9-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::231 Subject: [Qemu-devel] [RFC PATCH v1 8/9] util/qemu-thread-*: add qemu_lock, locked and unlock trace events X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" Signed-off-by: Alex Bennée --- include/qemu/thread.h | 7 +++++-- util/qemu-thread-posix.c | 11 +++++++++-- util/trace-events | 5 +++++ 3 files changed, 19 insertions(+), 4 deletions(-) -- 2.11.0 diff --git a/include/qemu/thread.h b/include/qemu/thread.h index 9910f49b3a..ddea4990c0 100644 --- a/include/qemu/thread.h +++ b/include/qemu/thread.h @@ -22,9 +22,12 @@ typedef struct QemuThread QemuThread; void qemu_mutex_init(QemuMutex *mutex); void qemu_mutex_destroy(QemuMutex *mutex); -void qemu_mutex_lock(QemuMutex *mutex); int qemu_mutex_trylock(QemuMutex *mutex); -void qemu_mutex_unlock(QemuMutex *mutex); +void qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, const int line); +void qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, const int line); + +#define qemu_mutex_lock(mutex) qemu_mutex_lock_impl(mutex, __FILE__, __LINE__) +#define qemu_mutex_unlock(mutex) qemu_mutex_unlock_impl(mutex, __FILE__, __LINE__) /* Prototypes for other functions are in thread-posix.h/thread-win32.h. */ void qemu_rec_mutex_init(QemuRecMutex *mutex); diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c index 73e3a0edf5..9da1c9e756 100644 --- a/util/qemu-thread-posix.c +++ b/util/qemu-thread-posix.c @@ -14,6 +14,7 @@ #include "qemu/thread.h" #include "qemu/atomic.h" #include "qemu/notify.h" +#include "trace.h" static bool name_threads; @@ -53,13 +54,17 @@ void qemu_mutex_destroy(QemuMutex *mutex) error_exit(err, __func__); } -void qemu_mutex_lock(QemuMutex *mutex) +void qemu_mutex_lock_impl(QemuMutex *mutex, const char *file, const int line) { int err; + trace_qemu_mutex_lock(mutex, file, line); + err = pthread_mutex_lock(&mutex->lock); if (err) error_exit(err, __func__); + + trace_qemu_mutex_locked(mutex, file, line); } int qemu_mutex_trylock(QemuMutex *mutex) @@ -67,13 +72,15 @@ int qemu_mutex_trylock(QemuMutex *mutex) return pthread_mutex_trylock(&mutex->lock); } -void qemu_mutex_unlock(QemuMutex *mutex) +void qemu_mutex_unlock_impl(QemuMutex *mutex, const char *file, const int line) { int err; err = pthread_mutex_unlock(&mutex->lock); if (err) error_exit(err, __func__); + + trace_qemu_mutex_unlock(mutex, file, line); } void qemu_rec_mutex_init(QemuRecMutex *mutex) diff --git a/util/trace-events b/util/trace-events index b44ef4f895..972d7e1786 100644 --- a/util/trace-events +++ b/util/trace-events @@ -34,6 +34,11 @@ qemu_co_mutex_lock_return(void *mutex, void *self) "mutex %p self %p" qemu_co_mutex_unlock_entry(void *mutex, void *self) "mutex %p self %p" qemu_co_mutex_unlock_return(void *mutex, void *self) "mutex %p self %p" +# util/qemu-thread.c +qemu_mutex_lock(void *mutex, const char *file, const int line) "waiting on mutex %p (%s:%d)" +qemu_mutex_locked(void *mutex, const char *file, const int line) "taken mutex %p (%s:%d)" +qemu_mutex_unlock(void *mutex, const char *file, const int line) "released mutex %p (%s:%d)" + # util/oslib-win32.c # util/oslib-posix.c qemu_memalign(size_t alignment, size_t size, void *ptr) "alignment %zu size %zu ptr %p" From patchwork Fri May 5 10:38:22 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Alex_Benn=C3=A9e?= X-Patchwork-Id: 98617 Delivered-To: patch@linaro.org Received: by 10.140.96.100 with SMTP id j91csp52860qge; Fri, 5 May 2017 03:48:11 -0700 (PDT) X-Received: by 10.200.36.131 with SMTP id s3mr45670235qts.289.1493981291782; Fri, 05 May 2017 03:48:11 -0700 (PDT) Return-Path: Received: from lists.gnu.org (lists.gnu.org. [2001:4830:134:3::11]) by mx.google.com with ESMTPS id c186si4201174qkd.44.2017.05.05.03.48.11 for (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 05 May 2017 03:48:11 -0700 (PDT) Received-SPF: pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) client-ip=2001:4830:134:3::11; Authentication-Results: mx.google.com; dkim=fail header.i=@linaro.org; spf=pass (google.com: domain of qemu-devel-bounces+patch=linaro.org@nongnu.org designates 2001:4830:134:3::11 as permitted sender) smtp.mailfrom=qemu-devel-bounces+patch=linaro.org@nongnu.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([::1]:46214 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6amP-00021g-Cd for patch@linaro.org; Fri, 05 May 2017 06:48:09 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:52636) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1d6acT-0001of-Ml for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:56 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1d6acS-0006ol-KJ for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:53 -0400 Received: from mail-wm0-x233.google.com ([2a00:1450:400c:c09::233]:35415) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1d6acS-0006nD-BJ for qemu-devel@nongnu.org; Fri, 05 May 2017 06:37:52 -0400 Received: by mail-wm0-x233.google.com with SMTP id w64so20094989wma.0 for ; Fri, 05 May 2017 03:37:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=ktf4cSj6hYYBwLb7A+y78uFZPtRBL2oqzEnVaZhJNgM=; b=eBonx0OpItGlfYWfbdBhThcRqYcQtu9CuJzs1oQtB6nagcd0/JGBYSMNtqIEchwzxJ dDs+JaF9MHNbKVfse6XLVz7zhOe192ImZV7CPvmP9Pr70Z8+Nad573NDuWtwmliwDEan 2D7cJf8Iv92sM4jPaJHAhfV67cfL6Lfa2iLOA= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=ktf4cSj6hYYBwLb7A+y78uFZPtRBL2oqzEnVaZhJNgM=; b=J3wLhLydo41Ep1xn/+KPtpUMNFcMd8jwqTl67KSSRlps3pSqY/Jowr6vK4KIQY8KrA zedx6S7umQbRvd3ODw+IgvWFCI2JvCVRWkHl5GZde6jVDOvhGerGVeUaUFVw9IheINcQ 99g9QXUP4HQqoyJuE2cDS1wnJLTt4X53ZzINDxOZ6+mytT0trsV5937B15rbg6nOH0bX esdsMQOSkqbSkqo4Jg4gePOpeZn7eH+wAM/wXhJP4GQBhk4dklJlLmhbPoucPAiUfFXl tmnMsvCA6rFFCN0MnnCmRH3sv62gl3OJNQDKHv6ejRsKrlu3kYwr8vvMLx3IzfNwtK5T rKww== X-Gm-Message-State: AODbwcAok3gnDb/jXvb76d25aOZlvEkzujEOyJ8+gr7KaMGK2NMxpgEJ 2/633xJ5/0eLpJXV X-Received: by 10.28.161.68 with SMTP id k65mr2588441wme.134.1493980671209; Fri, 05 May 2017 03:37:51 -0700 (PDT) Received: from zen.linaro.local ([81.128.185.34]) by smtp.gmail.com with ESMTPSA id 72sm2062449wmx.23.2017.05.05.03.37.44 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 May 2017 03:37:48 -0700 (PDT) Received: from zen.linaroharston (localhost [127.0.0.1]) by zen.linaro.local (Postfix) with ESMTP id 5D7353E024F; Fri, 5 May 2017 11:38:23 +0100 (BST) From: =?utf-8?q?Alex_Benn=C3=A9e?= To: pbonzini@redhat.com, boost.lists@gmail.com, pavel.dovgaluk@ispras.ru Date: Fri, 5 May 2017 11:38:22 +0100 Message-Id: <20170505103822.20641-10-alex.bennee@linaro.org> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170505103822.20641-1-alex.bennee@linaro.org> References: <20170505103822.20641-1-alex.bennee@linaro.org> MIME-Version: 1.0 X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a00:1450:400c:c09::233 Subject: [Qemu-devel] [RFC PATCH v1 9/9] scripts/analyse-locks-simpletrace.py: script to analyse lock times X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: cota@braap.org, =?utf-8?q?Alex_Benn=C3=A9e?= , qemu-devel@nongnu.org Errors-To: qemu-devel-bounces+patch=linaro.org@nongnu.org Sender: "Qemu-devel" This script allows analysis of mutex acquisition and hold times based on a trace file. Given a trace control file of: qemu_mutex_lock qemu_mutex_locked qemu_mutex_unlock And running with: $QEMU $QEMU_ARGS -trace events=./lock-trace You can analyse the results with: ./scripts/analyse-locks-simpletrace.py trace-events-all ./trace-21812 Signed-off-by: Alex Bennée --- scripts/analyse-locks-simpletrace.py | 99 ++++++++++++++++++++++++++++++++++++ 1 file changed, 99 insertions(+) create mode 100755 scripts/analyse-locks-simpletrace.py -- 2.11.0 diff --git a/scripts/analyse-locks-simpletrace.py b/scripts/analyse-locks-simpletrace.py new file mode 100755 index 0000000000..b72c951cd9 --- /dev/null +++ b/scripts/analyse-locks-simpletrace.py @@ -0,0 +1,99 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- +# +# Analyse lock events and +# +# Author: Alex Bennée +# + +import os +import simpletrace +import argparse +import numpy as np + +class MutexAnalyser(simpletrace.Analyzer): + "A simpletrace Analyser for checking locks." + + def __init__(self): + self.locks = 0 + self.locked = 0 + self.unlocks = 0 + self.mutex_records = {} + + def _get_mutex(self, mutex): + if not mutex in self.mutex_records: + self.mutex_records[mutex] = {"locks": 0, + "lock_time": 0, + "acquire_times": [], + "locked": 0, + "locked_time": 0, + "held_times": [], + "unlocked": 0} + + return self.mutex_records[mutex] + + def qemu_mutex_lock(self, timestamp, mutex, filename, line): + self.locks += 1 + rec = self._get_mutex(mutex) + rec["locks"] += 1 + rec["lock_time"] = timestamp[0] + rec["lock_loc"] = (filename, line) + + def qemu_mutex_locked(self, timestamp, mutex, filename, line): + self.locked += 1 + rec = self._get_mutex(mutex) + rec["locked"] += 1 + rec["locked_time"] = timestamp[0] + acquire_time = rec["locked_time"] - rec["lock_time"] + rec["locked_loc"] = (filename, line) + rec["acquire_times"].append(acquire_time) + + def qemu_mutex_unlock(self, timestamp, mutex, filename, line): + self.unlocks += 1 + rec = self._get_mutex(mutex) + rec["unlocked"] += 1 + held_time = timestamp[0] - rec["locked_time"] + rec["held_times"].append(held_time) + rec["unlock_loc"] = (filename, line) + + +def get_args(): + "Grab options" + parser = argparse.ArgumentParser() + parser.add_argument("--output", "-o", type=str, help="Render plot to file") + parser.add_argument("events", type=str, help='trace file read from') + parser.add_argument("tracefile", type=str, help='trace file read from') + return parser.parse_args() + +if __name__ == '__main__': + args = get_args() + + # Gather data from the trace + analyser = MutexAnalyser() + simpletrace.process(args.events, args.tracefile, analyser) + + print ("Total locks: %d, locked: %d, unlocked: %d" % + (analyser.locks, analyser.locked, analyser.unlocks)) + + # Now dump the individual lock stats + for key, val in sorted(analyser.mutex_records.iteritems(), + key=lambda (k,v): v["locks"]): + print ("Lock: %#x locks: %d, locked: %d, unlocked: %d" % + (key, val["locks"], val["locked"], val["unlocked"])) + + acquire_times = np.array(val["acquire_times"]) + if len(acquire_times) > 0: + print (" Acquire Time: min:%d median:%d avg:%.2f max:%d" % + (acquire_times.min(), np.median(acquire_times), + acquire_times.mean(), acquire_times.max())) + + held_times = np.array(val["held_times"]) + if len(held_times) > 0: + print (" Held Time: min:%d median:%d avg:%.2f max:%d" % + (held_times.min(), np.median(held_times), + held_times.mean(), held_times.max())) + + # Check if any locks still held + if val["locks"] > val["locked"]: + print (" LOCK HELD (%s:%s)" % (val["locked_loc"])) + print (" BLOCKED (%s:%s)" % (val["lock_loc"]))