From patchwork Mon Mar 20 14:43:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 75E00C6FD1D for ; Mon, 20 Mar 2023 14:44:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231846AbjCTOoh (ORCPT ); Mon, 20 Mar 2023 10:44:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231839AbjCTOog (ORCPT ); Mon, 20 Mar 2023 10:44:36 -0400 Received: from mail-pj1-x102c.google.com (mail-pj1-x102c.google.com [IPv6:2607:f8b0:4864:20::102c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ED6E6AD22; Mon, 20 Mar 2023 07:44:34 -0700 (PDT) Received: by mail-pj1-x102c.google.com with SMTP id e15-20020a17090ac20f00b0023d1b009f52so16743708pjt.2; Mon, 20 Mar 2023 07:44:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qOKgIqKype2FtcfBEs8DzVnGdB8jDwzY52kLns1gHeQ=; b=VnF+6G2rX3V3uiMnbJNUCShKHEg3qSXYFZW2BadyVGNW0EATEl8lpaMebXjsuGk82j 6uNC/eXNRKXC6LHgJEGqwFVgVpKr8Ft2sQSWlWoJNn2mcPP6CjiXAakf5VgjfGk741GP sIHFQgcHBE6gPMKsa9eAIRJpEMN/RO4X6UIOevibBEhmVCdoZdol18YJKBu4jexGjC/y xW2lLDNDlqC2S0GLUL1feKz6u5E6vwZDYvzQtXJzzQd8X1JJtWfI4To24/vDTwEUXx1m F3Vs7SviqMHjK5T10Q782LncWJWcxm7mshYIhZNh48C6PEae62pRbnFZfPIvZvxobGNO Zrbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qOKgIqKype2FtcfBEs8DzVnGdB8jDwzY52kLns1gHeQ=; b=pi5qHJ7SjUuUHV7uDaBhNVfXi9oBe4l/WhLwJKLH9IJJbrzhjhxnvl4Ldbc5ZIUIha ygAck0Yw9TvKkYRPonfFYrLX482BhupzwffIG+ouG33UhpzbA3q9G0nrsaMXKaOH4LDm GygrmLrtD7bFkmKlDUqWRJAyICU7gN03YbvG5tjRLhCXWrIvTdHo/hzhtXZDELLBokM5 Ziao/0n5+gfVz41UEXx3yM72fQmmcGw6k9zGoabNriXVRBrmjAJM8PWx45Aov2/GWIl/ J7+OJwA9c28Z4WoYtsdZUiwdKYVV+GnlMiS5NE8G8+R0IILiGJSvAplga1Wa+z94W1Gx 4euQ== X-Gm-Message-State: AO0yUKUYmOgai/ksN/n9YzU5wyuNM7Vr0HwV81XtQmWBLUi/H2xL5Nat pLQktBieAi1moKoka0C8EsE= X-Google-Smtp-Source: AK7set87GLedaxfP6MTlGWAVIi2V/FMub1LUUnVtxXaItSj0lTXwOwSmnXTsGsDYl2P+bImVpBwAtQ== X-Received: by 2002:a05:6a20:bb12:b0:da:24f5:ff25 with SMTP id fc18-20020a056a20bb1200b000da24f5ff25mr1307876pzb.48.1679323474431; Mon, 20 Mar 2023 07:44:34 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id c26-20020a631c5a000000b00507249cde91sm6186039pgm.91.2023.03.20.07.44.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:34 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 02/23] drm/msm: Move submit bo flags update from obj lock Date: Mon, 20 Mar 2023 07:43:24 -0700 Message-Id: <20230320144356.803762-3-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark The flags are only accessed (1) when submit is constructed, before enqueuing to gpu sched (ie. when still visible to only the task calling the submit ioctl), (2) here, where we own a reference to the submit and are serialized on the gpu sched thread, and (3) after the submit is retired and last reference is dropped, which is serialized on the submit's reference count. Hence locking is unneeded here. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c index a62b45e5a8c3..a80447c8764e 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.c +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c @@ -26,8 +26,8 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) msm_gem_lock(obj); msm_gem_unpin_vma_fenced(submit->bos[i].vma, fctx); msm_gem_unpin_locked(obj); - submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED); msm_gem_unlock(obj); + submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED); } /* TODO move submit path over to using a per-ring lock.. */ From patchwork Mon Mar 20 14:43:26 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665264 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77583C7618A for ; Mon, 20 Mar 2023 14:44:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231851AbjCTOon (ORCPT ); Mon, 20 Mar 2023 10:44:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50478 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231857AbjCTOok (ORCPT ); Mon, 20 Mar 2023 10:44:40 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B0B2DCA1A; Mon, 20 Mar 2023 07:44:38 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id u5so12652623plq.7; Mon, 20 Mar 2023 07:44:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xALpMXlLTSMvOfS7mQNvHfzG9+8QM3cmx9b7Fz6Q2XU=; b=cxu8fcc47sheJUocs3NFK/ej2kra9Ypkl7pCx1H1P5vOekErq1/GNU3/ZBsh7fug/U LK5SzMY1CdLJhSlItGMRB+oN2kKcpG9ZezdZTN4n8Us/IaCGxAzVEcp9zoIzcbsHXM8l Qze0Vuq2K4bx7XB8PZONqv84P/6H9hG3kl7ovKb5I6Z0JnHdaUA2grKNAlEuYnpDmN+B Y9CLZCVJ6ye3i6QM7sDrald94pji9aWZ+2cy0EOHNDHwDIaL6b6wQ2WpqdFMnc1TBHeR qnuRIQoDVOVCXCBRpQ2sohXgh4ltYqi2IkT24bat3p9pR9f1dZscLLR8ov0SK99wX8qm HqPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xALpMXlLTSMvOfS7mQNvHfzG9+8QM3cmx9b7Fz6Q2XU=; b=sNva6z4hfKo7CcVhs4Ogj8oEn1utR9+RmVVISSiwsyw6v8wHdyiFLjzmje9aCvHVR9 uEV3FvdYvpKRZuRr/mp/1Z5cNl8HjsFq2HGIePxF7F9uzPQ3e9G/ZYw4dEuWl3dEMQOD 5MpUHWolH/sXuD93CU3A9OpC7CnIP7DEIV2JCKuc1+9QMcjNY0qDECxDgB0pPyLr91/K ij1+jFWMKoBLpxxcdNOars3/oEq7C3X/PItbVNpo5LC8eVcaudvtDXn7irWgNoJZEnSz U955qNJb/CoLE+zh9GpeIhx/FTcY7MqmK0z4vm6/GNlUwn3Ddoca0ksgRpFCkxDtX4N4 Lq3Q== X-Gm-Message-State: AO0yUKVVxLXhK86X99fHkVGr1nayN+mhDOhSUbIfKmEAmvpW+4cDfrQ7 EEJrWS0jgyqLESi+D+1VEa0= X-Google-Smtp-Source: AK7set/Og7pcuw4IzsENyS7Re6THWUo/30QuP9+MbL2JloNr5T9/L0e5eyWIxk080vd7BjvX4FSQ2A== X-Received: by 2002:a05:6a20:7b05:b0:da:39a5:6e66 with SMTP id s5-20020a056a207b0500b000da39a56e66mr920491pzh.18.1679323478303; Mon, 20 Mar 2023 07:44:38 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id r13-20020a63e50d000000b004fb26a80875sm6389111pgh.22.2023.03.20.07.44.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:37 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 04/23] drm/msm: Decouple vma tracking from obj lock Date: Mon, 20 Mar 2023 07:43:26 -0700 Message-Id: <20230320144356.803762-5-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark We need to use the inuse count to track that a BO is pinned until we have the hw_fence. But we want to remove the obj lock from the job_run() path as this could deadlock against reclaim/shrinker (because it is blocking the hw_fence from eventually being signaled). So split that tracking out into a per-vma lock with narrower scope. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gem_vma.c | 44 ++++++++++++++++++++++++---- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- 3 files changed, 40 insertions(+), 7 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index d3219c523034..1929f09c5b0d 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -59,6 +59,7 @@ struct msm_fence_context; struct msm_gem_vma { struct drm_mm_node node; + spinlock_t lock; uint64_t iova; struct msm_gem_address_space *aspace; struct list_head list; /* node in msm_gem_object::vmas */ diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c b/drivers/gpu/drm/msm/msm_gem_vma.c index 2827679dc39a..98287ed99960 100644 --- a/drivers/gpu/drm/msm/msm_gem_vma.c +++ b/drivers/gpu/drm/msm/msm_gem_vma.c @@ -40,19 +40,28 @@ msm_gem_address_space_get(struct msm_gem_address_space *aspace) bool msm_gem_vma_inuse(struct msm_gem_vma *vma) { + bool ret = true; + + spin_lock(&vma->lock); + if (vma->inuse > 0) - return true; + goto out; while (vma->fence_mask) { unsigned idx = ffs(vma->fence_mask) - 1; if (!msm_fence_completed(vma->fctx[idx], vma->fence[idx])) - return true; + goto out; vma->fence_mask &= ~BIT(idx); } - return false; + ret = false; + +out: + spin_unlock(&vma->lock); + + return ret; } /* Actually unmap memory for the vma */ @@ -73,8 +82,7 @@ void msm_gem_vma_purge(struct msm_gem_vma *vma) vma->mapped = false; } -/* Remove reference counts for the mapping */ -void msm_gem_vma_unpin(struct msm_gem_vma *vma) +static void vma_unpin_locked(struct msm_gem_vma *vma) { if (GEM_WARN_ON(!vma->inuse)) return; @@ -82,13 +90,23 @@ void msm_gem_vma_unpin(struct msm_gem_vma *vma) vma->inuse--; } +/* Remove reference counts for the mapping */ +void msm_gem_vma_unpin(struct msm_gem_vma *vma) +{ + spin_lock(&vma->lock); + vma_unpin_locked(vma); + spin_unlock(&vma->lock); +} + /* Replace pin reference with fence: */ void msm_gem_vma_unpin_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx) { + spin_lock(&vma->lock); vma->fctx[fctx->index] = fctx; vma->fence[fctx->index] = fctx->last_fence; vma->fence_mask |= BIT(fctx->index); - msm_gem_vma_unpin(vma); + vma_unpin_locked(vma); + spin_unlock(&vma->lock); } /* Map and pin vma: */ @@ -103,7 +121,9 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot, return -EINVAL; /* Increase the usage counter */ + spin_lock(&vma->lock); vma->inuse++; + spin_unlock(&vma->lock); if (vma->mapped) return 0; @@ -113,11 +133,22 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot, if (!aspace) return 0; + /* + * NOTE: iommu/io-pgtable can allocate pages, so we cannot hold + * a lock across map/unmap which is also used in the job_run() + * path, as this can cause deadlock in job_run() vs shrinker/ + * reclaim. + * + * Revisit this if we can come up with a scheme to pre-alloc pages + * for the pgtable in map/unmap ops. + */ ret = aspace->mmu->funcs->map(aspace->mmu, vma->iova, sgt, size, prot); if (ret) { vma->mapped = false; + spin_lock(&vma->lock); vma->inuse--; + spin_unlock(&vma->lock); } return ret; @@ -148,6 +179,7 @@ struct msm_gem_vma *msm_gem_vma_new(struct msm_gem_address_space *aspace) if (!vma) return NULL; + spin_lock_init(&vma->lock); vma->aspace = aspace; return vma; diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c index 44a22b283730..31b4fbf96c36 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.c +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c @@ -23,8 +23,8 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) for (i = 0; i < submit->nr_bos; i++) { struct drm_gem_object *obj = &submit->bos[i].obj->base; - msm_gem_lock(obj); msm_gem_vma_unpin_fenced(submit->bos[i].vma, fctx); + msm_gem_lock(obj); msm_gem_unpin_locked(obj); msm_gem_unlock(obj); submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED); From patchwork Mon Mar 20 14:43:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665262 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE9E9C761AF for ; Mon, 20 Mar 2023 14:45:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231418AbjCTOpY (ORCPT ); Mon, 20 Mar 2023 10:45:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:50940 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231987AbjCTOpG (ORCPT ); Mon, 20 Mar 2023 10:45:06 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D06FECA1E; Mon, 20 Mar 2023 07:44:48 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id gp15-20020a17090adf0f00b0023d1bbd9f9eso16772440pjb.0; Mon, 20 Mar 2023 07:44:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=weYL3fqvUQCe/MGsBiphMuUm7IiUfAgwOkX0hsG5azI=; b=XoB4+mM4oASSRnco8nkZwARRG9zcXJcuiyLTtkB1/Em3LEQ2A1GcFBX4c9QpNQQF1j NODZTXwBb1ZsRgqzzncmT68xS+JsVOMXu4/yL4tqZacnngCN5gnrIyIgLpg67GCQaWu2 3RGsxmBtiA8aCU5oOdtb6AigAsxMRRyYT/P3Xh3WrVDvz6jvLttylpc7USgWWQ44GVqe rYzg55l+AaLbimu//o2q7VRZZJ2UA2rlR+D7WRutg8GbWIEPrBcIPRitI3OqhFoohqaA 20Pha7DZD60SGbvPicWbg+l+r5k+WAzhnRYWrhfrBafJ9eHbu4hwWd94ZwQX/0whZg/7 88jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323487; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=weYL3fqvUQCe/MGsBiphMuUm7IiUfAgwOkX0hsG5azI=; b=4SU08Y7DjTy3Hb9zVvuTKqLxJNA1kKe3NBIh8EW3M7yWFGFxSyvv9LSJqlSZnsw2Ma u9ecKaMa4RsrSkF0xwRPfaXIhzB8o738x+IoxG418NJ0iV6H+G000/Qy8XLF564F8F5k rZcfEU3puAxbiKe3/rYjqBBsKNPNl1w/oL1rno52+5oHCCOHHGdGVcvvNQ8T7is65qsr r5JWdydfTatPMCzzwLQoCbNws34vJ9DR+coGfaVGpjvpd3mbchvQLoXS43ECAjubx1N8 /9edCEaa1bAZqpI6jgbfk6vllktxLKFzEYXjmsxd9+VaK6/ZJLUS86+hP6MrTOEwntr+ S8Lw== X-Gm-Message-State: AO0yUKUfUmMwyAowKOjglmlPB9/pXIOyYJTY0uwhF9TVv9chD3SttSJW qw/ivGhFtuG2DVJOXuh7A/s= X-Google-Smtp-Source: AK7set/q6mxFL2SNpoyCJqlGoIsp6zXx5THF6Tar0/8ALnqXrSzyWxWyrmITe8Cih+IgIxzET11mjA== X-Received: by 2002:a17:902:d2ca:b0:19d:1c6e:d31e with SMTP id n10-20020a170902d2ca00b0019d1c6ed31emr21226848plc.60.1679323487099; Mon, 20 Mar 2023 07:44:47 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id g7-20020a170902934700b0019d397b0f18sm6777259plp.214.2023.03.20.07.44.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:46 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Sumit Semwal , =?utf-8?q?Christian_K=C3=B6nig?= , linux-kernel@vger.kernel.org (open list), linux-media@vger.kernel.org (open list:DMA BUFFER SHARING FRAMEWORK), linaro-mm-sig@lists.linaro.org (moderated list:DMA BUFFER SHARING FRAMEWORK) Subject: [PATCH v2 09/23] drm/msm/gem: Avoid obj lock in job_run() Date: Mon, 20 Mar 2023 07:43:31 -0700 Message-Id: <20230320144356.803762-10-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Now that everything that controls which LRU an obj lives in *except* the backing pages is protected by the LRU lock, add a special path to unpin in the job_run() path, we we are assured that we already have backing pages and will not be racing against eviction (because the GEM object's dma_resv contains the fence that will be signaled when the submit/job completes). Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 44 +++++++++++++++++++++++----- drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_ringbuffer.c | 4 +-- 3 files changed, 39 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index d0ac3e704b66..9628e8d8dd02 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -61,18 +61,14 @@ static void sync_for_cpu(struct msm_gem_object *msm_obj) dma_unmap_sgtable(dev, msm_obj->sgt, DMA_BIDIRECTIONAL, 0); } -static void update_lru_locked(struct drm_gem_object *obj) +static void update_lru_active(struct drm_gem_object *obj) { struct msm_drm_private *priv = obj->dev->dev_private; struct msm_gem_object *msm_obj = to_msm_bo(obj); - msm_gem_assert_locked(&msm_obj->base); - - if (!msm_obj->pages) { - GEM_WARN_ON(msm_obj->pin_count); + GEM_WARN_ON(!msm_obj->pages); - drm_gem_lru_move_tail_locked(&priv->lru.unbacked, obj); - } else if (msm_obj->pin_count) { + if (msm_obj->pin_count) { drm_gem_lru_move_tail_locked(&priv->lru.pinned, obj); } else if (msm_obj->madv == MSM_MADV_WILLNEED) { drm_gem_lru_move_tail_locked(&priv->lru.willneed, obj); @@ -83,6 +79,22 @@ static void update_lru_locked(struct drm_gem_object *obj) } } +static void update_lru_locked(struct drm_gem_object *obj) +{ + struct msm_drm_private *priv = obj->dev->dev_private; + struct msm_gem_object *msm_obj = to_msm_bo(obj); + + msm_gem_assert_locked(&msm_obj->base); + + if (!msm_obj->pages) { + GEM_WARN_ON(msm_obj->pin_count); + + drm_gem_lru_move_tail_locked(&priv->lru.unbacked, obj); + } else { + update_lru_active(obj); + } +} + static void update_lru(struct drm_gem_object *obj) { struct msm_drm_private *priv = obj->dev->dev_private; @@ -489,6 +501,24 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) mutex_unlock(&priv->lru.lock); } +/* Special unpin path for use in fence-signaling path, avoiding the need + * to hold the obj lock by only depending on things that a protected by + * the LRU lock. In particular we know that that we already have backing + * and and that the object's dma_resv has the fence for the current + * submit/job which will prevent us racing against page eviction. + */ +void msm_gem_unpin_active(struct drm_gem_object *obj) +{ + struct msm_drm_private *priv = obj->dev->dev_private; + struct msm_gem_object *msm_obj = to_msm_bo(obj); + + mutex_lock(&priv->lru.lock); + msm_obj->pin_count--; + GEM_WARN_ON(msm_obj->pin_count < 0); + update_lru_active(obj); + mutex_unlock(&priv->lru.lock); +} + struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj, struct msm_gem_address_space *aspace) { diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 0057e8e8fa13..2bd6846c83a9 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -128,6 +128,7 @@ struct msm_gem_object { uint64_t msm_gem_mmap_offset(struct drm_gem_object *obj); int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma); void msm_gem_unpin_locked(struct drm_gem_object *obj); +void msm_gem_unpin_active(struct drm_gem_object *obj); struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj, struct msm_gem_address_space *aspace); int msm_gem_get_iova(struct drm_gem_object *obj, diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c index 31b4fbf96c36..b60199184409 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.c +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c @@ -24,9 +24,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job) struct drm_gem_object *obj = &submit->bos[i].obj->base; msm_gem_vma_unpin_fenced(submit->bos[i].vma, fctx); - msm_gem_lock(obj); - msm_gem_unpin_locked(obj); - msm_gem_unlock(obj); + msm_gem_unpin_active(obj); submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED); } From patchwork Mon Mar 20 14:43:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AD35C76195 for ; Mon, 20 Mar 2023 14:45:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231883AbjCTOp1 (ORCPT ); Mon, 20 Mar 2023 10:45:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51522 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231984AbjCTOpG (ORCPT ); Mon, 20 Mar 2023 10:45:06 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50481AD2A; Mon, 20 Mar 2023 07:44:50 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id j13so12352669pjd.1; Mon, 20 Mar 2023 07:44:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323489; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=y74KF7H2nPAn1GkAkMf3ALBkHME3mvlc137TFEwpI5M=; b=KDNXAyKxSPxmqCM1goz2ei4OMneXayGq4rDfE6Qgq1VIyE6guRUWAO1mZzPWwgIP26 JYt+JAaoGORRqU5CpZfLxhpnaTqa15rJPVrROZd2PP5/Wrv+L91VA5PRNhE4DIVA3ZzB OFZzSBuKsPmYl+P7Ixx+dBJft1d8cUSoVWPBMVB/TZ2/y4+0WwyogGQnGsgpycK2T+HC 4AeAWwpByzuLtol49nW1nmtPSxhPBjJAzk4GYtsHsq6Xqba8j2nOOgS7lDgcUfg6PO4d 9Babeqe7rFqMFE/ieZ97Gbuj0CUvKp6/l1e4DET+Plb6nHmPrw2kqMuYyB3AqKl+6N11 miDQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323489; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=y74KF7H2nPAn1GkAkMf3ALBkHME3mvlc137TFEwpI5M=; b=sq3IHapkhrPVovMiDukpUvwyomSC4duPHeo1n081YL97mpu5si+sNgRcOsTrIvOIST PZrdqBCbqiCmZT0NoUMqcB98RBKGNpfuLZSrXE/gBXR8wmqHMAhZVaXju+F06UC4ffZR FhQ+0nevaXMSLWxlEqnOjCuf4PwHNFLxbDPN7YFVidxHo3FXXKZza6vJ2CSUyzIYFPN7 sIarBEzLAzgILIqQI2vyHTuts8omwayEPQtr+3kFWLSqEX25kIPbNfKPbviCAvog4B85 CRV0SklwNateH/CFda7Zu+4NUSnyynA9toohccnDq0655HqDOnFPPQmQ25IGczhpuHX5 +LTw== X-Gm-Message-State: AO0yUKVZ8VwqgciSVukiAqf9K4f+mVoPYiIicJ7D05IbWQazxWrdkV9Q ySBaTfiAh0UK1QN7ttr93A8= X-Google-Smtp-Source: AK7set9ZUleUp5mEZ/nimYP6FtanqkhrK9z8SGzJ4pCwCjW9CpDQ3yDpgBO1I4IgNMmovbqGg0F/ng== X-Received: by 2002:a17:902:c40c:b0:19a:a815:2868 with SMTP id k12-20020a170902c40c00b0019aa8152868mr21846958plk.44.1679323488934; Mon, 20 Mar 2023 07:44:48 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id j3-20020a170902c3c300b001a072be70desm6828123plj.41.2023.03.20.07.44.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:48 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 10/23] drm/msm: Switch idr_lock to spinlock Date: Mon, 20 Mar 2023 07:43:32 -0700 Message-Id: <20230320144356.803762-11-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Needed to idr_preload() which returns with preemption disabled. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_drv.c | 6 ++---- drivers/gpu/drm/msm/msm_gem_submit.c | 10 +++++----- drivers/gpu/drm/msm/msm_gpu.h | 2 +- drivers/gpu/drm/msm/msm_submitqueue.c | 2 +- 4 files changed, 9 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index aca48c868c14..ce1a77b607d1 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -918,13 +918,11 @@ static int wait_fence(struct msm_gpu_submitqueue *queue, uint32_t fence_id, * retired, so if the fence is not found it means there is nothing * to wait for */ - ret = mutex_lock_interruptible(&queue->idr_lock); - if (ret) - return ret; + spin_lock(&queue->idr_lock); fence = idr_find(&queue->fence_idr, fence_id); if (fence) fence = dma_fence_get_rcu(fence); - mutex_unlock(&queue->idr_lock); + spin_unlock(&queue->idr_lock); if (!fence) return 0; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 1d8e7c2a8024..b9d81e5acb42 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -79,9 +79,9 @@ void __msm_gem_submit_destroy(struct kref *kref) unsigned i; if (submit->fence_id) { - mutex_lock(&submit->queue->idr_lock); + spin_lock(&submit->queue->idr_lock); idr_remove(&submit->queue->fence_idr, submit->fence_id); - mutex_unlock(&submit->queue->idr_lock); + spin_unlock(&submit->queue->idr_lock); } dma_fence_put(submit->user_fence); @@ -882,7 +882,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->nr_cmds = i; - mutex_lock(&queue->idr_lock); + spin_lock(&queue->idr_lock); /* * If using userspace provided seqno fence, validate that the id @@ -892,7 +892,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, */ if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) && idr_find(&queue->fence_idr, args->fence)) { - mutex_unlock(&queue->idr_lock); + spin_unlock(&queue->idr_lock); ret = -EINVAL; goto out; } @@ -926,7 +926,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, INT_MAX, GFP_KERNEL); } - mutex_unlock(&queue->idr_lock); + spin_unlock(&queue->idr_lock); if (submit->fence_id < 0) { ret = submit->fence_id; diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index fc1c0d8611a8..5929ecaa1fcd 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -499,7 +499,7 @@ struct msm_gpu_submitqueue { struct msm_file_private *ctx; struct list_head node; struct idr fence_idr; - struct mutex idr_lock; + struct spinlock idr_lock; struct mutex lock; struct kref ref; struct drm_sched_entity *entity; diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c b/drivers/gpu/drm/msm/msm_submitqueue.c index c6929e205b51..0e803125a325 100644 --- a/drivers/gpu/drm/msm/msm_submitqueue.c +++ b/drivers/gpu/drm/msm/msm_submitqueue.c @@ -200,7 +200,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx, *id = queue->id; idr_init(&queue->fence_idr); - mutex_init(&queue->idr_lock); + spin_lock_init(&queue->idr_lock); mutex_init(&queue->lock); list_add_tail(&queue->node, &ctx->submitqueues); From patchwork Mon Mar 20 14:43:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AA5A4C7619A for ; Mon, 20 Mar 2023 14:45:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231861AbjCTOpU (ORCPT ); Mon, 20 Mar 2023 10:45:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51546 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231995AbjCTOpH (ORCPT ); Mon, 20 Mar 2023 10:45:07 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E734D10405; Mon, 20 Mar 2023 07:44:52 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id z18so6709253pgj.13; Mon, 20 Mar 2023 07:44:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323490; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=O4Xvkkg7F6CjyOF2RZo8Gtbu7JFTSy/bgLszOXTif/c=; b=pdh6pDbj0hMlQMJiqcjb8EcHHB++Bz4iZs1fWn8cufOZ+Nlx3Cw+6HTaCNHQc1riJo sNuYVoJZKGIWIeMkxx9NXtGIedo/FV+PKDgDHWaf9G0Teyo5+mk7ELb+tQFBYD1G0S21 0PtNuq9gBlinRoOXPX9RrFMMstWiCfJbX6w0UiLfyeU6VyzeB4va8oEX0kp4G/PqgrZ4 J8x7zIf81il9FWDLwkwoPzsnDUwEcI479IVGKAZjOeVZVZk4yNQNiDf7cUZrYqgllCFZ rRGRxpH+E0sP+A0ffVoJH4/hiO7565ux39p3UDwlawoDk1SaGvLvjUCn4t/eGacpTtyc c0Uw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323490; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=O4Xvkkg7F6CjyOF2RZo8Gtbu7JFTSy/bgLszOXTif/c=; b=7+Qhj1vSKnOh9qzyJAuKL3aSa4SXi12ShCNe1fQKfCAOdCj0TzckLX37agv7svEKna Fj8SANYxyCkcCtS5mpO52l+f1hdlSBVsT/PEFai8K0yZKsk2lFq/1o6lSzDlIA+ZNhY/ iAYpW7A+y8pT98r19jyNBXDxoZD9NWvkzoski3gtJoxz+BX8sH/Zrx4gHEkSoX9PmHUx IwD96l2dh8Xrdl73SB0dDLfIMQ6ppJov1gTiP4sKkSAj5mkXeS+Y5JAaSD09r6zWfbSh Uv5bqBEcIsGsOgYZSGjYB+x2igwFkBizf693qUMBLPgoElggdwIx8qzpiNQruoxu9SEV onQA== X-Gm-Message-State: AO0yUKWSEnuaTUa6DdHsztDNp4P0r0CqEUKI2HYgbuvt+irUnVAbQBUE iVuIoxZb4357KQvUOjxKUAI= X-Google-Smtp-Source: AK7set8KWfMWUfN/rUSfAdvgv538dmuC8YG9xJe+nc38O3y7lCz5M7md+3TF5MvUHCrG2tMwgVLAcQ== X-Received: by 2002:aa7:9f91:0:b0:627:fc31:1de with SMTP id z17-20020aa79f91000000b00627fc3101demr3121841pfr.7.1679323490696; Mon, 20 Mar 2023 07:44:50 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id v22-20020a62a516000000b0058bc7453285sm6389779pfm.217.2023.03.20.07.44.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:50 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 11/23] drm/msm: Use idr_preload() Date: Mon, 20 Mar 2023 07:43:33 -0700 Message-Id: <20230320144356.803762-12-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Avoid allocation under idr_lock, to prevent deadlock against the job_free() path (which runs on same thread as job_run(), which makes it also part of the fence-signaling path. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index b9d81e5acb42..0ab62cb4ed69 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -882,6 +882,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->nr_cmds = i; + idr_preload(GFP_KERNEL); + spin_lock(&queue->idr_lock); /* @@ -893,6 +895,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) && idr_find(&queue->fence_idr, args->fence)) { spin_unlock(&queue->idr_lock); + idr_preload_end(); ret = -EINVAL; goto out; } @@ -910,7 +913,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->fence_id = args->fence; ret = idr_alloc_u32(&queue->fence_idr, submit->user_fence, &submit->fence_id, submit->fence_id, - GFP_KERNEL); + GFP_NOWAIT); /* * We've already validated that the fence_id slot is valid, * so if idr_alloc_u32 failed, it is a kernel bug @@ -923,10 +926,11 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, */ submit->fence_id = idr_alloc_cyclic(&queue->fence_idr, submit->user_fence, 1, - INT_MAX, GFP_KERNEL); + INT_MAX, GFP_NOWAIT); } spin_unlock(&queue->idr_lock); + idr_preload_end(); if (submit->fence_id < 0) { ret = submit->fence_id; From patchwork Mon Mar 20 14:43:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665260 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 35213C7618D for ; Mon, 20 Mar 2023 14:45:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231398AbjCTOp3 (ORCPT ); Mon, 20 Mar 2023 10:45:29 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232029AbjCTOpQ (ORCPT ); Mon, 20 Mar 2023 10:45:16 -0400 Received: from mail-pl1-x636.google.com (mail-pl1-x636.google.com [IPv6:2607:f8b0:4864:20::636]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7180710273; Mon, 20 Mar 2023 07:44:55 -0700 (PDT) Received: by mail-pl1-x636.google.com with SMTP id kq3so265939plb.13; Mon, 20 Mar 2023 07:44:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323493; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ldFupt5wrZbRxkmdbQJ7PqzJnkF6w4MSZ/CZLw9J9jU=; b=nLqncyImNrlz2RYRgSTcGk81CIP+FxeIhwn8gsHuejhfPlvxGvZoYbDo0a811aEXK9 V1LlsnULXd0sVZPzFucY4idd1TfEuzWSGRr8oLl/nGNtw0w61RogcCNsUN5LYoaVQgy8 A9Zjp7rXg0KtTQarA5j7PNz52fbJgT60Emcaw3O4Td7B/lgNjsxPkom0BwnCRmFdnQOz ay5YoqOYuFncYBGa78Ckg/Fc8t6MF4OaZhtzlc9O16bNsgB0fKDDkrlbvblGBXx62ZqL kkIj3+KSXWEXR+AMbc3PB0j7Fhhv2yHH3MBrVP4JTcpba0E8hu6lf6FUt8KEpiKFKbzS Nr0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323493; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ldFupt5wrZbRxkmdbQJ7PqzJnkF6w4MSZ/CZLw9J9jU=; b=4Ijn8dGLsYTuXuJN1KE8BnX95ftX4uYEwdWyzrbL1iFJatug5Mdr7SSOCqcFx+YGV8 Aqr09L4EpiXJxnW2JhK+3ZdG5BvRQ/kppHDkZT73XfwXgi8zY5R8Cochq/5u9/KutYLf N9lv5c/kpTkIUjcUkEfaC8PZF8BRyphYykkW5UZoPFF8j9eI1p1qFLZGQ8TL5NPK1w/7 OnaM5474dNWIiDTRiUOUrkYBPIZtbbLlqMn26z0L9pQnA6owu3f7ibpLr0hix5s2MOrT yScQZp2L80CQf4WqL5Tg02DqoHRAY9lGbBqWyIBucAKb+csgf8kLpSckSi/jOOgqaCrB SZmQ== X-Gm-Message-State: AO0yUKUBD+mWInhycCZqPa4ZV0Lj3Pj3mu59AuLSBBuqP9bs+cp5NgLS dBuS3crU8CWc5vB+qwX4Hxo= X-Google-Smtp-Source: AK7set8ban8EVPY/GlOSyVWpuGWIu9Ktpe8j0t2q1ieP66G7xMsPkz9aTzAfar0y6DwailYRK/6igQ== X-Received: by 2002:a05:6a20:bc9e:b0:d9:f086:e756 with SMTP id fx30-20020a056a20bc9e00b000d9f086e756mr1931648pzb.39.1679323493697; Mon, 20 Mar 2023 07:44:53 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id j8-20020aa78dc8000000b00571f66721aesm6454473pfr.42.2023.03.20.07.44.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:44:53 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Akhil P Oommen , Chia-I Wu , Dmitry Osipenko , Luca Weiss , Maximilian Luz , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 12/23] drm/msm/gpu: Move fw loading out of hw_init() path Date: Mon, 20 Mar 2023 07:43:34 -0700 Message-Id: <20230320144356.803762-13-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark It is already a no-op, since we've already loaded the fw from adreno_load_gpu(), so drop the redundant call. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c index 817599766329..28cc5685ba96 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c @@ -503,16 +503,9 @@ struct drm_gem_object *adreno_fw_create_bo(struct msm_gpu *gpu, int adreno_hw_init(struct msm_gpu *gpu) { - struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); - int ret, i; - VERB("%s", gpu->name); - ret = adreno_load_fw(adreno_gpu); - if (ret) - return ret; - - for (i = 0; i < gpu->nr_rings; i++) { + for (int i = 0; i < gpu->nr_rings; i++) { struct msm_ringbuffer *ring = gpu->rb[i]; if (!ring) From patchwork Mon Mar 20 14:43:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665258 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EA960C7618A for ; Mon, 20 Mar 2023 14:46:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231954AbjCTOqS (ORCPT ); Mon, 20 Mar 2023 10:46:18 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231904AbjCTOpu (ORCPT ); Mon, 20 Mar 2023 10:45:50 -0400 Received: from mail-pl1-x634.google.com (mail-pl1-x634.google.com [IPv6:2607:f8b0:4864:20::634]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 98488B459; Mon, 20 Mar 2023 07:45:04 -0700 (PDT) Received: by mail-pl1-x634.google.com with SMTP id w4so4587895plg.9; Mon, 20 Mar 2023 07:45:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323502; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iRudoYKJ/9RBEgJ+2j3GsDNIAjaLf24fOR0Git+DXzM=; b=FSiCBrQqb/DttiCvh4W10nxnr4IwYmxqhvZrOxgMDQkWUirVLq87/u2vJrBad/daDh qw++XRKpNrVXUMtZedrPPKhpDSyZukdY/4ufBj5U2makZ08JuIbMHnJSOiC4xm1bzrjS HzPLQJOPxkS0Er/frUYuRF9+wzmCTAdEXPRxpH9m4E1e4+TSidc5nWZdbGAXgA9UJObW LQ8eG98up+TO8k88KCpl5YaCgqinGhFAOq+NXZiPa6fHGsmnWtfhPJltAZ0vlG4R9Kn2 hJvQqOIM1WcqtU43N+oX6hFcHJ9/IKX54VNK/yLNmLJEvxRuIOe1TvBoZvweJE5xupP+ mKsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323502; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iRudoYKJ/9RBEgJ+2j3GsDNIAjaLf24fOR0Git+DXzM=; b=0OJdm5aZeCUW885rLQNB4S0YOREn1Bvqq50NBa7CMwF4xgxHmCZO68vSC5f6jFfZyW F/RT2OwtjMNrOvSwOQI9kPpjbFBxdc++/7xDL0D31Bs04aItgRgO85shf6vDt17Cc+fx 9UrcRNzHjy4bcP+KL7VKBygoQQxTisjfr4jZ0m2C+WjM3seM39E6BkNEKBvNyv8nXMU6 6UUD3TP5rj6aYwZL6xcmYqAXvUO9Ut8AFMlQ089XP91uE9KnlhuyyZa/kuo4cwIBtt87 CrES5JU/7IJDN+NiJH0UaxYFFPg7HQlWUmFbIAH2M9fsmukCQwR909bcacD09Y8kfo1X cjUg== X-Gm-Message-State: AO0yUKU5v1IqvoBWPlf/+jBHMsd2KzlwAsEjrL5mdLGOhCHzbhv0XX3V eQMiO4U1BoISrZcNODwTKOg= X-Google-Smtp-Source: AK7set9ulfPXo9jK6KpDY96RxgLMtzdpXoEn31dcDtWIB8HP/mch1ngRHsDe2j2Y+tM8AsYX9Ahm6g== X-Received: by 2002:a17:902:e5c3:b0:19e:7d67:84e6 with SMTP id u3-20020a170902e5c300b0019e7d6784e6mr21119604plf.0.1679323502456; Mon, 20 Mar 2023 07:45:02 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id az8-20020a170902a58800b0019aa5528a5csm6793088plb.144.2023.03.20.07.45.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:45:01 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Akhil P Oommen , Konrad Dybcio , Geert Uytterhoeven , Douglas Anderson , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 14/23] drm/msm/a6xx: Move ioremap out of hw_init path Date: Mon, 20 Mar 2023 07:43:36 -0700 Message-Id: <20230320144356.803762-15-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Move the one-time RPMh setup to a6xx_gmu_init(). To get rid of the hack for one-time init vs start, add in an extra a6xx_rpmh_stop() at the end of the init sequence. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c index f3c9600221d4..30a1bf39ea83 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -621,6 +621,8 @@ static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) /* ensure no writes happen before the uCode is fully written */ wmb(); + a6xx_rpmh_stop(gmu); + err: if (!IS_ERR_OR_NULL(pdcptr)) iounmap(pdcptr); @@ -753,7 +755,6 @@ static int a6xx_gmu_fw_load(struct a6xx_gmu *gmu) static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) { - static bool rpmh_init; struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; int ret; @@ -776,15 +777,9 @@ static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) /* Turn on register retention */ gmu_write(gmu, REG_A6XX_GMU_GENERAL_7, 1); - /* We only need to load the RPMh microcode once */ - if (!rpmh_init) { - a6xx_gmu_rpmh_init(gmu); - rpmh_init = true; - } else { - ret = a6xx_rpmh_start(gmu); - if (ret) - return ret; - } + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; ret = a6xx_gmu_fw_load(gmu); if (ret) @@ -1633,6 +1628,9 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node) /* Set up the HFI queues */ a6xx_hfi_init(gmu); + /* Initialize RPMh */ + a6xx_gmu_rpmh_init(gmu); + gmu->initialized = true; return 0; From patchwork Mon Mar 20 14:43:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A00D0C6FD1D for ; Mon, 20 Mar 2023 14:46:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231942AbjCTOqR (ORCPT ); Mon, 20 Mar 2023 10:46:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51850 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231825AbjCTOpu (ORCPT ); Mon, 20 Mar 2023 10:45:50 -0400 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DB78D1024A; Mon, 20 Mar 2023 07:45:05 -0700 (PDT) Received: by mail-pf1-x42b.google.com with SMTP id fd25so7071697pfb.1; Mon, 20 Mar 2023 07:45:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323504; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Qu7alhJ6wWy9WzDcFrk58GfmlRJKvb1Npis5WJImQUk=; b=e/FNCZpTNWAV+E5t5npeOCsIakIkV+ragj5nuKWTyQ0ksvH7Wj0f/MjaAqvkOI3bxN kYww4vhk4UrVxjm8CqRtiK7B32r1TSYWhFbxIvO52TOAJVcilv0wjdmq+8V0WVhlEqJ+ 5L89sNZ4JEHFpnBDbudoUjmsIlFDaVzEgB4MSpsl4K0iZWnLddZ6epdjRw33YKTV96Lo VUOaLxw6oOD5rwbZkxcbQE9bjykyh2CfLHh3qeLlNO0+40+2IkwbKSXKfuz4GfS2hBBQ WNh1RbhlF9XONK9T6JHNdlW27qhvTVJTCY3UcsZIUBcKFPSuEuAmhvrT7xU9Wr/s2qw7 A7Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323504; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Qu7alhJ6wWy9WzDcFrk58GfmlRJKvb1Npis5WJImQUk=; b=u4X4pSMYjkYM6HMpYipMKje5e3MBfiwineCYlJTC4zClC4dbhh9EXHGg839xRplztW alvfniAcaxHcUh3eM7CcuB/etWtHrgzXzYej0kzdeJcq3XItMyB/4aUtbashUgF7cShh NY340Z6AtVD651D0JGgMoxDJ1Xk1WxUGR/iFH6C1J03Rzo7t7JPYNqrNWe4JuEoqGp13 ISWiMNbtJXsKMpufmPmL2hQfilOlMEE5b24AfCPilplaZD5dGMpHI9US4aKY0xQhvzzw girjiFxXRaqRqcp3bzrBPPS3MmYPTAiyp2hjIS22UtE9U6qQXszFhPOT9Sp8jz2JRI6P AHEA== X-Gm-Message-State: AO0yUKUuGX2mGPLml/IIgtPJL9/XMp9Y1xwyRcJC4XwuBxY59LuZ8QRF XM6Oe0HoBS2NyDpC9hQURdE7SJMKMvI= X-Google-Smtp-Source: AK7set//Qk9cXDTXMYexJ0g5upjPAMn6/IlbJvD3ot8I7O6eFbD1nrN/RI0I2HcbbYaxb7YioZLATw== X-Received: by 2002:a62:7b95:0:b0:625:e051:e462 with SMTP id w143-20020a627b95000000b00625e051e462mr15333738pfc.15.1679323504322; Mon, 20 Mar 2023 07:45:04 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id r14-20020a62e40e000000b00627ee6dcb84sm3045993pfh.203.2023.03.20.07.45.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:45:03 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , MyungJoo Ham , Kyungmin Park , Chanwoo Choi , linux-pm@vger.kernel.org (open list:DEVICE FREQUENCY (DEVFREQ)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 15/23] PM / devfreq: Drop unneed locking to appease lockdep Date: Mon, 20 Mar 2023 07:43:37 -0700 Message-Id: <20230320144356.803762-16-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark In the process of adding lockdep annotation for GPU job_run() path to catch potential deadlocks against the shrinker/reclaim path, I turned up this lockdep splat: ====================================================== WARNING: possible circular locking dependency detected 6.2.0-rc8-debug+ #556 Not tainted ------------------------------------------------------ ring0/123 is trying to acquire lock: ffffff8087219078 (&devfreq->lock){+.+.}-{3:3}, at: devfreq_monitor_resume+0x3c/0xf0 but task is already holding lock: ffffffd6f64e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #3 (dma_fence_map){++++}-{0:0}: __dma_fence_might_wait+0x74/0xc0 dma_resv_lockdep+0x1f4/0x2f4 do_one_initcall+0x104/0x2bc kernel_init_freeable+0x344/0x34c kernel_init+0x30/0x134 ret_from_fork+0x10/0x20 -> #2 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}: fs_reclaim_acquire+0x80/0xa8 slab_pre_alloc_hook.constprop.0+0x40/0x25c __kmem_cache_alloc_node+0x60/0x1cc __kmalloc+0xd8/0x100 topology_parse_cpu_capacity+0x8c/0x178 get_cpu_for_node+0x88/0xc4 parse_cluster+0x1b0/0x28c parse_cluster+0x8c/0x28c init_cpu_topology+0x168/0x188 smp_prepare_cpus+0x24/0xf8 kernel_init_freeable+0x18c/0x34c kernel_init+0x30/0x134 ret_from_fork+0x10/0x20 -> #1 (fs_reclaim){+.+.}-{0:0}: __fs_reclaim_acquire+0x3c/0x48 fs_reclaim_acquire+0x54/0xa8 slab_pre_alloc_hook.constprop.0+0x40/0x25c __kmem_cache_alloc_node+0x60/0x1cc __kmalloc_node_track_caller+0xb8/0xe0 kstrdup+0x70/0x90 kstrdup_const+0x38/0x48 kvasprintf_const+0x48/0xbc kobject_set_name_vargs+0x40/0xb0 dev_set_name+0x64/0x8c devfreq_add_device+0x31c/0x55c devm_devfreq_add_device+0x6c/0xb8 msm_devfreq_init+0xa8/0x16c msm_gpu_init+0x38c/0x570 adreno_gpu_init+0x1b4/0x2b4 a6xx_gpu_init+0x15c/0x3e4 adreno_bind+0x218/0x254 component_bind_all+0x114/0x1ec msm_drm_bind+0x2b8/0x608 try_to_bring_up_aggregate_device+0x88/0x1a4 __component_add+0xec/0x13c component_add+0x1c/0x28 dsi_dev_attach+0x28/0x34 dsi_host_attach+0xdc/0x124 mipi_dsi_attach+0x30/0x44 devm_mipi_dsi_attach+0x2c/0x70 ti_sn_bridge_probe+0x298/0x2c4 auxiliary_bus_probe+0x7c/0x94 really_probe+0x158/0x290 __driver_probe_device+0xc8/0xe0 driver_probe_device+0x44/0x100 __device_attach_driver+0x64/0xdc bus_for_each_drv+0xa0/0xc8 __device_attach+0xd8/0x168 device_initial_probe+0x1c/0x28 bus_probe_device+0x38/0xa0 deferred_probe_work_func+0xc8/0xe0 process_one_work+0x2d8/0x478 process_scheduled_works+0x4c/0x50 worker_thread+0x218/0x274 kthread+0xf0/0x100 ret_from_fork+0x10/0x20 -> #0 (&devfreq->lock){+.+.}-{3:3}: __lock_acquire+0xe00/0x1060 lock_acquire+0x1e0/0x2f8 __mutex_lock+0xcc/0x3c8 mutex_lock_nested+0x30/0x44 devfreq_monitor_resume+0x3c/0xf0 devfreq_simple_ondemand_handler+0x54/0x7c devfreq_resume_device+0xa4/0xe8 msm_devfreq_resume+0x78/0xa8 a6xx_pm_resume+0x110/0x234 adreno_runtime_resume+0x2c/0x38 pm_generic_runtime_resume+0x30/0x44 __rpm_callback+0x15c/0x174 rpm_callback+0x78/0x7c rpm_resume+0x318/0x524 __pm_runtime_resume+0x78/0xbc pm_runtime_get_sync.isra.0+0x14/0x20 msm_gpu_submit+0x58/0x178 msm_job_run+0x78/0x150 drm_sched_main+0x290/0x370 kthread+0xf0/0x100 ret_from_fork+0x10/0x20 other info that might help us debug this: Chain exists of: &devfreq->lock --> mmu_notifier_invalidate_range_start --> dma_fence_map Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(dma_fence_map); lock(mmu_notifier_invalidate_range_start); lock(dma_fence_map); lock(&devfreq->lock); *** DEADLOCK *** 2 locks held by ring0/123: #0: ffffff8087201170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150 #1: ffffffd6f64e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150 stack backtrace: CPU: 6 PID: 123 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #556 Hardware name: Google Lazor (rev1 - 2) with LTE (DT) Call trace: dump_backtrace.part.0+0xb4/0xf8 show_stack+0x20/0x38 dump_stack_lvl+0x9c/0xd0 dump_stack+0x18/0x34 print_circular_bug+0x1b4/0x1f0 check_noncircular+0x78/0xac __lock_acquire+0xe00/0x1060 lock_acquire+0x1e0/0x2f8 __mutex_lock+0xcc/0x3c8 mutex_lock_nested+0x30/0x44 devfreq_monitor_resume+0x3c/0xf0 devfreq_simple_ondemand_handler+0x54/0x7c devfreq_resume_device+0xa4/0xe8 msm_devfreq_resume+0x78/0xa8 a6xx_pm_resume+0x110/0x234 adreno_runtime_resume+0x2c/0x38 pm_generic_runtime_resume+0x30/0x44 __rpm_callback+0x15c/0x174 rpm_callback+0x78/0x7c rpm_resume+0x318/0x524 __pm_runtime_resume+0x78/0xbc pm_runtime_get_sync.isra.0+0x14/0x20 msm_gpu_submit+0x58/0x178 msm_job_run+0x78/0x150 drm_sched_main+0x290/0x370 kthread+0xf0/0x100 ret_from_fork+0x10/0x20 The issue is that we cannot be holding any lock while doing memory allocations that is also needed in the job_run (and in the case of devfreq, this means runpm_resume()) because lockdep sees this as a potential dependency. Fortunately there is really no reason to hold the devfreq lock when we are creating the devfreq device, as it is not yet visible to any other task. The only reason it was needed was for a lockdep assert in devfreq_get_freq_range(). Instead, split this up into an internal fxn that is used in the devfreq_add_device() (where the lock is not required). Signed-off-by: Rob Clark --- drivers/devfreq/devfreq.c | 46 ++++++++++++++++++--------------------- 1 file changed, 21 insertions(+), 25 deletions(-) diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c index 817c71da391a..11b774048bd2 100644 --- a/drivers/devfreq/devfreq.c +++ b/drivers/devfreq/devfreq.c @@ -111,23 +111,13 @@ static unsigned long find_available_max_freq(struct devfreq *devfreq) return max_freq; } -/** - * devfreq_get_freq_range() - Get the current freq range - * @devfreq: the devfreq instance - * @min_freq: the min frequency - * @max_freq: the max frequency - * - * This takes into consideration all constraints. - */ -void devfreq_get_freq_range(struct devfreq *devfreq, - unsigned long *min_freq, - unsigned long *max_freq) +static void __get_freq_range(struct devfreq *devfreq, + unsigned long *min_freq, + unsigned long *max_freq) { unsigned long *freq_table = devfreq->freq_table; s32 qos_min_freq, qos_max_freq; - lockdep_assert_held(&devfreq->lock); - /* * Initialize minimum/maximum frequency from freq table. * The devfreq drivers can initialize this in either ascending or @@ -158,6 +148,23 @@ void devfreq_get_freq_range(struct devfreq *devfreq, if (*min_freq > *max_freq) *min_freq = *max_freq; } + +/** + * devfreq_get_freq_range() - Get the current freq range + * @devfreq: the devfreq instance + * @min_freq: the min frequency + * @max_freq: the max frequency + * + * This takes into consideration all constraints. + */ +void devfreq_get_freq_range(struct devfreq *devfreq, + unsigned long *min_freq, + unsigned long *max_freq) +{ + lockdep_assert_held(&devfreq->lock); + + __get_freq_range(devfreq, min_freq, max_freq); +} EXPORT_SYMBOL(devfreq_get_freq_range); /** @@ -810,7 +817,6 @@ struct devfreq *devfreq_add_device(struct device *dev, } mutex_init(&devfreq->lock); - mutex_lock(&devfreq->lock); devfreq->dev.parent = dev; devfreq->dev.class = devfreq_class; devfreq->dev.release = devfreq_dev_release; @@ -823,17 +829,14 @@ struct devfreq *devfreq_add_device(struct device *dev, if (devfreq->profile->timer < 0 || devfreq->profile->timer >= DEVFREQ_TIMER_NUM) { - mutex_unlock(&devfreq->lock); err = -EINVAL; goto err_dev; } if (!devfreq->profile->max_state || !devfreq->profile->freq_table) { - mutex_unlock(&devfreq->lock); err = set_freq_table(devfreq); if (err < 0) goto err_dev; - mutex_lock(&devfreq->lock); } else { devfreq->freq_table = devfreq->profile->freq_table; devfreq->max_state = devfreq->profile->max_state; @@ -841,19 +844,17 @@ struct devfreq *devfreq_add_device(struct device *dev, devfreq->scaling_min_freq = find_available_min_freq(devfreq); if (!devfreq->scaling_min_freq) { - mutex_unlock(&devfreq->lock); err = -EINVAL; goto err_dev; } devfreq->scaling_max_freq = find_available_max_freq(devfreq); if (!devfreq->scaling_max_freq) { - mutex_unlock(&devfreq->lock); err = -EINVAL; goto err_dev; } - devfreq_get_freq_range(devfreq, &min_freq, &max_freq); + __get_freq_range(devfreq, &min_freq, &max_freq); devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev); devfreq->opp_table = dev_pm_opp_get_opp_table(dev); @@ -865,7 +866,6 @@ struct devfreq *devfreq_add_device(struct device *dev, dev_set_name(&devfreq->dev, "%s", dev_name(dev)); err = device_register(&devfreq->dev); if (err) { - mutex_unlock(&devfreq->lock); put_device(&devfreq->dev); goto err_out; } @@ -876,7 +876,6 @@ struct devfreq *devfreq_add_device(struct device *dev, devfreq->max_state), GFP_KERNEL); if (!devfreq->stats.trans_table) { - mutex_unlock(&devfreq->lock); err = -ENOMEM; goto err_devfreq; } @@ -886,7 +885,6 @@ struct devfreq *devfreq_add_device(struct device *dev, sizeof(*devfreq->stats.time_in_state), GFP_KERNEL); if (!devfreq->stats.time_in_state) { - mutex_unlock(&devfreq->lock); err = -ENOMEM; goto err_devfreq; } @@ -896,8 +894,6 @@ struct devfreq *devfreq_add_device(struct device *dev, srcu_init_notifier_head(&devfreq->transition_notifier_list); - mutex_unlock(&devfreq->lock); - err = dev_pm_qos_add_request(dev, &devfreq->user_min_freq_req, DEV_PM_QOS_MIN_FREQUENCY, 0); if (err < 0) From patchwork Mon Mar 20 14:43:40 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665256 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4472C6FD1D for ; Mon, 20 Mar 2023 14:46:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232009AbjCTOqw (ORCPT ); Mon, 20 Mar 2023 10:46:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52748 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231899AbjCTOqR (ORCPT ); Mon, 20 Mar 2023 10:46:17 -0400 Received: from mail-pj1-x1030.google.com (mail-pj1-x1030.google.com [IPv6:2607:f8b0:4864:20::1030]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id F3C79D302; Mon, 20 Mar 2023 07:45:10 -0700 (PDT) Received: by mail-pj1-x1030.google.com with SMTP id p3-20020a17090a74c300b0023f69bc7a68so8146731pjl.4; Mon, 20 Mar 2023 07:45:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323509; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4LG2BAueGKKjRd5oOzkHW01KAXz7hJtO0YTeAZX7ebQ=; b=JYMfuqbtpiJILVkp0uoqa+oBsFme/4zOTzeVWYi3j3wR1ZQJWuMq7gV27NVroQ4Cof DvrYwJDRdr8vjUe7AoOTjec4wKPvjzpSGWLEwjaFkMJH0s8z+PQvoCMmcpojocLE7ybO oo+/Hl2TVSIoCQa+s4zQECFi1GZGH4VbTC86Yqlf91CDneoARc4TOJ4Z2guCXmZDIpSu /196tgzmV3xQEvnInxfSyu1LNUDP3vSbz96LAF3Eqc21Kvo8zow4uMY7aNpcZ4+NT4Yg MDg9DNeVPHKmg+hxMcvtXfab3OuON64cpe+DAyLDIvSqDWJdJRf7qPJfP/BVFmTravPm rQuw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323509; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4LG2BAueGKKjRd5oOzkHW01KAXz7hJtO0YTeAZX7ebQ=; b=UrNvbjjWzYc5ua1kh5fogAC/kGvNkeEO/mH/8s/RbZZnP5042J0uXChEEoknNxGfyL UH0zwFqJX5eCROH24/M7uqIub0l6hBYTbUytYna8balEm6JVHnQ11PrV41dFdzSDCIY8 R0ra9MXwCNmEKUHi231Ff/MMMmnYXA+w2Ohl9AtMfi+iYijV0rdbnsbLxlnJonftPWBi vNQy1B4vI7N7juWymju54QnKUyavLrZlede47FenT4ZvGXRY5U0MPlL2eMJLgFKBylXX pyBojVIwStxyzXRLswXcxcsIJ/XrVDiYBLfNkJTLVGORmOWVQmZBb6I7HPruWKnV38Jb IesA== X-Gm-Message-State: AO0yUKWBbmQ1kD6WIYSIHf1CX+ZiTOVHuCM010AlE4lYEfCYVXjrExMj dF4i+Z6WxaAITxPJYwZc1Uw= X-Google-Smtp-Source: AK7set8ZMssF9rVaP9Q2tyqSeIvwgvafCYLFB4nuIVnqISbFD6NVpbGZbgq3RCYIRz5Qz1iGPiJ1mg== X-Received: by 2002:a17:902:c40a:b0:1a1:b9e0:fa1c with SMTP id k10-20020a170902c40a00b001a1b9e0fa1cmr10839593plk.0.1679323509573; Mon, 20 Mar 2023 07:45:09 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id io7-20020a17090312c700b001a1ca6dc38csm2920651plb.118.2023.03.20.07.45.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:45:09 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , "Rafael J. Wysocki" , Len Brown , Pavel Machek , Greg Kroah-Hartman , linux-pm@vger.kernel.org (open list:POWER MANAGEMENT CORE), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 18/23] PM / QoS: Decouple request alloc from dev_pm_qos_mtx Date: Mon, 20 Mar 2023 07:43:40 -0700 Message-Id: <20230320144356.803762-19-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Similar to the previous patch, move the allocation out from under dev_pm_qos_mtx, by speculatively doing the allocation and handle any race after acquiring dev_pm_qos_mtx by freeing the redundant allocation. Signed-off-by: Rob Clark --- drivers/base/power/qos.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c index f3e0c6b65635..9cba334b3729 100644 --- a/drivers/base/power/qos.c +++ b/drivers/base/power/qos.c @@ -922,12 +922,16 @@ s32 dev_pm_qos_get_user_latency_tolerance(struct device *dev) */ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val) { + struct dev_pm_qos_request *req = NULL; int ret; ret = dev_pm_qos_constraints_ensure_allocated(dev); if (ret) return ret; + if (!dev->power.qos->latency_tolerance_req) + req = kzalloc(sizeof(*req), GFP_KERNEL); + mutex_lock(&dev_pm_qos_mtx); if (!dev->power.qos->latency_tolerance_req) { @@ -940,7 +944,6 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val) ret = -EINVAL; goto out; } - req = kzalloc(sizeof(*req), GFP_KERNEL); if (!req) { ret = -ENOMEM; goto out; @@ -952,6 +955,13 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val) } dev->power.qos->latency_tolerance_req = req; } else { + /* + * If we raced with another thread to allocate the request, + * simply free the redundant allocation and move on. + */ + if (req) + kfree(req); + if (val < 0) { __dev_pm_qos_drop_user_request(dev, DEV_PM_QOS_LATENCY_TOLERANCE); ret = 0; From patchwork Mon Mar 20 14:43:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 85E61C7618D for ; Mon, 20 Mar 2023 14:46:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231934AbjCTOqh (ORCPT ); Mon, 20 Mar 2023 10:46:37 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230238AbjCTOqO (ORCPT ); Mon, 20 Mar 2023 10:46:14 -0400 Received: from mail-pj1-x1032.google.com (mail-pj1-x1032.google.com [IPv6:2607:f8b0:4864:20::1032]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4C980B763; Mon, 20 Mar 2023 07:45:13 -0700 (PDT) Received: by mail-pj1-x1032.google.com with SMTP id e15-20020a17090ac20f00b0023d1b009f52so16746144pjt.2; Mon, 20 Mar 2023 07:45:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QfAHpBWoMxTHMa2AnmzhjkXruv6iv5Hi+uoFH6JuoB0=; b=dvZaXVTGYKr1InrytzOA1sJ4NQK/BXq3NKHKzdudDkqaq0XSBSXf9z3k/Luykto2iP c+ZneJL4VSdLkTAQNEKxqGKTXzz+oLy7ySCL//+bGiDqqS8Kos40r9xWpaOKAqeVAX1I 8ud9G7quF6etshvv3cC+Cfw4AzbUl/5lJl5hQSMV5zMZN1hbdM58wMEWfU4eTlKS0D+h 8Km10S+bHJjWVXYJC7OKKLW+B6GYh+XLF9XGIOetq1OYyvj5k7FAok2JG+bMRN1OYN8H qUwQuvMHK6z5XutxhtjFHU09oWcsffFuyKjkKHs5HxjpqzlLs1cvryGa/JrKx16Z+bb/ eArw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QfAHpBWoMxTHMa2AnmzhjkXruv6iv5Hi+uoFH6JuoB0=; b=XnmPL5XTe1//zrDlUU4S2MPP1SDN/vVFqhA4pu4MyQi9HWggeyEN60cUNZW/wWwQp7 WcDs6p2KNRFpbnnm1LeQBj1bLmVa3/4fYhxcYdXqPbQ2SEnrftK3pYQMouWPHOz1/Dp5 19txSRBXPI1v5PgpeXNkthgkJLdTIDuacSG6Oput7ybzfPBDDGGM6KiTEth8CtYaiFh8 5YZWSE2HLqA4JRdR3x+gmKKA/LFCmWY67ubM7ymJtwULpq7VzRLeeqgWz7geiKOcKcFa CZAHeKZ/k/9w7/9/NotHqk6pCjiYDqwOgl2oMMc40GtXDQN9onRfJmCNDlNFnn3xa69j WGTA== X-Gm-Message-State: AO0yUKW5V0N4AdMn32iNAjPhiiFqXKNhXC26dJBCbrHKqaGkocLMxexw 1EP8G+qev7mqLiGlAR8E5fI= X-Google-Smtp-Source: AK7set/kgpnfgYQ39cq6tSqCbmQJlYOTfrb5Xz1qqT6kTEByPM9h/PbOYeIY23ga+5pEL0JwAhoBsw== X-Received: by 2002:a17:902:f9cf:b0:19c:dedd:2ace with SMTP id kz15-20020a170902f9cf00b0019cdedd2acemr15267590plb.18.1679323512781; Mon, 20 Mar 2023 07:45:12 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id jc7-20020a17090325c700b001a0667822c8sm6818324plb.94.2023.03.20.07.45.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:45:12 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Andy Gross , Bjorn Andersson , Konrad Dybcio , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 20/23] soc: qcom: smd-rpm: Use GFP_ATOMIC in write path Date: Mon, 20 Mar 2023 07:43:42 -0700 Message-Id: <20230320144356.803762-21-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Preparing for better lockdep annotations for things that happen in runpm suspend/resume path vs shrinker/reclaim in the following patches, we need to avoid allocations that can trigger reclaim in the icc_set_bw() path. In the RPMh case, rpmh_write_batch() already uses GFP_ATOMIC, so it should be reasonable to use in the smd-rpm case as well. Alternatively, 256bytes is small enough for a function that isn't called recursively to allocate on-stack. Signed-off-by: Rob Clark --- drivers/soc/qcom/smd-rpm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c index 7e3b6a7ea34c..478da981d9fb 100644 --- a/drivers/soc/qcom/smd-rpm.c +++ b/drivers/soc/qcom/smd-rpm.c @@ -113,7 +113,7 @@ int qcom_rpm_smd_write(struct qcom_smd_rpm *rpm, if (WARN_ON(size >= 256)) return -EINVAL; - pkt = kmalloc(size, GFP_KERNEL); + pkt = kmalloc(size, GFP_ATOMIC); if (!pkt) return -ENOMEM; From patchwork Mon Mar 20 14:43:44 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 665255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45FDFC6FD1D for ; Mon, 20 Mar 2023 14:46:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231909AbjCTOq4 (ORCPT ); Mon, 20 Mar 2023 10:46:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52012 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231950AbjCTOqS (ORCPT ); Mon, 20 Mar 2023 10:46:18 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A47C5FF1F; Mon, 20 Mar 2023 07:45:16 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id fd25so7072120pfb.1; Mon, 20 Mar 2023 07:45:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; t=1679323516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZTj6lDaB879i6vS4ZDrKb4b2c9OODf1NGI0zVIt2OIQ=; b=Bj+oH3CQChclMtHQOjFJOIB+Z7POqoHz3M05DrKcE49Bnf7ohSE3pDsx71PYs3ECvr oTHseaJiODm/MyHSlBMBASGteA5HzjOenIVagUXFeZlt7Q+3YFMZoD7rUejgRISy+aVY 2odaelScwYvdePOWDQYmQtu9g1I2XR+9uN9ba8aIcs04yVVbk2zqIX1fxAN1D/wbDGbE GKnk4Dos7F1ERtbzl7QoDAY8Tv0VU4CBkNuZAfbMf6wRdovcBhxA0rKvl1ZusIAFOTqG Ji6tDLvUkL6ZHoI/zPauDcA1VF/2Q7lzDyCG6Rg7kTodWzwhyU4JJcSU+8FUk5u7Mgzx O7Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1679323516; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZTj6lDaB879i6vS4ZDrKb4b2c9OODf1NGI0zVIt2OIQ=; b=fvJT19qqknclywQ8N5iPBs4hvjycnTKC7F5IriZQ581YZxPMea6PxZ/g2thCQJZ4iC cUd/DVF2jQTB3+Bt4xAPX3lYGnBaRX5gpPJYXA6oyEK0iU1OFY2FN2gY6j4fOXtnlQDU 7yj2/w1Qe2k67XRV2sua+WD1NoFK4nAOEaFyy+dNC44kUuoxszekUKIO7rP707kWZUCu 9vmkcwsspFNHKf/c7cVQvTXOB9xxsQWJwRyt5Ts2I2VAgVAjxlc1/abbV6SSGNJOSoUK /r51+/3Eao5L0xO+0lzJP2DhO1GhwYcmqJngWoAzNmBDLDT2mlaybGwVTQTWYkPwgHLz h5Rw== X-Gm-Message-State: AO0yUKUUHwpPEzNxKViOLlcC4vgvNTz2Dz3lPlcgNDBonC8oX7N5hSXN o8RaDMMcT1ekio0oaBr7dufZ6TfKbp4= X-Google-Smtp-Source: AK7set+YZdGvBmz/yJ5ACJx//DaOCsv2sHayWOfLM7STeuIIzbdJFihoX6ELSY7Jku8dzTLjKv9Llg== X-Received: by 2002:a62:7946:0:b0:625:2636:9cd2 with SMTP id u67-20020a627946000000b0062526369cd2mr19429597pfc.18.1679323516022; Mon, 20 Mar 2023 07:45:16 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:61b:48ed:72ab:435b]) by smtp.gmail.com with ESMTPSA id x48-20020a056a000bf000b005a79596c795sm6428405pfu.29.2023.03.20.07.45.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 20 Mar 2023 07:45:15 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: Daniel Vetter , freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Georgi Djakov , linux-pm@vger.kernel.org (open list:INTERCONNECT API), linux-kernel@vger.kernel.org (open list) Subject: [PATCH v2 22/23] interconnect: Teach lockdep about icc_bw_lock order Date: Mon, 20 Mar 2023 07:43:44 -0700 Message-Id: <20230320144356.803762-23-robdclark@gmail.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230320144356.803762-1-robdclark@gmail.com> References: <20230320144356.803762-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Teach lockdep that icc_bw_lock is needed in code paths that could deadlock if they trigger reclaim. Signed-off-by: Rob Clark --- drivers/interconnect/core.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/interconnect/core.c b/drivers/interconnect/core.c index f7251784765f..5619963ee85c 100644 --- a/drivers/interconnect/core.c +++ b/drivers/interconnect/core.c @@ -1127,13 +1127,21 @@ void icc_sync_state(struct device *dev) } } } + mutex_unlock(&icc_bw_lock); mutex_unlock(&icc_lock); } EXPORT_SYMBOL_GPL(icc_sync_state); static int __init icc_init(void) { - struct device_node *root = of_find_node_by_path("/"); + struct device_node *root; + + /* Teach lockdep about lock ordering wrt. shrinker: */ + fs_reclaim_acquire(GFP_KERNEL); + might_lock(&icc_bw_lock); + fs_reclaim_release(GFP_KERNEL); + + root = of_find_node_by_path("/"); providers_count = of_count_icc_providers(root); of_node_put(root);