From patchwork Tue Aug 2 15:51:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595387 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2468C19F2C for ; Tue, 2 Aug 2022 15:51:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236174AbiHBPvX (ORCPT ); Tue, 2 Aug 2022 11:51:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54266 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232990AbiHBPvW (ORCPT ); Tue, 2 Aug 2022 11:51:22 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 986DE18367; Tue, 2 Aug 2022 08:51:21 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id 12so12707365pga.1; Tue, 02 Aug 2022 08:51:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=0sMFYlUXA/wvNrdz6yghIiBT/tIS2Mu/kejgpgcpcUk=; b=QhffYyjcWnjYLPwBjsriZPzqgu1BVBqKuCmvl6T++1mJy47oPOFf0+O+nUMsJcr7k+ iv6xi+pVebH6fO1Co6ZrG26mIE/wFc//uVYI3wTapnnRB0zoKv3giOauEVcR68HIDs6O iQZjnfU+B+MdeNbAT/HlqplnYniHOFBw1Dp3dwcG9dq/G9ig3u1AgaBDBA94SJoze+CU 0Rbwl1sM+VC7Mo2Vz+ybyPmKzSok4oDvyGR+wYYEYd7IMdx+3/Vj8Nb2pbRO0/iD/0AX bWDjNV89jF6M2fU4zFsEeH6UaXCPfUKymx2zedVyMfylgaFlofvwUC3b5oMS+ryBIn0i +9pA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=0sMFYlUXA/wvNrdz6yghIiBT/tIS2Mu/kejgpgcpcUk=; b=ksbI/F9AlzVgNJjns5FDiE/2LXOrJqe7FuqjEt3uiTPH9jQC+a1R6I4cNuucWDzzsv kYtY2Dlab4DZ+yu9ihItYHSEHi3s6op6D0cnp37iJhbcm7T0qc/9Pbxej6y1vnv7LrpC Rmw4fEQzz08zX0G780S8ExyTVkkjXpwdWsD8ojd5BeciFqC9qDhqRqRIMnnt7CX5D/HT 98AhDSKF8e9ifq55qCulow9072rBZFycounq2OtemSX60DX8J/knjTAHJJD8on3cFPb2 Cj1wfeo/DpMCuVtes0g5EQpw93zlrpSD2rjoV1V8JepXWjFuCz2l10EnI/FBclYztgls Uq0Q== X-Gm-Message-State: ACgBeo2WFTNRl8XeNgczOPT/iBeTcN/P4SSgzw3V65gagTrHWgxoW+Rv XV51aI4aqa2R2MAcI4QvEX4= X-Google-Smtp-Source: AA6agR6rcDJDhvHJ166iKlgdxWV6djHRkq1yorJlZyS1sgkkWqCGcAl4IEbOloIhRp46PbcH+Fn4mQ== X-Received: by 2002:a63:83c1:0:b0:41c:192e:68f2 with SMTP id h184-20020a6383c1000000b0041c192e68f2mr8179318pge.606.1659455481019; Tue, 02 Aug 2022 08:51:21 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id x16-20020a17090a165000b001f334aa9170sm9537315pje.48.2022.08.02.08.51.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:20 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 01/15] drm/msm: Reorder lock vs submit alloc Date: Tue, 2 Aug 2022 08:51:34 -0700 Message-Id: <20220802155152.1727594-2-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark This lets us drop the NORETRY. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 24 ++++++++++-------------- 1 file changed, 10 insertions(+), 14 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index c9e4aeb14f4a..b7c61a99d274 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -36,7 +36,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, if (sz > SIZE_MAX) return ERR_PTR(-ENOMEM); - submit = kzalloc(sz, GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY); + submit = kzalloc(sz, GFP_KERNEL); if (!submit) return ERR_PTR(-ENOMEM); @@ -771,25 +771,21 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, trace_msm_gpu_submit(pid_nr(pid), ring->id, submitid, args->nr_bos, args->nr_cmds); - ret = mutex_lock_interruptible(&queue->lock); - if (ret) - goto out_post_unlock; - if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) { out_fence_fd = get_unused_fd_flags(O_CLOEXEC); if (out_fence_fd < 0) { ret = out_fence_fd; - goto out_unlock; + return ret; } } - submit = submit_create(dev, gpu, queue, args->nr_bos, - args->nr_cmds); - if (IS_ERR(submit)) { - ret = PTR_ERR(submit); - submit = NULL; - goto out_unlock; - } + submit = submit_create(dev, gpu, queue, args->nr_bos, args->nr_cmds); + if (IS_ERR(submit)) + return PTR_ERR(submit); + + ret = mutex_lock_interruptible(&queue->lock); + if (ret) + goto out_post_unlock; submit->pid = pid; submit->ident = submitid; @@ -965,9 +961,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (ret && (out_fence_fd >= 0)) put_unused_fd(out_fence_fd); mutex_unlock(&queue->lock); +out_post_unlock: if (submit) msm_gem_submit_put(submit); -out_post_unlock: if (!IS_ERR_OR_NULL(post_deps)) { for (i = 0; i < args->nr_out_syncobjs; ++i) { kfree(post_deps[i].chain); From patchwork Tue Aug 2 15:51:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594903 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 470D1C00140 for ; Tue, 2 Aug 2022 15:51:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237111AbiHBPvc (ORCPT ); Tue, 2 Aug 2022 11:51:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54314 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236975AbiHBPvZ (ORCPT ); Tue, 2 Aug 2022 11:51:25 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 37F6718382; Tue, 2 Aug 2022 08:51:24 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id t2-20020a17090a4e4200b001f21572f3a4so15806298pjl.0; Tue, 02 Aug 2022 08:51:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=FlC/bVN5qW7CT1CMAFt2x0HFIxhESZPM94joMv7rafM=; b=evbmRmyO/2+6RJ8/YJry12RHQKfHTrzxGc4UpXeZiB+6x6VMHvdsW+Hml5aWSUMWr6 t1QvkmZTCGcB7M3M73EQ1alGAugs9B4J5K96GjfPpVvWI052CTibRhV+jBuuyp6Rqcx0 Qaget8gCqyN1xaOimZURmkWthQu6ejcX+f1Hr68GpjrG+zSAuWCfFuqrNVXEiwJ4FlV7 gkk5U5eOoIqBe+YN4DC1BwjZBbuqzu8VuK51rYMuBXgloD+TokdPWEraW64oSx1epTk2 VOiB0RubwU0c/j6l1y35t2UHpPhp6KKkwMnL8raxTZm70tgINrIi8Hd9w61dv4uYXZxX v77A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=FlC/bVN5qW7CT1CMAFt2x0HFIxhESZPM94joMv7rafM=; b=zNKNQvMxIaPtdV2+tbjd28v6HTOdLAHXXCtS6kGmR0Mvpqo5FlG8SaO/ci5wO/BPGC 07p8ibhS2ksbZcZ9jekb24YrQyOFpeFdwGry+kDDBeHsX4YFnsWt+TkwwyzDySkKtXlW CGoimCrcIEconNf7NmOOOnaCtCspgK25Jo3jgPS5uCP/W6igTzgs2z/oPBPLTnfIXRyR WAh/5TrlnG10xJ0HgF6WanS55txgWAtnYDKXBZPe5kKYv56sHSNJWrjdC1f3zx2La6tB TmCjiPufkfvXOMz1h3ZOlQcVAr2Qn8Tfg2q8AsfvtOvN3nkKQSIBq4+3LpkgqF2XIURN 9sRw== X-Gm-Message-State: ACgBeo3JNQ75/6CDICl0s20jjq9bMqR4iR44Ae1dI12ZiMA6bIfZbeZW QrWmPDLGpxTv20TMjrKIPw0= X-Google-Smtp-Source: AA6agR7IYOquDSmc/HYkIRdC61n3AqniQbgtQGM8PmI0u+AJIiq0w4H/Dh1mgtBAyFYSYRq/TrPpiA== X-Received: by 2002:a17:903:328e:b0:16e:fa5f:37ae with SMTP id jh14-20020a170903328e00b0016efa5f37aemr6936035plb.148.1659455483649; Tue, 02 Aug 2022 08:51:23 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id a1-20020a170902900100b0016c46ff9741sm11905866plp.67.2022.08.02.08.51.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:22 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 02/15] drm/msm: Small submit cleanup Date: Tue, 2 Aug 2022 08:51:35 -0700 Message-Id: <20220802155152.1727594-3-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Move more initialization into submit_create(). Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index b7c61a99d274..c7819781879c 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -26,6 +26,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, struct msm_gpu_submitqueue *queue, uint32_t nr_bos, uint32_t nr_cmds) { + static atomic_t ident = ATOMIC_INIT(0); struct msm_gem_submit *submit; uint64_t sz; int ret; @@ -52,9 +53,13 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, submit->gpu = gpu; submit->cmd = (void *)&submit->bos[nr_bos]; submit->queue = queue; + submit->pid = get_pid(task_pid(current)); submit->ring = gpu->rb[queue->ring_nr]; submit->fault_dumped = false; + /* Get a unique identifier for the submission for logging purposes */ + submit->ident = atomic_inc_return(&ident) - 1; + INIT_LIST_HEAD(&submit->node); return submit; @@ -718,7 +723,6 @@ static void msm_process_post_deps(struct msm_submit_post_dep *post_deps, int msm_ioctl_gem_submit(struct drm_device *dev, void *data, struct drm_file *file) { - static atomic_t ident = ATOMIC_INIT(0); struct msm_drm_private *priv = dev->dev_private; struct drm_msm_gem_submit *args = data; struct msm_file_private *ctx = file->driver_priv; @@ -729,10 +733,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, struct msm_submit_post_dep *post_deps = NULL; struct drm_syncobj **syncobjs_to_reset = NULL; int out_fence_fd = -1; - struct pid *pid = get_pid(task_pid(current)); bool has_ww_ticket = false; unsigned i; - int ret, submitid; + int ret; if (!gpu) return -ENXIO; @@ -764,12 +767,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (!queue) return -ENOENT; - /* Get a unique identifier for the submission for logging purposes */ - submitid = atomic_inc_return(&ident) - 1; - ring = gpu->rb[queue->ring_nr]; - trace_msm_gpu_submit(pid_nr(pid), ring->id, submitid, - args->nr_bos, args->nr_cmds); if (args->flags & MSM_SUBMIT_FENCE_FD_OUT) { out_fence_fd = get_unused_fd_flags(O_CLOEXEC); @@ -783,13 +781,13 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, if (IS_ERR(submit)) return PTR_ERR(submit); + trace_msm_gpu_submit(pid_nr(submit->pid), ring->id, submit->ident, + args->nr_bos, args->nr_cmds); + ret = mutex_lock_interruptible(&queue->lock); if (ret) goto out_post_unlock; - submit->pid = pid; - submit->ident = submitid; - if (args->flags & MSM_SUBMIT_SUDO) submit->in_rb = true; From patchwork Tue Aug 2 15:51:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595386 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79813C19F2C for ; Tue, 2 Aug 2022 15:51:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237116AbiHBPvd (ORCPT ); Tue, 2 Aug 2022 11:51:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237051AbiHBPvb (ORCPT ); Tue, 2 Aug 2022 11:51:31 -0400 Received: from mail-pj1-x102e.google.com (mail-pj1-x102e.google.com [IPv6:2607:f8b0:4864:20::102e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D1EB71EADB; Tue, 2 Aug 2022 08:51:26 -0700 (PDT) Received: by mail-pj1-x102e.google.com with SMTP id e8-20020a17090a280800b001f2fef7886eso15748314pjd.3; Tue, 02 Aug 2022 08:51:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=AwYCVdrGkHnM2D+4h4BxfmYuAccPj8wFXhKV37+GyLo=; b=ARajg9SXrp7L34xI/L1wA9184aueBkou7taRAYgfMmeFpXW82dWb/YI4KTnJKaEbNV OFtbKltU38+EurNqiYyNdr4uf/qnSQs7hT6xr+aJWWNNHwJSEj9k80xdZwBARN4jMAyg 9ur5vywUmCBQxIUPt2ql9Lw7EVeGMxHeF7/riXMIDRvMEjFM6MYHfbJo14nYlEgzahac TqX5N9JilHHTxo4Vzl9QxoRMXXlVC6TEHxhIfIxaxnszL+KVAuUSUjhT5+Z5DvzazeZI CkDOCDTemgWzlJi0uCdM1SP8gBncRJER24+oONSfIMWh9gLn2/gfKDYTzza4Yp9UYmG/ 8EgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=AwYCVdrGkHnM2D+4h4BxfmYuAccPj8wFXhKV37+GyLo=; b=TaFcRz911J1xNSsIfmsAjPoaurByUKX1ktGDMBHXvhNhxHMXblp4TVbmmd4oBoEeep tx0fDFiZCPxc9ng1pMUPxSkjmaA3c6bfeuGMpwxfpzKtCeh+QNPidSts+YFqCQ5aNVBU 724jE1EUXUKQ2NJqilGduPWxlhmjJtaqV+A20sfs9yXlCtJO9vqSSG933IO/UtDhVa/c Ks3TnXSjBM58y2kmQ2lhNEeHifCUa4WIcrC67hcCFNzBQdVUTjKcbdCweyK50hfm8mwR K216rLi7x7HrzZ45EF2ehucVletGtnX/dWdBEs5J2q9uqYDS3Cx1H5XQ+MfJvz6S6qzI pHKw== X-Gm-Message-State: ACgBeo2ekDTn8slexYp+qFob/X1V8lqbdWdewW8QTawDTkwnm3krOKxg gXvftz3HKU45eI3JqqIjSjA= X-Google-Smtp-Source: AA6agR5ybwbYaWs8UaMfNMG/agnFBgQ19TL8DiPTztRpVe/R+rveUXnDpdfNDBUn4PmfyOb+0vRVOQ== X-Received: by 2002:a17:902:e5d1:b0:16b:ece4:79e6 with SMTP id u17-20020a170902e5d100b0016bece479e6mr21971931plf.83.1659455486238; Tue, 02 Aug 2022 08:51:26 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id y1-20020a17090a2b4100b001e292e30129sm11127180pjc.22.2022.08.02.08.51.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:25 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 03/15] drm/msm: Split out idr_lock Date: Tue, 2 Aug 2022 08:51:36 -0700 Message-Id: <20220802155152.1727594-4-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Otherwise if we hit reclaim pinning objects in the submit path, we'll be blocking retire_worker trying to free a submit. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_drv.c | 4 ++-- drivers/gpu/drm/msm/msm_gem_submit.c | 10 ++++++++-- drivers/gpu/drm/msm/msm_gpu.h | 4 +++- drivers/gpu/drm/msm/msm_submitqueue.c | 1 + 4 files changed, 14 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 1ed4cd09dbf8..d7ca025457b6 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -883,13 +883,13 @@ static int wait_fence(struct msm_gpu_submitqueue *queue, uint32_t fence_id, * retired, so if the fence is not found it means there is nothing * to wait for */ - ret = mutex_lock_interruptible(&queue->lock); + ret = mutex_lock_interruptible(&queue->idr_lock); if (ret) return ret; fence = idr_find(&queue->fence_idr, fence_id); if (fence) fence = dma_fence_get_rcu(fence); - mutex_unlock(&queue->lock); + mutex_unlock(&queue->idr_lock); if (!fence) return 0; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index c7819781879c..16c662808522 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -72,9 +72,9 @@ void __msm_gem_submit_destroy(struct kref *kref) unsigned i; if (submit->fence_id) { - mutex_lock(&submit->queue->lock); + mutex_lock(&submit->queue->idr_lock); idr_remove(&submit->queue->fence_idr, submit->fence_id); - mutex_unlock(&submit->queue->lock); + mutex_unlock(&submit->queue->idr_lock); } dma_fence_put(submit->user_fence); @@ -881,6 +881,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->nr_cmds = i; + mutex_lock(&queue->idr_lock); + /* * If using userspace provided seqno fence, validate that the id * is available before arming sched job. Since access to fence_idr @@ -889,6 +891,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, */ if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) && idr_find(&queue->fence_idr, args->fence)) { + mutex_unlock(&queue->idr_lock); ret = -EINVAL; goto out; } @@ -921,6 +924,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, submit->user_fence, 1, INT_MAX, GFP_KERNEL); } + + mutex_unlock(&queue->idr_lock); + if (submit->fence_id < 0) { ret = submit->fence_id; submit->fence_id = 0; diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 4d935fedd2ac..962d2070bcdf 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -466,7 +466,8 @@ static inline int msm_gpu_convert_priority(struct msm_gpu *gpu, int prio, * @node: node in the context's list of submitqueues * @fence_idr: maps fence-id to dma_fence for userspace visible fence * seqno, protected by submitqueue lock - * @lock: submitqueue lock + * @idr_lock: for serializing access to fence_idr + * @lock: submitqueue lock for serializing submits on a queue * @ref: reference count * @entity: the submit job-queue */ @@ -479,6 +480,7 @@ struct msm_gpu_submitqueue { struct msm_file_private *ctx; struct list_head node; struct idr fence_idr; + struct mutex idr_lock; struct mutex lock; struct kref ref; struct drm_sched_entity *entity; diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c b/drivers/gpu/drm/msm/msm_submitqueue.c index f486a3cd4e55..c6929e205b51 100644 --- a/drivers/gpu/drm/msm/msm_submitqueue.c +++ b/drivers/gpu/drm/msm/msm_submitqueue.c @@ -200,6 +200,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx, *id = queue->id; idr_init(&queue->fence_idr); + mutex_init(&queue->idr_lock); mutex_init(&queue->lock); list_add_tail(&queue->node, &ctx->submitqueues); From patchwork Tue Aug 2 15:51:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5CD44C00140 for ; Tue, 2 Aug 2022 15:51:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237135AbiHBPve (ORCPT ); Tue, 2 Aug 2022 11:51:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237157AbiHBPvc (ORCPT ); Tue, 2 Aug 2022 11:51:32 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A69A8252AE; Tue, 2 Aug 2022 08:51:29 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id f28so5701440pfk.1; Tue, 02 Aug 2022 08:51:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=6E5wGlWgwtmRN55cq/vUaSJZuL1NyuRgvoYtJGWwhUY=; b=GEfOJB7gdA4SxlaauM6Ugt1tZF4JKhQxQH244oEQIqhh9LV1XJGzasRjPRwFOhk6nj LqCY2Zm8HojS7Ghpd7BFD0y/3AEM4H/VSjoCC7jnTy5WxAZF4Cis+szcEaa3F0BtcWaQ OEgtEdaN7O56vF8AlFJTMSQ70vMFGNsQpwIfD0E1IK2s8rLZwRFXNm6iFW5CfwrN72LX bxegJs1R/Myor9Wf2/nRrbu5F5Cj3JEI9p3qBIL38QLw56hBPFlHBSy3rKEQTflMdgXn 6xoFT/4AAos4XOP2Vc0dT+/Xa2uSn43q5vGyGQClCkgeFbcxV2t3ev+fOYpno61xx7eH i5cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=6E5wGlWgwtmRN55cq/vUaSJZuL1NyuRgvoYtJGWwhUY=; b=7PIcWq6WoVm+icsuZvHPrpAvDgFPRqqOThHC0gidendsSpryyyuFie/KF3qolNy7M1 sD2ORkLmWXazd5YpfFI8FAFreFcc/JZjSmSDOkKnyXPcBeILbvisQZf1VhKBKYAKHjok 9H921q+fcOof2d3WBdvNeeIHBTPxX42O0ohB98zx1WITa5n2Vnee5kgXQW9IH3akzeKo lkUNzynOpyyGIZG/EMwvCPYT0xBgKgj6JnQqE8mHqsUbD7RDTFu06XnrLSlBFQDGFHVQ Ep9sLqd1tyCL0HHWJ5P2uX2QoqMCSTNSIoQGlum2b1vsDhZ/d/3aIBS8X0B06vBleuFy BJtw== X-Gm-Message-State: ACgBeo0rVQkAl1XSqHA5B6z1E8ze3F0iwE2pTm3p+StpaR81OoBygb/G mD+uHg4YM8F8xp+iSCTzVrw= X-Google-Smtp-Source: AA6agR5hyY6tEw30Evo0Mcd6KS/1UZLdeHQ2xVV06qRPr03qk2v+YkINNRhwe6EVTfVeOpSWQkvuWw== X-Received: by 2002:a63:448:0:b0:41b:b3a6:c005 with SMTP id 69-20020a630448000000b0041bb3a6c005mr14712235pge.201.1659455488843; Tue, 02 Aug 2022 08:51:28 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id c189-20020a624ec6000000b0052b6ed5ca40sm4871314pfb.192.2022.08.02.08.51.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:27 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 04/15] drm/msm/gem: Check for active in shrinker path Date: Tue, 2 Aug 2022 08:51:37 -0700 Message-Id: <20220802155152.1727594-5-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Currently in our shrinker path we shouldn't be encountering anything that is active, but this will change in subsequent patches. So check if there are unsignaled fences. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 10 ++++++++++ drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gem_shrinker.c | 6 ++++++ 3 files changed, 17 insertions(+) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 8ddbd2e001d4..b55d252aef17 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -870,6 +870,16 @@ static void update_inactive(struct msm_gem_object *msm_obj) mutex_unlock(&priv->mm_lock); } +bool msm_gem_active(struct drm_gem_object *obj) +{ + GEM_WARN_ON(!msm_gem_is_locked(obj)); + + if (to_msm_bo(obj)->pin_count) + return true; + + return !dma_resv_test_signaled(obj->resv, dma_resv_usage_rw(true)); +} + int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout) { bool write = !!(op & MSM_PREP_WRITE); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 432032ad4aed..0ab0dc4f8c25 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -173,6 +173,7 @@ void msm_gem_put_vaddr(struct drm_gem_object *obj); int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv); void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu); void msm_gem_active_put(struct drm_gem_object *obj); +bool msm_gem_active(struct drm_gem_object *obj); int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout); int msm_gem_cpu_fini(struct drm_gem_object *obj); int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file, diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 6e39d959b9f0..ea8ed74982c1 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -43,6 +43,9 @@ purge(struct msm_gem_object *msm_obj) if (!is_purgeable(msm_obj)) return false; + if (msm_gem_active(&msm_obj->base)) + return false; + /* * This will move the obj out of still_in_list to * the purged list @@ -58,6 +61,9 @@ evict(struct msm_gem_object *msm_obj) if (is_unevictable(msm_obj)) return false; + if (msm_gem_active(&msm_obj->base)) + return false; + msm_gem_evict(&msm_obj->base); return true; From patchwork Tue Aug 2 15:51:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595385 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D3BFC19F2C for ; Tue, 2 Aug 2022 15:51:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237265AbiHBPvk (ORCPT ); Tue, 2 Aug 2022 11:51:40 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54540 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbiHBPve (ORCPT ); Tue, 2 Aug 2022 11:51:34 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7931A1ADA1; Tue, 2 Aug 2022 08:51:32 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id z19so13854216plb.1; Tue, 02 Aug 2022 08:51:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=mxFT+vmlgH8OMaHZf0MGM1sQcdQOD7qQazzf52IZIdo=; b=WN+R3EX30Oabvdp1q7dZCqfGguppdYSI4OwmuufkGnHDq2mRy4WcoOGHk63Zj8uGVv BMncLyRJeFdmKS+j1O0CiGCa3yZVVru13iDEQKGjvusLjaxi4dehJMcY5j0AN747MhS/ FtaY50KoEXxF0riKhylVa4yF7/tNv9ZLyUDqoFULnqtiEJYJZbxemb/SaflrZTC6upC0 xw82VGZlbO8bYPREZvAAI9Z//xn5akLF6lMZrou4Eldc/zvpXFlKXDaEdOG+tpTpooeP 5al8QBx/WTUTbwVhLTtfHOJ9ir2BhsZj8ViV0f9nhYuW7cNvHhquBnOe6E8MB/Ihab3R Rk5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=mxFT+vmlgH8OMaHZf0MGM1sQcdQOD7qQazzf52IZIdo=; b=4SQfRkpC6zedX6FmwWG53/4qu4YlflCqM0OsRqoWWwBawtZXnz9+poG+Kl6MAAqAHX hpZjUGXJBZkFMq0mDnKZ+zWJjfWzRuosItpj4CXDVwTWohGMTLsQcXZUMVEbMrK+yB8D 4RqrZZiY89lUDOzbb6SuTKJWhx67Abm8Qmt7BQ7JO/NC8RL0BiToFkESBlnBoCQV3rqU KXWuo4SEMAjrNdmZEEqXTnNbxV189lCxAwNvxeO+sst8pRB3gehwdsXWlAypJCITgFKd ymixJqGcm4JhrMspRg5HaqC0IHVAp5pzznO21D5xUWsEe9rPhe6hXreY0GAmjCtO0NYm cjPw== X-Gm-Message-State: ACgBeo0X//U1JmqsjDC8CU4vJb53TTGJAyhUybLIq85WhiCTjFfsHP4d lCSG3QmUaHvfYkgNTfjN1pjsYuc8bp8= X-Google-Smtp-Source: AA6agR7pFLKO2uUqgp9rOlOUlR1Ah2c0yTxM50D66GE+OjvyQPhVV2ETDK0/78IyRlsx9x0Cxt8rvg== X-Received: by 2002:a17:90a:4291:b0:1f2:2a19:fc95 with SMTP id p17-20020a17090a429100b001f22a19fc95mr114509pjg.29.1659455491353; Tue, 02 Aug 2022 08:51:31 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id l5-20020a170903120500b0016d62ba5665sm12235660plh.254.2022.08.02.08.51.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:30 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 05/15] drm/msm/gem: Rename update_inactive Date: Tue, 2 Aug 2022 08:51:38 -0700 Message-Id: <20220802155152.1727594-6-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Really what this is doing is updating various LRU lists. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index b55d252aef17..97467364dc0a 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -19,7 +19,7 @@ #include "msm_gpu.h" #include "msm_mmu.h" -static void update_inactive(struct msm_gem_object *msm_obj); +static void update_lru(struct drm_gem_object *obj); static dma_addr_t physaddr(struct drm_gem_object *obj) { @@ -132,7 +132,7 @@ static struct page **get_pages(struct drm_gem_object *obj) if (msm_obj->flags & MSM_BO_WC) sync_for_device(msm_obj); - update_inactive(msm_obj); + update_lru(obj); } return msm_obj->pages; @@ -193,7 +193,7 @@ struct page **msm_gem_get_pages(struct drm_gem_object *obj) if (!IS_ERR(p)) { msm_obj->pin_count++; - update_inactive(msm_obj); + update_lru(obj); } msm_gem_unlock(obj); @@ -207,7 +207,7 @@ void msm_gem_put_pages(struct drm_gem_object *obj) msm_gem_lock(obj); msm_obj->pin_count--; GEM_WARN_ON(msm_obj->pin_count < 0); - update_inactive(msm_obj); + update_lru(obj); msm_gem_unlock(obj); } @@ -449,7 +449,7 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) msm_obj->pin_count--; GEM_WARN_ON(msm_obj->pin_count < 0); - update_inactive(msm_obj); + update_lru(obj); } struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj, @@ -658,7 +658,7 @@ static void *get_vaddr(struct drm_gem_object *obj, unsigned madv) goto fail; } - update_inactive(msm_obj); + update_lru(obj); } return msm_obj->vaddr; @@ -730,7 +730,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) * between inactive lists */ if (msm_obj->active_count == 0) - update_inactive(msm_obj); + update_lru(obj); msm_gem_unlock(obj); @@ -757,7 +757,7 @@ void msm_gem_purge(struct drm_gem_object *obj) put_iova_vmas(obj); msm_obj->madv = __MSM_MADV_PURGED; - update_inactive(msm_obj); + update_lru(obj); drm_gem_free_mmap_offset(obj); @@ -792,7 +792,7 @@ void msm_gem_evict(struct drm_gem_object *obj) put_pages(obj); - update_inactive(msm_obj); + update_lru(obj); } void msm_gem_vunmap(struct drm_gem_object *obj) @@ -835,13 +835,14 @@ void msm_gem_active_put(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); if (--msm_obj->active_count == 0) { - update_inactive(msm_obj); + update_lru(obj); } } -static void update_inactive(struct msm_gem_object *msm_obj) +static void update_lru(struct drm_gem_object *obj) { - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; + struct msm_drm_private *priv = obj->dev->dev_private; + struct msm_gem_object *msm_obj = to_msm_bo(obj); GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); From patchwork Tue Aug 2 15:51:39 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594901 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1AD4C00140 for ; Tue, 2 Aug 2022 15:51:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237343AbiHBPvx (ORCPT ); Tue, 2 Aug 2022 11:51:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237336AbiHBPvi (ORCPT ); Tue, 2 Aug 2022 11:51:38 -0400 Received: from mail-pf1-x432.google.com (mail-pf1-x432.google.com [IPv6:2607:f8b0:4864:20::432]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B887D2655D; Tue, 2 Aug 2022 08:51:34 -0700 (PDT) Received: by mail-pf1-x432.google.com with SMTP id f192so4488063pfa.9; Tue, 02 Aug 2022 08:51:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=4TbOfcqFkaHon89dsPDId7NNakdXJT5mDYYEM5ZbQu4=; b=UyEXg/3ai9YzpZukbK9CvD9ZVPgJH40NoiJi5Axn4y3jSqW1cVHKtbMxI8kAGkz53D LMuhDKxpOjOcozuAxu+omHCrBQapma1xG9OjqziQIEQJILB8Gb7HXpr30omM4CR9PTjh OUM6SRhfgVFlyYvmuj936u8lIRPUO4RVMF8ZzsOR/6lZYLqNWfUfIarEO9KAS2j2r7S2 lzFxOHbEfqRkfSpHCsRrLtSkCip/4uEAHYkz1CdsFmnTGG+WpWu6ZSKAZ6BTIpBkUE+c 5pP5pupwqxkUO6Xy1sGGZe0DzvJf0+Li27xCWpeeyhTrfOTz2dzkZVjUTyN1WTRNKZw9 2f0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=4TbOfcqFkaHon89dsPDId7NNakdXJT5mDYYEM5ZbQu4=; b=6y31ufRauQjd/wQJLQ4la/tZmyiY890k1R53cv1+2bXA4LSxi7HO2D3Iimveqt2/Y8 Gk8yN9Ok0aQRT8LMXk9WrPx6LFqixnDccSGLte0udmEmsu1a4yRJ94Q3lHgAMaECmrJC 4OW+5qCk/Y9ZJAEMTy9vLH7s7O/fqrIJWNsTI1+bJrKCcWhYpyQlhyid+JKhr5qJPwCp n6G8WzdN7cDzErN+AUyFJnied+jKASp5oKM5wCppdizkFY1n8CgRcYnM9+eVmtgKPBhh 92C4C8NKsuDphjTcCN4IpE1RfNHPgCEndlX5pUuz3BrsUZ7jF/DGvD0zvtnBGk4rReMk ZwvQ== X-Gm-Message-State: ACgBeo3w4Ytn9q7dDP/iyaXywCRPOVHt7r+LdGROTb3tP2gKZZL8+UIt Z7/rFFm/3jqwLn8ty6hUIp0= X-Google-Smtp-Source: AGRyM1s0NU+nmxXJRj8gxCaZl5yCJjQc3jGxLXaMIsICaXH6zKfyNT1ow1l63tJAMkwWopfrltWwkg== X-Received: by 2002:a05:6a00:10c1:b0:52a:d621:138a with SMTP id d1-20020a056a0010c100b0052ad621138amr21568328pfu.67.1659455493900; Tue, 02 Aug 2022 08:51:33 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id w12-20020a170902e88c00b0016bff65d5cbsm12146299plg.194.2022.08.02.08.51.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:32 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 06/15] drm/msm/gem: Rename to pin/unpin_pages Date: Tue, 2 Aug 2022 08:51:39 -0700 Message-Id: <20220802155152.1727594-7-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Since that is what these fxns actually do.. they are getting *pinned* pages (as opposed to cases where we need pages, but don't need them pinned, like CPU mappings). Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 18 +++++++++++++----- drivers/gpu/drm/msm/msm_gem.h | 4 ++-- drivers/gpu/drm/msm/msm_gem_prime.c | 4 ++-- 3 files changed, 17 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 97467364dc0a..3da64c7f65a2 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -177,30 +177,38 @@ static void put_pages(struct drm_gem_object *obj) } } -struct page **msm_gem_get_pages(struct drm_gem_object *obj) +static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); struct page **p; - msm_gem_lock(obj); + GEM_WARN_ON(!msm_gem_is_locked(obj)); if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) { - msm_gem_unlock(obj); return ERR_PTR(-EBUSY); } p = get_pages(obj); - if (!IS_ERR(p)) { msm_obj->pin_count++; update_lru(obj); } + return p; +} + +struct page **msm_gem_pin_pages(struct drm_gem_object *obj) +{ + struct page **p; + + msm_gem_lock(obj); + p = msm_gem_pin_pages_locked(obj); msm_gem_unlock(obj); + return p; } -void msm_gem_put_pages(struct drm_gem_object *obj) +void msm_gem_unpin_pages(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 0ab0dc4f8c25..6fe521ccda45 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -159,8 +159,8 @@ int msm_gem_get_and_pin_iova(struct drm_gem_object *obj, struct msm_gem_address_space *aspace, uint64_t *iova); void msm_gem_unpin_iova(struct drm_gem_object *obj, struct msm_gem_address_space *aspace); -struct page **msm_gem_get_pages(struct drm_gem_object *obj); -void msm_gem_put_pages(struct drm_gem_object *obj); +struct page **msm_gem_pin_pages(struct drm_gem_object *obj); +void msm_gem_unpin_pages(struct drm_gem_object *obj); int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev, struct drm_mode_create_dumb *args); int msm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev, diff --git a/drivers/gpu/drm/msm/msm_gem_prime.c b/drivers/gpu/drm/msm/msm_gem_prime.c index dcc8a573bc76..c1d91863df05 100644 --- a/drivers/gpu/drm/msm/msm_gem_prime.c +++ b/drivers/gpu/drm/msm/msm_gem_prime.c @@ -63,12 +63,12 @@ struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev, int msm_gem_prime_pin(struct drm_gem_object *obj) { if (!obj->import_attach) - msm_gem_get_pages(obj); + msm_gem_pin_pages(obj); return 0; } void msm_gem_prime_unpin(struct drm_gem_object *obj) { if (!obj->import_attach) - msm_gem_put_pages(obj); + msm_gem_unpin_pages(obj); } From patchwork Tue Aug 2 15:51:40 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595384 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A4742C00140 for ; Tue, 2 Aug 2022 15:51:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237426AbiHBPv5 (ORCPT ); Tue, 2 Aug 2022 11:51:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54494 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237420AbiHBPvk (ORCPT ); Tue, 2 Aug 2022 11:51:40 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04A6E2FFF9; Tue, 2 Aug 2022 08:51:37 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id 23so12684593pgc.8; Tue, 02 Aug 2022 08:51:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=UVam+OtTXmVRJOOFcNCglGVq9lRc/LFnzlVeuJ/h90Y=; b=BoCEnVzSBi7RAIzSPvWu1Dvtj8JjCONeM/LqyJBSilhAbjziNWpZAwTchX2ZR8ZiDl 4wVEJGxjI0oMdrAoAVmNDiKqKWp34jLzZGHz53YcrQgAkT3efp2mNUK2jXutL9fskOJp bhr528oiHsnAwhtqEx2f9ZGY8lFl9K876+DzX+e6F9xJpjz8WsAde4ikphP+JohNbOtO /ox2q+RvnzA9tcWwlJaiDlYoeaQpH2rvcvkd7jF2OVfBLSazlc4J3YSBF0pueFMSmGjH yHzndD6KRy9zu/X+UMSDSrsyuoUFv8zd5XBAr5fE8PFRT6SlKALwRjIU3szAUegzLGl3 zD9Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=UVam+OtTXmVRJOOFcNCglGVq9lRc/LFnzlVeuJ/h90Y=; b=tuvevTNVaHleFDBhYrjVqpM8HLyU1NnFrxOklh6TDtLdizhKB9pP4l8T4yvLSJ0ZUW AZpNssFrouF1RWsj8tvY1Mx2EuU5TJ1/6M0gYHhfdPYiopyliv2Nzti5TEAH0AF6WwX0 fW8/vPCx8R0E1JLVVmYN1XnOhwVvZTyrWHX7gq3mXwqbYkp/HQ8ryIld6IjeUy3+uiHi /SXI+r2dI3tBTB7xKBEXfpkRYCJeJkHN1UiprVXRMOSXMTM494v8WXmmYoGrDhbyjzUU o7xQP308kUpVnTp4yzb2h06BMypmnE0hwXXWlcAqQxer7wF4FR/YXeHGpGtGZz/nf9Ao PVuQ== X-Gm-Message-State: AJIora/oGwpLP+rWunlmMT+ySUBI8ebzPiC7BZeusQVkO6wddiZDL2D9 bAUNm4fyYrP71GzlRPBFzkTZxQ8xEVI= X-Google-Smtp-Source: AGRyM1suvrACZI+8um7dDawoDINq49y8ixC51+7RqQbVR7tOyRb4TqHa90P8pk/gPS8JWY8Y7cVpeA== X-Received: by 2002:a05:6a00:b4c:b0:52a:f6e7:818b with SMTP id p12-20020a056a000b4c00b0052af6e7818bmr21411240pfo.26.1659455496396; Tue, 02 Aug 2022 08:51:36 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id ck16-20020a17090afe1000b001f209699d89sm11202793pjb.18.2022.08.02.08.51.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:35 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 07/15] drm/msm/gem: Consolidate pin/unpin paths Date: Tue, 2 Aug 2022 08:51:40 -0700 Message-Id: <20220802155152.1727594-8-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Avoid having multiple spots where we increment/decrement pin_count (and associated LRU updating) Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 3da64c7f65a2..407b18a24dc4 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -190,7 +190,7 @@ static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) p = get_pages(obj); if (!IS_ERR(p)) { - msm_obj->pin_count++; + to_msm_bo(obj)->pin_count++; update_lru(obj); } @@ -213,9 +213,7 @@ void msm_gem_unpin_pages(struct drm_gem_object *obj) struct msm_gem_object *msm_obj = to_msm_bo(obj); msm_gem_lock(obj); - msm_obj->pin_count--; - GEM_WARN_ON(msm_obj->pin_count < 0); - update_lru(obj); + msm_gem_unpin_locked(obj); msm_gem_unlock(obj); } @@ -436,14 +434,13 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma) if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) return -EBUSY; - pages = get_pages(obj); + pages = msm_gem_pin_pages_locked(obj); if (IS_ERR(pages)) return PTR_ERR(pages); ret = msm_gem_map_vma(vma->aspace, vma, prot, msm_obj->sgt, obj->size); - - if (!ret) - msm_obj->pin_count++; + if (ret) + msm_gem_unpin_locked(obj); return ret; } From patchwork Tue Aug 2 15:51:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594900 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6D633C00140 for ; Tue, 2 Aug 2022 15:52:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237532AbiHBPwJ (ORCPT ); Tue, 2 Aug 2022 11:52:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55220 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237284AbiHBPvu (ORCPT ); Tue, 2 Aug 2022 11:51:50 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 956923191E; Tue, 2 Aug 2022 08:51:39 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id y141so13934024pfb.7; Tue, 02 Aug 2022 08:51:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=0Nc8381KvCAKtcJgWAO7GB/O/Xk1BB4fgkOydTh0/AI=; b=R52EHyBE6ongfYe7aJolfdbMhRwvpIueJmyManF1MVN9eBpcjiyYELrFK2WI6aC5aT jZblUe50feRmtvtfSv2cLZD7XXc4+NWT15y6NBCM+kupJTkQgc/Por2nGnEPWZhq3VvC AE5YdNbctsrgpyXbYiQBjKt82xD+2beFLccr+dhzAvBEJ+Q7MrQDHAbkj2fnn8XxBIut v0ZVseGFEM66xgQgDS7I0Veu31qGT7XnD478l+TcnAF74qvzG7nmRui41Rq7Pp86eO/y AZJXeVXZFtHe9cdFjFLKHsXXhqJagP/RwqBdnTbN1gZhW6UagOz+NkgaUMUjjlEyAoMV usrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=0Nc8381KvCAKtcJgWAO7GB/O/Xk1BB4fgkOydTh0/AI=; b=YAJOqMYqDy1vawL9VKtqdZh3HEEI4MKnW4siOY/fsXgpsp7j0sJBZ7aNOUMHQTkWZ9 CL/2wegHeeKKY+2so40xdy9ZEzLiHRQ9yWERqsOSZMVOx91u4furTskynaq5GIjp59gb Mr+zFjPg756klFZo5ubLk6N3xs0tNyXARL1S2KgIpIqsJ5y93A9lndoHXQISm7IYzXPt +ffY2Rr5EDhYLH5DJcplFRmSvF5+pNRomnukkzTQyW9y+1sDAPw+7uNoBYB2eLInMWqd ZxPHIMNghG6Hs2aqTshTf6BPAP7O8FcRQC9qOFwpDusV9guVlDbPoixOqvs3OkEzgimi nXqQ== X-Gm-Message-State: ACgBeo2rzScVlP1zezIcEj28Rh7su0C/gopuhSkTOZ+tVWPsT3mT9+Gm RkoipUrS+SbiBptZ+lBd/TU= X-Google-Smtp-Source: AA6agR4BKfq0tkllqgz2OHakvgQGIYnIki+6EYKklq90oWNCNuxMDN2TiqgC2tQeCu5qABBONZ9hsw== X-Received: by 2002:a05:6a00:986:b0:52d:8c68:7b4 with SMTP id u6-20020a056a00098600b0052d8c6807b4mr9058675pfg.35.1659455499054; Tue, 02 Aug 2022 08:51:39 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id p6-20020a170902780600b0016dbc169316sm7586883pll.187.2022.08.02.08.51.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:37 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 08/15] drm/msm/gem: Remove active refcnt Date: Tue, 2 Aug 2022 08:51:41 -0700 Message-Id: <20220802155152.1727594-9-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark At this point the pinned refcnt is sufficient, and the shrinker is already prepared to encounter objects which are still active according to fences attached to the resv. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 45 ++-------------------------- drivers/gpu/drm/msm/msm_gem.h | 14 ++------- drivers/gpu/drm/msm/msm_gem_submit.c | 22 ++------------ 3 files changed, 8 insertions(+), 73 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 407b18a24dc4..209438744bab 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -734,8 +734,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv) /* If the obj is inactive, we might need to move it * between inactive lists */ - if (msm_obj->active_count == 0) - update_lru(obj); + update_lru(obj); msm_gem_unlock(obj); @@ -788,7 +787,6 @@ void msm_gem_evict(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); GEM_WARN_ON(is_unevictable(msm_obj)); GEM_WARN_ON(!msm_obj->evictable); - GEM_WARN_ON(msm_obj->active_count); /* Get rid of any iommu mapping(s): */ put_iova_spaces(obj, false); @@ -813,37 +811,6 @@ void msm_gem_vunmap(struct drm_gem_object *obj) msm_obj->vaddr = NULL; } -void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu) -{ - struct msm_gem_object *msm_obj = to_msm_bo(obj); - struct msm_drm_private *priv = obj->dev->dev_private; - - might_sleep(); - GEM_WARN_ON(!msm_gem_is_locked(obj)); - GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED); - GEM_WARN_ON(msm_obj->dontneed); - - if (msm_obj->active_count++ == 0) { - mutex_lock(&priv->mm_lock); - if (msm_obj->evictable) - mark_unevictable(msm_obj); - list_move_tail(&msm_obj->mm_list, &gpu->active_list); - mutex_unlock(&priv->mm_lock); - } -} - -void msm_gem_active_put(struct drm_gem_object *obj) -{ - struct msm_gem_object *msm_obj = to_msm_bo(obj); - - might_sleep(); - GEM_WARN_ON(!msm_gem_is_locked(obj)); - - if (--msm_obj->active_count == 0) { - update_lru(obj); - } -} - static void update_lru(struct drm_gem_object *obj) { struct msm_drm_private *priv = obj->dev->dev_private; @@ -851,9 +818,6 @@ static void update_lru(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - if (msm_obj->active_count != 0) - return; - mutex_lock(&priv->mm_lock); if (msm_obj->dontneed) @@ -926,7 +890,7 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, stats->all.count++; stats->all.size += obj->size; - if (is_active(msm_obj)) { + if (msm_gem_active(obj)) { stats->active.count++; stats->active.size += obj->size; } @@ -954,7 +918,7 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, } seq_printf(m, "%08x: %c %2d (%2d) %08llx %p", - msm_obj->flags, is_active(msm_obj) ? 'A' : 'I', + msm_obj->flags, msm_gem_active(obj) ? 'A' : 'I', obj->name, kref_read(&obj->refcount), off, msm_obj->vaddr); @@ -1037,9 +1001,6 @@ static void msm_gem_free_object(struct drm_gem_object *obj) list_del(&msm_obj->mm_list); mutex_unlock(&priv->mm_lock); - /* object should not be on active list: */ - GEM_WARN_ON(is_active(msm_obj)); - put_iova_spaces(obj, true); if (obj->import_attach) { diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 6fe521ccda45..420ba49bf21a 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -138,7 +138,6 @@ struct msm_gem_object { char name[32]; /* Identifier to print for the debugfs files */ - int active_count; int pin_count; }; #define to_msm_bo(x) container_of(x, struct msm_gem_object, base) @@ -171,8 +170,6 @@ void *msm_gem_get_vaddr_active(struct drm_gem_object *obj); void msm_gem_put_vaddr_locked(struct drm_gem_object *obj); void msm_gem_put_vaddr(struct drm_gem_object *obj); int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv); -void msm_gem_active_get(struct drm_gem_object *obj, struct msm_gpu *gpu); -void msm_gem_active_put(struct drm_gem_object *obj); bool msm_gem_active(struct drm_gem_object *obj); int msm_gem_cpu_prep(struct drm_gem_object *obj, uint32_t op, ktime_t *timeout); int msm_gem_cpu_fini(struct drm_gem_object *obj); @@ -245,12 +242,6 @@ msm_gem_is_locked(struct drm_gem_object *obj) return dma_resv_is_locked(obj->resv) || (kref_read(&obj->refcount) == 0); } -static inline bool is_active(struct msm_gem_object *msm_obj) -{ - GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - return msm_obj->active_count; -} - /* imported/exported objects are not purgeable: */ static inline bool is_unpurgeable(struct msm_gem_object *msm_obj) { @@ -391,9 +382,8 @@ struct msm_gem_submit { /* make sure these don't conflict w/ MSM_SUBMIT_BO_x */ #define BO_VALID 0x8000 /* is current addr in cmdstream correct/valid? */ #define BO_LOCKED 0x4000 /* obj lock is held */ -#define BO_ACTIVE 0x2000 /* active refcnt is held */ -#define BO_OBJ_PINNED 0x1000 /* obj (pages) is pinned and on active list */ -#define BO_VMA_PINNED 0x0800 /* vma (virtual address) is pinned */ +#define BO_OBJ_PINNED 0x2000 /* obj (pages) is pinned and on active list */ +#define BO_VMA_PINNED 0x1000 /* vma (virtual address) is pinned */ uint32_t flags; union { struct msm_gem_object *obj; diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index 16c662808522..adf358fb8e9d 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -243,17 +243,13 @@ static void submit_cleanup_bo(struct msm_gem_submit *submit, int i, if (flags & BO_OBJ_PINNED) msm_gem_unpin_locked(obj); - if (flags & BO_ACTIVE) - msm_gem_active_put(obj); - if (flags & BO_LOCKED) dma_resv_unlock(obj->resv); } static void submit_unlock_unpin_bo(struct msm_gem_submit *submit, int i) { - unsigned cleanup_flags = BO_VMA_PINNED | BO_OBJ_PINNED | - BO_ACTIVE | BO_LOCKED; + unsigned cleanup_flags = BO_VMA_PINNED | BO_OBJ_PINNED | BO_LOCKED; submit_cleanup_bo(submit, i, cleanup_flags); if (!(submit->bos[i].flags & BO_VALID)) @@ -358,18 +354,6 @@ static int submit_pin_objects(struct msm_gem_submit *submit) submit->valid = true; - /* - * Increment active_count first, so if under memory pressure, we - * don't inadvertently evict a bo needed by the submit in order - * to pin an earlier bo in the same submit. - */ - for (i = 0; i < submit->nr_bos; i++) { - struct drm_gem_object *obj = &submit->bos[i].obj->base; - - msm_gem_active_get(obj, submit->gpu); - submit->bos[i].flags |= BO_ACTIVE; - } - for (i = 0; i < submit->nr_bos; i++) { struct drm_gem_object *obj = &submit->bos[i].obj->base; struct msm_gem_vma *vma; @@ -521,7 +505,7 @@ static void submit_cleanup(struct msm_gem_submit *submit, bool error) unsigned i; if (error) - cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED | BO_ACTIVE; + cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED; for (i = 0; i < submit->nr_bos; i++) { struct msm_gem_object *msm_obj = submit->bos[i].obj; @@ -540,7 +524,7 @@ void msm_submit_retire(struct msm_gem_submit *submit) msm_gem_lock(obj); /* Note, VMA already fence-unpinned before submit: */ - submit_cleanup_bo(submit, i, BO_OBJ_PINNED | BO_ACTIVE); + submit_cleanup_bo(submit, i, BO_OBJ_PINNED); msm_gem_unlock(obj); drm_gem_object_put(obj); } From patchwork Tue Aug 2 15:51:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595383 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D07EC00140 for ; Tue, 2 Aug 2022 15:52:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237550AbiHBPwM (ORCPT ); Tue, 2 Aug 2022 11:52:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54744 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237467AbiHBPvx (ORCPT ); Tue, 2 Aug 2022 11:51:53 -0400 Received: from mail-pj1-x102d.google.com (mail-pj1-x102d.google.com [IPv6:2607:f8b0:4864:20::102d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0DAB83CBE7; Tue, 2 Aug 2022 08:51:43 -0700 (PDT) Received: by mail-pj1-x102d.google.com with SMTP id q7-20020a17090a7a8700b001f300db8677so15753988pjf.5; Tue, 02 Aug 2022 08:51:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=Le752qOQHMV5YKNomZZhvKAccL9F4fI/bs+OgheaM0E=; b=ZNmkoFJQCUZt/2K96EzfkPzt9hq2E2EaSqg5Iw2fag3s0eYQQMl++JyuKjbMEiSDF5 nIHWuSk1zkzgQmlVeTromJairAKWW1CiLdtYFo9LSsi66uYoD9Oiq35ycvhOYrqdiNn4 US2PAzSSj3mjIrO585M5l5pkEIrd9DEiewZ8hn7SxBh0DJueyu2+w4rPwRRH1COp1AGC yl2nX3cVFnmYta09Sdp0KeoCr7G/C+6cVMY5Z4/R+ljdwmRGQfisKAuLJm6sORDzTwW1 67G/r6ywbgOz7sv1sso5TdI6BJaqhfdfozKHr7O4cVXyLoNwRr7KQ6U99AOBJQnL1Mu6 C3Ag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=Le752qOQHMV5YKNomZZhvKAccL9F4fI/bs+OgheaM0E=; b=1LHnYP+bBEMF9tEJClnTmKZmAxzEUoAObQSDom8Vb3w6HRadBcnmHexVvszyhcE4en QapUEFVNzI4cCNoFQ0v1u5nE8oHWB6d/tQWLvp3EGawjPIF55K71Qb+hDn8Z3UFrvjjH kHQhmqAg6PjWjcT8K7kK0OswAQMGnKdq7yva31hH2LZTPlsEAPUleUL6BMauDWAWq4Id y5mWwohEd8kD3VLc++Ix2T9C7ynRYJI6JJQ5xIr8OoYzxqh38qjiGKTdk7oAtdRPIxzY hUfBhhWjmJbnGQj1BZCdIHdE78MG2/WEdn6lSitn7p/G0ai4oKNkwJi79quR9hxGDNUZ bS6g== X-Gm-Message-State: ACgBeo0Ywxx5B3K4p4BjR45KTuWVEpj8t771bR2j+6pGWel5kk92LkjC 19f7IlfqBKudyFDXaJ7nZdMAVdEOc3w= X-Google-Smtp-Source: AA6agR5ifOOl6novGltLYp/+Nqhh+R7D/QWeM2OiTe0tVonfWJscREiz+fHcPxqwBLzSs5IbjVsN8g== X-Received: by 2002:a17:90b:4d8e:b0:1f3:160e:1a5c with SMTP id oj14-20020a17090b4d8e00b001f3160e1a5cmr155031pjb.10.1659455502353; Tue, 02 Aug 2022 08:51:42 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id a15-20020a170902710f00b0016d9468bb20sm11920276pll.129.2022.08.02.08.51.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:41 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Daniel Vetter , Thomas Zimmermann , Dmitry Osipenko , Maarten Lankhorst , Maxime Ripard , David Airlie , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 09/15] drm/gem: Add LRU/shrinker helper Date: Tue, 2 Aug 2022 08:51:42 -0700 Message-Id: <20220802155152.1727594-10-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Add a simple LRU helper to assist with driver's shrinker implementation. It handles tracking the number of backing pages associated with a given LRU, and provides a helper to implement shrinker_scan. A driver can use multiple LRU instances to track objects in various states, for example a dontneed LRU for purgeable objects, a willneed LRU for evictable objects, and an unpinned LRU for objects without backing pages. All LRUs that the object can be moved between must share a single lock. v2: lockdep_assert_held() instead of WARN_ON(!mutex_is_locked()) v3: make drm_gem_lru_move_tail_locked() static until there is a user Cc: Daniel Vetter Cc: Thomas Zimmermann Cc: Dmitry Osipenko Signed-off-by: Rob Clark Reviewed-by: Dmitry Osipenko --- drivers/gpu/drm/drm_gem.c | 170 ++++++++++++++++++++++++++++++++++++++ include/drm/drm_gem.h | 55 ++++++++++++ 2 files changed, 225 insertions(+) diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c index eb0c2d041f13..556714c10472 100644 --- a/drivers/gpu/drm/drm_gem.c +++ b/drivers/gpu/drm/drm_gem.c @@ -165,6 +165,7 @@ void drm_gem_private_object_init(struct drm_device *dev, obj->resv = &obj->_resv; drm_vma_node_reset(&obj->vma_node); + INIT_LIST_HEAD(&obj->lru_node); } EXPORT_SYMBOL(drm_gem_private_object_init); @@ -951,6 +952,7 @@ drm_gem_object_release(struct drm_gem_object *obj) dma_resv_fini(&obj->_resv); drm_gem_free_mmap_offset(obj); + drm_gem_lru_remove(obj); } EXPORT_SYMBOL(drm_gem_object_release); @@ -1274,3 +1276,171 @@ drm_gem_unlock_reservations(struct drm_gem_object **objs, int count, ww_acquire_fini(acquire_ctx); } EXPORT_SYMBOL(drm_gem_unlock_reservations); + +/** + * drm_gem_lru_init - initialize a LRU + * + * @lru: The LRU to initialize + * @lock: The lock protecting the LRU + */ +void +drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock) +{ + lru->lock = lock; + lru->count = 0; + INIT_LIST_HEAD(&lru->list); +} +EXPORT_SYMBOL(drm_gem_lru_init); + +static void +drm_gem_lru_remove_locked(struct drm_gem_object *obj) +{ + obj->lru->count -= obj->size >> PAGE_SHIFT; + WARN_ON(obj->lru->count < 0); + list_del(&obj->lru_node); + obj->lru = NULL; +} + +/** + * drm_gem_lru_remove - remove object from whatever LRU it is in + * + * If the object is currently in any LRU, remove it. + * + * @obj: The GEM object to remove from current LRU + */ +void +drm_gem_lru_remove(struct drm_gem_object *obj) +{ + struct drm_gem_lru *lru = obj->lru; + + if (!lru) + return; + + mutex_lock(lru->lock); + drm_gem_lru_remove_locked(obj); + mutex_unlock(lru->lock); +} +EXPORT_SYMBOL(drm_gem_lru_remove); + +static void +drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj) +{ + lockdep_assert_held_once(lru->lock); + + if (obj->lru) + drm_gem_lru_remove_locked(obj); + + lru->count += obj->size >> PAGE_SHIFT; + list_add_tail(&obj->lru_node, &lru->list); + obj->lru = lru; +} + +/** + * drm_gem_lru_move_tail - move the object to the tail of the LRU + * + * If the object is already in this LRU it will be moved to the + * tail. Otherwise it will be removed from whichever other LRU + * it is in (if any) and moved into this LRU. + * + * @lru: The LRU to move the object into. + * @obj: The GEM object to move into this LRU + */ +void +drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj) +{ + mutex_lock(lru->lock); + drm_gem_lru_move_tail_locked(lru, obj); + mutex_unlock(lru->lock); +} +EXPORT_SYMBOL(drm_gem_lru_move_tail); + +/** + * drm_gem_lru_scan - helper to implement shrinker.scan_objects + * + * If the shrink callback succeeds, it is expected that the driver + * move the object out of this LRU. + * + * If the LRU possibly contain active buffers, it is the responsibility + * of the shrink callback to check for this (ie. dma_resv_test_signaled()) + * or if necessary block until the buffer becomes idle. + * + * @lru: The LRU to scan + * @nr_to_scan: The number of pages to try to reclaim + * @shrink: Callback to try to shrink/reclaim the object. + */ +unsigned long +drm_gem_lru_scan(struct drm_gem_lru *lru, unsigned nr_to_scan, + bool (*shrink)(struct drm_gem_object *obj)) +{ + struct drm_gem_lru still_in_lru; + struct drm_gem_object *obj; + unsigned freed = 0; + + drm_gem_lru_init(&still_in_lru, lru->lock); + + mutex_lock(lru->lock); + + while (freed < nr_to_scan) { + obj = list_first_entry_or_null(&lru->list, typeof(*obj), lru_node); + + if (!obj) + break; + + drm_gem_lru_move_tail_locked(&still_in_lru, obj); + + /* + * If it's in the process of being freed, gem_object->free() + * may be blocked on lock waiting to remove it. So just + * skip it. + */ + if (!kref_get_unless_zero(&obj->refcount)) + continue; + + /* + * Now that we own a reference, we can drop the lock for the + * rest of the loop body, to reduce contention with other + * code paths that need the LRU lock + */ + mutex_unlock(lru->lock); + + /* + * Note that this still needs to be trylock, since we can + * hit shrinker in response to trying to get backing pages + * for this obj (ie. while it's lock is already held) + */ + if (!dma_resv_trylock(obj->resv)) + goto tail; + + if (shrink(obj)) { + freed += obj->size >> PAGE_SHIFT; + + /* + * If we succeeded in releasing the object's backing + * pages, we expect the driver to have moved the object + * out of this LRU + */ + WARN_ON(obj->lru == &still_in_lru); + WARN_ON(obj->lru == lru); + } + + dma_resv_unlock(obj->resv); + +tail: + drm_gem_object_put(obj); + mutex_lock(lru->lock); + } + + /* + * Move objects we've skipped over out of the temporary still_in_lru + * back into this LRU + */ + list_for_each_entry (obj, &still_in_lru.list, lru_node) + obj->lru = lru; + list_splice_tail(&still_in_lru.list, &lru->list); + lru->count += still_in_lru.count; + + mutex_unlock(lru->lock); + + return freed; +} +EXPORT_SYMBOL(drm_gem_lru_scan); diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h index 87cffc9efa85..f28a48a6f846 100644 --- a/include/drm/drm_gem.h +++ b/include/drm/drm_gem.h @@ -174,6 +174,41 @@ struct drm_gem_object_funcs { const struct vm_operations_struct *vm_ops; }; +/** + * struct drm_gem_lru - A simple LRU helper + * + * A helper for tracking GEM objects in a given state, to aid in + * driver's shrinker implementation. Tracks the count of pages + * for lockless &shrinker.count_objects, and provides + * &drm_gem_lru_scan for driver's &shrinker.scan_objects + * implementation. + */ +struct drm_gem_lru { + /** + * @lock: + * + * Lock protecting movement of GEM objects between LRUs. All + * LRUs that the object can move between should be protected + * by the same lock. + */ + struct mutex *lock; + + /** + * @count: + * + * The total number of backing pages of the GEM objects in + * this LRU. + */ + long count; + + /** + * @list: + * + * The LRU list. + */ + struct list_head list; +}; + /** * struct drm_gem_object - GEM buffer object * @@ -312,6 +347,20 @@ struct drm_gem_object { * */ const struct drm_gem_object_funcs *funcs; + + /** + * @lru_node: + * + * List node in a &drm_gem_lru. + */ + struct list_head lru_node; + + /** + * @lru: + * + * The current LRU list that the GEM object is on. + */ + struct drm_gem_lru *lru; }; /** @@ -420,4 +469,10 @@ void drm_gem_unlock_reservations(struct drm_gem_object **objs, int count, int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev, u32 handle, u64 *offset); +void drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock); +void drm_gem_lru_remove(struct drm_gem_object *obj); +void drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj); +unsigned long drm_gem_lru_scan(struct drm_gem_lru *lru, unsigned nr_to_scan, + bool (*shrink)(struct drm_gem_object *obj)); + #endif /* __DRM_GEM_H__ */ From patchwork Tue Aug 2 15:51:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595382 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id EBA54C00140 for ; Tue, 2 Aug 2022 15:52:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237501AbiHBPwY (ORCPT ); Tue, 2 Aug 2022 11:52:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237425AbiHBPv4 (ORCPT ); Tue, 2 Aug 2022 11:51:56 -0400 Received: from mail-pf1-x42e.google.com (mail-pf1-x42e.google.com [IPv6:2607:f8b0:4864:20::42e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90C4F3ED54; Tue, 2 Aug 2022 08:51:46 -0700 (PDT) Received: by mail-pf1-x42e.google.com with SMTP id q19so4594303pfg.8; Tue, 02 Aug 2022 08:51:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=zh/8/t0ZilHENXdeKCjYIuWBV3SQcdEuA5JHSYImukA=; b=L/CS5pN5XNZBpLJnvGQXmfnlryEskssDywWQufmmiQKnmQGK8cVwYr8/zwk06cM1bR 0CenCAwtl+w+USYLrxgYbblO1a2kJ/4DldIQvFGSd8bh2vBqQ/0stPD1zEHCGpu3JoEB zbnTe1RbGrezCMCNPgfdTe93VRajXWpvKWlAYd7V8gJ9xj0LLrGXaNKvixJH/tstB6YM hKGwyugtLdkkGPfGidAMHfX08rDAdyBjt2q/3Afgh+Z4+1ys7PiG/T7bphkBcdI0Z/fu qaUw6Y6BXGCJEyRIGeVC8GKl9iLEZqPJSpi5LGDOEHpkiYy+GB1A0grW//NTitRCr130 bY5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=zh/8/t0ZilHENXdeKCjYIuWBV3SQcdEuA5JHSYImukA=; b=so2Qvfx3DbcliSCvSgblAsN0XBqie9GLXFvJgBlfKG6AXJI1suWJ6HTF1A5A+fbqAq KgdXySsE6oeH2NsOkNbAj5XYm6Yf02sK8kBojOcAODQCHdS4HsITSGPjm0I4/X1gXwsP 3Cba8wxl5Fng1JXQOKbvFGuu1CKmhjFT6jd1Q0O6yB5cqU6nabuhFQsz7hN/cQyc3Omt TA7t+aSrRsMkaj10/KRWg7G02NdNtP2Jmu6Sw6hNpukAKYxgJStwUv/5X0cO/1gZIjtz iKaz8YGa7WfARGvFxjZSzTp4H/SiSaz+sK5HrObQi8xG33mbJITs32Zi7n97lPJJ4OzB BPqw== X-Gm-Message-State: AJIora/LxCk08ElWoDlMkRESVo+0uxOnYIGgpXr6ke4JDzYKL9gJOYX0 nHNwPXcexok69sKVAAkiNsk= X-Google-Smtp-Source: AGRyM1taB+ltecK2c7Xx37oHN+/8/NhyTUy+Kr/piIHySML88tJU6UOgJR/+cxCQDZw37VEAqhxFbA== X-Received: by 2002:a05:6a00:851:b0:52c:8e02:c95d with SMTP id q17-20020a056a00085100b0052c8e02c95dmr21210160pfk.31.1659455505239; Tue, 02 Aug 2022 08:51:45 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id u62-20020a626041000000b0052b94e757ecsm10907003pfb.213.2022.08.02.08.51.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:44 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 10/15] drm/msm/gem: Convert to using drm_gem_lru Date: Tue, 2 Aug 2022 08:51:43 -0700 Message-Id: <20220802155152.1727594-11-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark This converts over to use the shared GEM LRU/shrinker helpers. Note that it means we are no longer tracking purgeable or willneed buffers that are active separately. But the most recently pinned buffers should be at the tail of the various LRUs, and the shrinker is already prepared to encounter objects which are still active. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_drv.c | 14 +-- drivers/gpu/drm/msm/msm_drv.h | 70 +++++++++++---- drivers/gpu/drm/msm/msm_gem.c | 58 ++++-------- drivers/gpu/drm/msm/msm_gem.h | 93 -------------------- drivers/gpu/drm/msm/msm_gem_shrinker.c | 117 ++++++------------------- drivers/gpu/drm/msm/msm_gpu.c | 3 - drivers/gpu/drm/msm/msm_gpu.h | 6 -- 7 files changed, 104 insertions(+), 257 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index d7ca025457b6..1ca4a92ba96e 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -418,14 +418,18 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv) INIT_LIST_HEAD(&priv->objects); mutex_init(&priv->obj_lock); - INIT_LIST_HEAD(&priv->inactive_willneed); - INIT_LIST_HEAD(&priv->inactive_dontneed); - INIT_LIST_HEAD(&priv->inactive_unpinned); - mutex_init(&priv->mm_lock); + /* + * Initialize the LRUs: + */ + mutex_init(&priv->lru.lock); + drm_gem_lru_init(&priv->lru.unbacked, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.pinned, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.willneed, &priv->lru.lock); + drm_gem_lru_init(&priv->lru.dontneed, &priv->lru.lock); /* Teach lockdep about lock ordering wrt. shrinker: */ fs_reclaim_acquire(GFP_KERNEL); - might_lock(&priv->mm_lock); + might_lock(&priv->lru.lock); fs_reclaim_release(GFP_KERNEL); drm_mode_config_init(ddev); diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index b3689a2d27d7..208ae5bc5e6b 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -142,28 +142,60 @@ struct msm_drm_private { struct mutex obj_lock; /** - * LRUs of inactive GEM objects. Every bo is either in one of the - * inactive lists (depending on whether or not it is shrinkable) or - * gpu->active_list (for the gpu it is active on[1]), or transiently - * on a temporary list as the shrinker is running. + * lru: * - * Note that inactive_willneed also contains pinned and vmap'd bos, - * but the number of pinned-but-not-active objects is small (scanout - * buffers, ringbuffer, etc). + * The various LRU's that a GEM object is in at various stages of + * it's lifetime. Objects start out in the unbacked LRU. When + * pinned (for scannout or permanently mapped GPU buffers, like + * ringbuffer, memptr, fw, etc) it moves to the pinned LRU. When + * unpinned, it moves into willneed or dontneed LRU depending on + * madvise state. When backing pages are evicted (willneed) or + * purged (dontneed) it moves back into the unbacked LRU. * - * These lists are protected by mm_lock (which should be acquired - * before per GEM object lock). One should *not* hold mm_lock in - * get_pages()/vmap()/etc paths, as they can trigger the shrinker. - * - * [1] if someone ever added support for the old 2d cores, there could be - * more than one gpu object + * The dontneed LRU is considered by the shrinker for objects + * that are candidate for purging, and the willneed LRU is + * considered for objects that could be evicted. */ - struct list_head inactive_willneed; /* inactive + potentially unpin/evictable */ - struct list_head inactive_dontneed; /* inactive + shrinkable */ - struct list_head inactive_unpinned; /* inactive + purged or unpinned */ - long shrinkable_count; /* write access under mm_lock */ - long evictable_count; /* write access under mm_lock */ - struct mutex mm_lock; + struct { + /** + * unbacked: + * + * The LRU for GEM objects without backing pages allocated. + * This mostly exists so that objects are always is one + * LRU. + */ + struct drm_gem_lru unbacked; + + /** + * pinned: + * + * The LRU for pinned GEM objects + */ + struct drm_gem_lru pinned; + + /** + * willneed: + * + * The LRU for unpinned GEM objects which are in madvise + * WILLNEED state (ie. can be evicted) + */ + struct drm_gem_lru willneed; + + /** + * dontneed: + * + * The LRU for unpinned GEM objects which are in madvise + * DONTNEED state (ie. can be purged) + */ + struct drm_gem_lru dontneed; + + /** + * lock: + * + * Protects manipulation of all of the LRUs. + */ + struct mutex lock; + } lru; struct workqueue_struct *wq; diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index 209438744bab..d4e8af46f4ef 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -174,6 +174,7 @@ static void put_pages(struct drm_gem_object *obj) put_pages_vram(obj); msm_obj->pages = NULL; + update_lru(obj); } } @@ -210,8 +211,6 @@ struct page **msm_gem_pin_pages(struct drm_gem_object *obj) void msm_gem_unpin_pages(struct drm_gem_object *obj) { - struct msm_gem_object *msm_obj = to_msm_bo(obj); - msm_gem_lock(obj); msm_gem_unpin_locked(obj); msm_gem_unlock(obj); @@ -761,7 +760,6 @@ void msm_gem_purge(struct drm_gem_object *obj) put_iova_vmas(obj); msm_obj->madv = __MSM_MADV_PURGED; - update_lru(obj); drm_gem_free_mmap_offset(obj); @@ -786,7 +784,6 @@ void msm_gem_evict(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(obj)); GEM_WARN_ON(is_unevictable(msm_obj)); - GEM_WARN_ON(!msm_obj->evictable); /* Get rid of any iommu mapping(s): */ put_iova_spaces(obj, false); @@ -794,8 +791,6 @@ void msm_gem_evict(struct drm_gem_object *obj) drm_vma_node_unmap(&obj->vma_node, dev->anon_inode->i_mapping); put_pages(obj); - - update_lru(obj); } void msm_gem_vunmap(struct drm_gem_object *obj) @@ -818,26 +813,20 @@ static void update_lru(struct drm_gem_object *obj) GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); - mutex_lock(&priv->mm_lock); - - if (msm_obj->dontneed) - mark_unpurgeable(msm_obj); - if (msm_obj->evictable) - mark_unevictable(msm_obj); - - list_del(&msm_obj->mm_list); - if ((msm_obj->madv == MSM_MADV_WILLNEED) && msm_obj->sgt) { - list_add_tail(&msm_obj->mm_list, &priv->inactive_willneed); - mark_evictable(msm_obj); - } else if (msm_obj->madv == MSM_MADV_DONTNEED) { - list_add_tail(&msm_obj->mm_list, &priv->inactive_dontneed); - mark_purgeable(msm_obj); + if (!msm_obj->pages) { + GEM_WARN_ON(msm_obj->pin_count); + GEM_WARN_ON(msm_obj->vmap_count); + + drm_gem_lru_move_tail(&priv->lru.unbacked, obj); + } else if (msm_obj->pin_count || msm_obj->vmap_count) { + drm_gem_lru_move_tail(&priv->lru.pinned, obj); + } else if (msm_obj->madv == MSM_MADV_WILLNEED) { + drm_gem_lru_move_tail(&priv->lru.willneed, obj); } else { - GEM_WARN_ON((msm_obj->madv != __MSM_MADV_PURGED) && msm_obj->sgt); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - } + GEM_WARN_ON(msm_obj->madv != MSM_MADV_DONTNEED); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.dontneed, obj); + } } bool msm_gem_active(struct drm_gem_object *obj) @@ -995,12 +984,6 @@ static void msm_gem_free_object(struct drm_gem_object *obj) list_del(&msm_obj->node); mutex_unlock(&priv->obj_lock); - mutex_lock(&priv->mm_lock); - if (msm_obj->dontneed) - mark_unpurgeable(msm_obj); - list_del(&msm_obj->mm_list); - mutex_unlock(&priv->mm_lock); - put_iova_spaces(obj, true); if (obj->import_attach) { @@ -1160,13 +1143,6 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32 to_msm_bo(obj)->vram_node = &vma->node; - /* Call chain get_pages() -> update_inactive() tries to - * access msm_obj->mm_list, but it is not initialized yet. - * To avoid NULL pointer dereference error, initialize - * mm_list to be empty. - */ - INIT_LIST_HEAD(&msm_obj->mm_list); - msm_gem_lock(obj); pages = get_pages(obj); msm_gem_unlock(obj); @@ -1189,9 +1165,7 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32 mapping_set_gfp_mask(obj->filp->f_mapping, GFP_HIGHUSER); } - mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.unbacked, obj); mutex_lock(&priv->obj_lock); list_add_tail(&msm_obj->node, &priv->objects); @@ -1247,9 +1221,7 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev, msm_gem_unlock(obj); - mutex_lock(&priv->mm_lock); - list_add_tail(&msm_obj->mm_list, &priv->inactive_unpinned); - mutex_unlock(&priv->mm_lock); + drm_gem_lru_move_tail(&priv->lru.pinned, obj); mutex_lock(&priv->obj_lock); list_add_tail(&msm_obj->node, &priv->objects); diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 420ba49bf21a..0403b27ff779 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -93,16 +93,6 @@ struct msm_gem_object { */ uint8_t madv; - /** - * Is object on inactive_dontneed list (ie. counted in priv->shrinkable_count)? - */ - bool dontneed : 1; - - /** - * Is object evictable (ie. counted in priv->evictable_count)? - */ - bool evictable : 1; - /** * count of active vmap'ing */ @@ -114,17 +104,6 @@ struct msm_gem_object { */ struct list_head node; - /** - * An object is either: - * inactive - on priv->inactive_dontneed or priv->inactive_willneed - * (depending on purgeability status) - * active - on one one of the gpu's active_list.. well, at - * least for now we don't have (I don't think) hw sync between - * 2d and 3d one devices which have both, meaning we need to - * block on submit if a bo is already on other ring - */ - struct list_head mm_list; - struct page **pages; struct sg_table *sgt; void *vaddr; @@ -206,12 +185,6 @@ msm_gem_lock(struct drm_gem_object *obj) dma_resv_lock(obj->resv, NULL); } -static inline bool __must_check -msm_gem_trylock(struct drm_gem_object *obj) -{ - return dma_resv_trylock(obj->resv); -} - static inline int msm_gem_lock_interruptible(struct drm_gem_object *obj) { @@ -260,77 +233,11 @@ static inline bool is_vunmapable(struct msm_gem_object *msm_obj) return (msm_obj->vmap_count == 0) && msm_obj->vaddr; } -static inline void mark_purgeable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unpurgeable(msm_obj)) - return; - - if (GEM_WARN_ON(msm_obj->dontneed)) - return; - - priv->shrinkable_count += msm_obj->base.size >> PAGE_SHIFT; - msm_obj->dontneed = true; -} - -static inline void mark_unpurgeable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - GEM_WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unpurgeable(msm_obj)) - return; - - if (GEM_WARN_ON(!msm_obj->dontneed)) - return; - - priv->shrinkable_count -= msm_obj->base.size >> PAGE_SHIFT; - GEM_WARN_ON(priv->shrinkable_count < 0); - msm_obj->dontneed = false; -} - static inline bool is_unevictable(struct msm_gem_object *msm_obj) { return is_unpurgeable(msm_obj) || msm_obj->vaddr; } -static inline void mark_evictable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unevictable(msm_obj)) - return; - - if (WARN_ON(msm_obj->evictable)) - return; - - priv->evictable_count += msm_obj->base.size >> PAGE_SHIFT; - msm_obj->evictable = true; -} - -static inline void mark_unevictable(struct msm_gem_object *msm_obj) -{ - struct msm_drm_private *priv = msm_obj->base.dev->dev_private; - - WARN_ON(!mutex_is_locked(&priv->mm_lock)); - - if (is_unevictable(msm_obj)) - return; - - if (WARN_ON(!msm_obj->evictable)) - return; - - priv->evictable_count -= msm_obj->base.size >> PAGE_SHIFT; - WARN_ON(priv->evictable_count < 0); - msm_obj->evictable = false; -} - void msm_gem_purge(struct drm_gem_object *obj); void msm_gem_evict(struct drm_gem_object *obj); void msm_gem_vunmap(struct drm_gem_object *obj); diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index ea8ed74982c1..530b1102b46d 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -29,121 +29,61 @@ msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); - unsigned count = priv->shrinkable_count; + unsigned count = priv->lru.dontneed.count; if (can_swap()) - count += priv->evictable_count; + count += priv->lru.willneed.count; return count; } static bool -purge(struct msm_gem_object *msm_obj) +purge(struct drm_gem_object *obj) { - if (!is_purgeable(msm_obj)) + if (!is_purgeable(to_msm_bo(obj))) return false; - if (msm_gem_active(&msm_obj->base)) + if (msm_gem_active(obj)) return false; - /* - * This will move the obj out of still_in_list to - * the purged list - */ - msm_gem_purge(&msm_obj->base); + msm_gem_purge(obj); return true; } static bool -evict(struct msm_gem_object *msm_obj) +evict(struct drm_gem_object *obj) { - if (is_unevictable(msm_obj)) + if (is_unevictable(to_msm_bo(obj))) return false; - if (msm_gem_active(&msm_obj->base)) + if (msm_gem_active(obj)) return false; - msm_gem_evict(&msm_obj->base); + msm_gem_evict(obj); return true; } -static unsigned long -scan(struct msm_drm_private *priv, unsigned nr_to_scan, struct list_head *list, - bool (*shrink)(struct msm_gem_object *msm_obj)) -{ - unsigned freed = 0; - struct list_head still_in_list; - - INIT_LIST_HEAD(&still_in_list); - - mutex_lock(&priv->mm_lock); - - while (freed < nr_to_scan) { - struct msm_gem_object *msm_obj = list_first_entry_or_null( - list, typeof(*msm_obj), mm_list); - - if (!msm_obj) - break; - - list_move_tail(&msm_obj->mm_list, &still_in_list); - - /* - * If it is in the process of being freed, msm_gem_free_object - * can be blocked on mm_lock waiting to remove it. So just - * skip it. - */ - if (!kref_get_unless_zero(&msm_obj->base.refcount)) - continue; - - /* - * Now that we own a reference, we can drop mm_lock for the - * rest of the loop body, to reduce contention with the - * retire_submit path (which could make more objects purgeable) - */ - - mutex_unlock(&priv->mm_lock); - - /* - * Note that this still needs to be trylock, since we can - * hit shrinker in response to trying to get backing pages - * for this obj (ie. while it's lock is already held) - */ - if (!msm_gem_trylock(&msm_obj->base)) - goto tail; - - if (shrink(msm_obj)) - freed += msm_obj->base.size >> PAGE_SHIFT; - - msm_gem_unlock(&msm_obj->base); - -tail: - drm_gem_object_put(&msm_obj->base); - mutex_lock(&priv->mm_lock); - } - - list_splice_tail(&still_in_list, list); - mutex_unlock(&priv->mm_lock); - - return freed; -} - static unsigned long msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); + long nr = sc->nr_to_scan; unsigned long freed; - freed = scan(priv, sc->nr_to_scan, &priv->inactive_dontneed, purge); + freed = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); + nr -= freed; if (freed > 0) trace_msm_gem_purge(freed << PAGE_SHIFT); - if (can_swap() && freed < sc->nr_to_scan) { - int evicted = scan(priv, sc->nr_to_scan - freed, - &priv->inactive_willneed, evict); + if (can_swap() && nr > 0) { + unsigned long evicted; + + evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); + nr -= evicted; if (evicted > 0) trace_msm_gem_evict(evicted << PAGE_SHIFT); @@ -179,12 +119,12 @@ msm_gem_shrinker_shrink(struct drm_device *dev, unsigned long nr_to_scan) static const int vmap_shrink_limit = 15; static bool -vmap_shrink(struct msm_gem_object *msm_obj) +vmap_shrink(struct drm_gem_object *obj) { - if (!is_vunmapable(msm_obj)) + if (!is_vunmapable(to_msm_bo(obj))) return false; - msm_gem_vunmap(&msm_obj->base); + msm_gem_vunmap(obj); return true; } @@ -194,17 +134,18 @@ msm_gem_shrinker_vmap(struct notifier_block *nb, unsigned long event, void *ptr) { struct msm_drm_private *priv = container_of(nb, struct msm_drm_private, vmap_notifier); - struct list_head *mm_lists[] = { - &priv->inactive_dontneed, - &priv->inactive_willneed, - priv->gpu ? &priv->gpu->active_list : NULL, + struct drm_gem_lru *lrus[] = { + &priv->lru.dontneed, + &priv->lru.willneed, + &priv->lru.pinned, NULL, }; unsigned idx, unmapped = 0; - for (idx = 0; mm_lists[idx] && unmapped < vmap_shrink_limit; idx++) { - unmapped += scan(priv, vmap_shrink_limit - unmapped, - mm_lists[idx], vmap_shrink); + for (idx = 0; lrus[idx] && unmapped < vmap_shrink_limit; idx++) { + unmapped += drm_gem_lru_scan(lrus[idx], + vmap_shrink_limit - unmapped, + vmap_shrink); } *(unsigned long *)ptr += unmapped; diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index c2bfcf3f1f40..fba85f894314 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -846,7 +846,6 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, sched_set_fifo_low(gpu->worker->task); - INIT_LIST_HEAD(&gpu->active_list); mutex_init(&gpu->active_lock); mutex_init(&gpu->lock); init_waitqueue_head(&gpu->retire_event); @@ -974,8 +973,6 @@ void msm_gpu_cleanup(struct msm_gpu *gpu) DBG("%s", gpu->name); - WARN_ON(!list_empty(&gpu->active_list)); - for (i = 0; i < ARRAY_SIZE(gpu->rb); i++) { msm_ringbuffer_destroy(gpu->rb[i]); gpu->rb[i] = NULL; diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 962d2070bcdf..a0885ff99e5b 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -187,12 +187,6 @@ struct msm_gpu { */ int cur_ctx_seqno; - /* - * List of GEM active objects on this gpu. Protected by - * msm_drm_private::mm_lock - */ - struct list_head active_list; - /** * lock: * From patchwork Tue Aug 2 15:51:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594899 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8A7ADC19F2C for ; Tue, 2 Aug 2022 15:52:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237566AbiHBPwY (ORCPT ); Tue, 2 Aug 2022 11:52:24 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55498 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237504AbiHBPv4 (ORCPT ); Tue, 2 Aug 2022 11:51:56 -0400 Received: from mail-pl1-x633.google.com (mail-pl1-x633.google.com [IPv6:2607:f8b0:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 334373F337; Tue, 2 Aug 2022 08:51:48 -0700 (PDT) Received: by mail-pl1-x633.google.com with SMTP id p8so13801247plq.13; Tue, 02 Aug 2022 08:51:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=FzvQD1BdX2RVx+fHXlbnajcgEgfcaBeelT02bqIAfkE=; b=gyJP/xxOXSRqgltSY6MkFDlfxok3ZireuVXp4CDqtTjrutwuMplFbCh+3nG4pa0MYW O1i/IDxYXB5GWQEa2h/8BclQIu7fJFhdTb1UbBQuOqal0b/te/FXbS5DGyOENv8V2ZC3 e0FxaB4NpVSBOrK+1HbDBDDsV7CddlDJaVQEuSeWE19EVEK+HY5z9Gt5t+9MkDojAhJv NT80ariNQXjQpxcjVr2wjKnldhFOUsmLlGLtFKl7jsx+13EEMnneqEuAmigPPcXJvMPq DPZjy5mX6M7VCyL+DBo1Qdaq7sFVxWr1hTRW/zvw41YgBT/87nW7rBVrtjvsyRIuJxTN 4UjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=FzvQD1BdX2RVx+fHXlbnajcgEgfcaBeelT02bqIAfkE=; b=dMQV9S7JaSWLx08YYrJPoUsHueD93fdTFC0cqAIhrFvXAnNWV84px7DENR4JOOsd/O MafgHgl20L9FkGOaNRPrhfTR9GFmBQi4FvKaLFAOzZD2a/UB/wbYTxwn1fFf21WiQp8j CmlbxmZ0myoa9TYWMmv8Q2sTmM5SVqgiYiBedNuV9tNdnd7aoS6t9UBago7AxzE7BDDw 7acU6B6bGFL9fxYdf6ONl0Nf+gCKJ+Ndxvf+9suiQ7RzVV/NAnXekmw2Yjw6aFnyKjIR fMTNm0dlX3U8rcNESUK3FGZ339w3B/tIEeBwg2PNIFFtxajYvlmFamNO/VAGewqT+4o0 ZhXA== X-Gm-Message-State: ACgBeo0I5W8LzmOkrpnAvlphHCAeg2547BLqNtzusbONK4yoezuVIVSr JY58Trp5UOayVcbdsi3cCSU= X-Google-Smtp-Source: AA6agR5riFYgIxD5qt970Uy22d2sGuBlqC5eT/s6nbNG9HJYGy8kuF+I9Yeo95ehMIRdQhNhcsu4qA== X-Received: by 2002:a17:90a:1943:b0:1ef:8146:f32f with SMTP id 3-20020a17090a194300b001ef8146f32fmr137919pjh.112.1659455507557; Tue, 02 Aug 2022 08:51:47 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id i13-20020a17090332cd00b0016edd557412sm6301857plr.201.2022.08.02.08.51.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:46 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 11/15] drm/msm/gem: Unpin buffers earlier Date: Tue, 2 Aug 2022 08:51:44 -0700 Message-Id: <20220802155152.1727594-12-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark We've already attached the fences, so obj->resv (which shrinker checks) tells us whether they are still active. So we can unpin sooner, before we drop the queue lock. This also avoids the need to grab the obj lock in the retire path, avoiding potential for lock contention between submit and retire. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_submit.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c index adf358fb8e9d..5599d93ec0d2 100644 --- a/drivers/gpu/drm/msm/msm_gem_submit.c +++ b/drivers/gpu/drm/msm/msm_gem_submit.c @@ -501,11 +501,11 @@ static int submit_reloc(struct msm_gem_submit *submit, struct msm_gem_object *ob */ static void submit_cleanup(struct msm_gem_submit *submit, bool error) { - unsigned cleanup_flags = BO_LOCKED; + unsigned cleanup_flags = BO_LOCKED | BO_OBJ_PINNED; unsigned i; if (error) - cleanup_flags |= BO_VMA_PINNED | BO_OBJ_PINNED; + cleanup_flags |= BO_VMA_PINNED; for (i = 0; i < submit->nr_bos; i++) { struct msm_gem_object *msm_obj = submit->bos[i].obj; @@ -522,10 +522,6 @@ void msm_submit_retire(struct msm_gem_submit *submit) for (i = 0; i < submit->nr_bos; i++) { struct drm_gem_object *obj = &submit->bos[i].obj->base; - msm_gem_lock(obj); - /* Note, VMA already fence-unpinned before submit: */ - submit_cleanup_bo(submit, i, BO_OBJ_PINNED); - msm_gem_unlock(obj); drm_gem_object_put(obj); } } From patchwork Tue Aug 2 15:51:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594898 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 14282C3F6B0 for ; Tue, 2 Aug 2022 15:52:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237512AbiHBPw0 (ORCPT ); Tue, 2 Aug 2022 11:52:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230254AbiHBPv5 (ORCPT ); Tue, 2 Aug 2022 11:51:57 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5AD0A22296; Tue, 2 Aug 2022 08:51:50 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id t2-20020a17090a4e4200b001f21572f3a4so15807896pjl.0; Tue, 02 Aug 2022 08:51:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=CHnA8EA1zYWAltmKrCqgpnviThEUlN7Pg3SwIn6d3WM=; b=fP9YsqHrJUAD0NhV3Le0o57xbD7BFv8FZLg9ePwdwZdhWLvo+bJrPMJsHOs+QrwnZT kjQFGd2MJkmuTw4gjjBn+EG+ipUZv1rH6YMXiSG6DSbGXm1V2pddU1p77Iu9ob8eOJ2A mjs7THLbV2gMKVdVE50Jy//J8ZIB16y0tHOWMeIg3wu0ggLUJyMueKJ2V/r73qqyqkI5 jZW2KiLFaM+Ywt660i2TP2RqIHFCW8gCgBINNLd6FeUtm7jB+2NxHjKZXHadwQCSqNcD 3PviYFrtJDOsJx5hISLXBqPXBSX/NkBgpPq+lyYahHNrty9wp7NbAI25ttaigACtCYt2 sRpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=CHnA8EA1zYWAltmKrCqgpnviThEUlN7Pg3SwIn6d3WM=; b=sTtcNmDvfH+ZaZYZvkGih1ri6NH8cDfad3nE5bA8dhMr5sEWqwc3UAeRR6pkVCy4B/ Lrh2q9nKTLhJQjmHFTcVdcv9EEDGu+diqn9GNy2Ng7UavucphsCHu0VLH0ahtFOJR8y9 aZk7a7cDZdvtslC5AkGdhIBWzGkzAkeUl3oRKukMPacM2M52AaYK53rOS+P2yRXq8iyq uOa/x9SbgQA5KwAQAgzcQqv8yH15G5xmYQZroSCBPU8Nsan8EyOKiJseGEE/Z7DZPcn+ kh+yzKHYjsmVma0w8+tCDdUkr4ZjZzq5rkeJZuwzf0QggAvlu5yfH+9tANWuYDJUB8BR isag== X-Gm-Message-State: ACgBeo1xgNhQn0ylyJ9b0Nuo+iEeD04OMFpaaC5/+1hCAKREz6GxYtiW FPNS0Cgg7USoCTph7hf8r8k= X-Google-Smtp-Source: AA6agR4MoUqcOxvlaJDRZuPmLwFSY6Z3PzL0UzcR3mDRU/8Zo+wehtehFU9tTUKLFHqqEZIscAs4rQ== X-Received: by 2002:a17:90b:3149:b0:1f0:3539:e7ae with SMTP id ip9-20020a17090b314900b001f03539e7aemr147125pjb.80.1659455509941; Tue, 02 Aug 2022 08:51:49 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id c18-20020a631c12000000b0041c49af8156sm2577055pgc.6.2022.08.02.08.51.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:48 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 12/15] drm/msm/gem: Consolidate shrinker trace Date: Tue, 2 Aug 2022 08:51:45 -0700 Message-Id: <20220802155152.1727594-13-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Combine separate trace events for purge vs evict into one. When we add support for purging/evicting active buffers we'll just add more info into this one trace event, rather than adding a bunch more events. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_shrinker.c | 19 ++++++--------- drivers/gpu/drm/msm/msm_gpu_trace.h | 32 +++++++++++--------------- 2 files changed, 20 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 530b1102b46d..5cc05d669a08 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -71,25 +71,20 @@ msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); long nr = sc->nr_to_scan; - unsigned long freed; + unsigned long freed, purged, evicted = 0; - freed = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); - nr -= freed; - - if (freed > 0) - trace_msm_gem_purge(freed << PAGE_SHIFT); + purged = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); + nr -= purged; if (can_swap() && nr > 0) { - unsigned long evicted; - evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); nr -= evicted; + } - if (evicted > 0) - trace_msm_gem_evict(evicted << PAGE_SHIFT); + freed = purged + evicted; - freed += evicted; - } + if (freed) + trace_msm_gem_shrink(sc->nr_to_scan, purged, evicted); return (freed > 0) ? freed : SHRINK_STOP; } diff --git a/drivers/gpu/drm/msm/msm_gpu_trace.h b/drivers/gpu/drm/msm/msm_gpu_trace.h index ca0b08d7875b..8867fa0a0306 100644 --- a/drivers/gpu/drm/msm/msm_gpu_trace.h +++ b/drivers/gpu/drm/msm/msm_gpu_trace.h @@ -115,29 +115,23 @@ TRACE_EVENT(msm_gmu_freq_change, ); -TRACE_EVENT(msm_gem_purge, - TP_PROTO(u32 bytes), - TP_ARGS(bytes), +TRACE_EVENT(msm_gem_shrink, + TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted), + TP_ARGS(nr_to_scan, purged, evicted), TP_STRUCT__entry( - __field(u32, bytes) + __field(u32, nr_to_scan) + __field(u32, purged) + __field(u32, evicted) ), TP_fast_assign( - __entry->bytes = bytes; + __entry->nr_to_scan = nr_to_scan; + __entry->purged = purged; + __entry->evicted = evicted; ), - TP_printk("Purging %u bytes", __entry->bytes) -); - - -TRACE_EVENT(msm_gem_evict, - TP_PROTO(u32 bytes), - TP_ARGS(bytes), - TP_STRUCT__entry( - __field(u32, bytes) - ), - TP_fast_assign( - __entry->bytes = bytes; - ), - TP_printk("Evicting %u bytes", __entry->bytes) + TP_printk("nr_to_scan=%u pages, purged=%u pages, evicted=%u pages", + __entry->nr_to_scan, + __entry->purged, + __entry->evicted) ); From patchwork Tue Aug 2 15:51:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595381 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72D8FC19F2C for ; Tue, 2 Aug 2022 15:52:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237588AbiHBPwm (ORCPT ); Tue, 2 Aug 2022 11:52:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55242 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237529AbiHBPwJ (ORCPT ); Tue, 2 Aug 2022 11:52:09 -0400 Received: from mail-pg1-x536.google.com (mail-pg1-x536.google.com [IPv6:2607:f8b0:4864:20::536]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2C98F41994; Tue, 2 Aug 2022 08:51:53 -0700 (PDT) Received: by mail-pg1-x536.google.com with SMTP id 206so9189625pgb.0; Tue, 02 Aug 2022 08:51:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=kFWCPQ026Vm0ssc92OEIjelADHl+en2PNRs5WLmvYxM=; b=HGAytMNkJs+PiiKRsOJYl2t2RyafEOOCiFkupWytHV4W5pAS7xMTpuBmg9rUW0doOs /Wc2NE6YVKDMDzRyeQ79uCUX/Ekjc/fX/iBIEXUUPRBDiDwiWpT+kZjUGVQtnagRGr4i Y3K+C6zKq/Xc91RBy1xoRqInViCWUIHFsJDV5vhxPF8ERPmbBPiluEdbV3dlcovz6M1/ wBa3GqELCIJrSohEvvak1Vd+AbcXUYYdVnvpStDfltQ798w+ZWS7kVKG3KwpvAuDkEG6 Zvu8zd0fM72HoPm01yo9PzxO9cuOEyCasreaRaCJOLPielxARS/nQ8Jwovze9OUYhmqZ jBGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=kFWCPQ026Vm0ssc92OEIjelADHl+en2PNRs5WLmvYxM=; b=H+9m7Ri9OXZ6b3DmcjzSce5S3sNG2+VoVZMypsJkhP9liXADIjc5v6cBx3Y3Eo3yOj MQ9PUzgRvvxmKYZ7Ac8X+7rbCfSoEalerN0zNgvRzVZqFNAGpq3vJ9nlmAE7nstcBzsQ edi/eUVoEqX/hoajCKiZXtgCNUXC2/lhhVrJnn066Mc9x9ZYmp6JeUWJRNgWebV7BfrR szaEFCtw7FG20oQNxZnw8ngbrMU4E0YhRBT7rM3r8D/HSAP8TwDx/TpeWK7osKfgTdDY IBlU2IVXPmXWojMR6L2AgyrveRvu3Ypiui1aFJLZaBoOPhhVsZU0D9UEkFSJQSV/O1Dj Z2GA== X-Gm-Message-State: AJIora+EroxNBcxuOfZViYaoawy+/hefiGhNXEw5CqNens0mSoh0zGL7 W7/3P+2Uffxzrn5HCWoWWLE= X-Google-Smtp-Source: AA6agR4oR61wZ7xUWIXU+CtyFCFQdPK1uessy7b6wLzzRXgX42+hXqT1mT3T3t6+pmHAjUr/Z/oCgA== X-Received: by 2002:a05:6a00:8c8:b0:52c:887d:fa25 with SMTP id s8-20020a056a0008c800b0052c887dfa25mr21443316pfu.86.1659455512059; Tue, 02 Aug 2022 08:51:52 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id d9-20020a170902e14900b0016be9fa6807sm11739322pla.284.2022.08.02.08.51.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:51 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 13/15] drm/msm/gem: Evict active GEM objects when necessary Date: Tue, 2 Aug 2022 08:51:46 -0700 Message-Id: <20220802155152.1727594-14-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark If we are under enough memory pressure, we should stall waiting for active buffers to become idle in order to evict. v2: Check for __GFP_ATOMIC before blocking Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem_shrinker.c | 70 +++++++++++++++++++++----- drivers/gpu/drm/msm/msm_gpu_trace.h | 16 +++--- 2 files changed, 68 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem_shrinker.c b/drivers/gpu/drm/msm/msm_gem_shrinker.c index 5cc05d669a08..f31054d25314 100644 --- a/drivers/gpu/drm/msm/msm_gem_shrinker.c +++ b/drivers/gpu/drm/msm/msm_gem_shrinker.c @@ -24,6 +24,13 @@ static bool can_swap(void) return enable_eviction && get_nr_swap_pages() > 0; } +static bool can_block(struct shrink_control *sc) +{ + if (sc->gfp_mask & __GFP_ATOMIC) + return false; + return current_is_kswapd() || (sc->gfp_mask & __GFP_RECLAIM); +} + static unsigned long msm_gem_shrinker_count(struct shrinker *shrinker, struct shrink_control *sc) { @@ -65,26 +72,65 @@ evict(struct drm_gem_object *obj) return true; } +static bool +wait_for_idle(struct drm_gem_object *obj) +{ + enum dma_resv_usage usage = dma_resv_usage_rw(true); + return dma_resv_wait_timeout(obj->resv, usage, false, 1000) > 0; +} + +static bool +active_purge(struct drm_gem_object *obj) +{ + if (!wait_for_idle(obj)) + return false; + + return purge(obj); +} + +static bool +active_evict(struct drm_gem_object *obj) +{ + if (!wait_for_idle(obj)) + return false; + + return evict(obj); +} + static unsigned long msm_gem_shrinker_scan(struct shrinker *shrinker, struct shrink_control *sc) { struct msm_drm_private *priv = container_of(shrinker, struct msm_drm_private, shrinker); + struct { + struct drm_gem_lru *lru; + bool (*shrink)(struct drm_gem_object *obj); + bool cond; + unsigned long freed; + } stages[] = { + /* Stages of progressively more aggressive/expensive reclaim: */ + { &priv->lru.dontneed, purge, true }, + { &priv->lru.willneed, evict, can_swap() }, + { &priv->lru.dontneed, active_purge, can_block(sc) }, + { &priv->lru.willneed, active_evict, can_swap() && can_block(sc) }, + }; long nr = sc->nr_to_scan; - unsigned long freed, purged, evicted = 0; - - purged = drm_gem_lru_scan(&priv->lru.dontneed, nr, purge); - nr -= purged; - - if (can_swap() && nr > 0) { - evicted = drm_gem_lru_scan(&priv->lru.willneed, nr, evict); - nr -= evicted; + unsigned long freed = 0; + + for (unsigned i = 0; (nr > 0) && (i < ARRAY_SIZE(stages)); i++) { + if (!stages[i].cond) + continue; + stages[i].freed = + drm_gem_lru_scan(stages[i].lru, nr, stages[i].shrink); + nr -= stages[i].freed; + freed += stages[i].freed; } - freed = purged + evicted; - - if (freed) - trace_msm_gem_shrink(sc->nr_to_scan, purged, evicted); + if (freed) { + trace_msm_gem_shrink(sc->nr_to_scan, stages[0].freed, + stages[1].freed, stages[2].freed, + stages[3].freed); + } return (freed > 0) ? freed : SHRINK_STOP; } diff --git a/drivers/gpu/drm/msm/msm_gpu_trace.h b/drivers/gpu/drm/msm/msm_gpu_trace.h index 8867fa0a0306..ac40d857bc45 100644 --- a/drivers/gpu/drm/msm/msm_gpu_trace.h +++ b/drivers/gpu/drm/msm/msm_gpu_trace.h @@ -116,22 +116,26 @@ TRACE_EVENT(msm_gmu_freq_change, TRACE_EVENT(msm_gem_shrink, - TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted), - TP_ARGS(nr_to_scan, purged, evicted), + TP_PROTO(u32 nr_to_scan, u32 purged, u32 evicted, + u32 active_purged, u32 active_evicted), + TP_ARGS(nr_to_scan, purged, evicted, active_purged, active_evicted), TP_STRUCT__entry( __field(u32, nr_to_scan) __field(u32, purged) __field(u32, evicted) + __field(u32, active_purged) + __field(u32, active_evicted) ), TP_fast_assign( __entry->nr_to_scan = nr_to_scan; __entry->purged = purged; __entry->evicted = evicted; + __entry->active_purged = active_purged; + __entry->active_evicted = active_evicted; ), - TP_printk("nr_to_scan=%u pages, purged=%u pages, evicted=%u pages", - __entry->nr_to_scan, - __entry->purged, - __entry->evicted) + TP_printk("nr_to_scan=%u pg, purged=%u pg, evicted=%u pg, active_purged=%u pg, active_evicted=%u pg", + __entry->nr_to_scan, __entry->purged, __entry->evicted, + __entry->active_purged, __entry->active_evicted) ); From patchwork Tue Aug 2 15:51:47 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 594897 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9ADF9C19F2D for ; Tue, 2 Aug 2022 15:52:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237095AbiHBPwo (ORCPT ); Tue, 2 Aug 2022 11:52:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56032 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237539AbiHBPwK (ORCPT ); Tue, 2 Aug 2022 11:52:10 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B12A2CCBB; Tue, 2 Aug 2022 08:51:55 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id 130so6118664pfv.13; Tue, 02 Aug 2022 08:51:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=2lydIYqrd+W135f9r06NI6UhXQCS/e+8vxoGcpQKqzg=; b=loVsfXno3F4OkX6WtMi9HTFVA7zrZ11/TR0oHh25qr+zUexe9JPK6p8LM1oQ5W2kA3 6CyyTZ7Jebx2hDnXMe1RE5IaEbjXDEpfImPpM4m9/uFU2byrlRF0qoqzZmZvoXqkWxNv pLgmMR0mHXjyGznJZGD3NIQVN7AZEgjkqcsz8zdEqv0w0n0Ghc/s6ZMPxx+BcwuHu+OV 4G87t8SI2b+rauEavzuj62lQiunyBlDMOLNBFBGcsIMUKV6ZlOb7fzCeLSKW/7WH+qoD 51ku/1DM0zwSglyhFgCkkjNch1tdBtksG+axhSvdzS0f3FUtbV41Qf4ECv2RCGKkbC8x lijw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=2lydIYqrd+W135f9r06NI6UhXQCS/e+8vxoGcpQKqzg=; b=qz64tLEGqIPuujnz+9DUa3mhzDjl1APifrYZCyPvrzef/KSlYVhcaTxpymMWsGojTl nYL28eePma1dKCoMx/1z3b8w1BKdXKzQXzk6tGKTJN76qUHPa68FP0s2FFZNHVegTa3T WwnjNcfcYjhhRE8DQZKDyNdKejnTGUVTy4pHHi37sSDya6x9+/fFg7c72c7lpqT+46TO K3CVmcDX5ezqEZaJp8hNoeawXnX3856BBGIrAD1tPAErtsrk0Hiu9EVSTVAJV1CWF64G 5SY6joXrAiwaFlhZjNWlU+GMFXxySp2xwixSoouXI/7hBC4dzMW7BUL3HjsN7P1lq/Qh D72A== X-Gm-Message-State: AJIora+/KKPdZPPotoOOMoffksRKQf7+s2mAL+nQsENCx4sYkQ+sb+qg tDIb6m0HTCkr7dQx3r7i6Xg= X-Google-Smtp-Source: AGRyM1texTaTGh1BJJVTw7XWN8lXo9/2dAJgkRwDwtOI15rv31zNkE0tFjxshdmUZ9GLTkm3lCo1rQ== X-Received: by 2002:a05:6a00:1d26:b0:52b:f8ab:6265 with SMTP id a38-20020a056a001d2600b0052bf8ab6265mr21661696pfx.54.1659455514362; Tue, 02 Aug 2022 08:51:54 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id m4-20020a654384000000b0041c79a5f443sm961330pgp.9.2022.08.02.08.51.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:53 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 14/15] drm/msm/gem: Add msm_gem_assert_locked() Date: Tue, 2 Aug 2022 08:51:47 -0700 Message-Id: <20220802155152.1727594-15-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark All use of msm_gem_is_locked() is just for WARN_ON()s, so extract out into an msm_gem_assert_locked() patch. Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.c | 36 +++++++++++++++++------------------ drivers/gpu/drm/msm/msm_gem.h | 8 +++++++- 2 files changed, 25 insertions(+), 19 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c index d4e8af46f4ef..1dee0d18abbb 100644 --- a/drivers/gpu/drm/msm/msm_gem.c +++ b/drivers/gpu/drm/msm/msm_gem.c @@ -97,7 +97,7 @@ static struct page **get_pages(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (!msm_obj->pages) { struct drm_device *dev = obj->dev; @@ -183,7 +183,7 @@ static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj) struct msm_gem_object *msm_obj = to_msm_bo(obj); struct page **p; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) { return ERR_PTR(-EBUSY); @@ -278,7 +278,7 @@ static uint64_t mmap_offset(struct drm_gem_object *obj) struct drm_device *dev = obj->dev; int ret; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); /* Make it mmapable */ ret = drm_gem_create_mmap_offset(obj); @@ -307,7 +307,7 @@ static struct msm_gem_vma *add_vma(struct drm_gem_object *obj, struct msm_gem_object *msm_obj = to_msm_bo(obj); struct msm_gem_vma *vma; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); vma = kzalloc(sizeof(*vma), GFP_KERNEL); if (!vma) @@ -326,7 +326,7 @@ static struct msm_gem_vma *lookup_vma(struct drm_gem_object *obj, struct msm_gem_object *msm_obj = to_msm_bo(obj); struct msm_gem_vma *vma; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); list_for_each_entry(vma, &msm_obj->vmas, list) { if (vma->aspace == aspace) @@ -357,7 +357,7 @@ put_iova_spaces(struct drm_gem_object *obj, bool close) struct msm_gem_object *msm_obj = to_msm_bo(obj); struct msm_gem_vma *vma; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); list_for_each_entry(vma, &msm_obj->vmas, list) { if (vma->aspace) { @@ -375,7 +375,7 @@ put_iova_vmas(struct drm_gem_object *obj) struct msm_gem_object *msm_obj = to_msm_bo(obj); struct msm_gem_vma *vma, *tmp; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); list_for_each_entry_safe(vma, tmp, &msm_obj->vmas, list) { del_vma(vma); @@ -388,7 +388,7 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj, { struct msm_gem_vma *vma; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); vma = lookup_vma(obj, aspace); @@ -428,7 +428,7 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma) if (msm_obj->flags & MSM_BO_CACHED_COHERENT) prot |= IOMMU_CACHE; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (GEM_WARN_ON(msm_obj->madv != MSM_MADV_WILLNEED)) return -EBUSY; @@ -448,7 +448,7 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); msm_obj->pin_count--; GEM_WARN_ON(msm_obj->pin_count < 0); @@ -469,7 +469,7 @@ static int get_and_pin_iova_range_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma; int ret; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); vma = get_vma_locked(obj, aspace, range_start, range_end); if (IS_ERR(vma)) @@ -630,7 +630,7 @@ static void *get_vaddr(struct drm_gem_object *obj, unsigned madv) struct msm_gem_object *msm_obj = to_msm_bo(obj); int ret = 0; - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (obj->import_attach) return ERR_PTR(-ENODEV); @@ -703,7 +703,7 @@ void msm_gem_put_vaddr_locked(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); GEM_WARN_ON(msm_obj->vmap_count < 1); msm_obj->vmap_count--; @@ -745,7 +745,7 @@ void msm_gem_purge(struct drm_gem_object *obj) struct drm_device *dev = obj->dev; struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); GEM_WARN_ON(!is_purgeable(msm_obj)); /* Get rid of any iommu mapping(s): */ @@ -782,7 +782,7 @@ void msm_gem_evict(struct drm_gem_object *obj) struct drm_device *dev = obj->dev; struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); GEM_WARN_ON(is_unevictable(msm_obj)); /* Get rid of any iommu mapping(s): */ @@ -797,7 +797,7 @@ void msm_gem_vunmap(struct drm_gem_object *obj) { struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (!msm_obj->vaddr || GEM_WARN_ON(!is_vunmapable(msm_obj))) return; @@ -811,7 +811,7 @@ static void update_lru(struct drm_gem_object *obj) struct msm_drm_private *priv = obj->dev->dev_private; struct msm_gem_object *msm_obj = to_msm_bo(obj); - GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); + msm_gem_assert_locked(&msm_obj->base); if (!msm_obj->pages) { GEM_WARN_ON(msm_obj->pin_count); @@ -831,7 +831,7 @@ static void update_lru(struct drm_gem_object *obj) bool msm_gem_active(struct drm_gem_object *obj) { - GEM_WARN_ON(!msm_gem_is_locked(obj)); + msm_gem_assert_locked(obj); if (to_msm_bo(obj)->pin_count) return true; diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 0403b27ff779..3c6add51d13b 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -215,6 +215,12 @@ msm_gem_is_locked(struct drm_gem_object *obj) return dma_resv_is_locked(obj->resv) || (kref_read(&obj->refcount) == 0); } +static inline void +msm_gem_assert_locked(struct drm_gem_object *obj) +{ + GEM_WARN_ON(!msm_gem_is_locked(obj)); +} + /* imported/exported objects are not purgeable: */ static inline bool is_unpurgeable(struct msm_gem_object *msm_obj) { @@ -229,7 +235,7 @@ static inline bool is_purgeable(struct msm_gem_object *msm_obj) static inline bool is_vunmapable(struct msm_gem_object *msm_obj) { - GEM_WARN_ON(!msm_gem_is_locked(&msm_obj->base)); + msm_gem_assert_locked(&msm_obj->base); return (msm_obj->vmap_count == 0) && msm_obj->vaddr; } From patchwork Tue Aug 2 15:51:48 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rob Clark X-Patchwork-Id: 595380 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 581A4C00140 for ; Tue, 2 Aug 2022 15:52:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237605AbiHBPww (ORCPT ); Tue, 2 Aug 2022 11:52:52 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:54790 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237470AbiHBPwN (ORCPT ); Tue, 2 Aug 2022 11:52:13 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4416932446; Tue, 2 Aug 2022 08:51:57 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id s206so12694321pgs.3; Tue, 02 Aug 2022 08:51:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc; bh=tv9XZJtMf2UZGPK9r5daZO8lAjNs4ugv43Of9czCQIM=; b=ewcgdHYUnR0pjbQr7lLrbohv0CIdWnfTaVa/OdLqKa56REj1mZVDD3loCWUUXysHJb r1nv6LvBlTqUGW5Yn7MekuRqEHSPnRvgfxi8RbvnxLAPMdUnKJd3gAzv7ZK0ctBMmKFK Cx5FfFd7GebaOk14M/0AQVBAEON772Rfp5BU7zVsirwUcKBaErJ3PfTATLVq4G/Bw+B/ vKxsWxn33KGmbgUmEWsOn1uNznXGWHDoB+nDhkdNc08TKQ/kKKSY/K7/EoMlTGURdLwc r5hqBVOcuKvFTYPqyq2XRu1L0cPf/efn+WxvfX6OEEhuXH/KXeoqWzwOqXhYXUDDISLP 8bUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc; bh=tv9XZJtMf2UZGPK9r5daZO8lAjNs4ugv43Of9czCQIM=; b=68dfkfG8SJFu1kDPS5mMp5mH00Dupkd5w9UkwsbbM9/7zL4X0BAT2d3Nqz8kfQfBSS TNxOV1uAhfoehp+oola1czTcPbi+odBcvUdNeCYR4GA898RF+hbWnQXGIGkUuL/CPKMh 2NcmLWxQENsPQZh2ITNTFlpgrKRlQW1jx00e37bxDZoXogLVvo9VNv6b2vnm0bUW3CNp CHJh667quPOvAuMZz1fbB8tlUS5bAJLcId8NMu7h7RxNXZnkm0GP4iUE2vRzILOSjvYe vxlKPUMu5+ddwObdoDL825vZeG+1PiAsm4ftZYj29L7+H5ABFavOF9ewcDWWhDScFqEA b1ZQ== X-Gm-Message-State: ACgBeo2E1xbxP5MDZ9O/bykAd3zlFhXHVBg0uIlLYfHgnzFieLxCWKtS ELd9ojVCrY+JXN/Uvemt+Cw= X-Google-Smtp-Source: AA6agR7fjw8jn0jQNN1u5zHtoNHt/wIMLL6YxqIYFMtVGwrrRbd03oug9Bjkf/Ril+hPJDZo6qr8WQ== X-Received: by 2002:aa7:82c2:0:b0:52d:2317:8938 with SMTP id f2-20020aa782c2000000b0052d23178938mr15195307pfn.75.1659455516225; Tue, 02 Aug 2022 08:51:56 -0700 (PDT) Received: from localhost ([2a00:79e1:abd:4a00:2703:3c72:eb1a:cffd]) by smtp.gmail.com with ESMTPSA id s20-20020aa78bd4000000b0052d5e93fcb7sm5315612pfd.191.2022.08.02.08.51.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Aug 2022 08:51:55 -0700 (PDT) From: Rob Clark To: dri-devel@lists.freedesktop.org Cc: freedreno@lists.freedesktop.org, linux-arm-msm@vger.kernel.org, Rob Clark , Rob Clark , Abhinav Kumar , Dmitry Baryshkov , Sean Paul , David Airlie , Daniel Vetter , linux-kernel@vger.kernel.org (open list) Subject: [PATCH v4 15/15] drm/msm/gem: Convert to lockdep assert Date: Tue, 2 Aug 2022 08:51:48 -0700 Message-Id: <20220802155152.1727594-16-robdclark@gmail.com> X-Mailer: git-send-email 2.36.1 In-Reply-To: <20220802155152.1727594-1-robdclark@gmail.com> References: <20220802155152.1727594-1-robdclark@gmail.com> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org From: Rob Clark Signed-off-by: Rob Clark --- drivers/gpu/drm/msm/msm_gem.h | 15 ++++++--------- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h index 3c6add51d13b..c4844cf3a585 100644 --- a/drivers/gpu/drm/msm/msm_gem.h +++ b/drivers/gpu/drm/msm/msm_gem.h @@ -197,8 +197,8 @@ msm_gem_unlock(struct drm_gem_object *obj) dma_resv_unlock(obj->resv); } -static inline bool -msm_gem_is_locked(struct drm_gem_object *obj) +static inline void +msm_gem_assert_locked(struct drm_gem_object *obj) { /* * Destroying the object is a special case.. msm_gem_free_object() @@ -212,13 +212,10 @@ msm_gem_is_locked(struct drm_gem_object *obj) * Unfortunately lockdep is not aware of this detail. So when the * refcount drops to zero, we pretend it is already locked. */ - return dma_resv_is_locked(obj->resv) || (kref_read(&obj->refcount) == 0); -} - -static inline void -msm_gem_assert_locked(struct drm_gem_object *obj) -{ - GEM_WARN_ON(!msm_gem_is_locked(obj)); + lockdep_assert_once( + (kref_read(&obj->refcount) == 0) || + (lockdep_is_held(&obj->resv->lock.base) != LOCK_STATE_NOT_HELD) + ); } /* imported/exported objects are not purgeable: */