From patchwork Mon Nov 16 09:07:26 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9808BC4742C for ; Mon, 16 Nov 2020 09:40:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 28DB62225B for ; Mon, 16 Nov 2020 09:40:18 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q/s2XzIP" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728352AbgKPJIB (ORCPT ); Mon, 16 Nov 2020 04:08:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49639 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726291AbgKPJIB (ORCPT ); Mon, 16 Nov 2020 04:08:01 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517680; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1KTiUwVy+emA6v5nYKdFgzkrxs5yXDxztLIktrszjVo=; b=Q/s2XzIPMYSbBhWupxpeWANkfq+8l8qYmmDwP+SJlvRMvVrSLGx5MSxxdPc3pwyc/YVgeb 2H+nl7Z/+p6+j4Th4Pp1J0xU1vNdQJeFALEXX9YHbTx3JVmDqwpvI6McW5cMdM+B8VnVVA 50x2cuHS4SM965LIzgX3heUulCMTDV8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-249-SFsBCQvIP5ilj7Yn2hddFw-1; Mon, 16 Nov 2020 04:07:55 -0500 X-MC-Unique: SFsBCQvIP5ilj7Yn2hddFw-1 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3475F186DD2B; Mon, 16 Nov 2020 09:07:54 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A8F45D9CC; Mon, 16 Nov 2020 09:07:50 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" , Hannes Reinecke , Johannes Thumshirn Subject: [PATCH V4 01/12] sbitmap: remove sbitmap_clear_bit_unlock Date: Mon, 16 Nov 2020 17:07:26 +0800 Message-Id: <20201116090737.50989-2-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org No one uses this helper any more, so kill it. Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Cc: Hannes Reinecke Reviewed-by: Hannes Reinecke Reviewed-by: Johannes Thumshirn Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei --- include/linux/sbitmap.h | 6 ------ 1 file changed, 6 deletions(-) diff --git a/include/linux/sbitmap.h b/include/linux/sbitmap.h index e40d019c3d9d..51edc05489cb 100644 --- a/include/linux/sbitmap.h +++ b/include/linux/sbitmap.h @@ -320,12 +320,6 @@ static inline void sbitmap_deferred_clear_bit(struct sbitmap *sb, unsigned int b set_bit(SB_NR_TO_BIT(sb, bitnr), addr); } -static inline void sbitmap_clear_bit_unlock(struct sbitmap *sb, - unsigned int bitnr) -{ - clear_bit_unlock(SB_NR_TO_BIT(sb, bitnr), __sbitmap_word(sb, bitnr)); -} - static inline int sbitmap_test_bit(struct sbitmap *sb, unsigned int bitnr) { return test_bit(SB_NR_TO_BIT(sb, bitnr), __sbitmap_word(sb, bitnr)); From patchwork Mon Nov 16 09:07:28 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324968 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4E6A1C4742C for ; Mon, 16 Nov 2020 09:41:40 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id D37DB206A5 for ; Mon, 16 Nov 2020 09:41:39 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Uk/T31Si" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728395AbgKPJIM (ORCPT ); Mon, 16 Nov 2020 04:08:12 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49340 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728379AbgKPJIM (ORCPT ); Mon, 16 Nov 2020 04:08:12 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517690; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1FlWyHO+16o2KDl8pHqyk3MKvV5WZqHycr/u/6i+gg8=; b=Uk/T31SisANMwCBDIlchLGUdTticZINPZgC7OYMw9tfq+Vs4z8VsNt9afmZWnDI/QX5Nr6 QQRGSDgp78cY7tgUQKxy0wXYC/C9x0zBTf0KDKuStY0WacNmOzjOkZkh2QGGreOFkkLJEP Snx04KTLvMrfgt2Qhjg1/rL2czz3iMk= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-85-nrqfENkQO0SIlYnlC4ttLA-1; Mon, 16 Nov 2020 04:08:08 -0500 X-MC-Unique: nrqfENkQO0SIlYnlC4ttLA-1 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 5C8CF879519; Mon, 16 Nov 2020 09:08:06 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id ACCEE60C84; Mon, 16 Nov 2020 09:08:02 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" , Hannes Reinecke Subject: [PATCH V4 03/12] sbitmap: add helpers for updating allocation hint Date: Mon, 16 Nov 2020 17:07:28 +0800 Message-Id: <20201116090737.50989-4-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Add helpers for updating allocation hint, so that we can avoid to duplicate code. Prepare for moving allocation hint into sbitmap. Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Cc: Hannes Reinecke Reviewed-by: Hannes Reinecke Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei --- lib/sbitmap.c | 93 ++++++++++++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 39 deletions(-) diff --git a/lib/sbitmap.c b/lib/sbitmap.c index 8d920d66d42a..4e4423414f4d 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -9,6 +9,55 @@ #include #include +static int init_alloc_hint(struct sbitmap_queue *sbq, gfp_t flags) +{ + unsigned depth = sbq->sb.depth; + + sbq->alloc_hint = alloc_percpu_gfp(unsigned int, flags); + if (!sbq->alloc_hint) + return -ENOMEM; + + if (depth && !sbq->sb.round_robin) { + int i; + + for_each_possible_cpu(i) + *per_cpu_ptr(sbq->alloc_hint, i) = prandom_u32() % depth; + } + + return 0; +} + +static inline unsigned update_alloc_hint_before_get(struct sbitmap_queue *sbq, + unsigned int depth) +{ + unsigned hint; + + hint = this_cpu_read(*sbq->alloc_hint); + if (unlikely(hint >= depth)) { + hint = depth ? prandom_u32() % depth : 0; + this_cpu_write(*sbq->alloc_hint, hint); + } + + return hint; +} + +static inline void update_alloc_hint_after_get(struct sbitmap_queue *sbq, + unsigned int depth, + unsigned int hint, + unsigned int nr) +{ + if (nr == -1) { + /* If the map is full, a hint won't do us much good. */ + this_cpu_write(*sbq->alloc_hint, 0); + } else if (nr == hint || unlikely(sbq->sb.round_robin)) { + /* Only update the hint if we used it. */ + hint = nr + 1; + if (hint >= depth - 1) + hint = 0; + this_cpu_write(*sbq->alloc_hint, hint); + } +} + /* * See if we have deferred clears that we can batch move */ @@ -363,17 +412,11 @@ int sbitmap_queue_init_node(struct sbitmap_queue *sbq, unsigned int depth, if (ret) return ret; - sbq->alloc_hint = alloc_percpu_gfp(unsigned int, flags); - if (!sbq->alloc_hint) { + if (init_alloc_hint(sbq, flags) != 0) { sbitmap_free(&sbq->sb); return -ENOMEM; } - if (depth && !round_robin) { - for_each_possible_cpu(i) - *per_cpu_ptr(sbq->alloc_hint, i) = prandom_u32() % depth; - } - sbq->min_shallow_depth = UINT_MAX; sbq->wake_batch = sbq_calc_wake_batch(sbq, depth); atomic_set(&sbq->wake_index, 0); @@ -426,24 +469,10 @@ int __sbitmap_queue_get(struct sbitmap_queue *sbq) unsigned int hint, depth; int nr; - hint = this_cpu_read(*sbq->alloc_hint); depth = READ_ONCE(sbq->sb.depth); - if (unlikely(hint >= depth)) { - hint = depth ? prandom_u32() % depth : 0; - this_cpu_write(*sbq->alloc_hint, hint); - } + hint = update_alloc_hint_before_get(sbq, depth); nr = sbitmap_get(&sbq->sb, hint); - - if (nr == -1) { - /* If the map is full, a hint won't do us much good. */ - this_cpu_write(*sbq->alloc_hint, 0); - } else if (nr == hint || unlikely(sbq->sb.round_robin)) { - /* Only update the hint if we used it. */ - hint = nr + 1; - if (hint >= depth - 1) - hint = 0; - this_cpu_write(*sbq->alloc_hint, hint); - } + update_alloc_hint_after_get(sbq, depth, hint, nr); return nr; } @@ -457,24 +486,10 @@ int __sbitmap_queue_get_shallow(struct sbitmap_queue *sbq, WARN_ON_ONCE(shallow_depth < sbq->min_shallow_depth); - hint = this_cpu_read(*sbq->alloc_hint); depth = READ_ONCE(sbq->sb.depth); - if (unlikely(hint >= depth)) { - hint = depth ? prandom_u32() % depth : 0; - this_cpu_write(*sbq->alloc_hint, hint); - } + hint = update_alloc_hint_before_get(sbq, depth); nr = sbitmap_get_shallow(&sbq->sb, hint, shallow_depth); - - if (nr == -1) { - /* If the map is full, a hint won't do us much good. */ - this_cpu_write(*sbq->alloc_hint, 0); - } else if (nr == hint || unlikely(sbq->sb.round_robin)) { - /* Only update the hint if we used it. */ - hint = nr + 1; - if (hint >= depth - 1) - hint = 0; - this_cpu_write(*sbq->alloc_hint, hint); - } + update_alloc_hint_after_get(sbq, depth, hint, nr); return nr; } From patchwork Mon Nov 16 09:07:30 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324967 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 97ABCC4742C for ; Mon, 16 Nov 2020 09:43:42 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2249920782 for ; Mon, 16 Nov 2020 09:43:42 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gJ8Hw8W6" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728477AbgKPJIZ (ORCPT ); Mon, 16 Nov 2020 04:08:25 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:60938 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728461AbgKPJIZ (ORCPT ); Mon, 16 Nov 2020 04:08:25 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517703; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cOrWYdtv2sn5+hknxf7bjqdB5haJNia72F3gWe8QNnc=; b=gJ8Hw8W6PYsnKuz6PwknEb+hp/DchS17acb/Ku6JeTNcLem2aT79PQ4WF43ouLzxHXlMLZ SUlObeOVe6pxG/0g+ZEOo69jn1ntmiBR+eDh1uG68+Ht5LkL+k63wWlGU/CQvPHAlpSLFR lDDG4lZRi2YeSd+57bpjql4EzdIPq8U= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-31-RItekd-qNEelLU-hsqMY8w-1; Mon, 16 Nov 2020 04:08:21 -0500 X-MC-Unique: RItekd-qNEelLU-hsqMY8w-1 Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 73B4F879516; Mon, 16 Nov 2020 09:08:19 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id A834255762; Mon, 16 Nov 2020 09:08:15 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" , Hannes Reinecke Subject: [PATCH V4 05/12] sbitmap: export sbitmap_weight Date: Mon, 16 Nov 2020 17:07:30 +0800 Message-Id: <20201116090737.50989-6-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org SCSI's .device_busy will be converted to sbitmap, and sbitmap_weight is needed, so export the helper. Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Reviewed-by: Hannes Reinecke Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei --- include/linux/sbitmap.h | 9 +++++++++ lib/sbitmap.c | 11 ++++++----- 2 files changed, 15 insertions(+), 5 deletions(-) diff --git a/include/linux/sbitmap.h b/include/linux/sbitmap.h index 103b41c03311..34343ce3ef6c 100644 --- a/include/linux/sbitmap.h +++ b/include/linux/sbitmap.h @@ -346,6 +346,15 @@ static inline int sbitmap_test_bit(struct sbitmap *sb, unsigned int bitnr) */ void sbitmap_show(struct sbitmap *sb, struct seq_file *m); + +/** + * sbitmap_weight() - Return how many real bits set in a &struct sbitmap. + * @sb: Bitmap to check. + * + * Return: How many real bits set + */ +unsigned int sbitmap_weight(const struct sbitmap *sb); + /** * sbitmap_bitmap_show() - Write a hex dump of a &struct sbitmap to a &struct * seq_file. diff --git a/lib/sbitmap.c b/lib/sbitmap.c index dcd6a89b4d2f..fb1d3c2f70a2 100644 --- a/lib/sbitmap.c +++ b/lib/sbitmap.c @@ -342,20 +342,21 @@ static unsigned int __sbitmap_weight(const struct sbitmap *sb, bool set) return weight; } -static unsigned int sbitmap_weight(const struct sbitmap *sb) +static unsigned int sbitmap_cleared(const struct sbitmap *sb) { - return __sbitmap_weight(sb, true); + return __sbitmap_weight(sb, false); } -static unsigned int sbitmap_cleared(const struct sbitmap *sb) +unsigned int sbitmap_weight(const struct sbitmap *sb) { - return __sbitmap_weight(sb, false); + return __sbitmap_weight(sb, true) - sbitmap_cleared(sb); } +EXPORT_SYMBOL_GPL(sbitmap_weight); void sbitmap_show(struct sbitmap *sb, struct seq_file *m) { seq_printf(m, "depth=%u\n", sb->depth); - seq_printf(m, "busy=%u\n", sbitmap_weight(sb) - sbitmap_cleared(sb)); + seq_printf(m, "busy=%u\n", sbitmap_weight(sb)); seq_printf(m, "cleared=%u\n", sbitmap_cleared(sb)); seq_printf(m, "bits_per_word=%u\n", 1U << sb->shift); seq_printf(m, "map_nr=%u\n", sb->map_nr); From patchwork Mon Nov 16 09:07:32 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324966 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E5031C4742C for ; Mon, 16 Nov 2020 09:45:44 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8078920782 for ; Mon, 16 Nov 2020 09:45:44 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bBrhXXWV" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728494AbgKPJIj (ORCPT ); Mon, 16 Nov 2020 04:08:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:32912 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728492AbgKPJIi (ORCPT ); Mon, 16 Nov 2020 04:08:38 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517717; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CriNmAXBW7mzc+PjogMXuYh2RxVXtOSsoBGshC1pNVQ=; b=bBrhXXWVRC9UeJpFnfNvh1oBpVYNKtH07/xentc93dxJCJCy8yCMCFI7+puFA61F4I7SRy ckTBpd7NxgpuS03ekyAJr85HyePgK1rVtRGhpxingB3W+c6/ndom+DXJFdJvK986s8+GZg vu4xJaK+Py6B/Fbr6EbqWQQzDii+sA8= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-101-0LmrNOXzMsun9g0ax_wePg-1; Mon, 16 Nov 2020 04:08:34 -0500 X-MC-Unique: 0LmrNOXzMsun9g0ax_wePg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 32E481007B01; Mon, 16 Nov 2020 09:08:33 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id C34936EF55; Mon, 16 Nov 2020 09:08:28 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" , Hannes Reinecke Subject: [PATCH V4 07/12] blk-mq: add callbacks for storing & retrieving budget token Date: Mon, 16 Nov 2020 17:07:32 +0800 Message-Id: <20201116090737.50989-8-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org SCSI is the only driver which requires dispatch budget, and it isn't fair to add one field into 'struct request' for storing budget token which will be used in the following patches for improving scsi's device busy scalability. Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Cc: Hannes Reinecke Reviewed-by: Hannes Reinecke Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei --- drivers/scsi/scsi_lib.c | 18 ++++++++++++++++++ include/linux/blk-mq.h | 9 +++++++++ include/scsi/scsi_cmnd.h | 2 ++ 3 files changed, 29 insertions(+) diff --git a/drivers/scsi/scsi_lib.c b/drivers/scsi/scsi_lib.c index 60c7a7d74852..022ed2991463 100644 --- a/drivers/scsi/scsi_lib.c +++ b/drivers/scsi/scsi_lib.c @@ -1635,6 +1635,20 @@ static bool scsi_mq_get_budget(struct request_queue *q) return false; } +static void scsi_mq_set_rq_budget_token(struct request *req, int token) +{ + struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(req); + + cmd->budget_token = token; +} + +static int scsi_mq_get_rq_budget_token(struct request *req) +{ + struct scsi_cmnd *cmd = blk_mq_rq_to_pdu(req); + + return cmd->budget_token; +} + static blk_status_t scsi_queue_rq(struct blk_mq_hw_ctx *hctx, const struct blk_mq_queue_data *bd) { @@ -1845,6 +1859,8 @@ static const struct blk_mq_ops scsi_mq_ops_no_commit = { .cleanup_rq = scsi_cleanup_rq, .busy = scsi_mq_lld_busy, .map_queues = scsi_map_queues, + .set_rq_budget_token = scsi_mq_set_rq_budget_token, + .get_rq_budget_token = scsi_mq_get_rq_budget_token, }; @@ -1873,6 +1889,8 @@ static const struct blk_mq_ops scsi_mq_ops = { .cleanup_rq = scsi_cleanup_rq, .busy = scsi_mq_lld_busy, .map_queues = scsi_map_queues, + .set_rq_budget_token = scsi_mq_set_rq_budget_token, + .get_rq_budget_token = scsi_mq_get_rq_budget_token, }; struct request_queue *scsi_mq_alloc_queue(struct scsi_device *sdev) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index 794b2a33a2c3..d2c66b453d07 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -316,6 +316,15 @@ struct blk_mq_ops { */ void (*put_budget)(struct request_queue *); + /* + * @set_rq_budget_toekn: store rq's budget token + */ + void (*set_rq_budget_token)(struct request *, int); + /* + * @get_rq_budget_toekn: retrieve rq's budget token + */ + int (*get_rq_budget_token)(struct request *); + /** * @timeout: Called on request timeout. */ diff --git a/include/scsi/scsi_cmnd.h b/include/scsi/scsi_cmnd.h index 69ade4fb71aa..4884f300c896 100644 --- a/include/scsi/scsi_cmnd.h +++ b/include/scsi/scsi_cmnd.h @@ -75,6 +75,8 @@ struct scsi_cmnd { int eh_eflags; /* Used by error handlr */ + int budget_token; + /* * This is set to jiffies as it was when the command was first * allocated. It is used to time how long the command has From patchwork Mon Nov 16 09:07:34 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324965 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2AC91C55ABD for ; Mon, 16 Nov 2020 09:48:29 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id C10492225B for ; Mon, 16 Nov 2020 09:48:28 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iQlyVQjl" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728514AbgKPJIs (ORCPT ); Mon, 16 Nov 2020 04:08:48 -0500 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:34306 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728510AbgKPJIs (ORCPT ); Mon, 16 Nov 2020 04:08:48 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517726; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aYSjbzjWriJIsnEydUg72adUc1vhljl6EwJoN3Dib3Q=; b=iQlyVQjlIZMXphhDvrCLqBttfK7HSP078Oi10xBGCd3EI/vG1Klga483YC5Q4jI6BoacIM iLMEtZoRSL8V0NYpGkmX3IpSFreq/zIhRlV+2I+m49BbPZQElaq1aH6b2sEco/Ap17rSQO AA/1cMQNtKLr+W7LeU+zhRqAEU2xNo4= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-491-Ki4evKJuNPODmiKH-vQWKg-1; Mon, 16 Nov 2020 04:08:44 -0500 X-MC-Unique: Ki4evKJuNPODmiKH-vQWKg-1 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B7C975F9EC; Mon, 16 Nov 2020 09:08:42 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id F40566EF55; Mon, 16 Nov 2020 09:08:41 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" , Hannes Reinecke , Christoph Hellwig Subject: [PATCH V4 09/12] scsi: put hot fields of scsi_host_template into one cacheline Date: Mon, 16 Nov 2020 17:07:34 +0800 Message-Id: <20201116090737.50989-10-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org The following three fields of scsi_host_template are referenced in scsi IO submission path, so put them together into one cacheline: - cmd_size - queuecommand - commit_rqs Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Cc: Hannes Reinecke Reviewed-by: Hannes Reinecke Reviewed-by: Christoph Hellwig Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei --- include/scsi/scsi_host.h | 72 ++++++++++++++++++++++------------------ 1 file changed, 39 insertions(+), 33 deletions(-) diff --git a/include/scsi/scsi_host.h b/include/scsi/scsi_host.h index 701f178b20ae..2d6e3a1f5f0b 100644 --- a/include/scsi/scsi_host.h +++ b/include/scsi/scsi_host.h @@ -30,40 +30,15 @@ struct scsi_transport_template; #define MODE_TARGET 0x02 struct scsi_host_template { - struct module *module; - const char *name; - /* - * The info function will return whatever useful information the - * developer sees fit. If not provided, then the name field will - * be used instead. - * - * Status: OPTIONAL + * Put fields referenced in IO submission path together in + * same cacheline */ - const char *(* info)(struct Scsi_Host *); /* - * Ioctl interface - * - * Status: OPTIONAL - */ - int (*ioctl)(struct scsi_device *dev, unsigned int cmd, - void __user *arg); - - -#ifdef CONFIG_COMPAT - /* - * Compat handler. Handle 32bit ABI. - * When unknown ioctl is passed return -ENOIOCTLCMD. - * - * Status: OPTIONAL + * Additional per-command data allocated for the driver. */ - int (*compat_ioctl)(struct scsi_device *dev, unsigned int cmd, - void __user *arg); -#endif - - int (*init_cmd_priv)(struct Scsi_Host *shost, struct scsi_cmnd *cmd); - int (*exit_cmd_priv)(struct Scsi_Host *shost, struct scsi_cmnd *cmd); + unsigned int cmd_size; /* * The queuecommand function is used to queue up a scsi @@ -111,6 +86,41 @@ struct scsi_host_template { */ void (*commit_rqs)(struct Scsi_Host *, u16); + struct module *module; + const char *name; + + /* + * The info function will return whatever useful information the + * developer sees fit. If not provided, then the name field will + * be used instead. + * + * Status: OPTIONAL + */ + const char *(*info)(struct Scsi_Host *); + + /* + * Ioctl interface + * + * Status: OPTIONAL + */ + int (*ioctl)(struct scsi_device *dev, unsigned int cmd, + void __user *arg); + + +#ifdef CONFIG_COMPAT + /* + * Compat handler. Handle 32bit ABI. + * When unknown ioctl is passed return -ENOIOCTLCMD. + * + * Status: OPTIONAL + */ + int (*compat_ioctl)(struct scsi_device *dev, unsigned int cmd, + void __user *arg); +#endif + + int (*init_cmd_priv)(struct Scsi_Host *shost, struct scsi_cmnd *cmd); + int (*exit_cmd_priv)(struct Scsi_Host *shost, struct scsi_cmnd *cmd); + /* * This is an error handling strategy routine. You don't need to * define one of these if you don't want to - there is a default @@ -478,10 +488,6 @@ struct scsi_host_template { */ u64 vendor_id; - /* - * Additional per-command data allocated for the driver. - */ - unsigned int cmd_size; struct scsi_host_cmd_pool *cmd_pool; /* Delay for runtime autosuspend */ From patchwork Mon Nov 16 09:07:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 324964 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-9.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_PATCH, MAILING_LIST_MULTI, SIGNED_OFF_BY, SPF_HELO_NONE, SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 60102C55ABD for ; Mon, 16 Nov 2020 09:50:31 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E99FB22370 for ; Mon, 16 Nov 2020 09:50:30 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PaAlmlN1" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728534AbgKPJJB (ORCPT ); Mon, 16 Nov 2020 04:09:01 -0500 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:29091 "EHLO us-smtp-delivery-124.mimecast.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728531AbgKPJJA (ORCPT ); Mon, 16 Nov 2020 04:09:00 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1605517739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=NBlcwixDeij9jtY3WYF417Hx5FZ2dhJB1nxSwj3Zydc=; b=PaAlmlN1I0aeVQXEwmVvcN3HKRfzmhnqw+zqTJD8GJcp1GjiTWXYmqUhWWcUAV7kUCgH+t 7zZSdK8zzNMfo7f7j5vDZT+dr/OgU6j0J5YTaXlvevsLlZIq+mvidy7FEffiUbbdqyHXKz FjtDXM3daYyHhgIjn+FEPX2Sn2QEWgs= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-382-AIx48d-bO_C0i8O_Z6xzrQ-1; Mon, 16 Nov 2020 04:08:56 -0500 X-MC-Unique: AIx48d-bO_C0i8O_Z6xzrQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 45A2510B9CA1; Mon, 16 Nov 2020 09:08:55 +0000 (UTC) Received: from localhost (ovpn-13-166.pek2.redhat.com [10.72.13.166]) by smtp.corp.redhat.com (Postfix) with ESMTP id AA720171F4; Mon, 16 Nov 2020 09:08:51 +0000 (UTC) From: Ming Lei To: Jens Axboe , linux-block@vger.kernel.org, "Martin K . Petersen" , linux-scsi@vger.kernel.org Cc: Ming Lei , Omar Sandoval , Kashyap Desai , Sumanesh Samanta , "Ewan D . Milne" Subject: [PATCH V4 11/12] scsi: make sure sdev->queue_depth is <= max(shost->can_queue, 1024) Date: Mon, 16 Nov 2020 17:07:36 +0800 Message-Id: <20201116090737.50989-12-ming.lei@redhat.com> In-Reply-To: <20201116090737.50989-1-ming.lei@redhat.com> References: <20201116090737.50989-1-ming.lei@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org Limit scsi device's queue depth is less than max(host->can_queue, 1024) in scsi_change_queue_depth(), and 1024 is big enough for saturating current fast SCSI LUN(SSD, or raid volume on multiple SSDs). We need this patch for replacing sdev->device_busy with sbitmap which has to be pre-allocated with reasonable max depth. Cc: Omar Sandoval Cc: Kashyap Desai Cc: Sumanesh Samanta Cc: Ewan D. Milne Tested-by: Sumanesh Samanta Signed-off-by: Ming Lei Reviewed-by: Hannes Reinecke --- drivers/scsi/scsi.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/scsi/scsi.c b/drivers/scsi/scsi.c index 24619c3bebd5..a28d48c850cf 100644 --- a/drivers/scsi/scsi.c +++ b/drivers/scsi/scsi.c @@ -214,6 +214,15 @@ void scsi_finish_command(struct scsi_cmnd *cmd) scsi_io_completion(cmd, good_bytes); } + +/* + * 1024 is big enough for saturating the fast scsi LUN now + */ +static int scsi_device_max_queue_depth(struct scsi_device *sdev) +{ + return max_t(int, sdev->host->can_queue, 1024); +} + /** * scsi_change_queue_depth - change a device's queue depth * @sdev: SCSI Device in question @@ -223,6 +232,8 @@ void scsi_finish_command(struct scsi_cmnd *cmd) */ int scsi_change_queue_depth(struct scsi_device *sdev, int depth) { + depth = min_t(int, depth, scsi_device_max_queue_depth(sdev)); + if (depth > 0) { sdev->queue_depth = depth; wmb();