From patchwork Tue May 14 14:28:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ranjan Kumar X-Patchwork-Id: 796846 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7721416F8F7 for ; Tue, 14 May 2024 14:32:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715697128; cv=none; b=IViqih7y3x4/4rUoJzZozREO+MPbaGe2vGiMcEGyphP+J/8rvYY9bIs9rWGBT1zt4jNLKH/PCk4WQ2VCEDbbugAMdOmkeJb+/sMZnsMqvLDAfQITFJY5JDW+EEUWboKEBSUDdzXRGis3HVxCJszM8a6hRVWDqYXeMmhlKzAsDtM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715697128; c=relaxed/simple; bh=dFVSojSvcCs9YWmAzaz/FqqDEoIXdO3J5wCsXTjGa/4=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=XEfzecPFhValeB9xCtFXezEY7sqX+BvzQ5qi9FgmmREw2z3Yfz7/aZMmSTe4WOboEn1M5CIQM72ieC3ohvGjBvCWWvtA05dM5kT6yA0Ubp9btGsKHFH4ckCqK5/5iwdkVrdLle7ODeER65t7HMtYNsoj873BQ3g5umcWsm1sJfU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com; spf=fail smtp.mailfrom=broadcom.com; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b=X/1qCmaf; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=broadcom.com Authentication-Results: smtp.subspace.kernel.org; spf=fail smtp.mailfrom=broadcom.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=broadcom.com header.i=@broadcom.com header.b="X/1qCmaf" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-6f4603237e0so3949812b3a.0 for ; Tue, 14 May 2024 07:32:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=broadcom.com; s=google; t=1715697125; x=1716301925; darn=vger.kernel.org; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:from:to:cc:subject:date:message-id:reply-to; bh=PnIojsW+He4g6BM+XF4FcW1AtL2GhjB69lIPCsf8iwU=; b=X/1qCmafKblKgAvxvPpUr//8kiwVFKtGZq9+PaEk5jDCs4yelbmQq32S4/gxyS3a+8 frVpGvomsBddJHa1fAaUvnWGinBJ0lygYhoa0jbrnL8KvLOzmWQhyRBCR9szerPDYmCM IflxF41PD8AHTmzfU+ItGUR0dIP0fCz/hTFpo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715697125; x=1716301925; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=PnIojsW+He4g6BM+XF4FcW1AtL2GhjB69lIPCsf8iwU=; b=jEI8jxqBSZ/8cn/6g7gLYc87C0UV5TNh6QUrO+/ZxAX7srE2sHkxCWCloiDLQxXS5L oFRKs4hQfOiNNW96zppytEuQoK1UGFpf80/LHNe/93txh/uHibizGtEjcbQKDi3QqCvO eysECNK/oAPtbc4+iQ8Cby/5DxdlybVAxcwPZdekHONKX+fm5pWHSFfvHzDrIB3tWZuf +GdRAKX7ugpVfjGjNX+xMBkHPDbXFJOhO7NzCb04J/GTGIrvW3WE8c1Geq86rZKOrfsV PRz+Q1fTmrSqFr0qWHr8JvCAiH3vJf+TSyoV+c0QcyDguMZaNGLOImzj+93jtzWM4CyP Prgw== X-Gm-Message-State: AOJu0Yx3JgSvqHhc9Pd7rH9KS43UPqrDy17SgBT640lLmTAeVlqAvSpw 0F1+GCCTEFEl++H+o6LUpX8ENSZccNHzcYtFNvQZk7eItvvEDz26ub1tWm4rA4o6dGZq0y0gYxh SROEDBucVNjsQZONTWJkLZxKTz8+PHrheMnP+GHJGSW3l56+wMbgGxr3aHkK12VhpSR3GEelUd5 O4UPwxTrOyfiIcccv6BKp222aGErHQwVAd3+plB8PR+sltkA== X-Google-Smtp-Source: AGHT+IEsVe5JQggVespjLot0f0JXo0X87Py1njBReaSJigwUN5GiQYoaDCFrtG2OoNEmw+6vS2kDbA== X-Received: by 2002:a05:6a20:3cab:b0:1af:fff9:1c59 with SMTP id adf61e73a8af0-1affff91d50mr6416079637.2.1715697121756; Tue, 14 May 2024 07:32:01 -0700 (PDT) Received: from localhost.localdomain ([192.19.234.250]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-63560731046sm7849104a12.67.2024.05.14.07.31.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 May 2024 07:32:00 -0700 (PDT) From: Ranjan Kumar To: linux-scsi@vger.kernel.org, martin.petersen@oracle.com Cc: rajsekhar.chundru@broadcom.com, sathya.prakash@broadcom.com, sumit.saxena@broadcom.com, chandrakanth.patil@broadcom.com, prayas.patel@broadcom.com, Ranjan Kumar Subject: [PATCH v1 2/6] mpi3mr: Driver buffer allocation and posting Date: Tue, 14 May 2024 19:58:54 +0530 Message-Id: <20240514142858.51992-3-ranjan.kumar@broadcom.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20240514142858.51992-1-ranjan.kumar@broadcom.com> References: <20240514142858.51992-1-ranjan.kumar@broadcom.com> Precedence: bulk X-Mailing-List: linux-scsi@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 This patch adds support for allocating a driver diagnostic buffer and posting it to the firmware for capturing the driver logs in the controller for further debugging. Signed-off-by: Sathya Prakash Signed-off-by: Ranjan Kumar --- drivers/scsi/mpi3mr/mpi/mpi30_tool.h | 13 +++ drivers/scsi/mpi3mr/mpi3mr.h | 21 ++++ drivers/scsi/mpi3mr/mpi3mr_fw.c | 154 +++++++++++++++++++++++++++ 3 files changed, 188 insertions(+) diff --git a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h index 3b960893870f..495933856006 100644 --- a/drivers/scsi/mpi3mr/mpi/mpi30_tool.h +++ b/drivers/scsi/mpi3mr/mpi/mpi30_tool.h @@ -7,6 +7,7 @@ #define MPI3_DIAG_BUFFER_TYPE_TRACE (0x01) #define MPI3_DIAG_BUFFER_TYPE_FW (0x02) +#define MPI3_DIAG_BUFFER_TYPE_DRIVER (0x10) #define MPI3_DIAG_BUFFER_ACTION_RELEASE (0x01) struct mpi3_diag_buffer_post_request { @@ -40,5 +41,17 @@ struct mpi3_diag_buffer_manage_request { __le16 reserved0e; }; +struct mpi3_driver_buffer_header { + __le32 signature; + __le16 header_size; + __le16 rtt_file_header_offset; + __le32 flags; + __le32 circular_buffer_size; + __le32 logical_buffer_end; + __le32 logical_buffer_start; + __le32 ioc_use_only18[2]; + __le32 reserved20[760]; + __le32 reserved_rttrace[256]; +}; #endif diff --git a/drivers/scsi/mpi3mr/mpi3mr.h b/drivers/scsi/mpi3mr/mpi3mr.h index 4ef96c39c832..dc7e8f461826 100644 --- a/drivers/scsi/mpi3mr/mpi3mr.h +++ b/drivers/scsi/mpi3mr/mpi3mr.h @@ -37,6 +37,7 @@ #include #include #include +#include #include #include @@ -195,6 +196,13 @@ extern atomic64_t event_counter; #define MPI3MR_HDB_TRIGGER_TYPE_GLOBAL 3 +/* Driver Host Diag Buffer (drv_db) */ +#define MPI3MR_MIN_DIAG_HOST_BUFFER_SZ ((32 * 1024) + \ + sizeof(struct mpi3_driver_buffer_header)) +#define MPI3MR_DEFAULT_DIAG_HOST_BUFFER_SZ ((512 * 1024) + \ + sizeof(struct mpi3_driver_buffer_header)) +#define MPI3MR_UEFI_DIAG_HOST_BUFFER_OFFSET (16 * 1024) + /* SGE Flag definition */ #define MPI3MR_SGEFLAGS_SYSTEM_SIMPLE_END_OF_LIST \ (MPI3_SGE_FLAGS_ELEMENT_TYPE_SIMPLE | MPI3_SGE_FLAGS_DLAS_SYSTEM | \ @@ -218,6 +226,12 @@ extern atomic64_t event_counter; #define MPI3MR_WRITE_SAME_MAX_LEN_256_BLKS 256 #define MPI3MR_WRITE_SAME_MAX_LEN_2048_BLKS 2048 +/* Driver diag buffer levels */ +enum mpi3mr_drv_db_level { + MRIOC_DRV_DB_DISABLED = 0, + MRIOC_DRV_DB_MINI = 1, + MRIOC_DRV_DB_FULL = 2, +}; /** * struct mpi3mr_nvme_pt_sge - Structure to store SGEs for NVMe @@ -1113,6 +1127,10 @@ struct scmd_priv { * @ioctl_chain_sge: DMA buffer descriptor for IOCTL chain * @ioctl_resp_sge: DMA buffer descriptor for Mgmt cmd response * @ioctl_sges_allocated: Flag for IOCTL SGEs allocated or not + * @drv_diag_buffer: Diagnostic host buffer virtual address + * @drv_diag_buffer_dma: Diagnostic host buffer DMA address + * @drv_diag_buffer_sz: Diagnostic host buffer size + * */ struct mpi3mr_ioc { struct list_head list; @@ -1310,6 +1328,9 @@ struct mpi3mr_ioc { struct diag_buffer_desc diag_buffers[MPI3MR_MAX_NUM_HDB]; struct mpi3_driver_page2 *driver_pg2; spinlock_t trigger_lock; + void *drv_diag_buffer; + dma_addr_t drv_diag_buffer_dma; + u32 drv_diag_buffer_sz; }; /** diff --git a/drivers/scsi/mpi3mr/mpi3mr_fw.c b/drivers/scsi/mpi3mr/mpi3mr_fw.c index fbd6f32f79ce..5937054b3cdb 100644 --- a/drivers/scsi/mpi3mr/mpi3mr_fw.c +++ b/drivers/scsi/mpi3mr/mpi3mr_fw.c @@ -22,6 +22,17 @@ static int poll_queues; module_param(poll_queues, int, 0444); MODULE_PARM_DESC(poll_queues, "Number of queues for io_uring poll mode. (Range 1 - 126)"); +int drv_db_level = 1; +module_param(drv_db_level, int, 0444); +MODULE_PARM_DESC(drv_db_level, "Driver diagnostic buffer level(Default=1).\n\t\t" + "options:\n\t\t" + "0 = disabled: Driver diagnostic buffer not captured\n\t\t" + "1 = minidump: Driver diagnostic buffer captures prints\n\t\t" + "related to specific mrioc instance\n\t\t" + "2 = fulldump: Driver diagnostic buffer captures prints\n\t\t" + "related to specific mrioc instance and complete dmesg logs" + ); + #if defined(writeq) && defined(CONFIG_64BIT) static inline void mpi3mr_writeq(__u64 b, volatile void __iomem *addr) { @@ -872,6 +883,31 @@ static int mpi3mr_setup_isr(struct mpi3mr_ioc *mrioc, u8 setup_one) return retval; } +static const struct { + enum mpi3mr_drv_db_level value; + char *name; +} mpi3mr_drv_db[] = { + { MRIOC_DRV_DB_DISABLED, "disabled (uefi dump is enabled)" }, + { MRIOC_DRV_DB_MINI, "minidump" }, + { MRIOC_DRV_DB_FULL, "fulldump" }, +}; +static const char *mpi3mr_drv_db_name(enum mpi3mr_drv_db_level drv_db_level) +{ + int i; + char *name = NULL; + + /* Start with Disabled */ + name = mpi3mr_drv_db[0].name; + + for (i = 0; i < ARRAY_SIZE(mpi3mr_drv_db); i++) { + if (mpi3mr_drv_db[i].value == drv_db_level) { + name = mpi3mr_drv_db[i].name; + break; + } + } + return name; +} + static const struct { enum mpi3mr_iocstate value; char *name; @@ -1238,6 +1274,102 @@ static int mpi3mr_issue_and_process_mur(struct mpi3mr_ioc *mrioc, return retval; } +/** + * mpi3mr_alloc_issue_host_diag_buf - Allocate and send host diag buffer + * @mrioc: Adapter instance reference + * + * Issue diagnostic buffer post (unconditional) MPI request through admin queue + * and wait for the completion of it or time out. + * + * Return: 0 on success non-zero on failure + */ +static int mpi3mr_alloc_issue_host_diag_buf(struct mpi3mr_ioc *mrioc) +{ + struct mpi3_diag_buffer_post_request diag_buf_post_req; + dma_addr_t buf_dma_addr; + u32 buf_sz; + int retval = -1; + + ioc_info(mrioc, "driver diag buffer level = %s.\n", + mpi3mr_drv_db_name(drv_db_level)); + + if (!mrioc->drv_diag_buffer) { + mrioc->drv_diag_buffer_sz = + MPI3MR_DEFAULT_DIAG_HOST_BUFFER_SZ; + mrioc->drv_diag_buffer = + dma_alloc_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, + &mrioc->drv_diag_buffer_dma, GFP_KERNEL); + if (!mrioc->drv_diag_buffer) { + mrioc->drv_diag_buffer_sz = + MPI3MR_MIN_DIAG_HOST_BUFFER_SZ; + mrioc->drv_diag_buffer = + dma_alloc_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, + &mrioc->drv_diag_buffer_dma, GFP_KERNEL); + } + if (!mrioc->drv_diag_buffer) { + ioc_warn(mrioc, "%s:%d:failed to allocate buffer\n", + __func__, __LINE__); + mrioc->drv_diag_buffer_sz = 0; + return retval; + } + /* TBD - memset to Zero once feature is stable */ + memset(mrioc->drv_diag_buffer, 0x55, mrioc->drv_diag_buffer_sz); + } + + buf_dma_addr = mrioc->drv_diag_buffer_dma; + buf_sz = mrioc->drv_diag_buffer_sz; + + memset(&diag_buf_post_req, 0, sizeof(diag_buf_post_req)); + mutex_lock(&mrioc->init_cmds.mutex); + if (mrioc->init_cmds.state & MPI3MR_CMD_PENDING) { + ioc_err(mrioc, "sending driver diag buffer post is failed due to command in use\n"); + mutex_unlock(&mrioc->init_cmds.mutex); + return retval; + } + mrioc->init_cmds.state = MPI3MR_CMD_PENDING; + mrioc->init_cmds.is_waiting = 1; + mrioc->init_cmds.callback = NULL; + diag_buf_post_req.host_tag = cpu_to_le16(MPI3MR_HOSTTAG_INITCMDS); + diag_buf_post_req.function = MPI3_FUNCTION_DIAG_BUFFER_POST; + diag_buf_post_req.type = MPI3_DIAG_BUFFER_TYPE_DRIVER; + diag_buf_post_req.address = le64_to_cpu(buf_dma_addr); + diag_buf_post_req.length = le32_to_cpu(buf_sz); + + init_completion(&mrioc->init_cmds.done); + retval = mpi3mr_admin_request_post(mrioc, &diag_buf_post_req, + sizeof(diag_buf_post_req), 1); + if (retval) { + ioc_err(mrioc, "posting driver diag buffer failed\n"); + goto out_unlock; + } + wait_for_completion_timeout(&mrioc->init_cmds.done, + (MPI3MR_INTADMCMD_TIMEOUT * HZ)); + if (!(mrioc->init_cmds.state & MPI3MR_CMD_COMPLETE)) { + ioc_err(mrioc, "posting driver diag buffer timed out\n"); + mpi3mr_check_rh_fault_ioc(mrioc, + MPI3MR_RESET_FROM_DIAG_BUFFER_POST_TIMEOUT); + retval = -1; + goto out_unlock; + } + retval = 0; + if ((mrioc->init_cmds.ioc_status & MPI3_IOCSTATUS_STATUS_MASK) + != MPI3_IOCSTATUS_SUCCESS) + ioc_warn(mrioc, + "driver diag buffer post returned with ioc_status(0x%04x) log_info(0x%08x)\n", + (mrioc->init_cmds.ioc_status & MPI3_IOCSTATUS_STATUS_MASK), + mrioc->init_cmds.ioc_loginfo); + else + ioc_info(mrioc, "driver diag buffer of size %dKB posted successfully\n", + mrioc->drv_diag_buffer_sz / 1024); + +out_unlock: + mrioc->init_cmds.state = MPI3MR_CMD_NOTUSED; + mutex_unlock(&mrioc->init_cmds.mutex); + return retval; +} + /** * mpi3mr_revalidate_factsdata - validate IOCFacts parameters * during reset/resume @@ -4168,6 +4300,13 @@ int mpi3mr_init_ioc(struct mpi3mr_ioc *mrioc) goto out_failed; } + dprint_reset(mrioc, "posting driver diag buffer\n"); + retval = mpi3mr_alloc_issue_host_diag_buf(mrioc); + if (retval) { + ioc_err(mrioc, "failed to post driver diag buffer\n"); + goto out_failed; + } + ioc_info(mrioc, "controller initialization completed successfully\n"); return retval; out_failed: @@ -4358,6 +4497,13 @@ int mpi3mr_reinit_ioc(struct mpi3mr_ioc *mrioc, u8 is_resume) } else ioc_info(mrioc, "port enable completed successfully\n"); + dprint_reset(mrioc, "posting driver diag buffer\n"); + retval = mpi3mr_alloc_issue_host_diag_buf(mrioc); + if (retval) { + ioc_err(mrioc, "failed to post driver diag buffer\n"); + goto out_failed; + } + ioc_info(mrioc, "controller %s completed successfully\n", (is_resume)?"resume":"re-initialization"); return retval; @@ -4669,6 +4815,14 @@ void mpi3mr_free_mem(struct mpi3mr_ioc *mrioc) } } + if (mrioc->drv_diag_buffer) { + dma_free_coherent(&mrioc->pdev->dev, + mrioc->drv_diag_buffer_sz, mrioc->drv_diag_buffer, + mrioc->drv_diag_buffer_dma); + mrioc->drv_diag_buffer = NULL; + mrioc->drv_diag_buffer_sz = 0; + } + kfree(mrioc->throttle_groups); mrioc->throttle_groups = NULL;