From patchwork Mon Jun 24 15:09:25 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Robert Richter X-Patchwork-Id: 167610 Delivered-To: patch@linaro.org Received: by 2002:a92:4782:0:0:0:0:0 with SMTP id e2csp4347782ilk; Mon, 24 Jun 2019 08:09:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqxOm8HGHqreCLqyARML7t7Z2PbihSbWZ5zpOMiqQOfLFHzA8ZF5XI0CUxCYd9mp+LiA0rZF X-Received: by 2002:a17:902:4b:: with SMTP id 69mr3381272pla.89.1561388981114; Mon, 24 Jun 2019 08:09:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1561388981; cv=none; d=google.com; s=arc-20160816; b=iEm4dEGYm3NpLyiVQRR4jxuMk+cei4a4Zu8/8+TZ2KLMXCc3yTVyHTjuvL5aOe4l9w vY58hYiZE2BDulaPqNiSIqLQ4c9ZTGVktZuLnlGa9XMldDK2ILhcKppl7Ya/Qxy+ZC1E eaygbFycNO8eTrvTIICoCVEH1J8HSPUWbsYwbel1abZH7SPNtZnMyA7tdn9atfSl/cd1 zf6niEkivRa7PTgVBIDTm6spO7q053ztwXH8mG34S/KAKsMkLkW/ZMxEQYh7Z+uenaKD 8LoOfKJBrcGptjUgNMaMVw6NGAwLOT++kbveygKLqTW1NsVYgqz552xzHc42YHsZxPGU x2Gw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:content-transfer-encoding :content-language:accept-language:in-reply-to:references:message-id :date:thread-index:thread-topic:subject:cc:to:from:dkim-signature :dkim-signature; bh=fGiKqzPfUc9AXTHSQs9EVcWX6jy3P7JI7GKuDwheurI=; b=wKXvVP/XOmQv/wbHv4YkACcqfWsUUiLc5DdQmLL6mUfiVXlsJgTvHd1hIukFnUgXoX Ec9lCC/cy9rxFXakPa/lZAR41lVFIUtm+g/Ewgg50/JQ1I1ql6y9Uh/Pfmax6+AQVe0N tnPlAunzIjkcGlk9eDON7HfkTinj7uuLeQl6L9e5MHv4sjqYdv9fI9mvm05jeIzOY0uf QnG6mDAj8hhAvEDk3x1XcN38S4L+ZAbPU/G5R1Szqsc9NEwz+gbAJK7wOzIhUjcHxYO9 bzpqYAt4nv04RrlV+6Dy0zFUBmmJWMehAfEV0TmiJ8awWgoIalLPtlBwDVxHgEWgOEuz J4rQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@marvell.com header.s=pfpt0818 header.b=CKgBRbYA; dkim=pass header.i=@marvell.onmicrosoft.com header.s=selector2-marvell-onmicrosoft-com header.b=X8AieWew; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=marvell.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j1si10963534pfr.52.2019.06.24.08.09.40; Mon, 24 Jun 2019 08:09:41 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@marvell.com header.s=pfpt0818 header.b=CKgBRbYA; dkim=pass header.i=@marvell.onmicrosoft.com header.s=selector2-marvell-onmicrosoft-com header.b=X8AieWew; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=marvell.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730739AbfFXPJj (ORCPT + 30 others); Mon, 24 Jun 2019 11:09:39 -0400 Received: from mx0a-0016f401.pphosted.com ([67.231.148.174]:62922 "EHLO mx0b-0016f401.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1730682AbfFXPJf (ORCPT ); Mon, 24 Jun 2019 11:09:35 -0400 Received: from pps.filterd (m0045849.ppops.net [127.0.0.1]) by mx0a-0016f401.pphosted.com (8.16.0.27/8.16.0.27) with SMTP id x5OF0720018628; Mon, 24 Jun 2019 08:09:27 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-transfer-encoding : mime-version; s=pfpt0818; bh=fGiKqzPfUc9AXTHSQs9EVcWX6jy3P7JI7GKuDwheurI=; b=CKgBRbYARYn1hUyrxb0dCbUlA+fBle9XyaE7mtHsQAdDczARz4sb13sPLW2fGnzNbr9a 81FLfzSnbZlyqRs6pdEuoiC3oIPLMrc/6Q+aFI11KbdoYrG963yU0bPx38nXlo56uFhD qoL4UT/cweY8L+S/LiQ+JbPSeG7bLVQPj4k+IXmQnrb1nawy/2LUqbfE7IOQatoPaBCG FQU3gTLV5kE7NkgvJemlnBVHaQwCa0Ur8IUvAVSj0tAO9ArqGCEw91PLZ1YS8U8vWOJe 4Xttguaq3Eg9LChUD63/n+ayMsjGGhGEbGu0YsDERkY9vKnG0ORkiXEedHhv5YcBntYg Fw== Received: from sc-exch04.marvell.com ([199.233.58.184]) by mx0a-0016f401.pphosted.com with ESMTP id 2tarxr9tb6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Mon, 24 Jun 2019 08:09:27 -0700 Received: from SC-EXCH01.marvell.com (10.93.176.81) by SC-EXCH04.marvell.com (10.93.176.84) with Microsoft SMTP Server (TLS) id 15.0.1367.3; Mon, 24 Jun 2019 08:09:26 -0700 Received: from NAM03-BY2-obe.outbound.protection.outlook.com (104.47.42.57) by SC-EXCH01.marvell.com (10.93.176.81) with Microsoft SMTP Server (TLS) id 15.0.1367.3 via Frontend Transport; Mon, 24 Jun 2019 08:09:26 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=marvell.onmicrosoft.com; s=selector2-marvell-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fGiKqzPfUc9AXTHSQs9EVcWX6jy3P7JI7GKuDwheurI=; b=X8AieWewTvPsg2KjquFqgxlERr3aYybwMejjJy4RaUii+fRytIFor0ZLNz6R2U5seg7BvZ5zS6uw0IHzvtVLKyGA9buoxCqsHNw66OabqSGWKChwajRPZ792bIFKLR3tX4MGFKCNngG/7P01tyG6HU4NFG9IX/jz5d0PUHMo5a4= Received: from MN2PR18MB3408.namprd18.prod.outlook.com (10.255.238.217) by MN2PR18MB3197.namprd18.prod.outlook.com (10.255.236.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.2008.13; Mon, 24 Jun 2019 15:09:26 +0000 Received: from MN2PR18MB3408.namprd18.prod.outlook.com ([fe80::d3:794c:1b94:cf3]) by MN2PR18MB3408.namprd18.prod.outlook.com ([fe80::d3:794c:1b94:cf3%4]) with mapi id 15.20.2008.014; Mon, 24 Jun 2019 15:09:26 +0000 From: Robert Richter To: Borislav Petkov , James Morse , "Mauro Carvalho Chehab" CC: "linux-edac@vger.kernel.org" , "linux-kernel@vger.kernel.org" , Robert Richter Subject: [PATCH v2 15/24] EDAC, ghes: Extract numa node information for each dimm Thread-Topic: [PATCH v2 15/24] EDAC, ghes: Extract numa node information for each dimm Thread-Index: AQHVKp7PrQ3keExIu02YFGQ2PMgwJg== Date: Mon, 24 Jun 2019 15:09:25 +0000 Message-ID: <20190624150758.6695-16-rrichter@marvell.com> References: <20190624150758.6695-1-rrichter@marvell.com> In-Reply-To: <20190624150758.6695-1-rrichter@marvell.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-clientproxiedby: HE1P190CA0035.EURP190.PROD.OUTLOOK.COM (2603:10a6:7:52::24) To MN2PR18MB3408.namprd18.prod.outlook.com (2603:10b6:208:16c::25) x-ms-exchange-messagesentrepresentingtype: 1 x-mailer: git-send-email 2.20.1 x-originating-ip: [92.254.182.202] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: c4a9dddb-7085-4244-8bb2-08d6f8b5f23b x-microsoft-antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600148)(711020)(4605104)(1401327)(2017052603328)(7193020); SRVR:MN2PR18MB3197; x-ms-traffictypediagnostic: MN2PR18MB3197: x-microsoft-antispam-prvs: x-ms-oob-tlc-oobclassifiers: OLM:2887; x-forefront-prvs: 007814487B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(366004)(136003)(396003)(346002)(376002)(199004)(189003)(36756003)(110136005)(107886003)(54906003)(81156014)(81166006)(68736007)(8676002)(25786009)(5660300002)(8936002)(305945005)(7736002)(4326008)(1076003)(186003)(6116002)(14454004)(476003)(486006)(2616005)(11346002)(256004)(14444005)(446003)(3846002)(50226002)(26005)(2906002)(52116002)(76176011)(71200400001)(71190400001)(99286004)(386003)(6506007)(102836004)(478600001)(316002)(86362001)(6512007)(66066001)(53936002)(6486002)(73956011)(66946007)(66476007)(66556008)(64756008)(66446008)(6436002); DIR:OUT; SFP:1101; SCL:1; SRVR:MN2PR18MB3197; H:MN2PR18MB3408.namprd18.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; A:1; MX:1; received-spf: None (protection.outlook.com: marvell.com does not designate permitted sender hosts) x-ms-exchange-senderadcheck: 1 x-microsoft-antispam-message-info: Jk+lQP38n/+InFfn3SYvE+rahkc5CtHtaqlAdTQkcQkdcAWRmdNHNP5me/DZB5vaTMA3IqTNNJrhXsLL/dIt/VEjZXBexQb7RJSg2qU3254Eptzp/Z5RPvvNnM7sqRS+thg8ct2T1GJvV/zEJScL5ur5Jas0gZIAXm/Shes4cNDbJzcrDuEQF+RXMTja8f1SEAV3MUsJsQjJdLYMJIdQp2aaxMsLD8UdXa5pSg/d0xsfVNO1dganmDBfTjzF0HV9CGiN/u75GutuWdS5vyNYcwPsYnBRysG9uGcYQEmbmjDaXM8/jGi98pvRic7iJsy2G4m5o+csj/m+p4HVx4Ko+2q/JlJbDxEBaJA3aFFcMlKIXulK8rurUMz6jzVssXd7YMze+lpSuiZ+wFKYuJESMhtujVDX/aUD02l9u6Wn9kw= MIME-Version: 1.0 X-MS-Exchange-CrossTenant-Network-Message-Id: c4a9dddb-7085-4244-8bb2-08d6f8b5f23b X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jun 2019 15:09:25.9659 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 70e1fb47-1155-421d-87fc-2e58f638b6e0 X-MS-Exchange-CrossTenant-mailboxtype: HOSTED X-MS-Exchange-CrossTenant-userprincipalname: rrichter@marvell.com X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR18MB3197 X-OriginatorOrg: marvell.com X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-06-24_10:, , signatures=0 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org In a later patch we want to have one mc device per node. This patch extracts the numa node information for each dimm. This is done by collecting the physical address ranges from the DMI table (Memory Array Mapped Address - Type 19 of SMBIOS spec). The node information for a physical address is already know to a numa aware system (e.g. by using the ACPI _PXM method or the ACPI SRAT table), so based on the PA we can assign the node id to the dimms. A fallback that disables numa is implemented in case the node information is inconsistent. E.g., on a ThunderX2 system the following node mappings are found based on the DMI table: EDAC DEBUG: mem_info_setup: DIMM0: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM1: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM2: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM3: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM4: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM5: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM6: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM7: Found mem range [0x0000008800000000-0x0000009ffcffffff] on node 0 EDAC DEBUG: mem_info_setup: DIMM8: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM9: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM10: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM11: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM12: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM13: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM14: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 EDAC DEBUG: mem_info_setup: DIMM15: Found mem range [0x0000009ffd000000-0x000000bffcffffff] on node 1 Signed-off-by: Robert Richter --- drivers/edac/ghes_edac.c | 98 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 97 insertions(+), 1 deletion(-) -- 2.20.1 diff --git a/drivers/edac/ghes_edac.c b/drivers/edac/ghes_edac.c index 44bfb499b147..793362bea044 100644 --- a/drivers/edac/ghes_edac.c +++ b/drivers/edac/ghes_edac.c @@ -65,14 +65,32 @@ struct memdev_dmi_entry { u16 conf_mem_clk_speed; } __attribute__((__packed__)); +/* Memory Array Mapped Address - Type 19 of SMBIOS spec */ +struct memarr_dmi_entry { + u8 type; + u8 length; + u16 handle; + u32 start; + u32 end; + u16 phys_mem_array_handle; + u8 partition_width; + u64 ext_start; + u64 ext_end; +} __attribute__((__packed__)); + struct ghes_dimm_info { struct dimm_info dimm_info; int idx; + int numa_node; + phys_addr_t start; + phys_addr_t end; + u16 phys_handle; }; struct ghes_mem_info { - int num_dimm; + int num_dimm; struct ghes_dimm_info *dimms; + int dimms_per_node[MAX_NUMNODES]; }; static struct ghes_mem_info mem_info; @@ -108,12 +126,52 @@ static int ghes_dimm_info_init(int num) for_each_dimm(dimm) { dimm->idx = idx; + dimm->numa_node = NUMA_NO_NODE; idx++; } return 0; } +static void ghes_edac_set_nid(const struct dmi_header *dh, void *arg) +{ + struct memarr_dmi_entry *entry = (struct memarr_dmi_entry *)dh; + struct ghes_dimm_info *dimm; + phys_addr_t start, end; + int nid; + + if (dh->type != DMI_ENTRY_MEM_ARRAY_MAPPED_ADDR) + return; + + /* only support SMBIOS 2.7+ */ + if (entry->length < sizeof(*entry)) + return; + + if (entry->start == 0xffffffff) + start = entry->ext_start; + else + start = entry->start; + if (entry->end == 0xffffffff) + end = entry->ext_end; + else + end = entry->end; + + if (!pfn_valid(PHYS_PFN(start))) + return; + + nid = pfn_to_nid(PHYS_PFN(start)); + if (nid < 0 || nid >= MAX_NUMNODES || !node_possible(nid)) + nid = NUMA_NO_NODE; + + for_each_dimm(dimm) { + if (entry->phys_mem_array_handle == dimm->phys_handle) { + dimm->numa_node = nid; + dimm->start = start; + dimm->end = end; + } + } +} + static int get_dimm_smbios_index(u16 handle) { struct mem_ctl_info *mci = ghes_pvt->mci; @@ -135,6 +193,8 @@ static void ghes_edac_dmidecode(const struct dmi_header *dh, void *arg) struct dimm_info *dimm = &mi->dimm_info; u16 rdr_mask = BIT(7) | BIT(13); + mi->phys_handle = entry->phys_mem_array_handle; + if (entry->size == 0xffff) { pr_info("Can't get DIMM%i size\n", mi->idx); dimm->nr_pages = MiB_TO_PAGES(32);/* Unknown */ @@ -224,8 +284,23 @@ static void ghes_edac_dmidecode(const struct dmi_header *dh, void *arg) } } +static void mem_info_disable_numa(void) +{ + struct ghes_dimm_info *dimm; + + for_each_dimm(dimm) { + if (dimm->numa_node != NUMA_NO_NODE) + mem_info.dimms_per_node[dimm->numa_node] = 0; + dimm->numa_node = 0; + } + + mem_info.dimms_per_node[0] = mem_info.num_dimm; +} + static int mem_info_setup(void) { + struct ghes_dimm_info *dimm; + bool enable_numa = true; int num = 0; int idx = 0; int ret; @@ -238,6 +313,25 @@ static int mem_info_setup(void) return ret; dmi_walk(ghes_edac_dmidecode, &idx); + dmi_walk(ghes_edac_set_nid, NULL); + + for_each_dimm(dimm) { + if (dimm->numa_node == NUMA_NO_NODE) + enable_numa = false; + else + mem_info.dimms_per_node[dimm->numa_node]++; + + edac_dbg(1, "DIMM%i: Found mem range [%pa-%pa] on node %d\n", + dimm->idx, &dimm->start, &dimm->end, dimm->numa_node); + } + + if (enable_numa) + return 0; + + /* something went wrong, disable numa */ + if (num_possible_nodes() > 1) + pr_warn("Can't get numa info, disabling numa\n"); + mem_info_disable_numa(); return 0; } @@ -258,6 +352,8 @@ static int mem_info_setup_fake(void) dimm->dtype = DEV_UNKNOWN; dimm->edac_mode = EDAC_SECDED; + mem_info_disable_numa(); + return 0; }