From patchwork Tue Jan 24 14:07:56 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Srinivas Kandagatla X-Patchwork-Id: 92360 Delivered-To: patch@linaro.org Received: by 10.140.20.99 with SMTP id 90csp1730105qgi; Tue, 24 Jan 2017 06:09:51 -0800 (PST) X-Received: by 10.28.221.11 with SMTP id u11mr17294495wmg.75.1485266991344; Tue, 24 Jan 2017 06:09:51 -0800 (PST) Return-Path: Received: from bombadil.infradead.org (bombadil.infradead.org. [65.50.211.133]) by mx.google.com with ESMTPS id b9si22998941wrc.48.2017.01.24.06.09.51 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jan 2017 06:09:51 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 65.50.211.133 as permitted sender) client-ip=65.50.211.133; Authentication-Results: mx.google.com; dkim=neutral (body hash did not verify) header.i=@linaro.org; spf=pass (google.com: best guess record for domain of linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org designates 65.50.211.133 as permitted sender) smtp.mailfrom=linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cW1nA-0005ho-Mg; Tue, 24 Jan 2017 14:09:48 +0000 Received: from mail-wm0-f47.google.com ([74.125.82.47]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cW1mh-0005eE-CH for linux-arm-kernel@lists.infradead.org; Tue, 24 Jan 2017 14:09:24 +0000 Received: by mail-wm0-f47.google.com with SMTP id c206so211233804wme.0 for ; Tue, 24 Jan 2017 06:08:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:subject:to:cc:message-id:date:user-agent:mime-version :content-transfer-encoding; bh=Tx75uVq4gJ7pKERdFoIkKmjp0dUwFf8znPr0TBD3FTI=; b=iUH1CosQ+2IVU+7g+fMtLUZx1nFfmocCxqU4h5J3TVEbiBB3AeQIu8Gsjc2uedvpZz 9w06j4ZCnQ5G6gGe9XB+yyi3r3SKgaFjoOO8vvnYzyko7kgHn3XSqPiwP3FpmChXryIg SmbrtfHNWI9V3ZrishzeM5Xvypk3v+CKQFvXo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:subject:to:cc:message-id:date:user-agent :mime-version:content-transfer-encoding; bh=Tx75uVq4gJ7pKERdFoIkKmjp0dUwFf8znPr0TBD3FTI=; b=uJnFga4Y00gszCDAmOH5LKmb22c7OpoxUdrgUVYjW6lQwQpsiKw5YHO1mVjSDNFTAs rMXZyqHDj7PyGBLqaPXJukHhuzq81diKuj8QokhXqDYPcYui3/CvFU7j5dMEr3S9VNLv A4y2H50KHFflw3RZ9K/IUOP/SoxabmhrtWsjqU1fGDbtqiSg2Iow3PyPOn+hHbxVmXii rkyxgZptOZZCAYDkjLxDa2xz1HnNz3ecHSDWqz6oZ2Mpc3FUQZFwdJlWZ/rBd6sVO+5R pBblEaPDqq6eN2t+xk9v0N4d5AChLwVP32gwFmUold6kxxMVsAluE8ASj5dHjAYo/wuL Ue0w== X-Gm-Message-State: AIkVDXIh5wEgq58O01HoL326NVqdbZ6X6h2YNRKgwblQvUpHKKzll97vUPOLxZ6YT841o6Ja X-Received: by 10.28.46.74 with SMTP id u71mr17041918wmu.136.1485266877686; Tue, 24 Jan 2017 06:07:57 -0800 (PST) Received: from [192.168.1.180] (host-2-98-105-38.as13285.net. [2.98.105.38]) by smtp.googlemail.com with ESMTPSA id s26sm20091576wra.26.2017.01.24.06.07.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 24 Jan 2017 06:07:57 -0800 (PST) From: Srinivas Kandagatla Subject: pcie-designware: Incorrect programming of msi address register on some setups. To: Jingoo Han , Joao Pinto , Bjorn Helgaas Message-ID: <922851d0-a847-762a-5067-88c92d416d58@linaro.org> Date: Tue, 24 Jan 2017 14:07:56 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170124_060919_588272_03835E43 X-CRM114-Status: GOOD ( 15.45 ) X-Spam-Score: -1.5 (-) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-1.5 points) pts rule name description ---- ---------------------- -------------------------------------------------- 0.5 RCVD_IN_SORBS_SPAM RBL: SORBS: sender is a spam source [74.125.82.47 listed in dnsbl.sorbs.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [74.125.82.47 listed in list.dnswl.org] -0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [74.125.82.47 listed in wl.mailspike.net] -0.0 SPF_PASS SPF: sender matches SPF record -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 RCVD_IN_MSPIKE_WL Mailspike good senders X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-pci@vger.kernel.org, "linux-arm-kernel@lists.infradead.org" , Arnd Bergmann , Bjorn Andersson Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patch=linaro.org@lists.infradead.org Hi, Recently I hit a bug with pcie-designware driver with triggers a board reboot, this is my analysis so far on the issue. Issue is because designware driver is programming msi address which is above 32 address space eventhough one of the pcie endpoint indicated that it does not support 64 bit msi addresses. My setup is arm64 board DB820c with 3 root complexes, and the system has memory which spans across 4GB address space. First root complex is connected to a WLAN Chip (QCA6174) only supports 32 bit addresses. Second one is connected to a Ethernet Controller (ATL1c) value of MSI control register aka PCI_MSI_FLAGS (0x2) for device QCA6174 is 0x106 value of MSI control register aka PCI_MSI_FLAGS (0x2) for device ATL1C is 0x80 Now the problem is that on Synopsis IP, the MSI address register is programmed at two places 1> one at controller config dbi address space itself in PCIE_MSI_ADDR_LO(0x820) & PCIE_MSI_ADDR_HI (0x824) 2> at PCI_MSI_ADDRESS_LO (0x2) and PCI_MSI_ADDRESS_HI of the device cfg. If step 1 allocates msi address which is above 32 bit address space, then step 2 would truncate it to 32 bit address if it discovers endpoint which does not support 64 bit msi address. Then if endpoint tries to read/write this address as part of MSI mechanism it would fault and crash system. Am not sure how we can solve this correctly because we do not know about endpoints at step 1. On the other hand if we can force step 1 to allocate addresses with in 32 bit address space than it would work for both 32 bit and 64 bit. Has anyone hit this issue before? Do we already have a solution? Any suggestions? Below ugly Hack is what Am using to workaround this issue on my platform but am not sure how this will influence other platforms. ---------------------------->cut<------------------------------- ---------------------------->cut<------------------------------- Thanks, srini _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel diff --git a/drivers/pci/host/pcie-designware.c b/drivers/pci/host/pcie-designware.c index 4a81b72..d694e66 100644 --- a/drivers/pci/host/pcie-designware.c +++ b/drivers/pci/host/pcie-designware.c @@ -286,9 +286,8 @@ void dw_pcie_msi_init(struct pcie_port *pp) { u64 msi_target; - pp->msi_data = __get_free_pages(GFP_KERNEL, 0); + pp->msi_data = __get_free_pages(GFP_KERNEL | GFP_DMA, 0); msi_target = virt_to_phys((void *)pp->msi_data); /* program the msi_data */ dw_pcie_wr_own_conf(pp, PCIE_MSI_ADDR_LO, 4, (u32)(msi_target & 0xffffffff));