From patchwork Sat Jan 19 20:05:19 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Arnd Bergmann X-Patchwork-Id: 14119 Return-Path: X-Original-To: patchwork@peony.canonical.com Delivered-To: patchwork@peony.canonical.com Received: from fiordland.canonical.com (fiordland.canonical.com [91.189.94.145]) by peony.canonical.com (Postfix) with ESMTP id 402C223F02 for ; Sat, 19 Jan 2013 20:06:29 +0000 (UTC) Received: from mail-vc0-f171.google.com (mail-vc0-f171.google.com [209.85.220.171]) by fiordland.canonical.com (Postfix) with ESMTP id D092EA18E47 for ; Sat, 19 Jan 2013 20:06:28 +0000 (UTC) Received: by mail-vc0-f171.google.com with SMTP id fk10so3154019vcb.16 for ; Sat, 19 Jan 2013 12:06:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:x-forwarded-to:x-forwarded-for:delivered-to:x-received :received-spf:from:to:date:user-agent:references:in-reply-to :mime-version:message-id:x-provags-id:cc:subject:x-beenthere :x-mailman-version:precedence:list-id:list-unsubscribe:list-archive :list-post:list-help:list-subscribe:content-type :content-transfer-encoding:sender:errors-to:x-gm-message-state; bh=d3v2LzMxaG8084bkF4bqU7Hzm5JnibQf/L8F5Cakv8Q=; b=HC61jlpkSedve+QhKC3Pt0FcmLw424G3Pza3upSInF/2dm4dhuhXaLDGGcsm8Hp2FR HVc+yOtjBWUy5hDvM7ffHCfJ1uS43oqMt2YoTRWyws39kTO607F64AJLLJggeOf4h4sj dvxgGpMHvis5yncObb5lW2fhgkYntxbQbj2KPZ2/EBrbm4FpxfgocagiZQIiySo6Q3AS h2HcquVBj5YtXaxj95XrFrulHKV4OYcUp0pUc/fNsZLC22sazp9JyKLbGhrjC0gMGwsk GtBZRiaAatu1M2rRoE4Xi71q0S004jNrQx8d60lLHQLum7hmjjD2URK55mCGM4xKCR64 Es5A== X-Received: by 10.52.70.205 with SMTP id o13mr12538146vdu.75.1358625988213; Sat, 19 Jan 2013 12:06:28 -0800 (PST) X-Forwarded-To: linaro-patchwork@canonical.com X-Forwarded-For: patch@linaro.org linaro-patchwork@canonical.com Delivered-To: patches@linaro.org Received: by 10.58.145.101 with SMTP id st5csp141977veb; Sat, 19 Jan 2013 12:06:27 -0800 (PST) X-Received: by 10.14.3.137 with SMTP id 9mr29941905eeh.2.1358625985514; Sat, 19 Jan 2013 12:06:25 -0800 (PST) Received: from mombin.canonical.com (mombin.canonical.com. [91.189.95.16]) by mx.google.com with ESMTP id r1si14606423eeo.75.2013.01.19.12.06.23; Sat, 19 Jan 2013 12:06:25 -0800 (PST) Received-SPF: neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) client-ip=91.189.95.16; Authentication-Results: mx.google.com; spf=neutral (google.com: 91.189.95.16 is neither permitted nor denied by best guess record for domain of linaro-mm-sig-bounces@lists.linaro.org) smtp.mail=linaro-mm-sig-bounces@lists.linaro.org Received: from localhost ([127.0.0.1] helo=mombin.canonical.com) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1TwegE-0004kR-Ps; Sat, 19 Jan 2013 20:06:18 +0000 Received: from moutng.kundenserver.de ([212.227.126.187]) by mombin.canonical.com with esmtp (Exim 4.71) (envelope-from ) id 1TwegD-0004kM-SL for linaro-mm-sig@lists.linaro.org; Sat, 19 Jan 2013 20:06:17 +0000 Received: from klappe2.localnet (HSI-KBW-46-223-90-92.hsi.kabel-badenwuerttemberg.de [46.223.90.92]) by mrelayeu.kundenserver.de (node=mreu2) with ESMTP (Nemesis) id 0M36tX-1T6TYJ1tf5-00sRod; Sat, 19 Jan 2013 21:05:25 +0100 From: Arnd Bergmann To: Soeren Moch Date: Sat, 19 Jan 2013 20:05:19 +0000 User-Agent: KMail/1.12.2 (Linux/3.7.0-7-generic; KDE/4.3.2; x86_64; ; ) References: <20121119144826.f59667b2.akpm@linux-foundation.org> <201301172026.45514.arnd@arndb.de> <50FABBED.1020905@web.de> In-Reply-To: <50FABBED.1020905@web.de> MIME-Version: 1.0 Message-Id: <201301192005.20093.arnd@arndb.de> X-Provags-ID: V02:K0:hpLIMFHhe9r79Fne8XRQlnUUXylLigmVGhwB9vqj3MU T+g/v0qhk2OcyO5QjZCQ0q69z7PEHJrV0oEj2zDlfzGbeFCCWu ZOTzcH4dzSDcv5f+qqc/C18qISWZzmvL5nyQrNgs+4V/Hkwp2e dx8/hq0ZZBD/zkaZUvmau6SE/cbdBpMKHG7CAhlxsODsu7SEZN wcPeOScN8jx4JetBAd12X+D5wWCa7pdL+aG3iAskb836hzN/bw 8Qz2YCxJOfb/OsZHcT4eMYWcxvpge6TDCdYIPwdK2FIrEeAo2I Gfq0+qkQVmGvoYzQsMswDezigNLR1FAzcCpwDHyMe6aYRYTvdi 0A/arZZ32mvPnkV9ZjBA= Cc: Thomas Petazzoni , Andrew Lunn , Jason Cooper , linux-arm-kernel@lists.infradead.org, Greg KH , linux-kernel@vger.kernel.org, Michal Hocko , linux-mm@kvack.org, Kyungmin Park , Mel Gorman , Andrew Morton , Sebastian Hesselbarth , linaro-mm-sig@lists.linaro.org, KAMEZAWA Hiroyuki Subject: Re: [Linaro-mm-sig] [PATCH v2] mm: dmapool: use provided gfp flags for all dma_alloc_coherent() calls X-BeenThere: linaro-mm-sig@lists.linaro.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: "Unified memory management interest group." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: linaro-mm-sig-bounces@lists.linaro.org Errors-To: linaro-mm-sig-bounces@lists.linaro.org X-Gm-Message-State: ALoCoQnhE60bTQQKOjXemQl3IvdKR+J/YVfNGSBLKNXs1XvYAbd6SoS0UiyJ86xz45vtMSJDt4OE On Saturday 19 January 2013, Soeren Moch wrote: > What I can see in the log: a lot of coherent mappings from sata_mv and > orion_ehci, a few from mv643xx_eth, no other coherent mappings. > All coherent mappings are page aligned, some of them (from orion_ehci) > are not really small (as claimed in __alloc_from_pool). Right. Unfortunately, the output does not show which of the mappings are atomic, so we still need to look through those that can be atomic to understand what's going on. There are a few megabytes of coherent mappings in total according to the output, but it seems that a significant portion of them is atomic, which is a bit unexpected. > I don't believe in a memory leak. When I restart vdr (the application > utilizing the dvb sticks) then there is enough dma memory available > again. I found at least one source line that incorrectly uses an atomic allocation, in ehci_mem_init(): dma_alloc_coherent (ehci_to_hcd(ehci)->self.controller, ehci->periodic_size * sizeof(__le32), &ehci->periodic_dma, 0); The last argument is the GFP_ flag, which should never be zero, as that is implicit !wait. This function is called only once, so it is not the actual culprit, but there could be other instances where we accidentally allocate something as GFP_ATOMIC. The total number of allocations I found for each type are sata_mv: 66 pages (270336 bytes) mv643xx_eth: 4 pages == (16384 bytes) orion_ehci: 154 pages (630784 bytes) orion_ehci (atomic): 256 pages (1048576 bytes) from the distribution of the numbers, it seems that there is exactly 1 MB of data allocated between bus addresses 0x1f90000 and 0x1f9ffff, allocated in individual pages. This matches the size of your pool, so it's definitely something coming from USB, and no single other allocation, but it does not directly point to a specific line of code. One thing I found was that the ARM dma-mapping code seems buggy in the way that it does a bitwise and between the gfp mask and GFP_ATOMIC, which does not work because GFP_ATOMIC is defined by the absence of __GFP_WAIT. I believe we need the patch below, but it is not clear to me if that issue is related to your problem or now. 8<------- There is one more code path I could find, which is usb_submit_urb() => usb_hcd_submit_urb => ehci_urb_enqueue() => submit_async() => qh_append_tds() => qh_make(GFP_ATOMIC) => ehci_qh_alloc() => dma_pool_alloc() => pool_alloc_page() => dma_alloc_coherent() So even for a GFP_KERNEL passed into usb_submit_urb, the ehci driver causes the low-level allocation to be GFP_ATOMIC, because qh_append_tds() is called under a spinlock. If we have hundreds of URBs in flight, that will exhaust the pool rather quickly. Arnd diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c index 6b2fb87..c57975f 100644 --- a/arch/arm/mm/dma-mapping.c +++ b/arch/arm/mm/dma-mapping.c @@ -640,7 +641,7 @@ static void *__dma_alloc(struct device *dev, size_t size, dma_addr_t *handle, if (is_coherent || nommu()) addr = __alloc_simple_buffer(dev, size, gfp, &page); - else if (gfp & GFP_ATOMIC) + else if (!(gfp & __GFP_WAIT)) addr = __alloc_from_pool(size, &page); else if (!IS_ENABLED(CONFIG_CMA)) addr = __alloc_remap_buffer(dev, size, gfp, prot, &page, caller); @@ -1272,7 +1273,7 @@ static void *arm_iommu_alloc_attrs(struct device *dev, size_t size, *handle = DMA_ERROR_CODE; size = PAGE_ALIGN(size); - if (gfp & GFP_ATOMIC) + if (!(gfp & __GFP_WAIT)) return __iommu_alloc_atomic(dev, size, handle); pages = __iommu_alloc_buffer(dev, size, gfp, attrs);