From patchwork Fri May 2 15:20:35 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Salter X-Patchwork-Id: 29555 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ve0-f199.google.com (mail-ve0-f199.google.com [209.85.128.199]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id E294E202E7 for ; Fri, 2 May 2014 15:21:40 +0000 (UTC) Received: by mail-ve0-f199.google.com with SMTP id jy13sf16237707veb.10 for ; Fri, 02 May 2014 08:21:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:from:to:cc:subject :date:message-id:in-reply-to:references:sender:precedence:list-id :x-original-sender:x-original-authentication-results:mailing-list :list-post:list-help:list-archive:list-unsubscribe; bh=+WrP8/W+1jqBai9EQP3aj+8NXNlrfUgQz5pTE+3Mzrs=; b=VI2o9WHSVD+wu+hEHWL7StX5SWzUtRs4oT7LZqjPH/bdG6vXCgCxXfKXs1zwtFymup 0c1JclCqAPYbmmKHzrZwJGs01OclUu9fizrNnQ1myUGlaFTrbi5dKPuDYq9aKMTEj65s UGiYzi5CakiWh2LsxVl8U7jip1kRhP2ekQHdb9u+NrFMhGVDFoIM5DL0eQsLzHrRSCAX i52of0n1xydgXuwJq78bOjj1UtyPikcX55LOmdEhgM5YwNDv+QbC8yuGC1NxPpXM1C2o 80z7QkwG/POZ2nSx2DAdDIciISsrARDnmJ1bivqNme39z99Fm7utEnukaSKSHOCpeEFd zdGw== X-Gm-Message-State: ALoCoQnOxdoZh7qjybvtwu2CNWXHx7G1yDLPMnFnlK0UTAmFY490Pu2sh9hrti/ZKt3EOKDPt2dw X-Received: by 10.236.209.130 with SMTP id s2mr7683125yho.18.1399044100664; Fri, 02 May 2014 08:21:40 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.100.164 with SMTP id s33ls1573372qge.57.gmail; Fri, 02 May 2014 08:21:40 -0700 (PDT) X-Received: by 10.58.132.228 with SMTP id ox4mr442271veb.54.1399044100534; Fri, 02 May 2014 08:21:40 -0700 (PDT) Received: from mail-vc0-f173.google.com (mail-vc0-f173.google.com [209.85.220.173]) by mx.google.com with ESMTPS id ys8si6772859veb.214.2014.05.02.08.21.40 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Fri, 02 May 2014 08:21:40 -0700 (PDT) Received-SPF: none (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) client-ip=209.85.220.173; Received: by mail-vc0-f173.google.com with SMTP id ik5so5515299vcb.18 for ; Fri, 02 May 2014 08:21:40 -0700 (PDT) X-Received: by 10.58.112.98 with SMTP id ip2mr447593veb.35.1399044100453; Fri, 02 May 2014 08:21:40 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp110728vcb; Fri, 2 May 2014 08:21:40 -0700 (PDT) X-Received: by 10.182.33.131 with SMTP id r3mr16086228obi.40.1399044099851; Fri, 02 May 2014 08:21:39 -0700 (PDT) Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id sd1si24220542obb.154.2014.05.02.08.21.39; Fri, 02 May 2014 08:21:39 -0700 (PDT) Received-SPF: none (google.com: linux-kernel-owner@vger.kernel.org does not designate permitted sender hosts) client-ip=209.132.180.67; Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752716AbaEBPVN (ORCPT + 28 others); Fri, 2 May 2014 11:21:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:11825 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752596AbaEBPVJ (ORCPT ); Fri, 2 May 2014 11:21:09 -0400 Received: from int-mx02.intmail.prod.int.phx2.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) by mx1.redhat.com (8.14.4/8.14.4) with ESMTP id s42FKjBX030309 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Fri, 2 May 2014 11:20:45 -0400 Received: from deneb.redhat.com (ovpn-113-147.phx2.redhat.com [10.3.113.147]) by int-mx02.intmail.prod.int.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id s42FKhJQ018333; Fri, 2 May 2014 11:20:44 -0400 From: Mark Salter To: Catalin Marinas , Will Deacon Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Mark Salter Subject: [PATCH 2/2] arm64: fix soft lockup due to large tlb flush range Date: Fri, 2 May 2014 11:20:35 -0400 Message-Id: <1399044035-11274-3-git-send-email-msalter@redhat.com> In-Reply-To: <1399044035-11274-1-git-send-email-msalter@redhat.com> References: <1399044035-11274-1-git-send-email-msalter@redhat.com> X-Scanned-By: MIMEDefang 2.67 on 10.5.11.12 Sender: linux-kernel-owner@vger.kernel.org Precedence: list List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: msalter@redhat.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: patch+caf_=patchwork-forward=linaro.org@linaro.org does not designate permitted sender hosts) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Post: , List-Help: , List-Archive: List-Unsubscribe: , Under certain loads, this soft lockup has been observed: BUG: soft lockup - CPU#2 stuck for 22s! [ip6tables:1016] Modules linked in: ip6t_rpfilter ip6t_REJECT cfg80211 rfkill xt_conntrack ebtable_nat ebtable_broute bridge stp llc ebtable_filter ebtables ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw ip6table_filter ip6_tables iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw vfat fat efivarfs xfs libcrc32c CPU: 2 PID: 1016 Comm: ip6tables Not tainted 3.13.0-0.rc7.30.sa2.aarch64 #1 task: fffffe03e81d1400 ti: fffffe03f01f8000 task.ti: fffffe03f01f8000 PC is at __cpu_flush_kern_tlb_range+0xc/0x40 LR is at __purge_vmap_area_lazy+0x28c/0x3ac pc : [] lr : [] pstate: 80000145 sp : fffffe03f01fbb70 x29: fffffe03f01fbb70 x28: fffffe03f01f8000 x27: fffffe0000b19000 x26: 00000000000000d0 x25: 000000000000001c x24: fffffe03f01fbc50 x23: fffffe03f01fbc58 x22: fffffe03f01fbc10 x21: fffffe0000b2a3f8 x20: 0000000000000802 x19: fffffe0000b2a3c8 x18: 000003fffdf52710 x17: 000003ff9d8bb910 x16: fffffe000050fbfc x15: 0000000000005735 x14: 000003ff9d7e1a5c x13: 0000000000000000 x12: 000003ff9d7e1a5c x11: 0000000000000007 x10: fffffe0000c09af0 x9 : fffffe0000ad1000 x8 : 000000000000005c x7 : fffffe03e8624000 x6 : 0000000000000000 x5 : 0000000000000000 x4 : 0000000000000000 x3 : fffffe0000c09cc8 x2 : 0000000000000000 x1 : 000fffffdfffca80 x0 : 000fffffcd742150 The __cpu_flush_kern_tlb_range() function looks like: ENTRY(__cpu_flush_kern_tlb_range) dsb sy lsr x0, x0, #12 lsr x1, x1, #12 1: tlbi vaae1is, x0 add x0, x0, #1 cmp x0, x1 b.lo 1b dsb sy isb ret ENDPROC(__cpu_flush_kern_tlb_range) The above soft lockup shows the PC at tlbi insn with: x0 = 0x000fffffcd742150 x1 = 0x000fffffdfffca80 So __cpu_flush_kern_tlb_range has 0x128ba930 tlbi flushes left after it has already been looping for 23 seconds!. Looking up one frame at __purge_vmap_area_lazy(), there is: ... list_for_each_entry_rcu(va, &vmap_area_list, list) { if (va->flags & VM_LAZY_FREE) { if (va->va_start < *start) *start = va->va_start; if (va->va_end > *end) *end = va->va_end; nr += (va->va_end - va->va_start) >> PAGE_SHIFT; list_add_tail(&va->purge_list, &valist); va->flags |= VM_LAZY_FREEING; va->flags &= ~VM_LAZY_FREE; } } ... if (nr || force_flush) flush_tlb_kernel_range(*start, *end); So if two areas are being freed, the range passed to flush_tlb_kernel_range() may be as large as the vmalloc space. For arm64, this is ~240GB for 4k pagesize and ~2TB for 64kpage size. This patch works around this problem by adding a loop limit. If the range is larger than the limit, use flush_tlb_all() rather than flushing based on individual pages. The limit chosen is arbitrary and would be better if based on the actual size of the tlb. I looked through the ARM ARM but didn't see any easy way to get the actual tlb size, so for now the arbitrary limit is better than the soft lockup. Signed-off-by: Mark Salter --- arch/arm64/include/asm/tlbflush.h | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/tlbflush.h b/arch/arm64/include/asm/tlbflush.h index 8b48203..34ea52a 100644 --- a/arch/arm64/include/asm/tlbflush.h +++ b/arch/arm64/include/asm/tlbflush.h @@ -99,10 +99,32 @@ static inline void flush_tlb_page(struct vm_area_struct *vma, } /* - * Convert calls to our calling convention. + * The flush by range functions may take a very large range. + * If we need to invalidate a large range, it may be better + * to invalidate all tlb entries at once rather than looping + * through and invalidating individual entries. + * + * Here, we just use a fixed (arbitrary) number. It would be + * better if this was based on the actual size of the tlb... */ -#define flush_tlb_range(vma,start,end) __cpu_flush_user_tlb_range(start,end,vma) -#define flush_tlb_kernel_range(s,e) __cpu_flush_kern_tlb_range(s,e) +#define MAX_TLB_LOOP 128 + +static inline void flush_tlb_range(struct vm_area_struct *vma, + unsigned long start, unsigned long end) +{ + if (((end - start) >> PAGE_SHIFT) < MAX_TLB_LOOP) + __cpu_flush_user_tlb_range(start, end, vma); + else + flush_tlb_mm(vma->vm_mm); +} + +static inline void flush_tlb_kernel_range(unsigned long start, unsigned long end) +{ + if (((end - start) >> PAGE_SHIFT) < MAX_TLB_LOOP) + __cpu_flush_kern_tlb_range(start, end); + else + flush_tlb_all(); +} /* * On AArch64, the cache coherency is handled via the set_pte_at() function.