From patchwork Wed Mar 26 13:38:49 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ian Campbell X-Patchwork-Id: 27130 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-qa0-f71.google.com (mail-qa0-f71.google.com [209.85.216.71]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id DD0C420062 for ; Wed, 26 Mar 2014 13:44:47 +0000 (UTC) Received: by mail-qa0-f71.google.com with SMTP id j7sf3998931qaq.6 for ; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:from:to:date:message-id:in-reply-to :references:mime-version:cc:subject:precedence:list-id :list-unsubscribe:list-post:list-help:list-subscribe:sender :errors-to:x-original-sender:x-original-authentication-results :mailing-list:list-archive:content-type:content-transfer-encoding; bh=MI/jS+BP0SXbqKO2OOwiYGAaNC5ijqUBFmB3KDYSTi8=; b=OrlOO6d0x0tKcU2QESPxHpVHxzk0xYjwJuqUu7n3EJeIXm9upvUiz5/8jeRdYn8w2q NjYXWj9y2RYhRlvCB8UeZ/Eu6bFll4ndw9QpTV3XHXKyoryKfGlLHqPBEdUOLosZVEwv AxReVfdWSCkF1LlKdcpnm6Nr2tmaOMV8NndlDz3xIpNYe9LDyMNN7d6qXoqYNdh9SgXw qlFmTC075XQm5f+j0/cKbq/dg/6pap8fsCBi41djy6KVBC72x5Hx/7sZl4ePmaLDjaCs X489mBajHH0lTJ8gopWdihm/pKkvjSNjZMXUk2k8mPvaehncJq1rjIkOzU4Nq4B0Yxwh HS8g== X-Gm-Message-State: ALoCoQk2jIc3qzDRBVnlp42ZEE2phS172pZqRUOAxgzoLYWMMzmKb/tQLTrEyDMnDsSK2QDtFyfe X-Received: by 10.236.83.197 with SMTP id q45mr20236154yhe.10.1395841487701; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.34.233 with SMTP id l96ls641280qgl.90.gmail; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-Received: by 10.220.191.134 with SMTP id dm6mr61292519vcb.16.1395841487600; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) Received: from mail-ve0-f172.google.com (mail-ve0-f172.google.com [209.85.128.172]) by mx.google.com with ESMTPS id sh5si4637896vdc.86.2014.03.26.06.44.47 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 26 Mar 2014 06:44:47 -0700 (PDT) Received-SPF: neutral (google.com: 209.85.128.172 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=209.85.128.172; Received: by mail-ve0-f172.google.com with SMTP id jx11so2302846veb.17 for ; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-Received: by 10.220.191.134 with SMTP id dm6mr61292516vcb.16.1395841487494; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.78.9 with SMTP id i9csp47433vck; Wed, 26 Mar 2014 06:44:47 -0700 (PDT) X-Received: by 10.58.150.68 with SMTP id ug4mr103685veb.50.1395841486988; Wed, 26 Mar 2014 06:44:46 -0700 (PDT) Received: from lists.xen.org (lists.xen.org. [50.57.142.19]) by mx.google.com with ESMTPS id sv2si4637474vdc.93.2014.03.26.06.44.46 for (version=TLSv1 cipher=RC4-SHA bits=128/128); Wed, 26 Mar 2014 06:44:46 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xen.org designates 50.57.142.19 as permitted sender) client-ip=50.57.142.19; Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WSo6h-0006Ch-Mo; Wed, 26 Mar 2014 13:43:03 +0000 Received: from mail6.bemta14.messagelabs.com ([193.109.254.103]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1WSo6g-0006CJ-VW for xen-devel@lists.xen.org; Wed, 26 Mar 2014 13:43:03 +0000 Received: from [193.109.254.147:16111] by server-13.bemta-14.messagelabs.com id 95/8E-23211-669D2335; Wed, 26 Mar 2014 13:43:02 +0000 X-Env-Sender: Ian.Campbell@citrix.com X-Msg-Ref: server-11.tower-27.messagelabs.com!1395841373!4263562!4 X-Originating-IP: [66.165.176.63] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogNjYuMTY1LjE3Ni42MyA9PiAzMDYwNDg=\n X-StarScan-Received: X-StarScan-Version: 6.11.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 28910 invoked from network); 26 Mar 2014 13:43:01 -0000 Received: from smtp02.citrix.com (HELO SMTP02.CITRIX.COM) (66.165.176.63) by server-11.tower-27.messagelabs.com with RC4-SHA encrypted SMTP; 26 Mar 2014 13:43:01 -0000 X-IronPort-AV: E=Sophos;i="4.97,735,1389744000"; d="scan'208";a="113734825" Received: from accessns.citrite.net (HELO FTLPEX01CL01.citrite.net) ([10.9.154.239]) by FTLPIPO02.CITRIX.COM with ESMTP; 26 Mar 2014 13:43:00 +0000 Received: from norwich.cam.xci-test.com (10.80.248.129) by smtprelay.citrix.com (10.13.107.78) with Microsoft SMTP Server id 14.2.342.4; Wed, 26 Mar 2014 09:43:00 -0400 Received: from drall.uk.xensource.com ([10.80.16.71] helo=drall.uk.xensource.com.) by norwich.cam.xci-test.com with esmtp (Exim 4.72) (envelope-from ) id 1WSo2g-00074X-4i; Wed, 26 Mar 2014 13:38:54 +0000 From: Ian Campbell To: Date: Wed, 26 Mar 2014 13:38:49 +0000 Message-ID: <1395841133-2223-14-git-send-email-ian.campbell@citrix.com> X-Mailer: git-send-email 1.7.10.4 In-Reply-To: <1395841009.12547.11.camel@kazak.uk.xensource.com> References: <1395841009.12547.11.camel@kazak.uk.xensource.com> MIME-Version: 1.0 X-DLP: MIA1 Cc: julien.grall@linaro.org, tim@xen.org, Ian Campbell , stefano.stabellini@eu.citrix.com Subject: [Xen-devel] [PATCH v2 14/17] xen: arm64: assembly optimised mem* and str* X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: List-Unsubscribe: , List-Post: , List-Help: , List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: ian.campbell@citrix.com X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 209.85.128.172 is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 List-Archive: Taken from Linux v3.14-rc7. Signed-off-by: Ian Campbell Acked-by: Julien Grall --- xen/arch/arm/arm64/lib/Makefile | 2 ++ xen/arch/arm/arm64/lib/memchr.S | 43 +++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memcpy.S | 52 +++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memmove.S | 56 ++++++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/memset.S | 52 +++++++++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/strchr.S | 41 ++++++++++++++++++++++++++++ xen/arch/arm/arm64/lib/strrchr.S | 42 ++++++++++++++++++++++++++++ xen/include/asm-arm/string.h | 4 +-- 8 files changed, 290 insertions(+), 2 deletions(-) create mode 100644 xen/arch/arm/arm64/lib/memchr.S create mode 100644 xen/arch/arm/arm64/lib/memcpy.S create mode 100644 xen/arch/arm/arm64/lib/memmove.S create mode 100644 xen/arch/arm/arm64/lib/memset.S create mode 100644 xen/arch/arm/arm64/lib/strchr.S create mode 100644 xen/arch/arm/arm64/lib/strrchr.S diff --git a/xen/arch/arm/arm64/lib/Makefile b/xen/arch/arm/arm64/lib/Makefile index 32c02c4..9f3b236 100644 --- a/xen/arch/arm/arm64/lib/Makefile +++ b/xen/arch/arm/arm64/lib/Makefile @@ -1 +1,3 @@ +obj-y += memcpy.o memmove.o memset.o memchr.o obj-y += bitops.o find_next_bit.o +obj-y += strchr.o strrchr.o diff --git a/xen/arch/arm/arm64/lib/memchr.S b/xen/arch/arm/arm64/lib/memchr.S new file mode 100644 index 0000000..3cc1b01 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memchr.S @@ -0,0 +1,43 @@ +/* + * Based on arch/arm/lib/memchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find a character in an area of memory. + * + * Parameters: + * x0 - buf + * x1 - c + * x2 - n + * Returns: + * x0 - address of first occurrence of 'c' or 0 + */ +ENTRY(memchr) + and w1, w1, #0xff +1: subs x2, x2, #1 + b.mi 2f + ldrb w3, [x0], #1 + cmp w3, w1 + b.ne 1b + sub x0, x0, #1 + ret +2: mov x0, #0 + ret +ENDPROC(memchr) diff --git a/xen/arch/arm/arm64/lib/memcpy.S b/xen/arch/arm/arm64/lib/memcpy.S new file mode 100644 index 0000000..c8197c6 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memcpy.S @@ -0,0 +1,52 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Copy a buffer from src to dest (alignment handled by the hardware) + * + * Parameters: + * x0 - dest + * x1 - src + * x2 - n + * Returns: + * x0 - dest + */ +ENTRY(memcpy) + mov x4, x0 + subs x2, x2, #8 + b.mi 2f +1: ldr x3, [x1], #8 + subs x2, x2, #8 + str x3, [x4], #8 + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + ldr w3, [x1], #4 + sub x2, x2, #4 + str w3, [x4], #4 +3: adds x2, x2, #2 + b.mi 4f + ldrh w3, [x1], #2 + sub x2, x2, #2 + strh w3, [x4], #2 +4: adds x2, x2, #1 + b.mi 5f + ldrb w3, [x1] + strb w3, [x4] +5: ret +ENDPROC(memcpy) diff --git a/xen/arch/arm/arm64/lib/memmove.S b/xen/arch/arm/arm64/lib/memmove.S new file mode 100644 index 0000000..1bf0936 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memmove.S @@ -0,0 +1,56 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Move a buffer from src to test (alignment handled by the hardware). + * If dest <= src, call memcpy, otherwise copy in reverse order. + * + * Parameters: + * x0 - dest + * x1 - src + * x2 - n + * Returns: + * x0 - dest + */ +ENTRY(memmove) + cmp x0, x1 + b.ls memcpy + add x4, x0, x2 + add x1, x1, x2 + subs x2, x2, #8 + b.mi 2f +1: ldr x3, [x1, #-8]! + subs x2, x2, #8 + str x3, [x4, #-8]! + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + ldr w3, [x1, #-4]! + sub x2, x2, #4 + str w3, [x4, #-4]! +3: adds x2, x2, #2 + b.mi 4f + ldrh w3, [x1, #-2]! + sub x2, x2, #2 + strh w3, [x4, #-2]! +4: adds x2, x2, #1 + b.mi 5f + ldrb w3, [x1, #-1] + strb w3, [x4, #-1] +5: ret +ENDPROC(memmove) diff --git a/xen/arch/arm/arm64/lib/memset.S b/xen/arch/arm/arm64/lib/memset.S new file mode 100644 index 0000000..25a4fb6 --- /dev/null +++ b/xen/arch/arm/arm64/lib/memset.S @@ -0,0 +1,52 @@ +/* + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Fill in the buffer with character c (alignment handled by the hardware) + * + * Parameters: + * x0 - buf + * x1 - c + * x2 - n + * Returns: + * x0 - buf + */ +ENTRY(memset) + mov x4, x0 + and w1, w1, #0xff + orr w1, w1, w1, lsl #8 + orr w1, w1, w1, lsl #16 + orr x1, x1, x1, lsl #32 + subs x2, x2, #8 + b.mi 2f +1: str x1, [x4], #8 + subs x2, x2, #8 + b.pl 1b +2: adds x2, x2, #4 + b.mi 3f + sub x2, x2, #4 + str w1, [x4], #4 +3: adds x2, x2, #2 + b.mi 4f + sub x2, x2, #2 + strh w1, [x4], #2 +4: adds x2, x2, #1 + b.mi 5f + strb w1, [x4] +5: ret +ENDPROC(memset) diff --git a/xen/arch/arm/arm64/lib/strchr.S b/xen/arch/arm/arm64/lib/strchr.S new file mode 100644 index 0000000..9e265e4 --- /dev/null +++ b/xen/arch/arm/arm64/lib/strchr.S @@ -0,0 +1,41 @@ +/* + * Based on arch/arm/lib/strchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find the first occurrence of a character in a string. + * + * Parameters: + * x0 - str + * x1 - c + * Returns: + * x0 - address of first occurrence of 'c' or 0 + */ +ENTRY(strchr) + and w1, w1, #0xff +1: ldrb w2, [x0], #1 + cmp w2, w1 + ccmp w2, wzr, #4, ne + b.ne 1b + sub x0, x0, #1 + cmp w2, w1 + csel x0, x0, xzr, eq + ret +ENDPROC(strchr) diff --git a/xen/arch/arm/arm64/lib/strrchr.S b/xen/arch/arm/arm64/lib/strrchr.S new file mode 100644 index 0000000..3791754 --- /dev/null +++ b/xen/arch/arm/arm64/lib/strrchr.S @@ -0,0 +1,42 @@ +/* + * Based on arch/arm/lib/strrchr.S + * + * Copyright (C) 1995-2000 Russell King + * Copyright (C) 2013 ARM Ltd. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +#include + +/* + * Find the last occurrence of a character in a string. + * + * Parameters: + * x0 - str + * x1 - c + * Returns: + * x0 - address of last occurrence of 'c' or 0 + */ +ENTRY(strrchr) + mov x3, #0 + and w1, w1, #0xff +1: ldrb w2, [x0], #1 + cbz w2, 2f + cmp w2, w1 + b.ne 1b + sub x3, x0, #1 + b 1b +2: mov x0, x3 + ret +ENDPROC(strrchr) diff --git a/xen/include/asm-arm/string.h b/xen/include/asm-arm/string.h index 7d8b35a..3242762 100644 --- a/xen/include/asm-arm/string.h +++ b/xen/include/asm-arm/string.h @@ -3,8 +3,6 @@ #include -#if defined(CONFIG_ARM_32) - /* * We don't do inline string functions, since the * optimised inline asm versions are not small. @@ -29,6 +27,8 @@ extern void * memset(void *, int, __kernel_size_t); #define __HAVE_ARCH_MEMCHR extern void * memchr(const void *, int, __kernel_size_t); +#if defined(CONFIG_ARM_32) + extern void __memzero(void *ptr, __kernel_size_t n); #define memset(p,v,n) \