From patchwork Tue Jun 12 11:36:38 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Julien Grall X-Patchwork-Id: 138334 Delivered-To: patch@linaro.org Received: by 2002:a2e:970d:0:0:0:0:0 with SMTP id r13-v6csp5235090lji; Tue, 12 Jun 2018 04:39:36 -0700 (PDT) X-Google-Smtp-Source: ADUXVKLoIPphfz0CwUtoxUCG4i5wWwSZcI1lR9zq2ylXChwj1zHI24Ih874uU0jii+LTwKlJ22+D X-Received: by 2002:a6b:b846:: with SMTP id i67-v6mr2946653iof.21.1528803576656; Tue, 12 Jun 2018 04:39:36 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1528803576; cv=none; d=google.com; s=arc-20160816; b=mJ3gRFdtwVOYges0lcO6rDcgM/Stgm/dLcfLTlIj/SVjXxYzYC40/ayv1y6HmdlUJ3 luIvRkpchopBq2zYY7i4hZrFHBKLfMw6WRLvUdyaV58YlTCOPJzmHWOjBZecvyr38ZuZ 8n/hzwttVpZ1+HXEsxA3tZoIONxugVcvrTM5SidQSozR8xjqR7aMCym+dyq5jRErGIPx AM8cdHzQWo6SfHY1b/9eDDOV1YeV07OQi8bTmaEHzf/kPzl7TdelZ1iG+dkhYD/dzi8g lm5UmpuMUyIMh0Cn3Rwm6csUxqFDKG+XUJQ4g5t4zVQqSNic/npI/VzwIxDmJh2Z10OE qsrQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:content-transfer-encoding:mime-version:cc :list-subscribe:list-help:list-post:list-unsubscribe:list-id :precedence:subject:references:in-reply-to:message-id:date:to:from :arc-authentication-results; bh=TOWGXZUFkWM4cifKOWg8lYWbqud3GPyZRBgJQT0NJ2A=; b=QUq+qaEz0lX0XHcZ5BIaMkW8kzTjhrEhu3bPsiOt0Aq4VZLqBv9S2u8p2Cokam8B/Z pnsICF1AGkSE1N81N2qrQ9VHDFIajpM13/xRXry1aDCaJ6msPc6Qs14oxEJ469pgkQ12 G4N7dzojec2H/kai+KNGliHqrQRCQb+LWcAuf48oUws+aDQl697NQw8sDSfzPOsczG1Z s+A2AK/8JxuPXnDgCgfksTW5MU7RXtTML54eQRxC3k9q0/mMz3uHwW86IJLK2G8SF6kx 1UPJKY8p84mDKWLwi6tCS0nKim9MVnPY2aR4jvHtgHFZ4kvrKUFTrCKO53cBmJdakpmL dJNQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Return-Path: Received: from lists.xenproject.org (lists.xenproject.org. [192.237.175.120]) by mx.google.com with ESMTPS id y196-v6si702454iod.32.2018.06.12.04.39.36 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Tue, 12 Jun 2018 04:39:36 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of xen-devel-bounces@lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1fShbi-0000rD-CC; Tue, 12 Jun 2018 11:37:02 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1fShbg-0000qX-DX for xen-devel@lists.xenproject.org; Tue, 12 Jun 2018 11:37:00 +0000 X-Inumbo-ID: edf55b21-6e34-11e8-bc1d-65256ead4e3a Received: from foss.arm.com (unknown [217.140.101.70]) by us1-amaz-eas1.inumbo.com (Halon) with ESMTP id edf55b21-6e34-11e8-bc1d-65256ead4e3a; Tue, 12 Jun 2018 11:37:04 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9BA4B1435; Tue, 12 Jun 2018 04:36:59 -0700 (PDT) Received: from e108454-lin.cambridge.arm.com (e108454-lin.cambridge.arm.com [10.1.206.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id B1BB83F318; Tue, 12 Jun 2018 04:36:58 -0700 (PDT) From: Julien Grall To: xen-devel@lists.xenproject.org Date: Tue, 12 Jun 2018 12:36:38 +0100 Message-Id: <20180612113643.32020-9-julien.grall@arm.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20180612113643.32020-1-julien.grall@arm.com> References: <20180612113643.32020-1-julien.grall@arm.com> Subject: [Xen-devel] [PATCH v3 08/13] xen/arm: alternatives: Add dynamic patching feature X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: andre.przywara@arm.com, Julien Grall , sstabellini@kernel.org MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This is based on the Linux commit dea5e2a4c5bc "arm64: alternatives: Add dynamic patching feature" written by Marc Zyngier: We've so far relied on a patching infrastructure that only gave us a single alternative, without any way to provide a range of potential replacement instructions. For a single feature, this is an all or nothing thing. It would be interesting to have a more flexible grained way of patching the kernel though, where we could dynamically tune the code that gets injected. In order to achive this, let's introduce a new form of dynamic patching, assiciating a callback to a patching site. This callback gets source and target locations of the patching request, as well as the number of instructions to be patched. Dynamic patching is declared with the new ALTERNATIVE_CB and alternative_cb directives: asm volatile(ALTERNATIVE_CB("mov %0, #0\n", callback) : "r" (v)); or alternative_cb callback mov x0, #0 alternative_cb_end where callback is the C function computing the alternative. Reviewed-by: Christoffer Dall Reviewed-by: Catalin Marinas Signed-off-by: Marc Zyngier This is part of XSA-263. Signed-off-by: Julien Grall Acked-by: Stefano Stabellini --- Changes in v2: - Fix typo in the commit message - Add Stefano's acked-by --- xen/arch/arm/alternative.c | 48 +++++++++++++++++++++++++++++---------- xen/include/asm-arm/alternative.h | 44 +++++++++++++++++++++++++++++++---- 2 files changed, 75 insertions(+), 17 deletions(-) diff --git a/xen/arch/arm/alternative.c b/xen/arch/arm/alternative.c index 936cf04956..52ed7edf69 100644 --- a/xen/arch/arm/alternative.c +++ b/xen/arch/arm/alternative.c @@ -30,6 +30,8 @@ #include #include #include +/* XXX: Move ARCH_PATCH_INSN_SIZE out of livepatch.h */ +#include #include /* Override macros from asm/page.h to make them work with mfn_t */ @@ -94,6 +96,23 @@ static u32 get_alt_insn(const struct alt_instr *alt, return insn; } +static void patch_alternative(const struct alt_instr *alt, + const uint32_t *origptr, + uint32_t *updptr, int nr_inst) +{ + const uint32_t *replptr; + unsigned int i; + + replptr = ALT_REPL_PTR(alt); + for ( i = 0; i < nr_inst; i++ ) + { + uint32_t insn; + + insn = get_alt_insn(alt, origptr + i, replptr + i); + updptr[i] = cpu_to_le32(insn); + } +} + /* * The region patched should be read-write to allow __apply_alternatives * to replacing the instructions when necessary. @@ -105,33 +124,38 @@ static int __apply_alternatives(const struct alt_region *region, paddr_t update_offset) { const struct alt_instr *alt; - const u32 *replptr, *origptr; + const u32 *origptr; u32 *updptr; + alternative_cb_t alt_cb; printk(XENLOG_INFO "alternatives: Patching with alt table %p -> %p\n", region->begin, region->end); for ( alt = region->begin; alt < region->end; alt++ ) { - u32 insn; - int i, nr_inst; + int nr_inst; - if ( !cpus_have_cap(alt->cpufeature) ) + /* Use ARM_CB_PATCH as an unconditional patch */ + if ( alt->cpufeature < ARM_CB_PATCH && + !cpus_have_cap(alt->cpufeature) ) continue; - BUG_ON(alt->alt_len != alt->orig_len); + if ( alt->cpufeature == ARM_CB_PATCH ) + BUG_ON(alt->alt_len != 0); + else + BUG_ON(alt->alt_len != alt->orig_len); origptr = ALT_ORIG_PTR(alt); updptr = (void *)origptr + update_offset; - replptr = ALT_REPL_PTR(alt); - nr_inst = alt->alt_len / sizeof(insn); + nr_inst = alt->orig_len / ARCH_PATCH_INSN_SIZE; - for ( i = 0; i < nr_inst; i++ ) - { - insn = get_alt_insn(alt, origptr + i, replptr + i); - *(updptr + i) = cpu_to_le32(insn); - } + if ( alt->cpufeature < ARM_CB_PATCH ) + alt_cb = patch_alternative; + else + alt_cb = ALT_REPL_PTR(alt); + + alt_cb(alt, origptr, updptr, nr_inst); /* Ensure the new instructions reached the memory and nuke */ clean_and_invalidate_dcache_va_range(origptr, diff --git a/xen/include/asm-arm/alternative.h b/xen/include/asm-arm/alternative.h index 4e33d1cdf7..9b4b02811b 100644 --- a/xen/include/asm-arm/alternative.h +++ b/xen/include/asm-arm/alternative.h @@ -3,6 +3,8 @@ #include +#define ARM_CB_PATCH ARM_NCAPS + #ifndef __ASSEMBLY__ #include @@ -18,16 +20,24 @@ struct alt_instr { }; /* Xen: helpers used by common code. */ -#define __ALT_PTR(a,f) ((u32 *)((void *)&(a)->f + (a)->f)) +#define __ALT_PTR(a,f) ((void *)&(a)->f + (a)->f) #define ALT_ORIG_PTR(a) __ALT_PTR(a, orig_offset) #define ALT_REPL_PTR(a) __ALT_PTR(a, alt_offset) +typedef void (*alternative_cb_t)(const struct alt_instr *alt, + const uint32_t *origptr, uint32_t *updptr, + int nr_inst); + void __init apply_alternatives_all(void); int apply_alternatives(const struct alt_instr *start, const struct alt_instr *end); -#define ALTINSTR_ENTRY(feature) \ +#define ALTINSTR_ENTRY(feature, cb) \ " .word 661b - .\n" /* label */ \ + " .if " __stringify(cb) " == 0\n" \ " .word 663f - .\n" /* new instruction */ \ + " .else\n" \ + " .word " __stringify(cb) "- .\n" /* callback */ \ + " .endif\n" \ " .hword " __stringify(feature) "\n" /* feature bit */ \ " .byte 662b-661b\n" /* source len */ \ " .byte 664f-663f\n" /* replacement len */ @@ -45,15 +55,18 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en * but most assemblers die if insn1 or insn2 have a .inst. This should * be fixed in a binutils release posterior to 2.25.51.0.2 (anything * containing commit 4e4d08cf7399b606 or c1baaddf8861). + * + * Alternatives with callbacks do not generate replacement instructions. */ -#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled) \ +#define __ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg_enabled, cb) \ ".if "__stringify(cfg_enabled)" == 1\n" \ "661:\n\t" \ oldinstr "\n" \ "662:\n" \ ".pushsection .altinstructions,\"a\"\n" \ - ALTINSTR_ENTRY(feature) \ + ALTINSTR_ENTRY(feature,cb) \ ".popsection\n" \ + " .if " __stringify(cb) " == 0\n" \ ".pushsection .altinstr_replacement, \"a\"\n" \ "663:\n\t" \ newinstr "\n" \ @@ -61,11 +74,17 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en ".popsection\n\t" \ ".org . - (664b-663b) + (662b-661b)\n\t" \ ".org . - (662b-661b) + (664b-663b)\n" \ + ".else\n\t" \ + "663:\n\t" \ + "664:\n\t" \ + ".endif\n" \ ".endif\n" #define _ALTERNATIVE_CFG(oldinstr, newinstr, feature, cfg, ...) \ - __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg)) + __ALTERNATIVE_CFG(oldinstr, newinstr, feature, IS_ENABLED(cfg), 0) +#define ALTERNATIVE_CB(oldinstr, cb) \ + __ALTERNATIVE_CFG(oldinstr, "NOT_AN_INSTRUCTION", ARM_CB_PATCH, 1, cb) #else #include @@ -126,6 +145,14 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en 663: .endm +.macro alternative_cb cb + .set .Lasm_alt_mode, 0 + .pushsection .altinstructions, "a" + altinstruction_entry 661f, \cb, ARM_CB_PATCH, 662f-661f, 0 + .popsection +661: +.endm + /* * Complete an alternative code sequence. */ @@ -135,6 +162,13 @@ int apply_alternatives(const struct alt_instr *start, const struct alt_instr *en .org . - (662b-661b) + (664b-663b) .endm +/* + * Callback-based alternative epilogue + */ +.macro alternative_cb_end +662: +.endm + #define _ALTERNATIVE_CFG(insn1, insn2, cap, cfg, ...) \ alternative_insn insn1, insn2, cap, IS_ENABLED(cfg)