| Message ID | 1404137113-25101-1-git-send-email-julien.grall@linaro.org |
|---|---|
| State | Rejected, archived |
| Headers | show |
On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: > Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). > > Kernel are usually using 1MB (or 2MB) mapping for the early page table. If > the kernel doesn't relocate itself this would require to virtual address > starting at 0xXXXX8000. This is not part of the zImage protocol, but a > convenience for Linux after the decompressor stage. Linux is able to > load at any address in the memory and it will relocate itself to respect > it own constraint. Yes, because the boot protocol does not guarantee 2MB alignment, so if the OS wants to rely on that then it is obliged to take care of relocating itself. > Load the zImage at rambase address + 2MB to make life easier for other > kernel (such as FreeBSD, Mini-OS). Whether or not it is easier these OSes *must* be prepared to be loaded at any 4k aligned address. It sounds to me like you are hoping that these OSes can *rely* on 2MB alignment, which is not the case. If they make that assumption then they are buggy, regardless of whether it happens to currently work with some loader or not. > FWIW, this is already the case for DOM0. On that basis I'm more inclined to change dom0 to use 32kb alignment. Ian.
On 07/03/2014 11:01 AM, Ian Campbell wrote: > On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: >> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). >> >> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If >> the kernel doesn't relocate itself this would require to virtual address >> starting at 0xXXXX8000. This is not part of the zImage protocol, but a >> convenience for Linux after the decompressor stage. Linux is able to >> load at any address in the memory and it will relocate itself to respect >> it own constraint. > > Yes, because the boot protocol does not guarantee 2MB alignment, so if > the OS wants to rely on that then it is obliged to take care of > relocating itself. > >> Load the zImage at rambase address + 2MB to make life easier for other >> kernel (such as FreeBSD, Mini-OS). > > Whether or not it is easier these OSes *must* be prepared to be loaded > at any 4k aligned address. It sounds to me like you are hoping that > these OSes can *rely* on 2MB alignment, which is not the case. If they > make that assumption then they are buggy, regardless of whether it > happens to currently work with some loader or not. Using 4K alignment make impossible to use 1MB or 2MB mapping for early page table. Which make the code (usually in assembly) even harder to write or to impose relocation in the assembly code. >> FWIW, this is already the case for DOM0. > > On that basis I'm more inclined to change dom0 to use 32kb alignment. It's not really 32kb alignment but an address in 0xXXXX8000. If the kernel doesn't want to do relocation (because it's a pain to write in assembly code), it has to compile the kernel with a 0x8000 offset (see mini-os patch series). This solution, make the kernel more tight to Xen changes. I would prefer if we relax this change allowing easier support of new kernel to 2MB alignment. Regards,
On 03/07/14 11:12, Julien Grall wrote: > On 07/03/2014 11:01 AM, Ian Campbell wrote: >> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: >>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). >>> >>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If >>> the kernel doesn't relocate itself this would require to virtual address >>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a >>> convenience for Linux after the decompressor stage. Linux is able to >>> load at any address in the memory and it will relocate itself to respect >>> it own constraint. >> >> Yes, because the boot protocol does not guarantee 2MB alignment, so if >> the OS wants to rely on that then it is obliged to take care of >> relocating itself. >> >>> Load the zImage at rambase address + 2MB to make life easier for other >>> kernel (such as FreeBSD, Mini-OS). >> >> Whether or not it is easier these OSes *must* be prepared to be loaded >> at any 4k aligned address. It sounds to me like you are hoping that >> these OSes can *rely* on 2MB alignment, which is not the case. If they >> make that assumption then they are buggy, regardless of whether it >> happens to currently work with some loader or not. > > Using 4K alignment make impossible to use 1MB or 2MB mapping for early > page table. Which make the code (usually in assembly) even harder to > write or to impose relocation in the assembly code. This isn't the case for Mini-OS, at least. We already use a 1MB mapping with the 0x8000 offset just fine. The translation table (using 1MB sections) is 16K, which would fit nicely in the 0x8000 gap (although currently we don't use that space). Starting at 2MB would create an inconvenient 2MB of free memory just before the kernel, with no obvious way to tell xmalloc about it. >>> FWIW, this is already the case for DOM0. >> >> On that basis I'm more inclined to change dom0 to use 32kb alignment. > > It's not really 32kb alignment but an address in 0xXXXX8000. If the > kernel doesn't want to do relocation (because it's a pain to write in > assembly code), it has to compile the kernel with a 0x8000 offset (see > mini-os patch series). > > This solution, make the kernel more tight to Xen changes. I would prefer > if we relax this change allowing easier support of new kernel to 2MB > alignment. > > Regards, >
On 03/07/14 11:12, Julien Grall wrote: > On 07/03/2014 11:01 AM, Ian Campbell wrote: >> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: >>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). >>> >>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If >>> the kernel doesn't relocate itself this would require to virtual address >>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a >>> convenience for Linux after the decompressor stage. Linux is able to >>> load at any address in the memory and it will relocate itself to respect >>> it own constraint. >> >> Yes, because the boot protocol does not guarantee 2MB alignment, so if >> the OS wants to rely on that then it is obliged to take care of >> relocating itself. >> >>> Load the zImage at rambase address + 2MB to make life easier for other >>> kernel (such as FreeBSD, Mini-OS). >> >> Whether or not it is easier these OSes *must* be prepared to be loaded >> at any 4k aligned address. It sounds to me like you are hoping that >> these OSes can *rely* on 2MB alignment, which is not the case. If they >> make that assumption then they are buggy, regardless of whether it >> happens to currently work with some loader or not. > > Using 4K alignment make impossible to use 1MB or 2MB mapping for early > page table. Which make the code (usually in assembly) even harder to > write or to impose relocation in the assembly code. This isn't the case for Mini-OS, at least. We already use a 1MB mapping with the 0x8000 offset just fine. The translation table (using 1MB sections) is 16K, which would fit nicely in the 0x8000 gap (although currently we don't use that space). Starting at 2MB would create an inconvenient 2MB of free memory just before the kernel, with no obvious way to tell xmalloc about it. >>> FWIW, this is already the case for DOM0. >> >> On that basis I'm more inclined to change dom0 to use 32kb alignment. > > It's not really 32kb alignment but an address in 0xXXXX8000. If the > kernel doesn't want to do relocation (because it's a pain to write in > assembly code), it has to compile the kernel with a 0x8000 offset (see > mini-os patch series). > > This solution, make the kernel more tight to Xen changes. I would prefer > if we relax this change allowing easier support of new kernel to 2MB > alignment. > > Regards, >
On 07/03/2014 11:37 AM, Thomas Leonard wrote: >> Using 4K alignment make impossible to use 1MB or 2MB mapping for early >> page table. Which make the code (usually in assembly) even harder to >> write or to impose relocation in the assembly code. > > This isn't the case for Mini-OS, at least. We already use a 1MB mapping > with the 0x8000 offset just fine. The translation table (using 1MB > sections) is 16K, which would fit nicely in the 0x8000 gap (although > currently we don't use that space). As I said in Mini-OS series, the spec doesn't say anything about the start offset of the guest kernel for the Linux boot protocol. 0x8000 offset is very inconvenient for most of the other OS than Linux. As Linux is able to relocate itself, I don't see any known issue about starting at an 2MB-aligned address. > Starting at 2MB would create an inconvenient 2MB of free memory just > before the kernel, with no obvious way to tell xmalloc about it. That's a bug in Mini-OS in this case. Why can't you use the whole memory? Regards,
On Thu, 2014-07-03 at 11:12 +0100, Julien Grall wrote: > On 07/03/2014 11:01 AM, Ian Campbell wrote: > > On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: > >> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). > >> > >> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If > >> the kernel doesn't relocate itself this would require to virtual address > >> starting at 0xXXXX8000. This is not part of the zImage protocol, but a > >> convenience for Linux after the decompressor stage. Linux is able to > >> load at any address in the memory and it will relocate itself to respect > >> it own constraint. > > > > Yes, because the boot protocol does not guarantee 2MB alignment, so if > > the OS wants to rely on that then it is obliged to take care of > > relocating itself. > > > >> Load the zImage at rambase address + 2MB to make life easier for other > >> kernel (such as FreeBSD, Mini-OS). > > > > Whether or not it is easier these OSes *must* be prepared to be loaded > > at any 4k aligned address. It sounds to me like you are hoping that > > these OSes can *rely* on 2MB alignment, which is not the case. If they > > make that assumption then they are buggy, regardless of whether it > > happens to currently work with some loader or not. > > Using 4K alignment make impossible to use 1MB or 2MB mapping for early > page table. Which make the code (usually in assembly) even harder to > write or to impose relocation in the assembly code. yes, but that is what the protocol requires you to do, or to handle realigning yourself early. > >> FWIW, this is already the case for DOM0. > > > > On that basis I'm more inclined to change dom0 to use 32kb alignment. > > It's not really 32kb alignment but an address in 0xXXXX8000. I meant 32kb alignment. > If the > kernel doesn't want to do relocation (because it's a pain to write in > assembly code), it has to compile the kernel with a 0x8000 offset (see > mini-os patch series). Yes, if a kernel doesn't want to do relocaiton then it must give an address. Conversely if a kernel does do relocation then it must do it properly, not assuming 2MB alignment. > This solution, make the kernel more tight to Xen changes. I would prefer > if we relax this change allowing easier support of new kernel to 2MB > alignment. Easier isn't relevant here. The protocol requires you to deal with this if you claim to be relocatable. > > Regards, >
On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote: > On 03/07/14 11:12, Julien Grall wrote: > > On 07/03/2014 11:01 AM, Ian Campbell wrote: > >> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: > >>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). > >>> > >>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If > >>> the kernel doesn't relocate itself this would require to virtual address > >>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a > >>> convenience for Linux after the decompressor stage. Linux is able to > >>> load at any address in the memory and it will relocate itself to respect > >>> it own constraint. > >> > >> Yes, because the boot protocol does not guarantee 2MB alignment, so if > >> the OS wants to rely on that then it is obliged to take care of > >> relocating itself. > >> > >>> Load the zImage at rambase address + 2MB to make life easier for other > >>> kernel (such as FreeBSD, Mini-OS). > >> > >> Whether or not it is easier these OSes *must* be prepared to be loaded > >> at any 4k aligned address. It sounds to me like you are hoping that > >> these OSes can *rely* on 2MB alignment, which is not the case. If they > >> make that assumption then they are buggy, regardless of whether it > >> happens to currently work with some loader or not. > > > > Using 4K alignment make impossible to use 1MB or 2MB mapping for early > > page table. Which make the code (usually in assembly) even harder to > > write or to impose relocation in the assembly code. > > This isn't the case for Mini-OS, at least. We already use a 1MB mapping > with the 0x8000 offset just fine. The translation table (using 1MB > sections) is 16K, which would fit nicely in the 0x8000 gap (although > currently we don't use that space). > > Starting at 2MB would create an inconvenient 2MB of free memory just > before the kernel, with no obvious way to tell xmalloc about it. As far as I know you need to cope with arbitrary 4K alignment I'm afraid, that's all which is guaranteed. You can't assume that just because we use 0x8000 or 2MB offsets today it will always be that way. Ian.
On Thu, 2014-07-03 at 12:12 +0100, Julien Grall wrote: > On 07/03/2014 11:37 AM, Thomas Leonard wrote: > >> Using 4K alignment make impossible to use 1MB or 2MB mapping for early > >> page table. Which make the code (usually in assembly) even harder to > >> write or to impose relocation in the assembly code. > > > > This isn't the case for Mini-OS, at least. We already use a 1MB mapping > > with the 0x8000 offset just fine. The translation table (using 1MB > > sections) is 16K, which would fit nicely in the 0x8000 gap (although > > currently we don't use that space). > > As I said in Mini-OS series, the spec doesn't say anything about the > start offset of the guest kernel for the Linux boot protocol. > > 0x8000 offset is very inconvenient for most of the other OS than Linux. > As Linux is able to relocate itself, I don't see any known issue about > starting at an 2MB-aligned address. The known issue is that you cannot rely going forward on the address always being 2MB aligned. You *cannot* assume this. Ian.
On 07/03/2014 12:18 PM, Ian Campbell wrote: > On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote: >> On 03/07/14 11:12, Julien Grall wrote: >>> On 07/03/2014 11:01 AM, Ian Campbell wrote: >>>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: >>>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). >>>>> >>>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If >>>>> the kernel doesn't relocate itself this would require to virtual address >>>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a >>>>> convenience for Linux after the decompressor stage. Linux is able to >>>>> load at any address in the memory and it will relocate itself to respect >>>>> it own constraint. >>>> >>>> Yes, because the boot protocol does not guarantee 2MB alignment, so if >>>> the OS wants to rely on that then it is obliged to take care of >>>> relocating itself. >>>> >>>>> Load the zImage at rambase address + 2MB to make life easier for other >>>>> kernel (such as FreeBSD, Mini-OS). >>>> >>>> Whether or not it is easier these OSes *must* be prepared to be loaded >>>> at any 4k aligned address. It sounds to me like you are hoping that >>>> these OSes can *rely* on 2MB alignment, which is not the case. If they >>>> make that assumption then they are buggy, regardless of whether it >>>> happens to currently work with some loader or not. >>> >>> Using 4K alignment make impossible to use 1MB or 2MB mapping for early >>> page table. Which make the code (usually in assembly) even harder to >>> write or to impose relocation in the assembly code. >> >> This isn't the case for Mini-OS, at least. We already use a 1MB mapping >> with the 0x8000 offset just fine. The translation table (using 1MB >> sections) is 16K, which would fit nicely in the 0x8000 gap (although >> currently we don't use that space). >> >> Starting at 2MB would create an inconvenient 2MB of free memory just >> before the kernel, with no obvious way to tell xmalloc about it. > > As far as I know you need to cope with arbitrary 4K alignment I'm > afraid, that's all which is guaranteed. I think the zImage is also be able to cope on loading at 0xXXXXX200. There is no 4k alignment requirements :). > You can't assume that just because we use 0x8000 or 2MB offsets today it > will always be that way. That's all which is guaranteed if we are loading the kernel on a any bootloader. Even though, on u-boot you can choose your loading address (via u-boot image). That's why Xen is boot perfectly even if we have a 2MB-align address. I don't find mad to define a standard for booting on Xen saying the address will always be N MB align. For Mini-OS it will help to have a simpler code which doesn't need relocation. Regards,
On Thu, 2014-07-03 at 13:21 +0100, Julien Grall wrote: > On 07/03/2014 12:18 PM, Ian Campbell wrote: > > On Thu, 2014-07-03 at 11:39 +0100, Thomas Leonard wrote: > >> On 03/07/14 11:12, Julien Grall wrote: > >>> On 07/03/2014 11:01 AM, Ian Campbell wrote: > >>>> On Mon, 2014-06-30 at 15:05 +0100, Julien Grall wrote: > >>>>> Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). > >>>>> > >>>>> Kernel are usually using 1MB (or 2MB) mapping for the early page table. If > >>>>> the kernel doesn't relocate itself this would require to virtual address > >>>>> starting at 0xXXXX8000. This is not part of the zImage protocol, but a > >>>>> convenience for Linux after the decompressor stage. Linux is able to > >>>>> load at any address in the memory and it will relocate itself to respect > >>>>> it own constraint. > >>>> > >>>> Yes, because the boot protocol does not guarantee 2MB alignment, so if > >>>> the OS wants to rely on that then it is obliged to take care of > >>>> relocating itself. > >>>> > >>>>> Load the zImage at rambase address + 2MB to make life easier for other > >>>>> kernel (such as FreeBSD, Mini-OS). > >>>> > >>>> Whether or not it is easier these OSes *must* be prepared to be loaded > >>>> at any 4k aligned address. It sounds to me like you are hoping that > >>>> these OSes can *rely* on 2MB alignment, which is not the case. If they > >>>> make that assumption then they are buggy, regardless of whether it > >>>> happens to currently work with some loader or not. > >>> > >>> Using 4K alignment make impossible to use 1MB or 2MB mapping for early > >>> page table. Which make the code (usually in assembly) even harder to > >>> write or to impose relocation in the assembly code. > >> > >> This isn't the case for Mini-OS, at least. We already use a 1MB mapping > >> with the 0x8000 offset just fine. The translation table (using 1MB > >> sections) is 16K, which would fit nicely in the 0x8000 gap (although > >> currently we don't use that space). > >> > >> Starting at 2MB would create an inconvenient 2MB of free memory just > >> before the kernel, with no obvious way to tell xmalloc about it. > > > > As far as I know you need to cope with arbitrary 4K alignment I'm > > afraid, that's all which is guaranteed. > > I think the zImage is also be able to cope on loading at 0xXXXXX200. > There is no 4k alignment requirements :). Then it's even more flexible than I thought. Fine (this is the problem with de facto standards). > > You can't assume that just because we use 0x8000 or 2MB offsets today it > > will always be that way. > > That's all which is guaranteed if we are loading the kernel on a any > bootloader. No, nothing is of that sort is guaranteed or required by anything. > Even though, on u-boot you can choose your loading address > (via u-boot image). That's why Xen is boot perfectly even if we have a > 2MB-align address. This is a bug in Xen. We currently get away with it. But if a bootloader comes along which doesn't load us at a 2MB boundary then that is *our* bug, not the bootloader's. > I don't find mad to define a standard for booting on Xen saying the > address will always be N MB align. We are using someonelse's standard, which does not say this. We are not going to deviate from that (admittedly somewhat de facto) standard, sorry. You are of course free to go and have FreeBSD make whatever assumptions about alignment which you like about load addresses. But those are not guaranteed by the zImage protocol and when the behaviour of Xen changes within that spec and breaks FreeBSD then that will be a FreeBSD bug, not a Xen bug. If you want to define your own standard then go ahead and pick your own magic number and write down what the bootloader must guarantee and what the kernel must be capable of dealing with when that protocol isused. But the Linux zImage protocol is not ours to mess with. Ian.
diff --git a/tools/libxc/xc_dom_armzimageloader.c b/tools/libxc/xc_dom_armzimageloader.c index 2b28781..2cf95a5 100644 --- a/tools/libxc/xc_dom_armzimageloader.c +++ b/tools/libxc/xc_dom_armzimageloader.c @@ -87,7 +87,7 @@ static int xc_dom_parse_zimage32_kernel(struct xc_dom_image *dom) zimage = (uint32_t *)dom->kernel_blob; /* Do not load kernel at the very first RAM address */ - v_start = rambase + 0x8000; + v_start = rambase + 0x200000; if ( dom->kernel_size > UINT64_MAX - v_start ) {
Currently libxc is loading the kernel zImage at rambase + 32KB (0x8000). Kernel are usually using 1MB (or 2MB) mapping for the early page table. If the kernel doesn't relocate itself this would require to virtual address starting at 0xXXXX8000. This is not part of the zImage protocol, but a convenience for Linux after the decompressor stage. Linux is able to load at any address in the memory and it will relocate itself to respect it own constraint. Load the zImage at rambase address + 2MB to make life easier for other kernel (such as FreeBSD, Mini-OS). FWIW, this is already the case for DOM0. Signed-off-by: Julien Grall <julien.grall@linaro.org> --- This patch is candidate for backporting. Xen 4.4 is suitable to boot any guest OS, but without this patch it will require some modification in the guest kernel. Hence, with this patch DOM0 and the guest will have the same requirement for booting. --- tools/libxc/xc_dom_armzimageloader.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)