[RFC,1/2] android-tools: import android-tools from meta-smartphone into oe-core

Message ID 1440604110-4306-2-git-send-email-nicolas.dechesne@linaro.org
State New
Headers show

Commit Message

Nicolas Dechesne Aug. 26, 2015, 3:48 p.m.
The following changes were done to trim down the recipe size:
* removed adbd support
* removed android-tools-conf

The main idea is to keep only what is strictly needed for make_ext4fs tool.

Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org>
---
 .../android-tools/disable-selinux-support.patch    | 137 +++++++++++++++++++++
 .../android-tools/reboot-syscall.patch             |  25 ++++
 .../android-tools/remove-libselinux.patch          |  13 ++
 .../android-tools/android-tools_4.2.2.bb           |  51 ++++++++
 4 files changed, 226 insertions(+)
 create mode 100644 meta/recipes-devtools/android-tools/android-tools/disable-selinux-support.patch
 create mode 100644 meta/recipes-devtools/android-tools/android-tools/reboot-syscall.patch
 create mode 100644 meta/recipes-devtools/android-tools/android-tools/remove-libselinux.patch
 create mode 100644 meta/recipes-devtools/android-tools/android-tools_4.2.2.bb

Comments

Burton, Ross Aug. 28, 2015, 8:39 p.m. | #1
On 26 August 2015 at 16:48, Nicolas Dechesne <nicolas.dechesne@linaro.org>
wrote:

> The main idea is to keep only what is strictly needed for make_ext4fs tool.
>

Assuming that make_ext4fs is a tool to create an ext4 file system, what's
the difference between that and our mke2fs which is used to create ext4
file systems?

Ross
Richard Purdie Aug. 29, 2015, 12:36 p.m. | #2
On Fri, 2015-08-28 at 21:39 +0100, Burton, Ross wrote:
> 
> On 26 August 2015 at 16:48, Nicolas Dechesne
> <nicolas.dechesne@linaro.org> wrote:
>         The main idea is to keep only what is strictly needed for
>         make_ext4fs tool.
>         
> 
> Assuming that make_ext4fs is a tool to create an ext4 file system,
> what's the difference between that and our mke2fs which is used to
> create ext4 file systems?
> 
We've worked quite hard to land the ext4 rootfs creation code into
upstream e2fsprogs. I'd like to understand if we're missing anything for
that to work with fastboot. If we are, I suspect it wouldn't be hard to
add and should save maintenance of a separate tool (support in e2fsprogs
means we get any other ext* improvements for free).

Cheers,

Richard
Nicolas Dechesne Aug. 31, 2015, 1:38 p.m. | #3
On Sat, Aug 29, 2015 at 2:36 PM, Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>>
>> Assuming that make_ext4fs is a tool to create an ext4 file system,
>> what's the difference between that and our mke2fs which is used to
>> create ext4 file systems?
>>
> We've worked quite hard to land the ext4 rootfs creation code into
> upstream e2fsprogs. I'd like to understand if we're missing anything for
> that to work with fastboot. If we are, I suspect it wouldn't be hard to
> add and should save maintenance of a separate tool (support in e2fsprogs
> means we get any other ext* improvements for free).

thanks for the feedback.

I sent this as RFC , because I wanted to initiate a discussion... I
decided to simply move existing recipes from meta-smartphone/android
for that purpose.

compared to 'regular' ext4 images , fastboot added the (convenient)
ability to support sparse images, to reduce the size of what is
transmitted over the USB, when flashing (and reduce the size of
writting to eMMC). For devices with 8GB+ eMMC that is a big deal
whether we flash a raw ext4 image, or its sparse'd version. More info
about fastboot sparse format at [1].

Unfortunately I haven't found any tool to 'sparse' an existing image,
make_ext4fs is doing the FS as well, so as you noticed it duplicates
what mke2fs does. make_ext4fs generates sparse image optionally (with
-s flag), but I don't know precisely what's the difference between the
generated image vs what mke2fs would do. The non sparse image from
make_ext4fs doesn't have the sparse header, and as far as I can tell
it is a valid 'ext4' image, which I can mount with mount -o loop, for
example (or inspect with dumpe2fs).

I've looked further into that today, and :

* when transferring an image to be flashed over USB, the target uses a
RAM buffer. if the image file size is smaller than the RAM buffer,
then our standard ext4 image can be flashed just fine.

* when transferring an image larger than the internal target RAM
buffer, then the host side fastboot program will split the images into
chunks that can be individually transferred and flashed. In that case
I am able to load a standard OE ext4 image *only* if I round/pad the
image to 4096 bytes, otherwise fastboot fails (with cryptic assertion
which I don't quite understand for now). If anyone has experience with
that, please let us know.

* Of course images from make_ext4fs whether they are sparse, non
sparse, large or not, always work.

* the size if this intermediate buffer is platform specific, the host
side fastboot queries the target to know the size of this buffer.

so in short.. OE ext4 image seems to be working fine (with the file
size limitation), but we are missing the specific sparse support.
Maybe that's good enough for OE core, and we can keep the
customization/optimizations in BSP layers..


[1] http://www.2net.co.uk/tutorial/android-sparse-image-format
Otavio Salvador Aug. 31, 2015, 2:12 p.m. | #4
On Mon, Aug 31, 2015 at 10:38 AM, Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
> On Sat, Aug 29, 2015 at 2:36 PM, Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>>>
>>> Assuming that make_ext4fs is a tool to create an ext4 file system,
>>> what's the difference between that and our mke2fs which is used to
>>> create ext4 file systems?
>>>
>> We've worked quite hard to land the ext4 rootfs creation code into
>> upstream e2fsprogs. I'd like to understand if we're missing anything for
>> that to work with fastboot. If we are, I suspect it wouldn't be hard to
>> add and should save maintenance of a separate tool (support in e2fsprogs
>> means we get any other ext* improvements for free).
>
> thanks for the feedback.
>
> I sent this as RFC , because I wanted to initiate a discussion... I
> decided to simply move existing recipes from meta-smartphone/android
> for that purpose.
>
> compared to 'regular' ext4 images , fastboot added the (convenient)
> ability to support sparse images, to reduce the size of what is
> transmitted over the USB, when flashing (and reduce the size of
> writting to eMMC). For devices with 8GB+ eMMC that is a big deal
> whether we flash a raw ext4 image, or its sparse'd version. More info
> about fastboot sparse format at [1].

It does help for images it seems; did you see if someone has propose a
patch for e2fstools to enable this?
Nicolas Dechesne Aug. 31, 2015, 2:28 p.m. | #5
On Mon, Aug 31, 2015 at 4:12 PM, Otavio Salvador
<otavio.salvador@ossystems.com.br> wrote:
>> compared to 'regular' ext4 images , fastboot added the (convenient)
>> ability to support sparse images, to reduce the size of what is
>> transmitted over the USB, when flashing (and reduce the size of
>> writting to eMMC). For devices with 8GB+ eMMC that is a big deal
>> whether we flash a raw ext4 image, or its sparse'd version. More info
>> about fastboot sparse format at [1].
>
> It does help for images it seems; did you see if someone has propose a
> patch for e2fstools to enable this?

no, i haven't seen anything there. Note that it would be a
fastboot-specific sparse format.
Yi Qingliang Aug. 31, 2015, 3:07 p.m. | #6
can we compile android system using oe?
the Makefile system of android is too complicated to load/use.

On Mon, Aug 31, 2015 at 10:28 PM, Nicolas Dechesne
<nicolas.dechesne@linaro.org> wrote:
> On Mon, Aug 31, 2015 at 4:12 PM, Otavio Salvador
> <otavio.salvador@ossystems.com.br> wrote:
>>> compared to 'regular' ext4 images , fastboot added the (convenient)
>>> ability to support sparse images, to reduce the size of what is
>>> transmitted over the USB, when flashing (and reduce the size of
>>> writting to eMMC). For devices with 8GB+ eMMC that is a big deal
>>> whether we flash a raw ext4 image, or its sparse'd version. More info
>>> about fastboot sparse format at [1].
>>
>> It does help for images it seems; did you see if someone has propose a
>> patch for e2fstools to enable this?
>
> no, i haven't seen anything there. Note that it would be a
> fastboot-specific sparse format.
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
Nicolas Dechesne Aug. 31, 2015, 3:15 p.m. | #7
On Mon, Aug 31, 2015 at 5:07 PM, Yi Qingliang <niqingliang2003@gmail.com> wrote:
> can we compile android system using oe?
> the Makefile system of android is too complicated to load/use.

I believe this is fairly off topic. I am not trying to rebuild Android
here. Just trying to load a regular OE image on devices that typically
run Android.

Patch

diff --git a/meta/recipes-devtools/android-tools/android-tools/disable-selinux-support.patch b/meta/recipes-devtools/android-tools/android-tools/disable-selinux-support.patch
new file mode 100644
index 0000000..738f575
--- /dev/null
+++ b/meta/recipes-devtools/android-tools/android-tools/disable-selinux-support.patch
@@ -0,0 +1,137 @@ 
+diff --git a/extras/ext4_utils/make_ext4fs.c b/extras/ext4_utils/make_ext4fs.c
+index b2d1426..94e92d6 100644
+--- a/extras/ext4_utils/make_ext4fs.c
++++ b/extras/ext4_utils/make_ext4fs.c
+@@ -59,9 +59,11 @@
+ 
+ #else
+ 
++#if 0
+ #include <selinux/selinux.h>
+ #include <selinux/label.h>
+ #include <selinux/android.h>
++#endif
+ 
+ #define O_BINARY 0
+ 
+@@ -178,6 +180,7 @@ static u32 build_directory_structure(const char *full_path, const char *dir_path
+ 			error("can't set android permissions - built without android support");
+ #endif
+ 		}
++#if 0
+ #ifndef USE_MINGW
+ 		if (sehnd) {
+ 			if (selabel_lookup(sehnd, &dentries[i].secon, dentries[i].path, stat.st_mode) < 0) {
+@@ -188,6 +191,7 @@ static u32 build_directory_structure(const char *full_path, const char *dir_path
+ 				printf("Labeling %s as %s\n", dentries[i].path, dentries[i].secon);
+ 		}
+ #endif
++#endif
+ 
+ 		if (S_ISREG(stat.st_mode)) {
+ 			dentries[i].file_type = EXT4_FT_REG_FILE;
+@@ -229,10 +233,12 @@ static u32 build_directory_structure(const char *full_path, const char *dir_path
+ 		dentries[0].file_type = EXT4_FT_DIR;
+ 		dentries[0].uid = 0;
+ 		dentries[0].gid = 0;
++#if 0
+ 		if (sehnd) {
+ 			if (selabel_lookup(sehnd, &dentries[0].secon, dentries[0].path, dentries[0].mode) < 0)
+ 				error("cannot lookup security context for %s", dentries[0].path);
+ 		}
++#endif
+ 		entries++;
+ 		dirs++;
+ 	}
+@@ -270,9 +276,11 @@ static u32 build_directory_structure(const char *full_path, const char *dir_path
+ 			dentries[i].mtime);
+ 		if (ret)
+ 			error("failed to set permissions on %s\n", dentries[i].path);
++#if 0
+ 		ret = inode_set_selinux(entry_inode, dentries[i].secon);
+ 		if (ret)
+ 			error("failed to set SELinux context on %s\n", dentries[i].path);
++#endif
+ 
+ 		free(dentries[i].path);
+ 		free(dentries[i].full_path);
+@@ -562,6 +570,7 @@ int make_ext4fs_internal(int fd, const char *_directory,
+ 	root_mode = S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH | S_IXOTH;
+ 	inode_set_permissions(root_inode_num, root_mode, 0, 0, 0);
+ 
++#if 0
+ #ifndef USE_MINGW
+ 	if (sehnd) {
+ 		char *secontext = NULL;
+@@ -578,6 +587,7 @@ int make_ext4fs_internal(int fd, const char *_directory,
+ 		freecon(secontext);
+ 	}
+ #endif
++#endif
+ 
+ 	ext4_update_free();
+ 
+diff --git a/extras/ext4_utils/make_ext4fs_main.c b/extras/ext4_utils/make_ext4fs_main.c
+index b6c740d..ce31764 100644
+--- a/extras/ext4_utils/make_ext4fs_main.c
++++ b/extras/ext4_utils/make_ext4fs_main.c
+@@ -29,6 +29,7 @@
+ #include <private/android_filesystem_config.h>
+ #endif
+ 
++#if 0
+ #ifndef USE_MINGW
+ #include <selinux/selinux.h>
+ #include <selinux/label.h>
+@@ -36,6 +37,7 @@
+ #else
+ struct selabel_handle;
+ #endif
++#endif
+ 
+ #include "make_ext4fs.h"
+ #include "ext4_utils.h"
+@@ -72,9 +74,11 @@ int main(int argc, char **argv)
+ 	int exitcode;
+ 	int verbose = 0;
+ 	struct selabel_handle *sehnd = NULL;
++#if 0
+ #ifndef USE_MINGW
+ 	struct selinux_opt seopts[] = { { SELABEL_OPT_PATH, "" } };
+ #endif
++#endif
+ 
+ 	while ((opt = getopt(argc, argv, "l:j:b:g:i:I:L:a:S:fwzJsctv")) != -1) {
+ 		switch (opt) {
+@@ -131,6 +135,7 @@ int main(int argc, char **argv)
+ 			fprintf(stderr, "Warning: -t (initialize inode tables) is deprecated\n");
+ 			break;
+ 		case 'S':
++#if 0
+ #ifndef USE_MINGW
+ 			seopts[0].value = optarg;
+ 			sehnd = selabel_open(SELABEL_CTX_FILE, seopts, 1);
+@@ -139,6 +144,7 @@ int main(int argc, char **argv)
+ 				exit(EXIT_FAILURE);
+ 			}
+ #endif
++#endif
+ 			break;
+ 		case 'v':
+ 			verbose = 1;
+@@ -149,6 +155,7 @@ int main(int argc, char **argv)
+ 		}
+ 	}
+ 
++#if 0
+ #if !defined(HOST)
+ 	// Use only if -S option not requested
+ 	if (!sehnd && mountpoint) {
+@@ -160,6 +167,7 @@ int main(int argc, char **argv)
+ 		}
+ 	}
+ #endif
++#endif
+ 
+ 	if (wipe && sparse) {
+ 		fprintf(stderr, "Cannot specifiy both wipe and sparse\n");
diff --git a/meta/recipes-devtools/android-tools/android-tools/reboot-syscall.patch b/meta/recipes-devtools/android-tools/android-tools/reboot-syscall.patch
new file mode 100644
index 0000000..0055416
--- /dev/null
+++ b/meta/recipes-devtools/android-tools/android-tools/reboot-syscall.patch
@@ -0,0 +1,25 @@ 
+Subject: port android_reboot() to call the reboot syscall via syscall() (glibc) rather than __reboot (bionic)
+Author: Loïc Minier <loic.minier@ubuntu.com>
+
+--- a/core/libcutils/android_reboot.c
++++ b/core/libcutils/android_reboot.c
+@@ -21,6 +21,8 @@
+ #include <fcntl.h>
+ #include <stdio.h>
+ #include <string.h>
++#include <linux/reboot.h>
++#include <sys/syscall.h>
+ 
+ #include <cutils/android_reboot.h>
+ 
+@@ -121,8 +123,8 @@
+             break;
+ 
+         case ANDROID_RB_RESTART2:
+-            ret = __reboot(LINUX_REBOOT_MAGIC1, LINUX_REBOOT_MAGIC2,
+-                           LINUX_REBOOT_CMD_RESTART2, arg);
++            ret = syscall(SYS_reboot, LINUX_REBOOT_MAGIC1, LINUX_REBOOT_MAGIC2,
++                          LINUX_REBOOT_CMD_RESTART2, arg);
+             break;
+ 
+         default:
diff --git a/meta/recipes-devtools/android-tools/android-tools/remove-libselinux.patch b/meta/recipes-devtools/android-tools/android-tools/remove-libselinux.patch
new file mode 100644
index 0000000..24690b2
--- /dev/null
+++ b/meta/recipes-devtools/android-tools/android-tools/remove-libselinux.patch
@@ -0,0 +1,13 @@ 
+diff --git a/debian/makefiles/ext4_utils.mk b/debian/makefiles/ext4_utils.mk
+index cb64916..60e81d6 100644
+--- a/debian/makefiles/ext4_utils.mk
++++ b/debian/makefiles/ext4_utils.mk
+@@ -36,7 +36,7 @@ CPPFLAGS+= -I/usr/include
+ CPPFLAGS+= -I../../core/include
+ CPPFLAGS+= -I../../core/libsparse/include/
+ 
+-LIBS+= -lz -lselinux
++LIBS+= -lz
+ 
+ OBJS= $(SRCS:.c=.o)
+ 
diff --git a/meta/recipes-devtools/android-tools/android-tools_4.2.2.bb b/meta/recipes-devtools/android-tools/android-tools_4.2.2.bb
new file mode 100644
index 0000000..7780e09
--- /dev/null
+++ b/meta/recipes-devtools/android-tools/android-tools_4.2.2.bb
@@ -0,0 +1,51 @@ 
+DESCRIPTION = "Different utilities from Android - based on the corresponding ubuntu \
+package"
+SECTION = "console/utils"
+LICENSE = "Apache-2.0 & GPL-2.0 & BSD-2-Clause & BSD-3-Clause"
+LIC_FILES_CHKSUM = " \
+  file://${COMMON_LICENSE_DIR}/Apache-2.0;md5=89aea4e17d99a7cacdbeed46a0096b10 \
+  file://${COMMON_LICENSE_DIR}/GPL-2.0;md5=801f80980d171dd6425610833a22dbe6 \
+  file://${COMMON_LICENSE_DIR}/BSD-2-Clause;md5=8bef8e6712b1be5aa76af1ebde9d6378 \
+  file://${COMMON_LICENSE_DIR}/BSD-3-Clause;md5=550794465ba0ec5312d6919e203a55f9 \
+  file://${WORKDIR}/debian/copyright;md5=141efd1050596168ca05ced04e4f498b \
+"
+
+DEPENDS = "zlib openssl"
+
+# Use same version than ubuntu does here
+BASE_PV = "4.2.2+git20130218"
+PV = "${BASE_PV}-3ubuntu13"
+
+BBCLASSEXTEND += "native"
+
+SRC_URI = " \
+    https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_${BASE_PV}.orig.tar.xz;name=source \
+    https://launchpad.net/ubuntu/+archive/primary/+files/android-tools_${PV}.debian.tar.gz;name=debian \
+    file://reboot-syscall.patch \
+    file://disable-selinux-support.patch \
+    file://remove-libselinux.patch;patchdir=.. \
+"
+S = "${WORKDIR}/android-tools"
+
+SRC_URI[source.md5sum] = "0e653b129ab0c95bdffa91410c8b55be"
+SRC_URI[source.sha256sum] = "9bfba987e1351b12aa983787b9ae4424ab752e9e646d8e93771538dc1e5d932f"
+SRC_URI[debian.md5sum] = "5e409d01caf3c33fc60a2100464754ff"
+SRC_URI[debian.sha256sum] = "320757edc8af015f40335c41dc96bf37e2d50c9f3a40a31e64264ff6e2dba5e3"
+
+do_compile() {
+    # Setting both variables below causing our makefiles to not work with implicit make
+    # rules
+    unset CFLAGS
+    unset CPPFLAGS
+
+    sed -i "s%^CPPFLAGS+= -I/usr/include%# we don't want to include headers from host CPPFLAGS+= -I/usr/include%g" ${WORKDIR}/debian/makefiles/ext4_utils.mk
+
+    oe_runmake -f ${WORKDIR}/debian/makefiles/ext4_utils.mk -C ${S}/extras/ext4_utils clean
+    oe_runmake -f ${WORKDIR}/debian/makefiles/ext4_utils.mk -C ${S}/extras/ext4_utils
+}
+
+do_install() {
+    install -d ${D}${bindir}
+    install -m 0755 ${S}/extras/ext4_utils/make_ext4fs ${D}${bindir}
+
+}