| Message ID | 20180116132559.27794-1-ross.burton@intel.com |
|---|---|
| State | New |
| Headers | show |
| Series | [RFC] cmake: use Ninja backend instead of Make | expand |
On 01/16/2018 03:25 PM, Ross Burton wrote: > The Ninja backend is faster than Make for large projects (takes three minutes > off webkitgtk:do_compile for me), so use it by default. You do know why Ninja was invented in the first place, right? :) http://neugierig.org/software/chromium/notes/2011/02/ninja.html Alex -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com> wrote: > The Ninja backend is faster than Make for large projects (takes three minutes > off webkitgtk:do_compile for me), so use it by default. > > Signed-off-by: Ross Burton <ross.burton@intel.com> I tried to test this but it did not apply in master. Could you rebase it? -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On 2018-01-17 11:18 AM, Otavio Salvador wrote: > On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com> wrote: >> The Ninja backend is faster than Make for large projects (takes three minutes >> off webkitgtk:do_compile for me), so use it by default. >> >> Signed-off-by: Ross Burton <ross.burton@intel.com> > > I tried to test this but it did not apply in master. Could you rebase it? > On my 64+64 core system that is shared, it reduced a webkit compile from ~ 17 minutes to ~12 minutes for just the compile stage. YMMV. $ cat webkit-before-compile.log 3.29user 0.50system 17:39.98elapsed 0%CPU (0avgtext+0avgdata 28532maxresident)k 0inputs+176outputs (0major+16335minor)pagefaults 0swaps $ cat webkit-after-compile.log 2.70user 0.47system 12:09.24elapsed 0%CPU (0avgtext+0avgdata 28292maxresident)k 0inputs+160outputs (0major+14634minor)pagefaults 0swaps I'm going to try to build world (oe-core + meta-oe + ...). Octavio, it's a two line change! :) This applied to master for me: commit 3914c66f6e7123a64ab73e2b6707452bbfd10d02 Author: Ross Burton <ross.burton@intel.com> Date: Wed Jan 17 13:09:40 2018 -0500 cmake: use Ninja backend instead of Make The Ninja backend is faster than Make for large projects (it takes three minutes off webkitgtk:do_compile for example) so use it by default. Acked-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass index a4079dab0d..c46117acde 100644 --- a/meta/classes/cmake.bbclass +++ b/meta/classes/cmake.bbclass @@ -1,7 +1,7 @@ # Path to the CMake file to process. OECMAKE_SOURCEPATH ??= "${S}" -DEPENDS_prepend = "cmake-native " +DEPENDS_prepend = "cmake-native ninja-native " B = "${WORKDIR}/build" # We need to unset CCACHE otherwise cmake gets too confused @@ -117,7 +117,7 @@ cmake_do_configure() { oecmake_sitefile= fi - cmake \ + cmake -GNinja \ $oecmake_sitefile \ ${OECMAKE_SOURCEPATH} \ -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ -- # Randy MacLeod. WR Linux # Wind River an Intel Company -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On 17 January 2018 at 20:57, Randy MacLeod <randy.macleod@windriver.com> wrote: > On 2018-01-17 11:18 AM, Otavio Salvador wrote: > >> On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <ross.burton@intel.com> >> wrote: >> >>> The Ninja backend is faster than Make for large projects (takes three >>> minutes >>> off webkitgtk:do_compile for me), so use it by default. >>> >>> Signed-off-by: Ross Burton <ross.burton@intel.com> >>> >> >> I tried to test this but it did not apply in master. Could you rebase it? >> >> > On my 64+64 core system that is shared, it reduced a webkit compile > from ~ 17 minutes to ~12 minutes for just the compile stage. YMMV. > > $ cat webkit-before-compile.log > 3.29user 0.50system 17:39.98elapsed 0%CPU > (0avgtext+0avgdata 28532maxresident)k > 0inputs+176outputs (0major+16335minor)pagefaults 0swaps > > > $ cat webkit-after-compile.log > 2.70user 0.47system 12:09.24elapsed 0%CPU > (0avgtext+0avgdata 28292maxresident)k > 0inputs+160outputs (0major+14634minor)pagefaults 0swaps > > I'm going to try to build world (oe-core + meta-oe + ...). > Good win :) There appears to be a small number of upstreams that break with ninja, so I'll rework this patch so that the backend is an option but default to Ninja. Ross <div dir="ltr">On 17 January 2018 at 20:57, Randy MacLeod <span dir="ltr"><<a href="mailto:randy.macleod@windriver.com" target="_blank">randy.macleod@windriver.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class="">On 2018-01-17 11:18 AM, Otavio Salvador wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> On Tue, Jan 16, 2018 at 11:25 AM, Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>> wrote:<br> <blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"> The Ninja backend is faster than Make for large projects (takes three minutes<br> off webkitgtk:do_compile for me), so use it by default.<br> <br> Signed-off-by: Ross Burton <<a href="mailto:ross.burton@intel.com" target="_blank">ross.burton@intel.com</a>><br> </blockquote> <br> I tried to test this but it did not apply in master. Could you rebase it?<br> <br> </blockquote> <br></span> On my 64+64 core system that is shared, it reduced a webkit compile<br> from ~ 17 minutes to ~12 minutes for just the compile stage. YMMV.<br> <br> $ cat webkit-before-compile.log<br> 3.29user 0.50system 17:39.98elapsed 0%CPU<br> (0avgtext+0avgdata 28532maxresident)k<br> 0inputs+176outputs (0major+16335minor)pagefaults 0swaps<br> <br> <br> $ cat webkit-after-compile.log<br> 2.70user 0.47system 12:09.24elapsed 0%CPU<br> (0avgtext+0avgdata 28292maxresident)k<br> 0inputs+160outputs (0major+14634minor)pagefaults 0swaps<br> <br> I'm going to try to build world (oe-core + meta-oe + ...).<br></blockquote><div><br></div><div>Good win :)</div><div><br></div><div>There appears to be a small number of upstreams that break with ninja, so I'll rework this patch so that the backend is an option but default to Ninja.</div><div><br></div><div>Ross </div></div></div></div> -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On Wed, Jan 17, 2018 at 7:43 PM, Burton, Ross <ross.burton@intel.com> wrote: > There appears to be a small number of upstreams that break with ninja, so > I'll rework this patch so that the backend is an option but default to > Ninja. When doing so, please rebase on top of my patches and send it as official patch (no RFC). Besides test, it is very welcome. -- Otavio Salvador O.S. Systems http://www.ossystems.com.br http://code.ossystems.com.br Mobile: +55 (53) 9981-7854 Mobile: +1 (347) 903-9750 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On 2018-01-17 04:43 PM, Burton, Ross wrote: > There appears to be a small number of upstreams that break with ninja, > so I'll rework this patch so that the backend is an option but default > to Ninja. The only failure that involved a cmake based package was piglit for my qemux86-64 world build. It didn't install: make: Entering directory '/.../tmp-glibc/work/core2-64-wrs-linux/piglit/1.0+gitrAUTOINC+c8f4fd9eeb-r0/build' make: *** No rule to make target 'install/strip'. Stop. so that might not be related to this change. I haven't manually done a world build on master recently. As expected using ninja for the back-end of cmake just makes the builds faster. -- # Randy MacLeod. WR Linux # Wind River an Intel Company -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core
On 2018-01-17 09:59 PM, Randy MacLeod wrote: > On 2018-01-17 04:43 PM, Burton, Ross wrote: >> There appears to be a small number of upstreams that break with ninja, >> so I'll rework this patch so that the backend is an option but default >> to Ninja. > > The only failure that involved a cmake based package was piglit > for my qemux86-64 world build. > > It didn't install: > make: Entering directory > '/.../tmp-glibc/work/core2-64-wrs-linux/piglit/1.0+gitrAUTOINC+c8f4fd9eeb-r0/build' > > make: *** No rule to make target 'install/strip'. Stop. > > so that might not be related to this change. I haven't manually done > a world build on master recently. Apparently already fixed by Ross in master-next: commit 9fec5ef920bd63fe3cc2e623add0c7aead95ecae Author: Ross Burton <ross.burton@intel.com> Date: Tue Jan 16 13:25:16 2018 +0000 piglit: override install target name instead of do_install ../Randy > > As expected using ninja for the back-end of cmake just makes the > builds faster. >
diff --git a/meta/classes/cmake.bbclass b/meta/classes/cmake.bbclass index 10c4edcf1a5..ab29b1f1c23 100644 --- a/meta/classes/cmake.bbclass +++ b/meta/classes/cmake.bbclass @@ -1,7 +1,7 @@ # Path to the CMake file to process. OECMAKE_SOURCEPATH ??= "${S}" -DEPENDS_prepend = "cmake-native " +DEPENDS_prepend = "cmake-native ninja-native " B = "${WORKDIR}/build" # We need to unset CCACHE otherwise cmake gets too confused @@ -120,7 +120,7 @@ cmake_do_configure() { oecmake_sitefile= fi - cmake \ + cmake -GNinja \ $oecmake_sitefile \ ${OECMAKE_SOURCEPATH} \ -DCMAKE_INSTALL_PREFIX:PATH=${prefix} \ @@ -141,7 +141,7 @@ cmake_do_configure() { -Wno-dev } -do_compile[progress] = "percent" +do_compile[progress] = "outof:^\[(\d+)/(\d+)\]\s+" cmake_do_compile() { bbnote VERBOSE=1 cmake --build '${B}' --target ${OECMAKE_TARGET_COMPILE} -- ${EXTRA_OECMAKE_BUILD} VERBOSE=1 cmake --build '${B}' --target ${OECMAKE_TARGET_COMPILE} -- ${EXTRA_OECMAKE_BUILD}
The Ninja backend is faster than Make for large projects (takes three minutes off webkitgtk:do_compile for me), so use it by default. Signed-off-by: Ross Burton <ross.burton@intel.com> --- meta/classes/cmake.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) -- 2.11.0 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core