| Message ID | 20221205132207.94775-1-andrew@aj.id.au |
|---|---|
| Headers | show |
| Series | Convert the build from autotools to meson | expand |
On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: > > Hello, > > Based on a recent poke [1] and in-between meetings I've put together a > WIP series that converts libgpiod's build from autotools to meson. As > far as I'm aware the meson build supports all the significant options to > enable or disable features exposed by the autotools build: > > * Tests > * Tools > * Interactive gpioset > * Bindings > * C++ > * Python > * Rust > * Documentation > * Manpages > * Doxygen > > [1] https://lore.kernel.org/all/CAMRc=Mda8UnyH+_GxeX_4MyKd+DPN0BVH5K+J+VWnMJNC1vwTQ@mail.gmail.com/ > > Meson has pretty good support for handling python and so the patch does > away with setup.py entirely. Eek! No, please do keep setup.py. Autotools too is capable of building python C extensions on its own and it's what we use in v1 but I want the python code to be built the standard python way. I actually plan to post libgpiod v2 on pypi and split out building python bindings into a separate bitbake recipe in meta-openembedded using the setuptools3 class. So let's keep setup.py and just call it from meson. > However, the rust case isn't quite so > simple. In order to handle the dependencies of the rust bindings I've > called out to cargo through a custom target. It's not great, but from > what I could see it seems to be the path of least resistance given > meson's support for rust. > > There's no support for installing the rust bindings through meson, but > this is not worse than the support we appeared to have under autotools. > I think Viresh too wants to keep cargo as the building agent for the rust code. > It's worth noting that you'll probably want to disable the rust bindings > if you need to run the install phase for libgpiod under e.g. sudo but > have used rustup to install cargo for your unpriviledged user. > Current autotools setup doesn't install rust bindings at all, can we keep it this way? > Also, if you've used rustup to install the rust toolchain you may also > need to install clang in order to pick up C toolchain headers for > consumption by bindgen. > > Anyway, feedback on the rust part is definitely appreciated. Maybe > there's a better approach? > Cc'ed Viresh and Kent. > Moving along, the following tests pass in their entirety in my test VM: > > * gpiod-test > * gpiod-cxx-test > * python -m gpiod.test > > I've also briefly compared the install trees for the autotools and meson > builds under some configurations. The differences are accounted for by > meson defaulting to multi-arch installation paths for shared objects and > picking the generic rather than interpreter-version-specific python3 > dist-packages directory under $PREFIX. Let me know if those seem > problematic. > > A complete meson setup invocation looks as follows: > > ``` > $ meson setup -Dbindings=cxx,python,rust -Ddocumentation=man,inline -Dexamples=true -Dtests=true -Dtools=true build > ``` > > Subsequently the build can be performed with: > > ``` > $ meson compile -C build > ``` > > Meson defaults to using ninja as its backend, and automatically exploits > ccache[2] when available to keep repeated builds speedy. > It does show! Full rebuild with autotools: real 0m43,902s user 2m40,010s sys 0m20,172s Full rebuild with meson: real 0m10,001s user 1m1,334s sys 0m12,205s More than 4x faster now. > [2] https://ccache.dev/ > > We end up with a net reduction of 254 LOC for the build system, and, > IMO, a single and fairly readable language to express it. Along with > that comes easy integration as a dependency in other (meson) projects > and a straight-forward path for their cross-compilation. > > Let me know what you think. Meson has a steep learning curve but I really want to move over to it now and will put in the time to learn it. Thanks for doing it. The patches are functional from what I tested so far. One thing I'd love to see changed is: put all API and ABI version number next to each other in a single place so that there's less risk of forgetting to update one of them when making a release. Is that possible? Bart > > Andrew > > Andrew Jeffery (2): > Introduce meson as a build system > Remove autotools in favour of meson > > Doxyfile.in | 2 +- > Makefile.am | 43 ---- > autogen.sh | 17 -- > bindings/Makefile.am | 22 -- > bindings/cxx/Makefile.am | 48 ---- > bindings/cxx/examples/Makefile.am | 26 --- > bindings/cxx/examples/meson.build | 9 + > bindings/cxx/gpiodcxx/Makefile.am | 20 -- > bindings/cxx/gpiodcxx/meson.build | 19 ++ > bindings/cxx/meson.build | 49 ++++ > bindings/cxx/tests/Makefile.am | 32 --- > bindings/cxx/tests/meson.build | 26 +++ > bindings/meson.build | 14 ++ > bindings/python/Makefile.am | 35 --- > bindings/python/examples/Makefile.am | 10 - > bindings/python/examples/meson.build | 12 + > bindings/python/gpiod/Makefile.am | 17 -- > bindings/python/gpiod/ext/Makefile.am | 11 - > bindings/python/gpiod/ext/meson.build | 14 ++ > bindings/python/gpiod/meson.build | 17 ++ > bindings/python/meson.build | 16 ++ > bindings/python/setup.py | 47 ---- > bindings/python/tests/Makefile.am | 17 -- > bindings/python/tests/gpiosim/Makefile.am | 7 - > bindings/python/tests/gpiosim/meson.build | 12 + > bindings/python/tests/meson.build | 17 ++ > bindings/rust/Makefile.am | 19 -- > bindings/rust/gpiosim-sys/build.rs | 9 +- > bindings/rust/libgpiod-sys/build.rs | 9 +- > bindings/rust/meson.build | 33 +++ > configure.ac | 272 ---------------------- > include/Makefile.am | 4 - > include/meson.build | 7 + > lib/Makefile.am | 27 --- > lib/meson.build | 30 +++ > man/Makefile.am | 16 -- > man/meson.build | 21 ++ > meson.build | 91 ++++++++ > meson_options.txt | 9 + > tests/Makefile.am | 34 --- > tests/gpiosim/Makefile.am | 16 -- > tests/gpiosim/meson.build | 24 ++ > tests/meson.build | 30 +++ > tools/Makefile.am | 39 ---- > tools/meson.build | 69 ++++++ > 45 files changed, 532 insertions(+), 786 deletions(-) > delete mode 100644 Makefile.am > delete mode 100755 autogen.sh > delete mode 100644 bindings/Makefile.am > delete mode 100644 bindings/cxx/Makefile.am > delete mode 100644 bindings/cxx/examples/Makefile.am > create mode 100644 bindings/cxx/examples/meson.build > delete mode 100644 bindings/cxx/gpiodcxx/Makefile.am > create mode 100644 bindings/cxx/gpiodcxx/meson.build > create mode 100644 bindings/cxx/meson.build > delete mode 100644 bindings/cxx/tests/Makefile.am > create mode 100644 bindings/cxx/tests/meson.build > create mode 100644 bindings/meson.build > delete mode 100644 bindings/python/Makefile.am > delete mode 100644 bindings/python/examples/Makefile.am > create mode 100644 bindings/python/examples/meson.build > delete mode 100644 bindings/python/gpiod/Makefile.am > delete mode 100644 bindings/python/gpiod/ext/Makefile.am > create mode 100644 bindings/python/gpiod/ext/meson.build > create mode 100644 bindings/python/gpiod/meson.build > create mode 100644 bindings/python/meson.build > delete mode 100644 bindings/python/setup.py > delete mode 100644 bindings/python/tests/Makefile.am > delete mode 100644 bindings/python/tests/gpiosim/Makefile.am > create mode 100644 bindings/python/tests/gpiosim/meson.build > create mode 100644 bindings/python/tests/meson.build > delete mode 100644 bindings/rust/Makefile.am > create mode 100644 bindings/rust/meson.build > delete mode 100644 configure.ac > delete mode 100644 include/Makefile.am > create mode 100644 include/meson.build > delete mode 100644 lib/Makefile.am > create mode 100644 lib/meson.build > delete mode 100644 man/Makefile.am > create mode 100644 man/meson.build > create mode 100644 meson.build > create mode 100644 meson_options.txt > delete mode 100644 tests/Makefile.am > delete mode 100644 tests/gpiosim/Makefile.am > create mode 100644 tests/gpiosim/meson.build > create mode 100644 tests/meson.build > delete mode 100644 tools/Makefile.am > create mode 100644 tools/meson.build > > -- > 2.37.2 >
+ Miguel/Bjorn. On 05-12-22, 19:55, Bartosz Golaszewski wrote: > On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: > > > > Hello, > > > > Based on a recent poke [1] and in-between meetings I've put together a > > WIP series that converts libgpiod's build from autotools to meson. As > > far as I'm aware the meson build supports all the significant options to > > enable or disable features exposed by the autotools build: > > > > * Tests > > * Tools > > * Interactive gpioset > > * Bindings > > * C++ > > * Python > > * Rust > > * Documentation > > * Manpages > > * Doxygen > > > > [1] https://lore.kernel.org/all/CAMRc=Mda8UnyH+_GxeX_4MyKd+DPN0BVH5K+J+VWnMJNC1vwTQ@mail.gmail.com/ > > > > Meson has pretty good support for handling python and so the patch does > > away with setup.py entirely. > > Eek! No, please do keep setup.py. Autotools too is capable of building > python C extensions on its own and it's what we use in v1 but I want > the python code to be built the standard python way. I actually plan > to post libgpiod v2 on pypi and split out building python bindings > into a separate bitbake recipe in meta-openembedded using the > setuptools3 class. > > So let's keep setup.py and just call it from meson. > > > However, the rust case isn't quite so > > simple. In order to handle the dependencies of the rust bindings I've > > called out to cargo through a custom target. It's not great, but from > > what I could see it seems to be the path of least resistance given > > meson's support for rust. > > > > There's no support for installing the rust bindings through meson, but > > this is not worse than the support we appeared to have under autotools. > > > > I think Viresh too wants to keep cargo as the building agent for the rust code. I am not the best guy to ask around Rust tooling in general and probably Kent / Miguel can help here. Sorry for the stupid question, but what does "installing the rust bindings" mean here ? FWIW, for me the only thing that matters is that we are able to build the rust bindings, along with Make, and run tests somehow to make sure nothing broke. Since this is a library crate, the user crate will mark its dependency and do the build itself too. > > It's worth noting that you'll probably want to disable the rust bindings > > if you need to run the install phase for libgpiod under e.g. sudo but > > have used rustup to install cargo for your unpriviledged user. > > > > Current autotools setup doesn't install rust bindings at all, can we > keep it this way? > > > Also, if you've used rustup to install the rust toolchain you may also > > need to install clang in order to pick up C toolchain headers for > > consumption by bindgen. Yeah, from what I remember, we do need clang support for bindgen. > > Anyway, feedback on the rust part is definitely appreciated. Maybe > > there's a better approach?
On Tue, Dec 06, 2022 at 09:10:34AM +0530, Viresh Kumar wrote: > On 06-12-22, 10:56, Andrew Jeffery wrote: > > My experience with rust is (unfortunately) superficial; the meson > > conversion achieves the same outcome as the autotools integration (runs > > `cargo build ...`). If that's enough then I don't think there are any > > further issues. > > That's all I care about at the moment. Lets see if Kent have something > to add to this, else it looks okay. > I've got nothing to add - calling cargo makes sense to me. Cheers, Kent.
On Tue, Dec 6, 2022 at 4:55 AM Kent Gibson <warthog618@gmail.com> wrote: > > I've got nothing to add - calling cargo makes sense to me. We do that too in Kbuild in some cases (though not for building). I think it is fine to delegate where it makes sense. One concern may be handling `-j` properly between different build systems. Not sure what Meson provides there. Cargo supports the GNU Make jobserver as a client. Cheers, Miguel
On Mon, Dec 05, 2022 at 07:55:29PM +0100, Bartosz Golaszewski wrote: > On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: ... > > Meson defaults to using ninja as its backend, and automatically exploits > > ccache[2] when available to keep repeated builds speedy. ...which is a bad idea for a clean build. > It does show! Full rebuild with autotools: > > real 0m43,902s > user 2m40,010s > sys 0m20,172s > > Full rebuild with meson: > > real 0m10,001s > user 1m1,334s > sys 0m12,205s > > More than 4x faster now. And risk to have a badly formed binaries (yes, very little risk, but > 0). > > [2] https://ccache.dev/ ccache has downside of its own use. If we have a common storage for ccache -- the collision is just matter of time (yes, have seen that in real life). OTOH requiring per-project ccache storage makes a little sense for the end user as they quite likely won't rebuild it many times.
Hi Andy, On Wed, 7 Dec 2022, at 01:24, Andy Shevchenko wrote: > On Mon, Dec 05, 2022 at 07:55:29PM +0100, Bartosz Golaszewski wrote: >> On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: > > ... > >> > Meson defaults to using ninja as its backend, and automatically exploits >> > ccache[2] when available to keep repeated builds speedy. > > ...which is a bad idea for a clean build. > >> It does show! Full rebuild with autotools: >> >> real 0m43,902s >> user 2m40,010s >> sys 0m20,172s >> >> Full rebuild with meson: >> >> real 0m10,001s >> user 1m1,334s >> sys 0m12,205s >> >> More than 4x faster now. > > And risk to have a badly formed binaries (yes, very little risk, but > 0). > >> > [2] https://ccache.dev/ > > ccache has downside of its own use. If we have a common storage for ccache -- > the collision is just matter of time (yes, have seen that in real life). > > OTOH requiring per-project ccache storage makes a little sense for the end user > as they quite likely won't rebuild it many times. Valid points. However I think they're addressed by: 1. Not installing ccache on the system, or 2. Overriding the auto-detection behaviour of `meson setup ...` Regarding 2, you can specify the CC and CXX environment variables to force its hand: ``` $ command -v ccache /usr/bin/ccache $ CC=cc CXX=c++ meson setup -Dbindings=cxx build The Meson build system Version: 0.63.0 ... C compiler for the host machine: cc (gcc 12.2.0 "cc (Ubuntu 12.2.0-3ubuntu1) 12.2.0") ... C++ compiler for the host machine: c++ (gcc 12.2.0 "c++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0") ... ``` Compared to the default behaviour: ``` $ meson setup -Dbindings=cxx build The Meson build system Version: 0.63.0 ... C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Ubuntu 12.2.0-3ubuntu1) 12.2.0") ... C++ compiler for the host machine: ccache c++ (gcc 12.2.0 "c++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0") ... ``` This use of the CC and CXX variables is covered in the documentation: https://mesonbuild.com/Feature-autodetection.html#ccache Andrew
On Wed, Dec 07, 2022 at 08:34:20AM +1030, Andrew Jeffery wrote: > On Wed, 7 Dec 2022, at 01:24, Andy Shevchenko wrote: > > On Mon, Dec 05, 2022 at 07:55:29PM +0100, Bartosz Golaszewski wrote: > >> On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: ... > >> > Meson defaults to using ninja as its backend, and automatically exploits > >> > ccache[2] when available to keep repeated builds speedy. > > > > ...which is a bad idea for a clean build. > > > >> It does show! Full rebuild with autotools: > >> > >> real 0m43,902s > >> user 2m40,010s > >> sys 0m20,172s > >> > >> Full rebuild with meson: > >> > >> real 0m10,001s > >> user 1m1,334s > >> sys 0m12,205s > >> > >> More than 4x faster now. > > > > And risk to have a badly formed binaries (yes, very little risk, but > 0). > > > >> > [2] https://ccache.dev/ > > > > ccache has downside of its own use. If we have a common storage for ccache -- > > the collision is just matter of time (yes, have seen that in real life). > > > > OTOH requiring per-project ccache storage makes a little sense for the end user > > as they quite likely won't rebuild it many times. > > Valid points. However I think they're addressed by: > > 1. Not installing ccache on the system, or > 2. Overriding the auto-detection behaviour of `meson setup ...` > > Regarding 2, you can specify the CC and CXX environment variables to force its hand: > > ``` > $ command -v ccache > /usr/bin/ccache > $ CC=cc CXX=c++ meson setup -Dbindings=cxx build > The Meson build system > Version: 0.63.0 > ... > C compiler for the host machine: cc (gcc 12.2.0 "cc (Ubuntu 12.2.0-3ubuntu1) 12.2.0") > ... > C++ compiler for the host machine: c++ (gcc 12.2.0 "c++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0") > ... > ``` > > Compared to the default behaviour: > > ``` > $ meson setup -Dbindings=cxx build > The Meson build system > Version: 0.63.0 > ... > C compiler for the host machine: ccache cc (gcc 12.2.0 "cc (Ubuntu 12.2.0-3ubuntu1) 12.2.0") > ... > C++ compiler for the host machine: ccache c++ (gcc 12.2.0 "c++ (Ubuntu 12.2.0-3ubuntu1) 12.2.0") > ... > ``` > > This use of the CC and CXX variables is covered in the documentation: > > https://mesonbuild.com/Feature-autodetection.html#ccache Right, my point that ccache should be opt-in and not opt-out. For example, Buidroot project has ccache support (as opt-in).
On Thu, 8 Dec 2022, at 19:57, Bartosz Golaszewski wrote: > On Thu, Dec 8, 2022 at 5:23 AM Andrew Jeffery <andrew@aj.id.au> wrote: >> >> >> >> On Tue, 6 Dec 2022, at 05:25, Bartosz Golaszewski wrote: >> > On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: >> >> >> >> Hello, >> >> >> >> Based on a recent poke [1] and in-between meetings I've put together a >> >> WIP series that converts libgpiod's build from autotools to meson. As >> >> far as I'm aware the meson build supports all the significant options to >> >> enable or disable features exposed by the autotools build: >> >> >> >> * Tests >> >> * Tools >> >> * Interactive gpioset >> >> * Bindings >> >> * C++ >> >> * Python >> >> * Rust >> >> * Documentation >> >> * Manpages >> >> * Doxygen >> >> >> >> [1] https://lore.kernel.org/all/CAMRc=Mda8UnyH+_GxeX_4MyKd+DPN0BVH5K+J+VWnMJNC1vwTQ@mail.gmail.com/ >> >> >> >> Meson has pretty good support for handling python and so the patch does >> >> away with setup.py entirely. >> > >> > Eek! No, please do keep setup.py. Autotools too is capable of building >> > python C extensions on its own and it's what we use in v1 but I want >> > the python code to be built the standard python way. I actually plan >> > to post libgpiod v2 on pypi and split out building python bindings >> > into a separate bitbake recipe in meta-openembedded using the >> > setuptools3 class. >> > >> > So let's keep setup.py and just call it from meson. >> >> I've poked at this for a little while and it's not a great experience. >> Meson's design pushes back against calling out in this way, and I don't >> really have the motivation to carry on fighting it to make it do what >> you request. Unless someone else has that motivation, I think there are >> two options if meson is still desired: >> >> 1. Use the meson python support as posted in this series >> 2. Split out the python (and probably rust) bindings, keeping the >> dependency relationships pointing in one direction and using the >> language's own package management tooling. >> >> Given there's nothing to do in the install phase for rust we don't have >> as big of an issue there, but it is problematic for python. >> >> Let me know which way you want to go, including if you want to abandon >> meson :) >> > > No, I don't want to abandon it. What is the problem exactly? Is meson > unable to simply add external commands to its ninja output? Not as far as I'm aware. I think it's best covered by this policy description: https://mesonbuild.com/Mixing-build-systems.html There are some things that might make it sound feasible but aren't actually appropriate: 1. run_command(): https://mesonbuild.com/Reference-manual_functions.html#run_command 2. run_target(): https://mesonbuild.com/Reference-manual_functions.html#run_target 3. custom_target(): https://mesonbuild.com/Reference-manual_functions.html#custom_target run_command() isn't appropriate as it executes in the `meson setup` phase. run_target() isn't appropriate as it disregards any output artifacts and so has no impact in the `meson install` phase. custom_target() is probably closest to what is required, but there's a lot of pain in trying to get the artifacts to line up for correct deployment in the `meson install` phase. This is exacerbated by the requirement that setup.py be run from its containing directory in the source tree. Further, I couldn't get all the options to line up such that setuptools would relocate its output into meson's own build tree (and out of the source tree). Here's a not entirely working attempt at abusing custom_target() to that end: ``` diff --git a/bindings/python/meson.build b/bindings/python/meson.build index 26f7ff13e0dd..136d10824345 100644 --- a/bindings/python/meson.build +++ b/bindings/python/meson.build @@ -3,14 +3,31 @@ python = import('python') python3 = python.find_installation('python3') -python3_dep = python3.dependency() -subdir('gpiod') +python_build_dir = 'python-build' +python_install_dir = 'python-install' +python_include_dirs = '../../include:../../tests/gpiosim' +python_lib_dirs = '@0@/lib:@0@/tests/gpiosim'.format(meson.project_build_root()) +python_install_cmd = [ python3.full_path(), '@INPUT@', '--no-user-cfg', + 'build_ext', '--include-dirs', python_include_dirs, '--library-dirs', python_lib_dirs, + 'install', '--root', python_build_dir, '--prefix', get_option('prefix')] -if get_option('examples') - subdir('examples') -endif +python_env = environment() +python_env.set('GPIOD_WITH_TESTS', get_option('tests').to_string()) -if get_option('tests') - subdir('tests') -endif +python_setuptools = custom_target('python-setuptools', + input: 'setup.py', + output: python_build_dir, + depends: [gpiod, gpiosim], + env: python_env, + command: python_install_cmd) + +cp = find_program('cp') + +custom_target('python-install', + input: 'setup.py', + output: python_install_dir, + depends: python_setuptools, + command: [ cp, '-r', meson.current_source_dir() / python_build_dir, meson.current_build_dir() / python_install_dir ], + install: true, + install_dir: get_option('prefix')) diff --git a/bindings/python/setup.py b/bindings/python/setup.py index ec8f99d4013d..9eddae7466a1 100644 --- a/bindings/python/setup.py +++ b/bindings/python/setup.py @@ -1,9 +1,14 @@ # SPDX-License-Identifier: GPL-2.0-or-later # SPDX-FileCopyrightText: 2022 Bartosz Golaszewski <brgl@bgdev.pl> -from os import environ +import os +import sys + +from os import environ, path from setuptools import setup, Extension, find_packages +os.chdir(path.dirname(sys.argv[0]) or '.') + gpiod_ext = Extension( "gpiod._ext", sources=[ ``` This commits a bunch of crimes: 1. Assumes the structure of the meson build directory via the paths in python_lib_dirs 2. Adds a chdir() in setup.py to relocate the process out of the meson build directory back into the source tree 3. Assumes the chdir() operation in the setup of python_include_dirs rather than relying on meson's built-in dependency tracking as the inc objects are opaque[1] 4. Hacks the setuptools output back into the meson build directory using a crufty target invoking `cp` so meson can locate the artifacts in the `meson install` phase 5. Still doesn't correctly install the artifacts in the end due to restrictions on path mangling (can't strip off the parent directory) and the fact that we're trying to install an entire tree rather than specific files. [1] https://mesonbuild.com/Reference-manual_returned_inc.html It might feel like install_data() or install_subdir() could be used here, but from experiment their behaviour also seems unfit to be used in this context. At least, that's what I've experimented with. Maybe others can see the way through here, but it really is fighting against the policy linked earlier. Andrew
On Thu, Dec 8, 2022 at 12:11 PM Andrew Jeffery <andrew@aj.id.au> wrote: > > > > On Thu, 8 Dec 2022, at 19:57, Bartosz Golaszewski wrote: > > On Thu, Dec 8, 2022 at 5:23 AM Andrew Jeffery <andrew@aj.id.au> wrote: > >> > >> > >> > >> On Tue, 6 Dec 2022, at 05:25, Bartosz Golaszewski wrote: > >> > On Mon, Dec 5, 2022 at 2:22 PM Andrew Jeffery <andrew@aj.id.au> wrote: > >> >> > >> >> Hello, > >> >> > >> >> Based on a recent poke [1] and in-between meetings I've put together a > >> >> WIP series that converts libgpiod's build from autotools to meson. As > >> >> far as I'm aware the meson build supports all the significant options to > >> >> enable or disable features exposed by the autotools build: > >> >> > >> >> * Tests > >> >> * Tools > >> >> * Interactive gpioset > >> >> * Bindings > >> >> * C++ > >> >> * Python > >> >> * Rust > >> >> * Documentation > >> >> * Manpages > >> >> * Doxygen > >> >> > >> >> [1] https://lore.kernel.org/all/CAMRc=Mda8UnyH+_GxeX_4MyKd+DPN0BVH5K+J+VWnMJNC1vwTQ@mail.gmail.com/ > >> >> > >> >> Meson has pretty good support for handling python and so the patch does > >> >> away with setup.py entirely. > >> > > >> > Eek! No, please do keep setup.py. Autotools too is capable of building > >> > python C extensions on its own and it's what we use in v1 but I want > >> > the python code to be built the standard python way. I actually plan > >> > to post libgpiod v2 on pypi and split out building python bindings > >> > into a separate bitbake recipe in meta-openembedded using the > >> > setuptools3 class. > >> > > >> > So let's keep setup.py and just call it from meson. > >> > >> I've poked at this for a little while and it's not a great experience. > >> Meson's design pushes back against calling out in this way, and I don't > >> really have the motivation to carry on fighting it to make it do what > >> you request. Unless someone else has that motivation, I think there are > >> two options if meson is still desired: > >> > >> 1. Use the meson python support as posted in this series > >> 2. Split out the python (and probably rust) bindings, keeping the > >> dependency relationships pointing in one direction and using the > >> language's own package management tooling. > >> > >> Given there's nothing to do in the install phase for rust we don't have > >> as big of an issue there, but it is problematic for python. > >> > >> Let me know which way you want to go, including if you want to abandon > >> meson :) > >> > > > > No, I don't want to abandon it. What is the problem exactly? Is meson > > unable to simply add external commands to its ninja output? > > Not as far as I'm aware. I think it's best covered by this policy > description: > > https://mesonbuild.com/Mixing-build-systems.html > > There are some things that might make it sound feasible but aren't > actually appropriate: > > 1. run_command(): https://mesonbuild.com/Reference-manual_functions.html#run_command > 2. run_target(): https://mesonbuild.com/Reference-manual_functions.html#run_target > 3. custom_target(): https://mesonbuild.com/Reference-manual_functions.html#custom_target > > run_command() isn't appropriate as it executes in the `meson setup` > phase. run_target() isn't appropriate as it disregards any output > artifacts and so has no impact in the `meson install` phase. > > custom_target() is probably closest to what is required, but there's a > lot of pain in trying to get the artifacts to line up for correct > deployment in the `meson install` phase. This is exacerbated by the > requirement that setup.py be run from its containing directory in the > source tree. Further, I couldn't get all the options to line up such > that setuptools would relocate its output into meson's own build tree > (and out of the source tree). Here's a not entirely working attempt at > abusing custom_target() to that end: > > ``` > diff --git a/bindings/python/meson.build b/bindings/python/meson.build > index 26f7ff13e0dd..136d10824345 100644 > --- a/bindings/python/meson.build > +++ b/bindings/python/meson.build > @@ -3,14 +3,31 @@ > > python = import('python') > python3 = python.find_installation('python3') > -python3_dep = python3.dependency() > > -subdir('gpiod') > +python_build_dir = 'python-build' > +python_install_dir = 'python-install' > +python_include_dirs = '../../include:../../tests/gpiosim' > +python_lib_dirs = '@0@/lib:@0@/tests/gpiosim'.format(meson.project_build_root()) > +python_install_cmd = [ python3.full_path(), '@INPUT@', '--no-user-cfg', > + 'build_ext', '--include-dirs', python_include_dirs, '--library-dirs', python_lib_dirs, > + 'install', '--root', python_build_dir, '--prefix', get_option('prefix')] > > -if get_option('examples') > - subdir('examples') > -endif > +python_env = environment() > +python_env.set('GPIOD_WITH_TESTS', get_option('tests').to_string()) > > -if get_option('tests') > - subdir('tests') > -endif > +python_setuptools = custom_target('python-setuptools', > + input: 'setup.py', > + output: python_build_dir, > + depends: [gpiod, gpiosim], > + env: python_env, > + command: python_install_cmd) > + > +cp = find_program('cp') > + > +custom_target('python-install', > + input: 'setup.py', > + output: python_install_dir, > + depends: python_setuptools, > + command: [ cp, '-r', meson.current_source_dir() / python_build_dir, meson.current_build_dir() / python_install_dir ], > + install: true, > + install_dir: get_option('prefix')) > diff --git a/bindings/python/setup.py b/bindings/python/setup.py > index ec8f99d4013d..9eddae7466a1 100644 > --- a/bindings/python/setup.py > +++ b/bindings/python/setup.py > @@ -1,9 +1,14 @@ > # SPDX-License-Identifier: GPL-2.0-or-later > # SPDX-FileCopyrightText: 2022 Bartosz Golaszewski <brgl@bgdev.pl> > > -from os import environ > +import os > +import sys > + > +from os import environ, path > from setuptools import setup, Extension, find_packages > > +os.chdir(path.dirname(sys.argv[0]) or '.') > + > gpiod_ext = Extension( > "gpiod._ext", > sources=[ > ``` > > This commits a bunch of crimes: > > 1. Assumes the structure of the meson build directory via the paths in > python_lib_dirs > 2. Adds a chdir() in setup.py to relocate the process out of the meson > build directory back into the source tree > 3. Assumes the chdir() operation in the setup of python_include_dirs > rather than relying on meson's built-in dependency tracking as the > inc objects are opaque[1] > 4. Hacks the setuptools output back into the meson build directory using > a crufty target invoking `cp` so meson can locate the artifacts in the > `meson install` phase > 5. Still doesn't correctly install the artifacts in the end due to > restrictions on path mangling (can't strip off the parent directory) > and the fact that we're trying to install an entire tree rather than > specific files. > > [1] https://mesonbuild.com/Reference-manual_returned_inc.html > > It might feel like install_data() or install_subdir() could be used > here, but from experiment their behaviour also seems unfit to be used > in this context. > > At least, that's what I've experimented with. Maybe others can see the > way through here, but it really is fighting against the policy linked > earlier. > > Andrew I see. I understand that meson doesn't like dealing with other build-systems. The thing I like about the current autotools setup is that with the following one-liner: ./autogen.sh --prefix=/tmp/gpio/inst --enable-bindings-cxx --enable-examples --enable-tests --enable-tools --enable-gpioset-interactive --enable-bindings-python --enable-bindings-rust && make -j16 && sudo ./tests/gpiod-test && sudo ./bindings/cxx/tests/gpiod-cxx-test && sudo PYTHONPATH=./bindings/python LD_LIBRARY_PATH=./lib/.libs/:./tests/gpiosim/.libs/:bindings/python/ python -B -m tests && cd bindings/rust/; sudo CARGO_TARGET_DIR=/tmp/libgpiod-rust PATH=/home/brgl/.cargo/bin/:$PATH /home/brgl/.cargo/bin/cargo test; cd ../.. && sudo ./tools/gpio-tools-test I can configure, build and test the entire code base while also using the language specific build tools for python and rust. I will try to play with your patches and maybe figure it out or even a close approximation of the current functionality but then again: I'm not well versed with meson yet. Between it and rust and dayjob my cup runneth over... Bartosz