| Message ID | 20241018130852.931509-1-peter.maydell@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
| Series | [v4] scripts/symlink-install-tree.py: Fix MESONINTROSPECT parsing | expand |
On 10/18/24 06:08, Peter Maydell wrote: > From: Akihiko Odaki <akihiko.odaki@daynix.com> > > The arguments in MESONINTROSPECT are quoted with shlex.quote() so it > must be parsed with shlex.split(). Otherwise the script will fail if > the build directory has a character like "~" in it. > > Note: this fix cannot be backported directly to any stable branch > that doesn't require Meson version 1.4.0 or better; otherwise it will > work OK on Linux but will break on Windows hosts. > > (Unfortunately, Meson prior to version 1.4.0 was inconsistent between > host OSes about how it quoted arguments, and used a different quoting > process on Windows hosts. Our current git trunk already requires > 1.5.0 as of commit 07f0d32641e ("Require meson version 1.5.0"), but > the stable branches are still on older Meson.) > > Fixes: cf60ccc330 ("cutils: Introduce bundle mechanism") > Reported-by: Michael Tokarev <mjt@tls.msk.ru> > Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> > [PMM: Updated commit message to give all the detail about the > Meson version compability requirements.] > Signed-off-by: Peter Maydell <peter.maydell@linaro.org> > --- > This is essentially back to version 1 of Akihiko's patch, now we > have a new enough Meson; I just updated the commit message. > https://patchew.org/QEMU/20230812061540.5398-1-akihiko.odaki@daynix.com/ > (I have dropped the various reviewed-by and tested-by headers because > I figured the passage of time was enough to make them moot.) > > scripts/symlink-install-tree.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/scripts/symlink-install-tree.py b/scripts/symlink-install-tree.py > index 8ed97e3c943..b72563895c5 100644 > --- a/scripts/symlink-install-tree.py > +++ b/scripts/symlink-install-tree.py > @@ -4,6 +4,7 @@ > import errno > import json > import os > +import shlex > import subprocess > import sys > > @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: > return str(PurePath(d1, *PurePath(d2).parts[1:])) > > introspect = os.environ.get('MESONINTROSPECT') > -out = subprocess.run([*introspect.split(' '), '--installed'], > +out = subprocess.run([*shlex.split(introspect), '--installed'], > stdout=subprocess.PIPE, check=True).stdout > for source, dest in json.loads(out).items(): > bundle_dest = destdir_join('qemu-bundle', dest) Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Tested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> I confirm this fixes the error when the path to qemu src tree contains '~'.
Hi, Le lun. 21 oct. 2024, 19:27, Pierrick Bouvier <pierrick.bouvier@linaro.org> a écrit : > On 10/18/24 06:08, Peter Maydell wrote: > > From: Akihiko Odaki <akihiko.odaki@daynix.com> > > > > The arguments in MESONINTROSPECT are quoted with shlex.quote() so it > > must be parsed with shlex.split(). Otherwise the script will fail if > > the build directory has a character like "~" in it. > > > > Note: this fix cannot be backported directly to any stable branch > > that doesn't require Meson version 1.4.0 or better; otherwise it will > > work OK on Linux but will break on Windows hosts. > > > > (Unfortunately, Meson prior to version 1.4.0 was inconsistent between > > host OSes about how it quoted arguments, and used a different quoting > > process on Windows hosts. Our current git trunk already requires > > 1.5.0 as of commit 07f0d32641e ("Require meson version 1.5.0"), but > > the stable branches are still on older Meson.) > > > > Fixes: cf60ccc330 ("cutils: Introduce bundle mechanism") > > Reported-by: Michael Tokarev <mjt@tls.msk.ru> > > Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> > > [PMM: Updated commit message to give all the detail about the > > Meson version compability requirements.] > > Signed-off-by: Peter Maydell <peter.maydell@linaro.org> > > --- > > This is essentially back to version 1 of Akihiko's patch, now we > > have a new enough Meson; I just updated the commit message. > > > https://patchew.org/QEMU/20230812061540.5398-1-akihiko.odaki@daynix.com/ > > (I have dropped the various reviewed-by and tested-by headers because > > I figured the passage of time was enough to make them moot.) > > > > scripts/symlink-install-tree.py | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/scripts/symlink-install-tree.py > b/scripts/symlink-install-tree.py > > index 8ed97e3c943..b72563895c5 100644 > > --- a/scripts/symlink-install-tree.py > > +++ b/scripts/symlink-install-tree.py > > @@ -4,6 +4,7 @@ > > import errno > > import json > > import os > > +import shlex > > import subprocess > > import sys > > > > @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: > > return str(PurePath(d1, *PurePath(d2).parts[1:])) > > > > introspect = os.environ.get('MESONINTROSPECT') > > -out = subprocess.run([*introspect.split(' '), '--installed'], > > +out = subprocess.run([*shlex.split(introspect), '--installed'], > > stdout=subprocess.PIPE, check=True).stdout > > for source, dest in json.loads(out).items(): > > bundle_dest = destdir_join('qemu-bundle', dest) > > Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> > Tested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> > > I confirm this fixes the error when the path to qemu src tree contains '~'. > I confirm this fixes the problem we detected in our CI (where some build dirnames have '~' Thanks >
On 10/18/24 06:08, Peter Maydell wrote: > From: Akihiko Odaki <akihiko.odaki@daynix.com> > > The arguments in MESONINTROSPECT are quoted with shlex.quote() so it > must be parsed with shlex.split(). Otherwise the script will fail if > the build directory has a character like "~" in it. > > Note: this fix cannot be backported directly to any stable branch > that doesn't require Meson version 1.4.0 or better; otherwise it will > work OK on Linux but will break on Windows hosts. > > (Unfortunately, Meson prior to version 1.4.0 was inconsistent between > host OSes about how it quoted arguments, and used a different quoting > process on Windows hosts. Our current git trunk already requires > 1.5.0 as of commit 07f0d32641e ("Require meson version 1.5.0"), but > the stable branches are still on older Meson.) > > Fixes: cf60ccc330 ("cutils: Introduce bundle mechanism") > Reported-by: Michael Tokarev <mjt@tls.msk.ru> > Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> > [PMM: Updated commit message to give all the detail about the > Meson version compability requirements.] > Signed-off-by: Peter Maydell <peter.maydell@linaro.org> > --- > This is essentially back to version 1 of Akihiko's patch, now we > have a new enough Meson; I just updated the commit message. > https://patchew.org/QEMU/20230812061540.5398-1-akihiko.odaki@daynix.com/ > (I have dropped the various reviewed-by and tested-by headers because > I figured the passage of time was enough to make them moot.) > > scripts/symlink-install-tree.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/scripts/symlink-install-tree.py b/scripts/symlink-install-tree.py > index 8ed97e3c943..b72563895c5 100644 > --- a/scripts/symlink-install-tree.py > +++ b/scripts/symlink-install-tree.py > @@ -4,6 +4,7 @@ > import errno > import json > import os > +import shlex > import subprocess > import sys > > @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: > return str(PurePath(d1, *PurePath(d2).parts[1:])) > > introspect = os.environ.get('MESONINTROSPECT') > -out = subprocess.run([*introspect.split(' '), '--installed'], > +out = subprocess.run([*shlex.split(introspect), '--installed'], > stdout=subprocess.PIPE, check=True).stdout > for source, dest in json.loads(out).items(): > bundle_dest = destdir_join('qemu-bundle', dest) Hi, would that be possible to pull this patch please? It's currently blocking the devs who reported it initially. Thanks, Pierrick
On Mon, 28 Oct 2024 at 19:43, Pierrick Bouvier <pierrick.bouvier@linaro.org> wrote: > > On 10/18/24 06:08, Peter Maydell wrote: > > From: Akihiko Odaki <akihiko.odaki@daynix.com> > > > > The arguments in MESONINTROSPECT are quoted with shlex.quote() so it > > must be parsed with shlex.split(). Otherwise the script will fail if > > the build directory has a character like "~" in it. > > > > Note: this fix cannot be backported directly to any stable branch > > that doesn't require Meson version 1.4.0 or better; otherwise it will > > work OK on Linux but will break on Windows hosts. > > > > (Unfortunately, Meson prior to version 1.4.0 was inconsistent between > > host OSes about how it quoted arguments, and used a different quoting > > process on Windows hosts. Our current git trunk already requires > > 1.5.0 as of commit 07f0d32641e ("Require meson version 1.5.0"), but > > the stable branches are still on older Meson.) > > > > Fixes: cf60ccc330 ("cutils: Introduce bundle mechanism") > > Reported-by: Michael Tokarev <mjt@tls.msk.ru> > > Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> > > [PMM: Updated commit message to give all the detail about the > > Meson version compability requirements.] > > Signed-off-by: Peter Maydell <peter.maydell@linaro.org> > > --- > > This is essentially back to version 1 of Akihiko's patch, now we > > have a new enough Meson; I just updated the commit message. > > https://patchew.org/QEMU/20230812061540.5398-1-akihiko.odaki@daynix.com/ > > (I have dropped the various reviewed-by and tested-by headers because > > I figured the passage of time was enough to make them moot.) > > > > scripts/symlink-install-tree.py | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > diff --git a/scripts/symlink-install-tree.py b/scripts/symlink-install-tree.py > > index 8ed97e3c943..b72563895c5 100644 > > --- a/scripts/symlink-install-tree.py > > +++ b/scripts/symlink-install-tree.py > > @@ -4,6 +4,7 @@ > > import errno > > import json > > import os > > +import shlex > > import subprocess > > import sys > > > > @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: > > return str(PurePath(d1, *PurePath(d2).parts[1:])) > > > > introspect = os.environ.get('MESONINTROSPECT') > > -out = subprocess.run([*introspect.split(' '), '--installed'], > > +out = subprocess.run([*shlex.split(introspect), '--installed'], > > stdout=subprocess.PIPE, check=True).stdout > > for source, dest in json.loads(out).items(): > > bundle_dest = destdir_join('qemu-bundle', dest) > > Hi, > > would that be possible to pull this patch please? > It's currently blocking the devs who reported it initially. Yes, I'll put it in my upcoming pullreq. thanks -- PMM
On 10/29/24 03:04, Peter Maydell wrote: > On Mon, 28 Oct 2024 at 19:43, Pierrick Bouvier > <pierrick.bouvier@linaro.org> wrote: >> >> On 10/18/24 06:08, Peter Maydell wrote: >>> From: Akihiko Odaki <akihiko.odaki@daynix.com> >>> >>> The arguments in MESONINTROSPECT are quoted with shlex.quote() so it >>> must be parsed with shlex.split(). Otherwise the script will fail if >>> the build directory has a character like "~" in it. >>> >>> Note: this fix cannot be backported directly to any stable branch >>> that doesn't require Meson version 1.4.0 or better; otherwise it will >>> work OK on Linux but will break on Windows hosts. >>> >>> (Unfortunately, Meson prior to version 1.4.0 was inconsistent between >>> host OSes about how it quoted arguments, and used a different quoting >>> process on Windows hosts. Our current git trunk already requires >>> 1.5.0 as of commit 07f0d32641e ("Require meson version 1.5.0"), but >>> the stable branches are still on older Meson.) >>> >>> Fixes: cf60ccc330 ("cutils: Introduce bundle mechanism") >>> Reported-by: Michael Tokarev <mjt@tls.msk.ru> >>> Signed-off-by: Akihiko Odaki <akihiko.odaki@daynix.com> >>> [PMM: Updated commit message to give all the detail about the >>> Meson version compability requirements.] >>> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> >>> --- >>> This is essentially back to version 1 of Akihiko's patch, now we >>> have a new enough Meson; I just updated the commit message. >>> https://patchew.org/QEMU/20230812061540.5398-1-akihiko.odaki@daynix.com/ >>> (I have dropped the various reviewed-by and tested-by headers because >>> I figured the passage of time was enough to make them moot.) >>> >>> scripts/symlink-install-tree.py | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> diff --git a/scripts/symlink-install-tree.py b/scripts/symlink-install-tree.py >>> index 8ed97e3c943..b72563895c5 100644 >>> --- a/scripts/symlink-install-tree.py >>> +++ b/scripts/symlink-install-tree.py >>> @@ -4,6 +4,7 @@ >>> import errno >>> import json >>> import os >>> +import shlex >>> import subprocess >>> import sys >>> >>> @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: >>> return str(PurePath(d1, *PurePath(d2).parts[1:])) >>> >>> introspect = os.environ.get('MESONINTROSPECT') >>> -out = subprocess.run([*introspect.split(' '), '--installed'], >>> +out = subprocess.run([*shlex.split(introspect), '--installed'], >>> stdout=subprocess.PIPE, check=True).stdout >>> for source, dest in json.loads(out).items(): >>> bundle_dest = destdir_join('qemu-bundle', dest) >> >> Hi, >> >> would that be possible to pull this patch please? >> It's currently blocking the devs who reported it initially. > > Yes, I'll put it in my upcoming pullreq. > > thanks > -- PMM Thanks a lot Peter :).
diff --git a/scripts/symlink-install-tree.py b/scripts/symlink-install-tree.py index 8ed97e3c943..b72563895c5 100644 --- a/scripts/symlink-install-tree.py +++ b/scripts/symlink-install-tree.py @@ -4,6 +4,7 @@ import errno import json import os +import shlex import subprocess import sys @@ -14,7 +15,7 @@ def destdir_join(d1: str, d2: str) -> str: return str(PurePath(d1, *PurePath(d2).parts[1:])) introspect = os.environ.get('MESONINTROSPECT') -out = subprocess.run([*introspect.split(' '), '--installed'], +out = subprocess.run([*shlex.split(introspect), '--installed'], stdout=subprocess.PIPE, check=True).stdout for source, dest in json.loads(out).items(): bundle_dest = destdir_join('qemu-bundle', dest)