| Message ID | 20200924185414.28642-5-vsementsov@virtuozzo.com |
|---|---|
| State | Superseded |
| Headers | show |
| Series | coroutines: generate wrapper code | expand |
On 9/24/20 1:54 PM, Vladimir Sementsov-Ogievskiy wrote: > We have a very frequent pattern of creating a coroutine from a function > with several arguments: > > - create a structure to pack parameters > - create _entry function to call original function taking parameters > from struct > - do different magic to handle completion: set ret to NOT_DONE or > EINPROGRESS or use separate bool field > - fill the struct and create coroutine from _entry function with this > struct as a parameter > - do coroutine enter and BDRV_POLL_WHILE loop > > Let's reduce code duplication by generating coroutine wrappers. > > This patch adds scripts/block-coroutine-wrapper.py together with some > friends, which will generate functions with declared prototypes marked > by the 'generated_co_wrapper' specifier. > > The usage of new code generation is as follows: > > 1. define the coroutine function somewhere > > int coroutine_fn bdrv_co_NAME(...) {...} > > 2. declare in some header file > > int generated_co_wrapper bdrv_NAME(...); > > with same list of parameters (generated_co_wrapper is > defined in "include/block/block.h"). > > 3. Make sure the block_gen_c delaration in block/meson.build declaration > mentions the file with your marker function. > > Still, no function is now marked, this work is for the following > commit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > +++ b/docs/devel/block-coroutine-wrapper.rst > @@ -0,0 +1,54 @@ > +======================= > +block-coroutine-wrapper > +======================= > + > +A lot of functions in QEMU block layer (see ``block/*``) can only be > +called in coroutine context. Such functions are normally marked by the > +coroutine_fn specifier. Still, sometimes we need to call them from > +non-coroutine context; for this we need to start a coroutine, run the > +needed function from it and wait for coroutine finish in to finish in a > +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one > +void* argument. So for each coroutine_fn function which needs a > +non-coroutine interface, we should define a structure to pack the > +parameters, define a separate function to unpack the parameters and > +call the original function and finally define a new interface function > +with same list of arguments as original one, which will pack the > +parameters into a struct, create a coroutine, run it and wait in > +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand, > +so we have a script to generate them. > + > +++ b/scripts/block-coroutine-wrapper.py Reviewed-by: Eric Blake <eblake@redhat.com> -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3226 Virtualization: qemu.org | libvirt.org
24.09.2020 21:54, Vladimir Sementsov-Ogievskiy wrote: > We have a very frequent pattern of creating a coroutine from a function > with several arguments: > > - create a structure to pack parameters > - create _entry function to call original function taking parameters > from struct > - do different magic to handle completion: set ret to NOT_DONE or > EINPROGRESS or use separate bool field > - fill the struct and create coroutine from _entry function with this > struct as a parameter > - do coroutine enter and BDRV_POLL_WHILE loop > > Let's reduce code duplication by generating coroutine wrappers. > > This patch adds scripts/block-coroutine-wrapper.py together with some > friends, which will generate functions with declared prototypes marked > by the 'generated_co_wrapper' specifier. > > The usage of new code generation is as follows: > > 1. define the coroutine function somewhere > > int coroutine_fn bdrv_co_NAME(...) {...} > > 2. declare in some header file > > int generated_co_wrapper bdrv_NAME(...); > > with same list of parameters (generated_co_wrapper is > defined in "include/block/block.h"). > > 3. Make sure the block_gen_c delaration in block/meson.build > mentions the file with your marker function. > > Still, no function is now marked, this work is for the following > commit. > > Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> > --- > docs/devel/block-coroutine-wrapper.rst | 54 +++++++ > docs/devel/index.rst | 1 + > block/block-gen.h | 49 +++++++ > include/block/block.h | 10 ++ > block/meson.build | 8 ++ > scripts/block-coroutine-wrapper.py | 188 +++++++++++++++++++++++++ > 6 files changed, 310 insertions(+) > create mode 100644 docs/devel/block-coroutine-wrapper.rst > create mode 100644 block/block-gen.h > create mode 100644 scripts/block-coroutine-wrapper.py > > diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst > new file mode 100644 > index 0000000000..d09fff2cc5 > --- /dev/null > +++ b/docs/devel/block-coroutine-wrapper.rst > @@ -0,0 +1,54 @@ > +======================= > +block-coroutine-wrapper > +======================= > + > +A lot of functions in QEMU block layer (see ``block/*``) can only be > +called in coroutine context. Such functions are normally marked by the > +coroutine_fn specifier. Still, sometimes we need to call them from > +non-coroutine context; for this we need to start a coroutine, run the > +needed function from it and wait for coroutine finish in > +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one > +void* argument. So for each coroutine_fn function which needs a > +non-coroutine interface, we should define a structure to pack the > +parameters, define a separate function to unpack the parameters and > +call the original function and finally define a new interface function > +with same list of arguments as original one, which will pack the > +parameters into a struct, create a coroutine, run it and wait in > +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand, > +so we have a script to generate them. > + > +Usage > +===== > + > +Assume we have defined the ``coroutine_fn`` function > +``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it, > +called ``bdrv_foo(<same args>)``. In this case the script can help. To > +trigger the generation: > + > +1. You need ``bdrv_foo`` declaration somewhere (for example, in > + ``block/coroutines.h``) with the ``generated_co_wrapper`` mark, > + like this: > + > +.. code-block:: c > + > + int generated_co_wrapper bdrv_foo(<some args>); > + > +2. You need to feed this declaration to block-coroutine-wrapper script. > + For this, add the .h (or .c) file with the declaration to the > + ``input: files(...)`` list of ``block_gen_c`` target declaration in > + ``block/meson.build`` > + > +You are done. During the build, coroutine wrappers will be generated in > +``<BUILD_DIR>/block/block-gen.c``. > + > +Links > +===== > + > +1. The script location is ``scripts/block-coroutine-wrapper.py``. > + > +2. Generic place for private ``generated_co_wrapper`` declarations is > + ``block/coroutines.h``, for public declarations: > + ``include/block/block.h`` > + > +3. The core API of generated coroutine wrappers is placed in > + (not generated) ``block/block-gen.h`` > diff --git a/docs/devel/index.rst b/docs/devel/index.rst > index 04773ce076..cb0abe1e69 100644 > --- a/docs/devel/index.rst > +++ b/docs/devel/index.rst > @@ -31,3 +31,4 @@ Contents: > reset > s390-dasd-ipl > clocks > + block-coroutine-wrapper > diff --git a/block/block-gen.h b/block/block-gen.h > new file mode 100644 > index 0000000000..f80cf4897d > --- /dev/null > +++ b/block/block-gen.h > @@ -0,0 +1,49 @@ > +/* > + * Block coroutine wrapping core, used by auto-generated block/block-gen.c > + * > + * Copyright (c) 2003 Fabrice Bellard > + * Copyright (c) 2020 Virtuozzo International GmbH > + * > + * Permission is hereby granted, free of charge, to any person obtaining a copy > + * of this software and associated documentation files (the "Software"), to deal > + * in the Software without restriction, including without limitation the rights > + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > + * copies of the Software, and to permit persons to whom the Software is > + * furnished to do so, subject to the following conditions: > + * > + * The above copyright notice and this permission notice shall be included in > + * all copies or substantial portions of the Software. > + * > + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL > + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, > + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN > + * THE SOFTWARE. > + */ > + > +#ifndef BLOCK_BLOCK_GEN_H > +#define BLOCK_BLOCK_GEN_H > + > +#include "block/block_int.h" > + > +/* Base structure for argument packing structures */ > +typedef struct BdrvPollCo { > + BlockDriverState *bs; > + bool in_progress; > + int ret; > + Coroutine *co; /* Keep pointer here for debugging */ > +} BdrvPollCo; > + > +static inline int bdrv_poll_co(BdrvPollCo *s) > +{ > + assert(!qemu_in_coroutine()); > + > + bdrv_coroutine_enter(s->bs, s->co); > + BDRV_POLL_WHILE(s->bs, s->in_progress); > + > + return s->ret; > +} > + > +#endif /* BLOCK_BLOCK_GEN_H */ > diff --git a/include/block/block.h b/include/block/block.h > index 81d591dd4c..0f0ddc51b4 100644 > --- a/include/block/block.h > +++ b/include/block/block.h > @@ -10,6 +10,16 @@ > #include "block/blockjob.h" > #include "qemu/hbitmap.h" > > +/* > + * generated_co_wrapper > + * > + * Function specifier, which does nothing but mark functions to be > + * generated by scripts/block-coroutine-wrapper.py > + * > + * Read more in docs/devel/block-coroutine-wrapper.rst > + */ > +#define generated_co_wrapper > + > /* block.c */ > typedef struct BlockDriver BlockDriver; > typedef struct BdrvChild BdrvChild; > diff --git a/block/meson.build b/block/meson.build > index a3e56b7cd1..88ad73583a 100644 > --- a/block/meson.build > +++ b/block/meson.build > @@ -107,6 +107,14 @@ module_block_h = custom_target('module_block.h', > command: [module_block_py, '@OUTPUT0@', modsrc]) > block_ss.add(module_block_h) > > +wrapper_py = find_program('../scripts/block-coroutine-wrapper.py') > +block_gen_c = custom_target('block-gen.c', > + output: 'block-gen.c', > + input: files('../include/block/block.h', > + 'coroutines.h'), > + command: [wrapper_py, '@OUTPUT@', '@INPUT@']) > +block_ss.add(block_gen_c) > + > block_ss.add(files('stream.c')) > > softmmu_ss.add(files('qapi-sysemu.c')) > diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py > new file mode 100644 > index 0000000000..505e070660 > --- /dev/null > +++ b/scripts/block-coroutine-wrapper.py > @@ -0,0 +1,188 @@ > +#! /usr/bin/env python3 > +"""Generate coroutine wrappers for block subsystem. > + > +The program parses one or several concatenated c files from stdin, > +searches for functions with the 'generated_co_wrapper' specifier > +and generates corresponding wrappers on stdout. > + > +Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]... > + > +Copyright (c) 2020 Virtuozzo International GmbH. > + > +This program is free software; you can redistribute it and/or modify > +it under the terms of the GNU General Public License as published by > +the Free Software Foundation; either version 2 of the License, or > +(at your option) any later version. > + > +This program is distributed in the hope that it will be useful, > +but WITHOUT ANY WARRANTY; without even the implied warranty of > +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the > +GNU General Public License for more details. > + > +You should have received a copy of the GNU General Public License > +along with this program. If not, see <http://www.gnu.org/licenses/>. > +""" > + > +import sys > +import re > +import subprocess > +import json > +from typing import Iterator > + > + > +def prettify(code: str) -> str: > + """Prettify code using clang-format if available""" > + > + try: > + style = json.dumps({ > + 'IndentWidth': 4, > + 'BraceWrapping': {'AfterFunction': True}, > + 'BreakBeforeBraces': 'Custom', > + 'SortIncludes': False, > + 'MaxEmptyLinesToKeep': 2, > + }) > + p = subprocess.run(['clang-format', f'-style={style}'], check=True, > + encoding='utf-8', input=code, > + stdout=subprocess.PIPE) > + return p.stdout > + except FileNotFoundError: > + return code > + > + > +def gen_header(): > + copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL) > + copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE) > + copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE) > + return f"""\ > +/* > + * File is generated by scripts/block-coroutine-wrapper.py > + * > +{copyright} > + */ > + > +#include "qemu/osdep.h" > +#include "block/coroutines.h" > +#include "block/block-gen.h" > +#include "block/block_int.h"\ > +""" > + > + > +class ParamDecl: > + param_re = re.compile(r'(?P<decl>' > + r'(?P<type>.*[ *])' > + r'(?P<name>[a-z][a-z0-9_]*)' > + r')') > + > + def __init__(self, param_decl: str) -> None: > + m = self.param_re.match(param_decl.strip()) > + if m is None: > + raise ValueError(f'Wrong parameter declaration: "{param_decl}"') > + self.decl = m.group('decl') > + self.type = m.group('type') > + self.name = m.group('name') > + > + > +class FuncDecl: > + def __init__(self, return_type: str, name: str, args: str) -> None: > + self.return_type = return_type.strip() > + self.name = name.strip() > + self.args = [ParamDecl(arg.strip()) for arg in args.split(',')] > + > + def gen_list(self, format: str) -> str: > + return ', '.join(format.format_map(arg.__dict__) for arg in self.args) > + > + def gen_block(self, format: str) -> str: > + return '\n'.join(format.format_map(arg.__dict__) for arg in self.args) > + > + > +# Match wrappers declared with a generated_co_wrapper mark > +func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*' > + r'(?P<wrapper_name>[a-z][a-z0-9_]*)' > + r'\((?P<args>[^)]*)\);$', re.MULTILINE) > + > + > +def func_decl_iter(text: str) -> Iterator: > + for m in func_decl_re.finditer(text): > + yield FuncDecl(return_type='int', > + name=m.group('wrapper_name'), > + args=m.group('args')) > + > + > +def snake_to_camel(func_name: str) -> str: > + """ > + Convert underscore names like 'some_function_name' to camel-case like > + 'SomeFunctionName' > + """ > + words = func_name.split('_') > + words = [w[0].upper() + w[1:] for w in words] > + return ''.join(words) > + > + > +def gen_wrapper(func: FuncDecl) -> str: > + assert func.name.startswith('bdrv_') > + assert not func.name.startswith('bdrv_co_') > + assert func.return_type == 'int' > + assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *'] > + > + name = 'bdrv_co_' + func.name[5:] > + bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs' > + struct_name = snake_to_camel(name) > + > + return f"""\ > +/* > + * Wrappers for {name} > + */ > + > +typedef struct {struct_name} {{ > + BdrvPollCo poll_state; > +{ func.gen_block(' {decl};') } > +}} {struct_name}; > + > +static void coroutine_fn {name}_entry(void *opaque) > +{{ > + {struct_name} *s = opaque; > + > + s->poll_state.ret = {name}({ func.gen_list('s->{name}') }); > + s->poll_state.in_progress = false; > + > + aio_wait_kick(); > +}} > + > +int {func.name}({ func.gen_list('{decl}') }) > +{{ > + if (qemu_in_coroutine()) {{ > + return {name}({ func.gen_list('{name}') }); > + }} else {{ > + {struct_name} s = {{ > + .poll_state.bs = {bs}, > + .poll_state.in_progress = true, > + > +{ func.gen_block(' .{name} = {name},') } > + }}; > + > + s.poll_state.co = qemu_coroutine_create({name}_entry, &s); > + > + return bdrv_poll_co(&s.poll_state); > + }} > +}}""" > + > + > +def gen_wrappers(input_code: str) -> str: > + res = '' > + for func in func_decl_iter(input_code): > + res += '\n\n\n' > + res += gen_wrapper(func) > + > + return prettify(res) # prettify to wrap long lines > + > + > +if __name__ == '__main__': > + if len(sys.argv) < 3: > + exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...') > + > + with open(sys.argv[1], 'w') as f_out: > + f_out.write(gen_header()) > + for fname in sys.argv[2:]: > + with open(fname) as f_in: > + f_out.write(gen_wrappers(f_in.read())) > + f_out.write('\n') > probably we want encoding='utf-8' for both open calls.
diff --git a/docs/devel/block-coroutine-wrapper.rst b/docs/devel/block-coroutine-wrapper.rst new file mode 100644 index 0000000000..d09fff2cc5 --- /dev/null +++ b/docs/devel/block-coroutine-wrapper.rst @@ -0,0 +1,54 @@ +======================= +block-coroutine-wrapper +======================= + +A lot of functions in QEMU block layer (see ``block/*``) can only be +called in coroutine context. Such functions are normally marked by the +coroutine_fn specifier. Still, sometimes we need to call them from +non-coroutine context; for this we need to start a coroutine, run the +needed function from it and wait for coroutine finish in +BDRV_POLL_WHILE() loop. To run a coroutine we need a function with one +void* argument. So for each coroutine_fn function which needs a +non-coroutine interface, we should define a structure to pack the +parameters, define a separate function to unpack the parameters and +call the original function and finally define a new interface function +with same list of arguments as original one, which will pack the +parameters into a struct, create a coroutine, run it and wait in +BDRV_POLL_WHILE() loop. It's boring to create such wrappers by hand, +so we have a script to generate them. + +Usage +===== + +Assume we have defined the ``coroutine_fn`` function +``bdrv_co_foo(<some args>)`` and need a non-coroutine interface for it, +called ``bdrv_foo(<same args>)``. In this case the script can help. To +trigger the generation: + +1. You need ``bdrv_foo`` declaration somewhere (for example, in + ``block/coroutines.h``) with the ``generated_co_wrapper`` mark, + like this: + +.. code-block:: c + + int generated_co_wrapper bdrv_foo(<some args>); + +2. You need to feed this declaration to block-coroutine-wrapper script. + For this, add the .h (or .c) file with the declaration to the + ``input: files(...)`` list of ``block_gen_c`` target declaration in + ``block/meson.build`` + +You are done. During the build, coroutine wrappers will be generated in +``<BUILD_DIR>/block/block-gen.c``. + +Links +===== + +1. The script location is ``scripts/block-coroutine-wrapper.py``. + +2. Generic place for private ``generated_co_wrapper`` declarations is + ``block/coroutines.h``, for public declarations: + ``include/block/block.h`` + +3. The core API of generated coroutine wrappers is placed in + (not generated) ``block/block-gen.h`` diff --git a/docs/devel/index.rst b/docs/devel/index.rst index 04773ce076..cb0abe1e69 100644 --- a/docs/devel/index.rst +++ b/docs/devel/index.rst @@ -31,3 +31,4 @@ Contents: reset s390-dasd-ipl clocks + block-coroutine-wrapper diff --git a/block/block-gen.h b/block/block-gen.h new file mode 100644 index 0000000000..f80cf4897d --- /dev/null +++ b/block/block-gen.h @@ -0,0 +1,49 @@ +/* + * Block coroutine wrapping core, used by auto-generated block/block-gen.c + * + * Copyright (c) 2003 Fabrice Bellard + * Copyright (c) 2020 Virtuozzo International GmbH + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL + * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + +#ifndef BLOCK_BLOCK_GEN_H +#define BLOCK_BLOCK_GEN_H + +#include "block/block_int.h" + +/* Base structure for argument packing structures */ +typedef struct BdrvPollCo { + BlockDriverState *bs; + bool in_progress; + int ret; + Coroutine *co; /* Keep pointer here for debugging */ +} BdrvPollCo; + +static inline int bdrv_poll_co(BdrvPollCo *s) +{ + assert(!qemu_in_coroutine()); + + bdrv_coroutine_enter(s->bs, s->co); + BDRV_POLL_WHILE(s->bs, s->in_progress); + + return s->ret; +} + +#endif /* BLOCK_BLOCK_GEN_H */ diff --git a/include/block/block.h b/include/block/block.h index 81d591dd4c..0f0ddc51b4 100644 --- a/include/block/block.h +++ b/include/block/block.h @@ -10,6 +10,16 @@ #include "block/blockjob.h" #include "qemu/hbitmap.h" +/* + * generated_co_wrapper + * + * Function specifier, which does nothing but mark functions to be + * generated by scripts/block-coroutine-wrapper.py + * + * Read more in docs/devel/block-coroutine-wrapper.rst + */ +#define generated_co_wrapper + /* block.c */ typedef struct BlockDriver BlockDriver; typedef struct BdrvChild BdrvChild; diff --git a/block/meson.build b/block/meson.build index a3e56b7cd1..88ad73583a 100644 --- a/block/meson.build +++ b/block/meson.build @@ -107,6 +107,14 @@ module_block_h = custom_target('module_block.h', command: [module_block_py, '@OUTPUT0@', modsrc]) block_ss.add(module_block_h) +wrapper_py = find_program('../scripts/block-coroutine-wrapper.py') +block_gen_c = custom_target('block-gen.c', + output: 'block-gen.c', + input: files('../include/block/block.h', + 'coroutines.h'), + command: [wrapper_py, '@OUTPUT@', '@INPUT@']) +block_ss.add(block_gen_c) + block_ss.add(files('stream.c')) softmmu_ss.add(files('qapi-sysemu.c')) diff --git a/scripts/block-coroutine-wrapper.py b/scripts/block-coroutine-wrapper.py new file mode 100644 index 0000000000..505e070660 --- /dev/null +++ b/scripts/block-coroutine-wrapper.py @@ -0,0 +1,188 @@ +#! /usr/bin/env python3 +"""Generate coroutine wrappers for block subsystem. + +The program parses one or several concatenated c files from stdin, +searches for functions with the 'generated_co_wrapper' specifier +and generates corresponding wrappers on stdout. + +Usage: block-coroutine-wrapper.py generated-file.c FILE.[ch]... + +Copyright (c) 2020 Virtuozzo International GmbH. + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License as published by +the Free Software Foundation; either version 2 of the License, or +(at your option) any later version. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +GNU General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program. If not, see <http://www.gnu.org/licenses/>. +""" + +import sys +import re +import subprocess +import json +from typing import Iterator + + +def prettify(code: str) -> str: + """Prettify code using clang-format if available""" + + try: + style = json.dumps({ + 'IndentWidth': 4, + 'BraceWrapping': {'AfterFunction': True}, + 'BreakBeforeBraces': 'Custom', + 'SortIncludes': False, + 'MaxEmptyLinesToKeep': 2, + }) + p = subprocess.run(['clang-format', f'-style={style}'], check=True, + encoding='utf-8', input=code, + stdout=subprocess.PIPE) + return p.stdout + except FileNotFoundError: + return code + + +def gen_header(): + copyright = re.sub('^.*Copyright', 'Copyright', __doc__, flags=re.DOTALL) + copyright = re.sub('^(?=.)', ' * ', copyright.strip(), flags=re.MULTILINE) + copyright = re.sub('^$', ' *', copyright, flags=re.MULTILINE) + return f"""\ +/* + * File is generated by scripts/block-coroutine-wrapper.py + * +{copyright} + */ + +#include "qemu/osdep.h" +#include "block/coroutines.h" +#include "block/block-gen.h" +#include "block/block_int.h"\ +""" + + +class ParamDecl: + param_re = re.compile(r'(?P<decl>' + r'(?P<type>.*[ *])' + r'(?P<name>[a-z][a-z0-9_]*)' + r')') + + def __init__(self, param_decl: str) -> None: + m = self.param_re.match(param_decl.strip()) + if m is None: + raise ValueError(f'Wrong parameter declaration: "{param_decl}"') + self.decl = m.group('decl') + self.type = m.group('type') + self.name = m.group('name') + + +class FuncDecl: + def __init__(self, return_type: str, name: str, args: str) -> None: + self.return_type = return_type.strip() + self.name = name.strip() + self.args = [ParamDecl(arg.strip()) for arg in args.split(',')] + + def gen_list(self, format: str) -> str: + return ', '.join(format.format_map(arg.__dict__) for arg in self.args) + + def gen_block(self, format: str) -> str: + return '\n'.join(format.format_map(arg.__dict__) for arg in self.args) + + +# Match wrappers declared with a generated_co_wrapper mark +func_decl_re = re.compile(r'^int\s*generated_co_wrapper\s*' + r'(?P<wrapper_name>[a-z][a-z0-9_]*)' + r'\((?P<args>[^)]*)\);$', re.MULTILINE) + + +def func_decl_iter(text: str) -> Iterator: + for m in func_decl_re.finditer(text): + yield FuncDecl(return_type='int', + name=m.group('wrapper_name'), + args=m.group('args')) + + +def snake_to_camel(func_name: str) -> str: + """ + Convert underscore names like 'some_function_name' to camel-case like + 'SomeFunctionName' + """ + words = func_name.split('_') + words = [w[0].upper() + w[1:] for w in words] + return ''.join(words) + + +def gen_wrapper(func: FuncDecl) -> str: + assert func.name.startswith('bdrv_') + assert not func.name.startswith('bdrv_co_') + assert func.return_type == 'int' + assert func.args[0].type in ['BlockDriverState *', 'BdrvChild *'] + + name = 'bdrv_co_' + func.name[5:] + bs = 'bs' if func.args[0].type == 'BlockDriverState *' else 'child->bs' + struct_name = snake_to_camel(name) + + return f"""\ +/* + * Wrappers for {name} + */ + +typedef struct {struct_name} {{ + BdrvPollCo poll_state; +{ func.gen_block(' {decl};') } +}} {struct_name}; + +static void coroutine_fn {name}_entry(void *opaque) +{{ + {struct_name} *s = opaque; + + s->poll_state.ret = {name}({ func.gen_list('s->{name}') }); + s->poll_state.in_progress = false; + + aio_wait_kick(); +}} + +int {func.name}({ func.gen_list('{decl}') }) +{{ + if (qemu_in_coroutine()) {{ + return {name}({ func.gen_list('{name}') }); + }} else {{ + {struct_name} s = {{ + .poll_state.bs = {bs}, + .poll_state.in_progress = true, + +{ func.gen_block(' .{name} = {name},') } + }}; + + s.poll_state.co = qemu_coroutine_create({name}_entry, &s); + + return bdrv_poll_co(&s.poll_state); + }} +}}""" + + +def gen_wrappers(input_code: str) -> str: + res = '' + for func in func_decl_iter(input_code): + res += '\n\n\n' + res += gen_wrapper(func) + + return prettify(res) # prettify to wrap long lines + + +if __name__ == '__main__': + if len(sys.argv) < 3: + exit(f'Usage: {sys.argv[0]} OUT_FILE.c IN_FILE.[ch]...') + + with open(sys.argv[1], 'w') as f_out: + f_out.write(gen_header()) + for fname in sys.argv[2:]: + with open(fname) as f_in: + f_out.write(gen_wrappers(f_in.read())) + f_out.write('\n')
We have a very frequent pattern of creating a coroutine from a function with several arguments: - create a structure to pack parameters - create _entry function to call original function taking parameters from struct - do different magic to handle completion: set ret to NOT_DONE or EINPROGRESS or use separate bool field - fill the struct and create coroutine from _entry function with this struct as a parameter - do coroutine enter and BDRV_POLL_WHILE loop Let's reduce code duplication by generating coroutine wrappers. This patch adds scripts/block-coroutine-wrapper.py together with some friends, which will generate functions with declared prototypes marked by the 'generated_co_wrapper' specifier. The usage of new code generation is as follows: 1. define the coroutine function somewhere int coroutine_fn bdrv_co_NAME(...) {...} 2. declare in some header file int generated_co_wrapper bdrv_NAME(...); with same list of parameters (generated_co_wrapper is defined in "include/block/block.h"). 3. Make sure the block_gen_c delaration in block/meson.build mentions the file with your marker function. Still, no function is now marked, this work is for the following commit. Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com> --- docs/devel/block-coroutine-wrapper.rst | 54 +++++++ docs/devel/index.rst | 1 + block/block-gen.h | 49 +++++++ include/block/block.h | 10 ++ block/meson.build | 8 ++ scripts/block-coroutine-wrapper.py | 188 +++++++++++++++++++++++++ 6 files changed, 310 insertions(+) create mode 100644 docs/devel/block-coroutine-wrapper.rst create mode 100644 block/block-gen.h create mode 100644 scripts/block-coroutine-wrapper.py