| Message ID | 1599661096-127913-1-git-send-email-zhengchuan@huawei.com |
|---|---|
| Headers | show |
| Series | *** A Method for evaluating dirty page rate *** | expand |
Chuan Zheng <zhengchuan@huawei.com> 于2020年9月9日周三 下午10:14写道： > > Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be called > > Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> > --- > migration/dirtyrate.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++ > qapi/migration.json | 50 +++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 112 insertions(+) > > diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c > index 2f9ac34..e9e9e35 100644 > --- a/migration/dirtyrate.c > +++ b/migration/dirtyrate.c > @@ -61,6 +61,24 @@ static int dirtyrate_set_state(int *state, int old_state, int new_state) > } > } > > +static struct DirtyRateInfo *query_dirty_rate_info(void) > +{ > + int64_t dirty_rate = DirtyStat.dirty_rate; > + struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo)); > + > + if (CalculatingState == DIRTY_RATE_STATUS_MEASURED) { Should we use atomic read 'CalculatingState'? The qmp thread maybe run with the sampled thread. > + info->dirty_rate = dirty_rate; > + } else { > + info->dirty_rate = -1; > + } > + > + info->status = CalculatingState; > + info->start_time = DirtyStat.start_time; > + info->calc_time = DirtyStat.calc_time; > + > + return info; > +} > + > static void reset_dirtyrate_stat(void) > { > DirtyStat.total_dirty_samples = 0; > @@ -331,6 +349,8 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) > > msec = config.sample_period_seconds * 1000; > msec = set_sample_page_period(msec, initial_time); > + DirtyStat.start_time = initial_time / 1000; > + DirtyStat.calc_time = msec / 1000; > > rcu_read_lock(); > if (compare_page_hash_info(block_dinfo, block_index) < 0) { > @@ -362,3 +382,45 @@ void *get_dirtyrate_thread(void *arg) > DIRTY_RATE_STATUS_MEASURED); > return NULL; > } > + > +void qmp_calc_dirty_rate(int64_t calc_time, Error **errp) > +{ > + static struct DirtyRateConfig config; > + QemuThread thread; > + int ret; > + > + /* > + * If the dirty rate is already being measured, don't attempt to start. > + */ > + if (CalculatingState == DIRTY_RATE_STATUS_MEASURING) { atomic read? > + error_setg(errp, "the dirty rate is already being measured."); > + return; > + } > + > + if (!get_sample_page_period(calc_time)) { > + error_setg(errp, "calc-time is out of range[%d, %d].", > + MIN_FETCH_DIRTYRATE_TIME_SEC, > + MAX_FETCH_DIRTYRATE_TIME_SEC); > + return; > + } > + > + /* > + * Init calculation state as unstarted. > + */ > + ret = dirtyrate_set_state(&CalculatingState, CalculatingState, > + DIRTY_RATE_STATUS_UNSTARTED); > + if (ret == -1) { > + error_setg(errp, "init dirty rate calculation state failed."); > + return; > + } > + > + config.sample_period_seconds = calc_time; > + config.sample_pages_per_gigabytes = DIRTYRATE_DEFAULT_SAMPLE_PAGES; > + qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, > + (void *)&config, QEMU_THREAD_DETACHED); > +} > + > +struct DirtyRateInfo *qmp_query_dirty_rate(Error **errp) > +{ > + return query_dirty_rate_info(); > +} > diff --git a/qapi/migration.json b/qapi/migration.json > index 061ff25..4b980a0 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -1737,3 +1737,53 @@ > ## > { 'enum': 'DirtyRateStatus', > 'data': [ 'unstarted', 'measuring', 'measured'] } > + > +## > +# @DirtyRateInfo: > +# > +# Information about current dirty page rate of vm. > +# > +# @dirty-rate: @dirtyrate describing the dirty page rate of vm > +# in units of MB/s. > +# If this field return '-1', it means querying is not > +# start or not complete. > +# > +# @status: status containing dirtyrate query status includes > +# 'unstarted' or 'measuring' or 'measured' > +# > +# @start-time: start time in units of second for calculation > +# > +# @calc-time: time in units of second for sample dirty pages > +# > +# Since: 5.2 > +# > +## > +{ 'struct': 'DirtyRateInfo', > + 'data': {'dirty-rate': 'int64', > + 'status': 'DirtyRateStatus', > + 'start-time': 'int64', > + 'calc-time': 'int64'} } > + > +## > +# @calc-dirty-rate: > +# > +# start calculating dirty page rate for vm > +# > +# @calc-time: time in units of second for sample dirty pages > +# > +# Since: 5.2 > +# > +# Example: > +# {"command": "calc-dirty-rate", "data": {"calc-time": 1} } > +# > +## > +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} } > + > +## > +# @query-dirty-rate: > +# > +# query dirty page rate in units of MB/s for vm > +# > +# Since: 5.2 > +## > +{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' } > -- > 1.8.3.1 > >
* Chuan Zheng (zhengchuan@huawei.com) wrote: > Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be called > > Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> THis is looking OK by me, Eric: are you happy with the qapi stuff now? Dave > --- > migration/dirtyrate.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++ > qapi/migration.json | 50 +++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 112 insertions(+) > > diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c > index 2f9ac34..e9e9e35 100644 > --- a/migration/dirtyrate.c > +++ b/migration/dirtyrate.c > @@ -61,6 +61,24 @@ static int dirtyrate_set_state(int *state, int old_state, int new_state) > } > } > > +static struct DirtyRateInfo *query_dirty_rate_info(void) > +{ > + int64_t dirty_rate = DirtyStat.dirty_rate; > + struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo)); > + > + if (CalculatingState == DIRTY_RATE_STATUS_MEASURED) { > + info->dirty_rate = dirty_rate; > + } else { > + info->dirty_rate = -1; > + } > + > + info->status = CalculatingState; > + info->start_time = DirtyStat.start_time; > + info->calc_time = DirtyStat.calc_time; > + > + return info; > +} > + > static void reset_dirtyrate_stat(void) > { > DirtyStat.total_dirty_samples = 0; > @@ -331,6 +349,8 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) > > msec = config.sample_period_seconds * 1000; > msec = set_sample_page_period(msec, initial_time); > + DirtyStat.start_time = initial_time / 1000; > + DirtyStat.calc_time = msec / 1000; > > rcu_read_lock(); > if (compare_page_hash_info(block_dinfo, block_index) < 0) { > @@ -362,3 +382,45 @@ void *get_dirtyrate_thread(void *arg) > DIRTY_RATE_STATUS_MEASURED); > return NULL; > } > + > +void qmp_calc_dirty_rate(int64_t calc_time, Error **errp) > +{ > + static struct DirtyRateConfig config; > + QemuThread thread; > + int ret; > + > + /* > + * If the dirty rate is already being measured, don't attempt to start. > + */ > + if (CalculatingState == DIRTY_RATE_STATUS_MEASURING) { > + error_setg(errp, "the dirty rate is already being measured."); > + return; > + } > + > + if (!get_sample_page_period(calc_time)) { > + error_setg(errp, "calc-time is out of range[%d, %d].", > + MIN_FETCH_DIRTYRATE_TIME_SEC, > + MAX_FETCH_DIRTYRATE_TIME_SEC); > + return; > + } > + > + /* > + * Init calculation state as unstarted. > + */ > + ret = dirtyrate_set_state(&CalculatingState, CalculatingState, > + DIRTY_RATE_STATUS_UNSTARTED); > + if (ret == -1) { > + error_setg(errp, "init dirty rate calculation state failed."); > + return; > + } > + > + config.sample_period_seconds = calc_time; > + config.sample_pages_per_gigabytes = DIRTYRATE_DEFAULT_SAMPLE_PAGES; > + qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, > + (void *)&config, QEMU_THREAD_DETACHED); > +} > + > +struct DirtyRateInfo *qmp_query_dirty_rate(Error **errp) > +{ > + return query_dirty_rate_info(); > +} > diff --git a/qapi/migration.json b/qapi/migration.json > index 061ff25..4b980a0 100644 > --- a/qapi/migration.json > +++ b/qapi/migration.json > @@ -1737,3 +1737,53 @@ > ## > { 'enum': 'DirtyRateStatus', > 'data': [ 'unstarted', 'measuring', 'measured'] } > + > +## > +# @DirtyRateInfo: > +# > +# Information about current dirty page rate of vm. > +# > +# @dirty-rate: @dirtyrate describing the dirty page rate of vm > +# in units of MB/s. > +# If this field return '-1', it means querying is not > +# start or not complete. > +# > +# @status: status containing dirtyrate query status includes > +# 'unstarted' or 'measuring' or 'measured' > +# > +# @start-time: start time in units of second for calculation > +# > +# @calc-time: time in units of second for sample dirty pages > +# > +# Since: 5.2 > +# > +## > +{ 'struct': 'DirtyRateInfo', > + 'data': {'dirty-rate': 'int64', > + 'status': 'DirtyRateStatus', > + 'start-time': 'int64', > + 'calc-time': 'int64'} } > + > +## > +# @calc-dirty-rate: > +# > +# start calculating dirty page rate for vm > +# > +# @calc-time: time in units of second for sample dirty pages > +# > +# Since: 5.2 > +# > +# Example: > +# {"command": "calc-dirty-rate", "data": {"calc-time": 1} } > +# > +## > +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} } > + > +## > +# @query-dirty-rate: > +# > +# query dirty page rate in units of MB/s for vm > +# > +# Since: 5.2 > +## > +{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' } > -- > 1.8.3.1 >
On 2020/9/10 22:27, Li Qiang wrote: > Chuan Zheng <zhengchuan@huawei.com> 于2020年9月9日周三 下午10:14写道： >> >> Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function which could be called >> >> Signed-off-by: Chuan Zheng <zhengchuan@huawei.com> >> --- >> migration/dirtyrate.c | 62 +++++++++++++++++++++++++++++++++++++++++++++++++++ >> qapi/migration.json | 50 +++++++++++++++++++++++++++++++++++++++++ >> 2 files changed, 112 insertions(+) >> >> diff --git a/migration/dirtyrate.c b/migration/dirtyrate.c >> index 2f9ac34..e9e9e35 100644 >> --- a/migration/dirtyrate.c >> +++ b/migration/dirtyrate.c >> @@ -61,6 +61,24 @@ static int dirtyrate_set_state(int *state, int old_state, int new_state) >> } >> } >> >> +static struct DirtyRateInfo *query_dirty_rate_info(void) >> +{ >> + int64_t dirty_rate = DirtyStat.dirty_rate; >> + struct DirtyRateInfo *info = g_malloc0(sizeof(DirtyRateInfo)); >> + >> + if (CalculatingState == DIRTY_RATE_STATUS_MEASURED) { > > Should we use atomic read 'CalculatingState'? The qmp thread maybe run > with the sampled thread. > OK, will optimize it in V8 >> + info->dirty_rate = dirty_rate; >> + } else { >> + info->dirty_rate = -1; >> + } >> + >> + info->status = CalculatingState; >> + info->start_time = DirtyStat.start_time; >> + info->calc_time = DirtyStat.calc_time; >> + >> + return info; >> +} >> + > > >> static void reset_dirtyrate_stat(void) >> { >> DirtyStat.total_dirty_samples = 0; >> @@ -331,6 +349,8 @@ static void calculate_dirtyrate(struct DirtyRateConfig config) >> >> msec = config.sample_period_seconds * 1000; >> msec = set_sample_page_period(msec, initial_time); >> + DirtyStat.start_time = initial_time / 1000; >> + DirtyStat.calc_time = msec / 1000; >> >> rcu_read_lock(); >> if (compare_page_hash_info(block_dinfo, block_index) < 0) { >> @@ -362,3 +382,45 @@ void *get_dirtyrate_thread(void *arg) >> DIRTY_RATE_STATUS_MEASURED); >> return NULL; >> } >> + >> +void qmp_calc_dirty_rate(int64_t calc_time, Error **errp) >> +{ >> + static struct DirtyRateConfig config; >> + QemuThread thread; >> + int ret; >> + >> + /* >> + * If the dirty rate is already being measured, don't attempt to start. >> + */ >> + if (CalculatingState == DIRTY_RATE_STATUS_MEASURING) { > > atomic read? > >> + error_setg(errp, "the dirty rate is already being measured."); >> + return; >> + } >> + >> + if (!get_sample_page_period(calc_time)) { >> + error_setg(errp, "calc-time is out of range[%d, %d].", >> + MIN_FETCH_DIRTYRATE_TIME_SEC, >> + MAX_FETCH_DIRTYRATE_TIME_SEC); >> + return; >> + } >> + >> + /* >> + * Init calculation state as unstarted. >> + */ >> + ret = dirtyrate_set_state(&CalculatingState, CalculatingState, >> + DIRTY_RATE_STATUS_UNSTARTED); >> + if (ret == -1) { >> + error_setg(errp, "init dirty rate calculation state failed."); >> + return; >> + } >> + >> + config.sample_period_seconds = calc_time; >> + config.sample_pages_per_gigabytes = DIRTYRATE_DEFAULT_SAMPLE_PAGES; >> + qemu_thread_create(&thread, "get_dirtyrate", get_dirtyrate_thread, >> + (void *)&config, QEMU_THREAD_DETACHED); >> +} >> + >> +struct DirtyRateInfo *qmp_query_dirty_rate(Error **errp) >> +{ >> + return query_dirty_rate_info(); >> +} >> diff --git a/qapi/migration.json b/qapi/migration.json >> index 061ff25..4b980a0 100644 >> --- a/qapi/migration.json >> +++ b/qapi/migration.json >> @@ -1737,3 +1737,53 @@ >> ## >> { 'enum': 'DirtyRateStatus', >> 'data': [ 'unstarted', 'measuring', 'measured'] } >> + >> +## >> +# @DirtyRateInfo: >> +# >> +# Information about current dirty page rate of vm. >> +# >> +# @dirty-rate: @dirtyrate describing the dirty page rate of vm >> +# in units of MB/s. >> +# If this field return '-1', it means querying is not >> +# start or not complete. >> +# >> +# @status: status containing dirtyrate query status includes >> +# 'unstarted' or 'measuring' or 'measured' >> +# >> +# @start-time: start time in units of second for calculation >> +# >> +# @calc-time: time in units of second for sample dirty pages >> +# >> +# Since: 5.2 >> +# >> +## >> +{ 'struct': 'DirtyRateInfo', >> + 'data': {'dirty-rate': 'int64', >> + 'status': 'DirtyRateStatus', >> + 'start-time': 'int64', >> + 'calc-time': 'int64'} } >> + >> +## >> +# @calc-dirty-rate: >> +# >> +# start calculating dirty page rate for vm >> +# >> +# @calc-time: time in units of second for sample dirty pages >> +# >> +# Since: 5.2 >> +# >> +# Example: >> +# {"command": "calc-dirty-rate", "data": {"calc-time": 1} } >> +# >> +## >> +{ 'command': 'calc-dirty-rate', 'data': {'calc-time': 'int64'} } >> + >> +## >> +# @query-dirty-rate: >> +# >> +# query dirty page rate in units of MB/s for vm >> +# >> +# Since: 5.2 >> +## >> +{ 'command': 'query-dirty-rate', 'returns': 'DirtyRateInfo' } >> -- >> 1.8.3.1 >> >> > > . >
v6-> v7: fix minior comments and coding style by review add review-by for patches v5 -> v6: fix coding style according to review use TARGET_PAGE_SIZE and TARGET_PAGE_BITS instead of self-defined macros return start-time and calc-time by qmp command v4 -> v5: fix git apply failed due to meson-build add review-by for patches in v3 v3 -> v4: use crc32 to get hash result instead of md5 add DirtyRateStatus to denote calculation status add some trace_calls to make it easier to debug fix some comments accroding to review v2 -> v3: fix size_t compile warning fix codestyle checked by checkpatch.pl v1 -> v2: use g_rand_new() to generate rand_buf move RAMBLOCK_FOREACH_MIGRATABLE into migration/ram.h add skip_sample_ramblock to filter sampled ramblock fix multi-numa vm coredump when query dirtyrate rename qapi interface and rename some structures and functions succeed to compile by appling each patch add test for migrating vm Sometimes it is neccessary to evaluate dirty page rate before migration. Users could decide whether to proceed migration based on the evaluation in case of vm performance loss due to heavy workload. Unlikey simulating dirtylog sync which could do harm on runnning vm, we provide a sample-hash method to compare hash results for samping page. In this way, it would have hardly no impact on vm performance. Evaluate the dirtypage rate both on running and migration vm. The VM specifications for migration are as follows: - VM use 4-K page; - the number of VCPU is 32; - the total memory is 32Gigabit; - use 'mempress' tool to pressurize VM(mempress 4096 1024); - migration bandwidth is 1GB/s +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | | running | migrating | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | no mempress | 4MB/s | 8MB/s (migrated success) | ------------------------------------------------------------------------------------------- | mempress 4096 1024 | 1060MB/s | 456MB/s ~ 1142MB/s (cpu throttle triggered) | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ | mempress 4096 4096 | 4114MB/s | 688MB/s ~ 4132MB/s (cpu throttle triggered) | +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Test dirtyrate by qmp command like this: 1. virsh qemu-monitor-command [vmname] '{"execute":"calc-dirty-rate", "arguments": {"calc-time": [sleep-time]}}'; 2. sleep specific time which is a bit larger than sleep-time 3. virsh qemu-monitor-command [vmname] '{"execute":"query-dirty-rate"}' The qmp command returns like this: {"return":{"status":"measured","dirty-rate":374,"start-time":3718293,"calc-time":1},"id":"libvirt-15"} Further test dirtyrate by libvirt api like this: virsh getdirtyrate [vmname] [sleep-time] Chuan Zheng (12): migration/dirtyrate: setup up query-dirtyrate framwork migration/dirtyrate: add DirtyRateStatus to denote calculation status migration/dirtyrate: Add RamblockDirtyInfo to store sampled page info migration/dirtyrate: Add dirtyrate statistics series functions migration/dirtyrate: move RAMBLOCK_FOREACH_MIGRATABLE into ram.h migration/dirtyrate: Record hash results for each sampled page migration/dirtyrate: Compare page hash results for recorded sampled page migration/dirtyrate: skip sampling ramblock with size below MIN_RAMBLOCK_SIZE migration/dirtyrate: Implement set_sample_page_period() and get_sample_page_period() migration/dirtyrate: Implement calculate_dirtyrate() function migration/dirtyrate: Implement qmp_cal_dirty_rate()/qmp_get_dirty_rate() function migration/dirtyrate: Add trace_calls to make it easier to debug migration/dirtyrate.c | 435 +++++++++++++++++++++++++++++++++++++++++++++++++ migration/dirtyrate.h | 70 ++++++++ migration/meson.build | 2 +- migration/ram.c | 11 +- migration/ram.h | 10 ++ migration/trace-events | 8 + qapi/migration.json | 67 ++++++++ 7 files changed, 592 insertions(+), 11 deletions(-) create mode 100644 migration/dirtyrate.c create mode 100644 migration/dirtyrate.h