| Message ID | 20180828112628.9462-1-rafael.tinoco@linaro.org |
|---|---|
| State | New |
| Headers | show |
| Series | tests/heapshrink: disable malloc per-thread cache for accuracy | expand |
On Tue, Aug 28, 2018 at 8:26 AM Rafael David Tinoco <rafael.tinoco@linaro.org> wrote: > > After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a new > per-thread caching mechanism that will NOT allow this test to > successfully measure if heap has shrunk, or not, due to the fact that > heap won't have its sized reduced right away, as the tests expects. > > Only way to test heap shrinkness in an accurate way is to disable this > per-thread cache. In order to disable it you need to have an environment > tunable GLIBC in place: > > GLIBC_TUNABLES=glibc.malloc.tcache_count=0 > > Unfortunately it requires to be set before program is loaded, since it > is evaluated during malloc() initialization, and there is no way to > re-initialize malloc() from the program context (not even using > constructor functions). > > This commit sets GLIBC_TUNABLES environment variable for the heapshrink > tests in order to make the test to succeed in environments with newer > GLIBC. Older environments won't suffer any consequences since the > environment tunable will just be ignored. > > Link: https://bugs.linaro.org/show_bug.cgi?id=3950 > Signed-off-by: Rafael David Tinoco <rafael.tinoco@linaro.org> > --- > tests/heapshrink.c | 28 +++++++++++++++++++++++++--- > tests/run_tests.py | 41 ++++++++++++++++++++++++++++++++++------- > 2 files changed, 59 insertions(+), 10 deletions(-) Anyone able to review/push this fix for me ? Is this list the correct place for such suggestion ? Thanks, -Rafael -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout.
On Tue, 28 Aug 2018, Rafael David Tinoco wrote: > After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a new > per-thread caching mechanism that will NOT allow this test to > successfully measure if heap has shrunk, or not, due to the fact that > heap won't have its sized reduced right away, as the tests expects. > > Only way to test heap shrinkness in an accurate way is to disable this > per-thread cache. In order to disable it you need to have an environment > tunable GLIBC in place: > > GLIBC_TUNABLES=glibc.malloc.tcache_count=0 > > Unfortunately it requires to be set before program is loaded, since it > is evaluated during malloc() initialization, and there is no way to > re-initialize malloc() from the program context (not even using > constructor functions). > > This commit sets GLIBC_TUNABLES environment variable for the heapshrink > tests in order to make the test to succeed in environments with newer > GLIBC. Older environments won't suffer any consequences since the > environment tunable will just be ignored. > > Link: https://bugs.linaro.org/show_bug.cgi?id=3950 > Signed-off-by: Rafael David Tinoco <rafael.tinoco@linaro.org> Applied, thanks Eric -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout.
Sorry for taking so long to apply this. I just finished an international move and my workspace is now finally setup. Eric -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout.
No problem Eric! I was just confirming, for any future needs, if any. Tks a lot! All the best in your new place! On Tue, Sep 18, 2018 at 12:16 AM Eric B Munson <emunson@mgebm.net> wrote: > > Sorry for taking so long to apply this. I just finished an > international move and my workspace is now finally setup. > > Eric > > -- > You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. > To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. > To post to this group, send email to libhugetlbfs@googlegroups.com. > Visit this group at https://groups.google.com/group/libhugetlbfs. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout.
diff --git a/tests/heapshrink.c b/tests/heapshrink.c index 16f233b..d2934aa 100644 --- a/tests/heapshrink.c +++ b/tests/heapshrink.c @@ -33,7 +33,7 @@ int main(int argc, char **argv) { - int is_huge, have_env, shrink_ok, have_helper; + int is_huge, have_env, shrink_ok, have_helper, tcache_enabled; unsigned long long mapping_size; void *p; long size = MAX(32*1024*1024, kernel_default_hugepage_size()); @@ -45,6 +45,23 @@ int main(int argc, char **argv) p = getenv("LD_PRELOAD"); have_helper = p != NULL && strstr(p, "heapshrink") != NULL; + /* + * After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a + * new per-thread caching mechanism that will NOT allow this test to + * successfully measure if heap has shrunk or not due to the fact that + * heap won't have its sized reduced right away. + * + * In order to disable it you need to have the tunable GLIBC in place. + * Unfortunately, it requires to be set before program is loaded, as an + * environment variable, since we can't re-initialize malloc() from the + * program context (not even with a constructor function), and the + * tunable is only evaluated during malloc() initialization. + * + * GLIBC_TUNABLES=glibc.malloc.tcache_count=0 + */ + p = getenv("GLIBC_TUNABLES"); + tcache_enabled = p != NULL && strstr(p, "malloc.tcache_count=0"); + p = malloc(size); if (!p) { if (shrink_ok && have_helper) { @@ -68,7 +85,12 @@ int main(int argc, char **argv) free(p); mapping_size = get_mapping_page_size(p+size-1); - if (shrink_ok && mapping_size > MIN_PAGE_SIZE) - FAIL("Heap did not shrink"); + if (shrink_ok && mapping_size > MIN_PAGE_SIZE) { + if (tcache_enabled) + FAIL("Heap did not shrink"); + else + FAIL("Heap didn't shrink. Check malloc.tcache_count=0"); + } + PASS(); } diff --git a/tests/run_tests.py b/tests/run_tests.py index 617ed93..3c95a03 100755 --- a/tests/run_tests.py +++ b/tests/run_tests.py @@ -572,15 +572,42 @@ def functional_tests(): do_test("malloc_manysmall") do_test("malloc_manysmall", LD_PRELOAD="libhugetlbfs.so", HUGETLB_MORECORE="yes") - do_test("heapshrink") - do_test("heapshrink", LD_PRELOAD="libheapshrink.so") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so", HUGETLB_MORECORE="yes") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so libheapshrink.so", + + # After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a + # new per-thread caching mechanism that will NOT allow heapshrink test to + # successfully measure if heap has shrunk or not due to the fact that + # heap won't have its sized reduced right away. + # + # In order to disable it, you need to have the tunable GLIBC in place. + # Unfortunately, it requires to be set before program is loaded, as an + # environment variable, since we can't re-initialize malloc() from the + # program context (not even with a constructor function), and the tunable + # is only evaluated during malloc() initialization. + + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libheapshrink.so") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so", + HUGETLB_MORECORE="yes") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so libheapshrink.so", HUGETLB_MORECORE="yes") - do_test("heapshrink", LD_PRELOAD="libheapshrink.so", HUGETLB_MORECORE="yes", + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libheapshrink.so", + HUGETLB_MORECORE="yes", HUGETLB_MORECORE_SHRINK="yes") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so libheapshrink.so", - HUGETLB_MORECORE="yes", HUGETLB_MORECORE_SHRINK="yes") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so libheapshrink.so", + HUGETLB_MORECORE="yes", + HUGETLB_MORECORE_SHRINK="yes") + do_test("heap-overflow", HUGETLB_VERBOSE="1", HUGETLB_MORECORE="yes") # Run the remapping tests' up-front checks
After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a new per-thread caching mechanism that will NOT allow this test to successfully measure if heap has shrunk, or not, due to the fact that heap won't have its sized reduced right away, as the tests expects. Only way to test heap shrinkness in an accurate way is to disable this per-thread cache. In order to disable it you need to have an environment tunable GLIBC in place: GLIBC_TUNABLES=glibc.malloc.tcache_count=0 Unfortunately it requires to be set before program is loaded, since it is evaluated during malloc() initialization, and there is no way to re-initialize malloc() from the program context (not even using constructor functions). This commit sets GLIBC_TUNABLES environment variable for the heapshrink tests in order to make the test to succeed in environments with newer GLIBC. Older environments won't suffer any consequences since the environment tunable will just be ignored. Link: https://bugs.linaro.org/show_bug.cgi?id=3950 Signed-off-by: Rafael David Tinoco <rafael.tinoco@linaro.org> --- tests/heapshrink.c | 28 +++++++++++++++++++++++++--- tests/run_tests.py | 41 ++++++++++++++++++++++++++++++++++------- 2 files changed, 59 insertions(+), 10 deletions(-) -- 2.18.0 -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout.