From patchwork Tue Aug 28 11:26:28 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Rafael David Tinoco X-Patchwork-Id: 145306 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp1066203ljw; Tue, 28 Aug 2018 04:26:33 -0700 (PDT) X-Received: by 2002:a6b:ba45:: with SMTP id k66-v6mr874903iof.252.1535455593329; Tue, 28 Aug 2018 04:26:33 -0700 (PDT) ARC-Seal: i=3; a=rsa-sha256; t=1535455593; cv=pass; d=google.com; s=arc-20160816; b=wabxmNyKQUR0E9PV4EzNfijbCnMUPUU4bvlkAK+L+m/TcT2I5A7XhBhQg17FnC31/j bu900kVes2tzvt0yuMT9/k0yBhAb6wEhA2qlpjYFdpszo58h0EQtUNbTHoe3yXWKU4jq qLMmZA3HRNskf+N0AjIgSc47ekkwB5yGgTEXW56Fl53rsutdOqNuJUWtBPnJXeb/L4Om YgjcjkzuPjB6xDYA0xX33Bhuu4chF0NfsHxhcO3wLyXXsTotOJtOTUi7aFBNOvagnkQ7 ZxkH/EEP/GAm62Hf+/GxEE9y5hBc763bPg1irnBrkcVloqOHzBOLMidd5JNXD1QluCNf da9A== ARC-Message-Signature: i=3; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:message-id:date:subject:cc:to:from :arc-authentication-results:arc-message-signature:mime-version :sender:dkim-signature:arc-authentication-results :arc-message-signature:arc-authentication-results; bh=rw84EINWPmMp158CR5c6Cs/ZfJ71kVA1nXyNROQfGD0=; b=t8RTj4tP2483seZbTD86rbP2SUQutsLtMyO1diHYqpMqKJwVI/SJRFuNu8nKyz3d5s 5/dnetrJ6ZItVBFbzgZZ4ty6UTVhLXwC0XBhJvzRz9XqLctvtVcY7bVKfeIVY9QgoC33 yMTt6S7sSSg8pZ3t9iFKpWNQXe6w1zajUwWUvqqnDRMclC9uBFqH3GjgtgbiEzZQREsK QxVdnMHSzcl5B6yT57lUHt3uF0a0MIvXLRyOGvdpTaQk6ES6gpiCH1daloQuu+PHgjlc ymSQdIIS8fQf0hfOAMFUFrpVxgyFfEzVBnYKVOR3e+Zm8ciAD9MHIB2w7QfvojRd1h+p ECnA== ARC-Authentication-Results: i=3; mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=QV1xrzxo; arc=pass (i=2 spf=pass spfdomain=linaro.org dkim=pass dkdomain=linaro.org dmarc=pass fromdomain=linaro.org); spf=pass (google.com: domain of libhugetlbfs+bncbdi3h5fvwqjbb2pcstoakgqeywru3ea@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=libhugetlbfs+bncBDI3H5FVWQJBB2PCSTOAKGQEYWRU3EA@googlegroups.com; dmarc=fail (p=NONE sp=NONE dis=NONE arc=pass) header.from=linaro.org Return-Path: Received: from mail-sor-f55.google.com (mail-sor-f55.google.com. [209.85.220.55]) by mx.google.com with SMTPS id p6-v6sor271557iod.82.2018.08.28.04.26.33 (Google Transport Security); Tue, 28 Aug 2018 04:26:33 -0700 (PDT) Received-SPF: pass (google.com: domain of libhugetlbfs+bncbdi3h5fvwqjbb2pcstoakgqeywru3ea@googlegroups.com designates 209.85.220.55 as permitted sender) client-ip=209.85.220.55; Authentication-Results: mx.google.com; dkim=pass header.i=@googlegroups.com header.s=20161025 header.b=QV1xrzxo; arc=pass (i=2 spf=pass spfdomain=linaro.org dkim=pass dkdomain=linaro.org dmarc=pass fromdomain=linaro.org); spf=pass (google.com: domain of libhugetlbfs+bncbdi3h5fvwqjbb2pcstoakgqeywru3ea@googlegroups.com designates 209.85.220.55 as permitted sender) smtp.mailfrom=libhugetlbfs+bncBDI3H5FVWQJBB2PCSTOAKGQEYWRU3EA@googlegroups.com; dmarc=fail (p=NONE sp=NONE dis=NONE arc=pass) header.from=linaro.org ARC-Seal: i=2; a=rsa-sha256; t=1535455593; cv=pass; d=google.com; s=arc-20160816; b=SRigts4t4/ioBPeFyyo4ym8c0samqQ0LrB3b/udVOnzNMlB/wVXrutxLA1bQlZQ76/ Rb2YRQ2wYGtsEFwH3zvJiQGQUVsPXRlsxS4f7dXxPA0jGi0GyEPTao22byHYABDzbAsr k8VRJEGQC9L58HaanS/d6alP3O8ByQZEMdPXGcTRxGyKV3S39h5tuYdurjLyxTdKtHq9 CbvTkQC+yRSNrJrgazSIcPAKHGazZJQaPeoe9iKcswCRGGUvmnhj7GWP/8ZXxu1OJNX9 NMyB2rpyU26n2m9ub/1mPwuG/F/JUiJFb+jL+O4uRPsCkr1ZpYWAKlvMUT4Lw+BX+60K waTA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:message-id:date:subject:cc:to:from :arc-authentication-results:arc-message-signature:mime-version :sender:dkim-signature:arc-authentication-results; bh=rw84EINWPmMp158CR5c6Cs/ZfJ71kVA1nXyNROQfGD0=; b=CpC87j593js3JrrE/KVI1ZgKj025IwOMDbg7mKcW8ie60rkeccQouCF+wDW8OMrwPl 1OyvStH8FlWvy1lGfDNRthZ0rsLQkt5U1vpsnb9CSDN/XeAiZsvkdJCTPDLAwhlqwz6p xG48t2n6zN0lTvzAZziAhX4cQSSQ8DTzICQ0HWAG+T56MyCfnFGvrM1WG8npncePRvOz gW6Cooi4ic3qWkMqoQsXK2rPpRMhfdFatoPPlJmeGTu9itIJFrocNFoTPm1316pjqu7z //RIuCU0/W7xjd1FIy142fSJJy/N+wyDIH0xbWZDzLQ+I1pnVLmkV/xpizJ44lWWxp4W 4NAw== ARC-Authentication-Results: i=2; gmr-mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gtp4z12B; spf=pass (google.com: domain of rafael.tinoco@linaro.org designates 2607:f8b0:400d:c09::242 as permitted sender) smtp.mailfrom=rafael.tinoco@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20161025; h=sender:mime-version:from:to:cc:subject:date:message-id :x-original-sender:x-original-authentication-results:precedence :mailing-list:list-id:list-post:list-help:list-archive :list-subscribe:list-unsubscribe; bh=rw84EINWPmMp158CR5c6Cs/ZfJ71kVA1nXyNROQfGD0=; b=QV1xrzxoyeBKhW3hk4fkgPBsrOCiFS/iInvRo3+HYrd+RiGJrrnsaC338Jg30kBPOS 2eBbq9VSEK8yIEI63Pl76LG/CD2PFr26L4kjehKBOTvZqvRi61l5TKBhtiF8YaChP/SL mM3Ior5NjMTgpYzJfKbCcelGAyqkq/JExeywrgvYFsQniXOe+d59+26LgzJckYSO4lGV dNEWU8uqsnBAM5B6SFA6H06Z8ixQpjYWyms+hGXRcFGc5aOxdnVhlomv+OrDCkwTFBfm DIiRVktb3VYIhbLFc4U5Tz58qNsRWbceVuIYnMe2KUxgpCitmxdqpKYVjMrR2JNjOHx9 PRKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=sender:x-gm-message-state:mime-version:from:to:cc:subject:date :message-id:x-original-sender:x-original-authentication-results :precedence:mailing-list:list-id:x-spam-checked-in-group:list-post :list-help:list-archive:list-subscribe:list-unsubscribe; bh=rw84EINWPmMp158CR5c6Cs/ZfJ71kVA1nXyNROQfGD0=; b=BipI/GDowenEpQLGhOFIVYGv99AZo+q0c+MltSics9thgQqjtk9LQ7A3pXW5so5TUl mdecVYzg1NGTv5kNQVIo7jmsCOrcR2rRWFTEmYUXLrzDTRdYZJN2NVkUNjIp9WbcLC3S htGl9rEZ4vhdv2dcH3JR0KrNz1F1bn4flX8Dgcj5/Zgi79oIJmlWRLKV70eMMcH41+Q8 oh0z+4sKfDeYRndMf12SHZEJ2PCD4IE+CV5lcu8F74MrNhPXARF1JKsIr7OIbLTxdJxY 1h3FxHBR8CpU6ACtFd6yvMmPzgIPMm7UODL5/kFM4MulJFWE+7FF4qaFxtQQ+K2BxLGM 64fw== Sender: libhugetlbfs@googlegroups.com X-Gm-Message-State: APzg51B+WsC/c02pAp15kXL/Frjh+nbyw3jXPM4COL+08Ef2bRFGmMSK V0kkVIJ9q5CWki8M8AFnkVc= X-Google-Smtp-Source: ANB0VdaJEbQ9HvOb8XqSACbyqgrxe1d6xJ+UHbY1etuYUHdfNcUKcudWMLFvrpOGqyVd0EhxvPRplw== X-Received: by 2002:a6b:6a18:: with SMTP id x24-v6mr7813iog.5.1535455593096; Tue, 28 Aug 2018 04:26:33 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: libhugetlbfs@googlegroups.com Received: by 2002:a6b:f604:: with SMTP id n4-v6ls171507ioh.22.gmail; Tue, 28 Aug 2018 04:26:32 -0700 (PDT) X-Received: by 2002:a6b:7a45:: with SMTP id k5-v6mr421165iop.16.1535455592870; Tue, 28 Aug 2018 04:26:32 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1535455592; cv=none; d=google.com; s=arc-20160816; b=MDoCoMVpH4XgzyyfVOTVrBQyjGXEeQHN6ic4Nw4M5s3lWgFav3FAxUVfWBto2vTIoe n/lU5p/zT5aJsONqJijOn01jGw4lN7krmBtsIhmhgCKllBV5RBrWvZVAkZCBE4RRToWy ton+A6Ya1afBusac7UqIv3tI8UrJxTxL2Q/k+kW6NuMdqLASPE2UJvGogmTtJ7/SI0pm 0qpJ1fijFPlVloWy0wO4C8UGgieUm1Y3StHVAngmvVF88JHgil9KPdoqVOE8o5MHAy4N OEWtyP10dv5RRZS/pwjVnT4839QHu985M8sG/yPLggw5H8d7Kxfj2zg4MIEmHHnhyN5i lOjg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature :arc-authentication-results; bh=WEBawJTUPjx/bpSpodBcroV2mc541ws4heTaHCpvoi8=; b=gW/kOqKCDBwMs3/k7OeYUYa9udOF8BFzBQeoKfJt6FCpFgK8gLCtJcwoyO/dWf+6Mk /ns3aJxRgywL+ZE6j1XJMjjCdm/FnFQdLnsmj9a4jD/Y45pQIWpBL5ijXZ8ZAa/YIRqt A+qR0Fd1hLLGlrxOVaAruEWREgvbVf1/EDKPlT0qNcmzzwJo+H3fcCsBuPEqLHt43bVf BUPnE9bzCuRL1QfrgTnW4R4XttBZqJF5at2PptbHsoeSZNvFSlPAXrYHX7PIf4Se4UOJ V+UZsV73POrYK3AX4fjsIMSHSBYzSZ/XvcIYG2w/lD7rhJSKDYfVQIqttv/XqkktSXag Xjsw== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gtp4z12B; spf=pass (google.com: domain of rafael.tinoco@linaro.org designates 2607:f8b0:400d:c09::242 as permitted sender) smtp.mailfrom=rafael.tinoco@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: from mail-qk0-x242.google.com (mail-qk0-x242.google.com. [2607:f8b0:400d:c09::242]) by gmr-mx.google.com with ESMTPS id p16-v6si160481itf.0.2018.08.28.04.26.32 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Aug 2018 04:26:32 -0700 (PDT) Received-SPF: pass (google.com: domain of rafael.tinoco@linaro.org designates 2607:f8b0:400d:c09::242 as permitted sender) client-ip=2607:f8b0:400d:c09::242; Received: by mail-qk0-x242.google.com with SMTP id 93-v6so704207qks.3 for ; Tue, 28 Aug 2018 04:26:32 -0700 (PDT) X-Received: by 2002:a37:a904:: with SMTP id s4-v6mr948159qke.171.1535455591875; Tue, 28 Aug 2018 04:26:31 -0700 (PDT) Received: from workstation.celeiro.br ([168.194.163.45]) by smtp.gmail.com with ESMTPSA id r67-v6sm370131qkd.10.2018.08.28.04.26.30 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 28 Aug 2018 04:26:31 -0700 (PDT) From: Rafael David Tinoco To: libhugetlbfs@googlegroups.com Cc: rafael.tinoco@linaro.org Subject: [libhugetlbfs] [PATCH] tests/heapshrink: disable malloc per-thread cache for accuracy Date: Tue, 28 Aug 2018 08:26:28 -0300 Message-Id: <20180828112628.9462-1-rafael.tinoco@linaro.org> X-Mailer: git-send-email 2.18.0 X-Original-Sender: rafael.tinoco@linaro.org X-Original-Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=Gtp4z12B; spf=pass (google.com: domain of rafael.tinoco@linaro.org designates 2607:f8b0:400d:c09::242 as permitted sender) smtp.mailfrom=rafael.tinoco@linaro.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Precedence: list Mailing-list: list libhugetlbfs@googlegroups.com; contact libhugetlbfs+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: libhugetlbfs@googlegroups.com X-Google-Group-Id: 864002673528 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a new per-thread caching mechanism that will NOT allow this test to successfully measure if heap has shrunk, or not, due to the fact that heap won't have its sized reduced right away, as the tests expects. Only way to test heap shrinkness in an accurate way is to disable this per-thread cache. In order to disable it you need to have an environment tunable GLIBC in place: GLIBC_TUNABLES=glibc.malloc.tcache_count=0 Unfortunately it requires to be set before program is loaded, since it is evaluated during malloc() initialization, and there is no way to re-initialize malloc() from the program context (not even using constructor functions). This commit sets GLIBC_TUNABLES environment variable for the heapshrink tests in order to make the test to succeed in environments with newer GLIBC. Older environments won't suffer any consequences since the environment tunable will just be ignored. Link: https://bugs.linaro.org/show_bug.cgi?id=3950 Signed-off-by: Rafael David Tinoco --- tests/heapshrink.c | 28 +++++++++++++++++++++++++--- tests/run_tests.py | 41 ++++++++++++++++++++++++++++++++++------- 2 files changed, 59 insertions(+), 10 deletions(-) -- 2.18.0 -- You received this message because you are subscribed to the Google Groups "libhugetlbfs" group. To unsubscribe from this group and stop receiving emails from it, send an email to libhugetlbfs+unsubscribe@googlegroups.com. To post to this group, send email to libhugetlbfs@googlegroups.com. Visit this group at https://groups.google.com/group/libhugetlbfs. For more options, visit https://groups.google.com/d/optout. diff --git a/tests/heapshrink.c b/tests/heapshrink.c index 16f233b..d2934aa 100644 --- a/tests/heapshrink.c +++ b/tests/heapshrink.c @@ -33,7 +33,7 @@ int main(int argc, char **argv) { - int is_huge, have_env, shrink_ok, have_helper; + int is_huge, have_env, shrink_ok, have_helper, tcache_enabled; unsigned long long mapping_size; void *p; long size = MAX(32*1024*1024, kernel_default_hugepage_size()); @@ -45,6 +45,23 @@ int main(int argc, char **argv) p = getenv("LD_PRELOAD"); have_helper = p != NULL && strstr(p, "heapshrink") != NULL; + /* + * After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a + * new per-thread caching mechanism that will NOT allow this test to + * successfully measure if heap has shrunk or not due to the fact that + * heap won't have its sized reduced right away. + * + * In order to disable it you need to have the tunable GLIBC in place. + * Unfortunately, it requires to be set before program is loaded, as an + * environment variable, since we can't re-initialize malloc() from the + * program context (not even with a constructor function), and the + * tunable is only evaluated during malloc() initialization. + * + * GLIBC_TUNABLES=glibc.malloc.tcache_count=0 + */ + p = getenv("GLIBC_TUNABLES"); + tcache_enabled = p != NULL && strstr(p, "malloc.tcache_count=0"); + p = malloc(size); if (!p) { if (shrink_ok && have_helper) { @@ -68,7 +85,12 @@ int main(int argc, char **argv) free(p); mapping_size = get_mapping_page_size(p+size-1); - if (shrink_ok && mapping_size > MIN_PAGE_SIZE) - FAIL("Heap did not shrink"); + if (shrink_ok && mapping_size > MIN_PAGE_SIZE) { + if (tcache_enabled) + FAIL("Heap did not shrink"); + else + FAIL("Heap didn't shrink. Check malloc.tcache_count=0"); + } + PASS(); } diff --git a/tests/run_tests.py b/tests/run_tests.py index 617ed93..3c95a03 100755 --- a/tests/run_tests.py +++ b/tests/run_tests.py @@ -572,15 +572,42 @@ def functional_tests(): do_test("malloc_manysmall") do_test("malloc_manysmall", LD_PRELOAD="libhugetlbfs.so", HUGETLB_MORECORE="yes") - do_test("heapshrink") - do_test("heapshrink", LD_PRELOAD="libheapshrink.so") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so", HUGETLB_MORECORE="yes") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so libheapshrink.so", + + # After upstream commit: (glibc-2.25.90-688-gd5c3fafc43) glibc has a + # new per-thread caching mechanism that will NOT allow heapshrink test to + # successfully measure if heap has shrunk or not due to the fact that + # heap won't have its sized reduced right away. + # + # In order to disable it, you need to have the tunable GLIBC in place. + # Unfortunately, it requires to be set before program is loaded, as an + # environment variable, since we can't re-initialize malloc() from the + # program context (not even with a constructor function), and the tunable + # is only evaluated during malloc() initialization. + + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libheapshrink.so") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so", + HUGETLB_MORECORE="yes") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so libheapshrink.so", HUGETLB_MORECORE="yes") - do_test("heapshrink", LD_PRELOAD="libheapshrink.so", HUGETLB_MORECORE="yes", + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libheapshrink.so", + HUGETLB_MORECORE="yes", HUGETLB_MORECORE_SHRINK="yes") - do_test("heapshrink", LD_PRELOAD="libhugetlbfs.so libheapshrink.so", - HUGETLB_MORECORE="yes", HUGETLB_MORECORE_SHRINK="yes") + do_test("heapshrink", + GLIBC_TUNABLES="glibc.malloc.tcache_count=0", + LD_PRELOAD="libhugetlbfs.so libheapshrink.so", + HUGETLB_MORECORE="yes", + HUGETLB_MORECORE_SHRINK="yes") + do_test("heap-overflow", HUGETLB_VERBOSE="1", HUGETLB_MORECORE="yes") # Run the remapping tests' up-front checks