From patchwork Tue Sep 11 14:00:39 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Github ODP bot X-Patchwork-Id: 146461 Delivered-To: patch@linaro.org Received: by 2002:a2e:1648:0:0:0:0:0 with SMTP id 8-v6csp3682717ljw; Tue, 11 Sep 2018 07:07:29 -0700 (PDT) X-Google-Smtp-Source: ANB0Vdb0eOBlbmkAeyCuiwNgnaZixk7repvUQR1zLM1VdkPWIXdLxfUJLNwcrXpI8yQD5lXpZSef X-Received: by 2002:a37:6e01:: with SMTP id j1-v6mr5107404qkc.70.1536674849542; Tue, 11 Sep 2018 07:07:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536674849; cv=none; d=google.com; s=arc-20160816; b=oG+yXso3i51nuRypM0NMkFd5uQIAdY1bEl695y1g0aRoR69jOEDgWtN7qPjZdiPYWR kI25RxL5VSFgLAiMX1HfCEKN8s+frsVqRiges4eiEZO1OgjRO2++ywc11Ju9igU6FaMn 2H4QCrpgCfYlZ3FpM0y6YiDMI7fKDvoS5FZJmsJie/nzs3RR+62DFPxMbZ14IOBvmkVH IALEOJOmO5DFauUAPjjLBvpbr6FsL+qtcdZ4GAU8FLSBmavyzDIq+o+3favirEqU7lOu 5e7uUVLTOXkyUjP9jrXsA29So+SI0fKPkqzH1sFcNnEFRbML8eJM80a5H75FWbOzCmfB b5aA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=sender:errors-to:list-subscribe:list-help:list-post:list-archive :list-unsubscribe:list-id:precedence:subject:github-pr-num :references:in-reply-to:message-id:date:to:from:delivered-to; bh=8pIbwfKE2tu10ClN1xfYUCeRvH3woy/UuAXTtJcC9U0=; b=d/oCuvwHXXzPGJfVigy50CiSpYoQR/0EMN1MnLpxFdM9FjBTjsJP3thC1hUbsV++y1 6DERq55NwO5DTr7n6jVKwy2K1F+iSoh9mUFEyFpmcPCDuQJiHts3psSYYFr+sEi1b28i +CiYEEgiTNwY2S0VHukHzMIq10OcXEarNjDHAXzxfkZcHsHLURDlFbs0WmQtUVjvptym 32/x5MkfnosKbetjFmqIXXE34xHyr46sZqZLeSuuGSa6YhYOkX62v+gK2UHeCs2nSQbc 7vZ0y4jzARWeJwBBtaQZVt8RlrXWl48EnpjEr+Hhxh/v/vpwoTZmDsVl1AFfPxqH3/zo IpqQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Return-Path: Received: from lists.linaro.org (ec2-54-197-127-237.compute-1.amazonaws.com. [54.197.127.237]) by mx.google.com with ESMTP id 57-v6si1969818qtz.128.2018.09.11.07.07.29; Tue, 11 Sep 2018 07:07:29 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) client-ip=54.197.127.237; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.197.127.237 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=yandex.ru Received: by lists.linaro.org (Postfix, from userid 109) id 3772761C13; Tue, 11 Sep 2018 14:07:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-3.6 required=5.0 tests=BAYES_00,FREEMAIL_FROM, MAILING_LIST_MULTI, RCVD_IN_DNSWL_LOW autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 4164661C2A; Tue, 11 Sep 2018 14:02:42 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 089AB61C11; Tue, 11 Sep 2018 14:02:04 +0000 (UTC) Received: from forward105o.mail.yandex.net (forward105o.mail.yandex.net [37.140.190.183]) by lists.linaro.org (Postfix) with ESMTPS id 73AC661C25 for ; Tue, 11 Sep 2018 14:00:54 +0000 (UTC) Received: from mxback12j.mail.yandex.net (mxback12j.mail.yandex.net [IPv6:2a02:6b8:0:1619::87]) by forward105o.mail.yandex.net (Yandex) with ESMTP id E13AD44460B2 for ; Tue, 11 Sep 2018 17:00:46 +0300 (MSK) Received: from smtp2o.mail.yandex.net (smtp2o.mail.yandex.net [2a02:6b8:0:1a2d::26]) by mxback12j.mail.yandex.net (nwsmtp/Yandex) with ESMTP id 2uEwRl84pO-0kseKJn2; Tue, 11 Sep 2018 17:00:46 +0300 Received: by smtp2o.mail.yandex.net (nwsmtp/Yandex) with ESMTPSA id XLXHUeLpaf-0i0mPlad; Tue, 11 Sep 2018 17:00:44 +0300 (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (Client certificate not present) From: Github ODP bot To: lng-odp@lists.linaro.org Date: Tue, 11 Sep 2018 14:00:39 +0000 Message-Id: <1536674439-8532-3-git-send-email-odpbot@yandex.ru> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1536674439-8532-1-git-send-email-odpbot@yandex.ru> References: <1536674439-8532-1-git-send-email-odpbot@yandex.ru> Github-pr-num: 685 Subject: [lng-odp] [PATCH v5 2/2] linux-gen: ishm: make huge page cache size dynamic X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" From: Josep Puigdemont Signed-off-by: Josep Puigdemont --- /** Email created from pull request 685 (joseppc:fix/cache_huge_pages) ** https://github.com/Linaro/odp/pull/685 ** Patch: https://github.com/Linaro/odp/pull/685.patch ** Base sha: 33fbc04b6373960ec3f84de4e7e7b34c49d71508 ** Merge commit sha: 9826130fb2849a5c4088572ca285b00e358be707 **/ config/odp-linux-generic.conf | 16 ++++--- platform/linux-generic/odp_ishm.c | 73 +++++++++++++++++++------------ 2 files changed, 56 insertions(+), 33 deletions(-) diff --git a/config/odp-linux-generic.conf b/config/odp-linux-generic.conf index 0dd2a6c13..bddc92dd4 100644 --- a/config/odp-linux-generic.conf +++ b/config/odp-linux-generic.conf @@ -18,14 +18,20 @@ odp_implementation = "linux-generic" config_file_version = "0.0.1" -# Internal shared memory allocator +# Shared memory options shm: { - # ODP will try to reserve as many huge pages as the number indicated - # here, up to 64. A zero value means that no pages should be reserved. + # Number of cached default size huge pages. These pages are allocated + # during odp_init_global() and freed back to the kernel in + # odp_term_global(). A value of zero means no pages are cached. + # No negative values should be used here, they are reserved for future + # implementations. + # + # ODP will reserve as many huge pages as possible, which may be less + # than requested here if the system does not have enough huge pages + # available. + # # When using process mode threads, this value should be set to 0 # because the current implementation won't work properly otherwise. - # These pages will only be freed when the application calls - # odp_term_global(). num_cached_hp = 0 } diff --git a/platform/linux-generic/odp_ishm.c b/platform/linux-generic/odp_ishm.c index aeda50bec..11fbe8ef0 100644 --- a/platform/linux-generic/odp_ishm.c +++ b/platform/linux-generic/odp_ishm.c @@ -239,15 +239,15 @@ typedef struct { } ishm_ftable_t; static ishm_ftable_t *ishm_ftbl; -#define HP_CACHE_SIZE 64 struct huge_page_cache { uint64_t len; + int max_fds; /* maximum amount requested of pre-allocated huge pages */ int total; /* amount of actually pre-allocated huge pages */ int idx; /* retrieve fd[idx] to get a free file descriptor */ - int fd[HP_CACHE_SIZE]; /* list of file descriptors */ + int fd[]; /* list of file descriptors */ }; -static struct huge_page_cache hpc; +static struct huge_page_cache *hpc; #ifndef MAP_ANONYMOUS #define MAP_ANONYMOUS MAP_ANON @@ -301,19 +301,14 @@ static void hp_init(void) char filename[ISHM_FILENAME_MAXLEN]; char dir[ISHM_FILENAME_MAXLEN]; int count; - - hpc.total = 0; - hpc.idx = -1; - hpc.len = odp_sys_huge_page_size(); + void *addr; if (!_odp_libconfig_lookup_ext_int("shm", NULL, "num_cached_hp", &count)) { return; } - if (count > HP_CACHE_SIZE) - count = HP_CACHE_SIZE; - else if (count <= 0) + if (count <= 0) return; ODP_DBG("Init HP cache with up to %d pages\n", count); @@ -339,55 +334,77 @@ static void hp_init(void) dir, odp_global_data.main_pid); + addr = mmap(NULL, + sizeof(struct huge_page_cache) + sizeof(int) * count, + PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0); + if (addr == MAP_FAILED) { + ODP_ERR("Unable to mmap memory for huge page cache\n."); + return; + } + + hpc = addr; + + hpc->max_fds = count; + hpc->total = 0; + hpc->idx = -1; + hpc->len = odp_sys_huge_page_size(); + for (int i = 0; i < count; ++i) { int fd; - fd = hp_create_file(hpc.len, filename); - if (fd == -1) + fd = hp_create_file(hpc->len, filename); + if (fd == -1) { + do { + hpc->fd[i++] = -1; + } while (i < count); break; - hpc.total++; - hpc.fd[i] = fd; + } + hpc->total++; + hpc->fd[i] = fd; } - hpc.idx = hpc.total - 1; + hpc->idx = hpc->total - 1; ODP_DBG("HP cache has %d huge pages of size 0x%08" PRIx64 "\n", - hpc.total, hpc.len); + hpc->total, hpc->len); } static void hp_term(void) { - for (int i = 0; i < hpc.total; i++) { - if (hpc.fd[i] != -1) - close(hpc.fd[i]); + if (NULL == hpc) + return; + + for (int i = 0; i < hpc->total; i++) { + if (hpc->fd[i] != -1) + close(hpc->fd[i]); } - hpc.total = 0; - hpc.idx = -1; - hpc.len = 0; + hpc->total = 0; + hpc->idx = -1; + hpc->len = 0; } static int hp_get_cached(uint64_t len) { int fd; - if (hpc.idx < 0 || len != hpc.len) + if (NULL == hpc || hpc->idx < 0 || len != hpc->len) return -1; - fd = hpc.fd[hpc.idx]; - hpc.fd[hpc.idx--] = -1; + fd = hpc->fd[hpc->idx]; + hpc->fd[hpc->idx--] = -1; return fd; } static int hp_put_cached(int fd) { - if (odp_unlikely(++hpc.idx >= hpc.total)) { - hpc.idx--; + if (NULL == hpc || odp_unlikely(++hpc->idx >= hpc->total)) { + hpc->idx--; ODP_ERR("Trying to put more FD than allowed: %d\n", fd); return -1; } - hpc.fd[hpc.idx] = fd; + hpc->fd[hpc->idx] = fd; return 0; }