| Message ID | 1462891527-29359-1-git-send-email-maxim.uvarov@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
On 11 May 2016 at 10:37, Savolainen, Petri (Nokia - FI/Espoo) < petri.savolainen@nokia.com> wrote: > Other option would be to loop though the /sys/kernel/mm/hugepages > directory and see which size are available (nr_hugepages > 0). available 1 Gb page and 1000 2Mb page what is your decision which allocation give what memory? I think in api we need modify call to return list of available HPs with it's sizes. And if size is closer to maximum HP size that we can give that HP for requested allocation. For now quickest way is to use default HP and maybe change api for Tiger release. > I'd expect that /sys/kernel/mm/... content is better "standardized" than > proc file content. > Does all platforms (x86, arm, mips, power, ...) have the same content in > proc/meminfo? yes, it has to. > Does every platform spell "Hugepagesize:" the same way, or is there small > variation like "HugePageSize:"... > > It's common file for all arches: look at hugetlb_report_node_meminfo() https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/mm/hugetlb.c?id=refs/tags/v4.6-rc7 which called directly when you read meminfo fs/proc/meminfo.c Maxim. > -Petri > > > > -----Original Message----- > > From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of > Maxim > > Uvarov > > Sent: Tuesday, May 10, 2016 5:45 PM > > To: lng-odp@lists.linaro.org > > Subject: [lng-odp] [PATCH] linux-generic: use default huge page size > > > > odp_shm_reserve() relays on huge page size to round up > > requested size. If 1 Gb pages present than parser takes > > it first as first alphabetical file name in sysfs. That > > lead to issue where all allocations wants 1 GB HP. This > > patch takes system default huge pages which are usually 2Mb, > > and has to be enough for existence odp example apps and > > validation tests. > > > > Reported-by: B.Nousilal <bnousilal@gmail.com> > > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> > > --- > > platform/linux-generic/odp_system_info.c | 46 > +++++++++++++-------------- > > ----- > > 1 file changed, 18 insertions(+), 28 deletions(-) > > > > diff --git a/platform/linux-generic/odp_system_info.c b/platform/linux- > > generic/odp_system_info.c > > index 0f1f3c7..14244c3 100644 > > --- a/platform/linux-generic/odp_system_info.c > > +++ b/platform/linux-generic/odp_system_info.c > > @@ -24,12 +24,9 @@ > > #include <sys/types.h> > > #include <dirent.h> > > > > - > > #define CACHE_LNSZ_FILE \ > > "/sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size" > > > > -#define HUGE_PAGE_DIR "/sys/kernel/mm/hugepages" > > - > > /* > > * Report the number of logical CPUs detected at boot time > > */ > > @@ -77,37 +74,30 @@ static int systemcpu_cache_line_size(void) > > #endif > > > > > > -static int huge_page_size(void) > > +static int default_huge_page_size(void) > > { > > - DIR *dir; > > - struct dirent *dirent; > > - int size = 0; > > + char str[1024]; > > + char *pos; > > + uint64_t sz; > > + FILE *file; > > > > - dir = opendir(HUGE_PAGE_DIR); > > - if (dir == NULL) { > > - ODP_ERR("%s not found\n", HUGE_PAGE_DIR); > > - return 0; > > - } > > - > > - while ((dirent = readdir(dir)) != NULL) { > > - int temp = 0; > > - > > - if (sscanf(dirent->d_name, "hugepages-%i", &temp) != 1) > > - continue; > > - > > - if (temp > size) > > - size = temp; > > - } > > + file = fopen("/proc/meminfo", "rt"); > > > > - if (closedir(dir)) { > > - ODP_ERR("closedir failed\n"); > > - return 0; > > + while (fgets(str, sizeof(str), file) != NULL) { > > + pos = strstr(str, "Hugepagesize:"); > > + if (pos) { > > + *(pos - 1) = '\0'; > > + if (sscanf(pos, "@ %ld kB", &sz) == 1) { > > + fclose(file); > > + return sz * 1024; > > + } > > + } > > } > > > > - return size * 1024; > > + fclose(file); > > + return 0; > > } > > > > - > > /* > > * Analysis of /sys/devices/system/cpu/ files > > */ > > @@ -137,7 +127,7 @@ static int systemcpu(odp_system_info_t *sysinfo) > > return -1; > > } > > > > - sysinfo->huge_page_size = huge_page_size(); > > + sysinfo->huge_page_size = default_huge_page_size(); > > > > return 0; > > } > > -- > > 2.7.1.250.gff4ea60 > > > > _______________________________________________ > > lng-odp mailing list > > lng-odp@lists.linaro.org > > https://lists.linaro.org/mailman/listinfo/lng-odp >
On 11 May 2016 at 11:43, Maxim Uvarov <maxim.uvarov@linaro.org> wrote: > > > On 11 May 2016 at 10:37, Savolainen, Petri (Nokia - FI/Espoo) < > petri.savolainen@nokia.com> wrote: > >> Other option would be to loop though the /sys/kernel/mm/hugepages >> directory and see which size are available (nr_hugepages > 0). > > > available 1 Gb page and 1000 2Mb page what is your decision which > allocation give what memory? I think in api we need modify call to return > list of available HPs with it's sizes. And if size is closer to maximum HP > size that we can give that HP for requested allocation. For now quickest > way is to use default HP and maybe change api for Tiger release. > > >> I'd expect that /sys/kernel/mm/... content is better "standardized" than >> proc file content. >> Does all platforms (x86, arm, mips, power, ...) have the same content in >> proc/meminfo? > > > yes, it has to. > > >> Does every platform spell "Hugepagesize:" the same way, or is there small >> variation like "HugePageSize:"... >> >> > It's common file for all arches: > look at hugetlb_report_node_meminfo() > hugetlb_report_meminfo() function. Maxim. > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/mm/hugetlb.c?id=refs/tags/v4.6-rc7 > which called directly when you read meminfo fs/proc/meminfo.c > > Maxim. > > > >> -Petri >> >> >> > -----Original Message----- >> > From: lng-odp [mailto:lng-odp-bounces@lists.linaro.org] On Behalf Of >> Maxim >> > Uvarov >> > Sent: Tuesday, May 10, 2016 5:45 PM >> > To: lng-odp@lists.linaro.org >> > Subject: [lng-odp] [PATCH] linux-generic: use default huge page size >> > >> > odp_shm_reserve() relays on huge page size to round up >> > requested size. If 1 Gb pages present than parser takes >> > it first as first alphabetical file name in sysfs. That >> > lead to issue where all allocations wants 1 GB HP. This >> > patch takes system default huge pages which are usually 2Mb, >> > and has to be enough for existence odp example apps and >> > validation tests. >> > >> > Reported-by: B.Nousilal <bnousilal@gmail.com> >> > Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> >> > --- >> > platform/linux-generic/odp_system_info.c | 46 >> +++++++++++++-------------- >> > ----- >> > 1 file changed, 18 insertions(+), 28 deletions(-) >> > >> > diff --git a/platform/linux-generic/odp_system_info.c b/platform/linux- >> > generic/odp_system_info.c >> > index 0f1f3c7..14244c3 100644 >> > --- a/platform/linux-generic/odp_system_info.c >> > +++ b/platform/linux-generic/odp_system_info.c >> > @@ -24,12 +24,9 @@ >> > #include <sys/types.h> >> > #include <dirent.h> >> > >> > - >> > #define CACHE_LNSZ_FILE \ >> > "/sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size" >> > >> > -#define HUGE_PAGE_DIR "/sys/kernel/mm/hugepages" >> > - >> > /* >> > * Report the number of logical CPUs detected at boot time >> > */ >> > @@ -77,37 +74,30 @@ static int systemcpu_cache_line_size(void) >> > #endif >> > >> > >> > -static int huge_page_size(void) >> > +static int default_huge_page_size(void) >> > { >> > - DIR *dir; >> > - struct dirent *dirent; >> > - int size = 0; >> > + char str[1024]; >> > + char *pos; >> > + uint64_t sz; >> > + FILE *file; >> > >> > - dir = opendir(HUGE_PAGE_DIR); >> > - if (dir == NULL) { >> > - ODP_ERR("%s not found\n", HUGE_PAGE_DIR); >> > - return 0; >> > - } >> > - >> > - while ((dirent = readdir(dir)) != NULL) { >> > - int temp = 0; >> > - >> > - if (sscanf(dirent->d_name, "hugepages-%i", &temp) != 1) >> > - continue; >> > - >> > - if (temp > size) >> > - size = temp; >> > - } >> > + file = fopen("/proc/meminfo", "rt"); >> > >> > - if (closedir(dir)) { >> > - ODP_ERR("closedir failed\n"); >> > - return 0; >> > + while (fgets(str, sizeof(str), file) != NULL) { >> > + pos = strstr(str, "Hugepagesize:"); >> > + if (pos) { >> > + *(pos - 1) = '\0'; >> > + if (sscanf(pos, "@ %ld kB", &sz) == 1) { >> > + fclose(file); >> > + return sz * 1024; >> > + } >> > + } >> > } >> > >> > - return size * 1024; >> > + fclose(file); >> > + return 0; >> > } >> > >> > - >> > /* >> > * Analysis of /sys/devices/system/cpu/ files >> > */ >> > @@ -137,7 +127,7 @@ static int systemcpu(odp_system_info_t *sysinfo) >> > return -1; >> > } >> > >> > - sysinfo->huge_page_size = huge_page_size(); >> > + sysinfo->huge_page_size = default_huge_page_size(); >> > >> > return 0; >> > } >> > -- >> > 2.7.1.250.gff4ea60 >> > >> > _______________________________________________ >> > lng-odp mailing list >> > lng-odp@lists.linaro.org >> > https://lists.linaro.org/mailman/listinfo/lng-odp >> > >
diff --git a/platform/linux-generic/odp_system_info.c b/platform/linux-generic/odp_system_info.c index 0f1f3c7..14244c3 100644 --- a/platform/linux-generic/odp_system_info.c +++ b/platform/linux-generic/odp_system_info.c @@ -24,12 +24,9 @@ #include <sys/types.h> #include <dirent.h> - #define CACHE_LNSZ_FILE \ "/sys/devices/system/cpu/cpu0/cache/index0/coherency_line_size" -#define HUGE_PAGE_DIR "/sys/kernel/mm/hugepages" - /* * Report the number of logical CPUs detected at boot time */ @@ -77,37 +74,30 @@ static int systemcpu_cache_line_size(void) #endif -static int huge_page_size(void) +static int default_huge_page_size(void) { - DIR *dir; - struct dirent *dirent; - int size = 0; + char str[1024]; + char *pos; + uint64_t sz; + FILE *file; - dir = opendir(HUGE_PAGE_DIR); - if (dir == NULL) { - ODP_ERR("%s not found\n", HUGE_PAGE_DIR); - return 0; - } - - while ((dirent = readdir(dir)) != NULL) { - int temp = 0; - - if (sscanf(dirent->d_name, "hugepages-%i", &temp) != 1) - continue; - - if (temp > size) - size = temp; - } + file = fopen("/proc/meminfo", "rt"); - if (closedir(dir)) { - ODP_ERR("closedir failed\n"); - return 0; + while (fgets(str, sizeof(str), file) != NULL) { + pos = strstr(str, "Hugepagesize:"); + if (pos) { + *(pos - 1) = '\0'; + if (sscanf(pos, "@ %ld kB", &sz) == 1) { + fclose(file); + return sz * 1024; + } + } } - return size * 1024; + fclose(file); + return 0; } - /* * Analysis of /sys/devices/system/cpu/ files */ @@ -137,7 +127,7 @@ static int systemcpu(odp_system_info_t *sysinfo) return -1; } - sysinfo->huge_page_size = huge_page_size(); + sysinfo->huge_page_size = default_huge_page_size(); return 0; }
odp_shm_reserve() relays on huge page size to round up requested size. If 1 Gb pages present than parser takes it first as first alphabetical file name in sysfs. That lead to issue where all allocations wants 1 GB HP. This patch takes system default huge pages which are usually 2Mb, and has to be enough for existence odp example apps and validation tests. Reported-by: B.Nousilal <bnousilal@gmail.com> Signed-off-by: Maxim Uvarov <maxim.uvarov@linaro.org> --- platform/linux-generic/odp_system_info.c | 46 +++++++++++++------------------- 1 file changed, 18 insertions(+), 28 deletions(-)