From patchwork Thu Apr 17 12:30:35 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Newton X-Patchwork-Id: 28577 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-ob0-f200.google.com (mail-ob0-f200.google.com [209.85.214.200]) by ip-10-151-82-157.ec2.internal (Postfix) with ESMTPS id E443F20674 for ; Thu, 17 Apr 2014 12:31:01 +0000 (UTC) Received: by mail-ob0-f200.google.com with SMTP id uz6sf1814096obc.3 for ; Thu, 17 Apr 2014 05:31:01 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:delivered-to:mailing-list :precedence:list-id:list-unsubscribe:list-subscribe:list-archive :list-post:list-help:sender:delivered-to:from:to:subject:date :message-id:x-original-sender:x-original-authentication-results; bh=mZUZYsWAYhdfvj/741MZBmJfcCFhZ+fozR0C++cBa6U=; b=DyR5sceHiYowfdhNayAYvIqHBVVT1QXVhYRgO5kqAFMOvl+Ai0l1UOsmulF0H1JjXo E5C6RR3o5MK9oenduItHXFD599Rr5vAAi5B871djEdfzhNQOzLLZ+N2VnPmbqr5DFUSV iOktlQ8ZLkczyZv/xo1wF1dgNohDsKwzk2VXajUJIVOhSG1PDAS2lTQPLR1jJzgmrfVy uAXBKQ70xAo5aDzsztszXXlJ8wpE6PCRutUFPZXSu9g32Op9Mo+TmIzdR+I3c8Fa4uFZ ed535biV7ZqQ1ojA/aUqLgNXXHmYoLM1Q1VDg+Lo5z0jCJxTU2sThbB8T1HDpV3CYd+g WGvQ== X-Gm-Message-State: ALoCoQkKDEM1jAgQuDB+4mQE0rV5E6AIS7w6vcLC2dJIc1jMyzE0ulHUQaZP0yUWzC6RkkM2cSCg X-Received: by 10.50.50.211 with SMTP id e19mr7656657igo.4.1397737861391; Thu, 17 Apr 2014 05:31:01 -0700 (PDT) MIME-Version: 1.0 X-BeenThere: patchwork-forward@linaro.org Received: by 10.140.49.179 with SMTP id q48ls1014084qga.21.gmail; Thu, 17 Apr 2014 05:31:01 -0700 (PDT) X-Received: by 10.220.161.8 with SMTP id p8mr8405529vcx.4.1397737861274; Thu, 17 Apr 2014 05:31:01 -0700 (PDT) Received: from mail-ve0-x22f.google.com (mail-ve0-x22f.google.com [2607:f8b0:400c:c01::22f]) by mx.google.com with ESMTPS id eh16si4421618vcb.126.2014.04.17.05.31.01 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 17 Apr 2014 05:31:01 -0700 (PDT) Received-SPF: neutral (google.com: 2607:f8b0:400c:c01::22f is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) client-ip=2607:f8b0:400c:c01::22f; Received: by mail-ve0-f175.google.com with SMTP id oz11so397013veb.6 for ; Thu, 17 Apr 2014 05:31:01 -0700 (PDT) X-Received: by 10.58.230.103 with SMTP id sx7mr1615617vec.28.1397737860948; Thu, 17 Apr 2014 05:31:00 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.220.221.72 with SMTP id ib8csp31375vcb; Thu, 17 Apr 2014 05:31:00 -0700 (PDT) X-Received: by 10.66.158.132 with SMTP id wu4mr15411305pab.66.1397737859425; Thu, 17 Apr 2014 05:30:59 -0700 (PDT) Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id wh4si14540042pbc.391.2014.04.17.05.30.58 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 17 Apr 2014 05:30:58 -0700 (PDT) Received-SPF: pass (google.com: domain of libc-alpha-return-49120-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Received: (qmail 24348 invoked by alias); 17 Apr 2014 12:30:46 -0000 Mailing-List: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org Precedence: list List-Id: List-Unsubscribe: , List-Subscribe: List-Archive: List-Post: , List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 24336 invoked by uid 89); 17 Apr 2014 12:30:46 -0000 X-Virus-Found: No X-Spam-SWARE-Status: No, score=-2.1 required=5.0 tests=AWL, BAYES_00, RCVD_IN_DNSWL_LOW, SPF_PASS autolearn=ham version=3.3.2 X-HELO: mail-we0-f180.google.com X-Received: by 10.180.90.140 with SMTP id bw12mr12185606wib.18.1397737840240; Thu, 17 Apr 2014 05:30:40 -0700 (PDT) From: Will Newton To: libc-alpha@sourceware.org Subject: [PATCH v2] benchtests: Add malloc microbenchmark Date: Thu, 17 Apr 2014 13:30:35 +0100 Message-Id: <1397737835-15868-1-git-send-email-will.newton@linaro.org> X-Original-Sender: will.newton@linaro.org X-Original-Authentication-Results: mx.google.com; spf=neutral (google.com: 2607:f8b0:400c:c01::22f is neither permitted nor denied by best guess record for domain of patch+caf_=patchwork-forward=linaro.org@linaro.org) smtp.mail=patch+caf_=patchwork-forward=linaro.org@linaro.org; dkim=pass header.i=@sourceware.org X-Google-Group-Id: 836684582541 Add a microbenchmark for measuring malloc and free performance. The benchmark allocates and frees buffers of random sizes in a random order and measures the overall execution time and RSS. Variants of the benchmark are run with 8, 32 and 64 threads to measure the effect of concurrency on allocator performance. The random block sizes used follow an inverse square distribution which is intended to mimic the behaviour of real applications which tend to allocate many more small blocks than large ones. ChangeLog: 2014-04-15 Will Newton * benchtests/Makefile: (benchset): Add malloc benchmarks. Link threaded malloc benchmarks with libpthread. * benchtests/bench-malloc-threads-32.c: New file. * benchtests/bench-malloc-threads-64.c: Likewise. * benchtests/bench-malloc-threads-8.c: Likewise. * benchtests/bench-malloc.c: Likewise. --- benchtests/Makefile | 7 +- benchtests/bench-malloc-threads-32.c | 20 +++ benchtests/bench-malloc-threads-64.c | 20 +++ benchtests/bench-malloc-threads-8.c | 20 +++ benchtests/bench-malloc.c | 236 +++++++++++++++++++++++++++++++++++ 5 files changed, 302 insertions(+), 1 deletion(-) create mode 100644 benchtests/bench-malloc-threads-32.c create mode 100644 benchtests/bench-malloc-threads-64.c create mode 100644 benchtests/bench-malloc-threads-8.c create mode 100644 benchtests/bench-malloc.c Changes in v2: - Move random number generation out of the loop and use arrays of random values. This reduces the overhead of the benchmark loop to 10% or less. diff --git a/benchtests/Makefile b/benchtests/Makefile index a0954cd..f38380d 100644 --- a/benchtests/Makefile +++ b/benchtests/Makefile @@ -37,9 +37,11 @@ string-bench := bcopy bzero memccpy memchr memcmp memcpy memmem memmove \ strspn strstr strcpy_chk stpcpy_chk memrchr strsep strtok string-bench-all := $(string-bench) +malloc-bench := malloc malloc-threads-8 malloc-threads-32 malloc-threads-64 + stdlib-bench := strtod -benchset := $(string-bench-all) $(stdlib-bench) +benchset := $(string-bench-all) $(stdlib-bench) $(malloc-bench) CFLAGS-bench-ffs.c += -fno-builtin CFLAGS-bench-ffsll.c += -fno-builtin @@ -47,6 +49,9 @@ CFLAGS-bench-ffsll.c += -fno-builtin $(addprefix $(objpfx)bench-,$(bench-math)): $(common-objpfx)math/libm.so $(addprefix $(objpfx)bench-,$(bench-pthread)): \ $(common-objpfx)nptl/libpthread.so +$(objpfx)bench-malloc-threads-8: $(common-objpfx)nptl/libpthread.so +$(objpfx)bench-malloc-threads-32: $(common-objpfx)nptl/libpthread.so +$(objpfx)bench-malloc-threads-64: $(common-objpfx)nptl/libpthread.so diff --git a/benchtests/bench-malloc-threads-32.c b/benchtests/bench-malloc-threads-32.c new file mode 100644 index 0000000..463ceb7 --- /dev/null +++ b/benchtests/bench-malloc-threads-32.c @@ -0,0 +1,20 @@ +/* Measure malloc and free functions with threads. + Copyright (C) 2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define NUM_THREADS 32 +#include "bench-malloc.c" diff --git a/benchtests/bench-malloc-threads-64.c b/benchtests/bench-malloc-threads-64.c new file mode 100644 index 0000000..61d8c10 --- /dev/null +++ b/benchtests/bench-malloc-threads-64.c @@ -0,0 +1,20 @@ +/* Measure malloc and free functions with threads. + Copyright (C) 2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define NUM_THREADS 64 +#include "bench-malloc.c" diff --git a/benchtests/bench-malloc-threads-8.c b/benchtests/bench-malloc-threads-8.c new file mode 100644 index 0000000..ac4ff79 --- /dev/null +++ b/benchtests/bench-malloc-threads-8.c @@ -0,0 +1,20 @@ +/* Measure malloc and free functions with threads. + Copyright (C) 2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#define NUM_THREADS 8 +#include "bench-malloc.c" diff --git a/benchtests/bench-malloc.c b/benchtests/bench-malloc.c new file mode 100644 index 0000000..dc4fe17 --- /dev/null +++ b/benchtests/bench-malloc.c @@ -0,0 +1,236 @@ +/* Benchmark malloc and free functions. + Copyright (C) 2013-2014 Free Software Foundation, Inc. + This file is part of the GNU C Library. + + The GNU C Library is free software; you can redistribute it and/or + modify it under the terms of the GNU Lesser General Public + License as published by the Free Software Foundation; either + version 2.1 of the License, or (at your option) any later version. + + The GNU C Library is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + Lesser General Public License for more details. + + You should have received a copy of the GNU Lesser General Public + License along with the GNU C Library; if not, see + . */ + +#include +#include +#include +#include +#include +#include + +#include "bench-timing.h" +#include "json-lib.h" + +#define BENCHMARK_ITERATIONS 40000000 +#define RAND_SEED 88 + +#ifndef NUM_THREADS +#define NUM_THREADS 1 +#endif + +/* Maximum memory that can be allocated at any one time is: + + NUM_THREADS * WORKING_SET_SIZE * MAX_ALLOCATION_SIZE + + However due to the distribution of the random block sizes + the typical amount allocated will be much smaller. */ +#define WORKING_SET_SIZE 1024 + +#define MIN_ALLOCATION_SIZE 4 +#define MAX_ALLOCATION_SIZE 32768 + +/* Get a random block size with an inverse square distribution. */ +static unsigned int +get_block_size (unsigned int rand_data) +{ + /* Inverse square. */ + float exponent = -2; + /* Minimum value of distribution. */ + float dist_min = MIN_ALLOCATION_SIZE; + /* Maximum value of distribution. */ + float dist_max = MAX_ALLOCATION_SIZE; + + float min_pow = powf (dist_min, exponent + 1); + float max_pow = powf (dist_max, exponent + 1); + + float r = (float) rand_data / RAND_MAX; + + return (unsigned int) powf ((max_pow - min_pow) * r + min_pow, 1 / (exponent + 1)); +} + +#define NUM_BLOCK_SIZES 8000 +#define NUM_OFFSETS ((WORKING_SET_SIZE) * 4) + +static unsigned int random_block_sizes[NUM_BLOCK_SIZES]; +static unsigned int random_offsets[NUM_OFFSETS]; + +static void +init_random_values (void) +{ + size_t i; + + for (i = 0; i < NUM_BLOCK_SIZES; i++) + random_block_sizes[i] = get_block_size (rand ()); + + for (i = 0; i < NUM_OFFSETS; i++) + random_offsets[i] = rand () % WORKING_SET_SIZE; +} + +static unsigned int +get_random_block_size (unsigned int *state) +{ + unsigned int idx = *state; + + if (idx >= NUM_BLOCK_SIZES - 1) + idx = 0; + else + idx++; + + *state = idx; + + return random_block_sizes[idx]; +} + +static unsigned int +get_random_offset (unsigned int *state) +{ + unsigned int idx = *state; + + if (idx >= NUM_OFFSETS - 1) + idx = 0; + else + idx++; + + *state = idx; + + return random_offsets[idx]; +} + +/* Allocate and free blocks in a random order. */ +static void +malloc_benchmark_loop (size_t iters, void **ptr_arr) +{ + size_t i; + unsigned int offset_state = 0, block_state = 0; + + for (i = 0; i < iters; i++) + { + unsigned int next_idx = get_random_offset (&offset_state); + unsigned int next_block = get_random_block_size (&block_state); + + free (ptr_arr[next_idx]); + + ptr_arr[next_idx] = malloc (next_block); + } +} + +static void *working_set[NUM_THREADS][WORKING_SET_SIZE]; + +#if NUM_THREADS > 1 +static pthread_t threads[NUM_THREADS]; + +struct thread_args +{ + size_t iters; + void **working_set; +}; + +static void * +benchmark_thread (void *arg) +{ + struct thread_args *args = (struct thread_args *) arg; + size_t iters = args->iters; + void *thread_set = args->working_set; + + malloc_benchmark_loop (iters, thread_set); + + return NULL; +} +#endif + +static void +do_benchmark (size_t iters) +{ +#if NUM_THREADS == 1 + malloc_benchmark_loop (iters, working_set[0]); +#else + struct thread_args args[NUM_THREADS]; + + size_t i; + + for (i = 0; i < NUM_THREADS; i++) + { + args[i].iters = iters; + args[i].working_set = working_set[i]; + pthread_create(&threads[i], NULL, benchmark_thread, &args[i]); + } + + for (i = 0; i < NUM_THREADS; i++) + pthread_join(threads[i], NULL); +#endif +} + +int +main (int argc, char **argv) +{ + timing_t start, stop, cur; + size_t iters = BENCHMARK_ITERATIONS; + unsigned long res; + json_ctx_t json_ctx; + double d_total_s, d_total_i; + + init_random_values (); + + json_init (&json_ctx, 0, stdout); + + json_document_begin (&json_ctx); + + json_attr_string (&json_ctx, "timing_type", TIMING_TYPE); + + json_attr_object_begin (&json_ctx, "functions"); + + json_attr_object_begin (&json_ctx, "malloc"); + + json_attr_object_begin (&json_ctx, ""); + + TIMING_INIT (res); + + (void) res; + + TIMING_NOW (start); + do_benchmark (iters); + TIMING_NOW (stop); + + struct rusage usage; + getrusage(RUSAGE_SELF, &usage); + + TIMING_DIFF (cur, start, stop); + + d_total_s = cur; + d_total_i = iters * NUM_THREADS; + + json_attr_double (&json_ctx, "duration", d_total_s); + json_attr_double (&json_ctx, "iterations", d_total_i); + json_attr_double (&json_ctx, "mean", d_total_s / d_total_i); + json_attr_double (&json_ctx, "max_rss", usage.ru_maxrss); + + json_attr_double (&json_ctx, "threads", NUM_THREADS); + json_attr_double (&json_ctx, "min_size", MIN_ALLOCATION_SIZE); + json_attr_double (&json_ctx, "max_size", MAX_ALLOCATION_SIZE); + json_attr_double (&json_ctx, "random_seed", RAND_SEED); + + json_attr_object_end (&json_ctx); + + json_attr_object_end (&json_ctx); + + json_attr_object_end (&json_ctx); + + json_document_end (&json_ctx); + + return 0; +}