From patchwork Sat Aug 20 07:46:01 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christophe Milard X-Patchwork-Id: 74367 Delivered-To: patch@linaro.org Received: by 10.140.29.52 with SMTP id a49csp661468qga; Sat, 20 Aug 2016 00:55:54 -0700 (PDT) X-Received: by 10.55.176.1 with SMTP id z1mr4445860qke.165.1471679754922; Sat, 20 Aug 2016 00:55:54 -0700 (PDT) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id y63si7193903qkc.251.2016.08.20.00.55.53; Sat, 20 Aug 2016 00:55:54 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dmarc=pass (p=NONE dis=NONE) header.from=linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 594BD60D36; Sat, 20 Aug 2016 07:55:53 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 2202561EF4; Sat, 20 Aug 2016 07:48:10 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 352D061F56; Sat, 20 Aug 2016 07:48:00 +0000 (UTC) Received: from mail-lf0-f51.google.com (mail-lf0-f51.google.com [209.85.215.51]) by lists.linaro.org (Postfix) with ESMTPS id 73987617CD for ; Sat, 20 Aug 2016 07:46:26 +0000 (UTC) Received: by mail-lf0-f51.google.com with SMTP id f93so46700805lfi.2 for ; Sat, 20 Aug 2016 00:46:26 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3fhZbxRUzdowy+jeLpkuq29gsezH5/CRbyFUqU0xr/o=; b=Awj6Xn19vbdUMoCLRC+QX4kpBh/mPag0D+unwEoNPXttnlXvrzeP9ZMNby+ZJ6UnKd SC6UDsy13wEWlJgH+6+MyoUf1GAhsmEaO/Q9dq2/9cJ20E+4Ivz+/eMpRsiUkQEbsELc OpPx22knYOOoVK0slCKWzNsZvC3agTSoGmUQNp33rQltQI17bBXUT45zZsxpnGoztRAn xskDXJBrBxqocQYefTU8I56E4yev347pvjarvCYC6GrKKB+H6uXEoAMSgQwLo+Vu1EUq uMjEClcyKvR20AV/2Y/8PAsz9MbYd/3Xe159AFKuqzRBA0gomJyXtdbEBX0SecIdF8H+ 2qkw== X-Gm-Message-State: AEkoouvjWrhrDNuXajSdOuX5tbJ5hcFRS8853rFDHldUXGFAYOUYga2a9t3uuFJyF8te6J1yohQ= X-Received: by 10.25.126.2 with SMTP id z2mr3342069lfc.195.1471679184884; Sat, 20 Aug 2016 00:46:24 -0700 (PDT) Received: from localhost.localdomain (c-83-233-76-66.cust.bredband2.com. [83.233.76.66]) by smtp.gmail.com with ESMTPSA id 17sm1854092ljj.49.2016.08.20.00.46.23 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Sat, 20 Aug 2016 00:46:24 -0700 (PDT) From: Christophe Milard To: bill.fischofer@linaro.org, mike.holmes@linaro.org, lng-odp@lists.linaro.org Date: Sat, 20 Aug 2016 09:46:01 +0200 Message-Id: <1471679163-17240-12-git-send-email-christophe.milard@linaro.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1471679163-17240-1-git-send-email-christophe.milard@linaro.org> References: <1471679163-17240-1-git-send-email-christophe.milard@linaro.org> X-Topics: patch Subject: [lng-odp] [API-NEXT PATCHv3 11/13] test: validation: drv: shmem: basic tests X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Adding simple tests for shmem operations performed on the driver (drv) interface. Does not to assume pthread as odpthreads (the barrier is now part of the reserved area, rather than a global which only works for pthreads). Tests simple functionanlity, but regardless of reserve() time (i.e. memory reservation occurs both before and after odpthread creation here). Signed-off-by: Christophe Milard --- test/common_plat/m4/configure.m4 | 3 +- test/common_plat/validation/drv/Makefile.am | 3 +- .../common_plat/validation/drv/drvshmem/.gitignore | 1 + .../validation/drv/drvshmem/Makefile.am | 10 + .../common_plat/validation/drv/drvshmem/drvshmem.c | 351 +++++++++++++++++++++ .../common_plat/validation/drv/drvshmem/drvshmem.h | 25 ++ .../validation/drv/drvshmem/drvshmem_main.c | 12 + test/linux-generic/Makefile.am | 1 + 8 files changed, 404 insertions(+), 2 deletions(-) create mode 100644 test/common_plat/validation/drv/drvshmem/.gitignore create mode 100644 test/common_plat/validation/drv/drvshmem/Makefile.am create mode 100644 test/common_plat/validation/drv/drvshmem/drvshmem.c create mode 100644 test/common_plat/validation/drv/drvshmem/drvshmem.h create mode 100644 test/common_plat/validation/drv/drvshmem/drvshmem_main.c -- 2.7.4 diff --git a/test/common_plat/m4/configure.m4 b/test/common_plat/m4/configure.m4 index 20c3a3b..1fc350d 100644 --- a/test/common_plat/m4/configure.m4 +++ b/test/common_plat/m4/configure.m4 @@ -32,4 +32,5 @@ AC_CONFIG_FILES([test/common_plat/Makefile test/common_plat/validation/api/timer/Makefile test/common_plat/validation/api/traffic_mngr/Makefile test/common_plat/validation/drv/Makefile - test/common_plat/validation/drv/drvatomic/Makefile]) + test/common_plat/validation/drv/drvatomic/Makefile + test/common_plat/validation/drv/drvshmem/Makefile]) diff --git a/test/common_plat/validation/drv/Makefile.am b/test/common_plat/validation/drv/Makefile.am index eedbad5..bcdb92e 100644 --- a/test/common_plat/validation/drv/Makefile.am +++ b/test/common_plat/validation/drv/Makefile.am @@ -1,4 +1,5 @@ -ODPDRV_MODULES = drvatomic +ODPDRV_MODULES = drvatomic \ + drvshmem SUBDIRS = $(ODPDRV_MODULES) diff --git a/test/common_plat/validation/drv/drvshmem/.gitignore b/test/common_plat/validation/drv/drvshmem/.gitignore new file mode 100644 index 0000000..726dea3 --- /dev/null +++ b/test/common_plat/validation/drv/drvshmem/.gitignore @@ -0,0 +1 @@ +drvshmem_main diff --git a/test/common_plat/validation/drv/drvshmem/Makefile.am b/test/common_plat/validation/drv/drvshmem/Makefile.am new file mode 100644 index 0000000..816a5e6 --- /dev/null +++ b/test/common_plat/validation/drv/drvshmem/Makefile.am @@ -0,0 +1,10 @@ +include ../Makefile.inc + +noinst_LTLIBRARIES = libtestdrvshmem.la +libtestdrvshmem_la_SOURCES = drvshmem.c + +test_PROGRAMS = drvshmem_main$(EXEEXT) +dist_drvshmem_main_SOURCES = drvshmem_main.c +drvshmem_main_LDADD = libtestdrvshmem.la $(LIBCUNIT_COMMON) $(LIBODP) + +EXTRA_DIST = drvshmem.h diff --git a/test/common_plat/validation/drv/drvshmem/drvshmem.c b/test/common_plat/validation/drv/drvshmem/drvshmem.c new file mode 100644 index 0000000..42ad844 --- /dev/null +++ b/test/common_plat/validation/drv/drvshmem/drvshmem.c @@ -0,0 +1,351 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include +#include +#include +#include "drvshmem.h" +#include + +#define ALIGN_SIZE (128) +#define MEM_NAME "test_shmem" +#define TEST_SHARE_FOO (0xf0f0f0f0) +#define TEST_SHARE_BAR (0xf0f0f0f) +#define SMALL_MEM 10 +#define MEDIUM_MEM 4096 +#define BIG_MEM 16777216 + +typedef struct { + odpdrv_barrier_t test_barrier1; + odpdrv_barrier_t test_barrier2; + uint32_t foo; + uint32_t bar; + odpdrv_atomic_u32_t index; + odpdrv_shm_t shm[MAX_WORKERS]; +} shared_test_data_t; + +/* memory stuff expected to fit in a single page */ +typedef struct { + int data[SMALL_MEM]; +} shared_test_data_small_t; + +/* memory stuff expected to fit in a huge page */ +typedef struct { + int data[MEDIUM_MEM]; +} shared_test_data_medium_t; + +/* memory stuff expected to fit in many huge pages */ +typedef struct { + int data[BIG_MEM]; +} shared_test_data_big_t; + +/* + * thread part for the drvshmem_test_basic test + */ +static int run_test_basic_thread(void *arg ODP_UNUSED) +{ + odpdrv_shm_info_t info; + odpdrv_shm_t shm; + shared_test_data_t *shared_test_data; + int thr; + + thr = odp_thread_id(); + printf("Thread %i starts\n", thr); + + shm = odpdrv_shm_lookup_by_name(MEM_NAME); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + shared_test_data = odpdrv_shm_addr(shm); + CU_ASSERT(NULL != shared_test_data); + + odpdrv_barrier_wait(&shared_test_data->test_barrier1); + odpdrv_shm_print_all("(from thread) After lookup for the global shmem"); + CU_ASSERT(TEST_SHARE_FOO == shared_test_data->foo); + CU_ASSERT(TEST_SHARE_BAR == shared_test_data->bar); + CU_ASSERT(0 == odpdrv_shm_info(shm, &info)); + CU_ASSERT(0 == strcmp(MEM_NAME, info.name)); + CU_ASSERT(0 == info.flags); + CU_ASSERT(shared_test_data == info.addr); + CU_ASSERT(sizeof(shared_test_data_t) <= info.size); +#ifdef MAP_HUGETLB + CU_ASSERT(odp_sys_huge_page_size() == info.page_size); +#else + CU_ASSERT(odp_sys_page_size() == info.page_size); +#endif + odpdrv_shm_print_all("(from thread) About to end"); + + fflush(stdout); + return CU_get_number_of_failures(); +} + +/* + * test basic things: shmem creation, info, share, and free + */ +void drvshmem_test_basic(void) +{ + int base; /* memory usage when test starts */ + pthrd_arg thrdarg; + odpdrv_shm_t shm; + shared_test_data_t *shared_test_data; + odp_cpumask_t unused; + + base = odpdrv_shm_print_all("Before drvshmem_test_basic"); + shm = odpdrv_shm_reserve(MEM_NAME, + sizeof(shared_test_data_t), ALIGN_SIZE, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + CU_ASSERT(odpdrv_shm_to_u64(shm) != + odpdrv_shm_to_u64(ODPDRV_SHM_INVALID)); + + CU_ASSERT(0 == odpdrv_shm_free_by_handle(shm)); + CU_ASSERT(ODPDRV_SHM_INVALID == odpdrv_shm_lookup_by_name(MEM_NAME)); + + shm = odpdrv_shm_reserve(MEM_NAME, + sizeof(shared_test_data_t), ALIGN_SIZE, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + + shared_test_data = odpdrv_shm_addr(shm); + CU_ASSERT_FATAL(NULL != shared_test_data); + shared_test_data->foo = TEST_SHARE_FOO; + shared_test_data->bar = TEST_SHARE_BAR; + + CU_ASSERT(odpdrv_shm_free_by_address((char *)shared_test_data - 1) < 0); + + thrdarg.numthrds = odp_cpumask_default_worker(&unused, 0); + + if (thrdarg.numthrds > MAX_WORKERS) + thrdarg.numthrds = MAX_WORKERS; + + odpdrv_barrier_init(&shared_test_data->test_barrier1, thrdarg.numthrds); + odp_cunit_thread_create(run_test_basic_thread, &thrdarg); + CU_ASSERT(odp_cunit_thread_exit(&thrdarg) >= 0); + + CU_ASSERT(0 == odpdrv_shm_free_by_handle(shm)); + CU_ASSERT(odpdrv_shm_print_all("Test completion") == base); +} + +/* + * thread part for the drvshmem_test_reserve_after_fork + */ +static int run_test_reserve_after_fork(void *arg ODP_UNUSED) +{ + odpdrv_shm_t shm; + shared_test_data_t *glob_data; + int thr; + int thr_index; + char *name; + int name_len; + int size; + shared_test_data_small_t *pattern_small; + shared_test_data_medium_t *pattern_medium; + shared_test_data_big_t *pattern_big; + int i; + + thr = odp_thread_id(); + printf("Thread %i starts\n", thr); + + shm = odpdrv_shm_lookup_by_name(MEM_NAME); + glob_data = odpdrv_shm_addr(shm); + + /* + * odp_thread_id are not guaranteed to be consecutive, so we create + * a consecutive ID + */ + thr_index = odpdrv_atomic_fetch_inc_u32(&glob_data->index); + + /* allocate some memory (of different sizes) and fill with pattern */ + name_len = strlen(MEM_NAME) + 20; + name = malloc(name_len); + snprintf(name, name_len, "%s-%09d", MEM_NAME, thr_index); + switch (thr_index % 3) { + case 0: + size = sizeof(shared_test_data_small_t); + shm = odpdrv_shm_reserve(name, size, 0, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + glob_data->shm[thr_index] = shm; + pattern_small = odpdrv_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL(pattern_small); + for (i = 0; i < SMALL_MEM; i++) + pattern_small->data[i] = i; + break; + case 1: + size = sizeof(shared_test_data_medium_t); + shm = odpdrv_shm_reserve(name, size, 0, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + glob_data->shm[thr_index] = shm; + pattern_medium = odpdrv_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL(pattern_medium); + for (i = 0; i < MEDIUM_MEM; i++) + pattern_medium->data[i] = (i << 2); + break; + case 2: + size = sizeof(shared_test_data_big_t); + shm = odpdrv_shm_reserve(name, size, 0, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + glob_data->shm[thr_index] = shm; + pattern_big = odpdrv_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL(pattern_big); + for (i = 0; i < BIG_MEM; i++) + pattern_big->data[i] = (i >> 2); + break; + } + free(name); + + /* print block address */ + printf("In thread: Block index: %d mapped at %lx\n", + thr_index, (long int)odpdrv_shm_addr(shm)); + + odpdrv_barrier_wait(&glob_data->test_barrier1); + odpdrv_barrier_wait(&glob_data->test_barrier2); + + fflush(stdout); + return CU_get_number_of_failures(); +} + +/* + * test sharing memory reserved after odp_thread creation (e.g. fork()): + */ +void drvshmem_test_reserve_after_fork(void) +{ + int base; /* memory usage when test starts */ + pthrd_arg thrdarg; + odpdrv_shm_t shm; + odpdrv_shm_t thr_shm; + shared_test_data_t *glob_data; + odp_cpumask_t unused; + char *name; + int name_len; + int thr_index; + int i; + void *address; + shared_test_data_small_t *pattern_small; + shared_test_data_medium_t *pattern_medium; + shared_test_data_big_t *pattern_big; + + base = odpdrv_shm_print_all("Before drvshmem_test_reserve_after_fork"); + shm = odpdrv_shm_reserve(MEM_NAME, sizeof(shared_test_data_t), 0, 0); + CU_ASSERT(ODPDRV_SHM_INVALID != shm); + glob_data = odpdrv_shm_addr(shm); + CU_ASSERT_PTR_NOT_NULL(glob_data); + + thrdarg.numthrds = odp_cpumask_default_worker(&unused, 0); + if (thrdarg.numthrds > MAX_WORKERS) + thrdarg.numthrds = MAX_WORKERS; + + odpdrv_barrier_init(&glob_data->test_barrier1, thrdarg.numthrds + 1); + odpdrv_barrier_init(&glob_data->test_barrier2, thrdarg.numthrds + 1); + odpdrv_atomic_store_u32(&glob_data->index, 0); + + odp_cunit_thread_create(run_test_reserve_after_fork, &thrdarg); + + /* wait until all threads have made their shm_reserve: */ + odpdrv_barrier_wait(&glob_data->test_barrier1); + CU_ASSERT(odpdrv_shm_print_all("After all thread reserve") + == base + thrdarg.numthrds + 1); + + /* perform a lookup of all memories, by handle or name: */ + name_len = strlen(MEM_NAME) + 20; + name = malloc(name_len); + for (thr_index = 0; thr_index < thrdarg.numthrds; thr_index++) { + if (thr_index % 2) { + snprintf(name, name_len, "%s-%09d", + MEM_NAME, thr_index); + thr_shm = odpdrv_shm_lookup_by_name(name); + CU_ASSERT(thr_shm == glob_data->shm[thr_index]); + } else { + odpdrv_shm_lookup_by_handle(glob_data->shm[thr_index]); + } + } + + /* check that the patterns are correct: */ + for (thr_index = 0; thr_index < thrdarg.numthrds; thr_index++) { + switch (thr_index % 3) { + case 0: + pattern_small = + odpdrv_shm_addr(glob_data->shm[thr_index]); + CU_ASSERT_PTR_NOT_NULL(pattern_small); + for (i = 0; i < SMALL_MEM; i++) + CU_ASSERT(pattern_small->data[i] == i); + break; + case 1: + pattern_medium = + odpdrv_shm_addr(glob_data->shm[thr_index]); + CU_ASSERT_PTR_NOT_NULL(pattern_medium); + for (i = 0; i < MEDIUM_MEM; i++) + CU_ASSERT(pattern_medium->data[i] == (i << 2)); + break; + case 2: + pattern_big = + odpdrv_shm_addr(glob_data->shm[thr_index]); + CU_ASSERT_PTR_NOT_NULL(pattern_big); + for (i = 0; i < BIG_MEM; i++) + CU_ASSERT(pattern_big->data[i] == (i >> 2)); + break; + } + } + + /* + * print the mapping address of the blocks + */ + for (thr_index = 0; thr_index < thrdarg.numthrds; thr_index++) { + address = odpdrv_shm_addr(glob_data->shm[thr_index]); + printf("In main Block index: %d mapped at %lx\n", + thr_index, (long int)address); + } + + CU_ASSERT(odpdrv_shm_print_all("After main lookup of thread shmem") + == base + thrdarg.numthrds + 1); + + /* unblock the threads and let them terminate (no free is done): */ + odpdrv_barrier_wait(&glob_data->test_barrier2); + + /* at the same time, (race),free of all memories, by handle or name: */ + for (thr_index = 0; thr_index < thrdarg.numthrds; thr_index++) { + if (thr_index % 2) { + thr_shm = glob_data->shm[thr_index]; + CU_ASSERT(odpdrv_shm_free_by_handle(thr_shm) == 0); + } else { + snprintf(name, name_len, "%s-%09d", + MEM_NAME, thr_index); + CU_ASSERT(odpdrv_shm_free_by_name(name) == 0); + } + } + free(name); + + /* wait for all thread endings: */ + CU_ASSERT(odp_cunit_thread_exit(&thrdarg) >= 0); + + /* just glob_data should remain: */ + CU_ASSERT(odpdrv_shm_print_all("After all threads end") == base + 1); + + CU_ASSERT(0 == odpdrv_shm_free_by_handle(shm)); + CU_ASSERT(odpdrv_shm_print_all("Test completion") == base); +} + +odp_testinfo_t drvshmem_suite[] = { + ODP_TEST_INFO(drvshmem_test_basic), + ODP_TEST_INFO(drvshmem_test_reserve_after_fork), + ODP_TEST_INFO_NULL, +}; + +odp_suiteinfo_t drvshmem_suites[] = { + {"Shared Memory", NULL, NULL, drvshmem_suite}, + ODP_SUITE_INFO_NULL, +}; + +int drvshmem_main(int argc, char *argv[]) +{ + int ret; + + /* parse common options: */ + if (odp_cunit_parse_options(argc, argv)) + return -1; + + ret = odp_cunit_register(drvshmem_suites); + + if (ret == 0) + ret = odp_cunit_run(); + + return ret; +} diff --git a/test/common_plat/validation/drv/drvshmem/drvshmem.h b/test/common_plat/validation/drv/drvshmem/drvshmem.h new file mode 100644 index 0000000..12d738e --- /dev/null +++ b/test/common_plat/validation/drv/drvshmem/drvshmem.h @@ -0,0 +1,25 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#ifndef _ODP_TEST_DRVSHMEM_H_ +#define _ODP_TEST_DRVSHMEM_H_ + +#include + +/* test functions: */ +void drvshmem_test_basic(void); +void drvshmem_test_reserve_after_fork(void); + +/* test arrays: */ +extern odp_testinfo_t drvshmem_suite[]; + +/* test registry: */ +extern odp_suiteinfo_t drvshmem_suites[]; + +/* main test program: */ +int drvshmem_main(int argc, char *argv[]); + +#endif diff --git a/test/common_plat/validation/drv/drvshmem/drvshmem_main.c b/test/common_plat/validation/drv/drvshmem/drvshmem_main.c new file mode 100644 index 0000000..566ccec --- /dev/null +++ b/test/common_plat/validation/drv/drvshmem/drvshmem_main.c @@ -0,0 +1,12 @@ +/* Copyright (c) 2016, Linaro Limited + * All rights reserved. + * + * SPDX-License-Identifier: BSD-3-Clause + */ + +#include "drvshmem.h" + +int main(int argc, char *argv[]) +{ + return drvshmem_main(argc, argv); +} diff --git a/test/linux-generic/Makefile.am b/test/linux-generic/Makefile.am index 23b8e03..2b7cbc3 100644 --- a/test/linux-generic/Makefile.am +++ b/test/linux-generic/Makefile.am @@ -35,6 +35,7 @@ TESTS = validation/api/pktio/pktio_run.sh \ $(ALL_API_VALIDATION_DIR)/shmem/shmem_main$(EXEEXT) \ $(ALL_API_VALIDATION_DIR)/system/system_main$(EXEEXT) \ $(ALL_DRV_VALIDATION_DIR)/drvatomic/drvatomic_main$(EXEEXT) \ + $(ALL_DRV_VALIDATION_DIR)/drvshmem/drvshmem_main$(EXEEXT) \ ring/ring_main$(EXEEXT) SUBDIRS += validation/api/pktio\