From patchwork Wed Feb 3 09:05:37 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Maxim Uvarov X-Patchwork-Id: 61062 Delivered-To: patch@linaro.org Received: by 10.112.43.199 with SMTP id y7csp171763lbl; Wed, 3 Feb 2016 01:05:52 -0800 (PST) X-Received: by 10.140.223.9 with SMTP id t9mr376099qhb.65.1454490352680; Wed, 03 Feb 2016 01:05:52 -0800 (PST) Return-Path: Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id w142si4800967qhb.59.2016.02.03.01.05.52; Wed, 03 Feb 2016 01:05:52 -0800 (PST) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Authentication-Results: mx.google.com; spf=pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) smtp.mailfrom=lng-odp-bounces@lists.linaro.org; dkim=neutral (body hash did not verify) header.i=@linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id 0799A6173D; Wed, 3 Feb 2016 09:05:52 +0000 (UTC) Authentication-Results: lists.linaro.org; dkim=fail reason="verification failed; unprotected key" header.d=linaro.org header.i=@linaro.org header.b=HhPL2e9y; dkim-adsp=none (unprotected policy); dkim-atps=neutral X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252 X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, T_DKIM_INVALID, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from [127.0.0.1] (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id 9A919615DD; Wed, 3 Feb 2016 09:05:46 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id C9761615EE; Wed, 3 Feb 2016 09:05:44 +0000 (UTC) Received: from mail-lf0-f50.google.com (mail-lf0-f50.google.com [209.85.215.50]) by lists.linaro.org (Postfix) with ESMTPS id 82863615D9 for ; Wed, 3 Feb 2016 09:05:43 +0000 (UTC) Received: by mail-lf0-f50.google.com with SMTP id j78so9237588lfb.1 for ; Wed, 03 Feb 2016 01:05:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=7hxmex31JYtTdz/yQXslEWN897u46GyvZ/k//uwnVdg=; b=HhPL2e9yVajSZBPnonCMEuGR/ghNTvpnEqW4cj8nNH/qdA98C8gC07UCGPVDlp0HCx TXqIzR59eMUdTs1g42wcqBxZ3IGcoDeEpPTI8WTydof7h6WLr+oWpJS7nUuD4o1BldsX 7gDCgBOoldEGl/8PsjnvVXsgcE4/casOU8xG4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=7hxmex31JYtTdz/yQXslEWN897u46GyvZ/k//uwnVdg=; b=Ra7owC0awjEV9JZqjzA/oeCdGJs6S+t7banM7BPwV09OqmEHqprcaD47Z95OPws4CT gR1wtIDN2WYhtfijQHs5XCguyZZl63rPW3NkdFgIfjts3LpKf9vScIZ/dph+GQ01p6Z8 cJVaWNnT902/5/IQfrUI1/8Qz4EBQ1mlFDuCu+Ld7t3UFjGVT3xvsukTv78vsDitLHwh nJEaC8e78hoR1tZLD4ek3FH+tek0Y1zqEzInHqWMztEcykm00gAYk1CFYWk9ssztpOaT tcxo4njnxSqictoNvG4Z8PpqgOmG8DhfE2A6pRlSTmPCpi0T7GMKH4zSE3xcLljDx1tR uxpQ== X-Gm-Message-State: AG10YOSObQNEkqriCE/YxfGqwz2VlwQhGPLVUZmHuB/a1EdRubxDTAn+OtZGYxW6vjhPu0P3FOs= X-Received: by 10.25.138.2 with SMTP id m2mr205344lfd.59.1454490342103; Wed, 03 Feb 2016 01:05:42 -0800 (PST) Received: from localhost.localdomain (ppp91-76-173-134.pppoe.mtu-net.ru. [91.76.173.134]) by smtp.gmail.com with ESMTPSA id jx8sm763911lbc.29.2016.02.03.01.05.41 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Wed, 03 Feb 2016 01:05:41 -0800 (PST) From: Maxim Uvarov To: lng-odp@lists.linaro.org Date: Wed, 3 Feb 2016 12:05:37 +0300 Message-Id: <1454490337-25997-1-git-send-email-maxim.uvarov@linaro.org> X-Mailer: git-send-email 1.9.1 X-Topics: timers patch Subject: [lng-odp] [PATCH] linux-generic: timer use SIGEV_THREAD_ID X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: "The OpenDataPlane \(ODP\) List" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" Switch timer to use SIGEV_THREAD_ID instead of SIGEV_THREAD. I.e. do not start timer handle thread on each timer action. Start timer handle manually and wait for signal there. This patch also fixes nasty bug with hanging timer threads, which wants to access to timer pool and free shm on pool destroy action. Signed-off-by: Maxim Uvarov --- platform/linux-generic/odp_timer.c | 106 ++++++++++++++++++++++++++++++------- 1 file changed, 87 insertions(+), 19 deletions(-) diff --git a/platform/linux-generic/odp_timer.c b/platform/linux-generic/odp_timer.c index 1001af8..8bc8cc8 100644 --- a/platform/linux-generic/odp_timer.c +++ b/platform/linux-generic/odp_timer.c @@ -27,6 +27,10 @@ #include #include #include +#include +#include +#include + #include #include #include @@ -159,7 +163,6 @@ typedef struct odp_timer_pool_s { tick_buf_t *tick_buf; /* Expiration tick and timeout buffer */ odp_timer *timers; /* User pointer and queue handle (and lock) */ odp_atomic_u32_t high_wm;/* High watermark of allocated timers */ - odp_spinlock_t itimer_running; odp_spinlock_t lock; uint32_t num_alloc;/* Current number of allocated timers */ uint32_t first_free;/* 0..max_timers-1 => free timer */ @@ -169,6 +172,9 @@ typedef struct odp_timer_pool_s { odp_shm_t shm; timer_t timerid; int notify_overrun; + pthread_t timer_thread; /* pthread_t of timer thread */ + pid_t timer_thread_id; /* gettid() for timer thread */ + int timer_thread_exit; /* request to exit for timer thread */ } odp_timer_pool; #define MAX_TIMER_POOLS 255 /* Leave one for ODP_TIMER_INVALID */ @@ -254,26 +260,47 @@ static odp_timer_pool *odp_timer_pool_new( } tp->tp_idx = tp_idx; odp_spinlock_init(&tp->lock); - odp_spinlock_init(&tp->itimer_running); timer_pool[tp_idx] = tp; if (tp->param.clk_src == ODP_CLOCK_CPU) itimer_init(tp); return tp; } +static void block_sigalarm(void) +{ + sigset_t sigset; + + sigemptyset(&sigset); + sigaddset(&sigset, SIGALRM); + sigprocmask(SIG_BLOCK, &sigset, NULL); +} + +static void stop_timer_thread(odp_timer_pool *tp) +{ + int ret; + + tp->timer_thread_exit = 1; + ret = pthread_join(tp->timer_thread, NULL); + if (ret != 0) + ODP_ABORT("unable to join thread, err %d\n", ret); +} + static void odp_timer_pool_del(odp_timer_pool *tp) { odp_spinlock_lock(&tp->lock); timer_pool[tp->tp_idx] = NULL; - /* Wait for itimer thread to stop running */ - odp_spinlock_lock(&tp->itimer_running); + + /* Stop timer triggering */ + if (tp->param.clk_src == ODP_CLOCK_CPU) + itimer_fini(tp); + + stop_timer_thread(tp); + if (tp->num_alloc != 0) { /* It's a programming error to attempt to destroy a */ /* timer pool which is still in use */ ODP_ABORT("%s: timers in use\n", tp->name); } - if (tp->param.clk_src == ODP_CLOCK_CPU) - itimer_fini(tp); int rc = odp_shm_free(tp->shm); if (rc != 0) ODP_ABORT("Failed to free shared memory (%d)\n", rc); @@ -632,10 +659,10 @@ static unsigned odp_timer_pool_expire(odp_timer_pool_t tpid, uint64_t tick) * Functions that use Linux/POSIX per-process timers and related facilities *****************************************************************************/ -static void timer_notify(sigval_t sigval) +static void timer_notify(odp_timer_pool *tp) { int overrun; - odp_timer_pool *tp = (odp_timer_pool *)sigval.sival_ptr; + int64_t prev_tick; if (tp->notify_overrun) { overrun = timer_getoverrun(tp->timerid); @@ -653,32 +680,72 @@ static void timer_notify(sigval_t sigval) for (i = 0; i < 32; i += ODP_CACHE_LINE_SIZE / sizeof(array[0])) PREFETCH(&array[i]); #endif - uint64_t prev_tick = odp_atomic_fetch_inc_u64(&tp->cur_tick); - /* Attempt to acquire the lock, check if the old value was clear */ - if (odp_spinlock_trylock(&tp->itimer_running)) { - /* Scan timer array, looking for timers to expire */ - (void)odp_timer_pool_expire(tp, prev_tick); - odp_spinlock_unlock(&tp->itimer_running); - } + prev_tick = odp_atomic_fetch_inc_u64(&tp->cur_tick); + + /* Scan timer array, looking for timers to expire */ + (void)odp_timer_pool_expire(tp, prev_tick); + /* Else skip scan of timers. cur_tick was updated and next itimer * invocation will process older expiration ticks as well */ } +static void *timer_thread(void *arg) +{ + odp_timer_pool *tp = (odp_timer_pool *)arg; + sigset_t sigset; + int ret; + struct timespec tmo; + siginfo_t si; + + tp->timer_thread_id = (pid_t)syscall(SYS_gettid); + + tmo.tv_sec = 0; + tmo.tv_nsec = ODP_TIME_MSEC_IN_NS * 100; + + sigemptyset(&sigset); + sigaddset(&sigset, SIGALRM); + + while (1) { + ret = sigtimedwait(&sigset, &si, &tmo); + if (tp->timer_thread_exit) { + tp->timer_thread_id = 0; + return NULL; + } + if (ret == 0) + timer_notify(tp); + } + + return NULL; +} + static void itimer_init(odp_timer_pool *tp) { struct sigevent sigev; struct itimerspec ispec; uint64_t res, sec, nsec; + int ret; ODP_DBG("Creating POSIX timer for timer pool %s, period %" PRIu64" ns\n", tp->name, tp->param.res_ns); + tp->timer_thread_id = 0; + ret = pthread_create(&tp->timer_thread, NULL, timer_thread, tp); + if (ret) + ODP_ABORT("unable to create timer thread\n"); + + /* wait thread set tp->timer_thread_id */ + do { + sched_yield(); + } while (tp->timer_thread_id == 0); + + /* Block sigalarm in current thread */ + block_sigalarm(); + memset(&sigev, 0, sizeof(sigev)); - memset(&ispec, 0, sizeof(ispec)); - - sigev.sigev_notify = SIGEV_THREAD; - sigev.sigev_notify_function = timer_notify; + sigev.sigev_notify = SIGEV_THREAD_ID; sigev.sigev_value.sival_ptr = tp; + sigev._sigev_un._tid = tp->timer_thread_id; + sigev.sigev_signo = SIGALRM; if (timer_create(CLOCK_MONOTONIC, &sigev, &tp->timerid)) ODP_ABORT("timer_create() returned error %s\n", @@ -688,6 +755,7 @@ static void itimer_init(odp_timer_pool *tp) sec = res / ODP_TIME_SEC_IN_NS; nsec = res - sec * ODP_TIME_SEC_IN_NS; + memset(&ispec, 0, sizeof(ispec)); ispec.it_interval.tv_sec = (time_t)sec; ispec.it_interval.tv_nsec = (long)nsec; ispec.it_value.tv_sec = (time_t)sec;