From patchwork Wed Oct 30 20:00:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 178127 Delivered-To: patch@linaro.org Received: by 2002:a92:409a:0:0:0:0:0 with SMTP id d26csp1755015ill; Wed, 30 Oct 2019 13:02:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqzR6U+msbZTYGIQAKc7Zx4vTBkJ92e8U784rp6aLDHZGI6wyIZgGzb0KblWTgA+PWbUzoqv X-Received: by 2002:a17:907:20f3:: with SMTP id rh19mr1330371ejb.289.1572465730607; Wed, 30 Oct 2019 13:02:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572465730; cv=none; d=google.com; s=arc-20160816; b=L8h3ASvnhD+pPLYkBXjSSnvkn3YXB5bn1fuEuNXWyPNBkAfMQTeeYO7tvABQMGPmeT u/fHhxLOEbUJh1p674uHg3sm0grLSYNr4J8FzFAv+vNhlwHoCJxKtrtZwmqvgJN+iEQ6 VqeAT9AdnShIiXSNm7/kFQb4sZVis0WzbmtmWlpuRAWFz7ozBuuXsRIEX1N61rskS4K7 oParKvjV5e5/rMVYHtQio6IUTsAKARLNIdaZNeAi0mOVbp+dczz/7wP/TcYOAL8YdInW bRmZHE4OWHMRvBIWrzAptcA2oWNvzNYshiqr6kg5w1dSA911j5QdweVNh5oxEBESr0Qc dFQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:delivered-to:sender:list-help:list-post:list-archive :list-subscribe:list-unsubscribe:list-id:precedence:mailing-list :dkim-signature:domainkey-signature; bh=voaG7fzUcufsQHM3gig2Rk66hRvZi32m9xy6rCDoox8=; b=mJ2XAEOpiFmFAyRemp9xNDKVQIXlcIi8vi3G8zKHJ/LH4LfAF6HDhJEdwvBfsXm6i1 FGDRAPs2WSCQtNK0TicO71flhmHgWQF8Tb/Yd9zBnxI/PBCSuh+5+ean0hdCe58Thpn8 GcQh8p0+BfoIxMhvYGHzfYBAdorNbjYJUW6rq+OJg+a3ldtOyFCXh8h9crGYpx5zX6X6 1OsBEekqNPqka10MPMjUoeQEdI8cFh/yARdseHMHffZA1Z01Li+baifa8T669eD9vi4c 8ep6yOw+nCLUo3EvfUhgsUgLBlSzByF3fOxatDuEUzHTbGUbgpbTrI1Y6g7mc/eue2ei sIgA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=qEyraRX4; dkim=pass header.i=@linaro.org header.s=google header.b=E3zguMUX; spf=pass (google.com: domain of libc-alpha-return-106479-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="libc-alpha-return-106479-patch=linaro.org@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id f11si1961766ejd.111.2019.10.30.13.02.09 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 30 Oct 2019 13:02:10 -0700 (PDT) Received-SPF: pass (google.com: domain of libc-alpha-return-106479-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=qEyraRX4; dkim=pass header.i=@linaro.org header.s=google header.b=E3zguMUX; spf=pass (google.com: domain of libc-alpha-return-106479-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="libc-alpha-return-106479-patch=linaro.org@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:in-reply-to :references; q=dns; s=default; b=azywEbBoR1xm0+OqEG8H+c8ChBALZsl vdrida6IqgVVj//6+JnuhYHjL8jQED4AqxMHDNxh4qfkJ9leVQ+jLjtXz0lfLXw5 dUt88G+lGVNua8y2pGOFKgz5hQLYN0WInSLAbYr3ZLYKRJUYIpDZNMzTBWhmhHBk h9cl0j2YN424= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id:in-reply-to :references; s=default; bh=xPT/ZBEdyWx7c9P766JyqYnfPHI=; b=qEyra RX4LoU+k2DCmmwnEwgOh7z/agcug7NZd215qYlMIVRg/KUxYXjEkkmBX2Y55yw9b WKMWZYzF/wh9QuNqPVSuKyQQVZ7LTr529J5c4UmJ3sDyo6jf+ZQG6ClVQeNb3n0u Ftn0vdhLjTu2avnU85QHoFFZ0p9hH5gxdYfMmM= Received: (qmail 83239 invoked by alias); 30 Oct 2019 20:01:36 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 79438 invoked by uid 89); 30 Oct 2019 20:01:21 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-21.6 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.1 spammy=ac, caller's X-HELO: mail-qk1-f193.google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=voaG7fzUcufsQHM3gig2Rk66hRvZi32m9xy6rCDoox8=; b=E3zguMUX15j1FzMdVGJ/FDtyfS/5Fi73KzAdN00OsCeRdEn7JOjfBpcFzdE/dbI1if 07wPXKvkB//8AmxtiQjROq14NH35pyybUODyOpgGqsc5XGWdr1/sKuI9/j/jeddPPp2j HUkqS3YmxR2MQUd9MWSsrfaYjwinUKs1PezC4ghTUboiGHWajalNjH93RRduFvDuV8KX eV3XKnEYnW8mLT7HsH3Of5H526RArn6jXUNeG8XHEfO1n0GdKbqmG3Lnc6FhBEYgj+5t TtlRwZ7MIZilswF6I6iqd7EVn5qKERRfG7tFO6sWzHwOcR2j5V4JCpFsMBWqxkFxZNpB B8ag== Return-Path: From: Adhemerval Zanella To: libc-alpha@sourceware.org Cc: Alistair Francis Subject: [PATCH 5/5] Refactor PI mutexes internal defintions Date: Wed, 30 Oct 2019 17:00:52 -0300 Message-Id: <20191030200052.497-5-adhemerval.zanella@linaro.org> In-Reply-To: <20191030200052.497-1-adhemerval.zanella@linaro.org> References: <20191030200052.497-1-adhemerval.zanella@linaro.org> This patch adds the generic futex_lock_pi and futex_unlock_pi to wrap around the syscall machinery required to issue the syscall calls. It simplifies a bit the futex code required to implement PI mutexes. No function changes, checked on x86_64-linux-gnu. --- nptl/pthread_mutex_init.c | 13 ++--- nptl/pthread_mutex_lock.c | 24 +++------ nptl/pthread_mutex_timedlock.c | 60 +++++++++------------ nptl/pthread_mutex_trylock.c | 8 ++- nptl/pthread_mutex_unlock.c | 6 +-- sysdeps/nptl/futex-internal.h | 86 +++++++++++++++++++++++++++++++ sysdeps/nptl/lowlevellock-futex.h | 9 ++++ 7 files changed, 137 insertions(+), 69 deletions(-) -- 2.17.1 Reviewed-by: Carlos O'Donell diff --git a/nptl/pthread_mutex_init.c b/nptl/pthread_mutex_init.c index fe4eeee37c..20800b80f5 100644 --- a/nptl/pthread_mutex_init.c +++ b/nptl/pthread_mutex_init.c @@ -24,6 +24,7 @@ #include "pthreadP.h" #include #include +#include #include @@ -37,19 +38,13 @@ static const struct pthread_mutexattr default_mutexattr = static bool prio_inherit_missing (void) { -#ifdef __NR_futex static int tpi_supported; - if (__glibc_unlikely (tpi_supported == 0)) + if (__glibc_unlikely (atomic_load_relaxed (&tpi_supported) == 0)) { - int lock = 0; - INTERNAL_SYSCALL_DECL (err); - int ret = INTERNAL_SYSCALL (futex, err, 4, &lock, FUTEX_UNLOCK_PI, 0, 0); - assert (INTERNAL_SYSCALL_ERROR_P (ret, err)); - tpi_supported = INTERNAL_SYSCALL_ERRNO (ret, err) == ENOSYS ? -1 : 1; + int e = futex_unlock_pi (&(unsigned int){0}, 0); + atomic_store_relaxed (&tpi_supported, e == ENOSYS ? -1 : 1); } return __glibc_unlikely (tpi_supported < 0); -#endif - return true; } int diff --git a/nptl/pthread_mutex_lock.c b/nptl/pthread_mutex_lock.c index ace436d5a6..05bba50666 100644 --- a/nptl/pthread_mutex_lock.c +++ b/nptl/pthread_mutex_lock.c @@ -24,7 +24,7 @@ #include #include "pthreadP.h" #include -#include +#include #include #ifndef lll_lock_elision @@ -416,21 +416,16 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex) int private = (robust ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex) : PTHREAD_MUTEX_PSHARED (mutex)); - INTERNAL_SYSCALL_DECL (__err); - int e = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock, - __lll_private_flag (FUTEX_LOCK_PI, - private), 1, 0); - - if (INTERNAL_SYSCALL_ERROR_P (e, __err) - && (INTERNAL_SYSCALL_ERRNO (e, __err) == ESRCH - || INTERNAL_SYSCALL_ERRNO (e, __err) == EDEADLK)) + int e = futex_lock_pi ((unsigned int *) &mutex->__data.__lock, + NULL, private); + if (e == ESRCH || e == EDEADLK) { - assert (INTERNAL_SYSCALL_ERRNO (e, __err) != EDEADLK + assert (e != EDEADLK || (kind != PTHREAD_MUTEX_ERRORCHECK_NP && kind != PTHREAD_MUTEX_RECURSIVE_NP)); /* ESRCH can happen only for non-robust PI mutexes where the owner of the lock died. */ - assert (INTERNAL_SYSCALL_ERRNO (e, __err) != ESRCH || !robust); + assert (e != ESRCH || !robust); /* Delay the thread indefinitely. */ while (1) @@ -479,11 +474,8 @@ __pthread_mutex_lock_full (pthread_mutex_t *mutex) /* This mutex is now not recoverable. */ mutex->__data.__count = 0; - INTERNAL_SYSCALL_DECL (__err); - INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock, - __lll_private_flag (FUTEX_UNLOCK_PI, - PTHREAD_ROBUST_MUTEX_PSHARED (mutex)), - 0, 0); + futex_unlock_pi ((unsigned int *) &mutex->__data.__lock, + PTHREAD_ROBUST_MUTEX_PSHARED (mutex)); /* To the kernel, this will be visible after the kernel has acquired the mutex in the syscall. */ diff --git a/nptl/pthread_mutex_timedlock.c b/nptl/pthread_mutex_timedlock.c index 490064d8cf..112175eb11 100644 --- a/nptl/pthread_mutex_timedlock.c +++ b/nptl/pthread_mutex_timedlock.c @@ -25,6 +25,7 @@ #include #include #include +#include #include @@ -377,39 +378,29 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex, int private = (robust ? PTHREAD_ROBUST_MUTEX_PSHARED (mutex) : PTHREAD_MUTEX_PSHARED (mutex)); - INTERNAL_SYSCALL_DECL (__err); - - int e = INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock, - __lll_private_flag (FUTEX_LOCK_PI, - private), 1, - abstime); - if (INTERNAL_SYSCALL_ERROR_P (e, __err)) + int e = futex_lock_pi ((unsigned int *) &mutex->__data.__lock, + abstime, private); + if (e == ETIMEDOUT) + return ETIMEDOUT; + else if (e == ESRCH || e == EDEADLK) { - if (INTERNAL_SYSCALL_ERRNO (e, __err) == ETIMEDOUT) - return ETIMEDOUT; - - if (INTERNAL_SYSCALL_ERRNO (e, __err) == ESRCH - || INTERNAL_SYSCALL_ERRNO (e, __err) == EDEADLK) - { - assert (INTERNAL_SYSCALL_ERRNO (e, __err) != EDEADLK - || (kind != PTHREAD_MUTEX_ERRORCHECK_NP - && kind != PTHREAD_MUTEX_RECURSIVE_NP)); - /* ESRCH can happen only for non-robust PI mutexes where - the owner of the lock died. */ - assert (INTERNAL_SYSCALL_ERRNO (e, __err) != ESRCH - || !robust); - - /* Delay the thread until the timeout is reached. - Then return ETIMEDOUT. */ - do - e = lll_timedwait (&(int){0}, 0, clockid, abstime, - private); - while (e != ETIMEDOUT); - return ETIMEDOUT; - } - - return INTERNAL_SYSCALL_ERRNO (e, __err); + assert (e != EDEADLK + || (kind != PTHREAD_MUTEX_ERRORCHECK_NP + && kind != PTHREAD_MUTEX_RECURSIVE_NP)); + /* ESRCH can happen only for non-robust PI mutexes where + the owner of the lock died. */ + assert (e != ESRCH || !robust); + + /* Delay the thread until the timeout is reached. Then return + ETIMEDOUT. */ + do + e = lll_timedwait (&(int){0}, 0, clockid, abstime, + private); + while (e != ETIMEDOUT); + return ETIMEDOUT; } + else if (e != 0) + return e; oldval = mutex->__data.__lock; @@ -447,11 +438,8 @@ __pthread_mutex_clocklock_common (pthread_mutex_t *mutex, /* This mutex is now not recoverable. */ mutex->__data.__count = 0; - INTERNAL_SYSCALL_DECL (__err); - INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock, - __lll_private_flag (FUTEX_UNLOCK_PI, - PTHREAD_ROBUST_MUTEX_PSHARED (mutex)), - 0, 0); + futex_unlock_pi ((unsigned int *) &mutex->__data.__lock, + PTHREAD_ROBUST_MUTEX_PSHARED (mutex)); /* To the kernel, this will be visible after the kernel has acquired the mutex in the syscall. */ diff --git a/nptl/pthread_mutex_trylock.c b/nptl/pthread_mutex_trylock.c index 87e87c013a..d24bb58a8b 100644 --- a/nptl/pthread_mutex_trylock.c +++ b/nptl/pthread_mutex_trylock.c @@ -21,6 +21,7 @@ #include #include "pthreadP.h" #include +#include #ifndef lll_trylock_elision #define lll_trylock_elision(a,t) lll_trylock(a) @@ -346,11 +347,8 @@ __pthread_mutex_trylock (pthread_mutex_t *mutex) /* This mutex is now not recoverable. */ mutex->__data.__count = 0; - INTERNAL_SYSCALL_DECL (__err); - INTERNAL_SYSCALL (futex, __err, 4, &mutex->__data.__lock, - __lll_private_flag (FUTEX_UNLOCK_PI, - PTHREAD_ROBUST_MUTEX_PSHARED (mutex)), - 0, 0); + futex_unlock_pi ((unsigned int *) &mutex->__data.__lock, + PTHREAD_ROBUST_MUTEX_PSHARED (mutex)); /* To the kernel, this will be visible after the kernel has acquired the mutex in the syscall. */ diff --git a/nptl/pthread_mutex_unlock.c b/nptl/pthread_mutex_unlock.c index 71038f92e4..53f8b868e4 100644 --- a/nptl/pthread_mutex_unlock.c +++ b/nptl/pthread_mutex_unlock.c @@ -22,6 +22,7 @@ #include "pthreadP.h" #include #include +#include #ifndef lll_unlock_elision #define lll_unlock_elision(a,b,c) ({ lll_unlock (a,c); 0; }) @@ -277,9 +278,8 @@ __pthread_mutex_unlock_full (pthread_mutex_t *mutex, int decr) if (((l & FUTEX_WAITERS) != 0) || (l != THREAD_GETMEM (THREAD_SELF, tid))) { - INTERNAL_SYSCALL_DECL (__err); - INTERNAL_SYSCALL (futex, __err, 2, &mutex->__data.__lock, - __lll_private_flag (FUTEX_UNLOCK_PI, private)); + futex_unlock_pi ((unsigned int *) &mutex->__data.__lock, + private); break; } } diff --git a/sysdeps/nptl/futex-internal.h b/sysdeps/nptl/futex-internal.h index 76921466f0..2a8e3fb323 100644 --- a/sysdeps/nptl/futex-internal.h +++ b/sysdeps/nptl/futex-internal.h @@ -381,4 +381,90 @@ futex_wake (unsigned int* futex_word, int processes_to_wake, int private) } } +/* The operation checks the value of the futer, if the value is 0, then the + it is atomically set to the caller's thread ID. If the futex value is + nonzero, it is atomically sets the FUTEX_WAITERS bit, which signals wrt + other futex owner that it cannot unlock the futex in user space + atomically by setting its value to 0. + + If more than one wait operations is issued, the enqueueing of the waiters + are done in descending priority order. + + The ABSTIME arguments provides an absolute timeout (measured against the + CLOCK_REALTIME clock). If TIMEOUT is NULL, the operation will block + indefinitely. + + Returns: + + - 0 if woken by a PI unlock operation or spuriously. + - EAGAIN if the futex owner thread ID is about to exit, but has not yet + handled the state cleanup. + - EDEADLK if the futex is already locked by the caller. + - ESRCH if the thread ID int he futex does not exist. + - EINVAL is the state is corrupted or if there is a waiter on the + futex. + - ETIMEDOUT if the ABSTIME expires. +*/ +static __always_inline int +futex_lock_pi (unsigned int *futex_word, const struct timespec *abstime, + int private) +{ + int err = lll_futex_timed_lock_pi (futex_word, abstime, private); + switch (err) + { + case 0: + case -EAGAIN: + case -EINTR: + case -ETIMEDOUT: + case -ESRCH: + case -EDEADLK: + case -EINVAL: /* This indicates either state corruption or that the kernel + found a waiter on futex address which is waiting via + FUTEX_WAIT or FUTEX_WAIT_BITSET. This is reported on + some futex_lock_pi usage (pthread_mutex_timedlock for + instance). */ + return -err; + + case -EFAULT: /* Must have been caused by a glibc or application bug. */ + case -ENOSYS: /* Must have been caused by a glibc bug. */ + /* No other errors are documented at this time. */ + default: + futex_fatal_error (); + } +} + +/* Wakes the top priority waiter that called a futex_lock_pi operation on + the futex. + + Returns the same values as futex_lock_pi under those same conditions; + additionally, returns EPERM when the caller is not allowed to attach + itself to the futex. */ +static __always_inline int +futex_unlock_pi (unsigned int *futex_word, int private) +{ + int err = lll_futex_timed_unlock_pi (futex_word, private); + switch (err) + { + case 0: + case -EAGAIN: + case -EINTR: + case -ETIMEDOUT: + case -ESRCH: + case -EDEADLK: + case -ENOSYS: + case -EPERM: /* The caller is not allowed to attach itself to the futex. + Used to check if PI futexes are supported by the + kernel. */ + return -err; + + case -EINVAL: /* Either due to wrong alignment or due to the timeout not + being normalized. Must have been caused by a glibc or + application bug. */ + case -EFAULT: /* Must have been caused by a glibc or application bug. */ + /* No other errors are documented at this time. */ + default: + futex_fatal_error (); + } +} + #endif /* futex-internal.h */ diff --git a/sysdeps/nptl/lowlevellock-futex.h b/sysdeps/nptl/lowlevellock-futex.h index 392277e8a5..ff0fd4edc4 100644 --- a/sysdeps/nptl/lowlevellock-futex.h +++ b/sysdeps/nptl/lowlevellock-futex.h @@ -140,6 +140,15 @@ /* Priority Inheritance support. */ +#define lll_futex_timed_lock_pi(futexp, abstime, private) \ + lll_futex_syscall (4, futexp, \ + __lll_private_flag (FUTEX_LOCK_PI, private), \ + 0, abstime) + +#define lll_futex_timed_unlock_pi(futexp, private) \ + lll_futex_syscall (4, futexp, \ + __lll_private_flag (FUTEX_UNLOCK_PI, private), \ + 0, 0) /* Like lll_futex_wait (FUTEXP, VAL, PRIVATE) but with the expectation that lll_futex_cmp_requeue_pi (FUTEXP, _, _, MUTEX, _, PRIVATE) will