From patchwork Thu Oct 25 17:41:02 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Adhemerval Zanella Netto X-Patchwork-Id: 149540 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp1635042ljp; Thu, 25 Oct 2018 10:41:23 -0700 (PDT) X-Google-Smtp-Source: AJdET5fIizn5RJwqZ3VIJKPN86xd4vV4Y7xLVdtPSq1vDSGcPB7Z0Bk3KqvQW5lldXcnnLQqndX0 X-Received: by 2002:a17:902:be11:: with SMTP id r17-v6mr145395pls.95.1540489283758; Thu, 25 Oct 2018 10:41:23 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1540489283; cv=none; d=google.com; s=arc-20160816; b=EmMyc9tJmVuoI4A6pSgr3dpxYdUipkEsEdBi330Qjb4C48v6JS3y1wlVigPV925cvq EmBkeYrVpsFREae2HrujMIkj/jbEkAL+Orp6Ga1enn+sVQFaGbstXfBW2A+QpcdyIBKr tgAecPB8UARphV7g3E2IXCW+Eb4AdotAldbM4Fnw9BFzHrWC/Wsc479JA/LPaU4r0lFU CU5IfDwmODvUttjg1TMnrCDsT6uYyBRTC6d21NrAw69ngyf8HIoXKOrU6TwGo9MO2GvN LpQGKBooQgsEFNyFv8TL7+NmFsveISxfFv+8Yuh1detv1MyM2hn5Uvl5bbWMvIpnbx1C gIzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=message-id:date:subject:cc:to:from:dkim-signature:delivered-to :sender:list-help:list-post:list-archive:list-subscribe :list-unsubscribe:list-id:precedence:mailing-list:dkim-signature :domainkey-signature; bh=VVxbSTJiXPo/pA8pgavLue1cDhdzwquhYQphqnLohwI=; b=Um8modkG1uGHOg6iXRLkcbZNxSzsHuzOu46inUmvSaJ5K39sb1y23J2zvwUszepEAp ALHrsYFv73FtRmoywlbqnS9o08RcZmu+lCExnxsDfdRvcJIwSVVXv1Xkoy++IQ651WRk upuevyRbYvt4WU2esIv4FXSbDnjBYcX+dZD5/Wefa3DXahIfdzB4IjMC7sDYk3lqzMwt q6CzTIET25GaarEEJo0TVImPWciB3k8lvIzuIqmL5ZG9axW9e+R+0UhsTVqabRr9hs05 HzV18m/LymCocvCY3PbFA8s6LtKbeEG3TPKGRUhLOorjUHAj26FM4KlRHJUn2Zb76VZ8 mh2g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=Ygazj3eN; dkim=pass header.i=@linaro.org header.s=google header.b=LLF8BQ13; spf=pass (google.com: domain of libc-alpha-return-96743-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="libc-alpha-return-96743-patch=linaro.org@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from sourceware.org (server1.sourceware.org. [209.132.180.131]) by mx.google.com with ESMTPS id g198-v6si8202370pfb.165.2018.10.25.10.41.23 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 25 Oct 2018 10:41:23 -0700 (PDT) Received-SPF: pass (google.com: domain of libc-alpha-return-96743-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) client-ip=209.132.180.131; Authentication-Results: mx.google.com; dkim=pass header.i=@sourceware.org header.s=default header.b=Ygazj3eN; dkim=pass header.i=@linaro.org header.s=google header.b=LLF8BQ13; spf=pass (google.com: domain of libc-alpha-return-96743-patch=linaro.org@sourceware.org designates 209.132.180.131 as permitted sender) smtp.mailfrom="libc-alpha-return-96743-patch=linaro.org@sourceware.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org DomainKey-Signature: a=rsa-sha1; c=nofws; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id; q=dns; s= default; b=A9Dt08+f8rFVYdJPN0yGz4HUCy4l9kqv8kmHcz+kdnlQiGakIr/1y DFAzKfBCSJMop05HsB1ZZFmCha0GITxMu7qA8DVWICnOHz+wbNY0Kz3HU1mkMlGX J2/2h2JegOUGV2EUBOFEGO9IZvGfQNgleKlfgNtdqe/v5odj2sAAvw= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=sourceware.org; h=list-id :list-unsubscribe:list-subscribe:list-archive:list-post :list-help:sender:from:to:cc:subject:date:message-id; s=default; bh=Dr9p2HfvSI26Ef4N8dBxzmMJmKI=; b=Ygazj3eNujfNYPZKwA0btRGQO3ae LKHMxzAiAMSKW1c3S9qvcw5jV9Ml5S5ZQXC71Ul+p6IbxbIb89iL6sCsqzpkoDfQ cu/IhsRqxjtHLi1bWKm5kUm/LccLeNoqpVGXkKRpEywZth93iqsAy9/mm9sHzYG2 TnOr8g5rmtww6HI= Received: (qmail 120902 invoked by alias); 25 Oct 2018 17:41:14 -0000 Mailing-List: contact libc-alpha-help@sourceware.org; run by ezmlm Precedence: bulk List-Id: List-Unsubscribe: List-Subscribe: List-Archive: List-Post: List-Help: , Sender: libc-alpha-owner@sourceware.org Delivered-To: mailing list libc-alpha@sourceware.org Received: (qmail 120887 invoked by uid 89); 25 Oct 2018 17:41:13 -0000 Authentication-Results: sourceware.org; auth=none X-Spam-SWARE-Status: No, score=-25.1 required=5.0 tests=AWL, BAYES_00, GIT_PATCH_0, GIT_PATCH_1, GIT_PATCH_2, GIT_PATCH_3, KAM_ASCII_DIVIDERS, RCVD_IN_DNSWL_NONE, SPF_PASS autolearn=ham version=3.3.2 spammy=lazily X-HELO: mail-qt1-f195.google.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=VVxbSTJiXPo/pA8pgavLue1cDhdzwquhYQphqnLohwI=; b=LLF8BQ130m4n8R8izY2TDGyNh7scfGyWeufEfU4Q9zOWYq/wScjE+s/Jx38JkfiJJx Mv//6I28Zqee8r9xOMUXUJvq114s3u1S4fCAmSyfIWXJMHfJoOhMG7CR8N89jqU5y+qW p23+bOILYFVWAs+W+Im6Yb3Yj4Vo4n5dXrOfs= Return-Path: From: Adhemerval Zanella To: libc-alpha@sourceware.org Cc: fweimer@redhat.com Subject: [PATCH v3 1/2] posix: Use posix_spawn on popen Date: Thu, 25 Oct 2018 14:41:02 -0300 Message-Id: <20181025174103.31596-1-adhemerval.zanella@linaro.org> This patch uses posix_spawn on popen instead of fork and execl. On Linux this has the advantage of much lower memory consumption (usually 32 Kb minimum for the mmap stack area). Two issues are also fixed with this change: * BZ#17490: although POSIX pthread_atfork description only list 'fork' as the function that should execute the atfork handlers, popen description states that: '[...] shall be *as if* a child process were created within the popen() call using the fork() function [...]' Other libc/system seems to follow the idea atfork handlers should not be executed for popen: libc/system | run atfork handles | notes ------------|----------------------|--------------------------------------- Freebsd | no | uses vfork Solaris 11 | no | MacOSX 11 | no | implemented through posix_spawn syscall ------------|----------------------|---------------------------------------- Similar to posix_spawn and system, popen idea is to spawn a different binary so all the POSIX rationale to run the atfork handlers to avoid internal process inconsistency is not really required and in some cases might be unsafe. * BZ#22834: now that proc_file_chain is not copied on another process, it just require to access is through the proc_file_chain_lock. Checked on x86_64-linux-gnu and i686-linux-gnu. [BZ #22834] [BZ #17490] * NEWS: Add new semantic for atfork with popen and system. * libio/iopopen.c (_IO_new_proc_open): use posix_spawn instead of fork and execl. --- ChangeLog | 6 +++ NEWS | 6 +++ libio/iopopen.c | 116 +++++++++++++++++++++++++++++++----------------- 3 files changed, 87 insertions(+), 41 deletions(-) -- 2.17.1 diff --git a/NEWS b/NEWS index f054dc0433..c76813d12c 100644 --- a/NEWS +++ b/NEWS @@ -30,6 +30,12 @@ Major new features: HTM state is saved and restore lazily (the state being saved even when the process actually does not use HTM). +* The popen and system do not run atfork handlers anymore (BZ#17490). + Although it is a possible POSIX violation, the POSIX rationale in + pthread_atfork documentation regarding atfork handlers is to handle + incosistent mutex state after fork call in multithread environment. + In both popen and system there is no direct access to user-defined mutexes. + Deprecated and removed features, and other changes affecting compatibility: * The glibc.tune tunable namespace has been renamed to glibc.cpu and the diff --git a/libio/iopopen.c b/libio/iopopen.c index 2eff45b4c8..da24e60eef 100644 --- a/libio/iopopen.c +++ b/libio/iopopen.c @@ -34,7 +34,8 @@ #include #include #include -#include +#include +#include struct _IO_proc_file { @@ -64,8 +65,8 @@ _IO_new_proc_open (FILE *fp, const char *command, const char *mode) { int read_or_write; int parent_end, child_end; + int child_std_end; int pipe_fds[2]; - pid_t child_pid; int do_read = 0; int do_write = 0; @@ -108,59 +109,92 @@ _IO_new_proc_open (FILE *fp, const char *command, const char *mode) if (do_read) { - parent_end = pipe_fds[0]; - child_end = pipe_fds[1]; + parent_end = 0; + child_end = 1; read_or_write = _IO_NO_WRITES; + child_std_end = 1; } else { - parent_end = pipe_fds[1]; - child_end = pipe_fds[0]; + parent_end = 1; + child_end = 0; read_or_write = _IO_NO_READS; + child_std_end = 0; } - ((_IO_proc_file *) fp)->pid = child_pid = __fork (); - if (child_pid == 0) - { - int child_std_end = do_read ? 1 : 0; - struct _IO_proc_file *p; - - if (child_end != child_std_end) - __dup2 (child_end, child_std_end); - else - /* The descriptor is already the one we will use. But it must - not be marked close-on-exec. Undo the effects. */ - __fcntl (child_end, F_SETFD, 0); - /* POSIX.2: "popen() shall ensure that any streams from previous - popen() calls that remain open in the parent process are closed - in the new child process." */ - for (p = proc_file_chain; p; p = p->next) - { - int fd = _IO_fileno ((FILE *) p); + { + posix_spawn_file_actions_t fa; + /* posix_spawn_file_actions_init does not fail. */ + __posix_spawn_file_actions_init (&fa); - /* If any stream from previous popen() calls has fileno - child_std_end, it has been already closed by the dup2 syscall - above. */ - if (fd != child_std_end) - __close_nocancel (fd); - } + /* The descriptor is already the one the child will use. In this case + it must be moved to another one otherwise, there is no safe way to + remove the close-on-exec flag in the child without creating a FD leak + race in the parent. */ + if (pipe_fds[child_end] == child_std_end) + { + int tmp = __fcntl (child_std_end, F_DUPFD_CLOEXEC, 0); + if (tmp < 0) + goto spawn_failure; + __close_nocancel (pipe_fds[child_end]); + pipe_fds[child_end] = tmp; + } - execl ("/bin/sh", "sh", "-c", command, (char *) 0); - _exit (127); - } - __close_nocancel (child_end); - if (child_pid < 0) - { - __close_nocancel (parent_end); - return NULL; - } + if (__posix_spawn_file_actions_adddup2 (&fa, pipe_fds[child_end], + child_std_end) != 0) + goto spawn_failure; + + /* POSIX.2: "popen() shall ensure that any streams from previous popen() + calls that remain open in the parent process are closed in the new + child process." */ + bool addclose_failure = false; +#ifdef _IO_MTSAFE_IO + _IO_cleanup_region_start_noarg (unlock); + _IO_lock_lock (proc_file_chain_lock); +#endif + for (struct _IO_proc_file *p = proc_file_chain; p; p = p->next) + { + int fd = _IO_fileno ((FILE *) p); + + /* If any stream from previous popen() calls has fileno + child_send, it has been already closed by the dup2 syscall + above. */ + if (fd != child_std_end + && __posix_spawn_file_actions_addclose (&fa, fd) != 0) + { + addclose_failure = true; + break; + } + } +#ifdef _IO_MTSAFE_IO + _IO_lock_unlock (proc_file_chain_lock); + _IO_cleanup_region_end (0); +#endif + if (addclose_failure) + goto spawn_failure; + + if (__posix_spawn (&((_IO_proc_file *) fp)->pid, _PATH_BSHELL, &fa, 0, + (char *const[]){ (char*) "sh", (char*) "-c", + (char *) command, NULL }, __environ) != 0) + { + spawn_failure: + __posix_spawn_file_actions_destroy (&fa); + __close_nocancel (pipe_fds[child_end]); + __close_nocancel (pipe_fds[parent_end]); + __set_errno (ENOMEM); + return NULL; + } + + __posix_spawn_file_actions_destroy (&fa); + } + __close_nocancel (pipe_fds[child_end]); if (!do_cloexec) /* Undo the effects of the pipe2 call which set the close-on-exec flag. */ - __fcntl (parent_end, F_SETFD, 0); + __fcntl (pipe_fds[parent_end], F_SETFD, 0); - _IO_fileno (fp) = parent_end; + _IO_fileno (fp) = pipe_fds[parent_end]; /* Link into proc_file_chain. */ #ifdef _IO_MTSAFE_IO