From patchwork Thu Aug 20 16:44:17 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stuart Haslam X-Patchwork-Id: 52600 Return-Path: X-Original-To: linaro@patches.linaro.org Delivered-To: linaro@patches.linaro.org Received: from mail-lb0-f200.google.com (mail-lb0-f200.google.com [209.85.217.200]) by patches.linaro.org (Postfix) with ESMTPS id 9083722E4F for ; Thu, 20 Aug 2015 16:45:06 +0000 (UTC) Received: by lbcue2 with SMTP id ue2sf12629123lbc.1 for ; Thu, 20 Aug 2015 09:45:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:delivered-to:delivered-to:from:to:date :message-id:in-reply-to:references:subject:precedence:list-id :list-unsubscribe:list-archive:list-post:list-help:list-subscribe :mime-version:content-type:content-transfer-encoding:errors-to :sender:x-original-sender:x-original-authentication-results :mailing-list; bh=7VS+hl3FKTHiuhGSWRFMMU0hFDH1L+sOMqytoHJ9f3o=; b=adr7JWywIqVFyZKgm5Sh511qqQk3EqxPErJJBegrHZzGtS3CdM4F6/x2nWkDKSTf7k VxsndwTREvAOISOMoVLo+FZ4tQ2o15q5YG4Le/dv3z0nHmdCp0r/fL3S/LQ1XrXr448v MCelw7sOnH9BY5pyAV24tWFw768Q2OIRTPxGw+tn8UWasMKH29VkprQKv2wLAYMc5egK V8pCa55H4y+GGx6I8lVQRvDlyktbOx6YY4xPlN6NZc7cqGKdGM3G+HaVxBCLuHU8GfmR Z7Nfk2iw+eqOupoIAeL30YggyJd0x372le1QWcsTa9BDYdn1k2smNlD1YgQudcapXbIY +YUQ== X-Gm-Message-State: ALoCoQmdOh4mGwwAhwoTydj+P5C4SCkyCZkwigEsIbGtjG3fFd51RkVTEUen5fr9eJE3ABiQ50Kz X-Received: by 10.112.99.37 with SMTP id en5mr1100382lbb.7.1440089105181; Thu, 20 Aug 2015 09:45:05 -0700 (PDT) X-BeenThere: patchwork-forward@linaro.org Received: by 10.152.21.3 with SMTP id r3ls212591lae.89.gmail; Thu, 20 Aug 2015 09:45:05 -0700 (PDT) X-Received: by 10.112.16.135 with SMTP id g7mr3649780lbd.19.1440089105026; Thu, 20 Aug 2015 09:45:05 -0700 (PDT) Received: from mail-lb0-f177.google.com (mail-lb0-f177.google.com. [209.85.217.177]) by mx.google.com with ESMTPS id t9si4048572laj.106.2015.08.20.09.45.05 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 20 Aug 2015 09:45:05 -0700 (PDT) Received-SPF: pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.177 as permitted sender) client-ip=209.85.217.177; Received: by lbbsx3 with SMTP id sx3so27637706lbb.0 for ; Thu, 20 Aug 2015 09:45:04 -0700 (PDT) X-Received: by 10.152.243.9 with SMTP id wu9mr1632886lac.41.1440089104912; Thu, 20 Aug 2015 09:45:04 -0700 (PDT) X-Forwarded-To: patchwork-forward@linaro.org X-Forwarded-For: patch@linaro.org patchwork-forward@linaro.org Delivered-To: patch@linaro.org Received: by 10.112.162.200 with SMTP id yc8csp111198lbb; Thu, 20 Aug 2015 09:45:03 -0700 (PDT) X-Received: by 10.170.180.65 with SMTP id w62mr4689119ykd.60.1440089103687; Thu, 20 Aug 2015 09:45:03 -0700 (PDT) Received: from lists.linaro.org (lists.linaro.org. [54.225.227.206]) by mx.google.com with ESMTP id c33si8226798qga.59.2015.08.20.09.45.02; Thu, 20 Aug 2015 09:45:03 -0700 (PDT) Received-SPF: pass (google.com: domain of lng-odp-bounces@lists.linaro.org designates 54.225.227.206 as permitted sender) client-ip=54.225.227.206; Received: by lists.linaro.org (Postfix, from userid 109) id 67B63620F3; Thu, 20 Aug 2015 16:45:02 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on ip-10-142-244-252.ec2.internal X-Spam-Level: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_LOW, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL, T_FRT_STOCK2, URIBL_BLOCKED autolearn=disabled version=3.4.0 Received: from ip-10-142-244-252.ec2.internal (localhost [127.0.0.1]) by lists.linaro.org (Postfix) with ESMTP id B0CAC6205E; Thu, 20 Aug 2015 16:44:34 +0000 (UTC) X-Original-To: lng-odp@lists.linaro.org Delivered-To: lng-odp@lists.linaro.org Received: by lists.linaro.org (Postfix, from userid 109) id D896462063; Thu, 20 Aug 2015 16:44:27 +0000 (UTC) Received: from mail-wi0-f177.google.com (mail-wi0-f177.google.com [209.85.212.177]) by lists.linaro.org (Postfix) with ESMTPS id 54DAA61DEF for ; Thu, 20 Aug 2015 16:44:26 +0000 (UTC) Received: by widdq5 with SMTP id dq5so11429065wid.0 for ; Thu, 20 Aug 2015 09:44:25 -0700 (PDT) X-Received: by 10.180.108.39 with SMTP id hh7mr15397065wib.3.1440089065610; Thu, 20 Aug 2015 09:44:25 -0700 (PDT) Received: from e106441.emea.arm.com ([2001:41d0:a:3cb4::abcd]) by smtp.gmail.com with ESMTPSA id pu6sm6281514wjc.34.2015.08.20.09.44.24 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Thu, 20 Aug 2015 09:44:25 -0700 (PDT) From: Stuart Haslam To: lng-odp@lists.linaro.org Date: Thu, 20 Aug 2015 17:44:17 +0100 Message-Id: <1440089058-12833-2-git-send-email-stuart.haslam@linaro.org> X-Mailer: git-send-email 2.1.1 In-Reply-To: <1440089058-12833-1-git-send-email-stuart.haslam@linaro.org> References: <1440089058-12833-1-git-send-email-stuart.haslam@linaro.org> X-Topics: patch Subject: [lng-odp] [PATCH 2/3] linux-generic: pktio: handle transmit errors correctly X-BeenThere: lng-odp@lists.linaro.org X-Mailman-Version: 2.1.16 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: , List-Help: , List-Subscribe: , MIME-Version: 1.0 Errors-To: lng-odp-bounces@lists.linaro.org Sender: "lng-odp" X-Removed-Original-Auth: Dkim didn't pass. X-Original-Sender: stuart.haslam@linaro.org X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of patch+caf_=patchwork-forward=linaro.org@linaro.org designates 209.85.217.177 as permitted sender) smtp.mailfrom=patch+caf_=patchwork-forward=linaro.org@linaro.org Mailing-list: list patchwork-forward@linaro.org; contact patchwork-forward+owners@linaro.org X-Google-Group-Id: 836684582541 Errors which occur while sending packets via odp_pktio_send() aren't handled correctly or even consistently across the two socket based implementations. The problems being addressed are; - calls may block indefinitely in certain error conditions (mmsg) - packets may be freed and reported as being sent correctly even though they weren't really sent (mmap) - return value doesn't always accurately reflect number of packets sent - inconsistent use of __odp_errno Signed-off-by: Stuart Haslam --- .../linux-generic/include/odp_packet_io_internal.h | 6 ++ platform/linux-generic/pktio/socket.c | 27 +++--- platform/linux-generic/pktio/socket_mmap.c | 96 +++++++++++----------- 3 files changed, 70 insertions(+), 59 deletions(-) diff --git a/platform/linux-generic/include/odp_packet_io_internal.h b/platform/linux-generic/include/odp_packet_io_internal.h index a5d324e..09d67f5 100644 --- a/platform/linux-generic/include/odp_packet_io_internal.h +++ b/platform/linux-generic/include/odp_packet_io_internal.h @@ -28,6 +28,12 @@ extern "C" { #include #include +/** Determine if a socket read/write error should be reported. Transient errors + * that simply require the caller to retry are ignored, the _send/_recv APIs + * are non-blocking and it is the caller's responsibility to retry if the + * requested number of packets were not handled. */ +#define SOCK_ERR_REPORT(e) (e != EAGAIN && e != EWOULDBLOCK && e != EINTR) + /* Forward declaration */ struct pktio_if_ops; diff --git a/platform/linux-generic/pktio/socket.c b/platform/linux-generic/pktio/socket.c index 45040fd..a96ec90 100644 --- a/platform/linux-generic/pktio/socket.c +++ b/platform/linux-generic/pktio/socket.c @@ -373,9 +373,7 @@ static int sock_mmsg_send(pktio_entry_t *pktio_entry, struct iovec iovecs[ODP_PACKET_SOCKET_MAX_BURST_TX][ODP_BUFFER_MAX_SEG]; int ret; int sockfd; - unsigned i; - unsigned sent_msgs = 0; - unsigned flags; + unsigned n, i; if (odp_unlikely(len > ODP_PACKET_SOCKET_MAX_BURST_TX)) return -1; @@ -389,17 +387,24 @@ static int sock_mmsg_send(pktio_entry_t *pktio_entry, iovecs[i]); } - flags = MSG_DONTWAIT; - for (i = 0; i < len; i += sent_msgs) { - ret = sendmmsg(sockfd, &msgvec[i], len - i, flags); - sent_msgs = ret > 0 ? (unsigned)ret : 0; - flags = 0; /* blocking for next rounds */ + for (i = 0; i < len; ) { + ret = sendmmsg(sockfd, &msgvec[i], len - i, MSG_DONTWAIT); + if (odp_unlikely(ret <= -1)) { + if (i == 0 && SOCK_ERR_REPORT(errno)) { + __odp_errno = errno; + ODP_ERR("sendmmsg(): %s\n", strerror(errno)); + return -1; + } + break; + } + + i += ret; } - for (i = 0; i < len; i++) - odp_packet_free(pkt_table[i]); + for (n = 0; n < i; ++n) + odp_packet_free(pkt_table[n]); - return len; + return i; } /* diff --git a/platform/linux-generic/pktio/socket_mmap.c b/platform/linux-generic/pktio/socket_mmap.c index 7d42678..b394c2c 100644 --- a/platform/linux-generic/pktio/socket_mmap.c +++ b/platform/linux-generic/pktio/socket_mmap.c @@ -174,49 +174,70 @@ static inline unsigned pkt_mmap_v2_tx(int sock, struct ring *ring, { union frame_map ppd; uint32_t pkt_len; - unsigned frame_num, next_frame_num; + unsigned first_frame_num, frame_num, frame_count; int ret; - unsigned i = 0; + uint8_t *buf; + unsigned n, i = 0; + unsigned nb_tx = 0; + int send_errno; - frame_num = ring->frame_num; + first_frame_num = ring->frame_num; + frame_num = first_frame_num; + frame_count = ring->rd_num; while (i < len) { - if (mmap_tx_kernel_ready(ring->rd[frame_num].iov_base)) { - ppd.raw = ring->rd[frame_num].iov_base; + ppd.raw = ring->rd[frame_num].iov_base; + if (!odp_unlikely(mmap_tx_kernel_ready(ppd.raw))) + break; - next_frame_num = (frame_num + 1) % ring->rd_num; + pkt_len = odp_packet_len(pkt_table[i]); + ppd.v2->tp_h.tp_snaplen = pkt_len; + ppd.v2->tp_h.tp_len = pkt_len; - pkt_len = odp_packet_len(pkt_table[i]); - ppd.v2->tp_h.tp_snaplen = pkt_len; - ppd.v2->tp_h.tp_len = pkt_len; + buf = (uint8_t *)ppd.raw + TPACKET2_HDRLEN - + sizeof(struct sockaddr_ll); + odp_packet_copydata_out(pkt_table[i], 0, pkt_len, buf); - odp_packet_copydata_out(pkt_table[i], 0, pkt_len, - (uint8_t *)ppd.raw + - TPACKET2_HDRLEN - - sizeof(struct sockaddr_ll)); + mmap_tx_user_ready(ppd.raw); - mmap_tx_user_ready(ppd.raw); + if (++frame_num >= frame_count) + frame_num = 0; - odp_packet_free(pkt_table[i]); - frame_num = next_frame_num; - i++; - } else { + i++; + } + + ret = sendto(sock, NULL, 0, MSG_DONTWAIT, NULL, 0); + send_errno = errno; + + /* On success, the return value indicates the number of bytes sent. On + * failure a value of -1 is returned, even if the failure occurred + * after some of the packets in the ring have already been sent, so we + * need to inspect the packet status to determine which were sent. */ + for (n = first_frame_num; n < first_frame_num + i; ++n) { + struct tpacket2_hdr *hdr = ring->rd[n].iov_base; + + if (odp_likely(hdr->tp_status == TP_STATUS_AVAILABLE)) { + nb_tx++; + } else if (hdr->tp_status & TP_STATUS_WRONG_FORMAT) { + /* status will be cleared on the next send request */ break; } } - ring->frame_num = frame_num; + ring->frame_num = (ring->frame_num + nb_tx) % frame_count; - ret = sendto(sock, NULL, 0, MSG_DONTWAIT, NULL, 0); - if (ret == -1) { - if (errno != EAGAIN) { - __odp_errno = errno; - ODP_ERR("sendto(pkt mmap): %s\n", strerror(errno)); - return -1; - } + if (odp_unlikely(ret == -1 && + nb_tx == 0 && + SOCK_ERR_REPORT(send_errno))) { + __odp_errno = send_errno; + ODP_ERR("sendto(pkt mmap): %s\n", strerror(send_errno)); + return -1; } - return i; + for (i = 0; i < nb_tx; ++i) + odp_packet_free(pkt_table[i]); + + return nb_tx; } static void mmap_fill_ring(struct ring *ring, odp_pool_t pool_hdl, int fanout) @@ -260,21 +281,6 @@ static void mmap_fill_ring(struct ring *ring, odp_pool_t pool_hdl, int fanout) ring->flen = ring->req.tp_frame_size; } -static int mmap_set_packet_loss_discard(int sock) -{ - int ret, discard = 1; - - ret = setsockopt(sock, SOL_PACKET, PACKET_LOSS, (void *)&discard, - sizeof(discard)); - if (ret == -1) { - __odp_errno = errno; - ODP_ERR("setsockopt(PACKET_LOSS): %s\n", strerror(errno)); - return -1; - } - - return 0; -} - static int mmap_setup_ring(int sock, struct ring *ring, int type, odp_pool_t pool_hdl, int fanout) { @@ -284,12 +290,6 @@ static int mmap_setup_ring(int sock, struct ring *ring, int type, ring->type = type; ring->version = TPACKET_V2; - if (type == PACKET_TX_RING) { - ret = mmap_set_packet_loss_discard(sock); - if (ret != 0) - return -1; - } - mmap_fill_ring(ring, pool_hdl, fanout); ret = setsockopt(sock, SOL_PACKET, type, &ring->req, sizeof(ring->req));