From patchwork Fri Dec 14 08:59:00 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ilias Apalodimas X-Patchwork-Id: 153787 Delivered-To: patch@linaro.org Received: by 2002:a2e:299d:0:0:0:0:0 with SMTP id p29-v6csp1820091ljp; Fri, 14 Dec 2018 00:59:09 -0800 (PST) X-Google-Smtp-Source: AFSGD/Vyz8eEJeQd/aqWqOGZgqc4nBFBlZ//uy9WSAvrjdsn1275+IDm4E6AOzE/diK+jL8OAzwd X-Received: by 2002:a62:6385:: with SMTP id x127mr2058063pfb.15.1544777948971; Fri, 14 Dec 2018 00:59:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544777948; cv=none; d=google.com; s=arc-20160816; b=uIhF10T+xxvwK+2plLpaJ0JL3dCsHh3STajOT94cbL2u6w1uUP5cVUfqAU9hNm0+HV oNxDlj2qCw5L+T7VkEfsr/OryEQ4HfdndB3AEutZLgy/oB0hcoeQmw7ZfXkd7LkolNT5 4e9+M0sOLLn2lBqtaTfeFW/4ZGL5AdQkoWBQnGx9Uul9DE3srLJlgvxiOgLBZdegOLFo a/cKStxMifLX63GEAzBJ6r74WbppWQMWOKBMqQEb5B1EDngF1+HDtEp13fuWxPNxx1vc BTHhabNum17/n/7KbJcyfx6uq/peLz5jQjw88YOcgYCnxYnREO9+XyCLBsgXpVPgYZUn gm3Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from :dkim-signature; bh=fY/v3IB0GNY/tR3vpHOvysWTo7lSLQ74e18oMOLGu/8=; b=xraAC9SvfjKxRqBSvhj1qRh3dnL29ePQnDvIUhGDH0Fvcl4Y/JROj0fDoWk4CHTTQL POk4zL+lkaEqN0JztWFvucws8qnWQRXYo3T0pUADnBvGAYbriP0bSS6hqYlMujZMX66N bK0IMggXO0aa952MkRd8yVA7R4u3XRcwmrLhJPUWusUzazDgDezU7cigmI4aWgimI+9r PdJjZ9R7r9sezpKWvlTJ2A/nTu/CAomUvtHMJItp6SHpWDYvQZlq2qjJ+3FERLfDehnZ rgSy5YQ7Y8u00bs5KCHbqyTf/71pwYRcKbIQO4YZceUKzmgqh5kNQjaqTfZ1CtgX1rai 5Gwg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=G37MWiF9; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v69si763613pgd.284.2018.12.14.00.59.08; Fri, 14 Dec 2018 00:59:08 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=G37MWiF9; spf=pass (google.com: best guess record for domain of netdev-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=netdev-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728678AbeLNI7H (ORCPT + 10 others); Fri, 14 Dec 2018 03:59:07 -0500 Received: from mail-wr1-f67.google.com ([209.85.221.67]:44480 "EHLO mail-wr1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726520AbeLNI7H (ORCPT ); Fri, 14 Dec 2018 03:59:07 -0500 Received: by mail-wr1-f67.google.com with SMTP id z5so4613764wrt.11 for ; Fri, 14 Dec 2018 00:59:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=from:to:cc:subject:date:message-id; bh=fY/v3IB0GNY/tR3vpHOvysWTo7lSLQ74e18oMOLGu/8=; b=G37MWiF9VmGCHqR9S43YY9Xk4ldFFx/tsRJg/Ds+t7tYBS/7EO8lo9iU4C1EhUmUxS XOLYK0A+dfgwPYN86LOXsBBcNjGHmeT8dp/8xshjuFnKaUSJE7mej/b0NQTxLJyINRNh gz/EdrZNySLu2GI87cvgil3uKwtrxQPrmrvbs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=fY/v3IB0GNY/tR3vpHOvysWTo7lSLQ74e18oMOLGu/8=; b=W5xEAamgY4EfmxHGd5FHZlNCW+DVyHFtoREE0tYu2GWvNX4zJkCdsNhUvZl8Chh787 B3bzGW6TM/gb5buczdxJYMUXFQXd1bJHqbtPJVYnvhFNxn/RQMG3cHZsQlJKJgl0mFoi IwmDNi4lqx2MTJciLvHhf0IP7gFpjWIwAfYlzpDQeE0p1q5e5kB9wNWhfnA9vVXFvJw6 Q4CEErwV0kJY3dkEPq3skpAxygcD7fEil8hK91B4dHc9iQPse0KBDzZsHmWkfIoQ/ChZ 0tPJ86EXMerC392Ld0GStwkAW7NGj/NwgVHrTkZDTpULiVSOrRDLtZvoFF1/9rVTyTvq UyNw== X-Gm-Message-State: AA+aEWb61K/k6aG5ut8L0ICPZKYcWL9IjJIMgqiJgpVGOa9bDXfLbRs8 a8FvRadAu1ptc9OBqiLwZmVAH33XgM4= X-Received: by 2002:adf:e608:: with SMTP id p8mr1840519wrm.166.1544777944969; Fri, 14 Dec 2018 00:59:04 -0800 (PST) Received: from apalos.lan (ppp-94-64-136-149.home.otenet.gr. [94.64.136.149]) by smtp.gmail.com with ESMTPSA id o82sm4373808wmo.29.2018.12.14.00.59.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Fri, 14 Dec 2018 00:59:04 -0800 (PST) From: Ilias Apalodimas To: netdev@vger.kernel.org, jaswinder.singh@linaro.org, ard.biesheuvel@linaro.org Cc: masami.hiramatsu@linaro.org, masahisa.kojima@linaro.org, Ilias Apalodimas Subject: [net-next, PATCH 1/2] net: socionext: correctly recover txq after being full Date: Fri, 14 Dec 2018 10:59:00 +0200 Message-Id: <1544777941-24083-1-git-send-email-ilias.apalodimas@linaro.org> X-Mailer: git-send-email 2.7.4 Sender: netdev-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org Running pktgen with packets sizes > 512b ends up in the interface Txq getting stuck. "netsec 522d0000.ethernet eth0: netsec_netdev_start_xmit: TxQFull!" appears on dmesg but the interface never recovers. It requires an ifconfig down/up to make the interface usable again. The reason that triggers this, is a race condition between .ndo_start_xmit and the napi completion. The available budget is calculated first and indicates the queue is full. Due to a costly netif_err() the queue is not stopped in time while the napi completion runs, clears the irq and frees up descriptors, thus the queue never wakes up again. Fix this by moving the print after stopping the queue, make the print ratelimited, add barriers and check for cleaned descriptors.. Signed-off-by: Ilias Apalodimas --- drivers/net/ethernet/socionext/netsec.c | 56 ++++++++++++++++++++----- 1 file changed, 45 insertions(+), 11 deletions(-) -- 2.19.1 diff --git a/drivers/net/ethernet/socionext/netsec.c b/drivers/net/ethernet/socionext/netsec.c index bba9733b5119..584a6b3f6542 100644 --- a/drivers/net/ethernet/socionext/netsec.c +++ b/drivers/net/ethernet/socionext/netsec.c @@ -656,8 +656,13 @@ static int netsec_process_tx(struct netsec_priv *priv, int budget) budget -= new; } while (new); - if (done && netif_queue_stopped(ndev)) + if (done && netif_queue_stopped(ndev)) { + /* Make sure we update the value, anyone stopping the queue + * after this will read the proper consumer idx + */ + smp_wmb(); netif_wake_queue(ndev); + } return done; } @@ -877,6 +882,41 @@ static void netsec_set_tx_de(struct netsec_priv *priv, dring->head = (dring->head + 1) % DESC_NUM; } +static int netsec_desc_used(struct netsec_desc_ring *dring) +{ + int used; + + if (dring->head >= dring->tail) + used = dring->head - dring->tail; + else + used = dring->head + DESC_NUM - dring->tail; + + return used; +} + +static int netsec_check_stop_tx(struct netsec_priv *priv, int used) +{ + struct netsec_desc_ring *dring = &priv->desc_ring[NETSEC_RING_TX]; + + /* keep tail from touching the queue */ + if (DESC_NUM - used < 2) { + netif_stop_queue(priv->ndev); + + /* Make sure we read the updated value in case + * descriptors got freed + */ + smp_rmb(); + + used = netsec_desc_used(dring); + if (DESC_NUM - used < 2) + return NETDEV_TX_BUSY; + + netif_wake_queue(priv->ndev); + } + + return 0; +} + static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, struct net_device *ndev) { @@ -887,16 +927,10 @@ static netdev_tx_t netsec_netdev_start_xmit(struct sk_buff *skb, u16 tso_seg_len = 0; int filled; - /* differentiate between full/emtpy ring */ - if (dring->head >= dring->tail) - filled = dring->head - dring->tail; - else - filled = dring->head + DESC_NUM - dring->tail; - - if (DESC_NUM - filled < 2) { /* if less than 2 available */ - netif_err(priv, drv, priv->ndev, "%s: TxQFull!\n", __func__); - netif_stop_queue(priv->ndev); - dma_wmb(); + filled = netsec_desc_used(dring); + if (netsec_check_stop_tx(priv, filled)) { + net_warn_ratelimited("%s %s Tx queue full\n", + dev_name(priv->dev), ndev->name); return NETDEV_TX_BUSY; }