From: "Charles (Chas) Williams" <ciwillia@brocade.com>
To: <dev@dpdk.org>
Cc: <skhare@vmware.com>, <mrohilla@brocade.com>
Subject: Re: [dpdk-dev] [PATCH 5/6] net/vmxnet3: receive queue lockup and memleak
Date: Thu, 1 Jun 2017 08:24:16 -0400 [thread overview]
Message-ID: <f91a63d3-6e37-0da7-88d9-b2904a7d29b0@brocade.com> (raw)
In-Reply-To: <1495216560-12920-5-git-send-email-ciwillia@brocade.com>
While looking at another issue, I think one of the issues fixed in this
commit has already been fixed in the last DPDK release by:
commit 8fce14b789aecdb4345a62f6980e7b6e7f4ba245
Author: Stefan Puiu <stefan.puiu@gmail.com>
Date: Mon Dec 19 11:40:53 2016 +0200
net/vmxnet3: fix Rx deadlock
Our use case is that we have an app that needs to keep mbufs around
for a while. We've seen cases when calling vmxnet3_post_rx_bufs() from
vmxet3_recv_pkts(), it might not succeed to add any mbufs to any RX
descriptors (where it returns -err). Since there are no mbufs that the
virtual hardware can use, no packets will be received after this; the
driver won't refill the mbuf after this so it gets stuck in this
state. I call this a deadlock for lack of a better term - the virtual
HW waits for free mbufs, while the app waits for the hardware to
notify it for data (by flipping the generation bit on the used Rx
descriptors). Note that after this, the app can't recover.
...
The mbuf leak due to an error during receive still exists. That can be
refactored into a new commit.
On 05/19/2017 01:55 PM, Charles (Chas) Williams wrote:
> From: Mandeep Rohilla <mrohilla@brocade.com>
>
> The receive queue can lockup if all the rx descriptors have lost
> their mbufs and temporarily there are no mbufs available. This
> can happen if there is an mbuf leak or if the application holds
> on to the mbuf for a while.
>
> This also addresses an mbuf leak in an error condition during
> packet receive.
>
> Signed-off-by: Mandeep Rohilla <mrohilla@brocade.com>
> ---
> drivers/net/vmxnet3/vmxnet3_rxtx.c | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> diff --git a/drivers/net/vmxnet3/vmxnet3_rxtx.c b/drivers/net/vmxnet3/vmxnet3_rxtx.c
> index d8713a1..d21679d 100644
> --- a/drivers/net/vmxnet3/vmxnet3_rxtx.c
> +++ b/drivers/net/vmxnet3/vmxnet3_rxtx.c
> @@ -731,6 +731,7 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
> uint16_t nb_rx;
> uint32_t nb_rxd, idx;
> uint8_t ring_idx;
> + uint8_t i;
> vmxnet3_rx_queue_t *rxq;
> Vmxnet3_RxCompDesc *rcd;
> vmxnet3_buf_info_t *rbi;
> @@ -800,6 +801,12 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
> (int)(rcd - (struct Vmxnet3_RxCompDesc *)
> rxq->comp_ring.base), rcd->rxdIdx);
> rte_pktmbuf_free_seg(rxm);
> + if (rxq->start_seg) {
> + struct rte_mbuf *start = rxq->start_seg;
> +
> + rxq->start_seg = NULL;
> + rte_pktmbuf_free(start);
> + }
> goto rcd_done;
> }
>
> @@ -893,6 +900,18 @@ vmxnet3_recv_pkts(void *rx_queue, struct rte_mbuf **rx_pkts, uint16_t nb_pkts)
> }
> }
>
> + /*
> + * Try to replenish the rx descriptors with the new mbufs
> + */
> + for (i = 0; i < VMXNET3_RX_CMDRING_SIZE; i++) {
> + vmxnet3_post_rx_bufs(rxq, i);
> + if (unlikely(rxq->shared->ctrl.updateRxProd)) {
> + VMXNET3_WRITE_BAR0_REG(hw,
> + rxprod_reg[i] +
> + (rxq->queue_id * VMXNET3_REG_ALIGN),
> + rxq->cmd_ring[i].next2fill);
> + }
> + }
> return nb_rx;
> }
>
>
next prev parent reply other threads:[~2017-06-01 12:24 UTC|newest]
Thread overview: 30+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-19 17:55 [dpdk-dev] [PATCH 1/6] net/vmxnet3: retain counters on restart Charles (Chas) Williams
2017-05-19 17:55 ` [dpdk-dev] [PATCH 2/6] net/vmxnet3: Implement retrieval of extended stats Charles (Chas) Williams
2017-05-24 0:17 ` Shrikrishna Khare
2017-05-19 17:55 ` [dpdk-dev] [PATCH 3/6] net/vmxnet3: Generate link-state change notifications Charles (Chas) Williams
2017-05-19 17:55 ` [dpdk-dev] [PATCH 4/6] net/vmxnet3: Make vmxnet3_process_events less noisy Charles (Chas) Williams
2017-05-23 21:44 ` Shrikrishna Khare
2017-05-19 17:55 ` [dpdk-dev] [PATCH 5/6] net/vmxnet3: receive queue lockup and memleak Charles (Chas) Williams
2017-06-01 12:24 ` Charles (Chas) Williams [this message]
2017-05-24 21:09 ` [dpdk-dev] [PATCH 1/6] net/vmxnet3: retain counters on restart Shrikrishna Khare
2017-05-25 18:31 ` Nachi Prachanda
2017-05-25 20:27 ` Shrikrishna Khare
2017-05-25 22:08 ` Nachi Prachanda
2017-05-26 17:29 ` Shrikrishna Khare
2017-05-26 19:01 ` Nachi Prachanda
2017-05-26 17:31 ` Shrikrishna Khare
2017-06-15 12:16 ` [dpdk-dev] [PATCH V2 0/6] some local vmxnet3 patches Charles (Chas) Williams
2017-06-15 12:16 ` [dpdk-dev] [PATCH v2 1/6] net/vmxnet3: retain counters on restart Charles (Chas) Williams
2017-06-15 12:16 ` [dpdk-dev] [PATCH v2 2/6] net/vmxnet3: Implement retrieval of extended stats Charles (Chas) Williams
2017-06-21 1:42 ` Shrikrishna Khare
2017-06-15 12:16 ` [dpdk-dev] [PATCH v2 3/6] net/vmxnet3: Generate link-state change notifications Charles (Chas) Williams
2017-06-27 13:52 ` Ferruh Yigit
2017-06-15 12:16 ` [dpdk-dev] [PATCH v2 4/6] net/vmxnet3: Make vmxnet3_process_events less noisy Charles (Chas) Williams
2017-06-28 11:30 ` [dpdk-dev] [PATCH V2 0/6] some local vmxnet3 patches Ferruh Yigit
2017-06-28 12:52 ` Ferruh Yigit
2017-06-28 13:09 ` Charles (Chas) Williams
2017-06-28 17:15 ` Charles (Chas) Williams
2017-06-28 17:54 ` Ferruh Yigit
2017-06-15 12:17 ` [dpdk-dev] [PATCH v2 5/6] net/vmxnet3: receive queue memory leak Charles (Chas) Williams
2017-06-23 23:00 ` Shrikrishna Khare
2017-06-15 12:17 ` [dpdk-dev] [PATCH v2 6/6] net/vmxnet3: preserve configured MAC address Charles (Chas) Williams
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f91a63d3-6e37-0da7-88d9-b2904a7d29b0@brocade.com \
--to=ciwillia@brocade.com \
--cc=dev@dpdk.org \
--cc=mrohilla@brocade.com \
--cc=skhare@vmware.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).