DPDK patches and discussions
 help / color / mirror / Atom feed
From: Flavio Leitner <fbl@sysclose.org>
To: Ilya Maximets <i.maximets@ovn.org>
Cc: Maxime Coquelin <maxime.coquelin@redhat.com>,
	dev@dpdk.org, Shahaf Shuler <shahafs@mellanox.com>,
	David Marchand <david.marchand@redhat.com>,
	Tiwei Bie <tiwei.bie@intel.com>,
	Obrembski MichalX <michalx.obrembski@intel.com>,
	Stokes Ian <ian.stokes@intel.com>
Subject: Re: [dpdk-dev] [PATCH v5] vhost: add support for large buffers
Date: Wed, 16 Oct 2019 11:14:46 -0300	[thread overview]
Message-ID: <20191016111446.31d5163d@p50.lan> (raw)
In-Reply-To: <ba266dc1-f578-c5e9-af4d-83a814f9138e@ovn.org>

On Wed, 16 Oct 2019 16:08:54 +0200
Ilya Maximets <i.maximets@ovn.org> wrote:

> On 16.10.2019 16:02, Flavio Leitner wrote:
> > On Wed, 16 Oct 2019 15:46:15 +0200
> > Maxime Coquelin <maxime.coquelin@redhat.com> wrote:
> >   
> >> On 10/16/19 3:32 PM, Ilya Maximets wrote:  
> >>> On 16.10.2019 13:13, Maxime Coquelin wrote:  
> >>>>
> >>>>
> >>>> On 10/15/19 8:59 PM, Flavio Leitner wrote:  
> >>>>> The rte_vhost_dequeue_burst supports two ways of dequeuing data.
> >>>>> If the data fits into a buffer, then all data is copied and a
> >>>>> single linear buffer is returned. Otherwise it allocates
> >>>>> additional mbufs and chains them together to return a multiple
> >>>>> segments mbuf.
> >>>>>
> >>>>> While that covers most use cases, it forces applications that
> >>>>> need to work with larger data sizes to support multiple segments
> >>>>> mbufs. The non-linear characteristic brings complexity and
> >>>>> performance implications to the application.
> >>>>>
> >>>>> To resolve the issue, add support to attach external buffer
> >>>>> to a pktmbuf and let the host provide during registration if
> >>>>> attaching an external buffer to pktmbuf is supported and if
> >>>>> only linear buffer are supported.
> >>>>>
> >>>>> Signed-off-by: Flavio Leitner <fbl@sysclose.org>
> >>>>> ---
> >>>>>    doc/guides/prog_guide/vhost_lib.rst |  35 +++++++++
> >>>>>    lib/librte_vhost/rte_vhost.h        |   4 +
> >>>>>    lib/librte_vhost/socket.c           |  22 ++++++
> >>>>>    lib/librte_vhost/vhost.c            |  22 ++++++
> >>>>>    lib/librte_vhost/vhost.h            |   4 +
> >>>>>    lib/librte_vhost/virtio_net.c       | 109
> >>>>> ++++++++++++++++++++++++---- 6 files changed, 182 insertions(+),
> >>>>> 14 deletions(-)
> >>>>>
> >>>>> - Changelog:
> >>>>>     v5:
> >>>>>       - fixed to destroy mutex if incompatible flags
> >>>>>     v4:
> >>>>>       - allow to use pktmbuf if there is exact space
> >>>>>       - removed log message if the buffer is too big
> >>>>>       - fixed the length to include align padding
> >>>>>       - free allocated buf if shinfo fails
> >>>>>     v3:
> >>>>>       - prevent the new features to be used with zero copy
> >>>>>       - fixed sizeof() usage
> >>>>>       - fixed log msg indentation
> >>>>>       - removed/replaced asserts
> >>>>>       - used the correct virt2iova function
> >>>>>       - fixed the patch's title
> >>>>>       - OvS PoC code:
> >>>>>         https://github.com/fleitner/ovs/tree/rte_malloc-v3
> >>>>>     v2:
> >>>>>       - Used rte_malloc() instead of another mempool as
> >>>>> suggested by Shahaf.
> >>>>>       - Added the documentation section.
> >>>>>       - Using driver registration to negotiate the features.
> >>>>>       - OvS PoC code:
> >>>>>        
> >>>>> https://github.com/fleitner/ovs/commit/8fc197c40b1d4fda331686a7b919e9e2b670dda7
> >>>>>     
> >>>>
> >>>> Applied to dpdk-next-virtio/master.  
> >>>
> >>> Thanks Maxime,
> >>>
> >>> But can we return back the mbuf allocation failure message?  
> >>
> >> Good point, I missed it.
> >>  
> >>> I mean:
> >>>
> >>> diff --git a/lib/librte_vhost/virtio_net.c
> >>> b/lib/librte_vhost/virtio_net.c index 66f0c7206..f8af4e0b3 100644
> >>> --- a/lib/librte_vhost/virtio_net.c
> >>> +++ b/lib/librte_vhost/virtio_net.c
> >>> @@ -1354,8 +1354,11 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
> >>> *dev, struct rte_mempool *mp,
> >>>   {
> >>>       struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
> >>>   
> >>> -    if (unlikely(pkt == NULL))
> >>> +    if (unlikely(pkt == NULL)) {
> >>> +        RTE_LOG(ERR, VHOST_DATA,
> >>> +            "Failed to allocate memory for mbuf.\n");
> >>>           return NULL;
> >>> +    }
> >>>   
> >>>       if (rte_pktmbuf_tailroom(pkt) >= data_len)
> >>>           return pkt;
> >>> ---
> >>>
> >>> It's a hard failure that highlights some significant issues with
> >>> memory pool size or a mbuf leak.  
> >>
> >> I agree, we need this error message.  
> > 
> > If it runs out of memory and there are many packets coming, then it
> > will flood the logs. Maybe we could use something to report in a
> > more moderated way, for example:
> > 
> > diff --git a/lib/librte_vhost/virtio_net.c
> > b/lib/librte_vhost/virtio_net.c index da69ab1db..b1572b3a4 100644
> > --- a/lib/librte_vhost/virtio_net.c
> > +++ b/lib/librte_vhost/virtio_net.c
> > @@ -1354,8 +1354,14 @@ virtio_dev_pktmbuf_alloc(struct virtio_net
> > *dev, struct rte_mempool *mp, {
> >   	struct rte_mbuf *pkt = rte_pktmbuf_alloc(mp);
> >   
> > -	if (unlikely(pkt == NULL))
> > +	if (unlikely(pkt == NULL)) {
> > +		static int rate_lim = 0;
> > +
> > +		if (rate_lim++ % 1000 == 0)
> > +			RTE_LOG...
> > +
> >   		return NULL;
> > +	}
> >   
> >   	if (rte_pktmbuf_tailroom(pkt) >= data_len)
> >   		return pkt;
> > 
> > 
> > Or track this at mempool level and keep stats of failed requests and
> > report that in OvS. That would cover other points too.  
> 
> OVS is already rete-limiting DPDK logs.  Basically, I introduced this
> functionality to limit exactly this message.
> See: 9fd38f6867df ("netdev-dpdk: Limit rate of DPDK logs.")

Thanks, I missed that commit.
fbl

  reply	other threads:[~2019-10-16 14:14 UTC|newest]

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-10-01 22:19 [dpdk-dev] [PATCH] vhost: add support to large linear mbufs Flavio Leitner
2019-10-01 23:10 ` Flavio Leitner
2019-10-02  4:45 ` Shahaf Shuler
2019-10-02  8:04   ` David Marchand
2019-10-02  9:00     ` Shahaf Shuler
2019-10-02 12:58       ` Flavio Leitner
2019-10-02 17:50         ` Shahaf Shuler
2019-10-02 18:15           ` Flavio Leitner
2019-10-03 16:57             ` Ilya Maximets
2019-10-03 21:25               ` Flavio Leitner
2019-10-02  7:51 ` Maxime Coquelin
2019-10-04 20:10 ` [dpdk-dev] [PATCH v2] vhost: add support for large buffers Flavio Leitner
2019-10-06  4:47   ` Shahaf Shuler
2019-10-10  5:12   ` Tiwei Bie
2019-10-10 12:12     ` Flavio Leitner
2019-10-11 17:09   ` [dpdk-dev] [PATCH v3] " Flavio Leitner
2019-10-14  2:44     ` Tiwei Bie
2019-10-15 16:17     ` [dpdk-dev] [PATCH v4] " Flavio Leitner
2019-10-15 17:41       ` Ilya Maximets
2019-10-15 18:44         ` Flavio Leitner
2019-10-15 18:59       ` [dpdk-dev] [PATCH v5] " Flavio Leitner
2019-10-16 10:02         ` Maxime Coquelin
2019-10-16 11:13         ` Maxime Coquelin
2019-10-16 13:32           ` Ilya Maximets
2019-10-16 13:46             ` Maxime Coquelin
2019-10-16 14:02               ` Flavio Leitner
2019-10-16 14:08                 ` Ilya Maximets
2019-10-16 14:14                   ` Flavio Leitner [this message]
2019-10-16 14:05               ` Ilya Maximets
2019-10-29  9:02         ` David Marchand
2019-10-29 12:21           ` Flavio Leitner
2019-10-29 16:19             ` David Marchand

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20191016111446.31d5163d@p50.lan \
    --to=fbl@sysclose.org \
    --cc=david.marchand@redhat.com \
    --cc=dev@dpdk.org \
    --cc=i.maximets@ovn.org \
    --cc=ian.stokes@intel.com \
    --cc=maxime.coquelin@redhat.com \
    --cc=michalx.obrembski@intel.com \
    --cc=shahafs@mellanox.com \
    --cc=tiwei.bie@intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).