From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 79AB0A054F; Mon, 15 Mar 2021 12:29:40 +0100 (CET) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 30AEE24262E; Mon, 15 Mar 2021 12:29:40 +0100 (CET) Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [63.128.21.124]) by mails.dpdk.org (Postfix) with ESMTP id CF24F4003C for ; Mon, 15 Mar 2021 12:29:38 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615807778; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sLAu6aHP1ZYdEJRxqteXzmL6/9DnB9aMMj3uokqvnvI=; b=H1QxgV9r9fUSVoaVUNOU4eoqSYqyzK8ruyjYbtePmSnHFy4vKI6xHMouS71g6MxRiU6oBf wGPZ3Y6geOYgH3Aqbjd5ylKg77ygpSd1teBm8U/TUG0BlWjUYotwn39v6dXsjuAfl1mHZc 76S61ta4FHynL08OMH/qzC7ECuxQKWM= Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-240-W19LwwsKODC9Csz2Rcw_Ag-1; Mon, 15 Mar 2021 07:29:34 -0400 X-MC-Unique: W19LwwsKODC9Csz2Rcw_Ag-1 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 47E86363A2; Mon, 15 Mar 2021 11:29:33 +0000 (UTC) Received: from [10.36.110.10] (unknown [10.36.110.10]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C85526C32F; Mon, 15 Mar 2021 11:29:28 +0000 (UTC) To: "Xia, Chenbo" , "dev@dpdk.org" , "amorenoz@redhat.com" , "david.marchand@redhat.com" , "olivier.matz@6wind.com" References: <20201221161456.31696-1-maxime.coquelin@redhat.com> <20201221161456.31696-3-maxime.coquelin@redhat.com> From: Maxime Coquelin Message-ID: <58fb579b-6186-7018-e5a3-d9bb887475d5@redhat.com> Date: Mon, 15 Mar 2021 12:29:26 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=maxime.coquelin@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Subject: Re: [dpdk-dev] [PATCH 2/3] net/virtio: allocate fake mbuf in Rx queue X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 1/11/21 3:50 AM, Xia, Chenbo wrote: > Hi Maxime, > >> -----Original Message----- >> From: Maxime Coquelin >> Sent: Tuesday, December 22, 2020 12:15 AM >> To: dev@dpdk.org; Xia, Chenbo ; amorenoz@redhat.com; >> david.marchand@redhat.com; olivier.matz@6wind.com >> Cc: Maxime Coquelin >> Subject: [PATCH 2/3] net/virtio: allocate fake mbuf in Rx queue >> >> While it is worth clarifying whether the fake mbuf >> in virtnet_rx struct is really necessary, it is sure >> that it heavily impacts cache usage by being part of >> the struct. Indeed, it takes uses cachelines, and > > Did you mean 'uses cachelines'? I don't know what I meant here :) I will rework it! >> requires alignement on a cacheline. > > Alignment? > > With above fixed: > > Reviewed-by: Chenbo Xia > >> >> Before this series, it means it took 120 bytes in >> virtnet_rx struct: >> >> struct virtnet_rx { >> struct virtqueue * vq; /* 0 8 */ >> >> /* XXX 56 bytes hole, try to pack */ >> >> /* --- cacheline 1 boundary (64 bytes) --- */ >> struct rte_mbuf fake_mbuf __attribute__((__aligned__(64))); >> /* 64 128 */ >> /* --- cacheline 3 boundary (192 bytes) --- */ >> >> This patch allocates it using malloc in order to optimize >> virtnet_rx cache usage and so virtqueue cache usage. >> >> Signed-off-by: Maxime Coquelin >> --- >> drivers/net/virtio/virtio_ethdev.c | 10 ++++++++++ >> drivers/net/virtio/virtio_rxtx.c | 8 +++----- >> drivers/net/virtio/virtio_rxtx.h | 2 +- >> 3 files changed, 14 insertions(+), 6 deletions(-) >> >> diff --git a/drivers/net/virtio/virtio_ethdev.c >> b/drivers/net/virtio/virtio_ethdev.c >> index 297c01a70d..a1351b36ca 100644 >> --- a/drivers/net/virtio/virtio_ethdev.c >> +++ b/drivers/net/virtio/virtio_ethdev.c >> @@ -539,6 +539,7 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t >> queue_idx) >> } >> >> if (queue_type == VTNET_RQ) { >> + struct rte_mbuf *fake_mbuf; >> size_t sz_sw = (RTE_PMD_VIRTIO_RX_MAX_BURST + vq_size) * >> sizeof(vq->sw_ring[0]); >> >> @@ -550,10 +551,18 @@ virtio_init_queue(struct rte_eth_dev *dev, uint16_t >> queue_idx) >> goto fail_q_alloc; >> } >> >> + fake_mbuf = malloc(sizeof(*fake_mbuf)); >> + if (!fake_mbuf) { >> + PMD_INIT_LOG(ERR, "can not allocate fake mbuf"); >> + ret = -ENOMEM; >> + goto fail_q_alloc; >> + } >> + >> vq->sw_ring = sw_ring; >> rxvq = &vq->rxq; >> rxvq->port_id = dev->data->port_id; >> rxvq->mz = mz; >> + rxvq->fake_mbuf = fake_mbuf; >> } else if (queue_type == VTNET_TQ) { >> txvq = &vq->txq; >> txvq->port_id = dev->data->port_id; >> @@ -636,6 +645,7 @@ virtio_free_queues(struct virtio_hw *hw) >> >> queue_type = virtio_get_queue_type(hw, i); >> if (queue_type == VTNET_RQ) { >> + free(vq->rxq.fake_mbuf); >> rte_free(vq->sw_ring); >> rte_memzone_free(vq->rxq.mz); >> } else if (queue_type == VTNET_TQ) { >> diff --git a/drivers/net/virtio/virtio_rxtx.c >> b/drivers/net/virtio/virtio_rxtx.c >> index 1fcce36cbd..d147d7300a 100644 >> --- a/drivers/net/virtio/virtio_rxtx.c >> +++ b/drivers/net/virtio/virtio_rxtx.c >> @@ -703,11 +703,9 @@ virtio_dev_rx_queue_setup_finish(struct rte_eth_dev *dev, >> uint16_t queue_idx) >> virtio_rxq_vec_setup(rxvq); >> } >> >> - memset(&rxvq->fake_mbuf, 0, sizeof(rxvq->fake_mbuf)); >> - for (desc_idx = 0; desc_idx < RTE_PMD_VIRTIO_RX_MAX_BURST; >> - desc_idx++) { >> - vq->sw_ring[vq->vq_nentries + desc_idx] = >> - &rxvq->fake_mbuf; >> + memset(rxvq->fake_mbuf, 0, sizeof(*rxvq->fake_mbuf)); >> + for (desc_idx = 0; desc_idx < RTE_PMD_VIRTIO_RX_MAX_BURST; desc_idx++) { >> + vq->sw_ring[vq->vq_nentries + desc_idx] = rxvq->fake_mbuf; >> } >> >> if (hw->use_vec_rx && !virtio_with_packed_queue(hw)) { >> diff --git a/drivers/net/virtio/virtio_rxtx.h >> b/drivers/net/virtio/virtio_rxtx.h >> index 7f1036be6f..6ce5d67d15 100644 >> --- a/drivers/net/virtio/virtio_rxtx.h >> +++ b/drivers/net/virtio/virtio_rxtx.h >> @@ -19,7 +19,7 @@ struct virtnet_stats { >> >> struct virtnet_rx { >> /* dummy mbuf, for wraparound when processing RX ring. */ >> - struct rte_mbuf fake_mbuf; >> + struct rte_mbuf *fake_mbuf; >> uint64_t mbuf_initializer; /**< value to init mbufs. */ >> struct rte_mempool *mpool; /**< mempool for mbuf allocation */ >> >> -- >> 2.29.2 >