From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout1.w1.samsung.com (mailout1.w1.samsung.com [210.118.77.11]) by dpdk.org (Postfix) with ESMTP id 21C806CC3; Wed, 19 Oct 2016 11:47:53 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout1.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OFA004HBGJQE570@mailout1.w1.samsung.com>; Wed, 19 Oct 2016 10:47:50 +0100 (BST) Received: from eusmges3.samsung.com (unknown [203.254.199.242]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20161019094749eucas1p17a481c6bd34c1acbf68b9558103e0bb9~_5d4YuXyM1369713697eucas1p17; Wed, 19 Oct 2016 09:47:49 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges3.samsung.com (EUCPMTA) with SMTP id DF.A2.11330.54147085; Wed, 19 Oct 2016 10:47:49 +0100 (BST) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20161019094749eucas1p165f3dad8d3ba7dece68c110150ac6520~_5d3wFAgY0081800818eucas1p15; Wed, 19 Oct 2016 09:47:49 +0000 (GMT) X-AuditID: cbfec7f2-f79556d000002c42-98-58074145245c Received: from eusync1.samsung.com ( [203.254.199.211]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id C6.CE.10494.12147085; Wed, 19 Oct 2016 10:47:13 +0100 (BST) Received: from [106.109.129.180] by eusync1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OFA007WLGJNEY70@eusync1.samsung.com>; Wed, 19 Oct 2016 10:47:48 +0100 (BST) To: Jan Blunck Cc: dev@dpdk.org, Declan Doherty , Heetae Ahn , Yuanhan Liu , Eric Kinzie , Bernard Iremonger , stable@dpdk.org, Dyasly Sergey From: Ilya Maximets Message-id: Date: Wed, 19 Oct 2016 12:47:47 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.2.0 MIME-version: 1.0 In-reply-to: Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrAKsWRmVeSWpSXmKPExsWy7djP87qujuwRBgs2qFlsbhS2ePOgicXi 3aftTBbzTu1lt5j2+Ta7xZX2n+wWrQ9bmSwmz5ay+Nfxh93i+oQLrA5cHr8WLGX12DnrLrvH 5hVaHov3vGTymHcy0KNvyyrGALYoLpuU1JzMstQifbsEroyri/4wFnTrVHTvdW9g/KDUxcjJ ISFgIrHz5wxGCFtM4sK99WxdjFwcQgJLGSXWbNnPCuF8ZpRYP+E/M0zH71OvoaqWMUocvPUP quoFo8S5Z9fAqoQFwiV2X1gINldEQE1ixsllYDazwHwmiY1TM0BsNgEdiVOrj4DFeQXsJM5u AhnEwcEioCqx478YiCkqECGx+24qRIWgxI/J91hAbE6BYIndnz6zQEzUlHjxZRKULS+xec1b ZpBzJAT2sUs82b6IBWSOhICsxKYDUPe7SPy4cZ4JwhaWeHV8CzuELSNxeXI3C4RdLTFxaxs7 xJwWRomFE3+wQiTsJU7dvMoEsYxPYtK26cwQ83klOtqEIEo8JP7f3Qa1y1HiSuMHaPDcZZJ4 Ov0/8wRG+VlI/pmF5IdZSH5YwMi8ilEktbQ4Nz212FivODG3uDQvXS85P3cTIzDdnP53/NMO xq8nrA4xCnAwKvHwelizRQixJpYVV+YeYpTgYFYS4X1mxR4hxJuSWFmVWpQfX1Sak1p8iFGa g0VJnHfPgivhQgLpiSWp2ampBalFMFkmDk6pBkbrgI257Kk3t538sMIo4X4T7/4nKVFqL+Yp mJ99+ZajUjPs4p31T4zX7Hr8fJ6J7az/XLMLNPaVO77J1v8l9fNig8u2iyeEds0vCW+c8O6I xKuEs5Iil5aFzxGfopTUwRF/bT/vmyBjUWXFm4krQ85yddkmz7Q67BmZ3bAzwOmdHHeJEPMF /xolluKMREMt5qLiRAAD0D8EMwMAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrAIsWRmVeSWpSXmKPExsVy+t/xy7qKjuwRBpu+MltsbhS2ePOgicXi 3aftTBbzTu1lt5j2+Ta7xZX2n+wWrQ9bmSwmz5ay+Nfxh93i+oQLrA5cHr8WLGX12DnrLrvH 5hVaHov3vGTymHcy0KNvyyrGALYoN5uM1MSU1CKF1Lzk/JTMvHRbpdAQN10LJYW8xNxUW6UI Xd+QICWFssScUiDPyAANODgHuAcr6dsluGVcXfSHsaBbp6J7r3sD4welLkZODgkBE4nfp16z QdhiEhfurQeyuTiEBJYwSjxYvJkZwnnBKPH8SA8TSJWwQLjE7gsLGUFsEQE1iRknlzFCFN1l kpi09SpYB7PAfCaJxlM/warYBHQkTq0+AmbzCthJnN30j7WLkYODRUBVYsd/MZCwqECExK1V H6FKBCV+TL7HAmJzCgRLrPu1ggWknFlAXWLKlFyQMLOAvMTmNW+ZJzAKzELSMQuhahaSqgWM zKsYRVJLi3PTc4uN9IoTc4tL89L1kvNzNzECY2/bsZ9bdjB2vQs+xCjAwajEw+thzRYhxJpY VlyZe4hRgoNZSYT3mRV7hBBvSmJlVWpRfnxRaU5q8SFGU6APJjJLiSbnA9NCXkm8oYmhuaWh kbGFhbmRkZI479QPV8KFBNITS1KzU1MLUotg+pg4OKUaGCv0HPdc8WeJEQ4W3WXmvSl6z+e9 35/NS3balvU2U+uY/32RV0Gx31oOFvxfq7xikYzrP3UftcOfpF8r60TUJPY62d42k3raH5o0 w2iCc+mf5FW6PKcWPBVdsTLjp0Xc7gbuULnan5Mmby44fEyF7+e9Na08mTlLTbcdavK4rZt5 wkdbqfQsuxJLcUaioRZzUXEiAMbSr0PTAgAA X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20161019094749eucas1p165f3dad8d3ba7dece68c110150ac6520 X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G+yCvOyEseyghOyekBtFbmdpbmVlcg==?= X-Global-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G1NhbXN1bmcgRWxlY3Ryb25pY3MbRW5naW5lZXI=?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20161018122856eucas1p22493fbd48031126de6093fd312e7a0fc X-RootMTR: 20161018122856eucas1p22493fbd48031126de6093fd312e7a0fc References: <1473251290-22053-1-git-send-email-i.maximets@samsung.com> Subject: Re: [dpdk-dev] [PATCH] Revert "bonding: use existing enslaved device queues" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Oct 2016 09:47:53 -0000 On 18.10.2016 18:19, Jan Blunck wrote: > On Tue, Oct 18, 2016 at 2:49 PM, Ilya Maximets wrote: >> On 18.10.2016 15:28, Jan Blunck wrote: >>> If the application already configured queues the PMD should not >>> silently claim ownership and reset them. >>> >>> What exactly is the problem when changing MTU? This works fine from >>> what I can tell. >> >> Following scenario leads to APP PANIC: >> >> 1. mempool_1 = rte_mempool_create() >> 2. rte_eth_rx_queue_setup(bond0, ..., mempool_1); >> 3. rte_eth_dev_start(bond0); >> 4. mempool_2 = rte_mempool_create(); >> 5. rte_eth_dev_stop(bond0); >> 6. rte_eth_rx_queue_setup(bond0, ..., mempool_2); >> 7. rte_eth_dev_start(bond0); >> * RX queues still use 'mempool_1' because reconfiguration doesn't affect them. * >> 8. rte_mempool_free(mempool_1); >> 9. On any rx operation we'll get PANIC because of using freed 'mempool_1': >> PANIC in rte_mempool_get_ops(): >> assert "(ops_index >= 0) && (ops_index < RTE_MEMPOOL_MAX_OPS_IDX)" failed >> >> You may just start OVS 2.6 with DPDK bonding device and attempt to change MTU via 'mtu_request'. >> Bug is easily reproducible. >> > > I see. I'm not 100% that this is expected to work without leaking the > driver's queues though. The driver is allowed to do allocations in > its rx_queue_setup() function that are being freed via > rx_queue_release() later. But rx_queue_release() is only called if you > reconfigure the > device with 0 queues. From what I understand there is no other way to > reconfigure a device to use another mempool. > > But ... even that wouldn't work with the bonding driver right now: the > bonding master only configures the slaves during startup. I can put > that on my todo list. > > Coming back to your original problem: changing the MTU for the bond > does work through rte_eth_dev_set_mtu() for slaves supporting that. In > any other case you could (re-)configure rxmode.max_rx_pkt_len (and > jumbo_frame / enable_scatter accordingly). This does work without a > call to rte_eth_rx_queue_setup(). Thanks for suggestion, but using of rte_eth_dev_set_mtu() without reconfiguration will require to have mempools with huge mbufs (9KB) for all ports from the start. This is unacceptable because leads to significant performance regressions because of fast cache exhausting. Also this will require big work to rewrite OVS reconfiguration code this way. Anyway, it isn't the MTU only problem. Number of rx/tx descriptors also can't be changed in runtime. I'm not fully understand what is the use case for this 'reusing' code. Could you, please, describe situation where this behaviour is necessary? Best regards, Ilya Maximets. >> >>> >>> On Wed, Sep 7, 2016 at 2:28 PM, Ilya Maximets wrote: >>>> This reverts commit 5b7bb2bda5519b7800f814df64d4e015282140e5. >>>> >>>> It is necessary to reconfigure all queues every time because configuration >>>> can be changed. >>>> >>>> For example, if we're reconfiguring bonding device with new memory pool, >>>> already configured queues will still use the old one. And if the old >>>> mempool be freed, application likely will panic in attempt to use >>>> freed mempool. >>>> >>>> This happens when we use the bonding device with OVS 2.6 while MTU >>>> reconfiguration: >>>> >>>> PANIC in rte_mempool_get_ops(): >>>> assert "(ops_index >= 0) && (ops_index < RTE_MEMPOOL_MAX_OPS_IDX)" failed >>>> >>>> Cc: >>>> Signed-off-by: Ilya Maximets >>>> --- >>>> drivers/net/bonding/rte_eth_bond_pmd.c | 10 ++-------- >>>> 1 file changed, 2 insertions(+), 8 deletions(-) >>>> >>>> diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c >>>> index b20a272..eb5b6d1 100644 >>>> --- a/drivers/net/bonding/rte_eth_bond_pmd.c >>>> +++ b/drivers/net/bonding/rte_eth_bond_pmd.c >>>> @@ -1305,8 +1305,6 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, >>>> struct bond_rx_queue *bd_rx_q; >>>> struct bond_tx_queue *bd_tx_q; >>>> >>>> - uint16_t old_nb_tx_queues = slave_eth_dev->data->nb_tx_queues; >>>> - uint16_t old_nb_rx_queues = slave_eth_dev->data->nb_rx_queues; >>>> int errval; >>>> uint16_t q_id; >>>> >>>> @@ -1347,9 +1345,7 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, >>>> } >>>> >>>> /* Setup Rx Queues */ >>>> - /* Use existing queues, if any */ >>>> - for (q_id = old_nb_rx_queues; >>>> - q_id < bonded_eth_dev->data->nb_rx_queues; q_id++) { >>>> + for (q_id = 0; q_id < bonded_eth_dev->data->nb_rx_queues; q_id++) { >>>> bd_rx_q = (struct bond_rx_queue *)bonded_eth_dev->data->rx_queues[q_id]; >>>> >>>> errval = rte_eth_rx_queue_setup(slave_eth_dev->data->port_id, q_id, >>>> @@ -1365,9 +1361,7 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev, >>>> } >>>> >>>> /* Setup Tx Queues */ >>>> - /* Use existing queues, if any */ >>>> - for (q_id = old_nb_tx_queues; >>>> - q_id < bonded_eth_dev->data->nb_tx_queues; q_id++) { >>>> + for (q_id = 0; q_id < bonded_eth_dev->data->nb_tx_queues; q_id++) { >>>> bd_tx_q = (struct bond_tx_queue *)bonded_eth_dev->data->tx_queues[q_id]; >>>> >>>> errval = rte_eth_tx_queue_setup(slave_eth_dev->data->port_id, q_id, >>>> -- >>>> 2.7.4 >>>> >>> >>> >>> > > >