From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mailout3.w1.samsung.com (mailout3.w1.samsung.com [210.118.77.13]) by dpdk.org (Postfix) with ESMTP id F1896326C; Fri, 28 Oct 2016 08:14:21 +0200 (CEST) Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout3.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OFQ000AUUNWZH40@mailout3.w1.samsung.com>; Fri, 28 Oct 2016 07:14:20 +0100 (BST) Received: from eusmges5.samsung.com (unknown [203.254.199.245]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20161028061419eucas1p2212fe89bc4172c3098129a723e764d9b~BnXCqSG812348623486eucas1p24; Fri, 28 Oct 2016 06:14:19 +0000 (GMT) Received: from eucas1p1.samsung.com ( [182.198.249.206]) by eusmges5.samsung.com (EUCPMTA) with SMTP id 23.94.19540.BBCE2185; Fri, 28 Oct 2016 07:14:19 +0100 (BST) Received: from eusmgms2.samsung.com (unknown [182.198.249.180]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20161028061419eucas1p19d43234accc80b0e28efefbfbf6b22dd~BnXB7b_oa1317813178eucas1p13; Fri, 28 Oct 2016 06:14:19 +0000 (GMT) X-AuditID: cbfec7f5-f79ce6d000004c54-d4-5812ecbb3990 Received: from eusync1.samsung.com ( [203.254.199.211]) by eusmgms2.samsung.com (EUCPMTA) with SMTP id BF.3D.10494.A9CE2185; Fri, 28 Oct 2016 07:13:46 +0100 (BST) Received: from [106.109.129.180] by eusync1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OFQ00AGIUNTMQ70@eusync1.samsung.com>; Fri, 28 Oct 2016 07:14:18 +0100 (BST) To: Jan Blunck From: Ilya Maximets Message-id: <8507e1f4-328e-a13d-cee2-69db06fe30be@samsung.com> Date: Fri, 28 Oct 2016 09:14:17 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.3.0 MIME-version: 1.0 In-reply-to: <03e11be9-45b2-e05f-99f1-8be1bcc05f19@samsung.com> Content-type: text/plain; charset=utf-8 Content-transfer-encoding: 7bit X-Brightmail-Tracker: H4sIAAAAAAAAA02Se0hTcRTH+e3eu3s1F7dt1clH0tBCLTUpuESoQcH+EXpArghqtIuaTmVz 5tJA8202H7MHs2RiE9uSatosNBQRzfXQUlFJRbGHiZmlwnyW8yr43+dwvud7+B4OhQlNhDsV E5/EquLlcRK+K25rX/h4qHFKKAuu1AUwdRkiZsAcxkyN3sKZ6b8NPKbC/oZk7s1+IZne3AWS yR7L5jH6cndmNW+ZZPqLu4nwbdJFo4mQvjYMk9K6Gn9pVdNPnrSi84xUV29Gp/kXXY8r2LiY ZFYVFHrFNbpwel/iI98UW+kdlI5WvAqQCwX0EbA2zfI43gXdI8/4BciVEtImBFWmXB5XzCIo L+zib05862khuUY1grejnRvFBAJdVzPuVInoSGjsrkROFtP74UFnNXKKMLqRB9bsz+siPn0Q 7Ja2dZGADoWOH4VYAaIonPaF5SJPJ+6kZVD0/gCn2AEO/cj6pAsdBjW6WsLJGO0HE3OlOMfe UPf0F+ZcBXQ7CeXWPtLpA7QXWFswLsBJmGqxb7AIJjvqSY49oUd/G+c4FUpe5pCcTxaCyhIH wTXCwD7Yx+OWbYdS232M8xdAXo6Qk0jh37Btw/8E9GbMENx9dDh8H2wjipG3YUsew5YMhi0Z jAgzIzGrUSujWPXRQLVcqdbERwVeTVBa0drzvFvtmH+FTO3HWhFNIYmb4DkmlAkJebJaq2xF QGESsSBk7beEAoVce4NVJVxWaeJYdSvyoHDJbkGTsTdSSEfJk9hYlk1kVZtdHuXino66w6VK Rxl//G4WMbbytUE2oPV4MiHST8YE+F4wOCzBqS179ENBe6t9Pihx0VK+VpCmMMvOV0dk/rbI ogpuKv70lxmDG/iRKQ8fzyQQmRava7G2U+LVrKXpCo04P23I5/on+4v5S1W4v+Ps4pxfrWKq GEWcW3GPHNfmNYe4xUpwdbT8sD+mUsv/AyDXoEI4AwAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrCIsWRmVeSWpSXmKPExsVy+t/xy7qz3ghFGLSsZbLY3ChscWOVvcWb B00sFu8+bWeymHdqL7vFtM+32S2utP9kt2h92MpkMXm2lMW/jj/sFtcnXGB14Pb4tWApq8fO WXfZPTav0PJYvOclk8e8k4EefVtWMQawRbnZZKQmpqQWKaTmJeenZOal2yqFhrjpWigp5CXm ptoqRej6hgQpKZQl5pQCeUYGaMDBOcA9WEnfLsEto+edYsFc1Yptk3oZGxj/ynYxcnJICJhI PL18gB3CFpO4cG89WxcjF4eQwBJGiUUbnjBDOC8YJSZsPcAEUiUsEC6x+8JCRhBbREBNYsbJ ZYwQRX0sEneWdLOAOMwCu5kknpxYwQpSxSagI3Fq9RGwDl4BO4njz3uAxnJwsAioSvzplwEJ iwpESNx62MECUSIo8WPyPTCbU8BeYkXfWlaQcmYBdYkpU3JBwswC8hKb17xlnsAoMAtJxyyE qllIqhYwMq9iFEktLc5Nzy020itOzC0uzUvXS87P3cQIjMFtx35u2cHY9S74EKMAB6MSD+8G ZqEIIdbEsuLK3EOMEhzMSiK85c+BQrwpiZVVqUX58UWlOanFhxhNgT6YyCwlmpwPTA95JfGG JobmloZGxhYW5kZGSuK8Uz9cCRcSSE8sSc1OTS1ILYLpY+LglGpgvKR/KWX5avXO1M+JFx8s YXLucZqRcCdzqh3v001cjy4+cr5md4DXeeLlKP1PC9cr2LN/F1DNblqWHbI74lyM48qdga0z j1/6rrfEy/hxp+2Hvi8GEzT2t7/K3Gi5//aJLT/tpV2FZvv85GJQq2vfw7ijWu5O4Yy4zEsh 5V1mE0L3B768/WxarRJLcUaioRZzUXEiAIoE5M3XAgAA X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20161028061419eucas1p19d43234accc80b0e28efefbfbf6b22dd X-Msg-Generator: CA X-Sender-IP: 182.198.249.180 X-Local-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G+yCvOyEseyghOyekBtFbmdpbmVlcg==?= X-Global-Sender: =?UTF-8?B?SWx5YSBNYXhpbWV0cxtTUlItVmlydHVhbGl6YXRpb24gTGFi?= =?UTF-8?B?G1NhbXN1bmcgRWxlY3Ryb25pY3MbRW5naW5lZXI=?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20161018122856eucas1p22493fbd48031126de6093fd312e7a0fc X-RootMTR: 20161018122856eucas1p22493fbd48031126de6093fd312e7a0fc References: <1473251290-22053-1-git-send-email-i.maximets@samsung.com> <03e11be9-45b2-e05f-99f1-8be1bcc05f19@samsung.com> Cc: Dyasly Sergey , stable@dpdk.org, dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH] Revert "bonding: use existing enslaved device queues" X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Oct 2016 06:14:22 -0000 On 25.10.2016 09:26, Ilya Maximets wrote: > On 24.10.2016 17:54, Jan Blunck wrote: >> On Wed, Oct 19, 2016 at 5:47 AM, Ilya Maximets wrote: >>> On 18.10.2016 18:19, Jan Blunck wrote: >>>> On Tue, Oct 18, 2016 at 2:49 PM, Ilya Maximets wrote: >>>>> On 18.10.2016 15:28, Jan Blunck wrote: >>>>>> If the application already configured queues the PMD should not >>>>>> silently claim ownership and reset them. >>>>>> >>>>>> What exactly is the problem when changing MTU? This works fine from >>>>>> what I can tell. >>>>> >>>>> Following scenario leads to APP PANIC: >>>>> >>>>> 1. mempool_1 = rte_mempool_create() >>>>> 2. rte_eth_rx_queue_setup(bond0, ..., mempool_1); >>>>> 3. rte_eth_dev_start(bond0); >>>>> 4. mempool_2 = rte_mempool_create(); >>>>> 5. rte_eth_dev_stop(bond0); >>>>> 6. rte_eth_rx_queue_setup(bond0, ..., mempool_2); >>>>> 7. rte_eth_dev_start(bond0); >>>>> * RX queues still use 'mempool_1' because reconfiguration doesn't affect them. * >>>>> 8. rte_mempool_free(mempool_1); >>>>> 9. On any rx operation we'll get PANIC because of using freed 'mempool_1': >>>>> PANIC in rte_mempool_get_ops(): >>>>> assert "(ops_index >= 0) && (ops_index < RTE_MEMPOOL_MAX_OPS_IDX)" failed >>>>> >>>>> You may just start OVS 2.6 with DPDK bonding device and attempt to change MTU via 'mtu_request'. >>>>> Bug is easily reproducible. >>>>> >>>> >>>> I see. I'm not 100% that this is expected to work without leaking the >>>> driver's queues though. The driver is allowed to do allocations in >>>> its rx_queue_setup() function that are being freed via >>>> rx_queue_release() later. But rx_queue_release() is only called if you >>>> reconfigure the >>>> device with 0 queues. > > It's not true. Drivers usually calls 'rx_queue_release()' inside > 'rx_queue_setup()' function while reallocating the already allocated > queue. (See ixgbe driver for example). Also all HW queues are > usually destroyed inside 'eth_dev_stop()' and reallocated in > 'eth_dev_start()' or '{rx,tx}_queue_setup()'. > So, there is no leaks at all. > >>>> From what I understand there is no other way to >>>> reconfigure a device to use another mempool. >>>> >>>> But ... even that wouldn't work with the bonding driver right now: the >>>> bonding master only configures the slaves during startup. I can put >>>> that on my todo list. > > No, bonding master reconfigures new slaves in 'rte_eth_bond_slave_add()' > if needed. > >>>> Coming back to your original problem: changing the MTU for the bond >>>> does work through rte_eth_dev_set_mtu() for slaves supporting that. In >>>> any other case you could (re-)configure rxmode.max_rx_pkt_len (and >>>> jumbo_frame / enable_scatter accordingly). This does work without a >>>> call to rte_eth_rx_queue_setup(). >>> >>> Thanks for suggestion, but using of rte_eth_dev_set_mtu() without >>> reconfiguration will require to have mempools with huge mbufs (9KB) >>> for all ports from the start. This is unacceptable because leads to >>> significant performance regressions because of fast cache exhausting. >>> Also this will require big work to rewrite OVS reconfiguration code >>> this way. >>> Anyway, it isn't the MTU only problem. Number of rx/tx descriptors >>> also can't be changed in runtime. >>> >>> >>> I'm not fully understand what is the use case for this 'reusing' code. >>> Could you, please, describe situation where this behaviour is necessary? >> >> The device that is added to the bond was used before and therefore >> already has allocated queues. Therefore we reuse the existing queues >> of the devices instead of borrowing the queues of the bond device. If >> the slave is removed from the bond again there is no need to allocate >> the queues again. >> >> Hope that clarifies the usecase > > So, As I see, this is just an optimization that leads to differently > configured queues of same device and possible application crash if the > old configuration doesn't valid any more. > > With revert applied in your usecase while adding the device to bond > it's queues will be automatically reconfigured according to configuration > of the bond device. If the slave is removed from the bond all its' > queues will remain as configured by bond. You can reconfigure them if > needed. I guess, that in your case configuration of slave devices, > actually, matches configuration of bond device. In that case slave > will remain in the same state after removing from bond as it was > before adding. So, Jan, Declan, what do you think about this? Best regards, Ilya Maximets.