From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mails.dpdk.org (mails.dpdk.org [217.70.189.124]) by inbox.dpdk.org (Postfix) with ESMTP id 7C3E5A0C47; Wed, 1 Sep 2021 18:34:29 +0200 (CEST) Received: from [217.70.189.124] (localhost [127.0.0.1]) by mails.dpdk.org (Postfix) with ESMTP id 5D92040F35; Wed, 1 Sep 2021 18:34:29 +0200 (CEST) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by mails.dpdk.org (Postfix) with ESMTP id 3CA40406A3 for ; Wed, 1 Sep 2021 18:34:28 +0200 (CEST) X-IronPort-AV: E=McAfee;i="6200,9189,10094"; a="279810612" X-IronPort-AV: E=Sophos;i="5.84,369,1620716400"; d="scan'208";a="279810612" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Sep 2021 09:34:27 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.84,369,1620716400"; d="scan'208";a="466961536" Received: from orsmsx604.amr.corp.intel.com ([10.22.229.17]) by orsmga007.jf.intel.com with ESMTP; 01 Sep 2021 09:34:27 -0700 Received: from orsmsx607.amr.corp.intel.com (10.22.229.20) by ORSMSX604.amr.corp.intel.com (10.22.229.17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12; Wed, 1 Sep 2021 09:34:26 -0700 Received: from orsmsx603.amr.corp.intel.com (10.22.229.16) by ORSMSX607.amr.corp.intel.com (10.22.229.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Wed, 1 Sep 2021 09:34:26 -0700 Received: from ORSEDG601.ED.cps.intel.com (10.7.248.6) by orsmsx603.amr.corp.intel.com (10.22.229.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.12 via Frontend Transport; Wed, 1 Sep 2021 09:34:26 -0700 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (104.47.66.45) by edgegateway.intel.com (134.134.137.102) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2242.10; Wed, 1 Sep 2021 09:34:25 -0700 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ViiT4UXgHBoL4shwZVkpWt5cr7hVGKWb9+5/7yRujdVPprDV1h+waEHlQ2N9vGMAgQt4dQIhLVP54gARf5gfcnrod3V6j1VNhtF4M9aC/dvceguXfZ2ukxkTlNyGoonu0LRdRXnGN16iqnBG0aYM9vRekmGGRtDbrXtcjnoWVnrJiJZSkM5l4yo5qL5kTDU0tGaS9R788h9YY+Pgw6v2vDFOeerxJVXOFZlklI5oXUSQ9Xa80og+MwCT703CCvZuL3buni7Q8bSzv4h+/YGEAt3+ruCsdv40tKSQeuYR9WgCKNI4zjoDUwJOX7azjU4jOwrthXGsF/hXFtT6TpWctQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJ3CwTwRsH4IbgP8XFqcwrsZsciTiU2CpuHZpTrPsw4=; b=iMxogkh8WTa3LILJiK7aSFeANa1abpmHMfwu3i54xVJloaFP7HUJAlqZnLcaSuwnj6Htrh0BnAP0AkAD/21baF0uIWyyZSjYmO3OAOQuV9egCIS4YDfVL2mkz54/UbxGv4lId6dwv4m76Oe/3n0m1YZEt4/mZ3DfkK5OD8fi4FwxZhZCjRZoTpCdw4M8rXhKCafy4S+z4MrlSlSAoKRORRyNFUr5lOWxAr8khB7sKRdQVH1oImorEDcVsn0E1YNvk4HLQ7ZupJOjJeTeMK+NfG0GB6sxUwKEFWO6OXTYlpwYKPLlmr7SIoVSFpTn32+Cdes9dhBF82DE8g+OcxDmfw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=intel.com; dmarc=pass action=none header.from=intel.com; dkim=pass header.d=intel.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=intel.onmicrosoft.com; s=selector2-intel-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=UJ3CwTwRsH4IbgP8XFqcwrsZsciTiU2CpuHZpTrPsw4=; b=a2NQ9hAlX4IAdhApfr4NHhvLchd7pPxN3G5atckaHJZsoZR3zxZEK3E322M8spn5DifcO3vPR0vjAWKpT7YREleSH/WtgleZpXBXQdTCPk0gCkpSnxcGNYV6qYlOTkPSkW4ZehnE2QT7LnSK+9r/DbO3DYvNbCkYT1KqtNxBFUA= Authentication-Results: gmail.com; dkim=none (message not signed) header.d=none;gmail.com; dmarc=none action=none header.from=intel.com; Received: from PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) by PH0PR11MB4966.namprd11.prod.outlook.com (2603:10b6:510:42::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4457.24; Wed, 1 Sep 2021 16:34:25 +0000 Received: from PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::747b:3a08:d1ec:31fc]) by PH0PR11MB5000.namprd11.prod.outlook.com ([fe80::747b:3a08:d1ec:31fc%5]) with mapi id 15.20.4478.020; Wed, 1 Sep 2021 16:34:25 +0000 To: Tudor Cornea , CC: , , Mihai Pogonaru References: <1629466761-127333-1-git-send-email-tudor.cornea@gmail.com> From: Ferruh Yigit X-User: ferruhy Message-ID: <3154f4e8-9f0e-b04f-6e8f-096dc39f489c@intel.com> Date: Wed, 1 Sep 2021 17:34:20 +0100 In-Reply-To: <1629466761-127333-1-git-send-email-tudor.cornea@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-ClientProxiedBy: DB6PR0301CA0059.eurprd03.prod.outlook.com (2603:10a6:4:54::27) To PH0PR11MB5000.namprd11.prod.outlook.com (2603:10b6:510:41::19) MIME-Version: 1.0 Received: from [192.168.0.206] (37.228.236.146) by DB6PR0301CA0059.eurprd03.prod.outlook.com (2603:10a6:4:54::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4478.17 via Frontend Transport; Wed, 1 Sep 2021 16:34:23 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1c7b75f9-f1e5-4a5d-4e2c-08d96d665c36 X-MS-TrafficTypeDiagnostic: PH0PR11MB4966: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:10000; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: aXn2zmh4WNAaVSwTrxIMuVONvPdiyEuF9RBOvnnai/0+X67kxitKHwGAXG16Z11TcSjT+XD/2FSIeQiysKFTJqHqVJspDmSNJnqfjUsilx4izWUtbdJB3fqCUWNYOs/vP2DVzTaobYbDLJYy9VPEVEbi9/PPorKYy5/2QZMQxAPcIXDEf3mU6TJ31p4M2PmjT75Ul6w5yXYZgXAJ/pZG+qUsADbrJ6b+MD3SJd4j/GWr9RLtkV70xbEU4TN72l2iF7igPYDd5BwGdckMgxJ+c7kpG23J7Xcft4B2WKGs+3CkJD0dUxve2oBdEy0ULPsg2Jeoa+Bm4dDpPyl7dHMV46kFDx7wiotW/wl9nzLthrheP1UyHhVC7o0IND4TprDzpO1YP3g15OdyEyOlyjoCsvlYDnLXFpx8J0Ot9OizaFaPpukzpWGlt6te2U+zr09ccfxDKek+gVw+VvXT4dPxYdYHMhLuJBw7s5JxR/oc26TKjJ73NbRAibdAANg5H807ncK70V39t1SOW9u8btco4xQ4sHk8fPDQSr7g65oeZeRdCZcipyJjHNLHNF44h+mMOu7qav7xLqQ4zDJ2F83VQmBLyVv7OeTVH0sdlvAM1fTHkYaxV7iklMHixDRjlAJbZ8qewSqIZGHos8nvXO8UYlWnu2s+ILIfSDs9+/rkT+6xA+qdV0jK+T6AbKBNG8wvNZjw0aGRcIpOAEUYJAP5Jg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:PH0PR11MB5000.namprd11.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(366004)(136003)(39860400002)(346002)(396003)(376002)(44832011)(66556008)(66476007)(2906002)(26005)(5660300002)(36756003)(956004)(31686004)(83380400001)(6486002)(38100700002)(31696002)(316002)(86362001)(4326008)(186003)(2616005)(66946007)(478600001)(8676002)(8936002)(53546011)(16576012)(45980500001); DIR:OUT; SFP:1102; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?aUxKeTJIOWkvTUd4NWxxSU1UUzRiZWgrVWVaRStFTW0zQjB3bjJIdW5JTmRM?= =?utf-8?B?WUNZa1dQQVE3MmFJeE1GYXZRV0VHOTBBcXJQZWZudXVub3lGRmlJcG05SW1o?= =?utf-8?B?djFoWlFycVFvSUFsSWs3R0pYdmRXeDUwUHJuR1ZEUVFRZ3ZOWTdVMmNpZkZp?= =?utf-8?B?UVc1cG0yV2pkcVE3N1NzSEYrazRXWFlESU9kTzBmeXZpNk92Rk8remVMUVN5?= =?utf-8?B?WTNZL1g0anh6SVdTR1VzS1B3NUlyaHdpdjlEVVB5N281SEhlUlhOeVR2U1dh?= =?utf-8?B?eG9WK3o3TjFDdUVzaGNWUmtVR1ZWUytkdWd6WlBISjk5UHhTYkpnblczWGFi?= =?utf-8?B?clRGUVJZNHI0aEFYN2hIQ1A1alJqb2dUZUQ2UVIzd2YzNFJkVldEbUgyekFy?= =?utf-8?B?VTJ1NktJRC96WVFlKzIrcTBpdTJUcndRekQ5K3NYVThYaXpNL0lacnFnYmdt?= =?utf-8?B?aWsxYWxhTTJLbjM0SGFTZ1c1TmlyN2pnSnIwWUVkcnBZY1dQNndSai81dURz?= =?utf-8?B?UWNFRTkyczM3YW5PSnZZTDZvQmpqbkVqa1NYWmFLSXhXK016MUxCcUMvOFV2?= =?utf-8?B?RFJGZzM4TWwwMHhodDRtejB1MXJNUG54RTNIRXlXQjlCRVQwcG5vWWRCVTd2?= =?utf-8?B?VzJMZVpRbWQrRVorSUxnSUV0ak9RL3podnpCUDZFL0gvaDlVVEVvaEYvNEVQ?= =?utf-8?B?SmZMSkF3NlJQTzIrTG9SdEIyeXp1bTZnRFlQNG12cVhhTHlwM1c4UmFUN2JE?= =?utf-8?B?QVFnbHRhWTJQUUozYnBuSU82QUFwSDBxQW1kZk05SUZvQzZQa3M0bVN0TVd0?= =?utf-8?B?b2xkMDFwUlJFVkNja1NwMzBhQ0NQRWRXSGtLL1dGRC82aFhtaDc1Q2Q0QU9I?= =?utf-8?B?UlJJekN1S1YxK2pHeW5UZDNCb3JpV0Vvc05kSjU2NlMrdTg1b014Wll1bE1J?= =?utf-8?B?bm55ZEd4dXJ2Z3c1VGJMZk0xTklKVEg5dFF5SGp1Mkc2MFpxMVF1SzZEdnZB?= =?utf-8?B?ZStBRHM0cDRDV0VoRUt5aUlvZ050SUU1a2c3UkF1LzdDUStNTGpZd2tETkpj?= =?utf-8?B?cE90N0hZYi9FKzFTY081ZFdZdklXL2tWK2dmVUZIbUFnUVFrSEhpenpvK2gr?= =?utf-8?B?bm5lSlJYeS9sSlJhSERhWVFBYW92RkxkUGg3OEd6d0V1aTR3RFpKNHozZC94?= =?utf-8?B?d3N4ZVc2NEVnM2JSWXpZUnZJZWl0YXJ4Nm1neGdGaDZzbTJDVCs0b3IySUZG?= =?utf-8?B?THVYZzhKYWlJbW8vbFhCQjJSdFBpS0R1T1J3VzdLd3VxNmF2RFZzMjZwRlJw?= =?utf-8?B?dFdoTGJTdTBJOWFEK3ZWdHl6MEEwL2dSemZKcWEvTzFIMmMreWhyaFBhNDJn?= =?utf-8?B?Z2ZCdXlqTHJUN2RYa0h2UUhxVlVlUlN0LzdtNlFxbjQrVE8xaXEyb2xIMGtx?= =?utf-8?B?eGs0ZFdab1pGTldGSmRDRTVuUjBNZ09NajA2cjNpVThjdVdHa3NJTnIzT1JE?= =?utf-8?B?aWxRbEVwam5xR1lNc1c2dFRyc09tM0hpSkc4L2RiRHdTVHJtU2pURjF4amFG?= =?utf-8?B?VHlHTnlLUXAwSTlWQ2Ryb2dKZXIzclphMVhkVnRhaVlwRDRyK3cxNFNuMzNO?= =?utf-8?B?blh5ZEJmVjdVajFlQ0dsZXcwRHlNMlJUQmNmZHJ2ck82cVhlQllPUDB0T1NU?= =?utf-8?B?WU5jT3hyZTRVQ0VseWhLSjM1eXkwYnRMNmZBbkVPSHlLeHFpUWNXVE9xNFJD?= =?utf-8?Q?rP400/DFQpLiOnFcEvYLDFAnf3L4DaClQUYXq4e?= X-MS-Exchange-CrossTenant-Network-Message-Id: 1c7b75f9-f1e5-4a5d-4e2c-08d96d665c36 X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5000.namprd11.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Sep 2021 16:34:25.2940 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xM5Z8GrhT46e4TcS7Vrrc4mU8IBMHFwp9HohaDN1gQH0B2NwGVSGgKt6+Cd8z7HtyzoH4Tti3EWpuNIhOoBt4A== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4966 X-OriginatorOrg: intel.com Subject: Re: [dpdk-dev] [PATCH] net/af_packet: fix ignoring full ring on tx X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dev-bounces@dpdk.org Sender: "dev" On 8/20/2021 2:39 PM, Tudor Cornea wrote: > The poll call can return POLLERR which is ignored, or it can return > POLLOUT, even if there are no free frames in the mmap-ed area. > > We can account for both of these cases by re-checking if the next > frame is empty before writing into it. > > We also now eliminate the timestamp status from the frame status. > Hi Tudor, Would you mind separate timestamp status fix to its own patch? I think better to fix 'ignoring full Tx ring' first, to not make it dependent to timestamp patch. > Signed-off-by: Mihai Pogonaru > Signed-off-by: Tudor Cornea > --- > drivers/net/af_packet/rte_eth_af_packet.c | 47 +++++++++++++++++++++++++++++-- > 1 file changed, 45 insertions(+), 2 deletions(-) > > diff --git a/drivers/net/af_packet/rte_eth_af_packet.c b/drivers/net/af_packet/rte_eth_af_packet.c > index b73b211..3845df5 100644 > --- a/drivers/net/af_packet/rte_eth_af_packet.c > +++ b/drivers/net/af_packet/rte_eth_af_packet.c > @@ -167,6 +167,12 @@ eth_af_packet_rx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > return num_rx; > } > > +static inline __u32 tx_ring_status_remove_ts(volatile __u32 *tp_status) > +{ > + return *tp_status & > + ~(TP_STATUS_TS_SOFTWARE | TP_STATUS_TS_RAW_HARDWARE); I can see 'TP_STATUS_TS_SYS_HARDWARE' is deprecated, and I assume in the kernel versions the bug exists, this flag is not set, but can you please confirm? > +} > + > /* > * Callback to handle sending packets through a real NIC. > */ > @@ -211,9 +217,41 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > } > } > > + /* > + * We must eliminate the timestamp status from the packet > + * status. This should only matter if timestamping is enabled > + * on the socket, but there is a BUG in the kernel which is > + * fixed in newer releases. > + > + * For interfaces of type 'veth', the sent skb is forwarded > + * to the peer and back into the network stack which timestamps > + * it on the RX path if timestamping is enabled globally > + * (which happens if any socket enables timestamping). > + > + * When the skb is destructed, tpacket_destruct_skb() is called > + * and it calls __packet_set_timestamp() which doesn't check > + * the flags on the socket and returns the timestamp if it is > + * set in the skb (and for veth it is, as mentioned above). > + */ > + Can you give some more details on this bug, any link etc.. And does it only seen with veth, if so I wonder if we can ignore it, not sure how common to use af_packet PMD over veth interface, do you have this usecase? And if only specific kernel versions impacted from this bug, what do you think about adding kernel version check to reduce the scope of the fix in af_packet PMD. > /* point at the next incoming frame */ > - if ((ppd->tp_status != TP_STATUS_AVAILABLE) && > - (poll(&pfd, 1, -1) < 0)) > + if ((tx_ring_status_remove_ts(&ppd->tp_status) > + != TP_STATUS_AVAILABLE) && (poll(&pfd, 1, -1) < 0)) > + break; > + > + /* > + * Poll can return POLLERR if the interface is down or POLLOUT, > + * even if there are no extra buffers available. > + * This happens, because packet_poll() calls datagram_poll() > + * which checks the space left in the socket buffer and in the > + * case of packet_mmap the default socket buffer length > + * doesn't match the requested size for the tx_ring so there > + * is always space left in socket buffer, which doesn't seem > + * to be correlated to the requested size for the tx_ring > + * in packet_mmap. > + */ > + if (tx_ring_status_remove_ts(&ppd->tp_status) > + != TP_STATUS_AVAILABLE) > break; In this case should we break or poll again? If 'POLLERR' is received when interface is down, it makes sense to break. Do you know if is there any other case that 'POLLERR' is returned? And for 'POLLOUT', when exactly this event sent? If 'POLLOUT' received, can we poll again to wait Tx ring available to send more packets? > > /* copy the tx frame data */ > @@ -242,6 +280,11 @@ eth_af_packet_tx(void *queue, struct rte_mbuf **bufs, uint16_t nb_pkts) > rte_pktmbuf_free(mbuf); > } > > + /* > + * We might have to ignore a few more errnos here since the packets > + * remain in the mmap-ed queue and will be sent later, presumably. > + */ > + Can you please describe above comment more? What errors ignored? And won't all packets sent will below sendto() call? > /* kick-off transmits */ > if (sendto(pkt_q->sockfd, NULL, 0, MSG_DONTWAIT, NULL, 0) == -1 && > errno != ENOBUFS && errno != EAGAIN) { >