From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-wm0-f43.google.com (mail-wm0-f43.google.com [74.125.82.43]) by dpdk.org (Postfix) with ESMTP id 431331F5 for ; Tue, 23 May 2017 14:26:19 +0200 (CEST) Received: by mail-wm0-f43.google.com with SMTP id 123so18174966wmg.1 for ; Tue, 23 May 2017 05:26:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=6wind-com.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=z1CJWa9YFsr1Le2WX5C3eL8NLjNEwuhTciGryxtMh6g=; b=Pl75H7NgqpA82GbuwmRPoigXk8/BqSS/46NhDoEAZVBX9ai9yacfQ/zkCFQGH3Rdw+ SBVa7oUYKrb1Yl458O7BAqBM9/+O+t0Ns6tNzyAN2R7AEiMKWNCt4fkPucXjDruz6kXv m6xF7KxHYll57nJa8r1/537FhErDD4pys/iy24wqRxPCwpcHvxRvWvoE47hrLbW6tshf 6KzsiZwsZ4o/QUH5Tvydmopd3yGYh3NZJvTnlXQmGgLeSeifpZESpTYLXg5GIF+/afBr GJoGYq9sEKDkr5gKRgSTue6X+vJiQWAxf63jbtqAmtk+U28nvhyLxmpoqoW/pTZSdz6M rdGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=z1CJWa9YFsr1Le2WX5C3eL8NLjNEwuhTciGryxtMh6g=; b=TP+nsF6C1+2V8g7I5beiG46+5oPeQ1daWYy8c2WPibY1SSGdLBX6ftscNhDcz+ra+I qskDS2aETiYjgJwnpHFkm7wes6iWJMSYpGv4sk/0VLfsIsUuq6M/Ruv62CVltTtRLLSH eNY9ecaMGmfhLAB0vdfQQ0cZgr7e94lZfYEGSt34qNCoGRQi/JDGvdtrTAdThsQ9njBo Xwr7k7Gi288tPiZwhmdkNHGmiYrgf3SUm8J/ZDfpcXOq475hQhkjKAtdB/6Juu165NqB RzOfD2u6fTf2PBflQsFEgCmI/nOmeAq4SAFGFujEpun4ruW596auflje3SKPuzwIG4Me pkLA== X-Gm-Message-State: AODbwcAR+NwfT0thCPbSirGyUU8pIY+SM+TLa/jqLtBky3BKQxo4/Uzm 86zrKYOumj8Uriil X-Received: by 10.223.179.19 with SMTP id j19mr15255695wrd.118.1495542378779; Tue, 23 May 2017 05:26:18 -0700 (PDT) Received: from 6wind.com (host.78.145.23.62.rev.coltfrance.com. [62.23.145.78]) by smtp.gmail.com with ESMTPSA id 203sm854971wmv.18.2017.05.23.05.26.17 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 23 May 2017 05:26:17 -0700 (PDT) Date: Tue, 23 May 2017 14:26:12 +0200 From: Adrien Mazarguil To: Ferruh Yigit Cc: "Iremonger, Bernard" , "dev@dpdk.org" , "Mcnamara, John" , "Tahhan, Maryam" Message-ID: <20170523122612.GC1758@6wind.com> References: <20170420185448.19162-1-ferruh.yigit@intel.com> <20170518181203.15244-1-ferruh.yigit@intel.com> <8CEF83825BEC744B83065625E567D7C224D5F20E@IRSMSX108.ger.corp.intel.com> <7428261c-4276-1d28-63c2-3a51a583cdf9@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <7428261c-4276-1d28-63c2-3a51a583cdf9@intel.com> Subject: Re: [dpdk-dev] [RFC v2] Flow classification library X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 23 May 2017 12:26:19 -0000 On Mon, May 22, 2017 at 02:53:28PM +0100, Ferruh Yigit wrote: > On 5/19/2017 5:30 PM, Iremonger, Bernard wrote: > > Hi Ferruh, > > > >> -----Original Message----- > >> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Ferruh Yigit > >> Sent: Thursday, May 18, 2017 7:12 PM > >> To: dev@dpdk.org > >> Cc: Yigit, Ferruh ; Mcnamara, John > >> ; Tahhan, Maryam > >> > >> Subject: [dpdk-dev] [RFC v2] Flow classification library > >> > >> DPDK works with packets, but some network administration tools works > >> based on flow information. > >> > >> This library is suggested to provide helper API to convert packet based > >> information to the flow records. > >> > >> Basically the library consist of a single API that gets packets, flow definition > >> and action as parameter and provides flow stats based on action. Application > >> should call the API for all received packets. > >> > >> Library header file has more comments on how library works and provided > >> APIs. > >> > >> Packets to flow conversion will cause performance drop, that is why > >> conversion done on demand by an API call provided by this library. > >> > >> Initial implementation in mind is to provide support for IPFIX metering > >> process but library planned to be as generic as possible. And flow information > >> provided by this library is missing to implement full IPFIX features, but this is > >> planned to be initial step. > >> > >> Flows are defined using rte_flow, also measurements (actions) are provided > >> by rte_flow. To support more IPFIX measurements, the implementation may > >> require extending rte_flow addition to implementing this library. > > > > Do you know what extensions are needed to the rte_flow code? > > The extension may be required on two fields: > 1- Defining the flow > 2- Available actions > > For defining the flow, an update may not be required, specially at first > version of the library. > > But for action, there may be some updates. > > IPFIX RFC defines Metering process as [1], (in [2]). This library should > provide helper APIs to metering process. > > Currently only action can be used in rte_flow is COUNT, more actions can > be added to help "packet header capturing, timestamping, sampling, > classifying" tasks of the metering process. > > The exact list depends on the what will be implemented in this release. > > > [1] > Metering Process > > The Metering Process generates Flow Records. Inputs to the > process are packet headers, characteristics, and Packet Treatment > observed at one or more Observation Points. > > The Metering Process consists of a set of functions that includes > packet header capturing, timestamping, sampling, classifying, and > maintaining Flow Records. > > The maintenance of Flow Records may include creating new records, > updating existing ones, computing Flow statistics, deriving > further Flow properties, detecting Flow expiration, passing Flow > Records to the Exporting Process, and deleting Flow Records. > > [2] > https://tools.ietf.org/html/rfc7011 Since I did not take this into account in my previous answer [3], I now understand several of these requirements cannot be met by hardware (at least in the near future). Therefore I think it makes sense to leave IPFIX and more generally the maintenance of software data associated with flows to separate libraries, instead of adding many new rte_flow actions that can only be implemented in software. A hybrid solution as described in [3] is still needed regardless to offload flow recognition, so that only flows of interest are reprocessed in software to compute IPFIX and other data. You suggested at one point to take flow rules in addition to mbufs as input to handle that. Well, that's actually a nice approach. For this to work, rte_flow_classify would have to use opaque handles like rte_flow, provided back by the application when attempting to classify traffic. If the handle is not known (e.g. MARK is unsupported), a separate API function could take a mbuf as input and spit the related rte_flow_classify object if any. To be clear: 1. Create classifier object: classify = rte_flow_classify_create([some rte_flow pattern], [classify-specific actions list, associated resources]); 2. Create some flow rule with a MARK action to identify it uniquely. This step might fail and flow can be NULL, that's not an issue: flow = rte_flow_create([the same pattern], MARK with id 42) 3. For each received packet: /* * Attempt HW and fall back on SW for flow identification in order to * update classifier flow-related data. */ if (flow) { if (mbuf->ol_flags & PKT_RX_FDIR_ID && mbuf->hash.fdir.hi == 42) tmp_classify = classify; } else { tmp_classify = rte_flow_classify_lookup([classifier candidates], mbuf); } if (tmp_classify) rte_flow_classify_update(tmp_classify, mbuf); 4. At some point, retrieve computed data from the classifier object itself: rte_flow_classify_stats_get(classify, [output buffer(s)]) On the RX path, the MARK action can be enough to implement the above. When not supported, it could also be emulated through the "sw_fallback" bit described in [3] however if the above approach is fine, no need for that. It's a bit more complicated to benefit from rte_flow on the TX path since no MARK data can be returned. There is currently no other solution than doing it all in software anyway. [3] http://dpdk.org/ml/archives/dev/2017-May/066177.html -- Adrien Mazarguil 6WIND