From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from NAM03-DM3-obe.outbound.protection.outlook.com (mail-dm3nam03on0076.outbound.protection.outlook.com [104.47.41.76]) by dpdk.org (Postfix) with ESMTP id 35B4C1B20D for ; Wed, 10 Jan 2018 11:26:27 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=CAVIUMNETWORKS.onmicrosoft.com; s=selector1-cavium-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=MppIKk/5gTwp9CBXK6GNY/02KLuxkEtfBtts2G511PA=; b=WItE7KMsP5oGHOeQuqMFHlIx1wKpDDCltiMLObtWKc/To7MFZW4g32wExVnL2VWrHAX8t3Swhe6QGa0xXNN/7h40womkmC9hA4ozASiPq1ypN3JymPuddJD/mfFE3CRjl/O3Suv8a9fFwnSAN+o1+nNgVLpxWf3sY5xOSODEU2Y= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=Pavan.Bhagavatula@cavium.com; Received: from Pavan-LT (111.93.218.67) by BN6PR07MB3457.namprd07.prod.outlook.com (10.161.153.20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.386.5; Wed, 10 Jan 2018 10:26:22 +0000 Date: Wed, 10 Jan 2018 15:56:16 +0530 From: Pavan Nikhilesh To: Gage Eads , jerin.jacob@caviumnetworks.com, harry.van.haaren@intel.com, santosh.shukla@caviumnetworks.com Cc: dev@dpdk.org Message-ID: <20180110102615.42jh4hkprgombzki@Pavan-LT> References: <1515514775-10859-1-git-send-email-gage.eads@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1515514775-10859-1-git-send-email-gage.eads@intel.com> User-Agent: NeoMutt/20170609 (1.8.3) X-Originating-IP: [111.93.218.67] X-ClientProxiedBy: MWHPR15CA0054.namprd15.prod.outlook.com (10.174.254.16) To BN6PR07MB3457.namprd07.prod.outlook.com (10.161.153.20) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4b2fb26f-73b6-4207-f5fa-08d558149a2e X-Microsoft-Antispam: UriScan:; BCL:0; PCL:0; RULEID:(5600026)(4604075)(4534020)(4602075)(4627115)(201703031133081)(201702281549075)(2017052603307)(7153060)(7193020); SRVR:BN6PR07MB3457; X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3457; 3:0rR1c+CapBqJWYIaHcXy+33KXlbgdZUFjiWgVhXSr7unZEeftmbEy6dtUhEjMyxHOtCjGpOmst6IJrSv+0dWsH/g1WFdoJUVETy26whHTPjB940vN+KPKVN8ykhPl3swL5RHD8iq0/ymP0qPbXkPELp8gJfKdNoJuBQTiNBXCYv0jKP6HuxrMbVT8Df3/GA0kviafNEsYl1Bhpw+voB8FBWhKsD9jFtfHYf7o/namOa/xbYUDYQKi9SwSeT3OwNQ; 25:Grg/Fi3R7/+sOZ8zEwqoWNOzTpxSykwP1ZTsjIcVbz7yhzWdLlNUCthGdNC4PY9m3WYxg94a6DSnH4Jdn8KH5BGfX7xNe2S2RCIF4NoM8yEqecUdbr5WL6yamhQGB5NChTCbCMK3MaW10hCg7rU7pwQabfHkBdqBhQc91aLdRUQPd/kgQI5xXgznHk2i91Xax6Yv1QI7HEJwamF4VNBC3q/02KIdy34ODxjpAUNCu2qXzQ5S4FDKNzdRbVb00Ok/Ab3Nn/5O2f8cuJ44BF9vzG8W8TrPLJ9iIvA1AeMWvO+aFyht/TbyTKxctHZxNoXPOmHviNjAJh6PqZGnHLWFJw==; 31:+oGjn/zRL0AbLQb376bFd30CNZCiQlJUfWv05xYzjgEUfbBmS8nNq4H8oBwS69qIHQrZv+M8zwhZGyRJHzkwn/FLfMJ/Jrh0zH6seikO9X7b15aT/hAKbbifORfYpNDvhih6uaXqhbJIdhH1eIcv1xQbQ1dLMrhpQ76ejdgwPhKagAPulsx/KmvbIx8G/FiAKIvFGMVIqV4/P6uNz2Nw2OyJ06zrwyiJW80wSVJ4J6o= X-MS-TrafficTypeDiagnostic: BN6PR07MB3457: X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3457; 20:PVS+ZXR/RQlnITuOAYGkQBX9avnUdvTWle6t+f2Pzprv2W3RaSKPmZDVIUBswR9RPAJ3B+iwmQa4BbhLOS/tq7jj3VPGqedImcr7/2OjQmGl3RkqSbZ546TwzwXknu25gKsNpbufdbWbTQkWRAz4mPUeWpDFsrzHGpE0DkFAB1BwIe3alEh2Ht1G8z6fnZ+1uMeXPObKSzzUBo4JAViNnk+xChA8juYLXPl2iRe3xR0S3tdHZpJVD02ni/rxXscn5t9+GCXPb1jlle40X/YBgdXwqV0eAtLXjLkP2fly0tN5PmwoceP8npNErQbo47zxQW7vTsgmBwPXiCFof8+aX3/uVoL4acnpAQflVW5/V+odwyeN5tSpQwharqa2R58zVXIZtXor5Z191rUlJrvpYKzRChNrsf8BK2T16nqEPb4YBx/y5y2tm2Fr+cW4ywLvuN6I75A3xNm7jM+uA0+3XpJ5L3Rh5Y/7R/JTRvjMZJI9EhHPjSxDqUcABz23KBbcm6CxQu3EY8slomSO3YmqwFS9d5xv8heid1es3xef17AnZF6c9HVflbQtM2hTu2MkY9jMmy3sL992OCV6623gakxnqNOj0XxtB61XoXnwV+k=; 4:PXb3gbDk8eaxqyTSk3srgRkAy66yvDx21Cs6SRi9o4LjN0T7qbf3U7CzfUTONGZWu/KRr2gLu9bd4TEw7CdM4K76ePh+1QkuWQhokgRpBHk/24ek57lopWJ7i0e0yrVY9Kod2n2eEpXVzKGxV31iLfwGRH7ngQg8EtPdT2KX1m8R/rSekYqbJ5ft+pQjaEzQT8QO3RcnpzxMZuKu+k2xwFxVz68PeExtoWGYOnWVa0YUTUI3xspS1+6kojzcIsCB8Wk1pahcZGkgL/dprwFFv04FxtEQI9RmqC2Hspd12lKc5ya1SiDL2jiF/y9BOEHC X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(228905959029699); X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3002001)(93006095)(10201501046)(3231023)(944501075)(6041268)(20161123560045)(20161123564045)(20161123558120)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(6072148)(201708071742011); SRVR:BN6PR07MB3457; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:BN6PR07MB3457; X-Forefront-PRVS: 0548586081 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10009020)(39380400002)(346002)(366004)(396003)(376002)(39860400002)(24454002)(189003)(199004)(6116002)(478600001)(4326008)(68736007)(316002)(16526018)(23726003)(47776003)(386003)(72206003)(3846002)(83506002)(106356001)(1076002)(55016002)(105586002)(53936002)(16586007)(33896004)(76176011)(97736004)(9686003)(25786009)(229853002)(58126008)(6246003)(7736002)(81166006)(50466002)(305945005)(6496006)(8936002)(52116002)(81156014)(66066001)(2906002)(6666003)(8676002)(6636002)(5009440100003)(42882006)(5660300001)(2950100002)(33716001)(107986001)(42262002); DIR:OUT; SFP:1101; SCL:1; SRVR:BN6PR07MB3457; H:Pavan-LT; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: cavium.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; BN6PR07MB3457; 23:10/8wn4Wo5/w131IQB5rO/2+bGOhp/5h6Lm4SWAog?= =?us-ascii?Q?q7AnCtAxCxC1zjc35LRGiyfKsdo2P0frLkDOjf2sAEpmY2GKzztpb+75315S?= =?us-ascii?Q?HuvizbNx0WqIO8N6TXlDtLBZE1mnZcgo1LDnU9suYtTkTta7KhazNvietkgr?= =?us-ascii?Q?crnrPkwMvvAmB7dV/xUeYqauYi3gWWqoXo+R4IJ2gWfIe9+TWmfd4FFD0SXC?= =?us-ascii?Q?4Edi86YPODievVeH4huqlyN+D0fl0PdFQ9BqYJwpDd2/Xo2ZwxT6k52cekLi?= =?us-ascii?Q?LsDeZVu6yRHp20zOzpaqXzAzppjdtofoZOTr5heebcofT60uxKdYBMH//3nt?= =?us-ascii?Q?mG2zV2GMpzOyaa+ptoI8VsPGhtPDHae/Ko2RQ5+bn/1ZDXdxfZdVaJdmxcI4?= =?us-ascii?Q?9CHXPLzzVMka0TL5LRJtxkwdGr+i2jfOZVeVkB0BfykyYRq/v5ShYDwdN7tX?= =?us-ascii?Q?nd+mUeEwZgRYHd7fI7dv1eiSsJKTC/m4r8kwmmr+M9R9rDcpzitQYcn2SqGG?= =?us-ascii?Q?caOtmPEnO9e2tkbbE9foXpf8IkSxqurkBVRJODlGtRPRT1e/44R1PF6KpZDC?= =?us-ascii?Q?3ZnQP+RibsoPIVhD2inYl5vJW3akDSnEBUW7va15M2U6UnHJRfO3gT29xnsU?= =?us-ascii?Q?KgNu8g5ZsqEDNkW3HvK6N042I0Y5SEvAOMt34QtfrfWyxHOTcd/YAYtwir5q?= =?us-ascii?Q?wJ2p6/uk6EUv6s8eikinFNynQZU1LTlk4FIhoZdJHVYN28lr5qrdALuwZa6q?= =?us-ascii?Q?srdNGWRWTHTR3nnvHVUjZlgoXqWxK6wq3SP8MB94sAG9ZsLfOjBdvaqdMQza?= =?us-ascii?Q?wwfKEwPU11aUtKLesUsql3b13UsZQConNWQkzRBgjVqYinr8mHoYtDRtcM4d?= =?us-ascii?Q?2P8IFfNbF02P6/s2rtCs9ULPHXaNV6JLcjLEMZgDpFZ/T5uF5gD6GGsxu3vP?= =?us-ascii?Q?+hATYthKTUuijPEQF70Oj0bzqManApNOqhkgP5lD8YsbbRVnlo1oM6tEMjDc?= =?us-ascii?Q?Xhqe7/faoZFcktWlX2mm6UKRc9FUGiKqQ5CIV8M9yE6+59NfFc4k6Q/KEg7j?= =?us-ascii?Q?nGhZiZn/CRq9C7yM2d6d7onhhNnwFXRr27UCUXyc24JmMrPq9ckyxJWZghOf?= =?us-ascii?Q?c9vtAQru1l0thVz1CnV1CazZTmSpYUcRMxwOc6NfYT5b3OO9D4XWR0ljZifd?= =?us-ascii?Q?kGgB/Kd9diCduG9bMajM73BCT/4zQ+cgkv0FZt1Fl67f76rzr0cFk8KT3TS7?= =?us-ascii?Q?GGArSKsNMHfe0NIViSxGLmSHEsiTFRoh27IwaSk?= X-Microsoft-Exchange-Diagnostics: 1; BN6PR07MB3457; 6:IpXQGrzgJ/YbQe5grHwovSRkm4COTHMyY4LpJQ5Uz8Gc+eQLIjqOmVFjX9VzYD9Fi6mpo4Jg7OMG6eFAuVJpA7EXYJOjHC75pgPY5vjfHvxB9ltBr4c4RrPYaLrMdzn2Pw7cN3j7nXiSGVz/4pM1+W0fdnxvwdzp4hGcDnQEX/A12cGViG1cnqVBS+y7kms/Av2FormGmfEEvaQW0cIL9mtRVjZQy0tzLWALBBzHaQuptLi7dZP7rlRvQ5mBpwktPQScsyuvmg9OTGrI63hgwvuGWHR0N6MspaG2UdK/LwC8phrgLAAlE+D0ZHrfVFOw5NLWJfOmJOrH/B0LrPWwlQ==; 5:JWKk2LR3c1x/N3NpmTGgzhiDNwA+j7ok4TduOBxJfIdNv5yGm2Lx7imX64scEs89yFfsLbTR3/Ao1pWov9ZdNv1Qs+wATB/2hBwEpPJVi+4fGr7AT2jqDqBJH2mf9wae0hLxfImkA27CAHP3StT+uKtGEIC0LoKv7oDynUkc4sI=; 24:FYLU+2N5r6/p6DkGfHwz0iiidbkPDTqKbzgSB4YUAMZEOYEEBHi6eAiVnTl6nQRneyDNz0qeQvWJL+dvYP44gVhHw7xKRPij23Ic+4GwdsQ=; 7:+bFgpNbnAQrKgnVUG9i4sNTy32KqYQyNdSlO2w1ewF8WAjBOZXtirX14Lakrr+Gi/ZJK9cb/xBe/ovFyg93yKRDPLpTmzQCaa0Gs08DRfiyLzhzsaFr2GWr/j5fXbcjx05qIbi0WvJ70beUIEIapsdv4/t3PqVMeHZHHig6s4EEPipsOE55mEIrtulwaiAd5CyYWx4nHZeOLc5m3WGabvY//a/bG9EhDKr2mesZ9F1s8soYIL8y367OSu/FHPttP SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: caviumnetworks.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jan 2018 10:26:22.9165 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4b2fb26f-73b6-4207-f5fa-08d558149a2e X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 711e4ccf-2e9b-4bcf-a551-4094005b6194 X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR07MB3457 Subject: Re: [dpdk-dev] [PATCH] event/sw: remove stale IQ references when reconfigured X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Jan 2018 10:26:27 -0000 On Tue, Jan 09, 2018 at 10:19:35AM -0600, Gage Eads wrote: > This commit fixes a bug in which, when the sw PMD is reconfigured, it would > leave stale IQ chunk pointers in each queue's IQ structure. Now, the PMD > initializes all IQs at eventdev start time and releases all IQ chunk > pointers at eventdev stop time (which has the consequence that any events > in a queue when the eventdev is stopped will be lost). This approach should > be resilient to any reconfiguration done between the stop and start, such > as adding or removing queues. > > This commit also fixes two potential issues in iq_chunk.h. iq_init() > now initializes the IQ's count field to 0, and iq_dequeue_burst() sets > iq->head to the appropriate next pointer. > > Fixes: 5b5e476e59a4 ("event/sw: use dynamically-sized IQs") > Signed-off-by: Gage Eads > --- > drivers/event/sw/iq_chunk.h | 14 +++++++++- > drivers/event/sw/sw_evdev.c | 62 ++++++++++++++++++++++++++++++++------------- > 2 files changed, 57 insertions(+), 19 deletions(-) > > diff --git a/drivers/event/sw/iq_chunk.h b/drivers/event/sw/iq_chunk.h > index 29f5a35..2fa724c 100644 > --- a/drivers/event/sw/iq_chunk.h > +++ b/drivers/event/sw/iq_chunk.h > @@ -73,12 +73,24 @@ iq_free_chunk(struct sw_evdev *sw, struct sw_queue_chunk *chunk) > } > > static __rte_always_inline void > +iq_free_chunk_list(struct sw_evdev *sw, struct sw_queue_chunk *head) > +{ > + while (head) { > + struct sw_queue_chunk *next; > + next = head->next; > + iq_free_chunk(sw, head); > + head = next; > + } > +} > + > +static __rte_always_inline void > iq_init(struct sw_evdev *sw, struct sw_iq *iq) > { > iq->head = iq_alloc_chunk(sw); > iq->tail = iq->head; > iq->head_idx = 0; > iq->tail_idx = 0; > + iq->count = 0; > } > > static __rte_always_inline void > @@ -154,7 +166,7 @@ iq_dequeue_burst(struct sw_evdev *sw, > > done: > if (unlikely(index == SW_EVS_PER_Q_CHUNK)) { > - struct sw_queue_chunk *next = iq->head->next; > + struct sw_queue_chunk *next = current->next; > iq_free_chunk(sw, current); > iq->head = next; > iq->head_idx = 0; > diff --git a/drivers/event/sw/sw_evdev.c b/drivers/event/sw/sw_evdev.c > index 1ef6340..7430a5d 100644 > --- a/drivers/event/sw/sw_evdev.c > +++ b/drivers/event/sw/sw_evdev.c > @@ -245,9 +245,6 @@ qid_init(struct sw_evdev *sw, unsigned int idx, int type, > char buf[IQ_ROB_NAMESIZE]; > struct sw_qid *qid = &sw->qids[idx]; > > - for (i = 0; i < SW_IQS_MAX; i++) > - iq_init(sw, &qid->iq[i]); > - > /* Initialize the FID structures to no pinning (-1), and zero packets */ > const struct sw_fid_t fid = {.cq = -1, .pcount = 0}; > for (i = 0; i < RTE_DIM(qid->fids); i++) > @@ -325,11 +322,6 @@ qid_init(struct sw_evdev *sw, unsigned int idx, int type, > return 0; > > cleanup: > - for (i = 0; i < SW_IQS_MAX; i++) { > - if (qid->iq[i].head) > - iq_free_chunk(sw, qid->iq[i].head); > - } > - > if (qid->reorder_buffer) { > rte_free(qid->reorder_buffer); > qid->reorder_buffer = NULL; > @@ -348,13 +340,6 @@ sw_queue_release(struct rte_eventdev *dev, uint8_t id) > { > struct sw_evdev *sw = sw_pmd_priv(dev); > struct sw_qid *qid = &sw->qids[id]; > - uint32_t i; > - > - for (i = 0; i < SW_IQS_MAX; i++) { > - if (!qid->iq[i].head) > - continue; > - iq_free_chunk(sw, qid->iq[i].head); > - } > > if (qid->type == RTE_SCHED_TYPE_ORDERED) { > rte_free(qid->reorder_buffer); > @@ -388,6 +373,41 @@ sw_queue_setup(struct rte_eventdev *dev, uint8_t queue_id, > } > > static void > +sw_init_qid_iqs(struct sw_evdev *sw) > +{ > + int i, j; > + > + /* Initialize the IQ memory of all configured qids */ > + for (i = 0; i < RTE_EVENT_MAX_QUEUES_PER_DEV; i++) { > + struct sw_qid *qid = &sw->qids[i]; > + > + if (!qid->initialized) > + continue; > + > + for (j = 0; j < SW_IQS_MAX; j++) > + iq_init(sw, &qid->iq[j]); > + } > +} > + > +static void > +sw_clean_qid_iqs(struct sw_evdev *sw) > +{ > + int i, j; > + > + /* Release the IQ memory of all configured qids */ > + for (i = 0; i < RTE_EVENT_MAX_QUEUES_PER_DEV; i++) { > + struct sw_qid *qid = &sw->qids[i]; > + > + for (j = 0; j < SW_IQS_MAX; j++) { > + if (!qid->iq[j].head) > + continue; > + iq_free_chunk_list(sw, qid->iq[j].head); > + qid->iq[j].head = NULL; > + } > + } > +} > + > +static void > sw_queue_def_conf(struct rte_eventdev *dev, uint8_t queue_id, > struct rte_event_queue_conf *conf) > { > @@ -434,7 +454,10 @@ sw_dev_configure(const struct rte_eventdev *dev) > num_chunks = ((SW_INFLIGHT_EVENTS_TOTAL/SW_EVS_PER_Q_CHUNK)+1) + > sw->qid_count*SW_IQS_MAX*2; > > - /* If this is a reconfiguration, free the previous IQ allocation */ > + /* If this is a reconfiguration, free the previous IQ allocation. All > + * IQ chunk references were cleaned out of the QIDs in sw_stop(), and > + * will be reinitialized in sw_start(). > + */ > if (sw->chunks) > rte_free(sw->chunks); > > @@ -667,8 +690,8 @@ sw_start(struct rte_eventdev *dev) > > /* check all queues are configured and mapped to ports*/ > for (i = 0; i < sw->qid_count; i++) > - if (sw->qids[i].iq[0].head == NULL || > - sw->qids[i].cq_num_mapped_cqs == 0) { > + if (!sw->qids[i].initialized || > + sw->qids[i].cq_num_mapped_cqs == 0) { > SW_LOG_ERR("Queue %d not configured\n", i); > return -ENOLINK; > } > @@ -689,6 +712,8 @@ sw_start(struct rte_eventdev *dev) > } > } > > + sw_init_qid_iqs(sw); > + > if (sw_xstats_init(sw) < 0) > return -EINVAL; > > @@ -702,6 +727,7 @@ static void > sw_stop(struct rte_eventdev *dev) > { > struct sw_evdev *sw = sw_pmd_priv(dev); > + sw_clean_qid_iqs(sw); > sw_xstats_uninit(sw); > sw->started = 0; > rte_smp_wmb(); > -- > 2.7.4 > Sw eventdev works well alongside Rx adapter now. Cheers, Pavan. Acked-by: Pavan Nikhilesh