From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <terry.montague.1980@btinternet.com> Received: from rgout02.bt.lon5.cpcloud.co.uk (rgout02.bt.lon5.cpcloud.co.uk [65.20.0.179]) by dpdk.org (Postfix) with ESMTP id CAE9C108F for <users@dpdk.org>; Tue, 5 Sep 2017 18:35:16 +0200 (CEST) X-OWM-Source-IP: 10.110.13.1 () X-OWM-Env-Sender: terry.montague.1980@btinternet.com X-Junkmail-Premium-Raw: score=11/50, refid=2.7.2:2017.9.5.161217:17:11.235, ip=, rules=__HAS_FROM, __FRAUD_WEBMAIL_FROM, __HAS_REPLYTO, __FRAUD_WEBMAIL_REPLYTO, __TO_MALFORMED_2, __TO_NO_NAME, __HAS_MSGID, __SANE_MSGID, INVALID_MSGID_NO_FQDN, __MIME_VERSION, __CT, __CTYPE_HAS_BOUNDARY, __CTYPE_MULTIPART, __CTYPE_MULTIPART_MIXED, __HAS_X_PRIORITY, __REPLYTO_SAMEAS_FROM_ADDY, __REPLYTO_SAMEAS_FROM_ACC, __REPLYTO_SAMEAS_FROM_DOMAIN, __MIME_TEXT_P2, __MIME_TEXT_H2, __STOCK_PHRASE_7, SUPERLONG_LINE, __HAS_HTML, __HTML_TAG_DIV, HTML_NO_HTTP, BODYTEXTP_SIZE_3000_LESS, BODY_SIZE_2000_2999, BODYTEXTH_SIZE_10000_LESS, __MIME_TEXT_H1, __MIME_TEXT_P1, __MIME_HTML, BODY_SIZE_5000_LESS, __FRAUD_WEBMAIL, NO_URI_FOUND, NO_CTA_URI_FOUND, PRIORITY_NO_NAME, __MIME_TEXT_H, __MIME_TEXT_P, REPLYTO_SAMEAS_FROM, NO_URI_HTTPS, BODY_SIZE_7000_LESS Received: from webmail12.bt.ext.cpcloud.co.uk (10.110.13.1) by rgout02.bt.lon5.cpcloud.co.uk (9.0.019.13-1) (authenticated as terry.montague.1980@btinternet.com) id 58482B971C799548 for users@dpdk.org; Tue, 5 Sep 2017 17:35:16 +0100 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=btinternet.com; s=btcpcloud; t=1504629317; bh=zqe16+sk3LxGBrrsgBnO7wC97tNzv9BA4OH6f5l/ELg=; h=Date:From:Reply-To:To:Message-ID:Subject:MIME-Version; b=esYzTvFDM4HHtRdmRoW3r55wrcnMAemdsz+SYKFUdy8LCNjNPODq8WyeYJxdEYfrS6kM9IJepB6CSU2QpkIk0mLgn1Mtfc1ip+9s7CIi5xuyM6jzal/kRs7h3kli5VERJHULQBCa4qCkdVHwDkLVmhNGRWoMRNP5kNE9dDn5Wuw= Date: Tue, 5 Sep 2017 17:35:16 +0100 (BST) From: "terry.montague.1980@btinternet.com" <terry.montague.1980@btinternet.com> To: users@dpdk.org Message-ID: <23108673.51650.1504629316520.JavaMail.defaultUser@defaultHost> MIME-Version: 1.0 Importance: 3 (Normal) X-Priority: 3 (Normal) X-Client-IP: IPv4[31.53.103.251] Epoch[1504629316487] Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.15 Subject: [dpdk-users] Linux descheduling DPDK transmit thread in SMP system? Any help greatly appreciated... X-BeenThere: users@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: terry.montague.1980@btinternet.com List-Id: DPDK usage discussions <users.dpdk.org> List-Unsubscribe: <http://dpdk.org/ml/options/users>, <mailto:users-request@dpdk.org?subject=unsubscribe> List-Archive: <http://dpdk.org/ml/archives/users/> List-Post: <mailto:users@dpdk.org> List-Help: <mailto:users-request@dpdk.org?subject=help> List-Subscribe: <http://dpdk.org/ml/listinfo/users>, <mailto:users-request@dpdk.org?subject=subscribe> X-List-Received-Date: Tue, 05 Sep 2017 16:35:17 -0000 Hi all, I'm running a DPDK transmit thread with the ixgbe PMD on Ubuntu 17.04, with dual Xeon E5-2623 v4 CPUs on an Intel S2600 motherboard. The cores in use by DPDK are isolated in the boot cmdline with "isolcpus" What I'm observing is I believe, on occasion, the DPDK transmit thread being descheduled by the system for between 20 and 60 milliseconds. I've tried running as Real time with max priority, and tweaking the scheduling through /proc/sys/kernel/sched_rt_runtime_us to be 99.999% available for real time thread, but so far no improvement I notice there are various kernel threads resident on the isolated cpus, cpuhp/watchdog/migration/ksoftirqd +multiple kworker threads. I can only assume the system is occasionally running these kernel side instead, although the CPU time taken (tens of milliseconds) seems very high. Note please that I amd NOT running with a modified kernel (nohz_full is not set). Does anyone have any advice to keep the DPDK transmit thread running for more of the time on this SMP system ? Many thanks Terry.