From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51]) by dpdk.org (Postfix) with ESMTP id 1E7F812A1 for ; Mon, 21 Apr 2014 22:10:02 +0200 (CEST) Received: by mail-pa0-f51.google.com with SMTP id kq14so4095706pab.10 for ; Mon, 21 Apr 2014 13:10:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:cc:subject:message-id:in-reply-to :references:mime-version:content-type:content-transfer-encoding; bh=Iz1dMIYtEd5tbVWGuHdH+5ViQgRItIiZFMZBfXpHaKg=; b=QkBCg+CniE7ptIeXTU+3nCw82jrES0J3hMERHBeUey7inQzzYe86wo72zV8CdwjkUj qi5+QoR/YcPaGNSShu7qjA48zQfNwMP36aUHYElUgxBphbN15vzVprzG1tzuP5oScN4Y DFObjEXgn9HtTBCqnwDqXBFt+0zecWH4tcKkEIpQPa58SNq5W6q28fW0V4jxKJ3ZwE// g6j89Zsae+D7dRKVludOso3mLixHj9KZ846wsfQ6YWfmu7EQfzXSgV1jTt7Bd7PgO1LR GCL4lSuA6ZLroKlonqyh29Frfbd0AqoqsZRGjaf59V4XHMGWQ4JDcNFiqi/Kga/wGmMQ LLxw== X-Gm-Message-State: ALoCoQmpexRlRbGeyMDQVwOP2jSdhVVt4QyJTro9EVuUB9aZvlrYPmbNlvZ4mY5w4FdUIzclmqWv X-Received: by 10.68.211.164 with SMTP id nd4mr40030955pbc.44.1398111004120; Mon, 21 Apr 2014 13:10:04 -0700 (PDT) Received: from nehalam.linuxnetplumber.net (static-50-53-83-51.bvtn.or.frontiernet.net. [50.53.83.51]) by mx.google.com with ESMTPSA id pb7sm190803518pac.10.2014.04.21.13.10.03 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 21 Apr 2014 13:10:03 -0700 (PDT) Date: Mon, 21 Apr 2014 13:10:00 -0700 From: Stephen Hemminger To: Neil Horman Message-ID: <20140421131000.743b6d9b@nehalam.linuxnetplumber.net> In-Reply-To: <1398092379-7679-1-git-send-email-nhorman@tuxdriver.com> References: <1397585169-14537-1-git-send-email-nhorman@tuxdriver.com> <1398092379-7679-1-git-send-email-nhorman@tuxdriver.com> X-Mailer: Claws Mail 3.9.3 (GTK+ 2.24.23; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: dev@dpdk.org Subject: Re: [dpdk-dev] [PATCH v5 00/14] dpdk: Separate compile time linkage between eal lib and pmd's X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: patches and discussions about DPDK List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 20:10:03 -0000 On Mon, 21 Apr 2014 10:59:25 -0400 Neil Horman wrote: > Disconnect compile time linkage between eal library / applications and pmd's > > I noticed that, while tinkering with dpdk, building for shared libraries still > resulted in all the test applications linking to all the built pmd's, despite > not actually needing them all. We are able to tell an application at run time > (via the -d/--blacklist/--whitelist/--vdev options) which pmd's we want to use, > and so have no need to link them at all. The only reason they get pulled in is > because rte_eal_non_pci_init_etherdev and rte_pmd_init_all contain static lists > to the individual pmd init functions. The result is that, even when building as > DSO's, we have to load all the pmd libraries, which is space inefficient and > defeating of some of the purpose of shared objects. > > To correct this, I developed this patch series, which introduces a new macro, > PMD_REGISTER_DRIVER, which wraps up Oliviers work using constructors on the > virtual device pmds, then expands on it to include the physical device pmds, > allowing us to break linkages between dpdk applications and pmd's almost > entirely (save for the ring and xenvirt drivers, which have additional api's > outside of the standard dpdk code that we need to further fix). This also > allows us to completely remove the rte_pmd_init_all routine, hiding its function > internally to the rte_eal_init path. > > I've tested this feature using the igb and pcap pmd's, both statically and > dynamically linked with the test and testpmd sample applications, and it seems > to work well. > > Note, I encountered a few bugs along the way, which I fixed and noted in the > series. > > Regards > Neil > > Change Notes: > > 1) Reposted the entire series. Recent changes permeated accross several > patches, and since a few patches already had multiple versions, I've reposted > the entire series and bumped the version number to 5, whcih skips a few > versions, but ensures this series is seen as the most recent. > > 2) Merged the PMD_REGISTER_DRIVER macro into rte_dev.h, which is a better > location for it. Required removing include of rte_pmd.h across most of the > patches in the series. > > 3) Cleaned up various leftover "virtual" comments in the driver registration api > that no longer belong there after making it generic > > 4) Fixed the LINK_USING_CC check in rte.lib.mk so it works like other uses > > 5) Fixed CPU_LDFLAGS conversion to use -Wl in case we link with gcc. I am ok with build device drivers as libraries, but there can be a performance impact. For performance, we link with link-time optimization and that is not possible as shared library. Haven't measured but at least in the kernel, modules add a performance tax because they aren't in same memory region and therefore cause a TLB miss. Might have same impact in user mode. You might be able to get some of this back by compiling with link-time-optimization on a device at a time, and also use the flags to tell compiler that only certain entry points are shared and need tob e global.