DPDK patches and discussions
 help / color / mirror / Atom feed
* [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
@ 2014-11-12  3:22 Chi, Xiaobo (NSN - CN/Hangzhou)
  2014-11-13 20:07 ` Igor Ryzhov
  2014-11-14 10:48 ` Bruce Richardson
  0 siblings, 2 replies; 6+ messages in thread
From: Chi, Xiaobo (NSN - CN/Hangzhou) @ 2014-11-12  3:22 UTC (permalink / raw)
  To: dev

Hi,
Background:
What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.

Problem:
Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
I have seen some others also met similar troubles.

Solution:
I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.

static void eal_secondary_mem_parse_args(int argc, char **argv)
{
        static struct option lgopts[] = {
                {OPT_HUGE_DIR, 1, 0, 0},
                {OPT_FILE_PREFIX, 1, 0, 0},
                {0, 0, 0, 0}
        };

        int opt;
        int option_index;

        while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {

        if (!opt ) {
         if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
            internal_config.hugepage_dir = optarg;
         }
         else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
            internal_config.hugefile_prefix = optarg;
         }
      }
   }
}

int rte_eal_secondary_mem_init( int argc, char **argv )
{
        static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
        const char *logid;

        if (!rte_atomic32_test_and_set(&run_once))
                return -1;

        logid = strrchr(argv[0], '/');
        logid = strdup(logid ? logid + 1: argv[0]);

        if (rte_eal_log_early_init() < 0)
                rte_panic("Cannot init early logs\n");

   memset( &internal_config, 0, sizeof( struct internal_config ) );
   /*this is only for secondary PRBs */
   internal_config.process_type = RTE_PROC_SECONDARY;
        internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
        internal_config.hugepage_dir = NULL;
   /* user can freely define the hugefile_prefix and hugepage_dir */
   eal_secondary_mem_parse_args( argc, argv );

   RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );

   /* To share memory config with PRIMARY process */
   internal_config.no_shconf = 0;
   rte_config_init();

        if (rte_eal_memory_init() < 0)
                rte_panic("Cannot init memory\n");

        if (rte_eal_memzone_init() < 0)
                rte_panic("Cannot init memzone\n");

        if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
                rte_panic("Cannot init logs\n");

        return 0;
}

brgs,
chi xiaobo

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
  2014-11-12  3:22 [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory Chi, Xiaobo (NSN - CN/Hangzhou)
@ 2014-11-13 20:07 ` Igor Ryzhov
  2014-11-14 10:48 ` Bruce Richardson
  1 sibling, 0 replies; 6+ messages in thread
From: Igor Ryzhov @ 2014-11-13 20:07 UTC (permalink / raw)
  To: Chi, Xiaobo (NSN - CN/Hangzhou); +Cc: dev

This is really useful, thank you!

Best regards,
Igor Ryzhov

> 12 нояб. 2014 г., в 6:22, Chi, Xiaobo (NSN - CN/Hangzhou) <xiaobo.chi@nsn.com> написал(а):
> 
> Hi,
> Background:
> What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.
> 
> Problem:
> Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
> I have seen some others also met similar troubles.
> 
> Solution:
> I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.
> 
> static void eal_secondary_mem_parse_args(int argc, char **argv)
> {
>        static struct option lgopts[] = {
>                {OPT_HUGE_DIR, 1, 0, 0},
>                {OPT_FILE_PREFIX, 1, 0, 0},
>                {0, 0, 0, 0}
>        };
> 
>        int opt;
>        int option_index;
> 
>        while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {
> 
>        if (!opt ) {
>         if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
>            internal_config.hugepage_dir = optarg;
>         }
>         else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
>            internal_config.hugefile_prefix = optarg;
>         }
>      }
>   }
> }
> 
> int rte_eal_secondary_mem_init( int argc, char **argv )
> {
>        static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
>        const char *logid;
> 
>        if (!rte_atomic32_test_and_set(&run_once))
>                return -1;
> 
>        logid = strrchr(argv[0], '/');
>        logid = strdup(logid ? logid + 1: argv[0]);
> 
>        if (rte_eal_log_early_init() < 0)
>                rte_panic("Cannot init early logs\n");
> 
>   memset( &internal_config, 0, sizeof( struct internal_config ) );
>   /*this is only for secondary PRBs */
>   internal_config.process_type = RTE_PROC_SECONDARY;
>        internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
>        internal_config.hugepage_dir = NULL;
>   /* user can freely define the hugefile_prefix and hugepage_dir */
>   eal_secondary_mem_parse_args( argc, argv );
> 
>   RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );
> 
>   /* To share memory config with PRIMARY process */
>   internal_config.no_shconf = 0;
>   rte_config_init();
> 
>        if (rte_eal_memory_init() < 0)
>                rte_panic("Cannot init memory\n");
> 
>        if (rte_eal_memzone_init() < 0)
>                rte_panic("Cannot init memzone\n");
> 
>        if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
>                rte_panic("Cannot init logs\n");
> 
>        return 0;
> }
> 
> brgs,
> chi xiaobo
> 
> 
> 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
  2014-11-12  3:22 [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory Chi, Xiaobo (NSN - CN/Hangzhou)
  2014-11-13 20:07 ` Igor Ryzhov
@ 2014-11-14 10:48 ` Bruce Richardson
  2014-11-14 13:54   ` Neil Horman
  1 sibling, 1 reply; 6+ messages in thread
From: Bruce Richardson @ 2014-11-14 10:48 UTC (permalink / raw)
  To: Chi, Xiaobo (NSN - CN/Hangzhou); +Cc: dev

On Wed, Nov 12, 2014 at 03:22:37AM +0000, Chi, Xiaobo (NSN - CN/Hangzhou) wrote:
> Hi,
> Background:
> What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.
> 
> Problem:
> Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
> I have seen some others also met similar troubles.
> 
> Solution:
> I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.
>

Rather than writing a whole new API and arg parsing function, might it be better
to implement this just as an additional EAL flag that prevents the existing
eal_init function from doing all tasks?

/Bruce


> static void eal_secondary_mem_parse_args(int argc, char **argv)
> {
>         static struct option lgopts[] = {
>                 {OPT_HUGE_DIR, 1, 0, 0},
>                 {OPT_FILE_PREFIX, 1, 0, 0},
>                 {0, 0, 0, 0}
>         };
> 
>         int opt;
>         int option_index;
> 
>         while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {
> 
>         if (!opt ) {
>          if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
>             internal_config.hugepage_dir = optarg;
>          }
>          else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
>             internal_config.hugefile_prefix = optarg;
>          }
>       }
>    }
> }
> 
> int rte_eal_secondary_mem_init( int argc, char **argv )
> {
>         static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
>         const char *logid;
> 
>         if (!rte_atomic32_test_and_set(&run_once))
>                 return -1;
> 
>         logid = strrchr(argv[0], '/');
>         logid = strdup(logid ? logid + 1: argv[0]);
> 
>         if (rte_eal_log_early_init() < 0)
>                 rte_panic("Cannot init early logs\n");
> 
>    memset( &internal_config, 0, sizeof( struct internal_config ) );
>    /*this is only for secondary PRBs */
>    internal_config.process_type = RTE_PROC_SECONDARY;
>         internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
>         internal_config.hugepage_dir = NULL;
>    /* user can freely define the hugefile_prefix and hugepage_dir */
>    eal_secondary_mem_parse_args( argc, argv );
> 
>    RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );
> 
>    /* To share memory config with PRIMARY process */
>    internal_config.no_shconf = 0;
>    rte_config_init();
> 
>         if (rte_eal_memory_init() < 0)
>                 rte_panic("Cannot init memory\n");
> 
>         if (rte_eal_memzone_init() < 0)
>                 rte_panic("Cannot init memzone\n");
> 
>         if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
>                 rte_panic("Cannot init logs\n");
> 
>         return 0;
> }
> 
> brgs,
> chi xiaobo
> 
> 
> 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
  2014-11-14 10:48 ` Bruce Richardson
@ 2014-11-14 13:54   ` Neil Horman
  2014-11-16  3:06     ` Chi, Xiaobo (NSN - CN/Hangzhou)
  0 siblings, 1 reply; 6+ messages in thread
From: Neil Horman @ 2014-11-14 13:54 UTC (permalink / raw)
  To: Bruce Richardson; +Cc: dev

On Fri, Nov 14, 2014 at 10:48:21AM +0000, Bruce Richardson wrote:
> On Wed, Nov 12, 2014 at 03:22:37AM +0000, Chi, Xiaobo (NSN - CN/Hangzhou) wrote:
> > Hi,
> > Background:
> > What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.
> > 
> > Problem:
> > Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
> > I have seen some others also met similar troubles.
> > 
> > Solution:
> > I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.
> >
> 
> Rather than writing a whole new API and arg parsing function, might it be better
> to implement this just as an additional EAL flag that prevents the existing
> eal_init function from doing all tasks?
> 
> /Bruce
> 
+1, that seems like a more compatible approach.
Neil

> 
> > static void eal_secondary_mem_parse_args(int argc, char **argv)
> > {
> >         static struct option lgopts[] = {
> >                 {OPT_HUGE_DIR, 1, 0, 0},
> >                 {OPT_FILE_PREFIX, 1, 0, 0},
> >                 {0, 0, 0, 0}
> >         };
> > 
> >         int opt;
> >         int option_index;
> > 
> >         while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {
> > 
> >         if (!opt ) {
> >          if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
> >             internal_config.hugepage_dir = optarg;
> >          }
> >          else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
> >             internal_config.hugefile_prefix = optarg;
> >          }
> >       }
> >    }
> > }
> > 
> > int rte_eal_secondary_mem_init( int argc, char **argv )
> > {
> >         static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
> >         const char *logid;
> > 
> >         if (!rte_atomic32_test_and_set(&run_once))
> >                 return -1;
> > 
> >         logid = strrchr(argv[0], '/');
> >         logid = strdup(logid ? logid + 1: argv[0]);
> > 
> >         if (rte_eal_log_early_init() < 0)
> >                 rte_panic("Cannot init early logs\n");
> > 
> >    memset( &internal_config, 0, sizeof( struct internal_config ) );
> >    /*this is only for secondary PRBs */
> >    internal_config.process_type = RTE_PROC_SECONDARY;
> >         internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
> >         internal_config.hugepage_dir = NULL;
> >    /* user can freely define the hugefile_prefix and hugepage_dir */
> >    eal_secondary_mem_parse_args( argc, argv );
> > 
> >    RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );
> > 
> >    /* To share memory config with PRIMARY process */
> >    internal_config.no_shconf = 0;
> >    rte_config_init();
> > 
> >         if (rte_eal_memory_init() < 0)
> >                 rte_panic("Cannot init memory\n");
> > 
> >         if (rte_eal_memzone_init() < 0)
> >                 rte_panic("Cannot init memzone\n");
> > 
> >         if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
> >                 rte_panic("Cannot init logs\n");
> > 
> >         return 0;
> > }
> > 
> > brgs,
> > chi xiaobo
> > 
> > 
> > 
> 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
  2014-11-14 13:54   ` Neil Horman
@ 2014-11-16  3:06     ` Chi, Xiaobo (NSN - CN/Hangzhou)
  2014-11-17 10:50       ` Bruce Richardson
  0 siblings, 1 reply; 6+ messages in thread
From: Chi, Xiaobo (NSN - CN/Hangzhou) @ 2014-11-16  3:06 UTC (permalink / raw)
  To: ext Neil Horman, Bruce Richardson; +Cc: dev

Hi, Bruce and Neil,
Do you indicate that to add one program arg, such as "--mem-only", to let the rte_eal_init()only to do those memory initializing things? I am worrying that this would make rte_eal_init() more complex due to the "--mem-only" should be judged here and there, even in some sub-functions. 

brgs,
chi xiaobo

-----Original Message-----
From: ext Neil Horman [mailto:nhorman@tuxdriver.com] 
Sent: Friday, November 14, 2014 9:54 PM
To: Bruce Richardson
Cc: Chi, Xiaobo (NSN - CN/Hangzhou); dev@dpdk.org
Subject: Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory

On Fri, Nov 14, 2014 at 10:48:21AM +0000, Bruce Richardson wrote:
> On Wed, Nov 12, 2014 at 03:22:37AM +0000, Chi, Xiaobo (NSN - CN/Hangzhou) wrote:
> > Hi,
> > Background:
> > What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.
> > 
> > Problem:
> > Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
> > I have seen some others also met similar troubles.
> > 
> > Solution:
> > I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.
> >
> 
> Rather than writing a whole new API and arg parsing function, might it be better
> to implement this just as an additional EAL flag that prevents the existing
> eal_init function from doing all tasks?
> 
> /Bruce
> 
+1, that seems like a more compatible approach.
Neil

> 
> > static void eal_secondary_mem_parse_args(int argc, char **argv)
> > {
> >         static struct option lgopts[] = {
> >                 {OPT_HUGE_DIR, 1, 0, 0},
> >                 {OPT_FILE_PREFIX, 1, 0, 0},
> >                 {0, 0, 0, 0}
> >         };
> > 
> >         int opt;
> >         int option_index;
> > 
> >         while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {
> > 
> >         if (!opt ) {
> >          if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
> >             internal_config.hugepage_dir = optarg;
> >          }
> >          else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
> >             internal_config.hugefile_prefix = optarg;
> >          }
> >       }
> >    }
> > }
> > 
> > int rte_eal_secondary_mem_init( int argc, char **argv )
> > {
> >         static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
> >         const char *logid;
> > 
> >         if (!rte_atomic32_test_and_set(&run_once))
> >                 return -1;
> > 
> >         logid = strrchr(argv[0], '/');
> >         logid = strdup(logid ? logid + 1: argv[0]);
> > 
> >         if (rte_eal_log_early_init() < 0)
> >                 rte_panic("Cannot init early logs\n");
> > 
> >    memset( &internal_config, 0, sizeof( struct internal_config ) );
> >    /*this is only for secondary PRBs */
> >    internal_config.process_type = RTE_PROC_SECONDARY;
> >         internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
> >         internal_config.hugepage_dir = NULL;
> >    /* user can freely define the hugefile_prefix and hugepage_dir */
> >    eal_secondary_mem_parse_args( argc, argv );
> > 
> >    RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );
> > 
> >    /* To share memory config with PRIMARY process */
> >    internal_config.no_shconf = 0;
> >    rte_config_init();
> > 
> >         if (rte_eal_memory_init() < 0)
> >                 rte_panic("Cannot init memory\n");
> > 
> >         if (rte_eal_memzone_init() < 0)
> >                 rte_panic("Cannot init memzone\n");
> > 
> >         if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
> >                 rte_panic("Cannot init logs\n");
> > 
> >         return 0;
> > }
> > 
> > brgs,
> > chi xiaobo
> > 
> > 
> > 
> 

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
  2014-11-16  3:06     ` Chi, Xiaobo (NSN - CN/Hangzhou)
@ 2014-11-17 10:50       ` Bruce Richardson
  0 siblings, 0 replies; 6+ messages in thread
From: Bruce Richardson @ 2014-11-17 10:50 UTC (permalink / raw)
  To: Chi, Xiaobo (NSN - CN/Hangzhou); +Cc: dev

On Sun, Nov 16, 2014 at 03:06:15AM +0000, Chi, Xiaobo (NSN - CN/Hangzhou) wrote:
> Hi, Bruce and Neil,
> Do you indicate that to add one program arg, such as "--mem-only", to let the rte_eal_init()only to do those memory initializing things? I am worrying that this would make rte_eal_init() more complex due to the "--mem-only" should be judged here and there, even in some sub-functions. 
> 

Where, apart from rte_eal_init() and parse_args, would it need to be added? Unless
it's a very long list of functions that need to be changed, I would still expect
the flag solution to be easier to maintain than a new set of functions.

Regards,
/Bruce

> brgs,
> chi xiaobo
> 
> -----Original Message-----
> From: ext Neil Horman [mailto:nhorman@tuxdriver.com] 
> Sent: Friday, November 14, 2014 9:54 PM
> To: Bruce Richardson
> Cc: Chi, Xiaobo (NSN - CN/Hangzhou); dev@dpdk.org
> Subject: Re: [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory
> 
> On Fri, Nov 14, 2014 at 10:48:21AM +0000, Bruce Richardson wrote:
> > On Wed, Nov 12, 2014 at 03:22:37AM +0000, Chi, Xiaobo (NSN - CN/Hangzhou) wrote:
> > > Hi,
> > > Background:
> > > What we are doing now is port make telecom network element to be cloud based.  For one of our product,  DPDK is applied not only for fastpath/dataplane processing, but also for Distributed Message eXchange (DMX) between different processes/applications which may located in different VM even different host.  for such a DMX system, in one VM, we have one DPDK based dmxdemo (which acts as the PRIMARY) which is in charge of distribute message between different applications, and dozens of applications (act as SECONDARY) to use DPDK based rte_tx_ring/rte_rx_ring/mempool/memzone to send receive messages to dmxdemo.
> > > 
> > > Problem:
> > > Here, these DPDK based SECONDARY processes need only the DPDK's hugepage based sharememory mechanism and it's upper libs (such as ring, mempool, etc.), they need not cpu core pinning, iopl privilege changing , pci device, timer, alarm, interrupt, shared_driver_list,  core_info, threads for each core, etc. Then, for such kind of SECONDARY processes, the current rte_eal_init() is too heavy.
> > > I have seen some others also met similar troubles.
> > > 
> > > Solution:
> > > I write one light weight rte_eal_init(), called rte_eal_secondary_mem_init() as following.  It only initializes shared memory and mandatory resources. I expect your review and hope these code can be merged into DPDK main branch.
> > >
> > 
> > Rather than writing a whole new API and arg parsing function, might it be better
> > to implement this just as an additional EAL flag that prevents the existing
> > eal_init function from doing all tasks?
> > 
> > /Bruce
> > 
> +1, that seems like a more compatible approach.
> Neil
> 
> > 
> > > static void eal_secondary_mem_parse_args(int argc, char **argv)
> > > {
> > >         static struct option lgopts[] = {
> > >                 {OPT_HUGE_DIR, 1, 0, 0},
> > >                 {OPT_FILE_PREFIX, 1, 0, 0},
> > >                 {0, 0, 0, 0}
> > >         };
> > > 
> > >         int opt;
> > >         int option_index;
> > > 
> > >         while ((opt = getopt_long(argc, argv, "", lgopts, &option_index)) != EOF) {
> > > 
> > >         if (!opt ) {
> > >          if (!strcmp(lgopts[option_index].name, OPT_HUGE_DIR)) {
> > >             internal_config.hugepage_dir = optarg;
> > >          }
> > >          else if (!strcmp(lgopts[option_index].name, OPT_FILE_PREFIX)) {
> > >             internal_config.hugefile_prefix = optarg;
> > >          }
> > >       }
> > >    }
> > > }
> > > 
> > > int rte_eal_secondary_mem_init( int argc, char **argv )
> > > {
> > >         static rte_atomic32_t run_once = RTE_ATOMIC32_INIT(0);
> > >         const char *logid;
> > > 
> > >         if (!rte_atomic32_test_and_set(&run_once))
> > >                 return -1;
> > > 
> > >         logid = strrchr(argv[0], '/');
> > >         logid = strdup(logid ? logid + 1: argv[0]);
> > > 
> > >         if (rte_eal_log_early_init() < 0)
> > >                 rte_panic("Cannot init early logs\n");
> > > 
> > >    memset( &internal_config, 0, sizeof( struct internal_config ) );
> > >    /*this is only for secondary PRBs */
> > >    internal_config.process_type = RTE_PROC_SECONDARY;
> > >         internal_config.hugefile_prefix = HUGEFILE_PREFIX_DEFAULT;
> > >         internal_config.hugepage_dir = NULL;
> > >    /* user can freely define the hugefile_prefix and hugepage_dir */
> > >    eal_secondary_mem_parse_args( argc, argv );
> > > 
> > >    RTE_LOG(INFO, EAL, "prefix=%s, dir=%s.\n",internal_config.hugefile_prefix, internal_config.hugepage_dir );
> > > 
> > >    /* To share memory config with PRIMARY process */
> > >    internal_config.no_shconf = 0;
> > >    rte_config_init();
> > > 
> > >         if (rte_eal_memory_init() < 0)
> > >                 rte_panic("Cannot init memory\n");
> > > 
> > >         if (rte_eal_memzone_init() < 0)
> > >                 rte_panic("Cannot init memzone\n");
> > > 
> > >         if (rte_eal_log_init(logid, LOG_DAEMON ) < 0)
> > >                 rte_panic("Cannot init logs\n");
> > > 
> > >         return 0;
> > > }
> > > 
> > > brgs,
> > > chi xiaobo
> > > 
> > > 
> > > 
> > 

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2014-11-17 10:40 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-11-12  3:22 [dpdk-dev] one lightwight rte_eal_init() for SECONDARY processes which only use sharedmemory Chi, Xiaobo (NSN - CN/Hangzhou)
2014-11-13 20:07 ` Igor Ryzhov
2014-11-14 10:48 ` Bruce Richardson
2014-11-14 13:54   ` Neil Horman
2014-11-16  3:06     ` Chi, Xiaobo (NSN - CN/Hangzhou)
2014-11-17 10:50       ` Bruce Richardson

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).