[dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Somnath Kotur]

Hello,
       I'm trying to launch a thread - lcore_mainloop( from
examples/timer/main.c ) that runs rte_manage_timer() every 2s from testpmd
to ensure the timers i've registered in my driver are checked for expiry  (
i even tried putting this thread in my  driver as well,  no difference in
results) and i see that while this thread is running, i somehow seem to
stop getting interrupts ..infact i don't even
see eal_intr_process_interrupts () being called.


diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
index ca4e1a4..a8d71d6 100644
--- a/app/test-pmd/testpmd.c
+++ b/app/test-pmd/testpmd.c
@@ -71,6 +71,8 @@
 #include <rte_pci.h>
 #include <rte_ether.h>
 #include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_timer.h>
 #include <rte_dev.h>
 #include <rte_string_fns.h>
 #ifdef RTE_LIBRTE_IXGBE_PMD
@@ -2524,6 +2526,30 @@ signal_handler(int signum)
        }
 }

+static int
+lcore_mainloop(__attribute__((unused)) void *arg)
+{
+       uint64_t prev_tsc = 0, cur_tsc, diff_tsc;
+       unsigned int lcore_id;
+
+       lcore_id = rte_lcore_id();
+       printf("Starting mainloop on core %u\n", lcore_id);
+
+       while (f_quit == 0) {
+               cur_tsc = rte_rdtsc();
+               diff_tsc = cur_tsc - prev_tsc;
+               /* Schedule every 2 seconds */
+               if (diff_tsc > rte_get_timer_hz() * 2) {
+                       rte_timer_manage();
+                       prev_tsc = cur_tsc;
+               } else
+                       sleep(1);
+       }
+       return 0;
+}
+
 int
 main(int argc, char** argv)
 {
@@ -2627,6 +2653,7 @@ main(int argc, char** argv)
        if (strlen(cmdline_filename) != 0)
                cmdline_read_from_file(cmdline_filename);

+       rte_eal_remote_launch(lcore_mainloop, NULL, 3);
        if (interactive == 1) {
                if (auto_start) {
                        printf("Start automatic packet forwarding\n");


My testpmd cmdline is like so:

testpmd -c 0xff -n 3 -- -i portmask=0x3 --nb-cores=3 --rxq=1 --txq=1

Any idea what could be the problem ? Is this something that is expected or
am i doing something wrong ?

Thanks
Som

[dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Somnath Kotur]

On Fri, Nov 2, 2018 at 9:30 AM Somnath Kotur <somnath.kotur@broadcom.com>
wrote:

> Hello,
>        I'm trying to launch a thread - lcore_mainloop( from
> examples/timer/main.c ) that runs rte_manage_timer() every 2s from testpmd
> to ensure the timers i've registered in my driver are checked for expiry  (
> i even tried putting this thread in my  driver as well,  no difference in
> results) and i see that while this thread is running, i somehow seem to
> stop getting interrupts ..infact i don't even
> see eal_intr_process_interrupts () being called.
>
>
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index ca4e1a4..a8d71d6 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -71,6 +71,8 @@
>  #include <rte_pci.h>
>  #include <rte_ether.h>
>  #include <rte_ethdev.h>
> +#include <rte_cycles.h>
> +#include <rte_timer.h>
>  #include <rte_dev.h>
>  #include <rte_string_fns.h>
>  #ifdef RTE_LIBRTE_IXGBE_PMD
> @@ -2524,6 +2526,30 @@ signal_handler(int signum)
>         }
>  }
>
> +static int
> +lcore_mainloop(__attribute__((unused)) void *arg)
> +{
> +       uint64_t prev_tsc = 0, cur_tsc, diff_tsc;
> +       unsigned int lcore_id;
> +
> +       lcore_id = rte_lcore_id();
> +       printf("Starting mainloop on core %u\n", lcore_id);
> +
> +       while (f_quit == 0) {
> +               cur_tsc = rte_rdtsc();
> +               diff_tsc = cur_tsc - prev_tsc;
> +               /* Schedule every 2 seconds */
> +               if (diff_tsc > rte_get_timer_hz() * 2) {
> +                       rte_timer_manage();
> +                       prev_tsc = cur_tsc;
> +               } else
> +                       sleep(1);
> +       }
> +       return 0;
> +}
> +
>  int
>  main(int argc, char** argv)
>  {
> @@ -2627,6 +2653,7 @@ main(int argc, char** argv)
>         if (strlen(cmdline_filename) != 0)
>                 cmdline_read_from_file(cmdline_filename);
>
> +       rte_eal_remote_launch(lcore_mainloop, NULL, 3);
>         if (interactive == 1) {
>                 if (auto_start) {
>                         printf("Start automatic packet forwarding\n");
>
>
> My testpmd cmdline is like so:
>
> testpmd -c 0xff -n 3 -- -i portmask=0x3 --nb-cores=3 --rxq=1 --txq=1
>
> Any idea what could be the problem ? Is this something that is expected or
> am i doing something wrong ?
>
> Thanks
> Som
>

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Burakov, Anatoly]

On 02-Nov-18 4:00 AM, Somnath Kotur wrote:
> Hello,
>         I'm trying to launch a thread - lcore_mainloop( from
> examples/timer/main.c ) that runs rte_manage_timer() every 2s from testpmd
> to ensure the timers i've registered in my driver are checked for expiry  (
> i even tried putting this thread in my  driver as well,  no difference in
> results) and i see that while this thread is running, i somehow seem to
> stop getting interrupts ..infact i don't even
> see eal_intr_process_interrupts () being called.
> 
> 
> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
> index ca4e1a4..a8d71d6 100644
> --- a/app/test-pmd/testpmd.c
> +++ b/app/test-pmd/testpmd.c
> @@ -71,6 +71,8 @@
>   #include <rte_pci.h>
>   #include <rte_ether.h>
>   #include <rte_ethdev.h>
> +#include <rte_cycles.h>
> +#include <rte_timer.h>
>   #include <rte_dev.h>
>   #include <rte_string_fns.h>
>   #ifdef RTE_LIBRTE_IXGBE_PMD
> @@ -2524,6 +2526,30 @@ signal_handler(int signum)
>          }
>   }
> 
> +static int
> +lcore_mainloop(__attribute__((unused)) void *arg)
> +{
> +       uint64_t prev_tsc = 0, cur_tsc, diff_tsc;
> +       unsigned int lcore_id;
> +
> +       lcore_id = rte_lcore_id();
> +       printf("Starting mainloop on core %u\n", lcore_id);
> +
> +       while (f_quit == 0) {
> +               cur_tsc = rte_rdtsc();
> +               diff_tsc = cur_tsc - prev_tsc;
> +               /* Schedule every 2 seconds */
> +               if (diff_tsc > rte_get_timer_hz() * 2) {
> +                       rte_timer_manage();
> +                       prev_tsc = cur_tsc;
> +               } else
> +                       sleep(1);
> +       }
> +       return 0;
> +}
> +
>   int
>   main(int argc, char** argv)
>   {
> @@ -2627,6 +2653,7 @@ main(int argc, char** argv)
>          if (strlen(cmdline_filename) != 0)
>                  cmdline_read_from_file(cmdline_filename);
> 
> +       rte_eal_remote_launch(lcore_mainloop, NULL, 3);
>          if (interactive == 1) {
>                  if (auto_start) {
>                          printf("Start automatic packet forwarding\n");
> 
> 
> My testpmd cmdline is like so:
> 
> testpmd -c 0xff -n 3 -- -i portmask=0x3 --nb-cores=3 --rxq=1 --txq=1
> 
> Any idea what could be the problem ? Is this something that is expected or
> am i doing something wrong ?
> 
> Thanks
> Som
> 

I may be completely off mark here, but as far as i understand, the EAL 
Alarm API uses the interrupt thread. The rte_timer API is a high 
performance timer API and is meant to be managed manually, by 
periodically[1] calling rte_timer_manage(). If you want something to be 
called every two seconds, just set up an rte_alarm - there's no need to 
use the timer API (unless you are on FreeBSD, where alarm API is not 
officially supported).

[1] as in, more frequently than every two seconds if you want to have 
any semblance of timer precision!

-- 
Thanks,
Anatoly

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Wiles, Keith]

> On Nov 2, 2018, at 6:31 AM, Burakov, Anatoly <anatoly.burakov@intel.com> wrote:
> 
> On 02-Nov-18 4:00 AM, Somnath Kotur wrote:
>> Hello,
>>        I'm trying to launch a thread - lcore_mainloop( from
>> examples/timer/main.c ) that runs rte_manage_timer() every 2s from testpmd
>> to ensure the timers i've registered in my driver are checked for expiry  (
>> i even tried putting this thread in my  driver as well,  no difference in
>> results) and i see that while this thread is running, i somehow seem to
>> stop getting interrupts ..infact i don't even
>> see eal_intr_process_interrupts () being called.
>> diff --git a/app/test-pmd/testpmd.c b/app/test-pmd/testpmd.c
>> index ca4e1a4..a8d71d6 100644
>> --- a/app/test-pmd/testpmd.c
>> +++ b/app/test-pmd/testpmd.c
>> @@ -71,6 +71,8 @@
>>  #include <rte_pci.h>
>>  #include <rte_ether.h>
>>  #include <rte_ethdev.h>
>> +#include <rte_cycles.h>
>> +#include <rte_timer.h>
>>  #include <rte_dev.h>
>>  #include <rte_string_fns.h>
>>  #ifdef RTE_LIBRTE_IXGBE_PMD
>> @@ -2524,6 +2526,30 @@ signal_handler(int signum)
>>         }
>>  }
>> +static int
>> +lcore_mainloop(__attribute__((unused)) void *arg)
>> +{
>> +       uint64_t prev_tsc = 0, cur_tsc, diff_tsc;
>> +       unsigned int lcore_id;
>> +
>> +       lcore_id = rte_lcore_id();
>> +       printf("Starting mainloop on core %u\n", lcore_id);
>> +
>> +       while (f_quit == 0) {
>> +               cur_tsc = rte_rdtsc();
>> +               diff_tsc = cur_tsc - prev_tsc;
>> +               /* Schedule every 2 seconds */
>> +               if (diff_tsc > rte_get_timer_hz() * 2) {
>> +                       rte_timer_manage();
>> +                       prev_tsc = cur_tsc;
>> +               } else
>> +                       sleep(1);
>> +       }
>> +       return 0;
>> +}
>> +
>>  int
>>  main(int argc, char** argv)
>>  {
>> @@ -2627,6 +2653,7 @@ main(int argc, char** argv)
>>         if (strlen(cmdline_filename) != 0)
>>                 cmdline_read_from_file(cmdline_filename);
>> +       rte_eal_remote_launch(lcore_mainloop, NULL, 3);
>>         if (interactive == 1) {
>>                 if (auto_start) {
>>                         printf("Start automatic packet forwarding\n");
>> My testpmd cmdline is like so:
>> testpmd -c 0xff -n 3 -- -i portmask=0x3 --nb-cores=3 --rxq=1 --txq=1
>> Any idea what could be the problem ? Is this something that is expected or
>> am i doing something wrong ?
>> Thanks
>> Som
> 
> I may be completely off mark here, but as far as i understand, the EAL Alarm API uses the interrupt thread. The rte_timer API is a high performance timer API and is meant to be managed manually, by periodically[1] calling rte_timer_manage(). If you want something to be called every two seconds, just set up an rte_alarm - there's no need to use the timer API (unless you are on FreeBSD, where alarm API is not officially supported).
> 
> [1] as in, more frequently than every two seconds if you want to have any semblance of timer precision!
> 

Unless we changed it I believe the 

> -- 
> Thanks,
> Anatoly

Regards,
Keith

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Wiles, Keith]

> On Nov 2, 2018, at 9:35 AM, Wiles, Keith <keith.wiles@intel.com> wrote:
> 
> 
Sorry, meant to hit cancel for my previous email, Anatoly answered it correctly.

Regards,
Keith

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Somnath Kotur]

Thank you Anatoly and keith for your replies. What i still don't get is why
the two  cannot seem to co-exist?
That is , whenever this while loop was running in the EAL thread that i
showed in the code, my driver would stop receiving interrupts/async events
...Do we understand that?
But you are right Anatoly, i need something to be called in my driver at a
cadence of ~2s, I was registering timers in my driver and running
rte_timer_manage() in this seperate thread to check  and ensure the
callbacks are executed on expiry.
so if you are saying, i can achieve same functionality using rte_alarm()
while i can get async events in my driver at the same time, i will explore
this option
Thanks a lot

Som

On Fri, Nov 2, 2018 at 8:07 PM Wiles, Keith <keith.wiles@intel.com> wrote:

>
>
> > On Nov 2, 2018, at 9:35 AM, Wiles, Keith <keith.wiles@intel.com> wrote:
> >
> >
> Sorry, meant to hit cancel for my previous email, Anatoly answered it
> correctly.
>
> Regards,
> Keith
>
>

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Somnath Kotur]

Hi Anatoly/keith,
                     I just tried using rte_eal_alarm_set() in the driver
like so:

rte_eal_alarm_set(US_PER_S * 2, timer0_cb, (void *)data);

The moment i start this off, i stop getting async events/interrupt
notifications in my driver ...I'm hitting the same problem as it was before
with rte_timer_manage() as well.
What is going wrong here ? Something we are missing ?

Thanks
Som

On Sat, Nov 3, 2018 at 8:06 AM Somnath Kotur <somnath.kotur@broadcom.com>
wrote:

> Thank you Anatoly and keith for your replies. What i still don't get is
> why the two  cannot seem to co-exist?
> That is , whenever this while loop was running in the EAL thread that i
> showed in the code, my driver would stop receiving interrupts/async events
> ...Do we understand that?
> But you are right Anatoly, i need something to be called in my driver at a
> cadence of ~2s, I was registering timers in my driver and running
> rte_timer_manage() in this seperate thread to check  and ensure the
> callbacks are executed on expiry.
> so if you are saying, i can achieve same functionality using rte_alarm()
> while i can get async events in my driver at the same time, i will explore
> this option
> Thanks a lot
>
> Som
>
> On Fri, Nov 2, 2018 at 8:07 PM Wiles, Keith <keith.wiles@intel.com> wrote:
>
>>
>>
>> > On Nov 2, 2018, at 9:35 AM, Wiles, Keith <keith.wiles@intel.com> wrote:
>> >
>> >
>> Sorry, meant to hit cancel for my previous email, Anatoly answered it
>> correctly.
>>
>> Regards,
>> Keith
>>
>>

Re: [dpdk-dev] Question about rte_manage_timer() and eal_intr_handle_interrupts

[Somnath Kotur]

Hello Keith/Anatoly,
                     Another strange thing, it appears that if i keep a
breakpoint (using gdb) on my async intr handler(bnxt_int_handler), then it
seems to hit it (after the alarm callback is invoked first) and then i can
see both this alarm going off every 2s and those async/interrupts arriving
on my driver....

Thanks
Som

On Sat, Nov 3, 2018 at 9:58 AM Somnath Kotur <somnath.kotur@broadcom.com>
wrote:

> Hi Anatoly/keith,
>                      I just tried using rte_eal_alarm_set() in the driver
> like so:
>
> rte_eal_alarm_set(US_PER_S * 2, timer0_cb, (void *)data);
>
> The moment i start this off, i stop getting async events/interrupt
> notifications in my driver ...I'm hitting the same problem as it was before
> with rte_timer_manage() as well.
> What is going wrong here ? Something we are missing ?
>
> Thanks
> Som
>
> On Sat, Nov 3, 2018 at 8:06 AM Somnath Kotur <somnath.kotur@broadcom.com>
> wrote:
>
>> Thank you Anatoly and keith for your replies. What i still don't get is
>> why the two  cannot seem to co-exist?
>> That is , whenever this while loop was running in the EAL thread that i
>> showed in the code, my driver would stop receiving interrupts/async events
>> ...Do we understand that?
>> But you are right Anatoly, i need something to be called in my driver at
>> a cadence of ~2s, I was registering timers in my driver and running
>> rte_timer_manage() in this seperate thread to check  and ensure the
>> callbacks are executed on expiry.
>> so if you are saying, i can achieve same functionality using rte_alarm()
>> while i can get async events in my driver at the same time, i will explore
>> this option
>> Thanks a lot
>>
>> Som
>>
>> On Fri, Nov 2, 2018 at 8:07 PM Wiles, Keith <keith.wiles@intel.com>
>> wrote:
>>
>>>
>>>
>>> > On Nov 2, 2018, at 9:35 AM, Wiles, Keith <keith.wiles@intel.com>
>>> wrote:
>>> >
>>> >
>>> Sorry, meant to hit cancel for my previous email, Anatoly answered it
>>> correctly.
>>>
>>> Regards,
>>> Keith
>>>
>>>