From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by dpdk.org (Postfix) with ESMTP id 1B9AC1B505 for ; Thu, 13 Dec 2018 17:46:52 +0100 (CET) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 13 Dec 2018 08:46:52 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,349,1539673200"; d="scan'208";a="101296867" Received: from dhunt5-mobl2.ger.corp.intel.com (HELO [10.237.221.66]) ([10.237.221.66]) by orsmga008.jf.intel.com with ESMTP; 13 Dec 2018 08:46:50 -0800 To: "Burakov, Anatoly" , dev@dpdk.org Cc: lei.a.yao@intel.com References: <20181122170220.55482-1-david.hunt@intel.com> <20181122170220.55482-4-david.hunt@intel.com> <472ceb5c-7c2c-854c-7ad4-16dd8702ffc1@intel.com> From: "Hunt, David" Message-ID: <62133fbe-9777-b9fb-0a6d-bb8509641b75@intel.com> Date: Thu, 13 Dec 2018 16:46:48 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.3.3 MIME-Version: 1.0 In-Reply-To: <472ceb5c-7c2c-854c-7ad4-16dd8702ffc1@intel.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US Subject: Re: [dpdk-dev] [PATCH v1 3/4] examples/power: allow vms to use lcores over 63 X-BeenThere: dev@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: DPDK patches and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Dec 2018 16:46:53 -0000 Hi Anatoly, On 10/12/2018 1:06 PM, Burakov, Anatoly wrote: > On 22-Nov-18 5:02 PM, David Hunt wrote: >> Extending the functionality to allow vms to power manage cores beyond >> 63. >> >> Signed-off-by: David Hunt >> --- >>   examples/vm_power_manager/channel_manager.c | 59 ++++++++------------- >>   examples/vm_power_manager/channel_manager.h | 30 ++--------- >>   examples/vm_power_manager/channel_monitor.c | 56 +++++++------------ >>   examples/vm_power_manager/vm_power_cli.c    |  4 +- >>   4 files changed, 48 insertions(+), 101 deletions(-) >> >> diff --git a/examples/vm_power_manager/channel_manager.c >> b/examples/vm_power_manager/channel_manager.c >> index 5af4996db..3d493c179 100644 >> --- a/examples/vm_power_manager/channel_manager.c >> +++ b/examples/vm_power_manager/channel_manager.c >> @@ -49,7 +49,7 @@ static bool global_hypervisor_available; >>    */ >>   struct virtual_machine_info { >>       char name[CHANNEL_MGR_MAX_NAME_LEN]; >> -    rte_atomic64_t pcpu_mask[CHANNEL_CMDS_MAX_CPUS]; >> +    uint16_t pcpu_map[CHANNEL_CMDS_MAX_CPUS]; >>       struct channel_info *channels[CHANNEL_CMDS_MAX_VM_CHANNELS]; >>       char channel_mask[POWER_MGR_MAX_CPUS]; >>       uint8_t num_channels; >> @@ -79,7 +79,7 @@ update_pcpus_mask(struct virtual_machine_info >> *vm_info) >>       virVcpuInfoPtr cpuinfo; >>       unsigned i, j; >>       int n_vcpus; >> -    uint64_t mask; >> +    uint16_t pcpu; >>         memset(global_cpumaps, 0, CHANNEL_CMDS_MAX_CPUS*global_maplen); >>   @@ -120,26 +120,23 @@ update_pcpus_mask(struct virtual_machine_info >> *vm_info) >>           vm_info->info.nrVirtCpu = n_vcpus; >>       } >>       for (i = 0; i < vm_info->info.nrVirtCpu; i++) { >> -        mask = 0; >> +        pcpu = 0; >>           for (j = 0; j < global_n_host_cpus; j++) { >>               if (VIR_CPU_USABLE(global_cpumaps, global_maplen, i, j) >> > 0) { >> -                mask |= 1ULL << j; >> +                pcpu = j; > > Not sure what this does. > > Initial code goes through all CPU's, marks all that are usable into > mask (i.e. code implies there can be several), and then sets up this > mask. > > Now, you're going through all CPU's, store *one* arbitrary CPU index, > and set the map up with that single index. > > That doesn't look like an equivalent replacement. > Good catch. mask was changed to an array, so I need to set element in the loop, not outside it. Fixed in next rev. >>               } >>           } >> -        rte_atomic64_set(&vm_info->pcpu_mask[i], mask); >> +        vm_info->pcpu_map[i] = pcpu; >>       } >>       return 0; >>   } >>     int >> -set_pcpus_mask(char *vm_name, unsigned int vcpu, char *core_mask) >> +set_pcpu(char *vm_name, unsigned int vcpu, unsigned int pcpu) > > See patch 1 comments - it would've made this change easier to parse if > set_pcpu was removed there, instead of here. > I couldn't change it back there, as the calling functions had not been updated. Now that they are gone, I can remove it in this patch. >>   { >>       unsigned i = 0; >>       int flags = VIR_DOMAIN_AFFECT_LIVE|VIR_DOMAIN_AFFECT_CONFIG; >>       struct virtual_machine_info *vm_info; >> -    char mask[POWER_MGR_MAX_CPUS]; >> - >> -    memcpy(mask, core_mask, POWER_MGR_MAX_CPUS); >>         if (vcpu >= CHANNEL_CMDS_MAX_CPUS) { >>           RTE_LOG(ERR, CHANNEL_MANAGER, "vCPU(%u) exceeds max >> allowable(%d)\n", >> @@ -166,17 +163,16 @@ set_pcpus_mask(char *vm_name, unsigned int >> vcpu, char *core_mask) >>           return -1; >>       } >>       memset(global_cpumaps, 0 , CHANNEL_CMDS_MAX_CPUS * global_maplen); >> -    for (i = 0; i < POWER_MGR_MAX_CPUS; i++) { >> -        if (mask[i] == 1) { >> -            VIR_USE_CPU(global_cpumaps, i); >> -            if (i >= global_n_host_cpus) { >> -                RTE_LOG(ERR, CHANNEL_MANAGER, "CPU(%u) exceeds the >> available " >> -                        "number of CPUs(%u)\n", >> -                        i, global_n_host_cpus); >> -                return -1; >> -            } >> -        } >> + >> +    VIR_USE_CPU(global_cpumaps, i); >> + >> +    if (pcpu >= global_n_host_cpus) { >> +        RTE_LOG(ERR, CHANNEL_MANAGER, "CPU(%u) exceeds the available " >> +                "number of CPUs(%u)\n", >> +                i, global_n_host_cpus); >> +        return -1; >>       } > > Some comments on what the above code does would have been nice. Why > the check was removed? function changed from set_pcpus_mask to the name of what used to be the wrapper function, and in doing so we changed the mask parameter to a pcpu (int) parameter, so there's not need for a loop any more. However, there was an error where I was calling VIR_USE_CPU with the incorrect value, so that's now fixed.  The check was removed because we're no longer checking for '1's in an array, we are being passed the actual pcpu to use. Also, there were an addition of a spinlock around reading the pcpu config data. > >> + >