From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from EUR01-HE1-obe.outbound.protection.outlook.com (mail-he1eur01on0080.outbound.protection.outlook.com [104.47.0.80]) by dpdk.org (Postfix) with ESMTP id 7598E1B365 for ; Fri, 22 Dec 2017 10:53:31 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=armh.onmicrosoft.com; s=selector1-arm-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=av/hGscyJJKDvqvac7ENgHg/H/UEntkLIcasgkMqKZg=; b=WclA/YLK66W+sF3SsKiSx10u+ir1i3Nu2O6sw1COTnt/Ot1TR7G7et5V94c53gMDtQwJbib0QobIILlIZGqxccNGq8m0ZJfWmWiRUsdhpeQKWjl9E7P2GQ7pVQibOlUg5hhbotZZk9ppjCVkzS7fB5BzjT/IOdsNgfHWt1TUwNo= Received: from HE1PR08MB2809.eurprd08.prod.outlook.com (10.170.246.148) by HE1PR08MB0923.eurprd08.prod.outlook.com (10.166.125.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.345.14; Fri, 22 Dec 2017 09:53:29 +0000 Received: from HE1PR08MB2809.eurprd08.prod.outlook.com ([fe80::90dc:2dac:4dcd:a0f9]) by HE1PR08MB2809.eurprd08.prod.outlook.com ([fe80::90dc:2dac:4dcd:a0f9%13]) with mapi id 15.20.0323.018; Fri, 22 Dec 2017 09:53:29 +0000 From: Herbert Guan To: "Liu, Yong" , "dts@dpdk.org" CC: Phil Yang , Jianbo Liu , nd Thread-Topic: [dts] [PATCH] framework: enable AArch64 and add more options for libvirt Thread-Index: AQHTevxAcYGHvVSxGUeZWCwVilwtuKNPBIhg Date: Fri, 22 Dec 2017 09:53:28 +0000 Message-ID: References: <1512452777-3982-1-git-send-email-herbert.guan@arm.com> <86228AFD5BCD8E4EBFD2B90117B5E81E62F3804E@SHSMSX103.ccr.corp.intel.com> In-Reply-To: <86228AFD5BCD8E4EBFD2B90117B5E81E62F3804E@SHSMSX103.ccr.corp.intel.com> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: spf=none (sender IP is ) smtp.mailfrom=Herbert.Guan@arm.com; x-originating-ip: [113.29.88.7] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; HE1PR08MB0923; 6:1CMItC9w+EJvS+FVcRyhELjO68RPRKHEtJe0wQ/8NEuFPjiCmO/GW5OjL8Oo3CBtnfiSC3qJc/nry1iX+jL5XMWKGUeWwUR0A9xnheHkOMHmeAj78JHFqMgP+ewiXG1UoWFfD7rwfrWfIWwfYiST5VwTVRUQk5gnjubGTikI5GocSV0L8A8ERuhsuwYSfLtLgnS4F5wTsICaVw5dQUXXdobcFOd5Eb2S//fo2mDqUz0Xy7F9yJHC05xAaj7qaiXkGUMA7vNpE50nbP2nC46I4xty3zmUZQ8RVfbe39K6y6sicdrpBG0bARCR0Ef8tbL8GPBPJnBmJWGCCwGfqBF15YsFQACit/FKvjphtJQnTbU=; 5:8sILVTQ6GoyTRu3p7kPGNgvLjbCTzbHt77LRRN956QpJCo9QyqiLSr9Q/pn+sMDfWRWjvN8EAbwR8fobyucIKT/TOB5CgfjK78KgA18g/y0U1APRX1jpwTddkUtSoY/3XIzi9JGsx9bWExXdZlK/IOWwUNwY+/x/Isif0B8OnVA=; 24:h8Igpn7fttyMNhTEplvOtBxmXIamZJO0R/26XrxGv75zurX5Sv7aNsIlgT/zQuzZocVA8H0HbG3v1a6kUC18ukmxe4GZ3LrVsZC55RDJzLk=; 7:ZBGZPFozJN3eBMB7m6rGyvmf7+eTYjjv/U4p6C3kfeNY1x4P4eGR94NWb8/k9vOhKDAqGg2MlEVsZ8WZh/i/Hkqs0e2qtfIYVBjzaJyvmGv2LUK8E/tBTmqKGWbtSTWQeRRwk/i40ULULu1OZpIar/MLnQCxvck1K+IrqyFTENqmO+LJbSlU7v6C05337RTi9MDSIwhHM7zO0Rjohy9gNKCvNPLTVrQHEmg/G26op/nl9+0oBxSrOLiFkuKcH/gQ x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-ms-office365-filtering-ht: Tenant x-ms-office365-filtering-correlation-id: 3eab7c3e-7a09-43ef-2be8-08d54921da89 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(48565401081)(4534020)(4602075)(7168020)(4627115)(201703031133081)(201702281549075)(5600026)(4604075)(3008031)(2017052603307)(7153060); SRVR:HE1PR08MB0923; x-ms-traffictypediagnostic: HE1PR08MB0923: nodisclaimer: True x-microsoft-antispam-prvs: x-exchange-antispam-report-test: UriScan:(180628864354917)(228905959029699); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(6040470)(2401047)(8121501046)(5005006)(3002001)(93006095)(93001095)(10201501046)(3231023)(944501041)(6055026)(6041268)(20161123562045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123558120)(20161123560045)(20161123564045)(6072148)(201708071742011); SRVR:HE1PR08MB0923; BCL:0; PCL:0; RULEID:(100000803101)(100110400095); SRVR:HE1PR08MB0923; x-forefront-prvs: 05299D545B x-forefront-antispam-report: SFV:NSPM; SFS:(10009020)(39860400002)(39380400002)(376002)(346002)(366004)(396003)(13464003)(189003)(199004)(6246003)(4326008)(74316002)(2950100002)(25786009)(478600001)(97736004)(7696005)(6506007)(76176011)(229853002)(86362001)(53936002)(3280700002)(7736002)(102836004)(305945005)(66066001)(3660700001)(2501003)(575784001)(5660300001)(5250100002)(551544002)(106356001)(33656002)(2900100001)(68736007)(105586002)(110136005)(72206003)(55236004)(14454004)(6116002)(8936002)(99286004)(316002)(81166006)(53546011)(59450400001)(2906002)(55016002)(53946003)(9686003)(3846002)(81156014)(6436002)(54906003); DIR:OUT; SFP:1101; SCL:1; SRVR:HE1PR08MB0923; H:HE1PR08MB2809.eurprd08.prod.outlook.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; received-spf: None (protection.outlook.com: arm.com does not designate permitted sender hosts) spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: arm.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3eab7c3e-7a09-43ef-2be8-08d54921da89 X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Dec 2017 09:53:28.9933 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: f34e5979-57d9-4aaa-ad4d-b122a662184d X-MS-Exchange-Transport-CrossTenantHeadersStamped: HE1PR08MB0923 Subject: Re: [dts] [PATCH] framework: enable AArch64 and add more options for libvirt X-BeenThere: dts@dpdk.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: test suite reviews and discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Dec 2017 09:53:31 -0000 Marvin, > -----Original Message----- > From: Liu, Yong [mailto:yong.liu@intel.com] > Sent: Friday, December 22, 2017 16:10 > To: Herbert Guan ; dts@dpdk.org > Cc: Phil Yang ; Jianbo Liu > Subject: RE: [dts] [PATCH] framework: enable AArch64 and add more options > for libvirt >=20 > Herbert, thanks for your patch. I'm not very clear about what is must fo= r > AArch64 support and what is for enabling your environment. > Please separate them in different functions. To make it more clear, are you requesting to split this patch into smaller = ones (maybe around 6 patches)? >=20 > Thanks, > Marvin >=20 > > -----Original Message----- > > From: dts [mailto:dts-bounces@dpdk.org] On Behalf Of Herbert Guan > > Sent: Tuesday, December 05, 2017 1:46 PM > > To: dts@dpdk.org > > Cc: phil.yang@arm.com; jianbo.liu@arm.com; Herbert Guan > > > > Subject: [dts] [PATCH] framework: enable AArch64 and add more options > > for libvirt > > > > 1) Add the libvirt support on AArch64 platforms > > 2) Add 'loader' and 'nvram' option support in 'os' section > > 3) Add 'opt_bus', 'opt_dev' and 'opt_controller' option support in > > 'disk' section. Change 'type' to 'opt_format' to get aligined > > with option naming in qemu_kvm. > > 4) Add serial 'serial_option' option support for serial console > > 5) Add domain parsing in __parse_pci() > > 6) Function add_vm_device() supports both VF (new) and PF devices now > > > > Signed-off-by: Herbert Guan > > --- > > conf/vm_power_manager.cfg | 4 +- > > framework/qemu_libvirt.py | 152 > > ++++++++++++++++++++++++++++++++++++++--- > > ----- > > 2 files changed, 128 insertions(+), 28 deletions(-) mode change > > 100644 =3D> 100755 conf/vm_power_manager.cfg mode change 100644 =3D> > > 100755 framework/qemu_libvirt.py > > > > diff --git a/conf/vm_power_manager.cfg b/conf/vm_power_manager.cfg > old > > mode 100644 new mode 100755 index 9c8f87a..7d5d9b6 > > --- a/conf/vm_power_manager.cfg > > +++ b/conf/vm_power_manager.cfg > > @@ -7,7 +7,7 @@ > > # size: 4096 > > # disk > > # file: absolute path to disk image > > -# type: disk image format > > +# opt_format: [ raw | qcow2 | ... ] #disk image format > > # login > > # user: user name to login into VM > > # password: passwork to login into VM > > @@ -25,7 +25,7 @@ cpu =3D > > mem =3D > > size=3D4096; > > disk =3D > > - file=3D/storage/vm-image/vm0.img,type=3Draw; > > + file=3D/storage/vm-image/vm0.img,opt_format=3Draw; > > login =3D > > user=3Droot,password=3Dtester; > > [vm1] > > diff --git a/framework/qemu_libvirt.py b/framework/qemu_libvirt.py old > > mode 100644 new mode 100755 index ed8e0e6..18dcd8b > > --- a/framework/qemu_libvirt.py > > +++ b/framework/qemu_libvirt.py > > @@ -57,6 +57,7 @@ class LibvirtKvm(VirtBase): > > > > # disk and pci device default index > > self.diskindex =3D 'a' > > + self.controllerindex =3D 0 > > self.pciindex =3D 10 > > > > # configure root element > > @@ -82,7 +83,12 @@ class LibvirtKvm(VirtBase): > > > > # set some default values for vm, > > # if there is not the values of the specified options > > - self.set_vm_default() > > + arch =3D self.host_session.send_expect('uname -m', '# ') > > + set_default_func =3D getattr(self, 'set_vm_default_' + arch) > > + if callable(set_default_func): > > + set_default_func() > > + else: > > + self.set_vm_default() >=20 > I'm not sure how much is the difference between x86 and AArch64 platform. > Look like somethings are common for example graphic and control interface= . > Could we separate arch related setting in separated function and keep tho= se > common in another function? >=20 How about this: call into common function set_vm_default() first, and then = call arch specific function set_vm_default_'x86_64/aarch64/...'()? =20 > > > > def get_qemu_emulator(self): > > """ > > @@ -98,6 +104,13 @@ class LibvirtKvm(VirtBase): > > """ > > check and setup host virtual ability > > """ > > + arch =3D self.host_session.send_expect('uname -m', '# ') > > + if arch =3D=3D 'aarch64': > > + out =3D self.host_session.send_expect('service libvirtd > > + status', > > "# ") > > + if 'active (running)' not in out: > > + return False > > + return True > > + >=20 > Herbert, libvirt status has been checked in function has_virtual_ability = when > initialize LibvirtKvm object. > Why do this check again? >=20 This is not double check but just modifying the logic of "has_virtual_abili= ty ". It defines the specific checking logic for arm platforms. And if it= 's not 'aarch64' platform, will go with the original logic.=20 > > out =3D self.host_session.send_expect('cat /proc/cpuinfo | gre= p > > flags', > > '# ') > > rgx =3D re.search(' vmx ', out) @@ -201,6 +214,47 @@ class > > LibvirtKvm(VirtBase): > > ',name=3Dorg.qemu.guest_agent.0'}) > > self.qga_sock_path =3D '/tmp/%s_qga0.sock' % self.vm_name > > > > + def add_vm_os(self, **options): > > + os =3D self.domain.find('os') > > + if 'loader' in options.keys(): > > + loader =3D ET.SubElement( > > + os, 'loader', {'readonly': 'yes', 'type': 'pflash'}) > > + loader.text =3D options['loader'] > > + if 'nvram' in options.keys(): > > + nvram =3D ET.SubElement(os, 'nvram') > > + nvram.text =3D options['nvram'] > > + >=20 > Where this function is called? It's called by load_local_config() in virt_base.py. There'll be an entry a= dded in case config file, such as: os =3D loader=3D/usr/share/AAVMF/AAVMF_CODE.fd,nvram=3D/var/lib/libvirt/qemu/n= vram/vm0_VARS.fd; >=20 > > + > > + def set_vm_default_aarch64(self): > > + os =3D ET.SubElement(self.domain, 'os') > > + type =3D ET.SubElement( > > + os, 'type', {'arch': 'aarch64', 'machine': 'virt'}) > > + type.text =3D 'hvm' > > + ET.SubElement(os, 'boot', {'dev': 'hd'}) > > + > > + features =3D ET.SubElement(self.domain, 'features') > > + ET.SubElement(features, 'acpi') > > + ET.SubElement(features, 'gic', {'version': '3'}) > > + > > + ET.SubElement(self.domain, 'cpu', > > + {'mode': 'host-passthrough', 'check': 'none'}) > > + > > + # qemu-kvm for emulator > > + device =3D ET.SubElement(self.domain, 'devices') > > + ET.SubElement(device, 'emulator').text =3D self.qemu_emulator > > + > > + # graphic device > > + #ET.SubElement(device, 'graphics', { > > + # 'type': 'vnc', 'port': '-1', 'autoport': 'yes'}= ) > > + # qemu guest agent > > + self.add_vm_qga(None) > > + > > + # add default control interface > > + if not self.__default_nic: > > + def_nic =3D {'type': 'nic', 'opt_hostfwd': ''} > > + self.add_vm_net(**def_nic) > > + self.__default_nic =3D True > > + > > def set_vm_default(self): > > os =3D ET.SubElement(self.domain, 'os') > > type =3D ET.SubElement( > > @@ -273,17 +327,44 @@ class LibvirtKvm(VirtBase): > > return False > > > > ET.SubElement(disk, 'source', {'file': options['file']}) > > - if 'type' not in options: > > + if 'opt_format' not in options: > > disk_type =3D 'raw' > > else: > > - disk_type =3D options['type'] > > + disk_type =3D options['opt_format'] > > > > ET.SubElement(disk, 'driver', {'name': 'qemu', 'type': > > disk_type}) > > > > + if 'opt_bus' not in options: > > + bus =3D 'virtio' > > + else: > > + bus =3D options['opt_bus'] > > + if 'opt_dev' not in options: > > + dev =3D 'vd%c' % self.diskindex > > + self.diskindex =3D chr(ord(self.diskindex) + 1) > > + else: > > + dev =3D options['opt_dev'] > > ET.SubElement( > > - disk, 'target', {'dev': 'vd%c' % self.diskindex, 'bus': > > 'virtio'}) > > + disk, 'target', {'dev': dev, 'bus': bus}) > > + > > + if 'opt_controller' in options: > > + controller =3D ET.SubElement(devices, 'controller', > > + {'type': bus, > > + 'index': hex(self.controllerindex)[2:], > > + 'model': options['opt_controller']}) > > + self.controllerindex +=3D 1 > > + ET.SubElement(controller, 'address', > > + {'type': 'pci', 'domain': '0x0000', 'bus': > > hex(self.pciindex), > > + 'slot': '0x00', 'function': '0x00'}) > > + self.pciindex +=3D 1 > > > > - self.diskindex =3D chr(ord(self.diskindex) + 1) > > + def add_vm_serial_port(self, **options): > > + if 'enable' in options.keys(): > > + if options['enable'].lower() =3D=3D 'yes': > > + devices =3D self.domain.find('devices') > > + serial =3D ET.SubElement(devices, 'serial', {'type': '= pty'}) > > + ET.SubElement(serial, 'target', {'port': '0'}) > > + console =3D ET.SubElement(devices, 'console', {'type': > > 'pty'}) > > + ET.SubElement(console, 'target', {'type': 'serial', > > 'port': '0'}) > > >=20 > In qemu kvm module, we are using unix domain socket for serial device. Ca= n > you align to this? > Please let device type should be configurable, we plan to add more types > later. OK, I'll add the unix socket support and make type configurable in the new = version. >=20 > > def add_vm_login(self, **options): > > """ > > @@ -305,14 +386,23 @@ class LibvirtKvm(VirtBase): > > def __parse_pci(self, pci_address): > > pci_regex =3D r"([0-9a-fA-F]{1,2}):([0-9a-fA-F]{1,2})" + \ > > ".([0-9a-fA-F]{1,2})" > > + pci_regex_domain =3D r"([0-9a-fA-F]{1,4}):([0-9a-fA-F]{1,2}):"= + \ > > + "([0-9a-fA-F]{1,2}).([0-9a-fA-F]{1,2})" > > m =3D re.match(pci_regex, pci_address) > > - if m is None: > > - return None > > - bus =3D m.group(1) > > - slot =3D m.group(2) > > - func =3D m.group(3) > > - > > - return (bus, slot, func) > > + if m is not None: > > + bus =3D m.group(1) > > + slot =3D m.group(2) > > + func =3D m.group(3) > > + dom =3D '0' > > + return (bus, slot, func, dom) > > + m =3D re.match(pci_regex_domain, pci_address) > > + if m is not None: > > + bus =3D m.group(2) > > + slot =3D m.group(3) > > + func =3D m.group(4) > > + dom =3D m.group(1) > > + return (bus, slot, func, dom) > > + return None > > > > def add_vm_device(self, **options): > > """ > > @@ -325,35 +415,45 @@ class LibvirtKvm(VirtBase): > > 'mode': 'subsystem', 'type': 'pci', > > 'managed': 'yes'}) > > > > - if 'pf_idx' not in options.keys(): > > - print utils.RED("Missing device index for device option!!!= ") > > - return False > > - > > - pf =3D int(options['pf_idx']) > > - if pf > len(self.host_dut.ports_info): > > - print utils.RED("PF device index over size!!!") > > + if 'pf_idx' in options.keys(): > > + pf =3D int(options['pf_idx']) > > + if pf > len(self.host_dut.ports_info): > > + print utils.RED("PF device index over size!!!") > > + return False >=20 > Previous check sentence is incorrect:) Should be "if pf >=3D > len(self.host_dut.ports_info):". > Most of suites are just using option "opt_host" to specify the pci addres= s of > device. So I'm planning to remove pf_idx option, what's your idea for tha= t? >=20 Fully agree switching to "opt_host" to specify the PCI address. This can m= atch the function name "add_vm_device" better, not limited to pf/vf NIC dev= ices. 'vf_idx' was added just because it only support pf before, but this = is quite complex and not aligned with vf_xx suites. I can make such change= if decided to go. Could you confirm that? > > + pci_addr =3D self.host_dut.ports_info[pf]['pci'] > > + elif 'vf_idx' in options.keys(): > > + vf =3D int(options['vf_idx']) > > + if vf > len(self.host_dut.ports_info): > > + print utils.RED("VF device index over size!!!") > > + return False > > + if 'vf_id' in options.keys(): > > + vf_id =3D int(options['vf_id']) > > + else: > > + vf_id =3D 0 > > + pci_addr =3D > > self.host_dut.ports_info[vf]['sriov_vfs_pci'][vf_id] > > + else: > > + print utils.RED("Missing pf/vf device index for device > > option!!!") > > return False > > > > - pci_addr =3D self.host_dut.ports_info[pf]['pci'] > > > > pci =3D self.__parse_pci(pci_addr) > > if pci is None: > > return False > > - bus, slot, func =3D pci > > + bus, slot, func, dom =3D pci > > > > source =3D ET.SubElement(hostdevice, 'source') > > ET.SubElement(source, 'address', { > > - 'domain': '0x0', 'bus': '0x%s' % bus, > > + 'domain': '0x%s' % dom, 'bus': '0x%s' % bus, > > 'slot': '0x%s' % slot, > > 'function': '0x%s' % func}) > > if 'guestpci' in options.keys(): > > pci =3D self.__parse_pci(options['guestpci']) > > if pci is None: > > return False > > - bus, slot, func =3D pci > > + bus, slot, func, dom =3D pci > > ET.SubElement(hostdevice, 'address', { > > - 'type': 'pci', 'domain': '0x0', 'bus': '0x%s= ' % > > bus, > > - 'slot': '0x%s' % slot, 'function': '0x%s' % > > func}) > > + 'type': 'pci', 'domain': '0x%s' % dom, 'bus': > > + '0x%s' % > > bus, > > + 'slot': '0x%s' % slot, 'function': '0x%s' % func}) > > # save host and guest pci address mapping > > pci_map =3D {} > > pci_map['hostpci'] =3D pci_addr @@ -397,7 +497,7 @@ class > > LibvirtKvm(VirtBase): > > pci =3D self.__parse_pci(options['opt_addr']) > > if pci is None: > > return False > > - bus, slot, func =3D pci > > + bus, slot, func, dom =3D pci > > ET.SubElement(qemu, 'qemu:arg', > > {'value': 'nic,model=3De1000,addr=3D0x%s' % = slot}) > > else: > > -- > > 1.8.3.1 Thanks, Herbert