Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.

DPDK patches and discussions
 help / color / mirror / Atom feed

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
@ 2015-08-10 16:16 Zhang, Helin
  0 siblings, 0 replies; 7+ messages in thread
From: Zhang, Helin @ 2015-08-10 16:16 UTC (permalink / raw)
  To: Xu, Qian Q, dev

Sorry for repeat! I should have removed the useless pieces!


> +Hardware platform Network Interface Card Essential requirements:
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ~~~~
> +
> +1. Get an high end Intel(r) NIC, e.g: Intel(r) XL710.
> +
> +2. Make sure each NIC has flashed the latest version of NVM/firmware, 
> +if there
> is.
> +
> +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 
> +slots
> can't provide enough bandwidth for 2x10G and above.
Pci gen2 x 8 may not provide enough, pci gen2 x 16 may support that.

> +
> +4. When inserting NICs to the PCI slots, be careful about the NUMA. 
> +If you will
+
> +Example
> +-------
> +
> +Below is an case of running dpdk l3fwd sample to get high performance 
> +with
> Intel platform and NIC.
> +
> +**Note**: The scenario is to get best performance with two 
> +Intel(r)XL710 40G
> ports. See below Figure1 as the performance test setup.
> +
> +.. figure:: img/pf_performance_test_setup.*
> +
> +**Figure 1. PF_Performance_Test_setup**
> +
> +
> +1. Insert two NICs(Intel(r)XL710) into the platform, and use one port 
> +per card to
> get best performance. The reason using two NICs is the PCIe Gen3's limitations.
> **Note**: As PCIe Gen3 can't provide 80G bandwidth for two 40G ports, 
> but two
Pci gen3 x 8 cannot provide ....

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
  2015-08-10 16:10   ` Zhang, Helin
@ 2015-08-10 20:37     ` Qiu, Michael
  0 siblings, 0 replies; 7+ messages in thread
From: Qiu, Michael @ 2015-08-10 20:37 UTC (permalink / raw)
  To: Zhang, Helin, Thomas Monjalon, Xu, Qian Q; +Cc: dev

On 2015/8/10 9:11, Zhang, Helin wrote:
>
>> -----Original Message-----
>> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
>> Sent: Monday, August 10, 2015 6:38 AM
>> To: Xu, Qian Q
>> Cc: dev@dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how
>> to get DPDK high perf on Intel platform.
>>

[../..]

>>> +1. For Intel® 40G NICs, special configurations should be set before compiling it,
>> as follows. **Note**: This is very important::
>>> +
>>> +      for at least DPDK release 1.8, 2.0 and 2.1, in
>> <dpdk_folder>/config/common_linuxapp
>>> +      CONFIG_RTE_PCI_CONFIG=y
>>> +      CONFIG_RTE_PCI_EXTENDED_TAG=”on”
>> Please insert it in a i40e doc instead of here. Then you can reference it.
> Good idea!

Yes, agree, actually, for RRC(fm10k), this configure is meaningless.

Thanks,
Michael 
> Regards,
> Helin
>
>

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
  2015-08-10 13:37 ` Thomas Monjalon
@ 2015-08-10 16:10   ` Zhang, Helin
  2015-08-10 20:37     ` Qiu, Michael
  0 siblings, 1 reply; 7+ messages in thread
From: Zhang, Helin @ 2015-08-10 16:10 UTC (permalink / raw)
  To: Thomas Monjalon, Xu, Qian Q; +Cc: dev



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Thomas Monjalon
> Sent: Monday, August 10, 2015 6:38 AM
> To: Xu, Qian Q
> Cc: dev@dpdk.org
> Subject: Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how
> to get DPDK high perf on Intel platform.
> 
> Hi,
> 
> Thanks for adding more helpful documentation.
> 
> 2015-08-10 14:34, Qian Xu:
> > Signed-off-by: Qian Xu <qian.q.xu@intel.com>
> 
> The Sign-off line must be below the commit message.
> 
> > Add a new guide doc under guides folder. This document is a
> > step-by-step guide about how to get high performance with DPDK on Intel's
> platform and NICs.
> > It is designed for users who are not familiar with DPDK but would like
> > to measure the best performance. It contains step-by-step instructions to set
> the platform and NICs to its best performance.
> > The document will add more sections with the DPDK features' increment.
> > Currently, the document has only one section about PF performance test setup,
> and will add below cases in near future.
> >
> > *  VF performance tuning.
> > *  Vhost/virtio performance tuning.
> > *  new features....
> 
> Good idea, thanks
> 
> > --- /dev/null
> > +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
> > +  <image
> > +     width="609.375"
> > +     height="349.6875"
> > +     preserveAspectRatio="none"
> > +     xlink:href="data:image/png;base64,
> 
> This is a PNG picture embedded in SVG.
> Please draw only some true SVG files.
> Inkscape is the preferred tool.
> 
> > +++ b/doc/guides/perf_tuning_guide/performance_tuning.rst
> 
> This file is Intel-specific. So it should be renamed
> 	doc/guides/perf_tuning/intel_platform.rst
Agree with this.

> 
> > +This document is a step-by-step guide for getting high DPDK performance
> with Intel's platform and NICs.
> 
> Could it be specific to Intel's CPU but allows some chapters for other NICs tuning
> (e.g. Chelsio, Cisco, Mellanox)?
> Ideally, the NIC tuning should refer to sections in doc/guides/nics/.
Yes, we want to have it as common as possible, and take Intel(r) NIC/Server as an example.

> 
> > +1. Get an high end Intel® NIC, e.g: Intel® XL710.
> 
> "Get an high end NIC that can sustain a high packet rate."?
Usually yes, talking about 1G is meaningless now, and 10G @ line rate is not so challenging
and complicated.
40G is more challenging, and complicated, so take it as an example.

> 
> > +2. Make sure each NIC has flashed the latest version of NVM/firmware, if
> there is.
> > +
> > +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots
> can't provide enough bandwidth for 2x10G and above.
> 
> Could you describe how to get this characteristics with lspci?
> 
> > +4. When inserting NICs to the PCI slots, be careful about the NUMA. If you will
> use 2 or more ports from different NICs, please make sure these NICs on the
> same CPU socket.
> 
> Could you describe how to know the CPU-PCI mapping? lstopo, lspci?
Below there already have the descriptions.

> 
> > +BIOS settings:
> > +~~~~~~~~~~~~~~
> > +
> > +1. To be sure, reset all the BIOS settings to default.
> > +
> > +2. Disable all power saving options, and set all options for best performance.
> > +
> > +3. Disable Turbo to ensure the performance scaling with core numbers
> increment.
> > +
> > +4. Set memory frequency to the highest number, NOT auto.
> > +
> > +5. Disable all Virtualization options when you test physical function of NIC, and
> turn on VT-d if you wants to use VFIO.
> 
> Could you describe usage of dmidecode to check this, please?
> 
> > +1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 - 64
> bits.
> 
> Why not describe build options and run-time options to use with any Linux?
Actually there is nothing special for performance testing, comparing to running
DPDK basically.
No need to duplicate here, as all the setup requirements are described in GSG.

> 
> > +3. Make sure the widely used and fully validated version of kernel installed,
> e.g. 3.18.
> 
> Reference to release notes required.
Agree with this.

> 
> > +4. Make sure the required components are enabled for some old version of
> kernels, before rebuilding the kernel. The kernel may need to be rebuilt if any
> DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org
> for more details.
> 
> Please use sphinx reference.
> 
> > +1. For Intel® 40G NICs, special configurations should be set before compiling it,
> as follows. **Note**: This is very important::
> > +
> > +      for at least DPDK release 1.8, 2.0 and 2.1, in
> <dpdk_folder>/config/common_linuxapp
> > +      CONFIG_RTE_PCI_CONFIG=y
> > +      CONFIG_RTE_PCI_EXTENDED_TAG=”on”
> 
> Please insert it in a i40e doc instead of here. Then you can reference it.
Good idea!

Regards,
Helin


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
  2015-08-10 16:00 ` Zhang, Helin
@ 2015-08-10 16:01   ` Thomas Monjalon
  0 siblings, 0 replies; 7+ messages in thread
From: Thomas Monjalon @ 2015-08-10 16:01 UTC (permalink / raw)
  To: Zhang, Helin; +Cc: dev

Hi Helin,
Your comment is unreadable. Please remove useless lines of your reply.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
  2015-08-10  6:34 Qian Xu
  2015-08-10 13:37 ` Thomas Monjalon
@ 2015-08-10 16:00 ` Zhang, Helin
  2015-08-10 16:01   ` Thomas Monjalon
  1 sibling, 1 reply; 7+ messages in thread
From: Zhang, Helin @ 2015-08-10 16:00 UTC (permalink / raw)
  To: Xu, Qian Q, dev



> -----Original Message-----
> From: dev [mailto:dev-bounces@dpdk.org] On Behalf Of Qian Xu
> Sent: Sunday, August 9, 2015 11:35 PM
> To: dev@dpdk.org
> Subject: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to
> get DPDK high perf on Intel platform.
> 
> Signed-off-by: Qian Xu <qian.q.xu@intel.com>
> 
> Add a new guide doc under guides folder. This document is a step-by-step guide
> about how to get high performance with DPDK on Intel's platform and NICs.
> It is designed for users who are not familiar with DPDK but would like to measure
> the best performance. It contains step-by-step instructions to set the platform
> and NICs to its best performance.
> The document will add more sections with the DPDK features' increment.
> Currently, the document has only one section about PF performance test setup,
> and will add below cases in near future.
> 
> *  VF performance tuning.
> *  Vhost/virtio performance tuning.
> *  new features....
> 
> 
> diff --git a/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
> b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
> new file mode 100644
> index 0000000..50ce92d
> --- /dev/null
> +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
> @@ -0,0 +1,375 @@
> +<?xml version="1.0" encoding="UTF-8" standalone="no"?>
> +<!-- Created with Inkscape (http://www.inkscape.org/) -->
> +
> +<svg
> +   xmlns:dc="http://purl.org/dc/elements/1.1/"
> +   xmlns:cc="http://creativecommons.org/ns#"
> +   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
> +   xmlns:svg="http://www.w3.org/2000/svg"
> +   xmlns="http://www.w3.org/2000/svg"
> +   xmlns:xlink="http://www.w3.org/1999/xlink"
> +   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
> +   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
> +   id="svg2"
> +   version="1.1"
> +   inkscape:version="0.91 r13725"
> +   width="609.375"
> +   height="349.6875"
> +   viewBox="0 0 609.375 349.6875"
> +   sodipodi:docname="Performance_test_setup.svg">
> +  <metadata
> +     id="metadata8">
> +    <rdf:RDF>
> +      <cc:Work
> +         rdf:about="">
> +        <dc:format>image/svg+xml</dc:format>
> +        <dc:type
> +           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
> +        <dc:title></dc:title>
> +      </cc:Work>
> +    </rdf:RDF>
> +  </metadata>
> +  <defs
> +     id="defs6" />
> +  <sodipodi:namedview
> +     pagecolor="#ffffff"
> +     bordercolor="#666666"
> +     borderopacity="1"
> +     objecttolerance="10"
> +     gridtolerance="10"
> +     guidetolerance="10"
> +     inkscape:pageopacity="0"
> +     inkscape:pageshadow="2"
> +     inkscape:window-width="1596"
> +     inkscape:window-height="1108"
> +     id="namedview4"
> +     showgrid="false"
> +     inkscape:zoom="0.56123077"
> +     inkscape:cx="308.2511"
> +     inkscape:cy="174.84375"
> +     inkscape:window-x="0"
> +     inkscape:window-y="27"
> +     inkscape:window-maximized="1"
> +     inkscape:current-layer="svg2" />
> +  <image
> +     width="609.375"
> +     height="349.6875"
> +     preserveAspectRatio="none"
> +
> +xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAooAAAF1
> CAIAA
> +ADgOlyKAAAAAXNSR0IArs4c6QAAAARnQU1BAACx
> +jwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAEUsSURBVHhe7d0JcBT3nS/w
> v3yA1htqkcx
> +l78qW
> +0AgjLHadxY84owBCCIOgEstOVlbWSSG24hF6rofGu6YCu7AyDxJIOVmPXGULydlF
> VHZjWS+xSVJI
> +HDKHeYy9xGych2BsNDqCtmxhsERCDvARvf/VPd093aM5ejQ9o++npobunp6r0cx
> 3ft3//v+
> +zxsbG
> +CAAAADjJTfJfAAAAcAzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAA
> A4DuIZAADAcRD
> +PAAAA
> +joN4BgAAcBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZA
> ADAcRDPAAAAjoN
> +4BgAA
> +cBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZAADAcRDPA
> AAAjoN4BgAAcBz
> +EMwAA
> +gOMgngEAABwH8QwAAOA4WWNjY3ISIEWysrLkFABAhoo1bVE9AwAAOA7iGQA
> AwHGwcxtST92
> +5jb9G
> +AMgwcX+/oXoGAABwHMQzAACA4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMA
> ADgO4hkAAMBxEM8
> +AMgwcX+AAACO
> +g3gGAABwHMQzAACA4yCeAQAAHCfBPrevNHme9/bJGaaiemwLqVtxYkHbhoY8
> uSxuwfY9RS8
> +Snx0P
> +BU6GPrcBIFOlss9tz47GsdeUy5ZiuTRBQydLV2wvGpjlkfMAAACTyATs3A7UrdieJS6
> 7AmJ
> +R167t
> +pe1XxLQGrcW31/n5xI6jJTT1DWHvfyVrxZ6mITkHAACQqZIdzzRxO3oef4LX1k/4Bj
> pEKlc
> +uW+gf
> +uKysoMb25fN9C6vcdGJGQ2tjC5vQcz8y9hp2dAMAQOazIZ5btynF8YpXuuQyhf91
> b9/CrTU
> +z+MyM
> +htqF/hdfZ+vkzXR3B9jEUKCjj5CBK0E67Q+0VhRXsjUBAAAmNXuPPT9iEq6FM4vkF
> E9lOVF
> +cXfjB
> +hSESPHWOPF7tI+cO0OmLH7gLZorbYXLKguSQ2xcA0kfyjz33Xe6VU7RWvsyOLDM
> z1paTjlO
> +hSESPHWOPF7tI+BA0dJ
> +dWmxZlrU2QAAAJNakuPZvdRXeHanbAV2pantrPvxpaLCdt01y3/0RAe5d20ecZXe
> S5RpAAA
> +ASEY8
> +F1dVXPLWiuPQMxpaq0tefJ4fmX7eW1B9Sh6Hpsld7Om7RMqLXXQ6r7iaXPIXzGD
> TEaDldqY
> +bA1vJ
> +zQoAaSjBbkkAbKAeHMVfo72wYQFSLu6PYfKPPQMAAECMEM8AAACOg3gGAAB
> wHMQzAACA4yC
> +eAQAA
> +HAfxDEl2/bqcAACAqCUYz3xAC9nhNrvw8aZsoY5eJac1zxLx1Gd2brSypjJAVrhg+57
> Ij5P
> +4CkCG
> +h8mTT5KHH5azAAAQNRuq51Cf223lPdtMR4qMxGJwSaPQs+yYpfR5Eu5K04mZv
> WK116o93R0
> +h8mTT5KHH5azAAAQNRuq51Cf223lPdtMR4qMxGJwSaPQs+mPxfG
> +HUk68RVABHNBAfH5UD0DAMTB1p3beUtO7VDGpGJCIz2rMUnDWKluWelJZ9d0
> Ez/rVizqSpR
> +3FLqf
> +PqBJJ2IzGrYsUboeK66qID0XDcFvMZJ0SOIrTG4IZgAAO9h97Nld7CFsKCqezepIz9V
> kG89
> +R/ytr
> +BsqV6paN3Fy5pbGzgrjZarEP5DzO8M+B/d2k5C7DGBsWI0mHJL7CZIVgBgCwTzKa
> hl06z5I
> +BsqV6paN3Fy5pbGzgrjZarEP5DzO8M+40FpY
> +vk/2sM164WYL82a6+9jYkQnhY0hXjROQ8scBcnQiIJgBAOyWjHievUBUtH1Hi5Sd22
> u6+RK
> +293uW
> +t5YvtG63Zap1m3yorG0f+NrMBpZWsZ3eHWRHY2j4DUgSu4M5K2ryDgAAGcrueK
> ZFM5k1T8R
> +t5YvtG63Zap1m3yorG0f+zobof
> +m11kIcv2SNPZJ3wDHTE1Igs1DYu8G5xm8zbSqT4dJAkqZgCAZLI1nodOlm5TRnR2L
> /WRo+s
> +MATx0
> +sklpI0aJA8NFBbP9A5fFkoQF6satrSFBsQbz8eO0KB7nEgfDI6TpBQDAgg3xHNrtXM
> uaNCu
> +7lLUj
> +PdOLPBWqQ678vJeUb+IFrqtmmae7I4aW26rwlttDV3rIJbnznF9kgW7Sxlsv8RUmj
> 4MHyf7
> +PdOLPBWqQ678vJeUb+9SaqY
> +x8a7AIxv9MzLzQYvnxmVN2r0HzG7gd/7SL+cA0gZjPcMcWlrI9u3k8FBORtBWRk5d
> kxOW1C
> +PJY/7
> +1xj9mkBNus1FA/dQUE6bca2qXzlXTkdcW78iQALi/hgmVj0//XRowq5pSAu1tWRgg
> OzdS/L
> +z5RK7
> +ddXRv2qmzqIPGqfBIfhU6z8i0jZ3cU29Qc3iXHZL8BDKYkgbiVXPyUhTJHTaiVxJx1c9
> 03Be0+Pr
> +PbX2QGmRt6RzrEW0JzBZ0zHWr2fXjY3J+8USMydvriToP/Ly6P2PLsqRs+FGz7z8Vs
> 6jaln
> +PbX2QGmRt6RzrEW0JzBZ0zHWr2fXjY3J+MqueR
> +xTWGe9CV2k/nonoGu8T9MUT1DAlLQiUdvNBDSMk8F3HNKyGk50KkHZYOQn+o
> FBSwnI5mrz9
> +D1/P5
> +yDvvyFlIyNyVkbKZylkUymZq7sr6+vB70JXqkc3gADj2DLYKr6Tjqp6DTUrRrJTRDbyz
> Vod
> +yDvvyFlIyNyVkbKZylkUymZq7sr6+Xz/Td
> +q+iPlqgqaZrndHPNn09WryarVrFr+0yy6hnAiVJUPUPyifOYli9nl/p6xxdaNlXSatHM
> y+i
> +q+iPlqgqaZrndHPNn09WryarVrFr+0FG0l
> +vW4du6b/r7SMrqwkOTnkq19ld756ld8MycNabgs4Ig0OhOrZ0fbvZ9/v2i/q7Gzy7L
> NkwwY
> +562ii
> +kqY5HU/LbV4/+/mkRz30nMxysL99ST3Ze6iGVukHvRUPtRPyV984x2dDjuye+nXR
> AR5Z3Lj
> +3pOs/
> +lgQfO1kve8kxVM/Sbbtvufmxr305T1bSR3azu2jvSNObxripBx4gDz1EqqpYbR2XSV
> k9R98
> +e22TN
> +3LBD0QAJivtjmHA8D50srT0qvkUJme3Z8ZUW9ww+QvPz3j65lC73tWm6+tLdhY2H
> oe99k93
> +3LBD0QAJivtjmHA8D50srT0qvkUJme3Z8ZUW9ww+QvPz3j65lC73tWm6+3fK3o
> +9ivi4+hEv2bU/K9ktc3sbVWHwJpo9Hv7s581L6LeeIN9daeH48fZ/u2Iov/zTVre+Ot
> XvU6
> +9ivi4+K8//B
> +V+MSCUqz88juerK5eaVcg+X3c3ed9Mnu6IL9Q665ur8w63heSqb+M/l999eq5e5u
> Y/wvX86
> +V+MSCUqz88juerK5eaVcg+X3c3ed9Mnu6IL9Q665ur8w63heSqb+2UgT0
> +PmLXN43qWCCe9cxOl+ItwUbkjAonVoFt4v8Y0jsk4Lyn/GnPKTlDZ30vXeYTl32Pa5
> af+jE
> +PmLXN43qWCCe9cxOl+p/3Fn
> +aFpz08XznRflpEJ7X+vHMYp+zajRB3n89V45kwK1tfT/0vxSVibXyQzyb5G+sfHI9aJY
> MxY
> +aFpz08XznRflpEJ7X+vHMYp+Xmx7c
> +1UWvG16i/929L3g2HOaL+17a8ELorzO0XHV41xfoCod3TZm9QlxuuY2+wlO3yNldN
> 9P/rNv
> +orZ5b
> +bhm7eTq/9bZddyy9OPADfkfV3r3y/3XcS3b2WFXV2BtvyDuOR24sesdJpO/wCy+8
> cLhPzin
> +Ml1Ij
> +b7W/8EL7WyN8hq/FmK0JEBf5IYz9Y5jYsWfWS5d28KjiBtNRKLSjTG4769mh6RA7r7
> gy+ho
> +30kjP
> +euozDgXqPEonYp6Tym/qQB29ezt9EMNySh2jek/TRbmIC41drRnMQ/c4rIcy9sIM6
> 8Tv7bf
> +lRLho
> +OspMo4vKsDz8ojIsT+RyS87Ofzi6uTKL/MM+Mj+LzP8m+X4Vv+ke8v1vhlZTl9+SU
> zF1TsX
> +OspMo4vKsDz8ojIsT+UHH9W
> +FXnzm+SWL5M/ftj90aXuT39fyF9aHvmU/0sGsm5h//zx91s/+YR8+ttvkJvZn+zlX5C
> CjR/
> +FXnzm+9V/Ov
> +Pr/ugu+Hl9gaNTXsoEU0rl9nl/vuk7Ngk5HT7fwYtFXZDZACicVz3owScnbNeFEUbD
> 8hx8k
> +wxnm8
> +xhnpWfOMeaSqVo6l0Vmg7QP8kvfoTD5ixxOavsHlMJR8/a+Qo2f5QupKk0ddbhj
> M45J3oJg
> +v5z2Y
> +tonHrPZ0d9Spu+/jNSnbBtEEjnyx36dXt5LpFVNmr79p6vezpg19+mmByFEyRdemi
> y6/aQr
> +tonHrPZ0d9Spu+7T/30
> +avdHH37jj2IpM0CmyCnq5mn/QegKNK1vyCWS+mg3f0zuuPbRjd++ee7ak//y38V
> fOTf8u5t
> +ZQkeD
> +BvOrr0ab5ZNUTm4uzdtRfV+do6MjhOTm6o8q86Zh+j3bSo8m2LMNqZdgy+3iFh5
> XsmTUlaG
> +BvOrr0ab5ZNUTm4uzdtRfV+do6MjhOTm6o8q86Zh+j3bSo8m2LMNqZdgy+hvriL
> +XpzV+RofpmLockKZNd5IzybPSIorlfUrly3Ujr3hqRUHlWesLVfG5GDDbS3cqvYZXru
> QT8j
> +XpzV+n1S73
> +v/i60pPVbN/XivnEzAWF6mMWV1WQnov64UBiF6FGojeNjWXOJQ6GR0jskvfx5e
> 4bw3ufqfl
> +G4Dd5
> +Y7977HufXz9ldsWUaeRndFZd7Xebz317H1tOL7d//6Z1ZGw/eeA75JPfrPvGNLbw5
> ttY24d
> +PPyU3
> +3c5np4pXSm66TT7a2DX2i3JK9q0r7l/617/+7pT/PnB3fvY7g9e/+o8D1x/j7bcjo//rx4
> 4
> +hm8eT
> +k8Pi+XS7pjV2/xEewrk5Vo2+XKt4KJudBg2QKomfWDWjoZXWi+xCy9MiTULzISCf8
> BUSOYY
> +k8Pi+VlTfT
> +LfdyxyD6kZ5NnpGQrl3q3dVqmFIGpTYonFkkp/S0y9m7mAgNDXIiXF2dnAD75G30
> iXbaeRs
> +P0bSm
> +l82Gc5Bd9S18Ob/43GTlZt762t3Ml5T9eSH5iJDfb6alM7+0fPKJdnbzzDnk2W9t/nAg
> r3v
> +fF/+T
> +3eWf32yZN+f2W4+fubbng+JxTkWjt9K6efp0OQvW5q5cxf4fg4f4/mqG77N2rTKW
> xKxbEgH
> +3eWf32yZN+f2W4+fubbng+FMjiQ
> +nec9V36t3N13uVfOCaLW/JE8Tsx2hl86H2M8RzvSs6R7xmD7njUDyrDTO5RqOALj6
> 1dolye
> +4DyBq
> +ZWVk82Y5rbV6dZqcWBU72SIiCvIOjuCvn8OORl+czfPYzBwazM+yE8K9Xl31S7O5ec
> tddOI7+y4N
> +13jEQhP0/rRujpzfEMJyl0e0gu2zRgRDmkksnv2vaNtAdf37UX949el+pLPikneHqKq
> LW3Y
> +spNVw
> +6LisfgRoe+iekZCCGfxzeqWpTVs9m2ENys7uVI9Dq9U2b5KmLGePo63Ok2rXLtLVF
> drLPX++XAJO
> +IqvnB2bLeS2rYFZVlU2/b95twx9+3PSZh+SicPQXGbI5RqHSmMI+a0hDicWz+5Heg
> hNy1/G
> +K7axO
> +NTtLuHJLtafvaJEIcvcjY23lPaEhos+dJ7Efox2v5bb6jK6aryiHxn9Easvd47TYKm5pKyd
> +NTtLuHJLtafvaJEIcvcjY23lPaEhos+yjOoT
> +Cx5Xq23t2NXPewuq9SdqJxf9Zv7FL+Thz0DAvJ4GBxo3mFWNnjvoddvrn1wvqxBL
> Qug96aW
> +Cx5Xq23t2NXPewuq9SdqJxf9Zv7FL+tjbz5
> +plwCAJMDeg2D1Etx7xn97Uvc3z8tpmv+9421r6vdkmg7BRuX2i0JDeZvfpMdgGCp
> HN2DFH/
> +plwCAJMDeg2D1Etx7xn97Uvc3z8tpmv+l3DuD
> +119d86uqHV+Wiyh6/1dfJYcOsf4+acjTtI9RIhtWdtch+udQu/pIg+46YuyWBCDJ4v4Y
> 2nn
> +119d86uqHV+sGSBN
> +lTTuVRt8HTzQfXr7+qmr2vlX/NBzqyrqj/CVohBeMesfzVLdI2x/TMtIibybsHcv23nyE
> N/
> +lTTuVRt8HTzQfXr7+pvX8/
> +XzRhRvuDI0pzKmUcZSp46OUz+lOWACA5UD1D6jm2ep769YFn/C3znqv4ydruZqJ2
> uF1IF27
> +XzRhRvuDI0pzKmUcZSp46OUz+sk7N/
> +V1Pxb+2yI+51/9rdOsW4WjTV89Vrn+Ysfzt7yk2j857P3vevbBHN5tpafiM7V4idBU9
> jP8Y
> +V1Pxb+2yI+j0Als
> +WF6Daktn1hv13H5aUROHd0uteeWaae1ShXhjplBng21QPQPET1s9y0Xc4satG+eS
> 1WvZIeF
> +WF6Daktn1hv13H5aUROHd0uteeWaae1ShXhjplBng21QPQPET1s9y0Xc4satG+gc
> EAs
> +JKQvwHt3X8zv1bxWTtz4QcXNU0xWi8b0aTeXLZp2/aM/HryfR3JjYyibKdHV9kQX0
> BLv0UO
> +cNCxO
> +KR5B+QwwARDPAFEqeMbPI5xGsjpOhokoVzNatugz9PonH+aR5mby9NNiobRqF
> bs+cYLPTAy
> +KR5B+QwwARDPAFEqeMbPI5xGsjpOhokoVzNatugz9PonH+aR5mby9NNiobRqF
> bs+17y2x
> +l5u4CjOvmFQbd7OTsJT+wvRnZAGkDuIZgPRsX8+60aYXr5+sXFrCZncflDdKrvqlATd
> fZ06
> +l5u4CjOvmFQbd7OTsJT+wvRnZAGkDuIZgPRsX8+60aYXr5+ko9HG
> +1SweLVxVGety5PiZayZnta/m/aNEHtXKZjlzXbm8K2rR6aVI5/6+IM0xQ9eYAJAUO
> PYMqZf
> +iY8+O
> +kbP87avXPh346cL8OzW9eAsFBWyE0V/8IlJfr2ES3LDqsVlxJFbXljuj8Dcqh3oW7xLH
> nsE
> +2cX8M
> +E4xnwyjLrIcvZTQq402ELGT9YJuND921a/sa2bZGo6J6bIvozlrgD1gw/kL+aPy55AJu
> nEG
> +mtQyv
> +PPGho9lIG61iUn1T/leythHjizQxYe86lRDPwsNP9e0/fvXZv/8L79+GdXEiTtt69lnWK
> Dx
> +q2LBR
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+o
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+C
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+S
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+d
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+F
> +MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWs
> AiRt+9
> +QtcbCRusQk4qAvtZ0qudfHE0C7XP21a9tXSclAq98h2zvLWvxNRPF01KzUagUdrRSl
> OZPZp
> +mtCvW
> +Exm/PRoT9a4htR5axvZvn/iv34pZnWXL2PWEHn7OFLQeFl1vN2tGydCT3XRLuYsfR
> DZD6tl
> +67Dlv
> +yakd2tGcwkQ5PrQ1T8WsUG+dNPnaPvBU6OsMf6C1sLzz8dn+owFlNZsGmWbG
> G/V5V0DsCfC
> +yakd2tGcwkQ5PrQ1T8WsUG+dNPnaPvBU6OsMf6C1sLzz8dn+zLsZ4
> +v2Z8tKtO+VNDM9rVUKCjoFhUuvQu8jEtukJLwbuGVHhg4Z/S6zfP/k7M6pSVseuJ
> 7TuM1pQ
> +v2Z8tKtO+hltHm
> +VGoqi6Pn49D2AYrCGRzB7qZhrNtq6zGpohsfOpJlS33k6DMiLGnyFSzbVMCnFV0nz
> rrLiyt
> +L73X3
> +nTsgXoblINO0rlVzkV/GeWGsDrYc9VmMHr2luHKLug+A7RIPXvwgbLQrvn3oTxllL3
> do0A6
> +rMT/s
> +fNdg5shu0XNIsLlOafY19Nyq3Qf72+ublb9muo5oPjanYomy8KA31ObroJffKnogUR
> 4wJvP
> +fNdg5shu0XNIsLlOafY19Nyq3Qf72+zs6dP
> +u3n4w4/pRS5S5eez4aqGh9llQhj39wYPpUlCKz8qQqmsNsrG4WRIJ8louR0akyo0F
> qSsCyO
> +NDx0d
> +VoC2ttE7siJSGWtZMXRyZ/fs6tIZtFKsLrzUcYrHp+UAU6GhMOXFdHc6L39ZzvEJi1G
> f1ZG
> +ex8N+
> +oOjRwFYz1ZKN7xrCDT13IP/cITaapKt+a/F3WeIGm3cGntIPKLlyszhd6lxjYYmL/oxiH
> Yo
> +91C5v
> +pHn8rXv42c9PDX6PNdjeTCfWq9Eetfvm3Uav337392JW54EH2PUEFdC8kbZLGQZZ
> nG4U7Eu
> +nCloN
> +ZctqWLd3QC/t9hVABkpGPIeGUtYce1brQsvxoaPlZqXkOs+PaBFpqDWDp875C+9d
> yxbOWFs
> ++W8an
> +jYNMW476bDF6dDhW1OqxIwKzvLX8uSKU7xPyriep/lOBe0qVX1d5G5vzv7Wqbv2
> 76yzOWvZ
> +/b//y
> +f2A3sWGhf1IjFrLeSHhm0z+SfBLk233l0pL9p2L9877vHh7PF/4gZnVEm+233+YzMB
> 52Vhi
> +f2A3sWGhf1IjFrLeSHhm0z+Hbkgh
> +Pdkdz/5AK5k1L7qsMhsfOhq8cu0jxiKSXDlw9BLpO1rEY7XoxUuE8GPGloNMR9q5
> bfbDIp5
> +Rn113
> +zQq7V9j2cT/Cn8iww9zArncNYfoGiUhWYe5dJb/sI/eY/xEHm/eRp1idHYW8YjIYazx
> /roT
> +F83/2
> +mB1+/tzn2PUvf8lnkm1uISuXD4mAo9iO7vTomkQ5ilyzOFcsUHPatCbW7CHQwG5
> wSD1b43n
> +mB1+oZOm2
> +iGMhRzM+dDRYnoUdphWNsGSmsktnBWk9QZ/OapDp6HZuq6Ie9bmoYLZ/4LKc4
> fdSDreb3Us
> +iGMhRzM+/4nXJ
> +XdZt5ex512CiR9S73EHvvmJ/9z+9u/M5k+9y//csqmqXq0A+SN9gaAdJcX50QR4id
> m6/M3h
> +dzOpM
> +bPVMU07Xf1banQmcs+hREbVqTgOkFRviObQfmJ/Goz2zVnPsmV5e6YpufOj4sOZ
> R+uRj1bk
> +bPVMU07Xf1banQmcs+4JcmW
> +QaZZnEc16rOrZpmnuyNLHm5n9+KzlvfqkC/seS8p3xRjY67kv+sJlRU1eQdbrFxa8q6
> M52B
> +QaZZnEc16rOrZpmnuyNLHm5n9+z3bfu
> +4f1s+9YF3OwgtNqh2JLmIZbcG9X/IXHsufuhOXUsyFc+9jf7+ZpfJ/8kB8AYCpC7Yv3z
> np+
> +4f1s+fnT3l
> +JhrP1z/6o1ykEq3DBgfZ8BgTQtuaOY3rSTWnURNDWkGvYZB6ccStvX+3B711Fza20
> FS2UbB
> +5d+eq
> +zXE85mf/NvD2hd+/sXe+OM9K5/OfZ03D3nhDNhMbj7ph8TE3wxuno38wSLK4P4
> bJaBoGkGb
> +UWtk2
> +R3avJ4/Fl/cilWlCi1mdid2/HerPI53aV7H22OGvlr8XtMeGdILqGVJPWz1H/oOcDOW
> g74e
> +XnvyX
> +//b+7exn//4v5CKVz0eefJL16/nss3JJRIlsLtH3tJzh0qSnS+MJ2xqolCEFUD0DZIj5+d
> n
> +02rx1
> +2Pz57Pqdd/hMUolxJNVmzayV2EiwP7POUAqd94yyGhwI8QxO1FVHf3HW8V5fgk
> 2lWVmlTTG
> +fH5+2
> +5uf/Cb0efO+GmNURO7cHB/lMUo2OjNBy+X6l1px7/+JcMjKSLvEc6pJEEbn5tnoG
> Gc6RBuf
> +Azm1I
> +PbOd2zSg17R6OnsX7CzylnSOtYjW6ZNh5zaVdf8Zej321iIxq5OTw1pu/+EPJJsV2ZEl
> sLm
> +MzaYy
> +thlV+E58BrvBwTbYuQ0ZprKl1+duXVPk9Xs6ZTZPHmL/9uB7H4lZnfx8dp30Alp0S/Jz
> WUy
> +thlV+E58BrvBwTbYuQ0ZprKl1+Onvl5
> +2nRLEg+l2tad6Q2QWqieIfXMqme+V5uGMyEetXZOqBxMJ5Ubgwf9vz7WMq9s0
> TS5SPXww2T
> +2nRLEg+l2tad6Q2QWqieIfXMqme+/fvLq
> +q6SqSi6xNkk2lxl9A7EIrdpQPUOSxf0xRDxD6pnFMw9n4vOVeL2toYCeJHlTv+vinh9
> f3tu
> +q6SqSi6xNkk2lxl9A7EIrdpQPUOSxf0xRDxD6pnFMw9n4vOVeL2toYCeJHlTv+YX/v
> F
> +2+Ui1ZYtZPdu1nLb65VLrCWwuSK0f6acnV7miRse0SbvMU1ap0M6iftjiJ3b4ERddb
> Rwdvv
> +2+2NTS0
> +dHpI65rJ1DSMuvuOKfR68H2z1mF3382uf/UrPgMm+t/i2Ux/Qij4PuuR029Zts9W1
> 0U2g3O
> +geobU
> +M6uezU2S6rn98MhX/3Gg9osz9jbyMNY6eJBUVrI926++KpdYS3xzqQVm+uzt5S/Z
> WAWLghq
> +7rCEF
> +UD0DZI78O6bSa/NzqyaoaRhFIy2085edepRGZwfn5uir4JycqIfFoPGOE6HBCRDPAI
> 6Tf6f
> +YuZ3C
> +lts0pMTxW7Hfl+8eDvalQ2jl5NIkNr7U/j76OyM3F3uuIY0gnsFZsiKSK2W6ObffSq+
> Hr3w
> +lts0pMTxW7Hfl+sZnWy
> +s8mcOezU5wkYt4qdbyT2BrPBq9LkrKOcuS6Wz4c0BXD/EbYXINc1F/EMaQTHniH1
> 4sjdjP+
> +7LfhS
> +z+B7N94/9JciqnU++1k2KkYgIPv4tKZu2Ng3V/+RI2Rlhh+oNWm5rcBRarBN3B9DV
> M8ATpQ
> +z+vGm9H
> +6JlkeJjPJMncTM9mAKdD9Qypl0CRl7HWb/9V28+uvPrdwqqy6XKRqr6e7NlD9u4lt
> bVyiYU
> +EN6z2
> +/OH0OSE40ZqY3x/VM9gG1TNARpk+7WZ6ffXap2JWZ/Zsdp3c6tnYt8fI6XYMFwE
> wkRDPAE4
> +0O/cW
> +em3eM4nYuZ3cnknUASXZUE/in0j9ejgNfcVqg3M+jd60Ie0gngGcSLQIu/ThJ2JWZy
> KOPRs
> +GlIzl
> +xOE0xxqpY882OADiGcCJxKnPwx+anVs1nR+NnoATqxSsT5JDwYw7b5h3QGIO3ZJ
> A6iGeAZw
> +xOE0xxqpY882OADiGcCJxKnPwx+anVs1nR+o0rFn
> +Ec9Jbrld6CIjpw+HDjdjsAiAiYWW25B6aLkdbvC9jwq+dDb/zqkDPy2Ri1S0bs7JYZ2T
> vP+
> +Ec9Jbrld6CIjpw+HDjdjsAiAiYWW25B6aLkdbvC9jwq++XGIB
> +GzZavCEc4b9A0HIb7IWW2wAZRezcvn7jj2JWZyKqZyPWATd2+QJMIMQzgENlT7nJ
> /NgzRUt
> +GzZavCEc4b9A0HIb7IWW2wAZRezcvn7jj2JWZyKqZyPWATd2+nKqmH
> +n/VHZvlJVuwYdDPOrwKYEIhncJAs0Lj+xl+TM/fLGYPh4Sy6uXJy5KwFuVnjMXrm5
> xZ9ewD
> +AhMCx
> +Z0i9xIIExhf7x5wdgR3RtAbjB2dzcUQWIFbq91usH0PEM6Qe4jnZJls8G7o8U6DBF6
> QA4hn
> +SWNx/
> +vpntq/840H545FjLvLJF0+Qi1Ve/StrbyUsvkZoauQQUFtlMmcYzb6ktpylEONgM8Qx
> pDPF
> +sSoyK
> +sbcxv/aLt8tFqvXrSVtbNKNiTD4inaM7Tds8yXGKN9gp7u83NA0DcKjsKfJTDbFg3ZE
> S1/+
> +IIl9H
> +zxxm2UzTmHfQzbG+uXXdsQCkCuIZwKFm385Gxbh6zazb7bvvZteDg3wG4sODPH
> fxg9okn7u
> +zxxm2UzTmHfQzbG+SB/QI
> +4hlSDvEM4GhXf2vWrydYysnNJST4c9S/kO4QzwAOlT3F+uMpOg779a/5TDKwTkk
> MHZCYLHK
> +inLku
> +ms8jp9tFfypahl7PeJDr92SPnnmZtRPLtME/IC0hngEcSowp+av3HTFolZQOe31zFj3
> KBqk
> +eX86i
> +B9l62iTnDcUM+7sBUgMttyH10HLbVNvPPly/fbD2izP2NvIjzVptbazxdm0ta7ydFIb
> TjbQ
> +B9l62iTnDcUM+y7NQj
> +nFgFyYWW2wCZJnsq+1SbNw0TfW5P7KgYGWruStloW0A2g1OgeobUQ/Vs6viZa8
> vrLpQtmna
> +nFgFyYWW2wCZJnsq+sZZ5c
> +pDp+nCxfTsrKyLFjconNWE2p7TUsfUyeuh/SA6pngMkkO5tdX7/OZ0DLUA3XR3cc
> GsBxUD1
> +pDp+D6qF6
> +NhWpeh4cJAUFJD+fDAzIJWBGW0qHdQaGOhsmAqpngEwzfdrN9PrqNZz3HB+av
> hGyGcDpUD1
> +NhWpeh4cJAUFJD+fDAzIJWBGW0qHdQaGOhsmAqpngEwzfdrN9PrqNZz3HB+D
> 6qF6
> +NjX43kcFXzqbf+fUgZ+WyEWqq1dJTg47vWoUvW+Y0RfG49bCbHXlSDu/K6pnsA2
> qZ4DJJFX
> +NjX43kcFXzqbf+fUgZ+WyEWqq1dJTg47vWoUvW+nPaeD
> +0TMvx5TNsisSAIdBPANAJul/SzMG1XjZzKJcjlk1EuxneyJGR0P3Bkgl7NyG1MPObV
> NXr32
> +as/zt
> +6dNuHj12n1ykJTYatphR9A2+QmvmLl6ce/p0DBU3QNTi/n5DPEPqIZ6tZN1/hl6Pvb
> VIzOo
> +6dNuHj12n1ykJTYatphR9A2+gnhMl
> +4llpNBYa+xnhDHZCPEMaQzxbQTwnU/+RI2QlghiSDPEMaQzxbAXxHLvod24DTATE
> M6QxxLM
> +4llpNBYa+VxHPs
> +4jn2HAZBDrZBPEMaQzxbQTzHjoduWLyaL9XRpjXiGWwT9/cbTqwCgEmOnV6l9jB
> GkxnjVoE
> +TIJ4B
> +IJPk5OYSMmLoTY2fzZyba9arZ6jFdu7iGiQzOAfiGQAySU4Oi+fT7Uf65QK225oHcG
> 5O5E6
> +IJPk5OYSMmLoTY2fzZyba9arZ6jFdu7iGiQzOAfiGQAySU4Oi+3x7sd
> +YELh2DOkHo49W8Gx57iYtvmKcDg5VEDzEhpjZ4Cd0DQM0hji2QriOW66iI4qc9E0D
> JIC8Qx
> +pDPFs
> +BfGcTKZFtoB4BtvE/f2GY88AAACOg+oZUg/VsxVUz7FDTQzOguoZAAAgcyCeASDj
> 8L5F6ut
> +XuZRp
> +PgWQThDPAGno+nV2nZ3NZwAgAyGeAdLQ8DC7njOHzwBABkI8AwAAOA5abkP
> qoeW2FcuW24O
> +PgWQThDPAGno+DpKCA
> +zJ9PAgG5BAAcCS23ASYTGs8Udm4DZC5Uz5B6qJ5NDX/48R2r/t+c2299/9BfykWq4
> 8fJ8uW
> +krIwc
> +OyaXgITznsFZUD0DZJrrN9iHOXuq9YcULbcBMhfiGSANoeW2pbkr+SnPFDvXWYzh
> jPOeIQ0
> +OyaXgITznsFZUD0DZJrrN9iHOXuq9YcULbcBMhfiGSANoeW2pbkr+hngHS
> +kDjvGayNnnnZah83QFpAPAOkrenT5QTo0GhulgM4jwT7R9mSUTmcM0DaQDwD
> ONTwhx/T6zm
> +33yJm
> +da5eZdd/9md8BrT6jyjRnLt4sYuMnG5vbhZLXIVoFwZpBPEM4FDXP/ojvc6eYvYhFf
> EMltg
> +x50cX
> +LVpZszhXLnGtQqttSCuIZ4C0hZ3bptgYGI8uyuHTOYse5U3D6nFKFaQbxDNAGrpxg
> 10jnk3
> +MXYkg
> +hoyAeAZIQ+LEKgDIXIhnAIdSmobdKmZNoHoGyFyIZwCHitRrmKieEc9R6z/S3Nx8
> pF/OAaQ
> +hoyAeAZIQ+BxDOA
> +Q4nqefq0m8Wsjmi5jV7DADIX4hnAocSJVX/2GevqGX1uA2QujFgFqYcRq0zV77q4
> 58eXn/3
> +7v/D+
> +7Wy5SJWTwwpobC4TGLEKnAUjVgFkmusfsQ/z9GkWvYbhwDNARkP1DKmH6tlU
> 5cbgQf+vX/1
> +uYVWZ
> +PomHh8kdd5D8fDIwIJfAeHhNjeoZUgDVM0CmuX6DHXs2aRomhqtCuzCAjIZ4Bn
> Coq9c+pdc
> +PomHh8kdd5D8fDIwIJfAeHhNjeoZUgDVM0CmuX6DHXs2aRomhqtCuzCAjIZ4Bn
> Coq9c+m8T
> +PomHh8kdd5D8fDIwIJfAeHhNjeoZUgDVM0CmuX6DHXs2aRomhqtCuzCAjIZ4Bn
> Coq9c+w4
> +yK6xcxsgoyGeARzq6m95PH8m7NiziGdUz7GYuxLdbkOawbFnSD0cezaVs/xtWkCP
> HrvPWED
> +v3k22
> +bCGNjeTpp+USCEHLbXAWHHsGyDSWO7cvXWLXqJ4BMhriGcCJLLOZEn2S5OfzGT
> DDxpSkVrm
> +bCGNjeTpp+UaT4F
> +kE4QzwBOpMSz2UnPOPYMMAkgngGc6OpvP6HX0z9jXT0jngEyGuIZwIki7dym1X
> N2NuIZILO
> +h5Tak
> +Hlpuh2v72Yfrtw/WfnHG3sa75SKBZnNBAZk/nwQCcgkAOBhabgNkFDFclWWfJGg
> XBpDpEM8
> +ATiQG
> +ezYZTRLxPJ7RMy83U0f62Uz/ETatzgKkD8QzgBP9+reses6/Y6qYDRHxfLd+jzeEjPYH
> R9j
> +ZVKwD
> +kv4jag8lwUMvnxmV0wDpAPEM4ESWTcPefZddo3q2NDrC0rmQ9Q7W38fCOXdx
> TX3N4lwyEux
> +HPkMa
> +QTwDONHgezfo9ZzbbxWzIdi5HbXRURrUNJ5zckhOTi4hIyOIZ0gjiGcAJxLHnk3i+Z1
> 32PX8+XwG
> +wuXksiQeHZV7uWUhDZB2EM8ATjT43kf0Ov/OKWJWGh4mV6+yM54xmqSlnLmu
> XDJyur25/XQ
> +onflu
> +7tzcHL4GQFpIPJ6vNHm2Z63QXHYFCAnUrdjTNCTXiJP/Ff1jAkwWV699ev2jP+bfad
> EuDKV
> +zRDmL
> +HlV72BYtxEbP/JwfhM5BPEMaSbxbEhrPz5+vbWxxy3mOxvOJBW0bGvLkfOyuNO
> 0KrN2yhH/
> +K6KN1
> +kB2Gp4DMoZ62D9Jt80jxD8nvzpJ31ssl3AZCmgnZQ0i9XBAt9PcCkCoO75aE5quxDu
> 7atb2
> +0/YqY
> +1mC1eJ2fTsxokNlMFVdVkJ6LfGVWUidclwM43C23s+sb7/OZkHv4NW+6DbHh5z/j
> 3GdIJxM
> +1mC1eJ2fTsxokNlMFVdVkJ6LfGVWUidclwM43C23s+QzzRx
> +O3oef2Lstcax157wDXSIVK5cttA/cFlZQY3ty+f7FlYZq+TA/m5SctcMNul+ZOy1RIpyg
> HQ
> +w9U52
> +/cmHfCZE7NTmbcMAIMPZE8+t25TieMUrXXKZwv+6t2/h1hoerrQmrl3of/F1tk7eT
> Hd3gE0
> +MBTr6
> +CBm4wg4O+QOtFcWVbE0V27NN0x17tieDMeA2/3MT3RrP7t4k5xWr+TAYXe+/L+
> fHwzcqAKQ
> +CBm4wg4O+le+LZ
> +s4NWxuLyiD5cucKZRXKKp7KcKK4u/ODCEAmeOkcer/aRcwfo9MUP3AUzxe0M25
> XNjjqfkuk
> +OMCm8
> +M3idXs/Pzxaz0vAwu0yfjrGqIlK78dRRew8DSBcTcuy573KvnKK18mV2ZJmZsbacdJ
> wKHDh
> +KqkuL
> +NdNKEtNs3kY6X0OLMJh0lLOq9C23336bXT/wAJ8BgAyX/Hh2L/UVnt0pW4FdaWo
> 76358qai
> +wXXfN
> +8h890UHuXZtHXKX3EmWaC9Rt+8DXZlaLA2Q68+pZxDPOqhrH3JX1JtRTrQDSRZLiu
> biq4pK
> +8h890UHuXZtHXKX3EmWaC9Rt+8DXZlaLA2Q68+3VhyH
> +ntHQWl3y4vP8yPTz3oLq0J5qd7Gn7xIpL2YfnLzianLJXzBDfoiGrvQQ+gjqIW2lmTda
> bkO
> +ntHQWl3y4vP8yPTz3oLq0J5qd7Gn7xIpL2YfnLzianLJXzBDfoiGrvQQ+GevPs
> +78TE8Icfm5/0/J//ya4/9zk+AwAZLvHzngESFfd5gRnjoP/XDz/Vf6xl3gML/5ROV24M
> rnb
> +/Wddz
> ++oqvoIB1SxIIRF9AY8Oq+o80Hwq6VtWzXkoAJpLDz3sGAEsim2nF/JMTV+ms+Z7t
> q1dZNk+
> ++oqvoIB1SxIIRF9AY8Oq+fjp3b
> +8eG7vJHNkE4Qz5BcNHXkFJhRs5lOt/2Mnej87q/YWFX33K3fuX38OLu+7z4+AwCZ
> D/EMyTL
> +84cdP
> +/st/0+yR8xDm+JlrajZTdIvRJW9f+D2dvm/ebWKhJA48L1vGZ8BU/5GXz0QcMnL0z
> MvoOAz
> +SBuIZ
> +7CeCueCLPb4fXrp+A9WzORrDDz/VZ9i78PLhUfOd22++ya5RPUfGBqpqbjYJaRrM
> 9AYxhhV
> +7CeCueCLPb4fXrp+AekDT
> +MLATDebv7Lu050eX1dQpWzTtWMs8MW1lErZgotm8vO7C1Wufynm9/DunDvy0
> RM5Q16+TnBx
> +2PToa
> +01CSk27D8gZgctoM2obBxIv7Y4h4BnuEB7OAeA4XOZup5i13bfiypvu848fJ8uWsUV
> ggtpFVJ+Hv
> +HoaWysY6OXdxzaOLMJwkpEIK45kNKOntkzOUx7aRH7VDVRqeZbZvvNEqg+17il4
> kEVabgBU
> +HoaWysY6OXdxzaOLMJwkpEIK45kNKOntkzOUx7aRH7VDVRqeZbZvvNEqg+mCat
> g
> +FhDPBuNms8kW27KF7N5NvF7y7LNySXQmaTwDOEncH0N7jj2H+txuK+/ZZjpSZC
> QWg0sahZ5
> +FhDPBuNms8kW27KF7N5NvF7y7LNySXQmaTwDOEncH0N7jj2H+txuK+lxyyl
> +zxMzQydLV2wvGpjlkfNhJmCFyUF3jNksm8Fg8L2PKv9XMEI2U2+e/Z1xY+7fz65Xr
> eIzADA
> +zxMzQydLV2wvGpjlkfNhJmCFyUF3jNksm8Fg8L2PKv9XMEI2U2+p2N00
> +LG/JqR3KmFRMaKRnPoozQ8NY6QuM9f9FZ9d0Ez/rVizq7sB4R6H76QOadCJ2pWn
> H0RIa5Fu
> +K5QKj
> +CVgh8yUpmOnPzGjItdMNzeblGy7QTSfnLdDtuf84OwFaGhwk77xDsrNJWZlcAgC
> TgD07t5V
> +d0AKN
> +5BML2C5fORYk78VTWTj0SlbbzN7WJdr+kGhC7ywQq2kZd25rniX8ScOpL0POh5m
> AFTJQ5F3
> +5BML2C5fORYk78VTWTj0SlbbzN7WJdr+ZEBH9
> +YRHVx03XZZjPR558kqxeTbosdxhZUX/KYOc2QKrE/TFM0olVl87TitYfaC0s3ydDl/XC
> zRb
> +mzXT3
> +sbEjE8LHkK6KlM2QLAf9v6G1HbI5dtFmM3X8rWuhHeA/+Qm7fughPgMAk0Uyq2
> daKG87K5d
> +xstUY
> +GymSL6+oFvuHo6yeY2oahuo5qdp+9uH2F98ffI/1bxWZjU3D0rQWpD9lvvqPA7r9
> 1VGQjbe
> +GymSL6+oFvuHo6yeY2oahuo5qdp+Hh8kd
> +d7A92++/H9MpVQKqZ4CUc1L1TItmMmueSKzC8l7RmItfZIS7H+GzT/gGOmJqRBZ
> qGvbaZEx
> +d7A92++ER6n9
> +4u0DPy3Z25hvMrAS6GVPuenV7xaOvbVo9Nh99JfKS98uaPTcQaOX/nAx9j2i8fJh3
> rVGWxu
> +7Xr06
> +jmwGgLRmdzwPnSzdpozo7F7qI0fXGQJ46GST0kaMKrmLVcxFBbP9A5fFEkgjSQ3p
> rrqsrLq
> +Yj7Y6
> +2fRpN9NIrnkw92nPnbQ4plEd+NG9NLYHfrqQTtPNSGOb3krXmXP7rcfPXGONyM
> Se7XXr+AM
> +2fRpN9NIrnkw92nPnbQ4plEd+NG9NLYHfrqQTtPNSGOb3krXmXP7rcfPXGONyM
> Se7XXr+Aw
> +2fRpN9NIrnkw92nPnbQ4plEd+NG9NLYHfrqQTtPNSGOb3krXmXP7rcfPXGONyM
> Se7XXr+CR
> +2fRpN9NIrnkw92nPnbQ4plEd+NG9NLYHfrqQTtPNSGOb3krXmXP7rcfPXGONyM
> Se7XXr+i
> +Tzy3blMaY9eyJs3KbmrtSM/0Ik+F6pArP+8l5Zt4Pe2qWebp7oih5bYq+uGfx10z8RU
> mqyS
> +ENIvm
> +Na1yJuPl3zmFRjLdjDS2aW1No/r9Q39JY3vO0DnWl+ecOax6BoBJBr2GgZ3Cj0nHfe
> yZRvQ
> +a0jnW
> +wnbECJPuSOrDD7MznjdvJrt2ySUxwrFngJRL3bHngwflxNtv2zYNaQvHpI2efJI8/TQ
> brTl
> +Ww8Ps
> +Q5GdTRoa5BIAmEwSjueWFvYlQjOVfg3ZNQ1pLi1Dur99yap2MZzCQW/F1DkVU5
> XZkCO72XJ
> ++WdI8
> +RGfZdWTnT5Lt20lBAeuYkyauuEs0d/zOd9gYGDU1bOc2AExCtNxOyB/+MFZVNVZ
> WNjY6ats
> +0ZJBj
> +b/1GTlmTf4uhv8ZOTW+pnk65MHw1G53a8OCuDQ0v9dLJw7u+8MJFtuzwrg2H2b
> 9S30tfaDg
> +lp8fG
> +evv4OuN6cBF93fKSnT3m9Y69/35Xw4ops1dMeZA/nalf/IKtTC+BgFwSF7m9krLFA
> CAq8kM
> +Y+8fQ
> +js8tTVY1U+2ahslE/vGO9+crV7I/bC42Pbiri17zeO59wSNTue+lDSKnudBylQjyw7tY
> 1vI
> +js8tTVY1U+LX+HU
> +BjlLH1Mfz/xy49apY4+Ufmn3Gf4QFuhPVboyzfLEyO1FHwoAUkR+CGP/GNrRcjs7
> O3RSpl3
> +ToBoe
> +Zgcvly9nl/p61v0y2Oqgdydp3hx922i569vrF7PBIHnG331juPtcYyFfkFf8V/xfMnChX
> 0z
> +oTPn4
> +Bnnl1P/Z5mbHcQYH5VItn4+1CKOfhW9+Uy4BgMnH7vOewV70a7q4mB28PH6cXf
> bsIZ/9LLs
> +G+6z2
> +bSX1NHHXb2r//vrmIZeroCfIDwz3DfbwFQS6/N8OsEhe7eu+4f/GYrGUGQhoBlQN
> Nv8HaWZ
> +p/ZMa
> +ucQUC2kawwUFZP16XUgfPCibX+zahaPOAJOZHSdWDZ0srT0qSwky27PjKy3uGe
> N0w6m7C3H
> +ucQUC2kawwUFZP16XUgfPCibX+LYTNU
> +8XXnGUfHn+PxmwzgMXFo3Uyz2bTR7xtvkAcekNPpTz3xIEo2/NGaGHrOe2qNr8
> ZFJ1at3/R
> +8XXnGUfHn+LuqTi
> +J8O6qjrYXHfvduUvrOZ/31j7+pLgYyfrh+rn/PO/8WV/94PuZrJ76te7+VzhM/6WjfX
> 3k8N
> +n+GxE
> +tbXsBKpXX2UtyKgETqbSwolVACkX98cw8XgO1K3oIKIzbT7b1D6zgWWtNmJ5zm0j
> na89wk5
> +iZdNn
> +Zf/b1FCgixRX6nLUGM/mj2MU/ZpRS20819dbFsplZeTYMTk9Md55h/1cSI7ly5fLKQ
> u0ipw
> +vJ5nG
> +xkY5NTFu3GDdg4xnJHjh938+7y/+RM5KcWw3Gsw0nu2AeAZIufg/hvQOCbn4urv
> 8x2rbWo3
> +xkY5NTFu3GDdg4xnJHjh938+Lvsef
> +9oQaup73lDf7WDsbOqFdbkp7X8PjKLOnfkzkA6osnvHiec/jT5Nyfnn8daWtLL/1Jfo
> ghuU
> +Ue4V8
> +OV9Bdxflcb59Xi7TP477pcv8hRnWidd999H/TFwm0eXWW8eeeEL+79tBfsjpIwNAi
> sgPYew
> +fQ3uq
> +51Zl7CkNXS0bbN9T9OIsVsuy3dqXt45T1FpXz8ZZLYtnJIEuf3ElX6gZGou/7MJyXhmz
> O3a
> +Uh5Yr
> +Y1Sz5V4Svo5xfeXtax+TTWt2KsSluNhBDcHmz0/eodDjx4+LibKyMjFhNH06+SvZ4C
> o1rF7
> +Y1Sz5V4Svo5xfeXtax+YuPbt
> +k8NaRDZjBtm4kWzaxBpI2gfVM0DKpa56ZljZyupFegnVmpqF7KJU2Ky4NK22tbR1s
> L4mjnR
> +3i2fU
> +oneXRS0rhdWH7X2pWS43PD6dFe8ofLmc5dWzLOJ1L7Xz27yYTkRtLf3PNL/QwjqD
> yL9F+r4
> +yT2Oj
> +7j8u/DJnztizz7KzCpNAblb6LACQIvJDGPvH0JaW2zMaWuVQj50FR4s8J9W+lvgQkE
> /4Cok
> +cw4rK
> +m+kmH1yIcWCJ0JAb2z7wtUWqvE2ekRfNyt2140/PXmDacKxwZpGc0tMuZ+8i+SK
> MU1RXJyc
> +m+gcf3t
> +S5S+wNjpUmqXXtH07TWu/v8rJ8JNn84OMw8MEK/X3qIZADKAzSdWVX6t3N13u
> VfOCTMaahf
> +S5S+6X/yR
> +HOspb0YJuXQ+xi+9GEd61j1jsH3PmgFl2OkdC/kKERlfv0K7fOiy2uw8icrKzJsIrV5NN
> my
> +HOspb0YJuXQ+xi+Q02CH
> +ksa9N4bZ2VA3fO6DB7pPb1+v9Og59NyqivojfKW4BE3PfabB3NjIgpn+/yKYAcBM
> wvHsfyV
> +ksa9N4bZ2VA3fO6DB7pPb1+v9Og59NyqivojfKW4BE3PfabB3NjIgpn+rV0BO
> +0zr134/6w6tP9yOdFZe8O0RVXdyyYyGthuvUfNOPAG0P3TMSUjCDN72+0tSmrZ7
> NuIs95Ox
> +OOUZ1
> +oE6ttt1LfYXqcvY42uo8iWh11dUVOodq/ny5BGzVQ/NYqZ5Xr61YTNP6UA39mzm9
> fR9p7n7
> +oAE/o
> +UIfbdc/RzFVm672hjrjDV3PNmyufQ1CD+emn2TQAgIWE49n9SG/BCbnreMV2Vq
> eanYZUuaX
> +a03e0
> +SAS5+5GxtvKe0BDR584TEXuxoD8LIo6+rD6jq+YrvoEO/lw/IrXl7u6O0C8DE8UtbeV
> EjlF
> +SAS5+5GxtvKe0BDR584TEXuxoD8LIo6+rD6jq+9YsHj
> +arWtHbv6eW9BNW8XNiForfzGG2SM9b9KAgG7TrkBLW31LBdxixu3bpxLaGDT6W
> BwQCwkpE/
> +0Q8JS
> +fLi7ea2cuPED89UkBDMAxEQegwZIHfm3mKq/Rn3f2tohMbRjY/S+sKupj81JUa729
> RWsk20
> ++EoZc
> +MoHkZsXHHCB15IcwRU3DANKbduc2Wbm0hM3uVkYgl1z1SwNuzU5sC8bV/uej2
> 6YvnvrS7IP
> +ooRMA
> +YmFHp54AicHpuUmCDQuQcnF/DFE9AwAAOA7iGQAAwHEQzwAAAI6DeAbgJyvz
> fkiCzXVKs6+
> +h51bt
> +PtjfXq92HBY6oblC7U3soDfUguygl98q+jNRHhAAID6Jx/OVJo9yBjO/aM4qNt6UteI
> V1qH
> +PtjfXq92HBY6oblC7U3soDfUguygl98q+G0MnS
> +0JI9dX520nOo303tRdPhCccfMIqF/NH4c2npnnd7qexjxJThlUc6wTp6wfY9uodip26
> HvUg
> +TE/au
> +J62h5w7kn+P9kLjqtxZ/lyVusHln4CndYM9k5WZxbvS5xsISVx7P74qH2uWNNI+/d
> Q8/+/m
> +J62h5w7kn+pwe/R
> +gKcrPzW4PvE+QQFg0hpLlH7ICja+pDoUhP4mSTcWBZ316ceNiDiShBj0Qj+OJBudw
> jB6oxz
> +gKcrPzW4PvE+QQFg0hpLlH7ICja+pDoUhP4mSTcWBZ316ceNiDiShBj0Qj+5Ufc4
> +fDXN6zzfqX0QI/0rN4yHEQXju+CbhXz7x5rxMyj6LNE87IS965SRf4s2/DXGxXDec99
> LX3j
> +Q84UG
> +vtUMNzGnNjz4kjr8aFfDri4+0fuCZ8NhPhW6i27NlJCbNVUbFgAS+BjavXM7b8mp
> HQv9L75
> +vtUMNzGnNjz4kjr8aFfDri4+0fuCZ8NhPhW6i27NlJCbNVUbFgAS+uWRQO
> +XekhC6tCXTMVN8TY/ZanYlaot05aRLZ94KmYLecEf6C1sLzz8dn+owFlNdY9p0c7vG
> NeceX
> +4fXcr
> +eI+e++VegUCdWoyGile6cE9TO62G2UJaxa7pJn7WxZiola807ThasqPROObmUKCj
> oFj0DKr
> +eI+e++ZeWBe
> +qafgXU8efYOEVcOKuXeV/LKP3GO+pYLN+8hTrM6OQl4xGcT+bQCITxKOPbNuq63
> HpGJDYpx
> +dY9xV
> +G4tlS33k6DMiLP2vewuWbSrg04quE2fd5cWVpfe6+84dEC/D+JtAZdiJHb4P2YCuz0
> aD5oN
> +G4tlS33k6DMiLP2vewuWbSrg04quE2fd5cWVpfe6+zPOEb
> +6NDsK77kPTqTDbyxpbhyS2NnBXGz1cQAHmxEL5OBn+lPGRHY/ldCg3ZYjflh57t2rq
> zoyLX
> +6NDsK77kPTqTDbyxpbhyS2NnBXGz1cQAHmxEL5OBn+t0xMM
> +/b0e9O4r9nf/07s7WcfaRv7vvbuueaWc0XK5CuSD9A328CVMcX50QQ4AYJSkpmG
> hMalCY0H
> +KurC4
> +hQebXKgZfTJqbECq1jZ6R1ZE+r5mKElP7uyeXV06g5Yu1YWXOk7x+LQcYCo0FKa8G
> ApcgcZ
> +hQebXKgZfTJqbECq1jZ6R1ZE+r5mKElP7uyeXV06g5Yu1YWXOk7x+hH885
> +PrFVlvtiXKzQfgJPrUln41HJm+lWM9WSje8a9FYuLXlXbv1gc9237uH9bPvWBdzsILTa
> odi
> +PrFVlvtiXKzQfgJPrUln41HJm+S5iGW
> +3BvV3zvi2HP3Q2KEjJWP/c1+vubXyT/Vi19YQwFyF+IZAOKTpHgODaVsNhak5fjQ0X
> KzUnK
> +d50e0
> +iDTUmsFT5/yF965lC2esLZ8t49PGQaYtR322GD06GuyIwCxvLX+uCOX7hLzrScn9EN
> knamV
> +XfctJ
> +Ga7u5uHNq+fWnBRDZQx30+WrfS00uRV5Gw+Jm8RCdVY2KAs2v67JcgCA2CQhn
> v2BVjJrXnRZZTY+
> +dDR45dpHjEUkuXLg6CXSd7SIx2rRi5cI4ceMLQeZjrRz23yQ6SSN+ux+hD+RYYe5gV
> 3v2rl
> +dDR45dpHjEUkuXLg6CXSd7SIx2rRi5cI4ceMLQeZjrRz23yQ6SSN+ux+hD+kiwgL
> +cqUkUGtl2xzZvZ48pslyAIDY2B3PQydLt0UcCzma8aGjwfIs7DAt3/ncKTOVXTorSOs
> J+nR
> +Wg0xH
> +t3NbFfWoz0UFs/0Dl+XMuPQjXpfcZd1Wzp53DeF4rSyn7bBys1KFAwDEw554Du0
> HrmVNlLV
> +jIWuO
> +PdPLK13RjQ8dH9Y8Sp+XrDrvPsGOedsyyDSL86hGfXbVLPN0d2RZNMMO1yFf2P
> NeUr4pxh2
> +PdPLK13RjQ8dH9Y8Sp+iyX/X
> +E62rLisrq44f1A82lWZllTah/TMATDYYsQpST22Mrfw10oBe0+rp7F2ws8hb0jnWIn58
> hK0
> +G48AW
> +A0i5uD+GiGdIPZM/X1o2F3n9hHjUcEbYxA5bDCDl4v4YJqnlNgAAAMQP8QwOFGx
> a5/W7fT4
> +A0i5uD+PaV0j
> +DkIDAEwuiGdwnK66IhbO+xoaWjpZQKNpGABMPjj2DKkX5bEZHEmNFbYYQMrh
> 2DMAAEDmQDw
> +DkIDAEwuiGdwnK66IhbO+DAAA4
> +DnZuQ+qpO3+ihD/aKGHnNkDKYec2AABA5kA8AwAAOA52bkPqYR9skmDDAqQc
> dm4DAABkDsQ
> +DnZuQ+qpO3+zAACA
> +4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMAADgO4hkAAMBxEM8AAACOg25JIP
> XU0/YhSfAxB0g
> +VdEsC
> +AACQORDPAAAAjoOd25B66BoaADIVdm4DAABkDsQzAACA4yCeAQAAHAfxDAA
> A4DiIZwAAAMd
> +BPAMA
> +ADgO4hkAAMBxcN4zpB469QSAjIfzngEAANIe4hkAAMBxsHMbAADAcVA9AwAA
> OA7iGQBSJNh
> +UmkXV
> +dcl5AAhBPEMmkd/3XGlTUC4N6aqTN1LydvUuPCTEnMwLOqM+iPaRTVcQtE9geA
> Wmj8Do7hN
> +arluf
> +L9atKGkeR38Xs7dvwFcPvRXjHS1fMCNuDN1Z3Eu9S2ht/aPoXlXXM96SzrGxsZZKu
> cAOJv/
> +FjMVG
> +Dn95dEX1Zn4nNqe7d2lTU4T/iK463XvUUR9PMN+GuufSrK0yvBjxZIYtb/Ug2tV0b1
> 1ZxfT
> +B9aua
> +qmvSvwD9A4ll4lHk7XRGs6H42sproPS36ujfGlvLfGWTF0AXqc/BXkzoXvIW7X2snn
> 8CIZ4
> +hk/Se
> +59/3TGeJd53+A0Y/kDsX9Mqb6e3ne9lSV8OpTg/xdKoh4fb5yM6wexZ1VKt3Hes0r
> qBFH0o
> +61eCS
> +yyS3Tz5Gr69njfj40y+ENUS9B33kNZrvBWV9tjr99qhs4XN0Lf6COV20RXz7Bvw9Ebc
> ys05
> +yyS3Tz5Gr69njfj40y+5f73V
> +HeyOUbxlkw1l3Jj03RWd3yofgqIP/oz6HRy80ONeUCRn7GH+X2y5kc1e3n5+D0q5k
> 3wr2v/
> +WBvP/
> +CHqPrKw1rXx1M137Wz0eT6t2m5lsQyr0XOylalJLoazQ6yMdB/i9w/6MzR/EsJrhD0
> wI3VH
> +8AYdt
> +JZ9brqJ9+w3aR7ba4Bbv12zLRBJ6heGfMcH8BVRWeVr3i7cZPMD++OXGY8/vXkD
> M/3hSCPE
> +JZ9brqJ9+w3aR7ba4Bbv12zLRBJ6heGfMcH8BVRWeVr3i7cZPMD++MmaSy
> +Rf12Klogo0fLH/rIaVY1WltV4g2lCP3wPuMlvn2a74HKFotvhWi5GrZ62GsJNu3s8fVq
> Xkh
> +lS+gb
> +VyeqIDN5+zSwNIUACxA2w8KXfuPuqxaL2XdV9Vrxjlxrq+kLaIrqLRs3lEHYu2PpoM
> zSXwR
> +VyeqIDN5+zSwNIUACxA2w8KXfuPuqxaL2XdV9Vrxjlxrq+ev99b
> +xCsb+nooVrPwkOA5Jyihwd+HKlSrhd6bFP5fbLWRu0xfXhWfkl/v2hvHx34+0eySc0
> b8ZWx
> +xCsb+nooVrPwkOA5Jyihwd+q2WT4
> +/428DdlL7YmYWiXzxv1LHP9BKKs/sIj/ieasNjj/jzF5vxZbJn5WL0CTz73nSfVW9pfOn
> pD
> +9UKxe
> +S99/dN8PEwbxDBmpq47Wftp4oejXSu+CnfJbPexrXatyk69H/sqW5JegGhIR7t26R
> qwSaSX
> +6/dHq
> +qaqkXxJ+49era16J+iXB04uiURrb74HQ26dvmhWFRfS10Mhh1cGptQd4No/37RPN
> Ww7fUFo
> +qaqkXxJ+49era16J+m7y7E
> +1bDP5+bVm3gh/hJeoLVU0icMFT607FGfV9330Olp7SD7+CT91tV+2Zv+F1tt5P1WL
> 4/+D65
> +1bDP5+p9XTq
> +N1A0/62RKD+BxM8fzSNE3oa6vweV8mKKvH65JDLTB6FM/8D07zTif6I5qw1+gU+
> GvV/LLWM
> +p9ArV
> +kl/H+mNFf7XyfO7a31O9tlI+oXwB0X8/TBTEM2QcVnvtrzLd7cXCSmCRZf7ZZmh1S
> 3Y2ab7
> +Pei7w
> +D6u4v3WFxBj3DWoo34ZZbL8x+/an3xbykVXaXb4ikDo9ytNHJfzts+8d+nZpNrOFX
> c/Q73T
> +D6u4v3WFxBj3DWoo34ZZbL8x+xBce+
> +3elU3QG5ok5UbzlsQ2mZvDtr6pum363sl4tAyx3TXFG+fem3Lv83JPy/2GojV1m9P
> LbZw/Y
> +3elU3QG5ok5UbzlsQ2mZvDtr6pum363sl4tAyx3TXFG+qR/hv
> +jQaNALGvgG10v65+jLgNzQ8BhF4MDZQogsTqOILpH5j+ncb0nyhYbfB5YtrwfiNsG
> SuhV2j+G9P6
> +Y0V/ArCbeDqLHwTne5WfB1S03w8TBPEMGUVmk+nHNlKjnTCVm6o7dnYo07RGi
> 3wkNzpq/ad
> +8xbPv
> +B6/2a4CWvd6Srfrv/8qWqL8rzN8+rUdpecQehG0A9Qi2CF36VdfSoKlbxJdVQ9RvW
> buhDOj
> +3cInu
> +3bFXMu77oN+tygFC+obYQUGTXLFg+l9stZErTV/efpH9dCtpK/dEKa3gBFr+6+pH6
> 21o9ve
> +3bFXMu77oN+tygFC+obYQUGTXLFg+gQ3/M
> +yClr4z1IxD+wOP4TrTa4+v+ofb+Rt0x8Inys2E0d63a2it93dK5n5zqZzjF9P0wMuVUA
> MkB
> +yClr4z1IxD+wOP4TrTa4+v+ofb+YjRf6
> +mc3QKiFEiUr1PnxdOqdmKFtdndE/tHxcw/PRpaZPIWgfWkd3n9Ar1q/PVlJvozOh9
> UIML4f
> +fgS9T
> +VuYzmgdl8/K20H0jv2UFvVFdMbSh1LuE1ta9O92jaB5C+2iU5j6al659PvVxDJtC92w
> WD6l
> +7Fbrl
> +7AbtA/K3w+asHla7sn4tzWJKv54yb3hP8oG1j2J8HM50Bf5SQ/NWD6JdTfv04h50o
> e6Oulc
> +Xoj4c
> +XaxOG16A/i5imdn71T4EI+fVRxO0qxjvYbWyyQvg+NrqPF/L7B6aTZMy6DUMAAD
> AcbBzGwA
> +AwHEQ
> +zwAAAI6DeAYAAHAcxDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7i
> GQAAwHEQzwA
> +AAI6D
> +eAYAAHAcxDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHE
> QzwAAAI6DeAY
> +AAHAc
> +xDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwAAAI6
> DeAYAAHAcxDM
> +AAIDj
> +IJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwCQiK66LI3SpqBcP
> g56t7ouOR0
> +SbCqV
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+w
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+4
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+o
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+W
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+v
> +D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiK
> FKX+X
> +q2KbpaijurfTIxcATAjEMwBwXc94/XyidU3cWVZZ5SE9F1hqmtTBrDStU6raNa30ef
> R1bdf
> ++Vrdv
> +UyWfdjVs9fg7Dljnr6vh1NbzRdEVxgm9KvZMY2OnGlxyFmCCIJ4BwDY0YUnJPBerg
> 2m9yUt
> +XXR3c
> +2rNAlrS0FvXQQlkTe8ELPfy+UtECt/98r5wxVdlCHyeajE7kVQGkCuIZALjKFrn71tPZI
> kr
> +2rNAlrS0FvXQQlkTe8ELPfy+YaPm9
> +NCJF9dnj66V3pnno2SojTlcHq0tjpql7dWHMMrq3uqMoPKEn4lUBJA/iGQAkXo6O
> jcUYztq
> +jvInU
> +na55JXIXNNd73u9eUCRn5C5mQX0OJbLXkX0mT2zPqwJIFcQzANitssrTulOWs8Gm
> na3u6rUm+ajJ
> +Yo7ey+99RhwStr6XxKKZxXIM6RvfqwJIEcQzANiusoXtcOalLWuMbZKfPIuLjDuq2S
> HhSPc
> +Yo7ey+KYdV0
> +rEVxfK8KIDWy6E9POQkAAADOgOoZAADAcRDPAAAAjoN4BgAAcBzEMwAAgOM
> gngEAABwH8Qw
> +AAOA4
> +iGcAAADHQTwDAAA4DuIZAADAcRDPAAAAjoN4BoBEdNXxTqylqHurpnerE6Nf6
> MlRqExvi4G
> +tr0oz
> +mGWirwsgaohnABCUFIo5gkJDN3aWeIviTzD2Aoo6qnvluNMR0XXHCV27XlVXHRs
> /gz8QG7E
> +Dw2XA
> +BEE8AwDX9YzXzyda18SdZZVVHjkio0nFyUrTOqWqXdNKn8dQ1/IxnaMchoquu/V
> 8UXSFcUK
> +vqmt/
> +q9u3SQyB7WrY6vF3HBj/GQESh3gGANvQLCMl81ys4qRVcHjF2dqzQJa0tEL20JI01j
> EhtSp
> +b6ONE
> +k9GJvKrghR5+X6logdt/vlfOACQT4hkAuMoWuVPZ09kiisVosSGSZfXZ4+uld6Z56N
> kqI05
> +k9GJvKrghR5+XcapL
> +Y6ape3VhzDKajeIcntAT8aoAkgfxDAASL0fHxmIMZ+1R3oSq4Yj4jm9BfQ4lsteRfSZ
> PbM+rcs0r
> +kTvGud7zfveCIjkDkEyIZwCwW2WVp3WnLGeDTTtb3dVrTfJRk3qxY9HMYjmG9I3
> vVdF7+b3
> +kTvGud7zfveCIjkDkEyIZwCwW2WVp3WnLGeDTTtb3dVrTfJRk3qxY9HMYjmG9I3
> vVdF7+Pi
> +kTvGud7zfveCIjkDkEyIZwCwW2WVp3WnLGeDTTtb3dVrTfJRk3qxY9HMYjmG9I3
> vVdF7+AP
> +kTvGud7zfveCIjkDkEyIZwCwW2WVp3WnLGeDTTtb3dVrTfJRk3qxY9HMYjmG9I3
> vVdF7+V
> +1vcCsB3iGQBsV9nCdjjz0pY1ezbJT556RVG17DLFqulYi+L4XhW9l69nTaR7ASRDFv3
> pKSc
> +1vcCsB3iGQBsV9nCdjjz0pY1ezbJT556RVG17DLFqulYi+BAADA
> +GVA9AwAAOA7iGQAAwHEQzwAAAI6DeAYAAHAcxDMAAIDjIJ4BAAAcB/EMAAD
> gOIhnAAAAx0E
> +8AwAA
> +OA7iGQAAwGEI+f9N9WD9hjqa+wAAAABJRU5ErkJggg==
> +"
> +     id="image10"
> +     x="0"
> +     y="0" />
> +</svg>
> diff --git a/doc/guides/perf_tuning_guide/index.rst
> b/doc/guides/perf_tuning_guide/index.rst
> new file mode 100644
> index 0000000..ff325e9
> --- /dev/null
> +++ b/doc/guides/perf_tuning_guide/index.rst
> @@ -0,0 +1,47 @@
> +..  BSD LICENSE
> +    Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
> +    All rights reserved.
> +
> +    Redistribution and use in source and binary forms, with or without
> +    modification, are permitted provided that the following conditions
> +    are met:
> +
> +    * Redistributions of source code must retain the above copyright
> +    notice, this list of conditions and the following disclaimer.
> +    * Redistributions in binary form must reproduce the above copyright
> +    notice, this list of conditions and the following disclaimer in
> +    the documentation and/or other materials provided with the
> +    distribution.
> +    * Neither the name of Intel Corporation nor the names of its
> +    contributors may be used to endorse or promote products derived
> +    from this software without specific prior written permission.
> +
> +    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> +    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> FOR
> +    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> +    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> +    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> +    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> USE,
> +    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
> ON ANY
> +    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> +    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +Performance Tuning Guide for Intel's platform
> +=============================================
> +
> +|today|
> +
> +Contents
> +
> +.. toctree::
> +    :maxdepth: 2
> +    :numbered:
> +
> +    intro
> +    performance_tuning
> +
> +
> +
> +
> diff --git a/doc/guides/perf_tuning_guide/intro.rst
> b/doc/guides/perf_tuning_guide/intro.rst
> new file mode 100644
> index 0000000..5672549
> --- /dev/null
> +++ b/doc/guides/perf_tuning_guide/intro.rst
> @@ -0,0 +1,44 @@
> +..  BSD LICENSE
> +    Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
> +    All rights reserved.
> +
> +    Redistribution and use in source and binary forms, with or without
> +    modification, are permitted provided that the following conditions
> +    are met:
> +
> +    * Redistributions of source code must retain the above copyright
> +    notice, this list of conditions and the following disclaimer.
> +    * Redistributions in binary form must reproduce the above copyright
> +    notice, this list of conditions and the following disclaimer in
> +    the documentation and/or other materials provided with the
> +    distribution.
> +    * Neither the name of Intel Corporation nor the names of its
> +    contributors may be used to endorse or promote products derived
> +    from this software without specific prior written permission.
> +
> +    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> +    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
> +    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
> FOR
> +    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> +    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> +    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> +    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF
> USE,
> +    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
> ON ANY
> +    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
> +    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> +    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +Introduction
> +============
> +
> +This document is a step-by-step guide about how to get high performance with
> DPDK on Intel's platform and NICs.
> +It is designed for users who are not familiar with DPDK but would like
> +to measure the best performance. It contains step-by-step instructions to set
> the platform and NICs to its best performance.
> +The document will add more sections with the DPDK features' increment.
> +Currently, the document has only one section about PF performance test setup,
> and will add below cases in near future.
> +
> +*  VF performance tuning.
> +*  Vhost/virtio performance tuning.
> +*  new features....
> +
> +
> diff --git a/doc/guides/perf_tuning_guide/performance_tuning.rst
> b/doc/guides/perf_tuning_guide/performance_tuning.rst
> new file mode 100644
> index 0000000..e701d48
> --- /dev/null
> +++ b/doc/guides/perf_tuning_guide/performance_tuning.rst
> @@ -0,0 +1,157 @@
> +Performance Tuning DPDK with Intel's Platform and NICs
> +======================================================
> +
> +This document is a step-by-step guide for getting high DPDK performance with
> Intel's platform and NICs.
> +
> +Prerequisites
> +-------------
> +
> +Hardware platform essential requirements:
> +
> +1. Use a standard Intel® Xeon® server system (e.g. Ivy Bridge, Haswell or
> newer).
> +
> +2. Ensure that each memory channel has at least one memory DIMM inserted,
> and the memory size for each can be 4GB or above (e.g: 8GB or 16GB). You can
> use ``dmidecode -t memory`` to check the memory status. **Note**: This is one
> important element to impact the performance.
> +
> +Hardware platform Network Interface Card Essential requirements:
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> ~~~~
> +
> +1. Get an high end Intel® NIC, e.g: Intel® XL710.
> +
> +2. Make sure each NIC has flashed the latest version of NVM/firmware, if there
> is.
> +
> +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots
> can't provide enough bandwidth for 2x10G and above.
Pci gen2 x 8 may not provide enough, pci gen2 x 16 may support that.

> +
> +4. When inserting NICs to the PCI slots, be careful about the NUMA. If you will
> use 2 or more ports from different NICs, please make sure these NICs on the
> same CPU socket.
> +
> +BIOS settings:
> +~~~~~~~~~~~~~~
> +
> +1. To be sure, reset all the BIOS settings to default.
> +
> +2. Disable all power saving options, and set all options for best performance.
> +
> +3. Disable Turbo to ensure the performance scaling with core numbers
> increment.
> +
> +4. Set memory frequency to the highest number, NOT auto.
> +
> +5. Disable all Virtualization options when you test physical function of NIC, and
> turn on VT-d if you wants to use VFIO.
> +
> +
> +Linux System Essential Requirements:
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 - 64 bits.
> +
> +2. Make sure to select as more components as possible during system
> installation, to avoid install necessary components again and again.
> +
> +3. Make sure the widely used and fully validated version of kernel installed, e.g.
> 3.18.
> +
> +4. Make sure the required components are enabled for some old version of
> kernels, before rebuilding the kernel. The kernel may need to be rebuilt if any
> DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org
> for more details.
> +
> +
> +Grub Parameters Essential Requirements:
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +1. Use the default grub file as a good start.
> +
> +2. Reserve 1G huge pages via grub configurations, e.g. add
> ``default_hugepagesz=1G hugepagesz=1G hugepages=8`` to reserve 8 huge
> pages in 1G size.
> +
> +3. Isolate CPU cores which will be used for DPDK from scheduler, e.g:
> +isolcpus=2,3,4,5,6,7,8
> +
> +4. If it wants to use VFIO, additional grub parameters are needed. e.g:
> +``iommu=pt intel_iommu=on``
> +
> +
> +Configurations before running DPDK
> +----------------------------------
> +
> +1. For Intel® 40G NICs, special configurations should be set before compiling it,
> as follows. **Note**: This is very important::
> +
> +      for at least DPDK release 1.8, 2.0 and 2.1, in
> <dpdk_folder>/config/common_linuxapp
> +      CONFIG_RTE_PCI_CONFIG=y
> +      CONFIG_RTE_PCI_EXTENDED_TAG=”on”
> +
> +2. Build DPDK target and reserve huge page, refer to GSG guide for more details.
> Below scripts are for your reference::
> +
> +      cd <dpdk_folder>
> +      make install T=x86_64-native-linuxapp-gcc -j          # Build DPDK
> target
> +      awk '/Hugepagesize/ {print $2}' /proc/meminfo         # Get the
> hugepage size
> +      awk '/Hugepage_Total/ {print $2} ' /proc/meminfo      # Get the total
> huge page numbers
> +      umount `awk '/hugetlbfs/ {print $2}' /proc/mounts`    # Umount
> +      mkdir -p /mnt/huge                                    # Create
> the hugepage mount folder
> +      mount -t hugetlbfs nodev /mnt/huge                    # Mount to
> the specific folder
> +
> +3. Check the CPU layout by dpdk tools or system commands ``lscpu``::
> +
> +      cd <dpdk_folder>/tools
> +      ./cpu_layout.py                       #Run the script to check your
> system's cpu layout.
> +
> +   Or run ``lscpu`` to check the the cores on each socket
> +
> +4. Check your NIC id and related socket id::
> +
> +      lspci -nn|grep Eth                    # List all the NICs with PCI
> address and device IDs.
> +
> +   e.g. Suppose your output is as below::
> +
> +      82:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller
> XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +      82:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller
> XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +      85:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller
> XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +      85:00.1 Ethernet controller [0200]: Intel Corporation Ethernet
> + Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +
> +   Check the PCI device related numa node id::
> +
> +      cat /sys/bus/pci/devices/0000\:xx\:00.x/numa_node
> +
> +   Usually ``8x:00.x`` is on socket 1, ``0x:00.x`` is on socket 0. **Note**: To get
> best performance, please make sure the core and NICs are in the same socket.
> Take ``85:00.0`` for example, it's on socket 1, then use cores on socket1 for best
> performance.
> +
> +5. Bind the test ports to igb_uio. For example bind two ports to dpdk compatible
> driver and check the status::
> +
> +      # Bind ports 82:00.0 and 85:00.0 to dpdk driver
> +
> +      ./<dpdk_folder>/tools/dpdk_nic_bind.py -b igb_uio 82:00.0 85:00.0
> +
> +      # Check the port driver status
> +
> +      ./<dpdk_folder>/tools/dpdk_nic_bind.py --st
> +
> +
> +Example
> +-------
> +
> +Below is an case of running dpdk l3fwd sample to get high performance with
> Intel platform and NIC.
> +
> +**Note**: The scenario is to get best performance with two Intel®XL710 40G
> ports. See below Figure1 as the performance test setup.
> +
> +.. figure:: img/pf_performance_test_setup.*
> +
> +**Figure 1. PF_Performance_Test_setup**
> +
> +
> +1. Insert two NICs(Intel®XL710) into the platform, and use one port per card to
> get best performance. The reason using two NICs is the PCIe Gen3's limitations.
> **Note**: As PCIe Gen3 can't provide 80G bandwidth for two 40G ports, but two
Pci gen3 x 8 cannot provide ....

> different PCIe Gen3 slot can. Refer to the sample NICs output above, then we
> can select 82:00.0 and 85:00.0 as test ports::
> +
> +      82:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller
> XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +      85:00.0 Ethernet controller [0200]: Intel Corporation Ethernet
> + Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
> +
> +2. Connect the ports to the traffic generator, such as IXIA and Spirent.
> +
> +3. Check the PCI devices numa node(socket id) and get the cores number on
> the exact socket id. In this case, 82:00.0 and 85:00.0 are both in socket1, and the
> cores on socket1 in the referenced platform is 18-35,54-71. Note: Don't use one
> core's 2 thread(e.g core18 has 2 lcores, lcore18 and lcore54), instead, use 2 logical
> cores from different cores(e.g core18 and core19).
> +
> +4. Bind these two ports to igb_uio.
> +
> +5. As it is known that XL710 40G port need at least two queue pairs to achieve
> best performance, then two queues per port will be required, and each queue
> pair will need a dedicated CPU core for receiving/transmitting packets.
> +
> +6. Basically l3fwd will be using for performance testing, with using two ports for
> bi-directional forwarding. Compile the l3fwd sample with default lpm mode.
> +
> +7. Final command line of running l3fwd could be as followings. That means use
> core 18 for port 0, queue pair 0 forwarding, core 19 for port 0, queue pair 1
> forwarding, core 20 for port 1, queue pair 0 forwarding, core 21 for port 1, queue
> pair 1 forwarding::
> +
> +      ./l3fwd -c 0x3c0000 -n 4 -w 82:00.0 -w 85:00.0 -- -p 0x3 --config
> '(0,0,18),(0,1,19),(1,0,20),(1,1,21)'
> +
> +8. Configure the traffic to a traffic generator such as IXIA or Spirent.
> +
> +* Start creating a stream on packet generator, e.g. IXIA.
> +* Set the Ethernet II type to 0x0800
> +* Set the protocols to IPV4.
> +* Do not set any L4 protocols, just keep it as none.**Note**: this is very
> important, if you set UDP or TCP protocol, you may get relative low performance
> since the l3fwd example default using none protocols for RSS enabling.
> +* The flow's DEST MAC, DEST IP, SRC IP's settings can be seen in the above
> figure. It's for the user's reference. Set the correct destination IP address
> according to "ipv4_l3fwd_route_array" in the l3fwd example code, such as 2.1.1.1
> for port0, then it will forward the packets to port1. Set the source IP as random,
> **Note**: this is very important to make sure the packets will be received in
> multiple queues.
> +
> +
> --
> 2.1.0


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
  2015-08-10  6:34 Qian Xu
@ 2015-08-10 13:37 ` Thomas Monjalon
  2015-08-10 16:10   ` Zhang, Helin
  2015-08-10 16:00 ` Zhang, Helin
  1 sibling, 1 reply; 7+ messages in thread
From: Thomas Monjalon @ 2015-08-10 13:37 UTC (permalink / raw)
  To: Qian Xu; +Cc: dev

Hi,

Thanks for adding more helpful documentation.

2015-08-10 14:34, Qian Xu:
> Signed-off-by: Qian Xu <qian.q.xu@intel.com>

The Sign-off line must be below the commit message.

> Add a new guide doc under guides folder. This document is a step-by-step guide about how to get high performance with
> DPDK on Intel's platform and NICs.
> It is designed for users who are not familiar with DPDK but would like to measure the best performance. It contains
> step-by-step instructions to set the platform and NICs to its best performance.
> The document will add more sections with the DPDK features' increment.
> Currently, the document has only one section about PF performance test setup, and will add below cases in near future.
> 
> *  VF performance tuning.
> *  Vhost/virtio performance tuning.
> *  new features....

Good idea, thanks

> --- /dev/null
> +++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
> +  <image
> +     width="609.375"
> +     height="349.6875"
> +     preserveAspectRatio="none"
> +     xlink:href="data:image/png;base64,

This is a PNG picture embedded in SVG.
Please draw only some true SVG files.
Inkscape is the preferred tool.

> +++ b/doc/guides/perf_tuning_guide/performance_tuning.rst

This file is Intel-specific. So it should be renamed
	doc/guides/perf_tuning/intel_platform.rst

> +This document is a step-by-step guide for getting high DPDK performance with Intel's platform and NICs.

Could it be specific to Intel's CPU but allows some chapters for other NICs
tuning (e.g. Chelsio, Cisco, Mellanox)?
Ideally, the NIC tuning should refer to sections in doc/guides/nics/.

> +1. Get an high end Intel® NIC, e.g: Intel® XL710.

"Get an high end NIC that can sustain a high packet rate."?

> +2. Make sure each NIC has flashed the latest version of NVM/firmware, if there is.
> +
> +3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots can't provide enough bandwidth for 2x10G and above.

Could you describe how to get this characteristics with lspci?

> +4. When inserting NICs to the PCI slots, be careful about the NUMA. If you will use 2 or more ports from different NICs, please make sure these NICs on the same CPU socket.

Could you describe how to know the CPU-PCI mapping? lstopo, lspci?

> +BIOS settings:
> +~~~~~~~~~~~~~~
> +
> +1. To be sure, reset all the BIOS settings to default.
> +
> +2. Disable all power saving options, and set all options for best performance.
> +
> +3. Disable Turbo to ensure the performance scaling with core numbers increment.
> +
> +4. Set memory frequency to the highest number, NOT auto.
> +
> +5. Disable all Virtualization options when you test physical function of NIC, and turn on VT-d if you wants to use VFIO.

Could you describe usage of dmidecode to check this, please?

> +1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 - 64 bits.

Why not describe build options and run-time options to use with any Linux?

> +3. Make sure the widely used and fully validated version of kernel installed, e.g. 3.18.

Reference to release notes required.

> +4. Make sure the required components are enabled for some old version of kernels, before rebuilding the kernel. The kernel may need to be rebuilt if any DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org for more details.

Please use sphinx reference.

> +1. For Intel® 40G NICs, special configurations should be set before compiling it, as follows. **Note**: This is very important::
> +
> +      for at least DPDK release 1.8, 2.0 and 2.1, in <dpdk_folder>/config/common_linuxapp
> +      CONFIG_RTE_PCI_CONFIG=y
> +      CONFIG_RTE_PCI_EXTENDED_TAG=”on”

Please insert it in a i40e doc instead of here. Then you can reference it.

^ permalink raw reply	[flat|nested] 7+ messages in thread

* [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform.
@ 2015-08-10  6:34 Qian Xu
  2015-08-10 13:37 ` Thomas Monjalon
  2015-08-10 16:00 ` Zhang, Helin
  0 siblings, 2 replies; 7+ messages in thread
From: Qian Xu @ 2015-08-10  6:34 UTC (permalink / raw)
  To: dev

Signed-off-by: Qian Xu <qian.q.xu@intel.com>

Add a new guide doc under guides folder. This document is a step-by-step guide about how to get high performance with
DPDK on Intel's platform and NICs.
It is designed for users who are not familiar with DPDK but would like to measure the best performance. It contains
step-by-step instructions to set the platform and NICs to its best performance.
The document will add more sections with the DPDK features' increment.
Currently, the document has only one section about PF performance test setup, and will add below cases in near future.

*  VF performance tuning.
*  Vhost/virtio performance tuning.
*  new features....


diff --git a/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
new file mode 100644
index 0000000..50ce92d
--- /dev/null
+++ b/doc/guides/perf_tuning_guide/img/pf_performance_test_setup.svg
@@ -0,0 +1,375 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   xmlns:dc="http://purl.org/dc/elements/1.1/"
+   xmlns:cc="http://creativecommons.org/ns#"
+   xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
+   xmlns:svg="http://www.w3.org/2000/svg"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:xlink="http://www.w3.org/1999/xlink"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   id="svg2"
+   version="1.1"
+   inkscape:version="0.91 r13725"
+   width="609.375"
+   height="349.6875"
+   viewBox="0 0 609.375 349.6875"
+   sodipodi:docname="Performance_test_setup.svg">
+  <metadata
+     id="metadata8">
+    <rdf:RDF>
+      <cc:Work
+         rdf:about="">
+        <dc:format>image/svg+xml</dc:format>
+        <dc:type
+           rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
+        <dc:title></dc:title>
+      </cc:Work>
+    </rdf:RDF>
+  </metadata>
+  <defs
+     id="defs6" />
+  <sodipodi:namedview
+     pagecolor="#ffffff"
+     bordercolor="#666666"
+     borderopacity="1"
+     objecttolerance="10"
+     gridtolerance="10"
+     guidetolerance="10"
+     inkscape:pageopacity="0"
+     inkscape:pageshadow="2"
+     inkscape:window-width="1596"
+     inkscape:window-height="1108"
+     id="namedview4"
+     showgrid="false"
+     inkscape:zoom="0.56123077"
+     inkscape:cx="308.2511"
+     inkscape:cy="174.84375"
+     inkscape:window-x="0"
+     inkscape:window-y="27"
+     inkscape:window-maximized="1"
+     inkscape:current-layer="svg2" />
+  <image
+     width="609.375"
+     height="349.6875"
+     preserveAspectRatio="none"
+     xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAooAAAF1CAIAAADgOlyKAAAAAXNSR0IArs4c6QAAAARnQU1BAACx
+jwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAEUsSURBVHhe7d0JcBT3nS/wv3yA1htqkcxl78qW
+0AgjLHadxY84owBCCIOgEstOVlbWSSG24hF6rofGu6YCu7AyDxJIOVmPXGULydlFVHZjWS+xSVJI
+HDKHeYy9xGych2BsNDqCtmxhsERCDvARvf/VPd093aM5ejQ9o++npobunp6r0cx3ft3//v+zxsbG
+CAAAADjJTfJfAAAAcAzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZAADAcRDPAAAA
+joN4BgAAcBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZAADAcRDPAAAAjoN4BgAA
+cBzEMwAAgOMgngEAABwH8QwAAOA4iGcAAADHQTwDAAA4DuIZAADAcRDPAAAAjoN4BgAAcBzEMwAA
+gOMgngEAABwH8QwAAOA4WWNjY3ISIEWysrLkFABAhoo1bVE9AwAAOA7iGQAAwHGwcxtST925jb9G
+AMgwcX+/oXoGAABwHMQzAACA4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMAADgO4hkAAMBxEM8AAACO
+g3gGAABwHMQzAACA4yCeAQAAHCfBPrevNHme9/bJGaaiemwLqVtxYkHbhoY8uSxuwfY9RS8Snx0P
+BU6GPrcBIFOlss9tz47GsdeUy5ZiuTRBQydLV2wvGpjlkfMAAACTyATs3A7UrdieJS67AmJR167t
+pe1XxLQGrcW31/n5xI6jJTT1DWHvfyVrxZ6mITkHAACQqZIdzzRxO3oef4LX1k/4BjpEKlcuW+gf
+uKysoMb25fN9C6vcdGJGQ2tjC5vQcz8y9hp2dAMAQOazIZ5btynF8YpXuuQyhf91b9/CrTUz+MyM
+htqF/hdfZ+vkzXR3B9jEUKCjj5CBK0E67Q+0VhRXsjUBAAAmNXuPPT9iEq6FM4vkFE9lOVFcXfjB
+hSESPHWOPF7tI+cO0OmLH7gLZorbYXLKguSQ2xcA0kfyjz33Xe6VU7RWvsyOLDMz1paTjlOBA0dJ
+dWmxZlrU2QAAAJNakuPZvdRXeHanbAV2pantrPvxpaLCdt01y3/0RAe5d20ecZXeS5RpAAAASEY8
+F1dVXPLWiuPQMxpaq0tefJ4fmX7eW1B9Sh6Hpsld7Om7RMqLXXQ6r7iaXPIXzGDTEaDldqYbA1vJ
+zQoAaSjBbkkAbKAeHMVfo72wYQFSLu6PYfKPPQMAAECMEM8AAACOg3gGAABwHMQzAACA4yCeAQAA
+HAfxDEl2/bqcAACAqCUYz3xAC9nhNrvw8aZsoY5eJac1zxLx1Gd2brSypjJAVrhg+57Ij5P4CkCG
+h8mTT5KHH5azAAAQNRuq51Cf223lPdtMR4qMxGJwSaPQs+yYpfR5Eu5K04mZvWK116o93R0mPxfG
+HUk68RVABHNBAfH5UD0DAMTB1p3beUtO7VDGpGJCIz2rMUnDWKluWelJZ9d0Ez/rVizqSpR3FLqf
+PqBJJ2IzGrYsUboeK66qID0XDcFvMZJ0SOIrTG4IZgAAO9h97Nld7CFsKCqezepIz9VkG89R/ytr
+BsqV6paN3Fy5pbGzgrjZarEP5DzO8M+B/d2k5C7DGBsWI0mHJL7CZIVgBgCwTzKahl06z5I40FpY
+vk/2sM164WYL82a6+9jYkQnhY0hXjROQ8scBcnQiIJgBAOyWjHievUBUtH1Hi5Sd22u6+RK293uW
+t5YvtG63Zap1m3yorG0f+NrMBpZWsZ3eHWRHY2j4DUgSu4M5K2ryDgAAGcrueKZFM5k1T8Rzobof
+m11kIcv2SNPZJ3wDHTE1Igs1DYu8G5xm8zbSqT4dJAkqZgCAZLI1nodOlm5TRnR2L/WRo+sMATx0
+sklpI0aJA8NFBbP9A5fFkoQF6satrSFBsQbz8eO0KB7nEgfDI6TpBQDAgg3xHNrtXMuaNCu7lLUj
+PdOLPBWqQ678vJeUb+IFrqtmmae7I4aW26rwlttDV3rIJbnznF9kgW7Sxlsv8RUmj4MHyf79SaqY
+x8a7AIxv9MzLzQYvnxmVN2r0HzG7gd/7SL+cA0gZjPcMcWlrI9u3k8FBORtBWRk5dkxOW1CPJY/7
+1xj9mkBNus1FA/dQUE6bca2qXzlXTkdcW78iQALi/hgmVj0//XRowq5pSAu1tWRggOzdS/Lz5RK7
+ddXRv2qmzqIPGqfBIfhU6z8i0jZ3cU29Qc3iXHZL8BDKYkgbiVXPyUhTJHTaiVxJx1c903Be0+Pr
+PbX2QGmRt6RzrEW0JzBZ0zHWr2fXjY3J+8USMydvriToP/Ly6P2PLsqRs+FGz7z8Vs6jalnMqueR
+xTWGe9CV2k/nonoGu8T9MUT1DAlLQiUdvNBDSMk8F3HNKyGk50KkHZYOQn+oFBSwnI5mrz9D1/P5
+yDvvyFlIyNyVkbKZylkUymZq7sr6+vB70JXqkc3gADj2DLYKr6Tjqp6DTUrRrJTRDbyzVodXz/Td
+q+iPlqgqaZrndHPNn09WryarVrFr+0yy6hnAiVJUPUPyifOYli9nl/p6xxdaNlXSatHMy+i0FG0l
+vW4du6b/r7SMrqwkOTnkq19ld756ld8MycNabgs4Ig0OhOrZ0fbvZ9/v2i/q7Gzy7LNkwwY562ii
+kqY5HU/LbV4/+/mkRz30nMxysL99ST3Ze6iGVukHvRUPtRPyV984x2dDjuye+nXRAR5Z3Lj3pOs/
+lgQfO1kve8kxVM/Sbbtvufmxr305T1bSR3azu2jvSNObxripBx4gDz1EqqpYbR2XSVk9R98e22TN
+3LBD0QAJivtjmHA8D50srT0qvkUJme3Z8ZUW9ww+QvPz3j65lC73tWm6+tLdhY2Hoe99k933fK3o
+9ivi4+hEv2bU/K9ktc3sbVWHwJpo9Hv7s581L6LeeIN9daeH48fZ/u2Iov/zTVre+OtXvU6K8//B
+V+MSCUqz88juerK5eaVcg+X3c3ed9Mnu6IL9Q665ur8w63heSqb+M/l999eq5e5uY/wvX862UgT0
+PmLXN43qWCCe9cxOl+ItwUbkjAonVoFt4v8Y0jsk4Lyn/GnPKTlDZ30vXeYTl32Pa5af+jEp/3Fn
+aFpz08XznRflpEJ7X+vHMYp+zajRB3n89V45kwK1tfT/0vxSVibXyQzyb5G+sfHI9aJYMxYXmx7c
+1UWvG16i/929L3g2HOaL+17a8ELorzO0XHV41xfoCod3TZm9QlxuuY2+wlO3yNldN9P/rNvorZ5b
+bhm7eTq/9bZddyy9OPADfkfV3r3y/3XcS3b2WFXV2BtvyDuOR24sesdJpO/wCy+8cLhPzinMl1Ij
+b7W/8EL7WyN8hq/FmK0JEBf5IYz9Y5jYsWfWS5d28KjiBtNRKLSjTG4769mh6RA7r7gy+ho30kjP
+euozDgXqPEonYp6Tym/qQB29ezt9EMNySh2jek/TRbmIC41drRnMQ/c4rIcy9sIM68Tv7bflRLho
+OspMo4vKsDz8ojIsT+RyS87Ofzi6uTKL/MM+Mj+LzP8m+X4Vv+ke8v1vhlZTl9+SUzF1TsXUHH9W
+FXnzm+SWL5M/ftj90aXuT39fyF9aHvmU/0sGsm5h//zx91s/+YR8+ttvkJvZn+zlX5CCjR/9V/Ov
+Pr/ugu+Hl9gaNTXsoEU0rl9nl/vuk7Ngk5HT7fwYtFXZDZACicVz3owScnbNeFEUbD8hx8kwxnm8
+xhnpWfOMeaSqVo6l0Vmg7QP8kvfoTD5ixxOavsHlMJR8/a+Qo2f5QupKk0ddbhjM45J3oJgv5z2Y
+tonHrPZ0d9Spu+/jNSnbBtEEjnyx36dXt5LpFVNmr79p6vezpg19+mmByFEyRdemiy6/aQr7T/30
+avdHH37jj2IpM0CmyCnq5mn/QegKNK1vyCWS+mg3f0zuuPbRjd++ee7ak//y38VfOTf8u5tZQkeD
+BvOrr0ab5ZNUTm4uzdtRfV+do6MjhOTm6o8q86Zh+j3bSo8m2LMNqZdgy+3iFh5XsmTUlaGhvriL
+XpzV+RofpmLockKZNd5IzybPSIorlfUrly3Ujr3hqRUHlWesLVfG5GDDbS3cqvYZXruQT8jn1S73
+v/i60pPVbN/XivnEzAWF6mMWV1WQnov64UBiF6FGojeNjWXOJQ6GR0jskvfx5e4bw3ufqflG4Dd5
+Y7977HufXz9ldsWUaeRndFZd7Xebz317H1tOL7d//6Z1ZGw/eeA75JPfrPvGNLbw5ttY24dPPyU3
+3c5np4pXSm66TT7a2DX2i3JK9q0r7l/617/+7pT/PnB3fvY7g9e/+o8D1x/j7bcjo//rx44hm8eT
+k8Pi+XS7pjV2/xEewrk5Vo2+XKt4KJudBg2QKomfWDWjoZXWi+xCy9MiTULzISCf8BUSOYYVlTfT
+LfdyxyD6kZ5NnpGQrl3q3dVqmFIGpTYonFkkp/S0y9m7mAgNDXIiXF2dnAD75G30iXbaeRsP0bSm
+l82Gc5Bd9S18Ob/43GTlZt762t3Ml5T9eSH5iJDfb6alM7+0fPKJdnbzzDnk2W9t/nAgr3vfF/+T
+3eWf32yZN+f2W4+fubbng+JxTkWjt9K6efp0OQvW5q5cxf4fg4f4/mqG77N2rTKWxKxbEgHFMjiQ
+nec9V36t3N13uVfOCaLW/JE8Tsx2hl86H2M8RzvSs6R7xmD7njUDyrDTO5RqOALj61dolye4DyBq
+ZWVk82Y5rbV6dZqcWBU72SIiCvIOjuCvn8OORl+czfPYzBwazM+yE8K9Xl31S7O5ectddOI7+y4N
+13jEQhP0/rRujpzfEMJyl0e0gu2zRgRDmkksnv2vaNtAdf37UX949el+pLPikneHqKqLW3YspNVw
+6LisfgRoe+iekZCCGfxzeqWpTVs9m2ENys7uVI9Dq9U2b5KmLGePo63Ok2rXLtLVFdrLPX++XAJO
+IqvnB2bLeS2rYFZVlU2/b95twx9+3PSZh+SicPQXGbI5RqHSmMI+a0hDicWz+5HeghNy1/GK7axO
+NTtLuHJLtafvaJEIcvcjY23lPaEhos+dJ7Efox2v5bb6jK6aryiHxn9Easvd47TYKm5pKydyjOoT
+Cx5Xq23t2NXPewuq9SdqJxf9Zv7FL+Thz0DAvJ4GBxo3mFWNnjvoddvrn1wvqxBLQug96aWtjbz5
+plwCAJMDeg2D1Etx7xn97Uvc3z8tpmv+9421r6vdkmg7BRuX2i0JDeZvfpMdgGCpHN2DFH/l3DuD
+119d86uqHV+Wiyh6/1dfJYcOsf4+acjTtI9RIhtWdtch+udQu/pIg+46YuyWBCDJ4v4Y2nnsGSBN
+lTTuVRt8HTzQfXr7+qmr2vlX/NBzqyrqj/CVohBeMesfzVLdI2x/TMtIibybsHcv23nyEN/pvX8/
+XzRhRvuDI0pzKmUcZSp46OUz+lOWACA5UD1D6jm2ep769YFn/C3znqv4ydruZqJ2uF1IF27sk7N/
+V1Pxb+2yI+51/9rdOsW4WjTV89Vrn+Ysfzt7yk2j857P3vevbBHN5tpafiM7V4idBU9jP8Yj0Als
+WF6Daktn1hv13H5aUROHd0uteeWaae1ShXhjplBng21QPQPET1s9y0Xc4satG+eS1WvZIeFgcEAs
+JKQvwHt3X8zv1bxWTtz4QcXNU0xWi8b0aTeXLZp2/aM/HryfR3JjYyibKdHV9kQX0BLv0UOcNCxO
+KR5B+QwwARDPAFEqeMbPI5xGsjpOhokoVzNatugz9PonH+aR5mby9NNiobRqFbs+cYLPTAy17y2x
+l5u4CjOvmFQbd7OTsJT+wvRnZAGkDuIZgPRsX8+60aYXr5+sXFrCZncflDdKrvqlATdfZ06ko9HG
+1SweLVxVGety5PiZayZnta/m/aNEHtXKZjlzXbm8K2rR6aVI5/6+IM0xQ9eYAJAUOPYMqZfiY8+O
+kbP87avXPh346cL8OzW9eAsFBWyE0V/8IlJfr2ES3LDqsVlxJFbXljuj8Dcqh3oW7xLHnsE2cX8M
+E4xnwyjLrIcvZTQq402ELGT9YJuND921a/sa2bZGo6J6bIvozlrgD1gw/kL+aPy55AJunEGmtQyv
+PPGho9lIG61iUn1T/leythHjizQxYe86lRDPwsNP9e0/fvXZv/8L79+GdXEiTtt69lnWKDxq2LBR
+MjYSk1ENYIO4P4Y27NwOdbrJ+xvRjOak7Y+TXmhyBOpqj5aEFi5bwAdtrNwil3RWsAiRt+oCSdF9
+QtcbCRusQk4qAvtZ0qudfHE0C7XP21a9tXSclAq98h2zvLWvxNRPF01KzUagUdrRSlOZPZpmtCvW
+Exm/PRoT9a4htR5axvZvn/iv34pZnWXL2PWEHn7OFLQeFl1vN2tGydCT3XRLuYsfRDZD6tl67Dlv
+yakd2tGcwkQ5PrQ1T8WsUG+dNPnaPvBU6OsMf6C1sLzz8dn+owFlNZsGmWbGG/V5V0DsCfCzLsZ4
+v2Z8tKtO+VNDM9rVUKCjoFhUuvQu8jEtukJLwbuGVHhg4Z/S6zfP/k7M6pSVseuJ7TuM1pQhltHm
+VGoqi6Pn49D2AYrCGRzB7qZhrNtq6zGpohsfOpJlS33k6DMiLGnyFSzbVMCnFV0nzrrLiytL73X3
+nTsgXoblINO0rlVzkV/GeWGsDrYc9VmMHr2luHKLug+A7RIPXvwgbLQrvn3oTxllL3do0A6rMT/s
+fNdg5shu0XNIsLlOafY19Nyq3Qf72+ublb9muo5oPjanYomy8KA31ObroJffKnogUR4wJvPzs6dP
+u3n4w4/pRS5S5eez4aqGh9llQhj39wYPpUlCKz8qQqmsNsrG4WRIJ8louR0akyo0FqSsCyONDx0d
+VoC2ttE7siJSGWtZMXRyZ/fs6tIZtFKsLrzUcYrHp+UAU6GhMOXFdHc6L39ZzvEJi1Gf1ZGex8N+
+oOjRwFYz1ZKN7xrCDT13IP/cITaapKt+a/F3WeIGm3cGntIPKLlyszhd6lxjYYmL/oxiHYo91C5v
+pHn8rXv42c9PDX6PNdjeTCfWq9Eetfvm3Uav337392JW54EH2PUEFdC8kbZLGQZZnG4U7EunCloN
+ZctqWLd3QC/t9hVABkpGPIeGUtYce1brQsvxoaPlZqXkOs+PaBFpqDWDp875C+9dyxbOWFs+W8an
+jYNMW476bDF6dDhW1OqxIwKzvLX8uSKU7xPyriep/lOBe0qVX1d5G5vzv7Wqbv276yzOWvZ/b//y
+f2A3sWGhf1IjFrLeSHhm0z+SfBLk233l0pL9p2L9877vHh7PF/4gZnVEm+233+YzMB52VhiHbkgh
+Pdkdz/5AK5k1L7qsMhsfOhq8cu0jxiKSXDlw9BLpO1rEY7XoxUuE8GPGloNMR9q5bfbDIp5Rn113
+zQq7V9j2cT/Cn8iww9zArncNYfoGiUhWYe5dJb/sI/eY/xEHm/eRp1idHYW8YjIYazx/roTF83/2
+mB1+/tzn2PUvf8lnkm1uISuXD4mAo9iO7vTomkQ5ilyzOFcsUHPatCbW7CHQwG5wSD1b43noZOm2
+iGMhRzM+dDRYnoUdphWNsGSmsktnBWk9QZ/OapDp6HZuq6Ie9bmoYLZ/4LKc4fdSDreb3Us/4nXJ
+XdZt5ex512CiR9S73EHvvmJ/9z+9u/M5k+9y//csqmqXq0A+SN9gaAdJcX50QR4idm6/M3hdzOpM
+bPVMU07Xf1banQmcs+hREbVqTgOkFRviObQfmJ/Goz2zVnPsmV5e6YpufOj4sOZR+uRj1bk4JcmW
+QaZZnEc16rOrZpmnuyNLHm5n9+KzlvfqkC/seS8p3xRjY67kv+sJlRU1eQdbrFxa8q6M52Bz3bfu
+4f1s+9YF3OwgtNqh2JLmIZbcG9X/IXHsufuhOXUsyFc+9jf7+ZpfJ/8kB8AYCpC7Yv3znp+fnT3l
+JhrP1z/6o1ykEq3DBgfZ8BgTQtuaOY3rSTWnURNDWkGvYZB6ccStvX+3B711Fza20FS2UbB5d+eq
+zXE85mf/NvD2hd+/sXe+OM9K5/OfZ03D3nhDNhMbj7ph8TE3wxuno38wSLK4P4bJaBoGkGbUWtk2
+R3avJ4/Fl/cilWlCi1mdid2/HerPI53aV7H22OGvlr8XtMeGdILqGVJPWz1H/oOcDOWg74eXnvyX
+//b+7exn//4v5CKVz0eefJL16/nss3JJRIlsLtH3tJzh0qSnS+MJ2xqolCEFUD0DZIj5+dn02rx1
+2Pz57Pqdd/hMUolxJNVmzayV2EiwP7POUAqd94yyGhwI8QxO1FVHf3HW8V5fgk2lWVmlTTGfH5+2
+5uf/Cb0efO+GmNURO7cHB/lMUo2OjNBy+X6l1px7/+JcMjKSLvEc6pJEEbn5tnoGGc6RBufAzm1I
+PbOd2zSg17R6OnsX7CzylnSOtYjW6ZNh5zaVdf8Zej321iIxq5OTw1pu/+EPJJsV2ZElsLmMzaYy
+thlV+E58BrvBwTbYuQ0ZprKl1+duXVPk9Xs6ZTZPHmL/9uB7H4lZnfx8dp30Alp0S/JzWUyOnvl5
+2nRLEg+l2tad6Q2QWqieIfXMqme+V5uGMyEetXZOqBxMJ5Ubgwf9vz7WMq9s0TS5SPXww2T/fvLq
+q6SqSi6xNkk2lxl9A7EIrdpQPUOSxf0xRDxD6pnFMw9n4vOVeL2toYCeJHlTv+vinh9f3tuYX/vF
+2+Ui1ZYtZPdu1nLb65VLrCWwuSK0f6acnV7miRse0SbvMU1ap0M6iftjiJ3b4ERddbRwdvv2NTS0
+dHpI65rJ1DSMuvuOKfR68H2z1mF3382uf/UrPgMm+t/i2Ux/Qij4PuuR029Zts9W10U2g3OgeobU
+M6uezU2S6rn98MhX/3Gg9osz9jbyMNY6eJBUVrI926++KpdYS3xzqQVm+uzt5S/ZWAWLghq7rCEF
+UD0DZI78O6bSa/NzqyaoaRhFIy2085edepRGZwfn5uir4JycqIfFoPGOE6HBCRDPAI6Tf6fYuZ3C
+lts0pMTxW7Hfl+8eDvalQ2jl5NIkNr7U/j76OyM3F3uuIY0gnsFZsiKSK2W6ObffSq+Hr3wsZnWy
+s8mcOezU5wkYt4qdbyT2BrPBq9LkrKOcuS6Wz4c0BXD/EbYXINc1F/EMaQTHniH14sjdjP+7LfhS
+z+B7N94/9JciqnU++1k2KkYgIPv4tKZu2Ng3V/+RI2Rlhh+oNWm5rcBRarBN3B9DVM8ATpQvGm9H
+6JlkeJjPJMncTM9mAKdD9Qypl0CRl7HWb/9V28+uvPrdwqqy6XKRqr6e7NlD9u4ltbVyiYUEN6z2
+/OH0OSE40ZqY3x/VM9gG1TNARpk+7WZ6ffXap2JWZ/Zsdp3c6tnYt8fI6XYMFwEwkRDPAE40O/cW
+em3eM4nYuZ3cnknUASXZUE/in0j9ejgNfcVqg3M+jd60Ie0gngGcSLQIu/ThJ2JWZyKOPRsGlIzl
+xOE0xxqpY882OADiGcCJxKnPwx+anVs1nR+NnoATqxSsT5JDwYw7b5h3QGIO3ZJA6iGeAZwo0rFn
+Ec9Jbrld6CIjpw+HDjdjsAiAiYWW25B6aLkdbvC9jwq+dDb/zqkDPy2Ri1S0bs7JYZ2TvP++XGIB
+GzZavCEc4b9A0HIb7IWW2wAZRezcvn7jj2JWZyKqZyPWATd2+QJMIMQzgENlT7nJ/NgzRUtnKqmH
+n/VHZvlJVuwYdDPOrwKYEIhncJAs0Lj+xl+TM/fLGYPh4Sy6uXJy5KwFuVnjMXrm5xZ9ewDAhMCx
+Z0i9xIIExhf7x5wdgR3RtAbjB2dzcUQWIFbq91usH0PEM6Qe4jnZJls8G7o8U6DBF6QA4hnSWNx/
+vpntq/840H545FjLvLJF0+Qi1Ve/StrbyUsvkZoauQQUFtlMmcYzb6ktpylEONgM8QxpDPFsSoyK
+sbcxv/aLt8tFqvXrSVtbNKNiTD4inaM7Tds8yXGKN9gp7u83NA0DcKjsKfJTDbFg3ZES1/+IIl9H
+zxxm2UzTmHfQzbG+uXXdsQCkCuIZwKFm385Gxbh6zazb7bvvZteDg3wG4sODPHfxg9okn7uSB/QI
+4hlSDvEM4GhXf2vWrydYysnNJST4c9S/kO4QzwAOlT3F+uMpOg779a/5TDKwTkkMHZCYLHKinLku
+ms8jp9tFfypahl7PeJDr92SPnnmZtRPLtME/IC0hngEcSowp+av3HTFolZQOe31zFj3KBqkeX86i
+B9l62iTnDcUM+7sBUgMttyH10HLbVNvPPly/fbD2izP2NvIjzVptbazxdm0ta7ydFIbTjbQy7NQj
+nFgFyYWW2wCZJnsq+1SbNw0TfW5P7KgYGWruStloW0A2g1OgeobUQ/Vs6viZa8vrLpQtmnasZZ5c
+pDp+nCxfTsrKyLFjconNWE2p7TUsfUyeuh/SA6pngMkkO5tdX7/OZ0DLUA3XR3ccGsBxUD1D6qF6
+NhWpeh4cJAUFJD+fDAzIJWBGW0qHdQaGOhsmAqpngEwzfdrN9PrqNZz3HB+avhGyGcDpUD1D6qF6
+NjX43kcFXzqbf+fUgZ+WyEWqq1dJTg47vWoUvW+Y0RfG49bCbHXlSDu/K6pnsA2qZ4DJJFXnPaeD
+0TMvx5TNsisSAIdBPANAJul/SzMG1XjZzKJcjlk1EuxneyJGR0P3Bkgl7NyG1MPObVNXr32as/zt
+6dNuHj12n1ykJTYatphR9A2+QmvmLl6ce/p0DBU3QNTi/n5DPEPqIZ6tZN1/hl6PvbVIzOognhMl
+4llpNBYa+xnhDHZCPEMaQzxbQTwnU/+RI2QlghiSDPEMaQzxbAXxHLvod24DTATEM6QxxLMVxHPs
+4jn2HAZBDrZBPEMaQzxbQTzHjoduWLyaL9XRpjXiGWwT9/cbTqwCgEmOnV6l9jBGkxnjVoETIJ4B
+IJPk5OYSMmLoTY2fzZyba9arZ6jFdu7iGiQzOAfiGQAySU4Oi+fT7Uf65QK225oHcG5O5E63x7sd
+YELh2DOkHo49W8Gx57iYtvmKcDg5VEDzEhpjZ4Cd0DQM0hji2QriOW66iI4qc9E0DJIC8QxpDPFs
+BfGcTKZFtoB4BtvE/f2GY88AAACOg+oZUg/VsxVUz7FDTQzOguoZAAAgcyCeASDj8L5F6utXuZRp
+PgWQThDPAGno+nV2nZ3NZwAgAyGeAdLQ8DC7njOHzwBABkI8AwAAOA5abkPqoeW2FcuW24ODpKCA
+zJ9PAgG5BAAcCS23ASYTGs8Udm4DZC5Uz5B6qJ5NDX/48R2r/t+c2299/9BfykWq48fJ8uWkrIwc
+OyaXgITznsFZUD0DZJrrN9iHOXuq9YcULbcBMhfiGSANoeW2pbkr+SnPFDvXWYzhjPOeIQ0hngHS
+kDjvGayNnnnZah83QFpAPAOkrenT5QTo0GhulgM4jwT7R9mSUTmcM0DaQDwDONTwhx/T6zm33yJm
+da5eZdd/9md8BrT6jyjRnLt4sYuMnG5vbhZLXIVoFwZpBPEM4FDXP/ojvc6eYvYhFfEMltgx50cX
+LVpZszhXLnGtQqttSCuIZ4C0hZ3bptgYGI8uyuHTOYse5U3D6nFKFaQbxDNAGrpxg10jnk3MXYkg
+hoyAeAZIQ+LEKgDIXIhnAIdSmobdKmZNoHoGyFyIZwCHitRrmKieEc9R6z/S3Nx8pF/OAaQBxDOA
+Q4nqefq0m8Wsjmi5jV7DADIX4hnAocSJVX/2GevqGX1uA2QujFgFqYcRq0zV77q458eXn/37v/D+
+7Wy5SJWTwwpobC4TGLEKnAUjVgFkmusfsQ/z9GkWvYbhwDNARkP1DKmH6tlU5cbgQf+vX/1uYVWZ
+PomHh8kdd5D8fDIwIJfAeHhNjeoZUgDVM0CmuX6DHXs2aRomhqtCuzCAjIZ4BnCoq9c+pdcm8Tw4
+yK6xcxsgoyGeARzq6m95PH8m7NiziGdUz7GYuxLdbkOawbFnSD0cezaVs/xtWkCPHrvPWEDv3k22
+bCGNjeTpp+USCEHLbXAWHHsGyDSWO7cvXWLXqJ4BMhriGcCJLLOZEn2S5OfzGTDDxpSkVrmUaT4F
+kE4QzwBOpMSz2UnPOPYMMAkgngGc6OpvP6HX0z9jXT0jngEyGuIZwIki7dym1XN2NuIZILOh5Tak
+Hlpuh2v72Yfrtw/WfnHG3sa75SKBZnNBAZk/nwQCcgkAOBhabgNkFDFclWWfJGgXBpDpEM8ATiQG
+ezYZTRLxPJ7RMy83U0f62Uz/ETatzgKkD8QzgBP9+reses6/Y6qYDRHxfLd+jzeEjPYHR9jZVKwD
+kv4jag8lwUMvnxmV0wDpAPEM4ESWTcPefZddo3q2NDrC0rmQ9Q7W38fCOXdxTX3N4lwyEuxHPkMa
+QTwDONHgezfo9ZzbbxWzIdi5HbXRURrUNJ5zckhOTi4hIyOIZ0gjiGcAJxLHnk3i+Z132PX8+XwG
+wuXksiQeHZV7uWUhDZB2EM8ATjT43kf0Ov/OKWJWGh4mV6+yM54xmqSlnLmuXDJyur25/XQonflu
+7tzcHL4GQFpIPJ6vNHm2Z63QXHYFCAnUrdjTNCTXiJP/Ff1jAkwWV699ev2jP+bfadEuDKVzRDmL
+HlV72BYtxEbP/JwfhM5BPEMaSbxbEhrPz5+vbWxxy3mOxvOJBW0bGvLkfOyuNO0KrN2yhH/K6KN1
+kB2Gp4DMoZ62D9Jt80jxD8nvzpJ31ssl3AZCmgnZQ0i9XBAt9PcCkCoO75aE5quxDu7atb20/YqY
+1mC1eJ2fTsxokNlMFVdVkJ6LfGVWUidclwM43C23s+sb7/OZkHv4NW+6DbHh5z/j3GdIJxMQzzRx
+O3oef2Lstcax157wDXSIVK5cttA/cFlZQY3ty+f7FlYZq+TA/m5SctcMNul+ZOy1RIpygHQw9U52
+/cmHfCZE7NTmbcMAIMPZE8+t25TieMUrXXKZwv+6t2/h1hoerrQmrl3of/F1tk7eTHd3gE0MBTr6
+CBm4wg4O+QOtFcWVbE0V27NN0x17tieDMeA2/3MT3RrP7t4k5xWr+TAYXe+/L+fHwzcqAKQle+LZ
+s4NWxuLyiD5cucKZRXKKp7KcKK4u/ODCEAmeOkcer/aRcwfo9MUP3AUzxe0M25XNjjqfkukOMCm8
+M3idXs/Pzxaz0vAwu0yfjrGqIlK78dRRew8DSBcTcuy573KvnKK18mV2ZJmZsbacdJwKHDhKqkuL
+NdNKEtNs3kY6X0OLMJh0lLOq9C23336bXT/wAJ8BgAyX/Hh2L/UVnt0pW4FdaWo76358qaiwXXfN
+8h890UHuXZtHXKX3EmWaC9Rt+8DXZlaLA2Q68+pZxDPOqhrH3JX1JtRTrQDSRZLiubiq4pK3VhyH
+ntHQWl3y4vP8yPTz3oLq0J5qd7Gn7xIpL2YfnLzianLJXzBDfoiGrvQQ+gjqIW2lmTdabkOGevPs
+78TE8Icfm5/0/J//ya4/9zk+AwAZLvHzngESFfd5gRnjoP/XDz/Vf6xl3gML/5ROV24Mrnb/Wddz
++oqvoIB1SxIIRF9AY8Oq+o80Hwq6VtWzXkoAJpLDz3sGAEsim2nF/JMTV+ms+Z7tq1dZNk+fjp3b
+8eG7vJHNkE4Qz5BcNHXkFJhRs5lOt/2Mnej87q/YWFX33K3fuX38OLu+7z4+AwCZD/EMyTL84cdP
+/st/0+yR8xDm+JlrajZTdIvRJW9f+D2dvm/ebWKhJA48L1vGZ8BU/5GXz0QcMnL0zMvoOAzSBuIZ
+7CeCueCLPb4fXrp+A9WzORrDDz/VZ9i78PLhUfOd22++ya5RPUfGBqpqbjYJaRrM9AYxhhVAekDT
+MLATDebv7Lu050eX1dQpWzTtWMs8MW1lErZgotm8vO7C1Wufynm9/DunDvy0RM5Q16+TnBx2PToa
+01CSk27D8gZgctoM2obBxIv7Y4h4BnuEB7OAeA4XOZup5i13bfiypvu848fJ8uWsUVggtpFVJ+Hv
+HoaWysY6OXdxzaOLMJwkpEIK45kNKOntkzOUx7aRH7VDVRqeZbZvvNEqg+17il4kEVabgBUmCatg
+FhDPBuNms8kW27KF7N5NvF7y7LNySXQmaTwDOEncH0N7jj2H+txuK+/ZZjpSZCQWg0sahZ5lxyyl
+zxMzQydLV2wvGpjlkfNhJmCFyUF3jNksm8Fg8L2PKv9XMEI2U2+e/Z1xY+7fz65XreIzADAp2N00
+LG/JqR3KmFRMaKRnPoozQ8NY6QuM9f9FZ9d0Ez/rVizq7sB4R6H76QOadCJ2pWnH0RIa5FuK5QKj
+CVgh8yUpmOnPzGjItdMNzeblGy7QTSfnLdDtuf84OwFaGhwk77xDsrNJWZlcAgCTgD07t5Vd0AKN
+5BML2C5fORYk78VTWTj0SlbbzN7WJdr+kGhC7ywQq2kZd25rniX8ScOpL0POh5mAFTJQ5F3ZEBH9
+YRHVx03XZZjPR558kqxeTbosdxhZUX/KYOc2QKrE/TFM0olVl87TitYfaC0s3ydDl/XCzRbmzXT3
+sbEjE8LHkK6KlM2QLAf9v6G1HbI5dtFmM3X8rWuhHeA/+Qm7fughPgMAk0Uyq2daKG87K5dxstUY
+GymSL6+oFvuHo6yeY2oahuo5qdp+9uH2F98ffI/1bxWZjU3D0rQWpD9lvvqPA7r91VGQjbeHh8kd
+d7A92++/H9MpVQKqZ4CUc1L1TItmMmueSKzC8l7RmItfZIS7H+GzT/gGOmJqRBZqGvbaZExER6n9
+4u0DPy3Z25hvMrAS6GVPuenV7xaOvbVo9Nh99JfKS98uaPTcQaOX/nAx9j2i8fJh3rVGWxu7Xr06
+jmwGgLRmdzwPnSzdpozo7F7qI0fXGQJ46GST0kaMKrmLVcxFBbP9A5fFEkgjSQ3prrqsrLqYj7Y6
+2fRpN9NIrnkw92nPnbQ4plEd+NG9NLYHfrqQTtPNSGOb3krXmXP7rcfPXGONyMSe7XXr+AMAwCRi
+Tzy3blMaY9eyJs3KbmrtSM/0Ik+F6pArP+8l5Zt4Pe2qWebp7oih5bYq+uGfx10z8RUmqySENIvm
+Na1yJuPl3zmFRjLdjDS2aW1No/r9Q39JY3vO0DnWl+ecOax6BoBJBr2GgZ3Cj0nHfeyZRvQa0jnW
+wnbECJPuSOrDD7MznjdvJrt2ySUxwrFngJRL3bHngwflxNtv2zYNaQvHpI2efJI8/TQbrTlWw8Ps
+Q5GdTRoa5BIAmEwSjueWFvYlQjOVfg3ZNQ1pLi1Dur99yap2MZzCQW/F1DkVU5XZkCO72XJ+WdI8
+RGfZdWTnT5Lt20lBAeuYkyauuEs0d/zOd9gYGDU1bOc2AExCtNxOyB/+MFZVNVZWNjY6ats0ZJBj
+b/1GTlmTf4uhv8ZOTW+pnk65MHw1G53a8OCuDQ0v9dLJw7u+8MJFtuzwrg2H2b9S30tfaDglp8fG
+evv4OuN6cBF93fKSnT3m9Y69/35Xw4ops1dMeZA/nalf/IKtTC+BgFwSF7m9krLFACAq8kMY+8fQ
+js8tTVY1U+2ahslE/vGO9+crV7I/bC42Pbiri17zeO59wSNTue+lDSKnudBylQjyw7tY1vILX+HU
+BjlLH1Mfz/xy49apY4+Ufmn3Gf4QFuhPVboyzfLEyO1FHwoAUkR+CGP/GNrRcjs7O3RSpl3ToBoe
+Zgcvly9nl/p61v0y2Oqgdydp3hx922i569vrF7PBIHnG331juPtcYyFfkFf8V/xfMnChX0zoTPn4
+Bnnl1P/Z5mbHcQYH5VItn4+1CKOfhW9+Uy4BgMnH7vOewV70a7q4mB28PH6cXfbsIZ/9LLsG+6z2
+bSX1NHHXb2r//vrmIZeroCfIDwz3DfbwFQS6/N8OsEhe7eu+4f/GYrGUGQhoBlQNNv8HaWZp/ZMa
+ucQUC2kawwUFZP16XUgfPCibX+zahaPOAJOZHSdWDZ0srT0qSwky27PjKy3uGeN0w6m7C3HLYTNU
+8XXnGUfHn+PxmwzgMXFo3Uyz2bTR7xtvkAcekNPpTz3xIEo2/NGaGHrOe2qNr8ZFJ1at3/RLuqTi
+J8O6qjrYXHfvduUvrOZ/31j7+pLgYyfrh+rn/PO/8WV/94PuZrJ76te7+VzhM/6WjfX3k8Nn+GxE
+tbXsBKpXX2UtyKgETqbSwolVACkX98cw8XgO1K3oIKIzbT7b1D6zgWWtNmJ5zm0jna89wk5iZdNn
+Zf/b1FCgixRX6nLUGM/mj2MU/ZpRS20819dbFsplZeTYMTk9Md55h/1cSI7ly5fLKQu0ipwvJ5nG
+xkY5NTFu3GDdg4xnJHjh938+7y/+RM5KcWw3Gsw0nu2AeAZIufg/hvQOCbn4urv8x2rbWo3Lvsef
+9oQaup73lDf7WDsbOqFdbkp7X8PjKLOnfkzkA6osnvHiec/jT5Nyfnn8daWtLL/1JfoghuUUe4V8
+OV9Bdxflcb59Xi7TP477pcv8hRnWidd999H/TFwm0eXWW8eeeEL+79tBfsjpIwNAisgPYewfQ3uq
+51Zl7CkNXS0bbN9T9OIsVsuy3dqXt45T1FpXz8ZZLYtnJIEuf3ElX6gZGou/7MJyXhmzO3aUh5Yr
+Y1Sz5V4Svo5xfeXtax+TTWt2KsSluNhBDcHmz0/eodDjx4+LibKyMjFhNH06+SvZ4Co1rF7YuPbt
+k8NaRDZjBtm4kWzaxBpI2gfVM0DKpa56ZljZyupFegnVmpqF7KJU2Ky4NK22tbR1sL4mjnR3i2fU
+oneXRS0rhdWH7X2pWS43PD6dFe8ofLmc5dWzLOJ1L7Xz27yYTkRtLf3PNL/QwjqDyL9F+r4yT2Oj
+7j8u/DJnztizz7KzCpNAblb6LACQIvJDGPvH0JaW2zMaWuVQj50FR4s8J9W+lvgQkE/4Cokcw4rK
+m+kmH1yIcWCJ0JAb2z7wtUWqvE2ekRfNyt2140/PXmDacKxwZpGc0tMuZ+8i+SKMU1RXJycgcf3t
+S5S+wNjpUmqXXtH07TWu/v8rJ8JNn84OMw8MEK/X3qIZADKAzSdWVX6t3N13uVfOCTMaahf6X/yR
+HOspb0YJuXQ+xi+9GEd61j1jsH3PmgFl2OkdC/kKERlfv0K7fOiy2uw8icrKzJsIrV5NNmyQ02CH
+ksa9N4bZ2VA3fO6DB7pPb1+v9Og59NyqivojfKW4BE3PfabB3NjIgpn+/yKYAcBMwvHsfyVrV0BO
+0zr134/6w6tP9yOdFZe8O0RVXdyyYyGthuvUfNOPAG0P3TMSUjCDN72+0tSmrZ7NuIs95OxOOUZ1
+oE6ttt1LfYXqcvY42uo8iWh11dUVOodq/ny5BGzVQ/NYqZ5Xr61YTNP6UA39mzm9fR9p7n7oAE/o
+UIfbdc/RzFVm672hjrjDV3PNmyufQ1CD+emn2TQAgIWE49n9SG/BCbnreMV2VqeanYZUuaXa03e0
+SAS5+5GxtvKe0BDR584TEXuxoD8LIo6+rD6jq+YrvoEO/lw/IrXl7u6O0C8DE8UtbeVEjlF9YsHj
+arWtHbv6eW9BNW8XNiForfzGG2SM9b9KAgG7TrkBLW31LBdxixu3bpxLaGDT6WBwQCwkpE/0Q8JS
+fLi7ea2cuPED89UkBDMAxEQegwZIHfm3mKq/Rn3f2tohMbRjY/S+sKupj81JUa729RWsk20+EoZc
+MoHkZsXHHCB15IcwRU3DANKbduc2Wbm0hM3uVkYgl1z1SwNuzU5sC8bV/uej26YvnvrS7IPooRMA
+YmFHp54AicHpuUmCDQuQcnF/DFE9AwAAOA7iGQAAwHEQzwAAAI6DeAbgJyvzfkiCzXVKs6+h51bt
+PtjfXq92HBY6oblC7U3soDfUguygl98q+jNRHhAAID6Jx/OVJo9yBjO/aM4qNt6UteIV1qHG0MnS
+0JI9dX520nOo303tRdPhCccfMIqF/NH4c2npnnd7qexjxJThlUc6wTp6wfY9uodip26HvUgTE/au
+J62h5w7kn+P9kLjqtxZ/lyVusHln4CndYM9k5WZxbvS5xsISVx7P74qH2uWNNI+/dQ8/+/mpwe/R
+gKcrPzW4PvE+QQFg0hpLlH7ICja+pDoUhP4mSTcWBZ316ceNiDiShBj0Qj+OJBudwjB6oxz5Ufc4
+fDXN6zzfqX0QI/0rN4yHEQXju+CbhXz7x5rxMyj6LNE87IS965SRf4s2/DXGxXDec99LX3jQ84UG
+vtUMNzGnNjz4kjr8aFfDri4+0fuCZ8NhPhW6i27NlJCbNVUbFgAS+BjavXM7b8mpHQv9L75uWRQO
+XekhC6tCXTMVN8TY/ZanYlaot05aRLZ94KmYLecEf6C1sLzz8dn+owFlNdY9p0c7vGNeceX4fXcr
+eI+e++VegUCdWoyGile6cE9TO62G2UJaxa7pJn7WxZiola807ThasqPROObmUKCjoFj0DKrZeWBe
+qafgXU8efYOEVcOKuXeV/LKP3GO+pYLN+8hTrM6OQl4xGcT+bQCITxKOPbNuq63HpGJDYpxdY9xV
+G4tlS33k6DMiLP2vewuWbSrg04quE2fd5cWVpfe6+84dEC/D+JtAZdiJHb4P2YCuz0aD5oNzPOEb
+6NDsK77kPTqTDbyxpbhyS2NnBXGz1cQAHmxEL5OBn+lPGRHY/ldCg3ZYjflh57t2rqzoyLXt0xMM
+/b0e9O4r9nf/07s7WcfaRv7vvbuueaWc0XK5CuSD9A328CVMcX50QQ4AYJSkpmGhMalCY0HKurC4
+hQebXKgZfTJqbECq1jZ6R1ZE+r5mKElP7uyeXV06g5Yu1YWXOk7x+LQcYCo0FKa8GApcgcZhH885
+PrFVlvtiXKzQfgJPrUln41HJm+lWM9WSje8a9FYuLXlXbv1gc9237uH9bPvWBdzsILTaodiS5iGW
+3BvV3zvi2HP3Q2KEjJWP/c1+vubXyT/Vi19YQwFyF+IZAOKTpHgODaVsNhak5fjQ0XKzUnKd50e0
+iDTUmsFT5/yF965lC2esLZ8t49PGQaYtR322GD06GuyIwCxvLX+uCOX7hLzrScn9ENknamVXfctJ
+Ga7u5uHNq+fWnBRDZQx30+WrfS00uRV5Gw+Jm8RCdVY2KAs2v67JcgCA2CQhnv2BVjJrXnRZZTY+
+dDR45dpHjEUkuXLg6CXSd7SIx2rRi5cI4ceMLQeZjrRz23yQ6SSN+ux+hD+RYYe5gV3v2rlkiwgL
+cqUkUGtl2xzZvZ48pslyAIDY2B3PQydLt0UcCzma8aGjwfIs7DAt3/ncKTOVXTorSOsJ+nRWg0xH
+t3NbFfWoz0UFs/0Dl+XMuPQjXpfcZd1Wzp53DeF4rSyn7bBys1KFAwDEw554Du0HrmVNlLVjIWuO
+PdPLK13RjQ8dH9Y8Sp+XrDrvPsGOedsyyDSL86hGfXbVLPN0d2RZNMMO1yFf2PNeUr4pxh2iyX/X
+E62rLisrq44f1A82lWZllTah/TMATDYYsQpST22Mrfw10oBe0+rp7F2ws8hb0jnWIn58hK0G48AW
+A0i5uD+GiGdIPZM/X1o2F3n9hHjUcEbYxA5bDCDl4v4YJqnlNgAAAMQP8QwOFGxa5/W7fT4PaV0j
+DkIDAEwuiGdwnK66IhbO+xoaWjpZQKNpGABMPjj2DKkX5bEZHEmNFbYYQMrh2DMAAEDmQDwDAAA4
+DnZuQ+qpO3+ihD/aKGHnNkDKYec2AABA5kA8AwAAOA52bkPqYR9skmDDAqQcdm4DAABkDsQzAACA
+4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMAADgO4hkAAMBxEM8AAACOg25JIPXU0/YhSfAxB0gVdEsC
+AACQORDPAAAAjoOd25B66BoaADIVdm4DAABkDsQzAACA4yCeAQAAHAfxDAAA4DiIZwAAAMdBPAMA
+ADgO4hkAAMBxcN4zpB469QSAjIfzngEAANIe4hkAAMBxsHMbAADAcVA9AwAAOA7iGQBSJNhUmkXV
+dcl5AAhBPEMmkd/3XGlTUC4N6aqTN1LydvUuPCTEnMwLOqM+iPaRTVcQtE9geAWmj8Do7hNarluf
+L9atKGkeR38Xs7dvwFcPvRXjHS1fMCNuDN1Z3Eu9S2ht/aPoXlXXM96SzrGxsZZKucAOJv/FjMVG
+Dn95dEX1Zn4nNqe7d2lTU4T/iK463XvUUR9PMN+GuufSrK0yvBjxZIYtb/Ug2tV0b11ZxfTB9aua
+qmvSvwD9A4ll4lHk7XRGs6H42sproPS36ujfGlvLfGWTF0AXqc/BXkzoXvIW7X2snn8CIZ4hk/Se
+59/3TGeJd53+A0Y/kDsX9Mqb6e3ne9lSV8OpTg/xdKoh4fb5yM6wexZ1VKt3Hes0rqBFH0o61eCS
+yyS3Tz5Gr69njfj40y+ENUS9B33kNZrvBWV9tjr99qhs4XN0Lf6COV20RXz7Bvw9Ebcys055f73V
+HeyOUbxlkw1l3Jj03RWd3yofgqIP/oz6HRy80ONeUCRn7GH+X2y5kc1e3n5+D0q5k3wr2v/WBvP/
+CHqPrKw1rXx1M137Wz0eT6t2m5lsQyr0XOylalJLoazQ6yMdB/i9w/6MzR/EsJrhD0wI3VH8AYdt
+JZ9brqJ9+w3aR7ba4Bbv12zLRBJ6heGfMcH8BVRWeVr3i7cZPMD++OXGY8/vXkDM/3hSCPEMmaSy
+Rf12Klogo0fLH/rIaVY1WltV4g2lCP3wPuMlvn2a74HKFotvhWi5GrZ62GsJNu3s8fVqXkhlS+gb
+VyeqIDN5+zSwNIUACxA2w8KXfuPuqxaL2XdV9Vrxjlxrq+kLaIrqLRs3lEHYu2PpoMzSXwRev99b
+xCsb+nooVrPwkOA5Jyihwd+HKlSrhd6bFP5fbLWRu0xfXhWfkl/v2hvHx34+0eySc0b8ZWxq2WT4
+/428DdlL7YmYWiXzxv1LHP9BKKs/sIj/ieasNjj/jzF5vxZbJn5WL0CTz73nSfVW9pfOnpD9UKxe
+S99/dN8PEwbxDBmpq47Wftp4oejXSu+CnfJbPexrXatyk69H/sqW5JegGhIR7t26RqwSaSX6/dHq
+qaqkXxJ+49era16J+iXB04uiURrb74HQ26dvmhWFRfS10Mhh1cGptQd4No/37RPNWw7fUFom7y7E
+1bDP5+bVm3gh/hJeoLVU0icMFT607FGfV9330Olp7SD7+CT91tV+2Zv+F1tt5P1WL4/+D65p9XTq
+N1A0/62RKD+BxM8fzSNE3oa6vweV8mKKvH65JDLTB6FM/8D07zTif6I5qw1+gU+GvV/LLWMp9ArV
+kl/H+mNFf7XyfO7a31O9tlI+oXwB0X8/TBTEM2QcVnvtrzLd7cXCSmCRZf7ZZmh1S3Y2ab7Pei7w
+D6u4v3WFxBj3DWoo34ZZbL8x+/an3xbykVXaXb4ikDo9ytNHJfzts+8d+nZpNrOFXc/Q73TxBce+
+3elU3QG5ok5UbzlsQ2mZvDtr6pum363sl4tAyx3TXFG+fem3Lv83JPy/2GojV1m9PLbZw/YqR/hv
+jQaNALGvgG10v65+jLgNzQ8BhF4MDZQogsTqOILpH5j+ncb0nyhYbfB5YtrwfiNsGSuhV2j+G9P6
+Y0V/ArCbeDqLHwTne5WfB1S03w8TBPEMGUVmk+nHNlKjnTCVm6o7dnYo07RGi3wkNzpq/ad8xbPv
+B6/2a4CWvd6Srfrv/8qWqL8rzN8+rUdpecQehG0A9Qi2CF36VdfSoKlbxJdVQ9RvWbuhDOj3cInu
+3bFXMu77oN+tygFC+obYQUGTXLFg+l9stZErTV/efpH9dCtpK/dEKa3gBFr+6+pH621o9vegQ3/M
+yClr4z1IxD+wOP4TrTa4+v+ofb+Rt0x8Inys2E0d63a2it93dK5n5zqZzjF9P0wMuVUAMkBYjRf6
+mc3QKiFEiUr1PnxdOqdmKFtdndE/tHxcw/PRpaZPIWgfWkd3n9Ar1q/PVlJvozOh9UIML4ffgS9T
+VuYzmgdl8/K20H0jv2UFvVFdMbSh1LuE1ta9O92jaB5C+2iU5j6al659PvVxDJtC92wWD6l7Fbrl
+7AbtA/K3w+asHla7sn4tzWJKv54yb3hP8oG1j2J8HM50Bf5SQ/NWD6JdTfv04h50oe6OulcXoj4c
+XaxOG16A/i5imdn71T4EI+fVRxO0qxjvYbWyyQvg+NrqPF/L7B6aTZMy6DUMAADAcbBzGwAAwHEQ
+zwAAAI6DeAYAAHAcxDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwAAAI6D
+eAYAAHAcxDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwAAAI6DeAYAAHAc
+xDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwAAAI6DeAYAAHAcxDMAAIDj
+IJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAAOA7iGQAAwHEQzwCQiK66LI3SpqBcPg56t7ouOR0SbCqV
+D2R2awxsfVXq60rsNQHEBPEMAIKSjTGHkNvXOyZ0lniL4s+wrroib0knf6BeX8+aiKFKX+w4oWvX
+q2KbpaijurfTIxcATAjEMwBwXc94/XyidU3cWVZZ5SE9F1hqmtTBrDStU6raNa30efR1bdf+Vrdv
+UyWfdjVs9fg7Dljnr6vh1NbzRdEVxgm9KvZMY2OnGlxyFmCCIJ4BwDY0YUnJPBerg2m9yUtXXR3c
+2rNAlrS0FvXQQlkTe8ELPfy+UtECt/98r5wxVdlCHyeajE7kVQGkCuIZALjKFrn71tPZIkrYaPm9
+NCJF9dnj66V3pnno2SojTlcHq0tjpql7dWHMMrq3uqMoPKEn4lUBJA/iGQAkXo6OjcUYztqjvInU
+na55JXIXNNd73u9eUCRn5C5mQX0OJbLXkX0mT2zPqwJIFcQzANitssrTulOWs8Gmna3u6rUm+ajJ
+Yo7ey+99RhwStr6XxKKZxXIM6RvfqwJIEcQzANiusoXtcOalLWuMbZKfPIuLjDuq2SHhSPcKYdV0
+rEVxfK8KIDWy6E9POQkAAADOgOoZAADAcRDPAAAAjoN4BgAAcBzEMwAAgOMgngEAABwH8QwAAOA4
+iGcAAADHQTwDAAA4DuIZAADAcRDPAAAAjoN4BoBEdNXxTqylqHurpnerE6Nf6MlRqExvi4Gtr0oz
+mGWirwsgaohnABCUFIo5gkJDN3aWeIviTzD2Aoo6qnvluNMR0XXHCV27XlVXHRs/gz8QG7EDw2XA
+BEE8AwDX9YzXzyda18SdZZVVHjkio0nFyUrTOqWqXdNKn8dQ1/IxnaMchoquu/V8UXSFcUKvqmt/
+q9u3SQyB7WrY6vF3HBj/GQESh3gGANvQLCMl81ys4qRVcHjF2dqzQJa0tEL20JI01jEhtSpb6ONE
+k9GJvKrghR5+X6logdt/vlfOACQT4hkAuMoWuVPZ09kiisVosSGSZfXZ4+uld6Z56NkqI05XcapL
+Y6ape3VhzDKajeIcntAT8aoAkgfxDAASL0fHxmIMZ+1R3oSq4Yj4jm9BfQ4lsteRfSZPbM+rcs0r
+kTvGud7zfveCIjkDkEyIZwCwW2WVp3WnLGeDTTtb3dVrTfJRk3qxY9HMYjmG9I3vVdF7+b3PiAPV
+1vcCsB3iGQBsV9nCdjjz0pY1ezbJT556RVG17DLFqulYi+L4XhW9l69nTaR7ASRDFv3pKScBAADA
+GVA9AwAAOA7iGQAAwHEQzwAAAI6DeAYAAHAcxDMAAIDjIJ4BAAAcB/EMAADgOIhnAAAAx0E8AwAA
+OA7iGQAAwGEI+f9N9WD9hjqa+wAAAABJRU5ErkJggg==
+"
+     id="image10"
+     x="0"
+     y="0" />
+</svg>
diff --git a/doc/guides/perf_tuning_guide/index.rst b/doc/guides/perf_tuning_guide/index.rst
new file mode 100644
index 0000000..ff325e9
--- /dev/null
+++ b/doc/guides/perf_tuning_guide/index.rst
@@ -0,0 +1,47 @@
+..  BSD LICENSE
+    Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Intel Corporation nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Performance Tuning Guide for Intel's platform
+=============================================
+
+|today|
+
+Contents
+
+.. toctree::
+    :maxdepth: 2
+    :numbered:
+
+    intro
+    performance_tuning
+    
+    
+    
+    
diff --git a/doc/guides/perf_tuning_guide/intro.rst b/doc/guides/perf_tuning_guide/intro.rst
new file mode 100644
index 0000000..5672549
--- /dev/null
+++ b/doc/guides/perf_tuning_guide/intro.rst
@@ -0,0 +1,44 @@
+..  BSD LICENSE
+    Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+    All rights reserved.
+
+    Redistribution and use in source and binary forms, with or without
+    modification, are permitted provided that the following conditions
+    are met:
+
+    * Redistributions of source code must retain the above copyright
+    notice, this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright
+    notice, this list of conditions and the following disclaimer in
+    the documentation and/or other materials provided with the
+    distribution.
+    * Neither the name of Intel Corporation nor the names of its
+    contributors may be used to endorse or promote products derived
+    from this software without specific prior written permission.
+
+    THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+    "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+    LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+    A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+    OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+    SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+    LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+    DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+    THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+    (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+    OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+Introduction
+============
+
+This document is a step-by-step guide about how to get high performance with DPDK on Intel's platform and NICs.
+It is designed for users who are not familiar with DPDK but would like to measure the best performance. It contains
+step-by-step instructions to set the platform and NICs to its best performance. 
+The document will add more sections with the DPDK features' increment. 
+Currently, the document has only one section about PF performance test setup, and will add below cases in near future. 
+
+*  VF performance tuning.
+*  Vhost/virtio performance tuning.
+*  new features....
+
+
diff --git a/doc/guides/perf_tuning_guide/performance_tuning.rst b/doc/guides/perf_tuning_guide/performance_tuning.rst
new file mode 100644
index 0000000..e701d48
--- /dev/null
+++ b/doc/guides/perf_tuning_guide/performance_tuning.rst
@@ -0,0 +1,157 @@
+Performance Tuning DPDK with Intel's Platform and NICs
+======================================================
+
+This document is a step-by-step guide for getting high DPDK performance with Intel's platform and NICs.
+
+Prerequisites
+-------------
+
+Hardware platform essential requirements:
+
+1. Use a standard Intel® Xeon® server system (e.g. Ivy Bridge, Haswell or newer).
+
+2. Ensure that each memory channel has at least one memory DIMM inserted, and the memory size for each can be 4GB or above (e.g: 8GB or 16GB). You can use ``dmidecode -t memory`` to check the memory status. **Note**: This is one important element to impact the performance.
+
+Hardware platform Network Interface Card Essential requirements:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Get an high end Intel® NIC, e.g: Intel® XL710.
+
+2. Make sure each NIC has flashed the latest version of NVM/firmware, if there is.
+
+3. Use PCIe Gen3 slots, such as Gen3 x8 or Gen3 x16 because PCIe Gen2 slots can't provide enough bandwidth for 2x10G and above.
+
+4. When inserting NICs to the PCI slots, be careful about the NUMA. If you will use 2 or more ports from different NICs, please make sure these NICs on the same CPU socket.
+
+BIOS settings:
+~~~~~~~~~~~~~~
+
+1. To be sure, reset all the BIOS settings to default.
+
+2. Disable all power saving options, and set all options for best performance.
+
+3. Disable Turbo to ensure the performance scaling with core numbers increment.
+
+4. Set memory frequency to the highest number, NOT auto.
+
+5. Disable all Virtualization options when you test physical function of NIC, and turn on VT-d if you wants to use VFIO.
+
+
+Linux System Essential Requirements:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Get an widely used 64 bits Linux distribution installed, e.g. Fedora 20 - 64 bits.
+
+2. Make sure to select as more components as possible during system installation, to avoid install necessary components again and again.
+
+3. Make sure the widely used and fully validated version of kernel installed, e.g. 3.18.
+
+4. Make sure the required components are enabled for some old version of kernels, before rebuilding the kernel. The kernel may need to be rebuilt if any DPDK component is missing. Refer to Getting Started Guide on www.dpdk.org for more details.
+
+
+Grub Parameters Essential Requirements:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+1. Use the default grub file as a good start.
+
+2. Reserve 1G huge pages via grub configurations, e.g. add ``default_hugepagesz=1G hugepagesz=1G hugepages=8`` to reserve 8 huge pages in 1G size.
+
+3. Isolate CPU cores which will be used for DPDK from scheduler, e.g: isolcpus=2,3,4,5,6,7,8
+
+4. If it wants to use VFIO, additional grub parameters are needed. e.g: ``iommu=pt intel_iommu=on``
+
+
+Configurations before running DPDK
+----------------------------------
+
+1. For Intel® 40G NICs, special configurations should be set before compiling it, as follows. **Note**: This is very important::
+
+      for at least DPDK release 1.8, 2.0 and 2.1, in <dpdk_folder>/config/common_linuxapp
+      CONFIG_RTE_PCI_CONFIG=y
+      CONFIG_RTE_PCI_EXTENDED_TAG=”on”
+
+2. Build DPDK target and reserve huge page, refer to GSG guide for more details. Below scripts are for your reference::
+
+      cd <dpdk_folder>
+      make install T=x86_64-native-linuxapp-gcc -j          # Build DPDK target
+      awk '/Hugepagesize/ {print $2}' /proc/meminfo         # Get the hugepage size
+      awk '/Hugepage_Total/ {print $2} ' /proc/meminfo      # Get the total huge page numbers
+      umount `awk '/hugetlbfs/ {print $2}' /proc/mounts`    # Umount
+      mkdir -p /mnt/huge                                    # Create the hugepage mount folder
+      mount -t hugetlbfs nodev /mnt/huge                    # Mount to the specific folder
+
+3. Check the CPU layout by dpdk tools or system commands ``lscpu``::
+
+      cd <dpdk_folder>/tools
+      ./cpu_layout.py                       #Run the script to check your system's cpu layout.
+
+   Or run ``lscpu`` to check the the cores on each socket
+
+4. Check your NIC id and related socket id::
+
+      lspci -nn|grep Eth                    # List all the NICs with PCI address and device IDs.
+
+   e.g. Suppose your output is as below::
+
+      82:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+      82:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+      85:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+      85:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+
+   Check the PCI device related numa node id::
+
+      cat /sys/bus/pci/devices/0000\:xx\:00.x/numa_node
+
+   Usually ``8x:00.x`` is on socket 1, ``0x:00.x`` is on socket 0. **Note**: To get best performance, please make sure the core and NICs are in the same socket. Take ``85:00.0`` for example, it's on socket 1, then use cores on socket1 for best performance.
+
+5. Bind the test ports to igb_uio. For example bind two ports to dpdk compatible driver and check the status::
+
+      # Bind ports 82:00.0 and 85:00.0 to dpdk driver
+
+      ./<dpdk_folder>/tools/dpdk_nic_bind.py -b igb_uio 82:00.0 85:00.0
+
+      # Check the port driver status
+
+      ./<dpdk_folder>/tools/dpdk_nic_bind.py --st
+
+
+Example
+-------
+
+Below is an case of running dpdk l3fwd sample to get high performance with Intel platform and NIC.
+
+**Note**: The scenario is to get best performance with two Intel®XL710 40G ports. See below Figure1 as the performance test setup. 
+
+.. figure:: img/pf_performance_test_setup.*
+
+**Figure 1. PF_Performance_Test_setup**
+
+
+1. Insert two NICs(Intel®XL710) into the platform, and use one port per card to get best performance. The reason using two NICs is the PCIe Gen3's limitations. **Note**: As PCIe Gen3 can't provide 80G bandwidth for two 40G ports, but two different PCIe Gen3 slot can. Refer to the sample NICs output above, then we can select 82:00.0 and 85:00.0 as test ports::
+
+      82:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+      85:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583] (rev 01)
+
+2. Connect the ports to the traffic generator, such as IXIA and Spirent.
+
+3. Check the PCI devices numa node(socket id) and get the cores number on the exact socket id. In this case, 82:00.0 and 85:00.0 are both in socket1, and the cores on socket1 in the referenced platform is 18-35,54-71. Note: Don't use one core's 2 thread(e.g core18 has 2 lcores, lcore18 and lcore54), instead, use 2 logical cores from different cores(e.g core18 and core19).
+
+4. Bind these two ports to igb_uio.
+
+5. As it is known that XL710 40G port need at least two queue pairs to achieve best performance, then two queues per port will be required, and each queue pair will need a dedicated CPU core for receiving/transmitting packets.
+
+6. Basically l3fwd will be using for performance testing, with using two ports for bi-directional forwarding. Compile the l3fwd sample with default lpm mode.
+
+7. Final command line of running l3fwd could be as followings. That means use core 18 for port 0, queue pair 0 forwarding, core 19 for port 0, queue pair 1 forwarding, core 20 for port 1, queue pair 0 forwarding, core 21 for port 1, queue pair 1 forwarding::
+
+      ./l3fwd -c 0x3c0000 -n 4 -w 82:00.0 -w 85:00.0 -- -p 0x3 --config '(0,0,18),(0,1,19),(1,0,20),(1,1,21)'
+ 
+8. Configure the traffic to a traffic generator such as IXIA or Spirent.
+
+* Start creating a stream on packet generator, e.g. IXIA.
+* Set the Ethernet II type to 0x0800
+* Set the protocols to IPV4.
+* Do not set any L4 protocols, just keep it as none.**Note**: this is very important, if you set UDP or TCP protocol, you may get relative low performance since the l3fwd example default using none protocols for RSS enabling. 
+* The flow's DEST MAC, DEST IP, SRC IP's settings can be seen in the above figure. It's for the user's reference. Set the correct destination IP address according to "ipv4_l3fwd_route_array" in the l3fwd example code, such as 2.1.1.1 for port0, then it will forward the packets to port1. Set the source IP as random, **Note**: this is very important to make sure the packets will be received in multiple queues.
+
+
-- 
2.1.0

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2015-08-10 20:37 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-08-10 16:16 [dpdk-dev] [PATCH]doc: Add performance tuning guide about how to get DPDK high perf on Intel platform Zhang, Helin
  -- strict thread matches above, loose matches on Subject: below --
2015-08-10  6:34 Qian Xu
2015-08-10 13:37 ` Thomas Monjalon
2015-08-10 16:10   ` Zhang, Helin
2015-08-10 20:37     ` Qiu, Michael
2015-08-10 16:00 ` Zhang, Helin
2015-08-10 16:01   ` Thomas Monjalon

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).