DPDK Story

Uncategorized 3 Minutes

Contents

  1. About dpdk
  2. APIs
  3. Installation on a Linux box
  4. Tunning
    1. hugepage
    2. CPU core
  5. Example application

About DPDK

DPDK is all about zero copy of data from kernel space to user space. Mainly used for networking devices where packet data needs to be copied from kernel space to user space for different networking applications like switching, routing, firewalls, deep packet inspection, packet header enhancement and so on. This is becoming driving force for moving towards virtual deployment of networking functions.

Quick Installation

///
  tar xf dpdk.tar.gz
  cd dpdk
 
///
  make config T=x86_64-native-linuxapp-gcc
  sed -ri 's,(PMD_PCAP=).*,\1y,' build/.config
 
///
  make
 
///
make -C examples RTE_SDK=$(pwd) RTE_TARGET=build O=$(pwd)/build/examples

// build single example
nsarkar@dpdkserver:~/work/dpdk/dpdk-stable-18.11.2$ make -C examples/helloworld/ RTE_SDK=$(pwd) RTE_TARGET=build O=$(pwd)/build/examples/helloworld/build

Tips

Error: numa.h file missing.

eal_memory.c:29:18: fatal error: numa.h: No such file or directory

Solution: sudo apt-get install libnuma-dev

Error: missing pcap.h

rte_eth_pcap.c:19:18: fatal error: pcap.h: No such file or directory

Solution: sudo apt-get install libpcap-dev

DPDK HelloWorld application

Build sample application

dpdk/dpdk-stable-18.11.2$ make -C examples RTE_SDK=$(pwd) RTE_TARGET=build O=$(pwd)/build/examples
 
//dpdk/examples/helloworld/main.c

With no huge-page (malloc is used instead)

nsarkar@dpdkserver:~/work/dpdk/dpdk-stable-18.11.2/build/examples/helloworld/build$ ./helloworld --no-huge
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /run/user/26841/dpdk/rte/mp_socket
EAL: Probing VFIO support...
EAL: Started without hugepages support, physical addresses not available
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
eth_virtio_pci_probe(): IOPL call failed - cannot use virtio PMD
EAL: PCI device 0000:00:04.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
eth_virtio_pci_probe(): IOPL call failed - cannot use virtio PMD
hello from core 1
hello from core 2
hello from core 3
hello from core 0
nsarkar@dpdkserver:~/work/dpdk/dpdk-stable-18.11.2/build/examples/helloworld/build$

Hugepage

Check hugepage info

sudo cat /proc/meminfo
 
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       67452 kB
DirectMap2M:     3078144 kB
DirectMap1G:     7340032 kB

Add free hugepage

sudo sysctl -w vm.nr_hugepages=128 

HugePages_Total:     128
HugePages_Free:      128
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       67452 kB
DirectMap2M:     3078144 kB
DirectMap1G:     7340032 kB

Running helloworld with hugepage support

nsarkar@dpdkserver:~/work/dpdk/dpdk-stable-18.11.2/build/examples/helloworld/build$ sudo ./helloworld --huge-dir /mnt/huge
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
EAL: PCI device 0000:00:04.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
hello from core 1
hello from core 2
hello from core 3
hello from core 0
 
======================
 
nsarkar@dpdkserver:~$ sudo ls -l /mnt/huge/
total 2048
-rw------- 1 root root 2097152 Jun 28 13:50 rtemap_0
nsarkar@dpdkserver:~$
 
HugePages_Total:     128
HugePages_Free:      127
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       67452 kB
DirectMap2M:     3078144 kB
DirectMap1G:     7340032 kB

1G Hugepage with system reboot support

mkdir /mnt/huge
mount -t hugetlbfs nodev /mnt/huge
 
##/etc/fstab
nodev /mnt/huge_1GB hugetlbfs pagesize=1GB 0 0

dpdk ethtool app

// dpdk-stable-18.11.2/build/examples/ethtool/ethtool-app/build$ sudo ./ethtool
EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no -> using unreliable clock cycles !
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
EAL: PCI device 0000:00:04.0 on NUMA socket -1
EAL:   Invalid NUMA socket, default to 0
EAL:   probe driver: 1af4:1000 net_virtio
Number of NICs: 0
EAL: Error - exiting with code: 1
  Cause: No available NIC ports!

Reference

http://cmdlinelinux.blogspot.com/2018/12/dpdk-no-ethernet-ports.html

nilanjan@nilanjan-H81M-S:~/work/dpdk-19.02/usertools$ ./dpdk-devbind.py -s

Network devices using kernel driver
===================================
0000:02:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=eth0 drv=r8169 unused= *Active*

No 'Crypto' devices detected
============================


sudo modprobe uio
sudo insmod kmod/igb_uio.ko

nilanjan@nilanjan-H81M-S:~/work/dpdk-19.02/usertools$ sudo ./dpdk-devbind.py --bind=igb_uio eth0
nilanjan@nilanjan-H81M-S:~/work/dpdk-19.02/usertools$ sudo ./dpdk-devbind.py -s

Network devices using DPDK-compatible driver
============================================
0000:02:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' drv=igb_uio unused=r8169

nilanjan@nilanjan-H81M-S:~/work/dpdk-19.02/usertools$ sudo ./dpdk-devbind.py --bind=r8169 "0000:02:00.0"
nilanjan@nilanjan-H81M-S:~/work/dpdk-19.02/usertools$ sudo ./dpdk-devbind.py -s

Network devices using kernel driver
===================================
0000:02:00.0 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 8168' if=eth0 drv=r8169 unused=igb_uio

DPDK Patch submission

https://core.dpdk.org/contribute/

Currently submitted patch: http://patches.dpdk.org/patch/57593/

Unknown's avatar

Published by nsarkar007

Published

Leave a comment