CentOS Question posted in
The official documentation can be found here.

Not able to run pktgen-dpdk (error: Illegal instruction) – CentOS

I have followed below steps to install and run pktgen-dpdk. But I am getting "Illegal instruction" error and application stops.

System Information (Centos 8)

$ uname -a
Linux localhost.localdomain 4.18.0-240.22.1.el8_3.x86_64 #1 SMP Thu Apr 8 19:01:30 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Installing DPDK

[local@localhost dpdk-stable-20.11.1]$ cd dpdk-stable-20.11.1/
[local@localhost dpdk-stable-20.11.1]$ meson build
[local@localhost dpdk-stable-20.11.1]$ ninja -C build
[local@localhost dpdk-stable-20.11.1]$  ninja -C build install
[local@localhost dpdk-stable-20.11.1]$ sudo ldconfig
# echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

Building PKTGEN

[local@localhost dpdk-stable-20.11.1]$ export PKG_CONFIG_PATH=/usr/local/lib64/pkgconfig
[local@localhost pktgen-dpdk]$ cd pktgen-dpdk/
[local@localhost pktgen-dpdk]$ cat VERSION 
21.01.2
[local@localhost pktgen-dpdk]$ make 
[local@localhost pktgen-dpdk]$ sudo vi /etc/ld.so.conf.d/x86_64-linux-gnu.conf 
    edited"/usr/local/lib64"
[local@localhost pktgen-dpdk]$ export RTE_SDK=/home/local/dpdk-stable-20.11.1/
[local@localhost pktgen-dpdk]$ export RTE_TARGET=build

# yum install dnf-plugins-core
# yum config-manager --set-enabled powertools
# yum repolist
repo id                                    repo name
appstream                                  CentOS Linux 8 - AppStream
baseos                                     CentOS Linux 8 - BaseOS
extras                                     CentOS Linux 8 - Extras
powertools                                 CentOS Linux 8 - PowerTools

Editing default.cfg file to match system settings

[local@localhost pktgen-dpdk]$ lspci | grep Eth
01:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
01:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
02:00.0 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
02:00.1 Ethernet controller: Broadcom Inc. and subsidiaries NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
04:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
04:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
05:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
05:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
43:00.0 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]
43:00.1 Ethernet controller: Mellanox Technologies MT27800 Family [ConnectX-5]

[local@localhost pktgen-dpdk]$ vi cfg/default.cfg

    'devices': (
            #'03:00.0', '05:00.0', '81:00.0', '84:00.0'
            '04:00.0', '04:00.1'
            ),

        'allowlist': (
                '04:00.0', '04:00.1'
                #'05:00.0',
                #'84:00.0',
                #'03:00.0', '81:00.0'
                ),

        'map': (
                '[3:4].0',
                '[5:6].1',
                #'[16:17].2',
                #'[18:19].3'
                ),

Setting up the ports and attach them to DPDK

[local@localhost pktgen-dpdk]$ ./tools/run.py -s default
>>> sdk '/home/local/dpdk-stable-20.11.1/', target 'build'
<module 'cfg' from 'cfg/default.cfg'>
Setup DPDK to run 'pktgen' application from cfg/default.cfg file
[sudo] password for local:

Run the default configuration

[local@localhost pktgen-dpdk]$ ./tools/run.py default
>>> sdk '/home/local/dpdk-stable-20.11.1/', target 'build'
<module 'cfg' from 'cfg/default.cfg'>
   Trying ./usr/local/bin/pktgen
sudo -E ./usr/local/bin/pktgen -l 2,3-4,5-6,16-17,18-19 -n 4 --proc-type auto --log-level 7 --file-prefix pg -a 04:00.0 -a 04:00.1 -- -v -T -P -j -m [3:4].0 -m [5:6].1 -f themes/black-yellow.theme 



Copyright (c) <2010-2020>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pg/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.0 (socket 0)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.1 (socket 0)
EAL: No legacy callbacks, legacy socket not created

Running the pktgen directly creating Illegal Instruction error

[local@localhost pktgen-dpdk]$ sudo -E ./usr/local/bin/pktgen -l 2,3-4,5-6,16-17,18-19 -n 4 --proc-type auto --log-level 7 --file-prefix pg -a 04:00.0 -a 04:00.1 -- -v -T -P -j -m [3:4].0 -m [5:6].1 -f themes/black-yellow.theme 



Copyright (c) <2010-2020>, Intel Corporation. All rights reserved. Powered by DPDK
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: Auto-detected process type: PRIMARY
EAL: Detected shared linkage of DPDK
EAL: Multi-process socket /var/run/dpdk/pg/mp_socket
EAL: Selected IOVA mode 'VA'
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL:   using IOMMU type 1 (Type 1)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.0 (socket 0)
EAL: Ignore mapping IO port bar(2)
EAL: Probe PCI driver: net_ixgbe (8086:10fb) device: 0000:04:00.1 (socket 0)
EAL: No legacy callbacks, legacy socket not created
Illegal instruction

I have also tried same steps with dpdk-stable-19.11.8, but it not working.

[Vipin Check below] 
[local@localhost pktgen-dpdk]$ find ./ -name pktgen
./Builddir/app/pktgen
./usr/local/bin/pktgen

rte_memcpy is sending illegal instruction error
emphasized text
/* Get a clean copy of the configuration structure */ │
rte_memcpy(&conf, &default_port_conf, sizeof(struct rte_eth_conf));

Thread 1 "pktgen" received signal SIGILL, Illegal instruction.
0x000000000045a51e in _mm256_loadu_si256 (__P=0x6b5cc0 <default_port_conf>) at /usr/lib/gcc/x86_64-redhat-linux/8/include/avxintrin.h:922
(gdb) up
#1  rte_mov32 (src=0x6b5cc0 <default_port_conf> "", dst=0x7fffffffc2f0 "30035537737737737737737721431201367377177") at /usr/local/include/rte_memcpy.h:319
#2  rte_memcpy_generic (n=3064, src=0x6b5cc0 <default_port_conf>, dst=0x7fffffffc2f0) at /usr/local/include/rte_memcpy.h:461
#3  rte_memcpy (n=3080, src=0x6b5cc0 <default_port_conf>, dst=0x7fffffffc2f0) at /usr/local/include/rte_memcpy.h:874
#4  pktgen_config_ports () at ../app/pktgen-port-cfg.c:311

lscpu output:

[local@localhost pktgen-dpdk]$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              24
On-line CPU(s) list: 0-23
Thread(s) per core:  2
Core(s) per socket:  6
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               45
Model name:          Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
Stepping:            7
CPU MHz:             1199.646
CPU max MHz:         2500.0000
CPU min MHz:         1200.0000
BogoMIPS:            4000.03
Virtualization:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            15360K
NUMA node0 CPU(s):   0,2,4,6,8,10,12,14,16,18,20,22
NUMA node1 CPU(s):   1,3,5,7,9,11,13,15,17,19,21,23
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid xsaveopt dtherm ida arat pln pts md_clear flush_l1d
Tags:

2

Answers


  1. 0

    Intel Xeon E5-2620 is Sandy Bridge CPU which officially supports AVX and not AVX2.

    DPDK 20.11 meson build, ninja -C build will generate code with AVX instructions and not AVX2. But (Based on the live debug) PKTGEN forces the compiler to add AVX2 to be inserted, thus causing illegal instruction.

    Solution: edit meson.build in line 22

    from

    if get_option('enable-avx2') and cc.has_argument('-mavx2')
    add_project_arguments('-mavx2', language: 'c')
endif

    to

    if get_option('enable-avx2') and cc.has_argument('-mavx2')
    add_project_arguments('-mavx', language: 'c')
endif
    Login or Signup to reply.
  2. 0

    The above change works for you, but it does not really work for all conditions.

    The enable-avx2 flag with this change only uses AVX instruction and not AVX2 instructions on a platform which does support AVX2.

    I will need to rework the meson.build and meson_options.txt to enable AVX or AVX2 instructions when running on a platform supporting only AVX. When a platform supports AVX2 then AVX is supported as well.

    Maybe the default is AVX and when enable-avx2 is true then use AVX2 flag instead.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search