dpdk l2fwd-nv: Match error CPU and GPU pointers

Question

I run l2fwd-nv with next command: ./l2fwdnv -l 0-3 -n 8 -a 07:00.0,txq_inline_max=0 -- m 1 -w 2 -b 64 -p 1 -v 0 z 0

Program output:

************ L2FWD-NV ************

EAL: Detected 4 lcore(s)
EAL: Detected 1 NUMA nodes
EAL: Multi-process socket /var/run/dpdk/rte/mp_socket
EAL: Selected IOVA mode 'PA'
EAL: No available hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support...
EAL: VFIO support initialized
EAL: WARNING! Base virtual address hint (0x100a96000 != 0x7f3b1fe00000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: WARNING! Base virtual address hint (0x1016f7000 != 0x7f371fc00000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: WARNING! Base virtual address hint (0x102358000 != 0x7f331fa00000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL: WARNING! Base virtual address hint (0x102fb9000 != 0x7f2f1f800000) not respected!
EAL:    This may cause issues with mapping memory into secondary processes
EAL:   Invalid NUMA socket, default to 0
EAL: Probe PCI driver: mlx5_pci (15b3:1017) device: 0000:07:00.0 (socket 0)
common_mlx5: RTE_MEM is selected.
mlx5_pci: Size 0xFFFF is not power of 2, will be aligned to 0x10000.
EAL: No legacy callbacks, legacy socket not created

Device driver name in use: mlx5_pci... 
EAL: Error - exiting with code: 1
  Cause: GPU pointer does not match CPU pointer pDev=0x2030c0000 pBuf=0x7f3b20400000

What is the error?

ps: https://github.com/NVIDIA/l2fwd-nv

Answer 1

Are you actually running with -m 1 on the last version of l2fwd-nv ? The device memory path enabled with -m 1, should work in your case. In your question, you lack the - of -m, is it only a typo ?

Your error comes from the host pinned memory path, which is according to the documentation enabled with -m 0. This path allocates memory on the CPU, and allows the GPU to access it using cudaHostGetDevicePointer.

I don't know why, but l2fwd-nv requires the device pointer returned by cudaGetHostPointer to be equal to the initial host pointer. This is the case when your GPU has the attribute cudaDevAttrCanUseHostPointerForRegisteredMem. In your case, it looks like your GPU does not have this attribute.