'rte_eth_tx_burst can not send packet out
A dpdk application which generate a few arp request packets and call rte_eth_tx_burst to send them out, some packets are not received by peer NIC port(this can be confirmed by using wireshark to capture the packets from the peer NIC), dpdk-proc-info shows no error count. But before call rte_eth_tx_burst let the app sleep 10s, it can send all the packets.
example codes:
main(){
port_init();
sleep(10);
gen_pkt(mbuf);
rte_eth_tx_burst(mbuf);
}
System setup: Ubuntu 20.04.2 LTS, dpdk-stable-20.11.3, I350 Gigabit Network Connection 1521, igb_uio driver
root@k8s-node:/home/dpdk-stable-20.11.3/build/app# ./dpdk-proc-info -- --xstats
EAL: No legacy callbacks, legacy socket not created
###### NIC extended statistics for port 0 #########
####################################################
rx_good_packets: 10
tx_good_packets: 32
rx_good_bytes: 1203
tx_good_bytes: 1920
rx_missed_errors: 0
rx_errors: 0
tx_errors: 0
rx_mbuf_allocation_errors: 0
rx_q0_packets: 0
rx_q0_bytes: 0
rx_q0_errors: 0
tx_q0_packets: 0
tx_q0_bytes: 0
rx_crc_errors: 0
rx_align_errors: 0
rx_symbol_errors: 0
rx_missed_packets: 0
tx_single_collision_packets: 0
tx_multiple_collision_packets: 0
tx_excessive_collision_packets: 0
tx_late_collisions: 0
tx_total_collisions: 0
tx_deferred_packets: 0
tx_no_carrier_sense_packets: 0
rx_carrier_ext_errors: 0
rx_length_errors: 0
rx_xon_packets: 0
tx_xon_packets: 0
rx_xoff_packets: 0
tx_xoff_packets: 0
rx_flow_control_unsupported_packets: 0
rx_size_64_packets: 4
rx_size_65_to_127_packets: 3
rx_size_128_to_255_packets: 3
rx_size_256_to_511_packets: 0
rx_size_512_to_1023_packets: 0
rx_size_1024_to_max_packets: 0
rx_broadcast_packets: 0
rx_multicast_packets: 10
rx_undersize_errors: 0
rx_fragment_errors: 0
rx_oversize_errors: 0
rx_jabber_errors: 0
rx_management_packets: 0
rx_management_dropped: 0
tx_management_packets: 0
rx_total_packets: 10
tx_total_packets: 32
rx_total_bytes: 1203
tx_total_bytes: 1920
tx_size_64_packets: 32
tx_size_65_to_127_packets: 0
tx_size_128_to_255_packets: 0
tx_size_256_to_511_packets: 0
tx_size_512_to_1023_packets: 0
tx_size_1023_to_max_packets: 0
tx_multicast_packets: 0
tx_broadcast_packets: 32
tx_tso_packets: 0
tx_tso_errors: 0
rx_sent_to_host_packets: 0
tx_sent_by_host_packets: 0
rx_code_violation_packets: 0
interrupt_assert_count: 0
####################################################
root@k8s-node:/home/dpdk-stable-20.11.3/build/app# ./dpdk-proc-info -- --stats
EAL: No legacy callbacks, legacy socket not created
######################## NIC statistics for port 0 ########################
RX-packets: 5 RX-errors: 0 RX-bytes: 785
RX-nombuf: 0
TX-packets: 32 TX-errors: 0 TX-bytes: 1920
Stats reg 0 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 1 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 2 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 3 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 4 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 5 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 6 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 7 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 8 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 9 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 10 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 11 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 12 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 13 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 14 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 15 RX-packets: 0 RX-errors: 0 RX-bytes: 0
Stats reg 0 TX-packets: 0 TX-bytes: 0
Stats reg 1 TX-packets: 0 TX-bytes: 0
Stats reg 2 TX-packets: 0 TX-bytes: 0
Stats reg 3 TX-packets: 0 TX-bytes: 0
Stats reg 4 TX-packets: 0 TX-bytes: 0
Stats reg 5 TX-packets: 0 TX-bytes: 0
Stats reg 6 TX-packets: 0 TX-bytes: 0
Stats reg 7 TX-packets: 0 TX-bytes: 0
Stats reg 8 TX-packets: 0 TX-bytes: 0
Stats reg 9 TX-packets: 0 TX-bytes: 0
Stats reg 10 TX-packets: 0 TX-bytes: 0
Stats reg 11 TX-packets: 0 TX-bytes: 0
Stats reg 12 TX-packets: 0 TX-bytes: 0
Stats reg 13 TX-packets: 0 TX-bytes: 0
Stats reg 14 TX-packets: 0 TX-bytes: 0
Stats reg 15 TX-packets: 0 TX-bytes: 0
############################################################################
update:
Thanks for your response, I modified the codes:
main(){
uint32_t port_mask = 0x1;
port_init();
check_all_ports_link_status(port_mask);
gen_pkt(mbuf);
rte_eth_tx_burst(mbuf);
}
got the print logs:
Checking link status...............................
done
Port0 Link Up. Speed 1000 Mbps - full-duplex
I think the NIC should have initallized completely, but the peer NIC port still missed a lot of packets.
Solution 1:[1]
In most working cases the Physical NIC is enumerated for Duplex (full/half), speed (1, 10, 25, 40, 50, 100, 200) and negotiated for (auto/disable) within 1 second. Anything exceeding 2 or 3 seconds is the sign of connected machine or switch not able to negotiated with Duplex, speed or auto-negotiation. Hence the recommendation is
- update the driver, firmware on both sides if the interfaces are NIC
- Test out the different connection cable as link-sense might not be reaching properly
- in case of hub or switch try fixing speed and auto-negotiation.
- I do not recommend changing from FULL duplex to Half duplex (as it could be cable or SFI issue).
As temporary work around for the time being you can use rte_eth_link_get which also states it might need It might need to wait up to 9 seconds.
Note: easy way to test if it is cable issue is running DPDK on both ends to check time required for link to be up.
Modified Code Snippet:
main(){
port_init();
RTE_ETH_FOREACH_DEV(portid) {
struct rte_eth_link link;
memset(&link, 0, sizeof(link));
do {
retval = rte_eth_link_get_nowait(port, &link);
if (retval < 0) {
printf("Failed link get (port %u): %s\n",
port, rte_strerror(-retval));
return retval;
} else if (link.link_status)
break;
printf("Waiting for Link up on port %"PRIu16"\n", port);
sleep(1);
} while (!link.link_status);
}
gen_pkt(mbuf);
rte_eth_tx_burst(mbuf);
}
or
main(){
port_init();
RTE_ETH_FOREACH_DEV(portid) {
struct rte_eth_link link;
memset(&link, 0, sizeof(link));
ret = rte_eth_link_get(portid, &link);
if (ret < 0) {
printf("Port %u link get failed: err=%d\n", portid, ret);
continue;
}
gen_pkt(mbuf);
rte_eth_tx_burst(mbuf);
}
Solution 2:[2]
It's no surprise that packets can't be sent until the physical link goes up. That takes some time, and one can use rte_eth_link_get() API to automate waiting.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Vipin Varghese |
| Solution 2 | stackinside |