'How to filter UDP packets from a specific source with linux's SO_ATTACH_REUSEPORT_CBPF

I have been trying to write a CBPF filter to direct UDP packets from specific sources (source IP+port) to specific listening sockets on the same port using SO_ATTACH_REUSEPORT_CBPF. I have a simple program which I have been using to test this and have verified that I can direct packets to individual threads with extremely simple CBPF programs (simply returning a static integer like ret 0x00000001).

Unfortunately I've tried to develop a more complex CBPF program (which ultimately I hope to programmatically change) and I haven't been able to get it to work. Below is the program:

l0: ldh     [12]                            /* load protocol */
l1: jeq     #0x800          jt l3  jf l12   /* make sure we're IPv4 */
l2: ld      [26]                            /* load IP */
l3: jeq     #0x7f000001     jt l4  jf l12   /* compare IP to 127.0.0.1, if not jump to default case */
l4: ldxb    4*([14]&0xf)                    /* set x = ip header len * 4 (x is our working register) */
l5: ldh     [x + 14]                        /* load offest at x+14 */
l6: jeq     #0x7530         jt l7  jf l8    /* jump to next port check if src port isn't 30000 */
l7: ret     #0x1                            /* if src port is 30000 assign packet to 2nd waiting thread */
l8: jeq     #0x7531         jt l9  jf l10   /* jump to next port check if src port isn't 30001 */
l9: ret     #0x2                            /* src port is 30001, assign to 3rd waiting thread */
l10: jeq    #0x7532         jt l11 jf l12   /* jump to next port check if src port isn't 30002 */
l11: ret    #0x3                            /* src port is 30002, assign to 3rd waiting thread */
l12: ret    #0x0                            /* default, assign packet to first waiting thread */

And in a C-like array:

{BPF_LD | BPF_H | BPF_ABS, 0, 0, 0x0000000c},
{BPF_JMP | BPF_JEQ, 0, 10, 0x00000800},
{BPF_LD | BPF_W | BPF_ABS, 0, 0, 0x0000001a},
{BPF_JMP | BPF_JEQ, 0, 8, 0x7f000001},
{BPF_LDX | BPF_B | BPF_MSH, 0, 0, 0x0000000e},
{BPF_LD | BPF_H | BPF_IND, 0, 0, 0x0000000e},
{BPF_JMP | BPF_JEQ, 0, 1, 0x00007530},
{BPF_RET, 0, 0, 0x00000001},
{BPF_JMP | BPF_JEQ, 0, 1, 0x00007531},
{BPF_RET, 0, 0, 0x00000002},
{BPF_JMP | BPF_JEQ, 0, 1, 0x00007532},
{BPF_RET, 0, 0, 0x00000002},
{BPF_RET, 0, 0, 0x00000000}

Frustratingly, setsockopt() doesn't return an error when I set this but it packets continue to be assigned to threads in a round-robin fashion instead of even hitting the default return at the end which seems to imply something is wrong with my filter.

Documentation and examples for this specific BPF application are sparse (only 3 stackoverflow entries even mention SO_ATTACH_REUSEPORT_CBPF!) so I've struggled to find a faster way to verify this. Any advice would be appreciated!

Edit: Adding a code snippet of how I set the socket options:

// open a socket with SO_REUSEPORT and print what's received
int sfd = socket(AF_INET, SOCK_DGRAM, 0);

int optval = 1;
setsockopt(sfd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));

struct sock_filter code[] = {
                                {BPF_LD | BPF_H | BPF_ABS, 0, 0, 0x0000000c},
                                {BPF_JMP | BPF_JEQ, 0, 10, 0x00000800},
                                {BPF_LD | BPF_W | BPF_ABS, 0, 0, 0x0000001a},
                                {BPF_JMP | BPF_JEQ, 0, 8, 0x7f000001},
                                {BPF_LDX | BPF_B | BPF_MSH, 0, 0, 0x0000000e},
                                {BPF_LD | BPF_H | BPF_IND, 0, 0, 0x0000000e},
                                {BPF_JMP | BPF_JEQ, 0, 1, 0x00007530},
                                {BPF_RET, 0, 0, 0x00000001},
                                {BPF_JMP | BPF_JEQ, 0, 1, 0x00007531},
                                {BPF_RET, 0, 0, 0x00000002},
                                {BPF_JMP | BPF_JEQ, 0, 1, 0x00007532},
                                {BPF_RET, 0, 0, 0x00000002},
                                {BPF_RET, 0, 0, 0x00000000}
                            };
struct sock_fprog bpf = {
                            .len = 1,
                            .filter = code,
                        };

if (setsockopt(sfd, SOL_SOCKET, SO_ATTACH_REUSEPORT_CBPF, &bpf, sizeof(bpf)) < 0 ) {
    lock_and_print(cout_mutex, "failed to set bpf");
}

// also set a read timeout
struct timeval tv;
tv.tv_sec = 1;  // seconds
setsockopt(sfd, SOL_SOCKET, SO_RCVTIMEO, (struct timeval *)&tv, sizeof(struct timeval));

struct sockaddr_in servaddr, cliaddr;
memset(&servaddr, 0, sizeof(servaddr));
memset(&servaddr, 0, sizeof(cliaddr));

// Filling server information
servaddr.sin_family    = AF_INET; // IPv4
servaddr.sin_addr.s_addr = INADDR_ANY;
servaddr.sin_port = htons(listen_port);

// bind given port on 0.0.0.0
bind(sfd, (const struct sockaddr *)&servaddr, sizeof(servaddr));


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source