container_of / CONTAINING_RECORD, not pulling full struct

Question

I was implementing my own version of a simple doubly linked list in c99 and while writing some tests for it I started to run into some strange behavior when implementing versions of list_entry which usually calls container_of. I understand what it should be doing, and the fact that it is similar to what would be used in the linux kernel and not working is what is strange to me.

I'll walk you through what I'm doing.

I'm using a bog standard doubly linked list like this:

// Simple List
typedef struct d_list {
    struct d_list *next;
    struct d_list *prev;
} d_list;

Now when testing I created a test struct to put try stuff.

// List Testing
struct ListTester {
    int id;
    struct d_list;
} ListTester;

Now this is basically what the test looks like.

struct ListTesting test1, test2; // For manipulation
struct d_list test_list;        // The list add things to
struct d_list *ptr;             // The eventual pointer
struct ListTesting *element;     // The element from container_of

list_init(&test1.list);
list_init(&test2.list);
test1.id = 1;
test2.id = 2;

list_init(&test_list);

list_prepend(&test_list, &test1.list);
list_prepend(&test_list, &test2.list);
ptr = list_pop_prev(&test_list);                        // Should return test2.list
element = container_of(ptr, struct ListTesting, list);  // Should be test2
assert(element->id == test2.id);

So when running this, element should be the test2 variable. However this fails as there is just a random value for the id. To me it seemed that container_of just didn't work on my system (Windows with MSYS2). A note here is that the list side of the struct that it pulls looks correct in my debugger, do it's just not working for what I assume would be other elements of the struct.

#define container_of(ptr, type, member) \
    ((type *) (char *) (ptr) - offsetof(type, member))

That seemed to be what the linux kernal uses, or close to it. So I did some looking and found CONTAINING_RECORD that is also used.

#define  CONTAINING_RECORD( ptr, type, member )                 \
    ( (type*) ((char *) (ptr) - (char *)(&((type *)0)->member)) )

Now replacing:

element = container_of(ptr, struct ListTesting, list);

With:

element = CONTAINING_RECORD(ptr, struct ListTesting, list);

The code suddenly works, I'm pulling the id correctly like container_of was supposed to. Now maybe there was something weird going on with offsetof producing results different than what I expect, No big deal. However I wanted to try another test.

struct ListPointed {
    int id;
    struct d_list *list;
} ListPointed;

Working with a struct that the list needs to be initialized with malloc. I'm not sure if this is what your supposed to do but I wanted to try the same test using a struct like that.

struct ListPointed test1, test2; // For manipulation
struct d_list test_list;        // The list add things to
struct d_list *ptr;             // The eventual pointer
struct ListPointed *element;     // The element from container_of

// I do some more checking but its basically just malloc
test1.list = malloc(sizeof(d_list));
test2.list = malloc(sizeof(d_list));

list_init(test1.list);
list_init(test2.list);
test1.id = 1;
test2.id = 2;

list_init(&test_list);

list_prepend(&test_list, test1.list);
list_prepend(&test_list, test2.list);
ptr = list_pop_prev(&test_list);                       // Should return test2.list
element = CONTAINING_RECORD(ptr, struct d_list, list); // Should be test2
assert(element->id == test2.id);

free(test1.list);
free(test2.list);

Now when I do this, it is back to not getting the full struct. The assert fails because it is just pulling a random number for the id. I understand what container_of and CONTAINING_RECORD are trying to do in practice, but I guess I don't really understand what is happening when it's a pointer.

Is it not accounting for the pointer size? Am I just trying to use it in the wrong context? Why does container_of fail, even though I think at the end offset gets expanded and the whole thing is roughly same as CONTAINING_RECORD.

Help understanding this behavior would be appreciated.

Answer 1

There is an error in your container_of macro.

#define container_of(ptr, type, member) \
    ((type *) (char *) (ptr) - offsetof(type, member))

The problem is the part:

(type *) (char *) (ptr)

The internal cast to (char*) is redundant, the final type is (type*). Next offsetof(...) is added to this pointer what shifts it by offsetof(...) * sizeof(type) bytes.

You need to add extra parenthesis to perform the pointer arithmetic on char*.

#define container_of(ptr, type, member) \
    ((type *) ((char *) (ptr) - offsetof(type, member)))
              ^                                        ^

-- EDIT --

The variant with ListPointer is never going to work.

The reason if that computed ptr is equal to test2.list which comes from malloc(). On the other hand test2 is an automatic variable. Those storage are unrelated, therefore test2.list cannot be used to compute &test2. One must use &test.list to do that.