Graph Active Directory Users API returning lots of duplicates

Question

I'm using the Graph API in an azure function to get all users from active directory. The code I am using shown below but pseudocode is

• if there is no previous deltaLink, get all users, else use the deltalink.
• Get the first page.
• While the userPage.NextPageRequest is not null, get the next page.
• save this page to a database.

first page of 200 users returns as expected. After this I expect each subsequent page will have its limit of 200 users, until the last page but this is not what I am seeing.

Very often the next page has less than 200 users, 187 to be exact and they are mostly duplicates of what I got in the first page.

Am I doing something incorrect here to cause this behavior?

var scopes = new[] { "https://graph.microsoft.com/.default" };
IUserDeltaCollectionPage usersPage;
UserDeltaCollectionPage lastDeltaPage = new UserDeltaCollectionPage();

if (deltaLink == null)//get all users, there is no delta link from a previous full load.
            {

                usersPage = await graphClient
                   .Users
                   .Delta()
                   .Request()
                   .GetAsync();

                SaveUsersToDatabase(usersPage, sqlConnString, "insert");

            }

            else//use delta query to look for updates since last load.
            {
                lastDeltaPage.InitializeNextPageRequest(graphClient, deltaLink.ToString());
                usersPage = await lastDeltaPage.NextPageRequest.GetAsync();

                SaveUsersToDatabase(usersPage, sqlConnString, "insert");    
            }

            while (usersPage.NextPageRequest != null)
            {
                usersPage = await usersPage.NextPageRequest.GetAsync();
                SaveUsersToDatabase(usersPage, sqlConnString, "insert");
                
            }

            if (usersPage.NextPageRequest == null)//get delta link if this is the last page
            {
                usersPage.AdditionalData.TryGetValue("@odata.deltaLink", out newDeltaLink);
            }

FYI - SaveUsersToDatabase just serializes the usersPage to json and sends it to a database.

Answer 1

Regarding user count, you can check at Azure AD for the count of users and may be duplicates are there itself. Another way to check that is to pull the data by providing Top to validate the count

   usersPage = await graphClient
               .Users
               .Delta()
               .Request()
               .Top(400)
               .GetAsync();

However, I would suggest you to use PageIterator to iterate over all the users as shown below in that case, you don't need to check the nextLink.

List<User> userResult = new List<User>();
users = await graphClient
                   .Users
                   .Delta()
                   .Request()
                   .GetAsync();



var userIterator = PageIterator<User>
                .CreatePageIterator(graphClient, users, user) =>
                {
                    userResult.Add(users);
                    return true;
                });
    //get all users
    await userIterator.IterateAsync();

Answer 2

Troubleshooting shows that only the delta query is returning the duplicates, and is also returning the same ones each time.

We are going to avoid delta query for the moment and may come back to it in future.

thanks for the advice all.