Restricting access to S3 bucket resource from CloudFront to certain users

Question

I have deployed a website on CloudFront that contains an HTML file and a JS file. The JS code uses the fetch API to read a resource called data.json from my S3 bucket and updates the DOM with data read from that file. The code is as simple as fetch('./data.json'). I want to restrict access to that file to a certain group of users, and show different web UI depending on whether the user has access to that resource. To illustrate this use case, think of data.json as containing a list of items in a shop, and only a certain group of more privileged users would be able to see this list, while another group can only see descriptions of each individual item.

I'm using Cognito for authentication of this CloudFront website. There are two groups of users, A and B. A corresponds to the more privileged users while B corresponds to the other group. I have set up an IAM role and attached a policy to it denying access to data.json in my origin bucket. I confirmed that the role and policy are set up correctly using the IAM permission simulator. However, I'm seeing that the request from users from group B still succeed in getting the JSON file. I wonder where I messed up, and how I can solve this issue. Thanks in advance!