Authorization concept "user has permissions according a special project" with oauth2.0/keycloak/OIDC/openapi/spring boot

Question

I try to secure a spring boot/React app, that communication is generated by openapi over keycloak with oauth2.0/OIDC.

But my security-architecture is more complex than keycloak provides (I think). We have exactly the same situation as in gitlab:

Problem 1: We have users thats can be part of groups (in our case companies) and that can be part of projects. The priviledges that the users have correspond to a group or a project.

For example: User A has the right to see ToDos for project X but not for project Y. User A has the right to edit the name for group S but not for group T.

Problem 2: We also have to synchronize with some other apps from our customers.

How would I achieve this?

Normally I could use scopes and/or permissions, but in my case the granulatrity is not enough.

At this moment I think we should just use OAuth2.0/OIDC for SSO and security features in communication and do the whole user/project/group-part in spring boot.

But Im not so familiar with the concepts of oauth, so I better ask the specialists at first ;D

Thanks and best regards, Tobias