AWS S3 policy restrict folder delete

Question

I have a S3 bucket named "uploads" with this structure:

uploads|
       |_products
       |_users
       |_categories
       |_...  

I want restrict users from deleting folders (products, users, ...) but they can delete objects inside those folers. My policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectTagging",
                "s3:GetBucketLocation",
                "s3:GetObject",
                "s3:GetObjectTagging"
            ],
            "Resource": [
                "arn:aws:s3:::uploads",
                "arn:aws:s3:::uploads/*"
            ]
        },

        {
            "Effect": "Allow",
            "Action": [
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::uploads/banners/*",
                "arn:aws:s3:::uploads/brands/*",
                "arn:aws:s3:::uploads/categories/*",
                "arn:aws:s3:::uploads/products/*",
                "arn:aws:s3:::uploads/users/*"
            ]
        }

    ]
} 

But i tested and user was able to delete folder, where did i go wrong?