AWS elastic beanstalk configuration for rolling updates

Question

Currently I'm trying to solve some mystery changing settings in an Elastic Beanstalk environment. I will try to explain the situation and what is happening.

We have a Jenkins server which we can command to create a build (artifact) to upload to an S3 bucket. The S3 bucket is watched by a cloudwatch event what will trigger a codepipeline and deploys the artifact to multiple ec2 instances.

Now on the EBS environment there are some settings, for example the rolling updates part. It seems that sometimes the DeploymentPolicy is 'magically' updated from rolling to allAtOnce. Me and my colleagues don't understand how this setting is changed, because no-one touches this part of the gui.

I've tried putting these settings into config files, but that didn't work out so well. I tried adding a env.yaml in the project root. The EBS is reading this file, but I get errors that the syntax is incorrect. I tried following this document, but I can't get it to work. https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/environment-cfg-manifest.html

Also I tried adding a file in .ebextensions/rolling_updates.config with the following content:

option_settings:
  aws:elasticbeanstalk:command:
    BatchSizeType: Percentage
    DeploymentPolicy: Immutable

But that also didn't work. (I tried some other values just to check if the update worked)

Is there somebody who has done this before and can help me explain what I should do to fix this?

One thought that has crossed my mind is putting a config file in an S3 bucket and install AWS-cli on the Jenkins server. Then every deployment we can update the EBS config before uploading the artifact to S3. But I'm not sure that's a correct approach as well.

Thanks all!