Angular Dynamic styleUrls with environment variable are not loaded by StorybookJS

Question

Currently I'm working on a project where we are building our application in multiple styles. Our components have one base component.scss file and multiple stylesheets for the project-specific styling. With an environment variable we let angular know which stylesheets to compile. So for every component the following stylesheets exist: button.component.scss, button.component.stylea.scss and button.component.styleb.scss. In the component itself, we use the environment variable to declare the specific file to use:

@Component({
  selector: 'button[app-button]',
  templateUrl: 'button.component.html',
  // Below should become button.component.stylea.scss
  styleUrls: ['button.component' + environment.componentScssExtension] 
})

In our environment file, we have:

componentScssExtension: '.stylea.scss',

For Angular to build the project, this works fine. We have two npm commands to choose which build config te use (either stylea or styleb). However, when running StorybookJS, it seems the environment variable is not being used and we get an unstyled button component, even though we do set the angular config to let storybook know which environment files to use:

"storybook": {
  "builder": "@storybook/angular:start-storybook",
  "options": {
    "browserTarget": "APP:build",
    "styles": [
      "src/styles.scss"
    ]
  },
  "configurations": {
    "stylea": {
      "browserTarget": "APP:build:dev-stylea"
    },
    "styleb": {
      "browserTarget": "APP:build:dev-styleb"
    }
  }
}

In dev-stylea and dev-styleb, we tell which environment to use.

"dev-stylea": {
  "assets": [
    "src/assets",
    "src/assets/images/favicon.ico"
  ],
  "fileReplacements": [
    {
      "replace": "src/environments/environment.ts",
      "with": "src/environments/environment.stylea.ts"
    }
  ],
  "styles": [
    "src/styles.scss",
    "src/styles/stylea/stylea.scss"
  ]
},

StorybookJS gives no errors when compiling and we do see the button, but dynamically setting the styleUrls in our component like this (styleUrls: ['button.component' + environment.componentScssExtension]) doesn't seem to work for StorybookJS. Does anyone have a solution?