Conditional indexing in metricbeat using Ingest node pipeline creates a datastream

Question

I am trying to achieve conditional indexing for namespaces in elastic using ingest node pipelines. I used the below pipeline but the index getting created when I add the pipeline in metricbeat.yml is in form of datastreams.

PUT _ingest/pipeline/sample-pipeline
{
  "processors": [
    {
      "set": {
        "field": "_index",
        "copy_from": "metricbeat-dev",
        "if": "ctx.kubernetes?.namespace==\"dev\"",
        "ignore_failure": true
      }
    }
  ]
}

Expected index name is metricbeat-dev but i am getting the value in _index as .ds-metricbeat-dev. This works fine when I test with one document but when I implement it in yml file I get the index name starting with .ds- why is this happening?

update for the template :

{
   "metricbeat" : {
     "order" : 1,
     "index_patterns" : [
       "metricbeat-*"
     ],
     "settings" : {
       "index" : {
         "lifecycle" : {
           "name" : "metricbeat",
           "rollover_alias" : "metricbeat-metrics"
         },