'Merge two nested JSON files using JQ

I'm trying to merge two JSON files. The main destination is to overwrite environment variables in the 1st file with environment variables in the 2nd.

1st file:

{
    "containerDefinitions": [
        {
            "name": "foo",
            "image": "nginx:latest",
            "cpu": 1024,
            "memory": 4096,
            "memoryReservation": 2048,
            "portMappings": [
                {
                    "containerPort": 8080,
                    "hostPort": 0,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [
                {
                    "name": "SERVER_PORT",
                    "value": "8080"
                },
                {
                    "name": "DB_NAME",
                    "value": "example_db"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/dev/ecs/example",
                    "awslogs-region": "us-west-1",
                    "awslogs-stream-prefix": "ecs"
                }
            }
        }
    ],
    "family": "bar",
    "taskRoleArn": "arn:aws:iam::111111111111:role/assume-ecs-role",
    "executionRoleArn": "arn:aws:iam::111111111111:role/ecs-task-execution-role",
    "networkMode": "bridge",
    "volumes": [],
    "placementConstraints": [],
    "requiresCompatibilities": [
        "EC2"
    ]
}

2nd file:

{
    "containerDefinitions": [
        {
            "environment": [
                {
                    "name": "SERVER_PORT",
                    "value": "8081"
                }
            ]
        }
    ]
}

The product has to be next:

{
    "containerDefinitions": [
        {
            "name": "foo",
            "image": "nginx:latest",
            "cpu": 1024,
            "memory": 4096,
            "memoryReservation": 2048,
            "portMappings": [
                {
                    "containerPort": 8080,
                    "hostPort": 0,
                    "protocol": "tcp"
                }
            ],
            "essential": true,
            "environment": [
                {
                    "name": "SERVER_PORT",
                    "value": "8081"
                },
                {
                    "name": "DB_NAME",
                    "value": "example_db"
                }
            ],
            "mountPoints": [],
            "volumesFrom": [],
            "logConfiguration": {
                "logDriver": "awslogs",
                "options": {
                    "awslogs-group": "/dev/ecs/example",
                    "awslogs-region": "us-west-1",
                    "awslogs-stream-prefix": "ecs"
                }
            }
        }
    ],
    "family": "bar",
    "taskRoleArn": "arn:aws:iam::111111111111:role/assume-ecs-role",
    "executionRoleArn": "arn:aws:iam::111111111111:role/ecs-task-execution-role",
    "networkMode": "bridge",
    "volumes": [],
    "placementConstraints": [],
    "requiresCompatibilities": [
        "EC2"
    ]
}

I tried to do next:

jq -s 'reduce .[] as $item ({}; reduce ($item | keys_unsorted[]) as $key (.; $item[$key] as $val | ($val | type) as $type | .[$key] = if ($type == "array") then (.[$key] + $val | unique) elif ($type == "object") then (.[$key] + $val) else $val end))'  1.json 2.json

But the result is:

{
  "containerDefinitions": [
    {
      "name": "foo",
      "image": "nginx:latest",
      "cpu": 1024,
      "memory": 4096,
      "memoryReservation": 2048,
      "portMappings": [
        {
          "containerPort": 8080,
          "hostPort": 0,
          "protocol": "tcp"
        }
      ],
      "essential": true,
      "environment": [
        {
          "name": "SERVER_PORT",
          "value": "8080"
        },
        {
          "name": "DB_NAME",
          "value": "example_db"
        }
      ],
      "mountPoints": [],
      "volumesFrom": [],
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/dev/ecs/example",
          "awslogs-region": "us-west-1",
          "awslogs-stream-prefix": "ecs"
        }
      }
    },
    {
      "environment": [
        {
          "name": "SERVER_PORT",
          "value": "8081"
        }
      ]
    }
  ],
  "family": "bar",
  "taskRoleArn": "arn:aws:iam::111111111111:role/assume-ecs-role",
  "executionRoleArn": "arn:aws:iam::111111111111:role/ecs-task-execution-role",
  "networkMode": "bridge",
  "volumes": [],
  "placementConstraints": [],
  "requiresCompatibilities": [
    "EC2"
  ]
}

Could anyone help to find out how to reach the right result?

json jq

Solution 1:^[1]

Something like this will do the trick:

 (input | .containerDefinitions[0].environment | from_entries) as $new_env
| input | .containerDefinitions[].environment |= ((from_entries + $new_env) | to_entries)

^{Online demo}

In case it's unclear, the invocation should look like so:

jq -n '...' 2.json 1.json

Solution 2:^[2]

Here's a solution using tostream and has(1) to read the values from the second file, and setpath to set them in the first file:

jq 'reduce (input | tostream | select(has(1))) as $i (.; setpath($i[0]; $i[1]))' \
  1.json 2.json

Demo

When providing the files in reversed order (2.json 1.json), the context . and input have to be swapped:

jq 'reduce (tostream | select(has(1))) as $i (input; setpath($i[0]; $i[1]))' \
  2.json 1.json

Demo

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	oguz ismail
Solution 2	pmf

'Merge two nested JSON files using JQ

Solution 1:[1]

Solution 2:[2]

Sources

Related Questions

Solution 1:^[1]

Solution 2:^[2]