remove null elements from list ansible

Question

Below is my JSON file:

[
   {
      "?xml": {
         "attributes": {
            "encoding": "UTF-8",
            "version": "1.0"
         }
      }
   },
   {
      "domain": [

         {
            "server": [
               {
                  "name": "myserv1"
               },
               {
                  "ssl": {
                     "name": "myserv1"
                  }
               },
               {
                  "log": [
                     {
                        "name": "myserv1"
                     },
                     {
                        "file-name": "/web/bea_logs/domains/mydom/myserv1/myserv1.log"
                     }
                  ]
               }
            ]
         },
         {
            "server": [
               {
                  "name": "myserv2"
               },
               {
                  "ssl": {
                     "name": "myserv2"
                  }
               },
               {
                  "log": [
                     {
                        "name": "myserv2"
                     },
                     {
                        "file-name": "/web/bea_logs/domains/mydom/myserv2/myserv2.log"
                     }
                  ]
               }
            ]
         }
      ]
   }
]

Here is my Ansible play that uses JMESPath query to read the json data.

- name: Server Names and log details
  set_fact:
    serverlog:  "{{ jsondata | json_query(jmesquery) }}"
  vars:
    jmesquery: '[].domain[].server[*].log[*].[name, "file-name"]'

- name: removing empty
  set_fact:
    serverlog:  "{{ serverlog|reject('match', '^$')|list }}"

- name: Print all server names with log details
  debug:
    msg: "{{ item }}"
  with_items:
    - "{{ serverlog }}"

Output:

TASK [Print all server names with log details] *********************************
Wednesday 02 March 2022  03:17:45 -0600 (0:00:00.100)       0:00:04.730 *******
ok: [localhost] => (item=[]) => {
    "msg": []
}
ok: [localhost] => (item=[[['myserv1', None], [None, '/web/bea_logs/domains/mydom/myserv1/myserv1.log']]]) => {
    "msg": [
        [
            [
                "myserv1",
                null
            ],
            [
                null,
                "/web/bea_logs/domains/mydom/myserv1/myserv1.log"
            ]
        ]
    ]
}

I wish to remove null from the output so that the variables are better formatted & easily accessible.

I wish to store the read data as Ansible variables in a file like below:

myserv1_log: "/web/bea_logs/domains/mydom/myserv1/myserv1.log"
myserv2_log: "/web/bea_logs/domains/mydom/myserv2/myserv2.log"

As you can see in the playbook I tried to remove the null in the playbook but it does not help.

Answer 1

You can achieve most of what you are looking for with this — quite complex — JMESPath query:

[].domain[].server[?log].log[].{
  name: join(`_`, [[?name].name | [0], `log`]), 
  filename: [?"file-name"]."file-name" | [0]
}

The trick here is to query the server list to find elements that do have a property log — with the help of server[?log], then in the log list, do the same for element that have a property name and a property file-name with [?name] and [?"file-name"] respectively.

Because those are lists, you will need to stop the projections with a pipe expression, before getting the first item of both those lists — so, with a construct like | [0].

In order to add the _log suffix, you can use the join function, by creating an array of two element, containing the name and the string log, that would be joined with an underscore: join(`_`, [[?name].name | [0], `log`]).

With all that, you end up with this list:

- filename: /web/bea_logs/domains/mydom/myserv1/myserv1.log
  name: myserv1_log
- filename: /web/bea_logs/domains/mydom/myserv2/myserv2.log
  name: myserv2_log

And now, we just need to make that a dictionary, with the help of items2dict.

All this together gives this set_fact task:

- set_fact:
  server_logs: >-
    {{
      jsondata | json_query('[].domain[].server[?log].log[].{
        name: join(`_`, [[?name].name | [0], `log`]), 
        filename: [?"file-name"]."file-name" | [0]
      }') | items2dict(key_name='name', value_name='filename')
    }}

Which would yield, if debug'ed:

TASK [debug] ******************************************************************
ok: [localhost] => 
  server_logs:
    myserv1_log: /web/bea_logs/domains/mydom/myserv1/myserv1.log
    myserv2_log: /web/bea_logs/domains/mydom/myserv2/myserv2.log