Pyspark - Fetch all non null columns into a single row

Question

I have below requirement, need to get all the non null columns into a single row DataFrame:

TRXN_CD TRXN_BR  CODE
      A    NULL  NULL
   NULL      CD  NULL
   NULL    NULL   MOR

Expected output as below so that it can be loaded in table. Output Dataframe:

TRXN_CD TRXN_BR  CODE
      A      CD   MOR

Answer 1

You can use the first function, ignoring null values.

import pyspark.sql.functions as F

...
df = df.select(*[F.first(c, True).alias(c) for c in df.columns])
df.show(truncate=False)