'create dataframe from read many path files

thanks for your time.

I need to read several file paths, which are divided into months and days (/mm/dd/*.json)

I've been trying to traverse the path associated with days, but my loop always sticks with the last read:

for i_dia in range(1, 9):
  df_json = spark.read.json('/mnt/datalake/'+Year+'/'+ Month +'/'+ str(0) + str(i_dia) +'/'+ '*', mode="PERMISSIVE",multiLine = "true")
  return df_json
 
display(df_json)

How should the correct reading be done? I want to read all files in only one big dataframe please.

From already thank you very much.

Regards

Solution 1:^[1]

import pandas as pd
df_json=pd.DataFrame()
for i_dia in range(1, 9):
        df_json= pd.concat([df_json,pd.read_json(i_dia )])

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution	Source
Solution 1	Y U

'create dataframe from read many path files

Solution 1:[1]

Sources

Related Questions

Solution 1:^[1]