'Add rows to dataframe in loop Python

I am extracting data from .nc files for 2757 GPS coordinates. For now I am able to extract data and create a csv file for each location individually, however it is not efficient.

I would like to have a code that is able to add lines to an existing dataset for each location within the already existing loop.

The code to individually extract csv files looks like this :

lat = data.variables['lat'][:]
lon = data.variables['lon'][:]

grid = pd.read_csv('E:/Master Thesis/SPEI/Coordinates_Africa.csv')
grid['CELLID'] = grid['CELLID'].astype('string')


for index, row in grid.iterrows():
    location = row['CELLID']
    location_lat = row['latitude_m']
    location_lon = row['longitude_']
    

    # Squared difference of lat and lon 
    sq_diff_lat = (lat - location_lat)**2
    sq_diff_lon = (lon - location_lon)**2
    
    # Identifying the index of the minimum value for lat and lon 
    min_index_lat = sq_diff_lat.argmin()
    min_index_lon = sq_diff_lon.argmin()
    
    spei = data.variables['spei']
    
    # Creating an empty pandas dataframe
    starting_date = '1901-1-15'
    ending_date = '2021-1-1'
    date_range = pd.date_range(start = starting_date, end = ending_date, freq="M")
    df = pd.DataFrame(0, columns = ['SPEI'], index = date_range)
    df.index.names = ['Date']
    
    dt = np.arange(0, data.variables['time'].size)
    
    for time_index in dt:
        df.iloc[time_index] = spei[time_index,min_index_lat ,min_index_lon]
    
    df = df.transpose()
    

    df.to_csv(location+'.csv')

The final idea would be to have a dataframe looking like this :


|                 | Date 1 | Date 2 | Date..... |
|-----------------|--------|--------|-----------|
|SPEI + location1 | .....  |........|...........|
|SPEI + location2 | .....  |........|...........|
|SPEI + location3 | .....  |........|...........|

and so on with 2757 rows and 1440 columns for the dates.



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source