'How to create a dataframe of one index of a dataset?
I have a dataset NearGrid with dimensions (index:225, time:25933) that contains daily temperature data for 225 locations.
How can I create a dataframe for the first location (index=0) where the columns are date and tmax and each row represents one day of data (i.e. 25933 rows x 2 columns)?
Here's what I'm trying:
#import libraries
import os
import matplotlib.pyplot as plt
from netCDF4 import Dataset as netcdf_dataset
import numpy as np
from cartopy import config
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import xarray as xr
import pandas as pd
#open netcdf files
df=xr.open_mfdataset('/glacier1/mmartin/data/NOAATmax/Tmax****.nc')
#open cm stations csv and create new dataset with only points closest to stations in CMStations
CMStations=pd.read_csv('Slope95.csv')
Lat=CMStations.lat
Lon=CMStations.lon
NearGrid=df.sel(lat=Lat.to_xarray(), lon=Lon.to_xarray(), method='nearest')
#create dataframe of first location in NearGrid
NearGrid.isel(index=0).to_dataframe()
but when I do this the code runs indefinitely and nothing happens.
Solution 1:[1]
The problem was the way the data was chunked. When I saved the subsetted data as a new netcdf file and then opened it in a new notebook, it worked. I did that through this:
#import libraries
import os
import matplotlib.pyplot as plt
from netCDF4 import Dataset as netcdf_dataset
import numpy as np
from cartopy import config
import cartopy.crs as ccrs
import cartopy.feature as cfeature
import xarray as xr
import pandas as pd
#open netcdf files
df=xr.open_mfdataset('/glacier1/mmartin/data/NOAATmax/Tmax****.nc')
#open cm stations csv and create new dataset with only points closest to stations in CMStations
CMStations=pd.read_csv('Slope95.csv')
Lat=CMStations.lat
Lon=CMStations.lon
NearGrid=df.sel(lat=Lat.to_xarray(), lon=Lon.to_xarray(), method='nearest')
#save as new netcdf file
NearGrid.to_netcdf('/glacier1/mmartin/data/NOAATmax/Tmax_CMStations_19510101-20211231.nc')
I then opened this file in a new notebook and manipulated the data there
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Megan Martin |
