Read .asc data and .dat in Python for project

Question

I have been trying to find a way to read data I found from:

https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1028

I was under the wrong impression I could treat ASC as a netCDF and found that out very quickly. Is there a way for me to read the data and put it into arrays that I can turn into graphs?

I'm trying to read the sediment flux of different rivers throughout the years and make a change over time graph at the least.

There are two types of files I could use, one is a .dat and the other is a .asc; any syntax for being able to read the data without seeing lines of "NAN" or "-999".

I have been trying to find a way to read data I found from:

https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1028

I was under the wrong impression I could treat ASC as a netCDF and found that out very quickly. Is there a way for me to read the data and put it into arrays that I can turn into graphs?

I'm trying to read the sediment flux of different rivers throughout the years and make a change over time graph at the least.

There are two types of files I could use, one is a .dat and the other is a .asc; any syntax for being able to read the data without seeing lines of "NAN" or "-999".

When using code for data

The code I used for the asc.

np.loadtxt('ocean_flux_tss_1d_map.asc', skiprows = 6)

I get this back. This is the map in data form, it has a shape of (180, 360) and I need to find a way to load the map in python. Is there any code for that? Is there an alternative way I need to load the data to accomplish this?

[[-99. -99. -99. ... -99. -99. -99.]
 [-99. -99. -99. ... -99. -99. -99.]
 [-99. -99. -99. ... -99. -99. -99.]
 ...
 [-88. -88. -88. ... -88. -88. -88.]
 [-88. -88. -88. ... -88. -88. -88.]
 [-88. -88. -88. ... -88. -88. -88.]]

Answer 1

Don't read in binary mode if you don't have to. The .dat files are plain text and look like this, which should be super easy to use. Just read them using

with open('ocean_flux_tss_2d.dat', 'r') as f:

-178.750,-77.0000,5.04768E-03,32128.7
-176.250,-77.0000,2.24318E-02,32128.7
-173.750,-77.0000,2.53902E-02,39113.5
-171.250,-77.0000,5.86083E-02,58934.0
-168.750,-77.0000,0.886519,179959.
-166.250,-77.0000,0.460672,692235.
-163.750,-77.0000,8.16516E-02,8815.86
-161.250,-77.0000,5.67409E-02,8815.86
-158.750,-77.0000,0.357408,46922.3

Or just load them into a pandas dataframe....

import pandas as pd

df = pd.read_csv("ocean_flux_co2_2d.dat", header=None)
print(df.head())
-------------------------------------------------------
        0     1         2         3
0 -178.75 -77.0  0.000003   32128.7
1 -176.25 -77.0  0.000599   32128.7
2 -173.75 -77.0  0.001649   39113.5
3 -171.25 -77.0  0.003838   58934.0
4 -168.75 -77.0  0.007192  179959.0

The .asc files are a text version of the images like this, it's doubtful you're going to make much sense of these without digging deeper into how these are made.

All in all, it's never a bad idea to just try opening your files in a plain text editor. If it's actually binary data, you'll immediately know by the symbols everywhere...