'How to remove rows containing nan in .fits file with Python?
I tried to remove rows where nan occurs in an astropy table like so
a[~np.isnan(a).any(axis=1)]
but i get following error-code:
TypeError: ufunc 'isnan' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
Solution 1:[1]
I'm guessing that you want to remove rows where any column in that row has a NaN. That would be like so:
In [17]: t = Table([[1.0, np.nan, 3.0], [10, 20, np.nan]])
In [18]: t
Out[18]:
<Table length=3>
col0 col1
float64 float64
------- -------
1.0 10.0
nan 20.0
3.0 nan
In [19]: bad = np.logical_or.reduce([np.isnan(col) for col in t.itercols()])
In [20]: bad
Out[20]: array([False, True, True])
In [21]: t[~bad]
Out[21]:
<Table length=1>
col0 col1
float64 float64
------- -------
1.0 10.0
Solution 2:[2]
What do the NaNs look like in the table/array? The error you are reporting suggests a mixture of numbers and characters. If you are reading a table with pandas you could use pandas.dropna() which worked great for me in the past.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Tom Aldcroft |
| Solution 2 | Kilian |
