python - Removing nan values from an array

ID : 10390

viewed : 30

Tags : pythonarraysnumpynanpython

Top 5 Answer for python - Removing nan values from an array

vote vote

90

If you're using numpy for your arrays, you can also use

x = x[numpy.logical_not(numpy.isnan(x))] 

Equivalently

x = x[~numpy.isnan(x)] 

[Thanks to chbrown for the added shorthand]

Explanation

The inner function, numpy.isnan returns a boolean/logical array which has the value True everywhere that x is not-a-number. As we want the opposite, we use the logical-not operator, ~ to get an array with Trues everywhere that x is a valid number.

Lastly we use this logical array to index into the original array x, to retrieve just the non-NaN values.

vote vote

88

filter(lambda v: v==v, x) 

works both for lists and numpy array since v!=v only for NaN

vote vote

79

Try this:

import math print [value for value in x if not math.isnan(value)] 

For more, read on List Comprehensions.

vote vote

65

For me the answer by @jmetz didn't work, however using pandas isnull() did.

x = x[~pd.isnull(x)] 
vote vote

57

As shown by others

x[~numpy.isnan(x)] 

works. But it will throw an error if the numpy dtype is not a native data type, for example if it is object. In that case you can use pandas.

x[~pandas.isna(x)] or x[~pandas.isnull(x)] 

Top 3 video Explaining python - Removing nan values from an array

Related QUESTION?