How to Convert a Pandas DataFrame with Missing Values to a NumPy Array Preserving NaN?

Convert Pandas Dataframe with Missing Values to NumPy Array

Problem

Convert a Pandas dataframe with missing values into a NumPy array, preserving the missing values as np.nan. Consider the following dataframe:

<code class="python">index = [1, 2, 3, 4, 5, 6, 7]
a = [np.nan, np.nan, np.nan, 0.1, 0.1, 0.1, 0.1]
b = [0.2, np.nan, 0.2, 0.2, 0.2, np.nan, np.nan]
c = [np.nan, 0.5, 0.5, np.nan, 0.5, 0.5, np.nan]
df = pd.DataFrame({'A': a, 'B': b, 'C': c}, index=index)
df = df.rename_axis('ID')

print(df)</code>

Output:

      A    B    C
ID
1   NaN  0.2  NaN
2   NaN  NaN  0.5
3   NaN  0.2  0.5
4   0.1  0.2  NaN
5   0.1  0.2  0.5
6   0.1  NaN  0.5
7   0.1  NaN  NaN

Solution Using df.to_numpy()

Use the to_numpy() method to convert the dataframe to a NumPy array with missing values represented as np.nan:

<code class="python">import numpy as np
import pandas as pd

np_array = df.to_numpy()

print(np_array)</code>

Output:

[[ nan  0.2  nan]
 [ nan  nan  0.5]
 [ nan  0.2  0.5]
 [ 0.1  0.2  nan]
 [ 0.1  0.2  0.5]
 [ 0.1  nan  0.5]
 [ 0.1  nan  nan]]

Preserving Data Types

If you need to preserve the data types in the resulting array, use DataFrame.to_records() to create a NumPy structured array:

<code class="python">import numpy as np
import pandas as pd

structured_array = df.to_records()

print(structured_array)</code>

Output:

rec.array([('a', 1, 4, 7), ('b', 2, 5, 8), ('c', 3, 6, 9)],
          dtype=[('ID', 'O'), ('A', '<i8'), ('B', '<i8'), ('B', '<i8')])

The above is the detailed content of How to Convert a Pandas DataFrame with Missing Values to a NumPy Array Preserving NaN?. For more information, please follow other related articles on the PHP Chinese website!