/Curriculum/IntermediateData Cleaning

28. Missing Values & NaNIntermediate

Detect and handle np.nan using isnan(), nanmean(), nanmax(), and nan_to_num().

14 mins

Concept Overview

`np.nan` represents missing or undefined floating-point values. Standard functions propagate NaN (`1 + nan = nan`), while `nan*` functions ignore missing entries.

Hardware Mental Model

A `NaN` is a blank page in a ledger. Standard `sum()` panics and returns NaN. `np.nansum()` skips the blank page and adds up all the valid numbers.

Key Concepts (1)Click snippet to load in editor

By IEEE 754 floating-point standard, `np.nan == np.nan` evaluates to `False`. You must ALWAYS use `np.isnan(arr)` to detect NaNs.

mask = np.isnan(arr); avg = np.nanmean(arr)

⚠ Common Mistakes & Pitfalls

Avoid these frequent beginner syntax and logic traps
❌ Incorrect:mask = (arr == np.nan) # ALWAYS returns all False!
✓ Correct:mask = np.isnan(arr)

NaN is never equal to anything, including itself. `== np.nan` will fail 100% of the time.

Pro Tip: Always use `np.isnan(arr)` to detect missing values.

📌 Quick Revision

Core takeaway points from this topic
`np.nan`: IEEE 754 Not-a-Number placeholder for missing data.
`np.isnan(arr)`: Detects missing values (returns boolean mask).
`np.nanmean()`, `np.nansum()`, `np.nanmax()`: Aggregations that ignore NaNs.
`np.nan_to_num(arr, nan=0)`: Replaces NaNs with zeros or default values.
Editor: 28. Missing Values & NaN
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state

🎯 Try It Yourself: Practice Challenge

Hands-on Mode

Solve the objective below using NumPy vectorized syntax

Objective:Calculate Mean Ignoring NaNs

Create `arr = np.array([10.0, np.nan, 20.0])`, calculate its mean ignoring NaNs using `np.nanmean(arr)`, and print.

Expected Output Target:15.0
Practice Workspace
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state