/Curriculum/Practical & ProjectsData Analysis

48. NumPy for Data AnalysisIntermediate

Detect outliers with 3-Sigma rule, filter datasets, and compute group statistics.

18 mins

Concept Overview

NumPy provides the core statistical tools to clean messy real-world datasets: detecting outliers using the 3-Sigma Rule ($|X - \mu| > 3\sigma$) and filling missing values.

Hardware Mental Model

The 3-Sigma rule is a statistical metal detector: 99.7% of normal data falls within 3 standard deviations; anything beyond is flagged as an outlier.

Key Concepts (1)Click snippet to load in editor

Values where `np.abs(data - mean) > 3 * std` are statistical anomalies that distort model training.

clean = data[np.abs(data - data.mean()) <= 3 * data.std()]

⚠ Common Mistakes & Pitfalls

Avoid these frequent beginner syntax and logic traps
❌ Incorrect:clean = [x for x in data if x < 50] # Slow list comprehension
✓ Correct:clean = data[data < 50]

NumPy boolean indexing is 50x faster than Python list filtering.

Pro Tip: Use `np.clip(arr, min_val, max_val)` to clamp outliers in place.

📌 Quick Revision

Core takeaway points from this topic
3-Sigma rule detects outliers beyond $\mu \pm 3\sigma$.
`np.clip(arr, min, max)`: Clamps values between minimum and maximum bounds.
Boolean masking enables rapid dataset filtering.
Editor: 48. NumPy for Data Analysis
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state

🎯 Try It Yourself: Practice Challenge

Hands-on Mode

Solve the objective below using NumPy vectorized syntax

Objective:Clip Outlier Values

Create `arr = np.array([5, 25, 80, 40])`, clip values into `[10, 50]` with `np.clip(arr, 10, 50)`, and print.

Expected Output Target:[10 25 50 40]
Practice Workspace
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state