/Curriculum/AdvancedMemory & Performance

43. NumPy Memory Mapping (np.memmap)Advanced

Process 100GB+ datasets directly on disk using virtual memory mapping.

14 mins

Concept Overview

`np.memmap` enables reading and writing portions of massive binary arrays stored on disk without loading the entire multi-gigabyte file into physical RAM.

Hardware Mental Model

Memory mapping is like using Google Maps: instead of downloading the satellite images for the entire Earth into your phone, your screen only requests the specific street tile you zoom into.

Key Concepts (1)Click snippet to load in editor

`mode='r'` (read-only), `mode='w+'` (create/overwrite), `mode='r+'` (read/write in place), `mode='c'` (copy-on-write).

arr = np.memmap('data.dat', dtype='float32', mode='r', shape=(5000, 5000))

⚠ Common Mistakes & Pitfalls

Avoid these frequent beginner syntax and logic traps
❌ Incorrect:mmap_arr[0] = 5 # Forgot to flush; changes might stay in OS page buffer
✓ Correct:mmap_arr[0] = 5; mmap_arr.flush()

Always call `.flush()` to ensure cached memory pages are written back to disk.

Pro Tip: Call `mmap_arr.flush()` after batch modifications.

📌 Quick Revision

Core takeaway points from this topic
`np.memmap()`: Accesses large disk files as virtual in-memory arrays.
Processes arrays larger than total system RAM.
`flush()`: Commits dirty memory pages to physical disk.
Editor: 43. NumPy Memory Mapping (np.memmap)
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state

🎯 Try It Yourself: Practice Challenge

Hands-on Mode

Solve the objective below using NumPy vectorized syntax

Objective:Create Memmap Array

Create a memmap array `m = np.memmap('test.dat', dtype='int32', mode='w+', shape=(5,))`, set `m[:] = [1, 2, 3, 4, 5]`, and print `m`.

Expected Output Target:[1 2 3 4 5]
Practice Workspace
Python ● Ready
Initializing Python IDE...
Ctrl+Enter
Execution Result
Click "RUN CODE" to execute and inspect array state