CONCEPT 04 · FOUNDATIONS
Laziness & Pipeline Fusion
"Process data only when you need it."
Plain English
Laziness means: don't compute a value until it's actually needed. Instead of processing everything upfront, you describe what to do and compute on demand. This lets you work with infinite sequences without running out of memory, and it lets multiple operations (map, filter, take) fuse into a single pass over the data.
Analogy
A recipe book vs. actually cooking. A recipe can say "all even squares from 1 to infinity." No memory is used until you ask for the first result. Compare to making all the food first and then throwing away what you don't need. Lazy evaluation is the recipe — you cook only what you serve.
You already know this
Python generators: yield produces one value at a time, on demandJavaScript async iterators: for await (const x of stream) — one chunk at a timeSQL query builders (Knex, Arel): build a query object, execute only when you call .fetch()React Suspense: component describes loading state, React decides when to renderRxJS Observables: describe a stream of events, only active when subscribed
Interactive Demo
Pipeline: map(×2) → filter(>10) → take(3) | Input: [1..10]
Strict processes everything first. Lazy processes one element at a time and stops early.
map(x×2)
1
2
3
4
5
6
7
8
9
10
filter(x>10)
1
2
3
4
5
6
7
8
9
10
take(3)
1
2
3
4
5
6
7
8
9
10
Code Example
JavaScript
// EAGER — processes ALL 10 million items immediately
const result = bigArray
.map(x => x * 2) // creates 10M-item array in memory
.filter(x => x > 10) // creates another large array
.slice(0, 3); // finally takes just 3 items
// Wasted: processed 9,999,997 items we didn't need
// LAZY (generator) — processes only what you need
function* lazyPipeline(data) {
for (const x of data) {
const doubled = x * 2;
if (doubled > 10) yield doubled; // produce one value
}
}
const gen = lazyPipeline(bigArray);
const first = gen.next().value; // processed until first match
const second = gen.next().value; // processed until second match
const third = gen.next().value; // stopped after 3 total
// Processed only as many items as needed!
// INFINITE sequence — impossible with eager, trivial with lazy
function* naturals() {
let n = 1;
while (true) yield n++; // "all natural numbers"
}
function* take(n, iter) {
for (const x of iter) {
if (n-- <= 0) return;
yield x;
}
}
const first10 = [...take(10, naturals())]; // [1,2,3,...,10]Apply when
▸Processing large files or datasets — read and process one record at a time, not all at once
▸Infinite sequences — pagination, event streams, sensor data feeds
▸Chaining multiple map/filter/take — lazy fusion avoids creating intermediate arrays
▸Expensive computations that might not be needed — compute only if the value is actually used
▸Pipeline stages — describe the full pipeline before deciding how much to run
Check Your Understanding
You need to find the first 5 users whose last login was over a year ago from a database with 10 million rows. Which approach is best?