Files, Streams, and the Command Line in Node.js

Everything in this lesson is Node-specific — none of it runs in a browser: reading arguments and environment variables, working with files and paths, and moving data through streams without loading it all into memory at once. Every example runs with node (Node v24.18.1); the .mjs extension is used throughout because every example uses import syntax, which a plain .js file only gets with "type": "module" set in package.json.

process.argv, util.parseArgs, and Environment Variables

process.argv is the raw argument list: the path to node, the path to the script, then everything the user typed after that. node:util’s parseArgs turns the part after the script into typed, named options.

// cli-args.mjs
import { parseArgs } from "node:util";

console.log("raw argv:", process.argv);

const { values, positionals } = parseArgs({
  args: process.argv.slice(2),
  options: {
    name: { type: "string", short: "n", default: "World" },
    verbose: { type: "boolean", short: "v", default: false },
  },
  allowPositionals: true,
});

console.log("parsed values:", values);
console.log("positionals:", positionals);
console.log("NODE_ENV:", process.env.NODE_ENV ?? "(not set)");

$ NODE_ENV=production node cli-args.mjs build --name Alice -v extra-arg
raw argv: [
  'C:\\Program Files\\nodejs\\node.exe',
  'C:\\tmp\\jsnode\\cli-args.mjs',
  'build',
  '--name',
  'Alice',
  '-v',
  'extra-arg'
]
parsed values: [Object: null prototype] { name: 'Alice', verbose: true }
positionals: [ 'build', 'extra-arg' ]
NODE_ENV: production

The first two entries of argv are always the Node binary and the script path — real tooling slices them off with process.argv.slice(2) before parsing. parseArgs sorted everything else into values (named options, using each option’s declared type) and positionals (everything that isn’t a flag), in the order they appeared. process.env is a plain object of strings set by whatever started the process — here, NODE_ENV=production before node on the command line (that VAR=value cmd form is bash; the PowerShell equivalent is $env:NODE_ENV="production"; node cli-args.mjs ..., two statements rather than one).

Reading, Writing, and Inspecting Files with fs/promises

node:fs/promises gives every filesystem operation a promise-based API; node:path builds paths correctly across operating systems instead of concatenating strings with /.

// fs-basics.mjs
import { writeFile, readFile, stat, mkdir } from "node:fs/promises";
import path from "node:path";

const dir = path.join(process.cwd(), "data");
await mkdir(dir, { recursive: true });

const filePath = path.join(dir, "notes.txt");
await writeFile(filePath, "line one\nline two\nline three\n", "utf8");

const contents = await readFile(filePath, "utf8");
console.log(contents);

const info = await stat(filePath);
console.log("size:", info.size, "bytes");
console.log("isFile:", info.isFile());
console.log(path.basename(filePath), path.extname(filePath), path.dirname(filePath));

$ node fs-basics.mjs
line one
line two
line three

size: 29 bytes
isFile: true
notes.txt .txt C:\tmp\jsnode\data

mkdir(..., { recursive: true }) doesn’t fail if the folder already exists, unlike a plain mkdir. stat reports the file’s size in bytes and lets you ask isFile()/isDirectory() rather than inferring it from a naming convention. path.join and the three path.* accessors build and take apart file paths with the operating system’s own separator — \ here, / on Linux or macOS — so the same code works on both.

Reading a File Line by Line with readline

readline’s interface is itself an async iterable: for await reads one line at a time, however large the underlying file is.

// readline-count.mjs
import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";

const rl = createInterface({ input: createReadStream("data/notes.txt") });

let lineNumber = 0;
for await (const line of rl) {
  lineNumber++;
  console.log(`${lineNumber}: ${line}`);
}
console.log("total lines:", lineNumber);

$ node readline-count.mjs
1: line one
2: line two
3: line three
total lines: 3

createInterface wraps the read stream from createReadStream, splitting whatever bytes arrive into lines and stripping the line endings. Nothing here reads the file into one big string first — each line becomes available to the loop as soon as readline has assembled it from the incoming stream.

Streaming a File Larger Than You’d Want to Load at Once

A 50 MB text file is small enough to load whole without much trouble, but it’s large enough to show the difference in how much memory each approach actually uses — the same gap holds at any file size. Generate one once; every example below reads it.

// make-big-file.mjs
import { createWriteStream } from "node:fs";
import { mkdir, stat } from "node:fs/promises";

await mkdir("data", { recursive: true });

const LINES = 2_000_000; // enough rows to land around 50 MB of text
const out = createWriteStream("data/big.txt");

function writeRow(i) {
  return new Promise((resolve) => {
    const ok = out.write(`row ${i},value-${i * 2}\n`);
    if (ok) resolve();
    else out.once("drain", resolve);
  });
}

for (let i = 0; i < LINES; i++) {
  await writeRow(i);
}
out.end();
await new Promise((resolve) => out.once("finish", resolve));

const info = await stat("data/big.txt");
console.log("wrote", LINES, "lines,", info.size, "bytes");

$ node make-big-file.mjs
wrote 2000000 lines, 50333335 bytes

writeRow already respects backpressure — it waits for 'drain' whenever write() returns false — the same mechanism the dedicated example further down explains in detail. With data/big.txt on disk, load it the straightforward way first:

// load-whole-file.mjs
import { readFile } from "node:fs/promises";

const before = process.memoryUsage().rss;
const data = await readFile("data/big.txt");
const after = process.memoryUsage().rss;

let lineCount = 0;
for (const byte of data) if (byte === 10) lineCount++;

console.log("lines:", lineCount, "bytes:", data.length);
console.log("process memory (RSS) grew by MB:", Math.round((after - before) / 1e6));

$ node load-whole-file.mjs
lines: 2000000 bytes: 50333335
process memory (RSS) grew by MB: 51

readFile hands back the entire 50 MB file as one Buffer before counting its lines even starts, and the process’s resident memory grows by almost exactly the file’s size — the whole file sat in RAM at once. Compare that to doing the same counting work as a stream:

// stream-pipeline.mjs
import { createReadStream } from "node:fs";
import { pipeline } from "node:stream/promises";
import { Transform, Writable } from "node:stream";

let lineCount = 0;
let byteCount = 0;

const countLines = new Transform({
  transform(chunk, _enc, callback) {
    byteCount += chunk.length;
    for (const byte of chunk) if (byte === 10) lineCount++;
    callback(null, chunk);
  },
});

const discard = new Writable({
  write(_chunk, _enc, callback) { callback(); },
});

const before = process.memoryUsage().rss;
await pipeline(createReadStream("data/big.txt"), countLines, discard);
const after = process.memoryUsage().rss;

console.log("lines:", lineCount, "bytes:", byteCount);
console.log("process memory (RSS) grew by MB:", Math.round((after - before) / 1e6));

$ node stream-pipeline.mjs
lines: 2000000 bytes: 50333335
process memory (RSS) grew by MB: 41

Same file, same lines/bytes counts as load-whole-file.mjs — the two scripts did identical work — but memory grew less, because createReadStream only ever holds a small chunk in memory at a time and the Transform counts bytes and newlines as each chunk passes through, rather than after the whole file has arrived. This number is noisier than load-whole-file.mjs’s: three consecutive runs here gave 32, 42, and 41 MB, because a stream’s RSS depends on chunk buffering and GC timing, not just the file’s size, while readFile’s one fixed-size allocation barely varies. Even the high end of that range is well under load-whole-file.mjs’s steady ~51 MB, and a file 100x larger widens the gap further, not closer — readFile’s memory use tracks the file size directly; the stream’s mostly doesn’t.

There’s a harder limit than memory, too — one no amount of RAM fixes. fs.promises.readFile refuses a file over 2 GiB outright:

// readfile-2gib-limit.mjs
import { open, readFile } from "node:fs/promises";

// truncate() only changes the file's declared size — a sparse
// file this size costs no real disk space to create
const fh = await open("data/huge.bin", "w");
await fh.truncate(2 * 1024 ** 3 + 1024); // a little over 2 GiB
await fh.close();

try {
  await readFile("data/huge.bin");
} catch (err) {
  console.log(err.constructor.name + ":", err.message);
  console.log("code:", err.code);
}

$ node readfile-2gib-limit.mjs
RangeError: File size (2147484672) is greater than 2 GiB
code: ERR_FS_FILE_TOO_LARGE

readFile checks the file’s size before reading a single byte and throws a RangeError (code: ERR_FS_FILE_TOO_LARGE) rather than attempting to allocate a multi-gigabyte buffer — it’s a hard ceiling, not a slow path. createReadStream, readline, and pipeline never hit this wall, because none of them cares about the file’s total size; they only ever handle one chunk at a time. Above 2 GiB, readFile isn’t slower or more memory-hungry than a stream — it simply doesn’t work.

stream/promises Pipeline and Error Propagation

pipeline, from node:stream/promises, connects streams the way .pipe() does, but returns a promise that rejects — and cleans up every stream in the chain — if any one of them fails.

// pipeline-error.mjs
import { Readable, Writable } from "node:stream";
import { pipeline } from "node:stream/promises";

const flaky = new Readable({
  read() {
    this.count = (this.count ?? 0) + 1;
    if (this.count <= 3) {
      this.push(`chunk ${this.count}\n`);
    } else {
      this.destroy(new Error("upstream connection reset"));
    }
  },
});

const sink = new Writable({
  write(chunk, _enc, callback) {
    console.log("received:", chunk.toString().trim());
    callback();
  },
});

try {
  await pipeline(flaky, sink);
  console.log("pipeline finished cleanly");
} catch (err) {
  console.log("pipeline failed:", err.message);
  console.log("sink destroyed too:", sink.destroyed);
}

$ node pipeline-error.mjs
received: chunk 1
received: chunk 2
received: chunk 3
pipeline failed: upstream connection reset
sink destroyed too: true

flaky sends three good chunks, then calls destroy(err) on itself instead of a fourth. pipeline’s await sees that rejection as soon as it happens — no 'error' listener needed on either stream — and the catch block runs. sink.destroyed is true without any explicit call to destroy it: pipeline tore down the other end of the chain automatically once the source failed, which is the main reason to prefer it over wiring streams together with .pipe() by hand.

Backpressure: When write() Returns false

A writable stream buffers what you give it. If you write faster than it can drain that buffer, write() starts returning false — the signal to stop and wait for a 'drain' event before writing more.

// backpressure.mjs
import { Writable } from "node:stream";

const sink = new Writable({
  highWaterMark: 16 * 1024, // 16 KB buffer
  write(_chunk, _enc, callback) {
    setTimeout(callback, 1); // simulate a slow consumer
  },
});

const chunk = "x".repeat(4096); // 4 KB per write
let i = 0;
let sawBackpressure = false;

function writeMore() {
  let ok = true;
  while (i < 20 && ok) {
    i++;
    ok = sink.write(chunk);
    if (!ok) {
      sawBackpressure = true;
      console.log(`write() returned false after ${i} writes (buffered ${sink.writableLength} bytes)`);
    }
  }
  if (i < 20) {
    sink.once("drain", () => {
      console.log(`drain fired, writableLength now ${sink.writableLength}`);
      writeMore();
    });
  } else {
    sink.end(() => console.log("done,", i, "writes, backpressure seen:", sawBackpressure));
  }
}

writeMore();

$ node backpressure.mjs
write() returned false after 4 writes (buffered 16384 bytes)
drain fired, writableLength now 0
write() returned false after 8 writes (buffered 16384 bytes)
drain fired, writableLength now 0
write() returned false after 12 writes (buffered 16384 bytes)
drain fired, writableLength now 0
write() returned false after 16 writes (buffered 16384 bytes)
drain fired, writableLength now 0
write() returned false after 20 writes (buffered 16384 bytes)
done, 20 writes, backpressure seen: true

Four 4 KB writes exactly fill the 16 KB highWaterMark, so the 4th write() on each round returns false — the number in writableLength confirms the buffer is full, not just “probably full.” writeMore stops writing and waits for 'drain', which fires once the slow write() handler (the setTimeout) has caught up and emptied the buffer back to 0, then resumes. Ignoring the false return and writing anyway would still “work” — Node doesn’t throw — but the buffer, and your process’s memory, would keep growing for as long as the producer outpaces the consumer.

Key Takeaways

• process.argv.slice(2) is the raw arguments; parseArgs turns them into typed options and positionals. process.env reads whatever the OS or shell set before the process started.
• fs/promises gives filesystem calls a promise API; path builds and inspects paths without hand-written string concatenation.
• readline’s interface is an async iterable — for await reads a file (or any readable stream) one line at a time.
• A stream processes data as it arrives; fs.readFile loads the whole thing first. The memory gap between the two grows with the file’s size, and readFile refuses outright above 2 GiB (ERR_FS_FILE_TOO_LARGE) regardless of available RAM.
• stream/promises’ pipeline propagates errors across the whole chain and destroys every stream in it on failure; plain .pipe() doesn’t.
• write() returning false means the buffer is at its highWaterMark; wait for 'drain' before writing more, or memory grows unbounded.

Practice

// p1.mjs
import { parseArgs } from "node:util";

const { values } = parseArgs({
  args: process.argv.slice(2),
  options: {
    tag: { type: "string", multiple: true, default: [] },
  },
});

console.log("tags:", values.tag);
console.log("count:", values.tag.length);

$ node p1.mjs --tag urgent --tag backend --tag bug
tags: [ 'urgent', 'backend', 'bug' ]
count: 3

// p2.mjs
import path from "node:path";

const input = "data/notes.txt";
const parsed = path.parse(input);
console.log(parsed);

const outputPath = path.format({ ...parsed, base: undefined, ext: ".json" });
console.log(outputPath);

$ node p2.mjs
{
  root: '',
  dir: 'data',
  base: 'notes.txt',
  ext: '.txt',
  name: 'notes'
}
data\notes.json

// p3.mjs
import { createReadStream } from "node:fs";
import { createInterface } from "node:readline";

const rl = createInterface({ input: createReadStream("data/notes.txt") });

let total = 0;
let matches = 0;
for await (const line of rl) {
  total++;
  if (line.includes("two")) {
    matches++;
    console.log("match:", line);
  }
}
console.log(`${matches} of ${total} lines matched`);

$ node p3.mjs
match: line two
1 of 3 lines matched

// p4.mjs
import { Writable } from "node:stream";

const sink = new Writable({
  highWaterMark: 64 * 1024, // 4x the lesson's buffer
  write(_chunk, _enc, callback) {
    setTimeout(callback, 1);
  },
});

const chunk = "x".repeat(4096);
let i = 0;
let backpressureEvents = 0;

function writeMore() {
  let ok = true;
  while (i < 20 && ok) {
    i++;
    ok = sink.write(chunk);
    if (!ok) backpressureEvents++;
  }
  if (i < 20) {
    sink.once("drain", writeMore);
  } else {
    sink.end(() => console.log("backpressure events:", backpressureEvents));
  }
}

writeMore();

$ node p4.mjs
backpressure events: 1