NDJSON to matrix

About this tool

NDJSON (also called JSON Lines or JSONL) is great for logs and streams because each line is a complete JSON record. It is less convenient when you need a rectangular table: one record may have user.tier, the next may omit it, and nested payloads can hide the numeric columns you actually want.

This tool parses each non-blank line independently, flattens nested objects into column paths such as user.geo.lat, takes the union of every path it sees, and writes one aligned row per record. Missing paths and JSON null become your chosen fill value, so ragged event streams become CSV, TSV, a whitespace-aligned matrix, or a JSON array of rows without writing a script.

Worked example

Input:

{"id":1,"latency_ms":12,"user":{"tier":"pro"}}
{"id":2,"latency_ms":940}
{"id":3,"latency_ms":31,"user":{"tier":"free"}}

With the defaults (format=csv, headers on, first-seen columns, blank fill) the output is:

id,latency_ms,user.tier
1,12,pro
2,940,
3,31,free

Turn on Numeric columns only, set Fill for missing cells to 0, and turn headers off when you need a bare numeric matrix for numpy.loadtxt, R, Octave, or a plotting package. Use Columns to keep when you want a stable schema in a specific order.

Controls that matter

• Output format — CSV, TSV, aligned matrix, or json array-of-arrays.
• Nested arrays — index arrays into columns (reading.0, reading.1), keep the full array as JSON in one cell, or skip array-valued columns.
• Column order — first seen in the stream, alphabetical for stable diffs, or coverage-first so the most-populated columns appear first.
• Max depth — cap flattening and keep deeper objects as compact JSON when a payload would explode into too many columns.
• Invalid lines — stop at the first bad line with its line number, or skip malformed lines and convert the rest.

Limits and edge cases

• Input is capped at 5,000,000 bytes, 50,000 non-blank lines, and 2,000 distinct column paths.
• JSON objects become rows keyed by flattened paths. Bare JSON arrays become positional columns 0, 1, 2 and bare scalars go into a single value column.
• numeric_only keeps columns whose present values are all finite numbers; numeric-looking JSON strings count, but booleans, labels and non-finite tokens do not.
• Duplicate paths inside one record keep the last value written, which can happen when path separators collide with literal key names.
• CSV output follows RFC 4180 quoting for delimiters, quotes and newlines inside cells.

FAQ