# Log to Metrics

Turn JSON, logfmt or CSV logs into per-group counts, throughput rates, error rates and exact p50/p95/p99 latency. Table, JSON, CSV or Prometheus output.

## Run it

- **CLI:** `gizza tool log-to-metrics '{"ts":"2024-05-06T07:00:00Z","route":"/api/users","status":200,"duration_ms":37}'`
- **Web:** https://gizza.ai/tools/log-to-metrics/
- **Agents:** machine-readable descriptor (parameters JSON Schema) at https://gizza.ai/tools/log-to-metrics/tool.json

## Inputs

- `data` — Log lines _(field)_
- `format` — Input format _(field)_
- `group_by` — Group by fields _(field)_
- `value_field` — Numeric field (latency, bytes…) _(field)_
- `percentiles` — Percentiles _(field)_
- `percentile_method` — Percentile method _(field)_
- `time_field` — Timestamp field _(field)_
- `rate_unit` — Rate unit _(field)_
- `error_field` — Error field _(field)_
- `error_values` — Values that count as errors _(field)_
- `limit` — Groups to show _(field)_
- `other` — Roll the remainder into an (other) row _(field)_
- `sort` — Sort groups by _(field)_
- `output` — Output format _(field)_
- `metric_prefix` — Prometheus metric prefix _(field)_

## Output

- Metrics (text)

## Query parameters

Open the tool pre-filled and auto-run via URL:

- `data` — Log lines
- `format` — Input format
- `group_by` — Group by fields
- `value_field` — Numeric field (latency, bytes…)
- `percentiles` — Percentiles
- `percentile_method` — Percentile method
- `time_field` — Timestamp field
- `rate_unit` — Rate unit
- `error_field` — Error field
- `error_values` — Values that count as errors
- `limit` — Groups to show
- `other` — Roll the remainder into an (other) row
- `sort` — Sort groups by
- `output` — Output format
- `metric_prefix` — Prometheus metric prefix

Example: `https://gizza.ai/tools/log-to-metrics/?data=%7B%22ts%22%3A%222024-05-06T07%3A00%3A00Z%22%2C%22route%22%3A%22%2Fapi%2Fusers%22%2C%22status%22%3A200%2C%22duration_ms%22%3A37%7D&format=auto&group_by=route&value_field=duration_ms&percentiles=50%2C95%2C99&percentile_method=linear&time_field=auto-detect%2C%20or%20none&rate_unit=auto&error_field=status&error_values=5%2A%2C%20%3E%3D500%2C%20error%2C%20fatal&limit=20&other=true&sort=count&output=table&metric_prefix=log`

---

## Logs into RED metrics

Paste structured log lines and aggregate them into the numbers you usually need for an incident review or a dashboard seed: request counts, traffic share, rate over the log's own time span, error counts, error percentage, and exact latency percentiles. The tool reads NDJSON (one JSON object per line), logfmt `key=value` records, and CSV/TSV blocks with a header row. It runs locally in the browser; nothing is uploaded.

### Worked example

Input NDJSON:

```
{"ts":"2024-05-06T07:00:00Z","route":"/api/users","status":200,"duration_ms":37}
{"ts":"2024-05-06T07:00:20Z","route":"/api/users","status":500,"duration_ms":412}
{"ts":"2024-05-06T07:00:40Z","route":"/api/users","status":200,"duration_ms":58}
{"ts":"2024-05-06T07:01:00Z","route":"/api/orders","status":200,"duration_ms":120}
```

Set **Group by fields** to `route`, **Numeric field** to `duration_ms`, and **Error field** to `status`. The output shows one row per route with `count`, `percent`, a rate column, `errors`, `error_%`, `min`, `avg`, `p50`, `p95`, `p99`, `max`, and `sum`. Sort by `p_top` when you want the slowest endpoints first; sort by `errors` when you want the noisiest groups first.

### Options that matter

- **Input format** can auto-detect JSON/NDJSON, logfmt, and CSV. Set it explicitly when a sample is mixed or ambiguous.
- **Group by fields** accepts up to five comma-separated fields. Nested JSON is flattened to dotted paths, so `http.status` works.
- **Numeric field** is optional. When present, plain numbers are used as-is and durations such as `250ms`, `1.5s`, `10us`, or `2m` are normalised to milliseconds.
- **Percentiles** are exact over the pasted batch. Choose `linear` interpolation for numpy/R-style values, or `nearest` when you want a percentile to be an observed value.
- **Timestamp field** drives the rate column. Leave it blank to auto-detect common timestamp names, or set it to `none` for count-only logs.
- **Error field** plus **Values that count as errors** handles HTTP classes (`5*`), numeric comparisons (`>=500`), and severity strings (`error`, `fatal`, `panic`).
- **Output format** can be a Markdown-style table, JSON report, CSV, or Prometheus text exposition.

### Limits and edge cases

- Maximum input is **2,000,000 characters**, **200,000 lines**, and **50,000 distinct groups** per run.
- This is a one-shot batch aggregator. It does not tail a file, store state, or build time-bucketed series.
- Missing group fields are labelled `(missing)` rather than dropped.
- Missing or non-numeric values in the numeric field are counted in the summary and not treated as zero.
- If **Roll the remainder into an (other) row** is enabled, that row recomputes percentiles from the merged raw values; it does not average the visible rows.

## FAQ

<details>
<summary>What log formats can I paste?</summary>

Use NDJSON with one JSON object per line, logfmt records such as `route=/api status=500 dur=42ms`, or a CSV/TSV table with a header row. JSON arrays are not the intended input here; split them to one object per line first.

</details>

<details>
<summary>How is the rate calculated?</summary>

The tool reads the earliest and latest parseable timestamps in the batch and divides each group's count by that span. It is a batch rate over the log window you pasted, not a moving average. Set **Timestamp field** to `none` if the input has no timestamps.

</details>

<details>
<summary>Are the percentiles approximate?</summary>

No. Every parsed numeric value is kept, sorted, and reduced exactly. The `linear` method interpolates between neighbouring values; `nearest` uses nearest-rank and always returns a value that occurred in the log.

</details>

<details>
<summary>What counts as an error?</summary>

Only rows whose **Error field** matches one of the rules. Blank rules use the built-in set: `5*`, `error`, `err`, `fatal`, `critical`, `crit`, `panic`, `emerg`, and `alert`. You can also write numeric comparisons such as `>=500` or prefixes such as `4*`.

</details>

<details>
<summary>Is my log data uploaded?</summary>

No. The parser and aggregator are compiled to WebAssembly and run in your browser tab. Logs often contain user IDs, hostnames, URLs, and tokens, so the page is designed for local-only processing.

</details>

## Related tools

- [Apply a Unified Diff to a File](https://gizza.ai/tools/apply-patch/): Paste a file and a unified diff to get the patched text in your browser, with reverse apply, fuzz matching, and per-hunk conflict reports.
- [Autocomplete Trie](https://gizza.ai/tools/autocomplete-trie/): Build a prefix trie from a pasted wordlist and get ranked autocomplete suggestions for any typed prefix. Weights, typo tolerance, trie stats, JSON. Runs locally.
- [Code Chunker](https://gizza.ai/tools/code-chunker/): Split Python, Rust, JavaScript, TypeScript, Go, Java, C/C++, C#, PHP, or Swift into line-ranged chunks that keep functions and classes intact.
- [Code Formatter](https://gizza.ai/tools/code-formatter/): Beautify and re-indent minified or messy HTML, CSS, JavaScript, or JSON. Auto-detect the language, choose spaces or tabs, and format locally in your browser.
- [Code language detector](https://gizza.ai/tools/code-language-detect/): Detect the likely programming language of a pasted code snippet with ranked alternatives, confidence and explainable signals.