Remove Duplicate Lines
Paste text and get it back with duplicate lines removed — order preserved. Optional case-insensitive matching, whitespace trim, keep-first or keep-last, consecutive-only (uniq) mode, and blank-line removal. Runs in your browser; nothing is uploaded.
About this tool
Remove Duplicate Lines takes pasted text and returns it with duplicate lines removed. By default the first occurrence of each line is kept and the original order is preserved — so the output is paste-ready.
- Keep occurrence — keep the first (default) or the last occurrence of each repeated line.
- Ignore case — treat
Foo,foo, andFOOas the same line. - Trim whitespace — ignore leading/trailing spaces and tabs when comparing, so
itemanditemcollapse together (the kept line is also trimmed). - Only consecutive duplicates (uniq) — only collapse runs of identical lines,
like the Unix
uniqcommand; a line that reappears later is kept. - Remove blank lines — also drop empty (or whitespace-only) lines.
Everything runs locally in your browser via WebAssembly — your text is never uploaded.
Handy for
- Cleaning up lists, exports, logs, and word/URL/email collections.
- Deduplicating a file before importing it somewhere.
- Collapsing repeated log lines with the consecutive-only (
uniq) mode.
Want to count the repeats instead of removing them? Use the Find Duplicate Lines tool, which lists each repeated line with its count.
FAQ
Where does a line end up when I choose "keep last"?
At the position of its last occurrence. With input a, b, a, keeping the first gives a, b; keeping the last gives b, a — the surviving a sits where the final repeat was, and the rest of the order is preserved around it.
How is the consecutive-only mode different from normal deduplication?
It behaves like Unix uniq: only runs of identical adjacent lines collapse to one. A line that shows up again later — after different lines in between — is kept. Use it to squash repeated log lines without losing legitimate re-occurrences.
If I combine "ignore case" and "trim", which version of the line is kept?
The occurrence chosen by your keep setting (first or last), with one tweak: when trim is on, the kept line is output with its leading/trailing whitespace removed. Ignore-case only affects matching — the kept line's original casing is untouched.
Is there a limit on how much text I can paste?
No fixed line limit — processing is a single pass in WebAssembly, so even large lists dedupe quickly, and nothing is sent to a server regardless of size.
Developer & Automation Access
Run it from the terminal
Same engine as this page, headless — via the gizza CLI:
gizza tool remove-duplicate-lines "Paste lines to remove duplicates from…"New to the CLI? Get gizza →
Open it by URL
Pre-fill and auto-run this tool with query parameters — the names match the API/CLI:
https://gizza.ai/tools/remove-duplicate-lines/?text=Paste%20lines%20to%20remove%20duplicates%20from%E2%80%A6&ignore_case=true&trim=true&adjacent_only=true&keep=first&remove_empty=trueMachine-readable descriptor: tool.json — title + parameters JSON Schema for agents.