Find duplicate columns in a table
Paste CSV/table data and detect columns with identical values. Keep the first column, report redundant copies, or output a cleaned CSV — all in your browser.
About this tool
Duplicate Column Detector finds repeated columns in CSV/table data. Two
columns are duplicates when their values match down every row. By default the
header name is ignored, so email and contact count as duplicates if their cell
values are identical; turn that off when you only want repeated header names with
matching values.
The tool keeps the first (leftmost) column in each duplicate group and treats later matches as redundant copies. Use the default report to see the groups, switch to CSV output to remove redundant columns, or choose JSON for scripting. Everything runs locally in your browser — your data is not uploaded.
Worked example
Input:
id,name,email,id_copy,contact
1,Alice,[email protected],1,[email protected]
2,Bob,[email protected],2,[email protected]
Default report:
Scanned 5 columns across 2 data rows.
Found 2 duplicate column groups; 2 redundant columns can be removed (3 columns remain unique).
Duplicate column groups (kept → redundant copies):
keep "id" (col 1) == drop "id_copy" (col 4)
keep "email" (col 3) == drop "contact" (col 5)
Use output=csv to get the table with the redundant columns removed.
With Output = Cleaned CSV, the result is:
id,name,email
1,Alice,[email protected]
2,Bob,[email protected]
Options
- First row is a header — on by default; header names are used in reports and preserved in cleaned CSV output.
- Delimiter — comma, tab, semicolon, or pipe.
- Ignore case — on by default, so
Aliceandalicecompare equal. - Ignore whitespace — on by default, so stray spaces and repeated whitespace do not prevent a match.
- Ignore header names — on by default; duplicate detection compares values rather than requiring names to match. Turn it off to require matching header names too.
- Output — human report, cleaned CSV, or JSON groups.
Limits
- This finds exact duplicate value sequences after optional case/whitespace normalization. It does not do fuzzy or semantic column matching.
- The first duplicate is kept. Reorder columns before pasting if a different copy should survive.
- Ragged rows are allowed; missing cells are compared as empty strings.
- Very large files should be handled in a data-cleaning script; this page is best for paste-sized tables and quick audits.
FAQ
Does it compare the header names?
By default, no. The common duplicate-column cleanup pattern compares the values, so differently named columns with the same cells are considered duplicates. Turn Ignore header names off when names must also match.
Which duplicate column is kept?
The first (leftmost) column in each duplicate group is kept and later copies are reported or removed. This is deterministic and mirrors common spreadsheet and pandas cleanup workflows.
Can it remove the duplicate columns for me?
Yes. Set Output to Cleaned CSV to emit the table with redundant columns removed. The header row is preserved when First row is a header is enabled.
Is my table uploaded anywhere?
No. The tool is compiled to WebAssembly and runs entirely in your browser. Your CSV/table data never leaves your device.
Developer & Automation Access
Run it from the terminal
Same engine as this page, headless — via the gizza CLI:
gizza tool duplicate-column-detector "id,name,email,id_copy,contact
1,Alice,[email protected],1,[email protected]
2,Bob,[email protected],2,[email protected]"New to the CLI? Get gizza →
Open it by URL
Pre-fill and auto-run this tool with query parameters — the names match the API/CLI:
https://gizza.ai/tools/duplicate-column-detector/?data=id%2Cname%2Cemail%2Cid_copy%2Ccontact%0A1%2CAlice%2Ca%40x.com%2C1%2Ca%40x.com%0A2%2CBob%2Cb%40y.com%2C2%2Cb%40y.com&header=true&delimiter=comma&ignore_case=true&ignore_whitespace=true&ignore_header_name=true&output=reportMachine-readable descriptor: tool.json — title + parameters JSON Schema for agents.