# Topic Modeler

Discover latent topics across pasted documents with deterministic browser-local LDA, including topic word lists, document mixtures, JSON, and CSV output.

## Run it

- **CLI:** `gizza tool topic-modeler "The recipe uses butter flour sugar and a hot oven.

The compiler checks module types and function signatures."`
- **Web:** https://gizza.ai/tools/topic-modeler/
- **Agents:** machine-readable descriptor (parameters JSON Schema) at https://gizza.ai/tools/topic-modeler/tool.json

## Inputs

- `documents` — Documents _(field)_
- `separator` — Document separator _(field)_
- `topics` — Topics _(field)_
- `words_per_topic` — Words per topic _(field)_
- `iterations` — Sampling iterations _(field)_
- `alpha` — Alpha (0 = auto) _(field)_
- `beta` — Beta _(field)_
- `remove_stopwords` — Remove English stopwords _(field)_
- `stopwords` — Extra stopwords _(field)_
- `min_word_length` — Minimum word length _(field)_
- `seed` — Random seed _(field)_
- `output` — Output format _(field)_

## Output

- Topic model (text)

## Query parameters

Open the tool pre-filled and auto-run via URL:

- `documents` — Documents
- `separator` — Document separator
- `topics` — Topics
- `words_per_topic` — Words per topic
- `iterations` — Sampling iterations
- `alpha` — Alpha (0 = auto)
- `beta` — Beta
- `remove_stopwords` — Remove English stopwords
- `stopwords` — Extra stopwords
- `min_word_length` — Minimum word length
- `seed` — Random seed
- `output` — Output format

Example: `https://gizza.ai/tools/topic-modeler/?documents=The%20recipe%20uses%20butter%20flour%20sugar%20and%20a%20hot%20oven.%0A%0AThe%20compiler%20checks%20module%20types%20and%20function%20signatures.&separator=blank-line&topics=5&words_per_topic=8&iterations=200&alpha=0&beta=0.01&remove_stopwords=true&stopwords=company%2C%20report%2C%20q3&min_word_length=3&seed=42&output=report`

---

## About this tool

Topic modelling finds recurring themes across a collection of documents without labels. This tool fits a small LDA (Latent Dirichlet Allocation) model directly from the text you paste, then reports each learned topic's top words and each document's mixture of topics. It is designed for meeting notes, support tickets, survey responses, research snippets, and other short corpora where you want a quick local map of repeated themes.

The model is deterministic for a given seed. It tokenises text, lowercases words, removes optional English stopwords plus any stopwords you add, prunes short tokens, and runs collapsed Gibbs sampling over the document-word matrix. No text is uploaded and no pretrained model is downloaded.

### Worked example

Paste four short documents separated by blank lines:

```text
Butter flour sugar and oven heat make a crisp pastry.
Baking dough with butter and sugar creates a golden crust.

Compiler modules check function signatures and return types.
Type errors appear when the module function returns the wrong value.
```

Set `topics = 2`, keep the default seed, and run the report output. You should see one topic whose words lean toward baking terms and one whose words lean toward compiler/module terms, followed by a document-mixture section that shows each document's strongest topic.

### Output formats

- `report` gives a readable summary: corpus size, effective priors, ranked topic labels, top words with weights, and document mixtures.
- `json` returns the full model with topics, word probabilities, document previews, and mixture weights.
- `csv` returns a topic-keys table followed by a document-topic matrix, matching the shape many topic-modelling CLIs produce.

### Limits and edge cases

This browser-safe implementation caps the corpus at 300 documents, 25,000 kept tokens, and 20,000 vocabulary terms. Very tiny corpora can produce unstable topics; use a fixed seed and try a few topic counts before treating the result as a real pattern. For PDFs, DOCX, EPUB, HTML, or transcripts, extract text with a separate tool first and paste the plain text here.

## FAQ

<details>
<summary>Is this the same as a hosted NLP topic-modelling service?</summary>

It uses the same broad LDA idea, but it is intentionally smaller and local. There are no uploads, accounts, dashboards, coherence plots, word clouds, or saved projects. The result is a quick topic word list plus a document-topic matrix you can copy elsewhere.

</details>

<details>
<summary>How many topics should I choose?</summary>

Start small. For a short pasted corpus, try 2–5 topics and increase only if the word lists merge unrelated themes. Too many topics on too little text usually creates duplicate or noisy topics.

</details>

<details>
<summary>What does alpha do?</summary>

Alpha controls how mixed each document is. Lower alpha makes each document prefer fewer topics; higher alpha allows each document to blend more topics. Leave `alpha` at `0` to use the common MALLET-style automatic value `50 / topics`.

</details>

<details>
<summary>Can I use non-English text?</summary>

Yes, if the text is whitespace-tokenised, but the built-in stopword list is English only. Turn off English stopwords or paste your own comma/space-separated stopword list for the language you are analysing.

</details>

<details>
<summary>Why did changing the seed change the topic words?</summary>

LDA sampling starts from random topic assignments. The seed makes that randomness reproducible. If a topic only appears for one seed, it may be weak; stable themes tend to reappear across nearby settings and seeds.

</details>

## Related tools

- [RAKE Keyword Extractor](https://gizza.ai/tools/rake-keywords/): Extract the top keywords and keyphrases from any document using the RAKE algorithm — ranked by relevance, right in your browser. Nothing is uploaded.
- [Extract Action Items from Meeting Notes](https://gizza.ai/tools/action-item-extractor/): Extract action items, owners, and decisions from meeting notes or daily notes with deterministic rules. Markdown checklist or JSON, private in-browser.
- [Add Line Numbers](https://gizza.ai/tools/add-line-numbers/): Add line numbers to every line of text online, like nl or cat -n — custom start, step, separator, and alignment. Free and private, runs in your browser.
- [ANSI Log Renderer](https://gizza.ai/tools/ansi-log-renderer/): Paste ANSI-colored terminal output or CI logs and render them as HTML, or strip escape codes to plain text. Handles 16-color, 256-color, and truecolor SGR codes.
- [API response diff](https://gizza.ai/tools/api-response-diff/): Compare two JSON API responses and see only the meaningful changes: ignore request ids, timestamps and UUIDs, match arrays by key or as sets, export a JSON Patch.