Document Term Frequency Weight Statistics Tool

Batch analyze word frequency distribution in text documents, with support for TF-IDF and BM25 weight calculation

Browser execution mode: Your data is processed in your browser and is not uploaded to the server.

Speed and Stability: Processing speed depends on your device and browser. For large batch work, the desktop version may be more stable.

Loading tool, please wait...

If the online tool fails to load or run, try the desktop tool.https://tools.yikeaigc.com/

Tool Usage

Return to old version

A desktop version is available with more powerful, stable features. Open desktop version

Select text and set analysis parameters before processing. For a quick try, load sample data. Batch files show only the first 20, but all will be processed.

1Choose input method

2Add Text Files

Click to choose files or drag them here

Supports TXT, TEXT, MD, CSV, SRT, LOG. File content is read and analyzed in the browser.

2Paste Text

No text entered yet.

2Load Sample

Sample data will replace the current selection.

3Selected Content Status

Documents0

Estimated Size0 KB

Waiting for text

Add TXT files, paste text, or load sample data first.

1Tokenization Rules

Chinese word segmentation method

Min word length

Chinese word combination length

2Weighting algorithm

Word frequencyCount total occurrences and document coverage TF-IDFEvaluate term distinctiveness in the corpus BM25Weighted score with document length correction Keyword densityOutput frequency share and coverage

3BM25 parameters

K1 saturation parameter

B length normalization parameter

Common values: K1 about 1.2-2.0, B about 0.75. Values affect BM25 score ranking.

4Filter and sort

Number of results

Default sort

Filter stop wordsFilter common function words and low-info words Keep numbersKeep years, IDs, amounts, and other numbers Case-sensitive EnglishSuitable for analyzing proper nouns Output single-document reportExport batch files by original filename

5Custom word list

Custom keep words (one per line)

Custom stop words (one per line)

1Confirm task

Analysis pending

After confirming the files and settings are correct, click Start Analysis. Keep this page open during processing.

2Processing progress

Progress 0%

Waiting to start...

No analysis results yet. Complete the previous steps and start analysis first.

Instructions

Software Usage Instructions

Select text content: Click “Select Files” to bulk import text files such as TXT, MD, CSV, SRT, and LOG, or switch to “Paste Text” to enter content directly; you can also load sample data to try the workflow.
Confirm the file list: After selecting files in bulk, the interface displays the first 20 files, while the remaining files are shown as a count. All files will still be included in the actual analysis.
Set word segmentation rules: Choose the Chinese word segmentation method, minimum word length, and Chinese compound word length; if you have fixed industry terms, enter one term per line in “Custom Reserved Words.”
Configure weighting algorithms:
- Word frequency statistics: Count word occurrences, document coverage, and keyword density.
- TF-IDF weight: Evaluate word distinctiveness based on term frequency and inverse document frequency.
- BM25 weight: Calculate relevance scores by combining term frequency, document frequency, and document length.
Adjust filtering options: You can enable stop word filtering, keep numbers, distinguish English letter case, and set the number of displayed results and the default sorting method.
Save settings: Click “Save Settings” to keep commonly used parameters, which will be automatically restored the next time you open the tool; click “Restore Defaults” if you need to reconfigure.
Start analysis: After entering the “Start Analysis” page, click “Start Analysis and Next.” The tool will read the text, segment words, count word frequencies, and calculate weights.
View and download results: After analysis is complete, you can search keywords in the results table and download the current CSV results; for batch files, you can download a ZIP archive. Single-document reports are exported using the original file names, and duplicate file names are automatically distinguished with numbers.

FAQ

A: TF-IDF is suitable for identifying words that better distinguish topics within a set of documents. When a word appears frequently in the current document but rarely in other documents, it usually receives a higher TF-IDF value.

A: BM25 can be regarded as a commonly used relevance calculation method in information retrieval scenarios. Based on term frequency and inverse document frequency, it adds term frequency saturation and document length normalization, making it more suitable for comparing keyword weights in documents of different lengths.

A: For regular Chinese content, set it to 2–3 characters. If the text contains many industry terms, product names, or long-tail keywords, set it to 2–4 characters and add important terms to the custom reserved words list.

A: Stop words are usually high-frequency but low-information words such as “的,” “了,” “在,” “我们,” and “可以.” After filtering, the results table can more easily show topic terms, business terms, and keyword clues.

A: The ZIP archive contains a corpus summary table and individual document reports. Individual document reports will preserve the original file name as much as possible, such as “Document A.csv.” If duplicate names occur, names such as “Document A_1.csv” and “Document A_2.csv” will be generated automatically.

A: Keyword density indicates the proportion of occurrences of a word within the current statistical scope. A higher density means the word appears frequently, but it should still be evaluated together with TF-IDF, BM25, and document coverage to avoid mistaking common words for topic terms.