Architecture
TrendRadar consists of two parts: the main program and the MCP analysis service. The main program handles data collection, processing, and pushing; the MCP service provides AI-based data query and analysis capabilities.
The main program's three-stage pipeline
The main program's core workflow is divided into three stages, executed in order on each run:
Stage 1: Data collection
Fetch raw information from multiple data sources. Trending-topic data is fetched via the newsnow API, covering real-time hot topics from platforms such as Zhihu, Weibo, Douyin, and Baidu. At the same time, the latest content is pulled from the RSS/Atom feeds you configure.
Stage 2: Data processing
The collected raw data goes through multiple layers of processing. First, relevant content is screened via keyword matching and AI smart filtering; then a weighted algorithm (combining factors such as overall ranking, frequency of appearance, and hotness score) scores and ranks the items; finally, duplicates are removed and newly appearing content is tagged "new" to implement incremental detection.
Stage 3: Output and push
The processed results are output in multiple forms. An HTML visual report is generated; messages are sent to the configured push channels (9 push channels are supported in total); if AI analysis is enabled, the analysis results are appended to the pushed content; and if translation is enabled, the content is translated into the specified language.
Data flow
The diagram below shows the complete journey of data from collection to final output:
Data sources (newsnow API / RSS feeds)
|
v
Data collection module
|
v
Raw news data
|
v
Keyword matching / AI filtering
|
v
Matched news items
|
v
Weighted scoring & ranking
|
v
Ranked results
|
+----+----+--------+
| | |
v v v
HTML report Push msg AI analysisMCP analysis service
The MCP Server is a standalone service that provides AI chat-based data analysis built on the Model Context Protocol. It runs separately from the main program and can be deployed independently.
Capabilities: provides 27 tool functions for querying and analyzing the local data that TrendRadar has collected. It supports data retrieval and trend analysis across dimensions such as date, keyword, and platform.
Communication: supports two communication protocols:
HTTP-- exposes an HTTP interface on port 3333, suitable for remote callsSTDIO-- standard input/output streams, suitable for local integration
Data source: MCP analyzes the data in the local
output/ directory and does not access the internet. Therefore you must first run the main program to complete data collection before MCP has any data to analyze.Configuration files
TrendRadar's behavior is controlled by the following configuration files, each responsible for a different aspect of the settings:
Configuration priority
When deploying with Docker, environment variables (the values in the
.env file) take priority over the configuration in config.yaml. We recommend keeping secrets (API Key, Webhook URL, etc.) in environment variables and keeping business-logic configuration in config.yaml.Storage architecture
TrendRadar supports two storage methods and chooses automatically based on the deployment environment:
Local storage
Uses a SQLite database, with files stored in the
output/ directory. Suitable for Docker and local deployment scenarios. The data files include the news database, the RSS database, HTML reports, and run logs.Remote storage
Supports S3-compatible cloud storage services (such as Cloudflare R2, Alibaba Cloud OSS, Tencent Cloud COS, etc.). Mainly used for GitHub Actions deployments, because Actions does not keep local files between runs and needs remote storage to persist data.
Module structure
The project's main directories and their corresponding functional modules: