Overview

AI Crypto News uses a multi-stage pipeline to collect, process, and analyze cryptocurrency news. Our system runs continuously, fetching new articles every 5 minutes, classifying them every 10 minutes, and generating entity summaries every 30 minutes.

1. News Collection

We aggregate RSS feeds from 19+ cryptocurrency news publications. Our collector:

• Fetches articles from all sources in parallel
• Deduplicates content using URL matching
• Enriches articles with Open Graph images when missing
• Stores articles in Redis cache for fast retrieval

Update Frequency: Every 5 minutes

2. Sentiment Classification

Each article is analyzed by our AI model (Llama 3.2) to determine market sentiment:

3. Importance Scoring

Our AI assigns importance scores from 1-10 based on potential market impact:

• 1-3 (Low): Minor updates, routine news, educational content
• 4-6 (Medium): Noteworthy developments, partnership announcements, technical milestones
• 7-9 (High): Significant market events, major protocol upgrades, regulatory news
• 10 (Critical): Market-moving events, security incidents, major institutional moves

4. Entity Extraction

We automatically identify and tag entities mentioned in articles:

• Cryptocurrencies: 84+ symbols (BTC, ETH, SOL, etc.) and full names
• ETFs: Bitcoin and Ethereum ETF tickers (IBIT, FBTC, ARKB, etc.)
• Key Terms: airdrop, listing, mainnet, halving, regulation, hack, etc.
• Protocols & Exchanges: Major DeFi protocols and centralized exchanges

5. Entity Summaries

For major cryptocurrencies (BTC, ETH, SOL, XRP, BNB, ADA, DOGE, TRX, XLM, LINK), we generate AI-powered market analysis summaries:

• Paragraph 1: Current developments and main news themes
• Paragraph 2: Market sentiment and trading implications
• Paragraph 3: Upcoming catalysts and market outlook

Update Frequency: Every 30 minutes, based on the latest 10 articles per entity

6. Caching Strategy

We use a multi-layer caching system for optimal performance:

• In-Memory Cache: Fastest access for frequently requested data
• Redis Cache: Persistent storage with configurable TTL
• Stale-While-Revalidate: Serve cached data while refreshing in background

This ensures you always get fast responses while maintaining data freshness.

AI Model Details

We use locally-hosted Ollama with Llama 3.2 models for privacy and speed:

• Classification: Llama 3.2 3B parameter model (higher accuracy)
• Summarization: Llama 3.2 1B parameter model (faster generation)
• Temperature: 0.3 (focused, consistent outputs)
• Processing: All AI inference runs locally - no data sent to external APIs

Limitations

While our AI system strives for accuracy, please be aware of these limitations:

• AI sentiment analysis may misinterpret sarcasm, nuance, or complex narratives
• Importance scores are algorithmic estimates, not guarantees of market impact
• Entity extraction may miss context-dependent references
• News aggregation depends on source RSS feed availability and formatting
• Market conditions can change faster than our update intervals

Always verify important information with original sources and conduct your own research.