Build a Social Listening Agent: Weibo + Douyin

TL;DR — A Python agent that monitors Weibo hot search + Douyin trending every 15 minutes, uses an LLM to classify relevance, and fires webhook alerts when it finds something your brand should care about. Cost: $0.48/day for data + ~$0.96/day for LLM classification = ~$1.44/day ($43/month). Full code included.

Brand crises don’t wait for your morning check-in. A negative viral post at 2 AM Tuesday can accumulate 50M views before your team sees it at 9 AM. The 7-hour gap is the difference between “we responded quickly” and “we were blindsided.”

This agent closes that gap to 15 minutes, for $43/month.

What the agent does

Every 15 minutes, 24/7:

1. Pulls Weibo’s current hot search list (50 topics with heat scores)
2. Pulls Douyin’s hot account/trending list
3. Sends both lists to an LLM with your brand keywords and classification rules
4. If anything is flagged as relevant: fires a webhook (Slack, Discord, email, PagerDuty)
5. Logs everything for trend analysis

Architecture

┌─────────────────┐     ┌──────────────┐     ┌─────────────┐     ┌──────────┐
│ Scheduler       │────▶│ Data Fetch   │────▶│ LLM Classify│────▶│ Alert    │
│ (every 15 min)  │     │ Weibo+Douyin │     │ Relevant?   │     │ Webhook  │
└─────────────────┘     └──────────────┘     └─────────────┘     └──────────┘
                              │                      │
                              ▼                      ▼
                        ┌──────────┐          ┌──────────┐
                        │ Raw Log  │          │ Alert Log│
                        └──────────┘          └──────────┘

Total API calls per cycle: ~5 (2 data + 1 LLM + optional 2 detail fetches) Total API calls per day: ~480 Daily cost: ~$1.44

Complete code

#!/usr/bin/env python3
"""Social listening agent: Weibo + Douyin.

Monitors hot search every 15 minutes, classifies relevance, sends alerts.
Run as a long-running process or via cron every 15 minutes.
"""

import json
import os
import time
from datetime import datetime, timezone
from pathlib import Path
from urllib.request import Request, urlopen

from openai import OpenAI

# --- Configuration ---
BRAND_KEYWORDS = ["你的品牌名", "YourBrand", "竞品A", "竞品B"]
INDUSTRY_KEYWORDS = ["AI", "大模型", "agent", "智能体"]
WEBHOOK_URL = os.environ.get("ALERT_WEBHOOK_URL", "")  # Slack/Discord webhook
CHECK_INTERVAL = 900  # 15 minutes in seconds
LOG_DIR = Path("./listening_logs")
LOG_DIR.mkdir(exist_ok=True)

client = OpenAI(
    base_url="https://api.sandbase.ai/v1",
    api_key=os.environ["SANDBASE_API_KEY"],
)


# --- Data fetching ---
def fetch_weibo_hot_search() -> list:
    """Get current Weibo hot search list."""
    resp = client.chat.completions.create(
        model="weibo/web-v2/hot-search",
        messages=[],
    )
    data = json.loads(resp.choices[0].message.content)
    # Normalize to list of {title, heat, url}
    if isinstance(data, dict):
        return data.get("data", data.get("items", []))
    return data if isinstance(data, list) else []


def fetch_douyin_trending() -> list:
    """Get Douyin hot account/trending list."""
    resp = client.chat.completions.create(
        model="douyin/billboard/hot-account-list",
        messages=[],
    )
    data = json.loads(resp.choices[0].message.content)
    if isinstance(data, dict):
        return data.get("data", data.get("items", []))
    return data if isinstance(data, list) else []


def fetch_weibo_topic_detail(topic_id: str) -> dict:
    """Get details for a specific hot topic."""
    resp = client.chat.completions.create(
        model="weibo/web-v2/topic-posts",
        messages=[],
        extra_body={"topic_id": topic_id, "count": 10},
    )
    return json.loads(resp.choices[0].message.content)


# --- Classification ---
def classify_relevance(weibo_topics: list, douyin_trending: list) -> dict:
    """Use LLM to classify which topics are relevant to our brand/industry."""
    prompt = f"""You are a brand monitoring analyst. Review these social media trending topics and classify each as:
- "brand_mention": directly mentions our brand or competitors
- "industry_relevant": relevant to our industry but not a direct mention
- "irrelevant": not related

Brand keywords: {json.dumps(BRAND_KEYWORDS)}
Industry keywords: {json.dumps(INDUSTRY_KEYWORDS)}

WEIBO HOT SEARCH (top 20):
{json.dumps(weibo_topics[:20], ensure_ascii=False, indent=2)}

DOUYIN TRENDING (top 10):
{json.dumps(douyin_trending[:10], ensure_ascii=False, indent=2)}

Respond in JSON format:
{{
  "brand_mentions": [{{"source": "weibo|douyin", "title": "...", "heat": N, "reason": "..."}}],
  "industry_relevant": [{{"source": "weibo|douyin", "title": "...", "heat": N, "reason": "..."}}],
  "summary": "One sentence overall assessment"
}}

Only include items that are genuinely relevant. Empty lists are fine if nothing matches."""

    resp = client.chat.completions.create(
        model="anthropic/claude-sonnet-4",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=1500,
    )
    try:
        return json.loads(resp.choices[0].message.content)
    except json.JSONDecodeError:
        return {"brand_mentions": [], "industry_relevant": [], "summary": "Parse error"}


# --- Alerting ---
def send_webhook_alert(classification: dict, timestamp: str):
    """Send alert via webhook if brand mentions found."""
    if not WEBHOOK_URL:
        print("  No webhook configured, skipping alert")
        return

    brand_mentions = classification.get("brand_mentions", [])
    industry = classification.get("industry_relevant", [])

    if not brand_mentions and not industry:
        return  # Nothing to alert

    # Format alert message
    blocks = []
    if brand_mentions:
        mentions_text = "\n".join(
            f"• [{m['source'].upper()}] {m['title']} (heat: {m.get('heat', '?')}) — {m['reason']}"
            for m in brand_mentions
        )
        blocks.append(f"🚨 **Brand Mentions**\n{mentions_text}")

    if industry:
        industry_text = "\n".join(
            f"• [{i['source'].upper()}] {i['title']} — {i['reason']}"
            for i in industry[:5]  # Cap industry alerts
        )
        blocks.append(f"📡 **Industry Signals**\n{industry_text}")

    message = {
        "text": f"Social Listening Alert — {timestamp}\n\n" + "\n\n".join(blocks) + f"\n\n_{classification.get('summary', '')}_"
    }

    req = Request(
        WEBHOOK_URL,
        data=json.dumps(message).encode(),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urlopen(req, timeout=10) as resp:
            print(f"  Alert sent: {resp.status}")
    except Exception as e:
        print(f"  Alert failed: {e}")


# --- Logging ---
def log_cycle(timestamp: str, weibo: list, douyin: list, classification: dict):
    """Log raw data and classification for trend analysis."""
    log_entry = {
        "timestamp": timestamp,
        "weibo_count": len(weibo),
        "douyin_count": len(douyin),
        "brand_mentions": len(classification.get("brand_mentions", [])),
        "industry_relevant": len(classification.get("industry_relevant", [])),
        "summary": classification.get("summary", ""),
    }

    log_file = LOG_DIR / f"listening-{timestamp[:10]}.jsonl"
    with open(log_file, "a") as f:
        f.write(json.dumps(log_entry, ensure_ascii=False) + "\n")


# --- Main loop ---
def run_cycle():
    """Execute one monitoring cycle."""
    timestamp = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
    print(f"\n[{timestamp}] Running social listening cycle...")

    # Fetch data (2 API calls, $0.002)
    print("  Fetching Weibo hot search...")
    weibo = fetch_weibo_hot_search()
    print(f"  Got {len(weibo)} Weibo topics")

    print("  Fetching Douyin trending...")
    douyin = fetch_douyin_trending()
    print(f"  Got {len(douyin)} Douyin items")

    # Classify (1 LLM call, ~$0.01)
    print("  Classifying relevance...")
    classification = classify_relevance(weibo, douyin)
    brand_count = len(classification.get("brand_mentions", []))
    industry_count = len(classification.get("industry_relevant", []))
    print(f"  Found: {brand_count} brand mentions, {industry_count} industry signals")

    # Alert if needed
    if brand_count > 0:
        print("  ⚠️ Brand mention detected! Sending alert...")
        send_webhook_alert(classification, timestamp)
    elif industry_count > 0:
        print("  📡 Industry signal detected, sending digest...")
        send_webhook_alert(classification, timestamp)
    else:
        print("  ✓ Nothing relevant this cycle")

    # Log
    log_cycle(timestamp, weibo, douyin, classification)

    # Cost tracking
    data_cost = 0.002  # 2 data calls
    llm_cost = 0.01    # 1 LLM classification
    total_cost = data_cost + llm_cost
    print(f"  Cycle cost: ${total_cost:.3f}")

    return classification


def run_continuous():
    """Run as a long-lived process with 15-minute intervals."""
    print("Social Listening Agent started")
    print(f"  Brand keywords: {BRAND_KEYWORDS}")
    print(f"  Industry keywords: {INDUSTRY_KEYWORDS}")
    print(f"  Check interval: {CHECK_INTERVAL}s")
    print(f"  Webhook: {'configured' if WEBHOOK_URL else 'NOT configured'}")
    print(f"  Estimated daily cost: ${(86400/CHECK_INTERVAL) * 0.012:.2f}")

    while True:
        try:
            run_cycle()
        except Exception as e:
            print(f"  ERROR: {e}")
        time.sleep(CHECK_INTERVAL)


if __name__ == "__main__":
    import sys
    if "--once" in sys.argv:
        run_cycle()  # Single cycle for testing
    else:
        run_continuous()  # Continuous monitoring

Cost breakdown

Component	Per cycle	Cycles/day (96)	Daily cost
Weibo hot search	$0.001	96	$0.096
Douyin trending	$0.001	96	$0.096
LLM classification	~$0.01	96	$0.960
Detail fetches (optional)	$0.002	~20	$0.040
Total	~$0.013		~$1.19/day

Monthly: ~$36/month for 24/7 monitoring of both platforms.

The LLM call is 80% of the cost. To optimize: only call the LLM when raw keyword matching finds a potential hit. Pre-filter with simple string matching, escalate to LLM only for ambiguous cases. This cuts LLM calls by 70%, bringing the total to **$12/month**.

Optimized version: keyword pre-filter

def quick_keyword_check(topics: list, keywords: list) -> bool:
    """Fast check: any keyword appears in any topic title?"""
    text = " ".join(str(t) for t in topics).lower()
    return any(kw.lower() in text for kw in keywords)

def run_cycle_optimized():
    weibo = fetch_weibo_hot_search()
    douyin = fetch_douyin_trending()

    # Quick keyword check (free, no API call)
    all_keywords = BRAND_KEYWORDS + INDUSTRY_KEYWORDS
    has_potential_hit = quick_keyword_check(weibo + douyin, all_keywords)

    if has_potential_hit:
        # Only use LLM when keyword detected
        classification = classify_relevance(weibo, douyin)
        send_webhook_alert(classification, timestamp)
    else:
        # Log as "clean cycle", no LLM cost
        log_cycle(timestamp, weibo, douyin, {"brand_mentions": [], "industry_relevant": [], "summary": "No keyword match"})

With this optimization and assuming 30% of cycles have keyword matches: ~$12/month.

Deployment options

Option 1: Always-on process (VPS/container)

SANDBASE_API_KEY=sk-... ALERT_WEBHOOK_URL=https://hooks.slack.com/... python3 listener.py

Keep alive with systemd or Docker restart policy. ~10MB memory, negligible CPU.

Option 2: Cron every 15 minutes

*/15 * * * * SANDBASE_API_KEY=sk-... ALERT_WEBHOOK_URL=... python3 listener.py --once

Simpler, but each invocation is cold — no state between runs unless you persist to disk.

Option 3: GitHub Actions (free tier)

on:
  schedule:
    - cron: '*/15 * * * *'  # Every 15 minutes

Note: GitHub Actions cron can have up to 15-minute delays. Not ideal for time-sensitive alerts.

What to do when an alert fires

The agent detects. A human decides. Recommended response flow:

1. Brand mention alert → Marketing/PR team reviews within 30 minutes
2. Negative sentiment + high heat → Escalate to crisis protocol
3. Industry signal → Add to weekly strategy briefing
4. Competitor mention → Log for competitive intelligence review

Extending the agent

Add sentiment depth: When a brand mention is detected, call weibo/web-v2/topic-posts to pull the top 10 posts and run sentiment analysis. Adds ~$0.01 per alert (not per cycle).

Add Xiaohongshu: Add xiaohongshu/web-v3/hot-list as a third source. Same pattern, one more data call per cycle (+$0.096/day).

Historical trend analysis: After 30 days of logs, ask an LLM: “What industry topics are gaining heat week-over-week?” Weekly strategic insight from accumulated monitoring data.

Limitations

• 15-minute granularity: Not real-time. A crisis can gain significant traction in 15 minutes. For higher urgency, reduce to 5-minute intervals (3× cost).
• LLM classification is imperfect: False positives happen. The agent over-alerts rather than under-alerts by design — a human makes the final call.
• Weibo hot search is not exhaustive: Topics rotate quickly. A relevant topic at minute 1 may drop off by minute 14. Consider checking sub-topics and brand-specific searches for deeper coverage.
• No Douyin video content analysis: The agent sees trending account metadata, not video content. Combine with douyin/app-v3/video-detail for deeper content analysis on flagged items.

FAQ

Can I monitor specific keywords instead of hot search?

Yes. Use weibo/web-v2/search with specific keywords instead of (or in addition to) the hot search list. This catches mentions that are not trending but still relevant.

How fast does this catch a brand crisis?

Worst case: 15 minutes after the topic appears on hot search. Best case: within the current cycle. Average: ~8 minutes from hot search entry to alert delivery.

What webhook services work?

Any URL that accepts a POST with JSON body. Tested with: Slack (incoming webhook), Discord (webhook), Lark/Feishu (bot webhook), PagerDuty (events API), generic HTTP endpoints.

Can I run this for multiple brands?

Yes. Either maintain separate instances with different keyword lists, or expand the keywords array and have the LLM classify per-brand. The latter is more cost-efficient.

For more on the monitoring data available, see Weibo & Xiaohongshu APIs and 310 Douyin APIs.