Why Sync-Only is the Right Default for Data APIs

TL;DR — SandBase chose sync-only for 571 social data operations. This wasn’t a limitation — it was a deliberate architecture decision that simplifies agent integration, eliminates state management complexity, and matches how LLM tool-calling actually works. Async is necessary for generation tasks (video, images), but data retrieval should be sync by default. Here’s the full trade-off analysis.

The opinion, stated clearly

For data APIs consumed by AI agents, synchronous request-response is the correct default. Not because async is bad, but because sync eliminates an entire category of integration complexity that agents don’t need — and that complexity has real costs in reliability, debuggability, and development speed.

This is an engineering opinion based on operating 571 social data operations (Douyin, TikTok, Weibo, Xiaohongshu) and observing how agent developers actually integrate with APIs. The pattern is consistent: sync APIs get adopted faster, break less often, and produce fewer support tickets.

How LLM tool-calling actually works

Before discussing API design, understand the execution model of the consumer.

When an LLM uses a tool (function calling), the flow is:

1. LLM generates a tool call: {"name": "get_user", "args": {"id": "123"}}
2. Runtime executes the function
3. Runtime returns the result to the LLM
4. LLM continues reasoning with the result

This is inherently synchronous from the LLM’s perspective. The model generates a call, pauses, receives a result, continues. There is no mechanism in standard tool-calling for:

“Here’s a task ID, poll later”
“I’ll call your webhook when ready”
“Check back in 30 seconds”

Yes, you can build these patterns on top of tool-calling. But every layer of async you add becomes complexity the agent developer must manage:

# What sync looks like in an agent tool
def get_douyin_user(user_id: str) -> dict:
    return api.get(f"/douyin/user/{user_id}").json()
    # Done. LLM gets the result immediately.

# What async looks like in an agent tool
def get_douyin_user(user_id: str) -> dict:
    task = api.post(f"/douyin/user/{user_id}/async")
    task_id = task.json()["task_id"]
    
    # Now what? The LLM is waiting.
    # Option A: Poll in a loop (blocking, wasteful)
    for _ in range(30):
        result = api.get(f"/tasks/{task_id}")
        if result.json()["status"] == "complete":
            return result.json()["data"]
        time.sleep(1)
    raise TimeoutError("Task didn't complete in 30s")
    
    # Option B: Return task_id to LLM (confusing)
    # LLM: "I got a task_id but I need the actual data..."
    # Now you need another tool call just to check status

The async version is 10x more code, introduces failure modes (timeout, lost tasks, partial results), and doesn’t match the LLM’s execution model.

The sync contract

SandBase’s 571 social data operations follow a simple contract:

Request  → Processing (100ms–2s) → Response
POST/GET → Server fetches data    → JSON result or error

Properties of this contract:

Bounded latency: Every operation completes within a known time window
Atomic: You get the full result or an error, never partial data
Stateless: No task IDs, no polling, no session management
Retryable: On failure, retry the same request with the same parameters
Observable: Latency = time between request and response. No hidden queues.

Why this simplifies agent architecture

1. No state management

Async APIs require tracking outstanding tasks:

# Async: You need a task manager
class TaskManager:
    def __init__(self):
        self.pending = {}  # task_id -> metadata
        self.results = {}  # task_id -> result
    
    async def submit(self, operation, params):
        task = await api.post(operation, params)
        self.pending[task.id] = {"submitted": time.time(), "params": params}
        return task.id
    
    async def check(self, task_id):
        if task_id in self.results:
            return self.results[task_id]
        result = await api.get(f"/tasks/{task_id}")
        if result.status == "complete":
            del self.pending[task_id]
            self.results[task_id] = result.data
            return result.data
        return None
    
    async def cleanup_stale(self):
        # Tasks that never completed...
        for tid, meta in self.pending.items():
            if time.time() - meta["submitted"] > 300:
                # Retry? Abandon? Log? All three?
                pass

Sync APIs need none of this:

# Sync: Just call and get
result = await api.get("/douyin/user/profile", params={"user_id": uid})
# Done. No state. No cleanup. No stale tasks.

For a single API call the difference seems trivial. For an agent making 50 calls per decision cycle across 4 platforms, the async state management becomes a significant source of bugs.

2. Error handling is straightforward

Sync errors are immediate and actionable:

try:
    result = await api.get("/douyin/user/profile", params={"user_id": uid})
except HTTPError as e:
    if e.status == 429:
        await asyncio.sleep(1)
        result = await api.get(...)  # Retry
    elif e.status == 404:
        result = {"error": "user_not_found"}  # Agent handles gracefully
    else:
        raise  # Unexpected, bubble up

Async errors are distributed across time and harder to attribute:

# Submit succeeds, but task fails 10 seconds later
task = await api.post("/async/douyin/user/profile", params={"user_id": uid})
# No error yet... task is "processing"

# Later, when checking:
result = await api.get(f"/tasks/{task.id}")
# result.status might be: "failed", "timeout", "partial", "expired"
# Which user_id was this for? Need to track that.
# Can I retry? Maybe. But the original context is gone.

3. Debugging is linear

When a sync agent tool fails, the trace is simple:

10:00:01.234 → Request: GET /douyin/user/profile?user_id=abc
10:00:01.891 → Response: 200 OK, 657ms
10:00:01.892 → LLM receives result, continues reasoning

When an async tool fails, the trace fragments:

10:00:01.234 → Submit: POST /async/douyin/user/profile
10:00:01.456 → Got task_id: task_xyz
10:00:05.000 → Poll: GET /tasks/task_xyz → "processing"
10:00:10.000 → Poll: GET /tasks/task_xyz → "processing"
10:00:15.000 → Poll: GET /tasks/task_xyz → "failed"
10:00:15.001 → Why did it fail? Check task details...
10:00:15.200 → GET /tasks/task_xyz/details → "upstream_timeout"
10:00:15.201 → Retry? Submit a new task...

Six log entries instead of two. Distributed across 15 seconds instead of 657ms. And the agent’s reasoning was blocked the entire time — or worse, continued with stale data.

4. Composability

Agents compose multiple tool calls into decision workflows. Sync makes composition natural:

# Agent's internal reasoning leads to this sequence:
user = await get_user(user_id)
videos = await get_user_videos(user_id, count=10)
engagement = calculate_engagement_rate(videos)
decision = "This creator has {engagement}% engagement, which is {above/below} threshold"

Each step completes before the next begins. The agent can reason about intermediate results and decide whether to continue or branch.

With async, composition requires orchestration:

# Submit all tasks
user_task = await submit_task("get_user", user_id)
video_task = await submit_task("get_videos", user_id)

# Wait for both (what if one fails?)
user = await wait_for_task(user_task, timeout=10)
videos = await wait_for_task(video_task, timeout=10)

# If video_task fails, do we still use user data?
# If user_task succeeds but video_task times out, retry?
# The branching logic explodes

The performance question: “Isn’t sync slower?”

Common objection: “With async, I can submit 100 tasks and get results as they arrive. Sync means I wait for each one sequentially.”

This conflates two things:

API design (sync vs async) — whether the server returns results immediately or via a task system
Client concurrency — whether the client makes requests in parallel

You can have sync APIs with parallel client calls:

# Sync API + concurrent client = fast
async def get_50_profiles(user_ids: list[str]) -> list[dict]:
    tasks = [api.get(f"/douyin/user/profile?user_id={uid}") for uid in user_ids]
    results = await asyncio.gather(*tasks)
    return [r.json() for r in results]
    # 50 sync calls in parallel, all complete in ~1 second

The sync API completes each request individually in 200ms–2s. The client fires 50 requests concurrently. Total wall-clock time: ~2 seconds for 50 profiles, not 50 × 2s.

This is strictly better than async for bounded-latency operations because:

You know immediately which calls succeeded and which failed
No task management overhead
No polling cost
The server doesn’t need to maintain task state

When async IS necessary

Sync-only is not always appropriate. Here’s when async is the right choice:

1. Generation tasks (video, images)

Text → Video generation: 30–120 seconds
Image generation: 5–30 seconds
These cannot return sync within HTTP timeout bounds.

Video and image generation on SandBase use async patterns because the underlying computation genuinely requires minutes. You submit a request, get a task ID, and poll or receive a webhook when complete.

2. Bulk exports

"Export all 10,000 comments on this video"
This requires paginated fetching that may take 30+ seconds.

When the data volume is too large for a single response and the fetching takes longer than reasonable HTTP timeouts, async with a download URL makes sense.

3. Multi-step aggregations

"Calculate 90-day engagement trend for this account"
Requires fetching 90 days of data + computation.

When the server needs to perform significant computation across multiple data fetches, async prevents HTTP timeout issues.

The pattern

Notice what these have in common: they involve creation or computation, not retrieval. Data retrieval — “give me this user’s profile,” “give me this video’s stats,” “search for this keyword” — is inherently fast. The underlying data exists; the API just needs to fetch and return it.

The line is clear:

Data retrieval → sync (the data exists, just fetch it)
Data generation/computation → async (the result doesn’t exist yet, must be created)

Architecture implications

For API providers

If you’re building a data API for agent consumption:

Default to sync. If the operation can complete in < 5 seconds, make it sync.
Set clear timeouts. Document max latency per operation. Agents need to know how long to wait.
Use HTTP semantics correctly. 200 for success, 4xx for client errors, 5xx for server errors, 429 for rate limits with Retry-After header.
Return complete results. Don’t return partial data with a “fetch more” pattern for simple queries.
Reserve async for generation. Only use task-based patterns when the operation genuinely takes > 5 seconds.

For agent developers

If you’re building agents that consume data APIs:

Prefer sync APIs. They’re simpler to integrate, debug, and maintain.
Use client-side concurrency for parallelism. asyncio.gather() gives you parallel execution with sync APIs.
Set aggressive timeouts. If an API call takes > 5s, it’s likely failing. Time out and retry.
Keep tool functions simple. A tool function should be: call API → return result. If it needs state management, the API design is wrong for your use case.
Separate sync and async workflows. If you need both (data retrieval + video generation), keep them in separate agent tools with different timeout/retry strategies.

The 571-operation evidence

SandBase operates 571 social data operations across 4 platforms with sync-only design. Operational data:

Metric	Value
Operations	571
Median latency (P50)	340ms
P99 latency	1.8s
Success rate	99.4%
Timeout rate (> 5s)	0.3%

The 0.3% timeout rate means that out of every 1,000 calls, 3 take longer than 5 seconds — and those are retryable. The remaining 997 calls complete well within HTTP timeout bounds and fit cleanly into agent tool-calling patterns.

If these operations were async, every single one of those 997 fast calls would carry unnecessary overhead: task submission, task storage, polling or webhook delivery, task cleanup. That overhead exists purely because of the API design, not because the operation requires it.

Objections addressed

“What about webhooks? They’re more efficient than polling.”

True, but webhooks require:

Public endpoint on the client side
Webhook verification and security
Retry logic for failed deliveries
State reconciliation when webhooks are lost

For an agent running on a developer’s laptop or in a serverless function, none of this infrastructure exists. Sync eliminates the need entirely.

“What if the upstream source is slow?”

The API provider’s problem, not the agent developer’s. A good data API has caching, connection pooling, and pre-fetching to ensure consistent latency. If the upstream is genuinely slow (> 5s), that operation should be async — but most data retrieval operations aren’t.

“Sync limits throughput.”

Only if you make sequential calls. With concurrent requests and rate limit awareness, sync APIs can deliver thousands of results per second. The limit is the rate limit, not the protocol.

Conclusion

Sync-only for data APIs is not a compromise. It’s the architecture that matches how AI agents actually consume data: request, receive, reason, repeat. Every async pattern you add to a data API is complexity tax on every agent developer who integrates with it — complexity that serves the API provider’s infrastructure convenience, not the developer’s productivity.

The principle is simple: if the data exists and can be fetched in under 5 seconds, return it synchronously. Save async for operations where the result genuinely doesn’t exist yet and must be created. Your agent developers will thank you — mostly by not filing support tickets about lost tasks, webhook failures, and polling timeouts.