Multimodal Agent with Artifacts (E2B Sandbox)

TL;DR — Build an agent that takes a creative brief, generates an image with Seedream 5 Pro, writes Python analysis code, executes it in an E2B sandbox, and produces a complete report artifact. This tutorial demonstrates how SandBase’s ecosystem composition works: multimodal generation + code sandbox + LLM reasoning in a single agent workflow. Complete code included.

What we’re building

A multimodal agent that:

Takes a creative brief (e.g., “Generate a product hero image for a smartphone launch”)
Generates an image using Seedream 5 Pro through SandBase
Writes analysis code to evaluate the image (color distribution, composition metrics)
Executes that code safely in an E2B sandbox
Produces a structured report artifact combining the image, analysis, and recommendations

This demonstrates the “ecosystem composition” pattern: one agent orchestrating multiple capability types (generation, reasoning, execution) through a unified platform.

Architecture overview

Creative Brief
     ↓
┌─────────────────────────────────────────────┐
│  Agent (Claude Sonnet 5 / GPT-4.1)         │
│                                             │
│  1. Parse brief → generation prompt         │
│  2. Call Seedream 5 Pro → image URL         │
│  3. Write analysis code                     │
│  4. Execute in E2B sandbox → results        │
│  5. Synthesize report artifact              │
└─────────────────────────────────────────────┘
     ↓              ↓              ↓
  Seedream       E2B Sandbox    Final Report
  (image gen)   (code exec)    (artifact)

All three capabilities (image generation, code execution, LLM reasoning) are accessed through SandBase’s unified API layer.

Prerequisites

pip install openai e2b-code-interpreter httpx Pillow

Required access:

SandBase API key (for Seedream 5 Pro + LLM)
E2B API key (for sandbox execution)

The complete agent

"""
Multimodal Agent with Artifacts — E2B Sandbox
Generates images, analyzes them with code, produces reports.
"""

import json
import base64
from pathlib import Path
from openai import OpenAI
from e2b_code_interpreter import Sandbox
import httpx

# --- Configuration ---
SANDBASE_API_KEY = "your-sandbase-api-key"
E2B_API_KEY = "your-e2b-api-key"
OUTPUT_DIR = Path("./artifacts")
OUTPUT_DIR.mkdir(exist_ok=True)

# --- Clients ---
client = OpenAI(
    base_url="https://api.sandbase.ai/v1",
    api_key=SANDBASE_API_KEY,
)

# --- Step 1: Parse Brief → Generation Prompt ---
def parse_brief(brief: str) -> dict:
    """Use LLM to convert a creative brief into structured generation parameters."""
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content": """You are a creative director AI. 
Convert the brief into a structured image generation prompt.
Output JSON with:
{
    "prompt": "detailed prompt for image generation (English, descriptive)",
    "negative_prompt": "things to avoid",
    "aspect_ratio": "16:9 or 1:1 or 9:16",
    "style": "photographic or illustration or 3d-render",
    "analysis_focus": ["color", "composition", "text_readability", "brand_fit"]
}
Only output JSON."""},
            {"role": "user", "content": brief}
        ]
    )
    return json.loads(response.choices[0].message.content)

# --- Step 2: Generate Image with Seedream ---
def generate_image(prompt: str, negative_prompt: str = "", 
                   aspect_ratio: str = "16:9") -> str:
    """Generate an image using Seedream 5 Pro via SandBase."""
    
    # Map aspect ratio to dimensions
    dimensions = {
        "16:9": (1280, 720),
        "1:1": (1024, 1024),
        "9:16": (720, 1280),
    }
    width, height = dimensions.get(aspect_ratio, (1280, 720))
    
    response = client.images.generate(
        model="seedream-5-pro",
        prompt=prompt,
        n=1,
        size=f"{width}x{height}",
        extra_body={
            "negative_prompt": negative_prompt,
        }
    )
    
    image_url = response.data[0].url
    print(f"✓ Image generated: {image_url}")
    return image_url

# --- Step 3: Write Analysis Code ---
def write_analysis_code(image_url: str, analysis_focus: list[str]) -> str:
    """Use LLM to write image analysis code based on the focus areas."""
    
    focus_str = ", ".join(analysis_focus)
    
    response = client.chat.completions.create(
        model="gpt-4.1-mini",
        messages=[
            {"role": "system", "content": f"""You are a Python developer specializing in image analysis.
Write a complete Python script that:
1. Downloads the image from the provided URL
2. Analyzes it for: {focus_str}
3. Produces a JSON output with structured results

Requirements:
- Use PIL/Pillow for image analysis
- Use numpy for numerical computations
- Use httpx to download the image
- Print the final JSON result to stdout
- Include color distribution (top 5 dominant colors as hex)
- Include basic composition metrics (rule of thirds alignment)
- Include brightness/contrast statistics
- Handle errors gracefully

The script should be self-contained and executable.
Output ONLY the Python code, no markdown fences."""},
            {"role": "user", "content": f"Analyze image at URL: {image_url}"}
        ]
    )
    
    code = response.choices[0].message.content
    # Strip markdown fences if LLM adds them anyway
    if code.startswith("```"):
        code = code.split("\n", 1)[1].rsplit("```", 1)[0]
    
    return code

# --- Step 4: Execute in E2B Sandbox ---
def execute_in_sandbox(code: str) -> dict:
    """Run the analysis code in an E2B sandbox and return results."""
    
    sandbox = Sandbox(api_key=E2B_API_KEY)
    
    try:
        # Install required packages
        sandbox.commands.run("pip install Pillow numpy httpx colorthief", timeout=30)
        
        # Write and execute the analysis script
        sandbox.files.write("/home/user/analyze.py", code)
        result = sandbox.commands.run("python /home/user/analyze.py", timeout=60)
        
        if result.exit_code != 0:
            print(f"⚠ Sandbox execution error: {result.stderr}")
            return {
                "status": "error",
                "error": result.stderr,
                "stdout": result.stdout
            }
        
        # Parse the JSON output
        try:
            analysis = json.loads(result.stdout)
            print("✓ Analysis complete")
            return {"status": "success", "analysis": analysis}
        except json.JSONDecodeError:
            # If output isn't pure JSON, return raw
            return {
                "status": "partial",
                "raw_output": result.stdout,
                "note": "Output was not valid JSON"
            }
    
    finally:
        sandbox.kill()

# --- Step 5: Produce Report Artifact ---
def generate_report(brief: str, params: dict, image_url: str, 
                    analysis: dict) -> str:
    """Use LLM to synthesize a final report from all components."""
    
    response = client.chat.completions.create(
        model="gpt-4.1",  # Use stronger model for final synthesis
        messages=[
            {"role": "system", "content": """You are a creative analysis AI.
Produce a professional report artifact in Markdown format that includes:
1. Executive summary
2. Brief interpretation
3. Generated image reference
4. Technical analysis results (from code execution)
5. Recommendations for iteration
6. Cost breakdown

Be specific and actionable. Reference actual numbers from the analysis."""},
            {"role": "user", "content": f"""
Creative Brief: {brief}

Generation Parameters:
{json.dumps(params, indent=2)}

Image URL: {image_url}

Code Analysis Results:
{json.dumps(analysis, indent=2)}

Produce the final report artifact."""}
        ]
    )
    
    return response.choices[0].message.content

# --- Orchestrator ---
def run_multimodal_agent(brief: str) -> Path:
    """Run the full multimodal agent pipeline."""
    
    print(f"\n{'='*60}")
    print(f"🎨 Multimodal Agent — Processing Brief")
    print(f"{'='*60}")
    print(f"Brief: {brief}\n")
    
    # Step 1: Parse brief
    print("📋 Step 1: Parsing brief...")
    params = parse_brief(brief)
    print(f"   Prompt: {params['prompt'][:80]}...")
    print(f"   Aspect: {params['aspect_ratio']}")
    print(f"   Focus: {params['analysis_focus']}")
    
    # Step 2: Generate image
    print("\n🖼️  Step 2: Generating image with Seedream 5 Pro...")
    image_url = generate_image(
        prompt=params["prompt"],
        negative_prompt=params.get("negative_prompt", ""),
        aspect_ratio=params.get("aspect_ratio", "16:9")
    )
    
    # Step 3: Write analysis code
    print("\n💻 Step 3: Writing analysis code...")
    code = write_analysis_code(image_url, params["analysis_focus"])
    print(f"   Generated {len(code.split(chr(10)))} lines of Python")
    
    # Step 4: Execute in sandbox
    print("\n🔒 Step 4: Executing in E2B sandbox...")
    sandbox_result = execute_in_sandbox(code)
    
    if sandbox_result["status"] == "error":
        print(f"   ⚠ Execution failed, attempting fix...")
        # Self-correction: ask LLM to fix the code
        fix_response = client.chat.completions.create(
            model="gpt-4.1-mini",
            messages=[
                {"role": "system", "content": "Fix this Python script. The error was: " 
                 + sandbox_result.get("error", "unknown")},
                {"role": "user", "content": code}
            ]
        )
        fixed_code = fix_response.choices[0].message.content
        if fixed_code.startswith("```"):
            fixed_code = fixed_code.split("\n", 1)[1].rsplit("```", 1)[0]
        sandbox_result = execute_in_sandbox(fixed_code)
    
    # Step 5: Generate report
    print("\n📊 Step 5: Generating report artifact...")
    report = generate_report(brief, params, image_url, sandbox_result)
    
    # Save artifact
    timestamp = __import__("datetime").datetime.now().strftime("%Y%m%d_%H%M%S")
    artifact_path = OUTPUT_DIR / f"report_{timestamp}.md"
    artifact_path.write_text(report)
    
    # Also save the analysis code for reference
    code_path = OUTPUT_DIR / f"analysis_{timestamp}.py"
    code_path.write_text(code)
    
    print(f"\n{'='*60}")
    print(f"✅ Artifact saved: {artifact_path}")
    print(f"   Analysis code: {code_path}")
    print(f"   Image: {image_url}")
    print(f"{'='*60}\n")
    
    return artifact_path

# --- Cost Tracking ---
class CostTracker:
    """Track costs across the pipeline."""
    
    def __init__(self):
        self.costs = {
            "llm_reasoning": 0.0,
            "image_generation": 0.0,
            "sandbox_execution": 0.0,
        }
    
    def estimate_pipeline_cost(self) -> dict:
        """Estimate cost for one full pipeline run."""
        return {
            "brief_parsing (GPT-4.1-mini)": "$0.002",
            "image_generation (Seedream 5 Pro)": "$0.03",
            "code_writing (GPT-4.1-mini)": "$0.003",
            "sandbox_execution (E2B)": "$0.005",
            "report_synthesis (GPT-4.1)": "$0.015",
            "total_estimated": "$0.055"
        }

# --- Main ---
if __name__ == "__main__":
    # Example briefs
    briefs = [
        "Create a hero image for a premium smartphone launch. "
        "The phone should appear floating with dramatic lighting. "
        "Target audience: tech enthusiasts, 25-40. Brand colors: deep blue and silver.",
        
        "Design a social media banner for a summer coffee collection. "
        "Lifestyle feel, warm tones, outdoor café setting. "
        "Needs to work as both 16:9 and 1:1 crops.",
    ]
    
    # Run agent on first brief
    artifact = run_multimodal_agent(briefs[0])
    
    # Print cost estimate
    tracker = CostTracker()
    print("\n💰 Cost breakdown:")
    for item, cost in tracker.estimate_pipeline_cost().items():
        print(f"   {item}: {cost}")

How ecosystem composition works

This agent uses three distinct capability types, all through SandBase:

1. LLM Reasoning (GPT-4.1-mini / GPT-4.1)

client.chat.completions.create(model="gpt-4.1-mini", ...)

Used for: brief parsing, code generation, report synthesis.

2. Image Generation (Seedream 5 Pro)

client.images.generate(model="seedream-5-pro", ...)

Used for: creating the visual asset from the prompt.

For more on image generation APIs available on SandBase, including Seedream, Qwen Image, and others.

3. Code Execution (E2B Sandbox)

sandbox = Sandbox(api_key=E2B_API_KEY)
sandbox.commands.run("python analyze.py")

Used for: running untrusted analysis code safely.

The agent decides what to generate, what to analyze, and how to present results. The platform provides the capabilities. This separation is the core of ecosystem composition.

The self-correction pattern

Notice Step 4’s error handling:

if sandbox_result["status"] == "error":
    # Ask LLM to fix the code based on the error message
    fix_response = client.chat.completions.create(...)
    fixed_code = fix_response.choices[0].message.content
    sandbox_result = execute_in_sandbox(fixed_code)

This is the “generate → execute → fix → re-execute” loop that makes agents more capable than static pipelines. The LLM sees the actual error from sandbox execution and adjusts. In practice, this catches:

Import errors (missing packages → add pip install)
URL access issues (timeout → add retry logic)
Data format mismatches (unexpected image format → add conversion)

Cost analysis

Full pipeline cost per run:

Step	Model/Service	Estimated cost
Brief parsing	GPT-4.1-mini (~800 in, 200 out)	$0.002
Image generation	Seedream 5 Pro (1 image)	$0.03
Code writing	GPT-4.1-mini (~500 in, 1000 out)	$0.003
Sandbox execution	E2B (~30s runtime)	$0.005
Report synthesis	GPT-4.1 (~3000 in, 800 out)	$0.015
Total		~$0.055

At scale:

10 briefs/day: $0.55/day → $16.50/month
50 briefs/day: $2.75/day → $82.50/month
200 briefs/day: $11/day → $330/month

Compare to a human creative analyst: $80–150/hour. One brief analysis takes 30–60 minutes manually. The agent does it in under 2 minutes at $0.055.

Extending to video

The same pattern works with video generation — just swap Step 2:

# Instead of image generation:
def generate_video(prompt: str) -> str:
    """Generate a video using Kling or MiniMax via SandBase."""
    # Video generation is async (takes 30-120s)
    response = client.chat.completions.create(
        model="kling-video-3-standard",
        messages=[{"role": "user", "content": json.dumps({
            "prompt": prompt,
            "duration": 5,
            "aspect_ratio": "16:9"
        })}],
        extra_body={"sandbase_api": "kling/video/generation"}
    )
    # Poll for completion...
    return video_url

For a detailed video generation agent tutorial, see our ad creative agent guide.

Production patterns

Batch processing

import asyncio

async def process_batch(briefs: list[str]) -> list[Path]:
    """Process multiple briefs concurrently."""
    # Image generation can be parallelized
    tasks = [asyncio.to_thread(run_multimodal_agent, brief) for brief in briefs]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    return [r for r in results if isinstance(r, Path)]

Artifact versioning

class ArtifactStore:
    def __init__(self, base_dir: Path):
        self.base_dir = base_dir
    
    def save(self, brief_id: str, version: int, report: str, 
             image_url: str, code: str) -> Path:
        artifact_dir = self.base_dir / brief_id / f"v{version}"
        artifact_dir.mkdir(parents=True, exist_ok=True)
        
        (artifact_dir / "report.md").write_text(report)
        (artifact_dir / "analysis.py").write_text(code)
        (artifact_dir / "metadata.json").write_text(json.dumps({
            "image_url": image_url,
            "timestamp": datetime.now().isoformat(),
            "version": version
        }))
        
        return artifact_dir

Human-in-the-loop iteration

def iterate_with_feedback(brief: str, feedback: str, 
                          previous_report: str) -> Path:
    """Re-run the agent incorporating human feedback."""
    enhanced_brief = f"""
Original brief: {brief}

Previous output summary: {previous_report[:500]}

Human feedback for iteration: {feedback}

Generate an improved version addressing the feedback.
"""
    return run_multimodal_agent(enhanced_brief)

Key patterns demonstrated

Ecosystem composition: One agent uses LLM + image gen + code sandbox. No separate integrations needed.
Self-correction: Agent fixes its own code errors using sandbox feedback.
Artifact production: The output is a structured document, not just text — it includes references, analysis, and recommendations.
Cost predictability: Each step has known costs. Total pipeline is bounded and trackable.
Capability separation: The agent decides what to do. The platform provides how to do it. See best models for autonomous agents for model selection guidance.

This pattern scales to any workflow combining generation + analysis + execution. Replace “image” with “video,” “document,” or “data visualization” and the architecture remains the same.