AI City Remix: Fact-Check an AI City Script with Wikidata and OSM

The Kyoto script MiniMax M3.1 Flash wrote for us opens its second era with “Around 960, Emperor Ichijō moved the imperial palace to its present location.” Emperor Ichijō reigned from 986 to 1011, and the palace has stood on its present site since 1331. The line passed the Skill’s validator and rendered cleanly over a lit map of Heian-kyō. Nothing in the pipeline was built to notice it.

That is the gap this guide fills. In AI City Remix, SandBase’s city-story video challenge, accuracy is worth 25% of the score and the AI workflow 30%, and the workflow criterion rewards less human intervention. A checker that runs between writing the script and paying for narration serves both. Below is the one we built: four checks against Wikidata, OpenStreetMap and Wikipedia, about $0.0005 of model time per city. We ran it on two scripts written by M3.1 Flash, one for Vienna and one for Kyoto, and report what it caught, what it missed, and what it got wrong.

Fact-checking an AI city script: Key takeaway

Four automatic checks catch most of what goes wrong in a city-through-time story.json: landmark coordinates against Wikidata, OSM names that match nothing, modern boundaries on old eras, and dated claims with no support in a reference article.
On our Vienna script it raised 9 flags. They covered 4 of the 6 problems we had found by hand, and missed 2.
On a new Kyoto script it raised 13 flags. Those included Kyoto Tower placed 6.4 km from the real tower, a wrong emperor in 960, and Oda Nobunaga entering Kyoto in 1573 instead of 1568.
It also raised false alarms on two true dates: 1333 (the fall of the Kamakura shogunate) is absent from “History of Kyoto”, and Vienna’s 1910 population appears in “History of Vienna” but in a sentence that shares no keyword with the claim.
Treat a flag as “look at this”, not “this is wrong”. The checker narrows an 8-era script to about a dozen flagged claims and manual items to read, before any media money is spent.

Why the validator is not enough

The city-through-time Skill splits the work cleanly. The director model writes story.json: eras, narration, coordinates, OpenStreetMap references and fallback outlines. Scripts fetch real geometry, generate narration and illustrations through SandBase, and render the film. Its resolve.mjs step validates the format before fetching anything.

Format is all it validates. The Skill’s own documentation says the language model gets facts and coordinates wrong, and that fallback outlines several kilometres off are common. In our AI City Remix guide, a Vienna script passed every format check and still needed six fixes. In our model comparison, M3.1 Flash at low effort returned valid JSON 9 times out of 9, while its wall outlines held only 5 of 15 requested landmarks.

The timing matters as much as the checks. Narration and illustrations for our Vienna story were estimated at $0.69. Fixing a line after generation means deleting its audio and paying again. Every check below runs before media.mjs --confirm.

The four checks

Check	Data source	What it flags	Cost
Coordinates	Wikidata coordinate property (P625)	Any `events[].at` more than 0.5 km from the matching item near the city	Free
OSM names	Overpass API count query	Any `shape.osm` reference with zero matches, so the film falls back to the model’s own polygon	Free
Boundaries	The story itself	A `boundary=administrative` reference on any era except the last (today’s shape on a past era)	Free
Dated claims	M3.1 Flash lists claims; Wikipedia “History of ” is searched	A claimed year that never appears in a sentence sharing a keyword with the claim	≈ $0.0005 per city

Coordinates from Wikidata

Event cards carry an at coordinate, and the model writes it from memory. Wikidata already stores coordinates for most landmarks. The lookup searches by the card’s title and keeps the first hit that has a coordinate within 40 km of the city centre, so a “Kyoto Tower” match cannot land on a namesake in another country:

import json, math, time, urllib.parse, urllib.request

UA = {"User-Agent": "citycheck/1.0 (your-name@your-domain)"}  # use a real contact; overpass-api.de rejects example.com (HTTP 406)

def get_json(url, data=None, timeout=60):
    req = urllib.request.Request(url, data=data, headers=UA)
    with urllib.request.urlopen(req, timeout=timeout) as r:
        return json.loads(r.read())

def km(a, b):
    """Great-circle distance between two [lat, lon] points."""
    la1, lo1, la2, lo2 = map(math.radians, (a[0], a[1], b[0], b[1]))
    h = math.sin((la2 - la1) / 2) ** 2 + math.cos(la1) * math.cos(la2) * math.sin((lo2 - lo1) / 2) ** 2
    return 2 * 6371.0088 * math.asin(math.sqrt(h))

def wikidata_coord(title, city_centre, max_km=40):
    q = urllib.parse.urlencode({"action": "wbsearchentities", "search": title,
                                "language": "en", "format": "json", "limit": 7})
    for hit in get_json("https://www.wikidata.org/w/api.php?" + q).get("search", []):
        ent = get_json(f"https://www.wikidata.org/wiki/Special:EntityData/{hit['id']}.json")["entities"][hit["id"]]
        claim = (ent.get("claims", {}).get("P625") or [{}])[0]
        c = claim.get("mainsnak", {}).get("datavalue", {}).get("value")
        if c and km((c["latitude"], c["longitude"]), city_centre) <= max_km:
            return hit["id"], c["latitude"], c["longitude"]
        time.sleep(0.2)  # stay polite to the public endpoint
    return None

Titles that describe an event rather than a place (“Electric streetcars arrive”, “Heian Court”) return nothing. The checker marks those for a human instead of guessing. That was 9 of 14 cards in Vienna and 8 of 16 in Kyoto.

OSM names that match nothing

When a shape.osm reference finds nothing, resolve.mjs prints a warning and quietly uses the model’s hand-drawn polygon. The film then prints that polygon’s area on screen as if it were measured. A count query catches this before you resolve anything:

def overpass_count(ref, centre):
    """Number of OSM ways and relations matching a story.json osm reference near the city."""
    r = ref.get("within_km", 25) * 1000
    f = "".join(f'["{k}"="{v}"]' for k, v in (ref.get("tags") or {}).items())
    if ref.get("name"):
        f += f'["name"~"{ref["name"]}"]'
    ql = (f"[out:json][timeout:60];(way{f}(around:{r},{centre[0]},{centre[1]});"
          f"relation{f}(around:{r},{centre[0]},{centre[1]}););out count;")
    d = get_json("https://overpass-api.de/api/interpreter",
                 data=urllib.parse.urlencode({"data": ql}).encode(), timeout=90)
    t = d["elements"][0]["tags"]
    return int(t.get("ways", 0)) + int(t.get("relations", 0))

Public Overpass servers are often busy. Our script tries a second mirror and reports UNKNOWN rather than a false zero when both fail. That happened once in the Kyoto run, for the 京都御所 reference, and resolve.mjs later found 0 points for it too. Use a real contact in the User-Agent: overpass-api.de returned HTTP 406 to a placeholder example.com address in our tests.

Today’s boundary on a past era

This one needs no network at all. An administrative boundary in OSM is the city’s current shape. Used for 1910 or 1889, it makes the film show growth that had not happened yet. The check flags any boundary=administrative reference outside the final era.

Dated claims against a reference article

The last check is the only one that uses a model, and the only one that judges text. M3.1 Flash, through SandBase at reasoning_effort: "low", gets every narration line in one request and returns the dated claims as JSON. The prompt says to list claims, not to judge them. The script then fetches the English Wikipedia article “History of ”. A claim counts as found only when its year appears in the same sentence as at least one content word from the claim. Our first version accepted the year anywhere in the article, and a bare “100” or “1910” elsewhere on the page passed claims it should not have.

This is deliberately narrow. It finds years with no support in one reference. It cannot prove a sentence true, and it cannot tell a missing fact from a wrong one.

Results on two real scripts

Both scripts were written by M3.1 Flash at low effort, following the Skill’s one-era-per-request recipe. Vienna is the script from our AI City Remix guide. We wrote Kyoto fresh on October 24: 9 calls, 245 seconds, $0.011. We then ran the checker on each.

Checker output for both scripts, rendered from vienna_report.json and kyoto_report.json (2026-10-24). Only flags are listed. “Not in History of …” means no sentence holds both the year and a keyword from the claim. The last two lines of each block give the coordinate and claim passes and the cost of the one model call.

We then checked every flag by hand against Wikipedia and Wikidata.

City	Flag	Verdict after reading the sources
Kyoto	Kyoto Tower card 6.4 km from Wikidata	Real error: the tower stands opposite Kyoto Station; the card sat in the north of the city
Kyoto	”Around 960, Emperor Ichijō moved the imperial palace”	Real error: Ichijō reigned 986–1011; the palace was used from 1331
Kyoto	”After Oda Nobunaga entered Kyoto in 1573”	Real error: Nobunaga entered Kyoto on 9 November 1568; the Ashikaga shogunate ended in 1573
Kyoto	Imperial Palace card 1.3 km off (two eras); Ginkaku-ji 1.4 km off	Real errors: the palace cards sit west of the real palace, the Ginkaku-ji card north of the temple
Kyoto	Suzaku Gate 1.0 km off	Unclear: the gate no longer exists, and Wikidata’s point and the model’s reconstruction differ
Kyoto	`二条御城`, `^(Heian-kyo\|平安京)$` and a level-6 `^京都市$` match nothing	Real: OSM uses `二条城`, has no Heian-kyō outline, and maps the city boundary at admin_level 7
Kyoto	`^京都市$` boundary on the 1889–1973 era	Real: it is the modern city limit
Kyoto	”Kyoto’s core population has fallen since the 1970s”	Needs a source: we did not confirm or refute it
Kyoto	”The Kamakura shogunate fell in 1333”	False alarm: true, just absent from “History of Kyoto”
Vienna	Town rights from Duke Henry II in 1143	Real error (found by hand too): capital 1155, duchy 1156, city rights 1221
Vienna	1857 expansion “brought nineteen suburbs”	Real error (found by hand too): the suburbs joined in 1850
Vienna	Celtic tribe c. 500 BC	Needs a source (found by hand too)
Vienna	`Wiener Stadtmauer` ×3 and the `Wien` boundary on 1910	Real: no OSM match, and today’s boundary on a past era
Vienna	Austrian State Treaty card 2.1 km off	Unclear: a treaty is not a place; Wikidata’s point and the model’s differ
Vienna	”More than two million people by 1910”	False alarm: true; “History of Vienna” gives 2,031,000 for 1910, but in a sentence with no shared keyword (“population” vs “million”, “people”)

Two problems we had found by hand in Vienna got past the checker. The Roman era’s “turn of the first century BC” was marked found, because the year-matching rule read the “1” in “during the 1st century” as the year 1, in a sentence that also contains “Romans”. Vindobona’s legionary camp actually dates from around AD 97. The Parliament card, pinned 11 km from the building, has a descriptive title (“A New Parliamentary Vienna”) that matched no Wikidata item, so it went to the manual list instead.

Scope of this test: two cities, one script each, one day. The flag counts depend on how the model words its titles and on how complete each “History of” article is. Treat them as one observation, not a benchmark.

What the flags look like on screen

Flags are easier to prioritise once you see where they land in the film. These stills are from a stills-only render of the unedited Kyoto script; nothing was paid for.

Still at 20 seconds: the c. 960 era with the Ichijō subtitle the claims check flagged. The street grid behind it is real OSM data; the date and the emperor are wrong.

Still at 67 seconds: the Kyoto Tower card pinned 6.4 km from the tower. The modern outline is also the model’s fallback polygon, because the level-6 boundary reference matched nothing in OSM.

Still at 40 seconds: the Nijō Castle card passed the coordinate check (50 m from Wikidata). Some cards come out right, and the checker leaves those alone.

Wiring it into an entry

The full checker is about 200 lines of Python using only the standard library. Here is the order we would run it in for an AI City Remix entry:

Write story.json with M3.1 Flash, one era per request at low effort.
Run the checker. Replace every flagged coordinate with the Wikidata value it prints, and fix or rename each zero-match OSM reference until a count query returns at least one.
Read the flagged claims and the manual list. This is the only step that needs a person, and it is now about a dozen lines instead of the whole script.
Run resolve.mjs, then film.mjs --stills at the eras you changed.
Only then run media.mjs --confirm.

If you publish the checker with your entry, it also speaks to the workflow criterion: less manual work, a new tool, and open source scores extra under the event rules. Keep the human read in step 3, though. In our two scripts the checker missed two real errors and raised two false alarms. Going entirely unattended would have shipped both misses.

FAQ

Does this need a paid API?

Only the claims step calls a model. That is one M3.1 Flash request per city through SandBase, which cost $0.0004–$0.0005 in our runs. Wikidata, Wikipedia and Overpass are free public services; keep your request rate low and send a descriptive User-Agent.

Can it tell me a claim is true?

No. It tells you whether a year appears next to a matching word in one reference article. A “found” claim can still be wrong, as the Vindobona miss showed, and a flagged claim can be true. Use it to decide what to read.

Why not ask the model to fact-check itself?

Because the checks that worked best here do not trust a model at all: coordinates come from Wikidata, outlines from OpenStreetMap. The one model call only lists claims. Judging them against a source is left to code and a person.

Will it work for cities other than Vienna and Kyoto?

The coordinate, OSM and boundary checks are city-independent. The claims check needs an English Wikipedia article titled “History of ”; for a smaller place, point it at the city’s main article or another reference you trust.

Does using this break the AI City Remix rules?

No. The rules require M3.1 Flash to direct the entry and allow any tool chain around it. This checker adds data lookups and uses M3.1 Flash for its one model step.

A city script is cheap to write and expensive to get wrong on screen. Four checks against open data turn “read every line” into “read about a dozen”. That is the part of an entry that automates well. The remaining reading is short enough to do properly, and that reading is where the accuracy points come from.