Primary metric
Median great-circle error (lower is better)
Repository
https://github.com/LostBee/pinpoint-eval
Tracks
open-world, urban, rural

Methodology

Pinpoint Eval tests multimodal models on single-frame geolocation without search tools or EXIF metadata. Active evaluation images stay encrypted in an offline vault to prevent training-set contamination; only aggregate distances and synthetic coordinate examples are exported to reports.

Latest results

From No. 001 (2026-11-01).

Show data
ModelMedian geolocation error (km)95% CIn
Gemini 3 Pro 118 km 94 km – 146 km 400
Claude Opus 4.5 142 km 116 km – 171 km 400
GPT-5 164 km 132 km – 198 km 400
Qwen3 Max 248 km 205 km – 294 km 400
Grok 4 312 km 261 km – 368 km 400
Llama 4 Maverick 425 km 370 km – 488 km 400
Fig. 1 — Median geolocation error on Pinpoint Eval. Lower is better.
Model Median geolocation error (km) 95% CI n Note
Gemini 3 Pro 118 km 94 km – 146 km 400 Strong coastal recall
Claude Opus 4.5 142 km 116 km – 171 km 400 —
GPT-5 164 km 132 km – 198 km 400 —
Qwen3 Max 248 km 205 km – 294 km 400 —
Grok 4 312 km 261 km – 368 km 400 —
Llama 4 Maverick 425 km 370 km – 488 km 400 —

Run timeline