- Primary metric
- Median great-circle error (lower is better)
- Repository
- https://github.com/LostBee/pinpoint-eval
- Tracks
- open-world, urban, rural
Methodology
Pinpoint Eval tests multimodal models on single-frame geolocation without search tools or EXIF metadata. Active evaluation images stay encrypted in an offline vault to prevent training-set contamination; only aggregate distances and synthetic coordinate examples are exported to reports.
Latest results
From No. 001 (2026-11-01).
Gemini 3 Pro
118 km
Claude Opus 4.5
142 km
GPT-5
164 km
Qwen3 Max
248 km
Grok 4
312 km
Llama 4 Maverick
425 km
Show data
| Model | Median geolocation error (km) | 95% CI | n |
|---|---|---|---|
| Gemini 3 Pro | 118 km | 94 km – 146 km | 400 |
| Claude Opus 4.5 | 142 km | 116 km – 171 km | 400 |
| GPT-5 | 164 km | 132 km – 198 km | 400 |
| Qwen3 Max | 248 km | 205 km – 294 km | 400 |
| Grok 4 | 312 km | 261 km – 368 km | 400 |
| Llama 4 Maverick | 425 km | 370 km – 488 km | 400 |
| Model | Median geolocation error (km) | 95% CI | n | Note |
|---|---|---|---|---|
| Gemini 3 Pro | 118 km | 94 km – 146 km | 400 | Strong coastal recall |
| Claude Opus 4.5 | 142 km | 116 km – 171 km | 400 | — |
| GPT-5 | 164 km | 132 km – 198 km | 400 | — |
| Qwen3 Max | 248 km | 205 km – 294 km | 400 | — |
| Grok 4 | 312 km | 261 km – 368 km | 400 | — |
| Llama 4 Maverick | 425 km | 370 km – 488 km | 400 | — |
Run timeline
- No. 001