t2s-metrics Documentation

A lightweight evaluation toolkit for Text-to-SPARQL systems.

Contents

• Getting Started
  • Installation Guide
  • Project Layout
• User Guides
  • Basic Workflow Guide
  • Advanced Usage Guide
  • Dashboard and Exports
• Developer Guide
  • Architecture
  • Extending Metrics
  • Contributing
• API Reference
  • CLI Reference
  • Python API Reference

What is t2s-metrics?

t2s-metrics provides a configurable evaluation pipeline for Text-to-SPARQL systems:

• Load query pairs from JSONL files.

• Run a metric suite (exact-match, answer-set, token/text, execution-aware, ranking, and distance metrics).

• Export reproducible JSON results.

• Explore outcomes in an interactive dashboard or static dashboard snapshot.

Quick Links

• Installation and local setup: Getting Started

• End-to-end execution flow: Basic Workflow

• Parallel runs, custom exports, and backend choices: Advanced Usage

• How internals fit together: Architecture