Universal Token Counter (UTC)

Overview

Universal Token Counter (UTC) is a high-precision, multilingual token counting API designed for LLM workflows.

It supports OpenAI-compatible encodings (e.g. o200k_base, cl100k_base) and returns token counts, character counts, and density in a unified JSON format.

• OpenAI-compatible encodings (e.g. gpt-4o family)
• Japanese and multilingual text support
• Simple REST API / FastAPI implementation
• result + meta two-layer response (UTC Spec v0.1)

Hosted API on RapidAPI

UTC is available on RapidAPI.

• Interactive testing in the browser
• API key management and rate limit visibility
• Free tier available
• Auto-generated snippets for cURL / JS / Python

🔗 RapidAPI Hub: https://rapidapi.com/APIronlab/api/universal-token-counter-utc

Endpoint

POST /utc/v0/token-count

Request example:

{
  "model": "gpt-4o",
  "text": "これはテストです"
}

Response example:

{
  "result": {
    "model": "gpt-4o",
    "encoding": "o200k_base",
    "char_count": 8,
    "token_count": 5,
    "token_per_char": 0.625
  },
  "meta": {
    "input_language": "ja",
    "input_size_bytes": 17,
    "token_density": 0.294,
    "model_family": "openai",
    "processing_time_ms": 3.24,
    "utc_timestamp": "2025-01-01T00:00:00Z",
    "version": "0.1.0"
  }
}

Supported Models (example)

See the GitHub repository or RapidAPI documentation for the actual list of supported models.

Use Cases

• Estimating and managing LLM inference cost
• Tracking token usage in logging systems
• Controlling prompt length against token limits
• Comparing “token efficiency” across models

Quick Start – Python Example

import requests

payload = {
    "model": "gpt-4o",
    "text": "これはテストです",
}

res = requests.post(
    "https://your-endpoint/utc/v0/token-count",  # RapidAPI / API Gateway etc.
    json=payload,
)
print(res.json())

Links

• GitHub: https://github.com/APIron-lab/Universal-Token-Counter
• RapidAPI: https://rapidapi.com/APIronlab/api/universal-token-counter-utc