# llmspeedtest

**Repository Path**: startvibecoding/llmspeedtest

## Basic Information

- **Project Name**: llmspeedtest
- **Description**: 测试 OpenAI Chat 和 Anthropic 格式模型的速度，测量首次响应延迟和每秒 Token 数。
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-05-23
- **Last Updated**: 2026-05-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# LLMSpeed - LLM Token Speed Testing Tool

Test the speed of OpenAI Chat and Anthropic format models, measuring first token latency and tokens per second.

## Features

- Support OpenAI Chat and Anthropic streaming APIs
- Measure first token latency (ms) and tokens per second
- Detect invalid models early via content-type check
- Support `reasoning_content` field for reasoning models
- Export results to CSV

## Config File Format (config.yaml)

```yaml
test_prompt: "Please introduce yourself"

models:
  - base_url: "https://api.openai.com/v1"
    type: "openai-chat"
    models:
      - "gpt-4o"
      - "gpt-4o-mini"
    api_key: "sk-xxx"

  - base_url: "https://api.anthropic.com"
    type: "anthropic"
    models:
      - "claude-3-5-sonnet-20241022"
    api_key: "sk-ant-xxx"
```

## Run

```bash
pip install openai anthropic pyyaml
python llmspeed.py
```

Use a custom config file:

```bash
python llmspeed.py my_config.yaml
```

## Output

Results are saved to `results.csv` with the following columns:

| Column | Description |
|--------|-------------|
| `base_url` | API endpoint |
| `type` | API type (`openai-chat` or `anthropic`) |
| `model` | Model name |
| `first_token_latency` | First token latency in ms (-1 if failed) |
| `tokens_per_second` | Tokens per second (-1 if failed) |