# simple-rpa-win

**Repository Path**: qty_java/simple-rpa-win

## Basic Information

- **Project Name**: simple-rpa-win
- **Description**: 轻量级Windows自动化工具客户端，基于RPA技术实现桌面任务自动化，开源易用，适合个人与企业高效办公。
- **Primary Language**: C#
- **License**: Apache-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2026-07-08
- **Last Updated**: 2026-08-01

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# Simple RPA Win

简体中文 | [English](#english)

## Project Overview

Simple RPA Win is a Windows desktop automation agent that provides remote control capabilities via the WebSocket protocol. It leverages Microsoft UI Automation (UIA) technology to automate Windows desktop applications, supporting window management, control operations, OCR text recognition, and more.

## Technology Stack

- **.NET 8.0** - Runtime environment
- **FlaUI** - UI automation framework (supports UIA2 and UIA3)
- **Windows OCR** - Text recognition engine
- **WebSocket** - Real-time communication protocol
- **ASP.NET Core** - Hosting framework

## Core Features

### 1. Window Management

| Command | Description |
|---------|-------------|
| `window.list` | List all top-level windows in the current desktop session |
| `window.find` | Find windows by title keyword or process name |
| `window.activate` | Activate a specified window |
| `window.wait` | Wait for a window to appear |
| `window.minimize` | Minimize a specified window |
| `window.maximize` | Maximize a specified window |

### 2. Control Operations

| Command | Description |
|---------|-------------|
| `tree.snapshot` | Get a snapshot of the control tree for a specified window |
| `element.find` | Find a control using a locator |
| `element.invoke` | Invoke a control via InvokePattern |
| `element.click` | Click a control (prioritizes Pattern; falls back to coordinates if failed) |
| `element.hover` | Move the mouse cursor to the center of the control |
| `element.setValue` | Set a control's value (prioritizes ValuePattern; falls back to clipboard if failed) |
| `element.getValue` | Read a control's value |
| `element.getText` | Read a control's text |

### 3. Input Control

| Command | Description |
|---------|-------------|
| `input.type` | Type text into the currently focused element |
| `input.paste` | Paste text into the currently focused element |
| `input.click` | Left-click at specified screen coordinates |
| `input.move` | Move the mouse to specified screen coordinates |
| `input.doubleClick` | Double-click at specified screen coordinates |
| `input.rightClick` | Right-click at specified screen coordinates |
| `input.scroll` | Scroll the mouse wheel at specified screen coordinates |
| `input.hotkey` | Execute a keyboard shortcut combination |

### 4. Application Management

| Command | Description |
|---------|-------------|
| `process.list` | List all current system processes |
| `app.start` | Launch an application by executable path |
| `app.kill` | Terminate an application by process name |
| `app.locate` | Automatically search for the application's executable path |
| `app.ensure` | Ensure an application is running and activated to the foreground |

### 5. OCR Recognition

| Command | Description |
|---------|-------------|
| `element.findText` | Locate text within a window using OCR, with optional auto-click |
| `ocr.recognizeWindow` | Perform full OCR recognition on a window region |

### 6. Verification & Waiting

| Command | Description |
|---------|-------------|
| `verify.waitText` | Wait for specified text to appear in a window's control tree |
| `verify.waitElement` | Wait for a specified control to appear |

### 7. Screen Operations

| Command | Description |
|---------|-------------|
| `screen.capture` | Capture the current desktop screen |
| `screen.cleanup` | Delete all screenshot files in the screenshot directory |

### 8. File Operations

| Command | Description |
|---------|-------------|
| `file.readText` | Read content from a text file (restricted access) |

## Control Locators

The project supports multiple control location methods:

```csharp
public class ElementLocatorModel
{
    public string? WindowId { get; set; }        // Window ID
    public string? WindowTitle { get; set; }      // Window title
    public string? ElementId { get; set; }       // Element ID
    public string? RuntimeId { get; set; }      // Runtime ID
    public string? AutomationId { get; set; }    // Automation ID
    public string? Name { get; set; }           // Control name
    public NameMatchKind NameMatch { get; set; } // Name matching method
    public string? ControlType { get; set; }    // Control type
    public string? ClassName { get; set; }      // Class name
    public string? Path { get; set; }           // Path
    public int? Index { get; set; }             // Index
    public string? NearName { get; set; }       // Nearby name
    public bool HoverBeforeFind { get; set; }   // Hover before finding
}
```

## Configuration

Configuration file: `appsettings.json`

```json
{
  "Agent": {
    "ListenUrl": "http://127.0.0.1:5057",
    "DefaultMaxDepth": 8,
    "DefaultMaxElements": 500,
    "DefaultTimeoutMs": 10000,
    "ActionDelayMs": 150,
    "RestoreClipboardAfterPaste": true,
    "MaximumTextLength": 4000,
    "ScreenshotDirectory": "C:\\rag\\images",
    "LogDirectory": "C:\\rag\\logs",
    "UiaInitializationTimeoutMs": 8000,
    "WebSocketServerUrl": "",
    "WebSocketReconnectSeconds": 5,
    "WebSocketHeartbeatSeconds": 30
  }
}
```

### Configuration Options

| Option | Default | Description |
|--------|---------|-------------|
| ListenUrl | http://127.0.0.1:5057 | Local listening address |
| DefaultMaxDepth | 8 | Default maximum depth of control tree |
| DefaultMaxElements | 500 | Default maximum number of controls in tree |
| DefaultTimeoutMs | 10000 | Default timeout (milliseconds) |
| ActionDelayMs | 150 | Delay between actions (milliseconds) |
| ScreenshotDirectory | C:\rag\images | Directory to save screenshots |
| LogDirectory | C:\rag\logs | Log directory |
| WebSocketServerUrl | - | WebSocket server address |
| WebSocketReconnectSeconds | 5 | Reconnection interval (seconds) |
| WebSocketHeartbeatSeconds | 30 | Heartbeat interval (seconds) |

## Build and Run

### Requirements

- Windows 10/11 or Windows Server 2019+
- .NET 8.0 SDK
- Visual Studio 2022 or higher (recommended)

### Build Steps

```bash
# Clone the project
git clone https://gitee.com/qty_java/simple-rpa-win.git
cd simple-rpa-win

# Build the project
dotnet build src/Simple.Automation.Agent/Simple.Automation.Agent.csproj -c Release
```

### Run

```bash
# Run the agent
dotnet src/Simple.Automation.Agent/bin/Release/net8.0-windows/win-x64/Simple.Automation.Agent.dll
```

Or directly execute the compiled binary:

```
src\Simple.Automation.Agent\bin\Release\net8.0-windows\win-x64\Simple.Automation.Agent.exe
```

## Communication Protocol

The agent communicates with clients via WebSocket, supporting the following message formats:

### Request Message

```json
{
  "type": "command",
  "id": "unique-id",
  "command": "window.list",
  "args": {}
}
```

### Response Message

```json
{
  "type": "result",
  "id": "unique-id",
  "command": "window.list",
  "success": true,
  "message": "Success",
  "data": [...]
}
```

### Batch Execution

```json
{
  "type": "batch",
  "payload": {
    "commands": [
      { "command": "app.start", "args": { "filePath": "notepad.exe" } },
      { "command": "window.wait", "args": { "titleKeyword": "无标题" } }
    ],
    "stopOnFailure": true,
    "minDelayMs": 0,
    "maxDelayMs": 2000
  }
}
```

## Project Structure

```
simple-rpa-win/
├── src/
│   └── Simple.Automation.Agent/
│       ├── Commands/                    # Command handlers
│       │   ├── CommandDispatcher.cs    # Command dispatcher
│       │   └── Handlers/               # Command handlers
│       ├── Common/
│       │   ├── Dtos/                   # Data transfer objects
│       │   ├── Errors/                 # Error definitions
│       │   └── Models/                 # Data models
│       ├── Infrastructure/
│       │   ├── Logging/                # Logging infrastructure
│       │   ├── Uia/                   # UI Automation infrastructure
│       │   ├── WebSocket/              # WebSocket client
│       │   └── Win32/                  # Win32 API wrappers
│       ├── Services/
│       │   ├── Elements/               # Control services
│       │   ├── Input/                 # Input services
│       │   ├── Ocr/                   # OCR services
│       │   ├── System/                # System services
│       │   ├── Verification/          # Verification services
│       │   └── Windows/               # Window services
│       ├── Options/                    # Configuration options
│       └── Program.cs                  # Entry point
└── simple-rpa-win.sln
```

## License

MIT License

---

## English

### Overview

Simple RPA Win is a Windows desktop automation agent that provides remote control capabilities via the WebSocket protocol. It leverages Microsoft UI Automation (UIA) technology to automate Windows desktop applications, supporting window management, control operations, OCR text recognition, and more.

### Features

- **Window Management**: List, find, activate, minimize, maximize windows
- **Control Operations**: Find, click, hover, set/get value, invoke controls
- **Input Simulation**: Type, paste, click, move, scroll, hotkey
- **Application Management**: Start, kill, locate applications
- **OCR Recognition**: Text recognition using Windows OCR engine
- **Screen Capture**: Screenshot functionality

### Requirements

- Windows 10/11 or Windows Server 2019+
- .NET 8.0 SDK

### Build

```bash
dotnet build src/Simple.Automation.Agent/Simple.Automation.Agent.csproj -c Release
```

### License

MIT License