> ## Documentation Index
> Fetch the complete documentation index at: https://docs.flowdeck.studio/llms.txt
> Use this file to discover all available pages before exploring further.

# Testing

> Run unit and UI tests from the command line

FlowDeck CLI provides comprehensive test execution capabilities, making it easy to run tests from the terminal or integrate into CI/CD pipelines.

## Quick Start

After running `flowdeck config set`, workspace, scheme, and simulator are saved to project state. Run tests with:

```bash theme={null}
flowdeck test
```

To make headless simulator execution explicit for an app-hosted or agent workflow:

```bash theme={null}
flowdeck test --headless
```

Simulator tests already run without opening Simulator.app. The flag records the caller's intent and is ignored for macOS and physical-device targets.

<Note>
  `flowdeck config set` saves runtime selection state. If you pass `--config`, that uses the explicit command config format, which is separate from `.flowdeck/config.json` and `.flowdeck/config.local.json`.
</Note>

## Running All Tests

```bash theme={null}
# With saved settings (after config set)
flowdeck test

# Or specify settings directly
flowdeck test -w MyApp.xcworkspace -s MyScheme -S "iPhone 16"
```

## Running Tests on macOS

Use `--device "My Mac"` for macOS-only test targets:

```bash theme={null}
flowdeck test -D "My Mac"
```

## Filtering Tests

### By Test Target

Run specific test targets:

```bash theme={null}
flowdeck test --test-targets MyAppTests

# Multiple targets
flowdeck test --test-targets "MyAppTests,MyAppUITests"
```

### By Test Class or Method

The `--only` option supports flexible matching:

```bash theme={null}
# Run all tests in a target/class
flowdeck test --only MyAppTests/LoginTests

# Run a specific test method
flowdeck test --only MyAppTests/LoginTests/testValidLogin

# Run a top-level Swift Testing function (no synthetic class segment)
flowdeck test --only 'MyAppTests/topLevelTest()'

# Full test path with spaces (use quotes)
flowdeck test --only "MyAppTests/Login Tests/test Login Success"
```

### Skipping Tests

Skip specific tests:

```bash theme={null}
flowdeck test --skip MyAppTests/SlowIntegrationTests
flowdeck test --skip MyAppTests/IntegrationTests
```

## Test Plans

Run a specific test plan by name or by path:

```bash theme={null}
flowdeck test --plan "Smoke"
flowdeck test --plan "TestPlans/Smoke.xctestplan"
flowdeck test --plan "Smoke" --only MyAppTests/LoginTests
```

Filters are resolved against the active plan. A test selected by `Smoke` can
be run with `--only` or `--test-cases` even when another plan skips it. Without
`--plan`, FlowDeck uses the scheme's default test plan.

If a selected plan is not attached to the scheme, its referenced file is
missing, or the plan is malformed, FlowDeck stops before test execution and
points to `flowdeck test plans` for the available plan names.

### List Test Plans

List test plans referenced by a scheme (no build required):

```bash theme={null}
# After config set
flowdeck test plans

# Explicit parameters
flowdeck test plans -w MyApp.xcworkspace -s MyScheme

# JSON output for tooling
flowdeck test plans --json
```

## Test Discovery

Discover all available tests before running. Test discovery uses static source parsing (like Xcode's Test Navigator) and does not require building the project.

```bash theme={null}
# After config set, discover tests with saved settings
flowdeck test discover

# Or specify settings directly
flowdeck test discover -w MyApp.xcworkspace -s MyScheme

# JSON format for tooling
flowdeck test discover --json

# Filter by name
flowdeck test discover --filter Login
flowdeck test discover -F Login

# Discover the tests enabled by a specific plan
flowdeck test discover --plan Smoke

# Include tests that are skipped in the scheme or test plan
flowdeck test discover --include-skipped-tests
```

By default, tests disabled by the scheme's default plan (or the plan selected
with `--plan`) are excluded from discovery results. Use
`--include-skipped-tests` to show excluded and unselected tests marked as
skipped in the output.

When `--plan` is present on both the parent `test` command and the `discover`
subcommand, the value after `discover` wins. Plan-selection failures are not
silently ignored.

**Example output:**

```
🧪 Tests in MyScheme

LoginTests
  - testValidLogin LoginTests.swift:23
  - testInvalidPassword LoginTests.swift:45

CartTests
  - testAddItem CartTests.swift:12
  - testRemoveItem CartTests.swift:34

Total: 4 test(s) in 2 class(es)
```

**With `--include-skipped-tests`:**

```
🧪 Tests in MyScheme

LoginTests
  - testValidLogin LoginTests.swift:23
  - testInvalidPassword LoginTests.swift:45
  - testSlowIntegration LoginTests.swift:67 (skipped)

CartTests
  - testAddItem CartTests.swift:12
  - testRemoveItem CartTests.swift:34

Total: 5 test(s) in 2 class(es) (1 skipped)
```

**JSON output with skipped tests:**

```json theme={null}
{
  "tests": [
    {
      "target": "MyAppTests",
      "class": "LoginTests",
      "method": "testSlowIntegration",
      "identifier": "MyAppTests/LoginTests/testSlowIntegration",
      "file": "LoginTests.swift",
      "filePath": "/path/to/LoginTests.swift",
      "lineNumber": 67,
      "isSkipped": true
    }
  ]
}
```

## JSON Output

For CI/CD integration, use JSON output:

```bash theme={null}
flowdeck test --json
```

**Example JSON output:**

```json theme={null}
{"type":"test_started","data":{"testId":"LoginTests/testValidLogin"}}
{"type":"test_passed","data":{"testId":"LoginTests/testValidLogin","duration":0.123}}
{"type":"test_failed","data":{"testId":"LoginTests/testInvalidPassword","failure_message":"Expected failure but got success"}}
{"schema":"flowdeck.test","schemaVersion":"1.1.0","success":false,"totalTests":2,"passedTests":1,"failedTests":1,"skippedTests":0,"resultBundlePath":"/Users/me/.flowdeck/logs/abc123/test.xcresult"}
```

The final `flowdeck.test` object is authoritative: its totals and verdict come from the finalized xcresult, reconciled with the real xcodebuild status. Live events remain useful for progress and richer failure locations. Any nonzero xcodebuild status—including 65—keeps `success` false.

## Finalized Result Bundles

Every real test run saves a finalized bundle to `~/.flowdeck/logs/<project-hash>/test.xcresult`. The optional absolute `resultBundlePath` in the final JSON points to that bundle. Supply `-resultBundlePath` through `--xcodebuild-options` to choose another location; the final supplied value wins and applies only to the test phase.

Verbose output preserves canonical `** TEST SUCCEEDED **`, `** TEST FAILED **`, and `** TEST EXECUTE FAILED **` verdict lines. Human output also branches on the overall `success` value, so a process or infrastructure failure cannot be labeled as passing merely because zero failed tests were parsed.

## CI/CD Integration

### GitHub Actions Example

```yaml theme={null}
name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: macos-14
    steps:
      - uses: actions/checkout@v4

      - name: Install FlowDeck
        run: curl -sSL https://flowdeck.studio/install.sh | sh

      - name: Run Tests
        env:
          FLOWDECK_LICENSE_KEY: ${{ secrets.FLOWDECK_LICENSE_KEY }}
        run: |
          flowdeck test \
            --workspace MyApp.xcworkspace \
            --simulator "iPhone 16" \
            --json > test-results.json

      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: test-results.json
```

### Parsing Results

```bash theme={null}
# Count passed tests
flowdeck test --json | \
  jq -s '[.[] | select(.type == "test_passed")] | length'

# List failed tests
flowdeck test --json | \
  jq -s '[.[] | select(.type == "test_failed")] | .[].data.testId'

# Read the finalized aggregate and bundle path
flowdeck test --json | \
  jq 'select(.schema == "flowdeck.test") | {success, totalTests, passedTests, failedTests, resultBundlePath}'
```

## Test Output

### Human-Readable Output

```
🧪 Running tests...
  Workspace: MyApp.xcworkspace
  Scheme: MyApp
  Configuration: Debug
  Simulator: iPhone 16 Pro

Test Results
============

Total Tests: 45
Passed: 43 ✅
Failed: 2 ❌
Skipped: 0 ⏭️
Duration: 12.34s

Failures:
  ❌ testLoginWithInvalidCredentials: Expected failure but got success
  ❌ testNetworkTimeout: Async expectation not fulfilled

❌ Some tests failed.
```

## Tips

<Tip>
  Use `--json` output in CI/CD for reliable parsing and integration with test reporting tools.
</Tip>

<Note>
  Test discovery uses static source parsing and doesn't require building. It finds `XCTestCase` subclasses, `test*` methods, `@Suite` structs, and `@Test` functions directly from source files.
</Note>

## Troubleshooting

### Tests Not Discovered

Test discovery parses source files directly. If tests aren't found:

1. Ensure test files are in the scheme's test targets
2. Verify test classes inherit from `XCTestCase` or use `@Test`/`@Suite`
3. Check that XCTest methods start with `test`

```bash theme={null}
# Verify scheme and targets
flowdeck context --json | jq '.schemes'
flowdeck test discover --filter "YourTestClass"
```

### Simulator Issues

If tests fail to start, try booting the simulator first:

```bash theme={null}
flowdeck simulator list --platform iOS
flowdeck simulator boot <UDID>
flowdeck test -S "iPhone 16"
```