Testing

FlowDeck CLI provides comprehensive test execution capabilities, making it easy to run tests from the terminal or integrate into CI/CD pipelines.

Quick Start

After running flowdeck config set, workspace, scheme, and simulator are saved to project state. Run tests with:

flowdeck test

To make headless simulator execution explicit for an app-hosted or agent workflow:

flowdeck test --headless

Simulator tests already run without opening Simulator.app. The flag records the caller’s intent and is ignored for macOS and physical-device targets.

flowdeck config set saves runtime selection state. If you pass --config, that uses the explicit command config format, which is separate from .flowdeck/config.json and .flowdeck/config.local.json.

Running All Tests

# With saved settings (after config set)
flowdeck test

# Or specify settings directly
flowdeck test -w MyApp.xcworkspace -s MyScheme -S "iPhone 16"

Running Tests on macOS

Use --device "My Mac" for macOS-only test targets:

flowdeck test -D "My Mac"

Filtering Tests

By Test Target

Run specific test targets:

flowdeck test --test-targets MyAppTests

# Multiple targets
flowdeck test --test-targets "MyAppTests,MyAppUITests"

By Test Class or Method

The --only option supports flexible matching:

# Run all tests in a target/class
flowdeck test --only MyAppTests/LoginTests

# Run a specific test method
flowdeck test --only MyAppTests/LoginTests/testValidLogin

# Run a top-level Swift Testing function (no synthetic class segment)
flowdeck test --only 'MyAppTests/topLevelTest()'

# Full test path with spaces (use quotes)
flowdeck test --only "MyAppTests/Login Tests/test Login Success"

Skipping Tests

Skip specific tests:

flowdeck test --skip MyAppTests/SlowIntegrationTests
flowdeck test --skip MyAppTests/IntegrationTests

Test Plans

Run a specific test plan by name or by path:

flowdeck test --plan "Smoke"
flowdeck test --plan "TestPlans/Smoke.xctestplan"
flowdeck test --plan "Smoke" --only MyAppTests/LoginTests

Filters are resolved against the active plan. A test selected by Smoke can be run with --only or --test-cases even when another plan skips it. Without --plan, FlowDeck uses the scheme’s default test plan. If a selected plan is not attached to the scheme, its referenced file is missing, or the plan is malformed, FlowDeck stops before test execution and points to flowdeck test plans for the available plan names.

List Test Plans

List test plans referenced by a scheme (no build required):

# After config set
flowdeck test plans

# Explicit parameters
flowdeck test plans -w MyApp.xcworkspace -s MyScheme

# JSON output for tooling
flowdeck test plans --json

Test Discovery

Discover all available tests before running. Test discovery uses static source parsing (like Xcode’s Test Navigator) and does not require building the project.

# After config set, discover tests with saved settings
flowdeck test discover

# Or specify settings directly
flowdeck test discover -w MyApp.xcworkspace -s MyScheme

# JSON format for tooling
flowdeck test discover --json

# Filter by name
flowdeck test discover --filter Login
flowdeck test discover -F Login

# Discover the tests enabled by a specific plan
flowdeck test discover --plan Smoke

# Include tests that are skipped in the scheme or test plan
flowdeck test discover --include-skipped-tests

By default, tests disabled by the scheme’s default plan (or the plan selected with --plan) are excluded from discovery results. Use --include-skipped-tests to show excluded and unselected tests marked as skipped in the output. When --plan is present on both the parent test command and the discover subcommand, the value after discover wins. Plan-selection failures are not silently ignored. Example output:

🧪 Tests in MyScheme

LoginTests
  - testValidLogin LoginTests.swift:23
  - testInvalidPassword LoginTests.swift:45

CartTests
  - testAddItem CartTests.swift:12
  - testRemoveItem CartTests.swift:34

Total: 4 test(s) in 2 class(es)

With --include-skipped-tests:

🧪 Tests in MyScheme

LoginTests
  - testValidLogin LoginTests.swift:23
  - testInvalidPassword LoginTests.swift:45
  - testSlowIntegration LoginTests.swift:67 (skipped)

CartTests
  - testAddItem CartTests.swift:12
  - testRemoveItem CartTests.swift:34

Total: 5 test(s) in 2 class(es) (1 skipped)

JSON output with skipped tests:

{
  "tests": [
    {
      "target": "MyAppTests",
      "class": "LoginTests",
      "method": "testSlowIntegration",
      "identifier": "MyAppTests/LoginTests/testSlowIntegration",
      "file": "LoginTests.swift",
      "filePath": "/path/to/LoginTests.swift",
      "lineNumber": 67,
      "isSkipped": true
    }
  ]
}

JSON Output

For CI/CD integration, use JSON output:

flowdeck test --json

Example JSON output:

{"type":"test_started","data":{"testId":"LoginTests/testValidLogin"}}
{"type":"test_passed","data":{"testId":"LoginTests/testValidLogin","duration":0.123}}
{"type":"test_failed","data":{"testId":"LoginTests/testInvalidPassword","failure_message":"Expected failure but got success"}}
{"schema":"flowdeck.test","schemaVersion":"1.1.0","success":false,"totalTests":2,"passedTests":1,"failedTests":1,"skippedTests":0,"resultBundlePath":"/Users/me/.flowdeck/logs/abc123/test.xcresult"}

The final flowdeck.test object is authoritative: its totals and verdict come from the finalized xcresult, reconciled with the real xcodebuild status. Live events remain useful for progress and richer failure locations. Any nonzero xcodebuild status—including 65—keeps success false.

Finalized Result Bundles

Every real test run saves a finalized bundle to ~/.flowdeck/logs/<project-hash>/test.xcresult. The optional absolute resultBundlePath in the final JSON points to that bundle. Supply -resultBundlePath through --xcodebuild-options to choose another location; the final supplied value wins and applies only to the test phase. Verbose output preserves canonical ** TEST SUCCEEDED **, ** TEST FAILED **, and ** TEST EXECUTE FAILED ** verdict lines. Human output also branches on the overall success value, so a process or infrastructure failure cannot be labeled as passing merely because zero failed tests were parsed.

CI/CD Integration

GitHub Actions Example

name: Tests
on: [push, pull_request]

jobs:
  test:
    runs-on: macos-14
    steps:
      - uses: actions/checkout@v4

      - name: Install FlowDeck
        run: curl -sSL https://flowdeck.studio/install.sh | sh

      - name: Run Tests
        env:
          FLOWDECK_LICENSE_KEY: ${{ secrets.FLOWDECK_LICENSE_KEY }}
        run: |
          flowdeck test \
            --workspace MyApp.xcworkspace \
            --simulator "iPhone 16" \
            --json > test-results.json

      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: test-results.json

Parsing Results

# Count passed tests
flowdeck test --json | \
  jq -s '[.[] | select(.type == "test_passed")] | length'

# List failed tests
flowdeck test --json | \
  jq -s '[.[] | select(.type == "test_failed")] | .[].data.testId'

# Read the finalized aggregate and bundle path
flowdeck test --json | \
  jq 'select(.schema == "flowdeck.test") | {success, totalTests, passedTests, failedTests, resultBundlePath}'

Test Output

Human-Readable Output

🧪 Running tests...
  Workspace: MyApp.xcworkspace
  Scheme: MyApp
  Configuration: Debug
  Simulator: iPhone 16 Pro

Test Results
============

Total Tests: 45
Passed: 43 ✅
Failed: 2 ❌
Skipped: 0 ⏭️
Duration: 12.34s

Failures:
  ❌ testLoginWithInvalidCredentials: Expected failure but got success
  ❌ testNetworkTimeout: Async expectation not fulfilled

❌ Some tests failed.

Tips

Use --json output in CI/CD for reliable parsing and integration with test reporting tools.

Test discovery uses static source parsing and doesn’t require building. It finds XCTestCase subclasses, test* methods, @Suite structs, and @Test functions directly from source files.

Troubleshooting

Tests Not Discovered

Test discovery parses source files directly. If tests aren’t found:

Ensure test files are in the scheme’s test targets
Verify test classes inherit from XCTestCase or use @Test/@Suite
Check that XCTest methods start with test

# Verify scheme and targets
flowdeck context --json | jq '.schemes'
flowdeck test discover --filter "YourTestClass"

Simulator Issues

If tests fail to start, try booting the simulator first:

flowdeck simulator list --platform iOS
flowdeck simulator boot <UDID>
flowdeck test -S "iPhone 16"

Overview

FlowDeck for AI

Interactive Mode (TUI)

Core Features

Command Reference

Advanced

Support

Quick Start

Running All Tests

Running Tests on macOS

Filtering Tests

By Test Target

By Test Class or Method

Skipping Tests

Test Plans

List Test Plans

Test Discovery

JSON Output

Finalized Result Bundles

CI/CD Integration

GitHub Actions Example

Parsing Results

Test Output

Human-Readable Output

Tips

Troubleshooting

Tests Not Discovered

Simulator Issues

​Quick Start

​Running All Tests

​Running Tests on macOS

​Filtering Tests

​By Test Target

​By Test Class or Method

​Skipping Tests

​Test Plans

​List Test Plans

​Test Discovery

​JSON Output

​Finalized Result Bundles

​CI/CD Integration

​GitHub Actions Example

​Parsing Results

​Test Output

​Human-Readable Output

​Tips

​Troubleshooting

​Tests Not Discovered

​Simulator Issues

Quick Start

Running All Tests

Running Tests on macOS

Filtering Tests

By Test Target

By Test Class or Method

Skipping Tests

Test Plans

List Test Plans

Test Discovery

JSON Output

Finalized Result Bundles

CI/CD Integration

GitHub Actions Example

Parsing Results

Test Output

Human-Readable Output

Tips

Troubleshooting

Tests Not Discovered

Simulator Issues