Skip to main content
FlowDeck ships built-in UI automation for macOS apps so you can drive native interfaces, validate UI state, and capture accessibility trees from the CLI. Use flowdeck ui mac to run automation commands.

​
What It Does

Use macOS automation to:
  • Capture screenshots and accessibility trees from any running macOS app.
  • Click elements, type text, scroll, drag, and navigate menus.
  • Wait for UI state changes and assert UI conditions.
  • Manage windows, launch/quit apps, and send keyboard shortcuts.

​
When to Use It

  • Smoke tests for macOS app flows.
  • Scripted demos or QA checks on native Mac apps.
  • AI-driven interaction loops that need consistent UI state.
  • Automating menu-driven workflows or window management.

​
Prerequisites

macOS automation requires two system permissions:
  1. Accessibility — allows FlowDeck to read the accessibility tree and inject input.
  2. Screen Recording — allows FlowDeck to capture window screenshots (macOS 14+).
Check and request permissions: 
flowdeck ui mac check-permissions
flowdeck ui mac request-permissions
After granting permissions in System Settings > Privacy & Security, restart your terminal for changes to take effect.

​
Quick Start

# 1) Check permissions
flowdeck ui mac check-permissions

# 2) List running apps
flowdeck ui mac list apps

# 3) Capture screenshot + accessibility tree
flowdeck ui mac screen --app "Safari" --json

# 4) Click an element
flowdeck ui mac click "Downloads" --app "Safari"

# 5) Type text
flowdeck ui mac type "hello world" --app "TextEdit"

# 6) Press a hotkey
flowdeck ui mac hotkey "cmd+s" --app "TextEdit"
The --app flag accepts an app name (e.g., "Safari"), bundle ID (e.g., "com.apple.Safari"), or PID (e.g., "12345"). It is required on most commands.

​
Tangible Workflows

​
1) Automate a macOS app form fill

Use this to fill out a form in a native macOS app and verify the result. 
flowdeck ui mac click "nameField" --app "MyApp" --by-id
flowdeck ui mac type "John Doe" --app "MyApp"
flowdeck ui mac click "emailField" --app "MyApp" --by-id
flowdeck ui mac type "john@example.com" --app "MyApp"
flowdeck ui mac click "submitButton" --app "MyApp" --by-id
flowdeck ui mac wait "Success" --app "MyApp" --timeout 10
flowdeck ui mac screen --app "MyApp" --output /tmp/form-result.png

​
2) Drive a menu-based workflow

flowdeck ui mac menu list --app "TextEdit"
flowdeck ui mac menu click "File > Export as PDF" --app "TextEdit"
flowdeck ui mac wait "Save" --app "TextEdit" --timeout 5

​
3) Window management

flowdeck ui mac window list --app "Safari"
flowdeck ui mac window move --app "Safari" --to 100,100
flowdeck ui mac window resize --app "Safari" --size 1200,800
flowdeck ui mac window focus --app "Safari" --index 1

​
4) Verify a build output visually

Build and run your macOS app, then capture its UI: 
flowdeck run -D "My Mac"
flowdeck ui mac screen --app "MyApp" --output /tmp/current-ui.png
Compare /tmp/current-ui.png with the expected design and iterate.

​
App Targeting

The --app flag resolves targets in this order:
  1. Numeric — treated as a PID (e.g., --app "12345")
  2. Contains a dot — treated as a bundle ID (e.g., --app "com.apple.Safari")
  3. Otherwise — fuzzy-matched against running app names (e.g., --app "Safari")
To see all running GUI apps: 
flowdeck ui mac list apps
flowdeck ui mac list apps --include-agents --include-system

​
Differences from iOS Simulator Automation

FeatureiOS (ui simulator)macOS (ui mac)
Target flag-S (simulator name/UDID)--app (name/bundle ID/PID)
Click/taptapclick (tap is a hidden alias)
Right-clickN/Aright-click
DragN/Adrag --from x,y --to x,y
Cursor movementN/Amove --point x,y
Keyboard shortcutsN/Ahotkey "cmd+s"
Key inputHID keycodes only--name (e.g., return) or --keycode
Menu interactionN/Amenu list / menu click
Window managementN/Awindow list/move/resize/focus
App lifecycleN/Alaunch / activate / quit
SessionsBackground capture sessionsNo sessions (use screen for one-off)
Assertassert visible/hidden/enabled/disabled/textassert visible/hidden/enabled/disabled/text
ScrollFraction-based --distanceTick-based --amount
Scroll untilscroll --untilscroll --until
SwipeDirection argument--direction flag
Multi-touchpinch, rotateN/A
Hardware buttonsbutton home/lock/...N/A
Deep linksopen-urlN/A
Appearanceset-appearance light/darkN/A
CoordinatesPoints (normalized)Screen-absolute points
PermissionsNone neededAccessibility + Screen Recording

​
Performance and Reliability Tips

  • Prefer accessibility identifiers and use --by-id for clicks, finds, and assertions (fastest and most reliable).
  • Use flowdeck ui mac screen --tree --json --app "MyApp" when you only need the accessibility tree (no screenshot).
  • Coordinates are screen-absolute points matching the values returned by find. Do not scale by Retina factors.
  • Use flowdeck ui mac find to discover element labels and IDs before interacting.
  • For smooth scrolling, use --smooth on the scroll command.
  • Tune input timing with environment variables:
    • FLOWDECK_HID_STABILIZATION_MS (default 25) for click/gesture stability
    • FLOWDECK_TYPE_DELAY_MS (default 1) for typing speed
For the complete command list and flags, see the macOS UI Automation command reference.