Back to docs Munk Test
Munk Test
Getting Started

Settings

Learn what each Settings module controls, how config.yaml is resolved, and copy a ready-to-edit reference configuration.

The Web UI Settings page edits the active config.yaml used by munk serve. That same file also drives CLI and MCP flows that share the local orchestration host.

You do not need to configure every field on day one. Start with a global provider, then add the modules below only when your workflow needs them.

Where the config lives

Settings always reads and writes the currently active config file in the discovery chain:

  1. --config on the CLI
  2. MUNK_CONFIG environment variable
  3. <workspace>/.munk/config.yaml
  4. <Munk Test profile home>/config/config.yaml

The page header shows the resolved path. Use Refresh to reload from disk, and Save to write the form back to that file.

Shared vs local layers

config.yaml uses two top-level sections:

SectionSynced to cloud?Typical contents
sharedYes (as Bundle team_config)provider choice, non-secret provider fields, runtime, orchestration, agents, test_env
localNeverapi_key, sudo_password, proxy, ios_bridge, and other machine-only secrets

Effective runtime config is deep_merge(shared, local). Cloud Pull replaces shared only and leaves local intact. Legacy flat files still load; saving from Settings (or a successful Pull) rewrites them into this layered shape.

Modules

Global Provider

This is the default model connection for the whole product.

  • Choose one active provider: openai_compatible or gemini.
  • Both provider sections can stay in the file as standby configs, but only the selected provider is used at runtime.
  • Typical fields include endpoint, model id, API key, and optional advanced options such as timeout, extra headers, or Vertex AI settings.

For a first setup, fill in the active provider's base_url / model / api_key (or Gemini equivalents) and save. That is enough to generate plans and run cases.

Agent Overrides

Each role can optionally use its own provider instead of inheriting the global one:

RoleUsed for
PlanGenerating and structuring test plans
RunnerDriving on-device / in-browser actions
JudgeScoring case outcomes and deciding retries
ReviewReviewing structured assets
AnalysisPost-run analysis

Leave a role disabled to inherit the global provider. Enable an override only when that role needs a different model, endpoint, or credentials.

Orchestration

Controls case-level retry policy after Judge returns a verdict:

  • how many automatic retries are allowed
  • whether failed / inconclusive can enter the retry branch
  • whether the workflow escalates after the retry budget is exhausted

Use this when you want more resilience on flaky UI flows, or stricter fail-fast behavior in CI.

Proxy

Routes external Python and LLM traffic through a local HTTP or SOCKS proxy.

Enable this when your network requires a proxy to reach model providers. Localhost targets stay direct by default; you can also maintain a no-proxy allowlist.

iOS Bridge

Configures sudo startup for the iOS real-device bridge, mainly for iOS 18+ tunnel creation.

Enable sudo startup only when your Mac requires elevated privileges to create the device tunnel. The password is stored under local.ios_bridge in config.yaml, so treat that file as sensitive on shared machines.

For the full iOS device path, see iOS Real Device Setup.

Test Environment

Registers shared resources that TestCase.setup may use before a run starts:

  • HTTP bases: named backend endpoints (URL + optional default headers) referenced by setup http steps via base
  • Allowed executables: command names permitted in setup command steps

This module is for preparing test data or backend state, not for model routing. For how a case declares setup steps, see Core Concepts · TestCase.

Runtime

Shared defaults for execution loops across start, run case, run plan, and verify change. The Settings UI groups them into:

  • Generation: model output scale and sampling style
  • Execution Loop: step / time limits, polling cadence, and settle timing
  • Vision: screenshot sizing and perception thresholds

Most users can keep the defaults. Tune these when runs are too short, too slow, or when vision input needs different image sizing.

  1. Configure Global Provider and save.
  2. Run a simple plan or case to confirm the model path works.
  3. Add Proxy or iOS Bridge only if your network or device path requires them.
  4. Add Test Environment when cases need HTTP/exec setup steps.
  5. Adjust Orchestration and Runtime after you have real run feedback.

Reference config.yaml

Copy the example below into <workspace>/.munk/config.yaml (or your active config path), then replace the placeholder secrets and endpoints with your own values.

shared:
  provider: openai_compatible
  openai_compatible:
    base_url: https://openrouter.ai/api/v1/
    model: google/gemma-4-26b-a4b-it
    timeout_sec: 60.0
    extra_headers: {}
    output_strategy: auto
    thinking: false
  gemini:
    vertexai: false
    model: gemini-2.5-flash
    base_url: https://generativelanguage.googleapis.com/
    timeout_sec: 60.0
  agents:
    runner:
      provider: openai_compatible
      openai_compatible:
        base_url: https://openrouter.ai/api/v1/
        model: google/gemma-4-26b-a4b-it
        timeout_sec: 60.0
  runtime:
    max_tokens: 16384
    temperature: 0.2
    max_steps: 30
    max_seconds: 300.0
    interval: 0.2
    settle_timeout: 6.0
    settle_mode: ratio
    settle_ocr_only: true
    settle_ratio_threshold: 0.1
    settle_delay_sec: 1.0
    max_side: 1024
    vl_max_side: 768
    icon_conf: 0.12
    runner_include_screenshot: true
  orchestration:
    max_retry_attempts: 1
    allow_retry_on_failed: true
    allow_retry_on_inconclusive: true
    escalate_after_max_attempts: false
  test_env:
    bases:
      test_backend:
        url: http://127.0.0.1:8080
        headers:
          Accept: application/json
    allowed_exec:
    - echo
    - python
local:
  openai_compatible:
    api_key: sk-or-v1-your-api-key
  gemini:
    api_key: your-gemini-api-key
  agents:
    runner:
      openai_compatible:
        api_key: sk-or-v1-your-api-key
  proxy:
    enabled: false
    url: http://127.0.0.1:7890
  ios_bridge:
    sudo_enabled: false
    sudo_password: your-local-sudo-password

Notes:

  • Keep only the sections you need. A minimal working file can be shared.provider plus one provider section under shared, with the matching api_key under local.
  • The agents block is optional. Delete it if every role should inherit the global provider.
  • Put secrets only under local. Do not commit real API keys or sudo passwords to shared repositories.
  • Flat legacy files still work for reading; prefer the layered shape above for cloud sync.