Skip to content

Forward detected AI coding-agent id into XCUITest simulator runs#2431

Draft
ctufts wants to merge 1 commit into
mainfrom
agent-tag-xcuitest-forwarding
Draft

Forward detected AI coding-agent id into XCUITest simulator runs#2431
ctufts wants to merge 1 commit into
mainfrom
agent-tag-xcuitest-forwarding

Conversation

@ctufts

@ctufts ctufts commented Jul 21, 2026

Copy link
Copy Markdown

Summary

Mapbox wants to measure how much SDK traffic across its client SDKs comes from AI coding agents (Claude Code, Codex, Cursor, and similar tools), by appending an agent/<id> token to the outbound User-Agent header whenever an agent can be detected. This repo doesn't build that header itself - that happens in a shared native HTTP layer this SDK depends on - but verifying the tagging end-to-end on iOS means a detected agent id needs a way to reach the running app process during a test run. This PR adds that path, for XCUITest simulator runs only.

  • scripts/agent-detect.sh detects the AI coding agent (if any) driving the host shell about to invoke xcodebuild, using the same allowlist, precedence order, and sanitized fallback rules as this project's sibling SDK implementations of the same detection logic (kept in sync across those SDKs).
  • scripts/run-uitests-with-agent-tag.sh runs the ExamplesUITests scheme on a simulator, forwarding the detected id to xcodebuild test via the TEST_RUNNER_MAPBOX_AGENT environment variable, which Xcode automatically strips and exposes as MAPBOX_AGENT to the XCUITest runner process.
  • ExamplesUITests.swift and MapLoadingUITest.swift read MAPBOX_AGENT from ProcessInfo.processInfo.environment and copy it into XCUIApplication().launchEnvironment before .launch(), since the app under test doesn't automatically inherit the test runner's environment.

A normal app run is unaffected, and a UI test run started any other way (without TEST_RUNNER_MAPBOX_AGENT set) behaves exactly as before - nothing gets added to launchEnvironment. No production SDK code is touched. Actually seeing the agent/<id> token show up server-side still depends on a separate, not-yet-merged change to the shared native HTTP layer's User-Agent construction; this PR just makes sure a detected id can reach the app process once that lands.

I ran scripts/agent-detect.sh directly against a battery of scenarios: each supported agent's indicator var, precedence when multiple are set at once, the VTCODE/TERM_PROGRAM exact-match cases, safe and unsafe fallback-variable handling (including a header-injection-shaped value), and the "nothing detected" case. All produced the expected output, and that same battery runs as a self-check via scripts/agent-detect.sh --self-test. Both scripts pass shellcheck and bash -n. I also parsed and typechecked the modified Swift files standalone with swiftc against the iOS Simulator SDK; the only errors reported are pre-existing and unrelated to this change (an XCTAssertTrue/XCTAssertEqual macro-import limitation specific to typechecking a single file outside the full Xcode project build).

What I couldn't verify here: a full xcodebuild test simulator run of ExamplesUITests, since it needs Carthage/CocoaPods binary dependency downloads gated on a Mapbox secret token and ~/.netrc, plus carthage/pod tooling not available in this environment. A reviewer with a normal local dev setup (per DEVELOPING.md) should run scripts/run-uitests-with-agent-tag.sh to confirm the app process actually receives MAPBOX_AGENT, and later, once the native HTTP layer change lands, that outbound User-Agent headers carry the agent/<id> token. Real-device and CI runs weren't exercised either.

Test plan

  • Run scripts/run-uitests-with-agent-tag.sh from a machine with an AI coding agent's environment indicator set (e.g. inside a Claude Code session) and confirm the launched app's process environment contains MAPBOX_AGENT.
  • Run the same script from a plain shell (no agent indicator set) and confirm no TEST_RUNNER_MAPBOX_AGENT argument is passed and the run behaves as before.
  • Once the corresponding native HTTP layer change lands, re-run the positive case and confirm outbound Mapbox request User-Agent headers include agent/<id>.

Pull request checklist:

  • Describe the changes in this PR, especially public API changes. (No public API changes - this is internal test tooling only.)
  • Include before/after visuals or gifs if this PR includes visual changes. N/A, no visual changes.
  • Write tests for all new functionality. Put tests in correct Test Plan (Unit, Integration, All)
    • scripts/agent-detect.sh --self-test covers the detection logic itself; the XCUITest-side plumbing is exercised by the manual Test plan below since it depends on a real simulator run.
  • Add documentation comments for any added or updated public APIs. N/A, no public APIs added or changed.
  • Add any new public, top-level symbols to the DocC custom catatlog (Sources/MapboxMaps/Documentation.docc/API Catalogs). N/A, no new public symbols.
  • Add a changelog entry to to bottom of the relevant section (typically the ## main heading near the top). Not added yet - open to adding one if maintainers want this test-tooling change logged.
  • Update the guides (internal access only), README.md, and DEVELOPING.md if their contents are impacted by these changes. Not yet updated; happy to add a short DEVELOPING.md note on the new scripts if useful.
  • If this PR is a v10.[version] release branch fix / enhancement, merge it to main first and then port to v10.[version] release branch. N/A, targeting main directly.

PRs must be submitted under the terms of our Contributor License Agreement CLA.

Adds test-tooling to verify that outbound Mapbox requests made during an
XCUITest simulator run carry an `agent/<id>` User-Agent token when the host
machine running xcodebuild is driven by a supported AI coding agent
(Claude Code, Codex, Cursor, etc.):

- scripts/agent-detect.sh detects the agent from the host shell's
  environment, using the same allowlist/precedence/sanitized-fallback rules
  as this project's sibling SDK implementations.
- scripts/run-uitests-with-agent-tag.sh runs the ExamplesUITests scheme on
  a simulator, forwarding the detected id to xcodebuild via
  TEST_RUNNER_MAPBOX_AGENT.
- ExamplesUITests.swift and MapLoadingUITest.swift copy MAPBOX_AGENT from
  the test runner's environment into XCUIApplication's launchEnvironment
  before launch, since the app under test does not inherit it automatically.

None of this affects a normal app run or a UI test run started any other
way: nothing is forwarded unless TEST_RUNNER_MAPBOX_AGENT was explicitly set
on the xcodebuild invocation.
@ctufts ctufts added the locationai-team-skills Opened via the location-ai PR creation skill label Jul 21, 2026
@CLAassistant

CLAassistant commented Jul 21, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

locationai-team-skills Opened via the location-ai PR creation skill

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants