Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 12 additions & 1 deletion apps/docs/content/docs/en/workflows/blocks/pi.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Review Code uses a disposable sandbox for the repository, but the Pi harness and
- Requires sandbox execution. The provider key stays in Sim, so hosted keys and BYOK are both supported.
- Needs a **GitHub token** that can clone the repo and submit reviews (see [Setup](#setup-cloud-code-review)).
- Needs the **Pull Request Number** to review.
- Does not load skills or memory, and never exposes shell, write, edit, or arbitrary network tools to the reviewer.
- Does not load skills or memory, and never exposes shell, write, or edit tools to the reviewer. Its only network access is [Internet Search](#internet-search), and only when you select a provider.
- Rechecks the PR immediately before submission and pins the review to the exact checked-out head commit.
- The deliverable is a **submitted review** — read `reviewUrl` and `commentsPosted`.

Expand All @@ -63,6 +63,16 @@ The model that drives the agent. Defaults to `claude-sonnet-4-6`. The dropdown c

Your key for the chosen provider. On hosted Sim it is optional for Local Dev and Review Code runs (a hosted key is used and metered to your workspace). **Create PR requires your own key** because its model client runs in the sandbox. When the provider supports workspace BYOK, you can store the key in **Settings → BYOK** instead of entering it on the block.

### Internet Search

Off by default. Pick a provider — **Exa**, **Serper**, **Parallel AI**, or **Firecrawl** — and the agent gains a single `web_search` tool that returns a handful of results, each with a title, URL, snippet, and (where the provider reports one) a publication date. It works the same way in all three modes, and it is the agent's only network access in Review Code. The tool accepts at most 20 calls per run, which bounds accidental tool loops and the quota one run can consume.

Search always uses **your own key** for the selected provider, never a Sim-hosted one, because Create PR places the key inside the coding sandbox. Enter it in **Search API Key** or store it in **Settings → BYOK**; the run fails with a setup error before any sandbox is created when neither is present. Switching providers clears the key field in the editor, so re-enter the key that belongs to the provider you picked.

Results are third-party data. The agent is instructed to treat them as quoted evidence and never to follow instructions found inside them — the same posture Pi takes toward repository contents.

Traffic goes both ways: the agent writes its own queries after reading the repository, so leave search on **None** in Review Code when the pull request comes from an untrusted fork of a private repo. Injected instructions in a diff could otherwise put repository text into a query sent to the provider.

### Repository (Create PR / Review Code)

- **Repository Owner / Repository Name** — the GitHub repo (for example `your-org` / `your-repo`).
Expand Down Expand Up @@ -176,6 +186,7 @@ Enable sandbox execution as for Create PR. BYOK is optional because the model cr
{ question: "Why does Local Dev need a public hostname?", answer: "Sim connects over raw SSH and blocks localhost, LAN, and private/reserved addresses for safety. Expose the machine with a TCP tunnel such as `ngrok tcp 22` and use the tunnel's host and port. Tailscale's private 100.x addresses won't work for the same reason." },
{ question: "What GitHub permissions does Create PR need?", answer: "A token that can clone, push, and open a PR. With a fine-grained token: select the repo and grant Contents: Read and write plus Pull requests: Read and write. With a classic token: the repo scope. For organization repos, the token must be SSO-authorized." },
{ question: "What GitHub permissions does Review Code need?", answer: "A token that can clone the repo and submit a review. With a fine-grained token: Contents: Read plus Pull requests: Read and write. Push permission is not required. With a classic token: the repo scope. For organization repos, the token must be SSO-authorized." },
{ question: "Can the agent search the web?", answer: "Only if you pick a provider under Internet Search — Exa, Serper, Parallel AI, or Firecrawl. That adds one web_search tool in every mode, backed by your own key for that provider (on the block or in Settings → BYOK); Sim never supplies a search key. Leave it on None and the agent has no search tool at all." },
{ question: "Can I give it Gmail, Slack, or other integrations?", answer: "Yes, in Local Dev via the Tools field. Selected Sim tools run through Sim with your connected credentials, the same as the Agent block, so the agent can act beyond the repo while it codes. MCP and custom tools aren't supported yet." },
{ question: "Where do the changes or feedback go?", answer: "In Create PR, to a new branch and a pull request (read prUrl and branch). In Review Code, to a submitted GitHub review on the existing PR (read reviewUrl and commentsPosted). In Local Dev, the files are edited in place on the target machine — review them with git there. Create PR and Local Dev also return changedFiles and a diff." },
{ question: "What happens when memory or context gets large?", answer: "For Create PR and Local Dev, Sim trims memory before the run based on the memory type, and Pi compacts older turns as needed. Review Code does not load or save memory because a malicious PR could otherwise expose or poison prior context." },
Expand Down
67 changes: 67 additions & 0 deletions apps/sim/blocks/blocks/pi.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
/**
* @vitest-environment node
*/
import { describe, expect, it, vi } from 'vitest'

// The registry lives server-side (`keys.ts` reaches BYOK, which reaches the database) and the block
// deliberately does not import it — no block imports from `@/executor`. This test is what ties the
// two copies together, so adding a provider to one and not the other fails here.
vi.mock('@/lib/api-key/byok', () => ({ getBYOKKey: vi.fn(), getApiKeyWithBYOK: vi.fn() }))

import { evaluateSubBlockCondition } from '@/lib/workflows/subblocks/visibility'
import { PiBlock } from '@/blocks/blocks/pi'
import { PI_SEARCH_PROVIDERS } from '@/executor/handlers/pi/keys'

const searchProviderField = PiBlock.subBlocks.find((subBlock) => subBlock.id === 'searchProvider')
const searchApiKeyField = PiBlock.subBlocks.find((subBlock) => subBlock.id === 'searchApiKey')

function searchKeyVisible(values: Record<string, unknown>): boolean {
return evaluateSubBlockCondition(searchApiKeyField?.condition, values)
}

describe('Pi block search fields', () => {
it('offers None plus exactly the providers the resolver knows, defaulting to None', () => {
expect(searchProviderField?.type).toBe('dropdown')
expect(searchProviderField?.defaultValue).toBe('none')

const options = searchProviderField?.options as { id: string; label: string }[]
expect(options.map(({ id }) => id)).toEqual(['none', ...Object.keys(PI_SEARCH_PROVIDERS)])
// Labels too: a mismatch here means the dropdown names a provider differently from the setup
// error the run fails with.
expect(options.slice(1).map(({ label }) => label)).toEqual(
Object.values(PI_SEARCH_PROVIDERS).map(({ label }) => label)
)
})

// The same handling this block already gives githubToken, password, and privateKey.
it('keeps the search key out of connections, references, and plain text', () => {
expect(searchApiKeyField?.password).toBe(true)
expect(searchApiKeyField?.paramVisibility).toBe('user-only')
expect(searchApiKeyField?.connectionDroppable).toBe(false)
})

it('declares the key as dependent on the provider, which is what clears it in the editor', () => {
expect(searchApiKeyField?.dependsOn).toEqual(['searchProvider'])
})

it('shows the key field only once a provider is selected', () => {
expect(searchKeyVisible({ searchProvider: 'exa' })).toBe(true)
expect(searchKeyVisible({ searchProvider: 'firecrawl' })).toBe(true)
expect(searchKeyVisible({ searchProvider: 'none' })).toBe(false)
expect(searchKeyVisible({ searchProvider: '' })).toBe(false)
})

// A Pi block saved before this field existed has no stored value, and the serializer does not
// inject subBlock defaults — so `undefined` has to behave like None here too.
it('hides the key field on blocks saved before the field existed', () => {
expect(searchKeyVisible({})).toBe(false)
expect(searchKeyVisible({ searchProvider: undefined })).toBe(false)
})

// `inputs` is the block's type map, not the delivery mechanism — the handler reads resolved
// params — but an undeclared input is a convention break the next block author would copy.
it('declares both fields in the block input map', () => {
expect(PiBlock.inputs.searchProvider).toBeDefined()
expect(PiBlock.inputs.searchApiKey).toBeDefined()
})
})
57 changes: 56 additions & 1 deletion apps/sim/blocks/blocks/pi.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,18 +53,45 @@ const AUTHORING_MODES: { field: 'mode'; value: Array<'cloud' | 'local'> } = {
}
const MEMORY_TYPES = ['conversation', 'sliding_window', 'sliding_window_tokens']

const SEARCH_PROVIDER_OPTIONS = [
{ label: 'None', id: 'none' },
{ label: 'Exa', id: 'exa' },
{ label: 'Serper', id: 'serper' },
{ label: 'Parallel AI', id: 'parallel' },
{ label: 'Firecrawl', id: 'firecrawl' },
]

/**
* Mirrors `getApiKeyCondition()` for the model key: the search key's visibility is computed rather
* than declarative. The never-matching sentinel is how `buildModelVisibilityCondition` hides the
* model key when nothing is selected, and it is what keeps the field hidden on a block saved before
* this field existed — such a block has no stored `searchProvider`, and the declarative negative
* form would show the field, because a scalar `not` condition evaluates `undefined !== 'none'` as
* true.
*/
function getSearchApiKeyCondition() {
return (values?: Record<string, unknown>) => {
const provider = typeof values?.searchProvider === 'string' ? values.searchProvider : ''
if (!provider || provider === 'none') {
return { field: 'searchProvider', value: '__no_search_provider__' }
}
return { field: 'searchProvider', value: provider }
}
}

export const PiBlock: BlockConfig<PiResponse> = {
type: 'pi',
name: 'Pi Coding Agent',
description: 'Run an autonomous coding agent on a repo',
authMode: AuthMode.ApiKey,
longDescription:
'The Pi Coding Agent runs the Pi harness against a real repository. Create PR spins up an isolated sandbox, clones a GitHub repo, edits with native shell + git, and opens a pull request. Review Code checks out a pinned PR snapshot with read-only tools and posts a structured review with optional inline comments. Local Dev edits files on your own machine over SSH. Create PR and Local Dev can reuse skills and multi-turn memory; Review Code runs without either because PR contents are untrusted.',
'The Pi Coding Agent runs the Pi harness against a real repository. Create PR spins up an isolated sandbox, clones a GitHub repo, edits with native shell + git, and opens a pull request. Review Code checks out a pinned PR snapshot with read-only tools and posts a structured review with optional inline comments. Local Dev edits files on your own machine over SSH. Create PR and Local Dev can reuse skills and multi-turn memory; Review Code runs without either because PR contents are untrusted. Any mode can optionally get one web_search tool backed by your own Exa, Serper, Parallel AI, or Firecrawl key; the agent writes its own queries, so repository content may reach the provider, and results are untrusted third-party data.',
bestPractices: `
- Use Create PR for hands-off changes against a GitHub repo where a reviewable PR is the deliverable.
- Use Review Code to analyze an existing PR and leave summary + inline review comments.
- Use Local Dev to edit a repo on your own machine; expose the machine on a public hostname/tunnel so Sim can reach it over SSH.
- Create PR requires your own provider API key because the model runs in the sandbox. Review Code keeps the model key in Sim and can use either BYOK or a hosted key.
- Internet Search is off by default and always needs your own key for the selected provider, from the block field or Settings > BYOK. Leave it on None unless the task genuinely needs external information.
`,
category: 'blocks',
integrationType: IntegrationType.AI,
Expand Down Expand Up @@ -122,6 +149,29 @@ export const PiBlock: BlockConfig<PiResponse> = {

...getProviderCredentialSubBlocks(),

{
id: 'searchProvider',
title: 'Internet Search',
type: 'dropdown',
defaultValue: 'none',
options: SEARCH_PROVIDER_OPTIONS,
tooltip:
'Gives the agent a single web_search tool backed by the selected provider. Search always uses your own key for that provider, never a Sim-hosted one, because Create PR places the key inside the coding sandbox.',
},
{
id: 'searchApiKey',
title: 'Search API Key',
type: 'short-input',
password: true,
paramVisibility: 'user-only',
connectionDroppable: false,
placeholder: 'Falls back to the key stored in Settings > BYOK',
tooltip:
'Key for the selected search provider. Switching providers clears this field, so re-enter the key for the provider you picked.',
condition: getSearchApiKeyCondition(),
dependsOn: ['searchProvider'],
},

{
id: 'owner',
title: 'Repository Owner',
Expand Down Expand Up @@ -434,6 +484,11 @@ export const PiBlock: BlockConfig<PiResponse> = {
conversationId: { type: 'string', description: 'Conversation ID for memory' },
slidingWindowSize: { type: 'string', description: 'Number of messages for sliding window' },
slidingWindowTokens: { type: 'string', description: 'Max tokens for token-based window' },
searchProvider: {
type: 'string',
description: 'Web search provider for the agent: none, exa, serper, parallel, or firecrawl',
},
searchApiKey: { type: 'string', description: 'API key for the selected search provider' },
...PROVIDER_CREDENTIAL_INPUTS,
},
outputs: {
Expand Down
22 changes: 22 additions & 0 deletions apps/sim/executor/handlers/pi/backend.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import type { TSchema } from 'typebox'
import type { SSHConnectionConfig } from '@/app/api/tools/ssh/utils'
import type { Message } from '@/executor/handlers/agent/types'
import type { PiEvent, PiRunTotals } from '@/executor/handlers/pi/events'
import type { PiSearchKeySource, PiSearchProvider } from '@/executor/handlers/pi/keys'
import type { PiSupportedProvider } from '@/providers/pi-provider-configs'

/** A conversation message seeded into the Pi run (subset of the Agent block's message). */
Expand All @@ -31,6 +32,7 @@ export type PiSshConnection = Pick<
/** Result of invoking a tool Pi called. */
export interface PiToolResult {
text: string
/** Reported to Pi by throwing `text` from the converted tool; see `toPiTool` for why. */
isError: boolean
}

Expand All @@ -43,9 +45,28 @@ export interface PiToolSpec {
name: string
description: string
parameters: TSchema
/**
* Guideline bullets Pi folds into the system prompt while the tool is active. This is the
* trusted channel: a `description` travels in the provider request's tool-definition array, so
* guidance placed there carries the same trust level as the payload it describes. Dropped in
* Review Code, which supplies a sealed `customPrompt` instead.
*/
promptGuidelines?: string[]
execute: (args: Record<string, unknown>) => Promise<PiToolResult>
}

/** Optional web search for a Pi run, resolved by the handler before mode dispatch. */
export interface PiSearchConfig {
provider: PiSearchProvider
apiKey: string
keySource: PiSearchKeySource
/**
* Host-side tool for the two SDK modes. Absent for `cloud`, which has no host in the loop and
* registers a sandbox extension instead, so a spec built there could never execute.
*/
tool?: PiToolSpec
}

interface PiRunBaseParams {
/** Sim's catalog ID, retained for billing and output. */
model: string
Expand All @@ -56,6 +77,7 @@ interface PiRunBaseParams {
isBYOK: boolean
task: string
thinkingLevel?: string
search?: PiSearchConfig
}

interface PiContextualRunParams extends PiRunBaseParams {
Expand Down
Loading
Loading