Skip to content

LearnCloud

Cloud evidence package and operator log

Brief

Every claim in a cloud report rests on a command we ran and the output it returned, so the log of those commands and the files that hold the outputs are the engagement's evidence, not an afterthought. This page sets the log line, the capture rule, the folder layout and the manifest, so a client or a grader can verify a claim in one command and find the same call in their own logs.

Track
Cloud
Stage
Report
Reviewed
On this page
  1. Check first
  2. What goes in one operator log line?
  3. How do I capture a command and its output without a secret value?
  4. How is the evidence package laid out?
  5. How do I hash the package and check it in one command?
  6. How does each finding point at its evidence?
  7. What does the grader check first, and what happens to the package afterwards?
  8. The defender sees
  9. Done when
  10. Not covered
  11. Tools
  12. Where this page fits
  13. Reviewed

All commands

Situation
A cloud test about to start, the test identities the client issued, the scope letter naming the accounts, subscriptions and the window in UTC, and an encrypted volume on our own machine for the package. Everything here is a convention and a few shell lines around the AWS and Azure CLIs; the only calls run against a cloud are checkip.amazonaws.com, sts get-caller-identity, az account show and one bucket policy read, with az storage account list shown as a log example, and nothing is written to a client system.
Goal
One evidence package: an operator log, a captured file for every command, a sha256 manifest, and a claims index that points each sentence of each finding at a path and a log line.
Outputs
Representative examples on placeholder hosts, trimmed to the lines that matter. Not captured from a public fixture yet.

Check first

  • The window in UTC from the scope letter.
  • The test identities by role ARN and user principal name.
  • Our source address for the day.
  • The redaction rule in writing: names, ids, counts and hashes go in; keys, tokens, signatures and secret values never do.
  • The retention date for the package and the retest.

Steps

What goes in one operator log line?

date -u +%Y-%m-%dT%H:%M:%SZ
curl -s https://checkip.amazonaws.com
aws sts get-caller-identity --query '[Account,Arn]' --output text
az account show --query '[tenantId,id,user.name]' --output tsv

Expected output

2026-10-29T13:02:11Z
203.0.113.24
111122223333	arn:aws:sts::111122223333:assumed-role/atarus-review/review-20261029   # <- role and session name, as CloudTrail will print them
8f0c2b1e-4d3a-4e6f-9b7c-1a2d3e4f5a6b	4a5b6c7d-8e9f-4a1b-2c3d-4e5f6a7b8c9d	review@example.onmicrosoft.com

The log is one tab-separated file, one line per call; two lines of it:

utc	source	identity	key_or_app	call	result	evidence
2026-10-29T13:02:11Z	203.0.113.24	arn:aws:sts::111122223333:assumed-role/atarus-review/review-20261029	ASIAIOSFODNN7EXAMPLE	aws s3api get-bucket-policy --bucket example-exports --query Policy --output text	0	evidence/aws/2026-10-29T1302Z-s3-get-bucket-policy-example-exports.txt
2026-10-29T13:04:40Z	203.0.113.24	review@example.onmicrosoft.com	04b07795-8ddb-461a-bbee-02f9e1bf7b46	az storage account list --query [].name --output tsv	0	evidence/azure/2026-10-29T1304Z-storage-account-list.txt

Time is UTC to the second, as CloudTrail's eventTime and Entra's createdDateTime print it. The fourth column is what the client filters on: the role session's temporary access key id in AWS, and in Azure the client application id, fixed for the Azure CLI. The call is the exact text typed, the result the exit code. Refused calls are logged too; a refusal shows a control works.

Negative result. A call with no line is the failure, and ours to fix the same hour.

Fix. Issue test identities with a session name or user principal name nobody else uses, so one filter separates our lines.

How do I capture a command and its output without a secret value?

ev() {  # ev <cloud> <label> <command...>; SRC, ID and KEY are set once per session
  local t n f rc; t=$(date -u +%Y-%m-%dT%H:%M:%SZ); n=$(date -u +%Y-%m-%dT%H%MZ)
  f="evidence/$1/$n-$2.txt"; shift 2
  printf '# %s %s %s\n# %s\n' "$t" "$SRC" "$ID" "$*" > "$f"
  "$@" >> "$f" 2>&1; rc=$?
  printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\n' "$t" "$SRC" "$ID" "$KEY" "$*" "$rc" "$f" >> operator-log.tsv
  cat "$f"
}
ev aws s3-get-bucket-policy-example-exports aws s3api get-bucket-policy --bucket example-exports --query Policy --output text

Expected output

# 2026-10-29T13:02:11Z 203.0.113.24 arn:aws:sts::111122223333:assumed-role/atarus-review/review-20261029
# aws s3api get-bucket-policy --bucket example-exports --query Policy --output text
{"Version":"2012-10-17","Statement":[{"Sid":"PartnerRead","Effect":"Allow","Principal":{"AWS":"arn:aws:iam::444455556666:root"},"Action":["s3:GetObject","s3:ListBucket"],"Resource":["arn:aws:s3:::example-exports","arn:aws:s3:::example-exports/*"]}]}   # <- the policy as the API returned it, which is what the finding quotes

The function writes two header lines, the output and the log line. Commands that print a value are not run: no secretsmanager get-secret-value, ssm get-parameter --with-decryption, kms decrypt, az keyvault secret show, az storage account keys list or az account get-access-token, and no --debug, which prints the signed Authorization header. Where a value cannot be avoided, such as the marker file on the impact page, its shasum -a 256 hash goes in the file, never the content.

Negative result. Every file in evidence/ has its two header lines and exactly one log line; a mismatch is fixed the same hour or recorded as lost.

Fix. A session policy or permissions boundary on the test role that denies secretsmanager:GetSecretValue, kms:Decrypt and ssm:GetParameter, so the redaction rule is enforced by the account, not habit.

How is the evidence package laid out?

ENG-2026-10-example/
  README.txt            scope letter reference, accounts and subscriptions, window in UTC, identities, source address, retention date
  operator-log.tsv      one line per call, header first
  evidence/
    aws/                2026-10-29T1302Z-s3-get-bucket-policy-example-exports.txt ...
    azure/              2026-10-29T1304Z-storage-account-list.txt ...
  findings/
    F-01-s3-partner-read/claims.txt
    F-02-sqs-open-send/claims.txt
  MANIFEST.sha256
find ENG-2026-10-example -type f | wc -l
ls ENG-2026-10-example/evidence/aws | head -3

Expected output

     218
2026-10-29T1302Z-s3-get-bucket-policy-example-exports.txt
2026-10-29T1302Z-sts-get-caller-identity.txt
2026-10-29T1305Z-accessanalyzer-list-findings-v2-us-east-1.txt   # <- the name says when, which service, which call and which target

One folder per engagement, outputs by cloud, findings by id, and nothing else: no scratch files, client data or screenshots showing a value. The folder lives on an encrypted volume on our machine and nowhere the client has not approved in writing.

Negative result. find and the log agree once the README, log, claims files and manifest are subtracted, and nothing under evidence/ is a client object.

Fix. Agree the layout in the rules of engagement, so the client knows what they will receive.

How do I hash the package and check it in one command?

cd ENG-2026-10-example
find README.txt operator-log.tsv evidence findings -type f -print0 | sort -z | xargs -0 shasum -a 256 > MANIFEST.sha256
shasum -c --quiet MANIFEST.sha256 && echo "manifest ok: $(wc -l < MANIFEST.sha256) files"
shasum -a 256 MANIFEST.sha256

Expected output

manifest ok:      217 files
7c9e1b2d4f6a8c0e3b5d7f9a1c3e5b7d9f1a3c5e7b9d1f3a5c7e9b1d3f5a7c9e  MANIFEST.sha256   # <- the one hash that goes in the report and the delivery message

The manifest hash is one line in the report and the delivery message, so anyone holding the package can confirm it is the one delivered. Sorting makes the manifest identical when rebuilt. It is written once, when the test closes; later additions go in a second, dated manifest.

Negative result. shasum: WARNING: 1 computed checksum did NOT match names the changed file; the package is not delivered until the change is explained in the README.

Fix. Keep the manifest hash with the report in the ticket system, so the retest and any dispute start from the same files.

How does each finding point at its evidence?

F-01  Account 444455556666 can list and read every object in example-exports (Confidential, 48,213 objects)
  claim 1  the bucket policy admits the account   evidence/aws/2026-10-29T1302Z-s3-get-bucket-policy-example-exports.txt   log 14   GetBucketPolicy 3c1e7a52-9d4b-4f8e-a2c6-0b5d71e9f384
  claim 2  the bucket is tagged Confidential      evidence/aws/2026-10-29T1306Z-s3-get-bucket-tagging-example-exports.txt   log 19   GetBucketTagging 8a2f40c3-61de-4b7a-95e0-c4d2b17e6a09
  claim 3  it holds 48,213 objects                evidence/aws/2026-10-29T1307Z-cloudwatch-numberofobjects-example-exports.txt   log 20   GetMetricStatistics e41c7b90-3f2a-4d68-8b5e-9a0c1d2e7f46
grep -n 'get-bucket-policy --bucket example-exports' operator-log.tsv | cut -f1,5

Expected output

14:2026-10-29T13:02:11Z	aws s3api get-bucket-policy --bucket example-exports --query Policy --output text   # <- the log line the finding cites

Every factual sentence in a finding has a row: the evidence file, the log line, and the event id from the client's own logs once matched. A claim with no row is rewritten or dropped. The index lives in the finding's claims.txt and in the report appendix; the resource policy review is the cleanest example: one policy read, one event, per claim.

Negative result. Every claim resolves to a manifest path and a log line inside the window.

Fix. Ask for this index in every third-party report, and send back findings that cannot name the call behind them.

What does the grader check first, and what happens to the package afterwards?

shasum -c --quiet MANIFEST.sha256 && echo manifest ok
awk -F'\t' 'NR>1 && ($1 < "2026-10-29T13:00:00Z" || $1 > "2026-10-30T17:00:00Z")' operator-log.tsv | wc -l
grep -rEl 'aws_secret_access_key|SessionToken|"SecretString"|sig=|Bearer ey' evidence findings || echo "no secret shapes"
awk -F'\t' 'NR>1 {print $7}' operator-log.tsv | while read -r f; do [ -f "$f" ] || echo "missing $f"; done

Expected output

manifest ok
       0   # <- no line outside the window
no secret shapes

A grader runs these four before reading a word: the manifest verifies, no log line sits outside the window, no file holds the shape of a key, token, signature or secret string, and every evidence path exists. Then one finding end to end, claim to file to log line to their own event.

Retention is a date, not a policy name, written in the README and the report. On that date the folder is removed from the encrypted volume; the manifest and its hash are kept, so a later question can still be answered by file name and hash.

Negative result. manifest ok, a zero, no secret shapes and no missing line; for retention, an engagement record line saying the package was deleted.

Fix. Name the retention date in the contract, and ask for the manifest hash before the package, so the two can be compared on arrival.

The defender sees

In AWS, the fourth column is the AccessKeyId lookup attribute, the first is eventTime to the second, and the call names the eventName, so a CloudTrail lookup on our key id gives every line its event id, a few seconds off our clock. In Azure, the identity column is the user principal name and the fourth is the Azure CLI's application id, so the sign-in logs filter on both; control-plane writes appear in the activity log by caller and correlation id through az monitor activity-log list, and reads do not, so a read-only review is matched through the sign-in logs and, where the tenant exports them, the Microsoft Graph activity logs and the resource diagnostic logs, as the Entra page shows. In the debrief, a line with an event id was recorded; a line without one is a data event nobody logs, a region the trail does not cover, or a call that never left our machine. The CloudTrail and GuardDuty page has the step by step, possible only because the log was kept from the first call, in UTC, with the key id.

Done when

Every command of the test has a log line and a captured file with no secret values, the manifest verifies with one command, every finding names its evidence paths and log lines, and the retention date is written in the package and the report.

Not covered

Proving reach (see cloud impact without damage), writing the finding itself (see the cloud finding write-up), screen recordings and browser evidence, and chain of custody for a legal matter; the outputs shown are representative examples on placeholder accounts, not captured runs.