CLI, SDK and IAM policies — DVA-C02 Mastery Guide
- Sources: course chapter 07 (AWS CLI, SDK, IAM roles and policies). The course serves as the syllabus only; all wording here is original. IMDS mechanics (token flow, hop limit, enforcement) live in
notes/ec2-mastery-guide.md§2 and instance profiles in §1; this guide adds only their CLI and SDK angles.- Written 2026-10-12; values checked against AWS docs on 2026-10-12. The marker
⚠️ Changed: <current>. Older material may say <old>.flags a value or behaviour that differs from the course or older practice tests: learn the current value.⚠️ volatileflags a value AWS has announced will change soon. Every change is listed in Changed vs older material.- Tier 1 (highest exam weight), bank target 30 questions. Domains: Security (primary), Development with AWS Services.
- How to use it:
/explain <topic>teaches from these sections (concept → exam facts → developer angle → traps → 3 check questions);/quiz cli-sdk-policiesasks bank questions whosesource_notelinks point at the## N.headings below;/drill cli-sdk-policiesfires short recall prompts fromnotes/cli-sdk-policies-cheat-sheet.md. Suggested loop: read a section and answer its practice questions before opening the answers →/explainwhat is still unclear →/quiz→/drill.
How CLI, SDK and IAM policies appears on DVA-C02
DVA-C02 assumes you write code and scripts that call AWS. This module covers the plumbing every one of those calls passes through:
- Identity on the wire: which credentials a CLI command or SDK client actually picks up, why it is sometimes not the one you expected, and how to keep long-term keys out of the picture.
- MFA and temporary credentials from the command line: session tokens, role profiles, MFA condition keys.
- Robust calls: Region and endpoint resolution, pagination, waiters, retries with exponential backoff and jitter, quotas and throttling.
- Authentication and authorization mechanics: SigV4 signing (header vs presigned URL), dry runs, the IAM policy simulator, decoding denied requests.
Typical question shapes: "the role is attached but the app still acts as an IAM user"; an error code to classify (ThrottlingException vs AccessDenied vs SignatureDoesNotMatch); "the CLI times out listing a bucket"; "check permissions without creating resources"; the MOST secure way to give a laptop, a pipeline or an on-premises server credentials.
Domain map: Security → §2, §3, §4, §5, §10, §11, §12 · Development with AWS Services → §1, §6, §7, §8, §9. Many questions are framed as troubleshooting (§3, §8, §9, §12).
Owned by other modules (cross-referenced, not repeated): IAM users, groups, policy grammar, MFA devices and access-key management, plus the policy simulator, IAM Access Analyzer checks and the access-denied message catalogue (see notes/iam-guide.md §6); STS AssumeRole, cross-account access, writing MFA conditions, policy evaluation logic and iam:PassRole (see notes/advanced-identity-guide.md §9–§11); Secrets Manager and Parameter Store (see notes/security-encryption-guide.md); presigned URL use cases and S3 request rates (see notes/s3-guide.md); IMDSv2 and instance profiles (see notes/ec2-mastery-guide.md §1–§2).
| # | Topic | Why it is on the exam |
|---|---|---|
| 1 | AWS CLI profiles and configuration files | Files, profiles and setting precedence decide which account, Region and endpoint a command hits |
| 2 | MFA with the CLI and session tokens | Long-term keys carry no MFA; get-session-token, mfa_serial and MFA condition keys |
| 3 | CLI credential provider chain | First match wins: why a stray key beats the instance role |
| 4 | SDK default credential provider chains | Same idea in code: Java 2.x and boto3 order, container, Lambda and IMDS providers |
| 5 | Credentials best practices | Roles on AWS, short-term credentials elsewhere; the MOST secure option in every scenario |
| 6 | Using the AWS SDKs in application code | Region, paginators, waiters, errors, endpoints, current SDK versions |
| 7 | CLI output filtering and pagination | --page-size vs --max-items, server-side filters vs --query |
| 8 | Service quotas and API rate limits | Quota vs throttle, Service Quotas, EC2 token buckets, what to do when limits bite |
| 9 | Retries and exponential backoff with jitter | What to retry, SDK retry modes and settings, idempotency |
| 10 | Signing requests with SigV4 | Header vs query-string signing, temporary credentials, clock skew, SigV4a |
| 11 | Testing permissions with dry run and the policy simulator | DryRunOperation vs UnauthorizedOperation; simulating drafts and principals |
| 12 | Decoding authorization failures | Reading access-denied messages, decode-authorization-message, credential vs permission errors |
1. AWS CLI profiles and configuration files
Concept
- The AWS CLI is an open-source command-line front end to the public AWS APIs: most
aws <service> <command>invocations map to one API operation, while custom commands such asaws s3 synccombine several. It is built on botocore, the Python core library that boto3 also uses, so signing (§10), retries (§9) and pagination (§7) are handled for you. - Configuration lives in two plaintext INI files under
~/.aws/(Windows:%UserProfile%\.aws\):credentialsholds secrets,configholds everything else (Region, output, role, SSO, console-login and retry settings). Both files are split into named profiles;defaultapplies when you name none. - Profiles let one machine work with several accounts, roles and Regions: pick one per command with
--profileor per shell withAWS_PROFILE. - ⚠️ Changed: use AWS CLI version 2. Version 1 entered maintenance mode on 2026-07-15 (critical bug and security fixes only, no new services, APIs or Regions) and reaches end of support on 2027-07-15. Older material may say v1 (installed with
pip) is the current CLI.
Exam facts
- Section names differ per file:
configuses[default],[profile dev],[sso-session corp]and[services local-test];credentialsuses[default]and[dev], never the wordprofile. aws configure(wizard) writes the access key and secret tocredentialsand the Region and output format toconfig; add--profile devfor a named profile.- If one profile has keys in both files, the keys in
credentialswin. Move the files withAWS_CONFIG_FILEandAWS_SHARED_CREDENTIALS_FILE(environment only: there is no option or profile setting for them). - Precedence for every setting: command-line option > environment variable > profile setting. For the Region:
--region>AWS_REGION>AWS_DEFAULT_REGION> profileregion. CLI v1 reads onlyAWS_DEFAULT_REGION. - No default Region. ⚠️ Changed: AWS docs define no default Region; you must set one, and a regional command without it fails with "You must specify a region" (SDK behaviour: §6). Older material may say us-east-1 is chosen when no Region is configured.
- On EC2 the CLI can also obtain configuration such as the Region from instance metadata, unless
AWS_EC2_METADATA_DISABLED=true. - Output formats (
--output,AWS_DEFAULT_OUTPUT, profileoutput):json(default),yaml,yaml-stream,text(tab-separated, for shell tools),table(for humans),off(prints nothing; scripts check only the exit code). aws configuresubcommands:list(each value with its type and location, for exampleenv+AWS_DEFAULT_REGION),list-profiles,get <setting>,set <setting> <value>,import --csv file://…(an IAM user's access-key CSV from the console, not IAM Identity Center),export-credentials --format process(default) orenv,env-no-export,powershell,windows-cmd,fish, plusssoandsso-session(§5).- Custom endpoints (DynamoDB Local, LocalStack, tests):
--endpoint-url>AWS_ENDPOINT_URL_<SERVICE>(for exampleAWS_ENDPOINT_URL_DYNAMODB) >AWS_ENDPOINT_URL>endpoint_urlunder a[services …]section >endpoint_urlin the profile > the service default.AWS_IGNORE_CONFIGURED_ENDPOINT_URLS=trueignores the configured ones (an explicit--endpoint-urlstill applies). --debugprints the credential lookup, the canonical request and the raw response: the first tool when a command misbehaves.
Developer angle
~/.aws/config with a named profile that sends DynamoDB calls to DynamoDB Local:
[default]
region = eu-west-1
output = json
[profile dev]
region = us-east-2
output = table
services = local-ddb
[services local-ddb]
dynamodb =
endpoint_url = http://localhost:8000
~/.aws/credentials (format only: prefer short-term credentials, §5):
[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws configure --profile dev # wizard: key, secret, Region, output
aws configure list --profile dev # each value and where it came from
aws configure list-profiles
aws configure get region --profile dev
aws configure set output table --profile dev
export AWS_PROFILE=dev # default profile for this shell
aws ec2 describe-vpcs --profile prod --region eu-central-1 # per-command override
AWS_ENDPOINT_URL_DYNAMODB=http://localhost:8000 aws dynamodb list-tables
aws configure export-credentials --profile dev --format env # hand short-term credentials to another tool
Traps
- Trap: "Write
[profile dev]in~/.aws/credentials." → Reality: theprofileprefix belongs only inconfig; incredentialsthe section is[dev]. - Trap: "
AWS_DEFAULT_REGIONoverrides--region." → Reality: the command-line option always wins, thenAWS_REGION, thenAWS_DEFAULT_REGION, then the profile. - Trap: "With no Region configured, calls go to us-east-1." → Reality: current tools require a Region; set it in the profile, the environment or the command.
- Trap: "Pointing the CLI at DynamoDB Local needs a code change." → Reality:
AWS_ENDPOINT_URL_DYNAMODBor a[services]section redirects one service only. - Trap: "
aws configure importbrings in IAM Identity Center credentials." → Reality: it imports an IAM user's access-key CSV; Identity Center usesaws configure sso(§5). - Trap: "Choosing a profile with
--profilealso wins over an exported Region variable." → Reality:--profileselects the profile, but an environment variable still beats that profile'sregion; pass--regionor unset the variable.
Practice questions
-
A developer runs
aws dynamodb list-tables --profile analyticsand sees the tables of us-east-2, although theanalyticsprofile in~/.aws/configcontainsregion = eu-west-1. The developer's shell startup file exportsAWS_DEFAULT_REGION=us-east-2. What should the developer do so that the command targets eu-west-1 without editing the shell startup file?- A. Rename the section in
~/.aws/configfrom[profile analytics]to[analytics]. - B. Add
--region eu-west-1to the command. - C. Move the
regionline into the[analytics]section of~/.aws/credentials. - D. Run
aws configure set region eu-west-1 --profile analyticsagain.
- A. Rename the section in
-
A team wants the AWS CLI and boto3 test scripts on developer laptops to send DynamoDB calls to DynamoDB Local on port 8000, while every other service still reaches AWS, without changing any code. Which TWO configurations meet this requirement? (Select TWO.)
- A. Export
AWS_ENDPOINT_URL_DYNAMODB=http://localhost:8000in the test shell. - B. Export
AWS_ENDPOINT_URL=http://localhost:8000in the test shell. - C. Add a
[services local-ddb]section with adynamodbendpoint_url, referenced byservices = local-ddb. - D. Add
--endpoint-url http://localhost:8000to everyaws dynamodbcommand andendpoint_url=to every boto3 DynamoDB client. - E. Export
AWS_IGNORE_CONFIGURED_ENDPOINT_URLS=truein the test shell and keep the default endpoints for every other profile.
- A. Export
Answers
- B. A command-line option beats the environment variable, which in turn beats the profile's
region. A breaks the profile (theprofileprefix is required inconfig); C changes nothing because the environment variable still wins; D rewrites the value that is already there. - A, C. Both are service-specific endpoint settings that leave other services on AWS. B redirects every service; D works only by editing every script, which the requirement rules out; E makes the CLI and SDK ignore configured endpoints.
2. MFA with the CLI and session tokens
Concept
- Requests signed with an IAM user's long-term access keys carry no MFA information, so a policy that demands MFA denies them. From the command line you first trade the keys plus a current six-digit code for temporary credentials whose session records the MFA, then sign the protected calls with those.
- Two CLI patterns:
aws sts get-session-token: same account, same permissions as the IAM user, plus MFA context. You export or store the three returned values yourself.- Role profile with
mfa_serial: the CLI prompts for the code, callsAssumeRolewith it, caches the role credentials and refreshes them; MFA is enforced in the role's trust policy.
- This section is about getting MFA-backed credentials into the CLI. Writing the MFA conditions themselves (
aws:MultiFactorAuthPresent,aws:MultiFactorAuthAge,BoolIfExists) is covered innotes/advanced-identity-guide.md§10.
Exam facts
- Parameters:
--serial-number= hardware device serial (for exampleGAHT12345678) or virtual device ARN (arn:aws:iam::123456789012:mfa/<device-name>);--token-code= exactly 6 digits;--duration-secondsis optional. - Duration: IAM user 900–129,600 s (15 min to 36 h), default 43,200 s (12 h). Root user 900–3,600 s, default 1 h (and root shouldn't be used for this).
- Returned credentials:
AccessKeyId(ASIA…),SecretAccessKey,SessionToken,Expiration. All three values are needed; the token goes inAWS_SESSION_TOKENoraws_session_token. - Limits of GetSessionToken credentials: permissions equal the IAM user's; IAM API calls work only if MFA information was included; the only STS calls allowed are
AssumeRoleandGetCallerIdentity. CallingGetSessionTokenneeds no permission; a wrong code returns an access denied error. - MFA-protected API access works only with temporary credentials from
GetSessionTokenorAssumeRole; not with root user credentials and not with U2F security keys. CLI and API work needs a virtual or hardware TOTP device. - What the policies see:
aws:MultiFactorAuthPresentexists only in requests signed with temporary credentials (calls signed with access keys lack it, which is why MFA Deny statements useBoolIfExists);aws:MultiFactorAuthAgecounts the seconds since the MFA check. Policy patterns:notes/advanced-identity-guide.md§10. - Which pattern for which policy: an MFA condition in an identity policy works with
get-session-tokencredentials; an MFA condition in a role's trust policy is met at assume time by a role profile withmfa_serial. An MFA condition in a resource-based policy (bucket, queue, topic) needsget-session-tokencredentials: the IAM User Guide states that credentials returned byAssumeRoledon't carry the MFA information into later calls. - Role profile settings (
configfile):role_arn,source_profileorcredential_source,mfa_serial,duration_seconds(default 3,600; from 900 up to the role's maximum session duration, at most 43,200),role_session_name,external_id. The CLI promptsEnter MFA code for <serial>:and caches the credentials in~/.aws/cli/cacheuntil they expire. SDK support varies: the SDK for Java 2.x doesn't supportmfa_serialorduration_secondsin profiles. - A role profile whose
source_profileis itself a role profile is role chaining: the session is limited to 1 hour.
Developer angle
# 1) Find the MFA device identifier of the IAM user
aws iam list-mfa-devices --user-name dev-user --query 'MFADevices[].SerialNumber' --output text
# 2) Trade long-term keys + the current code for 12 h of MFA-backed credentials
read -r AK SK ST <<< "$(aws sts get-session-token \
--serial-number arn:aws:iam::123456789012:mfa/dev-user-phone \
--token-code 123456 --duration-seconds 43200 --profile dev-longterm \
--query 'Credentials.[AccessKeyId,SecretAccessKey,SessionToken]' --output text)"
# 3) Store all three values in a separate profile and use it for protected calls
aws configure set aws_access_key_id "$AK" --profile dev-mfa
aws configure set aws_secret_access_key "$SK" --profile dev-mfa
aws configure set aws_session_token "$ST" --profile dev-mfa
aws ec2 stop-instances --instance-ids i-0123456789abcdef0 --profile dev-mfa
A role profile that prompts for the code and caches the role session:
[profile ops-admin]
role_arn = arn:aws:iam::123456789012:role/OpsAdmin
source_profile = dev-longterm
mfa_serial = arn:aws:iam::123456789012:mfa/dev-user-phone
role_session_name = dev-user
duration_seconds = 3600
Running aws s3 ls --profile ops-admin prompts Enter MFA code for arn:aws:iam::123456789012:mfa/dev-user-phone: once; later commands reuse the cached session in ~/.aws/cli/cache until it expires. The matching trust-policy and identity-policy conditions are shown in notes/advanced-identity-guide.md §10.
Traps
- Trap: "Add the MFA code to each protected command." → Reality: there is no per-command MFA option; get a session first (
get-session-token) or use a role profile withmfa_serial. - Trap: "Adding
mfa_serialto the profile that holds the access keys makes its calls MFA-backed." → Reality: the CLI usesmfa_serialonly in role profiles (role_arn); for calls with the user's own permissions, runget-session-tokenand use its credentials. - Trap: "Session credentials from
get-session-tokenlast at most 1 hour." → Reality: up to 36 h for IAM users (default 12 h); the 1-hour cap applies to root sessions and role chaining. - Trap: "
get-session-tokengives access to another account." → Reality: same account, same permissions as the user; cross-account access usesAssumeRole(seenotes/advanced-identity-guide.md). - Trap: "Only the access key and the secret from the response are needed." → Reality: temporary credentials always need the session token as well.
- Trap: "A FIDO security key can supply
--token-code." → Reality: MFA-protected API access needs a TOTP code from a virtual or hardware device; U2F keys aren't supported for it.
Practice questions
-
An administrator attached a policy to the
opsgroup that allowsec2:TerminateInstancesonly whenaws:MultiFactorAuthPresentistrue. A developer in the group runsaws ec2 terminate-instanceswith the access keys stored in thedefaultprofile and receivesUnauthorizedOperation. The developer has a virtual MFA device. What should the developer do without changing the policy?- A. Run
aws sts get-caller-identityto refresh the MFA status of thedefaultprofile. - B. Run
aws iam enable-mfa-deviceagain for the virtual device, then retry the command. - C. Add the device ARN as
mfa_serialin thedefaultprofile that holds the access keys, then retry the same command. - D. Call
aws sts get-session-tokenwith the device ARN and a code, then use the returned credentials.
- A. Run
-
A developer with an IAM user and a virtual MFA device works in two accounts from a laptop. In account A (the developer's own), a bucket policy allows
s3:DeleteObjectonly whenaws:MultiFactorAuthPresentistrue. In account B, theDeployerrole's trust policy requires MFA. The developer wants to type MFA codes as rarely as possible. Which TWO configurations meet these requirements? (Select TWO.)- A. For account A, run
aws sts get-session-tokenwith--serial-numberand--token-codeonce and store the three returned values in a separate profile. - B. For account A, add
mfa_serialto the profile that holds the IAM user's access keys. - C. For account B, create a profile with
role_arn,source_profileandmfa_serialso that the CLI prompts once and caches the role session. - D. For account B, append
--token-codeto every command that must run asDeployer. - E. For both accounts, register a FIDO security key and enter its output as the
--token-codevalue.
- A. For account A, run
Answers
- D. Only temporary credentials obtained with an MFA code carry
aws:MultiFactorAuthPresent = true. A doesn't change the credentials; B fails because the device is already enabled and enabling it again adds no MFA context; C does nothing becausemfa_serialapplies only to profiles that assume a role. - A, C. Session credentials from
get-session-tokencarry the MFA context that a resource-based condition needs and last up to 36 hours; the role profile prompts once and reuses its cached session. B fails becausemfa_serialis ignored withoutrole_arn; D fails because the CLI has no per-command MFA option; E fails because security keys can't be used for MFA-protected API access.
3. CLI credential provider chain
Concept
- For every command the CLI checks credential sources in a fixed order and uses the first one that returns credentials; it never merges sources. Region and other settings resolve separately (§1).
- The top of the chain is explicit and local (command line, environment); the bottom is ambient (container endpoint, instance metadata). Anything stray near the top silently wins: the classic reason an instance role "doesn't work".
- Several sources produce temporary credentials and refresh them for you: assume role, web identity, IAM Identity Center, console login and custom processes.
Exam facts
- AWS CLI v2 order (first match wins). ⚠️ Changed: 10 documented sources. Older material may say there are only 6 sources (command line, environment, credentials file, config file, container, instance profile).
- Command-line options:
--profile(also--region,--output) - Environment variables:
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY,AWS_SESSION_TOKEN - Assume role: a profile with
role_arn+source_profileorcredential_source - Assume role with web identity:
web_identity_token_file+role_arn, orAWS_WEB_IDENTITY_TOKEN_FILE+AWS_ROLE_ARN(+AWS_ROLE_SESSION_NAME) - IAM Identity Center:
aws configure sso+aws sso login - Credentials file
~/.aws/credentials - Custom process:
credential_process - Config file
~/.aws/config - Container credentials: ECS task role
- EC2 instance profile credentials through IMDS
- Command-line options:
- Console-login profiles (
login_session, written byaws login) are read from the profile; in boto3's published order they sit between the credentials file and the config file (§4). --profileon the command line skips the environment-variable keys (the--debuglog says "Skipping environment variable credential check because profile name was explicitly set").AWS_PROFILEonly selects the profile: exported keys still override that profile's keys.credential_sourcevalues:Environment,Ec2InstanceMetadata,EcsContainer; it can't be combined withsource_profilein one profile. Use it to assume a second role from an instance or task role without storing keys.- Caches: role credentials in
~/.aws/cli/cache(delete it to force new credentials after a session was revoked); Identity Center tokens in~/.aws/sso/cache; console-login credentials in~/.aws/login/cache(AWS_LOGIN_CACHE_DIRECTORY). - Container step: ECS sets
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI(appended to169.254.170.2); EKS Pod Identity setsAWS_CONTAINER_CREDENTIALS_FULL_URI+AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE. If both URIs are set, the relative one is used. - IMDS step (last): the CLI makes one attempt (
AWS_METADATA_SERVICE_NUM_ATTEMPTS, default 1) with a 1-second timeout (AWS_METADATA_SERVICE_TIMEOUT);AWS_EC2_METADATA_DISABLED=trueskips IMDS entirely (laptops, CI runners outside EC2). Token flow, hop limit and IMDSv2 enforcement: seenotes/ec2-mastery-guide.md§2. - Diagnose in this order:
aws sts get-caller-identity(which principal),aws configure list(which source:env,shared-credentials-file, …),--debug(Found credentials in …). - Typical failure: keys of an IAM user left behind on a host (an exported variable, a systemd unit file, an
.envfile or~/.aws/credentials) are found before the instance role, so the process keeps that user's permissions however narrow or broad the new role is. The fix is to delete the keys, not to edit the role.
Developer angle
# Which identity, and from which source?
aws sts get-caller-identity --query Arn --output text
aws configure list
aws s3 ls --debug 2>&1 | grep -i "found credentials"
# Stray keys on an EC2 host? Remove them so the instance profile is used
env | grep '^AWS_'
unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN
# Assume a second role from the instance role, with no keys on disk
aws configure set role_arn arn:aws:iam::123456789012:role/ReportsWriter --profile reports
aws configure set credential_source Ec2InstanceMetadata --profile reports
aws s3 cp report.csv s3://reports-bucket/2026/ --profile reports
# Force fresh role credentials after the session was revoked
rm -r ~/.aws/cli/cache
Outside EC2, stop the CLI from probing instance metadata before it reports "Unable to locate credentials":
export AWS_EC2_METADATA_DISABLED=true
Traps
- Trap: "Exporting
AWS_PROFILE=prodguarantees the prod credentials." → Reality: exported access keys still override the profile's keys;--profile prodon the command line skips them. - Trap: "The container or instance role is checked before the credentials file." → Reality: files and profiles come first; container credentials and IMDS are the last two steps.
- Trap: "Put the role ARN in
~/.aws/credentials." → Reality: role profiles live inconfig(role_arn+source_profileorcredential_source). - Trap: "Attaching a narrower instance profile immediately narrows what the app can do." → Reality: only if nothing higher in the chain supplies credentials; check
aws configure list. - Trap: "A revoked role session recovers by itself." → Reality: revoked cached credentials aren't renewed automatically; delete
~/.aws/cli/cacheafter fixing the cause. - Trap: "Set both
source_profileandcredential_sourceas a fallback." → Reality: one profile can't have both.
Practice questions
-
A CI runner on an EC2 instance has an instance profile whose role allows only
s3:GetObject. A build step runsaws s3 rm s3://artifacts/old/ --recursiveand succeeds.aws configure listshows the access key with typeenv. What explains the result?- A. Access keys exported as environment variables are used before the instance profile credentials.
- B. Instance profile credentials cached in
~/.aws/cli/cachestill carry the role's earlier permissions. - C. The
defaultprofile in~/.aws/configcontainscredential_source = Ec2InstanceMetadata. - D. A bucket policy grants
s3:DeleteObjectto the instance role and overrides its identity policy.
-
A developer laptop has
[default]keys in~/.aws/credentials, aprodprofile, andAWS_ACCESS_KEY_IDandAWS_SECRET_ACCESS_KEYexported for a sandbox IAM user. Which TWO statements about the credentials the AWS CLI uses are correct? (Select TWO.)- A.
aws s3 lswith no options uses the[default]keys from the credentials file. - B.
AWS_PROFILE=prod aws s3 lsuses the prod profile's credentials instead of the exported sandbox keys. - C.
aws s3 ls --profile produses the prod profile's credentials instead of the exported sandbox keys. - D.
aws s3 ls --profile prodfails while access keys are exported. - E.
aws s3 lswith no options uses the exported sandbox keys.
- A.
Answers
- A. The
envsource shows that IAM-user keys from the environment, step 2 of the chain, were used instead of the role (step 10). B is wrong because the CLI caches assume-role credentials there, not instance profile credentials; C would still use the role; D contradicts the evidence that the role wasn't the identity used. - C, E. Environment keys beat every file-based profile, and an explicit
--profileon the command line skips them. A ignores that environment variables come before the credentials file; B fails becauseAWS_PROFILEonly selects the profile and the exported keys still win; D is wrong because the two can coexist.
4. SDK default credential provider chains
Concept
- Every AWS SDK resolves credentials the way the CLI does: create a client without credentials and the SDK walks its default credential provider chain, caches what it finds and refreshes temporary credentials before they expire.
- The order differs a little per SDK but has one shape: explicit in code → environment → web identity and profile-based providers (assume role, IAM Identity Center, console login, process) → shared files → container → instance metadata.
- Let the chain work: the same build then runs on a laptop, in Lambda, on ECS and on EC2. Pin a provider in code only when everything else must be ignored.
Exam facts
- SDK for Java 2.x (
DefaultCredentialsProvider): 1 system propertiesaws.accessKeyId,aws.secretAccessKey,aws.sessionToken→ 2 environment variables → 3 web identity token (AWS_WEB_IDENTITY_TOKEN_FILEoraws.webIdentityTokenFile+AWS_ROLE_ARNoraws.roleArn) → 4 sharedcredentials/configprofile (web identity, IAM Identity Center,role_arnwithsource_profileorcredential_source, console loginlogin_session,credential_process, session keys, plain keys) → 5 ECS container credentials → 6 EC2 instance profile. Nothing found →SdkClientException: Unable to load credentials from any of the providers in the chain. ⚠️ Changed: these are the Java 2.x names and steps. Older material may say the secret property isaws.secretKeyand the chain has 5 steps (SDK for Java 1.x, now end of support). - boto3 (12 steps): parameters to
boto3.client()→ parameters toSession()→ environment → assume role → web identity → IAM Identity Center →~/.aws/credentials→ console login →~/.aws/config→ legacy Boto2 files (/etc/boto.cfg,~/.boto) → container → IMDS. Nothing found →NoCredentialsError("Unable to locate credentials"). - Select a profile with
AWS_PROFILE,boto3.Session(profile_name="dev"), JavaProfileCredentialsProvider.create("dev")or theaws.profilesystem property. In boto3, as in the CLI, a profile named in code skips the environment keys, whileAWS_PROFILEleaves exported keys in front (botocore source). - Lambda: the runtime sets the reserved variables
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEYandAWS_SESSION_TOKENfrom the execution role (plusAWS_REGIONandAWS_DEFAULT_REGION), so the environment provider finds them. With SnapStart, the Java, .NET and Python SDKs authenticate throughAWS_CONTAINER_CREDENTIALS_FULL_URI+AWS_CONTAINER_AUTHORIZATION_TOKEN. Execution roles: seenotes/lambda-guide.md. - ECS: the task role arrives through
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI(169.254.170.2), checked before IMDS. EKS: Pod Identity (AWS_CONTAINER_CREDENTIALS_FULL_URI+ token file) or a web identity token (IRSA). Task roles vs instance roles: seenotes/containers-guide.md. - EC2 (IMDS provider): SDKs use IMDSv2 first and fall back to IMDSv1 only after an HTTP 403, 404 or 405;
AWS_EC2_METADATA_V1_DISABLED=true(ec2_metadata_v1_disabled) removes the fallback; the Kotlin and Rust SDKs never fall back.AWS_EC2_METADATA_DISABLED=trueswitches the provider off. A client that already has valid credentials never calls IMDS. Endpointhttp://169.254.169.254, orhttp://[fd00:ec2::254]with endpoint modeIPv6. IMDS reveals the role name, never its policy documents (notes/ec2-mastery-guide.md§1). - Refresh: standardized providers renew credentials automatically.
credential_processoutput that includesExpirationis re-run before expiry (the SDK doesn't cache it); output withoutExpirationis treated as long-term. - Pinning a provider in code (for example
InstanceProfileCredentialsProviderin Java) makes stray keys on the host irrelevant. Passing keys as parameters is meant for credentials you just obtained (for example fromget_session_token), never for hard-coded secrets.
Developer angle
import boto3
# Default chain: no keys in code; runs unchanged on a laptop, Lambda, ECS and EC2
s3 = boto3.client("s3", region_name="eu-west-1")
# Pin a named profile for local development
dev = boto3.Session(profile_name="dev")
ddb = dev.client("dynamodb", region_name="eu-west-1")
# Which principal did the chain resolve?
print(boto3.client("sts", region_name="eu-west-1").get_caller_identity()["Arn"])
import software.amazon.awssdk.auth.credentials.DefaultCredentialsProvider;
import software.amazon.awssdk.auth.credentials.InstanceProfileCredentialsProvider;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
public class S3Clients {
// Same behaviour as passing no provider at all
static S3Client defaultChain() {
return S3Client.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(DefaultCredentialsProvider.builder().build())
.build();
}
// Ignore everything else on the host and use only the EC2 instance role
static S3Client instanceRoleOnly() {
return S3Client.builder()
.region(Region.EU_WEST_1)
.credentialsProvider(InstanceProfileCredentialsProvider.create())
.build();
}
}
import { S3Client, ListBucketsCommand } from "@aws-sdk/client-s3";
import { fromIni } from "@aws-sdk/credential-providers";
const s3 = new S3Client({ region: "eu-west-1" }); // default chain
const devS3 = new S3Client({ region: "eu-west-1", credentials: fromIni({ profile: "dev" }) });
const { Buckets } = await devS3.send(new ListBucketsCommand({}));
console.log(Buckets?.length);
Traps
- Trap: "SDKs read only
~/.aws/credentials." → Reality: they walk a whole chain; environment variables and profiles come first, the compute service's role last. - Trap: "Give an ECS task credentials by putting keys in the task definition's environment." → Reality: assign a task role; ECS sets the container-credentials variable the SDK reads.
- Trap: "Java 2.x reads the secret from
aws.secretKey." → Reality:aws.secretAccessKey(plusaws.sessionToken);aws.secretKeyis the retired 1.x name. - Trap: "Requiring IMDSv2 breaks current SDKs." → Reality: current SDKs try IMDSv2 first; only old libraries and hand-written calls need changes (see
notes/ec2-mastery-guide.md§2). - Trap: "
AWS_EC2_METADATA_DISABLED=truehardens an application on EC2." → Reality: it removes the instance-role provider;AWS_EC2_METADATA_V1_DISABLED=trueblocks only the v1 fallback. - Trap: "Lambda code should call
sts:AssumeRoleon its own execution role." → Reality: the runtime already supplies the execution role's credentials.
Practice questions
-
A Java 2.x service on EC2 must authenticate only with the instance role, even if an operator exports access keys on the host. Which change meets this requirement with the fewest code changes?
- A. Set
AWS_EC2_METADATA_V1_DISABLED=truein the service's environment. - B. Pass
ProfileCredentialsProvider.create("default")to every client builder. - C. Pass
InstanceProfileCredentialsProvider.create()to every client builder. - D. Start the JVM with the system property
aws.profile=instance.
- A. Set
-
A team runs the same Python (boto3) code on AWS Lambda, on Amazon ECS with AWS Fargate and on Amazon EC2, with no credentials in the code. Which TWO statements describe how boto3 obtains credentials? (Select TWO.)
- A. On Lambda, it reads the
AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEYandAWS_SESSION_TOKENvalues set from the execution role. - B. On ECS, it calls IMDS at
169.254.169.254before it checks any environment variable. - C. On EC2, it reads the instance role from
~/.aws/credentials, which the EC2 launch process writes for the root user at first boot. - D. On ECS, it gets the task role's credentials from the endpoint in
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI. - E. On all three, the code must call
sts:AssumeRolebefore it creates clients.
- A. On Lambda, it reads the
Answers
- C. A pinned instance-profile provider never consults environment variables or profiles. A only blocks the IMDSv1 fallback, so exported keys would still win; B reads the
defaultprofile, which is exactly what must be ignored; D names a profile that doesn't exist and still leaves environment keys in front. - A, D. Lambda exposes the execution role through reserved environment variables, and ECS through the container credentials endpoint. B reverses the order (environment and container come before IMDS); C is wrong because EC2 delivers role credentials only through IMDS; E is unnecessary because the default chain does this automatically.
5. Credentials best practices
Concept
- Credentials don't belong in source code, container images, AMIs, user data, committed
.envfiles, notebooks or CI logs. Code asks the provider chain; the environment supplies short-lived credentials. - Every AWS compute service can hand your code a role, so code running there needs no keys at all: instance profile (EC2), task role (ECS), execution role (Lambda), Pod Identity or IRSA (EKS). Mechanics:
notes/ec2-mastery-guide.md§1,notes/containers-guide.md,notes/lambda-guide.md. - Off AWS: people sign in (
aws login, IAM Identity Center); machines federate (IAM Roles Anywhere with X.509 certificates, OIDC tokens throughAssumeRoleWithWebIdentityfor CI/CD, SAML, ECS Anywhere, EKS Hybrid Nodes, Systems Manager hybrid activations). Long-term IAM user keys only where nothing else works. - Application secrets (database passwords, third-party API keys) belong in Secrets Manager or Parameter Store
SecureString, read at runtime with the role's permissions: seenotes/security-encryption-guide.md.
Exam facts
- IAM best practices (relevant items): require human users to use federation with temporary credentials; require workloads to use temporary credentials with IAM roles; require MFA; update access keys when long-term credentials can't be avoided (use last-used information to remove unused keys); apply least privilege (IAM Access Analyzer can generate a policy from CloudTrail activity).
- ⚠️ Changed: AWS ranks CLI authentication as (Recommended) console credentials with
aws login→ IAM Identity Center → IAM user short-term credentials → EC2 instance metadata → assuming a role; long-term IAM user keys and credentials kept in an external store are "Not recommended". Older material may say "outside AWS, use environment variables or named profiles with access keys". aws login: AWS CLI 2.32.0 or later; works for the root user, IAM users and IAM-federated identities; attach the managed policySignInLocalDevelopmentAccess(root needs nothing); browser flow,--remotefor a device without a browser; credentials refreshed every 15 minutes for up to the principal's session duration (maximum 12 h); profile settinglogin_session; cache~/.aws/login/cache;aws logout(--allfor every profile). Current SDKs use the same session; older tools can read it throughcredential_process = aws configure export-credentials --profile signin --format process.- IAM Identity Center:
aws configure ssowrites an[sso-session]section (sso_start_url,sso_region,sso_registration_scopes) and a profile (sso_session,sso_account_id,sso_role_name);aws sso login --profile <p>uses the PKCE browser flow by default since CLI 2.22.0 (--use-device-codefor the device flow); tokens are cached in~/.aws/sso/cache;aws sso logout. The refreshablesso-sessionformat is recommended over the legacy profile-only format. - IAM Roles Anywhere (servers outside AWS): register your CA (or AWS Private CA) as a trust anchor (certificates from public CAs can't be used), create a profile (roles plus an optional session policy), make the role trust
rolesanywhere.amazonaws.comforsts:AssumeRole,sts:TagSessionandsts:SetSourceIdentitywithaws:SourceArn= the trust anchor ARN; the workload runsaws_signing_helper credential-processthroughcredential_process. - CI/CD outside AWS: OIDC federation (
AssumeRoleWithWebIdentity) instead of stored keys; pipeline details innotes/cicd-guide.md. - If long-term keys are unavoidable: one IAM user per workload, least privilege, regular rotation, removal of unused keys, never shared, never committed.
- Precedence is a security issue: forgotten keys in the environment or
~/.aws/credentialsoverride the role (§3), so "remove the keys" is part of any migration to roles.
Developer angle
~/.aws/config for a developer (IAM Identity Center) and for an on-premises batch server (IAM Roles Anywhere):
[profile dev]
sso_session = corp
sso_account_id = 111122223333
sso_role_name = DeveloperAccess
region = eu-west-1
[sso-session corp]
sso_start_url = https://my-sso-portal.awsapps.com/start
sso_region = eu-west-1
sso_registration_scopes = sso:account:access
[profile onprem-batch]
credential_process = /usr/local/bin/aws_signing_helper credential-process --certificate /etc/pki/batch.pem --private-key /etc/pki/batch.key --trust-anchor-arn arn:aws:rolesanywhere:eu-west-1:111122223333:trust-anchor/TA_ID --profile-arn arn:aws:rolesanywhere:eu-west-1:111122223333:profile/PROFILE_ID --role-arn arn:aws:iam::111122223333:role/BatchUploader
region = eu-west-1
aws sso login --profile dev # browser sign-in, short-term credentials cached
aws login --profile console-dev # console credentials (CLI 2.32.0 or later)
aws s3 ls --profile onprem-batch # the helper trades the X.509 certificate for role credentials
Traps
- Trap: "Outside AWS, the recommended setup is an IAM user's keys in environment variables." → Reality: use short-term credentials (
aws login, Identity Center, Roles Anywhere, OIDC federation); long-term keys are the last resort. - Trap: "Keys in an encrypted AMI or a private repository are safe." → Reality: anyone who can read the image or the repository can read the keys; use roles.
- Trap: "An on-premises server can use an IAM role directly." → Reality: only through federation such as IAM Roles Anywhere or a hybrid activation.
- Trap: "One shared team IAM user keeps access simple." → Reality: per-person Identity Center access gives traceability and short-term credentials.
- Trap: "
aws loginhands out 12-hour static credentials." → Reality: 15-minute credentials refreshed for up to 12 h. - Trap: "Lambda environment variables are the place for a database password." → Reality: store it in Secrets Manager or Parameter Store and give the execution role read access.
Practice questions
-
A company runs a nightly job on on-premises Linux servers that uploads files to Amazon S3. The security team prohibits long-term access keys, and the company already operates an internal certificate authority. Which solution meets the requirement with the LEAST operational overhead?
- A. Create one IAM user per server and rotate its access keys every 30 days with a scheduled script that updates each server.
- B. Make the internal CA an IAM Roles Anywhere trust anchor and use its credential helper via
credential_process. - C. Run
aws loginon each server every morning and let the job reuse the cached session. - D. Store an IAM user's access keys in Secrets Manager and have the job fetch them from Secrets Manager each time it starts.
-
A new developer needs CLI access to two accounts in the company's AWS Organization. The company uses IAM Identity Center with permission sets. Which TWO steps follow AWS best practices? (Select TWO.)
- A. Create an IAM user in each account and run
aws configurewith its access keys. - B. Run
aws configure ssoto create ansso-sessionand one profile per account and permission set. - C. Run
aws sso login --profile <name>to obtain short-term credentials before running commands. - D. Run
aws configure import --csvwith the access keys shown in the AWS access portal. - E. Paste the portal's temporary keys into
~/.aws/credentialsand refresh them by hand each week.
- A. Create an IAM user in each account and run
Answers
- B. Roles Anywhere trades the existing certificates for temporary role credentials with no keys to manage. A and D still rely on long-term keys; C needs an interactive browser sign-in every day, which an unattended job can't do.
- B, C. Identity Center profiles plus
aws sso logingive each person refreshable short-term credentials. A creates long-term keys; D imports only IAM user CSV files; E is manual and leaves expiring credentials in a plaintext file.
6. Using the AWS SDKs in application code
Concept
- An SDK is the in-process library for calling AWS from application code: it builds and signs requests (§10), resolves credentials (§4), Region and endpoint, retries (§9), paginates, waits and turns error responses into exceptions. Application code calls AWS through the SDK (publishing to a topic, reading an object, starting a workflow); the CLI is for people and shell scripts.
- ⚠️ Changed: current SDKs cover C++, Go, Java, JavaScript (Node.js and browsers), Kotlin, .NET, PHP, Python (boto3), Ruby, Rust, Swift and SAP ABAP, plus the AWS Tools for PowerShell. Java 1.x, JavaScript v2, Go v1 and .NET 3.x have reached end of support: build on Java 2.x, JavaScript v3, Go v2 and .NET 4.x. Older material may say there are eight SDKs (Java, .NET, Node.js, PHP, Python, Go, Ruby, C++).
- The AWS CLI itself is built on botocore, the core library under boto3.
Exam facts
- Region: ⚠️ Changed: there is no built-in default. Set it in code, in
AWS_REGION(boto3 and CLI v1 read onlyAWS_DEFAULT_REGION, notAWS_REGION) or in the profile; otherwise boto3 raisesNoRegionError("You must specify a region") and Java 2.x client creation fails. Older material may say the SDK falls back to us-east-1. - Java 2.x Region chain: builder
.region()→ system propertyaws.region→AWS_REGION→ profileregion→ EC2 instance metadata → exception. Lambda setsAWS_REGIONfor you. - One client per Region: a Java 2.x client is immutable after
build(); create one client for each Region you call. - Pagination: list and describe APIs return one page plus a token (
NextToken,ContinuationToken,Marker). Paginators follow it for you: boto3client.get_paginator("list_objects_v2").paginate(..., PaginationConfig={"PageSize": 500, "MaxItems": 2000, "StartingToken": token}), andpage_iterator.search("<JMESPath>")filters items. S3 list calls return at most 1,000 keys per page. - Waiters poll until a resource reaches a state (
bucket_exists,object_exists,instance_running,table_exists); tune them withWaiterConfig={"Delay": 5, "MaxAttempts": 20}. Prefer them, or EventBridge state-change events, to hand-written polling loops. - Errors (boto3): service errors raise
botocore.exceptions.ClientError; readerr.response["Error"]["Code"],["Error"]["Message"],["ResponseMetadata"]["HTTPStatusCode"]and["ResponseMetadata"]["RequestId"]. Modeled exceptions hang off the client (s3.exceptions.NoSuchKey,kinesis.exceptions.LimitExceededException). Client-side failures:NoCredentialsError,NoRegionError,ParamValidationError,EndpointConnectionError,ConnectTimeoutError,ReadTimeoutError. - Timeouts and retries are client configuration (
botocore.config.Config(connect_timeout=…, read_timeout=…, retries=…)), not code around every call. - Endpoints: override per client (
endpoint_url=in boto3,.endpointOverride(URI)in Java) or throughAWS_ENDPOINT_URL_<SERVICE>for local testing (§1). - Reuse clients: they are thread-safe for normal use but must not be shared across processes; create them once (in Lambda, outside the handler: see
notes/lambda-guide.md).
Developer angle
import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
cfg = Config(region_name="eu-west-1", connect_timeout=5, read_timeout=10,
retries={"total_max_attempts": 5, "mode": "standard"})
s3 = boto3.client("s3", config=cfg)
# Paginator instead of handling ContinuationToken yourself
pages = s3.get_paginator("list_objects_v2").paginate(
Bucket="orders-archive", PaginationConfig={"PageSize": 500})
for key in pages.search("Contents[?Size > `1048576`].Key"):
print(key)
# Waiter instead of a sleep loop
s3.create_bucket(Bucket="orders-staging-111122223333",
CreateBucketConfiguration={"LocationConstraint": "eu-west-1"})
s3.get_waiter("bucket_exists").wait(Bucket="orders-staging-111122223333",
WaiterConfig={"Delay": 5, "MaxAttempts": 20})
try:
s3.get_object(Bucket="orders-archive", Key="missing.json")
except s3.exceptions.NoSuchKey:
print("object not found")
except ClientError as err:
print(err.response["Error"]["Code"], err.response["ResponseMetadata"]["HTTPStatusCode"])
# Local testing against DynamoDB Local
local_ddb = boto3.client("dynamodb", region_name="us-east-1", endpoint_url="http://localhost:8000")
Traps
- Trap: "With no Region configured the SDK uses us-east-1." → Reality: it raises an error; configure the Region (Lambda sets
AWS_REGIONandAWS_DEFAULT_REGIONfor you). - Trap: "Raise
MaxKeysto read 40,000 keys in one call." → Reality: S3 returns at most 1,000 keys per page; use a paginator. - Trap: "Wrap calls in your own
while/sleeploop until a resource is ready." → Reality: use waiters or EventBridge events; they back off and stop at a sensible limit. - Trap: "Catch every exception and retry it." → Reality: the SDK already retries what is retryable; inspect
Error.Codeand fail fast on validation or permission errors. - Trap: "Start new projects on the SDK for Java 1.x or JavaScript v2." → Reality: both are end of support; use Java 2.x and JavaScript v3.
- Trap: "Change a Java 2.x client's Region at runtime." → Reality: clients are immutable; build one per Region.
Practice questions
-
A boto3 script must process every object in a bucket that holds 40,000 objects, but each
list_objects_v2call returns only 1,000 keys. Which change returns every key with the LEAST custom code?- A. Set
MaxKeys=40000on thelist_objects_v2call. - B. Call
aws s3api list-objects-v2 --no-paginatethroughsubprocessand parse the JSON output in the script. - C. Pass
PaginationConfig={"MaxItems": 40000}to the existinglist_objects_v2call. - D. Iterate over the pages returned by
get_paginator("list_objects_v2").paginate(Bucket=...).
- A. Set
-
A developer is building a Java 2.x service that must run unchanged on a laptop (profile
dev), in AWS Lambda and on Amazon EC2, in whichever Region it is deployed. Which TWO practices meet this requirement? (Select TWO.)- A. Build clients without a Region so the default Region provider chain resolves it from the environment, profile or IMDS.
- B. Hard-code
Region.US_EAST_1in every client builder. - C. Read access keys from a properties file packaged in the JAR and pass them to
StaticCredentialsProviderin every client builder. - D. Create one client at startup and call
region()on it whenever the target Region changes. - E. Build clients without a credentials provider so
DefaultCredentialsProviderfinds environment, profile or role credentials.
Answers
- D. The paginator follows
ContinuationTokenuntil the listing ends. A is capped at 1,000 keys per page; B returns only the first page and adds a subprocess; C is a paginator option, not a parameter oflist_objects_v2. - A, E. The default Region and credential chains adapt to each environment without code changes. B pins one Region; C embeds long-term keys in the artifact; D fails because built clients are immutable.
7. CLI output filtering and pagination
Concept
- Services return long lists in pages; by default the CLI follows the page tokens and prints the complete result.
- Two kinds of filtering: server-side (
--filters,--filter-expression,--prefixand other API parameters) reduces what the service sends; client-side--query(JMESPath) reshapes what the CLI prints after it has received everything. Filter on the server first, then shape on the client. - Output formats:
jsonfor programs,textfor shell pipelines,tablefor humans.
Exam facts
- Pagination options (only where the API paginates):
--page-size N: fewer items per underlying API call and more calls, same total output; the fix for timeouts on very large listings.--max-items N: print at most N items; the output then contains aNextToken.--starting-token <NextToken>: continue where the previous--max-itemsrun stopped (the value can't be empty).--no-paginate: make only the first call and print only the first page.- Use the same value for
--page-sizeand--max-itemsto avoid missing or duplicated items.
- The default page size is set by each service (S3: 1,000 objects), so
list-objects-v2on 3,500 objects makes 4 calls behind the scenes. - Seven heavy EC2 describe calls (
DescribeInstances,DescribeVolumes,DescribeSecurityGroupsand others) draw from a smaller API token bucket when sent without pagination or filters (§8): paginate and filter in automation. --queryruns after the response arrives, so it doesn't reduce transfer or API calls. With--output textit runs once per page ([0]can print one item per page); withjsonoryamlit runs once on the whole result.--filter-expressionon a DynamoDB scan filters after the read, so the scan still consumes read capacity for every item (seenotes/dynamodb-guide.md).- Pager: CLI v2 sends output through a pager (
lesson Linux and macOS,moreon Windows); scripts disable it with--no-cli-pager,AWS_PAGER=""orcli_pager =in the profile. - Skeletons:
--generate-cli-skeleton(inputby default,yaml-input,output) prints every parameter of a command; fill it in and pass it back with--cli-input-json file://…or--cli-input-yaml file://…, optionally combined with command-line parameters. Custom commands such asaws s3 cpdon't support skeletons. - Parameter values can be read from files with the
file://prefix (for example policy documents).
Developer angle
# Filter on the server, shape on the client
aws ec2 describe-instances \
--filters "Name=tag:app,Values=orders" "Name=instance-state-name,Values=running" \
--query 'Reservations[].Instances[].{Id:InstanceId,Type:InstanceType,AZ:Placement.AvailabilityZone}' \
--output table
# A huge listing that times out: smaller pages, same result
aws s3api list-objects-v2 --bucket orders-archive --page-size 200 \
--query 'Contents[].Key' --output text
# Work in batches of 500 and resume later
aws s3api list-objects-v2 --bucket orders-archive --max-items 500 --page-size 500 > batch1.json
NEXT_TOKEN=$(jq -r '.NextToken' batch1.json)
aws s3api list-objects-v2 --bucket orders-archive --max-items 500 --page-size 500 \
--starting-token "$NEXT_TOKEN"
# Script-friendly: a single value, no pager
aws sts get-caller-identity --query Account --output text --no-cli-pager
# Reusable input file, validated with a dry run first (§11)
aws ec2 run-instances --generate-cli-skeleton input > launch.json
aws ec2 run-instances --cli-input-json file://launch.json --dry-run
Traps
- Trap: "
--page-size 100returns only 100 items." → Reality: it changes the size of each API call, not the total;--max-itemslimits the output. - Trap: "
--querymakes the API return less data." → Reality: it filters client-side after the full response; use server-side filters to cut transfer. - Trap: "
--no-paginatespeeds up a complete listing." → Reality: it returns only the first page. - Trap: "
--query 'Items[0]' --output textalways prints exactly one item." → Reality: with text output the query runs once per page. - Trap: "Generate a skeleton for
aws s3 cp." → Reality: custom high-level commands don't support skeletons; use the matchingaws s3apicommand. - Trap: "
--starting-tokenworks with any value from an earlier run." → Reality: pass theNextTokenprinted by a--max-itemsrun, and keep--page-sizeequal to--max-items.
Practice questions
-
A nightly script runs
aws s3api list-objects-v2against a bucket with two million objects and intermittently fails with a timeout error. The script must still process every object. What should the developer change?- A. Add
--page-size 200to the command. - B. Add
--max-items 200to the command. - C. Add
--no-paginateto the command. - D. Add
--query 'Contents[:200]'to the command.
- A. Add
-
A developer needs the IDs of running
t3.microinstances taggedenv=devas plain text for a shell loop, and wants the EC2 API to return as little data as possible. Which TWO options should the developer use together? (Select TWO.)- A.
--output table - B.
--filters "Name=instance-type,Values=t3.micro" "Name=tag:env,Values=dev" "Name=instance-state-name,Values=running" - C.
--no-paginate - D.
--query 'Reservations[].Instances[?InstanceType==`t3.micro`].InstanceId'with no server-side filter - E.
--query 'Reservations[].Instances[].InstanceId' --output text
- A.
Answers
- A. Smaller pages give each API call a better chance to finish while the CLI still fetches the full list. B and D cut the output to 200 items, so most objects are never processed; C returns only the first page.
- B, E. Server-side filters shrink the API response;
--queryplustextprints only the IDs. A is for humans; C would drop later pages; D filters only after the full response has been transferred.
8. Service quotas and API rate limits
Concept
- Two kinds of ceilings. Service quotas cap how much you have (vCPUs of running On-Demand instances, concurrent Lambda executions, number of buckets); exceeding one fails the create call. API rate limits cap how fast you call an operation; exceeding one throttles the call.
- Short bursts of throttling: let the SDK's backoff absorb them (§9). Persistent throttling or a quota that has been reached: make fewer calls (filter, paginate, cache, batch, react to events instead of polling) or ask AWS for a higher limit.
- Service Quotas is the central console and API to view default and applied values, request increases, follow requests and alarm on usage.
Exam facts
- Terms: AWS default value vs applied value (your account's value after an increase); adjustable vs fixed; account-level vs resource-level quotas. Many quotas apply per Region (each quota's description says so); global quotas are requested from US East (N. Virginia).
- Requesting: console "Request increase at account-level", API
RequestServiceQuotaIncrease, CLIaws service-quotas request-service-quota-increase --service-code … --quota-code … --desired-value …. Smaller increases are usually approved automatically; larger ones become an AWS Support case (approved, denied or partially approved; no priority handling). Follow them withget-requested-service-quota-changeorlist-requested-service-quota-change-history. Requesting from the CLI needsServiceQuotasFullAccessoriam:CreateServiceLinkedRoleadded to your policy. - Look-ups:
list-services,list-service-quotas --service-code ec2,get-service-quota,get-aws-default-service-quota. - Alarms: usage metrics in the
AWS/Usagenamespace (EC2:ResourceCountwithResource=vCPUandClass=Standard/OnDemand); graph the limit withSERVICE_QUOTA(m1)and alarm onm1/SERVICE_QUOTA(m1)*100above 80. The Service Quotas console can create the alarm for supported quotas. - Service Quotas Automatic Management: opt-in; notifies at 80 % and 95 % utilization (AWS Console Mobile Application, email, Slack, EventBridge events, AWS Health); mode Notify Only, or Notify and Auto-Adjust, which files increase requests for you (approval isn't guaranteed).
- Quota request templates: for AWS Organizations with all features enabled; up to 10 quota increases requested automatically for every new account (existing accounts are unchanged); commercial Regions only.
- EC2 On-Demand vCPU quotas: counted in vCPUs per instance family group. ⚠️ Changed: the AWS default for "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances" (
L-1216C47A) is 5 vCPUs, other groups 0; EC2 raises them automatically as you use them, and you can request more. Older material may say 1,152 vCPUs. Exceeding it returnsVcpuLimitExceeded(fix the quota; retries don't help). - EC2 API throttling: per account, per Region, per API action, as a token bucket: bucket capacity = burst, refill per second = sustained rate. ⚠️ Changed: non-mutating
Describe*,List*,Get*100 burst / 20 per second;DescribeInstances(and six other heavy describe calls) without filters or pagination 50 / 10; mutating actions 50 / 5;RunInstancesrequest bucket 5 / 2 plus a resource bucket of 1,000 instances refilled at 2 per second. Older material may sayDescribeInstancesallows 100 calls per second. Throttled calls fail withRequestLimitExceeded; the values are visible in Service Quotas; increases go through an AWS Support case (at most 3× per request, refill rate first). - Other services' limits are covered in their modules: S3 at least 3,500 writes and 5,500 reads per second per partitioned prefix, with
503 Slow Downwhile it scales (notes/s3-guide.md); KMS request quotas (notes/security-encryption-guide.md); Lambda concurrency (notes/lambda-guide.md); API Gateway throttling and 429 (notes/api-gateway-guide.md); DynamoDB capacity (notes/dynamodb-guide.md).
Developer angle
# Applied value vs AWS default for Standard On-Demand vCPUs
aws service-quotas get-service-quota --service-code ec2 --quota-code L-1216C47A --query 'Quota.Value'
aws service-quotas get-aws-default-service-quota --service-code ec2 --quota-code L-1216C47A --query 'Quota.Value'
# Ask for more and follow the request
aws service-quotas request-service-quota-increase \
--service-code ec2 --quota-code L-1216C47A --desired-value 256
aws service-quotas list-requested-service-quota-change-history --service-code ec2 \
--query 'RequestedQuotas[].{Quota:QuotaName,Status:Status,Wanted:DesiredValue}'
# EC2 API throttle quotas (burst = bucket capacity, sustained = refill rate)
aws service-quotas list-service-quotas --service-code ec2 \
--query "Quotas[?contains(QuotaName, 'DescribeInstances')].[QuotaName,Value]" --output table
Filtered, paginated describes draw from the larger non-mutating bucket:
import boto3
ec2 = boto3.client("ec2", region_name="eu-west-1")
pages = ec2.get_paginator("describe_instances").paginate(
Filters=[{"Name": "tag:app", "Values": ["orders"]}],
PaginationConfig={"PageSize": 100})
for page in pages:
for reservation in page["Reservations"]:
for instance in reservation["Instances"]:
print(instance["InstanceId"], instance["State"]["Name"])
Traps
- Trap: "Any throttling error means opening a Support case." → Reality: occasional throttling is handled by backoff; only sustained throttling justifies a higher limit, after cutting unnecessary calls.
- Trap: "
VcpuLimitExceededgoes away with retries." → Reality: it's a quota; request an increase or use another instance family. - Trap: "A quota increase in one Region applies everywhere." → Reality: a Regional quota, such as the EC2 vCPU quotas, must be raised in each Region that needs it; only global quotas are account-wide (requested from us-east-1).
- Trap: "Service Quotas raises EC2 API throttle limits." → Reality: you can view them there, but increases go through an AWS Support case.
- Trap: "A quota request template fixes the limits of existing member accounts." → Reality: it applies only to accounts created after the template is associated.
- Trap: "Polling an unfiltered
DescribeInstancesevery second is fine at 100 calls per second." → Reality: unfiltered, unpaginated calls use a 50-token bucket refilled at 10 per second; filter, paginate or use EventBridge state-change events.
Practice questions
-
An automation tool calls
DescribeInstancesonce per second without filters or pagination to check about 300 instances, and it often receivesRequestLimitExceeded. Which change should the developer make first, without waiting for AWS Support?- A. Open an AWS Support case to raise the
DescribeInstancesbucket refill rate for the account in this Region. - B. Call
DescribeInstanceswith filters and pagination and keep the SDK's backoff enabled. - C. Request a higher "Running On-Demand Standard instances" quota in Service Quotas.
- D. Add
--no-paginateso the tool makes a single call per cycle.
- A. Open an AWS Support case to raise the
-
A deployment pipeline fails in a newly created account with
VcpuLimitExceededwhen it launches twelvem7i.xlargeinstances. Which TWO actions fix the launch and warn the team before the limit is reached again? (Select TWO.)- A. Retry the launch with exponential backoff until capacity becomes available.
- B. Ask AWS Support to raise the
RunInstancesAPI refill rate. - C. Request an increase of the "Running On-Demand Standard (A, C, D, H, I, M, R, T, Z) instances" quota.
- D. Create a CloudWatch alarm on the
AWS/UsageResourceCountmetric (vCPU,Standard/OnDemand) at 80 % ofSERVICE_QUOTA. - E. Switch the launch template to IMDSv2-only metadata options.
Answers
- B. Filtered, paginated calls use the larger non-mutating bucket, and backoff absorbs the remaining bursts. A needs Support and leaves the wasteful polling in place; C raises an unrelated resource quota; D keeps the calls unpaginated, which is what puts them in the small bucket.
- C, D. The vCPU quota is the limit that failed (12 × 4 vCPUs = 48), and a usage alarm warns before the next shortfall. A retries a quota error that won't clear; B targets API throttling, not the vCPU quota; E has no effect on quotas.
9. Retries and exponential backoff with jitter
Concept
- Remote calls fail for reasons that fix themselves: a timeout, a 5xx from a busy host, a throttle. The cure is to retry with waits that grow exponentially and are randomized (jitter), capped in time and number, so thousands of clients don't retry in lockstep.
- AWS SDKs and the CLI implement this already; you choose the retry mode and the maximum attempts. Write your own loop only for raw HTTP calls or for application-level partial failures (for example
UnprocessedItemsfrom a DynamoDB batch write: seenotes/dynamodb-guide.md). - Retry only what can succeed unchanged: transient and throttling errors. Validation and permission errors fail the same way every time.
Exam facts
- Retried by the SDKs (standard mode): transient:
RequestTimeout,RequestTimeoutException, connection errors and HTTP 500, 502, 503, 504 (PriorRequestNotCompleteis retried too); throttling:Throttling,ThrottlingException,ThrottledException,RequestThrottledException,TooManyRequestsException,ProvisionedThroughputExceededException,TransactionInProgressException,RequestLimitExceeded,BandwidthLimitExceeded,LimitExceededException,RequestThrottled,SlowDown,EC2ThrottledException. Not retried:AccessDeniedException,ValidationException,ResourceNotFoundExceptionand similar. - ⚠️ Changed: retry transient and throttling errors even when their HTTP status is 4xx (
ThrottlingExceptionis HTTP 400, many throttles are HTTP 429, and an HTTP 400RequestTimeoutis transient); don't retry other 4xx errors. Older material may say "retry only 5xx and throttling, never a 4xx". - Retry modes:
standard: recommended and the AWS CLI v2 default; 3 attempts in total (1 call + 2 retries); exponential backoff with base 2 and a maximum backoff of 20 s; a retry quota (token bucket) stops retrying while failures are widespread, so the client fails fast.adaptive: standard plus a client-side rate limiter that can delay even the first request after throttling; meant for a client that hits one resource with frequent throttling, not for multi-tenant clients; the CLI guide still calls it experimental.legacy: older per-SDK behaviour; the AWS CLI v1 default (5 attempts, 10 for DynamoDB) and boto3's default (5 attempts).
- Configure:
AWS_RETRY_MODE/retry_mode,AWS_MAX_ATTEMPTS/max_attempts(counts the first call; 1 = no retries); precedence code > environment > config file > SDK default. In boto3,Config(retries={"mode": "standard", "total_max_attempts": 5})counts the first call, while the olderConfigkeymax_attemptscounts retries only. - ⚠️ volatile: AWS's updated retry behaviour (announced 2026-05-20) is opt-in today with
AWS_NEW_RETRIES_2026=trueand, per AWS, becomes the default in November 2026: standard mode in every SDK (Java, Python, Ruby, PHP, C++ and the CLI leave legacy), 3 attempts (4 for DynamoDB and DynamoDB Streams), delay = random(0, 1) × min(20 s, base × 2^retry) with base 50 ms for transient errors (25 ms for DynamoDB) and 1,000 ms for throttling, a retry quota of 500 tokens (14 per transient retry, 5 per throttling retry), and thex-amz-retry-afterheader honoured. Support in the AWS CLI v2 is still listed as pending. Re-check before relying on these numbers. - Jitter: "full jitter" picks a uniformly random delay between 0 and the exponential cap, which spreads the retries of many clients instead of synchronizing them (the thundering-herd problem).
- Idempotency: retried writes must not duplicate work. EC2 actions such as
RunInstancesaccept a client token (--client-token/ClientToken, case-sensitive, up to 64 ASCII characters); reusing a token with different parameters fails withIdempotentParameterMismatch;TerminateInstancesis idempotent by default. Without your own token,RunInstancesuses a random one for that request only, so pass a stable token when your code re-sends the operation (for example after a crash). - Polling is retrying too: wait before the first check, back off between checks, or replace polling with waiters (§6) or EventBridge events.
Developer angle
~/.aws/config and environment overrides:
[profile batch]
retry_mode = adaptive
max_attempts = 6
export AWS_RETRY_MODE=standard
export AWS_MAX_ATTEMPTS=5
aws dynamodb describe-table --table-name Orders --debug 2>&1 | grep -i retry
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
// One hot table with frequent throttling: let the SDK slow itself down
const ddb = new DynamoDBClient({ region: "eu-west-1", maxAttempts: 5, retryMode: "adaptive" });
Your own loop for a raw HTTPS call without an SDK, with full jitter (for AWS APIs, also treat a 400 whose error code is a throttling code as retryable):
import random
import time
import requests
RETRYABLE_STATUS = {429, 500, 502, 503, 504}
def get_with_backoff(url, max_attempts=5, base=0.5, cap=20.0):
for attempt in range(max_attempts):
resp = requests.get(url, timeout=5)
if resp.status_code not in RETRYABLE_STATUS or attempt == max_attempts - 1:
return resp
time.sleep(random.uniform(0, min(cap, base * 2 ** attempt))) # full jitter
Traps
- Trap: "Fixed one-second retries are good enough." → Reality: synchronized retries amplify an overload; use exponential backoff with jitter and a cap.
- Trap: "
max_attempts = 3means three retries." → Reality: in the config file and the environment it counts the first call (3 calls in total); only boto3'sConfigkeymax_attemptscounts retries. - Trap: "Adaptive mode is the best default for every client." → Reality: it slows the whole client after one resource throttles; AWS recommends standard unless one hot resource throttles often.
- Trap: "The SDK retries the failed items of a batch call." → Reality: partial failures come back inside a successful response; retry those items yourself with backoff.
- Trap: "More retries always help." → Reality: they add latency and load; the retry quota deliberately fails fast during an outage.
- Trap: "A 400 status is never worth retrying." → Reality: throttling errors often arrive as 400 or 429.
Practice questions
-
A Python worker using boto3 writes to a single DynamoDB table and regularly receives
ProvisionedThroughputExceededExceptionduring bursts. The team wants the SDK to lower its own request rate when throttling occurs, without writing retry code. Which client configuration meets this requirement?- A.
Config(retries={"mode": "legacy", "max_attempts": 0}) - B.
Config(read_timeout=60, connect_timeout=60) - C.
AWS_MAX_ATTEMPTS=1in the worker's environment - D.
Config(retries={"mode": "adaptive", "total_max_attempts": 5})
- A.
-
A developer calls an AWS service with raw HTTPS requests signed with SigV4, without an SDK. Which TWO error responses should the client retry with exponential backoff and jitter? (Select TWO.)
- A. HTTP 400 with the error code
ThrottlingException - B. HTTP 403 with the error code
AccessDeniedException - C. HTTP 503 with the error code
ServiceUnavailable - D. HTTP 400 with the error code
ValidationException - E. HTTP 403 with the error code
SignatureDoesNotMatch
- A. HTTP 400 with the error code
Answers
- D. Adaptive mode adds client-side rate limiting on top of standard retries, which suits one throttled table. A disables retries; B changes timeouts, not the request rate; C turns retries off entirely.
- A, C. Throttling (even with status 400) and a temporarily unavailable service can succeed unchanged after a wait. B and D fail the same way every time until the policy or the request changes; E is a signing or credential error that needs a fix, not a retry.
10. Signing requests with SigV4
Concept
- Each authenticated API request carries a signature: an HMAC computed from the request contents with a key derived from your secret access key. AWS repeats the calculation and compares. The secret never leaves your machine; the signature is bound to one date, Region and service, protects the signed parts from tampering and limits replay.
- SDKs and the CLI sign every request. Sign by hand only when you send raw HTTP (no SDK for your language, or full control over the request is needed), for example to call an API Gateway endpoint that uses IAM authorization from a generic HTTP client.
- The signature travels in the
Authorizationheader (normal calls) or in the query string (a presigned URL: the URL describes the whole request, so a browser or another program can use it without credentials until it expires). - SigV4a signs with an asymmetric ECDSA key derived from your secret, so one signature is valid in a set of Regions; it is required for multi-Region requests such as S3 Multi-Region Access Points, and the SDKs and CLI switch to it automatically.
- Anonymous requests (for example to public S3 objects) carry no signature.
Exam facts
- Header form:
Authorization: AWS4-HMAC-SHA256 Credential=<access-key-id>/<YYYYMMDD>/<region>/<service>/aws4_request, SignedHeaders=host;x-amz-date, Signature=<64 hex characters>, plusX-Amz-Date(UTCYYYYMMDDTHHMMSSZ),X-Amz-Security-Tokenwhen the credentials are temporary, andx-amz-content-sha256for Amazon S3 (the payload hash orUNSIGNED-PAYLOAD). - Query-string form (presigned URL):
X-Amz-Algorithm,X-Amz-Credential,X-Amz-Date,X-Amz-Expires(1–604,800 s = 7 days),X-Amz-SignedHeaders,X-Amz-Signature, plusX-Amz-Security-Tokenfor temporary credentials. - Steps: canonical request (method, URI, query, headers, signed headers, payload hash) → SHA-256 hash → string to sign (algorithm, timestamp, credential scope, hash) → signing key (
HMAC("AWS4" + secret, date)→ Region → service →"aws4_request") → signature = hex HMAC of the string to sign → added to the header or the query string. - SigV4a: algorithm
AWS4-ECDSA-P256-SHA256; a credential scope without a Region;X-Amz-Region-Setlists the Regions (us-east-1,us-west-1, or wildcards such asus-west-*); CLI settingAWS_SIGV4A_SIGNING_REGION_SET. - Clock: requests are timestamped, and AWS documents two windows: the SigV4 page of the IAM User Guide says that in most cases a request must reach AWS within 5 minutes of its timestamp, while the common
RequestExpirederror (STS and EC2 API references) allows 15 minutes either way (a late request or a timestamp in the future). The window depends on the service, so plan for the stricter 5 minutes. Clock skew shows up asSignature expired: … is now earlier than …(orSignature not yet currentfor a clock running ahead),RequestExpired, S3RequestTimeTooSkewed(403) orSignatureDoesNotMatch; fix it by syncing the clock (NTP), not by changing IAM. - Signing and credential errors:
SignatureDoesNotMatch(HTTP 403: wrong secret, a proxy that altered signed headers, a canonicalization bug);IncompleteSignature(400);MissingAuthenticationToken(403: request not signed);UnrecognizedClientException(403),InvalidClientTokenIdor S3InvalidAccessKeyId(403): unknown access key or session token;ExpiredTokenException(403; S3 returnsExpiredTokenwith 400): temporary credentials expired; scope errors such as "Credential should be scoped to a valid Region, not …". - SigV2 is deprecated: Regions launched after January 2014 accept only SigV4, and AWS's August 2026 update lists eight older Regions where Amazon S3 still accepts SigV2. Sign new code with SigV4 (the SDK default) and block SigV2 with the
s3:signatureversioncondition key. - Presigned URLs carry the creator's permissions and die with the creator's credentials: IAM user up to 7 days; role session, instance-profile or STS credentials only as long as those credentials last (an
AssumeRolesession defaults to 1 hour; EC2 instance-role credentials typically last about 6 hours).aws s3 presign --expires-indefaults to 3,600 s (maximum 604,800); the S3 console allows 1 minute to 12 hours. Use cases and bucket-policy conditions: seenotes/s3-guide.md.
Developer angle
Signing a request to an IAM-authorized API Gateway endpoint (service name execute-api; the caller also needs execute-api:Invoke, see notes/api-gateway-guide.md):
import boto3
import requests
from botocore.auth import SigV4Auth
from botocore.awsrequest import AWSRequest
region = "eu-west-1"
url = "https://abc123defg.execute-api.eu-west-1.amazonaws.com/prod/orders"
creds = boto3.Session().get_credentials().get_frozen_credentials()
request = AWSRequest(method="GET", url=url)
SigV4Auth(creds, "execute-api", region).add_auth(request) # adds Authorization, X-Amz-Date and the token
response = requests.get(url, headers=dict(request.headers), timeout=10)
print(response.status_code)
Presigned URLs (query-string signing done for you):
aws s3 presign s3://orders-archive/invoices/2026-10.pdf --expires-in 3600 --region eu-west-1
import boto3
s3 = boto3.client("s3", region_name="eu-west-1")
url = s3.generate_presigned_url(
"get_object", Params={"Bucket": "orders-archive", "Key": "invoices/2026-10.pdf"}, ExpiresIn=3600)
Traps
- Trap: "The secret access key is sent in the
Authorizationheader." → Reality: only the access key ID, the scope and the signature travel; the secret derives the signing key locally. - Trap: "A presigned URL always lasts its full
X-Amz-Expires." → Reality: it stops working when the signing credentials expire or are revoked, whichever comes first. - Trap: "One SigV4 signature works in every Region." → Reality: SigV4 is scoped to one Region and service; multi-Region calls need SigV4a.
- Trap: "
SignatureDoesNotMatchmeans the IAM policy lacks an action." → Reality: it's a signing or credential problem (wrong secret, altered headers, clock); permission problems returnAccessDenied. - Trap: "Temporary credentials sign with the key and the secret alone." → Reality: the session token must travel as
X-Amz-Security-Token. - Trap: "Hand-written SigV2 signing is fine because S3 still accepts it." → Reality: only eight Regions launched before 2014 still take SigV2 for S3; every newer Region needs SigV4, which is also what AWS asks you to migrate to.
Practice questions
-
An on-premises service signs requests to an AWS API with its own SigV4 code. Requests that worked yesterday now fail with
Signature expired: 20261012T101500Z is now earlier than 20261012T102500Z. Nothing in the code changed. What is the MOST likely cause?- A. The IAM user's access key was deactivated during a scheduled key rotation last night.
- B. The role's permissions policy was edited and no longer allows the action that the service calls.
- C. The server's clock has drifted outside the allowed time window.
- D. The target Region now accepts only SigV4a signatures.
-
A web application assumes an IAM role with
sts:AssumeRoleusing the default session duration and uses those credentials to create S3 presigned GET URLs withExpiresIn=86400. Users report that the links stop working after about an hour. Which TWO statements are correct? (Select TWO.)- A. An
ExpiresInvalue above 3,600 seconds is silently ignored for SigV4 presigned URLs created by the SDK. - B. Switching the URLs to SigV4a removes the dependency on the lifetime of the credentials that signed them.
- C. Adding an
s3:signatureAgecondition to the bucket policy extends the validity of every URL to the full 24 hours. - D. A presigned URL stops working when the credentials that signed it expire, even if
ExpiresInis longer. - E. Requesting a longer role session with
DurationSeconds(up to the role's maximum) lets the URLs work longer.
- A. An
Answers
- C. "Signature expired" compares the request timestamp with AWS's clock, so a drifting server clock is the cause. A would fail with a credential error such as
InvalidClientTokenId; B would return an access-denied error; D is not how SigV4 works (SigV4a is only for multi-Region requests). - D, E. The URL can't outlive the role session that signed it (1 hour by default), and a longer session extends it. A is wrong because SigV4 allows up to 604,800 s; B doesn't change credential lifetimes; C can only restrict signature age, never extend it.
11. Testing permissions with dry run and the policy simulator
Concept
- Two ways to ask "would this be allowed?" without side effects:
- DryRun (mainly the EC2 API family): the service authorizes the real request with your real credentials and then stops, returning an error code that tells you the verdict.
- IAM policy simulator (console and
aws iam simulate-*): IAM evaluates the policies for a principal, action, resource and context values you choose, without sending anything to the target service.
- Pick by question: a dry run proves the live answer for one EC2 call; the simulator covers other services, draft policies, other principals and condition values. Simulator depth, IAM Access Analyzer validation and custom policy checks:
notes/iam-guide.md§6.
Exam facts
- DryRun:
--dry-runin the CLI (DryRun=Truein SDKs) on EC2 actions such asRunInstances,StartInstances,StopInstancesandCreateVolume. Allowed → errorDryRunOperation("Request would have succeeded, but DryRun flag is set."); not allowed →UnauthorizedOperation, often with an encoded authorization message (§12).--no-dry-runturns it off explicitly, for example over a skeleton file that sets"DryRun": true. The CLI guide also treats aDryRunOperationresult as proof that the input file's parameter values are valid. aws s3 cp | mv | rm | sync --dryrun(no hyphen) is different: it only prints the operations it would run; nothing checks whether the copies or deletes would be authorized.- Lambda's
Invokehas its own variant:--invocation-type DryRunvalidates parameters and checks permission to invoke (HTTP 204, the function doesn't run). DynamoDB and S3 APIs offer no dry run: use the simulator.
| Question | Tool |
|---|---|
| May this role make this EC2 call, live? | --dry-run: DryRunOperation = yes, UnauthorizedOperation = no |
| May it call S3, DynamoDB or another non-EC2 action? | aws iam simulate-principal-policy |
| Does a draft policy work before it is attached? | aws iam simulate-custom-policy with --context-entries |
| May the caller invoke this Lambda function? | aws lambda invoke --invocation-type DryRun |
- Simulator essentials: results are
allowed,explicitDenyorimplicitDenywith the matching statements; the resources don't have to exist; with the CLI or API you pass a resource-based policy yourself; SCPs are evaluated, RCPs are not; results can differ from live behaviour. Calls neediam:SimulatePrincipalPolicyoriam:SimulateCustomPolicy.
Developer angle
# Would this launch be authorized? Nothing is created.
aws ec2 run-instances --launch-template LaunchTemplateName=web \
--subnet-id subnet-0123456789abcdef0 --dry-run
# An error occurred (DryRunOperation) when calling the RunInstances operation:
# Request would have succeeded, but DryRun flag is set.
# S3 has no dry run: ask the simulator whether the CI role may write and delete under releases/
aws iam simulate-principal-policy \
--policy-source-arn arn:aws:iam::111122223333:role/ci-deployer \
--action-names s3:PutObject s3:DeleteObject \
--resource-arns "arn:aws:s3:::release-bucket/releases/*" \
--query 'EvaluationResults[].[EvalActionName,EvalDecision]' --output table
The same dry run from boto3:
import boto3
from botocore.exceptions import ClientError
ec2 = boto3.client("ec2", region_name="eu-west-1")
try:
ec2.stop_instances(InstanceIds=["i-0123456789abcdef0"], DryRun=True)
except ClientError as err:
code = err.response["Error"]["Code"]
print("would be allowed" if code == "DryRunOperation" else f"not allowed or invalid: {code}")
Traps
- Trap: "
DryRunOperationmeans the call failed." → Reality: it means you have the permissions and the request would have succeeded. - Trap: "A successful dry run guarantees the launch." → Reality: it checks permissions and parameters without making the request, so launch-time problems such as missing capacity can still appear.
- Trap: "
aws s3 rm --dryrunconfirms you may delete the objects." → Reality: it only lists what it would do; there is no authorization check. - Trap: "Every AWS API supports
--dry-run." → Reality: it's mainly an EC2 feature (LambdaInvokehas--invocation-type DryRun); use the simulator elsewhere. - Trap: "The simulator runs the action in a sandbox." → Reality: it only evaluates policies; nothing reaches the service.
- Trap: "An
allowedsimulation guarantees production access." → Reality: RCPs, VPC endpoint policies and other live factors can still deny; confirm in the real environment.
Practice questions
-
A pipeline step must confirm that its role is authorized to launch instances from the
weblaunch template before the real deployment, without creating resources or incurring cost. Which approach meets this requirement?- A. Run
aws ec2 run-instances --launch-template LaunchTemplateName=web --dry-run;DryRunOperationmeans success. - B. Run
aws ec2 run-instances --launch-template LaunchTemplateName=weband terminate the instance as soon as it starts. - C. Run
aws ec2 describe-launch-templates --launch-template-names weband check that the template exists. - D. Run
aws s3 cpwith--dryrunagainst the deployment artifact bucket.
- A. Run
-
A release script removes old objects with
aws s3 rm --recursiveand then stops EC2 instances. Before the first production run, the team must confirm that the script's role is allowed to do both, without deleting or stopping anything. Which TWO actions meet this requirement? (Select TWO.)- A. Run the
aws s3 rmcommand with--dryrunand check that it lists the objects. - B. Run the
aws ec2 stop-instancescommand with--dry-runand expectDryRunOperation. - C. Run
aws iam simulate-principal-policyfor the role withs3:DeleteObjecton the objects' ARNs, passing the bucket policy if there is one. - D. Run the
aws ec2 stop-instancescommand with--generate-cli-skeleton output. - E. Run
aws sts decode-authorization-messagewith the role ARN as the encoded message.
- A. Run the
Answers
- A. A dry run authorizes the real request without launching anything. B creates a billable instance; C proves the template exists, not that the role may use it; D tests nothing about EC2 permissions.
- B, C. The EC2 dry run authorizes the real
StopInstancesrequest, and the simulator covers the S3 deletes, which have no dry run. A only prints the planned deletes without any authorization check; D prints an output template; E decodes encoded failure messages, not ARNs.
12. Decoding authorization failures
Concept
- A failed call is either an authentication or credential problem (bad, expired or missing credentials, a broken signature) or an authorization problem (the caller is known, but no policy allows the action or one denies it). Fix the right layer.
- For authorization failures, establish four facts: the principal actually used, the action, the resource, and the policy type that denied (explicitly or implicitly). Sources: the error message,
aws sts get-caller-identity,aws sts decode-authorization-messagefor encoded messages, CloudTrail (seenotes/monitoring-guide.md) and the simulator (§11).
Exam facts
- Plain-text access-denied messages (most services) already name the principal, action, resource and the denying policy type:
because no <type> policy allows …= implicit deny,with an explicit deny in a <type> policy= explicit deny; when several types deny, only one is named. Full catalogue:notes/iam-guide.md§6; evaluation logic:notes/advanced-identity-guide.md§11. - EC2
UnauthorizedOperation(HTTP 403) can carry an encoded authorization message. Decode it withaws sts decode-authorization-message --encoded-message …; the caller needssts:DecodeAuthorizationMessage, because the details can be privileged. Only some AWS operations return an encoded message, and their documentation says so. - Decoded content:
allowed,explicitDeny,matchedStatements,failures, andcontextwith the principal (ID, name, ARN), the action, the resource and the condition keys with their values in the request.EncodedMessagelength 1–10,240 characters;InvalidAuthorizationMessage(HTTP 400) if the message contains invalid characters such as line breaks or has expired. aws sts get-caller-identityneeds no permission and works even under an explicit deny; it returnsAccount,ArnandUserId(AIDA…for an IAM user;AROA…:<session-name>andarn:aws:sts::<account>:assumed-role/<role>/<session>for a role session).- Credential errors, not policy errors:
Unable to locate credentials(boto3NoCredentialsError, JavaSdkClientException: Unable to load credentials…): nothing in the chain;InvalidClientTokenId,UnrecognizedClientException, S3InvalidAccessKeyId: unknown key or token;SignatureDoesNotMatch: wrong secret, altered request or clock skew (§10);ExpiredTokenException(S3:ExpiredToken): refresh the session (aws sso login,aws login, a newget-session-token);MissingAuthenticationToken: unsigned request or wrong URL; EC2AuthFailure: the credentials couldn't be validated. - Diagnosis order:
get-caller-identity(right principal?) →aws configure list(right source? §3) → read or decode the message (which policy type?) → simulate (§11) → check CloudTrail. - IAM is eventually consistent: a policy change can take a moment to apply everywhere (see
notes/iam-guide.md).
Developer angle
aws ec2 stop-instances --instance-ids i-0123456789abcdef0
# An error occurred (UnauthorizedOperation) ... followed by an encoded authorization message
aws sts get-caller-identity --query Arn --output text
aws sts decode-authorization-message --encoded-message "$ENCODED" \
--query DecodedMessage --output text | python3 -m json.tool
Policy that lets a support role decode messages:
{
"Version": "2012-10-17",
"Statement": [{
"Effect": "Allow",
"Action": "sts:DecodeAuthorizationMessage",
"Resource": "*"
}]
}
A trimmed decoded message (field names as in the STS API reference, values invented):
{
"allowed": "false",
"explicitDeny": "true",
"context": {
"principal": {"id": "AROAEXAMPLEROLEID:alice", "arn": "arn:aws:sts::111122223333:assumed-role/dev-role/alice"},
"action": "ec2:StopInstances",
"resource": "arn:aws:ec2:eu-west-1:111122223333:instance/i-0123456789abcdef0",
"conditions": [
{"item": {"key": "ec2:ResourceTag/env", "values": ["prod"]}}
]
}
}
Classifying failures in code before deciding what to fix:
from botocore.exceptions import ClientError, NoCredentialsError
CREDENTIAL_CODES = {"InvalidClientTokenId", "UnrecognizedClientException", "SignatureDoesNotMatch",
"ExpiredToken", "ExpiredTokenException", "InvalidAccessKeyId"}
AUTHZ_CODES = {"AccessDenied", "AccessDeniedException", "UnauthorizedOperation"}
def classify(call):
try:
return call()
except NoCredentialsError:
return "no credentials found in the provider chain"
except ClientError as err:
code = err.response["Error"]["Code"]
if code in CREDENTIAL_CODES:
return f"credential or signing problem: {code}"
if code in AUTHZ_CODES:
return f"authorization problem: {err.response['Error']['Message']}"
raise
Traps
- Trap: "
get-caller-identityfails when the user has no STS permissions." → Reality: it needs no permission and can't be denied. - Trap: "Any developer can decode the message." → Reality: decoding requires
sts:DecodeAuthorizationMessage. - Trap: "The encoded message is plain base64; decode it locally." → Reality: only the STS API can decode it.
- Trap: "
SignatureDoesNotMatchorInvalidClientTokenId→ add IAM permissions." → Reality: fix the credentials (source, secret, token, clock). - Trap: "The policy type named in the message is the only one that denies." → Reality: when several types deny, only one is named; check the others too.
- Trap: "
ExpiredToken→ rotate the IAM user's access keys." → Reality: the temporary session expired; sign in again or request a new session.
Practice questions
-
A developer's
aws ec2 start-instancescall fails withUnauthorizedOperationand a long encoded message, although the developer's identity policy allowsec2:StartInstanceson all instances. Which action shows whether an explicit deny caused the failure?- A. Decode the message locally with
base64 --decode, then search the resulting JSON document for aDenyeffect. - B. Call
aws sts get-session-tokenand retry the command with the new credentials. - C. Call
aws sts decode-authorization-messagewith a principal allowedsts:DecodeAuthorizationMessage. - D. Rerun the command with
--dry-runand read the matched policy statements in the EC2 error output.
- A. Decode the message locally with
-
A script on a build server fails, and the team must decide whether to change IAM policies or fix the credential setup. Which TWO error codes point to the credential setup rather than to missing permissions? (Select TWO.)
- A.
AccessDeniedExceptionwith "because no identity-based policy allows the action" - B.
InvalidClientTokenId - C.
UnauthorizedOperationfrom EC2 with an encoded message - D.
SignatureDoesNotMatch - E.
AccessDenied"with an explicit deny in a service control policy"
- A.
Answers
- C. The STS API decodes the message and reports
explicitDenyalong with the principal, action, resource and condition values. A fails because only STS can decode it; B changes credentials without explaining the denial; D returnsUnauthorizedOperationagain and never prints policy statements. - B, D. An unknown key or token and a signature mismatch are credential or signing problems. A, C and E are authorization results that a policy change (or removing a deny) would fix.
Quick decision rules
Profiles, MFA and the credential chain (§1–§5)
- If you see "the instance role is attached but calls still act as an IAM user", think credential-chain precedence: environment variables or
~/.aws/credentialswin; checkaws configure list, then remove the keys. - If you see
--profile prodvsAWS_PROFILE=prodwhile keys are exported, think the command-line option skips the environment keys and the variable doesn't. - If you see "regional command fails with You must specify a region", think set
--region,AWS_REGION/AWS_DEFAULT_REGIONor the profileregion: there is no default. - If you see "point one service at a local emulator without code changes", think
AWS_ENDPOINT_URL_<SERVICE>or a[services]section. - If you see an MFA condition and access keys on a laptop, think
aws sts get-session-token --serial-number … --token-code …, or a role profile withmfa_serial. - If you see
mfa_serialin a profile withoutrole_arn, think it does nothing: runget-session-tokenfor MFA-backed user credentials. - If you see an MFA condition in a bucket policy, think GetSessionToken credentials (AssumeRole credentials carry no MFA context into later calls).
- If you see "developer laptop, short-term credentials", think
aws login(console credentials) or IAM Identity Center (aws configure sso,aws sso login). - If you see "on-premises server, no long-term keys, existing CA", think IAM Roles Anywhere through
credential_process. - If you see "CI/CD outside AWS without stored keys", think OIDC federation with
AssumeRoleWithWebIdentity. - If you see code on EC2, ECS or Lambda that needs credentials, think instance profile, task role or execution role, found by the SDK's default chain.
- If you see "use only the instance role even if keys get exported", think pin
InstanceProfileCredentialsProvider(and remove the keys). - If you see
NoCredentialsErroror "Unable to locate credentials", think nothing in the chain: no role attached, wrong profile name, IMDS disabled or unreachable.
SDK usage, CLI output and quotas (§6–§8)
- If you see "only the first 1,000 results", think a paginator in code or the CLI's automatic pagination, not a bigger
MaxKeys. - If you see "wait until the bucket, table or instance is ready", think a waiter (or EventBridge events), not a sleep loop.
- If you see a CLI timeout on a huge listing, think
--page-size. - If you see "process N items now and continue later", think
--max-itemswith--starting-token. - If you see "reduce the data the API returns", think server-side
--filters;--queryonly reshapes the printed output. - If you see
RequestLimitExceededfrom EC2, think API throttling: filter and paginate, keep backoff, then an AWS Support case for a higher bucket. - If you see
VcpuLimitExceeded, think an On-Demand vCPU quota increase in Service Quotas. - If you see "warn before a quota is reached", think a CloudWatch alarm on
AWS/UsagewithSERVICE_QUOTA(), or Service Quotas Automatic Management. - If you see "new accounts in the organization need higher quotas from day one", think a quota request template.
Retries, signing and permission testing (§9–§12)
- If you see throttling errors that come and go, think the SDK's built-in retries with jittered backoff; throttling that never lets up → cut the call rate or raise the limit.
- If you see one hot table throttling a single client, think retry mode
adaptive. - If you see
AccessDeniedExceptionorValidationException, think don't retry: fix the policy or the request. - If you see a retried create that must not run twice, think a client token (
--client-token) or another idempotency key. - If you see raw HTTP calls to AWS without an SDK, think SigV4 signing (for example botocore
SigV4Auth) plusX-Amz-Security-Tokenfor temporary credentials. - If you see
Signature expired,RequestExpiredor S3RequestTimeTooSkewedwith unchanged code, think clock skew: sync with NTP. - If you see presigned URLs dying before
ExpiresIn, think the signing credentials expired (an AssumeRole session defaults to 1 hour). - If you see a request that must be valid in several Regions (Multi-Region Access Points), think SigV4a.
- If you see "check EC2 permissions without creating anything", think
--dry-run:DryRunOperationmeans allowed. - If you see "test a draft policy or another principal's access", think the IAM policy simulator (
simulate-custom-policy,simulate-principal-policy). - If you see an encoded authorization failure message, think
aws sts decode-authorization-message(needssts:DecodeAuthorizationMessage). - If you see "which identity am I really using?", think
aws sts get-caller-identity(no permission needed).
Changed vs older material
Learn the Current column; older practice material may still show the Course / older material value. Confirmed rows are copied to notes/current-facts.md.
| Topic | Course / older material | Current (verified 2026-10-12) | Source |
|---|---|---|---|
| §1, §6 · Default Region when none is configured | us-east-1 is chosen by default | No default: set the Region in code, AWS_REGION (boto3 and CLI v1 read only AWS_DEFAULT_REGION, not AWS_REGION) or the profile; the CLI fails with "You must specify a region", boto3 raises NoRegionError, Java 2.x client creation fails |
AWS docs |
| §1 · AWS CLI version 1 | v1 (installed with pip) treated as current |
Maintenance mode since 2026-07-15 (critical bug and security fixes only, no new services, APIs or Regions); end of support 2027-07-15; use v2 | AWS blog |
| §3 · AWS CLI v2 credential precedence | 6 sources: command line, environment, credentials file, config file, container, instance profile | 10 sources: options → environment → assume role → web identity → IAM Identity Center → credentials file → credential_process → config file → container → EC2 instance profile |
AWS docs |
| §4 · SDK for Java default credential chain | Java 1.x: aws.accessKeyId / aws.secretKey, 5 steps |
Java 2.x DefaultCredentialsProvider: aws.accessKeyId / aws.secretAccessKey / aws.sessionToken → environment → web identity → shared profile (SSO, role, console login, process, keys) → ECS container → instance profile; Java 1.x is end of support |
AWS docs |
| §5 · Credentials for CLI/SDK use outside AWS | Environment variables or named profiles with IAM user access keys | Short-term credentials: aws login (CLI 2.32.0+, refreshed every 15 min for up to 12 h), IAM Identity Center, IAM Roles Anywhere, OIDC federation; long-term keys "Not recommended" |
AWS docs |
| §6 · Official AWS SDKs | Java, .NET, Node.js, PHP, Python, Go, Ruby, C++ | Also JavaScript v3 (Node.js and browsers), Kotlin, Rust, Swift, SAP ABAP and the Tools for PowerShell; Java 1.x, JavaScript v2, Go v1 and .NET 3.x are end of support | AWS docs |
§8 · EC2 DescribeInstances request rate |
100 calls per second | Token bucket per account and Region: without filters or pagination 50 burst / 10 per s; filtered or paginated (non-mutating bucket) 100 / 20 per s; throttled calls get RequestLimitExceeded |
AWS docs |
| §8 · Running On-Demand Standard instances quota | 1,152 vCPUs | AWS default 5 vCPUs (L-1216C47A); the applied value varies by account and EC2 raises it automatically with usage or on request |
AWS docs |
| §9 · Which errors to retry | Retry 5xx and throttling; never retry a 4xx | Retry transient and throttling errors, including throttles with a 4xx status (ThrottlingException 400, 429) and an HTTP 400 RequestTimeout; don't retry other 4xx such as AccessDeniedException or ValidationException |
AWS docs |
| §9 · SDK default retry behaviour ⚠️ volatile | Per-SDK legacy defaults (boto3 and CLI v1: legacy, 5 attempts; DynamoDB 10) | Updated behaviour is opt-in (AWS_NEW_RETRIES_2026=true) and, per AWS, the default from November 2026: standard mode everywhere, 3 attempts (DynamoDB 4), base delay 50 ms transient / 1,000 ms throttling, 20 s cap, 500-token retry quota (14 per transient, 5 per throttling retry) |
AWS docs · AWS blog |