SQS, SNS and Kinesis — Cheat sheet

Verified against AWS documentation on 2026-10-12 · ⚠️ = changed vs older material: learn the value shown here (older practice material may say otherwise; full list in notes/sqs-sns-kinesis-guide.md#changed-vs-older-material) · ⚠️ unverified = not confirmed in AWS docs, don't rely on it · explanations and examples: notes/sqs-sns-kinesis-guide.md.

SQS standard queues and the message lifecycle

Topic	Current facts	Memory hook
Delivery model	Pull: `SendMessage` → `ReceiveMessage` → `DeleteMessage`; receiving removes nothing	Read, then shred
Standard queue guarantees	Nearly unlimited throughput, at-least-once delivery, best-effort order → idempotent consumers	Twice is possible
Message size ⚠️	1 byte – 1 MiB (was 256 KB); `MaximumMessageSize` can only lower it
Retention	Default 4 days; 60 s – 14 days; a change can take 15 min to apply	Four by default, fourteen at most
Receipt handle	New on every receive; delete with the latest one, never with the `MessageId`	Handle, not ID
Batch actions	Up to 10 entries and 1 MiB per call; check the `Failed` list	Ten per trip
Billing	Each 64 KB chunk = 1 request (1 MiB = 16); first 1 million requests per month free
Standard in-flight limit	About 120,000; at the limit short polling returns `OverLimit`
FIFO in-flight limit ⚠️	120,000 messages (was 20,000)
`PurgeQueue`	Takes up to 60 s; another purge within 60 s → `PurgeQueueInProgress`
Fair queues ⚠️	Standard queue + `MessageGroupId` as tenant ID → quiet tenants keep low dwell time; no ordering
Backlog metric ⚠️	CloudWatch `ApproximateNumberOfMessagesVisible` (queue attribute name: `ApproximateNumberOfMessages`)

SQS visibility timeout and long polling

Topic	Current facts	Memory hook
Visibility timeout	Default 30 s, range 0 – 12 h; set per queue, per receive or per message	Hidden, not gone
`ChangeMessageVisibility`	New value counts from the call; 12 h ceiling from the first receive; 0 = release now	Restart the clock
Too short vs too long	Too short → processed twice; too long → slow retry after a crash
Heartbeat	Extend the timeout periodically while a long job runs
Lambda consumer	Queue visibility timeout ≥ 6 × function timeout (+ `MaximumBatchingWindowInSeconds`)	Six times the timeout
Long polling	Wait 1–20 s via `WaitTimeSeconds` (per call, overrides) or `ReceiveMessageWaitTimeSeconds` (queue)	Wait for it
Short polling	Default (wait 0); samples servers; empty answers even when messages exist
SDK client timeout	HTTP read timeout must exceed `WaitTimeSeconds`

SQS dead letter queues and redrive

Topic	Current facts	Memory hook
Redrive policy	Source queue attribute `RedrivePolicy`: `deadLetterTargetArn` + `maxReceiveCount` (1–1,000)
DLQ rules	Create it first; same type, same account, same Region	Twins live together
Redrive allow policy	On the DLQ: `allowAll` (default), `denyAll` or `byQueue` (up to 10 source ARNs)
Standard DLQ retention	Original enqueue time is kept → DLQ retention must exceed the source's	Age travels with the message
FIFO DLQ retention	Enqueue time resets when the message moves
DLQ alarm	`ApproximateNumberOfMessagesVisible` ≥ 1 (`NumberOfMessagesSent` ignores automatic moves)
Redrive ⚠️	`StartMessageMoveTask` (API, FIFO too); default back to source; custom destination of the same type	Fix first, then redrive
Redrive limits	Up to 500 msg/s; 1 active task per queue; 36 h per task; no filtering; new message IDs
Not redrivable	DLQs of SNS subscriptions or Lambda invocations

SQS delay queues and large messages

Topic	Current facts	Memory hook
Delay queue	`DelaySeconds` 0–900 s (default 0) hides every new message	Delay before, visibility after
Message timer	Per-message `DelaySeconds` overrides the queue delay; standard queues only
Changing the queue delay	Standard: new messages only; FIFO: also messages already waiting
Longer schedules	EventBridge Scheduler, not SQS
Extended Client Library ⚠️	Java and Python; payload in S3 up to 2 GB; synchronous clients only	Parcel in S3, label in the queue
Payload permissions	Producer `s3:PutObject`, consumer `s3:GetObject`, cleanup `s3:DeleteObject`

SQS FIFO queues

Topic	Current facts	Memory hook
Naming	`.fifo` suffix; `FifoQueue=true` only at creation; no conversion from standard
`MessageGroupId`	Required; order kept per group; one in-flight message blocks its group	Groups are lanes
Deduplication	`MessageDeduplicationId` or `ContentBasedDeduplication` (SHA-256 of the body); 5-minute interval	Five-minute memory
Normal throughput	300 TPS per API action; 3,000 msg/s with batches of 10
High throughput mode ⚠️	`DeduplicationScope=messageGroup` + `FifoThroughputLimit=perMessageGroupId`; up to 70,000 TPS in us-east-1, us-west-2, eu-west-1
Not supported on FIFO	Per-message delay; a standard DLQ
`ReceiveRequestAttemptId`	Retry a failed receive within 5 min and get the same messages

SQS security and access policies

Topic	Current facts	Memory hook
Default encryption ⚠️	SSE-SQS (SQS-owned keys) on new queues since 2022	Encrypted out of the box
SSE-KMS	`KmsMasterKeyId`; data key reuse 60 s – 24 h (default 300 s)	Longer reuse, fewer KMS calls
What SSE covers	Message body only; new messages only
KMS permissions	Consumer `kms:Decrypt`; producer `kms:GenerateDataKey` (+ `kms:Decrypt`: ⚠️ unverified, docs disagree)
AWS service producers	SNS, S3, EventBridge → encrypted queue needs a customer managed key whose policy allows the service	`aws/sqs` can't be edited
Queue policy	For other accounts and service principals; add `aws:SourceArn` / `aws:SourceAccount`
Cross-account access	Queue policy in the owner account + IAM policy in the caller's account	Both doors open
Not shareable cross-account	`CreateQueue`, `DeleteQueue`, `SetQueueAttributes`, `AddPermission`, tagging, message move tasks

Topic	Current facts	Memory hook
Model	Push pub/sub; a standard topic stores nothing (retries, then DLQ or drop)	Shout, don't store
Protocols	`sqs`, `lambda`, `firehose`, `http`/`https`, `email`, `email-json`, `sms`, `application`
Quotas	100,000 standard / 1,000 FIFO topics; 12,500,000 subs per standard topic, 100 per FIFO topic
Message size ⚠️	256 KiB default; `MaximumMessageSize` up to 1 MiB (then ≤ 100 SQS, Lambda or Firehose subs)
Confirmation	HTTP/S, email and cross-account endpoints confirm with `ConfirmSubscription` (token valid 2 days)
Retries to SQS and Lambda	100,015 attempts over 23 days
Retries to email, SMS, push	50 attempts over 6 hours
HTTP/S delivery policy	Customisable; total retry time ≤ 3,600 s; 5XX and 429 are retried
Subscription DLQ	`RedrivePolicy` on the subscription → SQS queue in the same account and Region
Raw message delivery	Strips the JSON envelope; SQS raw delivery passes ≤ 10 attributes
Mobile push ⚠️	FCM with HTTP v1 credentials (platform still named `GCM`), APNs, ADM

Topic	Current facts	Memory hook
Fan-out	One topic + one SQS queue per consuming service	One shout, many mailboxes
Queue policy for SNS	Allow `sns.amazonaws.com` `sqs:SendMessage` with `aws:SourceArn` = topic ARN
Cross-Region delivery	To SQS and Lambda; opt-in Region → principal `sns.<region>.amazonaws.com`
Filter scope ⚠️	`FilterPolicyScope` = `MessageAttributes` (default) or `MessageBody`
Filter logic	AND across keys, OR within a key's values, `$or` across conditions
Operators ⚠️	exact, `anything-but`, `prefix`, `suffix`, `equals-ignore-case`, `numeric`, `exists`, `cidr`, `wildcard`
Filter limits	5 keys, 150 combinations, 256 KB; 200 policies per topic, 10,000 per account
Filter propagation	Up to 15 minutes after a change

Topic	Current facts	Memory hook
Publishing	`MessageGroupId` required; deduplication ID or content-based; 5-minute interval
Subscribers ⚠️	SQS FIFO and SQS standard queues only (no Lambda, HTTP/S, email, SMS)	Queues only
Throughput ⚠️	300 msg/s per group; `Topic` scope 3,000 msg/s or 20 MB/s; `MessageGroup` scope higher, one-way
Exactly-once conditions	FIFO queue subscriber, delete before the visibility timeout, no filter policy
Archive ⚠️	`ArchivePolicy` `MessageRetentionPeriod` 1–365 days, FIFO topics only
Replay	`ReplayPolicy` (`Timestamp`, `StartingPoint`, optional `EndingPoint`); filters still apply	Rewind one subscriber

Kinesis Data Streams shards and capacity modes

Topic	Current facts	Memory hook
Shard capacity	Write 1 MB/s or 1,000 records/s; read 2 MB/s shared and 5 `GetRecords` calls/s
Partition key	1–256 characters, MD5 → shard; same key = same shard = ordered	Same key, same lane
Record size ⚠️	1 MiB default; up to 10 MiB per stream with `UpdateMaxRecordSize`
`PutRecords` ⚠️	500 records and 10 MiB per request
Retention	24 h default; up to 8,760 h (365 days); decreasing drops old data at once
Provisioned scaling	`UpdateShardCount` ≤ 10 times per 24 h, ≤ 2× up or ½ down per call; `SplitShard`/`MergeShards`
On-demand Standard	Starts at 4 MB/s write / 8 MB/s read; absorbs 2× the previous 30-day peak
On-demand Advantage ⚠️	No per-stream fee, data ≥ 60 % cheaper, 25 MiB/s commitment, warm throughput, 50 EFO consumers
Capacity mode switch	`UpdateStreamMode` at most twice per 24 h
`AUTO` distribution ⚠️	On-demand only; ignores partition keys and `ExplicitHashKey`
Encryption	SSE-KMS (`StartStreamEncryption`); producer `kms:GenerateDataKey`, consumer `kms:Decrypt`

Kinesis producers consumers and the KCL

Topic	Current facts	Memory hook
`PutRecords` partial failure	HTTP 200 + `FailedRecordCount`; resend only failed entries	200 isn't all clear
KPL	Aggregation, collection, retries; adds up to `RecordMaxBufferedTime` latency	Batch for throughput, not latency
Shard iterators	`TRIM_HORIZON`, `LATEST`, `AT_SEQUENCE_NUMBER`, `AFTER_SEQUENCE_NUMBER`, `AT_TIMESTAMP`; expire after 5 min
`GetRecords`	Up to 10,000 records or 10 MB per call; 5 calls/s per shard
KCL lease table	DynamoDB table named after the application; one lease per shard; checkpoints	One shard, one worker
KCL scaling	Workers beyond the shard count sit idle; each application needs a unique name
KCL and KPL versions ⚠️	KCL 3.x (CPU-based balancing) and KPL 1.x current; KCL 1.x and KPL 0.x end-of-support date 2026-01-30
Lambda consumer	`ParallelizationFactor` 1–10 batches per shard (lambda guide)

Kinesis throttling and enhanced fan out

Topic	Current facts	Memory hook
Write throttling	`WriteProvisionedThroughputExceeded` → backoff, spread keys, split shard, on-demand
Read throttling	`ReadProvisionedThroughputExceeded` → EFO, fewer polling consumers
Consumer lag	Rising `GetRecords.IteratorAgeMilliseconds` → add consumer parallelism	Old iterator, slow reader
Hot shard detection	Enhanced shard-level metrics (`EnableEnhancedMonitoring`, extra cost)
Enhanced fan-out	2 MB/s per shard per consumer, HTTP/2 push via `SubscribeToShard`, about 70 ms	Own pipe per consumer
EFO limits ⚠️	20 consumers per stream (50 with On-demand Advantage); a subscription lasts ≤ 5 min

Amazon Data Firehose

Topic	Current facts	Memory hook
Name ⚠️	Amazon Data Firehose (was Kinesis Data Firehose); API, CLI and IAM unchanged
Sources and destinations ⚠️	Direct PUT, Kinesis Data Streams or Amazon MSK in; S3, Redshift, OpenSearch (+ Serverless), Iceberg Tables, Snowflake, Splunk, HTTP endpoints, partners out
Record and batch size	1,000 KiB per record; `PutRecordBatch` 500 records or 4 MiB
Buffering ⚠️	`SizeInMBs` 1–128 MiB (5) and `IntervalInSeconds` 0–900 (300); first one reached wins	Size or time, whichever first
Lambda transformation	Return every `recordId` with `Ok`/`Dropped`/`ProcessingFailed` and base64 `data`; ≤ 5 min	Same IDs back
Format conversion	JSON → Parquet or ORC using an AWS Glue Data Catalog schema
Dynamic partitioning	S3 only; up to 50 keys (jq or Lambda); enable at creation only, can't be disabled
Failures	`processing-failed` S3 prefix; Direct PUT data kept 24 h while the destination is down
Replay	None; Flink reads KDS or MSK, never Firehose	Firehose only pours
Flink ⚠️	Managed Service for Apache Flink (was Kinesis Data Analytics); CLI `kinesisanalyticsv2`

Requirement	Pick	Memory hook
Buffer work items, workers scale on backlog	SQS standard
Order or no duplicates per entity	SQS FIFO (SNS FIFO + SQS FIFO for fan-out)
One event, several independent consumers	SNS + one SQS queue per consumer
Many real-time readers or replay	Kinesis Data Streams	Streams remember
Load into S3, Redshift, OpenSearch or Splunk without code	Amazon Data Firehose
Email, SMS or mobile push to people and devices	SNS

Numbers to know

Item	Value	Note
SQS message size	1 byte – 1 MiB ⚠️	Was 256 KB; larger via S3 pointer
Extended Client payload	Up to 2 GB	SQS and SNS libraries, payload in S3
SQS retention	Default 4 days; 60 s – 14 days	`MessageRetentionPeriod`
SQS visibility timeout	Default 30 s; 0 – 12 h	12 h counted from the first receive
SQS delay	0 – 900 s (15 min)	Queue `DelaySeconds` or per-message timer
Long polling wait	Up to 20 s	0 = short polling
`MaxNumberOfMessages`	1 – 10 (default 1)	Per `ReceiveMessage` call
SQS batch request	10 messages, 1 MiB total	Send, delete, change visibility
SQS message attributes	Up to 10	Also the SNS raw-delivery limit for SQS
SQS billing chunk	64 KB = 1 request	A 1 MiB message = 16 requests
In-flight messages	≈120,000 standard; 120,000 FIFO ⚠️	FIFO was 20,000
FIFO normal throughput	300 TPS per action; 3,000 msg/s batched	Per partition
FIFO high throughput	Up to 70,000 TPS (700,000 msg/s batched) ⚠️	us-east-1, us-west-2, eu-west-1
FIFO deduplication interval	5 minutes	SQS and SNS
`maxReceiveCount`	1 – 1,000	At least 5 with Lambda consumers
DLQ redrive velocity	Up to 500 msg/s	36 h per task, 1 task per queue
KMS data key reuse	60 s – 24 h (default 300 s)	SQS SSE-KMS
`PurgeQueue`	Up to 60 s	One purge per 60 s
SNS topics per account	100,000 standard; 1,000 FIFO
SNS subscriptions per topic	12,500,000 standard; 100 FIFO	Firehose: 5 per topic per owner
SNS message size	256 KiB default; up to 1 MiB ⚠️	≤ 100 SQS/Lambda/Firehose subs above 256 KiB
SNS `PublishBatch`	10 entries	Total within `MaximumMessageSize`
SNS retries to SQS/Lambda	100,015 attempts over 23 days
SNS HTTP/S retry ceiling	3,600 s total	`numRetries` 0–100
SNS filter policy	5 keys, 150 combinations, 256 KB	Up to 15 min to apply
SNS FIFO throughput	300 msg/s per group; 3,000 msg/s or 20 MB/s per topic ⚠️	`Topic` scope; `MessageGroup` scope higher
SNS FIFO archive	1 – 365 days	FIFO topics only
KDS shard write	1 MB/s or 1,000 records/s	Per shard
KDS shard read	2 MB/s and 5 `GetRecords`/s	Shared by standard consumers
KDS record size	1 MiB default; up to 10 MiB opt-in ⚠️	Key + data before base64
KDS `PutRecords`	500 records, 10 MiB ⚠️	Was 5 MB
KDS retention	24 h default; up to 8,760 h (365 days)
`GetRecords` per call	10,000 records or 10 MB	Iterator expires after 5 min
Enhanced fan-out	2 MB/s per shard per consumer; 20 consumers (50 with On-demand Advantage) ⚠️	~70 ms delivery
On-demand Standard start	4 MB/s write, 8 MB/s read	Absorbs 2× previous 30-day peak
Capacity mode switches	2 per stream per 24 h	`UpdateStreamMode`
Firehose record	1,000 KiB	`PutRecordBatch` 500 records or 4 MiB
Firehose buffering	1 – 128 MiB (5); 0 – 900 s (300) ⚠️	Interval 0 = zero buffering
Firehose Lambda transform	Up to 5 min; 0.2 – 3 MB batches (1 MB)	3 retries by default
Firehose outage buffer	24 hours	Direct PUT source

Decision rules

If you see a message processed twice while a slow job is still running, think longer visibility timeout or a ChangeMessageVisibility heartbeat.
If you see many empty receives or high SQS request cost, think long polling with a 20 s wait.
If you see a poison message that fails forever, think DLQ with maxReceiveCount and an alarm on ApproximateNumberOfMessagesVisible.
If you see "move DLQ messages back after the fix without code", think StartMessageMoveTask.
If you see messages expiring early in a standard DLQ, think DLQ retention longer than the source queue's.
If you see "hide new messages for a few minutes", think DelaySeconds (queue) or a message timer (per message, standard only).
If you see payloads above 1 MiB, think S3 plus a pointer with the Extended Client Library.
If you see order per entity plus parallelism, think FIFO with MessageGroupId = entity ID.
If you see duplicates from producer retries, think FIFO with a stable MessageDeduplicationId.
If you see FIFO traffic above 3,000 msg/s, think high throughput mode and many group IDs.
If you see duplicates on a queue that must stay standard, think idempotent consumer.
If you see S3, SNS or EventBridge unable to write to an encrypted queue, think customer managed key with a key policy for that service.
If you see cross-account queue access, think queue policy plus the caller's IAM policy.
If you see one event for several independent services, think SNS topic with one SQS queue each.
If you see a subscribed queue receiving nothing, think queue policy for sns.amazonaws.com and the KMS key policy.
If you see subscribers that need only some events, think filter policies (MessageBody scope for payload fields).
If you see an HTTP/S subscriber down for hours, think subscription DLQ.
If you see ordered, deduplicated fan-out, think SNS FIFO → SQS FIFO.
If you see "redeliver last week's events to one subscriber", think SNS FIFO archive plus ReplayPolicy.
If you see real-time data for several applications or replay, think Kinesis Data Streams.
If you see write throttling on one hot shard, think partition key distribution, then SplitShard.
If you see read throttling with several consumer apps, think enhanced fan-out.
If you see rising iterator age, think more consumer parallelism (shards, workers, ParallelizationFactor).
If you see idle KCL workers, think more workers than shards.
If you see unpredictable stream traffic and shard toil, think on-demand capacity mode.
If you see PutRecords with FailedRecordCount > 0, think resend only the failed entries.
If you see "load into S3, Redshift, OpenSearch or Splunk with no code", think Amazon Data Firehose.
If you see "JSON to Parquet on the way to S3", think Firehose format conversion with a Glue schema.
If you see SQL or Java analytics on a stream, think Managed Service for Apache Flink on KDS or MSK.

Memory hooks

Read, then shred: receiving never deletes; DeleteMessage does.
Handle, not ID: delete with the latest receipt handle.
Hidden, not gone: the visibility timeout only hides a message.
Restart the clock: ChangeMessageVisibility counts from now.
Delay before, visibility after: delay hides before the first receive.
Age travels with the message: a standard DLQ keeps the original enqueue time.
Twins live together: a DLQ matches its source's type, account and Region.
Groups are lanes: order inside a group, parallelism across groups.
Five-minute memory: FIFO deduplication forgets after 5 minutes.
Both doors open: cross-account needs the resource policy and the caller's IAM policy.
aws/sqs can't be edited: service producers need a customer managed key.
Shout, don't store: a standard SNS topic keeps nothing.
One shout, many mailboxes: SNS + one SQS queue per consumer.
Queues only: SNS FIFO topics deliver only to SQS queues.
Same key, same lane: one partition key always lands on one shard.
Streams remember: KDS keeps records until retention ends, for every reader.
200 isn't all clear: check FailedRecordCount after PutRecords.
One shard, one worker: extra KCL workers idle.
Old iterator, slow reader: rising iterator age means consumers lag.
Own pipe per consumer: EFO gives 2 MB/s per shard to each consumer.
Size or time, whichever first: Firehose buffering hints.
Same IDs back: a Firehose transformation returns every recordId.
Firehose only pours: no consumers, no replay.

SQS, SNS and Kinesis — Cheat Sheet