Free lesson · Developer on AWS

The AWS SDK model: clients, credentials chain, retries, pagination

Easy10 min readawscloudbackend

What The SDK Actually Is

An AWS SDK is a client library over an HTTP API. Every call you make becomes an HTTPS request, signed with SigV4 using your credentials, sent to a Regional endpoint, and answered with XML or JSON that the SDK deserialises into objects.

Almost every confusing SDK behaviour makes sense once you hold that model. Credential resolution is "how do I find the key to sign with." Region resolution is "which endpoint do I send to." Retries are "what do I do when the HTTP call fails." Pagination is "the API returned a page, not a result set."

This topic covers the four things that cause most developer pain with AWS SDKs, in roughly the order they bite: client lifecycle, the credentials chain, retries and timeouts, and pagination.

Clients Are Expensive To Create And Cheap To Reuse

Constructing a client does real work: resolving credentials, resolving the Region, loading service model metadata, and setting up an HTTP connection pool.

Create it once. Reuse it for the life of the process.

# Wrong - a client per call
def get_item(key):
    client = boto3.client("dynamodb")      # resolves credentials, builds a pool
    return client.get_item(...)

# Right - construct once, reuse
client = boto3.client("dynamodb")

def get_item(key):
    return client.get_item(...)

Important: In Lambda, construct clients outside the handler. Code outside the handler runs once per execution environment and is reused across invocations; code inside runs every time.

A client constructed inside the handler resolves credentials and builds a connection pool on every single invocation, adding latency to every request and discarding a warm TLS connection that could have been reused. On a high-volume function this is measurable in both latency and cost.

The same reasoning applies to anything else expensive and reusable: database connection pools, parsed configuration, and cached secrets. Course 1 Module 04 Topic 8 covers the execution model this depends on.

Clients are generally thread-safe for concurrent use in the major SDKs - verify for your specific SDK and version, but the intended pattern is one client shared across threads, not one per thread.

The Credentials Provider Chain

The SDK looks for credentials in a defined order and uses the first source that yields them. The order is the thing to memorise, because the surprises all come from something earlier in the chain than you expected.

graph TD
    A[1. Explicit parameters<br/>passed to the client] --> B[2. Environment variables<br/>AWS_ACCESS_KEY_ID etc]
    B --> C[3. Shared credentials file<br/>~/.aws/credentials]
    C --> D[4. Shared config file<br/>~/.aws/config, incl. SSO]
    D --> E[5. Container credentials<br/>ECS task role]
    E --> F[6. Instance metadata<br/>IMDS - EC2 instance role]
    F --> G[No credentials found<br/>→ error]
    A -.->|first match wins| W[Credentials used]
    B -.-> W
    C -.-> W

This diagram shows the AWS SDK credentials provider chain in order. First, explicit parameters passed to the client. Second, environment variables such as AWS underscore ACCESS underscore KEY underscore ID. Third, the shared credentials file at tilde slash dot aws slash credentials. Fourth, the shared config file, which includes single sign-on configuration. Fifth, container credentials from an ECS task role. Sixth, instance metadata from IMDS for an EC2 instance role. If none yield credentials, an error results. The first source that provides credentials wins.

Warning: Environment variables sit above your profile in the chain, and this causes the single most common credential confusion.

The scenario: you ran a command that exported AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY into your shell - an assume-role helper, a CI script you tested locally, a tutorial. Later, in the same shell, you run a command with --profile prod. The environment variables win. You are operating as whatever those keys belong to, and the profile flag did nothing.

The symptom is an access denied error that makes no sense given the profile's permissions, or worse, a successful operation against the wrong account.

Check env | grep AWS when credentials behave unexpectedly. It is the first diagnostic and it resolves this immediately. Unset them with unset AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN.

Two further chain behaviours worth knowing:

AWS_PROFILE selects the profile without a --profile flag, and is itself an environment variable - so it can be stale in the same way.

On EC2 and ECS, the chain ends at the instance or task role, which is the correct pattern: no credentials on the box, an identity attached to the compute. Course 1 Module 04 Topic 5 covers IMDSv2 and why the hop limit matters.

Region resolution follows a parallel chain: explicit client parameter, then AWS_REGION or AWS_DEFAULT_REGION, then the profile's region setting, then in some SDKs the instance metadata. A "could not connect to endpoint" or "you must specify a region" error is almost always this chain finding nothing, not a networking problem.

Retries

The SDK retries some failures automatically, and understanding which is the difference between resilient code and code that either gives up too early or hammers a struggling service.

Retried by default:

Throttling - ThrottlingException, TooManyRequestsException, ProvisionedThroughputExceededException
5xx responses - server-side errors
Transient network failures - connection resets, timeouts

Not retried:

4xx client errors - AccessDenied, ValidationException, ResourceNotFound. Retrying will not help; the request is wrong.

Retry modes differ by SDK and version, typically offering a legacy mode, a standard mode with exponential backoff and jitter and a defined attempt count, and an adaptive mode that additionally rate-limits client-side when it detects throttling.

Tip: Adaptive retry mode is worth enabling for workloads that hit throttling regularly, because it slows the client down rather than continuing to send requests that will be rejected. Standard mode retries a throttled request; adaptive mode also reduces the rate at which new requests are issued.

The caution: adaptive mode maintains state per client, which means it works as intended for a long-lived client and does nothing useful if you construct a client per request - which is another reason for the client lifecycle guidance above.

Warning: Retry amplification is the failure that turns a degradation into an outage. The SDK retries three times. Your application wraps the call in its own retry loop of three. Your framework or job runner retries the whole operation three times. That is twenty-seven attempts for one logical request, arriving at a service that is already struggling.

The multiplication is invisible in any single layer's code, which is why it survives review.

Know your total attempt budget and configure it in one place. Usually that means letting the SDK do the retrying - it implements backoff and jitter correctly - and having application-level code not retry at all, or retry only for cases the SDK does not cover. If you must have both, reduce the SDK's count accordingly.

Timeouts

Defaults are set for general use and are frequently wrong for yours.

Timeout	Controls
Connect timeout	Establishing the TCP and TLS connection
Read timeout	Waiting for a response after the request is sent
Total attempt duration	Some SDKs support an overall cap

Important: Default timeouts multiplied by default retries can exceed your Lambda's timeout, and the failure mode is confusing. With a read timeout of tens of seconds and three retry attempts, a single hung call can consume more time than a function configured with a 30-second budget has available.

What you see is the function timing out with no error from the SDK at all - the invocation is killed mid-retry, so nothing logs a failure reason. The logs show the request starting and the function ending, with nothing in between.

Set timeouts explicitly for any latency-sensitive context, and size them so that connect + read × attempts fits comfortably inside the caller's own budget. This is one of the highest-value SDK configurations and it is almost always left at defaults.

Pagination

AWS APIs return pages, not result sets. Code that ignores this works in development and silently processes a subset in production.

The mechanism: a response includes a continuation token - NextToken, Marker, NextContinuationToken, or LastEvaluatedKey depending on the service - and you pass it back to get the next page.

Warning: ListObjectsV2 returns at most 1,000 objects per call. Scan and Query in DynamoDB return at most 1 MB. Code that calls either once and processes the result is correct on a small bucket or table and quietly wrong on a large one.

This is the most dangerous bug in this topic because it does not fail. There is no exception, no error, no warning. The job runs, reports success, and processed the first thousand of forty thousand objects. It is discovered weeks later when someone notices missing data, and the cause is not obvious from the code, which looks entirely reasonable.

Use the paginator abstraction your SDK provides rather than handling tokens by hand - get_paginator in boto3, the paginator classes in the JS and Java SDKs, NewListObjectsV2Paginator in Go. They handle the token loop correctly and yield every result.

# Silently wrong on any bucket with more than 1000 objects
response = s3.list_objects_v2(Bucket="my-bucket")
for obj in response["Contents"]:
    process(obj)

# Correct
paginator = s3.get_paginator("list_objects_v2")
for page in paginator.paginate(Bucket="my-bucket"):
    for obj in page.get("Contents", []):
        process(obj)

Note page.get("Contents", []) rather than page["Contents"] - a page with no results omits the key entirely rather than returning an empty list, which is a second, smaller version of the same class of bug.

Server-side filtering beats client-side filtering wherever the API supports it. A Query with a key condition reads only matching items; a Scan with a filter expression reads everything and then discards - you pay for the full read and get the filtered result, which is a cost problem as well as a performance one.

Waiters

Many operations are asynchronous: creating a table, launching an instance, and starting an export all return before the resource is ready.

Waiters poll for a state with sensible backoff, and every SDK provides them.

# Reinventing it badly
while True:
    if table_is_active():
        break
    time.sleep(1)     # no backoff, no timeout, no failure detection

# Using the waiter
waiter = client.get_waiter("table_exists")
waiter.wait(TableName="my-table")

Waiters have configurable delay and maximum attempts, and they raise on failure states rather than looping forever. Hand-rolled polling loops typically have no timeout, no backoff, and no handling for the resource entering a failed state - which means they hang rather than error.

Errors

Error codes matter more than HTTP status codes. Two failures can both be 400 and mean entirely different things - one retryable, one not.

The SDK exposes the error code (ThrottlingException, ConditionalCheckFailedException, AccessDeniedException) and service-specific exception types. Catch specific exceptions, not a base exception, because handling ConditionalCheckFailedException as business logic and ThrottlingException as a retry are different behaviours that a broad catch conflates.

Tip: Log the AWS request ID on every failure. Every response carries one, in the response metadata or the exception. It is the identifier AWS Support needs to investigate anything, and it is unrecoverable after the fact - a support case without it starts with "we cannot find your request."

It also lets you correlate an SDK-side failure with CloudTrail. Logging it costs one field and it is the difference between a support case that goes somewhere and one that does not.

To see the actual HTTP traffic, enable the SDK's debug logging - boto3.set_stream_logger in Python, the equivalent in each SDK. When behaviour is genuinely inexplicable, the signed request and raw response usually explain it in seconds.

Common Failure Modes

Warning: A client per request. Credential resolution and pool setup on every call.

Clients constructed inside the Lambda handler.

Stale environment variables overriding a profile. Check env | grep AWS first.

Region resolution failing. Usually the chain, not the network.

Ignoring pagination. Silently processes a subset, never errors.

page["Contents"] on a page with no results. KeyError on an empty page.

Retry amplification across SDK, application, and framework.

Default timeouts inside a Lambda. Invocation killed mid-retry with no logged reason.

Catching a broad exception. Retryable and business-logic failures handled identically.

Request IDs not logged. Support cases with nothing to investigate.

Hand-rolled polling instead of waiters. No backoff, no timeout, no failure detection.

Scan with a filter where a Query would work. Pay for the full read.

Why This Matters In An Interview

Interview: These are the questions that separate people who have shipped against AWS from people who have followed tutorials, because every one of them is learned by being caught out.

What interviewers are listening for:

The credentials chain order, and environment variables sitting above profiles.

Pagination as a silent-failure risk rather than a performance concern.

Retry amplification across layers.

Client reuse, and specifically the Lambda handler placement.

Common follow-ups:

"Your job processed 1,000 records and there were 40,000. What happened?" → Pagination ignored. ListObjectsV2 caps at 1,000 per call and returns a continuation token. It does not error, which is why it survived testing.

"You specified --profile prod and got an access denied for a different account. Why?" → Environment variables sit above profiles in the credentials chain. Something exported keys into your shell.

"Your Lambda times out with nothing useful in the logs." → SDK timeouts times retry attempts exceeding the function's budget. The invocation is killed mid-retry so nothing logs a failure. Set explicit timeouts sized to fit inside the function's.

"How should retries be handled?" → Let the SDK do it - it implements exponential backoff with jitter correctly - and do not layer application retries on top without reducing the SDK's count. Know the total attempt budget.

Practice

Exercise: Starter. Enable SDK debug logging and make one API call. Read the signed request and the response, and find the request ID.

Intermediate. Write code that lists a bucket with more than 1,000 objects both without a paginator and with one. Confirm the first silently returns a subset.

Advanced. Set an artificially low read timeout and a high retry count in a Lambda with a short timeout. Observe that the function is killed with no SDK error logged, then configure timeouts to fit inside the function's budget and confirm you now get a proper exception.