Boto3 Session Management: Writing Correct Python Code

Clients, Resources, and Sessions

Boto3 exposes three levels of abstraction for interacting with AWS:

Session: The root object. Holds credentials and configuration. All clients and resources are created from a session.

Client: Low-level interface. Maps 1:1 to the AWS REST API. Returns raw dictionaries (JSON-like). Always complete and up-to-date with new AWS features.

import boto3

s3 = boto3.client('s3')
response = s3.list_buckets()
# response is a dict: {'Buckets': [{'Name': '...', 'CreationDate': ...}], ...}

Resource: High-level, object-oriented interface. More Pythonic. Returns objects with methods and attributes instead of raw dicts.

s3 = boto3.resource('s3')
bucket = s3.Bucket('my-bucket')
bucket.upload_file('local.txt', 'remote.txt')  # cleaner API

When to use which: Use resource for common operations where it exists (S3, EC2, DynamoDB, SQS). Use client when you need access to newer API features not yet in the resource interface, or when you need precise control over the raw API response.

The Global Session Trap in Multi-Threaded Code

boto3.client('s3') uses a module-level default session. For single-threaded scripts, this is fine. For multi-threaded applications (e.g., a web server handling concurrent requests, or a ThreadPoolExecutor processing files in parallel), sharing a single session across threads can cause race conditions in credential refresh logic.

Unsafe pattern (multi-threaded context):

import boto3
from concurrent.futures import ThreadPoolExecutor

# This client is shared across all threads — unsafe
s3 = boto3.client('s3')

def upload_file(filename):
    s3.upload_file(filename, 'my-bucket', filename)  # shared client

with ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(upload_file, file_list)

Safe pattern — create a client per thread:

import boto3
from concurrent.futures import ThreadPoolExecutor

def upload_file(filename):
    # Each thread creates its own session and client
    session = boto3.Session()
    s3 = session.client('s3')
    s3.upload_file(filename, 'my-bucket', filename)

with ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(upload_file, file_list)

Explicit Session for Testability

Creating an explicit session also makes your code easier to test. You can inject a mock session in unit tests without patching global state:

def list_buckets(session: boto3.Session) -> list:
    s3 = session.client('s3')
    response = s3.list_buckets()
    return [b['Name'] for b in response['Buckets']]

# In production
session = boto3.Session(profile_name='dev')
buckets = list_buckets(session)

# In tests (using moto or a mock)
mock_session = create_mock_session()
buckets = list_buckets(mock_session)

Key Point: The boto3.Session() constructor accepts profile_name, region_name, aws_access_key_id, aws_secret_access_key, and aws_session_token as explicit parameters. Passing credentials directly into the Session constructor is useful in Lambda functions that retrieve secrets from AWS Secrets Manager to assume cross-account roles — a common pattern in multi-account automation.