How should serverless applications handle secrets and encryption?

Separate secrets from ordinary configuration

A database password, API credential, signing material, or private key is secret. A feature flag, table name, region, or public endpoint usually is not. Classification determines storage, access, logging, rotation, and incident response.

Do not place secrets in source control, build logs, deployment output, URLs, event payloads, or plaintext configuration files. Environment variables are configuration delivery, not a complete secret-management lifecycle.

Retrieve secrets with workload identity

Grant the function or task role permission to retrieve a specific secret and, when required, use its KMS key. Retrieve it at runtime through the AWS SDK or a managed integration rather than shipping long-lived AWS credentials.

Caching can reduce latency and API calls, but define refresh, rotation, memory exposure, and failure behavior. Never log the returned value, authorization header, or decrypted payload during debugging.

Understand envelope encryption and key policy

Secrets Manager uses AWS KMS keys and data keys to encrypt secret values. A customer-managed KMS key can provide more granular policy and cross-account control, while the service-managed default may be sufficient for simpler same-account use.

KMS permission is separate from permission to call Secrets Manager. Scope both policies, use service and encryption-context conditions where appropriate, and audit decrypt activity for sensitive paths.

Design rotation and compromise response

Rotation changes a credential and every dependent system must tolerate the transition. Test overlap, connection refresh, failed rotation, rollback, and the time required for cached consumers to use the new value.

Document how to revoke a secret immediately, identify consumers, rotate downstream credentials, redeploy or flush caches, and verify that unauthorized use has stopped. Rotation automation is incomplete without recovery and evidence.