Migrating to the AWS SDK for Go V2
Minimum Go Version
The AWS SDK for Go V2 requires a minimum version of Go 1.15. Migration from AWS SDK for Go to AWS SDK for Go V2 might require you to upgrade your application by one or more Go versions. The latest version of Go can be downloaded on the Downloads page. See the Release History for more information about each Go version release, and relevant information required for upgrading.
Modularization
The AWS SDK for Go V2 has been updated to take advantage of the Go modules which became the default development mode in Go 1.13. A number of packages provided by the SDK have been modularized and are independently versioned and released respectively. This change enables improved application dependency modeling, and enables the SDK to provide new features and functionality that follows the Go module versioning strategy.
The following list are some Go modules provided by the SDK:
Module | Description |
---|---|
github.com/aws/aws-sdk-go-v2 |
The SDK core |
github.com/aws/aws-sdk-go-v2/config |
Shared Configuration Loading |
github.com/aws/aws-sdk-go-v2/credentials |
AWS Credential Providers |
github.com/aws/aws-sdk-go-v2/feature/ec2/imds |
Amazon EC2 Instance Metadata Service Client |
The SDK’s service clients and higher level utilities modules are nested under the following import paths:
Import Root | Description |
---|---|
github.com/aws/aws-sdk-go-v2/service/ |
Service Client Modules |
github.com/aws/aws-sdk-go-v2/feature/ |
High-Level utilities for services, for example the Amazon S3 Transfer Manager |
Configuration Loading
The session package and associated functionality are replaced with a simplified
configuration system provided by the config package. The config
package is a separate Go
module, and can be included in your applications dependencies by with go get
.
go get github.com/aws/aws-sdk-go-v2/config
The session.New, session.NewSession, NewSessionWithOptions, and session.Must must be migrated to config.LoadDefaultConfig.
The config
package provides several helper functions that aid in overriding the shared configuration loading
programmatically. These function names are prefixed with With
followed by option that they override. Let’s look at
some examples of how to migrate usage of the session
package.
For more information on loading shared configuration see Configuring the AWS SDK for Go V2.
Examples
Migrating from NewSession to LoadDefaultConfig
The following example shows how usage of session.NewSession
without additional argument parameters is migrated to
config.LoadDefaultConfig
.
// V1 using NewSession
import "github.com/aws/aws-sdk-go/aws/session"
// ...
sess, err := session.NewSession()
if err != nil {
// handle error
}
// V2 using LoadDefaultConfig
import "context"
import "github.com/aws/aws-sdk-go-v2/config"
// ...
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// handle error
}
Migrating from NewSession with aws.Config options
The example shows how to migrate overriding of aws.Config
values during configuration loading. One or more
config.With*
helper functions can be provided to config.LoadDefaultConfig
to override the loaded configuration
values. In this example the AWS Region is overridden to us-west-2
using the
config.WithRegion helper function.
// V1
import "github.com/aws/aws-sdk-go/aws"
import "github.com/aws/aws-sdk-go/aws/session"
// ...
sess, err := session.NewSession(aws.Config{
Region: aws.String("us-west-2")
})
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/config"
// ...
cfg, err := config.LoadDefaultConfig(context.TODO(),
config.WithRegion("us-west-2"),
)
if err != nil {
// handle error
}
Migrating from NewSessionWithOptions
This example shows how to migrate overriding values during configuration loading. Zero or more
config.With*
helper functions can be provided to config.LoadDefaultConfig
to override the loaded configuration
values. In this example we show how to override the target profile that is used when loading the AWS SDK shared
configuration.
// V1
import "github.com/aws/aws-sdk-go/aws"
import "github.com/aws/aws-sdk-go/aws/session"
// ...
sess, err := session.NewSessionWithOptions(aws.Config{
Profile: "my-application-profile"
})
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/config"
// ...
cfg, err := config.LoadDefaultConfig(context.TODO(),
config.WithSharedConfigProfile("my-application-profile"),
)
if err != nil {
// handle error
}
Credentials & Credential Providers
The aws/credentials package and associated credential providers have been
relocated to the credentials package location. The credentials
package is a Go module that
you retrieve by using go get
.
go get github.com/aws/aws-sdk-go-v2/credentials
The AWS SDK for Go V2 release updates the AWS Credential Providers to provide a consistent interface for retrieving
AWS Credentials. Each provider implements the aws.CredentialsProvider
interface, which defines a Retrieve
method that returns a (aws.Credentials, error)
.
aws.Credentials that is analogous to the AWS SDK for Go
credentials.Value type.
You must wrap aws.CredentialsProvider
objects with aws.CredentialsCache to
allow credential caching to occur. You use NewCredentialsCache to
construct a aws.CredentialsCache
object. By default, credentials configured by config.LoadDefaultConfig
are wrapped
with aws.CredentialsCache
.
The following table list the location changes of the AWS credential providers from AWS SDK for Go V1 to AWS SDK for Go V2.
Name | V1 Import | V2 Import |
---|---|---|
Amazon EC2 IAM Role Credentials | github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds |
github.com/aws/aws-sdk-go-v2/credentials/ec2rolecreds |
Endpoint Credentials | github.com/aws/aws-sdk-go/aws/credentials/endpointcreds |
github.com/aws/aws-sdk-go-v2/credentials/endpointcreds |
Process Credentials | github.com/aws/aws-sdk-go/aws/credentials/processcreds |
github.com/aws/aws-sdk-go-v2/credentials/processcreds |
AWS Security Token Service | github.com/aws/aws-sdk-go/aws/credentials/stscreds |
github.com/aws/aws-sdk-go-v2/credentials/stscreds |
Static Credentials
Applications that use credentials.NewStaticCredentials to construct static credential programmatically must use credentials.NewStaticCredentialsProvider.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/credentials"
// ...
appCreds := credentials.NewStaticCredentials(accessKey, secretKey, sessionToken)
value, err := appCreds.Get()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/aws"
import "github.com/aws/aws-sdk-go-v2/credentials"
// ...
appCreds := aws.NewCredentialsCache(credentials.NewStaticCredentialsProvider(accessKey, secretKey, sessionToken))
value, err := appCreds.Retrieve(context.TODO())
if err != nil {
// handle error
}
Amazon EC2 IAM Role Credentials
You must migrate usage of NewCredentials and NewCredentialsWithClient to use New.
The ec2rolecreds
package’s ec2rolecreds.New
takes functional options of
ec2rolecreds.Options as
input, allowing you override the specific Amazon EC2 Instance
Metadata Service client to use, or to override the credential expiry window.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/credentials/ec2rolecreds"
// ...
appCreds := ec2rolecreds.NewCredentials(sess)
value, err := appCreds.Get()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/aws"
import "github.com/aws/aws-sdk-go-v2/credentials/ec2rolecreds"
// ...
// New returns an object of a type that satisfies the aws.CredentialProvider interface
appCreds := aws.NewCredentialsCache(ec2rolecreds.New())
value, err := appCreds.Retrieve(context.TODO())
if err != nil {
// handle error
}
Endpoint Credentials
You must migrate usage of NewCredentialsClient and NewProviderClient to use New.
The endpointcreds
package’s New
function takes a string argument containing the
URL of an HTTP or HTTPS endpoint to retrieve credentials from, and functional
options of endpointcreds.Options to mutate the credentials provider and
override specific configuration settings.
Process Credentials
You must migrate usage of NewCredentials, NewCredentialsCommand, and NewCredentialsTimeout to use NewProvider or NewProviderCommand.
The processcreds
package’s NewProvider
function takes a string argument that is the
command to be executed in the host environment’s shell, and functional options
of Options to mutate the
credentials provider and override specific configuration settings.
NewProviderCommand
takes an implementation of the
NewCommandBuilder interface that defines
more complex process commands that might take one or more command-line arguments, or have certain execution environment
requirements. DefaultNewCommandBuilder implements
this interface, and defines a command builder for a process that requires multiple command-line arguments.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/credentials/processcreds"
// ...
appCreds := processcreds.NewCredentials("/path/to/command")
value, err := appCreds.Get()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/aws"
import "github.com/aws/aws-sdk-go-v2/credentials/processcreds"
// ...
appCreds := aws.NewCredentialsCache(processcreds.NewProvider("/path/to/command"))
value, err := appCreds.Retrieve(context.TODO())
if err != nil {
// handle error
}
AWS Security Token Service Credentials
AssumeRole
You must migrate usage of NewCredentials, and NewCredentialsWithClient to use NewAssumeRoleProvider.
The stscreds
package’s NewAssumeRoleProvider
function must be called with
a sts.Client, and the AWS Identity and Access Management Role ARN to be assumed from the provided sts.Client
’s
configured credentials. You can also provide a set of functional options of
AssumeRoleOptions to
modify other optional settings of the provider.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/credentials/stscreds"
// ...
appCreds := stscreds.NewCredentials(sess, "arn:aws:iam::123456789012:role/demo")
value, err := appCreds.Get()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/credentials/stscreds"
// ...
client := sts.NewFromConfig(cfg)
appCreds := stscreds.NewAssumeRoleProvider(client, "arn:aws:iam::123456789012:role/demo")
value, err := appCreds.Retrieve(context.TODO())
if err != nil {
// handle error
}
AssumeRoleWithWebIdentity
You must migrate usage of NewWebIdentityCredentials, NewWebIdentityRoleProvider, and NewWebIdentityRoleProviderWithToken to use NewWebIdentityRoleProvider.
The stscreds
package’s NewWebIdentityRoleProvider
function must be called with a sts.Client, and the
AWS Identity and Access Management Role ARN to be assumed using the provided sts.Client
’s configured credentials, and an
implementation of a IdentityTokenRetriever for
providing the OAuth 2.0 or OpenID Connect ID token.
IdentityTokenFile is an IdentityTokenRetriever
that can
be used to provide the web identity token from a file located on the application’s host file-system.
You can also provide a set of functional options of
WebIdentityRoleOptions to modify other optional
settings for the provider.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/credentials/stscreds"
// ...
appCreds := stscreds.NewWebIdentityRoleProvider(sess, "arn:aws:iam::123456789012:role/demo", "sessionName", "/path/to/token")
value, err := appCreds.Get()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/aws"
import "github.com/aws/aws-sdk-go-v2/credentials/stscreds"
// ...
client := sts.NewFromConfig(cfg)
appCreds := aws.NewCredentialsCache(stscreds.NewWebIdentityRoleProvider(
client,
"arn:aws:iam::123456789012:role/demo",
stscreds.IdentityTokenFile("/path/to/file"),
func(o *stscreds.WebIdentityRoleOptions) {
o.RoleSessionName = "sessionName"
}))
value, err := appCreds.Retrieve(context.TODO())
if err != nil {
// handle error
}
Service Clients
AWS SDK for Go V2 provides service client modules nested under the github.com/aws/aws-sdk-go-v2/service
import path.
Each service client is contained in a Go package using each service’s unique identifier. The following table provides
some examples of service import paths in the AWS SDK for Go V2.
Service Name | V1 Import Path | V2 Import Path |
---|---|---|
Amazon S3 | github.com/aws/aws-sdk-go/service/s3 |
github.com/aws/aws-sdk-go-v2/service/s3 |
Amazon DynamoDB | github.com/aws/aws-sdk-go/service/dynamodb |
github.com/aws/aws-sdk-go-v2/service/dynamodb |
Amazon CloudWatch Logs | github.com/aws/aws-sdk-go/service/cloudwatchlogs |
github.com/aws/aws-sdk-go-v2/service/cloudwatchlogs |
Each service client package is an independently versioned Go module. To add the service client as a dependency of your
application, use the go get
command with the service’s import path. For example, to add the Amazon S3
client to your dependencies use
go get github.com/aws/aws-sdk-go-v2/service/s3
Client Construction
You can construct clients in the AWS SDK for Go V2 using either the New
or NewFromConfig
constructor functions in the client’s package.
When migrating from the AWS SDK for Go we recommend that you use the NewFromConfig
variant, which will return a new
service client using values from an aws.Config
. The aws.Config
value will have been created while loading the SDK shared configuration using
config.LoadDefaultConfig
. For details on creating service clients see
Using AWS Services.
Example 1
// V1
import "github.com/aws/aws-sdk-go/aws/session"
import "github.com/aws/aws-sdk-go/service/s3"
// ...
sess, err := session.NewSession()
if err != nil {
// handle error
}
client := s3.New(sess)
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/config"
import "github.com/aws/aws-sdk-go-v2/service/s3"
// ...
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// handle error
}
client := s3.NewFromConfig(cfg)
Example 2: Overriding Client Settings
// V1
import "github.com/aws/aws-sdk-go/aws"
import "github.com/aws/aws-sdk-go/aws/session"
import "github.com/aws/aws-sdk-go/service/s3"
// ...
sess, err := session.NewSession()
if err != nil {
// handle error
}
client := s3.New(sess, &aws.Config{
Region: aws.String("us-west-2"),
})
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/config"
import "github.com/aws/aws-sdk-go-v2/service/s3"
// ...
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
// handle error
}
client := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.Region = "us-west-2"
})
Endpoints
The endpoints package no longer exists in the AWS SDK for Go V2. Each service client now embeds its required AWS endpoint metadata within the client package. This reduces the overall binary size of compiled applications by no longer including endpoint metadata for services not used by your application.
By default, service clients use their configured AWS Region to resolve the service endpoint for the target Region. If
your application requires a custom endpoint to be specified for a particular service and region, you can specify
a custom aws.EndpointResolver using the EndpointResolver
field on the
aws.Config
structure. If your application implements a custom
endpoints.Resolver you must migrate it to conform to the
aws.EndpointResolver
interface. aws.EndpointResolverFunc is provided as
a convenient way to wrap a resolver function to satisfy the aws.EndpointResolver
interface.
For more information on endpoints and implementing a custom resolver, see Configuring Client Endpoints.
Invoking API Operations
The number of service client operation methods have been reduced significantly. The <OperationName>Request
,
<OperationName>WithContext
, and <OperationName>
methods have all been consolidated into single operation method, <OperationName>
.
Example
The following example shows how calls to the <nil> PutObject operation would be migrated from AWS SDK for Go to AWS SDK for Go V2.
// V1
import "context"
import "github.com/aws/aws-sdk-go/service/s3"
// ...
client := s3.New(sess)
// Pattern 1
output, err := client.PutObject(&s3.PutObjectInput{
// input parameters
})
// Pattern 2
output, err := client.PutObjectWithContext(context.TODO(), &s3.PutObjectInput{
// input parameters
})
// Pattern 3
req, output := client.PutObjectRequest(context.TODO(), &s3.PutObjectInput{
// input parameters
})
err := req.Send()
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/service/s3"
// ...
client : s3.NewFromConfig(cfg)
output, err := client.PutObject(context.TODO(), &s3.PutObjectInput{
// input parameters
})
Service Data Types
The top-level input and output types of an operation are found in the service client package. The input and output type
for a given operation follow the pattern of <OperationName>Input
and <OperationName>Output
, where OperationName
is the name of the operation you are invoking. For example the input and output shape for the
Amazon S3 PutObject operation are PutObjectInput and
PutObjectOutput respectively.
All other service data types, other than input and output types, have been migrated to the types
package located
under the service client package import path hierarchy. For example, the
s3.AccessControlPolicy type is now located at
types.AccessControlPolicy.
Enumeration Values
The SDK now provides a typed experience for all API enumeration fields. Rather than using a string literal value copied
from the service API reference documentation, you can now use one of the concrete types found in the service client’s
types
package. For example, you can provide the Amazon S3 PutObjectInput operation
with an ACL to be applied on an object. In the AWS SDK for Go V1, this parameter was a *string
type. In the
{{& alias sdk-go %}} this parameter is now a
types.ObjectCannedACL. The types
package
provides generated constants for the valid enumeration values that can be assigned to this field. For example
types.ObjectCannedACLPrivate is the constant for the
“private” canned ACL value. This value can be used in place of managing string constants within your application.
Pointer Parameters
The AWS SDK for Go v1 required pointer references to be passed for all input parameters to service operations. The
AWS SDK for Go V2 has simplified the experience with most services by removing the need to pass input values as
pointers where possible. This change means that many service clients operations no longer require your application
to pass pointer references for the following types: uint8
, uint16
, uint32
, int8
, int16
, int32
, float32
,
float64
, bool
. Similarly, slice and map element types have been updated accordingly to reflect whether their
elements must be passed as pointer references.
The aws package contains helper functions for creating pointers for the Go built-in types, these
helpers should be used to more easily handle creating pointer types for these Go types. Similarly, helper methods are
provided for safely de-referencing pointer values for these types. For example, the
aws.String function converts from string
⇒ *string
. Inversely,
the aws.ToString converts from *string
⇒ string
. When upgrading your
application from AWS SDK for Go V1 to AWS SDK for Go V2, you must migrate usage of the helpers for
converting from the pointer types to the non-pointer variants. For example,
aws.StringValue must be updated to aws.ToString
.
Errors Types
The AWS SDK for Go V2 takes full advantage of the error wrapping functionality
introduced in Go 1.13. Services that model error responses have generated
types available in their client’s types
package that can be used to test whether a client operation error was caused
by one of these types. For example Amazon S3 GetObject
operation can return a NoSuchKey
error if
attempting to retrieve an object key that doesn’t exist. You can use errors.As to
test whether the returned operation error is a types.NoSuchKey error.
In the event a service does not model a specific type for an error, you can utilize the
smithy.APIError interface type for inspecting the returned error code and message
from the service. This functionality replaces awserr.Error and the other
awserr functionality from the AWS SDK for Go V1. For more details information on
handling errors see Handling Errors.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/awserr"
import "github.com/aws/aws-sdk-go/service/s3"
// ...
client := s3.New(sess)
output, err := s3.GetObject(&s3.GetObjectInput{
// input parameters
})
if err != nil {
if awsErr, ok := err.(awserr.Error); ok {
if awsErr.Code() == "NoSuchKey" {
// handle NoSuchKey
} else {
// handle other codes
}
return
}
// handle a error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/service/s3"
import "github.com/aws/aws-sdk-go-v2/service/s3/types"
import "github.com/aws/smithy-go"
// ...
client := s3.NewFromConfig(cfg)
output, err := s3.GetObject(context.TODO(), &s3.GetObjectInput{
// input parameters
})
if err != nil {
var nsk *types.NoSuchKey
if errors.As(err, &nsk) {
// handle NoSuchKey error
return
}
var apiErr smithy.APIError
if errors.As(err, &apiErr) {
code := apiErr.ErrorCode()
message := apiErr.ErrorMessage()
// handle error code
return
}
// handle error
return
}
Paginators
Service operation paginators are no longer invoked as methods on the service client. To use a paginator for an operation
you must construct a paginator for an operation using one of the paginator constructor methods. For example,
to use paginate over the Amazon S3 ListObjectsV2
operation you must construct its paginator using the
s3.NewListObjectsV2Paginator. This constructor returns a
ListObjectsV2Paginator which provides the methods HasMorePages
,
and NextPage
for determining whether there are more pages to retrieve and invoking the operation to retrieve the next
page respectively. More details on using the SDK paginators can be found at
.
Let’s look at an example of how to migrate from a AWS SDK for Go paginator to the AWS SDK for Go V2 equivalent.
Example
// V1
import "fmt"
import "github.com/aws/aws-sdk-go/service/s3"
// ...
client := s3.New(sess)
params := &s3.ListObjectsV2Input{
// input parameters
}
totalObjects := 0
err := client.ListObjectsV2Pages(params, func(output *s3.ListObjectsV2Output, lastPage bool) bool {
totalObjects += len(output.Contents)
return !lastPage
})
if err != nil {
// handle error
}
fmt.Println("total objects:", totalObjects)
// V2
import "context"
import "fmt"
import "github.com/aws/aws-sdk-go-v2/service/s3"
// ...
client := s3.NewFromConfig(cfg)
params := &s3.ListObjectsV2Input{
// input parameters
}
totalObjects := 0
paginator := s3.NewListObjectsV2Paginator(client, params)
for paginator.HasMorePages() {
output, err := paginator.NextPage(context.TODO())
if err != nil {
// handle error
}
totalObjects += len(output.Contents)
}
fmt.Println("total objects:", totalObjects)
Waiters
Service operation waiters are no longer invoked as methods on the service client. To use a waiter you first construct
the desired waiter type, and then invoke the wait method. For example,
to wait for a Amazon S3 Bucket to exist, you must construct a BucketExists
waiter. Use the
s3.NewBucketExistsWaiter constructor to create a
s3.BucketExistsWaiter. The s3.BucketExistsWaiter
provides a
Wait
method which can be used to wait for a bucket to become available.
Features
Amazon EC2 Instance Metadata Service
The AWS SDK for Go V2 provides an Amazon EC2 Instance Metadata Service (IMDS) client that you can use to query the local IMDS when executing your application on an Amazon EC2 instance. The IMDS client is a separate Go module that can be added to your application by using
go get github.com/aws/aws-sdk-go-v2/feature/ec2/imds
The client constructor and method operations have been updated to match the design of the other SDK service clients.
Example
// V1
import "github.com/aws/aws-sdk-go/aws/ec2metadata"
// ...
client := ec2metadata.New(sess)
region, err := client.Region()
if err != nil {
// handle error
}
// V2
import "context"
import "github.com/aws/aws-sdk-go-v2/feature/ec2/imds"
// ...
client := imds.NewFromConfig(cfg)
region, err := client.GetRegion(context.TODO())
if err != nil {
// handle error
}
Amazon S3 Transfer Manager
The Amazon S3 transfer manager is available for managing uploads and downloads of objects
concurrently. This package is located in a Go module outside the service client import path. This module
can be retrieved by using go get github.com/aws/aws-sdk-go-v2/feature/s3/manager
.
s3.NewUploader and s3.NewUploaderWithClient have been replaced with the constructor method manager.NewUploader for creating an Upload manager client.
s3.NewDownloader and s3.NewDownloaderWithClient have been replaced with a single constructor method manager.NewDownloader for creating a Download manager client.
Amazon CloudFront Signing Utilities
The AWS SDK for Go V2 provides Amazon CloudFront signing utilities in a Go module outside the service
client import path. This module can be retrieved by using go get
.
go get github.com/aws/aws-sdk-go-v2/feature/cloudfront/sign