Integrating SageMaker Inference Endpoint with API Gateway REST API

Takahiro Iwasa

Jul 13, 2023

3 min read

API Gateway SageMaker

SageMaker offers its inference endpoint. Users can access it not only directly but also indirectly through API Gateway’s REST API using Integration Request.

docs.aws.amazon.com

Set up an integration request in API Gateway - Amazon API Gateway

To set up an integration request, you perform the following required and optional tasks:

The example below uses no AWS Lambda functions.

You can find your inference endpoint in the SageMaker console by navigating to Endpoint summary > URL.

The endpoint format is https://runtime.sagemaker.<ENDPOINT_REGION>.amazonaws.com/endpoints/<ENDPOINT_NAME>/invocations. Because you are required to include a valid Authorization header, this endpoint works properly despite lack of your AWS account id.

docs.aws.amazon.com

InvokeEndpoint - Amazon SageMaker

After you deploy a model into production using Amazon SageMaker hosting services, your client applications use this API to get inferences from the model hosted at the specified endpoint.

Endpoints are scoped to an individual account, and are not public. The URL does not contain the account ID, but Amazon SageMaker determines the account ID from the authentication token that is supplied by the caller.

Select REST API.

Specify a name for the API.

Select Actions -> Create Method.

Select a method type. POST is used in the example below.

Specify the required parameters according to the following table.

Creating the method completed.

If your model expects binary data as input, add the MIME type like image/* to Binary Media Types.

If the MIME type is not added, you should see the following response.

{
    "ErrorCode": "CLIENT_ERROR_FROM_MODEL",
    "LogStreamArn": "arn:aws:logs:ap-northeast-1:xxxxxxxxxxxx:log-group:/aws/sagemaker/Endpoints/<ENDPOINT_NAME>",
    "Message": "Received client error (400) from primary with message \"unable to evaluate payload provided\". See https://ap-northeast-1.console.aws.amazon.com/cloudwatch/home?region=ap-northeast-1#logEventViewer:group=/aws/sagemaker/Endpoints/<ENDPOINT_NAME> in account xxxxxxxxxxxx for more information.",
    "OriginalMessage": "unable to evaluate payload provided",
    "OriginalStatusCode": 400
}

Deploy the API.

After deploying, you can find the API endpoint.

Access the API endpoint with the following command.

curl --location '<API_ENDPOINT>' \
  --header 'Content-Type: image/jpeg' \
  --header 'Accept: application/json' \
  --data-binary '@/path/to/image.jpg'

Integrating a SageMaker inference endpoint with a REST API will lead to low codes and reduced costs by using no Lambda functions. However, if requests and responses need to be transformed with complexity, it would be better to consider AWS Lambda functions based on the use case because they can avoid the complexity.

I hope you will find this post useful.