Calling SageMaker from API Gateway Without Lambda

Wiring API Gateway's integration request directly to a SageMaker inference endpoint, cutting the Lambda function out of the request path entirely.

Takahiro Iwasa

2023-07-13

3 min read

API Gateway’s integration request feature can call a SageMaker inference endpoint directly, eliminating the need for a Lambda function in between.

Diagram Overview

Finding SageMaker Inference Endpoint

Navigate to SageMaker Console and go to Endpoint summary > URL.

Endpoint Example

The endpoint format is as follows:

1
https://runtime.sagemaker.<ENDPOINT_REGION>.amazonaws.com/endpoints/<ENDPOINT_NAME>/invocations

ℹ️ Note

Ensure a valid Authorization header is included for the endpoint to work correctly. Read more on the official documentation.

Endpoints are scoped to an individual account, and are not public. The URL does not contain the account ID, but Amazon SageMaker determines the account ID from the authentication token that is supplied by the caller.

Building REST API Integrated with SageMaker Inference Endpoint

Select REST API in the API Gateway Console.

Assign a name to your API.

API Name Setup

Select Actions -> Create Method.

Choose the HTTP method type (POST is used in this example).

Method Type

Configure integration request:

Integration type: AWS Service
AWS Service: SageMaker Runtime (NOT SageMaker)
HTTP method: POST
Action Type: Use path override
Path override: endpoints/<ENDPOINT_NAME>/invocations
Execution role: IAM role for API (must include the sagemaker:InvokeEndpoint action)
Content Handling: Passthrough

Integration Request Configuration

If your model accepts binary input (e.g., images), add the relevant MIME type (e.g., image/*) in the Binary Media Types.

Binary Media Configuration

Without this configuration, you may encounter the following error:

1
{
2
  "ErrorCode": "CLIENT_ERROR_FROM_MODEL",
3
  "LogStreamArn": "arn:aws:logs:ap-northeast-1:xxxxxxxxxxxx:log-group:/aws/sagemaker/Endpoints/<ENDPOINT_NAME>",
4
  "Message": "Received client error (400) from primary with message \"unable to evaluate payload provided\". See https://ap-northeast-1.console.aws.amazon.com/cloudwatch/home?region=ap-northeast-1#logEventViewer:group=/aws/sagemaker/Endpoints/<ENDPOINT_NAME> in account xxxxxxxxxxxx for more information.",
5
  "OriginalMessage": "unable to evaluate payload provided",
6
  "OriginalStatusCode": 400
7
}

Select Deploy API in the API Gateway console.

Assign a stage for the deployment.

Deployment Steps

Once deployed, the API endpoint will be available for testing.

API Endpoint Example

Testing

The deployed API can be tested with curl:

curl --location '<API_ENDPOINT>' \
  --header 'Content-Type: image/jpeg' \
  --header 'Accept: application/json' \
  --data-binary '@/path/to/image.jpg'

Conclusion

Configuring API Gateway’s integration request to call a SageMaker inference endpoint directly, with the right path override and execution role, made image inference requests work with no Lambda function in the middle. Wiring API Gateway directly to the SageMaker Runtime service removes a Lambda function that would otherwise just forward the request body and headers unchanged, which cuts out one more hop of latency and one more piece of infrastructure to maintain. The tradeoff is that Lambda would normally be the natural place to validate or reshape a request before it reaches the model — without it, getting the Binary Media Types and Content Handling settings exactly right becomes the only line of defense against the “unable to evaluate payload provided” error shown above, so it’s worth testing with the exact content type the model expects before assuming the integration is broken.