Object Counting with SageMaker Object Detection

Labeling images with Ground Truth, training a SageMaker object detection model, and counting objects from the inference results.

Takahiro Iwasa

2023-07-18

7 min read

This example covers the following workflow:

Labeling images using Ground Truth
Training and deploying an object detection model
Performing inference

ℹ️ Note

Images in this post are illustrative and not associated with specific customer projects.

The example uses a SageMaker inference endpoint to test the trained model.

Labeling with Ground Truth

Creating a Labeling Workforce

Create a private workforce for the labeling task. Team members can authenticate with Amazon Cognito or an OIDC identity provider.

After the workforce is created, each worker receives an invitation email containing the labeling portal URL.

You can also retrieve the labeling portal URL by navigating to Private workforce summary > Labeling portal sign-in URL in the SageMaker management console.

Workers must follow the instructions in the invitation email to sign up and access the labeling portal.

An example of the invitation email is as follows:

1
Hi,
2

3
You are invited by [email protected] from <COMPANY> to work on a labeling project.
4

5
Click on the link below to log into your labeling project.
6
"https://<LABELING_PORTAL_URL>"
7

8
You will need the following username and temporary password provided below to login for the first time.
9
User name: <USER_NAME>
10
Temporary password: <PASSWORD>
11

12
Once you log in with your temporary password, you will be required to create a new password for your account.
13
After creating a new password, you can log into your private team to access your labeling project.
14

15
If you have any questions, please contact us at [email protected].

After accessing the URL, workers must enter the username and temporary password from the invitation email.

They will then be prompted to change their temporary password to a new one.

After signing in, workers see their assigned jobs on the labeling portal home page.

Creating a Labeling Job

Return to the SageMaker console and create a labeling job. Enter the required settings and click Complete data setup.

💡 Tip

After creating the labeling job, it cannot be deleted. Use a unique value, such as one generated with the following command: uuidgen | tr "[:upper:]" "[:lower:]".

For complex labeling tasks, consider specifying a longer value for the Task timeout parameter.

Starting Labeling

ℹ️ Note

It may take some time for the labeling job to appear in the list.

Follow the job instructions to draw bounding boxes and assign labels. The following image shows an example.

Once all workers have completed their tasks, stop the labeling job.

Checking Labeling Output

After the labeling job stops, Ground Truth writes its output to the configured S3 bucket. For this object detection task, the training data is in manifests/output/output.manifest. See the official documentation for the output structure.

annotation-tool/
annotations/
consolidated-annotation/
worker-response/
manifests/
intermediate/
output/
output.manifest
temp/

Ground Truth stores the labeling results as an Augmented Manifest. See the official documentation for its object detection fields.

Training with SageMaker

After labeling is complete, configure the SageMaker training job as follows:

Job settings
Job name: Use a unique value (e.g., uuidgen | tr "[:upper:]" "[:lower:]").
Algorithm source: SageMaker built-in algorithm
Choose an algorithm:
Algorithm: Vision - Object Detection (MXNet)
Input mode: Pipe
Resource configuration
Instance type: Use a GPU instance like ml.p2.xlarge.
Only GPU instances support SageMaker object detection algorithms.
Hyperparameters
num_classes: Set to the number of object classes (e.g., 1 in this post).
num_training_samples: Equal to the number of lines in the manifest file.
Input data configuration
Training channel
Channel name: train
Input mode: Pipe
Content type: application/x-recordio
Record wrapper: RecordIO
Data source: S3 (Augmented Manifest File)
Attribute names: Include attributes like source-ref and bounding box data keys.
S3 location: Specify the S3 URI for the training data manifest file.
Validation channel
Channel name: validation
Output data configuration
S3 location: Specify the S3 URI for storing model artifacts.

With an Augmented Manifest, the object detection algorithm can use Pipe input mode and the RecordIO wrapper without a separately generated RecordIO file. See the official documentation for details.

Inference

Creating a Model from the Training Job

To create a model from the completed training job, click Create model in the SageMaker console.

Deploying the Model

After creating the model, click Create endpoint to deploy it. For infrequent inference that fits its resource limits, consider a serverless endpoint.

Making Requests

Locate the SageMaker Runtime endpoint on the endpoint detail page. For testing, invoke it with the AWS SDK or a SigV4-capable client such as Postman.

🔥 Caution

SageMaker real-time endpoints can serve production inference, but they require AWS authentication. Do not expose AWS credentials in an untrusted client; place an authenticated application or API layer in front of the endpoint when necessary. The direct Postman call below is for testing.

Example: Postman Configuration

Configure AWS Signature Version 4 authentication with the following values:

AccessKey
SecretKey
Session Token: Use temporary credentials instead of long-term credentials.
AWS Region: The region of your SageMaker endpoint.
Service Name: sagemaker

Set Accept: application/json and the image content type expected by the model, such as Content-Type: application/x-image.

Send the image as the binary request body.

Example: Using AWS SDK (boto3)

The following example invokes the endpoint with boto3 invoke_endpoint:

1
import json
2

3
import boto3
4

5
# Initialize SageMaker runtime client
6
runtime = boto3.client('sagemaker-runtime')
7

8
# Define endpoint and input details
9
endpoint_name = '<YOUR_ENDPOINT_NAME>'
10
content_type = 'application/x-image'
11
payload = None
12

13
# Read the image file in binary mode
14
with open('/path/to/image.jpg', 'rb') as f:
15
    payload = f.read()
16

17
# Invoke the endpoint
18
response = runtime.invoke_endpoint(
19
    EndpointName=endpoint_name,
20
    ContentType=content_type,
21
    Body=payload
22
)
23

24
# Parse and display the response
25
body = response['Body'].read()
26
predictions = json.loads(body.decode())
27
print(json.dumps(predictions, indent=2))
28

29
# Save the response to a file
30
with open('./response.json', 'w') as f:
31
    json.dump(predictions, f, indent=2)

Checking Response

The JSON response contains the following values for each detection:

Class Label IDs
Confidence Scores
Bounding Box Coordinates

ℹ️ Note

The bounding box coordinates are relative to the actual image dimensions. For more details, refer to the official documentation.

1
{
2
  "prediction": [
3
    [
4
      0.0,
5
      0.9953756332397461,
6
      0.3821756839752197,
7
      0.007661208510398865,
8
      0.525381863117218,
9
      0.19436971843242645
10
    ],
11
    [
12
      0.0,
13
      0.9928023219108582,
14
      0.3435703217983246,
15
      0.23781903088092804,
16
      0.5533013343811035,
17
      0.6385164260864258
18
    ],
19
    [
20
      0.0,
21
      0.9911478757858276,
22
      0.15510153770446777,
23
...
24
      0.9990172982215881
25
    ]
26
  ]
27
}

Visualizing Response

To interpret the results of the inference visually, you can use Jupyter Notebook along with matplotlib.

The following Python script overlays the detected bounding boxes and count labels on the input image.

1
import json
2

3
import matplotlib.patches as patches
4
import matplotlib.pyplot as plt
5
from PIL import Image
6

7
# Configure plot
8
plt.figure()
9
axes = plt.axes()
10

11
# Read an image
12
im = Image.open('/path/to/image.jpg')
13
# Display the image
14
plt.imshow(im)
15

16
# Read SageMaker inference predictions
17
with open('response.json') as f:
18
    predictions = json.loads(f.read())['prediction']
19

20
# Set initial count
21
count = 0
22

23
# Create rectangles
24
for prediction in predictions:
25
    score = prediction[1]
26
    if score < 0.2:
27
        continue
28

29
    # Count up
30
    count += 1
31

32
    x = prediction[2] * im.width
33
    y = prediction[3] * im.height
34
    width = prediction[4] * im.width - x
35
    height = prediction[5] * im.height - y
36

37
    rect = patches.Rectangle((x, y), width, height, linewidth=1, edgecolor='r', facecolor='none')
38
    axes.annotate(count, (x + width / 2, y + height / 2), color='yellow', weight='bold', fontsize=18, ha='center', va='center')
39
    axes.add_patch(rect)
40

41
# Display the rectangles
42
plt.show()

The script filters detections by confidence score, converts normalized coordinates to pixels, draws each bounding box, and assigns a sequential count label.

Conclusion

Ground Truth labels and the SageMaker built-in object detection algorithm produce normalized bounding boxes that can be filtered, counted, and visualized with matplotlib.

The score < 0.2 filter determines which detections contribute to the count. Tune this confidence threshold against validation data instead of choosing it from a single test image.

Count accuracy also depends on consistent Ground Truth annotations, especially for overlapping or partially visible objects. Review labeling guidelines and validation results before using the counts at scale.