Building and Deploying Greengrass Components in a Dockerized Environment

Takahiro Iwasa

2/28/2023

8 min read

Greengrass Greengrass Development Kit IoT

This note describes how to develop AWS IoT Greengrass components in a local environment using the Docker image of Greengrass Core. For more details, refer to the official documentation.

In this guide, we’ll build a Greengrass component that sends messages to AWS IoT Core via MQTT every second. The component will be deployed to a Docker container locally using the Greengrass CLI.

By the end of this example, your project directory will look like this:

components/
├── mqtt_publisher/
│   ├── .gitignore
│   ├── gdk-config.json
│   ├── main.py
│   ├── recipe.yaml
│   ├── requirements.txt
docker/
├── greengrass-v2-credentials/
│   ├── credentials
├── .env
├── docker-compose.yml

Developing Greengrass Custom Component

Installing Greengrass Development Kit (GDK)

Install the Greengrass Development Kit (GDK):

pip install -U git+https://github.com/aws-greengrass/[email protected]

Starting Development

Initialize your Greengrass component by running gdk component init:

mkdir ./components
gdk component init \
  --language python \
  --template HelloWorld \
  --name components/mqtt_publisher

This command generates a basic component structure with the following files and directories:

components/
├── mqtt_publisher/
│   ├── src/
│   │   ├── greeter.py
│   ├── tests/
│   │   ├── test_greeter.py
│   ├── .gitignore
│   ├── gdk-config.json
│   ├── main.py
│   ├── README.md
│   ├── recipe.yaml

Configuring Component Metadata

Update the gdk-config.json file with the component metadata. If the component is not being published to an S3 bucket using gdk component publish, the publish.bucket field (line 10) does not need to be set.

Here’s an example of the updated gdk-config.json:

1
{
2
  "component": {
3
    "com.example.MqttPublisher": {
4
      "author": "wasabee.dev",
5
      "version": "0.0.1",
6
      "build": {
7
        "build_system": "zip"
8
      },
9
      "publish": {
10
        "bucket": "<PLACEHOLDER_BUCKET>",
11
        "region": "ap-northeast-1"
12
      }
13
    }
14
  },
15
  "gdk_version": "1.0.0"
16
}

For additional details, refer to the official documentation on the GDK CLI configuration file.

Writing Python Script

Create a main.py script for your component. This script will publish MQTT messages to the /mqtt-publisher topic every second.

1
import json
2
import random
3
from datetime import datetime
4
from time import sleep
5

6
import boto3
7

8
client = boto3.client('iot-data')
9

10

11
def main():
12
    payload = {
13
        "value": random.randint(1, 10000),
14
        "datetime": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
15
    }
16
    while True:
17
        client.publish(
18
            topic='/mqtt-publisher',
19
            payload=json.dumps(payload).encode(),
20
            qos=1,
21
            contentType='application/json',
22
        )
23
        print(f'Message was sent successfully: {payload}')
24
        sleep(1)
25

26

27
if __name__ == "__main__":
28
    main()

Create a requirements.txt file to specify the required dependencies for your component. These dependencies will be installed during the component installation process as defined in the recipe.yaml.

1
boto3==1.26.65

Component Recipe

Define your component’s recipe by creating a recipe.yaml file. The recipe specifies metadata, dependencies, and lifecycle hooks for the component. For more information about the component recipe specification, refer to the official documentation.

Here’s an example:

1
---
2
RecipeFormatVersion: "2020-01-25"
3
ComponentName: "{COMPONENT_NAME}"
4
ComponentVersion: "{COMPONENT_VERSION}"
5
ComponentDescription: "This is an mqtt publisher written in Python."
6
ComponentPublisher: "{COMPONENT_AUTHOR}"
7
ComponentDependencies:
8
  aws.greengrass.TokenExchangeService:
9
    VersionRequirement: '^2.0.0'
10
Manifests:
11
  - Platform:
12
      os: all
13
    Artifacts:
14
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/mqtt_publisher.zip"
15
        Unarchive: ZIP
16
    Lifecycle:
17
      Install: "pip3 install --user -r {artifacts:decompressedPath}/mqtt_publisher/requirements.txt"
18
      Run: "python3 -u {artifacts:decompressedPath}/mqtt_publisher/main.py"

Component Dependencies

This script communicates with AWS IoT Core using boto3. Therefore, specify aws.greengrass.TokenExchangeService component as a dependency in the ComponentDependencies section. The TokenExchangeService runs a local server that provides AWS credentials for your custom component.

For more details, refer to the official documentation.

AWS IoT Greengrass provides a public component, the token exchange service component, that you can define as a dependency in your custom component to interact with AWS services. The token exchange service provides your component with an environment variable, AWS_CONTAINER_CREDENTIALS_FULL_URI, that defines the URI to a local server that provides AWS credentials.

Lifecycle Hooks

The Lifecycle section specifies commands to execute during component installation and at runtime:

• Install: Installs Python libraries listed in requirements.txt.
• Run: Executes the main.py script when the component starts.

Placeholders in Recipe

Placeholders in the recipe (e.g., {COMPONENT_NAME}) are replaced with values from gdk-config.json during the build process. These placeholders include:

• {COMPONENT_NAME}
• {COMPONENT_VERSION}
• {COMPONENT_AUTHOR}
• Artifacts URI (BUCKET_NAME, COMPONENT_NAME, and COMPONENT_VERSION)

Building the Component

Use gdk component build to build the component with the Greengrass Development Kit:

cd components/mqtt_publisher
gdk component build

After building, the artifacts will be located in the greengrass-build directory. There is no need to run gdk component publish for local deployment to a Docker container.

Greengrass Core in Docker

This section explains how to set up Greengrass Core in a Docker container, configure credentials, and deploy components.

From here, work in <PROJECT_ROOT>/docker directory.

Security Credentials

Greengrass Core requires AWS security credentials to provision resources automatically. While permanent credentials can be used, temporary credentials via sts get-session-token are highly recommended for enhanced security.

The following AWS resources will be provisioned:

• AWS IoT
  • Greengrass Core Device
  • IoT Thing
  • IoT Thing Group
  • Certificate
  • Policies (two)
  • Token Exchange Role Alias
• AWS IAM
  • Token Exchange Role
  • Token Exchange Role Policy

Generate temporary credentials:

aws sts get-session-token

Save the credentials to a file:

mkdir ./greengrass-v2-credentials
nano ./greengrass-v2-credentials/credentials

Example content for credentials:

1
[default]
2
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
3
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
4
aws_session_token = AQoEXAMPLEH4aoAH0gNCAPy...truncated...zrkuWJOgQs8IZZaIv2BXIa2R4Olgk

Dot Env File

Create a .env file to configure environment variables for the Greengrass Core installer. Refer to the official documentation for more details.

Example .env file:

1
GGC_ROOT_PATH=/greengrass/v2
2
AWS_REGION=ap-northeast-1
3
PROVISION=true
4
THING_NAME=MyGreengrassCore
5
THING_GROUP_NAME=MyGreengrassCoreGroup
6
TES_ROLE_NAME=GreengrassV2TokenExchangeRole
7
TES_ROLE_ALIAS_NAME=GreengrassCoreTokenExchangeRoleAlias
8
COMPONENT_DEFAULT_USER=ggc_user:ggc_group

Running Greengrass Core

Create a docker-compose.yml file to run Greengrass Core in Docker. Refer to the documentation for further information.

Example docker-compose.yml:

1
version: '3.7'
2

3
services:
4
  greengrass:
5
    init: true
6
    container_name: aws-iot-greengrass
7
    image: amazon/aws-iot-greengrass:latest
8
    volumes:
9
      - ./greengrass-v2-credentials:/root/.aws/:ro
10
      - ../components:/root/components
11
    env_file: .env
12
    ports:
13
      - '8883:8883'

Run the container:

docker-compose up -d
docker-compose logs -f greengrass

You should see logs indicating the Nucleus has successfully launched:

1
aws-iot-greengrass | Launching Nucleus...
2
aws-iot-greengrass | Launched Nucleus successfully.

Deploying AWS-provided Components

Greengrass CLI

Install the Greengrass CLI component (aws.greengrass.Cli) for local deployments. After installation, it can be found in /greengrass/v2/bin.

docker-compose exec greengrass bash
cd /greengrass/v2
ls bin

Token Exchange Service

Deploy the aws.greengrass.TokenExchangeService component to enable your custom component to interact with AWS. This service provides temporary credentials via a local server.

https://docs.aws.amazon.com/greengrass/v2/developerguide/interact-with-aws-services.html

Greengrass core devices use X.509 certificates to connect to AWS IoT Core using TLS mutual authentication protocols. These certificates let devices interact with AWS IoT without AWS credentials, which typically comprise an access key ID and a secret access key.

Deploying from AWS IoT Greengrass Console

Deploy AWS-provided components, including Greengrass Nucleus, via the AWS IoT Greengrass Console.

Once deployed, you should see success logs in /greengrass/v2/logs/greengrass.log.

1
[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=SUCCEEDED, ThingName=MyGreengrassCore, JobId=}
2
[INFO] (pool-2-thread-11) com.aws.greengrass.status.FleetStatusService: fss-status-update-published. Status update published to FSS. {trigger=THING_GROUP_DEPLOYMENT, serviceName=FleetStatusService,
3
[INFO] (pool-2-thread-11) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-success}
4
[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyGreengrassCore}

Updating Token Exchange Role

Update the GreengrassV2TokenExchangeRole IAM policy to grant permissions for MQTT publishing:

1
{
2
  "Version": "2012-10-17",
3
  "Statement": [
4
    {
5
      "Effect": "Allow",
6
      "Action": "iot:Connect",
7
      "Resource": "*"
8
    },
9
    {
10
      "Effect": "Allow",
11
      "Action": "iot:Publish",
12
      "Resource": "arn:aws:iot:*:<AWS_ACCOUNT_ID>:topic//mqtt-publisher*"
13
    }
14
  ]
15
}

Attach the policy:

aws iam put-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-name IoTPolicy \
  --policy-document file://policy.json

Deploying Custom Component Locally

Deploy your custom component using the Greengrass CLI greengrass-cli deployment create inside the Docker container.

cd /greengrass/v2
bin/greengrass-cli deployment create \
  --recipeDir /root/components/mqtt_publisher/greengrass-build/recipes \
  --artifactDir /root/components/mqtt_publisher/greengrass-build/artifacts \
  --merge "com.example.MqttPublisher=0.0.1"

Check the deployment status using greengrass-cli deployment status:

bin/greengrass-cli deployment status -i <DEPLOYMENT_ID>

You should see the response:

1
INFO: Connection established with event stream RPC server
2
<DEPLOYMENT_ID>: SUCCEEDED

Monitor logs to ensure the component is running:

cd /greengrass/v2/logs
tail -f com.example.MqttPublisher.log

Expected log output:

1
[INFO] (Copier) com.example.MqttPublisher: stdout. Message was sent successfully: {'value': 31, 'datetime': '2023-02-27 12:31:35'}. {scriptName=services.com.example.MqttPublisher.lifecycle.Run, serviceName=com.example.MqttPublisher, currentState=RUNNING}

Testing with AWS IoT Test Client

Use the MQTT test client in the AWS IoT Console to verify that messages are being published to the /mqtt-publisher topic.

1. Navigate to the MQTT test client.
2. Input /# or /mqtt-publisher in the Topic filter field.
3. Click the Subscribe button.

You should see the published messages from your custom component.