Developing Greengrass Components Running in Greengrass Core in Docker

Takahiro Iwasa

Feb 28, 2023

8 min read

Greengrass Greengrass Development Kit IoT

This post describes how to develop Greengrass components in our local environment using the Docker image of AWS IoT Greengrass. Please refer to the official documentation for more details.

docs.aws.amazon.com

Run AWS IoT Greengrass Core software in a Docker container - AWS IoT Greengrass

Install and run AWS IoT Greengrass Core software v2.0 in a Docker container.

In this post, we will develop a Greengrass component that sends messages to AWS IoT Core via MQTT every second. This component will be deployed to a Docker container locally using Greengrass CLI.

At the end of this post, the project directory structure will look like the following:

/
|-- components/
|   `-- mqtt_publisher/
|        |-- .gitignore
|        |-- gdk-config.json
|        |-- main.py
|        |-- recipe.yaml
|        `-- requirements.txt
`-- docker/
  |-- greengrass-v2-credentials/
  |   `-- credentials
  |-- .env
  `-- docker-compose.yml

We will develop a Greengrass component that publishes messages to AWS IoT Core via MQTT.

Install Greengrass Development Kit (GDK).

docs.aws.amazon.com

AWS IoT Greengrass Development Kit Command-Line Interface - AWS IoT Greengrass

Use the AWS IoT Greengrass Development Kit Command-Line Interface to create AWS IoT Greengrass V2 components from templates and publish them to the AWS IoT Greengrass service.

$ pip install -U git+https://github.com/aws-greengrass/[email protected]

Start developing by running gdk component init.

docs.aws.amazon.com

component - AWS IoT Greengrass

Use the component command in the AWS IoT Greengrass Development Kit Command-Line Interface to create, build, and publish AWS IoT Greengrass V2 components.

$ mkdir ./components
$ gdk component init \
  --language python \
  --template HelloWorld \
  --name components/mqtt_publisher

The command generates the following files. The src and tests directories will not be used in this post.

./
|-- components/
|   |   |-- mqtt_publisher/
|   |   |   |-- src/
|   |   |   |   |-- greeter.py
|   |   |   |-- tests/
|   |   |   |   |-- test_greeter.py
|   |   |   |-- .gitignore
|   |   |   |-- gdk-config.json
|   |   |   |-- main.py
|   |   |   |-- README.md
|   |   |   |-- recipe.yaml

Configure the component metadata in gdk-config.json as follows. Because the component is not published using gdk component publish, you do not need to set your S3 bucket to publish.bucket.

Also refer to the official documentation for the configuration file.

docs.aws.amazon.com

Greengrass Development Kit CLI configuration file - AWS IoT Greengrass

Configure AWS IoT Greengrass V2 component source folders to use them with the Greengrass Development Kit CLI, which enables you to initialize, build, and publish components.

{
  "component": {
    "com.example.MqttPublisher": {
      "author": "wasabee.dev",
      "version": "0.0.1",
      "build": {
        "build_system": "zip"
      },
      "publish": {
        "bucket": "<PLACEHOLDER_BUCKET>",
        "region": "ap-northeast-1"
      }
    }
  },
  "gdk_version": "1.0.0"
}

Create main.py with the following code. This script will publish MQTT messages to the topic /mqtt-publisher every second.

import json
import random
from datetime import datetime
from time import sleep

import boto3

client = boto3.client('iot-data')


def main():
    payload = {
        "value": random.randint(1, 10000),
        "datetime": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
    }
    while True:
        client.publish(
            topic='/mqtt-publisher',
            payload=json.dumps(payload).encode(),
            qos=1,
            contentType='application/json',
        )
        print(f'Message was sent successfully: {payload}')
        sleep(1)


if __name__ == "__main__":
    main()

Create requirements.txt with the following content. The dependencies listed in this file will be installed during the component installation, which is specified in recipe.yaml described below.

boto3==1.26.65

Create recipe.yaml with the following content. Please refer to the official documentation for recipe specification.

docs.aws.amazon.com

AWS IoT Greengrass component recipe reference - AWS IoT Greengrass

Use the recipe reference to find explanations and examples for how to write component recipes for AWS IoT Greengrass V2.

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "This is an mqtt publisher written in Python."
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentDependencies:
  aws.greengrass.TokenExchangeService:
    VersionRequirement: '^2.0.0'
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/mqtt_publisher.zip"
        Unarchive: ZIP
    Lifecycle:
      Install: "pip3 install --user -r {artifacts:decompressedPath}/mqtt_publisher/requirements.txt"
      Run: "python3 -u {artifacts:decompressedPath}/mqtt_publisher/main.py"

You can specify Greengrass components at ComponentDependencies that must be installed with your custom component. Because the example Python script uses boto3, specify aws.greengrass.TokenExchangeService.

docs.aws.amazon.com

Interact with AWS services - AWS IoT Greengrass

Use the token exchange service to acquire AWS credentials that let custom AWS IoT Greengrass V2 components interact with AWS services.

AWS IoT Greengrass provides a public component, the token exchange service component, that you can define as a dependency in your custom component to interact with AWS services. The token exchange service provides your component with an environment variable, AWS_CONTAINER_CREDENTIALS_FULL_URI, that defines the URI to a local server that provides AWS credentials.

You can use lifecycle hooks to install Python libraries listed in requirements.txt.

The following placeholders will be replaced with the values configured in gdk-config.json during gdk component build.

• {COMPONENT_NAME}
• {COMPONENT_VERSION}
• {COMPONENT_AUTHOR}
• Artifacts.URI
  • BUCKET_NAME
  • COMPONENT_NAME
  • COMPONENT_VERSION

Build the component with gdk component build.

docs.aws.amazon.com

component - AWS IoT Greengrass

Use the component command in the AWS IoT Greengrass Development Kit Command-Line Interface to create, build, and publish AWS IoT Greengrass V2 components.

$ cd components/mqtt_publisher
$ gdk component build

After building the component, the built artifact is generated in the greengrass-build directory. You do not need to run gdk component publish, as the component will be deployed to a Docker container locally.

Work in <PROJECT_ROOT>/docker directory.

When setting up Greengrass Core in a Docker container with automatic provisioning mode, the Greengrass Core installer needs to use an AWS security credential to provision the necessary AWS resources. Although you can use an AWS permanent credential, I strongly recommend temporary credentials using sts get-session-token.

docs.aws.amazon.com

Run AWS IoT Greengrass in a Docker container with automatic resource provisioning - AWS IoT Greengrass

Start a Docker container that runs the AWS IoT Greengrass Core software v2.0 with automatically provisioned AWS resources.

The following AWS resources will be automatically provisioned:

• AWS IoT
  • Greengrass Core Device
  • IoT Thing
  • IoT Thing Group
  • Certificate
  • Policies (x2)
  • Token Exchange Role Alias
• AWS IAM
  • Token Exchange Role
  • Token Exchange Role Policy

Get a temporary credential with the following command.

$ aws sts get-session-token

Create a credential file used by the Greengrass Core installer.

$ mkdir ./greengrass-v2-credentials
$ nano ./greengrass-v2-credentials/credentials

Here is the example.

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token = AQoEXAMPLEH4aoAH0gNCAPy...truncated...zrkuWJOgQs8IZZaIv2BXIa2R4Olgk

Create .env to configure environment variables used by the Greengrass Core installer. Fore more information, please refer to the official documentation

docs.aws.amazon.com

Run AWS IoT Greengrass in a Docker container with automatic resource provisioning - AWS IoT Greengrass

Start a Docker container that runs the AWS IoT Greengrass Core software v2.0 with automatically provisioned AWS resources.

GGC_ROOT_PATH=/greengrass/v2
AWS_REGION=ap-northeast-1
PROVISION=true
THING_NAME=MyGreengrassCore
THING_GROUP_NAME=MyGreengrassCoreGroup
TES_ROLE_NAME=GreengrassV2TokenExchangeRole
TES_ROLE_ALIAS_NAME=GreengrassCoreTokenExchangeRoleAlias
COMPONENT_DEFAULT_USER=ggc_user:ggc_group

Create docker-compose.yml with the following content. For more information, please refer to the official documentation.

docs.aws.amazon.com

Run AWS IoT Greengrass in a Docker container with automatic resource provisioning - AWS IoT Greengrass

Start a Docker container that runs the AWS IoT Greengrass Core software v2.0 with automatically provisioned AWS resources.

version: '3.7'

services:
  greengrass:
    init: true
    container_name: aws-iot-greengrass
    image: amazon/aws-iot-greengrass:latest
    volumes:
      - ./greengrass-v2-credentials:/root/.aws/:ro
      - ../components:/root/components
    env_file: .env
    ports:
      - '8883:8883'

Run the container with the following command.

$ docker-compose up -d
$ docker-compose logs -f greengrass
...
aws-iot-greengrass | Launching Nucleus...
aws-iot-greengrass | Launched Nucleus successfully.

Deploy two AWS-provided components to your Docker container.

To deploy components locally, install Greengrass CLI component (aws.greengrass.Cli).

docs.aws.amazon.com

Greengrass Command Line Interface - AWS IoT Greengrass

Use the Greengrass CLI to perform local deployments and debugging functions on your AWS IoT Greengrass V2 core device.

We recommend that you use this component in only development environments, not production environments. This component provides access to information and operations that you typically won’t need in a production environment. Follow the principle of least privilege by deploying this component to only core devices where you need it.

After installing, you should find it in /greengrass/v2/bin.

$ docker-compose exec greengrass bash

$ cd /greengrass/v2
$ ls bin
greengrass-cli  greengrass-cli.cmd

Greengrass custom components interact with AWS using Token Exchange Service component (aws.greengrass.TokenExchangeService). It runs an ECS container instance as a local server that connects to the AWS IoT credentials provider using the Token Exchange Role Alias. When custom components create AWS SDK clients, the client use the local server to obtain temporary credentials.

docs.aws.amazon.com

Interact with AWS services - AWS IoT Greengrass

Use the token exchange service to acquire AWS credentials that let custom AWS IoT Greengrass V2 components interact with AWS services.

Greengrass core devices use X.509 certificates to connect to AWS IoT Core using TLS mutual authentication protocols. These certificates let devices interact with AWS IoT without AWS credentials, which typically comprise an access key ID and a secret access key.

You can deploy the AWS-provided components from the AWS IoT Greengrass console.

After deploying, you should see the following log records in /greengrass/v2/logs/greengrass.log.

[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=SUCCEEDED, ThingName=MyGreengrassCore, JobId=}
[INFO] (pool-2-thread-11) com.aws.greengrass.status.FleetStatusService: fss-status-update-published. Status update published to FSS. {trigger=THING_GROUP_DEPLOYMENT, serviceName=FleetStatusService,
[INFO] (pool-2-thread-11) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-success}
[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyGreengrassCore}

Because the example Python script publishes messages to AWS IoT Core, attach the following IAM policy to GreengrassV2TokenExchangeRole. Before saving, replace <AWS_ACCOUNT_ID> with your own AWS account ID.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iot:Connect",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "iot:Publish",
            "Resource": "arn:aws:iot:*:<AWS_ACCOUNT_ID>:topic//mqtt-publisher*"
        }
    ]
}

Attach the policy with the following command.

$ aws iam put-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-name IoTPolicy \
  --policy-document file://policy.json

Deploy your custom component locally with greengrass-cli deployment create in the Docker container.

docs.aws.amazon.com

deployment - AWS IoT Greengrass

Describes the deployment command for Greengrass CLI.

$ cd /greengrass/v2
$ bin/greengrass-cli deployment create \
  --recipeDir /root/components/mqtt_publisher/greengrass-build/recipes \
  --artifactDir /root/components/mqtt_publisher/greengrass-build/artifacts \
  --merge "com.example.MqttPublisher=0.0.1"
...
Local deployment submitted! Deployment Id: <DEPLOYMENT_ID>

Check the deployment status using greengrass-cli deployment status.

docs.aws.amazon.com

deployment - AWS IoT Greengrass

Describes the deployment command for Greengrass CLI.

$ bin/greengrass-cli deployment status -i <DEPLOYMENT_ID>
...
INFO: Connection established with event stream RPC server
<DEPLOYMENT_ID>: SUCCEEDED

You should see the following log record in /greengrass/v2/logs/com.example.MqttPublisher.log.

$ cd /greengrass/v2/logs
$ tail -f com.example.MqttPublisher.log
...
[INFO] (Copier) com.example.MqttPublisher: stdout. Message was sent successfully: {'value': 31, 'datetime': '2023-02-27 12:31:35'}. {scriptName=services.com.example.MqttPublisher.lifecycle.Run, serviceName=com.example.MqttPublisher, currentState=RUNNING}

Using the MQTT test client, input /# or /mqtt-publisher in the Topic filter field, click Subscribe button. You should see the messages sent from your custom component to AWS IoT Core.

With the AWS IoT Greengrass Docker image, you can easily develop Greengrass components.

I hope you will find this post useful.