Workflow with AWS Step Functions and Amazon Rekognition

Aug 15, 2024

In this workflow, it will be displayed a state machine using AWS Step Functions to detect the labels of images that are either already in or uploaded to a bucket, utilizing Amazon Rekognition label detection. The images will be stored by label in another bucket, and their metadata will be saved in DynamoDB.

https://github.com/Cuchuflim/AWS-StepFunctions-Amazon-Rekognition/tree/main

For this project, we will use the US East 1 (N. Virginia) region because it allows 50 transactions per second with Amazon Rekognition.

https://docs.aws.amazon.com/general/latest/gr/rekognition.html#limits_rekognition

The Amazon States Language provides several intrinsic functions, also known as intrinsics, that help you perform basic data processing operations without using a Task state.

https://docs.aws.amazon.com/step-functions/latest/dg/intrinsic-functions.html

Initial Preparation

Create S3 Buckets:

Bucket 1: For uploading and storing initial images.
Bucket 2: For storing images organized by labels.

Create a DynamoDB Table:

Partition Key: key_id (string)
Sort key: label (string)

Create AWS Lambda Function:

Create a Lambda Function:
- Create a Lambda function to capture the bucket name and image through a trigger (event) each time an image is uploaded to the bucket.
Assign Necessary Permissions to the Lambda Role:
- AmazonS3FullAccess
- AWSLambdaBasicExecutionRole
- AWSStepFunctionsFullAccess
Enrich or Extract Information (Optional):
- Within the Lambda function, you can enrich the data or extract the specific information needed before triggering the state machine.
Connect with AWS Step Functions:
- Use boto3 to connect with AWS Step Functions and execute the state machine, providing its ARN.

Create Rule in Amazon EventBridge:

Open the Amazon EventBridge Console:
- Navigate to the Amazon EventBridge console in AWS.
Create a New Rule:
- Click on "Create rule".
Define Rule Details:
- Rule name: Provide a meaningful name for your rule.
- Rule type: Select "Rule with an event pattern".
Configure the Event Pattern:
- Sample event type: Choose "AWS events".
- Sample events: Select "S3 Object Created".
Creation method:
- Select "Use pattern form".
Define Event Source and Event Pattern:
- Event source: Select "AWS services".
- AWS service: Choose "Simple Storage Service (S3)".
- Event type: Choose "Amazon S3 event notification".
Specify the Event Details:
- Event name: Choose "Object Created".
- Bucket name: Specify the name of the S3 bucket you want to monitor.
Set the Target:
- Target: Select "Step Function state machine" as the target for the event.
- State machine: Choose the state machine you want to trigger.
Assign Permissions:
- Execution role: Select the default execution role or use an existing one that has the necessary permissions to trigger the Step Function.
Additional Settings (Optional):
- Add any additional settings or customizations as needed.
Review and Create:
- Review your configuration and click "Create" to finalize the rule.

This rule will now monitor the specified S3 bucket for object creation events and trigger the selected Step Function state machine accordingly.

First State Machine

In this case, the images are already pre-loaded in the S3 bucket. When the State Machine is executed, a list of the files found in the bucket will be generated, and the workflow will be executed for each one of them.

Create the Workflow in AWS Step Functions:

Assign necessary permissions to the Step Functions role
- AmazonDynamoDBFullAccess
- AmazonRekognitionFullAccess
- AmazonS3FullAccess
- AWSLambda_FullAccess
- AWSLambdaBasicExecutionRole
- CloudWatchLogsDeliveryFullAccessPolicy
- XRayAccessPolicy
Define the workflow steps:
- Use listObjects to get a list of all objects in the bucket.
- Use Map State to iterate through the list, object by object.
- Get image attributes.
- Use Rekognition to get image labels.
- Use a Choice State to identify if a label has a parent name.
- Copy the image to a new bucket organized by label.
- Load the metadata into a DynamoDB table.

Second State Machine

When an image is uploaded to the S3 bucket, an event will be generated that triggers a Lambda function. This function will send the bucket name and file name to Step Functions, thereby initiating the workflow. Alternatively, you can use Amazon EventBridge to capture the event and trigger the Step Functions state machine directly, bypassing the need for an intermediate Lambda function if only event forwarding is required.

Create the State Machine in AWS Step Functions:

Assign necessary permissions to the Step Functions role:
Define the workflow steps:
- Get image attributes using GetObjectAttributes.
- Use Rekognition to get image labels.
- Use a Choice State to identify if a label has a parent name.
- Copy the image to a new bucket organized by label.
- Load the metadata into a DynamoDB table.

Third State Machine

For the third state machine, a custom labeling will be created using Amazon Rekognition, with the goal of obtaining the specific labels that are required.

Create S3 Buckets:

Bucket: For uploading images to build a dataset for model training.
Bucket Policy: Update the bucket policy with the following configuration to allow Rekognition to access the images in the dataset.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AWSRekognitionS3AclBucketRead20191011",
            "Effect": "Allow",
            "Principal": {
                "Service": "rekognition.amazonaws.com"
            },
            "Action": [
                "s3:GetBucketAcl",
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::bucket-name"
        },
        {
            "Sid": "AWSRekognitionS3GetBucket20191011",
            "Effect": "Allow",
            "Principal": {
                "Service": "rekognition.amazonaws.com"
            },
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:GetObjectTagging"
            ],
            "Resource": "arn:aws:s3:::bucket-name/*"
        },
        {
            "Sid": "AWSRekognitionS3ACLBucketWrite20191011",
            "Effect": "Allow",
            "Principal": {
                "Service": "rekognition.amazonaws.com"
            },
            "Action": "s3:GetBucketAcl",
            "Resource": "arn:aws:s3:::bucket-name"
        },
        {
            "Sid": "AWSRekognitionS3PutObject20191011",
            "Effect": "Allow",
            "Principal": {
                "Service": "rekognition.amazonaws.com"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::bucket-name/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }
    ]
}

Amazon Rekognition Custom Labels

Project Creation: Create project is initiated.
Dataset Generation: The dataset is created using the images stored in the bucket.
Label Detection: You can choose to automatically detect labels based on the folders in the bucket or select them manually.
Model Training: The model is trained using the created dataset.
Model Deployment: The model is launched for use through an API or, in this case, via Step Functions."

Configure AWS Lambda or Amazon EventBridge:

same process as the first state machine.

Create the Workflow in AWS Step Functions:

Assign necessary permissions to the Step Functions role
Define the workflow steps:
- Get image attributes using GetObjectAttributes.
- Use Rekognition to get image Custom labels.
- Use a Choice State to identify if a label has a parent name.
- Copy the image to a new bucket organized by label.
- Load the metadata into a DynamoDB table.

Monitoring and Verification

Monitor Executions:

Use the Step Functions console to verify the status and execution of the workflows.
Use CloudWatch Logs to review logs generated by Lambda functions and Step Functions.

Review Metadata in DynamoDB:

Verify that the table contains the metadata of processed images.

This workflow ensures that images are processed and labeled correctly, and their metadata is stored appropriately in DynamoDB for subsequent querying and analysis.

Henry’s Substack

Discussion about this post