Building an IoT backend from scratch means operating a message broker that handles millions of persistent connections, a device registry that tracks which devices exist and what state they are in, a certificate authority for device authentication, and a rules engine that routes device data to storage, analytics, and business logic. Each of these is a significant infrastructure problem individually.
AWS IoT Core handles all of them as a managed service. In this article I will walk through provisioning devices with certificates, the MQTT message broker, the rules engine for routing device data to downstream services, and Device Jobs for managing firmware updates across a fleet.
Device Provisioning and Certificates
Every device authenticates to IoT Core using an X.509 certificate. Each device has its own unique certificate. Certificates can be revoked individually when a device is decommissioned without affecting any other device. This is the correct model for IoT security: shared credentials mean compromising one device compromises all of them.
resource "aws_iot_certificate" "device" {
active = true
}
resource "aws_iot_thing" "sensor" {
name = "temperature-sensor-001"
attributes = {
location = "building-a-floor-3"
device_type = "temperature-sensor"
firmware_version = "2.1.0"
}
}
resource "aws_iot_policy" "device_policy" {
name = "temperature-sensor-policy"
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = ["iot:Connect"]
Resource = "arn:aws:iot:${var.region}:${data.aws_caller_identity.current.account_id}:client/temperature-sensor-*"
},
{
Effect = "Allow"
Action = ["iot:Publish"]
Resource = "arn:aws:iot:${var.region}:${data.aws_caller_identity.current.account_id}:topic/sensors/temperature/*"
},
{
Effect = "Allow"
Action = ["iot:Subscribe"]
Resource = "arn:aws:iot:${var.region}:${data.aws_caller_identity.current.account_id}:topicfilter/devices/temperature-sensor-001/commands"
},
{
Effect = "Allow"
Action = ["iot:Receive"]
Resource = "arn:aws:iot:${var.region}:${data.aws_caller_identity.current.account_id}:topic/devices/temperature-sensor-001/commands"
}
]
})
}
resource "aws_iot_thing_principal_attachment" "sensor_cert" {
principal = aws_iot_certificate.device.arn
thing = aws_iot_thing.sensor.name
}
resource "aws_iot_policy_attachment" "device_policy" {
policy = aws_iot_policy.device_policy.name
target = aws_iot_certificate.device.arn
}
The IoT policy restricts the device to connecting with only client IDs that match its pattern, publishing to only its own sensor topic subtree, subscribing to only its own command topic, and receiving from only its own command topic. A compromised device with this policy cannot read data from other devices or publish to topics it should not have access to. This granular topic-level authorization is one of the most important security controls in an IoT system.
Device Shadow for State Management
import boto3
import json
iot_data = boto3.client("iot-data", region_name="us-east-1")
def update_device_desired_state(thing_name: str, desired_state: dict):
payload = json.dumps({
"state": {
"desired": desired_state
}
})
iot_data.update_thing_shadow(
thingName = thing_name,
payload = payload.encode()
)
def get_device_state(thing_name: str) -> dict:
response = iot_data.get_thing_shadow(thingName=thing_name)
shadow = json.loads(response["payload"].read())
return shadow["state"]
update_device_desired_state(
thing_name = "temperature-sensor-001",
desired_state = {
"reporting_interval_seconds": 30,
"alert_threshold_celsius": 28.0
}
)
The Device Shadow is a JSON document that represents the last known state of a device and the desired state you want it to reach. When you update the desired state, IoT Core stores the delta and delivers it to the device when it next connects. The device reports its actual state back as the reported section. This pattern handles the fundamental challenge of IoT: devices are offline frequently and commands must be queued rather than sent synchronously.
Rules Engine for Data Routing
resource "aws_iot_topic_rule" "temperature_to_timestream" {
name = "temperature_sensor_to_timestream"
sql = "SELECT *, topic(3) as device_id FROM 'sensors/temperature/+'"
sql_version = "2016-03-23"
enabled = true
timestream {
role_arn = aws_iam_role.iot_timestream.arn
database_name = "iot-metrics"
table_name = "temperature-readings"
dimension {
name = "device_id"
value = "${device_id}"
}
dimension {
name = "location"
value = "${location}"
}
timestamp {
unit = "MILLISECONDS"
value = "${timestamp}"
}
}
error_action {
sqs {
role_arn = aws_iam_role.iot_error.arn
queue_url = aws_sqs_queue.iot_errors.url
use_base64 = false
}
}
}
resource "aws_iot_topic_rule" "high_temperature_alert" {
name = "high_temperature_alert"
sql = "SELECT *, topic(3) as device_id FROM 'sensors/temperature/+' WHERE temperature > 35"
sql_version = "2016-03-23"
enabled = true
sns {
role_arn = aws_iam_role.iot_sns.arn
target_arn = aws_sns_topic.temperature_alerts.arn
message_format = "JSON"
}
}
The rules engine evaluates SQL against every MQTT message and routes matching messages to configured actions. The WHERE clause in the alert rule filters to only messages where temperature exceeds 35 degrees. This filtering happens at the IoT Core layer before the SNS publish, so your downstream systems only receive events that require action rather than processing every reading.
Fleet Management with IoT Jobs
resource "aws_iot_thing_group" "building_a_sensors" {
name = "building-a-sensors"
}
resource "aws_iot_job" "firmware_update" {
job_id = "firmware-update-v2-2-0"
targets = [aws_iot_thing_group.building_a_sensors.arn]
document = jsonencode({
operation = "firmware-update"
firmware_version = "2.2.0"
download_url = "https://firmware.yourdomain.com/v2.2.0/temperature-sensor.bin"
checksum = "sha256:abc123..."
})
job_executions_rollout_config {
maximum_per_minute = 10
}
abort_config {
criteria_list {
action = "CANCEL"
failure_type = "FAILED"
min_number_of_executed_things = 10
threshold_percentage = 20
}
}
timeout_config {
in_progress_timeout_in_minutes = 30
}
}
The abort_config automatically cancels the firmware rollout if more than 20 percent of devices fail the update, once at least 10 devices have been attempted. This prevents a bad firmware build from propagating to your entire fleet. maximum_per_minute = 10 means only 10 devices attempt the update per minute, giving you time to observe the rollout before it reaches the full fleet.
Closing Thoughts
IoT Core handles the infrastructure complexity that would otherwise consume a significant engineering team to build and operate. Device certificates with topic-level authorization, Device Shadows for state management across intermittent connectivity, the rules engine for zero-code data routing, and Jobs for safe fleet management are the capabilities that make production IoT systems reliable.
Start with certificates and topic policies that enforce least privilege from day one. Use Device Shadows for all commands to devices so offline devices receive them on reconnection. Use the rules engine for data routing rather than building your own message forwarder. Configure abort criteria on all Jobs before you have learned about a bad firmware update the hard way.
Enjoy the cloud.
Osama
#AWS #IoTCore #IoT #CloudArchitecture #Terraform #InfrastructureAsCode #AmazonWebServices #SolutionsArchitect #CloudComputing #DeviceManagement #MQTT #CloudSecurity #EdgeComputing #TechBlog #CloudInfrastructure #FleetManagement #Timestream #RulesEngine #SmartDevices #DevOps
Leave a Reply