Device Shadows: Talking to Ghosts

The "Offline" Problem

IoT devices are unreliable by nature. They lose Wi-Fi. They run out of battery. They go into deep sleep to conserve power. If you send a command "Turn Light ON" while the device is offline, a raw MQTT message is simply dropped — the message is lost. The user thinks the light is on, but it's off. Your application has no way to know the true state of the device.

This is the state synchronization problem, and it's one of the hardest problems in distributed systems applied to physical hardware.

The Solution: Device Shadows

A Device Shadow (also called a "Thing Shadow") is a persistent JSON document stored in the cloud that represents the last known and desired state of a device. It acts as a proxy for the device — your application always talks to the shadow, never directly to the device. The shadow handles the synchronization.

The Shadow Document Structure:

{
  "state": {
    "desired": {
      "color": "Red",
      "brightness": 80
    },
    "reported": {
      "color": "Green",
      "brightness": 80
    },
    "delta": {
      "color": "Red"
    }
  },
  "metadata": { ... },
  "version": 12,
  "timestamp": 1700000000
}

• desired: What the application wants the device to be. Written by your app or backend.
• reported: What the device last confirmed its actual state to be. Written only by the device.
• delta: Automatically computed by IoT Core. Contains only the keys where desired ≠ reported. The device subscribes to the delta topic and acts on it.

The Full Synchronization Workflow:

Step 1: App sets desired state (device is offline)
  App → PATCH shadow → desired: {color: "Red"}
  Shadow stores it. Delta = {color: "Red"}.

Step 2: Device wakes up and connects
  Device → GET shadow → receives delta: {color: "Red"}
  Device changes LED to Red.

Step 3: Device confirms the change
  Device → PATCH shadow → reported: {color: "Red"}
  Shadow recomputes: desired == reported → delta disappears.

Step 4: App reads shadow
  App → GET shadow → sees reported: {color: "Red"} → confirms success.

The MQTT topics involved:

Named Shadows: A single device can have multiple shadows. A smart home hub might have one shadow for its network configuration, another for its firmware update state, and another for its user preferences. Named shadows use the topic prefix $aws/things/{name}/shadow/name/{shadowName}/.

Key architectural distinction: Use Device Shadows for state — configuration, mode, on/off status — things that must eventually be consistent. Use MQTT topics directly for transient telemetry — temperature readings, GPS coordinates, event streams — data where you care about the current value, not reconciling past commands.