Context & Why: Moving Beyond HTTP for IoT
Early in my career, I built IoT backends the way most web developers do: using REST APIs. It worked fine for a handful of devices. However, things fell apart once I hit the 100-device mark. HTTP headers are heavy. When thousands of sensors send 50-byte payloads every few seconds, a 500-byte header becomes a massive waste of bandwidth and battery life.
MQTT (Message Queuing Telemetry Transport) solves this by keeping a persistent, lightweight connection open. It is the industry standard for a reason. While raw MQTT can quickly turn into unmanageable spaghetti code, NestJS provides a microservices abstraction that keeps things clean. By using NestJS, you can treat MQTT messages like standard events, decoupling data ingestion from your core business logic.
Switching to this architecture in a recent project reduced our server CPU usage by 40%. The system stayed responsive even when device check-ins spiked during a firmware rollout. This guide walks through the exact setup I use to handle high-frequency sensor data without breaking a sweat.
Installation: Setting Up the Environment
You need a message broker before you can process any data. While EMQX is great for massive scale, Eclipse Mosquitto is my go-to for development and mid-sized workloads. It is incredibly lightweight, often using less than 10MB of RAM under light load.
1. Running Mosquitto with Docker
Docker is the fastest way to get a broker running without cluttering your local machine. Create a docker-compose.yml file to spin up the service:
version: '3.8'
services:
mosquitto:
image: eclipse-mosquitto
container_name: mosquitto_broker
ports:
- "1883:1883" # Standard MQTT port
- "9001:9001" # WebSockets port
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf
To keep things simple for testing, use a basic mosquitto.conf that allows anonymous connections:
persistence true
allow_anonymous true
listener 1883 0.0.0.0
2. Initializing the NestJS Projects
Start by installing the NestJS CLI. I recommend splitting your system into a Gateway and a Telemetry Service, but for this walkthrough, we will focus on the core microservice setup.
# Install NestJS CLI
npm install -g @nestjs/cli
# Create the project
nest new iot-backend
cd iot-backend
# Install the microservices and MQTT transport packages
npm install @nestjs/microservices mqtt
Configuration: Implementing the Microservice Logic
NestJS treats MQTT as a transport layer. You can handle incoming data using decorators like @MessagePattern or @EventPattern. For telemetry, @EventPattern is the better choice because it operates on a “fire-and-forget” basis, which is perfect for high-velocity sensor updates.
1. Configuring the Microservice Entry Point
Modify main.ts to tell NestJS to listen for MQTT messages instead of standard HTTP requests. This transforms your application into a dedicated message processor.
import { NestFactory } from '@nestjs/core';
import { Transport, MicroserviceOptions } from '@nestjs/microservices';
import { AppModule } from './app.module';
async function bootstrap() {
const app = await NestFactory.createMicroservice<MicroserviceOptions>(
AppModule,
{
transport: Transport.MQTT,
options: {
url: 'mqtt://localhost:1883',
// For production, use a unique clientId to track session state
},
},
);
await app.listen();
console.log('Telemetry Microservice is active');
}
bootstrap();
2. Handling Incoming Sensor Data
Next, create a controller to listen to a specific topic. Use the + wildcard to match any sensor ID. For example, sensors/sensor_01/data and sensors/sensor_02/data will both trigger the same handler.
import { Controller } from '@nestjs/common';
import { EventPattern, Payload, Ctx, MqttContext } from '@nestjs/microservices';
@Controller()
export class TelemetryController {
@EventPattern('sensors/+/data')
handleTelemetry(@Payload() data: any, @Ctx() context: MqttContext) {
const topic = context.getTopic();
const sensorId = topic.split('/')[1];
// Process the 10-20ms task here, like database insertion
console.log(`Processing Sensor [${sensorId}]:`, data);
this.processData(sensorId, data);
}
private processData(id: string, payload: any) {
// Business logic goes here
}
}
3. Sending Commands to Devices
Communication in IoT is a two-way street. You often need to push commands back to hardware, such as updating a display or toggling a relay. The ClientProxy makes this straightforward.
import { Injectable, Inject } from '@nestjs/common';
import { ClientProxy } from '@nestjs/microservices';
@Injectable()
export class CommandService {
constructor(
@Inject('MQTT_SERVICE') private client: ClientProxy,
) {}
sendCommand(deviceId: string, command: string) {
const pattern = `devices/${deviceId}/commands`;
const payload = { action: command, timestamp: new Date().toISOString() };
return this.client.emit(pattern, payload);
}
}
Verification & Scaling: Production Considerations
Writing the code is only half the battle. You need to ensure the system survives real-world traffic and connection drops.
1. Visibility with MQTT Explorer
Stop guessing if your messages are sent. Download MQTT Explorer and connect it to localhost:1883. It visualizes your topic tree in real-time. If your NestJS service isn’t reacting, check this tool first to see if the broker is actually receiving the packets.
2. Horizontal Scaling and Shared Subscriptions
One major hurdle with MQTT is that multiple instances of a service will each receive a copy of the same message. This leads to duplicate database entries. To fix this, use Shared Subscriptions. By prefixing your topic with $share/group_name/, the broker will load-balance messages across all active service instances. This allows you to scale to hundreds of thousands of messages per second.
3. Choosing the Right QoS
MQTT offers three Quality of Service (QoS) levels. For most telemetry, QoS 1 (At least once) is the sweet spot. It guarantees the message reaches your backend but adds minimal overhead. NestJS automatically sends the acknowledgment (PUBACK) once your controller function finishes. If your code crashes mid-process, the broker will retry delivery, ensuring no critical data is lost during a restart.
This architecture provides a clean separation of concerns. It allows your team to focus on processing data rather than managing socket heartbeats. For any long-term IoT project, the combination of NestJS and MQTT is a robust foundation that scales as your fleet grows.
