MQTT and RabbitMQ

This page covers two messaging protocols: MQTT for IoT and device traffic, and RabbitMQ (AMQP 0-9-1) for queue-based services. For Kafka see its own page: Kafka.

#MQTT

#What it does

Every VU is its own MQTT client, like a device: it connects on its first call, keeps its subscriptions for the run and reconnects on the next call if the connection is lost. There are three actions:

• Publish: sends a message to a topic. With QoS 1/2 the time runs until the broker acknowledges.
• Wait for a message (subscribe): the VU subscribes to the topic once; every iteration waits for the next message.
• Request/reply: subscribes to the response topic, publishes the message and waits for the reply; the time is the round trip.

#When to use it

To measure the capacity and latency of a broker (Mosquitto, EMQX, HiveMQ, the RabbitMQ MQTT plugin…) when thousands of devices connect at once and send telemetry.

#Creating the connection

1. Connections → Add connection, Type: MQTT.
2. Brokers (comma-separated): tcp://broker:1883, for TLS ssl://broker:8883, over WebSocket ws://broker:8083/mqtt. Required. Don't put a password in the address.
3. Client ID: a template; default spitfire-{{__VU}}. Each VU needs a unique client ID; when two clients connect with the same ID, the broker drops one of them.
4. Username and Password (a secret).
5. Keep-alive (s): default 30.
6. Clean session: on by default. When off, the broker keeps the session and persistent subscriptions.
7. Expand TLS for TLS.
8. Save, Test.

#Adding a step

1. Add step → Protocol: MQTT → pick the Connection.
2. Pick an Action.
3. Type the Topic. Wildcards (+, #) are only valid for Wait for a message (subscribe); on publish the save fails with wildcards (+ #) are only valid for subscribe.
4. Write the Payload; {{variables}} are allowed.
5. Pick QoS (0, 1 or 2) and retain if needed.
6. For Request/reply, write the Response topic.
7. Wait (default 10s): for Wait for a message and Request/reply, the step fails when it runs out.

[
  {"id": "telemetry", "name": "Telemetry", "protocol": "mqtt", "connection": "iot-broker",
   "mqtt": {"action": "publish", "topic": "devices/{{__VU}}/telemetry",
     "payload": "{\"temp\": {{$randInt 18 30}}}", "qos": 1}},
  {"id": "command", "name": "Wait for command", "protocol": "mqtt", "connection": "iot-broker",
   "mqtt": {"action": "subscribe", "topic": "devices/{{__VU}}/commands", "qos": 1, "wait": "10s"}}
]

#Metrics it produces

req_duration (publish: sending with QoS 0, until the broker acknowledges with QoS 1/2; subscribe: waiting for the next message; request/reply: the round trip), req_failed, reqs, data_sent, data_received.

#RabbitMQ (AMQP)

#What it does

Publishes a message to an exchange, takes a message from a queue, or does request/reply (RPC). The VUs of a runner share one connection (each with its own channels); optionally every VU opens its own connection. Every publish waits for its publisher confirm, so the measured time is the broker's.

#When to use it

To see how the workload that flows through RabbitMQ (orders, notifications, job queues) affects the broker and its consumers.

#Creating the connection

1. Connections → Add connection, Type: RabbitMQ (AMQP).
2. URL: amqp://rabbit:5672/, or with a vhost amqp://rabbit:5672/orders. For the TLS port usually amqps://rabbit:5671/. Required; don't put the password in the URL.
3. Username and Password (a secret).
4. Separate connection per VU: when ticked, each VU opens its own TCP connection (to imitate many clients). Default: one connection per runner, a channel per VU.
5. TLS section for TLS.
6. Save, Test.

#Adding a step

1. Add step → Protocol: RabbitMQ (AMQP) → pick the Connection.
2. Action:
   • Publish: write the Exchange (empty means the (default) exchange) and the Routing key. A message no queue receives (unroutable) is a failure.
   • Consume from queue: write the Queue. The queue must exist; each iteration takes one message and acknowledges it (ack).
   • RPC (request/reply): the request is sent with direct reply-to and the matching reply is awaited.
3. Write the Message; add Content-Type and Headers if needed.
4. Tick Persistent message to keep the message across a broker restart.
5. Wait (default 10s): for consume and RPC.

[
  {"id": "publish", "name": "Order event", "protocol": "amqp", "connection": "rabbit",
   "amqp": {"action": "publish", "exchange": "orders", "routingKey": "orders.created",
     "body": "{\"id\": \"{{$uuid}}\"}", "contentType": "application/json", "persistent": true}},
  {"id": "take", "name": "Take from queue", "protocol": "amqp", "connection": "rabbit",
   "amqp": {"action": "consume", "queue": "orders.created", "wait": "10s"}},
  {"id": "rpc", "name": "Ask price", "protocol": "amqp", "connection": "rabbit",
   "amqp": {"action": "rpc", "routingKey": "pricing.rpc", "body": "{\"sku\": \"A-1\"}", "wait": "5s"}}
]

#Metrics it produces

req_duration (publish: until the publisher confirm; consume: waiting for the message; RPC: the round trip), req_failed, reqs, data_sent, data_received.

#Common problems

Symptom: MQTT Test fails with connection refused or not authorized. Cause: Wrong broker address/port, or the username/password is refused. Fix: Check the scheme (tcp://, ssl://, ws://) and the port; type the password again.

Symptom: MQTT VUs keep disconnecting and reconnecting. Cause: Client IDs collide (e.g. a fixed Client ID); the broker drops the older connection with the same ID. Fix: Use {{__VU}} in the Client ID (default spitfire-{{__VU}}).

Symptom: Wait for a message times out every time. Cause: No message arrives on the topic, or the topic name/wildcard is wrong. Fix: Check the topic; add a publish step to the same test; raise the Wait.

Symptom: RabbitMQ ACCESS_REFUSED - Login was refused. Cause: Wrong username/password, or the user has no access to the vhost. Fix: Type the password again; give the user permissions on the vhost in the URL.

Symptom: The publish step fails with publish: unroutable: 312 NO_ROUTE. Cause: No queue bound to the exchange matches the routing key. Fix: Check the exchange name and the routing key; create the queue binding.

Symptom: Consume from queue fails with NOT_FOUND - no queue. Cause: The queue does not exist; Spitfire does not create queues. Fix: Create the queue in RabbitMQ beforehand.

#Related pages

• Protocols and connections
• Kafka
• Redis and MongoDB