Time to cook: 120min
Difficulty: +++++
IKEA is moving step-by-step towards the connected home.The TRADFRI product line includes smart light bulbs, dimming remotes, color temperature remotes and motion sensors. Recently they added smart speakers and smart curtains. As we know IKEA all of these smart products are affordable.
IKEA sells a TRADFRI hub to connect these products together and provides an
IKEA Smart Home APP to control them from your phone. But would it not be cool if we could make them part of our system-of-systems architecture?
Well stay tuned, this tutorial highlights how you can connect all of these affordable and smart products within our connected home set up and you even don’t need to invest in the IKEA TRADFRI HUB.
Ingredients
A Raspberry Pi model 3, 3B+,4 or Zero W
A RPI power supply
A 4GB or > SD card
A Zigbee dongle e.g. Conbee II
A IKEA Tradfri light bulb (TRADFRI LED GU10)
Prerequisites
Tools
Etcher
Google MQTT lens
Based on open standards
The IKEA TRADFRI line uses Zigbee between devices and CoAP/dTLS to talk to the gateway. This means you are not locked into a single vendor. You can pair it with Philips Hue Bulbs and other compatible vendors. What is more, is that we can use a standard Zigbee solution out there to integrate them with our open ecosystem.
Zigbee2MQTT
Zigbee2MQTT is a free-to-use open source zigbee solution. As the word says, it connects on one hand Zigbee devices and translates them into MQTT topics. As we already mentioned in this cookbook, MQTT is the glue between our system-of-systems architecture.
Zigbee2MQTT is maintained by Koen Kanters from the Netherlands and has built up a nice reputation over the last years resulting in a stable and widely supported device list. Currently the Zigbee2MQTT solution supports 951 devices from 158 vendors.
Source (https://www.zigbee2mqtt.io/information/supported_devices.html)
The zigbee2MQTT stack comes with detailed documentation and forum to help you out.
The Hardware
Zigbee2MQTT can run on bare-metal Linux, on Docker on Windows or in a virtual environment. For our purpose, the popular Raspberry Pi looks like a very good candidate as it is low-cost and already used for other recipes within this cookbook. In fact we can run it on the same physical Raspberry Pi besides OpenHAB.
To be able to communicate with the Zigbee devices, we need a Zigbee dongle (or coordinator). The CC2530 USB Sniffer is highly suitable for this as described in the zigbee2mqtt documentation, but the downside is that you need to flash this USB sniffer first with the correct firmware, which is quite cumbersome and also requires a CC debugger you need to purchase and probably don’t use it anymore further. The alternative is that you buy a pre-programmed CC2530 USB sniffer from someone on the internet or I recently bought a Conbee II Zigbee dongle which seems to work fine. It’s a bit more expensive (around €40) but on the other hand you don’t need to by the cc debugger and saves you a lot of hassle.
If you got all those pieces together, we can start building. This tutorial will explain how to connect an IKEA smart LED (TRADFRI LED GU10). In the next recipe we will cover the integration of a TRAFDRI remote.
The Building Plan
Most information is available in the getting started section of zigbee2mqtt.io but we will cover all steps here to get a complete step-by-step overview:
Preparing the environment
We are going to use the popular Raspberry Pi to install and run the zigbee2MQTT.
Step 1: Download the image
Downloading the latest Raspberry Pi OS Lite image based on Debian Buster
Step 2: Flash to SD Card
The easiest way to flash the Raspberry Pi OS image to your SD card is to use Etcher.
Download and install the latest version of Etcher.
Open Etcher and select the Debian-raspbian Buster.zip file you have downloaded. There is no need to extract the .zip file first.
Choose the drive your SD card has been inserted into.
Click Flash
Step 3: Enable SSH
After you flash (burn) the image, pull the SD card out then plug it back in. On a Mac it should appear on the desktop with the name boot. On Windows it should appear in File Explorer with the name boot followed by a drive letter.
For security reasons, ssh is no longer enabled by default. To enable it you need to place an empty file named ssh (no extension) in the root of the boot disk.
Step 4: Connect To Network
Ethernet
This is the easiest way to connect the Raspberry Pi to the network. Insert a network cable between the Raspberry Pi and your home router.
No additional configuration is required to connect. By default your Raspberry Pi will receive an IP address on your network via DHCP.
WiFi
Follow these steps to connect your device to WiFi:
Create a file in the root of boot called: wpa_supplicant.conf (instructions below). Then paste the following into it (adjusting for your ISO 3166 alpha-2 country code, network name and network password):
country=BE
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
network={
ssid="NETWORK-NAME"
psk="NETWORK-PASSWORD"
}Eject the micro SD card
Remove the mini-SD card from the adapter and plug it into the Raspberry Pi
Boot the Raspberry Pi without an Ethernet cable attached.
Step 5: Connect via SSH
Now try to connect to your Raspberry Pi using an SSH connection. On a mac, you can use a terminal session. On a Windows PC you can use for example Putty.
For more information about SSH and how to connect visit:
https://www.raspberrypi.org/documentation/remote-access/ssh/
Step 6: Change your hostname and password
At the Pi command line type:
sudo raspi-config
Select the options for changing the hostname and password. On a new image, I would also recommend
expanding the file system (now under the Advanced options). Once the changes are made, reboot.
Remember that once you reboot, you will need to use the new hostname to login over ssh. For example, if your new hostname is mypi you would connect like this on a Mac:
ssh pi@mypi.local
On Windows you would need to change the hostname in Putty to mypi.local.
Step 7: Update your Raspberry Pi
sudo apt-get update -y
sudo apt-get upgrade -y
This will update your Operating system to the latest version.
Installing the MQTT Broker
Before we can install zigbee2MQTT, make sure you have an MQTT broker installed on your system. There are many tutorials available on how to do this. Mosquitto is the recommended MQTT broker but others should also work fine.
Mosquitto is a very lightweight broker ideal to run on a Raspberry Pi and can easily cope with MQTT traffic for clients on a smart home network.
Step 1: Get the repo key
wget http://repo.mosquitto.org/debian/mosquitto-repo.gpg.key
sudo apt-key add mosquitto-repo.gpg.key
Step 2: Make the repository available
cd /etc/apt/sources.list.d/
sudo wget http://repo.mosquitto.org/debian/mosquitto-buster.list
Step 3: Update apt information
sudo apt-get update
sudo apt-get install mosquitto
sudo apt-get install mosquitto-clients
Step 4: Starting and Configuring Mosquitto
The install installs mosquitto as a service which starts automatically and runs on port 1883.
If you need to change the configuration then the configuration file is called mosquitto.conf and is located in the /etc/mosquitto folder.
On the Pi mosquitto can be controlled using the
systemctl command. You can stop,start and restart the service using.
sudo systemctl stop mosquitto.service
sudo systemctl start mosquitto.service
sudo systemctl restart mosquitto.service
Note:
When testing you will find it very useful to run mosquitto manually from the command line.
The first step is to stop mosquitto using:
sudo systemctl stop mosquitto.service
Then you can start it using:
mosquitto -v #start in verbose mode
Step 5: Set up username & password for Mosquitto
Although you can use the MQTT broker as-is, for security reasons, it is advisable to configure a username and password to access the MQTT broker.
Mosquitto includes a utility to generate a special password file called mosquitto_passwd. This command will prompt you to enter a password for the specified username, and place the results in /etc/mosquitto/passwd.
sudo mosquitto_passwd -c /etc/mosquitto/passwd admin
Open up a new configuration file for Mosquitto and tell it to use this password file to require logins for all connections:
sudo nano /etc/mosquitto/conf.d/default.conf
This should open an empty file. Paste in the following:
allow_anonymous false
password_file /etc/mosquitto/passwd
allow_anonymous false will disable all non-authenticated connections, and the password_file line tells Mosquitto where to look for user and password information.
Save and exit the file.
Now we need to restart Mosquitto and test our changes.
sudo systemctl restart mosquitto
Note: you can test if the authentication file works as follows:
Try to publish a message without a password:
mosquitto_pub -h localhost -t “test” -m “hello world”
The message should be rejected:
Output
Connection Refused: not authorised.
Error: The connection was refused.
Now try again, using the username and password this time:
mosquitto_sub -h localhost -t test -u “sammy” -P “password”
The message should go through
Installing Zigbee2MQTT
Step 1: Determine the location of the CC2531 USB sniffer or Conbee II USB dongle
We first need to determine the location of the CC2531 USB sniffer or Conbee II USB dongle. Connect the CC2531 USB or Conbee II to your Raspberry Pi. Most of the time the location is /dev/ttyACM0. This can be verified by:
pi@raspberry:~ $ ls -l /dev/ttyACM0
crw-rw---- 1 root dialout 166, 0 May 16 19:15 /dev/ttyACM0 # <-- CC2531 on /dev/ttyACM0
Step 2: Setup Node.js repository
sudo curl -sL https://deb.nodesource.com/setup_12.x | sudo -E bash -
NOTE 1: If you see the message below please follow: https://gist.github.com/Koenkk/11fe6d4845f5275a2a8791d04ea223cb.
You appear to be running on ARMv6 hardware. Unfortunately this is not currently supported by the NodeSource Linux distributions. Please use the ‘linux-armv6l’ binary tarballs available directly from nodejs.org for Node.js 4 and later.
IMPORTANT: In this case instead of the apt-get install mentioned below; do: sudo apt-get install -y git make g++ gcc
NOTE 2: On x86, Node.js 10 may not work. It’s recommended to install an unofficial Node.js 12 build which can be found here:
https://unofficial-builds.nodejs.org/download/release/ (e.g. v12.16.3)
Step 3: Setup Node.js repository
sudo apt-get install -y nodejs git make g++ gcc
Verify that the correct nodejs and npm (automatically installed with nodejs)
version has been installed
node --version # Should output v12.X or v10.X
npm --version # Should output 6.X
Step 4: Clone Zigbee2MQTT repository
sudo git clone https://github.com/Koenkk/zigbee2mqtt.git /opt/zigbee2mqtt
sudo chown -R pi:pi /opt/zigbee2mqtt
Step 5: Install dependencies (as user “pi”)
cd /opt/zigbee2mqtt
npm ci
If everything went correctly the output of npm ci is similar to (the number of packages and seconds is probably different on your device):
node-pre-gyp info ok
added 383 packages in 111.613s
Note that the npm ci produces some warning which can be ignored.
Step 6: Configure zigbee2MQTT
Before we can start Zigbee2MQTT we need to edit the configuration.yaml file. This file contains the configuration which will be used by Zigbee2MQTT.
Open the configuration file:
nano /opt/zigbee2mqtt/data/configuration.yaml
For a basic configuration, the default settings are probably good. The only thing we need to change is the MQTT server url and authentication (if applicable) and set the username and password for the MQTT broker. This can be done by changing the section below in your configuration.yaml.
# MQTT settings
mqtt:
# MQTT base topic for Zigbee2MQTT MQTT messages
base_topic: zigbee2mqtt
# MQTT server URL
server: 'mqtt://localhost'
# MQTT server authentication, uncomment if required:
# user: my_user
# password: my_passwordSave the file and exit.
It is recommended to use a custom network key. This can be done by adding the following to your configuration.yaml. With this Zigbee2MQTT will generate a network key on next startup.
advanced:
network_key: GENERATE
This can be done with the following command:
echo “\n\nadvanced:\n network_key: GENERATE” >> /opt/zigbee2mqtt/data/configuration.yaml
Step 7: Starting zigbee2MQTT
Now that we have setup everything correctly we can start Zigbee2MQTT.
cd /opt/zigbee2mqtt
npm start
When started successfully, you will see something like:
Zigbee2MQTT:info 2019-11-09T13:04:01: Logging to directory: ‘/opt/zigbee2mqtt/data/log/2019-11-09.14-04-01’
Zigbee2MQTT:info 2019-11-09T13:04:01: Starting Zigbee2MQTT version 1.6.0 (commit #720e393)
Zigbee2MQTT:info 2019-11-09T13:04:01: Starting zigbee-herdsman...
Zigbee2MQTT:info 2019-11-09T13:04:03: zigbee-herdsman started
Zigbee2MQTT:info 2019-11-09T13:04:03: Coordinator firmware version: ‘{“type”:”zStack30x”,”meta”:{“transportrev”:2,”product”:2,”majorrel”:2,”minorrel”:7,”maintrel”:2,”revision”:20190425}}’
Zigbee2MQTT:info 2019-11-09T13:04:03: Currently 0 devices are joined:
Zigbee2MQTT:warn 2019-11-09T13:04:03: `permit_join` set to `true` in configuration.yaml.
Zigbee2MQTT:warn 2019-11-09T13:04:03: Allowing new devices to join.
Zigbee2MQTT:warn 2019-11-09T13:04:03: Set `permit_join` to `false` once you joined all devices.
Zigbee2MQTT:info 2019-11-09T13:04:03: Zigbee: allowing new devices to join.
Zigbee2MQTT:info 2019-11-09T13:04:03: Connecting to MQTT server at mqtt://localhost
Zigbee2MQTT:info 2019-11-09T13:04:03: Connected to MQTT server
Zigbee2MQTT can be stopped by pressing CTRL + C.
Testing the installation
Now that everything is installed, let’s test the installation. Zigbee2MQTT converts all zigbee messages into MQTT topics. To test the installation we can use an MQTT client such as Google MQTTlens, but any other MQTT client would also work.
Step 1: Installing & running MQTTlens
Google MQTTLens is A Google Chrome application, which connects to a MQTT broker and is able to subscribe and publish to MQTT topic and you can install it into your chrome browser from the Chrome web store.
Step 2: configuring MQTTlens
In the connections tab, add a new connection.
Enter a connection name
Under hostname, enter the IP address or local name of your Raspberry Pi. If you configured a username and password, enter them under Credentials.
The client ID is randomly assigned and can be used as such.
Click ‘save changes’
When saved, the connection should indicate “green” as shown in the image:
Step 3: Subscribe on MQTT Topics
Next, we subscribe on all MQTT topics from zigbee2mqtt
Subscribe to the zigbee2MQTT topics by entering
zigbee2mqtt/# in the MQTT subscribe field
Click subscribe
The # sign is a wildcard which listens on all MQTT topics under zigbee2MQTT.
Let’s now pair our IKEA TRADFRI LED and then come back to this tool.
Pairing the TRADFRI LED
Now it's time to connect our IKEA TRADFRI light. You can pair this zigbee device by 6x quickly turn on/off the LED. (see https://www.zigbee2mqtt.io/devices/LED1837R5.html).
This will trigger a factory reset.
When you turn on the LED after the factory reset, you should see the following incoming messages in the MQTTLens application
{
“message”: “interview_started”,
“meta”: {
“friendly_name”: “0x680ae2fffe7155a0”
},
“type”: “pairing”
}
{
“message”: “interview_successful”,
“meta”: {
“description”: “TRADFRI LED bulb GU10 400 lumen, dimmable, white spectrum”,
“friendly_name”: “0x680ae2fffe7155a0”,
“model”: “LED1537R6/LED1739R5”,
“supported”: true,
“vendor”: “IKEA”
},
“type”: “pairing”
}
Zigbee2MQTT translates incoming zigbee messages into MQTT. A good reference document about the MQTT topics and message structure used by zigbee2MQTT can be found on the zigbee2mqtt.io website
For example to toggle the LED, we can publish the following message on topic:
zigbee2mqtt/0x680ae2fffe7155a0/set
{
"state": "TOGGLE"
}
This should toggle the IKEA TRADFRI LED bulb.
You can change the brightness by publishing:
{
"brightness": 10
}
The value can be between 0 and 255
The IKEA TRADFRI LED bulb also accept to set the color temp.
{
“state”: “TOGGLE”,
“color_temp”: 400
}
Color_temp value can vary between 50 and 400, with 400 the warmest color.
What’s Next?
Now we can receive and send commands from and to zigbee devices using MQTT messages we can integrate them into our systems-of-systems architecture.
One way is to use Node-RED. Within Node-RED, you can handle the MQTT messages and create intuitive dashboards and automation rules, allowing you to control your zigbee devices and act upon them.
Another way is to integrate the zigbee devices into our smart HUB by creating generic MQTT Things.
In another recipe we will show how to connect a TRADFRI remote and use it within a Node-RED environment.