andry/EMS-ESP32
1
0
Fork 0
mirror of https://github.com/emsesp/EMS-ESP32.git synced 2025-12-06 07:49:52 +03:00
Code Issues Packages Projects Releases Wiki Activity

1.5.7b

Browse Source
This commit is contained in:
proddy
2019-03-10 15:09:51 +01:00
parent b52794998e
commit 8e62a99417
46 changed files with 6145 additions and 5523 deletions
Download Patch File Download Diff File Expand all files Collapse all files

351
README.md
View File

@@ -1,42 +1,38 @@
# EMS-ESP-Boiler
# EMS-ESP
EMS-ESP-Boiler is a project to build a controller circuit running with an ESP8266 to communicate with EMS (Energy Management System) based Boilers and Thermostats from the Bosch range and compatibles such as Buderus, Nefit, Junkers etc.
EMS-ESP is a project to build an electronic controller circuit using an Espressif ESP8266 microcontroller to communicate with EMS (Energy Management System) based Boilers and Thermostats from the Bosch range and compatibles such as Buderus, Nefit, Junkers etc.
There are 3 parts to this project, first the design of the circuit, second the code for the ESP8266 microcontroller firmware and lastly an example configuration for Home Assistant to monitor the data and issue direct commands via MQTT.
There are 3 parts to this project, first the design of the circuit, secondly the code for the ESP8266 microcontroller firmware with telnet and MQTT support, and lastly an example configuration for Home Assistant to monitor the data and issue direct commands via a MQTT broker.
[![version](https://img.shields.io/badge/version-1.1.0-brightgreen.svg)](CHANGELOG.md)
[![branch](https://img.shields.io/badge/branch-dev-orange.svg)](https://github.org/xoseperez/espurna/tree/dev/)
[![license](https://img.shields.io/github/license/xoseperez/espurna.svg)](LICENSE)
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/b8880625bdf841d4adb2829732030887)](https://app.codacy.com/app/proddy/EMS-ESP?utm_source=github.com&utm_medium=referral&utm_content=proddy/EMS-ESP&utm_campaign=Badge_Grade_Settings)
[![version](https://img.shields.io/badge/version-1.5.5-brightgreen.svg)](CHANGELOG.md)
- [EMS-ESP-Boiler](#ems-esp-boiler)
- [EMS-ESP](#ems-esp)
- [Introduction](#introduction)
- [Supported Boilers Types](#supported-boilers-types)
- [Supported EMS Devices](#supported-ems-devices)
- [Supported ESP8266 devices](#supported-esp8266-devices)
- [Getting Started](#getting-started)
- [Monitoring The Output](#monitoring-the-output)
- [Building The Circuit](#building-the-circuit)
- [Powering The EMS Circuit](#powering-the-ems-circuit)
- [Adding external temperature sensors](#adding-external-temperature-sensors)
- [How The EMS Bus Works](#how-the-ems-bus-works)
- [EMS IDs](#ems-ids)
- [EMS Polling](#ems-polling)
- [EMS Broadcasting](#ems-broadcasting)
- [EMS Reading and Writing](#ems-reading-and-writing)
- [The ESP8266 Source Code](#the-esp8266-source-code)
- [Supported EMS Types](#supported-ems-types)
- [Supported Thermostats](#supported-thermostats)
- [RC20 (Moduline 300)](#rc20-moduline-300)
- [RC30 (Moduline 400)](#rc30-moduline-400)
- [RC35](#rc35)
- [TC100/TC200 (Nefit Easy)](#tc100tc200-nefit-easy)
- [Special EMS Types](#special-ems-types)
- [Which thermostats are supported?](#which-thermostats-are-supported)
- [Customizing The Code](#customizing-the-code)
- [Using MQTT](#using-mqtt)
- [The Basic Shower Logic](#the-basic-shower-logic)
- [Home Assistant Configuration](#home-assistant-configuration)
- [Building The Firmware](#building-the-firmware)
- [Using PlatformIO Standalone](#using-platformio-standalone)
- [Using ESPurna](#using-espurna)
- [Using Pre-built Firmware](#using-pre-built-firmware)
- [Building Using Arduino IDE](#building-using-arduino-ide)
- [Using the Pre-built Firmware](#using-the-pre-built-firmware)
- [Troubleshooting](#troubleshooting)
- [Known Issues](#known-issues)
- [Wish List](#wish-list)
- [Your Comments and Feedback](#your-comments-and-feedback)
@@ -44,82 +40,84 @@ There are 3 parts to this project, first the design of the circuit, second the c
## Introduction
My original intention for this home project with to build my own smart thermostat for my Nefit Trendline boiler and then have it controlled automatically via [Home Assistant](https://www.home-assistant.io/) on my mobile phone. I had a few ESP32s and ESP8266s lying around from previous IoT projects and building a specific circuit to decode the EMS messages was a nice challenge into designing complete end-to-end complex electronic circuits. I then began adding new features such as timing how long the shower would be running for and subsequently triggering an alarm (actually a shot of cold water) after a certain period.
The original intention for this home project was to build a custom smart thermostat that interfaces with my Nefit Trendline HRC30 boiler and have it controlled via a mobile app using MQTT. I had a few cheap ESP32s and ESP8266s microcontrollers lying around from previous IoT projects and learning how to build a circuit to decode the EMS bus messages seemed like a nice challenge.
Acknowledgments and kudos to the following people and their open-sourced projects that have helped me get this far:
Acknowledgments and kudos to the following people who have open-sourced their projects:
**susisstrolch** - Probably the first working version of the EMS bridge circuit I found designed for the ESP8266. I borrowed Juergen's [schematic](https://github.com/susisstrolch/EMS-ESP12) and parts of his code logic.
**susisstrolch** - One of the first working versions of the EMS bridge circuit I found designed for specifically for the ESP8266. I borrowed Juergen's [schematic](https://github.com/susisstrolch/EMS-ESP12) and parts of his code ideas for reading telegrams.
**bbqkees** - Kees built a [circuit](https://github.com/bbqkees/Nefit-Buderus-EMS-bus-Arduino-Domoticz) and some sample Arduino code to read from the EMS and push messages to Domoticz. His SMD board is also now available for purchase.
**bbqkees** - Kees built a working [circuit](https://shop.hotgoodies.nl/ems/) and his SMD board is available for purchase on his website.
**EMS Wiki** - A comprehensive [reference](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme) for decoding the EMS telegrams, which I found not always to be 100% accurate. It's in German so use Google Translate if you need help.
**EMS Wiki** - A comprehensive [reference](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme) (in German) for the EMS bus which is a little outdated, not always 100% accurate and sadly no longer maintained.
## Supported Boilers Types
## Supported EMS Devices
Most Bosch branded boilers that support the Logamatic EMS (and EMS+) bus protocols work with this design. Which are Nefit, Buderus, Worcester and Junkers and copyrighted. Please make sure you read the **Disclaimer** carefully before sending ambigious messages to your EMS bus as you cause device damage.
Most Bosch branded boilers that support the Logamatic EMS bus protocols work with this design. This includes Nefit, Buderus, Worcester and Junkers (all copyrighted). Please make sure you read the **Disclaimer** carefully before sending ambiguous messages to your EMS bus as you could cause serious damage to your equipment.
Note support for the later EMS Plus (EMS+ or EMS2) standard hasn't been added yet to the library. If you'd like to help please reach out.
## Supported ESP8266 devices
I've tested the code and circuit with a few ESP8266 development boards such as the Wemos D1 Mini, Wemos D1 Mini Pro, Nodemcu0.9 and Nodemcu2 boards. It will also work on bare ESP8266 chips such as the E-12s but do make sure you disabled the LED support and wire the UART correctly as the code doesn't use the normal Rx and Tx pins. This is explained below.
The code and circuit has been tested with a few ESP8266 development boards such as the Wemos D1 Mini, Wemos D1 Mini Pro, Nodemcu0.9 and Nodemcu2 dev boards. It will also work on bare ESP8266 chips such as the ESP-12E but do make sure you disabled the LED support and wire the UART correctly as the code doesn't use the normal Rx and Tx pins.
## Getting Started
1. Either build the circuit below or purchase a ready built board from bbqkees via his [GitHub](https://github.com/bbqkees/Nefit-Buderus-EMS-bus-Arduino-Domoticz) page or the [Domoticz forum](http://www.domoticz.com/forum/viewtopic.php?f=22&t=22079&start=20).
2. Get an ESP8266 dev board and connect the 2 EMS output lines from the boiler to the circuit and the Rx and Tx out to ESP pins D7 and D8 respectively. The EMS connection can either be the 12-15V AC direct from the thermostat bus line or from the 3.5" Service Jack at the front.
3. Optionally connect the three LEDs to show Rx and Tx traffic and Error codes to pins D1, D2, D3 respectively. I use 220 Ohm pull-down resistors. These pins are configurable in ``boiler.ino``. This is further explained in the **code** section below.
4. Build and upload the firmware to the ESP8266 device. I used Platformio with Visual Studio. Do make sure you set the MQTT and WiFi credentials correctly and if you're not using MQTT leave the MQTT_IP blank. The firmware supports OTA too with the default hostname as 'boiler' (or 'boiler.' depending on your OS and how the mdns resolves hostnames).
5. Power the ESP either via USB or direct into the 5v vin pin from an external power 5V volts supply with min 400mA.
6. Attach the 3v3 out on the ESP8266 to the DC power line on the EMS circuit as indicated in the schematics.
7. The WiFi connects via DHCP by default. Find the IP by from your router and then telnet (port 23) to it. Tip: to enable Telnet on Windows run `dism /online /Enable-Feature /FeatureName:TelnetClient` or install something like [putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html). If everything is working you should see the messages appear in the window as shown in the next section. However if you're unable to locate the IP of the ESP then probably the WiFi failed to instantiate. In this case add -DUSE_SERIAL to the build options, connect at USB, build, upload and then use a terminal to connect to the serial port to see the debug messages. A word of warning, do not use both a USB and power from the EMS at the same time.
1. Either build the circuit described below or purchase a ready built board from bbqkees.
2. Grab any ESP8266 dev board. The latest bbqkees boards have a Wemos D1 pre-mounted with a copy of this firmware.
3. Optionally add external Dallas temperature sensors and an external LED. The default pins for these are D1 and D5 respectively.
4. Decide whether to compile and upload the code yourself using PlatformIO or just upload the pre-baked firmware using the esptool (read these [instructions](#using-the-pre-built-firmware)). If you want to build yourself now is the time to customize your settings in `my_custom.h`. Upload the firmware.
5. Connect a USB 5v power supply to the ESP8266 board, either via laptop/PC or external power supply.
7. When the ESP8266 starts up for the first time the onboard LED will be flashing. This is because the EMS bus is not yet connected.
8. If you haven't hardcoded the WiFi credentials in step 4, the ESP8266 will boot up in a WiFi Access Point (AP) mode with the ssid name `ems-esp`. Now you can either use a laptop and connect to this AP using Telnet to `192.168.1.4` or if its powered from a computers USB use a Serial monitor tool to the ESP's COM port. Tip: to enable Telnet on Windows 10 run `dism /online /Enable-Feature /FeatureName:TelnetClient` or install something like [putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html).
9. Next is to change some of the settings. Type `set` to list the current stored settings. Use `set wifi` to add your wifi credentials and if you're using MQTT set the host, username and password. There is no need to reboot the device.
10. The `led_gpio` will default to the onboard LED (which is probably blinking now). Ignore `thermostat_type` and `boiler_type` as these will be auto-detected hopefully later on.
11. **Important**: If `serial` is set to `on` set it to `off` using `set serial off`. The EMS bus is disabled when the serial is on. This mode is only used for setting up a new board or debugging startup issues.
12. Hook up the ESP to the EMS board as follows:
| EMS board | ESP8266 dev board |
| ----------|------------------ |
| Ground/G/J2| GND/G |
| Rx/J2 | D7 |
| Tx/J2 | D8 |
| VC/J2 | 3v3 or 5v |
13. Connect the EMS lines to the ESP. This can be done via the two EMS wires or via the 3.5" service jack if you have an bbqkees board.
14. Reboot the ESP, either by the reset switch or pulling the power.
15. The ESP will first perform an autodetect to try and discover the EMS devices attached. If your boiler and thermostat are recognized it will set these types and store them for ever and ever. You can trace the output by telnet'ing to the board `telnet ems-esp.local`. Also type `info` to check what happened.
16. If your boiler/thermostat is not discovered create a GitHub issue stating the type and product ID. These will be added to the file `ems_devices.h` in a future release.
17. If all is well and there is traffic on the EMS bus the onboard LED will stop blinking and be permanently on. If this is annoying you can disable with `set led off`. To see the EMS messages type `set log v` for verbose logging.
18. And all is not well, check the wiring, make sure serial is off and look at the telnet session for errors. If in doubt, wipe the ESP with `pio run -t erase` and start again with step #3
## Monitoring The Output
Use the telnet client to inform you of all activity and errors real-time. This is an example of the telnet output:
![Telnet](doc/telnet/telnet_example.jpg)
![Telnet](doc/telnet/telnet_menu.jpg)
If you type 'l 4' and Enter, it will toggle verbose logging showing you more detailed messages. I use ANSI colors with white text for info messages, green for well formatted telegram packages (which have validated CRC checks), red for corrupt packages and yellow for send responses.
Type 'log v' and Enter and you'll be seeing verbose logging messages. ANSI colors with white text for info messages, green are for broadcast telegrams, yellow are the ones sent to us and red are for unknown data or telegrans which have failed the CRC check.
![Telnet](doc/telnet/telnet_verbose.PNG)
To see the current values of the Boiler and its parameters type 's' and hit Enter. Watch out for unsuccessful telegram packets in the #CrcErrors line.
To see the current stats and collected values type 'info'. Watch out for unsuccessful telegram packets in the #CrcErrors line.
![Telnet](doc/telnet/telnet_stats.PNG)
Commands can be issued directly to the EMS bus typing in a letter followed by an optional parameter and pressing Enter. Supported commands are:
- **b** to send a read command to the boiler. The 2nd parameter is the type. For example 'b 33' will request type UBAParameterWW and bring back the Warm Water temperatures from the Boiler.
- **t** is similar, but to send a read command to the thermostat.
- **T** set the thermostat temperature to the given celsius value
- **w** to adjust the temperature of the warm water from the boiler
- **a** to turn the warm tap water on and off
- **h** to list all the recognized EMS types
- **P** to toggle the Polling response on/off (note it's not necessary to have Polling enabled to work)
- **m** to set the thermostat mode to manual or auto
- **S** to toggle the Shower Timer functionality on/off
- **A** to toggle the Shower Timer Alert functionality on/off
**Disclaimer: be careful when sending values to the boiler. If in doubt you can always reset the boiler to its original factory settings by following the instructions in the user guide. On my **Nefit Trendline HRC30** that is done by holding down the Home and Menu buttons simultaneously for a few seconds, selecting factory settings from the scroll menu and lastly pressing the Reset button.**
**Disclaimer: be careful when sending values to the boiler. If in doubt you can always reset the boiler to its original factory settings by following the instructions in the user guide. For example on my Nefit Trendline that is done by holding down the Home and Menu buttons simultaneously for a few seconds, selecting factory settings from the scroll menu followed by pressing the Reset button.**
## Building The Circuit
The EMS circuit is really all credit to the hard work many people have done before me, noticeably *susisstrolch* with his ESP8266 [version](https://github.com/susisstrolch/EMS-ESP8266_12-PCB/tree/newmaster/Schematics/EMS-ESP8266-12).
I've included a prototype boards you can build yourself on a breadboard. One part for only reading values from the Boiler and an extension with the write logic so you can also send commands.
We need the Rx/Tx of the ESP8266 for flashing, so the code in ``emsuart.cpp`` switches the UART pins to use RX1 and TX1 (GPIO13/D7 and GPIO15/D8 respectively). This also prevents any bogus stack data being sent to EMS bus when the ESP8266 decides to crash like after a Watch Dog Reset.
Included is a prototype boards you can build yourself on a breadboard.
The breadboard layout was done using [DIY Layout Creator](https://github.com/bancika/diy-layout-creator) and sources files are included in this repo.
![Breadboard Circuit](doc/schematics/breadboard.png)
The schematic used (as designed by [susisstrolch](https://github.com/susisstrolch/EMS-ESP8266_12-PCB)):
The schematic used:
![Schematic](doc/schematics/circuit.png)
*Optionally I've also added 2 0.5A/72V polyfuses between the EMS and the two inductors L1 and L2 for extra protection.*
And lastly if you don't fancy building the circuit, [bbqkees](http://www.domoticz.com/forum/memberlist.php?mode=viewprofile&u=1736) can sell you one complete with SMD components which looks like the photo below when connected to a Wemos D1 Mini:
And here's a version using an early prototype board from **bbqkees**:
![WemosD1](doc/schematics/wemos_kees.png)
@@ -134,33 +132,35 @@ The EMS circuit will work with both 3.3V and 5V. It's easiest though to power di
| With Power Circuit |
| ------------------------------------------ |
| ![Power circuit](doc/schematics/Schematic_EMS-ESP-Boiler-supercap.png) |
| ![Power circuit](doc/schematics/Schematic_EMS-ESP-supercap.png) |
## Adding external temperature sensors
The code supports auto-detection of Dallas type temperature sensors. The default gpio pin used on the ESP8266 is D5 but this can be configured in the setting menu (`set dallas_gpio`). The dallas chips DS1822, DS18S20, DS18B20, DS1825 are supported including their parasite varieties.
## How The EMS Bus Works
Packages are sent to the EMS "bus" from the Boiler and any other compatible connected devices via serial TTL transmission. The protocol is 9600 baud, 8N1 (8 bytes, no parity, 1 stop bit). Each package is terminated with a break signal `<BRK>`, a 11-bit long low signal of zeros.
Packages are streamed to the EMS "bus" from any other compatible connected device via serial TTL transmission using protocol 9600 baud, 8N1 (8 bytes, no parity, 1 stop bit). Each package is terminated with a break signal `<BRK>`, a 11-bit long low signal of zeros.
A package can be a single byte (see Polling below) or a string of 6 or more bytes making up an actual data telegram. A telegram is always in the format:
``[src] [dest] [type] [offset] [data] [crc] <BRK>``
I reference the first 4 bytes as the *header* in this document.
The first 4 bytes is referenced as the *header* in this document.
### EMS IDs
Each device has a unique ID.
The Boiler has an ID of 0x08 (type MC10) and also referred to as the Bus Master or UBA.
In this example a UBA boiler has an ID of 0x08 (such as a MC10) and also referred to as the Bus Master.
My thermostat, which is a* Moduline 300* uses the RC30 protocol and has an ID 0x17. If you're using a RC35 type thermostat such as the newer Moduline 300s or 400s use 0x10 and make adjustments in the code as appropriate. bbqkees did a nice write-up on his github page [here](https://github.com/bbqkees/Nefit-Buderus-EMS-bus-Arduino-Domoticz/blob/master/README.md).
Our circuit acts as a service key and thus uses an ID 0x0B. This ID is reserved for special devices intended for installation engineers for maintenance work.
The circuit acts as a service key and thus uses an ID 0x0B. This ID is reserved for special devices intended for service engineers.
### EMS Polling
The bus master (boiler) sends out a poll request every second by sending out a sequential list of all possible IDs as a single byte followed by the break signal. The ID always has its high 8th bit (MSB) set so in the code we're looking for 1 byte messages matching the format `[dest|0x80] <BRK>`.
Any connected device can respond to a Polling call with an acknowledging by sending back a single byte with its own ID. In our case we would listen for a `[0x8B] <BRK>` (meaning us) and then send back `[0x0B] <BRK>` to say we're alive and ready. Although I found this is not needed for normal operation so it's disabled as default in the code.
Any connected device can respond to a Polling request with an acknowledgement by sending back a single byte with its own ID. In our case we would listen for a `[0x8B] <BRK>` (meaning us) and then send back `[0x0B] <BRK>` to say we're alive and ready.
Polling is also the trigger to start transmitting any packages queued for sending. It must be done within 200ms or the bus master will time out.
@@ -168,7 +168,7 @@ Polling is also the trigger to start transmitting any packages queued for sendin
When a device is broadcasting to everyone there is no specific destination needed. `[dest]` is always 0x00.
The tables below shows which types are broadcasted regularly by the boiler (ID 0x08) and thermostat (ID 0x17). The **data length** is excluding the 4 byte header and CRC and the **Name** references those in the [ems wiki](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme).
The tables below shows which types are broadcasted regularly by the boiler (in this case ID 0x08) and thermostat (ID 0x17). The **data length** is excluding the 4 byte header and CRC and the **Name** references those in the German [ems wiki](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme).
| Source (ID) | Type ID | Name | Description | Data length | Frequency |
| ------------- | ------- | ------------------- | -------------------------------------- | ----------- | ---------- |
@@ -189,106 +189,86 @@ Refer to the code in `ems.cpp` for further explanation on how to parse these mes
### EMS Reading and Writing
Telegram packets can only be sent after the Boiler sends a poll to the sending device. The response can be a read command to request data or a write command to send data. At the end of the transmission a poll response is sent from the client (`<ID> <BRK>`) to say we're all done and free up the bus for other clients.
Telegrams can only be sent after the Master (boiler) sends a poll to the receiving device. The response can be a read command to request data or a write command to send data. At the end of the transmission a poll response is sent from the client (`<ID> <BRK>`) to say we're all done and free up the bus for other clients.
When doing a request to read data the `[src]` is our device `(0x0B)` and the `[dest]` must have has it's MSB (8th bit) set. Say we were requesting data from the thermostat we would use `[dest] = 0x97` since RC30 has an ID of 0x17.
When executing a request to read data the `[src]` is our device `(0x0B)` and the `[dest]` must have has it's MSB (8th bit) set. Say we were requesting data from the thermostat we would use `[dest] = 0x97` since RC20 has an ID of 0x17.
Following a write request, the `[dest]` doesn't have the 8th bit set and after this write request the destination device will send either a single byte 0x01 for success or 0x04 for failure.
Every telegram sent is echo'd back to Rx.
Every telegram sent is echo'd back to Rx, along the same Bus used for all Rx/Tx transmissions.
## The ESP8266 Source Code
*Disclaimer*: This code here is really for reference only, I don't expect anyone to use "as is" since it's highly tailored to my environment and my needs. Most of the code however is self explanatory with comments here and there in the code.
`emsuart.cpp` handles the low level UART read and write logic to the bus. You shouldn't need to touch this. All receive commands from the EMS bus are handled asynchronously using a circular buffer via an interrupt. A separate function processes the buffer and extracts the telegrams.
The code is built on the Arduino framework and is dependent on these external libraries:
`ems.cpp` is the logic to read the EMS data packets (telegrams), validates them and process them based on the type.
- Time http://playground.arduino.cc/code/time
- PubSubClient http://pubsubclient.knolleary.net
- ArduinoJson https://github.com/bblanchon/ArduinoJson
- CRC32 https://github.com/bakercp/CRC32
`ems-esp.ino` is the Arduino code for the ESP8266 that kicks it all off. This is where we have specific logic such as the code to monitor and alert on the Shower timer and light up the LEDs.
`emsuart.cpp` handles the low level UART read and write logic. You shouldn't need to touch this. All receive commands from the EMS bus are handled asynchronously using a circular buffer via an interrupt. A separate function processes the buffer and extracts the telegrams. Since we don't send too many write commands this is done sequentially. I couldn't use the standard Arduino Serial implementation because of the 11-bit break signal causes a frame-error which gets ignored.
`my_config.h` has all the custom settings tailored to your environment. Specific values here are also stored in the ESP's SPIFFs (File system).
`ems.cpp` is the logic to read the EMS packets (telegrams), validates them and process them based on the type.
`ems_devices.h` has all the configuration for the known EMS devices currently supported.
`boiler.ino` is the Arduino code for the ESP8266 that kicks it all off. This is where we have specific logic such as the code to monitor and alert on the Shower timer and light up the LEDs. LED support is enabled by setting the -DUSE_LED build flag.
`MyESP.cpp` is my custom library to handle WiFi, MQTT and Telnet. Uses a modified version of [TelnetSpy](https://github.com/yasheena/telnetspy)
`ESPHelper.cpp` is my customized version of [ESPHelper](https://github.com/ItKindaWorks/ESPHelper) with added Telnet support and some other minor tweaking.
### Supported EMS Types
### Special EMS Types
`ems.cpp` defines callback functions that handle all the broadcast types listed above (e.g. 0x34, 0x18, 0x19 etc) plus these extra types:
| Source (ID) | Type ID | Name | Description |
| ----------------- | ---------------- | ----------------------------- | ---------------------------------------- |
| Boiler (0x08) | 0x33 | UBAParameterWW | reads selected & desired warm water temp |
| Boiler (0x08) | 0x14 | UBATotalUptimeMessage | |
| Boiler (0x08) | 0x15 | UBAMaintenanceSettingsMessage | |
| Boiler (0x08) | 0x16 | UBAParametersMessage | |
| Thermostat (0x17) | 0xA8 | RC20Set | sets operating modes for an RC20 |
| Thermostat (0x10) | 0xA7 | RC30Set | sets operating modes for an RC30 |
| Thermostat | 0x02 | Version | reads Version major/minor |
| Thermostat | 0x91, 0x41, 0x0A | Status Message | read monitor values |
| Source (ID) | Type ID | Name | Description |
| ------------- | ------- | ----------------------------- | ---------------------------------------- |
| Boiler (0x08) | 0x33 | UBAParameterWW | reads selected & desired warm water temp |
| Boiler (0x08) | 0x14 | UBATotalUptimeMessage | |
| Boiler (0x08) | 0x15 | UBAMaintenanceSettingsMessage | |
| Boiler (0x08) | 0x16 | UBAParametersMessage | |
In `boiler.ino` you can make calls to automatically send these read commands. See the function *regularUpdates()*
In `ems.cpp` you can add scheduled calls to specific EMS types in the functions `ems_getThermostatValues()` and `ems_getBoilerValues()`.
### Supported Thermostats
### Which thermostats are supported?
Modify `EMS_ID_THERMOSTAT` in `myconfig.h` to the thermostat type you want to support.
I am still working on adding more support to known thermostats. Any contributions here are welcome. The know types are listed in `ems_devices.h` and include
#### RC20 (Moduline 300)
Read and write of setpoint temp and mode supported.
#### RC30 (Moduline 400)
Read and write of setpoint temp and mode supported.
Type's 3F, 49, 53, 5D are identical. So are 4B, 55, 5F and mostly zero's. Types 40, 4A, 54 and 5E are also the same.
#### RC35
***not implemented yet***!
An RC35 thermostat can support up to 4 heating circuits each controlled with their own Monitor and Working Mode IDs.
Fetching the thermostats setpoint temp us by requesting 0x3E and looking at the 3rd byte in the data telegram (data[2]) and dividing by 2.
The mode is on type 0x47 (or 0x3D) and the 8th byte (data[7]). 0=off, 1=on, 2=auto
#### TC100/TC200 (Nefit Easy)
There is limited support for an Nefit Easy TC100/TC200 type thermostat. The current room temperature and setpoint temperature can be read. What I'm still figuring out is how to read the mode and set the temperature values without sending http post commands to their web server.
- RC20 and RC30, both are fully supported
- RC10 support is being added
- RC35 with support for the 1st heating circuit (HC1)
- TC100/TC200/Easy but only with support for reading the temperatures. There seems to be no way to set settings using EMS bus messages that I know of. One option is to send XMPP messages but a special server is needed and out of scope for this project.
### Customizing The Code
- To configure for your thermostat and specific boiler settings, modify `my_config.h`. Here you can
- set the thermostat type. The default ID is 0x17 for an RC30 Moduline 300.
- set flags for enabled/disabling functionality such as `BOILER_THERMOSTAT_ENABLED`, `BOILER_SHOWER_ENABLED` and `BOILER_SHOWER_TIMER`.
- Set WIFI and MQTT settings, instead of doing this in `platformio.ini`
- set flags for enabled/disabling functionality such as `BOILER_SHOWER_ENABLED` and `BOILER_SHOWER_TIMER`.
- Set WIFI and MQTT settings. The values can also be set from the telnet command menu using the **set** command.
- To add new handlers for EMS data types, first create a callback function and add to the `EMS_Types` array at the top of the file `ems.cpp` and modify `ems.h`
- To add new devices modify `ems_devices.h`
### Using MQTT
When the ESP8266 boots it will send a start signal via MQTT. This is picked up by Home Assistant and sends a notification informing me that the device has booted. Useful for knowing when the ESP gets reset.
The boiler data is collected and sent as a single JSON object to MQTT TOPIC `home/ems-esp/boiler_data`. The `home` preifx is the MQTT topic prefix and can be customized in `my_config.h`. A hash is generated (CRC32 based) to determine if the payload has changed, otherwise it will not be sent. An example payload looks like:
I'm using the standard PubSubClient client so make sure you set `-DMQTT_MAX_PACKET_SIZE=400` as the default package size is 128 and our JSON messages are around 300 bytes.
`{"wWSelTemp":"60","selFlowTemp":"5.0","outdoorTemp":"?","wWActivated":"on","wWComfort":"Comfort","wWCurTmp":"46.0","wWCurFlow":"0.0","wWHeat":"on","curFlowTemp":"54.2","retTemp":"51.5","burnGas":"off","heatPmp":"off","fanWork":"off","ignWork":"off","wWCirc":"off","selBurnPow":"0","curBurnPow":"0","sysPress":"1.2","boilTemp":"56.7","pumpMod":"0","ServiceCode":"0H"}`
I run Mosquitto on my Raspberry PI 3 as the MQTT broker.
Similarly the thermostat values are also sent as a JSON package with the topic `home/ems-esp/thermostat_data` along with the current mode, room temperature and set temperature:
The boiler data is collected and sent as a single JSON object to MQTT TOPIC `home/boiler/boiler_data` (or `{hostname}/boiler_data` for ESPurna). A hash is generated (CRC32 based) to determine if the payload has changed, otherwise don't send it. An example payload looks like:
`{"thermostat_currtemp":"19.8","thermostat_seltemp":"16.0","thermostat_mode":"manual"}`
`{"wWCurTmp":"43.0","wWHeat":"on","curFlowTemp":"51.7","retTemp":"48.0","burnGas":"off","heatPmp":"off","fanWork":"off","ignWork":"off","wWCirc":"off","selBurnPow":"0","curBurnPow":"0","sysPress":"1.6","boilTemp":"54.7","pumpMod":"4"}`
These incoming MQTT topics are also handled:
Similarly the thermostat values are sent as a json package under a topic named `home/boiler/thermostat_data` with the current mode, room temperature and set temperature.
| topic | ID in my_config.h | Payload | Description |
| ------------------- | ------------------------- | ---------------------- | ---------------------------------------- |
| thermostat_cmd_temp | TOPIC_THERMOSTAT_CMD_TEMP | temperature as a float | sets the thermostat current setpoint |
| thermostat_cmd_mode | TOPIC_THERMOSTAT_CMD_MODE | auto, day, night | sets the thermostat mode |
| wwactivated | TOPIC_BOILER_WWACTIVATED | 0 or 1 | turns boiler warm water on/off (not tap) |
| boiler_cmd_wwtemp | TOPIC_BOILER_CMD_WWTEMP | temperature as a float | sets the boiler wwtemp current setpoint |
These topics can be configured in the `TOPIC_*` defines in `boiler.ino`. Make sure you change the HA configuration too to match.
If MQTT is not used use 'set mqtt_host' to remove it.
Some home automation systems such as Domoticz and OpenHab have special formats for their MQTT messages so I would advise to use [node-red](https://nodered.org/) as a parser like in [this example](https://www.domoticz.com/forum/download/file.php?id=18977&sid=67d048f1b4c8833822175eac6b55ecff).
### The Basic Shower Logic
Checking whether the shower is running was tricky. We know when the warm water is on and being heated but need to distinguish between the central heating, shower, hot tap and bath. I found via trial and error the Selected Burner Max Power is between 80% and 115% when the shower is running and fixed at 75% if the central heating is on. Furthermore the Selected Flow Impulsion is 80 C for the heating.
Checking whether the shower is running is tricky. We know when the warm water is on and being heated but need to distinguish between the central heating, shower, hot tap and even a bath tap. So this code is a little experimental.
There is other logic in the code to compensate for ramp up and whether the shower is turned off and back on again quickly within a 10 second window.
There is other logic in the code to compensate for water heating up to shower temperature and whether the shower is turned off and back on again quickly within a 10 second window.
## Home Assistant Configuration
@@ -300,122 +280,85 @@ and the alerts on an iOS/Android device using PushBullet, PushOver or any notifi
![Home Assistant iPhone notify)](doc/home_assistant/ha_notify.jpg)
You can find the .yaml configuration files under `doc/ha`. See also https://community.home-assistant.io/t/thermostat-and-boiler-controller-for-ems-based-boilers-nefit-buderus-bosch-using-esp/53382
You can find the .yaml configuration files under `doc/ha`. See also this [HA forum post](https://community.home-assistant.io/t/thermostat-and-boiler-controller-for-ems-based-boilers-nefit-buderus-bosch-using-esp/53382).
## Building The Firmware
### Using PlatformIO Standalone
PlatformIO is my preferred way. The code uses a modified version [ESPHelper](https://github.com/ItKindaWorks/ESPHelper) which handles all the basic handling of the WiFi, MQTT, OTA and Telnet server. I switched from Atom to the marvelous Visual Studio Code, works on Windows, OSX and Linux.
**On Windows:**
- Download [Git](https://git-scm.com/download/win) (install using the default settings)
- Download and install [Visual Studio Code](https://code.visualstudio.com/docs/?dv=win) (VSC). It's like 40MB so don't confuse with the commercial Microsoft Visual Studio.
- Restart the PC (if using Windows) to apply the new PATH settings. It should now detect Git
- Install these VSC extensions: PlatformIO IDE & GitLens, and then click reload to activate them
- Install the VSC extension "PlatformIO IDE" then click reload to activate it
- Git clone this repo, eith using `git clone` from PlatformIO's terminal or the Git GUI interface
- Create a `platformio.ini` based on the `platformio.ini-example` making the necessary changes for your WiFi and MQTT credentials in the build flags. If you're not using MQTT leave MQTT_IP empty (`MQTT_IP=""`)
- Create a `platformio.ini` based on the `platformio.ini-example` making the necessary changes for your board type
**On Linux (e.g. Ubuntu under Windows10):**
**On Linux (e.g. Ubuntu under Windows 10):**
- make sure Python 2.7 is installed
- make sure you have a Linux distro installed (https://docs.microsoft.com/en-us/windows/wsl/install-win10)
- Do:
Make sure Python 2.7 is installed, then...
```python
% pip install -U platformio
% sudo platformio upgrade
% platformio platform update
% git clone https://github.com/proddy/EMS-ESP-Boiler.git
% cd EMS-ESP-Boiler
% git clone https://github.com/proddy/EMS-ESP.git
% cd EMS-ESP
% cp platformio.ini-example platformio.ini
```
- edit `platformio.ini` to set `env_default` and the flags `WIFI_SSID WIFI_PASSWORD, MQTT_IP, MQTT_USER, MQTT_PASS`. If you're not using MQTT leave MQTT_IP empty (`MQTT_IP=""`)
edit `platformio.ini` to set `env_default` to your board type, then
```c
% platformio run -t upload
```
### Using ESPurna
[ESPurna](https://github.com/xoseperez/espurna/wiki) is framework that handles most of the tedious tasks of building IoT devices so you can focus on the functionality you need. This replaces my ESPHelper code in the standalone version above. ESPurna is natively built on PlatformIO and Visual Studio Code too which is nice. The latest version I tested this on is 1.13.3. So if you're brave, follow these steps:
1. Download and install [NodeJS](https://nodejs.org/en/download). This gives you npm. Choose the LTS version
2. Download ESPurna by cloning the ESPurna git repository (command palette, git clone, https://github.com/xoseperez/espurna.git)
3. Restart VSC.
4. From VSC open the folder `espurna\code`. PlatformIO should detect and set some things up for you automagically
5. open a terminal window (*ctrl-`*)
6. Install the node modules: `npm install --only=dev`
7. Build the web interface: `node node_modules/gulp/bin/gulp.js`. This will create a compressed `code/espurna/static/index.html.gz.h`. If you get warnings about lf during the building edit `gulpfile.js` and change the line `'failOnError': true` to `false` as a temporary workaround.
8. Modify the platformio.ini file making sure you add `-DUSE_CUSTOM_H -DUSE_EXTRA` to the `debug_flags`
9. Copy the following files from EMS-ESP-Boiler repo to where you installed ESPurna
```c
espurna/index.html -> code/html/index.html
espurna/custom.h -> code/config/custom.h
espurna/boiler-espurna.ino -> code/espurna/boiler-espurna.ino
ems*.* -> code/espurna/
```
10. Now build and upload as you usually would with PlatformIO (or ctrl-arl-t and choose the right build). Look at my version of platformio.ini as an example.
11. When the firmware loads, use a wifi connected pc/mobile to connect to the Access Point called ESPURNA_XXXXXX. Use 'fibonacci' as the password. Navigate to `http://192.168.4.1` from a browser, set a new username and password when prompted, log off the wifi and reconnect to the AP using these new credentials. Again go to 192.168.4.1
12. In the ADMIN page enable Telnet and SAVE
13. In the WIFI page add your home wifi details, click SAVE and reboot, and go to the new IP
14. Configure MQTT
The Telnet functions are `BOILER.READ`, `BOILER.INFO` and a few others for reference. `HELP` will list them. Add your own functions to expand the functionality by calling the EMS* functions as in the examples.
If you run into issues refer to ESPurna's official setup instructions [here](https://github.com/xoseperez/espurna/wiki/Build-and-update-from-Visual-Studio-Code-using-PlatformIO) and [here](https://github.com/xoseperez/espurna/wiki/Configuration).
This is what ESPurna looks like with the custom boiler code:
![Example running in ESPurna](doc/espurna/example.PNG)
*Note: I didn't bother porting all the EMS Read and Write commands from the Telnet code to the Espurna, but its pretty straight forward if you want to extend the code.*
### Using Pre-built Firmware
pre-baked firmwares for some ESP8266 devices based on ESPurna are available in the directory `/firmware` which you can upload yourself using [esptool](https://github.com/espressif/esptool) bootloader. On Windows, follow these instructions:
1. Check if you have **python 2.7** installed. If not [download it](https://www.python.org/downloads/) and make sure you select the option to add Python to the windows PATH
2. Install the ESPTool by running `pip install esptool` from a command prompt
3. Connect the ESP via USB, figure out the COM port
4. run `esptool.py -p <com> write_flash 0x00000 <firmware>` where firmware is the `.bin` file and \<com\> is the COM port, e.g. `COM3`
now follow the steps in ESPurna section above from #10 on to configure the device.
### Building Using Arduino IDE
Porting to the Arduino IDE can be a little tricky but it is possible.
Porting to the Arduino IDE can be a little tricky but it did it once. Something along these lines:
- Add the ESP8266 boards (from Preferences add Additional Board URL `http://arduino.esp8266.com/stable/package_esp8266com_index.json`)
- Go to Boards Manager and install ESP8266 2.4.x platform
- Go to Boards Manager and install ESP8266 2.4.x platform. Make sure your board supports SPIFFS.
- Select your ESP8266 from Tools->Boards and the correct port with Tools->Port
- From the Library Manager install ArduinoJson 5.13.x, PubSubClient 2.6.x, CRC32 and Time
- The Arduino IDE doesn't have a common way to set build flags (ugh!) so you'll need to un-comment these lines in `boiler.ino`:
- From the Library Manager install the needed libraries from platformio.ini. Note make sure you pick ArduinoJson v5 (5.13.4 and above) and not v6. See https://arduinojson.org/v5/doc/
- Put all the files in a single sketch folder
- cross your fingers and hit CTRL-R to compile
```c
#define WIFI_SSID "<my_ssid>"
#define WIFI_PASSWORD "<my_password>"
#define MQTT_IP "<broker_ip>"
#define MQTT_USER "<broker_username>"
#define MQTT_PASS "<broker_password>"
```
## Using the Pre-built Firmware
- Put all the files in a single sketch folder (`ESPHelper.*, boiler.ino, ems.*, emsuart.*`)
- cross your fingers and hit CTRL-R to compile...
pre-baked firmware for the Wemos D1 mini is available in the GitHub [releases](https://github.com/proddy/EMS-ESP/releases) which you can upload yourself using the [esptool](https://github.com/espressif/esptool) bootloader like `esptool.py -p <com port> write_flash 0x00000 <firmware.bin file>`. Here's how to set it up on Windows:
1. Check if you have **python 2.7** installed. If not [download it](https://www.python.org/downloads/) and make sure you select the option to add Python to the windows PATH
2. Then install the ESPTool by running `pip install esptool` from a command prompt
The ESP8266 will start in Access Point (AP) mode. Connect via WiFi to the SSID **EMS-ESP** and telnet to **192.168.4.1**. Then use the `set wifi` command to configure your own network settings like `set wifi your_ssid your_password`. Alternatively connect the ESP8266 to your PC and open a Serial monitor (with baud 115200) to configure the settings. Make sure you disable Serial support before connecting the EMS lines using `set serial off`.
`set` wil list all currently stored settings.
`set erase` will clear all settings.
## Troubleshooting
When flashing for the first time the Serial port is enabled by default with baud 115200. You can then use a PC with USB to the ESP8266 to set the settings like wifi, mqtt etc and also monitor the boot up procedure. Remember to disable the serial (`set serial off`) when connecting to the EMS lines.
The onboard LED will flash if there is no connection with the EMS bus. You can disable LED support by the 'set led' command from the telnet client.
If you want to completely erase the ESP and rebuild the firmware then do a `pio run -t erase` which will wipe the onboard flash including the SPIFFs where all the settings are stored.
## Known Issues
Some annoying issues that need fixing:
- Very infrequently an EMS write command is not sent, probably due to a collision somewhere in the UART between an incoming Rx and a Poll. The retries in the code fix this for now.
- On newer EMS+ Boilers the Tx commands for reading and writing may not always work. I believe there is some handshake that needs to happen before the UBA3/Master is able to send a poll request to our service device.
## Wish List
- Measure amount of gas in m3 per day for the hot water vs the central heating, and convert this into cost in Home Assistant
- Support changing temps on an Nefit Easy. To do this you must send XMPP messages directly to the thermostat. See this project: https://github.com/robertklep/nefit-easy-core
- Store custom params like wifi credentials, mqtt, thermostat type on ESP8266 using SPIFFS
- Automatic detection of thermostat type
- Add support for a temperature sensor on the circuit (DS18B20)
- Measure amount of gas in m3 per day for the hot water vs the central heating, and convert this into cost.
- Support changing temperatures on an Nefit Easy. To do this we must send XMPP messages directly to the thermostat. There is already a TCP stack and a Wifi and Telnet server running in the code, so the building blocks are there to extend with another XMPP client. Here are a number of Python based projects that show how to do this:
- https://github.com/patvdleer/nefit-client-python
- https://github.com/marconfus/ha-nefit
- https://github.com/robertklep/nefit-easy-core
- Improve detection of Heating Off without checking for selFlowTemp (selected flow temperature)
- Split MQTT into smaller chunks. Now the messages can be up to 600 bytes which may cause issues. Preferably make the items configurable.
## Your Comments and Feedback
@@ -423,4 +366,4 @@ Any comments, suggestions or code contributions are very welcome. Please post a
## DISCLAIMER
This code and libraries were developed from information gathered on the internet and many hours of reverse engineering the communications between the EMS bus and thermostats. It is **not** based on any official documentation or supported libraries from Buderus/Junkers/Nefit (and associated companies) and therefore there are no guarantees whatsoever regarding the safety of your devices and/or their settings, or the accuracy of the information provided.
This code and libraries were developed from information gathered on the internet and many hours of reverse engineering the communications between the EMS bus and thermostats. It is **not** based on any official documentation or supported libraries from Buderus/Junkers/Nefit (and associated companies) and therefore there are no guarantees whatsoever regarding the safety of your devices and/or their settings, or the accuracy of the information provided.