text changes

This commit is contained in:
Derbyshire
2019-01-16 23:44:17 -02:00
parent 7c71096e38
commit 873330a31a
2 changed files with 52 additions and 62 deletions

112
README.md
View File

@@ -1,11 +1,11 @@
# EMS-ESP
EMS-ESP is a project to build a controller circuit running with an ESP8266 to communicate with EMS (Energy Management System) based Boilers and Thermostats from the Bosch range and compatibles such as Buderus, Nefit, Junkers etc.
EMS-ESP is a project to build an electronic controller circuit using an Espressif ESP8266 microcontroller to communicate with EMS (Energy Management System) based Boilers and Thermostats from the Bosch range and compatibles such as Buderus, Nefit, Junkers etc.
There are 3 parts to this project, first the design of the circuit, second the code for the ESP8266 microcontroller firmware and lastly an example configuration for Home Assistant to monitor the data and issue direct commands via MQTT.
There are 3 parts to this project, first the design of the circuit, secondly the code for the ESP8266 microcontroller firmware and lastly an example configuration for Home Assistant to monitor the data and issue direct commands via MQTT.
[![Codacy Badge](https://api.codacy.com/project/badge/Grade/b8880625bdf841d4adb2829732030887)](https://app.codacy.com/app/proddy/EMS-ESP?utm_source=github.com&utm_medium=referral&utm_content=proddy/EMS-ESP&utm_campaign=Badge_Grade_Settings)
[![version](https://img.shields.io/badge/version-1.3.0-brightgreen.svg)](CHANGELOG.md)
[![version](https://img.shields.io/badge/version-1.3.1-brightgreen.svg)](CHANGELOG.md)
- [EMS-ESP](#ems-esp)
- [Introduction](#introduction)
@@ -39,33 +39,33 @@ There are 3 parts to this project, first the design of the circuit, second the c
## Introduction
My original intention for this home project was to build a custom smart thermostat that interfaces with my Nefit Trendline HRC30 boiler and have it controlled via a mobile app using [Home Assistant](https://www.home-assistant.io/). I had a few cheap ESP32s and ESP8266s microcontrollers lying around from previous IoT projects and learning how to build a circuit to decode the EMS messages seemed like a nice challenge.
The original intention for this home project was to build a custom smart thermostat that interfaces with my Nefit Trendline HRC30 boiler and have it controlled via a mobile app using MQTT. I had a few cheap ESP32s and ESP8266s microcontrollers lying around from previous IoT projects and learning how to build a circuit to decode the EMS bus messages seemed like a nice challenge.
Firstly, some acknowledgments and kudos to the following people who have open-sourced their projects which have helped me tremendously:
Acknowledgments and kudos to the following people who have open-sourced their projects:
**susisstrolch** - Probably the first working version of the EMS bridge circuit I found designed for the ESP8266. I borrowed Juergen's [schematic](https://github.com/susisstrolch/EMS-ESP12) and parts of his code logic for reading telegrams.
**susisstrolch** - One of the first working versions of the EMS bridge circuit I found designed for specifically for the ESP8266. I borrowed Juergen's [schematic](https://github.com/susisstrolch/EMS-ESP12) and parts of his code ideas for reading telegrams.
**bbqkees** - Kees built a working [circuit](https://shop.hotgoodies.nl/ems/) and some sample Arduino code to read from the EMS and push messages to Domoticz. His SMD board is also available for purchase.
**bbqkees** - Kees built a working [circuit](https://shop.hotgoodies.nl/ems/) and his SMD board is available for purchase on his website.
**EMS Wiki** - A comprehensive [reference](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme) (in German) for the EMS bus which is a little outdated, not 100% accurate and unfortunately no longer maintained.
**EMS Wiki** - A comprehensive [reference](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme) (in German) for the EMS bus which is a little outdated, not always 100% accurate and sadly no longer maintained.
## Supported Boilers Types
Most Bosch branded boilers that support the Logamatic EMS (and EMS+) bus protocols work with this design. This includes Nefit, Buderus, Worcester and Junkers (all copyrighted). Please make sure you read the **Disclaimer** carefully before sending ambiguous messages to your EMS bus as you could cause serious damage to your boiler or thermostat.
Most Bosch branded boilers that support the Logamatic EMS (and EMS+) bus protocols work with this design. This includes Nefit, Buderus, Worcester and Junkers (all copyrighted). Please make sure you read the **Disclaimer** carefully before sending ambiguous messages to your EMS bus as you could cause serious damage to your equiptment.
## Supported ESP8266 devices
I've tested the code and circuit with a few ESP8266 development boards such as the Wemos D1 Mini, Wemos D1 Mini Pro, Nodemcu0.9 and Nodemcu2 dev boards. It will also work on bare ESP8266 chips such as the ESP-12E but do make sure you disabled the LED support and wire the UART correctly as the code doesn't use the normal Rx and Tx pins. Why is explained later.
The code and circuit has been tested with a few ESP8266 development boards such as the Wemos D1 Mini, Wemos D1 Mini Pro, Nodemcu0.9 and Nodemcu2 dev boards. It will also work on bare ESP8266 chips such as the ESP-12E but do make sure you disabled the LED support and wire the UART correctly as the code doesn't use the normal Rx and Tx pins.
## Getting Started
1. Either build the circuit described below or purchase a ready built board from bbqkees via [website](https://shop.hotgoodies.nl/ems/).
2. Get an ESP8266 dev board and connect the 2 EMS output lines from the boiler to the circuit and the Rx and Tx out to ESP pins D7 and D8 respectively. The EMS connection can either be the 12-15V AC direct from the thermostat bus line or from the 3.5" Service Jack at the front.
3. Optionally connect an external LED or decide to use the onboard ESP8266 LED. This will flash when there is an error on the EMS bus line or stay solid when it's connected.
4. Modify `my_custom.h`
5. Build and upload the firmware to the ESP8266 device. I used PlatformIO with Visual Studio Code but using Atom or a command-line is just as easy if you don't plan to make code changes. Do make sure you set the MQTT and WiFi credentials correctly in the build flags and if you're not using MQTT leave the MQTT_HOST blank. The firmware supports OTA too with the default hostname as 'boiler'.
6. Power the ESP either via USB or direct into the 5v vin pin from an external power 5V volts supply with min 400mA.
7. Attach the 3v3 out on the ESP8266 to the DC power line on the EMS circuit as indicated in the schematics.
1. Either build the circuit described below or purchase a ready built board
2. Get an ESP8266 dev board and connect the 2 EMS output lines from the boiler to the circuit and the Rx and Tx out to ESP pins D7 and D8 respectively. To prevent interference with the serial ports and debug messages we use pins RX1 and TX1 (GPIO13/D7 and GPIO15/D8 respectively). The EMS connection can either be the 12-15V AC direct from the thermostat bus line or from the 3.5" Service Jack at the front of the boiler.
3. Optionally connect an external LED or decide to use the onboard ESP8266 LED. This will flash when there is an error on the EMS bus line or stay solid when it's connected to the EMS bus.
4. Modify `my_custom.h` if needed
5. Build and upload the firmware to the ESP8266 device. I used PlatformIO with Visual Studio Code but using Atom or a command-line is just as easy if you don't plan to make many code changes.
6. Power the ESP either via USB or direct into the 5v vin pin from an external 5V power source (with min 300mA is required).
7. Attach the 3v3 out on the ESP8266 to the DC power line on the EMS circuit as indicated in the schematics to power the circuit.
8. The WiFi connects via DHCP by default. Find the IP by from your router and then telnet (port 23) to it. If a connection can't be made it will go into Access Point mode. Tip: to enable Telnet on Windows run `dism /online /Enable-Feature /FeatureName:TelnetClient` or install something like [putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html). If everything is working you should see the messages appear in the window as shown in the next section. However if you're unable to locate the IP of the ESP then something went wrong. Re-compile with the -DDEBUG_SUPPORT and connect via USB to a PC and check the Serial log for errors.
## Monitoring The Output
@@ -82,27 +82,23 @@ To see the current stats and collected values type 'info'. Watch out for unsucce
![Telnet](doc/telnet/telnet_stats.PNG)
**Disclaimer: be careful when sending values to the boiler. If in doubt you can always reset the boiler to its original factory settings by following the instructions in the user guide. On my **Nefit Trendline HRC30** that is done by holding down the Home and Menu buttons simultaneously for a few seconds, selecting factory settings from the scroll menu followed by pressing the Reset button.**
**Disclaimer: be careful when sending values to the boiler. If in doubt you can always reset the boiler to its original factory settings by following the instructions in the user guide. On my **Nefit Trendline** that is done by holding down the Home and Menu buttons simultaneously for a few seconds, selecting factory settings from the scroll menu followed by pressing the Reset button.**
## Building The Circuit
The EMS circuit is really all credit to the hard work many people have done before me, noticeably *susisstrolch* with his ESP8266 [version](https://github.com/susisstrolch/EMS-ESP8266_12-PCB/tree/newmaster/Schematics/EMS-ESP8266-12).
I've included a prototype boards you can build yourself on a breadboard. One part for only reading values from the Boiler and an extension with the write logic so you can also send commands.
We need the Rx/Tx of the ESP8266 for flashing, so the code in ``emsuart.cpp`` switches the UART pins to use RX1 and TX1 (GPIO13/D7 and GPIO15/D8 respectively). This also prevents any bogus stack data being sent to EMS bus when the ESP8266 decides to crash with a Watch Dog (WD) Reset.
Included is a prototype boards you can build yourself on a breadboard.
The breadboard layout was done using [DIY Layout Creator](https://github.com/bancika/diy-layout-creator) and sources files are included in this repo.
![Breadboard Circuit](doc/schematics/breadboard.png)
The schematic used (as designed by [susisstrolch](https://github.com/susisstrolch/EMS-ESP8266_12-PCB)):
The schematic used:
![Schematic](doc/schematics/circuit.png)
*Optionally I've also added 2 0.5A/72V polyfuses between the EMS and the two inductors L1 and L2 for extra protection.*
And here's a version using bbqkees' circuit
And here's a version using an early prototype board from **bbqkees**:
![WemosD1](doc/schematics/wemos_kees.png)
@@ -121,29 +117,27 @@ The EMS circuit will work with both 3.3V and 5V. It's easiest though to power di
## How The EMS Bus Works
Packages are sent to the EMS "bus" from the Boiler and any other compatible connected devices via serial TTL transmission. The protocol is 9600 baud, 8N1 (8 bytes, no parity, 1 stop bit). Each package is terminated with a break signal `<BRK>`, a 11-bit long low signal of zeros.
Packages are streamed to the EMS "bus" from any other compatible connected device via serial TTL transmission using protocol 9600 baud, 8N1 (8 bytes, no parity, 1 stop bit). Each package is terminated with a break signal `<BRK>`, a 11-bit long low signal of zeros.
A package can be a single byte (see Polling below) or a string of 6 or more bytes making up an actual data telegram. A telegram is always in the format:
``[src] [dest] [type] [offset] [data] [crc] <BRK>``
I reference the first 4 bytes as the *header* in this document.
The first 4 bytes is referenced as the *header* in this document.
### EMS IDs
Each device has a unique ID.
The Boiler has an ID of 0x08 (type MC10) and also referred to as the Bus Master or UBA.
In this example a UBA boiler has an ID of 0x08 (such as a MC10) and also referred to as the Bus Master.
My thermostat, which is a* Moduline 300* uses the RC30 protocol and has an ID 0x17. If you're using a RC35 type thermostat such as the newer Moduline 300s or 400s use 0x10 and make adjustments in the code as appropriate. bbqkees did a nice write-up on his github page [here](https://github.com/bbqkees/Nefit-Buderus-EMS-bus-Arduino-Domoticz/blob/master/README.md).
Our circuit acts as a service key and thus uses an ID 0x0B. This ID is reserved for special devices intended for installation engineers for maintenance work.
The circuit acts as a service key and thus uses an ID 0x0B. This ID is reserved for special devices intended for service engineers.
### EMS Polling
The bus master (boiler) sends out a poll request every second by sending out a sequential list of all possible IDs as a single byte followed by the break signal. The ID always has its high 8th bit (MSB) set so in the code we're looking for 1 byte messages matching the format `[dest|0x80] <BRK>`.
Any connected device can respond to a Polling call with an acknowledging by sending back a single byte with its own ID. In our case we would listen for a `[0x8B] <BRK>` (meaning us) and then send back `[0x0B] <BRK>` to say we're alive and ready. Although I found this is not needed for normal operation so it's disabled as default in the code.
Any connected device can respond to a Polling request with an acknowledgement by sending back a single byte with its own ID. In our case we would listen for a `[0x8B] <BRK>` (meaning us) and then send back `[0x0B] <BRK>` to say we're alive and ready.
Polling is also the trigger to start transmitting any packages queued for sending. It must be done within 200ms or the bus master will time out.
@@ -151,7 +145,7 @@ Polling is also the trigger to start transmitting any packages queued for sendin
When a device is broadcasting to everyone there is no specific destination needed. `[dest]` is always 0x00.
The tables below shows which types are broadcasted regularly by the boiler (ID 0x08) and thermostat (ID 0x17). The **data length** is excluding the 4 byte header and CRC and the **Name** references those in the [ems wiki](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme).
The tables below shows which types are broadcasted regularly by the boiler (in this case ID 0x08) and thermostat (ID 0x17). The **data length** is excluding the 4 byte header and CRC and the **Name** references those in the German [ems wiki](https://emswiki.thefischer.net/doku.php?id=wiki:ems:telegramme).
| Source (ID) | Type ID | Name | Description | Data length | Frequency |
| ------------- | ------- | ------------------- | -------------------------------------- | ----------- | ---------- |
@@ -172,31 +166,29 @@ Refer to the code in `ems.cpp` for further explanation on how to parse these mes
### EMS Reading and Writing
Telegram packets can only be sent after the Boiler sends a poll to the sending device. The response can be a read command to request data or a write command to send data. At the end of the transmission a poll response is sent from the client (`<ID> <BRK>`) to say we're all done and free up the bus for other clients.
Telegrams can only be sent after the Master (boiler) sends a poll to the receiving device. The response can be a read command to request data or a write command to send data. At the end of the transmission a poll response is sent from the client (`<ID> <BRK>`) to say we're all done and free up the bus for other clients.
When doing a request to read data the `[src]` is our device `(0x0B)` and the `[dest]` must have has it's MSB (8th bit) set. Say we were requesting data from the thermostat we would use `[dest] = 0x97` since RC30 has an ID of 0x17.
When executing a request to read data the `[src]` is our device `(0x0B)` and the `[dest]` must have has it's MSB (8th bit) set. Say we were requesting data from the thermostat we would use `[dest] = 0x97` since RC20 has an ID of 0x17.
Following a write request, the `[dest]` doesn't have the 8th bit set and after this write request the destination device will send either a single byte 0x01 for success or 0x04 for failure.
Every telegram sent is echo'd back to Rx.
Every telegram sent is echo'd back to Rx, along the same Bus used for all Rx/Tx transmissions.
## The ESP8266 Source Code
*Disclaimer*: This code here is really for reference only, I don't expect anyone to use "as is" since it's highly tailored to my environment and my needs. Most of the code however is self explanatory with comments here and there in the code.
`emsuart.cpp` handles the low level UART read and write logic to the bus. You shouldn't need to touch this. All receive commands from the EMS bus are handled asynchronously using a circular buffer via an interrupt. A separate function processes the buffer and extracts the telegrams.
`emsuart.cpp` handles the low level UART read and write logic. You shouldn't need to touch this. All receive commands from the EMS bus are handled asynchronously using a circular buffer via an interrupt. A separate function processes the buffer and extracts the telegrams. Since we don't send too many write commands this is done sequentially. I couldn't use the standard Arduino Serial implementation because of the 11-bit break signal causes a frame-error which gets ignored.
`ems.cpp` is the logic to read the EMS packets (telegrams), validates them and process them based on the type.
`ems.cpp` is the logic to read the EMS data packets (telegrams), validates them and process them based on the type.
`ems-esp.ino` is the Arduino code for the ESP8266 that kicks it all off. This is where we have specific logic such as the code to monitor and alert on the Shower timer and light up the LEDs.
`my_config.h` all the custom settings
`my_config.h` has all the custom settings tailored to your environment. Specific values here are also stored in the ESP's SPIFFs (File system).
`ems_devices.h` has all the configuration for the known EMS devices this supports
`ems_devices.h` has all the configuration for the known EMS devices currently supported.
`MyESP.cpp` is my custom library to handle WiFi, MQTT, MDNS and Telnet. Uses a modified version of TelnetSpy (https://github.com/yasheena/telnetspy)
`MyESP.cpp` is my custom library to handle WiFi, MQTT, MDNS and Telnet. Uses a modified version of [TelnetSpy](https://github.com/yasheena/telnetspy)
### Supported EMS Types
### Special EMS Types
`ems.cpp` defines callback functions that handle all the broadcast types listed above (e.g. 0x34, 0x18, 0x19 etc) plus these extra types:
@@ -206,12 +198,8 @@ Every telegram sent is echo'd back to Rx.
| Boiler (0x08) | 0x14 | UBATotalUptimeMessage | |
| Boiler (0x08) | 0x15 | UBAMaintenanceSettingsMessage | |
| Boiler (0x08) | 0x16 | UBAParametersMessage | |
| Thermostat (0x17) | 0xA8 | RC20Set | sets operating modes for an RC20 |
| Thermostat (0x10) | 0xA7 | RC30Set | sets operating modes for an RC30 |
| Thermostat | 0x02 | Version | reads Version major/minor |
| Thermostat | 0x91, 0x41, 0x0A | Status Message | read monitor values |
In `ems-esp.ino` you can make calls to automatically request these types in the function *regularUpdates()*.
In `ems.cpp` you can add scheduled calls to specific EMS types in the functions `ems_getThermostatValues()` and `ems_getBoilerValues()`.
### Which thermostats are supported?
@@ -225,7 +213,7 @@ I am still working on adding more support to known thermostats. Any contribution
- To configure for your thermostat and specific boiler settings, modify `my_config.h`. Here you can
- set flags for enabled/disabling functionality such as `BOILER_SHOWER_ENABLED` and `BOILER_SHOWER_TIMER`.
- Set WIFI and MQTT settings, instead of doing this in `platformio.ini`. The values can be set from the telnet command menu using the **set** command.
- Set WIFI and MQTT settings. The values can also be set from the telnet command menu using the **set** command.
- To add new handlers for EMS data types, first create a callback function and add to the `EMS_Types` array at the top of the file `ems.cpp` and modify `ems.h`
- To add new devices modify `ems_devices.h`
@@ -235,10 +223,12 @@ The boiler data is collected and sent as a single JSON object to MQTT TOPIC `hom
`{"wWSelTemp":"60","selFlowTemp":"5.0","outdoorTemp":"?","wWActivated":"on","wWComfort":"Comfort","wWCurTmp":"46.0","wWCurFlow":"0.0","wWHeat":"on","curFlowTemp":"54.2","retTemp":"51.5","burnGas":"off","heatPmp":"off","fanWork":"off","ignWork":"off","wWCirc":"off","selBurnPow":"0","curBurnPow":"0","sysPress":"1.2","boilTemp":"56.7","pumpMod":"0","ServiceCode":"0H"}`
Similarly the thermostat values are sent as a json package under a topic named `home/ems-esp/thermostat_data` with the current mode, room temperature and set temperature like
Similarly the thermostat values are sent as a JSON package under the topic `home/ems-esp/thermostat_data` with the current mode, room temperature and set temperature like
`{"thermostat_currtemp":"19.8","thermostat_seltemp":"16.0","thermostat_mode":"manual"}`
If MQTT is not used set the MQTT_HOST to `NULL`.
### The Basic Shower Logic
Checking whether the shower is running is tricky. We know when the warm water is on and being heated but need to distinguish between the central heating, shower, hot tap and even a bath tap. So this code is a little experimental.
@@ -255,7 +245,7 @@ and the alerts on an iOS/Android device using PushBullet, PushOver or any notifi
![Home Assistant iPhone notify)](doc/home_assistant/ha_notify.jpg)
You can find the .yaml configuration files under `doc/ha`. See also https://community.home-assistant.io/t/thermostat-and-boiler-controller-for-ems-based-boilers-nefit-buderus-bosch-using-esp/53382
You can find the .yaml configuration files under `doc/ha`. See also this [HA forum post](https://community.home-assistant.io/t/thermostat-and-boiler-controller-for-ems-based-boilers-nefit-buderus-bosch-using-esp/53382).
## Building The Firmware
@@ -266,15 +256,13 @@ You can find the .yaml configuration files under `doc/ha`. See also https://comm
- Download [Git](https://git-scm.com/download/win) (install using the default settings)
- Download and install [Visual Studio Code](https://code.visualstudio.com/docs/?dv=win) (VSC). It's like 40MB so don't confuse with the commercial Microsoft Visual Studio.
- Restart the PC (if using Windows) to apply the new PATH settings. It should now detect Git
- Install the VSC extensions "PlatformIO IDE" then click reload to activate it
- Install the VSC extension "PlatformIO IDE" then click reload to activate it
- Git clone this repo, eith using `git clone` from PlatformIO's terminal or the Git GUI interface
- Create a `platformio.ini` based on the `platformio.ini-example` making the necessary changes for your board type
**On Linux (e.g. Ubuntu under Windows 10):**
- make sure Python 2.7 is installed
- make sure you have a Linux distro installed (https://docs.microsoft.com/en-us/windows/wsl/install-win10)
- Then...
Make sure Python 2.7 is installed, then...
```python
% pip install -U platformio
% sudo platformio upgrade
@@ -284,7 +272,7 @@ You can find the .yaml configuration files under `doc/ha`. See also https://comm
% cd EMS-ESP
% cp platformio.ini-example platformio.ini
```
- edit `platformio.ini` to set `env_default` to your board type
edit `platformio.ini` to set `env_default` to your board type, then
```c
% platformio run -t upload
```
@@ -298,7 +286,7 @@ Porting to the Arduino IDE can be a little tricky but it is possible.
- Select your ESP8266 from Tools->Boards and the correct port with Tools->Port
- From the Library Manager install the needed libraries from platformio.ini
- Put all the files in a single sketch folder
- cross your fingers and hit CTRL-R to compile...
- cross your fingers and hit CTRL-R to compile
## Using the Pre-built Firmware
@@ -309,7 +297,9 @@ pre-baked firmwares for some ESP8266 devices are available in the directory `/fi
3. Connect the ESP via USB, figure out the COM port
4. run `esptool.py -p <com> write_flash 0x00000 <firmware>` where firmware is the `.bin` file and \<com\> is the COM port, e.g. `COM3`
The ESP8266 will start in Access Point (AP) mode, so connect via WiFi to the SSID **EMS-ESP** and telnet to 192.168.4.1. Then use the set command to configure your own network settings.
The ESP8266 will start in Access Point (AP) mode. Connect via WiFi to the SSID **EMS-ESP** and telnet to **192.168.4.1**. Then use the set command to configure your own network settings.
`set erase` will clear all settings. `set` wil show all settings.
## Troubleshooting
@@ -325,8 +315,8 @@ Some annoying issues that need fixing:
## Wish List
- Measure amount of gas in m3 per day for the hot water vs the central heating, and convert this into cost in Home Assistant
- Support changing temperatures on an Nefit Easy. To do this you must send XMPP messages directly to the thermostat. There is already a TCP stack and a Wifi and Telnet server running in the code, so the building blocks are there to extend with another XMPP client. Here are a number of Python based projects that show how to do this:
- Measure amount of gas in m3 per day for the hot water vs the central heating, and convert this into cost.
- Support changing temperatures on an Nefit Easy. To do this we must send XMPP messages directly to the thermostat. There is already a TCP stack and a Wifi and Telnet server running in the code, so the building blocks are there to extend with another XMPP client. Here are a number of Python based projects that show how to do this:
- https://github.com/patvdleer/nefit-client-python
- https://github.com/marconfus/ha-nefit
- https://github.com/robertklep/nefit-easy-core

View File

@@ -401,10 +401,10 @@ void MyESP::_consoleShowHelp() {
#endif
SerialAndTelnet.println("*\n\r* Commands:\n\r* ?=help, CTRL-D=quit, !=reboot");
SerialAndTelnet.println(FPSTR("* set"));
SerialAndTelnet.println(FPSTR("* set <wifi_ssid | wifi_password | mqtt_host | mqtt_username | mqtt_password> [value]"));
SerialAndTelnet.println(FPSTR("* set erase"));
SerialAndTelnet.println(FPSTR("* set led <on | off>"));
SerialAndTelnet.println(FPSTR("*"));
// print custom commands if available. Take from progmem