diff --git a/README.md b/README.md index 8b77b170d..28302370c 100644 --- a/README.md +++ b/README.md @@ -8,24 +8,25 @@ Note: Version 2.0 is not backward compatible with v1.0. The File system structur ## Major changes since version 1.9.x -### **Design & Coding principles** +### **Design & coding principles** -- The code can be built and run without an ESP microcontroller, which is useful with testing and simulating handling of telegrams. Make sure you have GNU make and g++ installed and use 'make' to build the image and execute the file `emsesp` (on linux). +- The code can be built and run without an ESP microcontroller, which is useful when testing and simulating handling of the different telegrams and devices. Make sure you have GNU make and g++ installed and use 'make' to build the image and execute the file `emsesp` (on linux). - I used C++11 containers where I could (std::string, std::deque, std::list, std::multimap etc). - The core is based off the great libraries from @nomis' and I adopted his general design pattens such as making everything as asynchronous as possible so that no one operation should starve another operation of it's time to execute (https://isocpp.org/wiki/faq/ctors#static-init-order). - All EMS devices (e.g. boiler, thermostat, solar modules etc) are derived from a factory base class and each class handles its own registering of telegram and mqtt handlers. This makes the EMS device code easier to manage and extend with new telegrams types and features. - Built to work with both EMS8266 and ESP32. +- Extended MQTT to use MQTT discovery on Home Assistant, just for the thermostat ### **Features** - A web interface built using React and TypeScript to be secure and cross-browser compatible. Each restful endpoint is protected and issues a JWT which is then sent using Bearer Authentication. Implements a Web captive portal. On first installs EMS-ESP starts an Access Point where system settings can be configured. Note, this is still in a separate repo and pending a merge into this project. -- A new console. Like 1.9.x it works with both Serial and Telnet but a lot more intuitive behaving like a Linux shell and secure. Multiple telnet sessions are supported now but watch out for slow connections and low memory. A password is need to change any settings. You can use TAB to auto-complete commands. Some key commands: +- A new console. Like 1.9.x it works with both Serial and Telnet but a lot more intuitive behaving like a Linux shell and secure. Multiple telnet sessions are supported now but watch out for slow connections and low memory. A password is need to change any settings. You can use TAB to auto-complete commands, ctrl-L, ctrl-U and the other typical console type shortcuts. Some key commands: * `help` lists the commands and keywords * some commands take you into a new context, a bit like a sub-menu. e.g. `system`, `mqtt`, `thermostat`. Use `help` to show which commands this context has and `exit` to get back to the root. * To change a setting use the `set` command. Typing `set` shows the current settings. * `show` shows the data specific to the context you're in. - * `su` to switch to Admin which enables more commands such as most of the `set` commands. The default password is "neo". When in Admin mode the command prompt switches from `$` to `#`. + * `su` to switch to Admin which enables more commands such as most of the `set` commands. The default password is "neo" which can be changed with `passwd` from the system menu. When in Admin mode the command prompt switches from `$` to `#`. * `log` sets the logging. `log off` disables logging. Use `log trace` to see the telegram traffic and `log debug` for very verbose logging. To watch a specific telegram ID or device ID use `log trace [id]`. - There is no "serial mode" anymore like with version 1.9. When the Wifi cannot connect to the SSID it will automatically enter a "safe" mode where the Serial console is activated, baud 115200. Note Serial is always available on the ESP32 because it has 2 UARTs. @@ -34,7 +35,7 @@ Note: Version 2.0 is not backward compatible with v1.0. The File system structur - on a new install you will want to enter `su` and then go to the `system` context. Use `set wifi ...` to set the network up. Then go to the `mqtt` context to set the mqtt up. -# Full Console +# Full Console Commands ``` common commands available in all contexts: @@ -43,15 +44,15 @@ common commands available in all contexts: log [level] [trace ID] su -top level/root +(top level) refresh show show version - ems (is a menu) - mqtt (is a menu) - system (is a menu) - boiler (is a menu) - thermostat (is a menu) + ems (enters a context) + mqtt (enters a context) + system (enters a context) + boiler (enters a context) + thermostat (enters a context) ems scan devices [deep] @@ -72,7 +73,7 @@ mqtt set enabled set heartbeat set ip - set nested_json + set format set password set publish_time set qos @@ -125,8 +126,8 @@ thermostat ``` TODO figure out why sometimes telnet on ESP32 (and sometimes ESP8266) has slow response times. After a manual reset it seems to fix itself. Perhaps the telnet service needs to start after the wifi is up & running. TODO Get the ESP32 UART code working. +TODO sometimes with tx_mode 0 there are a few CRC errors due to collision when waiting for a BRK signal. TODO console auto-complete with 'set' command in the system context is not showing all commands, only the hostname. -TODO sometimes get an error after a Tx send when first booting up. timeout perhaps? ``` ### **Features to add next** @@ -139,7 +140,6 @@ TODO validate 0xE9 with data from Koen. (https://github.com/proddy/EMS-ESP/issue ``` TODO replace vectors of class objects with shared pointers and use emplace_back since it instantiates during construction. It may have a performance gain. -TODO decide if we really need to store the timestamp of each incoming Rx telegram. TODO make more use of comparison operators in the Telegram class e.g. the compare like "friend inline bool operator==(const Telegram & lhs, const Telegram & rhs)" TODO exit from serial should be prevented? Because you never can really exit, just close it. TODO add real unit tests using platformio's test bed (https://docs.platformio.org/en/latest/plus/pio-remote.html) @@ -149,10 +149,6 @@ TODO See if it's easier to use timers instead of millis() timers, using https:// ### **These features to add next** ``` -TODO merge in the web code -TODO merge in NTP code -TODO make ascii colors in the console optional? -TODO decide what to do with gateways, switches and other bogus EMS devices -TODO add MQTT subscribe topic to toggle on/off the shower alert and timer. If really needed. +TODO merge in the web code which has the Captive AP and better wifi reconnect logic. Use IPV6 and NTP from lwip2 TODO decide if I want to port over the shower one-shot cold water logic. Don't think its used. ``` diff --git a/platformio.ini b/platformio.ini index bff78b4c3..41c00a9b5 100644 --- a/platformio.ini +++ b/platformio.ini @@ -2,7 +2,7 @@ ; For EMS-ESP [platformio] -;default_envs = esp8266 +default_envs = esp8266 ;default_envs = esp32 # override any settings with your own local ones in pio_local.ini @@ -45,11 +45,22 @@ libs_esp32 = [env] build_unflags = -fno-rtti ; for dynamic_cast<> -;lib_ldf_mode = chain+ +lib_ldf_mode = chain+ +lib_compat_mode = strict extra_scripts = scripts/main_script.py framework = arduino monitor_speed = 115200 upload_speed = 921600 +check_tool = cppcheck, clangtidy +check_severity = high, medium +check_flags = + cppcheck: --std=c++11 + clangtidy: --checks=-*,clang-analyzer-*,performance-* + +; example ports for OSX +;upload_port = /dev/cu.wchusbserial14403 +;upload_port = /dev/cu.usbserial-1440 +;upload_port = /dev/cu.SLAB_USBtoUART ; OTA upload_protocol = esptool @@ -63,8 +74,8 @@ upload_protocol = esptool build_type = release platform = espressif8266 ; https://github.com/platformio/platform-espressif8266/releases ;platform = espressif8266@2.4.0 ; Arduino core 2.6.3 -; board = esp12e -board = d1_mini ; https://github.com/platformio/platform-espressif8266/blob/master/boards/d1_mini.json +board = esp12e +; board = d1_mini ; https://github.com/platformio/platform-espressif8266/blob/master/boards/d1_mini.json lib_deps = ${common.libs_core} ${common.libs_esp8266} board_build.f_cpu = 160000000L ; 160MHz ;board_build.ldscript = eagle.flash.4m1m.ld ; 1019 KB sketch, 1000 KB SPIFFS. 4KB EEPROM, 4KB RFCAL, 12KB WIFI stack, 2052 KB OTA & buffer diff --git a/src/console.h b/src/console.h index c49d81e38..cb1868807 100644 --- a/src/console.h +++ b/src/console.h @@ -83,6 +83,7 @@ MAKE_PSTR_WORD(disconnect) MAKE_PSTR_WORD(debug) MAKE_PSTR_WORD(restart) MAKE_PSTR_WORD(reconnect) +MAKE_PSTR_WORD(format) // context menus MAKE_PSTR_WORD(mqtt) diff --git a/src/emsdevice.cpp b/src/emsdevice.cpp index 87aab3d5b..686ac800e 100644 --- a/src/emsdevice.cpp +++ b/src/emsdevice.cpp @@ -187,7 +187,7 @@ void EMSdevice::show_mqtt_handlers(uuid::console::Shell & shell) { } void EMSdevice::register_mqtt_topic(const std::string & topic, mqtt_function_p f) { - DEBUG_LOG(F("Registering new MQTT topic for device ID %02X"), this->device_id_); + DEBUG_LOG(F("Registering MQTT topic %s for device ID %02X"), topic.c_str(), this->device_id_); Mqtt::subscribe(this->device_id_, topic, f); } diff --git a/src/emsesp.cpp b/src/emsesp.cpp index 3064bdf32..009397c79 100644 --- a/src/emsesp.cpp +++ b/src/emsesp.cpp @@ -570,7 +570,8 @@ void EMSESP::incoming_telegram(uint8_t * data, const uint8_t length) { // are we waiting for a response from a recent Tx Read or Write? if (EMSbus::tx_waiting()) { - // if it's a single byte 1 or 4 then its a response from the last write + // if it's a single byte 1 or 4 then its maybe a response from the last write action + EMSbus::tx_waiting(false); // reset Tx wait state if (length == 1) { if (first_value == TxService::TX_WRITE_SUCCESS) { DEBUG_LOG(F("Last Tx write successful. Sending read request.")); @@ -581,13 +582,12 @@ void EMSESP::incoming_telegram(uint8_t * data, const uint8_t length) { DEBUG_LOG(F("Last Tx write rejected by host")); txservice_.send_poll(); // close the bus } else { -#ifdef EMSESP_DEBUG - logger_.err(F("Expecting Tx ACK (1/4) but got 0x%02X. Tx:%s"), first_value, txservice_.last_tx_to_string().c_str()); -#endif + // ignore it, it's probably a poll and we can wait for the next one + return; } - } else { - // got a telegram. See if the src/dest matches that from the last one we sent + // got a telegram with data in it. See if the src/dest matches that from the last one we sent + // and continue to process it uint8_t src = data[0]; uint8_t dest = data[1]; if (txservice_.is_last_tx(src, dest)) { @@ -605,18 +605,21 @@ void EMSESP::incoming_telegram(uint8_t * data, const uint8_t length) { } } } - EMSbus::tx_waiting(false); // reset Tx wait state } - // check for poll, if so, send Tx from the queue immediately - // if ht3 poll must be ems_bus_id else if Buderus poll must be (ems_bus_id | 0x80) - if ((length == 1) && ((first_value ^ 0x80 ^ rxservice_.ems_mask()) == txservice_.ems_bus_id())) { - EMSbus::last_bus_activity(millis()); // set the flag indication the EMS bus is active - txservice_.send(); + // check for poll + if (length == 1) { + // check for poll to us, if so send top message from Tx queue immediately and quit + // if ht3 poll must be ems_bus_id else if Buderus poll must be (ems_bus_id | 0x80) + if ((first_value ^ 0x80 ^ rxservice_.ems_mask()) == txservice_.ems_bus_id()) { + EMSbus::last_bus_activity(millis()); // set the flag indication the EMS bus is active + txservice_.send(); + } + return; + } else { + // add to RxQueue, what ever it is. + rxservice_.add(data, length); } - - // add to RxQueue - rxservice_.add(data, length); } // sends raw data of bytes along the Tx line diff --git a/src/mixing.cpp b/src/mixing.cpp index 22aa24277..c868a8b56 100644 --- a/src/mixing.cpp +++ b/src/mixing.cpp @@ -145,10 +145,15 @@ void Mixing::process_MMPLUSStatusMessage_WWC(std::shared_ptr tel } // Mixing on a MM10 - 0xAB -// We assume MM10 is on HC2 and WM10 is using HC1 - https://github.com/proddy/EMS-ESP/issues/270 +// e.g. Mixing Module -> All, type 0xAB, telegram: 21 00 AB 00 2D 01 BE 64 04 01 00 (CRC=15) #data=7 +// see also https://github.com/proddy/EMS-ESP/issues/386 void Mixing::process_MMStatusMessage(std::shared_ptr telegram) { type_ = Type::HC; - hc_ = 1; // fixed to circuit 1 + + // the heating circuit is determine by which device_id it is, 0x20 - 0x23 + // 0x21 is position 2. 0x20 is typically reserved for the WM10 switch module + // see https://github.com/proddy/EMS-ESP/issues/270 and https://github.com/proddy/EMS-ESP/issues/386#issuecomment-629610918 + hc_ = 0x22 - device_id(); telegram->read_value(flowTemp_, 1); // is * 10 telegram->read_value(pumpMod_, 3); telegram->read_value(flowSetTemp_, 0); diff --git a/src/mqtt.cpp b/src/mqtt.cpp index b46de93b3..e141cbe2a 100644 --- a/src/mqtt.cpp +++ b/src/mqtt.cpp @@ -24,7 +24,9 @@ MAKE_PSTR_WORD(qos) MAKE_PSTR_WORD(base) MAKE_PSTR_WORD(heartbeat) MAKE_PSTR_WORD(ip) -MAKE_PSTR_WORD(nested_json) +MAKE_PSTR_WORD(nested) +MAKE_PSTR_WORD(single) +MAKE_PSTR_WORD(ha) MAKE_PSTR_WORD(publish_time) MAKE_PSTR_WORD(publish) MAKE_PSTR_WORD(connected) @@ -38,7 +40,7 @@ MAKE_PSTR(mqtt_enabled_fmt, "MQTT is %s") MAKE_PSTR(mqtt_base_fmt, "Base = %s") MAKE_PSTR(mqtt_qos_fmt, "QOS = %ld") MAKE_PSTR(mqtt_retain_fmt, "Retain Flag = %s") -MAKE_PSTR(mqtt_nestedjson_fmt, "Use nested JSON = %s") +MAKE_PSTR(mqtt_format_fmt, "JSON format = %s") MAKE_PSTR(mqtt_heartbeat_fmt, "Heartbeat = %s") MAKE_PSTR(mqtt_publish_time_fmt, "Publish time = %d seconds") @@ -54,6 +56,7 @@ std::vector Mqtt::mqtt_functions_; bool Mqtt::mqtt_retain_; uint8_t Mqtt::mqtt_qos_; std::string Mqtt::mqtt_hostname_; // copy of hostname +uint8_t Mqtt::mqtt_format_; std::string Mqtt::mqtt_base_; uint16_t Mqtt::mqtt_publish_fails_ = 0; size_t Mqtt::maximum_mqtt_messages_ = Mqtt::MAX_MQTT_MESSAGES; @@ -71,9 +74,8 @@ Mqtt::QueuedMqttMessage::QueuedMqttMessage(uint16_t id, std::shared_ptr MQTT_PUBLISH_WAIT) { last_mqtt_poll_ = currentMillis; - publish_queue(); + process_queue(); } return; @@ -325,7 +332,7 @@ void Mqtt::on_message(char * topic, char * payload, size_t len) { return; } - // convert payload to a null-terminate char string + // convert payload to a null-terminated char string char message[len + 2]; strlcpy(message, payload, len + 1); @@ -333,7 +340,8 @@ void Mqtt::on_message(char * topic, char * payload, size_t len) { DEBUG_LOG(F("Received %s => %s (length %d)"), topic, message, len); #endif - char * topic_magnitude = strrchr(topic, '/'); // strip out everything until last / + // strip out everything until the last / + char * topic_magnitude = strrchr(topic, '/'); if (topic_magnitude != nullptr) { topic = topic_magnitude + 1; } @@ -341,7 +349,7 @@ void Mqtt::on_message(char * topic, char * payload, size_t len) { // Send message event to custom service // this will pick the first topic that matches, so for multiple devices of the same type it's gonna fail for (const auto & mf : mqtt_functions_) { - if (topic == mf.topic_) { + if (strcmp(topic, mf.topic_.c_str()) == 0) { (mf.mqtt_function_)(message); return; } @@ -470,7 +478,7 @@ void Mqtt::queue_publish_message(const char * topic, const JsonDocument & payloa std::string payload_text; serializeJson(payload, payload_text); - auto message = std::make_shared(uuid::get_uptime_ms(), Operation::PUBLISH, topic, payload_text, retain); + auto message = std::make_shared(Operation::PUBLISH, topic, payload_text, retain); // DEBUG_LOG(F("Adding JSON publish message created with topic %s, message %s"), topic, payload_text.c_str()); @@ -484,11 +492,12 @@ void Mqtt::queue_publish_message(const char * topic, const JsonDocument & payloa // add MQTT message to queue, payload is a string void Mqtt::queue_publish_message(const char * topic, const std::string & payload, const bool retain) { + // can't have bogus topics, but empty payloads are ok if (strlen(topic) == 0) { return; } - auto message = std::make_shared(uuid::get_uptime_ms(), Operation::PUBLISH, topic, payload, retain); + auto message = std::make_shared(Operation::PUBLISH, topic, payload, retain); // if the queue is full, make room but removing the last one if (mqtt_messages_.size() >= maximum_mqtt_messages_) { @@ -498,13 +507,13 @@ void Mqtt::queue_publish_message(const char * topic, const std::string & payload mqtt_messages_.emplace_back(mqtt_message_id_++, std::move(message)); } -// add MQTT message to queue, payload is a string +// add MQTT subscribe message to queue void Mqtt::queue_subscribe_message(const std::string & topic) { if (topic.empty()) { return; } - auto message = std::make_shared(uuid::get_uptime_ms(), Operation::SUBSCRIBE, topic, "", false); + auto message = std::make_shared(Operation::SUBSCRIBE, topic, "", false); DEBUG_LOG(F("Adding a subscription for %s"), topic.c_str()); // if the queue is full, make room but removing the last one @@ -537,16 +546,21 @@ void Mqtt::publish(const char * topic, const bool value) { queue_publish_message(topic, value ? "1" : "0", mqtt_retain_); } +// no payload +void Mqtt::publish(const char * topic) { + queue_publish_message(topic, "", mqtt_retain_); +} + // publish all queued messages to MQTT -void Mqtt::publish_all_queue() { +void Mqtt::process_all_queue() { while (!mqtt_messages_.empty()) { - publish_queue(); + process_queue(); } } // take top from queue and try and publish it // assumes there is an MQTT connection -void Mqtt::publish_queue() { +void Mqtt::process_queue() { if (mqtt_messages_.empty()) { return; } @@ -554,8 +568,16 @@ void Mqtt::publish_queue() { // fetch first from queue and create the full topic name auto mqtt_message = mqtt_messages_.front(); auto message = mqtt_message.content_; + + // append the hostname and base to the topic, unless we're doing native HA which has a different format char full_topic[MQTT_TOPIC_MAX_SIZE]; - make_topic(full_topic, message->topic); + + // if the topic starts with "homeassistant" we leave it untouched, otherwise append ho st and base + if (strncmp(message->topic.c_str(), "homeassistant/", 13) == 0) { + strcpy(full_topic, message->topic.c_str()); + } else { + make_topic(full_topic, message->topic); + } // if we're subscribing... if (message->operation == Operation::SUBSCRIBE) { @@ -586,7 +608,7 @@ void Mqtt::publish_queue() { #else uint16_t packet_id = 1; #endif - DEBUG_LOG(F("Published topic %s (#%02d, attempt #%d, pid %d)"), full_topic, mqtt_message.id_, mqtt_message.retry_count_ + 1, packet_id); + DEBUG_LOG(F("Publishing topic %s (#%02d, attempt #%d, pid %d)"), full_topic, mqtt_message.id_, mqtt_message.retry_count_ + 1, packet_id); if (packet_id == 0) { // it failed. if we retried n times, give up. remove from queue @@ -612,7 +634,7 @@ void Mqtt::publish_queue() { } mqtt_messages_.pop_front(); // remove the message from the queue -} +} // namespace emsesp // add console commands void Mqtt::console_commands() { @@ -641,25 +663,27 @@ void Mqtt::console_commands() { EMSESPShell::commands->add_command( ShellContext::MQTT, CommandFlags::ADMIN, - flash_string_vector{F_(set), F_(nested_json)}, - flash_string_vector{F_(bool_mandatory)}, + flash_string_vector{F_(set), F_(format)}, + flash_string_vector{F_(name_mandatory)}, [](Shell & shell, const std::vector & arguments) { Settings settings; - if (arguments[0] == read_flash_string(F_(on))) { - settings.mqtt_nestedjson(true); - settings.commit(); - shell.println(F("Please restart EMS-ESP")); - } else if (arguments[0] == read_flash_string(F_(off))) { - settings.mqtt_nestedjson(false); - settings.commit(); - shell.println(F("Please restart EMS-ESP")); + uint8_t value; + if (arguments[0] == read_flash_string(F_(single))) { + value = Settings::MQTT_format::SINGLE; + } else if (arguments[0] == read_flash_string(F_(nested))) { + value = Settings::MQTT_format::NESTED; + } else if (arguments[0] == read_flash_string(F_(ha))) { + value = Settings::MQTT_format::HA; } else { - shell.println(F("Must be on or off")); + shell.println(F("Must be single, nested or ha")); return; } + settings.mqtt_format(value); + settings.commit(); + shell.println(F("Please restart EMS-ESP")); }, [](Shell & shell __attribute__((unused)), const std::vector & arguments __attribute__((unused))) -> const std::vector { - return std::vector{read_flash_string(F_(on)), read_flash_string(F_(off))}; + return std::vector{read_flash_string(F_(single)), read_flash_string(F_(nested)), read_flash_string(F_(ha))}; }); EMSESPShell::commands->add_command(ShellContext::MQTT, @@ -818,11 +842,17 @@ void Mqtt::console_commands() { shell.printfln(F_(mqtt_base_fmt), settings.mqtt_base().empty() ? uuid::read_flash_string(F_(unset)).c_str() : settings.mqtt_base().c_str()); shell.printfln(F_(mqtt_qos_fmt), settings.mqtt_qos()); shell.printfln(F_(mqtt_retain_fmt), settings.mqtt_retain() ? F_(enabled) : F_(disabled)); - shell.printfln(F_(mqtt_nestedjson_fmt), settings.mqtt_nestedjson() ? F_(enabled) : F_(disabled)); + if (settings.mqtt_format() == Settings::MQTT_format::SINGLE) { + shell.printfln(F_(mqtt_format_fmt), F_(single)); + } else if (settings.mqtt_format() == Settings::MQTT_format::NESTED) { + shell.printfln(F_(mqtt_format_fmt), F_(nested)); + } else if (settings.mqtt_format() == Settings::MQTT_format::HA) { + shell.printfln(F_(mqtt_format_fmt), F_(ha)); + } shell.printfln(F_(mqtt_heartbeat_fmt), settings.mqtt_heartbeat() ? F_(enabled) : F_(disabled)); shell.printfln(F_(mqtt_publish_time_fmt), settings.mqtt_publish_time()); shell.println(); }); -} +} // namespace emsesp } // namespace emsesp diff --git a/src/mqtt.h b/src/mqtt.h index 40407a6e1..c61a5fe69 100644 --- a/src/mqtt.h +++ b/src/mqtt.h @@ -49,10 +49,9 @@ using mqtt_function_p = std::function; using namespace std::placeholders; // for `_1` struct MqttMessage { - MqttMessage(uint64_t uptime_ms, uint8_t operation, const std::string & topic, const std::string & payload, bool retain); + MqttMessage(uint8_t operation, const std::string & topic, const std::string & payload, bool retain); ~MqttMessage() = default; - const uint64_t uptime_ms; const uint8_t operation; const std::string topic; const std::string payload; @@ -75,6 +74,7 @@ class Mqtt { static void publish(const char * topic, const JsonDocument & payload); static void publish(const char * topic, const JsonDocument & payload, bool retain); static void publish(const char * topic, const bool value); + static void publish(const char * topic); static void show_topic_handlers(uuid::console::Shell & shell, const uint8_t device_id); @@ -139,8 +139,8 @@ class Mqtt { void on_message(char * topic, char * payload, size_t len); void on_connect(); static char * make_topic(char * result, const std::string & topic); - void publish_queue(); - void publish_all_queue(); + void process_queue(); + void process_all_queue(); void send_start_topic(); static void reconnect(); void init(); @@ -177,6 +177,7 @@ class Mqtt { static std::string mqtt_hostname_; static std::string mqtt_base_; static uint8_t mqtt_qos_; + static uint8_t mqtt_format_; std::string mqtt_ip_; std::string mqtt_user_; std::string mqtt_password_; diff --git a/src/sensors.cpp b/src/sensors.cpp index 62cd66a97..a7dd020c3 100644 --- a/src/sensors.cpp +++ b/src/sensors.cpp @@ -28,7 +28,7 @@ uuid::log::Logger Sensors::logger_{F_(logger_name), uuid::log::Facility::DAEMON} void Sensors::start() { // copy over values from MQTT so we don't keep on quering the filesystem - mqtt_nestedjson_ = Settings().mqtt_nestedjson(); + mqtt_format_ = Settings().mqtt_format(); #ifndef EMSESP_STANDALONE bus_.begin(SENSOR_GPIO); @@ -39,34 +39,36 @@ void Sensors::loop() { #ifndef EMSESP_STANDALONE if (state_ == State::IDLE) { if (millis() - last_activity_ >= READ_INTERVAL_MS) { - // DEBUG_LOG(F("Read sensor temperature")); + // DEBUG_LOG(F("Read sensor temperature")); // uncomment for debug if (bus_.reset()) { bus_.skip(); bus_.write(CMD_CONVERT_TEMP); state_ = State::READING; } else { - // logger_.err(F("Bus reset failed")); + // no sensors found + // logger_.err(F("Bus reset failed")); // uncomment for debug + devices_.clear(); // remove all know devices incase we have a disconnect } last_activity_ = millis(); } } else if (state_ == State::READING) { if (temperature_convert_complete()) { - // DEBUG_LOG(F("Scanning for sensors")); + // DEBUG_LOG(F("Scanning for sensors")); // uncomment for debug bus_.reset_search(); found_.clear(); state_ = State::SCANNING; last_activity_ = millis(); } else if (millis() - last_activity_ > READ_TIMEOUT_MS) { - // logger_.err(F("Sensor read timeout")); + logger_.err(F("Sensor read timeout")); state_ = State::IDLE; last_activity_ = millis(); } } else if (state_ == State::SCANNING) { if (millis() - last_activity_ > SCAN_TIMEOUT_MS) { - // logger_.err(F("Sensor scan timeout")); + logger_.err(F("Sensor scan timeout")); state_ = State::IDLE; last_activity_ = millis(); } else { @@ -84,8 +86,13 @@ void Sensors::loop() { found_.emplace_back(addr); found_.back().temperature_c_ = get_temperature_c(addr); - // char result[10]; - // DEBUG_LOG(F("Temperature of %s = %s"), found_.back().to_string().c_str(), Helpers::render_value(result, found_.back().temperature_c_, 2)); + /* + // comment out for debugging + char result[10]; + DEBUG_LOG(F("Temp of %s = %s"), + found_.back().to_string().c_str(), + Helpers::render_value(result, found_.back().temperature_c_, 2)); + */ break; default: @@ -99,7 +106,7 @@ void Sensors::loop() { bus_.depower(); devices_ = std::move(found_); found_.clear(); - // DEBUG_LOG(F("Found %zu sensor(s)"), devices_.size()); + // DEBUG_LOG(F("Found %zu sensor(s). Adding them."), devices_.size()); // uncomment for debug state_ = State::IDLE; last_activity_ = millis(); } @@ -220,14 +227,14 @@ void Sensors::publish_values() { // if we're not using nested JSON, send each sensor out seperately // sensor1, sensor2 etc... // e.g. sensor_1 = {"temp":20.2} - if (!mqtt_nestedjson_) { - StaticJsonDocument<20> doc; + if (mqtt_format_ != Settings::MQTT_format::NESTED) { + StaticJsonDocument<100> doc; for (const auto & device : devices_) { char s[5]; doc["temp"] = Helpers::render_value(s, device.temperature_c_, 2); char topic[60]; // sensors{1-n} - strlcpy(topic, "sensor_", 50); // create topic - strlcat(topic, device.to_string().c_str(), 50); + strlcpy(topic, "sensor_", 50); // create topic, e.g. home/ems-esp/sensor_28-EA41-9497-0E03-5F + strlcat(topic, device.to_string().c_str(), 60); Mqtt::publish(topic, doc); doc.clear(); // clear json doc so we can reuse the buffer again } diff --git a/src/sensors.h b/src/sensors.h index 2bae06749..a3b530037 100644 --- a/src/sensors.h +++ b/src/sensors.h @@ -74,7 +74,7 @@ class Sensors { static constexpr size_t SCRATCHPAD_TEMP_LSB = 0; static constexpr size_t SCRATCHPAD_CONFIG = 4; - // chips + // dallas chips static constexpr uint8_t TYPE_DS18B20 = 0x28; static constexpr uint8_t TYPE_DS18S20 = 0x10; static constexpr uint8_t TYPE_DS1822 = 0x22; @@ -102,7 +102,7 @@ class Sensors { std::vector found_; std::vector devices_; - bool mqtt_nestedjson_; + uint8_t mqtt_format_; }; } // namespace emsesp diff --git a/src/settings.cpp b/src/settings.cpp index 31a349be9..26213847d 100644 --- a/src/settings.cpp +++ b/src/settings.cpp @@ -50,7 +50,7 @@ namespace emsesp { EMSESP_SETTINGS_SIMPLE(std::string, "", mqtt_base, "", (), EMSESP_DEFAULT_MQTT_BASE) \ EMSESP_SETTINGS_SIMPLE(uint8_t, "", mqtt_qos, "", (), EMSESP_DEFAULT_MQTT_QOS) \ EMSESP_SETTINGS_SIMPLE(bool, "", mqtt_retain, "", (), EMSESP_DEFAULT_MQTT_RETAIN) \ - EMSESP_SETTINGS_SIMPLE(bool, "", mqtt_nestedjson, "", (), EMSESP_DEFAULT_MQTT_NESTEDJSON) \ + EMSESP_SETTINGS_SIMPLE(uint8_t, "", mqtt_format, "", (), EMSESP_DEFAULT_MQTT_FORMAT) \ EMSESP_SETTINGS_SIMPLE(bool, "", mqtt_heartbeat, "", (), EMSESP_DEFAULT_MQTT_HEARTBEAT) #define EMSESP_SETTINGS_SIMPLE EMSESP_SETTINGS_GENERIC diff --git a/src/settings.h b/src/settings.h index b8a025f74..64fc36619 100644 --- a/src/settings.h +++ b/src/settings.h @@ -51,7 +51,7 @@ #define EMSESP_DEFAULT_MQTT_PORT 1883 #define EMSESP_DEFAULT_MQTT_QOS 0 #define EMSESP_DEFAULT_MQTT_RETAIN false -#define EMSESP_DEFAULT_MQTT_NESTEDJSON true +#define EMSESP_DEFAULT_MQTT_FORMAT 2 // nested #define EMSESP_DEFAULT_MQTT_HEARTBEAT true #define EMSESP_DEFAULT_EMS_READ_ONLY false #define EMSESP_DEFAULT_SHOWER_TIMER false @@ -136,9 +136,6 @@ class Settings { bool mqtt_retain() const; void mqtt_retain(const bool & mqtt_retain); - bool mqtt_nestedjson() const; - void mqtt_nestedjson(const bool & mqtt_nestedjson); - bool mqtt_heartbeat() const; void mqtt_heartbeat(const bool & mqtt_heartbeat); @@ -151,6 +148,10 @@ class Settings { uint8_t master_thermostat() const; void master_thermostat(const uint8_t & master_thermostat); + enum MQTT_format : uint8_t { SINGLE = 1, NESTED, HA }; + uint8_t mqtt_format() const; + void mqtt_format(const uint8_t & mqtt_format); + private: static constexpr size_t BUFFER_SIZE = 2048; // max size for the settings file @@ -194,9 +195,9 @@ class Settings { static std::string mqtt_base_; static uint8_t mqtt_qos_; static bool mqtt_retain_; - static bool mqtt_nestedjson_; static bool mqtt_heartbeat_; - static uint16_t mqtt_publish_time_; // frequency of MQTT publish in seconds + static uint16_t mqtt_publish_time_; // seconds + static uint8_t mqtt_format_; }; } // namespace emsesp diff --git a/src/system.cpp b/src/system.cpp index c9bf2a11f..505b6771e 100644 --- a/src/system.cpp +++ b/src/system.cpp @@ -25,7 +25,6 @@ MAKE_PSTR_WORD(mark) MAKE_PSTR_WORD(level) MAKE_PSTR_WORD(host) MAKE_PSTR_WORD(passwd) -MAKE_PSTR_WORD(format) MAKE_PSTR_WORD(hostname) MAKE_PSTR_WORD(wifi) MAKE_PSTR_WORD(ssid) diff --git a/src/telegram.cpp b/src/telegram.cpp index e740aaeff..ea7443bcb 100644 --- a/src/telegram.cpp +++ b/src/telegram.cpp @@ -189,9 +189,8 @@ void Telegram::read_value8(int16_t & param, const uint8_t index) const { param = message_data[pos]; } -RxService::QueuedRxTelegram::QueuedRxTelegram(uint16_t id, uint32_t timestamp, std::shared_ptr && telegram) +RxService::QueuedRxTelegram::QueuedRxTelegram(uint16_t id, std::shared_ptr && telegram) : id_(id) - , timestamp_(timestamp) , telegram_(std::move(telegram)) { } @@ -238,11 +237,6 @@ void RxService::loop() { // data is the whole telegram, assuming last byte holds the CRC // for EMS+ the type_id has the value + 256. We look for these type of telegrams with F7, F9 and FF in 3rd byte void RxService::add(uint8_t * data, uint8_t length) { - // ignore any telegrams which are 1 byte - if (length <= 1) { - return; - } - // validate the CRC uint8_t crc = calculate_crc(data, length - 1); if (data[length - 1] != crc) { @@ -313,8 +307,8 @@ void RxService::add(uint8_t * data, uint8_t length) { } // add to queue, with timestamp - DEBUG_LOG(F("New Rx [%d] telegram added, length %d"), rx_telegram_id_, message_length); - rx_telegrams_.emplace_back(rx_telegram_id_++, millis(), std::move(telegram)); + DEBUG_LOG(F("New Rx [#%d] telegram added, length %d"), rx_telegram_id_, message_length); + rx_telegrams_.emplace_back(rx_telegram_id_++, std::move(telegram)); } diff --git a/src/telegram.h b/src/telegram.h index f67d8154c..6153bc5f8 100644 --- a/src/telegram.h +++ b/src/telegram.h @@ -187,11 +187,10 @@ class RxService : public EMSbus { class QueuedRxTelegram { public: - QueuedRxTelegram(uint16_t id, uint32_t timestamp, std::shared_ptr && telegram); + QueuedRxTelegram(uint16_t id, std::shared_ptr && telegram); ~QueuedRxTelegram() = default; - uint16_t id_; // sequential identifier - uint32_t timestamp_; // time it was received + uint16_t id_; // sequential identifier const std::shared_ptr telegram_; }; diff --git a/src/thermostat.cpp b/src/thermostat.cpp index bcd2026ff..1514c7f4d 100644 --- a/src/thermostat.cpp +++ b/src/thermostat.cpp @@ -114,24 +114,76 @@ Thermostat::Thermostat(uint8_t device_type, uint8_t device_id, uint8_t product_i uint8_t master_thermostat = settings.master_thermostat(); // what the user has defined uint8_t actual_master_thermostat = EMSESP::actual_master_thermostat(); // what we're actually using uint8_t num_devices = EMSESP::count_devices(EMSdevice::DeviceType::THERMOSTAT) + 1; // including this thermostat - mqtt_nested_json_ = settings.mqtt_nestedjson(); + mqtt_format_ = settings.mqtt_format(); // single, nested or ha - // if we're on auto mode, register this first one we find as we may find multiple + // if we're on auto mode, register this thermostat if it has a device id of 0x10 or 0x17 // or if its the master thermostat we defined - if (((num_devices == 1) && (actual_master_thermostat == EMSESP_DEFAULT_MASTER_THERMOSTAT)) || (master_thermostat == device_id)) { + // see https://github.com/proddy/EMS-ESP/issues/362#issuecomment-629628161 + if (((num_devices == 1) && (actual_master_thermostat == EMSESP_DEFAULT_MASTER_THERMOSTAT) && ((device_id == 0x10) || (device_id == 0x17))) + || (master_thermostat == device_id)) { EMSESP::actual_master_thermostat(device_id); DEBUG_LOG(F("Registering new thermostat with device ID 0x%02X (as the master)"), device_id); - - // MQTT callbacks - register_mqtt_topic("thermostat_cmd", std::bind(&Thermostat::thermostat_cmd, this, _1)); - register_mqtt_topic("thermostat_cmd_temp", std::bind(&Thermostat::thermostat_cmd_temp, this, _1)); - register_mqtt_topic("thermostat_cmd_mode", std::bind(&Thermostat::thermostat_cmd_mode, this, _1)); - + init_mqtt(); } else { DEBUG_LOG(F("Registering new thermostat with device ID 0x%02X"), device_id); } } +// for the master thermostat initialize the MQTT subscribes +void Thermostat::init_mqtt() { + register_mqtt_topic("thermostat_cmd", std::bind(&Thermostat::thermostat_cmd, this, _1)); // generic commands + + // if the MQTT format type is ha then send the config to HA (via the mqtt discovery service) + // for each of the heating circuits + if (mqtt_format_ == Settings::MQTT_format::HA) { + for (uint8_t hc = 0; hc < monitor_typeids.size(); hc++) { + std::string topic(100, '\0'); // e.g homeassistant/climate/hc1/thermostat/config + snprintf_P(&topic[0], topic.capacity() + 1, PSTR("homeassistant/climate/hc%d/thermostat/config"), hc + 1); + + // Mqtt::publish(topic.c_str()); // empty payload, this remove any previous config sent to HA + + StaticJsonDocument doc; + std::string payload(100, '\0'); + snprintf_P(&payload[0], payload.capacity() + 1, PSTR("thermostat_hc%d"), hc + 1); + + doc["name"] = payload; // "name": "thermostat_hc1" + doc["unique_id"] = payload; // "unique_id": "thermostat_hc1" + + snprintf_P(&payload[0], payload.capacity() + 1, PSTR("homeassistant/climate/hc%d/thermostat"), hc + 1); + doc["~"] = payload; // "homeassistant/climate/hc1/thermostat" + + doc["mode_cmd_t"] = "~/cmd_mode"; + doc["mode_stat_t"] = "~/state"; + doc["mode_stat_tpl"] = "{{value_json.mode}}"; + doc["temp_cmd_t"] = "~/cmd_temp"; + doc["temp_stat_t"] = "~/state"; + doc["temp_stat_tpl"] = "{{value_json.seltemp}}"; + doc["curr_temp_t"] = "~/state"; + doc["curr_temp_tpl"] = "{{value_json.currtemp}}"; + doc["min_temp"] = "5"; + doc["max_temp"] = "40"; + doc["temp_step"] = "0.5"; + + JsonArray modes = doc.createNestedArray("modes"); + modes.add("off"); + modes.add("heat"); + modes.add("auto"); + + Mqtt::publish(topic.c_str(), doc, true); // publish the config payload with retain flag + + // subscribe to the temp and mode commands + snprintf_P(&topic[0], topic.capacity() + 1, PSTR("homeassistant/climate/hc%d/thermostat/cmd_temp"), hc + 1); + register_mqtt_topic(topic, std::bind(&Thermostat::thermostat_cmd_temp, this, _1)); + snprintf_P(&topic[0], topic.capacity() + 1, PSTR("homeassistant/climate/hc%d/thermostat/cmd_mode"), hc + 1); + register_mqtt_topic(topic, std::bind(&Thermostat::thermostat_cmd_mode, this, _1)); + } + } else { + // these will be prefixed with hostname and base + register_mqtt_topic("thermostat_cmd_temp", std::bind(&Thermostat::thermostat_cmd_temp, this, _1)); + register_mqtt_topic("thermostat_cmd_mode", std::bind(&Thermostat::thermostat_cmd_mode, this, _1)); + } +} + // only add the menu for the master thermostat void Thermostat::add_context_menu() { if (device_id() != EMSESP::actual_master_thermostat()) { @@ -292,8 +344,8 @@ void Thermostat::publish_values() { JsonObject rootThermostat = doc.to(); JsonObject dataThermostat; - // optional, add external temp - if (flags == EMS_DEVICE_FLAG_RC35) { + // optional, add external temp. I don't think anyone actually is interested in this + if ((flags == EMS_DEVICE_FLAG_RC35) && (mqtt_format_ == Settings::MQTT_format::SINGLE)) { if (dampedoutdoortemp != EMS_VALUE_INT_NOTSET) { doc["dampedtemp"] = dampedoutdoortemp; } @@ -312,7 +364,7 @@ void Thermostat::publish_values() { } has_data = true; - if (mqtt_nested_json_) { + if (mqtt_format_ == Settings::MQTT_format::NESTED) { // create nested json for each HC char hc_name[10]; // hc{1-4} strlcpy(hc_name, "hc", 10); @@ -371,25 +423,54 @@ void Thermostat::publish_values() { } if (hc->mode != EMS_VALUE_UINT_NOTSET) { - dataThermostat["mode"] = mode_tostring(hc->get_mode(flags)); + uint8_t hc_mode = hc->get_mode(flags); + // if we're sending to HA the only valid mode types are heat, auto and off + if (mqtt_format_ == Settings::MQTT_format::HA) { + if ((hc_mode == HeatingCircuit::Mode::MANUAL) || (hc_mode == HeatingCircuit::Mode::DAY)) { + hc_mode = HeatingCircuit::Mode::HEAT; + } else if ((hc_mode == HeatingCircuit::Mode::NIGHT) || (hc_mode == HeatingCircuit::Mode::OFF)) { + hc_mode = HeatingCircuit::Mode::OFF; + } else { + hc_mode = HeatingCircuit::Mode::AUTO; + } + } + dataThermostat["mode"] = mode_tostring(hc_mode); } - if (hc->mode_type != EMS_VALUE_UINT_NOTSET) { + + // special handling of mode type, for the RC35 replace with summer/holiday + // https://github.com/proddy/EMS-ESP/issues/373#issuecomment-619810209 + if ((flags & 0x0F) == EMS_DEVICE_FLAG_RC35) { + if (hc->holiday_mode != EMS_VALUE_UINT_NOTSET) { + dataThermostat["modetype"] = F("holiday"); + } else if (hc->summer_mode != EMS_VALUE_UINT_NOTSET) { + dataThermostat["modetype"] = F("summer"); + } else if (hc->mode_type != EMS_VALUE_UINT_NOTSET) { + dataThermostat["modetype"] = mode_tostring(hc->get_mode_type(flags)); + } + } else if (hc->mode_type != EMS_VALUE_UINT_NOTSET) { dataThermostat["modetype"] = mode_tostring(hc->get_mode_type(flags)); } - // if its not nested, send immediately - if (!mqtt_nested_json_) { + + // if format is single, send immediately + // if its HA send it to the special topic + if (mqtt_format_ == Settings::MQTT_format::SINGLE) { char topic[30]; char s[3]; // for formatting strings strlcpy(topic, "thermostat_data", 30); strlcat(topic, Helpers::itoa(s, hc->hc_num()), 30); // append hc to topic Mqtt::publish(topic, doc); return; + } else if (mqtt_format_ == Settings::MQTT_format::HA) { + std::string topic(100, '\0'); + snprintf_P(&topic[0], topic.capacity() + 1, PSTR("homeassistant/climate/hc%d/thermostat/state"), hc->hc_num()); + Mqtt::publish(topic.c_str(), doc); + return; } } // if we're using nested json, send all in one go - if (mqtt_nested_json_ && has_data) { + if ((mqtt_format_ == Settings::MQTT_format::NESTED) && has_data) { Mqtt::publish("thermostat_data", doc); } } @@ -408,7 +489,7 @@ std::shared_ptr Thermostat::heating_circuit(const ui } // determine which heating circuit the type ID is referring too -// returns pointer to the HeatingCircuit +// returns pointer to the HeatingCircuit or nullptr if it can't be found std::shared_ptr Thermostat::heating_circuit(std::shared_ptr telegram) { // look through the Monitor and Set arrays to see if there is a match uint8_t hc_num = 0; @@ -430,9 +511,9 @@ std::shared_ptr Thermostat::heating_circuit(std::sha } } - // still didn't recognize it, assume its hc 1 + // still didn't recognize it, ignore it if (hc_num == 0) { - hc_num = DEFAULT_HEATING_CIRCUIT; + return nullptr; } // if we have the heating circuit already present, returns its object @@ -446,7 +527,6 @@ std::shared_ptr Thermostat::heating_circuit(std::sha // create a new heating circuit object // TODO do we need to create a new object if using emplace_back? heating_circuits_.emplace_back(new HeatingCircuit(hc_num, monitor_typeids[hc_num - 1], set_typeids[hc_num - 1])); - return heating_circuits_.back(); } @@ -826,8 +906,9 @@ void Thermostat::process_RC30Set(std::shared_ptr telegram) { // type 0x3E (HC1), 0x48 (HC2), 0x52 (HC3), 0x5C (HC4) - data from the RC35 thermostat (0x10) - 16 bytes void Thermostat::process_RC35Monitor(std::shared_ptr telegram) { - // exit if... - // - the 15th byte (second from last) is 0x00, which I think is flow temp, means HC is not is use + // exit if the 15th byte (second from last) is 0x00, which I think is calculated flow setpoint temperature + // with weather controlled RC35s this value can be zero and our setpoint temps will be incorrect + // see https://github.com/proddy/EMS-ESP/issues/373#issuecomment-627907301 if (telegram->message_data[14] == 0x00) { return; } diff --git a/src/thermostat.h b/src/thermostat.h index 2de2a4b0d..fe117fa5f 100644 --- a/src/thermostat.h +++ b/src/thermostat.h @@ -104,10 +104,11 @@ class Thermostat : public EMSdevice { static uuid::log::Logger logger_; void console_commands(); + void init_mqtt(); std::string datetime_; // date and time stamp - bool mqtt_nested_json_; + uint8_t mqtt_format_; // single, nested or ha // Installation parameters uint8_t ibaMainDisplay = diff --git a/src/version.h b/src/version.h index 675485602..0f1f2959d 100644 --- a/src/version.h +++ b/src/version.h @@ -1 +1 @@ -#define EMSESP_APP_VERSION "2.0.0a3" +#define EMSESP_APP_VERSION "2.0.0a4"