Monday, May 28, 2018

Spring Beans Discovery through Java Config in Practice

Spring's Java Config is great tool giving us precise way to control bootstrap and the wiring of an application. However it does not always work as expected and we are running sometimes into some strange issues, where particular bean it not being instantiated at expected time, or lazy loading is not working as expected, or whole configuration is being ignored for no reason.
So I've decided to go over some examples, to get better understanding of how things are in practice.

Examples can be found here: https://github.com/maciejmiklas/spring-context-test.git
Each example has name, like: exa (example A) or exb (example B). Within each example there are several exercises, like: exa has: exa01, exa02, ..., ex05.

Bean Loading Order - Single Config

exa01

This is the first exercise, it defines basic structure that will be modify in following chapters.
The main method loads spring context from configuration file Conf.java. This configuration file creates 3 beans. Each of those beans has two log statements, first one in constructor, second in @PostConstruct. We can see also the console output after execution of this example.
Nothing spectacular happens here: beans are initialised from A to C, right after constructor Spring executes also @PostConstruct - there are no dependencies between beans.


exa02

Lets modify order of factory methods in Conf.java

Bean initialisation order has changed. This means that method names do not matter, but the physical order within class. I can imagine, that this can change with JVM, so you definitely should not rely on it!


exa03

Now we will extract definition of BeanB into dedicated configuration file:
Spring loads bean definitions from imported configuration in the first place.


exa04

We will modify previous exercise, so that BeanA depends on BeanB. Spring loads dependent bean in the first place, it does not matter where it's located, Spring will scan all possible configurations to locate it.

exa05

The same as exa04, but BeanA does not have @DependsOn("beanB"), instead BeanA injects BeanB. We have logically similar dependency situation to exa4: BanA depends on BeanB. However @PostConstructs are called in different order: Spring creates instance of BeanA, during construction phase (constructor) references to BeanB are not set. Once the instance of BeanA is created, BeanB will be created and injected to BeanA. BeanA cannot access BeanB in constructor, first after full initialisation is done - in method @PostConstruct.
Spring had to modify order of @PostConstruct calls, to ensure that bean references are not null during initialisation phase.


Summary

  • within a single configuration class the bean instantiation order depends only on methods order within this class and not method names. However this might depend on JVM,
  • Spring loads in the first place imported bean definitions,
  • previous rule applies only to situation where direct bean definitions do not have dependencies on imported beans, if this is the case Spring will load dependent bean in the first place,
  • initialisation code has to be placed in @PostConstruct method and not in a constructor, otherwise references to some beans might be null.

Bean Loading Order - Mixed Config


exb01

BeanA, BeanB and BeanC are created through dedicated configuration classes: ConfBeanA, ConfBeanB and ConfBeanC. Beans are created in order: A, B, C.


exb02

Similar to exb01, but configuration class ConfBeanB has been renamed to ConfBeanXB
The instantiation order remains unchanged.


exb03

The same as exb01 but factory method in ConfBeanB has been renamed from beanB to xyz. The instantiation order remains unchanged.


exb04

The same as exb01 but BeanB has been renamed to BeanXB. This change did not influence instantiation order.


exb05

Similar to exb01 but we've changed import order from A, B, C into B, A, C. The instantiation order has changed as well.


Summary


  • when @Import statement contains multiple classes, Spring will load them from the left to the right side - it's iterating over array, names of configuration classes do not matter in this case,
  • within single configuration class physical order of methods matters, not their names.

Duplicated Bean Name

exc01

We have here 3 beans A, B, C with dedicated config.  BeanA injects BeanB and BeanC.


exc02

The same as exc01, but we've renamed factory method ConfBeanB#beanB() into ConfBeanB#beanC() In order to find out all possible bean definitions Spring goes over our @Import declaration, which is: ConfBeanA, ConfBeanB, ConfBeanC. There are two factory methods with the same name: ConfBeanB#beanC() and ConfBeanC#beanC(), so ConfBeanC overwrites bean created through ConfBeanB, because it creates bean with the same name.


exc03

Similar to exc02 but this time we've renamed method ConfBeanC#beanC() into ConfBeanC#beanB(), so that we have two methods beanB and not as it was in exc02 two methods with name: beanC The output is still the same, BeanB is missing. We've just changed name of factory method, and we already know that names do not change loading order, and we have still two beans with the same name.


exc04

Similar to exc02, but we've changed import order for config classes from A, B, C to A, C, B. Now ConfBeanB will get scanned on the end and it overwrites previous bean with the same name, so BeanC is missing.


exc05 

It's a modified exc02. We've set AllowBeanDefinitionOverriding to false on Application Context. Bean overwriting is disabled now, so instead of bean not found we are getting exception that we are trying to register two bean under the same name.


exc06

The same as exc02, there are still two factory methods: beanC. Additionally BeanB and BeanC are implementing common interface. Now BeanA does not inject directly BeanB and BeanC, but it injects collection of beans implementing our interface. In case of direct injection one of the beans was missing, now both are there!

Summary
In case of two beans with the same name the last one wins. You will not be able to directly inject such overwritten bean, but you can inject collection of such beans sharing common interface.

Lazy Loading


exd01

We have here three beans A, B, C, there are no dependencies between them. BeanC is being defined through dedicated configuration class. Nothing special here, initialisation order is as expected.


exd02

The same as exd02, but we've annotated BeanB as lazy. Spring does not load BeanB at all, there are no dependencies to that bean.


exd03

Now we've injected BeanB into BeanA. @Lazy is set on injection point and bean definition. Additionally we have defined method on BeanA that calls method on BeanB: ApplicationExD03 -> beanA.method() -> beanB.method() Spring postpones creation of BeanB until method is being called on it, @Lazy works as expected.


exd04

Same as exd03, but @Lazy has been removed on injection point. BeanB is not lazy any more, @Lazy on bean definition is not enough.


exd05

Same as exd03, but @Lazy has been removed from bean configuration. BeanB is not lazy any more, @Lazy on bean definition is not enough.


Summary

  • lazy annotation has to be provided on configuration and all injection points, otherwise spring eagerly initialises such bean,
  • lazy beans that are not referenced (injected) are not loaded at all,
  • you should not rely on @Lazy, unless you are 100% sure that you can control all possible injection points, otherwise one missing Lazy-Annotation will disable laziness on such bean.

Saturday, August 12, 2017

Thermostat based on Arduino

This time we are going to build a Thermostat based on Arduino, temperature sensor, and relay.
You can find it on GitHub

This Thermostat gives you the possibility to drive multiple devices in order to control temperature. In my case, I have installed two fans on my attic in order to cool it down in summer. There are no windows, so I had to force airflow. The first fan gets started when the temperature reaches 35 degrees, second over 45.

You can control any reasonable amount of units, it's all configurable. You have also access to basic statistics for past two weeks:
Runtime of whole system

ON time for each relay

Statistics for 14 days

Statistics for 7th day

Configuration

The whole configuration is stored in Config.h. You can change PINs controlling relays, buttons, input for reading temperature, thresholds or timings.

Hardware


Buttons

Two buttons are dedicated for menu navigation and the third one resets statistics. They are stored in EEPROM (Storage.cpp) once a day. Pins assigned to buttons can be found in Config.h

Configuring Relays

Let's assume that we would like to have 3 relays:

  • ID:0, PIN: 1, Temperature setpoint: 20
  • ID:1, PIN: 10, Temperature setpoint: 30
  • ID:2, PIN: 11, Temperature setpoint: 40
  • First, you have to make sure that PIN of your choice is not already taken. All pins can be found in Config.h, they are defined by variables starting with DIG_PIN.

    You have to edit Config.h and configure PINs, thresholds and amount of relays. Obviously, some properties already exist, so you have to just edit them.

    const static uint8_t DIG_PIN_RELAY_0 = 1;
    const static uint8_t DIG_PIN_RELAY_1 = 10;
    const static uint8_t DIG_PIN_RELAY_2 = 11;
    
    const static uint8_t RELAYS_AMOUNT = 3;
    
    const static int16_t RELAY_TEMP_SET_POINT_0 = 20;
    const static int16_t RELAY_TEMP_SET_POINT_1 = 30;
    const static int16_t RELAY_TEMP_SET_POINT_2 = 40;
    

    Now we have to set up relays and controller, this happens in RelayDriver.cpp


        initRelayHysteresisController(0, DIG_PIN_RELAY_0, RELAY_TEMP_SET_POINT_0);
        initRelayHysteresisController(1, DIG_PIN_RELAY_1, RELAY_TEMP_SET_POINT_1);
        initRelayHysteresisController(2, DIG_PIN_RELAY_2, RELAY_TEMP_SET_POINT_2);


    Choosing Controller

    There two controllers available Hysteresis and PID

    Hysteresis Controller

    It's the one chosen in the example above, it has few additional configurations:

    const static uint32_t RELAY_DELAY_AFTER_SWITCH_MS = 300000; // 5 minutes
    const static uint32_t RHC_RELAY_MIN_SWITCH_MS = 3600000;
    

    RELAY_DELAY_AFTER_SWITCH_MS gives wait time for switching next relay. Imagine that configuration from our example would start working in 40 degrees environment. This would result in enabling of all three relays at the same time. This could eventually lead to high power consumption - depending on what you are controlling, an electric engine, for example, consumes more power during start. In our case switching relays has following flow: the first relay goes, wait 5 minutes, second goes on, wait 5 minutes, third goes on.

    RHC_RELAY_MIN_SWITCH_MS defines hysteresis, it's the minimum frequency for particular relay to change its state. Once its on, it will remain on for at least this period of time, ignoring temperature changes. This is quite useful if you are controlling electric motors since each switch has a negative impact on live time.

    PID Controller

    This is an advanced topic. Implementing such controller is a simple task, finding right amplitude settings is a different story. 

    In order to use PID controller, you have to change initRelayHysteresisController(.....) to initRelayPiDController(....) and you need to find right settings for it. As usual, you will find them in Config.h 

    I've implemented simple simulator in Java so that it's possible to visualize the results. It can be found in the folder: pidsimulator.
    Below you can see simulations for two controllers PID a P. PID is not perfectly stable because I did not apply any sophisticated algorithm to find right values.

    On both plots required temperature is set to 30 (blue). Current temperature indicates read line. The relay has two states ON and OFF. When it's enabled temperature drops by 1.5, when it's disabled it rises by 0.5.

    Software Design

    Message Bus

    Different software modules have to communicate with each other, hopefully not both ways ;) 

    For example: 
    • statistics module has to know when the particular relay goes on and off,
    • pressing a button has to change display content and it also has to suspend services that would consume many CPU cycles, for example, temperature reading from the sensor,
    • after some time temperature reading has to be renewed, 
    • and so on...
    Every module is connected to Message Bus and can register for particular events, and can produce any events.

    For example, pressing Next button results in the following flow:
    Some components have some tasks that needs to be executed periodically. We could call their corresponding methods from the main loop since we have Message Bus it's only necessary to propagate right event:

    LIBS

    Following libs are required to compile Thermostat:
    • https://github.com/maciejmiklas/Thermostat
    • https://github.com/milesburton/Arduino-Temperature-Control-Library
    • https://github.com/maciejmiklas/ArdLog.git

    Wednesday, March 22, 2017

    Weather Station based on Arduino and NodeMCU

    We are about to build a weather station with forecast for three days, it will also include clock with local time and date.
    The whole project is based on Arduino, NodeMCU provides access to the Internet over WiFi. The display is built from single LEDs. Since they can be really bright it adopts illumination based on light conditions.


    Let's start from the beginning ;)
    I've found those 8x8 LED modules:

    so I've decided to combine few to build a display. In my case there are 3 lines, each consists of 8 modules, 24 it total, this give us 1532 single LEDs! 
    To drive single module I've chosen MAX72xx, I also wanted to improve my soldering skills, so I've decided to go for 24 PIN DIP chips and solder them to prototype boards:

    Well that worked out pretty well when it comes to those skills, but I would recommend to use LED modules combined with MAX Chip, this will save you at least few hours, not mentioning time spent afterwards when single cable gets loose ;) Such combo-module has only 3 wires instead of 16.

    So we have hardware part for our display: 24 led modules with drivers. Now it's time to light them up! I've decided to go for Arduino Mega, because it has two serial ports, so it's easier to debug things (one port will be used for communication with EPS8266). You could also use Uno, in this case you would have to daisy chain MAX chips and change addressing in software. I've used separate line for each Max chip, but Uno just does not have enough digital output pins.

    I was looking for API that will join all led modules into one canvas, so that you can print sprites on it without bothering with transitions between LED modules. I did not find anything that would make me happy so I've decided to implement one by myself. It provides not only simple canvas, but fonts and few animations. Basically everything that will be needed to display time and weather.

    So ... we have display and API to control it. Now we need to get date and weather. Arduino does not support Internet connectivity and it definitely does not have enough resources to process incoming data. So I've decided to use NodeMCU. With few Lua scripts I was able to implement simple API that is accessible over serial port. Arduino connects over it with NodeMCU, obtains time, date, weather and displays it.

    In order to provide date NodeMCU connects with NTP server and receives UTC time, afterwards it calculates local date from it. I could use one of Internet services to grab the local date, but I was looking for solution that will remain stable for long time. Those services can change their API or just go offline, but NTP protocol will remain unchanged. In worst case you will have to change server name. Current implementation supports also failover trough many different servers, so probably you will not have to bother with it soon ;)

    Getting the weather was a bit tricky, because I had to find API that will return response small enough so it could be parsed by NodeMCU. I've decided to use yahoo weather. They provide nice REST API with small and simple response. Hopefully they will keep interfaces stable for long time.....


    Putting things together

    Hardware

    First you have to build the display, I've already described it in this post: Arduino LED Display .You can use the same pin numbers on Mega, so that you will not have to alter software.

    After the display is ready, you should connect ESP8266 and photoresistor. Schematic below contains all hardware elements together:

    The area on right top corner is the display itself - 8x3 LED modules. Each single module exposes 3 PINs, those are MAX 72xxx PINs responsible for communication over SPI.
    Here you will find exact schematic of single module including SPI PINs:


    Next one is just another representation:
    Those last two schematics can be found here: https://github.com/maciejmiklas/LEDDisplay/tree/master/doc/fritzing

    Now it's time to build a software.

    Software for Arduino Mega

    You need to compile this project https://github.com/maciejmiklas/LEDClock and upload into Arduino.
    In order to compile it, you will need to include SPI module and two other libraries: https://github.com/maciejmiklas/LEDDisplay and https://github.com/maciejmiklas/LEDDisplay

    Here is a compiled software for those with the same hardware setup.

    You can use Sloeber for compilation, just follow those steps:
    1. Install Sloeber from http://eclipse.baeyens.it
    2. Create new Arduino sketch and name it: LEDClock 
    3. Confirm next screens until it will ask you about code, select cpp
    4. Finish it, and you should get something like that:
    5. Clone LEDClock project from https://github.com/maciejmiklas/LEDDisplay and move its content into Sloeber project folder. This operation should replace two files: LEDClock.h and LEDClock.cpp. Now we have Sloeber cpp project with right main files. Those wired steps were necessary, because I did not want to check in into github IDE specific files. This is our structure now:
    6. There are still compilation errors, we will now add missing libraries
    7. Clone LED Display API: https://github.com/maciejmiklas/LEDDisplay and import into the project:
    8. Repeat this procedure for Logger: https://github.com/maciejmiklas/ArdLog
    9. Imported project LEDDisplay has subfolder: examples. We have to exclude it from compilation, because it contains files with main-loop and this will disturb Sloeber. Select Properties on folder examples and check: Exclude resource from build:
    10. Now you should have following structure:
    11. You can upload it !

    Software for NodeMCU/ESP8266

    I am using EPS8266 with Lua interpreter, this combination is called NodeMCU.

    In order to provide weather and time to Arduino you will have to clone NodeMCUUtil, modify few scripts and finally upload those into NodeMCU. Below you will find exact instruction: 
    1. Compile firmware for NodeMCU so that it has all required modules. Here you will find instructions on it, and those are required modules: file, gpio, net, node, tmr, uart, wifi and cjson.
    2. Clone project containing Lua scripts: https://github.com/maciejmiklas/NodeMCUUtils
    3. Edit serialAPIClock.lua and set UTC offset for your location. This will be required to calculate local date from UTC time. For most European countries it's already set to correct value. For US you will have to replace require "dateformatEurope" with require "dateformatAmerica" and rename all method calls from setEuropeTime to setAmericaTime
    4. Edit yahooWeather.lua and provide city and country that you would like to have weather for.
    5. Create new file called: credentials.lua and specify login data for WiFi connection, it's just one line, for example: cred = {ssid = 'openwifi', password = '123456789'}
    6. Upload all Lua scirpts from main project's folder into NodeMCU:
      • credentials.lua
      • dateformat.lua
      • dateformatAmerica.lua
      • dateformatEurope.lua
      • ntp.lua
      • ntpClock.lua
      • serialAPI.lua
      • serialAPIClock.lua
      • serialAPIYahooWeather.lua
      • wlan.lua
      • yahooWeather.lua
    7. Now for the final touch we need the init-file that will be executed right after NodeMCU boots up. In our case we are using the only Serial Port in order to expose weather and clock API. This also means, that once our API is registered, it's impossible to execute standard NodeMCU commands, like file upload. For this reason init-script has two seconds delay, during this time you can still upload files, or just remove current init.lua file. Init-files are there: NodeMCUUtils/init/serialInit
      • init.lua
      • serialInit.lua

    Github repos used in this project

    Friday, November 18, 2016

    NodeMCU (ESP8266) as provider of Yahoo Weather

    I've developed small library that can be used to access Yahoo Weather over NodeMCU and than provide it over serial port. This can be connected to anything with serial port with - for example Arduino.  In such case you have simple way of accessing weather for your Arduino projects :)

    When connecting ESP8266 to Arduino you will need voltage divider to convert 5v from arduino to 3.3v.


    yahooWeather.lua

    yahooWeather.lua script provides access to Yahoo weather.
    yaw.start() will obtain weather immediately and keep refreshing it every yaw.syncPeriodSec seconds. Weather data itself is stored in yahooWeather.lua -> yaw.weather, you will find there further documentation. 

    In the example below you can see how to get wether for Munich and update it periodically:

    require "yahooWeather"
    require "wlan"
    
    yaw.city = "munich"
    yaw.country = "de"   
    wlan.setup("free wlan", "12345678")
    
    -- update weather every 17 minutes
    yaw.syncPeriodSec = 1020
    
    yaw.responseCallback = function()
        print("Weather for today:", yaw.weather[1].date) 
        print(yaw.weather[1].low, yaw.weather[1].high, yaw.weather[1].text)
    
        print("Weather for tomorrow:", yaw.weather[2].date) 
        print(yaw.weather[2].low, yaw.weather[2].high, yaw.weather[2].text)
    end
    
    -- start weather update timer
    yaw.start()
    

    Weather for today:  01 Sep 2016
    18  25  Partly Cloudy
       
    Weather for tomorrow:   02 Sep 2016
    16  25  Partly Cloudy
    
    

    Serial API

    Script below registers Serial API Interface providing access to weather. Weather itself will get updated every 17 minutes.

    require "serialAPI"
    require "serialAPIYahooWeather"
    
    yaw.city = "munich"
    yaw.country = "de"
    yaw.syncPeriodSec = 1020 -- 17 min
    -- setup wlan required by NTP clokc wlan.setup("fred", "1234567890") -- start serial API by enabling gpio and uart sapi.start() -- start yahoo weather with serial API yaw.start()

    Now you can execute few simple commands over serial:
    # weather description for today
    >YF1 text
    Rain And Snow
    
    # weather description for tomorrow
    >YF2 text
    Showers
    
    # not existing command
    >YF1 min
    ERR:serialAPIYahooWeather.lua:6: attempt to concatenate field '?' (a nil value)
    
    # max temp for tomorrow 
    >YF2 low
    1
    
    # weather date for today
    >YF1 date
    09 Nov 2016
    

    You can find full API description in serialAPIYahooWeather.lua

    Monday, September 19, 2016

    NodeMCU(ESP8266) as provider of NTP date and time over serial port

    I've boon looking for possibility to have current date and time on my Arduino projects. Arduino itself has limited resources and I would like to use them for the actual work. So I've decided to take NodeMCU with Lua language and use it to access date and time form the internet, format it in the way that meets my requirements and provide such result over serial port.

    There are many different ways to access local date and time from the internet, but mostly those interfaces change over time and code using them has to be adopted. I was looking for something that would last for years without interaction form my side. That's why I've decided to use Network Time Protocol. There is only one catch: it's UTC time without date. It was not such a big deal, because there are lot of algorithms out there so I've just adopted one of them.

    Now it's possible to obtain local date and time from NodeMCU over serial port and the whole thing is reliable and will last for many years without touching it :)

    Here is the source code: https://github.com/maciejmiklas/NodeMCUUtils

    The whole implementation is dividend into small building blocks, putting them together will give you desired serial API. This is just short overview:

    • Date Format - calculates local date and time based on timestamp
    • WiFi Access  - simple facade for connecting to WiFi
    • NTP Time - obtains UTC timestamp form give NTP server
    • NTP Clock - keeps actual UTC timestamp and synchronizes it periodically with NTP server
    • Serial API - finally this one provides API for date and time


    Date Format

    Provides functionality to get local date and time from timestamp given in seconds since 1970.01.01

    For such code:
    collectgarbage() print("heap before", node.heap())
    
    require "dateformat"
    require "dateformatEurope"
    local ts = 1463145687
    df.setEuropeTime(ts, 3600) -- function requires GMT offset for your city
    
    print(string.format("%04u-%02u-%02u %02u:%02u:%02d", 
        df.year, df.month, df.day, df.hour, df.min, df.sec))
    print("DayOfWeek: ", df.dayOfWeek)
    
    collectgarbage() print("heap after", node.heap())
    

    you will get this output:
    heap before 44704
    2016-05-13 15:21:27
    DayOfWeek:  6
    heap after  39280
    

    In order to format date for USA you have to replace require "dateformatEurope" with require "dateformatAmerica" and call setAmericaTime instead of setEuropeTime. Functionality is divided into small scripts in order to save some RAM.


    WiFi Access

    It's simple facade for connecting to WiFi. You have to provide connection credentials and function that will be executed after the connection has been established.

    execute(...) connects to WiFi and this can take some time. You can still call this method multiple times. In such case callbacks will be stored in the queue and executed after WiFi connection has been established.

    require "wlan"
    
    wlan.debug = true
    
    local function printAbc() 
        print("ABC")
    end
    
    wlan.setup("free wlan", "12345678")
    wlan.execute(printAbc)
    

    Configuring WiFi on:   free wlan
    status  1
    status  1
    status  5
    Got WiFi connection:   172.20.10.6 255.255.255.240 172.20.10.1
    ABC
    


    NTP Time

    This simple facade connects to given NTP server, request UTC time from it and once response has been received it calls given callback function. 

    Example below executes following chain: WiFi -> NTP -> Date Format. 
    So in the fist step we are creating WLAN connection and registering callback function that will be executed after connection has been established. This callback function requests time from NTP server (ntp.requestTime). 
    On the ntp object we are registering another function that will get called after NTP response has been received: printTime(ts).

    collectgarbage() print("RAM init", node.heap())
    
    require "wlan"
    require "ntp"
    require "dateformatEurope";
    
    collectgarbage() print("RAM after require", node.heap())
    
    ntp = NtpFactory:fromDefaultServer():withDebug()
    wlan.debug = true
    
    local function printTime(ts) 
        collectgarbage() print("RAM before printTime", node.heap())
        
        df.setEuropeTime(ts, 3600)
        
        print("NTP Local Time:", string.format("%04u-%02u-%02u %02u:%02u:%02d", 
            df.year, df.month, df.day, df.hour, df.min, df.sec))
        print("Summer Time:", df.summerTime)
        print("Day of Week:", df.dayOfWeek)
        
        collectgarbage() print("RAM after printTime", node.heap())
    end
    
    ntp:registerResponseCallback(printTime)
    
    wlan.setup("free wlan", "12345678")
    wlan.execute(function() ntp:requestTime() end)
    
    collectgarbage() print("RAM callbacks", node.heap())
    

    and console output:
    RAM init    43328
    RAM after require   30920
    Configuring WiFi on:   free wlan
    RAM callbacks   30688
    status  1
    status  1
    status  5
    Got WiFi connection:   172.20.10.6 255.255.255.240 172.20.10.1
    NTP request:    pool.ntp.org
    NTP request:    194.29.130.252
    NTP response:   11:59:34
    RAM before printTime    31120
    NTP Local Time: 2016-07-12 13:59:34
    Summer Time: 
    Day of Week:    3
    RAM after printTime 30928
    


    NTP Clock

    This script provides functionality to run a clock with precision of one second and to synchronise this clock every few hours with NTP server. 

    In the code below we first configure WiFi access. Once the WiFi access has been established it will call ntpc.start(). This function will start clock that will get synchronized with given NTP server every minute. Now you can access actual UTC time in seconds over ntpc.current. In order to show that it's working we have registered timer that will call printTime() every second. This function reads current time as ntpc.current and prints it as local time. 

    collectgarbage() print("RAM init", node.heap())
    
    require "dateformatEurope";
    require "ntpClock";
    require "wlan";
    
    collectgarbage() print("RAM after require", node.heap())
    
    ntpc.debug = true
    wlan.debug = true
    
    wlan.setup("free wlan", "12345678")
    wlan.execute(function() ntpc.start("pool.ntp.org", 60) end)
    
    local function printTime() 
        collectgarbage() print("RAM in printTime", node.heap())
        
        df.setEuropeTime(ntpc.current, 3600)
        
        print("Time:", string.format("%04u-%02u-%02u %02u:%02u:%02d", 
            df.year, df.month, df.day, df.hour, df.min, df.sec))
        print("Summer Time:", df.summerTime)
        print("Day of Week:", df.dayOfWeek)
    end
    
    tmr.alarm(2, 30000, tmr.ALARM_AUTO, printTime)
    

    so this is the output:
    RAM init    43784
    RAM after require   29408
    Configuring WiFi on:    free wlan
    status  1
    status  5
    Got WiFi connection:    192.168.2.113   255.255.255.0   192.168.2.1
    
    NTP request:    pool.ntp.org
    NTP request:    195.50.171.101
    NTP response:   17:09:46
    
    RAM in printTime    29664
    Time:   2016-08-08 19:10:08
    Summer Time:    true
    Day of Week:    2
    
    RAM in printTime    29808
    Time:   2016-08-08 19:10:38
    Summer Time:    true
    Day of Week:    2
    
    NTP request:    pool.ntp.org
    NTP request:    195.50.171.101
    NTP response:   17:10:46
    
    RAM in printTime    29680
    Time:   2016-08-08 19:11:08
    Summer Time:    true
    Day of Week:    2
    
    RAM in printTime    29808
    Time:   2016-08-08 19:11:38
    Summer Time:    true
    Day of Week:    2
    
    NTP request:    pool.ntp.org
    NTP request:    131.188.3.221
    NTP response:   17:11:46
    
    RAM in printTime    29680
    Time:   2016-08-08 19:12:08
    Summer Time:    true
    Day of Week:    2
    
    RAM in printTime    29808
    Time:   2016-08-08 19:12:38
    Summer Time:    true
    Day of Week:    2
    


    Serial API

    Serial API exposes simple interface that provides access to diagnostic info and date so that it can be accessed outside NodeMCU - for example by Arduino.

    Serial API is divided into few Lua scripts. Loading of each script will automatically add new API commands:
    - serialAPI.lua - has to be always loaded. It initializes serial interface with few diagnostics commands.
    - serialAPIClock.lua - access to clock including date formatter.

    Each script above registers set of commands as keys of scmd table - inside of each script you will find further documentation.

    Example below provides date access over serial port:

    require "credentials"
    require "serialAPI"
    require "serialAPIClock"
    
    ntpc.syncPeriodSec = 900 -- 15 min
    sapi.baud = 115200
    
    -- setup wlan required by NTP clock
    wlan.setup("free wlan", "12345678")
    
    -- start serial API by enabling gpio and uart
    sapi.start()
    
    -- start NTP synchronization
    ntpc.start("pool.ntp.org")
    

    Here are few Serial API commands and their responses:
    # free ram
    >GFR
    10664
    
    # WiFi status
    >GWS
    5
    
    # date and time (24h) in format: yyyy-mm-dd HHLmm:ss
    >CF1
    2016-09-16 10:45:25
    
    # date in format: yyyy-mm-dd
    >CH2
    10:45:59
    

    Firmware

    Executing multiple scripts can lead to out of memory issues on NodeMCU. One possibility to solve it is to build custom firmware containing only minimal set of node-mcu modules: cjson, file, gpio, net, node, tmr, uart, wifi. This blog provides detailed upgrade procedure: http://maciej-miklas.blogspot.de/2016/08/installing-nodemcu-v15-on-eps8266-esp.html

    Wednesday, August 17, 2016

    Upgrading NodeMCU Firmware to v1.5 on EPS8266 (ESP-12E)

    I've been updating firmware on my NodeMCU few times without issues - at least up to 0.9
    The installation procedure has changed with SDK 1.5 - I've spent few hours before I got it right, so maybe this post will help you to save some time.

    Here you will find all required information, but depending on your chip manufacturer you will need different combination of some options, and this is exactly the part when it's getting a bit tricky.

    Since things are changing rapidly, I will point out exact versions in this tutorial, so that it's clear what was working with what at time of this writing.

    Installing esptool

    Download esptool v1.1 https://github.com/themadinventor/esptool/releases/tag/v1.1 and unzip it.
    This will give you possibility to execute esptool.py form command line - we will use it below.

    Determining version of your ESP8266 chip

    As usual there are different versions, and probably you will have to modify flashing commands if you have different one.
    sudo python esptool.py -p /dev/tty.wchusbserial1410 flash_id
    
    esptool.py v1.2-dev
    Connecting...
    Manufacturer: e0
    Device: 4016
    

    Well it's obviously ESP-12E with 4MB flash:

    How do you tell? I've no idea - just google for "Manufacturer: e0 Device: 4016" and check what people are thinking.


    Building custom firmware 

    Now we have to obtain firmware containing Lua interpreter for our chip - it's everything that you will need in order to upload Lua scripts over serial. 

    Basically you will need two binary files: 
    • custom firmware - something like: nodemcu-dev-7-modules-2016-08-10-10-43-59-integer.bin
    • initial-data-block (esp_init_data_default.bin)
    Obviously you will need matching versions. 

    There are two options to get those files: 
    • you can download it from my github repo - this will give you SDK 1.5.4.1
    • you can also use cloud service to build latest release. The link to latest initial-data-block can be found here - just search for esp_init_data_default.bin
    I would prefer a latest version - as usual. But in case of some problems you can try using version provided on my github repo. At least this one was working in my case with ESP-12E, this might help you to locale a problem.

    Erase flash

    Put your chip into flash mode as described here. If you have development board it should have two buttons: flash and reset. In such case press flash and without releasing it press reset, after that release reset.

    Now you have to determine com port, and change it in all commands in this tutorial. In my case it's: /dev/tty.wchusbserial1410

    Let's finally erase flash!
    sudo python esptool.py -p /dev/tty.wchusbserial1410 erase_flash
    esptool.py v1.1
    Connecting...
    Erasing flash (this may take a while)...
    

    Now disconnect ESP from usb port, so that it can fully reset - pressing reset button does not have the same effect !

    Flashing new firmware

    Put your chip again into flash mode.
    I am assuming that you have in you current directory two binary files: one with firmware (nodemcu-dev-7-modules-2016-08-10-10-43-59-integer.bin) and second with init-stuff (esp_init_data_default.bin).
    sudo python esptool.py -p /dev/tty.wchusbserial1410 write_flash -fm dio -fs 32m 0x000000 nodemcu-master-7-modules-2016-08-10-10-45-03-integer.bin 0x3fc000  esp_init_data_default.bin

    esptool.py v1.1
    Connecting...
    Running Cesanta flasher stub...
    Flash params set to 0x0240
    Writing 385024 @ 0x0... 385024 (100 %)
    Wrote 385024 bytes at 0x0 in 33.4 seconds (92.2 kbit/s)...
    Writing 4096 @ 0x3fc000... 4096 (100 %)
    Wrote 4096 bytes at 0x3fc000 in 0.4 seconds (80.0 kbit/s)...
    Leaving...


    Testing....

    I've used ESPlorer, and it looks just fine.



    Thursday, May 12, 2016

    TS 555 timer with trigger network

    I wanted to pimp my son's bobby car by installing few lights, they should go on for a few minutes once he starts moving the car. For this I've installed few LEDs and an micro switch attached to rear axle. So that the micro switch will get triggered from time to time.

    Connecting LEDs was quiet simple, but I had some difficulties to build circuit that will enable the lights for few minutes. I've decided to use 555 timer - there are plenty of examples out there how to assemble it. I just wanted one, that will be energy efficient on stand by, and the lights should always go off after few minutes. This was not so easy, because you have to trigger the 555 by short impulse, otherwise it will never go off. My solution with micro switch has this one catch, that it can remain shorted for very long time. I've could ask my son to keep pushing the car until he hears the second click..... instead I've used a trigger network.

    On the end I had to find right values for all components in the way that it will guarantee the functionality and ensure low power consumption to not to drain batteries over few days. Actually I did not calculate those values, I've just used oscilloscope to make sure that timing on 555 pins is right. Probably I've got not a perfect values, but on the other hand side it does what is suppose to be doing and it runs on single battery for months :)



    Here is the Fritzing drawing.