SOAP is not very popular these days, primarily because of almost nonhuman readable WSDL and protocol that puts a lot of overhead into a simple message. It's tough to send a message using a browser or curl. Not mentioning the fact that we have to obtain WSDL somehow.
But on the other hand side, SOAP has some advantages over REST. It allows us to define a contract and specify the data it will process. Once you have such WSLD, you can throw it over the hedge, and you do not have to answer questions like how do I delete a created resource, what 403 means in this case, or is this call idempotent?
WSLD contains everything needed to define an RPC interface and document it. I would choose REST for most cases, but still, there are some good use cases for SOAP.
This one thing bothers me about SOAP: how do I get proper WSLD? Should I use a tool to generate it or write it? Honestly... I really do not want to dig into SOAP to just write WSLD, and I do not want to spend hours playing around with some tools generating WSLD, importing it into my project, and binding generated stuff to actual implementation only to find out about the end that it does not operate as expected.
The perfect solution would be to write Java code, annotate it, and generate WSDL out of it. Those annotations should provide enough flexibility to influence WSLD generation so that we archive something that can be delivered to the customers.
I've tried a few Java frameworks, and all provide a way to generate WSLD out of Java code, but they all need to include one fundamental feature: you cannot offer XSD for data types. So it's impossible to specify the content of an email field or provide a format for a date. We get only half of a possible functionality of a WSDL: method calls, exception handling, documentation, and security, but there is no way to specify the data format.
I would like to have WSLD that is as precise as possible so that the client can call the particular method and know precisely what is possible and is not allowed.
Let's start with the standard SOAP Service generated out of Java code. We will use Apache CXF and Spring Boot for it. The source code is here: https://github.com/maciejmiklas/apache-cxf-soap.
It's a simple application where users can submit registration as a SOAP Request on http://localhost:8080/soap/Registration:
This code outpost following WSDL:
As you can see, this WSDL has little info about data types (lines 9-22). The email is mandatory, but there is no way to provide any further assertions.
The problem lies in JAXB. It ignores annotations to influence generated XML schema. There is a pull request (jaxb-facets) that would solve this issue, but it's already a few years old and looks like it will only be integrated for a while.
The limitation is not caused by SOAP frameworks but by the fact that they are based on JAXB.
This does not change the fact that I will not write WSLD myself. There has to be a better way!
We will implement a simple extension for Apache CXF to combine generated WSLD with custom XSD. We have to write XSD in this case, but CXF will generate WSLD for us and include the given schema.
XSD is mighty, so I prefer writing it by hand because it's possible to specify the data format precisely. Using annotations to generate XSD would be convenient, but it would still cover only some common areas.
The implementation below has its limitations. You might run into some issues. It might stop working after the next CXF update. But! I use it, and it does what I need, so it might be something for you as well ;)
Now we are going to modify the first example. The idea is to write XSD that defines simple types, references those types in transfer classes, and generates WSLD, combining them all.
In the beginning, we have to write a schema that defines types for transfer objects:
The original source code will be modified only in a few places:
We will replace the default Apache CXF data binding with a custom implementation. It reads Schama from the file and integrates it into generated WSLD,
Some fields in transfer objects are annotated with @XmlSchemaType - this annotation provides a connection between Java types and types defined in XSD. For example, the ExRegistration#email is annotated with @XmlSchemaType(name = "email"), and XSD contains the email type. The email generated in WSDL references the type from a provided schema,
the classes following the pattern *Registration* have been renamed to *ExRegistration*
Spring Java Config is a great tool that gives us a precise way to control an application's bootstrap and wiring. However, it only sometimes works as expected. We occasionally run into strange issues where a particular bean is not instantiated at the expected time, lazy loading does not work as expected, or the whole configuration is ignored for no apparent reason.
This is the first exercise. It defines a basic structure that will be modified as we go along.
The main method loads Spring context from the configuration file Conf.java. This configuration instructs Spring to create 3 beans. Each of these beans has two log statements, the first in the constructor and the second in @PostConstruct. The console output after the execution of this example is shown below the code.
Nothing spectacular happens here: beans are initialized from A to C right after the constructor Spring executes also @PostConstruct. There are no dependencies between the beans.
exa02
Let's modify the order of factory methods in Conf.java.
The bean initialization order has changed. It means that method names do not matter, but the physical order within a class. It can change with JVM, so you should not rely on it!
exa03
Now we will extract the definition of BeanB into a dedicated configuration file:
Spring loads bean definitions from the imported configuration in the first place.
exa04
We will modify the previous exercise so that BeanB depends on BeanA.
Spring loads dependent beans in the first place. It does not matter where it's located. Spring will scan all possible configurations to find it.
exa05
The same as exa04, but BeanB does not have @DependsOn("beanA"). Instead, BeanB injects BeanA. We have a similar dependency situation to exa4: BanB depends on BeanA. However @PostConstructs are called in a different order: Spring creates an instance of BeanB, and during the construction phase (constructor), references to BeanA are not set. Once the instance of BeanB is created, BeanA will be created and injected into BeanB. BeanB cannot access BeanA in the constructor, first after a complete initialization is done - in the method @PostConstruct.
Spring had to modify the order of @PostConstruct calls to ensure that the bean references were not null during an initialization phase.
Summary
within a single configuration class, the bean instantiation order depends only on the methods order within this class, not method names. However, this might depend on JVM,
Spring loads, in the first place, imported bean definitions. The previous rule applies only to the situation where direct bean definitions do not depend on imported beans. If this is the case, Spring will load the dependent bean in the first place,
the initialization code has to be placed in the @PostConstruct method and not in a constructor. Otherwise, references to some beans might be null.
BeanA, BeanB, and BeanC are created through dedicated configuration classes: ConfBeanA, ConfBeanB, and ConfBeanC. BeanD is declared in Conf Beans are created in order: A, B, C, D.
exb02
Similar to exb01, but the configuration class ConfBeanB has been renamed to ConfBeanXB.
The instantiation order remains unchanged.
exb03
The same as exb01, but the factory method in ConfBeanB has been renamed from beanB to xyz. The instantiation order remains unchanged.
exb04
The same as exb01, but BeanB has been renamed to BeanXB. This change did not influence instantiation order.
exb05
Similar to exb01, but we've changed the import order from A, B, C into B, A, C. The instantiation order has changed as well.
exb06
Same as exb01, but BeanC has been injected into BeanA. The instantiation order has been changed because BeanC had to be created before BeanA
Summary
when the @Import statement contains multiple classes, Spring will load them from the left to the right side - it's iterating over an array, and names of configuration classes do not matter in this case,
within single configuration class physical order of methods matters, not their names.
Duplicated Bean Name
exc01
We have 3 beans, A, B, and C, with dedicated config here. BeanA injects BeanB and BeanC.
exc02
The same as exc01, but we've renamed factory method ConfBeanB#beanB() into ConfBeanB#beanC(). Spring goes over our @Import declaration to determine all possible bean definitions: ConfBeanA, ConfBeanB, ConfBeanC. There are two factory methods with the same name: ConfBeanB#beanC() and ConfBeanC#beanC(), so ConfBeanC overwrites the bean created through ConfBeanB because it creates a bean with the same name.
exc03
Similar to exc02, but this time we've renamed method ConfBeanC#beanC() into ConfBeanC#beanB(), so that we have two methods beanB, and not as it was in exc02 two methods with the name: beanC. The output is still the same, BeanB is missing. We've just changed the name of the factory method, and we already know that names do not change loading order, and we still have two beans with the same name.
exc04
Similar to exc02, but we've changed the import order for config classes from A, B, C to A, C, B. Now ConfBeanB will get scanned on end, and it overwrites the previous bean with the same name, so BeanC is missing.
exc05
It's a modified exc02. We've set AllowBeanDefinitionOverriding to false on Application Context. Bean overwriting is disabled now, so instead of bean not found, we are getting an exception that we are trying to register two beans under the same name.
exc06
The same as exc02, there are still two factory methods: beanC. Additionally, BeanB and BeanC are implementing the common interface. Now BeanA does not inject BeanB and BeanC directly but injects a collection of beans implementing our interface. In the case of direct injection, one of the beans was missing; now both are there!
Summary
In the case of two beans with the same name, the last one wins. You cannot directly inject such overwritten beans, but you can inject a collection of such beans sharing a common interface.
Lazy Loading
exd01
We have three beans: A, B, and C, and there are no dependencies between them. BeanC is being defined through a dedicated configuration class. Nothing special here. The initialization order is as expected.
exd02
The same as exd02, but we've annotated BeanB as lazy. Spring does not load BeanB at all. There are no dependencies to that bean.
exd03
Now we've injected BeanB into BeanA. @Lazy is set on injection point and bean definition. Additionally, we have defined a method on BeanA that calls a method on BeanB: ApplicationExD03 -> beanA.method() -> beanB.method()
Spring postpones the creation of BeanB until the method is being called on it, @Lazy works as expected.
exd04
Same as exd03, but @Lazy has been removed on the injection point. BeanB is not lazy anymore, @Lazy on bean definition is not enough.
exd05
Same as exd03, but @Lazy has been removed from the bean configuration. BeanB is not lazy anymore, @Lazy on bean definition is not enough.
Summary
lazy annotation has to be provided on configuration, and all injection points, otherwise Spring eagerly initializes such beans,
lazy beans that are not referenced (injected) are not loaded at all,
you should not rely on @Lazy unless you are 100% sure you can control all possible injection points. Otherwise, one missing Lazy-Annotation will disable laziness on the such bean.
We will build a Thermostat based on Arduino, a temperature sensor, and a relay this time.
You can find it on GitHub
This Thermostat gives you the possibility to drive multiple devices in order to control temperature. In my case, I have installed two fans in my attic in order to cool it down in summer. There are no windows, so I had to force airflow. The first fan starts when the temperature reaches 35 degrees, and the second over 45.
You can control any reasonable amount of units, it's all configurable. You have also had access to basic statistics for the past two weeks:
Runtime of whole system
ON time for each relay
Statistics for 14 days
Statistics for 7th day
Configuration
The whole configuration is stored in Config.h. You can change PINs controlling relays, buttons, and input for reading temperature, thresholds, or timings.
Hardware
Buttons
Two buttons are dedicated for menu navigation and the third one resets statistics. They are stored in EEPROM (Storage.cpp) once a day. Pins assigned to buttons can be found in Config.h
Configuring Relays
Let's assume that we would like to have 3 relays:
ID:0, PIN: 1, Temperature setpoint: 20
ID:1, PIN: 10, Temperature setpoint: 30
ID:2, PIN: 11, Temperature setpoint: 40
First, you have to make sure that PIN of your choice is not already taken. All pins can be found in Config.h, they are defined by variables starting with DIG_PIN.
You have to edit Config.h and configure PINs, thresholds and amount of relays. Obviously, some properties already exist, so you have to just edit them.
RELAY_DELAY_AFTER_SWITCH_MS gives a wait time for switching to the next relay. Imagine that configuration from our example would start working in 40 degrees environment. This would result in enabling all three relays at the same time. This could eventually lead to high power consumption - depending on what you are controlling, an electric engine, for example, consumes more power during the start. In our case, switching relays have the following flow: the first relay goes, waits 5 minutes, the second goes on, waits 5 minutes, third goes on.
RHC_RELAY_MIN_SWITCH_MS defines hysteresis, it's the minimum frequency for a particular relay to change its state. Once it's on, it will remain on for at least this period of time, ignoring temperature changes. This is quite useful if you control electric motors since each switch negatively impacts live time.
PID Controller
This is an advanced topic. Implementing such a controller is a simple task, finding the right amplitude settings is a different story.
To use the PID controller, you must change initRelayHysteresisController(.....) to initRelayPiDController(....) and find the right settings for it. You will find them in Config.h
I've implemented a simple simulator in Java so that it's possible to visualize the results. It can be found in the folder: pidsimulator.
Below you can see simulations for two controllers PID, and P. PID is not perfectly stable because I did not apply any sophisticated algorithm to find the right values.
On both plots required temperature is set to 30 (blue). Current temperature indicates read line. The relay has two states ON and OFF. When it's enabled, temperature drops by 1.5, when it's disabled, it rises by 0.5.
Software Design
Message Bus
Different software modules have to communicate with each other, hopefully not both ways ;)
For example:
statistics module has to know when the particular relay goes on and off,
pressing a button has to change display content, and it also has to suspend services that would consume many CPU cycles, for example, temperature reading from the sensor,
after some time temperature reading has to be renewed,
and so on...
Every module is connected to Message Bus and can register for particular events and can produce any events.
For example, pressing Next button results in the following flow:
Some components have some tasks that need to be executed periodically. We could call their corresponding methods from the main loop since we have Message Bus it's only necessary to propagate the right event:
LIBS
The following libs are required to compile Thermostat:
We are about to build a weather station with a forecast for three days, including a clock with local time and date.
The whole project is based on Arduino, and NodeMCU provides access to the Internet over WiFi. The display is builtfrom single LEDs. Since they can be really bright, it adopts illumination based on light conditions.
Let's start from the beginning ;)
I've found those 8x8 LED modules:
So I've decided to combine a few to build a display. In my case, there are 3 lines, each consisting of 8 modules, 24 in total, this gives us 1532 single LEDs!
To drive a single module, I've chosen MAX72xx, I also wanted to improve my soldering skills, so I've decided to go for 24 PIN DIP chips and solder them to prototype boards:
Well, that worked out pretty well when it comes to those skills, but I would recommend using LED modules combined with MAX Chip, this will save you at least a few hours, not to mention time spent afterward when a single cable gets loose ;) Such a combo module has only 3 wires instead of 16.
So we have a hardware part for our display: 24 led modules with drivers. Now it's time to light them up! I've decided to go for Arduino Mega because it has two serial ports, so it's easier to debug things (one port will be used for communication with EPS8266). You could also use Uno, in this case, you would have to daisy chain MAX chips and change addressing in the software. I've used separate lines for each Max chip, but Uno just does not have enough digital output pins.
I was looking for API that will join all led modules into one canvas, so you can print sprites without bothering with transitions between LED modules. I did not find anything that would make me happy, so I implemented one myself. It provides not only a simple canvas but fonts and a few animations. Basically, everything that will be needed to display time and weather.
So ... we have a display and API to control it. Now we need to get the date and weather. Arduino does not support Internet connectivity and does not have enough resources to process incoming data. So I've decided to use NodeMCU. With a few Lua scripts, I could implement a simple API accessible over the serial port. Arduino connects over it with NodeMCU, obtains time, date, and weather, and displays it.
To provide the date, NodeMCU connects with the NTP server and receives UTC time, afterwards it calculates the local date from it. I could use one of the Internet services to grab the date, but I was looking for a solution that would remain stable for a long time. Those services can change their API or just go offline, but the NTP protocol will remain unchanged. In the worst case, you will have to change the server name. The current implementation also supports failover through many different servers, so you will probably not have to bother with it soon ;)
Getting the weather was a bit tricky because I had to find an API that would return a response small enough so it could be parsed by NodeMCU. I've decided to use Yahoo Weather. They provide a nice REST API with a small and simple response. Hopefully, they will keep interfaces stable for a long time.....
Putting things together
Hardware
First, you have to build the display, I've already described it in this post: Arduino LED Display . You can use the same pin numbers on Mega to not have to alter the software.
After the display is ready, you should connect ESP8266 and the photoresistor. The Schematic below contains all hardware elements together:
The area on the top right corner is the display itself - 8x3 LED modules. Every single module exposes 3 PINs, those are MAX 72xxx PINs responsible for communication over SPI.
Here you will find an exact schematic of the single module, including SPI PINs:
Confirm the next screens until it asks you about code, select cpp:
Finish it, and you should get something like that:
Clone the LEDClock project from https://github.com/maciejmiklas/LEDDisplay and move its content into the Sloeber project folder. This operation should replace two files: LEDClock.h and LEDClock.cpp.Now we have the Sloeber cpp project with the right main files. Those wired steps were necessary because I did not want to check into GitHub IDE-specific files. This is our structure now:
There are still compilation errors, we will now add missing libraries
Imported project LEDDisplay has a subfolder: examples. We must exclude it from compilation because it contains files with the main-loop, which will disturb Sloeber. Select Properties on folder examples and check: Exclude resource from the build:
Now you should have the following structure:
You can upload it!
Software for NodeMCU/ESP8266
I am using EPS8266 with a Lua interpreter, this combination is called NodeMCU.
To provide weather and time to Arduino, you must clone NodeMCUUtil, modify a few scripts and finally upload those into NodeMCU. Below you will find exact instructions:
Compile firmware for NodeMCU so that it has all required modules. Here you will find instructions on it, which are the required modules: file, gpio, net, node, tmr, uart, wifi and cjson.
Edit serialAPIClock.lua andset UTC offset for your location. This will be required to calculate local date from UTC time. For most European countries it's already set to correct value. For US you will have to replace require "dateformatEurope" with require "dateformatAmerica" and rename all method calls from setEuropeTime to setAmericaTime
Edit yahooWeather.lua and provide city and country that you would like to have weather for.
Create a new file called: credentials.lua and specify login data for WiFi connection, it's just one line, for example cred = {ssid = 'openwifi', password = '123456789'}
Upload all Lua scirpts from main project's folder into NodeMCU:
credentials.lua
dateformat.lua
dateformatAmerica.lua
dateformatEurope.lua
ntp.lua
ntpClock.lua
serialAPI.lua
serialAPIClock.lua
serialAPIYahooWeather.lua
wlan.lua
yahooWeather.lua
Now for the final touch, we need the init-file that will be executed right after NodeMCU boots up. In our case, we use only the Serial Port to expose weather and clock API. This also means that once our API is registered, it's impossible to execute standard NodeMCU commands, like file upload. For this reason init-script has two seconds delay, during this time you can still upload files, or just remove current init.lua file. Init-files are there: NodeMCUUtils/init/serialInit
I've developed a small library that can be used to access Yahoo Weather over NodeMCU and then provide it over the serial port. This can be connected to anything with a serial port - for example, Arduino. In such case, you have an easy way of accessing the weather for your Arduino projects :)
When connecting ESP8266 to Arduino, you need a voltage divider to convert 5v from Arduino to 3.3v.
yahooWeather.lua script provides access to Yahoo Weather. yaw.start() will obtain weather immediately and refresh it every yaw.syncPeriodSec seconds. Weather data itself is stored in yahooWeather.lua -> yaw.weather, you will find their further documentation.
In the example below, you can see how to get wether for Munich and update it periodically:
Weather for today: 01 Sep 2016
18 25 Partly Cloudy
Weather for tomorrow: 02 Sep 2016
16 25 Partly Cloudy
Serial API
The script below registers Serial API Interface providing access to weather. The weather itself will get updated every 17 minutes.
require"serialAPI"require"serialAPIYahooWeather"
yaw.city ="munich"
yaw.country ="de"
yaw.syncPeriodSec =1020-- 17 min
-- setup wlan required by NTP clokc
wlan.setup("fred", "1234567890")
-- start serial API by enabling gpio and uart
sapi.start()
-- start yahoo weather with serial API
yaw.start()
Now you can execute a few simple commands over serial:
# weather description for today
>YF1 text
Rain And Snow
# weather description for tomorrow
>YF2 text
Showers
# not existing command
>YF1 min
ERR:serialAPIYahooWeather.lua:6: attempt to concatenate field '?'(a nil value)# max temp for tomorrow
>YF2 low
1
# weather date for today
>YF1 date
09 Nov 2016
You can find the whole API description in serialAPIYahooWeather.lua
I've been looking for a date and time source for my Arduino projects. Arduino has limited resources, and I would like to use them for the actual work. So I've decided to use NodeMCU with Lua to access date and time from the internet, format it in a way that meets my requirements, and provide such results over the serial port.
There are many ways to access date and time from the internet, but mainly those interfaces change over time, and code using them must be adopted. I was looking for something that would last for years without interaction from my side. That's why I've decided to use Network Time Protocol. There is only one catch: it's UTC time without a date. It was not such a big deal because there are a lot of algorithms out there, so I've just adopted one of them.
Now it's possible to obtain local date and time from NodeMCU over the serial port, and the whole thing is reliable and will last for many years without touching it :)
The implementation is divided into small building blocks; putting them together will give you the desired serial API. This is just a short overview:
Date Format - calculates local date and time based on the timestamp
WiFi Access - simple facade for connecting to WiFi
NTP Time - obtains UTC timestamp form given NTP server
NTP Clock - keeps actual UTC timestamp and synchronizes it periodically with the NTP server
Serial API - finally, this one provides API for date and time
Date Format
Provides functionality to get local date and time from timestamp given in seconds since 1970.01.01
For such code:
collectgarbage()print("heap before",node.heap())require"dateformat"require"dateformatEurope"localts=1463145687df.setEuropeTime(ts,3600)-- function requires GMT offset for your cityprint(string.format("%04u-%02u-%02u %02u:%02u:%02d",df.year,df.month,df.day,df.hour,df.min,df.sec))print("DayOfWeek: ",df.dayOfWeek)collectgarbage()print("heap after",node.heap())
You will get this output:
heap before 44704
2016-05-13 15:21:27
DayOfWeek: 6
heap after 39280
To format the date for the USA, you have to replace "dateformatEurope" with "dateformatAmerica" and call setAmericaTime instead of setEuropeTime. Functionality is divided into small scripts to save some RAM.
WiFi Access
It's a simple facade for connecting to WiFi. You have to provide connection credentials and a function that will be executed after the connection has been established.
execute(...) connects to WiFi, which can take some time. You can still call this method multiple times. In such cases, callbacks will be stored in the queue and executed after establishing a WiFi connection.
Configuring WiFi on: free wlan
status 1
status 1
status 5
Got WiFi connection: 172.20.10.6 255.255.255.240 172.20.10.1
ABC
NTP Time
This simple facade connects to a given NTP server, requests UTC time from it, and once a response has been received, it calls the given callback function.
The example below executes the following chain: WiFi -> NTP -> Date Format.
So in the first step, we create a WLAN connection and register a callback function that will be executed after the connection has been established. This callback function requests time from the NTP server (ntp.requestTime).
On the ntp object, we are registering another function that will get called after the NTP response has been received: printTime(ts).
collectgarbage()print("RAM init",node.heap())require"wlan"require"ntp"require"dateformatEurope";collectgarbage()print("RAM after require",node.heap())ntp=NtpFactory:fromDefaultServer():withDebug()wlan.debug=truelocalfunctionprintTime(ts)collectgarbage()print("RAM before printTime",node.heap())df.setEuropeTime(ts,3600)print("NTP Local Time:",string.format("%04u-%02u-%02u %02u:%02u:%02d",df.year,df.month,df.day,df.hour,df.min,df.sec))print("Summer Time:",df.summerTime)print("Day of Week:",df.dayOfWeek)collectgarbage()print("RAM after printTime",node.heap())endntp:registerResponseCallback(printTime)wlan.setup("free wlan","12345678")wlan.execute(function()ntp:requestTime()end)collectgarbage()print("RAM callbacks",node.heap())
and console output:
RAM init 43328
RAM after require 30920
Configuring WiFi on: free wlan
RAM callbacks 30688
status 1
status 1
status 5
Got WiFi connection: 172.20.10.6 255.255.255.240 172.20.10.1
NTP request: pool.ntp.org
NTP request: 194.29.130.252
NTP response: 11:59:34
RAM before printTime 31120
NTP Local Time: 2016-07-12 13:59:34
Summer Time:
Day of Week: 3
RAM after printTime 30928
NTP Clock
This script provides functionality to run a clock with the precision of one second and to synchronize this clock every few hours with the NTP server.
In the code below, we first configure WiFi access. Once the WiFi access has been established, it will call ntpc.start(). This function will start a clock synchronizing with a given NTP server every minute. Now you can access actual UTC time in seconds over ntpc.current. To show that it's working, we have registered a timer that will call: printTime() every second. This function reads current time as ntpc.current and prints it as local time.
collectgarbage()print("RAM init",node.heap())require"dateformatEurope";require"ntpClock";require"wlan";collectgarbage()print("RAM after require",node.heap())ntpc.debug=truewlan.debug=truewlan.setup("free wlan","12345678")wlan.execute(function()ntpc.start("pool.ntp.org",60)end)localfunctionprintTime()collectgarbage()print("RAM in printTime",node.heap())df.setEuropeTime(ntpc.current,3600)print("Time:",string.format("%04u-%02u-%02u %02u:%02u:%02d",df.year,df.month,df.day,df.hour,df.min,df.sec))print("Summer Time:",df.summerTime)print("Day of Week:",df.dayOfWeek)endtmr.alarm(2,30000,tmr.ALARM_AUTO,printTime)
so this is the output:
RAM init 43784
RAM after require 29408
Configuring WiFi on: free wlan
status 1
status 5
Got WiFi connection: 192.168.2.113 255.255.255.0 192.168.2.1
NTP request: pool.ntp.org
NTP request: 195.50.171.101
NTP response: 17:09:46
RAM in printTime 29664
Time: 2016-08-08 19:10:08
Summer Time: true
Day of Week: 2
RAM in printTime 29808
Time: 2016-08-08 19:10:38
Summer Time: true
Day of Week: 2
NTP request: pool.ntp.org
NTP request: 195.50.171.101
NTP response: 17:10:46
RAM in printTime 29680
Time: 2016-08-08 19:11:08
Summer Time: true
Day of Week: 2
RAM in printTime 29808
Time: 2016-08-08 19:11:38
Summer Time: true
Day of Week: 2
NTP request: pool.ntp.org
NTP request: 131.188.3.221
NTP response: 17:11:46
RAM in printTime 29680
Time: 2016-08-08 19:12:08
Summer Time: true
Day of Week: 2
RAM in printTime 29808
Time: 2016-08-08 19:12:38
Summer Time: true
Day of Week: 2
Serial API
Serial API exposes a simple interface that provides access to diagnostic info and date to be accessed outside NodeMCU - for example, by Arduino.
Serial API is divided into a few Lua scripts. Loading of each script will automatically add new API commands:
- serialAPI.lua - has to be always loaded. It initializes the serial interface with a few diagnostics commands.
- serialAPIClock.lua - access to clock including date formatter.
Each script above registers a set of commands as keys of scmd table and contains further documentation.
The example below provides date access over the serial port:
require"credentials"require"serialAPI"require"serialAPIClock"ntpc.syncPeriodSec=900-- 15 minsapi.baud=115200-- setup wlan required by NTP clockwlan.setup("free wlan","12345678")-- start serial API by enabling gpio and uartsapi.start()-- start NTP synchronizationntpc.start("pool.ntp.org")
Here are few Serial API commands and their responses:
# free ram
>GFR
10664
# WiFi status
>GWS
5
# date and time (24h) in format: yyyy-mm-dd HHLmm:ss
>CF1
2016-09-16 10:45:25
# date in format: yyyy-mm-dd
>CH2
10:45:59
Firmware
Executing multiple scripts can lead to out-of-memory issues on NodeMCU. One possibility to solve it is to build custom firmware containing only a minimal set of node-mcu modules: cjson, file, gpio, net, node, tmr, uart, and WiFi. This blog provides a detailed upgrade procedure: http://maciej-miklas.blogspot.de/2016/08/installing-nodemcu-v15-on-eps8266-esp.html
I've been updating firmware on my NodeMCU a few times without issues - at least up to 0.9
The installation procedure has changed with SDK 1.5 - I've spent a few hours before I got it right, so maybe this post will help you save time.
Here you will find all the required information, but depending on your chip manufacturer, you will need different combinations of some options, and this is exactly the part when it's getting tricky.
Since things are changing rapidly, I will point out exact versions in this tutorial so that it's clear what was working with what at the time of this writing.
How do you tell? I've no idea - just Google "Manufacturer: e0 Device: 4016" and check what people think.
Building custom firmware
Now we have to obtain firmware containing the Lua interpreter for our chip - it's everything you will need to upload Lua scripts over serial. Basically, you will need two binary files:
Obviously, you will need matching versions. There are two options to get those files:
you can download it from my github repo - this will give you SDK 1.5.4.1
you can also use cloud service to build latest release. The link to latest initial-data-block can be found here - just search for esp_init_data_default.bin
I would prefer the latest version - as usual. But in case of some problems, you can try using the version provided on my github repo. At least this one was working in my case with ESP-12E, this might help you to locate a problem.
Erase flash
Put your chip into flash mode as described here. If you have a development board, it should have two buttons: flash and reset. In such case, press flash, and without releasing it, press reset, after that, release flash.
Now you must determine and change the com port in all commands in this tutorial. In my case, it's /dev/tty.wchusbserial1410
Let's finally erase flash!
sudo python esptool.py -p /dev/tty.wchusbserial1410 erase_flash
esptool.py v1.1
Connecting...
Erasing flash (this may take a while)...
Now disconnect ESP from usb port, so that it can fully reset - pressing reset button does not have the same effect!
Flashing new firmware
Put your chip again into flash mode.
I am assuming that you have in your current directory two binary files: one with firmware (nodemcu-dev-7-modules-2016-08-10-10-43-59-integer.bin) and a second with init-stuff (esp_init_data_default.bin).
I wanted to pimp my son's bobby car by installing a few lights, they should go on for a few minutes once he starts moving the car. I've installed a few LEDs and a micro switch attached to the rear axle for this. So that the microswitch will get triggered from time to time.
Connecting LEDs was quite simple, but I had some difficulties building a circuit that would enable the lights for a few minutes. I've decided to use a 555 timer - plenty of examples exist of how to assemble it. I just wanted one that would be energy efficient on stand-by, and the lights should always go off after a few minutes. This was difficult because you have to trigger the 555 by short impulse, otherwise, it will never go off. My solution with micro switch has this one catch: it can remain shorted for a very long time. I could ask my son to keep pushing the car until he hears the second click..... instead, I've used a trigger network.
In the end, I had to find the right values for all components in a way that would guarantee the functionality and ensure low power consumption to not drain batteries over a few days. Actually, I did not calculate those values, I've used an oscilloscope to make sure that timing on 555 pins is right. Probably I've got not a perfect value, but on the other hand side it does what it is supposed to be doing, and it runs on a single battery for months :)
My latest project (https://github.com/maciejmiklas/LEDDisplay) contains the driver for 8x8 LED Modules controlled via
MAX722xx. It allows you to build a display of custom size that is only
limited by the hardware itself. Vertical and horizontal size can contain
up to 256 modules, but before reaching this limit, you would run out of
Slave Select lines for controlling MAX chips, or you would be limited by the amount of RAM. The fact is: you can control a reasonable amount of MAX
chips and build a display of custom size ;)
I've tested the whole idea on display which consist of 8 LED Modules
in horizontal and 3 in vertical position. This gives us 24 modules that
contain 1536 LEDs (88 * 38).
Hardware
First, let's start with the controller, actually, any Arduino will
work, I've used Mega due to the large number of digital output pins. You
could also use Nano with a shift register and alter the way of addressing
Select Slave lines in Display::send(...).
You will need an extra power supply for driving LEDs - assuming that you will use more than one LED Matrix.
Driving single LED Matrix Module
I've used the MAX7219 and 788BS LED Matrix, this is the one with a common
anode. The schematic below illustrates the wiring of LEDs, MAX and Arduino:
This one is equivalent, but instead of single LEDs we have PIN layout of LED Module:
It might happen that your LED Module has a common cathode, in this case, you have to rewire the connection to the MAX chip. You just have to remember that MAX has two sets of relevant pins: Dig0-Dig7
are supposed to be connected to cathodes (-) and SegA-SegG to anodes(+). Such a change will also swap rows with columns within the sine LED
module.
Connecting all LED Matrix together
In the previous chapter, we've seen how to connect a single LED Module with an MAX chip. Now we will connect multiple LED Modules together into one
large display.
Below is the physical display that I've used for testing and examples.
Each LED Module has label indicating its position and Select Slave line.
Each 3-PIN connector on the schematic above symbolizes one module
described in the previous chapter (LED Matrix + MAX72xx), now we've
connected all those modules together.
All MAX722xx chips share common MOSI and SCK lines, MISO is not used, and each chip occupies a separate Slave Select line.
The position of LED Matrix on the schematic above directly
corresponds to their location on the physical display that I've used for
testing. Additionally, each module has a description indicating its
position and Select Slave line, so for example: (2,1) SS: 35 gives us a second module on the third row (counting from zero) and PIN:35 on Arduino for the Select Slave line.
Software
Compiling
We are using standard Arduino libraries, which are already available, the only exception is ArdLog. You have to import this LIB into your IDE. This basically means that you have to download the right release: https://github.com/maciejmiklas/ArdLog/releases/tag/v1.0.0 and unzip it into a folder, where you usually place external libraries. In the case of standard Arduino IDE on Mac, it's ~/Documents/Arduino/libraries. In the end you should have the following structure:
$ pwd
~/Documents/Arduino/libraries/ArdLog
$ ls
ArdLog.cpp ArdLog.h LICENSE README.md
Communication with MAX72xxx
We are using the standard SPI library and Select Slave line on the MAX chip for
addressing. MAX is configured in LED Matrix mode - so there is nothing
special. The setup method can be found in Display::setup()
Setting things up
The main class of our interest will be the Display - it's responsible for the setup of MAX chips and provides API for painting.
Before we start painting, it's necessary to set things up. The code below
creates a 2D array containing Select Slave lines and initializes the display.
The display consists of 3 rows, each with 8 LED Modules.
You can choose any responsible size, but I will stick to this
one.
The layout of the mentioned 2D array corresponds to the physical display:
each LED Module has a dedicated MAX chip, and each chip has a dedicated
Select Slave line. The first dimension of our array indicates the physical row
on display, the second dimension indicates LED Module within this row, and
the value itself contains the PIN number for the Select Slave line.
#include <Display.h>Display*disp;/** * Orientation of LED Kits (8x8 LED matrix) on display that I've used for testing. * The numbers are indicating Select Slave line of MAX7219. * 48, 46, 49, 47, 45, 43, 41, 39 * 36, 34, 32, 30, 28, 26, 24, 22 * 37, 35, 33, 31, 29, 27, 25, 23 */ss_t**ss;ss_t**createSS(){ss_t**ss=alloc2DArray8(3,8);// first rowss[0][0]=48;ss[0][1]=46;ss[0][2]=49;ss[0][3]=47;ss[0][4]=45;ss[0][5]=43;ss[0][6]=41;ss[0][7]=39;// second rowss[1][0]=36;ss[1][1]=34;ss[1][2]=32;ss[1][3]=30;ss[1][4]=28;ss[1][5]=26;ss[1][6]=24;ss[1][7]=22;// third rowss[2][0]=37;ss[2][1]=35;ss[2][2]=33;ss[2][3]=31;ss[2][4]=29;ss[2][5]=27;ss[2][6]=25;ss[2][7]=23;returnss;}voidsetup(){util_setup();log_setup();ss=createSS();// Test display consist of 8x3 LED Modules (3 rows, each one 8 Modules)disp=newDisplay(8,3,ss);disp->setup();}
There is one more method worth mentioning: log_setup(). The whole project has quite a precise logger - so that you can see what is actually
happening. By default, it's disabled, in order to enable it to check out its
documentation: https://github.com/maciejmiklas/ArdLog
Painting on the display
The display consists of a few LED Modules, but from an API perspective, they are
connected together into one continuous canvas. You can place on this
canvas bitmap any position given by such coordinates:
(0,0) -----------------------------> (x)
|
|
|
|
|
|
| (x max, y max)
v (y)
The paint method has the following syntax: paint(pixel_t x, pixel_t y, pixel_t width, pixel_t height, uint8_t data).
It allows you to paint a bitmap on given coordinates with limited width
and height. So you can, for example, paint a bitmap on (3,4) that has
25x3 pixels. It might be larger than the actual display size - in this
case, it will get trimmed.
This is obvious and simple, but there is one catch - you must provide the right data.
This is a 2D array, where the first dimension indicates the vertical and the second
horizontal position on the display. Technically speaking, data is a flat array of pointers, and each pointer points to an array that represents one horizontal line on the display.
Moving over the first dimension of data traverses over lines of the display. The second data dimension represents horizontal pixels within a single line; each byte
represents 8 pixels. Since our display consists of simple LEDs, they can
be either on or off, so each pixel is not represented by
one byte but by one bit. In order to cover 16 pixels in a horizontal
position, we need two bytes, 24 pixels require 3 bytes, and so on.
For example, to fully cover a display consisting of 8x3 LED kits (one used in our examples), we would need data[3][8]. Usually, you will take an array small enough to fit your bitmap and not one that will cover up the whole display.
The paint(...) method updates the internal buffer, in order to send the content of this buffer to MAX chips, you have to call flush().
The idea behind this is to allow you to shift a few bitmaps into the display and then paint the result. You can program a few
independent routines that will update different display parts and
flush all changes at once.
Communication with MAX chips is not very fast, and sending the content of the whole display with every flush() is time-consuming. You can speed up this process by enabling double buffering (set DEOUBLE_BUFFER in Display.h to true). In this case, flush() method will send only bytes that have changed, so you can call flush()
with every loop and not worry about losing performance. The
only drawback is increased usage of RAM: we are creating 2D array that
allocates 8 bytes per each LED Kit plus a few pointers that are usually
required to maintain arrays.
2D arrays in this project have reduced memory footprint because to create a dynamic 2D array, we are creating a single continuous array with a calculated offset (see: alloc2DArray8(....) in DisplayUtil.h).
In this example, we will display a simple static bitmap with 8x8 pixels:
Here is the Arduino sketch: SimpleBitmap, now lest go over it:
First, we have to initialize the display, as we have done in the above chapter Setting things up.
Next, we have to create data that can hold our bitmap - it will have 8x2
bytes. This gives us up to 8 lines and 16 horizontal pixels. But the
size of our bitmap is 9x8 pixels (width x height), which will also be the size of the painted rectangle. It should be as small as possible so you can place another bitmap next to it.
The display will obviously only paint the rectangle given by width/height and not the whole data
array. This is normal that a data array can hold more pixels than the accrual size of our bitmap because size of the data is a multiplication o 8, and the bitmap is not necessary.
voidsetup(){util_setup();log_setup();ss=createSS();disp=newDisplay(8,3,ss);disp->setup();data=alloc2DArray8(8,2);data[0][0]=B01100001;data[0][1]=B10000000;data[1][0]=B01100001;data[1][1]=B10000000;data[2][0]=B01100001;data[2][1]=B10000000;data[3][0]=B01100001;data[3][1]=B10000000;data[4][0]=B01100001;data[4][1]=B10000000;data[5][0]=B00110011;data[5][1]=B00000000;data[6][0]=B00011110;data[6][1]=B00000000;data[7][0]=B00001100;data[7][1]=B00000000;disp->paint(27,9,9,8,data);}voidloop(){util_cycle();log_cycle();// Paint method updates only internal buffer, in order to send data to // MAX chips you have to flush display. disp->flush();delay(100000);}
Static Text
Now we will display static text, actually those are going to be two independent lines.
Here you can find an Arduino sketch containing the whole example: StaticText.
Your sketch needs a setup method, as we've already seen above (chapter: Setting things up), so we will not discuss it again.
In order to display text, you should use StaticText8x8.
The font is defined in Font8x8, each character has 8x8 pixels.
Your code could look like this one (plus initialization stuff from Setting things up):
Display*disp;ScrollingText8x8*message;constchar*textMessage;voidsetup(){util_setup();log_setup();ss=createSS();disp=newDisplay(8,3,ss);disp->setup();message=newScrollingText8x8(disp,48,50,5);message->init();textMessage="This is an example of multiple scorlling areas ;)";message->scroll(8,8,ScrollingText8x8::LOOP,textMessage);}voidloop(){util_cycle();log_cycle();message->cycle();disp->flush();}
The initialization of the display is the same as in the examples above, so it's omitted here.
In order to display scrolling text, we are using ScrollingText8x8. In setup(), we are creating an instance of this class and calling method scroll(...). This part only initializes scrolling but does not play the animation itself. In order to play the animation, you have to call cycle() and flush() in the main loop, and you must not have any additional delays there, otherwise, you might get jagged animation.
During the creation of ScrollingText8x8 we have provided the speed of animation - actually, it's a delay of 50ms per frame. Now calling cycle() in the main loop will produce animation frames according to provided delay. When the time comes, the method cycle() will update the display, and finally, method flush() will send updated content to MAX chips.
The whole implementation of ScrollingText8x8 is nonblocking and consumes CPU only when something is to be done. Internally it's using a simple State Machine.
There is one last thing: you must keep the text used for animation in the global variable to avoid deletion. It's not being copied in the scroll() to avoid memory fragmentation.
Scrolling Text Mixed
This example is similar to the one above, but this time we will display
several scrolling areas:
Here is the sketch.
This code is similar to one with one scrolling area, but this time we have a few:
voidsetup(){util_setup();log_setup();ss=createSS();disp=newDisplay(8,3,ss);disp->setup();uint8_tborderSpeed=20;textUpDown="* * * * * ";up=newScrollingText8x8(disp,64,borderSpeed,1);up->init();up->scroll(0,0,ScrollingText8x8::CONTINOUS_LOOP,textUpDown);down=newScrollingText8x8(disp,64,borderSpeed,2);down->init();down->scroll(0,16,ScrollingText8x8::CONTINOUS_LOOP,textUpDown);textLeftRight="* ";left=newScrollingText8x8(disp,8,borderSpeed,3);left->init();left->scroll(0,8,ScrollingText8x8::CONTINOUS_LOOP,textLeftRight);right=newScrollingText8x8(disp,8,borderSpeed,4);right->init();right->scroll(56,8,ScrollingText8x8::CONTINOUS_LOOP,textLeftRight);message=newScrollingText8x8(disp,48,50,5);message->init();textMessage="This is an example of multiple scrolling areas ;)";message->scroll(8,8,ScrollingText8x8::LOOP,textMessage);}voidloop(){util_cycle();log_cycle();up->cycle();down->cycle();right->cycle();message->cycle();left->cycle();disp->flush();}
We have created a few instances of ScrollingText8x8, each containing different text and positions on the display. In order to play an animation, you have to call cycle() on each instance, but you have to call only once flush(). Each call on cycle() will update its part of the display, and flush will send the changed display to MAX chips.
