Thursday, February 4, 2016

Top 10 Most Common Mistakes That Java Developers Make

I recently came across a very interesting article by a gentleman called Mikhail Selivanov describing a number of problems young developers struggle with.

Top 10 Most Common Mistakes That Java Developers Make

Even if you're an experienced developer you might find it interesting. Us pros we tend to forget what mistakes can be made. Going through them helps us understand our young colleagues better.

Have a nice day!

Monday, February 1, 2016

ESP8266 ESP-01 and LUA (NodeMCU)

Sometimes the faiths are just too kind. For example I have learned a few days ago about the ESP8266 chip. It has a complete 802.1b/g/n Wifi stack, a few general purpose I/O pins, loads of RAM, an 80 MHz 32 bit CPU and loads of stuff already in the ROM like RTOS to just name one big one.

As it usually is the case with all new hardware nice is pretty much always accompanied by some ugliness creeping around the corner. 8266 is no exception. By default it comes in 2 flavors: as a module that you talk to using AT commands (just like modems) and as NodeMCU module with LUA interpreter inside.

So let's say you bought the ESP-01 module, because it was the cheepest and you have played around with the AT commands for a while. It is really fun for the first hour or so but then with its unexpected reboots and what not programming something that communicates with it (like an Arduino) and making it resilient to all the quirks becomes just distasteful. Another reason for making the switch might be that doing all the communication directly in the chip (as opposed to waiting for user input via serial interface) is just a lot faster (20+ requests per second vs about 1!!!). Luckily enough you actually can load the LUA firmware to the ESP-01.

From this moment on if you're not a Linux user you need to start looking somewhere else because those instructions won't work for you.

First you need the NodeMCU firmware. You can do that locally but you also can (and I would strongly recommend you do that) use the cloud-based service that has been specifically designed for building it. You just enter your email, select the LUA modules you'd like to have available and presto! In a few minutes you'll be sent a link to the binaries.

Once you have it, it is time to upload the firmware to your ESP-01. To do that you'll need the esptool.py utility (just the one file!). Hooking up the board isn't all that difficult:

The diagram is a taken from https://importhack.wordpress.com/2014/11/22/how-to-use-ep8266-esp-01-as-a-sensor-web-client/

Once you've double-checked and triple-checked all the wires, made sure the voltage supplied is 3.3V and NOT 5V (I have BURNED 2 modules just because of that!) and when your module responds to commands in the terminal it is time to finally burn the LUA firmware:

./esptool.py --port /dev/ttyUSB0 write_flash 0x0000 nodemcu-*-float.bin

That will do the trick :) Now you get to learning LUA (I like the video tutorial by Derek Banas) and start hacking. Because LUA and NodeMCU are so popular there is a ton of examples on the Internet to learn from. For uploading I strongly recommend you get yourself a copy of ESPlorer which makes the whole experience a lot more approachable for mere mortals.

Happy hacking!

Friday, January 22, 2016

Intermission

Today was kind of a special day. You don't get too many of those days. We have had a company event with presentations and all. At the end of it we have had a pleasure of experiencing J.R. Hawker, a former pilot and leader of the R.A.F. Red Arrows team.

One thing that struck me down (again) was that he was a leader that actually grew out of the team he led. I find that really interesting with all the personas out there like Mark Zuckerberg, ‎Larry Page and ‎Sergey Brin, Jack Dorsey, Evan Williams, Biz Stone, and Noah Glass, Jeff Atwood and Joel Spolsky leading the business they have created. Even if you take a look at the icons of modern computer industry like Bill Gates and Steve Jobs one of the things they all have in common is they grew as a leader out of the business they created. They were down to the details, creating the first prototypes, doing all the dirty work, basically spinning of the product.

Well, I wonder what kind of leader does it take to jump in into a moving bullet-train and steer it in a direction other than a concrete wall... I'm sure it needs to be an exceptional character.

Anyways.. Thank you mr Hawker for the presentation. It was really great to see someone who speaks about something with a passion!

As a further reading I'd recommend the most popular TED talk ever, by Sir Ken Robinson. You'd find that one of the things both talks have in common is the acceptance of being wrong and using that as a tool to get better in what you do. Really inspiring!

Wednesday, January 20, 2016

Web platform performance comparison

I have developed a deeper interest in nodejs recently especially in the area of server applications. Node's HTTP module makes it extremely easy to create small and very fast applications. As a polyglot software developer I have decided to test how node compares to other frameworks. For this I have chosen, rack, Sinatra, node/http and due to my current occupation, Java servlets with Tomcat as the servlet container.

I started the benchmark with Siege and ab but have soon realized that there is something wrong with the world. As it turns out the problem was performance on the side of the request initiator. I have since switched to wrk which does a much better job all round.

Ruby

I started the test with Rack on Thin. It is the framework for Ruby so the idea was that it will be a good reference point for others.

Running 10s test @ http://localhost:9292
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     3.38ms  619.35us  16.05ms   92.73%
    Req/Sec     1.42k   200.93     1.59k    95.56%
  25527 requests in 10.02s, 3.12MB read
  Socket errors: connect 10, read 0, write 0, timeout 0
Requests/sec:   2546.72
Transfer/sec:    318.34KB

I ended up running this test about 10 times with different threading models on Thin and couldn't get it to pass the 3k rps mark

Sinatra

This is the most elegant solution of all. Sinatra is just hands down the cleanest one ever.

Performance-wise one might clearly say that under thin it performs very much OK

Running 10s test @ http://localhost:4567/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.90ms    0.91ms  13.55ms   84.49%
    Req/Sec     1.75k   120.16     1.99k    68.00%
  34765 requests in 10.01s, 7.43MB read
Requests/sec:   3474.70
Transfer/sec:    760.09KB

I mean almost 3.5k requests per second, nice API to program against... What more can we expect?

Node + http

Having established the baseline now was the time to start poking around nodejs.

Well, that is also very succinct, even though the asynchronous API might feel a bit weird at the beginning. Performance wise it was exceptionally good!

Running 10s test @ http://localhost:8080
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   627.53us    1.19ms  37.32ms   99.26%
    Req/Sec     8.95k     1.06k   18.06k    97.01%
  179056 requests in 10.10s, 22.03MB read
Requests/sec:  17729.12
Transfer/sec:      2.18MB

17.7k requests per second! Compared to Ruby it is a 5.1 times better performance!

The platform itself is single threaded so in order to make use of all the CPU power one would simply spin off a few instances on different ports and put a load balancer in front of them. Luckily there is a node module called loadbalancer which makes the whole experience quite approachable for mere mortals:

Running 10s test @ http://localhost:8080/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   658.55us    0.94ms  17.34ms   95.67%
    Req/Sec     9.55k     1.40k   19.58k    83.08%
  191007 requests in 10.10s, 23.50MB read
Requests/sec:  18912.15
Transfer/sec:      2.33MB

The setup is way more complex, there are 2 types of applications and all but the gain isn't what I would expect.

So I thought since JavaScript is slow I decided to give haproxy a go. After all it is a specialized application for balancing high-load traffic. I would expect way better performance than the JavaScript-based simplistic load balancer. And so I downloaded the latest sources, built it, configured and ran the tests.

And here are the test results

Running 10s test @ http://localhost:8090/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.91ms    0.92ms  12.88ms   93.20%
    Req/Sec     6.53k   729.78    11.21k    80.60%
  130444 requests in 10.10s, 13.06MB read
Requests/sec:  12915.95
Transfer/sec:      1.29MB

What what what??? HAProxy is 30% slower than a proxy written in JavaScript? Can this be true? I'm sure the configuration I came up with can be tuned so I'm going to call this one a draw and move on.

Node has one more option to choose from - it's the cluster. The idea is quite simply to fork the current process, bind the socket to the parent's port and let the parent distribute the load over the spawned children. It's brilliant in that it doesn't add the overhead of making additional proxy request. So it should be really fast!

As you can see it is very simple and also very expressive. If you add to it that this is actually the only file in the entire solution it starts to take a really nice shape. My computer has 4 cores so I'll be spawning 4 processes and let them process the requests in a round-robin way. Now let's take a look at the results:

Running 10s test @ http://localhost:8080/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   592.01us    2.58ms  80.36ms   97.85%
    Req/Sec    15.96k     2.56k   19.74k    87.62%
  320693 requests in 10.10s, 38.54MB read
Requests/sec:  31751.61
Transfer/sec:      3.82MB

Wow!! 31.7k requests per second! Fricken amazing performance! JavaScript's V8 engine rocks! Let's leave it at that.

Java

Now to get a sense of where Node with those 31.7k rps places itself on the landscape I decided to test Java servlets. I didn't went with spark or anything else of that sort since I wanted to compare only the most prominent solutions (or Sinatra since you just can't ignore that extremely beautiful framework).

As you can see we're doing a Maven project with just one servlet and the web.xml. Let's see the performance on that baby:

Running 10s test @ http://localhost:8080/example/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.90ms    5.34ms  85.90ms   92.74%
    Req/Sec    21.62k    12.25k   43.35k    50.50%
  430345 requests in 10.00s, 47.69MB read
Requests/sec:  43028.84
Transfer/sec:      4.77MB

Hold your horses! Yes it is faster but one needs to remember that Java needs time to get to the top performance. So I ran the test once again

Running 10s test @ http://localhost:8080/example/
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   804.67us    1.92ms  32.48ms   91.16%
    Req/Sec    31.80k     2.68k   37.12k    77.00%
  632656 requests in 10.00s, 70.10MB read
Requests/sec:  63258.42
Transfer/sec:      7.01MB

Now that is just unbelievable! 63.2k requests a second is twice the speed the fastest node solution was capable of yielding! Twice!

Post scriptum

In reality the performance of the platform doesn't really matter all that much. If you take into consideration the response times they all are below 3ms which in turn means that if you make one call to the database you already blew the performance as that is going to cost you way more than just a couple milliseconds. But it is really nice to know the characteristics and to know that performance-wise it really doesn't matter what you choose these days. The framework is going to perform on an acceptable level. Even Sinatra with the 3.5k is still fast enough to serve thousands of requests a minute which is more than enough for most corporate solutions out there.

For a much more complete comparison of many frameworks and platforms check out the techempowered site

Happy coding!

Thursday, January 7, 2016

ENC28J60 Ethernet Shield V1.0 for Arduino Nano 3.0 RJ45

Wouldn't it be great? You put a shield onto the Arduino Nano which then fits perfectly into the breadboard and then you can put together anything you'd like using just jumper wires.

p>Well, it is perfectly feasible with the ENC28J60 Ethernet Shield V1.0 for Arduino Nano 3.0 RJ45 (the so called Web Server module).

The only problem is that when you do stress it out (for example using ab or siege) then just after a few seconds and sometimes after few minutes or even after a few hours the module will just stop responding and die. Initially when I bought it I was pretty damn sure it must have some sort of manufacturing problem and so I have ordered another one. Oh boy was I crushed when I learned the new one after it was being shipped for over a month came to me with the same kind of problem... Damn!

Internet to the rescue

I started (yet again) digging into the matter using the most sophisticated tool mankind has ever created (that'd be google.com). Lo and behold I found this post where a gentleman who goes by the name of mmorcos found out that a similar issue occurred when connecting a similar module to a Duemilanove. The solution he found was to put a more powerful power source than the USB cable. I tried it using an old breadboard module and now the module is being bombarded (coming up on 11 hours) at a steady 150 requests a second without a single packet being lost! Yeah!

Alternatively you could try powering the hardware from a USB 3.0 which gives 900mAh over 500mAh from USB 2.0

Happy hacking!

Monday, January 4, 2016

Calculating pressure reduced to sea level (JavaScript)

Problem

The pressure as it is read from the BMP180 sensor is an absolute pressure. To be able to compare it with other sensors it is necessary to have a common level above the sea. For example the value of pressure as it is given in your weather forecast is reduced to the sea level. How do we do that in JavaScript?

The solution

The following calculateSeaLevelPressure() function is based on this blog post (in polish!) and validated using a PHP implementation

Happy coding!

Saturday, January 2, 2016

webled - a web-enabled light emitting diode

There are different styles of hello-world applications. Some just spit out to whatever output is provided the Hello, world! message, some read a digital input and react to it and some just toggle an LED. This one is the of the latter kind, using Raspberry Pi and a green LED.

Disclaimer

I am not responsible for any damage you might cause to your hardware. Use at your own risk.

The setup

It's really straight forward to wire everything up. You start by connecting PIN01 through a resistor (let's say 100Ohm) to anode (the longer lead) and then the cathode (the shorter lead) to PIN40. That's it.

Now we need a web application to work with that new high-end device of ours. We start by installing Flask - a Sinatra-inspired framework for Python.

pi@raspberrypi:~ $ sudo pip install flask

Code

That will take a while (~ a minute) and after it is done we finally get to do some coding.

The code is pretty straightforward. First we import flask and jsonify for later use, then we create a Flask application instance, map the root to hello function which when executed toggles the state and stores the new value to PIN40. In the initialization block first we setup the GPIO.BOARD mode to be able to refer to the pins as they are on the board (as opposed to the GPIO.BCM mode which refers to the CPU pins), set the PIN40 mode to GPIO.OUTPUT to be able to switch our device on and off, and initialize the initial state of that pin. When the application finishes we clean things up so that others can use the library.

The host = '0.0.0.0' makes it possible to access the application from other hosts as opposed to host = '127.0.0.1' (which is the default) which would only allow access from the board.

Test drive

To test drive it open up a web browser and navigate to the http://localhost:5000 URL and observe the LED to light up. When you refresh the page it will go dark again. And so on and so forth. Such is the purpose of a web-enabled LED (a webled)

Happy hacking!