Principles of software architecture

Hello folks! Recently, I've been doing some consulting for an e-commerce site. During the review it struck me that there are no ground rules there (or anywhere else for that matter) that would guide what the architecture should look like. So I came up with some points myself:

Principles of architecture

The following describes the principles of the desired state of architecture. Architecture is being viewed in this case not only as the system of interconnected components but as a complete framework for developers, testers, devops and business to know their way around the system.

Ease of development

The following principles will allow for streamlined and effective use of development resources:

• the technologies being used should be widely adopted to allow for easy sourcing of new resources (developers, designers, testers)
• both backend and frontend tooling should allow for quick turnaround between changing the code and observing the result
• manual work to get the changes to appear in local development environment should be eliminated
• all projects should have a unified API for starting the development environment

Ease of deployment

The following principles will allow for uninterrupted delivery of individual changes:

• deployment of changes should be automated which includes automated validation of changes as well as pubishing chnaged to production

Extensibility

The following principles will allow for continuous development of new features:

• the system should obey the law of simple design which dictates that finding the relevant part of the code should be obvious and easy
• the system should obey the law of extensibility which dictates that changing/adding/removing functionality should be possible without changing parts of the system that are not directly connected to the part being changed/added/removed (derived from the Open/Closed principle)

Maintainability

The following principles will allow for continuous improvements of existing features:

• the system should be and remain maintainable. This means that both development of new functionalities as well as bugfixes should be as easy and as universaly applicable as possible. For instance, the developer shouldn't worry where to get the data from to reproduce reported errors and how to connect the parts being developed

High availability

The following principles will allow to increase the presence of your app on the market:

• the system should be able to loose access to parts of it without breaking the user experience. That applies not only to services serving the data (such as the CMS) but also to services serving static resources and custom APIs.

Performance

The following principles will allow to increase the presence of your app on the market:

• the system should be performant enough to consistently serve every page update under 2 seconds. This leaves 25% margin for variation in delivery time due to connection speed

Security

The following principles will allow to eliminate any potential risks involving unauthorized use of the system thus eliminating losses:

• the system should not allow for unauthorized users to perform actions they would normally be allowed to accomplish. That includes both accessing frontend pages as well as backend services. Maintaining access to elements of the system should be easy and performant and shouldn't overshadow the essential functionality

Accounting

The following principles will allow to reason about the undesired behavior:

• the system should employ globally tracable logging facilities

So there you go - a few rules to guide you when you need to decide if what you're doing is good or bad.

The point of this list is not only to show what is important but also to prioritize which parts of it are more important than others so you go top to bottom. If your app isn't a critical one then performance and high availability isn't all that important. If it's being used by just 2 people in the entire company and the app is basically a glorified spreadsheet then security and accounting might not be a concern for you. Regardless of the project size its ease of development, maintainability and extensibility are always key points of any architecture.

Have fun!

There is no such thing as free beer - RC link comparison

Controlling a model in the air is not all that difficult. There are analog systems that use PWM modulation that we all know from the old ages where RC hobby was only available for the very few ones with deep pockets and then there are digital systems like DSM2/DSMX, ACCST, ACCESS, CRSF, ELRS and probably a dosen more that rely on some kind of analog signal modulation (like FLRC, LoRa or others) to send digital information.

You might have heard about the exceptional results of ExpressLRS system that Wezly got from his long range tests. I mean, going 35km out on 100mW output power is seriously a lot. Even the mighty Ghost from ImmersionRC or Tracer from Team Blacksheep struggled to get past the 30km mark in the real world tests. So what sets ExpressLRS apart? Are those OpenSource guys just better than commercial companies? Is this the end of the world as we know it?

Well, as the title of this post says, there's no such thing as free beer. Nor is there RF performance that comes for free. Here's the secret behind ExpressLRS exceptional performance.

All other links, besides ExpressLRS are called by the number of channels they support. For example Crossfire (both 900MHz and 2.4GHz) support up to 12 channels. Sure enough there is prioritization of which channels are being sent most often (that'd be the first 4 channels where control inputs are being transmitted) but all channels are being sent with full resolution. If you come to thing about it that's a huge waste of bandwidth if you only want to send binary (on/off) information. That can be encoded on just one bit instead of 10 bits which is the usual resolution of channels.

This is exactly what ExpressLRS does - it sends 4 full resolution channels at an extremely high packet rate (500Hz!) but the rest is sent as 1 byte (8 bits) and only conveys if a switch is on or off. That's the default mode for ExpressLRS and it works great with software such as Betaflight or INAV and for simple setups such as a racing quad or a small flying wing.

There is a second "mode" that you can select when building the firmware that is called "Hybrid switches" in which the 5th channel is a binary switch that is sent with every packet (and that shall be used for controlling arming the quad),then there are 6 channels that can take up to 6 positions (3 bits) and finally there's the 12th channel which has a 4 bit resolution which amounts to 16 different values being sent.

Yes, you read this right: there are only 4 full resolution, high performance channels in ExpressLRS. This is how the link can have much much much smaller packets than any other system. But that comes at a cost. If you'd like to control, let's say, a gimbal in 2 or 3 axes you're not going to be able to do that with ExpressLRS. It just doesn't have the technical capabilities at the core of the solution that would allow you to do that.

Then there is the telemetry thing...

Telemetry is data being sent from the receiver back to the radio. It can include anything from link statistics, battery voltage to GPS position, altitude, attitude of the craft and many many more. That holds true for everyone - besides ExpressLRS. Sure, the basics, such as the GPS position and link stats are being sent back but that is about it. This, however, further limits the amount of data that needs to be sent back and forth which in turn means ExpressLRS can send less data in the aloted time frame and they can do it a lot slower thus extending the range.

So we know what are the limitation. Who would then use such a crippled RC system? Is it good for anything?

Let's examine the requirements of a typical racing pilot. First, the faster the response from moving the sticks the better. There is no limit to it. Faster is better. But does a racing pilot need telemetry? No. does one need a gimbal on the quad that goes from zero to 100 in a second? Hello no! What is needed is a reliable channel for arming/disarming and 4 very fast, very accurate channels to control the vehicle. That's it.

That's 1:0 for ExpressLRS vs everyone else.

Let's see what a typical long-range pilot requires to operate a vehicle at 50km+. 4 control channels - check. A way to arm/disarm - check. A way to switch between 4 modes (launch, air, 3D cruise, RTL) - check (although it requires a bit of fiddling around to get it to work with the default setup with binary switches - in the hybrid mode it is quite simple though). And frankly speaking 500Hz update rate is just not that useful. The thing that is usually happening during the flight is that the plane flys itself and the pilot only enters minor course corrections to point the plane where it should go. So even if the packet rate is at about 5Hz it will still be way more than enough.

But if one would like to have a camera that moves around to be able to see if there is anything coming in at the aircraft from any direction, then the 16 position AUX8 is just not good enough.

That'd be 1.5:0.5 for ExpressLRS vs everyone else.

Now let's go ahead and see what the big boys need. Imagine you're flying something like a hexacopter, with a gimbal and a lot of other accessories. You're most probably going to be doing that with the help of some ground station to better see where you are. For that you need telemetry and a control link that can not only send analog stick positions but also commands to the aircraft (most probably using the two-way MAVLink protocol). ExpressLRS is just not capable of any of that.

That'd be 1.5:1.5.

So it all depends on what you need vs what you can live without. If, for example, you're flying a tiny whoop and just want to have decent range but not a lot of cables and antennas - ELRS has you covered. If you're racing or freestyling and need the fastest link possible - again ELRS is the way to go. The greay area starts with piloting anything that can be a bit more sophisticated than just doing crazy things around the tree. Planes tend to have much more functionality that sometimes needs more control. This is where ExpressLRS starts to fade and other, full-fledged control links come into play.

Hasura - querying fields conditionally

As it turns out GraphQL is great :) I know, I know,... it's an old story but with Hasura sitting on top of PostgreSQL it is a whole different world to explore.

For example, there might be times where you have some filter that might say Select variant: "A", "B", "C", "any". Previously it was kinda obvious, that you'd say the values were [ 'A', 'B', 'C', null ] which resulted in collapsing the expression to an empty object but with the advent of Hasura 2.0 this feature was removed. Are we left with nothing? Hell no!

Conditional expressions

So let's say you have a String field that you want to either query by a value or skip in the WHERE clausule completely. How would you do that if null is not traversed to {}?

First, let's make an example and then I'll exaplain everything in details.

query GetItems($field: String_comparison_exp!) {
  items({
    where: {
      field: $field
    }
  }) {
    id
    field
  }
}

And the accompanying variables section to go along with it:

 { field: value ? { _eq: value } : {} }

So what happened here? First, as you can see we're not passing the value of $field anymore. Instead we're passing a String_comparison_exp. This is a way of passing the actual expression in variables. Previously, in Hasura <2.0 when a value was null the actual epxpression evaluated to an empty object ({}) which in turn evaluates to TRUE in the SQL being generated. That last part still holds true, but null is no longer collapsed in Hasura 2.0. Instead you need to do that yourself. There was a ton of issues with people deleting whole content of tables and so they removed this particular functionality.

Remark: if your field is not of type String there will be an error telling you what type of expression_exp you need to use. Just read the effin error message :D.

Summary

So now you know how to manually do the collapsing and you're the boss of GraphQL again!

Happy coding!

Classes and inheritance with Vue CLI-generated Cypress tests

This is going to be a quick one but took forever to figure out. Brace yourself!

The setup

You have a project that is managed by vue-cli and you have end to end tests created with Cypress (all stock!). Now you want to move it to the next level and use wrappers as described in this tutorial.

The problem

When you try to create classes in separate modules the strangest thing happens:

import _typeof from "../../helpers/esm/typeof";
^
ParseError: 'import' and 'export' may appear only with 'sourceType: module'

This is easy to fix. Just install @cypress/webpack-preprocessor package and follow the instructions in tests/e2e/plugins/index.js - it's all there.

But the moment you'll try to use them... all of the sudden no tests are found.

The Solution

The actual issue here is transpilation and yes, you've guessed it - it has to do with Internet Explorer support. The fix is super easy.

Go to .browserslistrc and add the following line at the end of the file:

not ie > 0

And live is good again :) I so hope the masacre called Internet Explorer will be over really soon!

Happy coding!

ESP3D on SKR 1.4/1.4 Turbo with ESP01S

So you have the board. It's new, shiny and it has this strange looking 8 pin port labeled WiFi? Seriously?! WiFi on that board?

Yes! And it is quite easy to set it up! Curious? Let's begin!

Installation of required tools

For obvious reasons I'll tell you how to do it on Linux. It may work on MacOS too and I doubt it'll work on Windows. Linux is my go-to system and I don't use anything else.

Installing PlatformIO

First you need Platformio commandline tools installed. You can find the instruction on PlatformIO website here.

I've had to symlink the pio command to ~/bin/pio to have it available without specifying the full path. Your milage might vary.

Clone the repository

This is quite easy if you already have Git installed. If you don't have it check out this site for further information. Once Git is on your system clone the repository by issuing the following command:

$ git clone https://github.com/luc-github/ESP3D

Building

This is the coolest part :) There are a few commands that will get you to 100% in no time thanks to PlatformIO:

Erasing EEPROM:
$ pio run -e esp01s_160mhz -t erase

Uploading the firmware:
$ pio run -e esp01s_160mhz -t upload

Uploading files to the SPIFSS
$ pio run -e esp01s_160mhz -t uploadfs

Configuration of ESP3D

Configuring the module is pretty straightforward and described really well on this page so I won't duplicate the content. Clicking a link is easy enough :D

Configuration of Marlin firmware

Now this used to be a royal pain in the neck requiring you to modify framework code which was plain and simple ugly. Luckily enough this is now made super simple!

Find a place in Configuration.h that has the following line:

/**
 * Select a secondary serial port on the board to use for...
 * :[-1, 0, 1, 2, 3, 4, 5, 6, 7]
 */
//#define SERIAL_PORT_2 -1

and change it to look like this:

/**
 * Select a secondary serial port on the board to use for...
 * :[-1, 0, 1, 2, 3, 4, 5, 6, 7]
 */
#define SERIAL_PORT_2 3
#define LPC_PINCFG_UART3_P4_28

The #define LPC_PINCFG_UART3_P4_28 is the newest addition to the LPC1769 core and allows to use the correct pins for the second serial port so that the module can properly communicate with the board.

Compiling Marlin

Now that we have PlatformIO installed we can build Marlin from command line which I think is super cool and so easy! Switch to the root of the Marlin repository and issue the following command:

$ pio run -e LPC1769

And a few moments later... that's it! You have the latest firmware in .pio/build/LPC1769/firmware.bin

Have fun!

I solved the biggest problem I've had recently

One of the biggest headaches I've had recently was to get a software NLE (non linear video editor) to do friggen frame freezes. I edit videos in such a way that I can explain what I do on the screen which, obviously, sometimes requires to pause the video feed and let the audio do its work.

Up until yesterday I've been using Blender's F12 (or as you'll know it Render Frame) feature, then clicking "k" for cut and inserting the image in the place where I needed to do the freeze frame. What a colossal waste of time! And I've had that feeling for over two months but couldn't find a solution for it.

So I kept on doing it and I kept on searching for alternative software that would allow me to do what I need....

The solution

Sometimes it is better to look at the tools you already have at hand and learn them in more depth than just start looking for alternatives. This was the case Today with me and Blender. Blender is a fantastic NLE and I dare to say it satisfies now all the needs I'll ever ever will have. Especially now that I have learned the Shift-K (hard-cut) feature.

So the thing is there's a difference between a soft-cut (k) and a hard cut (shift-k). With soft-cut your clip will extend beyond the last frame when you extend it on the timeline. With hard cut it will essentially repeat the last frame which was what I was looking for in Blender for over A FRIGGEN MONTH!!!

God! I wish someone would have written an article about freezing frames in Blender....

In the mean time... keep on having fun!

Building a MIDI console with Arduino

I have recently been overwhelmed with what can be done with Audio on Linux. The DAWs, like Ardour, are not only free but also so extremely powerful you can actually do lots with it. I don't know if it would stand against the big guys but knowing there's no limitation for tracks, buses and the use of plugins it seems like the sound editing capabilities are only limited by the CPU power. Or are they?

Hardware to be made

In point of fact I am not a professional. I did take some piano lessons (my mum was a piano teacher and I was forced to follow in her footsteps for 5 years that I would rather forget) but beyond that and a few years of learning the theory of music I have no idea what I have set out myself to do. So what is it exactly that I'd like to do?

Well, it is not all that difficult. I think it is far more enjoyable to tweak all the knobs in a DAW if you have... welll.. knobs :D Moving them all with a mouse is just such a pain in the ass. So I have decided to make some hardware that will allow me to control the DAW with physical knobs.

Following are some basic ideas that I have that will allow me to complete my mission:

• Every mixer console has a slider for volume - I need one of those
• Every mixer console has a number of knobs that can be turned from left to right that will then change some parameter in the DAW - configurable. I need at least 8 if not 12 per channel
• I need a set of buttons that will control solo, mute and record per channel. I don't feel like pressing a key combination to achieve any of those
• I need a section that will have a master volume control as well as a set of knobs to control the output parameters. At least 8-10 for that purpose.

That brings me to a whopping 13 potentiometers per channel plus 3-5 push buttons and 11 for the master control plus some 10+ push buttons. No microcontroller can handle that amount of analog inputs alone. This is why I came up with the idea of modular design that will help me accomplish my goal (some time in the unforeseeable future :D).

The grand design

I don't want to be limited by the number of inputs at any stage. That'd be ridiculous. So instead of making an all-in-one design I am going for a modular, expand-as-you-go design that will allow me to basically define two types of modules:

• A channel module that will pretty much be a device on its own, handling all the analog and digital inputs and being an I2C slave that will allow a master to read the state of all the knobs at once
• A master module capable of recognizing all the connected slaves and sending their values over MIDI as they change

Each module will run its own ATMega382p (Arduino) with the necessary I2C hardware (like additional ADCs over I2C or I/O expanders if I choose to go with encoders rather than potentiometers). That way every module will be a slave and the master can ready the values of each control at a time of its choosing leaving the combining and transposition to MIDI commands to the master controller. Cool, ain't it?

Let the fun begin

So here I am, 3:40am, with that crazy idea in mind, putting it all on the blog. I really would like to make a big console for 32 channels but I think I'll start with 1 or 2 channels and the master to validate if the design scales.

Have fun! I know I will!