Support Internet Explorer using Babel Transpilation

 

In this post we will review the steps request to support IE (Internet Explorer) in a webpack based javascript project.

First, we create a babel configuration file. It should be in the same folder as the package.json file. Notice the file includes the list of browsers that you want to support.

babel.config.json:

{
  "sourceType": "unambiguous",
  "presets": [
    [
      "@babel/preset-env",
      {
        "debug": false,
        "targets": {
          "edge": "17",
          "firefox": "60",
          "chrome": "67",
          "safari": "11.1",
          "ie": "11"
        },
        "useBuiltIns": "usage",
        "corejs": {
          "version": 3
        }
      }
    ]
  ],
  "plugins": [
    [
      "@babel/plugin-proposal-decorators",
      {
        "decoratorsBeforeExport": true
      }
    ],
    [
      "@babel/plugin-proposal-class-properties"
    ],
    [
      "@babel/transform-runtime"
    ]
  ]
}

Next, install the babel dependencies:

NPM commands:

npm install --save-dev @babel/core
npm install --save-dev @babel/plugin-proposal-class-properties
npm install --save-dev @babel/plugin-proposal-decorators
npm install --save-dev @babel/plugin-transform-runtime
npm install --save-dev @babel/preset-env
npm install --save-dev @babel/preset-typescript
npm install --save-dev @babel/runtime
npm install --save-dev babel-loader
npm install --save core-js

Now we need to update the webpack configuration to run babel, and to include the core-js library to fill all of the missing ES6 APIs (Only the relevant part of the file is shown here). In case you have non supported nodes_modules, replace them in the exclude section.

webpack.config.js:

module.exports = {
  entry: ['core-js/stable', './main/index.js'],

target: 'es5',
module: {
    rules: [
      {
        test: /\.js$/,
        exclude: /node_modules\/(?!(exclusemelib1|exclusemelib2))/,
        loader: 'babel-loader',
      },
    ],
  },

To find the ES5 non supported library, you can use the following command:

are-you-es5 CLI:

npx are-you-es5 check -a PATH_TO_YOUR_PROJECT

However, not all of the output node_modules from these commands are really used. You should really guess, which ones are really relevant. You can also run your project in IE without minification, and find out according to the errors in the console, which modules are problematic.

Final Notes

I've wrote this post after scanning many sites, and it seems that the API for handling IE support keeps changing. I hope the description in the post will be stable enough to make it until you will need it...

Some of the good resources I've used are:

• Remeez Post
• are-you-es5
• transpile node modules

Persistence Volume Allocation on a Bare Metal Kubernetes

 

When we have a kubernetes based deployment, we usually need to use persistence volumes. This is usually required for databases such as Redis and ElasticSearch, but might be required for many other services.

In a cloud based kubernetes, such as GKE, and EKS, automatically provision the persistence volumes according to the persistence volume claims that our application deployments and statefulsets create.

But in a bare metal environments, which many use as a debugging environment, persistence volume are not provisioned automatically.

Until recently I have been using hostpath-provisioner to handle the persistence volumes allocation, but once I've upgarded my kubernetes to version 1.20, the hostpath-provisioner was broken with error: "selflink was empty". I could not find the reason for the failure.

However, I did find a simple bypass using HELM, and I think I should have used it anyways instead of using the hostpath-provisioner.

The idea is to use Helm (that is widely used for kubernetes deployments) to create the persistence volumes only when hostpath is specified as the storage class for the persistence volume.

For example, a statefulset would create a persistence volume claim:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: my-statefulset
spec:
  
  
    
  volumeClaimTemplates:
    - metadata:
        name: pvc
      spec:
        accessModes: [ "ReadWriteOnce" ]
        {{- if ne .Values.storageClass "" }}
        storageClassName: "{{ .Values.storageClass }}"
        {{- if eq .Values.storageClass "hostpath" }}
        volumeName: my-pv
        {{- end }}
        {{- end }}
        resources:
          requests:
            storage: {{ .Values.storageSize }}

Notice that in case the storage class is hostpath, we ask for a specific volume name.

Next we create the persistence volume, only if the storage class is hostpath:

{{- if eq .Values.storageClass "hostpath" }}
apiVersion: v1
kind: PersistentVolume
metadata:
  name: my-pv
  labels:
    type: local
spec:
  storageClassName: hostpath
  capacity:
    storage: {{ .Values.storageSize }}
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/my-folder"
{{- end }}

That's it. 

No need of any special tricks.

Also, our helm chart support both deployment on a bare metal kubernetes, and on a cloud based kubernetes with a change of a single helm value.

Use SSL for NGINX

 

In this post we will review the steps required to configure SSL for NGINX.

Unlike many other articles, this post includes BOTH the NGINX configuration, and the keys creation steps.

This is intended for development environment, hence we will use a self signed certificate.

Create the Keys

We will create a server key and certificate.

sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout nginx.key -out nginx.crt

The NGINX Configuratoin

The following runs both HTTP and HTTPS servers.

If required, the related HTTP listener can be removed.

user  nginx;
worker_processes  10;

error_log /dev/stdout debug;
pid       /var/run/nginx.pid;

load_module modules/ngx_http_js_module.so;
load_module modules/ngx_stream_js_module.so;

events {
    worker_connections  10240;
}

http {
    include         /etc/nginx/mime.types;
    default_type    application/octet-stream;
    include         /etc/nginx/resolvers.conf;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log          on;
    sendfile            on;
    port_in_redirect    off;
    proxy_max_temp_file_size 0;
    keepalive_requests 1000000;
    keepalive_timeout 300s;

    server {
        listen       8080;
        listen       8443 ssl;
        server_name  localhost;

        ssl_certificate /etc/ssl/nginx.crt;
        ssl_certificate_key /etc/ssl/nginx.key;

        location / {
            # your configuration here
        }
    }
}

Use SSL for NodeJS

In this post we will review the steps required to configure SSL for express server on NodeJS.

Unlike many other articles, this post includes BOTH the code changes, and the keys creation steps.

This is intended for development environment, hence we will use a self signed certificate.

Create the Keys

We will create a Certificate Authority (CA) key and certificate, and then we will create a server key and certificate.

rm -rf ./ssl
mkdir ./ssl
sudo openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=" -keyout ./ssl/key.pem -out ./ssl/cert.pem
sudo openssl req -newkey rsa:2048 -new -nodes -x509 -days 3650 -subj "/C=US/ST=Denial/L=Springfield/O=Dis/CN=" -keyout ./ssl/ca.key.pem -out ./ssl/ca.pem
sudo chmod -R 777 ./ssl

The NodeJS SSL Server

The following runs both HTTP and HTTPS servers.

If required, the related HTTP listener can be removed.

const express = require('express')
const fs = require('fs')
const cors = require('cors')
const https = require('https')

const app = express()

app.use(express.static('public'))
app.use(cors())

app.get('/', (req, res) => {
  res.json({msg: 'OK'})
})

app.listen(8080)

const options = {
  key: fs.readFileSync('/etc/ssl/key.pem'),
  cert: fs.readFileSync('/etc/ssl/cert.pem'),
  ca: fs.readFileSync('/etc/ssl/ca.pem'),
}

https.createServer(options, app).listen(8443)

Example for ANTLR usage in Python

 

In this post we will review usage of the ANTLR parser in a python application.

The code is base on the ANTLR Python documentation, and on some ANTLR samples.

For this post example, we will create a simple calculator that can parse simple expressions using the following operations:

• plus
• minus
• divide
• multiple

Installation

Install ANTLR:

sudo apt install antlr4

And then install the ANTLR python library:

pip3 install antlr4-python3-runtime

Create ANTLR Grammar

The ANTLR grammar file defines our language.

We create a calc.g4 file for it:

grammar calc;

expression
   : plusExpression
   | minusExpression
   ;

plusExpression
   : priorityExpression (PLUS  priorityExpression)*
   ;

minusExpression
   : priorityExpression (MINUS  priorityExpression)*
   ;

priorityExpression
    : mulExpression
    | divExpression
    ;

mulExpression
   : atom (TIMES atom)*
   ;

divExpression
   : atom (DIV atom)*
   ;

atom
   : NUMBER
   | LPAREN expression RPAREN
   ;

NUMBER
   : ('0' .. '9') +
   ;

LPAREN
   : '('
   ;


RPAREN
   : ')'
   ;


PLUS
   : '+'
   ;


MINUS
   : '-'
   ;


TIMES
   : '*'
   ;

DIV
   : '/'
   ;

The grammar file is based on a recursive definition of expression. Notice that this will cause the ANTLR to build a tree, that we will later use to analyze the expression, and calculate the result.

In our case we must ensure to build the parsed tree according to the order of the mathematics operations, hence the expression is first spit to plus/minus expression, and only then to multiply/divide expression.

Example for tree (from http://csci201.artifice.cc/notes/antlr.html)

The Python Code

First generate the python code for parsing based on the ANTLR grammar file:

antlr4 -Dlanguage=Python3 calc.g4

This create several files that we should import in our application:

from antlr4 import *

from calcLexer import calcLexer
from calcListener import calcListener
from calcParser import calcParser

Next we create a listener. The listener inherits the generated listener and can override its methods to run some logic. In our case the logic will be run upon exit of each rule, which means it will be run after the child nodes of the current expression were already scanned.

class MyListener(calcListener):
    def __init__(self):
        self.verbose = False
        self.stack = []

    def debug(self, *argv):
        if self.verbose:
            print(*argv)

    def debug_items(self, operation, items):
        if len(items) == 1:
            self.debug(operation, items[0])
        else:
            self.debug(operation.join(map(str, items)), "=", self.stack[-1])
        self.debug("stack is {}".format(self.stack))

    def exitAtom(self, ctx: calcParser.AtomContext):
        number = ctx.NUMBER()
        if number is not None:
            value = int(str(ctx.NUMBER()))
            self.stack.append(value)
            self.debug("atom {}".format(value))

    def exitPlusExpression(self, ctx: calcParser.PlusExpressionContext):
        elements = len(ctx.PLUS()) + 1
        items = self.stack[-elements:]
        items_result = sum(items)
        self.stack = self.stack[:-elements]
        self.stack.append(items_result)
        self.debug_items("+", items)

    def exitMinusExpression(self, ctx: calcParser.MinusExpressionContext):
        elements = len(ctx.MINUS()) + 1
        items = self.stack[-elements:]
        items_result = items[0] - sum(items[1:])
        self.stack = self.stack[:-elements]
        self.stack.append(items_result)
        self.debug_items("-", items)

    def exitMulExpression(self, ctx: calcParser.MulExpressionContext):
        elements = len(ctx.TIMES()) + 1
        items = self.stack[-elements:]
        items_result = math.prod(items)
        self.stack = self.stack[:-elements]
        self.stack.append(items_result)
        self.debug_items("*", items)

    def exitDivExpression(self, ctx: calcParser.DivExpressionContext):
        elements = len(ctx.DIV()) + 1
        items = self.stack[-elements:]
        if len(items) > 1:
            items_result = items[0] / math.prod(items[1:])
        else:
            items_result = items[0]
        self.stack = self.stack[:-elements]
        self.stack.append(items_result)
        self.debug_items("/", items)

Now we can parse an expression, and run our listener:

def parse(text, verbose=False):
    input_stream = InputStream(text)
    lexer = calcLexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = calcParser(stream)
    tree = parser.expression()
    listener = MyListener()
    listener.verbose = verbose
    walker = ParseTreeWalker()
    walker.walk(listener, tree)
    return listener.stack[0]

And finally, lets run some tests:

def test(text, expected):
    print(text)
    actual = parse(text)
    print(text, "=", actual)

    if actual != expected:
        print("=== rerun in verbose ===")
        parse(text, True)
        raise Exception("expected {} but actual is {}".format(expected, actual))


test("1", 1)
test("1+2", 3)
test("3*4", 12)
test("10-8", 2)
test("10/2", 5)
test("4+2+3", 9)
test("90-10-20", 60)
test("(1)", 1)
test("(1)+(2)", 3)
test("(1+2)*(3+4)", 21)
test("(10-8)*(1+2+3)*4", 48)
test("(11-1)-(10-5)", 5)
test("(11-1)/(10-5)", 2)

Final Note

ANTLR is a great tool, that can be used in python, and in more lanuguages.

The listener API of ANTLR enables us to run any interpretation of the grammar result, and run our own logic, hence getting the max out of the parsed text.

 

Custom Redux Middleware

 

In this post we will create a custom redux middleware. 

A redux middleware listens for actions, and choose either to ignore it and let redux handle it, or handle the action by itself.

Let review the steps for using a custom redux middleware. First, add our middleware to the store default middlewares:

import {configureStore, getDefaultMiddleware} from '@reduxjs/toolkit'
import counterReducer from '../features/counter/counterSlice'

export default configureStore({
  reducer: {
    counter: counterReducer,
  },
  middleware: [
    ...getDefaultMiddleware(),
    refreshMiddleware,
  ],
})

Next, activate our middleware only for a specific type of action:

function refreshMiddleware({dispatch, getState}) {
  return next => action => {
    if (action.type === 'refresh') {
      handleAction(action, dispatch, getState)
    } else {
      return next(action)
    }
  }
}

And lastly, we want to handle our action.

In this case we will send an async request, and update redux with its result using a new type of action.

function handleAction(action, dispatch, getState) {
  fetch('https://jsonplaceholder.typicode.com/todos/1')
    .then(response => response.json())
    .then(json => {
        const state = getState()
        dispatch({
          type: 'set-user',
          payload: {
            counter: state.counter.value,
            userId: json.userId,
          },
        })
      },
    )
}

Final Note

While easy to perform, custom middleware is poorly documented in the redux documentation. I hope this short post would assist you to implement your own middlware.

That's said, make sure to use redux middleware only as a last resort, a prefer using existing mechanisms such as simple reducers, and redux-thunk.

Web Sockets Using JavaScript Frontend and GO Backend

 

In this post we will review an example of using WebSockets.

We will use a JavaScript based client running on the browser, and sending requests to a GO backend. The JavasScript and the GO backend will send messages to each other over the web socket.

The JavaScript Side

To start our JavaScript project, it would be simple to begin with a predefined react based template:

npx create-react-app demo
cd demo
npm start

Then, in App.js, add our web socket handling code:

const ws = new WebSocket('ws://localhost:8080/ws')
ws.onmessage = function (evt) {
  console.log(`got from server: ${evt.data}`)
}

ws.onopen = function () {
  console.log('WS Connected')
  setInterval(() => {
    console.log(`sending to the server`)
    ws.send('I am your client')
  }, 1000)
}

We start by initiating a web socket connection. 

    ❗ we use the ws:// prefix, for a secure connection use wss:// prefix

Then, we wait for the web socket to establish, and send a message to the server.

In addition, whenever we get a message from the server, we print it.

The GO Side

The GO backend application uses an echo server.

First, let's configure the web server to handle the requests:

package main

import (
   "fmt"
   "github.com/gorilla/websocket"
   "github.com/labstack/echo/v4"
   "net/http"
   "time"
)

func main() {
   echoServer := echo.New()

   echoServer.GET("/ws", serveWebSocket)

   err := echoServer.Start(":8080")
   if err != nil {
      panic(err)
   }
}

We use the /ws URL path. This should match the path specified on the JavaScript side.

Next, communicate over the socket:

var upgrader = websocket.Upgrader{
   CheckOrigin: func(r *http.Request) bool {
      return true
   },
}

func serveWebSocket(ctx echo.Context) error {
   ws, err := upgrader.Upgrade(ctx.Response(), ctx.Request(), nil)
   if err != nil {
      return err
   }
   defer ws.Close()

   for {
      err := ws.WriteMessage(websocket.TextMessage, []byte("Hello from the server"))
      if err != nil {
         return err
      }
      _, msg, err := ws.ReadMessage()
      if err != nil {
         return err
      }
      fmt.Printf("got from the client: %v\n", string(msg))

      time.Sleep(time.Second)
   }
   return nil
}

The socket connection is upgraded to a web socket, and then we start the conversation.

    ❗ Implement CheckOrigin to return true to allow cross origin requests

The web socket library in use is the Gorilla WebSocket.

Debugging in Google Chrome

Google Chrome supplies a great logging of the entire web socket conversation.

NGINX

In case the JavaScript code is served using an NGINX, you will need to enable the web socket establishment. See the two upgrade related headers.

location /ws {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
    proxy_buffering off;
    proxy_pass http://backend/ws;
}

Final Note

In this post we have reviewed the steps required to create a simple application that uses web sockets.

In a real life scenario, it is possible to connect the web socket messages to redux store, and handle them as if they were part of any other application messages.