Using kind for Local Development

 

Overview

In this post we will review a method to use kind for local development, for example on the personal laptop. Kind stands for Kubernetes IN Docker, and uses docker containers to simulate kubernetes nodes. It is a great tool to replace bare metal kubernetes,which until recently was relatively easy to install, but then with the CRI changes in kubernetes got very complicated.

The steps below explain how to install kind cluster with NGINX based ingress. Then we show how to simplify the communication with the services deployed on the kubernetes cluster.

Install The Kind Binary

Kind is a standalone binary, we just download it:

curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64
chmod +x ./kind
sudo mv ./kind /usr/local/bin/kind

Create Kind Based Kubernetes Cluster

To install a kubernetes cluster, we use the following:

cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
  kubeadmConfigPatches:
  - |
    kind: InitConfiguration
    nodeRegistration:
      kubeletExtraArgs:
        node-labels: "ingress-ready=true"
  extraPortMappings:
  - containerPort: 80
    hostPort: 80
    protocol: TCP
  - containerPort: 443
    hostPort: 443
    protocol: TCP
EOF

This enables binding of ports 80 and 443 on the local machine, which will later be used for communication from the local development machine to the services running on the kubernetes cluster.

Deploy NGINX Ingress

Now we deploy NGINX based ingress, which is the most popular kubernetes ingress.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/main/deploy/static/provider/kind/deploy.yaml

sleep 5

kubectl wait --namespace ingress-nginx \
  --for=condition=ready pod \
  --selector=app.kubernetes.io/component=controller \
  --timeout=90s

Load Docker Images

In most cases we build the application docker images locally on the development machine, and hence to make these available for the kind based kubernetes we need to load them, for example:

kind load docker-image ${DockerTag}

This command should be integrated as part of the docker images build scripts on the local machine.

List Loaded Docker Images

While this is not a part of the regular build and test process, I've found it very useful command - to view the images that are already loaded to the kind:

docker exec -it $(kind get clusters | head -1)-control-plane crictl images

Communication With The Services

Great! We've deployed our services on the kubernetes cluster, and everything works, but hey? wait... How do we access these services? The best method would be a combination of local development kind dedicated ingress with hosts file update.

Let assume we have kubernetes services named my-service-1 and my-service-2. First we create an ingress to route the incoming requests to the services. Notice that this ingress should be deployed only as part of the development environment, and not on the production. This could be easily accomplish using helm flags. The ingress would be:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-kind
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  rules:
    - host: my-service-1
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: my-service-1
                port:
                  number: 80
    - host: my-service-2
      http:
        paths:
          - pathType: ImplementationSpecific
            backend:
              service:
                name: my-service-2
                port:
                  number: 80

Notice that we use the host name to select the target service to access. But how do we get the my-service-1 and my-service-2 to reach the kind based kubernetes ingress? Simple - we update the hosts file:

127.0.0.1 my-service-1
127.0.0.1 my-service-2

By setting the IP for these services to the localhost, we get to the binding of the cluster that we've previously configured as part of the cluster creation.

Final Note

Kind is a great and easy tool for local development on a kubernetes cluster, but there are some downsides. First, we need to wait for the docker images to be loaded into the kind cluster. This might last ~20 seconds, and while it is relatively short time, it is a bit annoying. Second, using ingress we can route only to HTTP/S based services. TCP based services (as Redis) cannot be addressed using this method.

React-Redux Application Coding Guidelines

 

In this post we will list some of the coding best practices for a react/redux application development. These thumb rules prevents a great pain during the actual application initial development, and a greater pain during the later stages of the application maintenance. 

create-react-app

Use the create-react-app and redux template to initialize your new application. See example in the post: Create Redux App with ASync Ajax Calls, which also includes a requests wrapper, and error handling.

When using create-react-app, the application build is super simplified. This reduces the pain involved in a JavaScript application build, and it is a real pain, trust me. In some cases, an issue that you encounter might require your to eject. Trying avoiding this as long as you can, even at a price of ugly bypass, as the alternative is for sure uglier.

Dockerize

Use two stages docker build to shorted the application build time. An example for a docker file is:

FROM node:16.3 as builder

WORKDIR /app
COPY src/package.json ./
COPY src/package-lock.json ./
RUN npm install

COPY src/ ./
RUN npm run build


FROM nginx:1.21.5
COPY --from=builder /app/build /gui
CMD ["nginx", "-g", "daemon off;"]

Flat Components

Keep all components as a flat folders list under the src folder.

This might seems counter intuitive at first sight, but in later stage of the application maintenance, when you often refactor components due to requirements changes, you are saved from trying to understand where to import your component from. 

Should I use:

import '../componentA/component.js'

Or:

import '../componentA/componentB/component.js'

Or:

import '../../componentA/componentB/component.js'

In addition, finding you component is much simpler when the entire list is flat.

Component Files

Each component folder should contain at the most 3 files:

• component.js
• component.module.css
• slice.js

For example, a super button component will have the following directories and files:

~/src/super-component/component.js

~/src/super-component/component.module.css

~/src/super-component/slice.js

This is a good standard, that all component look the same, and it is also a great time saver for refactors.

component.js Visualization

Use only visualization related code in the component. Any logic related code should be moved to the slice.js. 

For example, displaying a list of items that includes a search by item name text box (assuming the search is client side). The component.js should never do the filter action. The slice.js should handle the filtering and produce a new filter-result list that will be used in the component.js.

component.js Indentation

Handling GUI and css can be a real pain sometimes, hence we should strive to keep the component.js simple. Do not overload it with multiple levels of components, but instead keep only a flat list of items in the result of the component.

For example, this simple table component:

return (
  <div className={styles.root}>
    <div>My Table</div>

    <div>
      <div className={styles.tableHead}>
        <div className={styles.tableHeadColumn}>
          Header1
        </div>
        <div className={styles.tableHeadColumn}>
          Header2
        </div>
      </div>
      <div className={styles.tableRow}>
        <div className={styles.tableValue}>
          Value 1
        </div>
        <div className={styles.tableValue}>
          Value 2
        </div>
      </div>
    </div>
  </div>
)

is terrible!

It should be a flat list of items:

return (
  <div className={styles.root}>
    <div>My Table</div>
    <TableHeader/>
    <TableRows/>
  </div>
)

So we should break the complex table component into multiple small components.

Flex

CSS flex is a great method to position and stretch the components, but we should keep it neat and clean to make stuff work. When using flex, make sure to avoid a case when a parent flex CSS directive is handled in one component, and affects or combines with a flex CSS directive in another component.

The following guideline usually applies to flex usage:

return (
  <div className={styles.parent}>
    <div className={styles.nonStretchedChild}>
      <ChildComponent1/>
    </div>
    <div className={styles.stretchedChild}>
      <ChildComponent2/>
    </div>
  </div>
)

and the styles are:

.parent {
    width: 100%;
    display: flex;
    flex-direction: row;
}

.nonStretchedChild {
    width: 100px
}

.stretchedChild {
    flex: 1
}

slice.js Scope

Keep the slice.js small as possible, while it handles only the related component logic. When we have a slice which is very large, that's usually a sign that you need to break the component to multiple child components.

Thunk

When we split our logic to multiple slices, we probably have some (but not many) actions that require access to a state which is distributed among multiple slices. To access multiple slices, we can use the Thunk API. We can also use thunk API for async request/response handling for example:

export const graphClick = createAsyncThunk(
  'graph/clickGraph',
  async (input, thunkApi) => {
    const {graphId} = input
    const state = thunkApi.getState()
    const lastXLocation = state.graph.lastXLocation[graphId]

    const body = {
      x: lastXLocation,
    }

    thunkApi.dispatch(setLoading(true))
    const {ok, response} = await sendRequest('/get-my-data', body)
    thunkApi.dispatch(setLoading(false))

    if (!ok) {
      thunkApi.dispatch(addNotification(true, response))
      return
    }

    thunkApi.dispatch(setBuckets(response))
  },
)

Final Note

In this document we have listed some coding best practices. Looking on this list, a first impression might be that this is a "too much overhead" for an application, but trust me, it is not. Once starting to use these guidelines, coding, maintenance, refactoring, and bug fixes are all much simpler, and coding without these guidelines looks like a huge mess.

Create Excel File in Go

 

In the following post we will wrap Excel file creation in GO.

The excel wraps the Excelize library, and simplify the usage for applications, that need a simple excel. This is not suitable for complex excel sheets.

Upon creation of the wrapper, we create a new excel file, with some default styles.

package excel2

import (
 "fmt"
 "github.com/xuri/excelize/v2"
)

const sheetName = "Sheet1"

type Excel struct {
 excelFile     *excelize.File
 currentRow    int
 currentColumn int
 styleBold     int
 styleRed      int
 styleOrange   int
 styleNormal   int
}

func ProduceExcel() *Excel {
 excelFile := excelize.NewFile()

 const fontSize = 10
 styleBold, err := excelFile.NewStyle(&excelize.Style{
  Font: &excelize.Font{
   Bold:   true,
   Family: "Times New Roman",
   Size:   fontSize,
   Color:  "#000000",
  },
 })
 if err != nil {
  panic(err)
 }

 styleRed, err := excelFile.NewStyle(&excelize.Style{
  Font: &excelize.Font{
   Bold:   true,
   Family: "Times New Roman",
   Size:   fontSize,
   Color:  "#FF0000",
  },
 })
 if err != nil {
  panic(err)
 }

 styleOrange, err := excelFile.NewStyle(&excelize.Style{
  Font: &excelize.Font{
   Bold:   false,
   Family: "Times New Roman",
   Size:   fontSize,
   Color:  "#FFA500",
  },
 })
 if err != nil {
  panic(err)
 }

 styleNormal, err := excelFile.NewStyle(&excelize.Style{
  Font: &excelize.Font{
   Bold:   false,
   Family: "Times New Roman",
   Size:   fontSize,
   Color:  "#000000",
  },
 })
 if err != nil {
  panic(err)
 }

 err = excelFile.SetColWidth(sheetName, "A", "Z", 30)
 if err != nil {
  panic(err)
 }

 return &Excel{
  excelFile:     excelFile,
  currentRow:    1,
  currentColumn: -1,
  styleBold:     styleBold,
  styleRed:      styleRed,
  styleOrange:   styleOrange,
  styleNormal:   styleNormal,
 }
}

Next we handle add of data to the excel:

func (e *Excel) MoveToNextRow() {
 e.currentRow++
 e.currentColumn = -1
}

func (e *Excel) AddNextCell(text string) {
 e.currentColumn++

 axis := e.getCurrentCellAxis()
 err := e.excelFile.SetCellValue(sheetName, axis, text)
 if err != nil {
  panic(err)
 }
}

func (e *Excel) getCurrentCellAxis() string {
 columnChar := string(rune(int('A') + e.currentColumn))
 axis := fmt.Sprintf("%v%v", columnChar, e.currentRow)
 return axis
}

And provide a simple style wrapper:

func (e *Excel) SetCellStyle(bold bool, red bool, orange bool) {
 axis := e.getCurrentCellAxis()
 if bold {
  err := e.excelFile.SetCellStyle(sheetName, axis, axis, e.styleBold)
  if err != nil {
   panic(err)
  }
 } else if red {
  err := e.excelFile.SetCellStyle(sheetName, axis, axis, e.styleRed)
  if err != nil {
   panic(err)
  }
 } else if orange {
  err := e.excelFile.SetCellStyle(sheetName, axis, axis, e.styleOrange)
  if err != nil {
   panic(err)
  }
 } else {
  err := e.excelFile.SetCellStyle(sheetName, axis, axis, e.styleNormal)
  if err != nil {
   panic(err)
  }
 }
}

Last, we can save the file:

func (e *Excel) SaveAs(filePath string) {
 err := e.excelFile.SaveAs(filePath)
 if err != nil {
  panic(err)
 }
}

Create and Download Excel File in JavaScript

 

In this post we'll display a simple method to create Excel file from an HTML table, and download it.

First we create an HTML table based on our data:

function getHtmlTable() {
  const data = [
    ['Name', 'Age', 'Height'],
    ['Alice', 56, 180],
    ['Bob', 47, 176]
  ]
  const rows = data.map(row => {
    const columns = row.map(column => {
      return `<td>${column}</td>`
    })
    return `<tr>${columns.join('')}</tr>`
  })
  return `<table>${rows.join('')}</table>`
}

Next we create a download as an Excel file:

function exportToExcel() {
  const htmlTable = getHtmlTable()
  window.open('data:application/vnd.ms-excel,' + encodeURIComponent(htmlTable))
}

Final Note

Notice that while this method is extremely simple, we cannot have control over the excel file styling, and the downloaded file name.

 

Getting AWS Batch Logs in Go

 

In the previous post AWS Batch in Go, we've started AWS batches, and tracked their execution, waiting for the completion. Once an AWS batch had failed, why won't we simplify the problem investigation, and fetch the log?

To get a job log, we use first AWS batch API to get the job stream name:

input := batch.DescribeJobsInput{Jobs: []*string{aws.String(jobId)}}
output, err := awsBatch.DescribeJobs(&input)
if err != nil {
 panic(err)
}

streamName := output.Jobs[0].Container.LogStreamName

Then we use the AWS cloud watch API to get the events, while limit to get only tail of 100 events.

events, err := awsCloudWatchLog.GetLogEvents(&cloudwatchlogs.GetLogEventsInput{
 Limit:         aws.Int64(100),
 LogGroupName:  aws.String("/aws/batch/job"),
 LogStreamName: streamName,
})

The full function is:

func GetJobLogs(jobId string) string {
 awsSession, err := session.NewSession()
 if err != nil {
  panic(err)
 }

 awsBatch := batch.New(awsSession)

 input := batch.DescribeJobsInput{Jobs: []*string{aws.String(jobId)}}
 output, err := awsBatch.DescribeJobs(&input)
 if err != nil {
  panic(err)
 }

 streamName := output.Jobs[0].Container.LogStreamName

 awsCloudWatchLog := cloudwatchlogs.New(awsSession)
 events, err := awsCloudWatchLog.GetLogEvents(&cloudwatchlogs.GetLogEventsInput{
  Limit:         aws.Int64(100),
  LogGroupName:  aws.String("/aws/batch/job"),
  LogStreamName: streamName,
 })

 var lines []string
 for _, event := range events.Events {
  eventTime := time.UnixMilli(*event.Timestamp)
  line := fmt.Sprintf("%v %v", times.NiceTime(&eventTime), *event.Message)
  lines = append(lines, line)
 }

 return strings.Join(lines, "\n")
}