Using MongoDB

In this post we review the basic steps of start using an existing MongoDB. See this post for installing a MongoDB on a kubernetes cluster. 

To add a new document in a a new collection in a new database, we need to perform the following actions (see examples for commands below):

1. Use the new db
2. Create new user in the new db
3. Grant permissions to the new user in the new db
4. Switch to the new user
5. Add thew new document

Databases Commands

• use db1
  Set a database for the next operations
  The database is automatically created when we store data in it
  
  
• show dbs
  Show list of databases

Users Commands

• db.runCommand({connectionStatus : 1})
  Display current user information
  
  
• db.createUser({ user: 'alon', pwd: 'alon', roles: [] })
  Create a new user
  
  
• db.grantRolesToUser('alon', [{ role: 'readWrite', db: 'db1' }])
  Add permissions to user
  
  
• db.auth('alon','alon')
  Switch to a user in the current database

Documents Commands

• db.collection1.insertOne({name:'John', Age:45})
  Insert new document
  
  
• db.collection1.find()
  Get all documents from the collection
  
  
• db.collection1.find({'Age':45})
  Query for specific documents
  
  
• db.collection1.insertMany([{'Name':"Foo"},{"Name":"Bar"}])
  Insert multiple documents

Indexes Command

• db.collection1.createIndex({'name':1})
  Create new index

Go Implementation for IP range to CIDRs

 

In this post we will review creation of CIDRs from IP range. This was required in my case as we've received input as IP ranges, but the processing application had built a trie representation of the IPs using CIDRs.

We've started with a simple conversion functions:

func IntToIp(intIp uint32) net.IP {
 ip := make(net.IP, 4)
 binary.BigEndian.PutUint32(ip, intIp)
 return ip
}

func ParseIpv4(ipv4 string) net.IP {
 ip := net.ParseIP(ipv4)
 return ip[12:]
}

func IpToInt(ip net.IP) uint32 {
 return binary.BigEndian.Uint32(ip)
}

And the implemented a naive CIDR creation function:

func MakeCidr(
 fromIp net.IP,
 toIp net.IP,
) net.IPNet {
 fromIpInt := IpToInt(fromIp)
 toIpInt := IpToInt(toIp)

 ones := 32
 for diff := toIpInt - fromIpInt; diff > 0; diff = diff / 2 {
  ones--
 }

 mask := net.CIDRMask(ones, 32)
 maskInt := IpToInt(net.IP(mask))
 networkAddressInt := fromIpInt & maskInt
 networkAddressEndInt := networkAddressInt | (^maskInt)
 networkAddress := IntToIp(networkAddressInt)

 network := net.IPNet{
  IP:   networkAddress,
  Mask: mask,
 }
 if networkAddressInt != fromIpInt || networkAddressEndInt != toIpInt {
  errsimple.RaiseIfError(fmt.Errorf("invalid range %v-%v, network address is: %v",
   fromIp.String(), toIp.String(), network.String()))
 }
 return network
}

Just to be on the safe side, we've added a protection for non CIDR range. For example the IP range:

1.1.1.0 - 1.1.1.255

Is converted to the CIDR 1.1.1.0/24, but the IP range:

1.1.1.0 - 1.1.1.254

Cannot be converted to a single CIDR.

Then, we run the application, and found that the IP ranges that we have are indeed non-convertable to a single CIDR. To address this issue, we've create another implementation that can split IP range to multiple CIDRs.

func MakeMultipleCidrs(
 fromIp uint32,
 toIp uint32,
) []net.IPNet {

 ones := 0

 for checkBitMask := uint32(1) << 31; checkBitMask != 0 && fromIp&checkBitMask == toIp&checkBitMask; checkBitMask = checkBitMask >> 1 {
  ones++
 }

 mask := net.CIDRMask(ones, 32)
 maskInt := IpToInt(net.IP(mask))
 networkAddressStartInt := fromIp & maskInt
 networkAddressEndInt := networkAddressStartInt | (^maskInt)

 if networkAddressStartInt == fromIp && networkAddressEndInt == toIp {
  networkAddress := IntToIp(networkAddressStartInt)
  network := net.IPNet{
   IP:   networkAddress,
   Mask: mask,
  }
  return []net.IPNet{network}

 }
 size := networkAddressEndInt - networkAddressStartInt + 1
 halfSize := size / 2
 secondHalfStartInt := networkAddressStartInt + halfSize
 firstHalfEnd := secondHalfStartInt - 1

 cidrsFirst := MakeMultipleCidrs(fromIp, firstHalfEnd)
 cidrsSecond := MakeMultipleCidrs(secondHalfStartInt, toIp)
 cidrs := append(cidrsFirst, cidrsSecond...)
 return cidrs
}

This recursive function breaks the range into 2 parts, until it finds a CIDR that match the range.

An example of output for this function is below:

range 1.1.1.1 - 1.1.1.1, cidrs: [1.1.1.1/32]
range 1.1.1.0 - 1.1.1.1, cidrs: [1.1.1.0/31]
range 1.1.1.0 - 1.1.1.2, cidrs: [1.1.1.0/31 1.1.1.2/32]
range 1.1.1.1 - 1.1.1.3, cidrs: [1.1.1.1/32 1.1.1.2/31]
range 1.1.1.0 - 1.1.1.3, cidrs: [1.1.1.0/30]
range 1.1.1.0 - 1.1.1.255, cidrs: [1.1.1.0/24]
range 1.1.1.0 - 1.1.2.255, cidrs: [1.1.1.0/24 1.1.2.0/24]
range 1.1.2.0 - 1.1.3.255, cidrs: [1.1.2.0/23]

Notice that this implementation works only for IPv4, feel free to use the same idea for IPv6.