Don't use ethereum bootnode

Ethereum documentation recommends using bootnode for private ethereum networks.
Once the boot node is installed, any geth based miner specifies the boot node ip and key, for example:

geth --bootnodes "enode:// BOOT_NOT_KEY @ BOOT_NODE_IP :30301"

This way, the boot node is notified of any new miner that starts, and update the new miner, using the gossip protocol, of any other existing miners, hence allowing the new miner to join the ethereum network.

Sounds fine, right?

Well... Not perfect.

This design works only as long as the boot node is alive.
Once the boot node crashes, new nodes cannot connect to the ethereum network.
The boot node is a single point of failure.

You can state: that's fine. Run the boot node as part of kubernetes, or docker swarm, and let it automatically restart the boot node if it crashes.

But let's examine this:
The boot node will start, from scratch. No miner peers to talk with, so all the existing miners will be part of the old ethereum network, while new miners will establish a new ethereum network, separated from the old one.

What is the solution?

The solution involves storing a set of all miners in an external database, such as MongoDB, or in case running in kubernetes, the kubernetes ConfigMap can be used.
Wrap each miner startup with the following logic:

1. Generate node key using the bootnode -genkey command
2. Store the current miner enode address in the external set of miners. The enode address is in format: enode://NODE_KEY@NODE_IP:30303
3. Fetch all enodes addresses from the external set of miners, and run geth using all the addresses separated by comma. For example:
   geth --bootnodes "enode://BOOT_NOT_KEY1@BOOT_NODE_IP1 :30301,enode://BOOT_NOT_KEY2@BOOT_NODE_IP2 :30301"

Summary

Using the external set of miners has multiple advantages:

1. There is no single point of failure
2. Miners always join the same ethereum network, and do not create a separate one
3. The bootnode process is no longer required (though we do need an external location to save the set of miners)