I have a three node Cassandra cluster running. It’s three containers on one virtual machine. Pulsar is next. I decided on a new virtual machine from the same base image the Cassandra one started from. That way, it’s free of any “Cassandra pollution”. And I have been taking notes, so I know what I did to that image.
This caused me to have a miniphany about failure and disaster recovery. From that perspective, what I’m doing is a horrible mistake. By containerizing layers on a single machine, I’m creating single points of failure for the system. If the machine, virtual or otherwise, goes down, the system becomes non-functional because an entire layer is missing. The correct approach is to distribute the layer, containerized or not, across multiple machines and deploy the system vertically on each machine (to the extent that the hardware supports it).
In this test case, that would be Cassandra, Pulsar (and BookKeeper and ZooKeeper), and my application all in the same virtual machine, but three of them. In that situation, losing a machine brings down a “slice” of the system, but not an entire layer of it. It can remain functional.
Great. I’ve added another step to getting my infrastructure up and running. I’m still going to get Pulsar up and running in a new VM. For my own personal use, I’m not going to split BookKeeper and ZooKeeper into separate things – unless I must. They can be “Pulsar components”.
I’ll start playing with the application in a third VM, since I – not Docker – will be in control of network configuration. Once I get that talking to both Cassandra and Pulsar VMs, then I’ll deal with splitting up the layers and spin up three VMs each running all the layers, each in its own container.
If anyone, particularly a work person who stumbles on this, is wondering why I’m not doing this on work computers, it is because they are shared environments and I don’t want to break what anyone else is doing (this requires much “sudo”ing). Now that I have my new home computer, I have the resources to spare. My work desktop does not, nor do I expect it to. I also have much more screen space here, which is also a new thing. Running three VMs side-by-side, I get a GUI “full screen” view of each one, not just a terminal window. For years, I would not work from home because my office monitors were so much larger. If I were still going in to the office, I would probably ask for a monitor like this one for exactly the opposite reason. I would not consider asking for a computer like this one; it’s insane overkill and I do not expect anyone else to pay for that.
Marked Up 