Does it help to have the service containers on the same server as the application containers?
I have a cluster with two servers that are roughly in the same region. I have my production environment set up in such a way that all of the service containers (i.e., load balancer, discovery, and VPN) are located on one server and the main application containers are located on the other.
The "Neighbor Latency" panel says the latency between the servers is 8 ms.
With that being said, would it be better to have the service containers on the same server? Is there a standard practice for where the service containers should be deployed?
Any advice or insight would be appreciated!
Hey Andrew that's a great question and I'm sure one that lots of Cycle users have thought about.
The answer really is... it depends. But let's take a deeper look at what it depends on.
The first step here is to identify the needs of the application. For smaller applications, background services, and the like, it's ok to just run one service per environment. In these cases, the optimizations from putting services on each of the servers or moving the services onto the same server shouldn't be noticeable.
Now if you have an application that has deep network latency requirements, then yes you could start putting a service in each datacenter you're deploying containers to for that environment. Putting one on every server, on the other hand, would be overkill.
And in the middle of these two is a pretty wide range of needs that you can fill by making changes and seeing what the impact is on your service. If you feel like you'd like to really optimize for latency, try a few different strategies and check the logs to see what seems to be working best. (Great news here as well, more logging/graphs/monitoring is coming each month!)
Side Note
There are some languages, like nodeJS, that will create a DNS lookup for every request. This could really clog the discovery service if things get out of hand, so if that's a concern for you, increasing the number of discovery services is something that could help.
I'd love to hear more about what you end up trying out and what works for you!
I'll also add, if you have a highly available deployment and are concerned about getting traffic to your instances quickly, you can take advantage of the GeoDNS setting on individual DNS records to route that ingress traffic to the nearest load balancer instance. If you want to run fewer services but optimize for geographic distribution, this is a pretty handy option to enable.