Indexing performance boost and load balancing for Elasticsearch with NGINX as a sidecar of your application stack

Feeding Elasticsearch works via the HTTP endpoint reachable on Port 9200 by default. Basically you can feed your data with HTTP1.1 requests, which can be implemented with any language on any platform.

The benefit for the HTTP endpoint is the loosley coupling between your application and the Elasticsearch cluster and the stateless communication. A big minus is that a new connection needs to be established for every request. So how can we reuse the HTTP connection or keep the connections alive?

We need a proxy!!!

To keep the closed connections active we can use a transparent middleware, a reverse proxy. The simplest solution and also recommended from elastic is NGINX. With NGINX we have an open source webserver that scales well and supports multiple proxy setups.

Sidecar???

The sidecar pattern is used to place a utility/tool for you app beside it to support the application. In our case we can spin up an NGINX instance that speeds up the indexing performance of Elasticsearch.

Development setup

As usual for my development approach I’m going to use a docker based setup to get a…