Lessons learned to save costs on OpenSearch clusters at AWS

The OpenSearch Service on AWS can be expensive, especially when dealing with large volumes of data. In this context, many businesses are looking for ways to save costs on OpenSearch while still maintaining its performance and functionality. In this post, we will explore some of the strategies that businesses can use to optimize their OpenSearch costs on AWS, without sacrificing performance and scalability.

Select the right shard size and number of shards

The recommendations from AWS are:

• Shard size should be between 10 to 30 GB for search
• Shard size should be between 30 to 50gb for time series based data
• Never exceed 50gb per shard
• Don’t worry to use a single shard based on the size recommendations above, the more shards you have per index the more network communication between the shards(distributed over the nodes) and the more computation is necessary to combine distributed results.

Do you need replicas?

If you don’t care about downtimes(never happend for me on AWS so far), which means the node holding your single shard with data dies, you can avoid replicas. During indexing all the data is written to the primary shard…