CentOS Question posted in
The official documentation can be found here.

How to improve search speed of a query in Elastic Search? – CentOS

The host is AWS from ElasticSearch, I have 2TB of data stored in 6 nodes and in 30 indexes with 10 shards each. A simple search in all indexes is very very slow and takes a few minutes.

Where I made the mistake? Is that normal or I have bad settings or maybe I have too much data stored?

My cluster settings:

"search": {
     "max_queue_size": "1000",
     "queue_size": "1000",
     "size": "4",
     "auto_queue_frame_size": "2000",
     "target_response_time": "1s",
     "min_queue_size": "1000"
},

My nodes settings:

"os": {
     "refresh_interval_in_millis": 1000,
     "name": "Linux",
     "pretty_name": "CentOS Linux 7 (Core)",
     "arch": "amd64",
     "version": "4.15.0-1039-aws",
     "available_processors": 32,
     "allocated_processors": 2
}

Thank you!

Tags:

2

Answers


  1. 0

    Its a very broad question with very less information, can you please provide more information like:

    1. How much data you have and size of each shard.
    2. Sample search queries with their avg time taken.
    3. data nodes heap size and why you have allocated_processors very less(2) than available processors(32)
    4. How many documents you are fetching in your search query ie size param.
    5. Search latency wrt to no of search calls
    6. Search slow logs of elasticsearch(ES)

    You can refer to my 10 tips on improving search performance ,and also tell me the values of some param mentioned in the tips.

    Login or Signup to reply.
  2. 0 
    1)I have 2TB data. Every index has 50GB or more. Each has 10 shards and a replica

    That’s to much.
    The goal size for a shard should be around 50Gb. With your setting you are more around 5Gb Each.
    You can shrink to 5 shards or less and force merge to 1 segment.
    Performances should be improved a lot.

    After that, look at other good advises provided by Optsters in his blog. They are all relevant.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search