'Unable to search on whole database with searchkick as it limits to 10000 records

Unable to search on whole elastic search DB just by using

SearchData.search('yamaha', match: :word_middle,load: false)

This limits the search to 10000 records but in my DB there are more than a hundred thousand records so, how to search on the whole DB not just the first ten thousand records I'm not able to find anything a little help will be appreciated

ruby-on-railsrubysearchkick


Solution 1:[1]

I am not sure about searckick but Elasticsearch always search on entire index and not only 10000 recored but by default it returns only 10 documents in response. You can change response size by changing size parameters and max it return 10000 documents per request.

Also, if you have large index then it always show 10000 for hits.total.value in response and for getting actual number count you can set track_total_hits value to true.

{
  "track_total_hits": true,
  "query": {
    "match" : {
      "user.id" : "elkbee"
    }
  }
}

If you need to get more then 10000 documents from the Elasticsearch then you can use search_after or scroll API. you can refer this documentation for more details.

Solution 2:[2]

Deep Paging

By default, Elasticsearch and OpenSearch limit paging to the first 10,000 results. Here’s why. We don’t recommend changing this, but if you really need all results, you can use:

class Product < ApplicationRecord
  searchkick deep_paging: true
end

If you just need an accurate total count, you can instead use:

Product.search("pears", body_options: {track_total_hits: true})

Solution 3:[3]

Searhkick only limits the results to 10,000. It will still search on all records in your model, even if the number is well over 10,000.

If you aren't seeing the results you expect, there is something else wrong with your setup.

Have you included searchkick in your model and reindexed?

class YourModel < ApplicationRecord
  searchkick
end

And in a new console: YourModel.reindex

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 Sagar Patel
Solution 2 jainvikram444
Solution 3 Pseudo-Hippy

Related Questions