'Avoid using scroll in ReactiveElasticsearchTemplate or Clearing/closing scroll after use

We have a spring webflux application which is querying Elasticsearch using ReactiveElasticsearchTemplate like this

        final inline fun <reified R> getSearchMonoList(
        indexName: String,
        query: NativeSearchQuery,
        metricName: String
    ): Mono<List<SearchHit<R>>> {
        val startTime = getCurrentTime()
        recordThroughput(metricName, THROUGHPUT)
        return when (indexName == EMPTY_STRING) {
            true -> getEsClientTemplate().search(query, R::class.java).collectList()
            else -> getEsClientTemplate().search(query, R::class.java, IndexCoordinates.of(indexName)).collectList()
        }.doOnError {
            recordThroughput(metricName, FAILED_THROUGHPUT)
        }.doFinally {
            getEsClientTemplate()
            recordTime(metricName, startTime)
        }
    }

Client Configuration is

 public ReactiveElasticsearchClient reactiveElasticsearchClient() {
        ClientConfiguration clientConfiguration = ClientConfiguration.builder()
                .connectedTo(urls.toArray(String[]::new))
                .withWebClientConfigurer(webClient -> {
                    final ExchangeStrategies exchangeStrategies = ExchangeStrategies.builder()
                            .codecs(configurer -> configurer.defaultCodecs()
                                    .maxInMemorySize(-1))
                            .build();
                    return webClient.mutate().exchangeStrategies(exchangeStrategies).build();
                })
                .build();
        return ReactiveRestClients.create(clientConfiguration);
    } 

Our Problem

whenever Template queries ES Request body: {"from":0,"size":10000} is requesting ES for 10000 records which was way too much for a term query so we have fixed in query builder by using Pageable, However we do not want to use scroll as this is exhausting Max scroll connections as this API is very heavily used.

NativeSearchQueryBuilder().withQuery(boolQueryBuilders).withPageable(PageRequest.of(0,1))

i am aware of this spring documentation where this was the suggested solution

    SearchScrollHits<SampleEntity> scroll = template.searchScrollStart(1000, searchQuery, SampleEntity.class, index);

String scrollId = scroll.getScrollId();
List<SampleEntity> sampleEntities = new ArrayList<>();
while (scroll.hasSearchHits()) {
  sampleEntities.addAll(scroll.getSearchHits());
  scrollId = scroll.getScrollId();
  scroll = template.searchScrollContinue(scrollId, 1000, SampleEntity.class);
}
template.searchScrollClear(scrollId);

However in current production we can not update the libraries because of DMZ restrictions, We are using spring data elasticsearch 4.1.1

How can i disable scroll or clear the scroll after use any help would be appreciable ?



Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source