'How would I return results from an aggregation sorted by two fields from that aggregation that need to be equally weighted
I have an index with documents with the following format:
{
Id : unique string identifier,
Added: a positive integer,
Deleted: a positive integer
}
I was previously doing the following operations with transforms and update by query operations.
Transform:
Group by Id,
Churn = count of Id
Complexity = sum of (Added - Deleted)
After Transform Completes:
Query for min and max churn and complexity in resulting index from transform
UpdateByQuery all documents with this formula:
string linmapComplexity = $"(ctx._source.ElasticTransformDocument.Complexity - {minComplexity}) * 100 / ({maxComplexity} - {minComplexity})";
string linmapChurn = $"(ctx._source.Churn - {minChurn}) * 100 / ({maxChurn} - {minChurn})";
UpdateByQueryScript = $"ctx._source.Score = {linmapComplexity} * {linmapChurn};"
This formula has the effect of creating a composite score on which to sort results from so that we can get the top x complexity and churn values. We need to map complexity and churn to the same range since complexity is orders of magnitude larger than churn.
I was recently informed that Transforms or any process that would store the results in an extra index are not feasible in my use case and that this would all need to be done under one query. However as the score on which we sort values requires knowing the min and max of the previous aggregation and that aggregations themselves only return a subset of the data, making this data scaling useless I don't see a way to return the top x values in an aggregation based on two fields
Another issue I noticed is that when sorting for a single field in an aggregation that it would sort the subset of data returned by the aggregation and not the top x field values in the entire dataset. Is there a way to do the latter?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
