'What's the best way to handle efficient lookup for more than 64 separate fields with indexes in MongoDB?
I need to be able to search documents efficiently with the functionality of an index for over 64 fields (potentially thousands). MongoDB has a cap of 64 indices per collection, so my first instinct is to just split the data across 64+ collections that have 1 custom index each since MongoDB allows the creation of thousands of collections. However, that seems very cluttered to me and it would flood collection list when browsing the DB with software like NoSQLBooster. I'm wondering if there's a better way to do it.
Context (in case it's relevant)
I run a competitive service for a game. People can set scores on leaderboards. A collection of user-selected leaderboards can be created that will allow thousands (or even millions) of users to compete on the combination of leaderboards for an overall ranking for the collection of leaderboards. Every player can have a ranking in every user-defined set of leaderboards and I had indices set up that allowed me to efficiently search through players based on the numerical rating that determined their rank within the set of leaderboards.
Presently (for clarity), every user in the users collection looks something like this:
{
_id: ObjectID,
name: string,
avatar: string,
rankings: {
leaderboard_collection_a: float,
leaderboard_collection_b: float,
leaderboard_collection_c: float,
leaderboard_collection_d: float,
...
}
...
}
where the indices for the collection are ["rankings.leaderboard_collection_a_-1", "rankings.leaderboard_collection_b_-1", "rankings.leaderboard_collection_c_-1", "rankings.leaderboard_collection_d_-1", ...].
Here are the templates for other collections:
leaderboard_collections
{
_id: string,
name: string,
associated_leaderboards: [
leaderboard_id_a,
leaderboard_id_b,
leaderboard_id_c,
...
]
...
}
leaderboards
{
_id: string,
name: string,
metadata: {
author: string,
sub_name: string,
...
}
difficulty: {
leaderboard_collection_a: float,
leaderboard_collection_b: float,
leaderboard_collection_c: float,
...
}
...
}
scores
{
_id: ObjectID,
leaderboard_id: string,
user_id: ObjectID,
score: float,
value: {
leaderboard_collection_a: float,
leaderboard_collection_b: float,
leaderboard_collection_c: float,
...
}
...
}
The objective here is just to be able to efficiently search through users based on their total ranking score in a leaderboard collection. This functionality is used to look up a user's ranking and display ranked ladders. Normally you'd use an index for this, but since a user can have a ranking score in numerous leaderboard collections, the 64 index cap is quickly reached.
Aside from the index issue, the database is structured to accommodate:
- 1M+ leaderboards
- 1M+ users
- 100M+ scores
- 1K+ leaderboard collections
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
