'Find matching pattern on streaming data using Python
I'm receiving a stream of data in following format
a1 = {"name":"abc","city":"c1","code":"c1234"}
a1 = {"name":"abcd","city":"c1","code":"c3234"}
I need to find which pattern each of these belong to as they are received as message in KAFKA topic.
The patterns are like
p1 : name = 'abc' AND code="c1234"
p2 : city="c1"
p3 : name = 'abc'
Example 1 above should be mapped to p1 as it satisfies more conditions than p2, whereas second example should be mapped to p2.
I tried following
- Tried using for loops, but that takes lot of time as incoming data & rules are many.
- Tried to create rules in XML and use XPATH, but the nodes (name, city, code here) are not fixed. Rule can have only one or combination of many.
- Created matrix like one-hot encoding and tried to match them.
Is there any other way to find out which pattern/ rule the incoming message should be tagged to?
Can this be handled in KAFKA?
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
