Insert a Dynamic into Elastic Document through nest

Question

var doc = new ESDoc {
   Field1 = "test1",
   Field2 = 3,
   ExtraData = 'dynamic object',
   Index = "myindex"
};

ElasticClient.Index(doc, s => s.Index(doc.Index));

This is in essence what I am trying to do. I have a document object, and I am wanting to add to it a dynamic object that allows us to through whatever customer specific data we want in there. I have no need to ever search or do any querying on it, I just need to hold it for CYA issues.

This results in ExtraData having a value_kind = 1.

I tried to JsonSerializer.Serialize the data and it came out in a triple escaped string.

I have seen people trying to create a entire document of dynamic data, and using a object cast, but I feel that isnt the answer here because I have a document that I want to add a dynamic object too.

NEST and Elaticsearch.Net 7.16.0

Answer 1

You can do this with the following:

1. You need to have a type to model the dynamic behaviour of the field. Now, you could specify it as dynamic on the ESDoc class, but you'll need an instance of a type to assign to it and Nest deserializes JSON to a Dictionary<string, object> for a dynamic member. Because of this, you might want to use the DynamicDictionary type in Elasticsearch.Net (a dependency of Nest) that is a dictionary with dynamic access behaviour

   public class ESDoc
   {
       public string Field1 { get; set; }
   
       public int Field2 { get; set; }
   
       public DynamicDictionary ExtraData { get; set; }
   }
   

2. Map ExtraData as object data type with enabled: false, so that it is not parsed and indexed

   var createIndexResponse = client.Indices.Create("myindex", c => c
       .Map<ESDoc>(m => m
           .Properties(p => p
               .Text(t => t
                   .Name(f => f.Field1)
               )
               .Number(t => t
                   .Name(f => f.Field2)
                   .Type(NumberType.Integer)
               )
               .Object<object>(o => o
                   .Name(f => f.ExtraData)
                   .Enabled(false)
               )
           )
       )
   );
   

3. Now index a doc, refresh the index, and search for it to ensure it deserializes as expected

   dynamic extraData = new DynamicDictionary();
   extraData.foo = "Foo";
   extraData.bar = new DynamicDictionary
   {
       ["baz"] = new DynamicValue("Baz")
   };
   
   var doc = new ESDoc
   {
       Field1 = "test1",
       Field2 = 3,
       ExtraData = extraData
   };
   
   client.Index(doc, s => s.Index("myindex")); 
   client.Indices.Refresh("myindex");   
   var searchResponse = client.Search<ESDoc>(s => s.Index("myindex"));
   
   var firstDoc = searchResponse.Documents.First();
   var baz = firstDoc.ExtraData["bar"]["baz"];
   
   Console.WriteLine($"{baz}");