'What is the best way to design an ingestion pipeline that pulls data from Snowflake into ElasticSearch Cloud

I have newly subscribed to elastic platinum license to use elastic cloud services. I want to design the ingestion pipelines, which would pull data from snowflake data warehouse into the elasticsearch service. I have explored and found following methods, please suggest the best approach to design the pipeline. Also point me to any documentation available for it.

  1. Setup a logstash server, install logstash service in it. Define a pipeline which would pull data from snowflake into elastic service, using jdbc snowflake plugins.
  2. Use logstash pipelines, in elastic cloud. Trigger the pipeline through a rest api when ingesting. This cloud pipeline setup does not seem to have much documentation on how to implement.
  3. Ingesting through rest apis. I think this method is not the best, as configuring pipelines is not very flexible.
  4. Any other method such as Beats.
logstashdata-ingestionelastic-cloudelastic-pipeline


Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source

Related Questions