I am sending JSON telemetry data from Azure Stream Analytics to Azure Data Lake Gen2 serialized as .parquet files. From the data lake I've then created a view i
I'm trying to connect to synapse serverless pool via databricks. I need to create synapse views and external tables directly in databricks as part of an existin
I'm using Synapse Serverless Pool and get the following error trying to use CETAS Msg 15860, Level 16, State 5, Line 3 External table location path is not valid
I am building a datawarehouse in Azure Synapse where one of the sources are about 20 different types of XML files (with a different XSD scheme) and 1 base schem
I am trying to pass a whole directory of python files that are referenced in the main python file in Azure Synapse Spark Job Definition but the files are not ap
I am trying to transform data from ADLS by using Azure Synapse's Dataflow and store it in a table in Dedicated SQL Pool. I created a Dataset 'UserSinkDataset' p
I was working on connecting dedicated sql pool(formerly sql DWH) to synapse spark notebooks. I was using spark.read.synapsesql(). I'm able to write data as tabl
I am running a Synapse Notebook in a For Each activity in a Synapse Pipeline. The notebook loads some data from the datalake into the database and some custom
I have an ETL pipeline that gets triggers when file is uploaded into Datalake location "Processing" folder, Once data is processed I want to move the file to d
I'm trying read a file from a directory that contains square brackets in the path using a mapping dataflow in Azure Synapse, like this: /path/to/[a].[b]/some/fi
I am not able to find the way to connect ADLS Gen 1 in a synapse notebook. I already have existing linked service for ADLS Gen 1. I have seen some of the docume
I have a copy pipeline set up that connects to an SFTP server (on Azure Synapse). I have used it to copy csv files and this works fine, but now I have the compl
When I tried to create Azure Synapse Workspace I get below error: {"code":"DeploymentFailed","message":"At least one resource deployment operation failed. Pleas
Have had a look around but can't see any concrete information. Essentially, if anyone could help it would be great. We are building reporting in the cloud and l
I have a copy activity in Datafactory that dynamically maps the columns between files in tables A and B. Both tables, A and B are .parquet. Table A has 8 column
In Azure SYNAPSE I have two tables table A with 6 millions of records and Table B with 2 millions when I run a simple left join query it takes around 20 minutes
I would like to run spatial queries on large data sets; e.g. geopandas would be too slow. Inspiration I found here: https://anant-sharma.medium.com/apache-sedon
How to set the Synapse integrate pipeline parameter during deployment? I am using the Synapse deployment task with GIT to deploy the workspace to multiple envir
I'm experimenting with NLTK in an Azure Synapse notebook. When I try and run nltk.download('stopwords') I get the following error: ValueError: I/O operation on
New to Spark and Synapse....Need to do some transformation including adding a columns, changing datatypes, etc. I am reading a csv into a dataframe. I'd like t