I wonder how this query is executing successfully. As we know 'having' clause execute before the select one then here how alias name used in 'select' statement
I am trying to cleanup and recreate databricks delta table for integration tests. I want to run the tests on devops agent so i am using JDBC (Simba driver) bu
As you can see the library I'm using is asking to make an entry but there's no box/window where I can make the entry. How do I make an entry here amongst y/n/u/
I am working on Delta table using Databricks on Azure. The Delta table contains about 100 million records with many columns. One column data type of which is S
I need to find a way to delete multiple rows from a delta table/pyspark data frame given a list of ID's to identify the rows. As far as I can tell there isn't a
I am running databricks 7.3LTS and having errors while trying to use scala bulk copy. The error is: object sqldb is not a member of package com.microsoft. I hav
A peer of mine has created code that opens a restful api web service within an interactive spark job. The intent of our company is to use his code as a means o
I am working with Azure Databricks jupyter notebooks and have time-consuming jobs (complex queries, model training, loops over many items, etc.). Every time I c
I’m attempting to make a cURL patch request via a GitHub action that executes when I make a push. The cURL call works perfectly when I execute it in Windo
I have two ML projects on Azure Databricks that work almost the same except that they are for different clients. Essentially I want to use some management syste
I have a curious issue, when launching a databricks notebook from a caller notebook through dbutils.notebook.run (I am working in Azure Databricks). One intere
The error described below occurs when I run Spark job on Databricks the second time (the first less often). The sql query just performs create table as select
I am using Azure Purview for Data Governance, and Data Lineage. We use Databricks in our Data Architecture, but there isn't any native support for capturing Dat
I have been attempting to setup 'init scripts' on databricks, so I can install all of my python libraries and keep the environment controlled. Tried yesterday u
I am executing a Spark job in Databricks cluster. I am triggering the job via a Azure Data Factory pipeline and it execute at 15 minute interval so after the su
I have python variable created under %python in my jupyter notebook file in Azure Databricks. How can I access the same variable to make comparisons under %sql.
I am building an Azure Data Factory pipeline and I would like to know how to get this parameter into the python script. The python script is l
I'm using Azure Databricks with a custom configuration that uses vnet injection and I am unable to start a cluster in my workspace. The error message being give
I want to define an environment variable in Databricks init script and then read it in Pyspark notebook. I wrote this: dbutils.fs.put("/databricks/scripts/i
Trying to create Databricks workspace using terraform but unsupported arguments: resource "azurerm_databricks_workspace" "workspace" { name = "