'Programmatically running notebook with LIKE operator
In Databricks repos, say I want to programmatically run a notebook titled 'ExampleNotebook_45', but I want only use the part of the title 'ExampleNotebook', how do I write this in the command:
dbutils.notebook.run()
Similar language that comes to mind is the LIKE operator in sql where I can write along the lines of:
select * from table where name LIKE 'ar%'
Solution 1:[1]
After doing some research, this is the answer that solved the problem, getting the list of files in the folder via the workspace API
import requests
import json
ctx = json.loads(dbutils.entry_point.getDbutils().notebook().getContext.toJson())
my_pat = ctx['extraContext']['api_token']
workspace_url = 'https://'
def list_files(base_path: str):
lst = requests.request(method='get',
url=f"{workspace_url}/api/2.0/workspace/list",
headers={"Authentication": f"Bearer {my_pat}"},
json={"path": base_path}).json()["objects"]
results = []
for i in lst:
if i["object_type"] == "DIRECTORY" or i["object_type"] == "REPO":
results.extend(list_files(i["path"]))
else:
results.append(i["path"])
return results
all_files = list_files("base_path")
for i in all_files:
if 'ExampleNotebook' in i:
file_name = i
else:
continue
This will give me the file name (and path), that is similar to the LIKE sql operator.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|---|
| Solution 1 | Gitted |
