'How to use different uploaders in CKAN (local & cloud)
I have multiple workspaces on a single CKAN installation. I want the resources of some workspaces to be stored locally to my server and the resources of other ones to be stored on AWS. For uploading to AWS, I am using https://github.com/open-data/ckanext-cloudstorage After setting it up, each upload goes to AWS and nothing local.
I want CKAN (or a plugin) to decide when to upload based on the workspace. I have written this code, in a custom plugin
import ckan
import ckan.lib.uploader as uploader
import ckan.plugins as plugins
import ckan.plugins.toolkit as toolkit
from pprint import pprint
from ckanext.cloudstorage import storage
c = toolkit.c
class MyCustomPlugin(plugins.SingletonPlugin):
#IUploader
plugins.implements(plugins.IUploader)
def get_resource_uploader(self, data_dict):
catalogs_string = toolkit.config.get(
'ckanext.myplugin.catalogs_to_upload_to_cloud_storage', '')
catalogs = catalogs_string.split()
context = {'model': ckan.model, 'session': ckan.model.Session,
'user': c.user}
package = toolkit.get_action('package_show')(context, {"id":data_dict['package_id']})
organization = toolkit.get_action('organization_show')(context, {"id":package['organization']['id']})
groups = organization['groups']
upload_to_aws = False
for g in groups:
for cat in catalogs:
#print(g['name'])
if g['name'] == cat:
upload_to_aws = True
break
#package = ckanGet.package_show(c, {"package_id":resource['package_id']})
#print(package)
#print(organization)
#print(catalogs)
print(upload_to_aws)
if upload_to_aws == True:
return storage.ResourceCloudStorage(data_dict)
else:
return uploader.ResourceUpload(data_dict)
#for upload of files which are not resource, use default upload (local to the ckan installation)
def get_uploader(self, upload_to, old_filename = None):
return uploader.Upload(upload_to, old_filename)
On the config file (ckan.ini), I have added something like
ckanext.myplugin.catalogs_to_upload_to_cloud_storage = workspace1 workspace2
Let's suppose I have 4 workspaces. I want workspace1 and workspace2 to be uploaded to aws and workspace3 and workspace4 locally.
The plugin I have written does not work. its function get_resource_uploader() is being called, since I see the prints on the terminal. The method does its job, but the returned values seems to be skipped.
Any comment / suggestion / solution? I am working on CKAN 2.9.5
Thank you so much
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
