Handling File Uploads When Offline With Service Worker

Question

We have a web app (built using AngularJS) that we're gradually adding PWA 'features' too (service worker, launchable, notifications, etc). One of the features our web app has is the ability to complete a web form while offline. At the moment, we store the data in IndexedDB when offline, and simply encourage the user to push that data to the server once they're online ("This form is saved to your device. Now you're back online, you should save it to the cloud..."). We will do this automatically at some point, but that's not necessary at the moment.

We are adding a feature to these web forms, whereby the user will be able to attach files (images, documents) to the form, perhaps at several points throughout the form.

My question is this - is there a way for service worker to handle file uploads? To somehow - perhaps - store the path to the file to be uploaded, when offline, and push that file up once the connection has been restored? Would this work on mobile devices, as do we have access to that 'path' on those devices? Any help, advice or references would be much appreciated.

Answer 1

When the user selects a file via an <input type="file"> element, we are able to get the selected file(s) via fileInput.files. This gives us a FileList object, each item in it being a File object representing the selected file(s). FileList and File are supported by HTML5's Structured Clone Algorithm.

When adding items to an IndexedDB store, it creates a structured clone of the value being stored. Since FileList and File objects are supported by the structured clone algorithm, this means that we can store these objects in IndexedDB directly.

To perform those file uploads once the user goes online again, you can use the Background Sync feature of service workers. Here's an introductory article on how to do that. There are a lot of other resources for that as well.

In order to be able to include file attachments in your request once your background sync code runs, you can use FormData. FormDatas allow adding File objects into the request that will be sent to your backend, and it is available from within the service worker context.

Answer 2

The Cache API is designed to store a request (as the key) and a response (as the value) in order to cache a content from the server, for the web page. Here, we're talking about caching user input for future dispatch to the server. In other terms, we're not trying to implement a cache, but a message broker, and that's not currently something handled by the Service Worker spec (Source).

You can figure it out by trying this code:

HTML:

<button id="get">GET</button>
<button id="post">POST</button>
<button id="put">PUT</button>
<button id="patch">PATCH</button>

JavaScript:

if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/service-worker.js', { scope: '/' }).then(function (reg) {
    console.log('Registration succeeded. Scope is ' + reg.scope);
  }).catch(function (error) {
    console.log('Registration failed with ' + error);
  });
};

document.getElementById('get').addEventListener('click', async function () {
  console.log('Response: ', await fetch('50x.html'));
});

document.getElementById('post').addEventListener('click', async function () {
  console.log('Response: ', await fetch('50x.html', { method: 'POST' }));
});

document.getElementById('put').addEventListener('click', async function () {
  console.log('Response: ', await fetch('50x.html', { method: 'PUT' }));
});

document.getElementById('patch').addEventListener('click', async function () {
  console.log('Response: ', await fetch('50x.html', { method: 'PATCH' }));
});

Service Worker:

self.addEventListener('fetch', function (event) {
    var response;
    event.respondWith(fetch(event.request).then(function (r) {
        response = r;
        caches.open('v1').then(function (cache) {
            cache.put(event.request, response);
        }).catch(e => console.error(e));
        return response.clone();
    }));
});

Which throws:

TypeError: Request method 'POST' is unsupported

TypeError: Request method 'PUT' is unsupported

TypeError: Request method 'PATCH' is unsupported

Since, the Cache API can't be used, and following the Google guidelines, IndexedDB is the best solution as a data store for ongoing requests. Then, the implementation of a message broker is the responsibility of the developer, and there is no unique generic implementation that will cover all of the use cases. There are many parameters that will determine the solution:

• Which criteria will trigger the use of the message broker instead of the network? window.navigator.onLine? A certain timeout? Other?
• Which criteria should be used to start trying to forward ongoing requests on the network? self.addEventListener('online', ...)? navigator.connection?
• Should requests respect the order or should they be forwarded in parallel? In other terms, should they be considered as dependent on each other, or not?
• If run in parallel, should they be batched to prevent a bottleneck on the network?
• In case the network is considered available, but the requests still fail for some reason, which retry logic to implement? Exponential backoff? Other?
• How to notify the user that their actions are in a pending state while they are?
• ...

This is really very broad for a single StackOverflow answer.

That being said, here is a minimal working solution:

HTML:

<input id="file" type="file">
<button id="sync">SYNC</button>
<button id="get">GET</button>

JavaScript:

if ('serviceWorker' in navigator) {
  navigator.serviceWorker.register('/service-worker.js', { scope: '/' }).then(function (reg) {
    console.log('Registration succeeded. Scope is ' + reg.scope);
  }).catch(function (error) {
    console.log('Registration failed with ' + error);
  });
};

document.getElementById('get').addEventListener('click', function () {
  fetch('api');
});

document.getElementById('file').addEventListener('change', function () {
  fetch('api', { method: 'PUT', body: document.getElementById('file').files[0] });
});

document.getElementById('sync').addEventListener('click', function () {
  navigator.serviceWorker.controller.postMessage('sync');
});

Service Worker:

self.importScripts('https://unpkg.com/[email protected]/build/iife/index-min.js');

const { openDB, deleteDB, wrap, unwrap } = idb;

const dbPromise = openDB('put-store', 1, {
    upgrade(db) {
        db.createObjectStore('put');
    },
});

const idbKeyval = {
    async get(key) {
        return (await dbPromise).get('put', key);
    },
    async set(key, val) {
        return (await dbPromise).put('put', val, key);
    },
    async delete(key) {
        return (await dbPromise).delete('put', key);
    },
    async clear() {
        return (await dbPromise).clear('put');
    },
    async keys() {
        return (await dbPromise).getAllKeys('put');
    },
};

self.addEventListener('fetch', function (event) {
    if (event.request.method === 'PUT') {
        let body;
        event.respondWith(event.request.blob().then(file => {
            // Retrieve the body then clone the request, to avoid "body already used" errors
            body = file;
            return fetch(new Request(event.request.url, { method: event.request.method, body }));
        }).then(response => handleResult(response, event, body)).catch(() => handleResult(null, event, body)));

    } else if (event.request.method === 'GET') {
        event.respondWith(fetch(event.request).then(response => {
            return response.ok ? response : caches.match(event.request);
        }).catch(() => caches.match(event.request)));
    }
});

async function handleResult(response, event, body) {
    const getRequest = new Request(event.request.url, { method: 'GET' });
    const cache = await caches.open('v1');
    await idbKeyval.set(event.request.method + '.' + event.request.url, { url: event.request.url, method: event.request.method, body });
    const returnResponse = response && response.ok ? response : new Response(body);
    cache.put(getRequest, returnResponse.clone());
    return returnResponse;
}

// Function to call when the network is supposed to be available

async function sync() {
    const keys = await idbKeyval.keys();
    for (const key of keys) {
        try {
            const { url, method, body } = await idbKeyval.get(key);
            const response = await fetch(url, { method, body });
            if (response && response.ok)
                await idbKeyval.delete(key);
        }
        catch (e) {
            console.warn(`An error occurred while trying to sync the request: ${key}`, e);
        }
    }
}

self.addEventListener('message', sync);

Some words about the solution: it allows to cache the PUT request for future GET requests, and it also stores the PUT request into an IndexedDB database for future sync. About the key, I was inspired by Angular's TransferHttpCacheInterceptor which allows to serialize backend requests on the server-side rendered page for use by the browser-rendered page. It uses <verb>.<url> as the key. That supposes a request will override another request with the same verb and URL.

This solution also supposes that the backend does not return 204 No content as a response of a PUT request, but 200 with the entity in the body.

Answer 3

I was also stumbling upon it lately. Here is what I am doing to store in index db and return response when offline.

const storeFileAndReturnResponse = async function (request, urlSearchParams) {
  let requestClone = request.clone();

  let formData = await requestClone.formData();

  let tableStore = "fileUploads";

  let fileList = [];
  let formDataToStore = [];
  //Use formData.entries to iterate collection - this assumes you used input type= file
  for (const pair of formData.entries()) {
    let fileObjectUploaded = pair[1];
    //content holds the arrayBuffer (blob) of the uploaded file
    formDataToStore.push({
      key: pair[0],
      value: fileObjectUploaded,
      content: await fileObjectUploaded.arrayBuffer(),
    });

    let fileName = fileObjectUploaded.name;
    fileList.push({
      fileName: fileName,
    });
  }

  let payloadToStore = {
    parentId: parentId,
    fileList: fileList,
    formDataKeyValue: formDataToStore,
  };
  (await idbContext).put(tableStore, payloadToStore);

  return {
    UploadedFileList: fileList,
  };
};