We have a web app (built using AngularJS) that we’re gradually adding PWA ‘features’ too (service worker, launchable, notifications, etc). One of the features our web app has is the ability to complete a web form while offline. At the moment, we store the data in IndexedDB when offline, and simply encourage the user to push that data to the server once they’re online (“This form is saved to your device. Now you’re back online, you should save it to the cloud…”). We will do this automatically at some point, but that’s not necessary at the moment.
We are adding a feature to these web forms, whereby the user will be able to attach files (images, documents) to the form, perhaps at several points throughout the form.
My question is this – is there a way for service worker to handle file uploads? To somehow – perhaps – store the path to the file to be uploaded, when offline, and push that file up once the connection has been restored? Would this work on mobile devices, as do we have access to that ‘path’ on those devices? Any help, advice or references would be much appreciated.
Advertisement
Answer
One way to handle file uploads/deletes and almost everything, is by keeping track of all the changes made during the offline requests. We can create a sync
object with two arrays inside, one for pending files that will need to be uploaded and one for deleted files that will need to be deleted when we’ll get back online.
tl;dr
Key phases
Service Worker Installation
Service Worker Fetch
Handling the service worker
fetch
event, if the fetch fails, then we have to handle the requests for the files listing, the requests that upload a file to the server and the request that deletes a file from the server. If we don’t have any of these requests, then we return a match from the default cache.- Listing
GET
We get the cached object of the listing (in our case/uploads
) and thesync
object. Weconcat
the default listing files with thepending
files and we remove thedeleted
files and we return new response object with a JSON result as the server would have returned it. - Uloading
PUT
We get the cached listing files and thesync
pending
files from the cache. If the file isn’t present, then we create a new cache entry for that file and we use the mime type and theblob
from the request to create a newResponse
object that it will be saved to the default cache. - Deleting
DELETE
We check in the cached uploads and if the file is present we delete the entry from both the listing array and the cached file. If the file is pending we just delete the entry from thepending
array, else if it’s not already in thedeleted
array, then we add it. We update listing, files and sync object cache at the end.
- Listing
Syncing
When the
online
event gets triggered, we try to synchronize with the server. We read thesync
cache.- If there are pending files, then we get each file
Response
object from cache and we send aPUT
fetch
request back to the server. - If there are deleted files, then we send a
DELETE
fetch
request for each file to the server. - Finally, we reset the
sync
cache object.
- If there are pending files, then we get each file
Code implementation
(Please read the inline comments)
Service Worker Install
const cacheName = 'pwasndbx'; const syncCacheName = 'pwasndbx-sync'; const pendingName = '__pending'; const syncName = '__sync'; const filesToCache = [ '/', '/uploads', '/styles.css', '/main.js', '/utils.js', '/favicon.ico', '/manifest.json', ]; /* Start the service worker and cache all of the app's content */ self.addEventListener('install', function(e) { console.log('SW:install'); e.waitUntil(Promise.all([ caches.open(cacheName).then(async function(cache) { let cacheAdds = []; try { // Get all the files from the uploads listing const res = await fetch('/uploads'); const { data = [] } = await res.json(); const files = data.map(f => `/uploads/${f}`); // Cache all uploads files urls cacheAdds.push(cache.addAll(files)); } catch(err) { console.warn('PWA:install:fetch(uploads):err', err); } // Also add our static files to the cache cacheAdds.push(cache.addAll(filesToCache)); return Promise.all(cacheAdds); }), // Create the sync cache object caches.open(syncCacheName).then(cache => cache.put(syncName, jsonResponse({ pending: [], // For storing the penging files that later will be synced deleted: [] // For storing the files that later will be deleted on sync }))), ]) ); });
Service Worker Fetch
self.addEventListener('fetch', function(event) { // Clone request so we can consume data later const request = event.request.clone(); const { method, url, headers } = event.request; event.respondWith( fetch(event.request).catch(async function(err) { const { headers, method, url } = event.request; // A custom header that we set to indicate the requests come from our syncing method // so we won't try to fetch anything from cache, we need syncing to be done on the server const xSyncing = headers.get('X-Syncing'); if(xSyncing && xSyncing.length) { return caches.match(event.request); } switch(method) { case 'GET': // Handle listing data for /uploads and return JSON response break; case 'PUT': // Handle upload to cache and return success response break; case 'DELETE': // Handle delete from cache and return success response break; } // If we meet no specific criteria, then lookup to the cache return caches.match(event.request); }) ); }); function jsonResponse(data, status = 200) { return new Response(data && JSON.stringify(data), { status, headers: {'Content-Type': 'application/json'} }); }
Service Worker Fetch Listing GET
if(url.match(//uploads/?$/)) { // Failed to get the uploads listing // Get the uploads data from cache const uploadsRes = await caches.match(event.request); let { data: files = [] } = await uploadsRes.json(); // Get the sync data from cache const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName }); const sync = await syncRes.json(); // Return the files from uploads + pending files from sync - deleted files from sync const data = files.concat(sync.pending).filter(f => sync.deleted.indexOf(f) < 0); // Return a JSON response with the updated data return jsonResponse({ success: true, data }); }
Service Worker Fetch Uloading PUT
// Get our custom headers const filename = headers.get('X-Filename'); const mimetype = headers.get('X-Mimetype'); if(filename && mimetype) { // Get the uploads data from cache const uploadsRes = await caches.match('/uploads', { cacheName }); let { data: files = [] } = await uploadsRes.json(); // Get the sync data from cache const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName }); const sync = await syncRes.json(); // If the file exists in the uploads or in the pendings, then return a 409 Conflict response if(files.indexOf(filename) >= 0 || sync.pending.indexOf(filename) >= 0) { return jsonResponse({ success: false }, 409); } caches.open(cacheName).then(async (cache) => { // Write the file to the cache using the response we cloned at the beggining const data = await request.blob(); cache.put(`/uploads/${filename}`, new Response(data, { headers: { 'Content-Type': mimetype } })); // Write the updated files data to the uploads cache cache.put('/uploads', jsonResponse({ success: true, data: files })); }); // Add the file to the sync pending data and update the sync cache object sync.pending.push(filename); caches.open(syncCacheName).then(cache => cache.put(new Request(syncName), jsonResponse(sync))); // Return a success response with fromSw set to tru so we know this response came from service worker return jsonResponse({ success: true, fromSw: true }); }
Service Worker Fetch Deleting DELETE
// Get our custom headers const filename = headers.get('X-Filename'); if(filename) { // Get the uploads data from cache const uploadsRes = await caches.match('/uploads', { cacheName }); let { data: files = [] } = await uploadsRes.json(); // Get the sync data from cache const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName }); const sync = await syncRes.json(); // Check if the file is already pending or deleted const pendingIndex = sync.pending.indexOf(filename); const uploadsIndex = files.indexOf(filename); if(pendingIndex >= 0) { // If it's pending, then remove it from pending sync data sync.pending.splice(pendingIndex, 1); } else if(sync.deleted.indexOf(filename) < 0) { // If it's not in pending and not already in sync for deleting, // then add it for delete when we'll sync with the server sync.deleted.push(filename); } // Update the sync cache caches.open(syncCacheName).then(cache => cache.put(new Request(syncName), jsonResponse(sync))); // If the file is in the uplods data if(uploadsIndex >= 0) { // Updates the uploads data files.splice(uploadsIndex, 1); caches.open(cacheName).then(async (cache) => { // Remove the file from the cache cache.delete(`/uploads/${filename}`); // Update the uploads data cache cache.put('/uploads', jsonResponse({ success: true, data: files })); }); } // Return a JSON success response return jsonResponse({ success: true }); }
Synching
// Get the sync data from cache const syncRes = await caches.match(new Request(syncName), { cacheName: syncCacheName }); const sync = await syncRes.json(); // If the are pending files send them to the server if(sync.pending && sync.pending.length) { sync.pending.forEach(async (file) => { const url = `/uploads/${file}`; const fileRes = await caches.match(url); const data = await fileRes.blob(); fetch(url, { method: 'PUT', headers: { 'X-Filename': file, 'X-Syncing': 'syncing' // Tell SW fetch that we are synching so to ignore this fetch }, body: data }).catch(err => console.log('sync:pending:PUT:err', file, err)); }); } // If the are deleted files send delete request to the server if(sync.deleted && sync.deleted.length) { sync.deleted.forEach(async (file) => { const url = `/uploads/${file}`; fetch(url, { method: 'DELETE', headers: { 'X-Filename': file, 'X-Syncing': 'syncing' // Tell SW fetch that we are synching so to ignore this fetch } }).catch(err => console.log('sync:deleted:DELETE:err', file, err)); }); } // Update and reset the sync cache object caches.open(syncCacheName).then(cache => cache.put(syncName, jsonResponse({ pending: [], deleted: [] })));
Example PWA
I have created a PWA example that implements all these, which you can find and test here. I have tested it using Chrome and Firefox and using Firefox Android on a mobile device.
You can find the full source code of the application (including an express
server) in this Github repository: https://github.com/clytras/pwa-sandbox.