Serverless Data Apps

It’s not unusual to follow a link from a scientific publication only to arrive at a page for a once useful web application that no longer functions or exists – presumably because funding for the project has expired and the person who maintained the app is no longer working on it.

With this in mind, and particularly since Focused Research Organizations like EvE Bio are not designed to exist indefinitely, we tried to avoid having the data application we built ever ending up in this situation. Fortunately, a set of relatively new technologies provide a new way to create serverless interactive data apps. What began as a hypothesis that this could work has been proven out over the last year in EvE Bio’s Data Explorer, which is available to the public.

The tech stack for this application is DuckDB, WebAssembly (WASM), Quarto, and Observable JS, all of which are open source and freely available. The end result is a responsive app for data exploration that doesn’t require a backend server. It exists as a set of files that are uploaded to EvE’s website, and everything runs locally in the user’s browser. The app was built by a data scientist and requires no maintenance or monitoring unless new data needs to be added. This reduces the labor cost, and virtually eliminates infrastructure costs. The expertise needed to transition this to a new location is reduced to just copying a folder of files – the usual html, css, and images you’d expect for a website, plus a set of parquet data files.

This is the role of each part of the stack:

  • DuckDB allows for a set of data files in familiar formats to be used as a fast, lightweight analytical database
  • WASM makes it possible for DuckDB to work directly in the browser
  • Quarto is a technical publishing platform that combines code and text into website form
  • Observable JS provides a reactive framework to support user interaction

Since WASM is bundled with DuckDB, and Observable and DuckDB are accessible from Quarto, the developer can work entirely in Quarto.

We have been encouraged by the success of this approach and hope it provides inspiration for other apps using this stack!