Data management is becoming increasingly important in large-scale scientific applications such as computational astrophysics, severe weather monitoring, and genomics. In this talk, I present our recent work to address two major challenges raised by those scientific applications. The first challenge regards “data uncertainty”, due to the fact that scientific measurements are inherently noisy and uncertain. In particular, we address uncertain data management under the array model, which has gained popularity for large-scale scientific data processing due to performance benefits. We propose a suite of storage and evaluation strategies to support array operations under data uncertainty. Results from Sloan Digital Sky Survey (SDSS) datasets show that our techniques outperform state-of-the-art methods by 1.7x to 4.3x for the Subarray operation and 1 to 2 orders of magnitude for Structure-Join.

As scientific data continues to grow in size and diversity, it is becoming harder for the user to express her data interests precisely in a formal language like SQL. We refer to this second problem as “query uncertainty”. This leads to a strong need for “interactive data exploration,” a service that efficiently navigates the user through a large data space to identify the objects of interest. We present our initial work on interactive data exploration, with results suggesting that it is possible to predict user interests modeled by conjunctive queries with a small number of samples, while providing interactive performance.