The goal of the second phase of the collaboration is to create a collaborative distributed software ecosystem to manage data lifecycle and enable data analytics on distributed data sets and resources. Specifically, our goal is to build a dynamic software stack that is user- friendly, scalable, energy-efficient and fault tolerant. We plan to approach the problem from two dimensions.
- Distributed Infrastructure Support for Workflow and Data Management: Research to determine appropriate execution environments that allow users to seamlessly execute their end-to-end dynamic data analysis workflows in various resource environments and scales while meeting energy-efficiency, performance and fault tolerance goals.
- Deep partnerships with Scientific Collaborations: We will engage in deep partnerships with scientific teams and use a mix of user research with system software R&D to address specific challenges that these communities face. Our experience will in turn inform future research directions.