Continuing innovation in science and technology is vital for our society and
requires ever increasing computational resources. However, delivering such
resources has become intolerably complex, ad-hoc, costly and error prone due to an
enormous number of available design and optimization choices combined with the
complex interactions between all software and hardware components.
Auto-tuning, run-time adaptation and machine learning based approaches
have been demonstrating good promise to address above challenges for more than
a decade but are still far from the widespread production use due to unbearably
long exploration and training times, lack of a common experimental methodology,
and lack of public repositories for unified data collection, analysis and mining.
In this talk, I will present my long-term holistic and cooperative
methodology and infrastructure for systematic characterization and optimization
of computer systems through unified and scalable repositories of knowledge
and crowdsourcing. In this approach, multi-objective program and architecture
tuning to balance performance, power consumption, compilation time, code size
and any other important metric is transparently distributed among multiple
users while utilizing any available mobile, cluster or cloud computer services.
Collected information about program and architecture properties and behavior
is continuously processed using statistical and predictive modeling techniques
to build, keep and share only useful knowledge at multiple levels of granularity.
Gradually increasing and systematized knowledge can be used to predict most profitable
program optimizations, run-time adaptation scenarios and architecture configurations
depending on user requirements. I will also present a new version of the public,
open-source infrastructure and repository (cTuning3 aka Collective Mind)
for crowdsourcing auto-tuning using thousands of shared kernels, benchmarks
and datasets combined with online learning plugins. Finally, I will discuss
encountered challenges and some future collaborative research directions
on the way towards Exascale computing.
