High performance in software development

About

What are the ingredients of high-performing software?

Software development, especially for large high-performance systems, is one the most complex tasks mankind has ever tried. Technological change leads to huge opportunities but challenges our old ways of working.

Processing large data sets, possibly in real time or with other tight computational constraints, requires an efficient solution architecture. Efficiency requirements span from the distributed storage and large-scale organization of computation and data onto the lowest level of processor and data bus behavior. Integrating performance behavior over these levels is especially important when the computation is resource-bounded, as it is in numerics: physical simulation, machine learning, estimation of statistical models, etc.

For example, memory locality and utilization of vector processing are essential for harnessing the computing power of modern processor architectures due to the deep memory hierarchies of modern general-purpose computers. As a rule of thumb, a highly modularized software with distributed state leads to poor performance. At the same time, structure of the system naturally aligns to the communication structure of the organization designing it (Conway's law).

To achieve good design and high performance of complex systems require diverse collaboration with an empirical mindset: rapid validation and ability to change direction when our assumptions become disqualified. This is very different from traditional process-oriented engineering and requires different organizational structure and behavior.

In this talk we share our experiences from creating critical, large, fault-tolerant and high-performing systems during the last 20 years. We've been working with global companies such as HBO, Panasonic, Samsung and Supercell, that are all dependent on high-performing software. We show what has worked, what not and why. We'll cover the topics of productivity, efficiency and quality what are the…