Surviving Change

Ever increasing complexity means that people charged with maintaining production systems face an impossible task: they need to spend time developing new ways to manage their platform but are constantly fighting fires which prevent them from concentrating on the real challenges at hand. In essence, the solution to the problem is a human one. These issues are not unique to systems administration, so this paper takes a far reaching approach to addressing them. Expertise from software development, military operations, the scientific revolution, and even NASA are used. Three themes are dominant: we need a systematic way to learn from our experiences, bridging structure and flexibility is key, and above all, it is leadership which makes the difference. Using procedures for specific events offers a breakthrough approach. A formidable tool is created when procedures are used in a way that resists bureaucracy while encouraging knowledge transfer and learning