WHY DO SO MANY large-scale IT projects go wrong? How can we manage and control the behaviour of systems involving tens or hundreds of thousands of human users, interacting with tens of hundreds of thousands of interconnected computers, such as modern-day financial markets or future digitally enabled national healthcare systems? What happens when these systems grow to involve millions of people and machines, all interacting and adapting?
These are the kind of questions addressed by researchers in the UK national research and training programme - The Large-Scale Complex IT Systems (LSCITS) Initiative. From 2007 until early 2014, the UK Engineering and Physical Sciences Research Council (EPSRC) funded the LSCITS Initiative, which was aimed at developing a new community of researchers and practitioners skilled in dealing with the challenges of ultra-large-scale software dependent socio-technical systems. The funing paid for research conducted in a network of six UK universities by leading academic faculty members, PHD-qualified research fellows and a group of more than 50 PhD level research students, the last of whom is set to conclude their studies in 2016.
The scale and complexity of these socio-economically critical supersystems was increasing rapidly, and there was a genuine concern that the avility to manage and predict their behaviour would not keep pace. This could lead to situations in which the only way we would learn that we did not understand them would be when they suddenly malfunctioned or broke down, with potentially disastrous results. That is, there is a risk that we could find ourselves reliant on ultr-large-scale IT systems that we do not fully understand because of their complexity, and that we cannot effectively manage. Complexity in IT systems stems from their increasing size, the increasing involvement of many different organisations in their construction and use, and the increasing rate of business and social change that these systems have to accommodate. To manage and control complexity, there is a need for better technical tools and methods of systems development. There is also a need for better understanding of the human, social and organisational issues that affect the procurement, development, deployment and use of large-scale complex IT systems.
The LSCITS Initiative's coordinated national network linked industry and academia, providing the skills and knowledge appropriate to dealing with the problems of current and future large-scale complex IT systems across their lifecycles. The Initiative's training programme was intended to contribute to the next generation of systems engineers and technology innovation leaders.
One of the key research questions is then, how are the essential large-scale complex IT system properties maintained in the face of ongoing change and normal failure? Although there is still no simple answer to this, it is clearly necessary to reason at different levels, such as the implementation level, the intermediate level of development and operational processes, and the higher level of organisational dynamics.
In total, approximately 250 person-years of research will have been conducted by members of the LSCITS Initiative by the time that the programme concludes.
PUBLIC ENGAGEMENT AND OPEN-SOURCE RELEASES
The LSCITS Initiative team at the University of St Andrews has led the creation of the St Andrews Socio-Technical Systems Engineering Handbook, an online archive of key papers and overview/commentary articles in sociotechnical systems engineering (available from http://archive.cs.st-andrews.ac.uk/STSE-Handbook/index.html).
Via releases under creative commons licences on the SourceForge and Github source-code repositories, the University of Bristol group has made available the software necessary to replicate and extend their published experiments studying interactions between automated trading systems and human traders, and also exploring the dynamics of markets populated entirely by automated trading systems of varying types. Marco de Luca’s open Exchange system is available from (http://sourceforge.net/projects/open- exchange/), Dr John Cartlidge’s Exchange Portal system is available from (http:// sourceforge.net/projects/exchangeportal/) and Professor Dave Cliff’s BSE system is available from (https://github.com/davecliff/ BristolStockExchange). The combined total number of downloads of these three software packages is now more than one thousand.
The Bristol and St Andrews teams have also released source code used in their cloud- computing research. This includes Owen Rogers’ (Bristol) software for exploring financial brokerage and options contracts in the pricing of cloud computing services (http://sourceforge. net/projects/cloudoptions/), James Smith’s (St Andrews) Cloudmonitor system for predicting power usage in cloud computing systems (https:// github.com/jws7/Cloudmonitor) and the Cloud research Simulation Toolkit (CreST), developed at Bristol by Cartlidge, Cliff and various LSCITS Initiative-funded undergraduate interns (http:// sourceforge.net/projects/cloudresearch/).
Teaching material for core taught courses on the LSCITS Initiative Engineering doctorate training programme have also been made available on the web: Ian Sommerville (St Andrews) has released the slides for his course Systems Engineering for LSCITS on the web at (http://www.software-engin.com/teaching/ systems-engineering-for-lscits), and the slides for his course Socio-Technical Systems Engineering at (http://www.software-engin. com/teaching/socio-technical-systems-engineering). Cliff has added several pages to the LSCITS Initiative website from which it is possible to download the slides, reading lists and video lectures for the Technology Innovation LSCITS Engd module, which includes guest lecturers from industry talking about related technology innovation at Google, Hewlett Packard Labs and also in start-up companies, see: (http://lscits.cs.bris. ac.uk/techinnovation.html).