The installation of wikis and quality control of the wiki software is still generally done by hand. Obviously, this increases the costs for the customers. And the software producers do not find this repetitive work much fun either.
Docker, Puppet, Ansible: a wealth of new control systems
It is not remarkable that the topic of automatising software development was raised at last year’s MediaWiki conferences. The crucial point, however, is that there are many new technologies available. They have been developed in the cloud environment, and are available to distribute and maintain large amounts of servers and application software.
- Composer: A packet-administration system for a PHP script, similar to the extension installer in WordPress.
- Docker and Vagrant: software to create and control “virtual machines”.
- Puppet, SALT, Ansible, Chef: control systems which configure the system environment and processes.
These technologies revolutionise the whole area of software development. More precisely, they revolutionise the development of standard software. Like in a factory, the software is created, tested and delivered at the touch of a button. Distributions like Red Hat or SuSE create their distributions in this way. But the Wikimedia Foundation manages hundreds of wikis via Puppet and other control system too.
BlueSpice Factory – the new backbone of the software development
This development has not passed BlueSpice by. Under the title BlueSpice Factory, we are building a homogeneous, fully automated production and testing environment for BlueSpice. New builds of the software will be created, packetised, automatically tested and archived on a daily basis. BlueSpice Factory will become the backbone of our development with the delivery of BlueSpice 3 at the latest.
This is a big step for the stability of the product. Because as far as possible, BlueSpice Factory will automate the work-steps and make them uniform. In addition, many more automatic tests will take place via the system than today. Factory is to take over packeting, create free and pro packages, perhaps RPMs, and perhaps discard and file packages automatically. And, last but not least, product improvements should be able to be rolled out to the user more quickly.
We are also considering the idea that BlueSpice Factory could take on delivery in live systems and importing patches. Technologically, this is not so difficult. We are trying it out first with our own internal wikis.
The customer’s environment is decisive here: are the necessary technologies available? And how should a possible updating process be organised? After all, the customer should keep control over the updating process.
Our task is to offer as stable a solution as possible for the maintenance of wiki systems. So we take technology which is already being used in other places and make it available for the wiki world outside the Wikimedia universe.
In search of the right technology stack
The current challenge for us is getting the right combination of the tools available. Additionally, the steps we have to take on the production lines have not been definitely settled.
Currently we provide a basic system with Puppet. On this basis, we collect together everything we need for a wiki, the up-to-date software components and install a complete wiki. We are currently sorting out what should be tested when. The preferred solution is to call up the source with Puppet or the versioning management system Git. We build a package with a build script – which is also a Puppet system, install the package and then test it in the order: unit test, performance test and Selenium test. All this happens automatically. After this, the system is enabled. After the system is enabled, the next step is to distribute the package to the customers’ systems. This can also take place over Puppet. The whole process is orchestrated by the integration software Jenkins (Sauce Labs).
One can see that the exact constellation takes in a very wide range of technology. And because the whole field of software automation is very dynamic, we can be sure that there will be some changes here too. However, we have to accept the danger that we will have to change everything later on, just so that we can begin and gain experience. One cannot develop automation technology on the drawing board.