European Grid celebrates ten years

On the 11th of December 2001, the European DataGrid (EDG) project (one of EGI’s predecessors) announced at a meeting that a testbed of the first international grid infrastructure was up and running.

This pioneering grid integrated provided by four sites: Manchester University, and the Rutherford Appleton Laboratory in the United Kingdom, CERN, and CNAF site at Bologna. It wasn't long before they had company with CCIN2P3 in Lyon and NIKHEF in Amsterdam joining them within a week.

Ten years on, the European Grid Infrastructure (EGI) is the world’s largest multidisciplinary grid infrastructure, giving researchers access to state-of-the-art computing facilities across more than 300 sites worldwide.

By late November 2001, the EDG project was only 11 months old and brought together a collection of loosely affiliated computing clusters that people could submit work to by hand. This allowed them to test some of the tools being developed by the project but it was not a real grid. A grid should enable access to distributed remote computational resources when needed by the user but remove the complexity of making the user choose what specific resources to use.

One of the projects involved in EDG was GridPP, the UK’s contribution to the worldwide effort providing computational resources to the LHC experiments. "We had actually done a lot in a very short period of time", explains David Britton, GridPP's project manager at the time, "We had gone from a plan for a proposal in December 2000 to a fully funded infrastructure project, within less than a year. Now EDG and ourselves were about to make the first big step towards a proper grid".

That step was pretty big though; they needed two very important components, an information system and a resource broker. By monitoring the state of the resources on the infrastructure, and taking into account a user’s requirements, these would allow a user to submit jobs "into the grid" and have them automatically routed to a suitable site.

The 11 December announcement was only a humble beginning. The entire system only consisted of 14 machines, Manchester's entire cluster was a single worker node. Andrew McNab from Manchester was part of the integration team that was involved in the work, "It was a pretty exciting time as we were motivated by the idea of connecting hundreds of sites to do computing in a new way. The Integration Team had come to CERN for a fortnight from all across Europe to make it work, and I found out that the CERN and CNAF/Bologna sites had gone live on the Sunday afternoon. I put the final pieces in place on the Manchester site and then late on Sunday night successfully submitted a job to the Broker service at CERN, the job was then run at Manchester. It was like seeing the World Wide Web in the early 1990s: it wasn't very big and things were very rough round the edges, but you knew the technology would scale up enormously and change the way everyone worked.".

In the intervening decade the grid has become indispensable to many communities, the LHC experiments would be lost without the 200,000 machines they can access to do their analysis.