Today, the 6th of June 2012, is the official World IPv6 launch – a day organised to raise awareness and improve the deployment of IPv6. This new technology will extend the current Internet system to allow more and more devices to connect to the worldwide community. One of the worldwide events is taking place in Amsterdam and Steven Newhouse, director of EGI.eu, will be discussing EGI's plans for IPv6.
The first results have been published from a public challenge to automate the process of determining the shape and structure of a molecule from NMR data. Three of the teams who took up the challenge have now ported their work to the grid.
Today the European Grid Infrastructure (EGI) launched their new online video series “Stories from the grid” at the International Symposium on Grids and Clouds in Taipei. The first episode is about how a component of the toxic venom used by the cone snail is being modified using the aid of computer models to help produce new anaesthetics and alleviate the muscle spasms caused by the condition dystonia.
EGI is a pan-European project providing access to computing resources for researchers through a distributed computing infrastructure called a grid. The diverse range of science supported by EGI is being showcased in the series of short online videos. Each episode focuses on a particular piece of research that would have been near impossible without EGI.
The first video looks at how Henry Hocking from the CONCO project (see www.conco.eu) has used the grid to analyse naturally occurring molecules in venoms used by marine snails to immobilise their prey. They hope to be able to use their work to synthesise an artificial molecule that specifically targets and blocks the transmission of pain signals to create better muscle relaxants that have anaesthetic properties. “At the end of the day we just want to get our work done,” explains Henry, ”but science has changed and computing has become integral to what we do on a daily basis. Without the resources provided through EGI we would not be at the stage we are today.“
EGI prides itself on providing expert computing without the need to be a computer expert. So to help CONCO use the grid, EGI worked closely with the WeNMR project. “We have been working on the European grid since 2009,” explains Alexandre Bonvin from WeNMR, who is also featured in the video. “In that time we have developed tools to ensure that scientists coming to us get up and running as soon as possible. They are not interested in having to spend weeks getting to grips with a new technology, they just want their results.”
The video was premiered during Alexandre Bonvin’s keynote speech at the International Symposium on Grids and Clouds in Taipei this morning. The first episode of Stories from the grid, “The cone snail and the search for powerful new anaesthetics”, can be seen at http://go.egi.eu/conco
On the 11th of December 2001, the European DataGrid (EDG) project (one of EGI’s predecessors) announced at a meeting that a testbed of the first international grid infrastructure was up and running.
This pioneering grid integrated provided by four sites: Manchester University, and the Rutherford Appleton Laboratory in the United Kingdom, CERN, and CNAF site at Bologna. It wasn't long before they had company with CCIN2P3 in Lyon and NIKHEF in Amsterdam joining them within a week.
Ten years on, the European Grid Infrastructure (EGI) is the world’s largest multidisciplinary grid infrastructure, giving researchers access to state-of-the-art computing facilities across more than 300 sites worldwide.
By late November 2001, the EDG project was only 11 months old and brought together a collection of loosely affiliated computing clusters that people could submit work to by hand. This allowed them to test some of the tools being developed by the project but it was not a real grid. A grid should enable access to distributed remote computational resources when needed by the user but remove the complexity of making the user choose what specific resources to use.
One of the projects involved in EDG was GridPP, the UK’s contribution to the worldwide effort providing computational resources to the LHC experiments. "We had actually done a lot in a very short period of time", explains David Britton, GridPP's project manager at the time, "We had gone from a plan for a proposal in December 2000 to a fully funded infrastructure project, within less than a year. Now EDG and ourselves were about to make the first big step towards a proper grid".
That step was pretty big though; they needed two very important components, an information system and a resource broker. By monitoring the state of the resources on the infrastructure, and taking into account a user’s requirements, these would allow a user to submit jobs "into the grid" and have them automatically routed to a suitable site.
The 11 December announcement was only a humble beginning. The entire system only consisted of 14 machines, Manchester's entire cluster was a single worker node. Andrew McNab from Manchester was part of the integration team that was involved in the work, "It was a pretty exciting time as we were motivated by the idea of connecting hundreds of sites to do computing in a new way. The Integration Team had come to CERN for a fortnight from all across Europe to make it work, and I found out that the CERN and CNAF/Bologna sites had gone live on the Sunday afternoon. I put the final pieces in place on the Manchester site and then late on Sunday night successfully submitted a job to the Broker service at CERN, the job was then run at Manchester. It was like seeing the World Wide Web in the early 1990s: it wasn't very big and things were very rough round the edges, but you knew the technology would scale up enormously and change the way everyone worked.".
In the intervening decade the grid has become indispensable to many communities, the LHC experiments would be lost without the 200,000 machines they can access to do their analysis.
The MAPPER project (Multiscale Applications on European e-Infrastructures) has joined the official list of EGI partners signing a Memorandum of Understanding in August. This agreement ensures that the solutions developed by MAPPER and its partners are compatible with the European grid.
This is another step towards making EGI an integral part of the computing resources offered to European researchers and their international collaborators.
“MAPPER is a science driven project with the aim to exploit existing European e-infrastructures for a new mode of computing,” says project coordinator Alfons Hoekstra. “To do so MAPPER has brought together a strong European consortium covering the chain from the infrastructure to the science applications, and by closely collaborating with EGI and PRACE.”
The main goal of the MAPPER project is to develop “an environment for distributed multiscale computing, consisting of dedicated programming and execution tools, and production quality services that are needed to support this mode of computing,” explains Hoekstra.
The MAPPER focus on multiscale is important given that many areas of research focus on investigating phenomena at every level of the system, from the microscopic to the global and everything in between. Climate and weather modelling are important examples, being able to take into account the global state of the weather, but also local conditions, is important in understanding how the entire system works and its possible future direction.
The MAPPER project, started in October 2010, aims to develop tools, software and services to maximise the usage of e-infrastructures by this community.
Starting with 5 specific areas - fusion, clinical decision making, systems biology, nano-science and engineering - they will help each create the tools needed for a specific problem within that discipline. They will ensure that the solutions being offered to researchers are not only appropriate but also provide access to them.
The software provisioning team at the European Grid Infrastructure (EGI) has released the first version of the Unified Middleware Distribution (UMD-1). This is a major milestone in providing a consistent platform for all researchers in the European Research Area to gain access to suitable and integrated computing resources.
Every research community has different goals and requirements for their computing and data needs. Many have developed and maintained their own solutions, which work for them. For many years, the European grid has been offering researchers a distributed computing system across different middleware. The goal of the UMD is to provide a system that the community’s existing solutions can easily plug into, not replace, so that these solutions can be deployed at scale across Europe. It will offer a set of well-defined, stable and general-purpose software components to meet their needs.
So for the last year, EGI and its partners have been working with technology providers and users to work out what they need to offer as a sustainable base on which users can build. The chosen components are collectively known as UMD and have been verified to work in the environment used within EGI. Leading the work has been Michel Drescher; EGI’s Technical Manger; he is excited by what they have achieved: “Just getting this far has been a mammoth task. We have consulted a huge range of stakeholders, reviewed every software package on the grid and come up with what we believe provides the best support structure we can. This is however only a first step, we will learn a lot from this initial deployment to improve future releases”.
UMD 1.0.0 is the first release of UMD-1 for the European Grid Infrastructure. This initial release contains the most critical products from EGI's Technology Providers, as agreed by the Technology Coordination Board on behalf of the EGI community. Further versions will be released in the near future incorporating more features, including the full EMI (European Middleware Initiative) software stack and IGE (Initiative for Globus in Europe) components.
Earlier this month EGI.eu signed a Letter of Intent with two European Commission funded projects aiming to provide the arts and humanities with an e-Infrastructure of their own. The two projects, DARIAH and CLARIN, both signed the three-way agreement, which has the express intention of ensuring that technology developed by the two projects and the European Grid Infrastructure (EGI) are compatible and provides the best service to their users. The agreement also provides a blueprint for similar arrangements between EGI.eu and other user communities that are interested in exploring grid technology but are temporarily constrained from establishing a Virtual Research Community.
While there is a wide range of research using the European grid, the arts and humanities are currently under represented but both DARIAH and CLARIN aim to remedy this. CLARIN (Common Language Resources and Technology Infrastructure) is focussed on creating tools and resources for the language community, while DARIAH (Digital Research Infrastructure for the Arts and Humanities) supports the wider humanities and arts community. The two projects are part of the European Strategy Forum on Research Infrastructures (ESFRI) programme. This initiative helps to support a coherent approach to policy-making on research infrastructures in Europe so that researchers get the resources/technologies they need.
This latest agreement cements the relationship between the 2 ESFRI projects and EGI.eu, helping all three to develop common tools and technologies while exploring further opportunities for collaboration. Connecting EGI with the infrastructures developed by these two projects will have benefits for the sites within partner countries providing the resources, benefits for the countries themselves in terms of integrating the resource centres with the EGI monitoring and support services and finally, benefits for the research communities in terms of support, advice and influence over the evolving infrastructure. The first priority however is to understand the needs of the communities that DARIAH and CLARIN represent so that they can build and manage an infrastructure for their users. The areas of greatest concern are; data storage and availability, the hosting and monitoring of services as well as authentication and authorisation issues.
Chief Community Officer, Steve Brewer said: “For EGI this is the first step in establishing and welcoming a valuable new user community onto the European grid, DARIAH and CLARIN represent a broad range of interests across the arts, humanities and social sciences with, between them, a wealth of digitally-enabled methods and tools to integrate with the infrastructure.”
Outside the agreement with EGI, CLARIN and DARIAH have also agreed to co-organise the SDH 2011 conference in Copenhagen later in the year. The meeting, the full name of which is “Supporting the Digital Humanities: Answering the unaskable”, will be held in the Danish capital on the 17th and 18th of November. It will provide a forum for discussing the benefits of the e-Infrastructures to the arts and humanities alongside how new forms of research can be facilitated and supported.
One particular research area that benefits hugely from easy access to computational resources is the analysis of medical images, like brain scans or x-ray images. In a paper published in this month’s Nature Reviews Neurology various infrastructures on offer to the biomedical community are compared, including EGI’s partner neuGRID.
Research into neurodegenerative diseases uses imaging techniques to help diagnose, and track the progress of, numerous illnesses including Alzheimer’s. Until recently however the image collections used by the researchers have been usually collected locally, and only being 10s or 100s of images. However thanks to a growth in the availability and accessibility of clinical and research imaging data, the area is being transformed.
The teams now working on these illnesses have access to data sets made up of literally hundreds of thousands of individual images. Traditionally only a few laboratories had the expertise and computational resources required to make use of the data. However recent developments in e-Science have changed this and there are many solutions available to the community.
The paper is a review of the three major projects helping overcome the image overload; neuGRID, LONI and CBRAIN. All three use different techniques, technologies and image sets, making them equally useful to different researchers. LONI and CBRAIN focus on using High Performance Computing resources while neuGRID is based on the technology provided by EGI as well as using the computational resources of the infrastructure.
David Manset, from the neuGRID project and Technical Coordinator of its international chapter outGRID, is one of the paper’s authors “Putting this paper together has been a great experience. neuGRID has achieved a lot in the last 3 years and it has been interesting learning about the differences between our approach and the others. I also hope that the paper raises awareness of the tools on offer to the community by the three projects”.
The paper, Virtual imaging laboratories for marker discovery in neurodegenerative diseases, is published in July’s Nature Reviews Neurology – Doi:10.1038/nrneurol.2011.99
neuGRID launches its new phase later this month called “neuGRID for you (N4U): expansion of neuGRID services and outreach to new user communities”, led by Giovanni Frisoni, who was also principal investigator of neuGRID, which will aim to expand the services offered by neuGRID to their end users.
Last week the StratusLab project launched a production release of their eponymous software, StratusLab 1.0. This first stable distribution makes it even easier to turn available computing resources into a cloud environment, and in the process lowers one of the barriers to providing resources to the European grid.
The team at StratusLab has spent the last 12 months working with users and system administrators to provide a set of tools for creating and managing a private cloud-based infrastructure. For users still unsure about moving to the cloud the project also provide a reference infrastructure running their latest tools. This allows potential users to see what they can do with the system before investing time and effort in setting one up.
One of the clear benefits of this work for the European grid, however, is the ability to create and run a grid site on a StratusLab cloud. In fact the Greek National Grid already has one of their sites up and running like this using a previous version of the software. This makes it easier for institutes with limited resources to join and contribute to the European Grid Infrastructure (EGI).
Dr Charles Loomis, the Project coordinator, has been very happy with the progress: "thanks to everyone pulling together we have been able to ship four beta releases and this production release in our first year, which is brilliant. Alongside the speed of development is the quality of our product, demonstrated by the Greek NGI using our software to run a certified EGI grid site in the cloud. I'm looking forward to the next 12 months, which will bring a series of incremental releases featuring improvements to existing services and added functionality."
It is not only the developers who are happy with StratusLab, researchers from various disciplines have been interested in the benefits that this work could have for them. One of these, Christophe Blanchet of the Institut de Biologie et Chimie des Protéines (IBCP) France, hopes to see the work become a standard tool for bioinformatics research: “French bioinformatics platforms are planning to use StratusLab to help the deployment of their site inside our national grid infrastructure. This will also allow them to use the flexibility and elasticity of the cloud to provide bioinformaticians and biologists with a simple way to access the bioinformatics services and applications needed to analyse the massive biological datasets normal in modern research."
Currently the software offers the benefits of both systems, from cloud’s ease of adding new resources to grid’s openness and wealth of applications. The team are not finished yet though. They will continue to add more functionality to StratusLab, including the ability to interface with commercial cloud services.
The key features of StratusLab v1.0 are:
It is a full production-ready release with many bug-fixes and enhancements
Uses the OpenNebula 2.2 virtual infrastructure manager
includes the Claudia platform service management toolkit for dynamic service provisioning and scalability
has a marketplace allowing user communities to create, search for and share customised digitally-signed virtual machine images
numerous Base OS images are available in the Marketplace as well as grid and bioinformatics appliances
persistent storage allows users to create and access disks on remote cloud storage resources
provides a sophisticated web-monitoring tool
has easy-to-use command line tools for system administrators and users
uses an authentication proxy service which allows cloud authentication via X509 grid certificates
provides both manual and Quattor installation options
EGI signed a Memorandum of Understanding (MoU) yesterday with the gSLM project to collaborate in bringing EGI’s Service Level Management in line with industry best practices.
The EGI project is interested in giving European scientists, and their international collaborators, access to a sustainable distributed computing service. However, the European grid has many stakeholders; resource owners, users, and administrators, and it can be difficult to bring all these viewpoints into focus. Thomas Schaaf from gSLM explains, “Even once you reach agreement on what needs to be done and how, it still isn’t easy. Documenting, managing and policing rights and responsibilities in a fair but enforceable way is extremely complex in an environment like the grid. We hope to help EGI improve their Service Level Management by bringing lessons learned by industry to bear on the problem”.
The MoU, signed at ISC’11 in Hamburg, formalises the collaboration between EGI and gSLM so that the two can work together to provide concrete plans that can help shape the future of the infrastructure. Sergio Andreozzi has been working with gSLM to define the details of the collaboration plan, “Leveraging the experience and the best practices of Service Level Management from the commercial sector is just one more step towards improving EGI overall services. The gSLM project will bring the needed expertise to understand what can be improved and how”.
Some of the most important results of the collaboration will include:
Defining everyone’s role within EGI in relation to delivering a service to EGI users. This will mean that everyone within the project understand their place within the schema along with what is expected of them and what they can expect from others.
Creating a list of the current obstacles and requirements that the service providers have for providing services to users. EGI needs to know what are the issues for resource providers, this list will allow them to tackle these problems and hopefully make supporting new users as easy as possible.
Discovering the obstacles and requirements that existing and potential users have accessing grid services. Solving the problems being encountered by users now will help the project understand what needs to be done to make the grid more attractive to new users.
Building a training programme to improve the project’s IT Service Management. For EGI to gain from the collaboration with gSLM the lessons learned need to be disseminated as wide as possible. The training programme will do this alongside as explaining the new processes being used by the project.
The activity will start immediately by harmonising the terminology used and planning for a training session on IT Service Management at the next EGI Technical Forum.