GWDG, the IT provider for the University of Götingen and for the Max Planck Society and partner in MIKELANGELO project, wants to build a system that will enable resource providers to deploy an on-demand big data service on their cloud.
From a technical point of view there are a couple of hurdles to overcome when setting up such on-demand, configurable big data service, as described by Peter Chronz, the technical leader of MIKELANGELO at GWDG:
The first hurdle is the management of big data clusters in a cloud. Here OpenStack Sahara provides a good start, but more elaborate features are required to allow for an end-user driven deployment of big data clusters. Furthermore, since customers have diverse requirements for big data cluster, a high degree of flexibility needs to be allowed. This in turn leads to the requirements for end-users to configure their system by themselves, even without deep expertise in system administration. In addition, we expect that our customers will require a high degree of security, especially with regards to sensitive data. The last technical hurdle is one of efficiency. Although currently it is possible to run a big data cluster in a public cloud, the deployment will be inefficient. There are multiple reasons for this inefficiency, which in the end boil down to low I/O efficiency and poor cloud agility.
These issues lie at the core of MIKELANGELO’s motivation – to improve responsiveness, agility and security of the virtual infrastructure, using the lean guest operating system OSv and I/O-optimised hypervisor sKVM.
An elaboration on the technical hurdles, the concrete technical limitations of current clouds and virtual infrastructures in general and how to overcome them – gain I/O efficiency, faster bursting, better application management, and improved security – described in MIKELANGELO use cases report:
First Virtualised Big Data Use Case Implementation Strategy
MIKELANGELO is funded by the European Union’s flagship research program and innovation program Horizon 2020 and coordinated by XLAB. It involves a consortium of partners with research and technical expertise in Cloud and virtualized environments from Europe and worldwide; Huawei, IBM, Intel, Cloudius**,*** HLRS, GWDG, Ben Gurion University and Pipistrel.*
Project No. 645402
© Copyright Beneficiaries of the MIKELANGELO Project
