` October 14th 2016 | DevOps, Infrastructure-as-Code, TOSCA, Cloudify, Apache Storm
The first release of DICE tools has a lot of great features, the one we are most excited about, here at XLAB, are the DICE delivery tools, which simplify and speed up the development of big data applications.
When developing applications that use big data technology, such as Apache Storm, Spark or Hadoop, a number of time-consuming steps are needed to install, configure and start the application. Traditionally, developers and administrators would need to study the documentation for installation of these technologies then struggle through trial and error before succeeding in setting them up. For any changes and upgrades, the process would have to be repeated, causing further delays.
Instead, DICE recommends the DevOps approach for agile development, operations and quality assessment to enable building fast, resilient systems at scale.
To automate complex and repetitive tasks, the development team first describes the big data application in a TOSCA blueprint document, a text document which can be stored and maintained in the team’s version control system repository, together with the application’s code. The deployment itself is not a manual process, but an automated one thanks to the open source cloud orchestrator, Cloudify. Cloudify is used as base technology, bringing the blueprint to reality, and works with any private or public platform, which is also one of the deciding factors for its adoption. At this level, DICE provides ready-made packages of libraries, employing application and configuration managers such as Chef.
To show how to easily deploy a big data application, we provide the example of Apache Storm in the DICE delivery tools demo:
Credits:
Script by Matej Artač
[email protected] | LinkedIn
Voiceover by James Lopez
[email protected] | LinkedIn
Recording by Alja Leskovar
[email protected]
All DICE building blocks, the components, subcomponents and tools are described in more detail at DICE official site and at GitHub.
DICE tools were developed with funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 644869.
Consortium is comprised of experts from academia and industry working in areas such as quality assurance, agile software engineering, and model-based development: Imperial College London (coordinator), Polytechnic University of Milan, IeAT, Flexiant, ATC, Prodevelop, Netfective, University of Zaragoza and XLAB.
Tweet: Focus on what matters – Your #bigdata #application, leave #configuration #installation to @diceh2020 #DevOps tools goo.gl/0BnxkP
`
