Welcome to the SkyLark Monthly TechReview
Each edition will cover exciting new developments and success stories in the ecosystem of technologies we use here at VirtusLab.
This first issue focuses on the relatively new concept of Fast Data, also known as Big Data Analytics. But what does this actually mean and what good is it to you?
To make things clearer, we explain what it's all about and give some real-life case scenarios and recent success stories. Hopefully, this will whet your appetite for the exciting potential of this new technology.
If you have any suggestions for future newsletters or you'd like to get in touch for any reason at all, please mail info@virtuslab.com.
|
|
What's all this talk about Fast Data?
Fast Data is the logical progression from Big Data, which, put simply, means data sets that are too large or complex for traditional processing techniques. So, what do we do with this data?
The is where Fast Data comes into play. As explained in more detail here, Fast Data is the application of Big Data Analytics in near-real or real-time, in order to create business value. After all, having all this data is pointless if we can't do something useful with it, right?
Here's a short YouTube video by a very charming guy that sums it all up very nicely:
Big Data as Fast As Possible
 |
|
So what can Fast Data do for you?
So far these concepts might seem rather abstract, but Fast Data has massive real-world implications for all of us.
Imagine you work at NASA and have an ongoing mission to Mars. You've launched the space ship, and it's happily winging its way towards Mars, already millions of kilometres away from Earth. As well as your precious astronauts, this ship contains state-of-the-art computer and communications technology which can transmit back to earth data about the state of all the on-board systems, conditions inside and outside the craft, as well as crucial data about the health of the astronauts.
The amount of data generated is massive. It's constantly streaming into your Mission Control Data Centre, but it's impossible for any human or traditional technology to do anything more than store it. It needs to be analysed right now; even batch analysing it tonight might be too late to prevent a catastrophic problem. We're not just talking about simple variables such as the temperature of the hull, or the fuel level, but complex interactions between factors that could reveal other more complex problems.
So, we use Machine Learning to find problems in real time by searching for patterns and correlations that have previously been shown to be dangerous.
This is Fast Data at work!
|
|
|
|
Your astronauts landing safely on Mars, assisted by Fast Data.
|
|
Fast Data and us
As Scala experts, we love Apache Spark as it works beautifully with Scala and is up to 100x faster than Google's Hadoop. Fast Data is used by many organisations to gain a competitive edge, or simply to achieve what was previously impossible.
Virtusity, our training division, offers regular Spark and Scala training in Krakow in association with Lightbend Inc., the company behind Scala and the Typesafe Reactive Platform. |
|
It's already happening!
The mission to Mars scenario is imaginary, but Fast Data techniques are already being used at the NASA Center for Climate Simulation (NCCS) and you can be sure that it'll be a crucial factor in NASA's success when they finally do launch their Mars mission.
|
|
|
|
|
The Search for Extra-Terrestrial Life
In keeping with the cosmic theme, the SETI Project, which searches for signs of extra-terrestrial life by analysing signals from space, is already using fast data to increase their chances of success.
If you've never heard about the SETI project, here's some big news from last year that makes Fast Data a big issue for them going forward.
And here's a more recent article about how they use Spark to sift through massive amounts of data.
|
|
|
Fast Data in agriculture
Modern farms are bristling with high-tech machinery that constantly spits out huge amounts of data. Rapid analysis of this data leads to higher productivity, lower costs, and reduced risk of failure.
See how Monsanto is meeting the challenge of data analytics to help farmers and boost profits.
bloomberg.com
|
|
|
|
Enormous potential in the healthcare industry
Data Analytics has enormous potential in many areas of healthcare such as analysing and preventing disease outbreaks, monitoring hospital data in real time, or reducing inefficiency and fraud.
indiatimes.com
|
|
|
Rapid growth forecasts
Fast Data is just getting started, but big players like Airbnb, Lyft, Yelp and Splunk are already doing Big Data in the cloud, and 2000% growth is forecast over the next decade.
siliconangle.com
|
|
|
Big Data and Machine Learning
The science of training machines to learn is finally coming of age, but how to feed these data hungry beasts?
datanami.com
|
|
|
|
|
|
