NextGen Storage for Next Gen Apps
Some applications don’t use much source data, and may not even generate much output data. Instead they intensively process a limited amount of data. But for the most part, modern applications – especially the high-performance ones we call next-gen apps – crunch their way through a lot of data.
They crunch through so much data, and so many kinds of data – that next-gen apps call for NextGen storage to handle it all. Big Data has become a big buzzword. Big Data is not only big, it is complex. Indeed, its variety and complexity, even more than its sheer volume, is what sets Big Data off from older, conventional databases.
Data: Structured, Unstructured, and Demanding
Structured data, such as account records and other types of standardized records, remains a widespread and important class of data. This means that the familiar relational database and the SQL language for working with it, are anything but dead. Relational databases and SQL queries are going to be with us for a long time to come.
But next-gen apps, and therefore the storage that serves them, also now have to deal with vast new quantities of “unstructured” data.
This data is not truly unstructured. If it were, it would be incoherent and useless. Network traffic data, or sensor data from the “Internet of things,” conforms to its own structural rules. Even social media messages, such as tweets, follow grammatical rules and other rules that allow humans to write them and read them. But these structures do not easily fit the traditional relational database.
Storage in the Sky
Another key feature of next-gen storage is that much of it, perhaps most of it, is being stored in the cloud. And even though everyone still speaks of “the” cloud, in reality this means multiple clouds. The number of cloud vendors continues to grow, offering public, private, and hybrid clouds. These clouds exist on every scale from individual user storage to vast data centers.
All of this means that data storage is now as complex and dynamic an environment for DevOps and IT to work with as the hosting for application source code. Distributed data warehousing has become the name of the game.
Today’s application performance monitoring solutions must therefore be able to handle database architectures ranging from Cassandra and MySQL to VoltDB, and they must work with frameworks and languages, such as Flume, Hadoop, MapReduce, and Thrift.
The good news for the DevOps and IT communities is that next-gen tool sets capable of handling these tasks are now available.
Image Source: Flickr
- Twitter: @rocketpunkrick