.png)
by Achi Rotem, K2View CTO
If we had it to do all over again, we’d never architect enterprise systems the way we did 20, 10, or even five years ago, especially when it comes to data management. Why not? Take a typical Fortune 500 company, for example. It might have customer data in 60 or even 600 different systems, each containing bits and pieces of that data, or worse, mass duplications of it. And that’s just customer data, to say nothing of all the other business entities which these systems maintain.
With so much data scattered over all those systems and data sources, data management these days is no longer, well, manageable. Even with massive data integration projects, no company can possibly hope to keep the data synchronized across all those systems.
It gets worse. With every merger or acquisition, the number of legacy applications and their associated data can easily double or triple. Big Data initiatives—the collection and storage of every possible scrap of info for each transaction, phone call, data transfer, network incident, customer complaint, and more—massively increase data storage requirements and data fragmentation across multiple data warehouses and lakes. Add to that the fact that organizations are moving big chunks of their data to the cloud—private, public, hybrid and even multi clouds—and the complexity of the enterprise data landscape becomes nearly impossible to manage.
All this makes it increasingly difficult to provide excellent customer service to every single customer—and to earn their loyalty. To keep them from churning, an enterprise must deliver new services faster, and they must provide exceptional experiences through every encounter and every channel. With hundreds of systems to deal with and petabytes of data to sift through, organizations simply can’t move fast enough to provide the truly personalized experience today’s customers demand.
To truly deliver on the promise of a holistic 360-degree view of our customers and every other business entity we care about, we need to fundamentally change the approach to data organization and management. For years now, K2View’s approach to this has been to store all available information for each individually identifiable business entity in a separate logical unit. Now that the U.S Patent and Trademark Office has issued K2View a patent for our concept of a logical unit and its innovative use of micro-databases, a generational shift is underway.
Let’s take a look at earlier approaches to solve this problem, and why proliferation of big data doomed those efforts to fall short of our expectations. Then we’ll see how an innovative use of micro-databases not only solves these problems but will usher in a whole new approach to enterprise application development.
CTO Achi Rotem displays the patent awarded to K2View
The limitations of systems integration, data warehouses and lakes, and big data analytics
Attempts to integrate systems and collect masses of operational data just made data management a bigger nightmare. So, to bring all the disparate yet related data together in one place, we’ve turned to ever bigger and bigger data warehouses. Yes, we wanted a way to analyze the data to sense macro patterns and trends. But big data warehouses also promised to provide a single source from which to query everything about an individual customer. Or an order, a shipment, a network switch—whatever uniquely identified individual “unit” is most important to us in a given situation.
While data warehouses may have delivered on macro analytics of past collected data, they can’t, by design, provide up-to-the-second, 360-degree views of a customer or anything else.
Ever wondered why you’re put on hold by customer support while they “bring up your account information?” If their system has to query multiple systems to collect the info, it can take 15 seconds to 2 minutes - or longer. Each of those systems, while purportedly up-to-date, contains an enormous amount of data. (A telecom’s billing system alone, for example, can generate tens of millions of invoice records every month.)
If the support system queries a warehouse, it has to sift through even more sets of combined data, just to find the information on that single customer. If it relies on a distributed data lake, the data for that customer may exist in ten, 100, or 1,000 nodes, making the possibility of true real-time access just as remote. And then, once the data comes back, there’s always something that isn’t quite current, right? That’s because data warehouses and data lakes are only as up-to-date as the last batch refresh from hundreds of application data stores. At any given time, the customer’s data could be stale or out of sync with other data.
If we could only start over: The Logical Unit and the micro-database
These aren’t new issues, but until recently they’ve been nearly impossible to solve. Fortune 500s can spend years and 10s of millions of dollars and still fail to achieve true, real-time, 360-degree representations of the data they care about. That’s because every enterprise application has its own view of the world, its own data model, and its own “APIs” that allow external integration to access—but not own—its data. Which makes huge integration projects fail and, presumably, data warehouses and data lakes necessary. Yet these still can’t deliver the exact data organizations need, when they need it.
Getting real-time insights into individual customers and other entities requires a data-centric approach—one that doesn’t require complex integrations and synching disparate application data stores or sifting through massive data sets.
That’s why our recent patent for the innovative concept of a logical unit, and its associated micro-database technology, is so important.
Some years ago, K2View came to realize we needed a fundamental shift away from traditional application-centric data organization to a data-centric model. Using our patented approach, K2View Fabric can store all information about each customer (or other entity) in its own micro-database, and it can keep that information up-to-date with minimal impact to the systems from which it comes. That enables us to integrate, organize, store, access, and secure our customers’ data, delivering a holistic, 360-degree view of customer and other business data in real time—no massive integration or data warehouses required.
The Logical Unit, the micro-database, and 360X – It’s just the beginning
K2View Fabric is a thin layer that sits atop your existing application and database infrastructures, no matter how large or small. Using our patented logical unit concept and innovative micro-database techniques, we can bidirectionally connect to your existing data topology in weeks, not years. That provides holistic, real-time 360-degree views and control to whatever business entities are most important to your organization.
We call that “360X,” where “X” is an individual customer, an order, a shipment, network circuit, practically anything for which you have—but have never been able to organize and operationalize—the data.
With 360X, your organization can have a 360-degree, data-centric view today, even if all your systems still maintain an application-centric model. This means you can perform micro-analytics for each individual customer (or other “X”) on demand, so you can provide them exactly the experience they need right now.
For example, before a customer service rep even answers the phone, you might calculate the probable reason for the call and the customer’s likelihood to churn, based on the phone number and the customer’s current service status and history. You might determine the likelihood they’ll be interested in increasing their internet speed, upgrading their mobile phone to the latest model, or enrolling in paperless billing. You might even automatically route the call to a rep whose personality best fits the caller. Having every scrap of information about an individual customer in one place lets you answer hundreds or even thousands of questions about them in just milliseconds.
But 360X is just the beginning for this innovative application of the micro-database.
I said earlier, today’s enterprises would design their databases very differently than even a few years ago, given the chance. And as K2View customers begin to see the enormous advantage of our micro-database approach to data management, they’ll start to wonder: Why shouldn’t this be my system of record, rather than 200, 500 or 1,000 disparate systems, all vying for the privilege?
Indeed, that may be the tipping point where enterprise application developers also see that in a data-centric world, there is no longer a need to maintain closed, proprietary representations of customers—or of any “X”. They need only call a microservice—K2View Fabric provides these already—to retrieve or update “X” from or to the micro-database, instead of a local datastore. The result? Developers can focus on providing superior functionality, not on owning the data.
This new concept of data ownership becomes even more revolutionary in the contexts of data security and data privacy, such as CCPA and GDPR. Not only are data breaches at an all-time high, but more and more countries and states are regulating how companies are allowed to use and store personal data. A micro-database approach addresses both issues. Because we encrypt each micro-database using a different key, even if millions of micro-databases were leaked, the thief would have to break a key for every single customer’s data. As for data privacy, we deliver 360X data from the micro-database via microservices in real-time and only as needed, without the requesting application having to store it anywhere at all.
So, get ready. K2View’s patent for its micro-database technology will do more than just create a fundamentally new approach to data organization and management. It’s about to herald an entire generational shift in enterprise application architecture and design.
