Shadow Analytics – where end-users are extracting data, doing calculations on and relying on that data, and storing and using it in ways that don’t necessarily tie back to the data origins very well – came from the need to move more quickly than their IT teams can move in allowing access to data and information.
Here’s some more information about what this is all about.
I caught myself chuckling a little, thinking about that long-time conversation so man have had about “departmental” databases. It used to be a consistent struggle to work with Access databases and small SQL Server installations throughout a company. We (collective we here) would preach about these systems that we’d find and the fact that they needed to be managed – backups, controls, recoverable, etc.
It’s time to be thinking about getting in front of this (or as front of it as we can at this point), otherwise we risk some real vulnerabilities to our information systems. I say that because those issues of the departmental database are back. In fact, I’d say the issues are more acute because people are relying even more on their data sets and extractions…. and sharing them with others.
For a bit now there has been a good deal of coverage here about education for those accessing data stores. Explaining about USB keys, data extracts, security of information, reproducibility or results and calculations. If an audit were to happen, how would the process go? These same issues both require education, and need controls.
These make it ever more critical to mask critical information so the user simply never has it. Typically it’s not required for these types of uses, so the impact to functionality will be slight, if not non-existent.
Some excellent points in that post linked above, but this was a key point when trying to consider where to start:
“The analytics required really depends on the department or unit but you need an infrastructure layer on which models can run,” said Ramkumar Murali, Practice Head – Digital Operations at Brillio, a global technology consulting and business solutions company. “The best thing to do is decentralize analytics and centralize the plumbing layer. That way, it’s commoditized, you can apply governance, and address security standards.”
While it used to be about control, I really believe now it’s about providing the infrastructure (the plumbing) to SUPPORT the requirements, while supporting the best practices.
Is that possible? Do you do this now? If so, what has worked for you, and how intimately involved with data extracts are your users? Have you had surprises crop up?
It would be great to hear your thoughts and how you’ve worked through these issues.