Data… Information… They’re what drive almost every business and industry: from supply chain management and assembly line control in manufacturing to loan underwriting and trading operations in banking and finance. And yet, somehow, most businesses are terrible stewards of data and fail to demand more of their data processes.

Take banking. The dirty secret of finance is the degree to which Excel is embedded in core processes: reporting, automation, data storage, analysis… the list goes on. While Excel is a wonderful tool for quick and dirty analysis, is it really the best tool for managing hundreds of billions of dollars (literally – we have seen this)? Here’s how departments in such firms operate:

  • Store data in Excel worksheets, maybe use Access
  • Password protect the file(s), often with the password “password”
  • “Process” data via recorded macros
  • Exchange and share data by copy / paste into emails

Because every department has their own web of Excel unholiness, there is tremendous duplication of data, effort, and resources. Plus, firms fail to unlock the hidden insight from merging and linking various data.

Shouldn’t firms demand more?

Almost without exception, each of these workflows could be synthesized into automated processes running on servers using enterprise-level tools – both data storage and processing). Databases exist for a reason – they are really good at storing and organizing data, handling access privileges, and providing backup and audit. Coupled with programming tools running on servers (providing the capacity for both automated, repeated analysis and ad hoc exploration), this paradigm can be a powerful enterprise system. Moving to this paradigm facilitates having separate development (DEV), user-acceptance testing (UAT), and production (PROD) environments so that everything is rock-solid and robust. Plus, it is a simple and logical next step to layer on data visualization dashboards.

We’ve helped departments in multiple industries make this transition and become not only the most efficient department but also the envy of their colleagues.

The ironic thing is that most businesses implicitly, if not explicitly, acknowledge the value of information.

  • TRADING FLOORS: Why do they need so many screens? Why are they all watching TV? News, data, information – that’s what drives their business. Traders process enormous volumes of data to inform their decisions.
  • INDUSTRIAL CONTROL: Why do they need so many screens? What are all the gauges and monitors? Data matters and drives decisions.

Why do we use the word infrastructure? Once data and base analytic tools move from individual Excel / Access fiefdoms to accessible repositories, magical things start to happen: the problem of the commons becomes the surfeit of the commons.

  • Resources previously tasked with process execution and maintenance are instead able to actually analyze results and conduct follow-up / ad hoc analysis.
  • Data can be shared. You never know where insight may come from – it may be in another group.
  • Data can be contextualize with and juxtaposed against other data – the whole can be greater than the sum of the parts.

Data, along with base analytic tools, becomes infrastructure, much like roads, bridges, pipes, etc.: readily and easily accessible by the multitude.