The Data Decade: My Top 3 Data Resolutions After an Evening with a Dozen IT Leaders
Recently, I had the opportunity to attend an exclusive CIO dinner hosted by Silicon Valley Bank. The setting was to hear three industry executives — a practitioner (the CTO of SVB), a venture investor, and an entrepreneur (the CEO of Numerify, which is how I managed to get in) — discuss the past, present, and most importantly future of analytics. We were joined by a dozen IT leaders from top organizations in various markets, including IBM, Visa, MasterCard, GoDaddy, and the Golden State Warriors.
All companies that were present either produce or access vast amounts of data — in petabytes. We had an open forum discussing the challenges and opportunities that these increasing amounts of data bring to their business as well as society in general.
Data Resolutions for the New Year
I walked away with one overwhelming conclusion: We are in the midst of the data decade, and the amount of data is only going to grow from petabytes to yottabytes and beyond. This data tsunami will bring exciting potential, but also come with a new set of headaches. The top concerns I heard from event attendees were quite interesting, and led to three main resolutions for me, even before the new year begins.
#1: Data Security
No doubt the volume of data is increasing manifold every year. This exponentially multiplying data has to be secured and stored in such a manner that it is accessible for valid business reasons, but cannot be breached. Since the data contains information about individuals as well as corporations, it can be disastrous in the wrong hands. Yahoo’s recent announcement of its one billion accounts breach was fresh on everyone’s mind, and it was yet another new story from the past few months that has shone a spotlight on hacking and security.
Resolution #1 — Add a calendar reminder to start changing passwords on a quarterly basis and focus on securing all important accounts with two-factor authentication.
#2: Data Interpretation
Data by itself is dumb, and needs to be converted into insights. When used in a naive or simplistic manner, data, especially numerical data, can lead to incorrect and potentially dangerous conclusions. As the old saying goes, there are lies, damned lies, and statistics! Let me explain.
There was an interesting moment during the dinner when one attendee stated that, with the advent of machine learning and AI, the need for humans to interpret data was declining. In fact, several new technologies not only use data to understand new areas, but they also take defensive or proactive actions. Related to my first point above (the threat to intelligence systems), these changes in data interpretation are truly a blessing. AI-based systems are able to detect intrusions, which can be in the range of thousands on a daily basis, and respond without human involvement.
However, some of us felt that on the business and operational side of things, humans will still need to do a lot of the data interpretation, especially when it relates to understanding human behavior. My experience has largely been in the area where analytical algorithms, models, and simulations were used to understand and predict human behavior to accurately price products, promote the next best offer, or identify the root cause of an outage. In all of those cases, a skilled individual had to bring to bear their experience and expertise in combination with the data story to make an informed decision.
This debate is far from over. As machine learning algorithms and artificial intelligence take hold in our world, more and more tasks are going to be automated. Whether it is a self-driving car or a security detection alerting system, it is clear that computers are becoming increasingly adept at pattern matching and taking action based on insights. A recent McKinsey & Company study predicts that current technologies can automate some 45 percent of activities performed by today’s professionals across some 800 occupations.
Resolution #2 — Start reading up on machine learning/AI techniques in analytics and their widening application sphere.
#3: Data Insights
IT is embracing data for themselves, but it’s not easy. A common adage in the tech industry is about “drinking your own champagne.” While the phrase is used by technology vendors to describe the internal use of their own products, this concept also applies to IT. As some of the event attendees shared, IT must start using their own data to help their organization make smarter decisions. One of the challenges in doing so has been that IT lacked the critical data science and modeling skills available in other business areas such as the marketing division or the fraud department of an insurance company.
Security and automation of IT operations was the starting point, as almost everyone has deployed machine and log file data-based analytics systems as well as APM systems to monitor ports, hardware, services, virtual machines, and everything in between. Now, IT organizations are moving to tap non-traditional business data sources to optimize their operations.
For example, one CIO described how they were using data in their financial department via expense management/credit card bills to uncover shadow IT purchases. In his case, the percentage of spend going through credit and corporate purchasing cards was far less than average. He was nevertheless surprised to find that their environment uncovered new applications. As they say, “follow the money trail” and you will find the culprit.
Resolution #3 — Continue to spread the Numerify message of IT business analytics widely. (I had to plug our solutions!)
Everyone at the dinner agreed we are living in an exciting time where data is going to be constantly changing our work and personal lives in more ways than one. There is no function or industry that will not be impacted by the data deluge and insights that come from within. We still have three years to go in this data decade, and if you are in a data/analytics domain I suggest you strap in, because this is going to be a crazy ride!
[Image courtesy of flickr user NATS Press office.]