Big Data Hubris

November 12, 2014 BY Charles Fallon

20141029 - Big data hubrisI was in a meeting recently with the transportation team of a multi-billion dollar grocery chain and it got me thinking back to my days in engineering school.

My Fluid Mechanics professor did his best to make it an interesting topic.  Translated into common-speak, Fluid Dynamics means “How to efficiently pump salsa and other slurries through a pipe.”

The professor was known for his legendary digressions that could begin with the mathematics of laminar flow and end up with a critique of Hollywood blockbusters.  Sitting there with those grocery shippers, made me recall my professor pointing out the difference between knowing something and having access to that information.

“Too many engineers,” he would say, “confuse the knowledge in their heads with the knowledge printed in the books in their offices.  Just because you have the book, doesn’t mean you know what’s in it.  There’s a false confidence that builds in a person who takes for granted what they know and don’t know based on the textbooks they hung onto.”

We were discussing the company’s backhaul program, which represented about a quarter of all inbound volumes.  The director of outbound transportation had summarized their backhaul program in terms of loads, revenues and costs.  As looked at it something jumped out at me, so I asked, “if I take the number of dry backhauls you did and the costs you attribute to those loads, it comes out to about $40 per load.  The cost of your driver, truck and trailer sitting at your vendor’s site is more than $40 so how could the backhaul cost be so little?”

The director of inbound transportation looked at the summary carefully.  “That’s wrong,” she declared, “where did these numbers come from?”  That question precipitated a quarter-hour of agitation as the two directors debated the source data at length.  To be fair, the cost accounting made the whole thing a very tricky exercise and the only thing we could all agree on was that it was definitely not $40 a load.  Both directors also agreed that the necessary data was available and that they could get the right numbers if needed.

The point here is that the company had the data to figure out what their backhaul program was costing them but did not actually know what that number was.  They didn’t think it was necessary because they could, if required, demonstrate that whatever it was costing them, was well below the revenues generated from the program.  What they didn’t know, however, was how well they were integrating backhauls into their outbound routing. Satisfied with getting any profit margin from the program, they didn’t know whether they could potentially grow that margin and if so, by how much.

This was exactly what my fluid mechanics professor was talking about.  In the era of big data, companies – supply chains in particular – generate colossal amounts of data.  Companies have invested millions of dollars in technology to harvest that data and do wondrous things with it.  But for too many companies, the buck stops there. They are satisfied just with knowing they can access and analyze data if they wanted to, but often don’t get around to making use of the right data to make better decisions that could greatly impact the profitability and overall value of their business.

Does this sound familiar?  The odds are, it does. If so, I’d love to hear your thoughts on what you think can be done to solve this problem and if it doesn’t I’d love to hear how your company is leveraging big data to make better decisions.