Most Analytics Projects Don’t Require Much Data

In their headlong rush into advanced data science, big data, machine learning, and artificial intelligence, too many companies have ignored “small data.” This is a huge miss. The relative ease, ubiquity, and power of small data projects carry profound implications for all employees, managers, and leaders at all levels, in every department, in every organization. Small data projects involve teams of a handful of employees, addressing issues in their local workplaces using small data sets — hundreds of data points, not the millions or more used in big data projects. They are tightly focused and utilize basic analytic methods that are accessible to all. They can be completed in a few months by people working part-time and yield financial benefits of $10,000 to $250,000 annually per project. Companies are loaded with potential small data projects, and it is reasonable to expect that a 40-person department to complete 20 projects a year. The cumulative benefits are enormous. Unlike big data projects, which often involve dozens of people with disparate agendas, politics, enormous budgets, and high failure rates, the probability of success is high. Thus, small data projects build the organizational data muscle that helps the entire company learn what it takes to succeed with data, gain needed skills, build confidence, and breed the kind of culture that big data demands. And with many individuals worrying that they will be replaced by automation or that their jobs will change in ways beyond their control, participating in these projects enables everyone to take proactive steps toward building their data literacy and deal with their own fears. Plus, they are fun! A first-line manager, a veteran of 20 years in telecom, exalted at a celebration dinner we attended, after leading her team through a series of small data quality projects: “It was the best experience of my 20-year career. It was the only time I felt like I had any control on where I was going.” We’ve helped launch hundreds, maybe thousands, of such stories from all over the world. People revel in understanding the numbers, what they mean, and the detective work to sort out what is really going on. They love working on teams and seeing the results of their labor improve their work and their company’s performance. While the work to unlock the power of small data is not difficult, reorienting your thinking toward prioritizing these projects can be tough. We recommend jumping right in and taking the following steps. Get everyone involved — yourself included. Personally lead at least one small data project with your direct reports every year. You will learn a lot, experience the power of small data firsthand, gain credibility, and set a good example by doing so. Encourage your people to join your data initiative and empower them to put small data to work in their own unique ways. Once you know where to look, it won’t be hard to find small data opportunities. Three areas are often “target rich”: Eliminating hidden data factories: Poor data quality is the norm, and the goal is to reduce the non-value-added work needed to deal with errors. Reducing wasted time: We find that people waste a lot of time waiting for meetings to start, inputs from a colleague, a shipment to arrive, and so forth. The goal is to reduce that time. Simplifying handoffs: As work proceeds into, across, and out of your team, poor handoffs may increase complexity, cost, or time. The goal is to streamline these handoffs. As you get the hang of it, you may wish to focus, concentrating on one area, setting an aggressive target, and challenging your team to meet that target. Follow a disciplined approach. Many small data projects are quite simple, and it is tempting to leap directly to the solution. Still, we find it best to follow a disciplined, straightforward process, as follows: define the business problem, gather the needed data, analyze the data, make improvements, lock in the gains, identify the next opportunity, and repeat the cycle. Those familiar with lean, six sigma, the data analytic life cycle, or the scientific method will see similarities to these approaches. Provide training. Ensure that you and your team receive training that provides both practical experience and explains the “whys” and “hows” behind the methods. In one company we worked with, the data quality training consists of eight 90-minute workshops, each focused on an important topic such as measurement, and on-the-job-assignments in which attendees put those fundamentals into practice immediately. The in-class examples were specific to each individual’s discipline: Those in finance got finance examples, engineers got engineering examples, and so forth. This combination of fundamentals, relevant examples, and hands-on experience helped people in this company develop and apply the skills they learned very quickly. Define your unique area of expertise. Empower yourself to address at least one problem, then build on what you’ve learned to carve out a niche for yourself. Perhaps you can become facile with data quality measurement and improvement to become your team’s data quality expert. Or maybe you can build on your interest in customer relationships by developing expertise in the associated metrics and their implications. Doing so will both help you make valuable contributions to your team and help you build a satisfying career. As you take these steps, bear in mind that just because you’re focusing on small data projects doesn’t mean you won’t also have big data initiatives — now or in the future. Indeed, big data is needed to address big problems. But overlooking the low-hanging fruit in favor of the sexier big data projects that you are not yet prepared to complete is unwise. So think strategically. Especially when getting started, emphasize small data. We know that many people will find the direction proposed here counterintuitive given the rush to big data and artificial intelligence. But project yourself into your organization’s future ten years from now. We hope you see a cadre of elite data scientists and technologists, working on a few complex problems. But more so, we hope you also see a more egalitarian workplace, with everyone growing increasingly comfortable with data and contributing on scores of opportunities. The small data path leads you there. Get on it.

Most Analytics Projects Don’t Require Much Data

Summary.

Partner Center