There is no doubt that the vast troves of data residing on the web—“big data” as they have come to be known—hold enormous potential to inform business strategy and boost productivity. One recent report estimates that data could generate gains of 1 to 5 per cent, seemingly modest improvements which could translate into game-changing returns. Amid such great promise, rising demand is spawning dozens of specialized start-ups, several fueled by venture capital—and an entire field of research, computational enterprise analytics.

The biggest challenge right now is how to sift efficiently through this raw information, in order to piece together digestible and meaningful trends and patterns—a process that has been characterized as “wrangling” and “janitor work.” (A task, as you might expect, that can take up most of a typical data scientist’s day.)

“The leap from the tools to the insight is the weak link,” Professor David B. Yoffie of Harvard Business School recently observed. Echoing this sentiment, University of Washington computer science professor Jeffrey Heer cautioned, “It’s an absolute myth that you can send an algorithm over raw data and have insights pop up.”

These scholars raise an interesting question: where do institutional insights come from? Is it, as they seem to suggest, a matter of the scale and the integrity of data, and the sophistication of analysis? Do more data necessarily mean bigger or better insights?

By contrast, in-depth interviews live at the other end of the data spectrum—allowing organizations to learn about themselves and their operating environments by digging where they stand. They provide an opportunity to probe critical issues with the help of people who are knowledgeable and invested. As with all forms of research there is a certain amount of mess involved, but the richness of what organizations learn from such close looking is hard to match.

These conversations can also teach big data a thing or two: they consistently reveal how vital it is to be clear about what you’re looking for from the outset, to formulate meaningful questions, and to think carefully about whom to ask them. The results may be “small data” by comparison, but the insights they generate can be far deeper.

Today, few would question the value of assessing organizational impact and its promise of accountability. What was a bold crusade more than a decade ago has now become mainstream: assessment, evaluation, monitoring, and theories of change have become household terms, commonplace—even obligatory—in grant proposals, strategic plans, annual reports, and conference agendas. Robust assessment systems have come to be expected, and can confer competitive advantage.

Yet in some recent conversations, we have begun to hear questions about the effectiveness of these practices. Some sense a risk of measuring for measurement’s sake, without much thought about how the information generated will ultimately be used, and by whom. That the mere act of inquiry—especially when highly visible—can suffice as a signal of innovation and self-scrutiny. In short, these observers see a danger of assessment becoming an end itself, and in the process missing the point of doing it in the first place: to inform and improve what organizations do.

One factor at play, some suspect, is that for many organizations assessment is not native or organic—driven instead by funders, peer pressure, or even by internal efforts to justify predetermined strategy. In their view, one consequence of this pressure to assess is overcompensation, resulting in a vast surplus of information—information that no one is sure is actually being used. A recent examination by the World Bank, reported in the Washington Post, found that one third of Bank reports had never been downloaded, and another 40 per cent had been downloaded less than 100 times.

One answer may lie in another booming trend: big data. The Foundation Center, through its IssueLab, is beginning to explore ways of aggregating and analyzing data from those mountains of reports, to draw out lessons and discern broader trends. Perhaps these may even come in tweetable insights.