Giordano Dichter tells us why the numbers and statistics need to involve the human component and how it can be done

In June 2016, EBN published its annual Impact Report. This report, which can be downloaded from the EBN website (ebn.eu/impact), contains facts and figures coming from the annual survey that we undertake, primarily, to assess the compliance of the EU|BICs (the full members of EBN) to the EU|BIC Quality Mark Criteria. The effort to produce this publication, from data collection to editing, spans a good six months, going through validation, aggregation, statistical analysis, interpretation of the data, to design... and the effort is not inconsiderable. Usually once the document is printed and ready one imagines it is all over and the subject can be shelved for another six months.

This year, though things are different. What changed? Well, the first thing that changed (and not suddenly, but through a long dedicated process) has been EBN’s approach to quality assurance itself. We have been obsessed with numbers and key performance indicators (KPIs) for a long time. Numbers have been primarily what has driven us in the selection of the bestperforming EU|BICs. Percentages, mean values, medians, shares were the main ingredients whilst preparing the list of those EU|BICs that would undergo an audit. 

The truth of numbers

But, being trapped in this ‘numbers game’ is a risky business. Focussing solely on numbers has indeed led us to a point where we were unable to grasp the full picture. And the full picture is definitely more complex, with a set of qualitative nuances that need to be taken into account when assessing the true impact of a network such as ours. It is definitely too easy (and often too far-fetched) to simply judge an organisation as ‘not compliant’ because it falls short in the value of the KPIs when compared to the mean or median values of its peers in a given year. Without the qualitative component you would take uninformed decisions based on just one of the many components of the story.

The matter is not a trivial one, as it might affect the economic growth of a region and might impact the well-being of society at large. For example, we can see from the impact report, that on average each EU|BIC in 2015 has had a deal-flow (from enquiry to startup) that led to the creation of 32 startups. As for any data series there are outliers in both senses. At first sight we would be tempted to say that some EU|BICs have done way better than others. However, without a clear vision of the nature of the startups we would not have a clear understanding of the real situation and decisions might be taken that could be regretted. And beware, it is not just a quality-assurance matter (trademark or not). It is mostly a matter of policies, of funding, of providing the adequate information to stakeholders who, if not impressed by the numbers, might decide to quit supporting the incubation industry all together. And who knows what really highimpact innovations might never see the light of day because of this. The matter indeed is not a trivial one. It is easier to ‘sell’ an incubator that launches 50 startups than one that supports five… Yes, of course, but what if in those five there is a startup that will solve a real global challenge? And what if in the 50 there are just 50 new irrelevant apps on the iPhone store that will not have any real impact on the world?

As a general rule, quantity needs to go hand-in-hand with quality. Therefore a quantitative approach needs to be accompanied by a qualitative one. EBN has gone through this paradigm shift in the recent years and, of course, as a result, things got more complicated and less linear, but definitely more interesting. 

The human factor

Last year’s impact report, albeit not disregarding the quantitative results, shows innovators for what they are, and not for what they are wrongfully often seen as: innovators are often just that, not just indicators.

Innovators, obviously, do have a human component. While this is fundamentally true, unfortunately, while concentrating on impact measurement, we have a certain tendency to forget this. The hatching eggs in the EU|BIC infographics show us that, on average, an EU|BIC will need to assess 572 business ideas to deliver 28.8 sound startups (already discounting the 90 percent survival rate). From a purely quantitative perspective this means that there is a conversion factor from idea to startup of five percent. Taken alone, this figure can be conducive to different contrasting decisions. Some could perceive the five percent conversion rate as being too low, therefore requiring immediate action to reshuffle incubation programmes or cancel them altogether. Some could be willing to be more risktaking for which it may be viable to sacrifice such a high sustainability rate in order to get more companies up and running Others may be more conservative and prefer to keep things safe and are happy enough to know that those companies that have been started up will remain in the region without creating great distress and economic turmoil due to a larger amount of future failures and foreclosures. However, whatever the decision taken, it would be a flawed decision, as it would be purely based on a quantitative analysis. It wouldn’t take into account the (human) innovators, the innovations and the intrinsic value they represent and bring forward. 

Analysts who spend many hours crunching numbers trying to make something sensible out of them, are inevitably struck by waves of frustration when they realise that the innovation and the human factor, which is mostly unpredictable, come in to scramble the hard-earned results of a serious quantitative research. But ignoring the problem is not the solution, and at EBN we had to (and still have to) find ways to cope with this complexity. The best way to approach what appeared, initially, to be an insurmountable problem, seemed to be to rely on the EU|BIC community itself. EBN indeed is a community of over 150 certified EU|BICs that occupy prestigious positions in the innovation ecosystems at all levels, and have been doing so for over 33 years. The community is made of EU|BICs which are in turn made of respected professionals that know each other and have worked with each other for long periods of time, in part facilitated by the existence of the network itself. If we hadn’t been able to rely on the community, we surely would have a had a much larger problem than we initially envisaged. 

Community task force

We rightly assumed that a well-networked community would be the perfect ground for qualitative benchmarking. The quantitative benchmarking reports that each EU|BIC receives every year could be done sitting behind our solitary desks. But to incorporate the qualitative aspect we needed to blend our desk work with a purpose-built networking action, where trusted EU|BICs (the trademark helps build trust), would tell the full story.

A properly built suitable networking action, of course, takes time and efforts to create. From our end it required the amendment of our EU|BIC questionnaire, where we incorporated a qualitative section where the EU|BICs could detail innovation, startups and practices. It required the creation of a specific annual event, the EBN TechCamp, where incubation and acceleration practices, methods and tools are described, shared and put to test. It required opening up the current impact report to real stories, of real innovators supported by our members. There is still much to do, but we couldn’t have achieved what we have achieved today without the active community of EU|BICs ready to share their narrative.

Lessons and learning

We have learned much from this activity, however the most valuable lessons were two. The first one became abundantly clear when enlarging the scope of our zoom. Less focus on the direct KPIs, meant more clarity on the side-effects, which was clearly very important, as well, and assumed significance as stand-alone variables. The qualitative stories show us the other side of the coin, which is much heavier than the standard one we usually look at. A five percent conversion rate meant that 95 percent of the innovative ideas do not pass the so-called preincubation phase. Ideas that are incapable of finding an adequate business model, failing their proof of businesses. Ideas that are not technologically viable, ideas that could work with different teams, but fail to get anywhere themselves because of stubborn potential entrepreneurs... All this hardly represents a failure as it is a strong sign of dynamism, and of an EU|BIC’s capacity to contribute to shaping the much-needed entrepreneurial mindset.

How do we know this? Because we can rely on a quantitative figure: to reach the figure of 572 innovations assessed, each EU|BIC has been through a massive amount of lead-generating activities (check figure 21 of the impact report for proof of this). We know this because our EU|BIC staff members come to us with stories of innovators who fail the pre-incubation phase, and of how some come back with a better knowledge of what needs to be done, with clearer ideas. This qualitative impact could not be assessed otherwise.

The second lesson that we learned is that any quality system that aims at measuring the impact of the innovationbased incubation industry will be imperfect. But this doesn’t meant that we need to kill it altogether. It just means that we need to manage it with serious creativity, by conjugating the data dimension, with qualitative networking actions and with honest reflection of where all this is bringing us. We, as a team, hope that we are achieving this, to the best of our knowledge and abilities, in partnership with you.