by I am very pleased to have nominated the work of my colleague, Terry Grim of the Foresight Alliance, on the Foresight Maturity Model as a candidate for a 2013 APF Most Significant Futures Works award. The article “Foresight Maturity Model (FMM): Achieving best practices in the foresight field was published in the Journal of Futures Studies in 2009.
Her piece addresses a hugely important issue for the profession: how do we more effectively evaluate the contribution of our work? Terry adapts work she did at IBM in developing a Strategy Maturity Model to foresight. The approach advocates measuring the effectiveness of the practice, which makes sense give how challenging it is to measure outcomes of the work, given the time frame of much of our work. The approach defines best practices and then measures the competency of those practices. Organizations can assess their foresight capabilities against the model and identify where they need improvement. This model provides a much-needed measurement tool that I believe merits its nomination as a Most Significant Futures Work. Andy Hines
I agree with you, Andy! I have used the FMM in my doctoral research and found it extremely useful in assessing foresight competencies. Congrats to Terry!
Congratulations to Terry! Thank you Andy for sharing. I am a follower of your blog.
Btw, does Terry have a website or twitter account?
Be
Hi Andy,
I agree with you comment that more effectively evaluate the contribution of futures work is a key issue. However, Terry Grim’s work and model doesn’t actually help with that!! In her paper she explicitly states that she is assessing practices and not outcomes. If we don’t look at outcomes how can we evaluate our contribution?
I’m keen to hear your thoughts on how we might actually tackle this issue, rather than just assess processes. Some argue predictive accuracy should be central to evaluation; others argue contribution to particular outcomes (e.g. organisational success/performance) should be the focus. There’s certainly no consensus on the matter!
Full disclosure: I’ve recently started a PhD at the Institute for Sustainable Futures (a research institute at the University of Technology Sydney, in Australia) and in my research I’m looking to examine – and hopefully actually evaluate – the contribution of foresight activities to tackling “wicked” sustainability problems. So this is an issue I’m very interested in and hoping to make a contribution through my research to better addressing this challenge. I’m very keen to hear your thoughts on this…
Cheers
Stephen
Hi Stephen,
I hear you. I would say that evaluating the work/work processes is part, and an important part, but as you say, it’s not the whole enchilada. I took a stab at an outcomes framework in my dissertation. I will eventually try to write it up in an article. In the meantime, you could look here in “outcomes” https://www.andyhinesight.com/foresight-2/phd-the-role-of-an-organizational-futurist-in-integrating-foresight-into-organizations/
It’s great to hear about your dissertation topic — that sounds interesting, useful, and important!!!
andy
Thanks for the link to your dissertation (and congrats – the end product still feels very, very far away in my case!).
An initial question – which you identify – is what is a “successful outcome” in foresight? And there’s no doubt this will be context-dependent (as you argue). The published literature is currently dominated by a central focus on the impact of foresight on decision-making (or policy-making) rather than – say – specific organisational or wider societal outcomes/changes. A big question I think is, how might this be measured? How you prove such decisions were improved – beyond what would have been conceived and implemented without foresight?
Table 8 and the lit review are a useful start, but, as you say, it’s early days. Some of ‘measures’ you note are more indicators of success (e.g. improved strategic conversation) rather tangible measures
I used to work in advertising and if you’re not improving sales, or not having a measurable impact on brand image and perceptions, well, you lose your clients. Much more clear cut than futures activities! Your conclusion that the “specifics of success remain to be negotiated” is a really interesting and important one.
A related issue is the lack of empirical research evaluating the outcomes of foresight activities, and the complexities of attempting this. For example, only a couple of studies have tried to examine whether or not scenario planning improves organisational performance (that I can locate, anyways). An additional issue is the published literature tends to be practitioner accounts of their (supposed) successes – no reporting of negative results occurs (where the outcomes disappointed and discussing why), and rarely is any independent evaluation conducted and reported (i.e. not reported by the practitioners, whom it could be argued will be biased in their writing up of cases).
Food for thought!
Stephen