If you have ever tried to describe the logic and results of something you do in the form of a 'logical framework' (log-frame) for a donor, and are interested in how private sector growth contributes to development, then this is for you. But if log-frames mean nothing to you, this blog is probably neither useful or interesting for you.
I've just seen a summary of a discussion comparing log-frames with 'theories of change' on the Knowledge Brokers' Forum, highlighting the complexity that can get missed in a tabulated log-frame. It contrasts with another discussion I've had recently about whether employment is the best indicator for a donor-supported investment fund. This all prompts me to share our internal discussion about the challenges of log-frames for the Business Innovation Facility.
The logic behind for investing donor resources in both Business Innovation Facility and Innovations Against Poverty is clear, but the chain of logic is not short or simple. Donor input reduces risks and transaction costs for private sector activity that has high social impact, but would - if left to the market alone - probably be under-delivered. But BIF and IAP input is catalytic: we cannot say that $x delivers $y benefit. The ratio varies - some project thrive, and some don't. The benefit varies too - it's certainly not just measured in jobs, and not just in numbers of low-income consumers reached.
It would be a lot simpler if the results of BIF or IAP could simply be tracked as numbers of jobs - and make it easier to do a log-frame, which donors use to identify indicators of success and what needs to be tracked. Here are a few reasons why, in my role as Learning and Results Manager for BIF and IAP, I have argued against over focusing on employment as an indicator of success:
Various reasons why a simple 'increase in employment' indicator does not really work for BIF, or more generally for this type of project
- where projects reach producers in the value chain,such as groundnut farmers, they gain market access, secure sales, chance to diversity or take up improved varieties, but it is not 'new employment'. It may not even be increased income, but security, less fluctuation etc.
- roughly half of projects and much more than half of BoP beneficiaries will focus on the poor as consumers: access to communications, healthcare, information, more efficient or safer technology. This is not just about lower-priced goods and services, which could be converted into money saved indicators. Often it is about improving access or quality. eg mKrishi, ischool,
- our analysis of the Portfolio so far, suggests that direct impacts on people at the Base of the Pyramid (BoP) are just half the story. The most significant impacts in some projects will be systemic effects, on what others do and trajectories of sector development. In a few cases where this is replication of the business model, you could count this as 'scaling up' and add indirect beneficiaries to direct beneficiaries. But often it is more of a sideways impact. eg others adopting aflatoxin-management, others others investing in services for farmers that can be used via a mobile application
- even in projects that do directly benefit BOP people with jobs and income, it is nigh on impossible to do a baseline with those specific poor groups. Businesses do not start with fixed client groups and then serve them, as a classic donor project does. They start with a product then secure the suppliers or market. BIF could invest in more baseline analysis with likely BoP participants, but it could be an expensive/risky investment, and we would have to choose in advance where to focus.
- BIF's engagement is directly with companies and we rely on them for reporting results. Business should be able to report numbers of BoP people reached directly, as clients, buyers, suppliers, distributors etc - although even this reporting of BoP people reached is proving a challenge in some. Where payments are involved, they will know those. But to delve deeper into the incomes earned, the full-time equivalent employment in the supply chain, or which employment is net new jobs, is not their core business. To go further is an extra operating cost of the business, which affects competitiveness. As our job is usually to help companies reach break-even, it is hard to demand something more that will go against that.
- If companies themselves cannot report the data, then BIF and IAP could fund external assessment. Average BIF spend on a larger 'cost-sharing' project is £30,000 - £50,000 and only £10,000 or so on shorter engagements. To do an impact assessment at each project to assess BoP impacts could easily cost more than the grant. This is why we have opted to do deep dives' in a small selection of projects in Year 3.
- we have to ask what BIF is all about: a multiple of 40 + 100 projects, or catalysing up-take of inclusive business by exploring and demonstrating how it works. If the real impact is in developing inclusive business as a business practice (and a donor practice), then we should measure that. Now there is a new challenge.
Debates about what to measure and what framework to use can get technical and boring, but this is a live debate. Defining success affects what counts as successful. More and more initiatives that aim to combine private sector growth with development impact are being supported. Jim Tanburn asked in this Network, whether the Donor Committee on Enterprise Development should prioritise seeing through a smaller number of fully-fledged tracking systems, or engaging more people in the approach. The essence of the DCED approach, to my mind, is that it focusing you on the chain of logic, and the need to assess it. My belated answer to Jim, is that getting more of us to be clearer on our chain of logic and our story of change is the first priority.
Caroline, thank you for this post, I think it is much needed. You raise some excellent points about thinking about what might have happened otherwise (e.g. not tracking jobs 'created', etc.), and underscore the difficulties in measuring impact. Of course you do somehow have to assess if business is 'doing good'. I would definitely argue for more thinking through a theory of change (or chain of logic, causal chain, however you want to call it) to determine what exactly is 'inclusive' about the approach and possible ways of tracking that inclusivity (or at least some of it). Again, thank you.
Interesting points which all add up to conclude that employment is clearly a very limited indicator. In Malawi it has to be a very poor one - the chances of generating significant new employment in this tiny economy where most of the population are smallholder farmers is limited, and the first point you make about securing or increasing income is the appropriate one. I was amused the other day when I saw an application for donor funding which stated the "unemployment rate" in a particular area of something like 85% and how the business in question would be improving this by increasing employment. The statistic was a strange twist on the fact that 15% of the population had waged jobs - but the rest were smallholder farmers or ganyu (part-time) labourers (plus a few advantaged business owners) - not what I would describe as "unemployed". Or am I being unfair .. should we be thinking of these farmers as "unemployed"?? The term clearly means a hugely different thing in the western world than an agriculture-based LDC.
On another note, regarding your final point about up-take, yes, I agree that increase in donor practice in inclusive business is a target. I remain surprised about an "access to donor finance" piece of work we conducted recently finding it hard to locate charities in the Uk with an appetite for private sector development (yes, blog will follow!). However, I still struggle to think that many businesses consider "inclusive business" as a specific and novel option rather than it being a subset of all considered business solutions. If BOP consumers or producers make a good choice for your business model, then that's great - it's inclusive. If they don't, then not. Do we really believe that businesses fail to see opportunities that involve the poor and they need further education in this area? Again I find myself thinking about the grey area of CSR -> inclusive business ... but enough for now!!
A related blog on why BIF cannot well report cost per job created, and some thoughts on what universal or comparable metrics can be used, are in this blog: http://businessinnovationfacility.org/group/inclusive-business-impa...