What would have happened without the campaign?
SN-2026-102 · Methods note · Communities and governance
By Sofia Mendes · · 2 pages · 1,352 words
Fictional demonstration publication. Not peer reviewed. Still North Arctic Institute is a fictional organisation. This paper is a teaching synthesis or illustrative practice document, not a report of original institute fieldwork. Published sources are identified below; illustrative examples are not measured results.
Abstract
A favourable decision after collective action is not automatically an effect caused by that action. This methods note develops a counterfactual evaluation for a fictional workplace transport coalition, drawing on the Magenta Book, the GHG Protocol Policy and Action Standard, and the IPCC demand-side assessment. A pre-existing employer plan illustrates bounded contribution rather than exclusive credit, while invented arrival counts expose the assumptions behind before-and-after and comparison-group arithmetic. The analysis distinguishes mobilisation, decisions, implementation, sustained operation, and downstream climate effects. Participant protection and ethical constraints remain part of the design. No campaign result or emissions saving is claimed. The proposed approach supports honest learning from shared influence, partial gains, and unresolved demands without converting signatures into unsupported carbon totals.
Download the complete PDF (2 pages)
Full paper
The outcome is not the effect
A campaign can be followed by a favourable decision without having caused that decision. A council may already have budgeted a change; an employer may be responding to regulation; several organisations may have contributed over many years. Equally, a campaign that does not win its initial demand may change the timetable, preserve an option, or make an otherwise inaccessible decision open to scrutiny. Evaluation needs a counterfactual, not a victory label.
This methods note examines how a small workplace coalition could evaluate a transport-policy campaign. It combines HM Treasury’s Magenta Book, the GHG Protocol Policy and Action Standard, and the IPCC demand-side assessment. The approach is a selective synthesis of causal and governance principles, not a study of a real campaign. The organisation, decisions, and arithmetic in the worked case are fictional. No observed campaign effect or emissions saving is claimed.
The counterfactual is the relevant outcome that would have occurred without the specified action. It is not necessarily the status quo, a world without any climate policy, or the least favourable imaginable future. A fair evaluation describes an alternative that is credible given the information available, then explains why the evidence can or cannot distinguish that alternative from what happened.
Define the action narrowly enough to evaluate
Specify who acted, what they requested, who had authority, and the period of interest. A staff coalition might request a revised late-shift transport policy, supported by documented access problems and a costed proposal. Its action is not every conversation about transport in the organisation. Nor is the employer’s eventual entire travel policy automatically the coalition’s product.
Write a causal sequence before implementation: the proposal reaches the responsible manager; the manager considers a change; a budget and operating arrangement are approved; the service becomes usable; journey patterns may then change. Each step requires different evidence. A petition count measures mobilisation, a decision record measures an institutional response, and a journey survey measures reported travel. None directly measures avoided emissions.
The Magenta Book distinguishes process, impact, and value-for-money questions. Applied to a coalition, a process account can be valuable even where a causal estimate is unavailable. It can explain who was excluded, which decision routes worked, and why an agreed measure never became operational. Calling that process evidence is more accurate than inflating it into a quantified impact.
A fictional campaign with an existing plan
Imagine a fictional employer already considering a general commuter benefit before a staff network begins its campaign. The network asks for late-shift coverage and an accessible booking route, both absent from the initial draft. The employer later introduces a benefit containing those provisions. Claiming the whole scheme as caused by the network would ignore the pre-existing plan; claiming no contribution would ignore the specific amendments.
A useful record would preserve dated drafts, the network’s submitted request, meeting notes, the decision rationale, and the final operating terms. With consent and proportionate confidentiality, interviews could examine how the amendments were considered. Ask people to discuss concrete decisions and alternatives, not merely whether they appreciated the campaign. Retrospective praise is not a complete causal account.
Suppose the documentary record shows the requested clauses entering the draft after the proposal, while an independent manager confirms that an alternative source also raised accessibility. That would support a bounded contribution narrative, not exclusive attribution. The narrative should identify the alternative explanation and any unresolved chronology. The result of this fictional reasoning exercise is a better claim structure, not evidence about an actual employer.
The baseline should retain the general commuter benefit if it was already likely to proceed. The evaluation can then focus on whether the campaign changed coverage, timing, safeguards, or implementation. This narrower question may reveal substantial practical importance for late-shift staff without pretending that every participant or every subsequent journey is attributable to one campaign.
Why before and after is insufficient
Consider a separate fictional illustration. A workplace records forty car arrivals in a comparable observation window before a policy and thirty afterwards. The raw change is ten fewer arrivals. That difference is not yet a policy effect: staff attendance, weather, parking availability, fuel prices, or seasonal services may also have changed. Counting accurately does not remove the need to explain other causes.
If a suitable comparison workplace changed from forty arrivals to thirty-five over the same period, a simple difference-in-differences calculation would leave five fewer arrivals beyond the comparison change. All quantities are invented. Even in a real dataset, this calculation would depend on credible parallel trends, comparable measurement, and no relevant spillovers. One convenient comparison group does not make those assumptions true.
Repeated observations before and after the change, evidence about differing trends, and contextual information would matter. Staff who select into a new service may differ from non-users. A survey of users alone cannot establish how all staff would have travelled otherwise. Where there is no defensible design, report descriptive change and its limits rather than treating adjustment arithmetic as proof of causality.
Trace climate effects beyond the decision
The GHG Protocol Policy and Action Standard asks users to identify causal chains, define assessment boundaries, and compare baseline and intervention emissions. For a transport measure, this means examining displaced journeys, additional vehicle movements, service occupancy, and the relevant energy boundaries. A policy approval is an intermediate outcome, not a direct emissions measurement.
Do not multiply signatures by an assumed personal footprint or assign each supporter a fraction of the employer’s inventory. The campaign may change a condition that enables travel choices; actual effects depend on uptake and alternatives. It may also induce journeys or change other activity. Such pathways should be assessed where material, rather than excluded because they complicate a positive story.
Time matters too. A commitment can be delayed, reversed, underfunded, or restricted after approval. Define whether the evaluation concerns adoption, implementation, or sustained operation. A review date and named operational owner make it possible to see whether a promise survived the initial announcement. They also help distinguish an advocacy result from a durable public benefit.
Evaluate collective contribution fairly
The IPCC’s demand-side chapter places collective action within wider institutional and infrastructural change. Its discussion supports taking social organisation seriously, but it does not supply a conversion factor from meeting attendance to mitigation. A campaign’s mechanism and context must remain visible when moving from general evidence to a local evaluation.
Coalitions should agree how to describe shared work. Document partners’ roles, prior efforts, and affected people’s input. Avoid dividing credit into unsupported percentages merely because a funder asks for an attributable total. A specific contribution account can state which evidence supports influence, which steps remain uncertain, and what would contradict the interpretation.
Evaluation can itself create risks. Publishing personal travel details, employment complaints, or quotations may expose participants. Collect the minimum data needed, obtain appropriate consent, and protect routes for people who cannot safely speak publicly. Do not delay urgent safety or access changes solely to create a cleaner experiment. The strength of the claim must adapt to ethical constraints, rather than the other way round.
Limits and a useful definition of success
Political and organisational changes are rarely isolated interventions. Effects may be delayed, shared, and dependent on actions outside the coalition’s control. Documentary accounts can be incomplete, interviewees may rationalise decisions, and comparison groups may be imperfect. This method does not eliminate those limits. It provides a way to state them while retaining evidence about genuine institutional change.
Before beginning, write the counterfactual and the decision that new evidence will inform. After acting, report what changed, how it was implemented, who benefited or remained excluded, and how strongly the evidence supports a contribution. A campaign can then learn from partial gains and unsuccessful demands without needing a fabricated emissions total. Accountability becomes a route to better collective action, not a competition for the largest claim.
References
- HM Treasury and Evaluation Task Force (2026). Magenta Book: Central Government guidance on evaluation.
- World Resources Institute (2014). GHG Protocol Policy and Action Standard.
- Creutzig, F., Roy, J., and colleagues (2022). Demand, services, and social aspects of mitigation. IPCC AR6 Working Group III, Chapter 5.
Keywords: collective action; counterfactuals; contribution; campaign evaluation.
Browse and filter the research library · Download learning guides