The Year We Had Results and No Way to Explain Them
In 2001 the Singapore Armed Forces shooting contingent had dropped from second place to fourth at the ASEAN Armies Rifles Meet, and the national sports psychologist brought me in to work with them. Our shooters were full-time national servicemen, most of them barely eighteen. Across the firing points stood career marksmen from other armies with years of range time behind them. We had one year.
I video recorded eight shooters across five weapon types: two M16 gunners, two P155 gunners, two pistol shooters, two GPMG gunners and two carbine shooters. Something unexpected showed up. None of them had been trained in Neuro-Linguistic Programming, and all of them were running its patterns.
The carbine shooter practised in his sleep until his muscles twitched. He rehearsed every scenario inside his head, so when rain swept the range and the rest of the contingent stopped, he moved into his mental range and kept working. The GPMG gunner carried the heaviest weapon in the group and was not the biggest man in it. Before the three-hundred-metre rundown he would shout and breathe hard until he felt stronger, and he told me so himself. He also asked me a question I still enjoy hearing: "Who do you think you are? What are you trying to do?"
I wrote their patterns into a report with step-by-step flow charts, cut down to the level of observable behaviour, and handed it to the team. In 2002 the Singapore Armed Forces shooting contingent finished first at the ASEAN Armies Rifles Meet for the first time in its history.
Then somebody asked the question that turned the whole exercise interesting: how do you know the modelling caused that?
I did not hold an answer that would survive a laboratory. Eighteen-year-olds trained hard. The coaching staff changed other things that season. The competing teams had their own year. My result was real, and my explanation for it was assembled afterwards, which makes it a different kind of claim.
That gap is what this lesson is about. Founders pay real money for NLP training and cannot tell whether it changed anything. Skeptics read a review and conclude the discipline is theatre. The missing piece for both groups is a method of testing the thing sitting in front of you.
What the Published Record Actually Shows
Honesty first, because everything below depends on it.
In 2012 a team led by Jennifer Sturt published a systematic review in the British Journal of General Practice. They searched for controlled studies of NLP used in health settings and found ten: five randomised trials and five pre-post studies, covering anxiety, weight maintenance, morning sickness, substance misuse and claustrophobia during MRI scans. Their conclusion was that there is little evidence that NLP interventions improve health-related outcomes, and they attributed that to the small quantity and low quality of the research itself rather than to proof of no effect. Across the included studies, risk of bias was rated high or uncertain.
Two years earlier, Tomasz Witkowski reviewed thirty-five years of NLP research in the Polish Psychological Bulletin and reached a similar verdict: the empirical base under the field's claims is far thinner than the claims are loud. Going further back, Chris Sharpley's 1987 review in the Journal of Counseling Psychology found that studies testing NLP's central claims about eye movements and representational systems produced data that did not support them, and asked whether the theory as stated could be tested at all. Michael Heap reached the same ground in 1988.
Now the part most practitioners skip. The absence of evidence loses its scandal once you know how the discipline was built. Richard Bandler and John Grinder built NLP by modelling, and they said so from the start. They watched Milton Erickson, Virginia Satir and Fritz Perls work, extracted the verbal and behavioural patterns, and published them in The Structure of Magic in 1975 and Frogs into Princes in 1979. Modelling is a legitimate method for generating hypotheses about what effective people do. It is a weak method for proving that the pattern caused the result.
Both things hold at the same time. The packaged claims of NLP have not been demonstrated in controlled trials. The observable behaviour of a person changes when their internal strategy changes, and practitioners watch that happen every week. An honest skeptic and an honest practitioner can agree on the published record. The disagreement sits in the question being asked.
A systematic review answers a population question: does this intervention work on average, across studies, for a defined outcome? A founder makes a single-case decision: did this technique change this behaviour in my business, and how will I know inside a month? Those are different questions, and only the second one decides whether your money was well spent.
Two Ways a Founder Gets This Wrong
Let me be direct about the first error, because I have watched it cost people real money. It is buying the story.
A demonstration on a stage. A volunteer with a fear of public speaking, nine minutes of work, and a room on its feet. The story that follows is vivid, emotionally complete and completely uninformative about effect. There is no baseline, no comparison condition and no chance of disconfirmation. I have sat at the back of rooms and watched founders write five-figure cheques on the strength of a moment that could not have failed, because nobody gave it a chance to.
The second error runs the other way. A leader reads a review, decides the whole field is pseudoscience, and discards a tool that changes behaviour in the rooms he runs. Group-level evidence answers a question about averages. Your business runs at the level of one, which is why single-case research designs exist as a formal method in the social sciences (Kazdin, 2011, Single-Case Research Designs, Oxford University Press). If one conversational pattern gets you four more second meetings out of twenty calls, that matters whether or not a randomised trial has studied the pattern.
Three Things That Make a Coincidence Look Like Proof
Expectancy. In a review published in American Psychologist in 1985, Irving Kirsch showed that a person's expectation of improvement predicted a substantial share of the outcome, whichever treatment was being tested. Pay for a course, believe in the method, spend a week enthusiastic, and some improvement arrives on its own. Your belief operates as an ingredient in the result, and it inflates the reading you take afterwards.
Regression to the mean. Adrian Barnett and colleagues described this in the International Journal of Epidemiology in 2005. People ask for help at their worst point, and extreme scores drift back toward average with no intervention at all. The quarter in which you finally sign up for the training is the quarter most likely to improve on its own.
Selective memory. You remember the call where the reframe landed and forget the three where the buyer stayed flat. Nobody keeps a record of the failures, so memory settles into a ratio that flatters the technique.
These three are why an enthusiastic practitioner and a disappointed one can hold opposite views with the same evidence behind them.
What the Research Does Support
Several components inside NLP have been studied properly, and the results are worth knowing before you decide what to practise.
Mental rehearsal. Driskell, Copper and Moran pooled thirty-five studies in the Journal of Applied Psychology in 1994 and found that mental practice improved performance, with an average effect around half a standard deviation, and the strongest results where imagery was combined with physical practice. A twenty-four-year follow-up meta-analysis by Toth and colleagues in 2020 found the effect still present. That was the carbine shooter twitching in his sleep.
Feedback aimed at the task. Avraham Kluger and Angelo DeNisi reviewed hundreds of feedback studies in Psychological Bulletin in 1996. The average effect on performance was around 0.4 of a standard deviation, and in more than a third of the cases feedback made performance worse. The split follows where attention goes: feedback that directs attention to the task helps, and feedback that turns attention onto the person damages performance.
Deciding the when and where in advance. Gollwitzer and Sheeran's 2006 meta-analysis of ninety-four studies found that implementation intentions, the specific plan of when and where an action happens, produced an average effect around 0.65 of a standard deviation. This is the working mechanism underneath a well-formed outcome.
Specific goals. Locke and Latham's 2002 review in American Psychologist found that specific, difficult goals outperform vague goals and outperform instructions to do your best.
A design feature runs through all four. Each one names an observable behaviour and a measurable result before it begins. Techniques that arrive without a measurement plan are the ones that cannot be checked afterwards, and those are the ones defended with stories.
The Five-Step Trial: Build Your Own Evidence in Four Weeks
One: write the behaviour and the number before you change anything. "I want to be more confident" cannot be tested. "Second meetings booked per twenty dials" can. Choose one behaviour and one number, and put both somewhere you will see them.
Two: take a baseline three times. Three weeks of the existing number, measured the same way each week. This is the step people skip, and it is the step that makes the result visible. Without a baseline, an improvement of twenty per cent disappears into feeling.
Three: fix the window and change one variable. Four weeks, one technique, nothing else altered. Two techniques at once produce a result you can attribute to neither.
Four: run the ecology check before you accept the result. Ask what else moved when the number moved. A gain pulled out of another column has shifted a cost rather than removed one. A founder who doubles outreach and loses sleep will read the outreach number as a win for two weeks, then pay for it in the third.
Five: decide on the number, not the mood. At the end of the window, keep the technique, adjust it or drop it, and write the decision beside the number. That ledger is the one artefact separating a practice from a belief.
Understand what a trial of one cannot do. It cannot validate a theory, and it will not settle an argument on the internet. It answers the question you are actually paying to answer: whether this practice earns its place in your week.