WHEN WE INTERVIEW RESEARCH SUBJECTS, HOW DO WE KNOW WE’RE GETTING THE TRUTH?

I was having a drink with a couple of colleagues, who, like me, are engaged in case study research, and the conversation turned to interviews. One of my colleagues mentioned some questionable propositions that had been put to him in one of those interviews. “I don’t believe that,” he said, “but if that’s what they say, what are you going to do?”
I knew the answer to the question: triangulation. But it took some excavation of my own research experience to remember how I had arrived at that answer. The idea of triangulation never actually occurred to me. It presented itself, in the form of a puzzle I encountered as a graduate student immersing myself in my first primary research project, a study of Kenya’s Million-Acre Settlement Scheme, the starting point for a book I later published under the title Land and Class in Kenya.


The Million-Acre Scheme was a giant development project, in which a large proportion of land previously occupied by European settlers was subdivided and conveyed to African smallholders. As I cooled my heels in Nairobi, Kenya’s capital, waiting for research clearance from the Kenya government, I read everything I could get my hands on about Kenya’s recent history, its agriculture and rural society, in order to gain an understanding of the research I was embarking upon.
I gathered from my reading that, although there were two types of settlement schemes, high-density and low-density, the differences between them were relatively minor. All of them, I understood from the readings, had been settled by Africans in need of land. The high-density schemes had been less successful than the low-density, but – beyond the fact that low-density settlers made a down-payment on their land, while high-density settlers did not – the reasons were not altogether clear. A colonial agriculturalist, writing about African agriculture, explained them in terms of the fable of the grasshopper and the ant. Some Africans, in his opinion, were simply harder-working and more prudent than others.
Once I got my research clearance, I was in a position to do my own work on the settlement scheme. The heart of my planned research was two sets of interviews with large random samples of the farmers in one high-density and one low-density settlement scheme. As I began to talk to people, it quickly became evident that the differences between the two schemes, and the people who got the land, were dramatic. A preponderance of the low-density settlers were upwardly mobile: absentee landlords in many cases and often owners of multiple pieces of land or businesses in different parts of the country.
The high-density settlers, by contrast, were mostly poor. All settlers, high and low-density, were required to pay for their land in instalments, and although this clearly posed no problems for the low-density settlers, many of their high-density counterparts were scratching a living from manifestly inferior soil, and struggling, or failing, to meet the payments. In short, the literature said one thing and the farmers were telling me something very different. What to do?
My puzzlement drove me to the Kenya National Archives in search of documentary evidence of who these two types of settlers were and how they had come to the land they occupied. The archives helped me to solve the puzzle. The solution is complicated, but, long story short, in doing my two sets of interviews, I had stumbled upon Kenya’s rural class system, and found the key to a fascinating and disturbing story of colonial duplicity and manipulation. My discovery, and the follow-up work in the archives, transformed my study from a development politics case into a study of the evolution of Kenya’s agrarian class system through the colonial and post-colonial eras.
As you might guess, the farmers and the archives told different aspects of the same story. My archival sources confirmed the observations I made on the ground. The history I was able to read out of them confirmed that the two types of settlement schemes did indeed serve two different social classes and explained how that came to be. In short, the archives and the interview findings agreed that the secondary sources from which I gained my initial impressions were wrong.
My research taught me a great deal about Kenya’s history and its agrarian class system. It also focused my mind, once and for all, on the fact that, if you can find three independent sources for the same story, you have a very good chance of working out who’s got the story right. That was my first experience with triangulation, which, ever since then, has become my modus operandi. It has served me well. In this blog, I relate two cases, that of the Eaton Centre in Edmonton and the Norwood Bridge in Winnipeg, where triangulation helped me get to the bottom of stories in which facts were being shaded or misrepresented.
In the Kenya case, my interviewees had the story right. More frequently, it is interviews that are least likely to be a reliable source of factual information. As a rule, I treat interviews as an invaluable source of background and context, because no one understands a situation better than those who have lived through it.
However, in assembling the facts – times, places, precise questions at issue – I always bear in mind that the facts cited in interviews are likely to be shaded by the frailties of human memory and the reality that we are all prone to remembering things as we wish they were, not necessarily as they are. (I think I remember this story as it actually was!)
Accordingly, I look for confirmation in documents or secondary sources of any facts I glean from interviews, and teach my research assistants to do the same. Of course, neither documents nor secondary sources are infallible. The best way to get near the truth is to look to all three types of sources for mutual confirmation, and never to settle for less than two.
Therefore, my answer to my colleague, who was wondering what to do if you suspect your interviewees aren’t telling the truth is: Triangulate. Always suspect your sources, and always look to other, independent sources for confirmation or contradiction. This becomes particularly important when your interviewees are power holders who, as mu colleague Jerry Krase has pointed out, may control some of the “independent” sources you are using to check on their stories.
It is a reality of political research that it often impinges on power holders, who invariably have definite ideas about what they want you to say in your report of their activities. The realities of power pose a host of problems for researchers, and for the ethics of politics research, as I point out in another blog entry.
But come what may, it’s a rare half-truth or lie that survives a determined search for independent sources.
•••••••••••••••••
Want to find out more? Various parts of my Kenya story are documented in detail in:
Christopher Leo, Land and Class in Kenya. Toronto: University of Toronto Press 1984.
Christopher Leo, Land and Class in Kenya (Southern African Edition). Harare, Zimbabwe: Nehanda 1989.
Christopher Leo, “Failure of the ‘Progressive Farmer’ in Kenya’s Million-Acre Settlement Scheme”. Journal of Modern African Studies 16 (4) 1978, 619-38.
Christopher Leo, “Who Benefited from the Million-Acre Scheme? Toward a Class Analysis of Kenya’s Transition to Independence.” Canadian Journal of African Studies. 15 (2) 1981, 201-22.
Here are some useful sources on research methodology, including interviews:
Robert K. Yin, Case Study Research: Design and Methods. Thousand Oaks: Sage, 2003.
Robert E. Stake, The Art of Case Study Research. Thousand Oaks: Sage, 1995.
Raymond L. Gorden, Interviewing: Strategy, Techniques, and Tactics (Fourth Edition). Chicago: Dorsey Press, 1987.
Another source, pitched especially to ethnographic research, is:
John and Lyn H Lofland, Analysing Social Settings: A Guide to Qualitative Observation and Analysis (Third Edition). London: Wadsworth, 1995.

var sc_project=1651625;
var sc_invisible=1;
var sc_partition=15;
var sc_security=”bb02f132″;

unique visitor counter

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s