As I start to collate information for my final essay and a research paper, I have been revisiting my approach to determining an appropriate sample size for my research. This is particularly difficult to establish at the start of a research project when it is unknown (and hard to predict) how many participants are needed before the saturation point in data collection will be reached.
Several factors influence the sample size required to give sufficient, reliable results: the aim of the study (how focused it is); the nature of the participant group (homogenous or not); the quality of data received from participants; and how the data will be analysed.
My instincts, logic and understanding of the building design and delivery process told me that my research involves a relatively homogenous population of participants, i.e. the people involved in designing, delivering or procuring buildings. Therefore, I could reasonably suggest similar views from participants. My research aim is narrowly focused on assessing opinions of my developing design guidance. Those participating are likely to have strong views on the usefulness and style of the design guidance, as they are used to referring to design guidance whilst developing building projects – thereby providing good quality data. And my research doesn’t involve cross examining data from participants, but rather collecting data and using to identify common views to influence my final design guidance documents.
However, to give my research academic credibility, it can’t be based on my own assumptions! I have therefore used 3 methods to determine an appropriate sample size. In the research design stage, I used a table produced by a researcher in 2003 (Israel, 2003), which has been widely used in qualitative research projects since publication.
This told me that for a population of 1,127 architectural designers (architects and architectural technologists) in Wales, I should consider a sample size of 95 participants sufficient. By carrying out focus groups, this seemed a reasonable number to aim for, and in December 2019 I started to arrange face to face focus groups, designed as CPD seminars, as well as a few interviews.
Then, due to the national lockdown in March this year, I had to adapt my research method to run online focus groups instead of face to face sessions. There were pros and cons to this. One advantage was the ability to reach more people, increasing attendance at the sessions; but a downside was that this would broaden the sample group to a wider range of participants. My final study group included 242 participants, of which 61 were architectural designers (64% of the original sample size). The question was, did this matter to the study?
The second method I looked at was developed by researchers Malterud, Siersma and Guassora (2006), which they called the Information Power Method. This diagram illustrates their criteria for determining sample size:
Applying the characteristics of my project to this, gave me confidence that my sample group would have high Information Power, hence a smaller sample size should be needed before the saturation level is reached within the data.
This was validated during my data analysis when examining the data and setting out to determine the saturation point, i.e. the point at which no new data was found – in this case, when no new themes emerged from the data. The results are shown in this table and graph, where FG = Focus Group and Int = Interview:
It is clear from these results that, after just 5 sessions (focus groups or interviews) no new themes emerged, despite the number of themes present in the data and regardless of the number of participants. This suggests that a relatively small sample size would have provided sufficient data for this narrowly focused study. The wider range of individual participants did not have an impact on the data, with the same themes being discussed in each group, regardless of whether they were architectural designers, engineers, surveyors, building control officers, or project managers. This makes sense as the information I have put together in the design guidance (the Toolkit) is relevant to all involved in building projects.
Hopefully, this information will be useful to others involved in qualitative research.
[I will be running another webinar on 11th December. Please contact me for further details: firstname.lastname@example.org]