Sampling & Generalisability
Distinguish selecting a sample from assigning treatment, and judge whose experience the results actually represent.
Quick revision ↓ Medical Dictionary
01Population, frame and sample
The target population is the group to which the question refers. The sampling frame is the list or practical source from which participants can be selected, and the sample is the group actually studied. A frame limited to registered clinic attenders can miss people unable to access care. Define inclusion and exclusion criteria before recruitment and compare the realised sample with the intended population.
02Probability sampling
In probability sampling, eligible units have known non-zero selection probabilities. Simple random sampling selects from a complete frame; stratified sampling samples within relevant subgroups; cluster sampling first selects groups such as schools or villages. Each design has implications for analysis and weighting. People within a cluster may resemble one another, so treating clustered observations as fully independent can make confidence intervals misleadingly narrow.
03Convenience and recruitment
A convenience sample recruits those easiest to reach; consecutive recruitment includes all eligible people presenting during a defined period. These approaches may be practical but can limit representativeness. A large convenience sample does not necessarily correct systematic exclusion. Report where, when and how participants were approached, how many declined, and whether recruitment patterns differ across groups or centres.
04Non-response and missingness
Non-response can distort estimates if participation is related to the characteristic being studied. A high response rate is helpful but does not prove freedom from bias. Compare available characteristics of responders and non-responders, explain reasons for missingness and consider sensitivity analyses. Weighting can help under defensible assumptions; it cannot recover information about completely unrepresented people without additional knowledge.
05Internal and external validity
Internal validity asks whether the findings credibly describe the studied comparison. External validity asks whether they apply elsewhere. A rigorously conducted specialist-centre trial may not transfer directly to a rural service with different staffing, baseline risks and access. Random sampling supports population representation, whereas random assignment supports fair treatment comparison. These are distinct operations and neither should be inferred merely from the word random.
High-Yield Revision
- Random sampling selects people; randomisation assigns interventions.
- A larger sample does not repair systematic selection bias.
- Check setting, eligibility, participation and baseline risk before applying results.
Common Pitfalls
- Do not treat clustered observations as automatically independent.
- Do not assume a high response rate guarantees representativeness.
References & Further Reading
Use the full methods and relevant chapters when appraising a paper. These sources support further study; the examples and questions above are original teaching exercises.
- Oxford CEBM — Critical appraisal tools
- Cochrane Handbook — Methods for evidence synthesis
- CDC — Analysing and interpreting epidemiological data
- ASA — Statement on p-values and statistical significance
- Oxford CEBM — Likelihood ratios
- Cochrane — GRADE and summary of findings
- WMA — Declaration of Helsinki, 2024 revision
- EQUATOR — Reporting guideline library
- NIH OBSSR — Mixed-methods research resources
- Institute for Healthcare Improvement — Model for Improvement