One of the ways to identify and address potential validity threats is to conduct a pilot study prior to running the full study. The goal of a pilot study is to validate the study design. The goal of a pilot study is not to gather data to draw conclusions. Any data gatered during a pilot study should discarded. The pilot study provides an opportunity to identify potential problems with study execution before it is too late.
For example, assume a study design that includes a survey. Survey questions can easily be interpreted differently by respondents than by the survey author. Piloting the survey will allow the researcher to identify questions that may be unclear. If the researcher does not pilot the survey, they run the risk of receiving unusable results because the question was unclear. A small pilot study can help address this problem and increase the chances of valid data.