Exploratory Sequential Design

When this design earns its place

Use it when you cannot measure something because nobody has yet established what should be measured, in your setting.

The situations where it genuinely fits:

• No suitable instrument exists for the construct in your population, and you need one to answer your question.
• Existing instruments were developed elsewhere and may not transfer — a scale validated with North American undergraduates being applied to mid-career professionals in a different country, for instance.
• The construct itself is understood differently in your context, so items written elsewhere would measure something subtly different.
• You need to identify the relevant variables before you can model anything, because the literature does not tell you what matters here.

It is the wrong choice when a validated instrument already exists and fits — adapting one is far cheaper than building one — and when your timeline cannot absorb instrument development, which is most of the time if it was not planned for.

The building step is a project, not a transition

This is where the design succeeds or fails, and it is what distinguishes a strong exploratory sequential thesis from a weak one.

Turning qualitative findings into a measuring instrument involves, at minimum:

1. Deciding what becomes an item. Which themes, sub-themes or participant statements convert into measurable statements, and which do not. This is an analytic decision, not a clerical one, and it needs describing in detail.
2. Writing items. Often using participants’ own language, which is one of the design’s real advantages — the wording comes from the population rather than from a researcher’s vocabulary.
3. Deciding response format and scale. A decision with consequences for every analysis that follows.
4. Expert review. Specialists judging whether the items cover the construct and whether any are ambiguous or double-barrelled.
5. Cognitive interviewing. Sitting with a handful of people from your target population while they think aloud through the items. This reliably finds questions being read in ways you did not intend, and it is the cheapest fix available.
6. Piloting. A trial administration to check timing, completion and item behaviour.
7. Validation in phase two. Factor analysis to check the structure, reliability analysis, and evidence for validity.

Each step is ordinary practice in instrument development, and together they are a substantial piece of work. A proposal saying “findings from phase one will inform the development of a questionnaire” has compressed all of that into a single clause, and supervisors reading proposals should push back on it.

The methodology chapter has to make the translation traceable: a reader should be able to see which qualitative finding produced which item. A table mapping themes to items is the standard way of showing it, and its absence is one of the first things questioned.

Who phase two is for

A design decision that is easy to get wrong: phase two normally uses different participants from phase one.

The reason is straightforward. An instrument built from a group’s own accounts will fit that group unusually well, so testing it on the same people tells you very little about whether it works. Validation needs a fresh sample from the same population.

That has consequences worth planning for. You need access to two samples, not one, and the second needs to be large enough for whatever analysis you intend — factor analysis in particular requires substantially more participants than most qualitative phases involve. Where the population is small or hard to reach, this constraint sometimes rules the design out entirely, which is better discovered at the proposal stage than in year three.

Timing and approvals

Three phases, not two. Qualitative collection and analysis; instrument development and piloting; quantitative collection and analysis. Timelines that show two phases have hidden the middle one, and it is often the longest.

Ethics in stages. You cannot submit a questionnaire you have not yet written. As with any sequential design, the usual route is to describe the intended process in the original application and submit the instrument as an amendment. Confirm with your committee before submitting.

The instrument is a contribution. Worth saying because candidates undersell it: a validated instrument for a context that lacked one is a genuine, citable output, and often the part of the thesis with the longest life. Report its development fully rather than treating it as preliminary work.

Have a fallback. If the instrument performs poorly in validation — items loading unexpectedly, weak reliability — you need a plan that is not “the thesis fails”. Reporting honestly what did and did not work, and what that reveals about the construct in your population, is itself a finding. Examiners are considerably more receptive to that than to a scale reported as sound when the output shows otherwise.

Where this design fails

• Instrument development treated as a step, not a phase. The single most common failure, and it shows up as a missed deadline rather than a methodological criticism.
• No validation. A questionnaire built from interviews and then used to produce findings, with no evidence it measures anything reliably.
• The translation is not described. Items appear without any traceable link to the qualitative findings that produced them.
• Phase two run on phase one participants. Validation on the people whose words generated the items shows almost nothing.
• A phase two sample too small for the analysis. Particularly for factor analysis, where requirements are substantial.
• Existing instruments never checked. Building from scratch when an adaptable, validated measure already existed is expensive and hard to justify.