Research Methodology Consulting

Why the timing matters more than the advice

Almost every serious problem in a doctoral thesis was created before any data existed.

A cross-sectional survey cannot be made to demonstrate causation afterwards by any analysis. An instrument that was never validated in your population produces precise numbers about nothing, and no technique repairs it. A sample recruited through one organisation cannot support a claim about a sector. A first phase collected anonymously makes purposeful selection for a second phase impossible.

None of those are analysis problems, and all of them are usually presented as analysis problems — because that is the stage at which they become visible. By then the fix is not a better method; it is a narrower claim, a redesign, or another year.

The asymmetry is stark. An hour spent on the design before collection can save a year afterwards, and this is the single most repeated observation among people who have supervised and examined doctorates.

What gets examined

A design review works forwards through the chain, checking that each link supports the next.

• The question. Is it specific enough to imply a method? If two entirely different designs would both suit it, it is still too broad — and that is fixable now.
• The gap. Is it something the field cannot explain, or just something nobody has written about? Examiners press hardest here.
• The design. Can it establish what you intend to claim — particularly around causation, generalisation and change over time?
• Philosophical consistency. Whether the position stated in the methodology is contradicted by what the study actually does. A common and visible flaw.
• Sampling. Who, how many, chosen how, and whether that supports the claim. For quantitative work, whether the number was argued from an effect size worth detecting rather than from what seemed achievable.
• Instruments. Whether existing validated measures exist, whether they transfer to your population, and what modifying them costs you.
• The analysis plan. Whether the analysis you intend is possible with the data you will actually have — the check that catches the most expensive errors.
• Practical constraints. Access, timing, transcription load and what your ethics committee is likely to ask.

When it is worth doing

Before submitting a proposal or upgrade document. An independent read finds the questions your panel will ask while you can still answer them.

Before an ethics application. Committees push on consent, anonymity, data storage and vulnerable participants. Anticipating that is faster than being sent back, particularly in sequential designs where delay compounds.

Before you start recruiting. The last genuinely cheap moment. After this, changes cost data.

When your supervisor is not a methodologist. Common and not a criticism — supervisors are chosen for subject expertise, and may not have used the design you are proposing. A second opinion from someone who has is not disloyalty.

When you are choosing between two designs and can argue for both. Usually a sign the question needs narrowing, which is quicker to hear from outside.

When something feels wrong and you cannot name it. Often accurate instinct. Describing it aloud to someone who has examined doctorates tends to locate it quickly.

How it works

1. You send what exists — a proposal, a draft methodology chapter, or just a description of what you intend. Early and rough is fine; that is the point.
2. A first conversation, at no cost, to establish what you actually need. Sometimes that is a full review; sometimes it is one answer to one question, and you will be told if so.
3. You are matched with a PhD in your own field. Methodological conventions differ between disciplines — what counts as an adequate sample or a defensible design is not universal — so field knowledge matters as much as methods knowledge.
4. You receive a written review identifying what holds, what does not, and what each problem would cost if left. Prioritised, so you know what to fix first.
5. You discuss it, because a written review answers the questions someone thought to ask, and you will have others.

NDAs available before any detail is discussed. Your ideas and design remain yours.

What to expect from the answer

A useful review is often uncomfortable, and it is worth knowing that in advance.

You may be told the question is too broad, that your design will not support the claim you want, that the sample you can realistically access will not carry the analysis you planned, or that the instrument you intended to use has not been validated for your population. Those are the findings that justify the exercise. A review that agrees with everything has told you nothing you did not already believe.

You may also be told the design is sound and to proceed — which is worth knowing before you commit two years to it.

What a review will not do is make the decision for you. It identifies what each option costs and what each claim requires; the design remains yours to defend, which is the position you need to be in when the viva comes.