Reviewer Guidelines

1. Expertise: you can assess a substantial part of the manuscript. Accept for the part you can judge and tell the editor which part that is — a review that covers the method and says so is more useful than one that covers everything vaguely.
2. Competing interests: no shared institution, recent collaboration, co-authorship or funding relationship with the authors, and no commercial interest in the system, vendor or product evaluated. Declare anything that a reader might reasonably see as a conflict and let the editor judge it.
3. Time: you can return the review within the agreed period. If you cannot, decline promptly so the editor can invite someone else.
4. Confidentiality: you accept that the manuscript is confidential.

Suggesting an alternative reviewer when you decline is a substantial help to the editor.

Confidentiality

A manuscript under review is confidential. Do not share it, discuss it outside the review, or use its content, data or ideas in your own work before publication.

Involving a colleague

Ask the editor first. A co-reviewer, including a trainee, may take part only with the editor's agreement and is named in the report.

Review model

Single-blind by default; double-blind review is available on request.

Double-blind manuscripts

For double-blind manuscripts, reviewers must not attempt to identify the authors.

AI use

Reviewers must obtain journal permission before using AI to assist with a review. Any permitted use must be disclosed, and confidential manuscript content must not be uploaded where confidentiality cannot be assured.

Files

After submitting the review, reviewers should securely delete downloaded manuscript files and must not retain or use unpublished material.

Tone

Address the work, never the authors. A criticism worth making can always be made without disparagement.

Healthcare decision-science and methodological research

Evaluate the contribution to medical or health informatics, or healthcare decision science. For decision analysis, assess alternatives, assumptions, preferences or utilities, uncertainty and the relevance of conclusions to the decision. Formal, conceptual and methods papers should be judged by the reasoning and evaluation appropriate to their claims. Do not require a new digital system, clinical cohort, deployment or patient outcomes for every study. Apply the data, software and ethics questions below where relevant.

Machine-learning and AI studies

1. Dataset provenance: where the data came from, over what period, and how the labels or reference standard were derived.
2. Leakage: whether the split was made at the right unit, and whether any feature could encode the outcome or the time of it.
3. Validation: internal validation done properly, and external validation where the claim reaches beyond the development setting.
4. Comparator: current practice, a clinical score, or a simpler model. A result with no baseline cannot be interpreted.
5. Metrics: appropriate to the task and the class balance, reported with uncertainty.
6. Calibration, where the output is used as a probability.
7. Bias and fairness across the groups the system would be used on.
8. Generalizability: what would have to hold for this to work elsewhere.
9. Reproducibility: preprocessing, hyperparameters, seeds, code and model version.
10. Whether the claims exceed the evidence.

Clinical NLP and imaging informatics

1. Corpus or image set: size, source, and how representative it is of the intended setting.
2. Annotation: who annotated, against what guideline, and with what inter-annotator agreement.
3. The reference standard, and its own reliability.
4. Metrics suited to the task, at the right level — mention, document, study or patient.
5. Generalization to another institution's notes, scanners or protocols.
6. Error analysis: what the system gets wrong, and whether those errors matter clinically.

Clinical decision support

1. The decision, and the point in care at which it is made.
2. The intended user, and whether the output is usable by them.
3. The comparator: what that decision rests on today.
4. Workflow: how the recommendation is delivered, and what happens when it is overridden or ignored.
5. Whether the evaluation measures model performance, user behavior or care outcomes — and whether the conclusion matches which of the three it measured.
6. Risks: alert burden, automation bias, and the consequence of each type of error.

Information-system and implementation studies

1. Context: the organization, its scale, its existing systems and the period of deployment.
2. Evaluation design, and what it can and cannot establish.
3. Usability and workflow effects, measured rather than asserted.
4. Adoption: who used it, how much, and what happened when they did not.
5. Transferability: what is specific to this site.

1. Summary

   Two or three sentences on what the work does and what it contributes, so the editor and the authors can see you read it as intended.

2. Major points

   Numbered, each identifying the problem, where it is in the manuscript, and what would resolve it. Separate what must be fixed from what would strengthen the paper.

3. Minor points

   Numbered, for specific corrections and clarifications.

4. Recommendation

   Accept, minor revision, major revision, or decline — consistent with the report. A recommendation to decline is supported by the major points, and a recommendation to accept is not accompanied by unresolved ones.

5. Confidential comments to the editor

   Anything the editor needs and the authors should not see: a suspected duplicate, an integrity concern, the limits of your own expertise on part of the manuscript.

Where you suspect plagiarism, duplicate publication, image manipulation, fabricated data or an undisclosed conflict, do not raise it with the authors. Report it in the confidential comments with the evidence you have, and leave the investigation to the editor.