What Should a Credible Clinical Research Competency Assessment Actually Measure?

If clinical research needs better evidence of capability, the next question is unavoidable: what should a credible competency assessment actually measure? The easiest answer is knowledge. Knowledge matters. A person cannot interpret a protocol, protect a participant or document a trial correctly without knowing the governing requirements. But a knowledge test and a competency assessment make different claims. A long knowledge quiz may show that someone can recall terms, recognize rules or reason through selected options under stated conditions. It does not automatically show that the person can find the relevant fact inside an imperfect record, decide what matters, act within the boundaries of the role, produce a usable artifact and escalate the right issue under time pressure. The difference is not philosophical. It determines how far an employer, educator or professional may legitimately trust the result.

Start with the claim, not the question bank

Every assessment creates an inference. We observe a response, then decide what that response permits us to say about the person. The wider the claim, the stronger the evidence must be.

"The participant selected the correct response on a protocol question" is a narrow claim. "The participant can independently manage protocol exceptions as a clinical research coordinator" is much broader. The second claim requires more than a correct answer. It requires a defined role, a realistic task, observable performance, known conditions, a versioned scoring method and an interpretation that stays inside the evidence.

That is why a credible assessment begins with the work. Before writing a question, define:

• the role and its authority;
• the responsibility being assessed;
• the decisions the role must make;
• the records or work products the role must produce;
• the risks the person must recognize;
• the issues that require escalation;
• the conditions under which the work will be performed; and
• the conclusion the result is intended to support.

This is the practical value of job analysis. The assessment target is not "clinical research" in general. It is a defined responsibility performed by a defined role in a defined context.

A credible assessment narrows the claim before it strengthens the score.

Ten assessment design checks

The following checks translate BORAKA-VCS-CR-001 into practical questions for assessment designers, employers, educators and professionals.

1. Define the role and the performance claim

What exactly is the person being asked to demonstrate? A coordinator, monitor and trial assistant may work with the same protocol, but they do not hold the same authority. A credible task respects that boundary. It can reward recognition, inquiry, documentation and escalation when those are the correct responsibilities of the role. It must not reward someone for making an unauthorized medical, regulatory or employment decision. The target should be specific enough that two reviewers can agree on what competent performance would look like and what the result would not establish.

2. Make the task resemble the work

Work samples are useful because they ask a person to perform tasks that are identical or meaningfully similar to the job. In clinical research, that may mean reviewing a protocol extract, checking source evidence, resolving a data discrepancy, prioritizing monitoring findings, documenting a deviation or drafting an escalation. Authenticity does not mean copying a confidential trial file. It means reproducing the relevant materials, constraints, ambiguity, time pressure and decision opportunities with fictional or properly governed data. A realistic task should be tied to job analysis. It should measure a responsibility that matters on entry or at the stated level, not an impressive-looking activity chosen because it is easy to score.

3. Require observable evidence

Competency becomes more visible when the participant must produce something another person can inspect. BORAKA-VCS-CR-001 recognizes several useful forms of evidence:

• knowledge evidence, such as an explanation or selected response;
• decision evidence, including the choice, reasoning, authority boundary and consequence;
• work-product evidence, such as a source note, query, monitoring finding, reconciliation or action log;
• system-interaction evidence from role-appropriate activity in tools such as EDC, CTMS or eTMF;
• communication evidence captured as a written product, transcript or structured observation; and
• longitudinal evidence from repeated performance over time in a governed setting.

Knowledge evidence can be appropriate. It should not be mistaken for performance evidence when the claim concerns doing the work.

4. Examine the reasoning, not only the final answer

Two people can select the same action for very different reasons. One may have identified the governing requirement and weighed the risk. Another may have guessed or followed a familiar pattern that happens to work in this case. A credible task makes the decision trail visible. It asks what the person knows, what remains uncertain, which source controls the decision, what assumption is being made and why the next action is proportionate. Ambiguity should be deliberate. Real records contain incomplete notes, conflicting dates and missing information. The task should require judgment without becoming a guessing game about what the designer intended.

5. Treat documentation as part of performance

In clinical research, the work and the record of the work cannot be separated cleanly. A sound decision documented vaguely may still create risk. A polished note that hides an unsupported decision is not good performance either. Assessment should examine whether the record is attributable, contemporaneous, accurate, complete, consistent and traceable. It should show whether another qualified person can reconstruct what happened, why it happened and what must happen next. Documentation is not an administrative extra added after the real task. It is part of the evidence that the task was performed responsibly.

6. Make participant protection and data reliability decisive

ICH E6(R3) places the rights, safety and well-being of trial participants and the reliability of trial results at the center of Good Clinical Practice. Assessment design should reflect that priority. This means distinguishing ordinary weaknesses from critical failures. An assessment may combine several criteria into a score, but a high average should not erase a response that knowingly proceeds without valid consent, ignores urgent safety information, obscures an audit trail, fabricates a source entry or exceeds role authority in a way that creates material risk.

Critical rules should be defined before administration, applied consistently and accompanied by a documented outcome, remediation path and appeal or correction process.

7. Standardize the conditions that affect interpretation

If results will be compared, the conditions must be materially equivalent. Participants should encounter the same target competency, comparable source information, similar constraints and equivalent opportunities to demonstrate judgment. Timing, navigation, permitted resources, communication rules and attempt limits should be declared rather than inferred after the fact.

Fairness does not require pretending every person has identical needs. Appropriate accommodations may remove barriers that are unrelated to the competency. The assessment must document the adjustment and whether it changed the intended construct. Accent, voice quality, native-language fluency, disability-related characteristics or inferred emotion should never be scored unless a job analysis establishes a legitimate and lawful connection to the specific competency being assessed.

8. Preserve provenance and traceability

A score without provenance is difficult to audit and easy to overread. The evidence record should identify the role profile, competency, task, casebook, source artifacts, rubric, assessment mode, administration conditions and review version that produced the result.

Competency requirement to role profile to task to source material to candidate action to observable evidence to rubric judgment to human review to evidence record.

Versioning matters because protocols, regulations, technology, rubrics and role expectations change. A reader should be able to interpret an older result without pretending it was produced under today's requirements.

9. Keep artificial intelligence bounded and human accountability visible

AI can make practice more frequent and accessible. It can support role-play, transcription, structured observation and preliminary analysis. These uses may help a professional receive faster developmental feedback. They do not remove the need for accountability when a stronger claim is issued.

Under BORAKA-VCS-CR-001, an AI system may not independently issue, deny, revise or invalidate BORAKA Verified status. A qualified person must make the final decision against preserved evidence and a current rubric, with reviewer qualification, conflicts, moderation and appeal controls documented. This boundary is not an argument against AI. It is a statement about who remains answerable for a verified claim.

10. State what the result supports and what it does not

The most important discipline may come after scoring. An assessment result should say which role, task and conditions were assessed; what performance was observed; how strong the supporting evidence is; who reviewed it; and what limitations remain. One simulation does not establish universal readiness. A verified result is not a professional license, a regulatory authorization, a guarantee of future performance or a substitute for an employer's qualification, delegation and supervision decisions. This is consistent with established testing practice: validity attaches to an intended interpretation and use, not to a score in the abstract. Credibility depends on keeping the claim within the evidence that supports it.

Do not collapse three questions into one score

BORAKA-VCS-CR-001 separates three questions that are often hidden inside one number:

• 1. Performance level: How well did the person perform the defined work?
• 2. Evidence Assurance Level: How strong is the evidence behind that claim?
• 3. Evidence status: What is the record's current lifecycle state?

These dimensions can move independently. A participant may produce a strong response in an open practice exercise. The performance may be useful for learning, but the evidence assurance is limited because identity, administration and review controls were not present. Another participant may complete a standardized task and receive qualified human review, but a critical safety rule may prevent verified status.

"Human-reviewed" and "BORAKA Verified" are not synonyms. A summary Competency Score may help a reader navigate the record, but it cannot replace domain-level performance, critical-rule outcomes, evidence conditions and interpretation limits.

Practice, structured assessment and verification make different promises

The words used for an experience should match the controls behind it.

• Practice is for learning. It may be repeatable, coached, untimed and supported by immediate feedback. Practice does not create verified evidence.
• Structured assessment uses defined tasks, source materials and rubrics, but may not include every identity, timing, attempt, preservation or review control required for verification.
• Verified assessment adds authenticated linkage, standardized conditions, versioned materials, preserved evidence, qualified human final review, declared integrity controls and an appeal, correction and invalidation process.

BORAKA's present product status follows that distinction. Practice is open today. Human-reviewed evidence is operating as a controlled beta. A generally available formal assessment service and portable evidence record remain in design. Those are different stages, and they should be described without borrowing credibility from one another.

What better assessment changes

For professionals, a good assessment turns a vague career problem into a specific development problem. Instead of "I need more experience," the question becomes: Which responsibility can I not yet demonstrate? Is the gap knowledge, judgment, documentation, system use, communication, repetition or evidence assurance?

For employers, aggregate patterns could inform onboarding and development. If a group repeatedly misses the same safety escalation or documentation requirement, the response may be clearer procedures, targeted training, supervised practice or better role design. That does not mean an assessment result should make the hiring decision. It gives the hiring team better evidence to examine alongside experience, interviews, references, role requirements and lawful selection procedures.

Five questions to ask before trusting an assessment

Before relying on any clinical research competency result, ask:

• 1. What role and job responsibility does this assessment represent?
• 2. What observable evidence did the participant produce?
• 3. Under what conditions was that evidence produced and preserved?
• 4. How was the evidence scored and who remained accountable for the final judgment?
• 5. What conclusion does the result support, and what does it explicitly not prove?

If those questions cannot be answered, the assessment may be measuring activity without producing defensible evidence. Clinical research does not need more testing for its own sake. It needs clearer claims, better tasks, stronger evidence and more disciplined interpretation. That is the standard a credible competency assessment should meet.

Sources

• BORAKA Verified Competence Standard for Clinical Research, BORAKA-VCS-CR-001, Version 1.0.0: https://zenodo.org/records/22256856
• BORAKA public standard page: https://boraka.co/standard
• ICH E6(R3) Good Clinical Practice, Final Consolidated Guideline: https://database.ich.org/sites/default/files/ICH%20E6(R3)_Step4_FinalConsolidatedGuideline_2026_0616_.pdf
• Joint Task Force for Clinical Trial Competency, Domains and Leveled Core Competencies: https://mrctcenter.org/clinical-trial-competency/framework/domains/
• Standards for Educational and Psychological Testing: https://www.apa.org/science/programs/testing/standards.html
• U.S. Office of Personnel Management, Work Samples and Simulations: https://www.opm.gov/policy-data-oversight/assessment-and-selection/other-assessment-methods/work-samples-and-simulations/