# Bias Library

Curated offline mini-KB for the Critical Compass skill.

## Contents

- Cognitive
- Cultural
- AI
- Research Methodology

- Source of truth: `/Volumes/KyleSSD/CriticalThinking/db/critical-compass.sqlite`
- Use this when the MCP is unavailable or when you want a fast human-readable lookup.
- Signal phrases are KB-derived cues from signals, examples, or summaries, not invented labels.

## Cognitive

### Confirmation bias
- **ID:** ct:biases:cognitive:confirmation-bias
- **Definition:** Disconfirming data is discounted or ignored.
- **Signal phrase:** Including only favorable case studies in a proposal.
- **Example:** Including only favorable case studies in a proposal.
- **Mitigation:** Pre-register criteria
- **Confidence:** high
- **Three Cs:** 🔍

### Anchoring bias
- **ID:** ct:biases:cognitive:anchoring-bias
- **Definition:** Initial values unduly influence subsequent estimates and choices.
- **Signal phrase:** First offer sets the negotiating range.
- **Example:** First offer sets the negotiating range.
- **Mitigation:** Multiple independent anchors
- **Confidence:** high
- **Three Cs:** 🔍

### Sunk cost fallacy
- **ID:** ct:biases:cognitive:sunk-cost-fallacy
- **Definition:** Irrecoverable costs irrationally influence go/no-go decisions, escalating commitment to losing courses of action.
- **Signal phrase:** Keeping a failed initiative alive because 'we’ve already spent six
- **Example:** Keeping a failed initiative alive because 'we’ve already spent six months'.
- **Mitigation:** Base on expected future value
- **Confidence:** high
- **Three Cs:** 🔍

### Status quo bias
- **ID:** ct:biases:cognitive:status-quo-bias
- **Definition:** Loss aversion, regret aversion, and inertia make existing options feel safer, overweighting switch costs.
- **Signal phrase:** Keeping an outdated workflow 'because it works' despite persistent rework.
- **Example:** Keeping an outdated workflow 'because it works' despite persistent rework.
- **Mitigation:** Opt-out design with clear reversibility
- **Confidence:** high
- **Three Cs:** 🔍

### Negativity bias
- **ID:** ct:biases:cognitive:negativity-bias
- **Definition:** Attention and memory are tuned to threats and losses, skewing perception and decisions.
- **Signal phrase:** Overreacting to one bad review while ignoring dozens of good
- **Example:** Overreacting to one bad review while ignoring dozens of good ones.
- **Mitigation:** Separate risk review from ideation
- **Confidence:** high
- **Three Cs:** 🔍

### Belief bias
- **ID:** ct:biases:cognitive:belief-bias
- **Definition:** Reasoners accept invalid arguments with believable conclusions and reject valid ones with unbelievable conclusions.
- **Signal phrase:** Dismissing a valid but counterintuitive study because the result feels
- **Example:** Dismissing a valid but counterintuitive study because the result feels wrong.
- **Mitigation:** Blind logic checks
- **Confidence:** high
- **Three Cs:** 🔍

### Choice-supportive bias
- **ID:** ct:biases:cognitive:choice-supportive-bias
- **Definition:** Post-choice memory distorts to favor the selected option.
- **Signal phrase:** Recalling only positives about a purchased product.
- **Example:** Recalling only positives about a purchased product.
- **Mitigation:** Pre/post criteria comparison
- **Confidence:** high
- **Three Cs:** 🔍

### Disconfirmation bias
- **ID:** ct:biases:cognitive:disconfirmation-bias
- **Definition:** We apply higher standards to counter-attitudinal data, fueling motivated skepticism.
- **Signal phrase:** Demanding preregistration from opposing studies but not from aligned ones.
- **Example:** Demanding preregistration from opposing studies but not from aligned ones.
- **Mitigation:** Symmetric evidence quality checklists
- **Confidence:** high
- **Three Cs:** 🔍

### Information bias
- **ID:** ct:biases:cognitive:information-bias
- **Definition:** Preference for more data even when it lacks decision value.
- **Signal phrase:** Ordering tests that won’t alter treatment.
- **Example:** Ordering tests that won’t alter treatment.
- **Mitigation:** Value of information checks
- **Confidence:** high
- **Three Cs:** 🔍

### Attentional bias
- **ID:** ct:biases:cognitive:attentional-bias
- **Definition:** We notice information consistent with current concerns and miss other relevant cues.
- **Signal phrase:** Seeing only security risks during a design review and missing
- **Example:** Seeing only security risks during a design review and missing usability problems.
- **Mitigation:** Checklists
- **Confidence:** high
- **Three Cs:** 🔍

### Bias blind spot
- **ID:** ct:biases:cognitive:bias-blind-spot
- **Definition:** We detect and criticize bias in others more readily than in ourselves, inflating our sense of objectivity.
- **Signal phrase:** A reviewer flags others’ confirmation bias while ignoring their own.
- **Example:** A reviewer flags others’ confirmation bias while ignoring their own.
- **Mitigation:** Third-party audits
- **Confidence:** high
- **Three Cs:** 🔍

### Actor–observer bias
- **ID:** ct:biases:cognitive:actor-observer-bias
- **Definition:** Attribution asymmetry based on perspective and information access.
- **Signal phrase:** They’re late because they’re lazy; I’m late due to traffic.
- **Example:** They’re late because they’re lazy; I’m late due to traffic.
- **Mitigation:** Swap perspectives
- **Confidence:** high
- **Three Cs:** 🔍

### Groupthink
- **ID:** ct:biases:cognitive:groupthink
- **Definition:** Cohesive groups self-censor, assume unanimity, and pressure dissenters, degrading decision quality.
- **Signal phrase:** Rubber-stamping a risky launch because 'everyone seems aligned'.
- **Example:** Rubber-stamping a risky launch because 'everyone seems aligned'.
- **Mitigation:** Assign red teams
- **Confidence:** high
- **Three Cs:** 🔍

### Selection bias
- **ID:** ct:biases:cognitive:selection-bias
- **Definition:** Sample differs systematically from target population.
- **Signal phrase:** Only power-users respond to a feature survey.
- **Example:** Only power-users respond to a feature survey.
- **Mitigation:** Randomization
- **Confidence:** high
- **Three Cs:** 🔍

### Neglect of probability
- **ID:** ct:biases:cognitive:neglect-of-probability
- **Definition:** People focus on payoff magnitude and narratives, not likelihood, leading to poor expected-value choices.
- **Signal phrase:** Over-investing in edge-case features while neglecting the common path.
- **Example:** Over-investing in edge-case features while neglecting the common path.
- **Mitigation:** Pair magnitude with probability bands
- **Confidence:** high
- **Three Cs:** 🔍
## Cultural

### Authority bias
- **ID:** ct:biases:cultural:authority-bias
- **Definition:** Status cues beat evidence evaluation.
- **Signal phrase:** Accepting a senior exec’s plan without scrutiny.
- **Example:** Accepting a senior exec’s plan without scrutiny.
- **Mitigation:** Argument-from-evidence prompts
- **Confidence:** high
- **Three Cs:** 🔍

### Ingroup bias
- **ID:** ct:biases:cultural:ingroup-bias
- **Definition:** Preferential treatment and trust toward ingroup members.
- **Signal phrase:** Scoring candidates from your alma mater higher.
- **Example:** Scoring candidates from your alma mater higher.
- **Mitigation:** Blind evaluation
- **Confidence:** high
- **Three Cs:** 🔍

### Egocentric bias
- **ID:** ct:biases:cultural:egocentric-bias
- **Definition:** Self-focus inflates perceived agreement and effort share.
- **Signal phrase:** Teammates’ effort totals sum to more than 100%.
- **Example:** Teammates’ effort totals sum to more than 100%.
- **Mitigation:** Perspective-taking
- **Confidence:** high
- **Three Cs:** 🔍

### Omission bias
- **ID:** ct:biases:cultural:omission-bias
- **Definition:** Moral weight is unevenly assigned, excusing 'not acting' relative to acting.
- **Signal phrase:** Refusing a safe vaccine seems 'safer' than consenting to it.
- **Example:** Refusing a safe vaccine seems 'safer' than consenting to it.
- **Mitigation:** Compare counterfactuals
- **Confidence:** high
- **Three Cs:** 🔍

### Dehumanization bias
- **ID:** ct:biases:cultural:dehumanization-bias
- **Definition:** Language and framing strip out mind and moral worth, enabling harsher treatment.
- **Signal phrase:** Calling users 'traffic' or 'units' in planning.
- **Example:** Calling users 'traffic' or 'units' in planning.
- **Mitigation:** Use person-first language
- **Confidence:** high
- **Three Cs:** 🔍

### Height bias
- **ID:** ct:biases:cultural:height-bias
- **Definition:** Physical stature inappropriately influences social and economic outcomes.
- **Signal phrase:** Preferring taller candidates for sales leadership absent performance data.
- **Example:** Preferring taller candidates for sales leadership absent performance data.
- **Mitigation:** Define role criteria narrowly
- **Confidence:** high
- **Three Cs:** 🔍

### Bandwagon effect
- **ID:** ct:biases:cultural:bandwagon-effect
- **Definition:** Conformity and social proof drive uptake independent of evidence.
- **Signal phrase:** Picking a framework because 'everyone uses it'.
- **Example:** Picking a framework because 'everyone uses it'.
- **Mitigation:** Independent evaluation
- **Confidence:** high
- **Three Cs:** 🔍
## AI

### Availability heuristic
- **ID:** ct:biases:ai:availability-heuristic
- **Definition:** Salient or recent examples inflate perceived probability.
- **Signal phrase:** Overestimating plane crashes after news coverage.
- **Example:** Overestimating plane crashes after news coverage.
- **Mitigation:** Use base rates
- **Confidence:** high
- **Three Cs:** 🔍

### Anthropomorphism bias
- **ID:** ct:biases:ai:anthropomorphism-bias
- **Definition:** We over-infer intent or emotion in animals, devices, and algorithms, distorting expectations.
- **Signal phrase:** Assuming an assistant 'tried to ignore me' rather than hit
- **Example:** Assuming an assistant 'tried to ignore me' rather than hit a parser limit.
- **Mitigation:** Capability disclaimers
- **Confidence:** high
- **Three Cs:** 🔍

### Hindsight bias
- **ID:** ct:biases:ai:hindsight-bias
- **Definition:** Outcome knowledge reshapes memory of prior uncertainty.
- **Signal phrase:** 'We knew it all along' after a surprise launch flop.
- **Example:** 'We knew it all along' after a surprise launch flop.
- **Mitigation:** Time-stamped predictions
- **Confidence:** high
- **Three Cs:** 🔍

### Attractiveness (beauty premium) bias
- **ID:** ct:biases:ai:attractiveness-bias
- **Definition:** Appearance colors perceptions of competence, trustworthiness, and intelligence across domains.
- **Signal phrase:** Rating similar resumes higher when paired with an attractive photo.
- **Example:** Rating similar resumes higher when paired with an attractive photo.
- **Mitigation:** Blind work samples
- **Confidence:** high
- **Three Cs:** 🔍

### Framing effect
- **ID:** ct:biases:ai:framing-effect
- **Definition:** Gain vs. loss frames, or reference points, shift preferences.
- **Signal phrase:** Choosing a '95% survival' option over '5% mortality' despite equivalence.
- **Example:** Choosing a '95% survival' option over '5% mortality' despite equivalence.
- **Mitigation:** Neutral reframe checks
- **Confidence:** high
- **Three Cs:** 🔍

### Survivorship bias
- **ID:** ct:biases:ai:survivorship-bias
- **Definition:** Skips the denominator, inflating success narratives.
- **Signal phrase:** Studying only successful startups for strategy lessons.
- **Example:** Studying only successful startups for strategy lessons.
- **Mitigation:** Account for base rates
- **Confidence:** high
- **Three Cs:** 🔍

### Self-serving bias
- **ID:** ct:biases:ai:self-serving-bias
- **Definition:** Protects self-esteem at the cost of accuracy.
- **Signal phrase:** Blaming market conditions for missed targets, crediting skill for wins.
- **Example:** Blaming market conditions for missed targets, crediting skill for wins.
- **Mitigation:** External facilitation
- **Confidence:** high
- **Three Cs:** 🔍

### Halo effect
- **ID:** ct:biases:ai:halo-effect
- **Definition:** Global impressions bias evaluations of unrelated attributes.
- **Signal phrase:** Attractive candidates rated as more competent.
- **Example:** Attractive candidates rated as more competent.
- **Mitigation:** Attribute-by-attribute scoring
- **Confidence:** high
- **Three Cs:** 🔍

### Hallucination-confidence risk
- **ID:** ct:biases:ai-data:hallucination-confidence-risk
- **Definition:** Hallucination-confidence risk appears when generated or AI-mediated text sounds settled, complete, or definitive while citations, source coverage, or direct evidence remain invisible.
- **Signal phrase:** clearly shows; proves
- **Example:** An AI-generated brief says a study clearly proves statewide policy success but gives no citation trail.
- **Mitigation:** Verify the load-bearing claim against source material and revise certainty language until it matches the evidence.
- **Confidence:** medium
- **Three Cs:** 🔍

### Provenance-gap risk
- **ID:** ct:biases:ai-data:provenance-gap-risk
- **Definition:** Provenance-gap risk appears when a text asks readers to rely on a claim without showing where the claim came from, what sources were included, or what was excluded.
- **Signal phrase:** source not shown; report proves
- **Example:** A memo says the report proves the program works but does not identify the report, method, or quoted evidence.
- **Mitigation:** Add source coverage, citation trail, and explicit limits before relying on the claim.
- **Confidence:** medium
- **Three Cs:** 🔍

### Sycophancy risk
- **ID:** ct:biases:ai-data:sycophancy-risk
- **Definition:** Sycophancy risk appears when a generated response confirms the user's plan, opinion, or framing while under-surfacing counterarguments, limits, or disconfirming evidence.
- **Signal phrase:** confirms your plan; as requested
- **Example:** A generated response says 'your recommendation is clearly the best path' without naming any tradeoffs.
- **Mitigation:** Add one serious counterargument and one disconfirming-evidence check before accepting the recommendation.
- **Confidence:** medium
- **Three Cs:** 🔍

### Metric or benchmark bias
- **ID:** ct:biases:ai-data:metric-or-benchmark-bias
- **Definition:** Metric or benchmark bias appears when a score, benchmark, leaderboard, or evaluation result is used as if it directly proves practical performance for the user's context.
- **Signal phrase:** benchmark; score
- **Example:** A model is described as safe for community use because it scored highest on a narrow benchmark.
- **Mitigation:** State what the metric measures, what it does not measure, and whether it matches real-world use.
- **Confidence:** medium
- **Three Cs:** 🔍

### Dataset shift
- **ID:** ct:biases:ai-data:dataset-shift
- **Definition:** Dataset shift appears when the evidence base, training data, or observed sample differs meaningfully from the context where the claim will be used.
- **Signal phrase:** all residents; all users
- **Example:** A pilot in one county is described as proving the policy will work for all residents statewide.
- **Mitigation:** Bound the conclusion to the data context or gather evidence from the target setting before use.
- **Confidence:** medium
- **Three Cs:** 🔍

### Evaluation bias
- **ID:** ct:biases:ai-data:evaluation-bias
- **Definition:** Evaluation bias appears when the evaluation set, scoring rule, judge pool, or task definition privileges some cases while hiding failures that matter in use.
- **Signal phrase:** evaluation; test set
- **Example:** A tool is called best-in-class because it wins on a benchmark that excludes low-resource users.
- **Mitigation:** Disclose evaluation design, judge criteria, and real-world mismatch before presenting the score as readiness.
- **Confidence:** medium
- **Three Cs:** 🔍

### Representation bias
- **ID:** ct:biases:ai-data:representation-bias
- **Definition:** Representation bias appears when the people, contexts, languages, or cases present in the data do not adequately cover the population affected by the conclusion.
- **Signal phrase:** representative; underrepresented
- **Example:** A service is described as working for every community even though the dataset excludes rural and non-English users.
- **Mitigation:** Name coverage limits and gather evidence from underrepresented groups before broad use.
- **Confidence:** medium
- **Three Cs:** 🔍
## Research Methodology

### WEIRD bias
- **ID:** ct:biases:methodology:weird-bias
- **Definition:** WEIRD bias appears when a study, policy claim, or social-science argument treats a narrow participant pool, often university or online samples from wealthy Western contexts, as a universal human baseline.
- **Signal phrase:** WEIRD sample; college students
- **Example:** A study of U.S. college students is described as revealing how all humans make moral decisions.
- **Mitigation:** Do not call a study WEIRD-biased solely because it was conducted in the West; check whether the claim overgeneralizes beyond the sample.
- **Confidence:** medium
- **Three Cs:** 🔍

### Demand characteristics
- **ID:** ct:biases:methodology:demand-characteristics
- **Definition:** Demand characteristics occur when cues in the study design, researcher behavior, task wording, or setting reveal the expected response and participants adapt their behavior accordingly.
- **Signal phrase:** participants guessed; study expected
- **Example:** Participants give more generous answers after realizing the study is about empathy.
- **Mitigation:** Do not assume demand characteristics whenever participants know they are in a study; identify concrete cues or incentives.
- **Confidence:** medium
- **Three Cs:** 🔍

### Social desirability bias
- **ID:** ct:biases:methodology:social-desirability-bias
- **Definition:** Social desirability bias appears when survey, interview, or self-report responses are shaped by reputation, shame, status, politeness, or fear of judgment.
- **Signal phrase:** self-report; sensitive topic
- **Example:** Respondents overreport voting, volunteering, or healthy behavior because those answers feel respectable.
- **Mitigation:** Do not dismiss all self-report; weigh the sensitivity of the question and the safeguards used.
- **Confidence:** medium
- **Three Cs:** 🔍

### Overgeneralization
- **ID:** ct:biases:methodology:overgeneralization
- **Definition:** Overgeneralization occurs when a narrow observation is presented as a broad rule, universal pattern, or policy-ready conclusion without showing that the scope expansion is warranted.
- **Signal phrase:** all people; always
- **Example:** A pilot program in one city is described as proving what will work for all rural communities.
- **Mitigation:** Do not flag overgeneralization when the author clearly limits the claim to the studied population or setting.
- **Confidence:** medium
- **Three Cs:** 🔍

### Publication bias
- **ID:** ct:biases:methodology:publication-bias
- **Definition:** Publication bias appears when the accessible literature is skewed because null results, failed replications, negative findings, or unglamorous studies are less likely to be published or cited.
- **Signal phrase:** published studies show; only significant results
- **Example:** A review cites only positive trials and ignores registered studies with null outcomes.
- **Mitigation:** Do not infer publication bias from one missing citation; look for evidence-pool asymmetry or search-method gaps.
- **Confidence:** medium
- **Three Cs:** 🔍

### P-hacking
- **ID:** ct:biases:methodology:p-hacking
- **Definition:** P-hacking occurs when analytic flexibility is used to manufacture or exaggerate statistical significance, especially without preregistered hypotheses, correction for multiple comparisons, or transparent reporting of all analyses tried.
- **Signal phrase:** many outcomes; subgroup analysis
- **Example:** A paper reports one significant subgroup after testing many demographics without correction.
- **Mitigation:** Do not label all exploratory analysis as p-hacking; the problem is undisclosed flexibility presented as confirmatory evidence.
- **Confidence:** medium
- **Three Cs:** 🔍

### HARKing
- **ID:** ct:biases:methodology:harking
- **Definition:** HARKing, or hypothesizing after the results are known, turns exploratory interpretation into a falsely confirmatory research story.
- **Signal phrase:** hypothesis predicted; no preregistration
- **Example:** A study discovers a pattern in the data and writes the introduction as if that pattern was predicted all along.
- **Mitigation:** Do not penalize transparent exploratory work; penalize exploratory work disguised as confirmatory testing.
- **Confidence:** medium
- **Three Cs:** 🔍

### Nonresponse bias
- **ID:** ct:biases:methodology:nonresponse-bias
- **Definition:** Nonresponse bias occurs when missing respondents, dropouts, or unreachable participants differ in ways that matter for the measured outcome or claim.
- **Signal phrase:** low response rate; dropout
- **Example:** A community survey reports high satisfaction, but only people already engaged with the program responded.
- **Mitigation:** Do not flag nonresponse bias when response rate and respondent/nonrespondent differences are transparently handled.
- **Confidence:** medium
- **Three Cs:** 🔍

### Attrition bias
- **ID:** ct:biases:methodology:attrition-bias
- **Definition:** Attrition bias appears when participants who leave a study, program, or follow-up differ systematically from those who remain, especially when dropout relates to the outcome.
- **Signal phrase:** dropout; loss to follow-up
- **Example:** A training program reports strong results only for participants who completed every session, without describing who dropped out.
- **Mitigation:** Do not flag attrition bias if loss is minimal, balanced, and handled transparently.
- **Confidence:** medium
- **Three Cs:** 🔍

### Citation bias
- **ID:** ct:biases:research-evidence:citation-bias
- **Definition:** Citation bias appears when cited evidence is selected or weighted in a way that makes one view look more settled than the broader evidence base supports.
- **Signal phrase:** experts say; studies show
- **Example:** An article cites only favorable expert reports and omits known failed replications.
- **Mitigation:** Disclose source-selection criteria and include relevant dissent or null findings before using citations as proof.
- **Confidence:** medium
- **Three Cs:** 🔍

### Spin bias
- **ID:** ct:biases:research-evidence:spin-bias
- **Definition:** Spin bias appears when interpretation, headline, abstract, or summary language upgrades tentative results into stronger practical or causal conclusions.
- **Signal phrase:** proves; breakthrough
- **Example:** A small association is summarized as proof that the intervention works.
- **Mitigation:** Reframe the claim so interpretation matches outcome, uncertainty, and scope.
- **Confidence:** medium
- **Three Cs:** 🔍

### Outcome reporting bias
- **ID:** ct:biases:research-evidence:outcome-reporting-bias
- **Definition:** Outcome reporting bias appears when the text does not make clear whether the highlighted outcomes were pre-specified, complete, and representative of the full analysis.
- **Signal phrase:** we highlight; key metric
- **Example:** A grant report highlights graduation gains but omits attendance, dropout, and subgroup outcomes from the original evaluation plan.
- **Mitigation:** List planned outcomes and disclose omitted, null, or negative results before treating the highlighted metric as representative.
- **Confidence:** medium
- **Three Cs:** 🔍

### Sponsorship bias
- **ID:** ct:biases:research-evidence:sponsorship-bias
- **Definition:** Sponsorship bias appears when a claim depends on evidence produced or promoted by a party with a stake in the outcome and the text does not disclose or manage that conflict.
- **Signal phrase:** funded by; sponsored by
- **Example:** A vendor-funded benchmark is presented as neutral proof of superiority.
- **Mitigation:** Name the sponsor or incentive structure and compare with independent or dissenting evidence before relying on the claim.
- **Confidence:** medium
- **Three Cs:** 🔍

### Recall bias
- **ID:** ct:biases:research-evidence:recall-bias
- **Definition:** Recall bias appears when a claim relies on memory-based self-report and the text does not address timing, prompts, salience, or differential recall across groups.
- **Signal phrase:** remembered; self-reported
- **Example:** Participants report last year's service use from memory, and the claim treats it as exact usage data.
- **Mitigation:** Triangulate memory claims with records, timing, or contemporaneous evidence before treating them as measured outcomes.
- **Confidence:** medium
- **Three Cs:** 🔍

### Source monitoring error
- **ID:** ct:biases:research-evidence:source-monitoring-error
- **Definition:** Source monitoring error appears when source origin is confused, blended, or unstated, making a claim feel familiar without a trustworthy evidence trail.
- **Signal phrase:** it is well known; everyone remembers
- **Example:** A public brief says 'it is well known' but cannot distinguish report evidence from repeated commentary.
- **Mitigation:** Trace the claim to its source class and label unsupported familiarity as unverified rather than known.
- **Confidence:** medium
- **Three Cs:** 🔍
## Research Validity Companion

### Construct validity
- **ID:** ct:methodology:validity:construct-validity
- **Definition:** Construct validity asks whether an operational measure, proxy, scale, task, or model output represents the theoretical construct being claimed.
- **Signal phrase:** proxy measure; construct
- **Example:** A benchmark score is treated as measuring reasoning ability without validating that the task captures reasoning.
- **Mitigation:** Do not reject a proxy only because it is imperfect; ask whether the author has bounded its interpretation.
- **Confidence:** medium
- **Three Cs:** 🔍

### External validity
- **ID:** ct:methodology:validity:external-validity
- **Definition:** External validity asks how far a finding can travel beyond the specific people, places, tools, conditions, and time period studied.
- **Signal phrase:** generalizes; external validity
- **Example:** A study in one urban hospital is used to justify a national rural health policy without transportability evidence.
- **Mitigation:** Do not demand universal portability from a clearly bounded local study.
- **Confidence:** medium
- **Three Cs:** 🔍

### Measurement validity
- **ID:** ct:methodology:validity:measurement-validity
- **Definition:** Measurement validity asks whether the observed variable is accurate, reliable, interpretable, and appropriate for the inference being made.
- **Signal phrase:** measurement; endpoint
- **Example:** Engagement clicks are treated as learning without validating that clicks represent understanding.
- **Mitigation:** Do not demand perfect measurement; ask whether known error is acknowledged and bounded.
- **Confidence:** medium
- **Three Cs:** 🔍

### Internal validity
- **ID:** ct:methodology:validity:internal-validity
- **Definition:** Internal validity asks whether confounding, selection, history, maturation, measurement, or analytic choices could explain the observed result within the study.
- **Signal phrase:** caused; effect
- **Example:** A before-after program improvement is treated as causal without ruling out secular trends.
- **Mitigation:** Do not require randomized trials for every claim; match causal confidence to design strength.
- **Confidence:** medium
- **Three Cs:** 🔍

### Replicability
- **ID:** ct:methodology:reproducibility:replicability
- **Definition:** Replicability asks whether a finding is likely to recur under an independent but comparable design, population, measurement strategy, and analysis plan.
- **Signal phrase:** replication; replicate
- **Example:** A study reports a significant effect but provides no protocol, code, or materials for independent replication.
- **Mitigation:** Do not treat lack of replication as disproof when a study is new; report it as uncertainty.
- **Confidence:** medium
- **Three Cs:** 🔍
