Rule-based adjudication mirrors the logic payers use internally. By pre-evaluating claims against known plan rules, practices avoid unnecessary verification calls. Each rule fired is traceable and auditable, providing a clear decision trail for compliance and payer negotiations.

CDT codes are matched against payer-specific YAML rule definitions. Each rule type (frequency, age, bundling, waiting period, alternate benefit) implements a Matches(claim) interface. Rules fire independently; results are aggregated into a coverage determination. Conflict resolution: deny wins over cover when rules contradict.

The confidence score drives the predict-vs-call decision. Predictions above the routing threshold bypass manual verification, directly reducing operational cost. The score is decomposed into four auditable factors, enabling targeted data collection to improve weak areas. Practices typically see 60-70% of claims routed with confidence above 85%.

Weighted linear combination of four normalized factors:

Weights are configurable per payer. The model is intentionally linear for interpretability — each factor's contribution is directly readable from the decomposition.

The routing threshold is the business lever controlling the precision/recall trade-off. A higher threshold means fewer automated predictions but higher accuracy. A lower threshold means more automation but higher risk of incorrect predictions. The optimal threshold depends on the cost of a wrong prediction vs. the cost of a verification call.

Binary decision based on confidence threshold comparison:

The threshold can also be derived from the decision-theoretic Value of Information (VoI) calculation when cost parameters are available.

The rule match score directly reflects the depth of payer-specific knowledge in the system. A consistently low score for a payer indicates a gap in rule coverage — an actionable signal to invest in rule acquisition for that payer. Higher rule coverage correlates with reduced call volume and improved prediction accuracy.

Historical approval rates provide empirical grounding for predictions. This factor improves automatically as the system processes more claims, creating a data flywheel. Payers with more historical data produce more reliable predictions, directly reducing risk exposure over time.

Data completeness is the most actionable confidence factor — it identifies exactly which missing fields would improve prediction quality. Practices that achieve 90%+ data completeness see measurably higher prediction accuracy, directly reducing unnecessary payer calls and improving patient experience at check-in.

Payer reliability quantifies adjudication consistency across a payer's claim history. Low-reliability payers represent higher risk and should trigger manual verification more frequently. This metric also serves as a negotiation tool — demonstrating to payer representatives where their adjudication deviates from published guidelines.

The Brier Score is the standard metric used by the National Weather Service and insurance actuaries to evaluate probabilistic forecasts. It directly measures calibration — whether a "90% covered" prediction is actually covered 90% of the time. Tracking Brier Score over time demonstrates model reliability to auditors and payer partners. A score below 0.15 indicates clinical-grade prediction accuracy. Each 0.01 improvement translates to approximately 2-3% fewer unnecessary payer calls.

Brier, G.W. (1950). "Verification of forecasts expressed in terms of probability." Monthly Weather Review 78(1).

Log Loss is the industry-standard metric for evaluating probabilistic classifiers in machine learning and actuarial science. Unlike accuracy, it rewards well-calibrated probabilities and severely penalizes overconfident errors. Financial institutions and insurance companies use Log Loss to ensure risk models don't systematically underestimate uncertainty.

The Spherical Score provides a bounded, interpretable complement to Log Loss. Its 0-to-1 range makes it suitable for dashboards and executive reporting. As a proper scoring rule, it cannot be gamed — the best strategy is always to report true beliefs. Used in forecasting competitions and weather prediction evaluation.

The Brier Skill Score benchmarks prediction quality against a no-skill baseline. A BSS of 0.30 means the model reduces prediction error by 30% compared to always guessing the base rate. This directly quantifies the ROI of the prediction system — a BSS of 0 means no value added, while higher scores justify the investment in rule development and data collection.

Bayesian updating provides a principled framework for combining prior knowledge with observed data. Unlike simple averages, it naturally handles small sample sizes by incorporating prior beliefs, preventing wild swings from limited data. This approach is standard in actuarial science and clinical trials — domains where decisions must be made under uncertainty.

Conjugate prior for Bernoulli likelihood. Closed-form update — no MCMC required.

Credible intervals communicate prediction uncertainty in an intuitive way. A narrow interval (85% ± 3%) signals high precision and justifies automated routing. A wide interval (85% ± 20%) flags high uncertainty, warranting manual review. This transparency builds trust with clinical staff and supports defensible decision-making.

Interpretation: 95% probability that the true coverage rate lies within this interval, given the observed data and prior.

This metric directly addresses the business question: "Can we trust this prediction enough to skip the call?" Rather than comparing a point estimate to a threshold, it computes the probability of exceeding the threshold, incorporating uncertainty. A posterior probability of 95% means there's only a 5% chance the true rate is below the threshold — a quantified risk statement for decision-makers.

Fisher Information quantifies the diminishing returns of additional data. Early claims dramatically reduce uncertainty, but each additional claim provides less marginal value. This informs resource allocation: if the CRLB shows that 50 more claims would only reduce the interval by 1%, the investment in manual data collection may not be justified.

Shannon Entropy provides an information-theoretic measure of prediction uncertainty that's independent of the scoring method. High-entropy predictions represent the highest-value targets for data collection or rule development — they're the cases where additional information would have the greatest impact on routing efficiency.

KL Divergence quantifies the information advantage of the prediction system over having no information. It provides a single metric for the total value of the system's knowledge about a specific claim. Tracking average KL Divergence over time demonstrates increasing system knowledge and justifies continued investment in rule development.

Information Gain directly measures the value of the prediction in reducing decision uncertainty. An average Information Gain of 0.7 bits (out of 1.0 maximum) means the system resolves 70% of the uncertainty in each claim, translating to proportional reductions in the need for manual verification. This is the core value metric for the prediction system.

The Bayes Factor provides an intuitive strength-of-evidence measure for the prediction system vs. a null model. Unlike p-values, Bayes Factors have a natural interpretation: BF = 20 means "the data is 20 times more likely under our model than under random chance." Industry convention: BF > 10 is "strong evidence," BF > 100 is "decisive." This metric is directly presentable to non-technical stakeholders.

EMV converts prediction quality into direct financial terms. By comparing EMV(predict) vs. EMV(call), each routing decision has a quantified dollar impact. Aggregated across thousands of claims, this provides a clear ROI picture: "The prediction system saved $X by avoiding Y calls while correctly predicting Z% of outcomes."

Value of Information provides the economic foundation for the routing decision. When VoI > cost of verification call, calling the payer is justified. When VoI < call cost, the prediction is sufficient. This framework transforms a subjective "should we call?" into an objective, auditable financial decision. Tracking VoI across payers identifies where verification processes are most and least cost-effective.

Expected Value of Perfect Information caps the maximum investment justifiable for improving prediction quality. If aggregate EVPI across all claims is $50K/month, any investment in better rules, more data, or payer integrations that costs less than $50K/month and closes the gap has positive ROI. This is a standard decision-analysis metric used in healthcare and insurance portfolio management.

Monte Carlo simulation is the gold standard for risk analysis in financial services and insurance. By generating 1,000 scenarios, it reveals the full distribution of potential outcomes — including tail risks that point estimates miss. This approach is required by Basel III banking regulations and Solvency II insurance frameworks for risk quantification.

VaR and CVaR are regulatory-standard risk metrics used by every major financial institution. VaR(95%) sets the "normal worst case" boundary, while CVaR captures average tail losses — the scenarios that matter most. Presenting these alongside predictions demonstrates enterprise-grade risk management, supporting compliance requirements and payer contract negotiations.

The Sharpe Ratio (Nobel Prize-winning metric) and Sortino Ratio measure risk-adjusted returns. Applied to insurance predictions, they answer: "Is the prediction system generating savings efficiently, or are good results masking high variance?" A Sharpe > 1.0 is considered "good" in financial markets. The Sortino variant only penalizes downside risk, better matching the asymmetric cost structure of incorrect coverage predictions.

Factor decomposition provides full auditability for every prediction. Regulators, payers, and practice managers can see exactly why a prediction was made — not just the outcome. This SHAP-like decomposition is the standard for explainable AI in regulated industries, supporting compliance with emerging AI transparency requirements.

The Tornado Chart prioritizes improvement investments. By showing which factors are closest to flipping the routing decision, it directly identifies where to focus resources. If payer reliability is the swing factor for 30% of uncertain predictions, investing in payer-specific rule development has a quantifiable expected return.

The Murphy decomposition separates prediction quality into actionable components. High reliability means the system's probabilities are trustworthy. High resolution means it successfully differentiates cases. High uncertainty is an inherent property of the domain, not a system flaw. This decomposition is used by NOAA, the European Centre for Medium-Range Weather Forecasts, and actuarial certification boards.

Murphy, A.H. (1973). "A New Vector Partition of the Probability Score." Journal of Applied Meteorology 12(4).

The calibration diagram is the single most important visual for assessing prediction trustworthiness. A well-calibrated system (line near diagonal) means the probabilities can be taken at face value for decision-making. Systematic deviations reveal biases: above the diagonal means the system is under-confident (leaving value on the table), below means over-confident (creating risk). This visualization is standard in actuarial reporting.

The Hosmer-Lemeshow test is a standard goodness-of-fit measure for logistic and probabilistic models in healthcare and insurance. A passing result (p > 0.05) provides statistical evidence that the model is well-calibrated — a key requirement for regulatory submissions and actuarial certifications. This test is required by FDA guidance for clinical prediction models.

Hosmer, D.W. & Lemeshow, S. (1980). "Goodness of fit tests for the multiple logistic regression model." Communications in Statistics A9.

Cumulative scoring provides real-time performance monitoring. Convergence of the running Brier Score demonstrates prediction stability — essential for quality assurance. Sudden jumps indicate either a system issue or an unusual claim mix, triggering investigation. This continuous monitoring approach mirrors SPC (Statistical Process Control) used in manufacturing quality management.

Discrimination is the complement of calibration — it measures whether the system can distinguish good from bad outcomes. Two overlapping distributions indicate poor discrimination; separated distributions indicate strong discrimination. In medical testing terms, this is analogous to the sensitivity/specificity trade-off visible in a single chart.

The Confidence Trend chart provides a process-level view of system performance over time. Upward trends indicate improving predictions (data flywheel effect). Downward trends or increased variance signal potential issues — payer rule changes, unusual patient mix, or data quality degradation. This is the operational dashboard metric for day-to-day system health monitoring.