feat: add confidence-scoring reference v1.0.0

2026-05-05 13:38:08 -05:00
parent 5314e1d8d4
commit 78dee9582c
1 changed files with 62 additions and 0 deletions
@@ -0,0 +1,62 @@
+# Confidence Scoring Methodology
+
+How to assign and interpret confidence scores for generated brand guidelines.
+
+## Scoring Levels
+
+### High Confidence
+The guideline section is well-supported and actionable.
+
+**Criteria (must meet at least 3):**
+- 3+ corroborating sources
+- Explicit guidance found in at least one AUTHORITATIVE source
+- Consistent across document and conversation analysis
+- Specific, actionable instructions (not just vague principles)
+- No unresolved conflicts
+
+### Medium Confidence
+The section is reasonable but could benefit from more data or team confirmation.
+
+**Criteria (must meet at least 2):**
+- 1-2 corroborating sources
+- Inferred from patterns rather than explicit instruction
+- Minor inconsistencies resolved via recency or authority
+- Actionable but some interpretation was required
+- May have one unresolved conflict
+
+### Low Confidence
+The section is a best-effort recommendation. Team review strongly recommended.
+
+**Criteria (must meet at least 2):**
+- Single source only
+- Primarily inferred from indirect evidence
+- Significant interpretation required
+- Unresolved conflicts between sources
+- Limited specificity
+
+## Aggregate Confidence
+
+Calculate overall guideline confidence as the weighted average of section scores:
+
+| Section | Weight |
+|---------|--------|
+| Voice Attributes | 30% |
+| Messaging Framework | 25% |
+| Tone Matrix | 20% |
+| Terminology | 15% |
+| Language Patterns | 10% |
+
+Convert scores: High = 1.0, Medium = 0.6, Low = 0.3
+
+**Aggregate score thresholds:**
+- 0.85–1.0 = High
+- 0.60–0.84 = Medium
+- Below 0.60 = Low
+
+## Relationship to Open Questions
+
+- **Low confidence + conflict** = High Priority open question
+- **Low confidence + gap** = Medium Priority open question
+- **Medium confidence + minor inconsistency** = Low Priority open question
+
+Every open question includes a recommendation that, if confirmed, would raise the section's confidence score.