1.9 KiB
1.9 KiB
Confidence Scoring Methodology
How to assign and interpret confidence scores for generated brand guidelines.
Scoring Levels
High Confidence
The guideline section is well-supported and actionable.
Criteria (must meet at least 3):
- 3+ corroborating sources
- Explicit guidance found in at least one AUTHORITATIVE source
- Consistent across document and conversation analysis
- Specific, actionable instructions (not just vague principles)
- No unresolved conflicts
Medium Confidence
The section is reasonable but could benefit from more data or team confirmation.
Criteria (must meet at least 2):
- 1-2 corroborating sources
- Inferred from patterns rather than explicit instruction
- Minor inconsistencies resolved via recency or authority
- Actionable but some interpretation was required
- May have one unresolved conflict
Low Confidence
The section is a best-effort recommendation. Team review strongly recommended.
Criteria (must meet at least 2):
- Single source only
- Primarily inferred from indirect evidence
- Significant interpretation required
- Unresolved conflicts between sources
- Limited specificity
Aggregate Confidence
Calculate overall guideline confidence as the weighted average of section scores:
| Section | Weight |
|---|---|
| Voice Attributes | 30% |
| Messaging Framework | 25% |
| Tone Matrix | 20% |
| Terminology | 15% |
| Language Patterns | 10% |
Convert scores: High = 1.0, Medium = 0.6, Low = 0.3
Aggregate score thresholds:
- 0.85–1.0 = High
- 0.60–0.84 = Medium
- Below 0.60 = Low
Relationship to Open Questions
- Low confidence + conflict = High Priority open question
- Low confidence + gap = Medium Priority open question
- Medium confidence + minor inconsistency = Low Priority open question
Every open question includes a recommendation that, if confirmed, would raise the section's confidence score.