Today, AgentFlow treats all knowledge sources equally during retrieval. When multiple sources contain overlapping information (e.g. Custom Answers, indexed website URLs, PDFs, Knowledge Base articles), the model determines which source to retrieve from and there is no way for administrators to control source priority.
This creates challenges for enterprise deployments where customers have curated and approved answers that should always take precedence over website content.
Expected behavior:
If a customer asks a question that exists in the approved Q&A set, the AI should prioritize and return the approved answer.
Actual behavior:
The AI frequently ignores the Custom Answer and retrieves information from indexed website pages instead, generating a different response.
This results in:
  • Inconsistent answers.
  • Hallucinations when website content is interpreted incorrectly.
  • Difficulty validating AI behavior during UAT.
  • Loss of customer trust because approved answers are not consistently used.
  • Significant CS and SE effort spent troubleshooting behavior that cannot currently be configured.
Proposed Solution
**Introduce Knowledge Source Prioritization settings within AgentFlow.
Example:
Priority Order:
  • Custom Answers
  • PDFs
  • Indexed Website URLs
  • Live Web Search
Alternative approach:
Allow administrators to assign a priority score to each source type.
Benefits, Customer Benefits:
  • More predictable AI behavior
  • Greater trust in approved responses
  • Easier compliance and governance
  • Better enterprise adoption
  • Faster onboarding and implementation
  • Easier UAT validation
  • Reduced troubleshooting effort
  • Clearer expectation setting with customers
  • Improved answer quality
  • Reduced hallucination risk
  • Stronger enterprise readiness for regulated industries such as education, finance, healthcare, and government.
Additional Consideration:
A future enhancement could include displaying the source used for each answer and allowing admins to configure fallback behavior when higher-priority sources do not contain a relevant answer.