​

Testing Analysis Field Drafts

​

Accessing the Draft Tester

​

Setting Up a Test

1. Select Interaction Type:
   • Choose whether you want to test against Calls or Documents, matching the primary data type the field is intended to analyze. You might select this via a dropdown (Select component labeled “Select Interaction Type”).
   • (Optional) Select Analysis Configuration: If you want to compare the draft’s output directly against the results generated by a specific live configuration that uses this field, select that configuration from the dropdown. This is useful for seeing side-by-side comparisons with existing production analysis. If left blank, the test runs the draft logic in isolation.
2. Select Test Interactions:
   • Use the Search Component to find relevant interactions.
   • Choose wisely! Select a diverse set (usually up to 10) that represent different scenarios:
     • Examples where the current live field works well.
     • Examples where the current live field performs poorly or misses information.
     • Examples covering edge cases or specific situations mentioned in your new prompts.
   • Click on the calls/documents in the search results to add them to your test set. Selected items will appear in a list.
3. Run the Test:
   • Click the “Test Field” button.
   • Fabius will now run the analysis logic defined in your current draft against each of the selected interactions. This happens in the background via an API call (/analysis/testEnumerationFieldDraft).

​

Reviewing Test Results

1. Automatic Integration: Test results are automatically added to your chat conversation, allowing you to immediately discuss them with the AI assistant.
2. Visual Notification: You’ll see a notification confirming the results have been added to the chat.
3. Results Display: The results appear in the test section with an informational alert.

​

Result Views

• Comparison View (if live configuration was selected):
  • You’ll see a side-by-side comparison table:
    • Interaction: The Call or Document tested. Clicking the name opens a drawer with full details.
    • Existing Result: The output from the live field in the selected configuration.
    • Draft Result: The output from your current draft’s logic.
• Draft-Only View (if no configuration was selected):
  • Shows only the draft’s output for each selected interaction.

• Scores: Compare numeric scores, Pros, Cons, and Suggestions between versions. Check if changes align with your expectations.
• Values: Verify extracted data is more accurate, complete, and properly formatted.

​

Iterating Based on Results

1. Identify Issues: Review the test results in your chat. Note any discrepancies or missed information.
2. Discuss with AI: Without switching contexts, explain issues to the AI assistant directly in the chat (e.g., “In the test for Call X, the draft missed the competitor ‘BetaCorp’. Please update the Instructions to be more explicit about capturing all competitor names.”).
3. Review Changes: The AI will propose changes. Review the diff view for each field component.
4. Accept/Reject: Use the provided buttons to accept or reject changes for each component.
5. Re-test: After accepting changes, run the test again with the same interactions.
6. Verify Improvements: Check if the refined prompts fixed the identified issues.
7. Iterate: Continue this seamless cycle until the draft performs correctly across all test cases.