Cross-Cutting Work Groups
Description
Develop end-to-end, generalized evaluation scenarios for each recommended method/metric in the Patient Discharge Summarization Testing & Evaluation Framework on GitHub. For example, create a general step-by-step example that a developer or technical team would need to follow in order to objectively evaluate DocLens. This evaluation scenario will include sections such as datasets (e.g., demo, public, synthetic), code snippets a technical team can customize for evaluation (e.g., steps to practically assess and evaluate a particular method/metric), and the method/metric definition with examples.
Timeline
Phase 1: Q2 2025 - Q3 2025
Tasking:
Phase 1 underway: develop end-to-end, generalized evaluation scenarios for each recommended method/metric in the Patient Discharge Summarization Testing & Evaluation Framework on GitHub
Output:
Initial open-source tooling exercise (Phase 1) will be added to CHAI's GitHub Repository, initially for the Note Summarization - Patient Discharge Summary Testing & Evaluation Framework.
Link here once available
Work Group Members
Mayo Clinic
Mount Sinai
Providence
Citingale