Motivation for the SAGES CVS Challenge

presented by former SAGES President Dr. Michael Brunt, MD.

About the CVS Challenge….

The Society of American Gastrointestinal and Endoscopic Surgeons (SAGES) is hosting its first-ever biomedical challenge targeting the computational assessment of one of the most meaningful, clinically validated, surgical safety measures - the Critical View of Safety (CVS) in laparoscopic cholecystectomy.

Numerous AI models have been proposed to improve the safety of laparoscopic cholecystectomy by academic groups as well as MedTech companies. While these algorithms generally show promising results, thorough testing on large, diverse and representative datasets is needed to better assess real-world performance and build trust prior to clinical translation. Challenges, a popular format in AI conferences to address real-world problems, typically generate these much-needed large annotated datasets and call AI researchers to compete to solve a particular task of recognized importance.

The SAGES CVS Challenge provides a foundations for the development of clinically meaningful and effective AI solutions for intraoperative assessment of the CVS to assist surgeons in critical decision making and enhance surgical safety. AI solutions to achieve automated visual identification of the CVS and its sub-criteria can augment surgeon’s visual perception and cognitive capacity during laparoscopic cholecystectomy. This challenge offers a uniquely large and diverse dataset of 1000 laparoscopic cholecystectomy videos from across the globe, meticulously annotated by clinical experts for the development of accurate, robust and undertainty aware AI algorithms targeting real-time CVS assessment.

For more information on how to become involved in the challenge, contribute data or participate please email info@cvschallenge.org. 

About the Critical View of Safety…

Laparoscopic cholecystectomy is one of the most frequently performed, highly standardized surgical procedures worldwide, to remove the gallbladder in case of inflammation or other pathologies. The Critical View of Safety (CVS) is an internationally recognized, clinically validated, intraoperative surgical safety measure during laparoscopic cholecystectomy. The CVS was first introduced by Dr. Steven Strasberg in 1995 to help prevent bile duct injuries, one of the most severe complications of laparoscopic cholecystectomy. The CVS is achieved by isolating three visually distinct anatomic structures, reflected by three CVS criteria (C1, C2 and C3), before proceeding to remove the gallbladder. The optical assessment and differentiation of these three anatomic structures can be deceiving. Therefore AI-driven automated  assessment of the CVS criteria can enhance surgeons perception,  decrease their cognitive workload, and improve surgical safety.

  • The cystic duct and artery should be separated from other structures, with only their lower part still attached to the gallbladder. This ensures that the surgeon can clearly identify and dissect the correct structures.

  • The hepatocystic triangle is the area bordered by the common hepatic duct, the cystic duct, and the liver. Surgeons must adequately expose and clear this area of surrounding fat and connective tissue to visualize the essential structures accurately.

  • The lower part of the gallbladder must be separated from the liver bed, making the gallbladder-liver interface clearly visible. This ensures that the cystic duct and artery are the only structures connecting the gallbladder to the liver and other organs.

Two structures ✔️ + Hepatocystic Triangle ✔️ + Cystic Plate ✔️ = The Critical View of Safety ✅ ...............................................................

Two structures ✔️ + Hepatocystic Triangle ✔️ + Cystic Plate ✔️ = The Critical View of Safety ✅ ...............................................................

About the SAGES CVS Challenge Dataset…

The Data

The SAGES CVS challenge offers a global and diverse dataset of 1000 videos of laparoscopic cholecystectomy procedures, a routinely performed minimally invasive surgical procedure to remove the gallbladder due to inflammation or other pathologies.  The dataset was assembled through data contributions from 67 surgeons from 57 countries and 6 continents and encompasses a wide diversity of patient demographics and procedural quality to reflect the worldwide diversity in patients and surgeons. Moreover, the video data is acquired using a variety of laparoscopes from different manufacturers and displays an array of optical properties, which adds to the diversity of the data and ensures adequate representation of the real-world population. The dataset was carefully curated and annotated with respect to the three CVS criteria, following consensus-based protocols incorporating the expertise of worldwide leading clinicians, computer scientists, and industry through structured multidisciplinary Advisory Committees (AC).  

The Labels

Each video in the dataset represents a single patient's case, spanning a 90-second segment. This segment starts 90 seconds before one of the relevant anatomic structures composing the CVS (cystic duct or artery) is clipped ("point of no return"). Within each video (90-second segment), additional frames are extracted at five-second intervals (= 18 frames per video) to provide additional training data. 

The three criteria defining the CVS are annotated as binary labels (criterion achieved yes / no) for each video and each of the 18 frames per case. No semantic segmentation labels are provided. Moreover, we provide an overall rating of the annotation difficulty for each video, reflecting the annotators' subjective assessment. This does not reflect the surgical/operative difficulty of the case. The annotation complexity is provided as a 0-1 value, representing the annotator's confidence ( 0 = very difficult to annotate/ not confident at all; 1 = very easy to annotate/ very confident). For each video, we provide annotations:

  • Per Video (Were the 3 CVS criteria achieved in the overall video? Yes / No) = 3 Binary Labels x 3 annotators

  • Per Frame (Were the 3 CVS criteria achieved in this frame? Yes / No) = 3 Binary Labels x 18 Frames  x 3 annotators

  • Annotation Confidence (How sure was the annotation of the 3 CVS criteria for this case? 0-1 / not confident-confident) = 1 probabilistic label x 3 annotators

The Metadata

Besides the Labels (per video / per frame / annotation difficulty) we provide demographic and selected clinical metadata.

  • Country of data origin (anonymized) = the dataset was composed with contributions from 6 continents, to preserve data privacy the country name is anonymized

  • Laparoscope Type (category) = The type and manufacturer with which the video was recorded, which can influence the resolution, is provided in pseudonymized categories. 

  • Robotic Surgery (yes/no) = If the video was performed robotically (as opposed to conventional laparoscopy) is provided as a binary (yes/no) label.

  • Indocyanine Green (ICG) Use (yes/no) = During any laparoscopic cholecystectomy procedure, ICG can be used to better visualize and identify the relevant anatomy under fluorescent light. While ICG is routinely used in some institutions, in others, surgeons might selectively opt to use ICG for an enhanced overview of the operative field. In this case, ICG may indirectly indicate a more difficult case. While you may not see ICG or fluorescence in the provided 90-second video, the label indicates whether it was applied throughout the case. 

  • Intraoperative Cholangiography (IOC) (yes/no) = An intraoperative cholangiography is a measure to visualize the anatomy of the biliary ducts more clearly. During an IOC, the surgeon opens the structure identified as the cystic duct and introduces a catheter into the common bile duct for drainage or to inject dye. This can improve the understanding of the anatomic relations of the structures (cystic duct vs common bile duct) to each other and help to prevent accidental clipping of the common bile duct. Usually, an IOC indicates a more complex case (i.e., due to the clinical history of the patient, complex anatomy, an increased degree of inflammation of the gallbladder). Similarly to ICG, the IOC may be performed outside the 90-second video segment

The SAGES CVS Challenge is ONLY accessible to challenge participants, who registered through grand-challenge.org and signed the SAGES CVS Challenge participation agreement.

Join us in developing accurate, robust, uncertainty aware and widely generalizable AI to improve surgical safety in laparoscopic cholecystectomy by competing in the CVS Challenge. Register to participate here or reach out to info@cvschallenge.org with any questions!