Photo: Mike Gogulski
To date, the public discussion on ProPublica’s Surgeon Scorecard has mainly been by doctors and surgeons – many who criticize the Scorecard’s premature release and “bad data.” Now, the public will have an opportunity to weigh in on Friday, September 25th, at the Stanford MedicineX conference when it hosts speakers and a panel discussion on the Scorecard, including ProPublica.
Patients to Weigh In on Surgeon Scorecard at MedX
Including patients is central to the healthcare revolution today. Stanford MedicineX is known as a “patient-centered” conference that is a catalyst for new ideas about the future of medicine and health care. The conference will be livestreamed for the public, and anyone can engage in the discussion on Twitter with the #MedX hashtag. It will be great to have the patient point of view on the Scorecard.
The Surgeon Scorecard was developed as a tool for patients to evaluate surgeons using research data made available by the Centers for Medicare and Medicaid Services. ProPublica released this dramatic video to promote the Scorecard.
Medical experts were used by ProPublica to structure the study, but the methodology does not say if patients were included or asked about the qualities they search for in a surgeon.
Data Transparency or Bad Data?
Statistically literate: able to make sense of statistics, i.e. to think critically about the information being presented; to understand the context; and to be able to tell the story in the data.
ProPublica looked at two outcomes from surgery, death and hospital readmission rates, and assessed scores to surgeons. Complications for readmission rates were not detailed. According to ProPublica’s findings, there were some low rated surgeons at highly rated institutions.
The only uncontroversial thing that can be said about the Surgeon Scorecard is the health care community found it massively controversial. – Geoff Dougherty, Ben Harder, U.S. News and World Report
In summary, many doctors on the web and social media commend ProPublica on its intent, and say they value data transparency, but that the study falls short. Major problems, they say, include the use of only Medicare billing data, and a lack of context around the data to pass judgment. ProPublica says they could not provide specificity regarding complications because of an agreement with CMS to protect patient privacy. As a result, many questions surround the accuracy of the data and its statistical significance. Errors have also been highlighted, and some claim releasing the data could be bad for patients.
Links to further reading at the bottom of this post.
Will Doctors Avoid High-Risk Patients?
In a comment on the subject, Dr. Alan Fein says scoring systems and tools are exceedingly difficult to craft and can be misleading and damaging. He provides an example from another study, the first NYS study on cardiac surgery:
The “worst” surgeon on the first list produced was a young, bright and extremely talented cardiac surgeon who was assigned salvage cases because of his expertise. The system failed to capture the extreme morbidity of the cases he undertook and consequently he sank to the bottom of the listing … which was then made public causing him inordinate and totally unwarranted grief. The result was that immediately afterwards, he stopped taking salvage cases and focused on routine CABGs – his name rose to the top of the list the following year. The unintended consequence was that an untold number of critically ill patients were denied the possible benefits of his skills, most likely to die.
Scorecard Data Only on Medicare Patients
The Scorecard was only based on Medicare patient data. ProPublica says the results are still useful, and can be applied more broadly.
According to CMS’s Chronic Conditions among Medicare Beneficiaries Chartbook 2012, in 2010,
- Two-thirds of all Medicare beneficiaries had two or more chronic conditions.
- Medicare beneficiaries with two or more chronic conditions accounted for almost all (98 percent) of the total 1.9 million Medicare hospital readmissions.
- Beneficiaries with 6 or more chronic conditions accounted for a disproportionate share of these readmissions, with this 14 percent accounting for 70 percent of all Medicare readmissions.
ProPublica did not specify the number of chronic conditions for the patients in the study, but instead assigned “a health score” to patients. For this reason, I don’t make the leap at applying these findings to the general population.
ProPublica collected Medicare data from 2009 to 2013. Beginning October 1, 2012, the Centers for Medicare & Medicaid started reducing payments to hospitals with excess readmission rates. In recent years, advances in technology, like predictive analytics and real-time patient monitoring with mHealth, have exploded. There are new solutions to better analyze and address sources of complications.
Video from Scalabilty Experts.
Should some of these elective surgeries have taken place?
Also, in 2013, the American College of Surgeons, as part of the National Surgical Quality Improvement Program, introduced a Surgery Risk Calculator to help doctors and patients make better decisions for elective surgery. It may surprise you to run through the calculator. BMI alone is something that should be lowered before patients elect some surgeries.
After Surgery, I’ll Lose Weight
Dr. Howard Luks, an orthopedic surgeon, says he displays the Surgery Risk Calculator on a 50-inch screen when discussing elective knee surgery with patients. Many patients who are overweight have knee problems. They may also have chronic conditions like high blood pressure and diabetes. Dr. Luks says most know they could lower their risk of surgery complications by losing weight, “There’s no doubt that they profess a desire to improve, but they blame their knee osteoarthritis, etc., as a reason why they can’t do anything until after surgery.” But his “gung-ho” patients prove otherwise, he says, “They will go for pre-hab, etc. and work on their weight or A1c.”
A mobile app for the surgery risk calculator would be great for patients, but it has still not been made available. There are other medical app risk calculators specifically for doctors like Calculate by QxMD.
Please let us know in the comments if the Surgery Risk Calculator is part of any EHR.
Accepting a Complication for the End Result
According to urologist Dr. Ben Davies, “Readmissions, while unfortunate, are often not a result of a surgeon’s prowess or lack thereof.” Incidentally, Dr. Davies says his score was “absolutely perfect” – of the 33 radical prostatectomies he performed on Medicare patients, none were readmitted in 30 days.
He says many patients would accept a complication for the end result, “Would you trade a bad bout of constipation for an erection? I would. Would you trade an ileus for prostate cancer remission? Ditto.”
Alternatives Sources of Surgical Data
Another source that was featured on CNN using the same Medicare data as ProPublica, but received a lot less fanfare is SurgeonRatings by Consumers Checkbook.
Other Surgical Databases
Dr. Paul Kuriansky also commented, “Both the American College of Surgeons and the Society of Thoracic Surgeons have put considerable time, effort and expense into carefully developing clinically meaningful well-validated risk models that can be used to assess surgical performance–both have started entering the arena of public reporting with appropriate caution. Let’s do this the right way!
The Michigan Urological Surgery Improvement Collaborative
Dr. David Miller and Dr. Brian Stork will follow the talk by ProPublica, on September 25th at MedicineX, with an introduction to the Michigan Urological Surgery Improvement Collaborative or MUSIC.
They hope the statewide initiative dedicated to improving surgical care will inspire others to adopt a similar methodology focused on better care for patients. The effort is supported by Blue Cross and Blue Shield of Michigan. Watch Patient Reported Outcomes.
Save the Date
Join the #MedX ProPublica discussion on Friday, September 25th, beginning at 1:05 pm (PDT). Dr. Jordan Schlain moderates a panel with Marshall Allen and Olga Pierce from ProPublica, Dr. Robert Wachter from UCSF, Dr. David Miller from MUSIC, and patient advocate and ePatient scholar Breck Gamel.
Further Reading:
Scoring No Goal – Adventures in Transparency
Failing Grade for ProPublica’s Scorecard (Read the comments)
ProPublica’s Scorecard: Clickbait? Or Serious Data?
Surgeon Scorecard measures docs by complications
Urology Times – Do you know what your ‘surgeon score’ is
The U.S. News Take on ProPublica’s Surgeon Scorecard
Operating in the public eye: Life after the Surgeon Scorecard
How we measured surgical complications – ProPublica Methodology
