UPDATE: This story was updated 12:24 p.m. Oct. 6 to reflect changes in We Rate Covid Dashboards’ ratings for IU and several other Big Ten schools.
We Rate Covid Dashboards, a group of Yale University professors and medical students that evaluates university COVID-19 dashboards, assessed IU’s about a week ago. At first, it got a C+. Early this week, it was rescored to a B-.
“That’s a horrible dashboard,” said Howard Forman, professor of public health and director of Yale's health care management program, immediately after pulling up IU’s COVID-19 tracking website on his computer. “I mean, come on, they’re a big state university, why would they have such limited information? That’s disappointing.”
Forman, who founded We Rate Covid Dashboards with a colleague, said the site grades dashboards based on nine categories, including if they are easy to read, updated daily, provide separate student and faculty data, state how often people are tested and give a summary of how the campus is handling the coronavirus.
IU earned six out of 13 possible points, increasing by one point when it was reevaluated due to an error in We Rate Covid Dashboards original grade. While easy to read, the dashboard is not updated daily, does not include city or county data, does not state how soon tests come back and does not give a summary of the campus status.
University of Michigan, Ohio State University, Northwestern University and Purdue University also saw their dashboard grade increase in the recent reevaluation. Overall, five of the 14 schools in the Big Ten got a B+ — eight points — or higher, and IU had the worst score of all schools in the conference, scoring the same as Rutgers University and Northwestern University.
IU’s infrequent dashboard updates aren’t due to a lack of COVID-19-related data points. The university just won’t release them to the public.
Aaron Neal, associate vice president of enterprise systems at University Information Technology Services, oversees IU’s internal COVID-19 dashboards. Neal said his team built and maintains multiple internal dashboards to update public health experts on testing.
This internal data, including multiple metrics not present on the public-facing dashboard, flows into the internal ones as fast as Neal’s team can manage, often almost in real time as soon as COVID-19 test results are processed by Vault Health and other testing facilities.
But IU’s public-facing dashboard is only updated once a week.
While the dashboard was originally updated each Monday, IU announced Sept. 11 in an email it would move all future updates to Wednesdays to provide its public health experts more time to “analyze results and provide important context.” This means weekly data doesn’t reach public eyes until four days after the last day of testing included in it.
Asked repeatedly by the Indiana Daily Student, university officials have maintained a weekly COVID-19 update is the best way to present data to the public given IU’s mitigation testing of testing certain groups on a week-to-week basis.
“We would like to really focus on giving a much more accurate and reliable picture with the context of a week-to-week, which is a better way of looking at this through the public eyes, but also as an institution,” IU spokesperson Chuck Carney said.
Dr. Aaron Carroll, IU’s director of mitigation testing, agreed with Carney in a Sept. 16 webinar, adding daily data comes in “haphazardly,” not in regular batches.
“We have almost all of them back by Monday evening, then we can look at them holistically, make analyses, discuss changes, we get everything together for Tuesday, it goes up Wednesday,” Carroll said in the webinar. “I know people want more, but more isn’t necessarily better.”
Forman said IU’s claims that weekly updates are “more accurate” are untrue.
“To say that I’m going to give you less information because you don’t know how to use it is to infantilize people,” Forman said. “I think everybody has a right to be able to see the information.”
Forman said daily fluctuations in data are evident in nearly every COVID-19 dashboard, including state and national ones. He said releasing seven-day rolling averages each day can compensate for day-to-day discrepancies while still providing more frequent updates.
Carney said IU does not plan to release daily updates incorporating a seven-day rolling average.
“To just simply throw numbers up every day is not necessarily going to be helpful,” Carney said.
Despite IU’s C+ dashboard rating, Forman said the university’s mitigation testing strategy is a good public health approach. Forman co-authored a medical paper published in May with recommendations for communities like universities on how to monitor COVID-19, and he said IU’s response closely follows processes laid out in that paper.
“From what I can glean, IU has a strategy that is based in good public health science, but the lack of transparency makes it hard for me to know if it’s working or not and how effective it is,” Forman said.
Forman said if IU released data more frequently, not only would people living on and around campus be able to understand the health risks of the area where they live, but other universities could learn from IU.
“I think everybody deserves to know and it also helps other institutions,” Forman said. “It would be nice if IU shared that information so that other places could say, ‘Hey look, it’s working here, let’s model ourselves after them.’”
Forman said since We Rate Covid Dashboards was created, several institutions that initially received low ratings have improved their dashboards.
“If you agitate for change and you believe in something enough, people are going to respond to it,” Forman said.
In his weekly webinars, Carroll has frequently mentioned a goal of eventually testing students, staff and faculty one to two times every week after three new IU labs are added in October, allowing coronavirus tests to be processed locally. This would increase IU’s average weekly mitigation test load from around 10,000 tests per week to at least 40,000.
Testing every student at least once a week would provide a much more comprehensive picture of the state of COVID-19 at IU. Despite this, Carney said IU does not plan to update the dashboard more frequently even if Carroll’s goal is met.
Forman said he understands why IU would not want to release daily data because a spike in cases or other negative developments in the university’s COVID-19 situation could result in negative publicity.
“You want to be able to put fires out before anybody notices that there was a fire,” Forman said. “It doesn’t make them bad people to respond that way, but it’s why we have our Twitter, it’s why we have our website: we’re trying to shine light on this.”
