Israeli Wastewater Study A Study In Junk Science

Flawed Methodology Plus Flawed Initial Assumptions Equal Flawed Conclusions

The Israeli study of wastewater detection methods for the SARS-CoV-2 virus and variants by Karin Yaniv, et al, is a fascinating bit of research.

Unfortunately, being fascinating does not save it from being crap.

While the study attempts to make a compelling case for wastewater testing as a basis for charting viral evolution, it suffers from fatal flaws both in its methodology and its base assumptions. That is a shame, because the concept of wastewater testing itself has merit, and deserves greater use as an ongoing method of disease surveillance.

Background: The Origins Of Environmental Virology

Wastewater surveillance—aka “environmental virology”—is not a new concept. It began during the mid-20th century as a means for enhanced surveillance and detection of poliovirus, and has expanded since then to encompass a broad range of infectious pathogens, including hepatitis, rotavirus, as well as SARS-CoV-2.

Intriguingly, Israel happens to be the country with one of the more notable success stories for environmental virology. Israel established a wastewater surveillance program for poliovirus in 1989, and in 2013, that program identified rising prevalence of poliovirus n sewage from communities in Israel’s Negev Desert region. Alerted to the presence of the pathogen before a major outbreak occurred, Israel was able to mount a concentrated campaign of polio vaccination, with the result that no cases of polio resulting in paralysis were reported.

Wastewater surveillance, unlike so many other protocols established or emphasized for managing the COVID-19 pandemic, has demonstrated utility. In addition to the 2013 success story for poliovirus, there have been a number of pilot programs, proof of concept programs, and small scale monitoring programs around the world for tracking the SARS-CoV-2 virus in human effluvia.

• A study of sewage samples taken in Massachusetts during March of 2020 suggested a far broader spread of the virus than had been established by diagnostic testing.

• Dutch scientists conducted a similar proof-of-concept study in the city of Amersfoort also during the spring of 2020.

• In April of 2020, Australia announced a wide sewage testing program after a successful regional pilot program.

Wastewater surveillance is a demonstrably valid monitoring protocol for COVID-19, as there is ample evidence of fecal shedding of the SARS-CoV-2 virus, and fecal-oral transmission of the virus was implicated early on in the pandemic among cases in Shenzen, China. This fits with known behaviors of SARS-related coronaviruses (SARSr-CoV), as fecal-aerosol transmission was implicated in the spread of the original SARS virus through the Amoy Gardens residential complex in Hong Kong in 2003.

For its part, the CDC initiated the National Wastewater Surveillance System in the fall of 2020, explicitly in response to the COVID-19 pandemic. One of the few bright spots in the mishmash of bad science and junk data known as President Biden’s National COVID-19 Preparedness Plan is its support for enhanced CDC surveillance via NWSS and other methods.

This background on wastewater surveillance is important, because the Yaniv study attempts to use the tool in manifestly unsustainable ways, as noted (and, regarding the COVID-19 pandemic, notably contrarian) virologist Geert Vanden Bossche points out in his critique and peer review of the study.

The Bad Science In The Yaniv Study

The goals of the study are initially straightforward and in keeping with the established history and utility of wastewater surveillance.

The latest variant of concern, Omicron, is spreading swiftly around the world with record morbidity reports. Unlike the Delta variant, previously considered to be the main variant of concern in most countries, including Israel, the dynamics of the Omicron variant showed different characteristics. To enable quick assessment of the spread of this variant we developed an RT-qPCR primers-probe set for the direct detection of Omicron variant. Characterized as highly specific and sensitive, the new Omicron detection set was deployed on clinical and wastewater samples.

Had the study’s authors Yaniv, et al, been content with developing new and more efficient/effective Omicron monitoring tools, the study could have been an important contribution to the compendium of science supporting wastewater surveillance as a useful disease surveillance tool for monitoring against future outbreaks of SARS-CoV-2 infection.

Unfortunately, the authors encountered some unexpected data—the presence within the wastewater samples of higher than projected levels of the Delta variant.

In contrast to the expected dynamics whereupon the Delta variant diminishes as Omicron variant increases, representative results received from wastewater detection indicated a cryptic circulation of the Delta variant even with the increased levels of Omicron variant. Resulting wastewater data illustrated the very initial Delta-Omicron dynamics occurring in real time.

This is where the researchers began to wander off into the weeds, pursuing this canard of “cryptic circulation”. As I observed early on in the pandemic, while discussing the limitations of diagnostic testing as a disease tracking tool, “cryptic circulation”, alternatively known as “cryptic transmission”, does not actually happen.

In the case of CCPVirus testing, attempting to extrapolate from testing to model disease spread was the mismeasurement that led, in multiple instances, to the curious presumed phenomenon of "cryptic transmission." In Washington State, the curious 6-week lag between the first positive test and the second positive test, with both tests showing the viral strain to be related (meaning both patients were on a common chain of transmission), led doctors to quite naturally wonder where the transmission had been occurring, and how had they failed to miss it.

The answer it both simple and brutal: they missed the transmission because, contrary to their presumptions, they were not actually looking for it.

At the time of that article, the “cryptic transmission” scenario under scrutiny was a mysterious six-week lag within Washington State between the first detected case of COVID-19 and the second—mysterious only because the “experts” at the time failed to take into account a corresponding decline in general influenza-like illnesses within the state during that interval, indicating that the lag was due to the virus actually propagating somewhere else before returning to Washington.

In the case of Washington State's mystery "cryptic transmission", scrutiny of that state's Weekly Influenza Report revealed that, from mid January until mid February, patient visits for influenza like illnesses actually declined. Thus, there was no "cryptic transmission"—the disease quite literally was not spreading in the state during that period.

There is nothing “cryptic” about an infectious respiratory pathogen circulating in a manner undetected by mass diagnostic PCR testing. It merely means that the pathogen has moved from one untested host to another untested host to another, until finally moving to a host that is tested. The logistical infeasibility of concurrently testing everyone in all but the smallest communities virtually assures that there will always be at least some level of undetected transmission. Undetected is hardly “cryptic”.

Perversely, the finding of unexpected levels of Delta variant highlights an important virtue of wastewater surveillance. Since the sampling is carried out at a common collection point (e.g., a water treatment facility) for a particular community, nearly all viral shedding within that community can theoretically be detected, subject to the limitations of the test itself. The need to test every member of that community is reduced.

The tradeoff for this more encompassing detection ability is a lack of particularity in the findings, which brings us to Vanden Bossche’s criticism of the study.

These authors tend to believe that wastewater-based epidemiology combined with mathematical modeling allows for making predictions in regard of the evolutionary dynamics of this pandemic:

“According to the developed model, it can be expected that the Omicron levels will decrease until eliminated, while Delta variant will maintain its cryptic circulation. If this comes to pass, the mentioned cryptic circulation may result in the reemergence of a Delta morbidity wave or in the possible generation of a new threatening variant.”

One should - per definition -  always be careful and skeptical about conclusions and predictions proposed by scientists who don’t seem to have an in-depth understanding of the immunology involved!

A limitation of all modeling exercises is that models are themselves merely an approximation of reality. Modeling is an exercise in inference—taking specific data and building general predictions and expectations about broader aspects of the phenomenon being modeled (in this case, viral propagation and evolution). The inferences can only hold up so long as the estimates and assumptions about the underlying reality are valid. Vanden Bossche highlights one key assumption about the Yaniv study that is, in his view, invalid: that levels of viral shedding correlate to levels of infectiousness.

…Ab-mediated enhancement of infection with Omicron in vaccinees does not translate into enhanced viral shedding from the gastrointestinal tract, which is the primary source of wastewater contamination.  On the other hand, diminished shedding in vaccinees is likely compensated by its prolonged duration due to a delay in viral clearance.

Vanden Bossche is addressing the impact vaccination has on fecal shedding of Omicron (and for the complete picture of that impact I encourage everyone to read his entire discussion of the topic), but the study authors’ flawed understanding of levels of viral shedding also point to another flaw in the study itself: there is no discussion of the assumed ratio of total cases to detected cases (detected by individual diagnostic testing). This is a key assumption, as prior research on wastewater surveillance and modeling has asserted—particularly the recent research by Daniele Proverbio, et al:

Another parameter to be estimated is the average ratio of total and detected cases at day t, ηt. This is necessary to link the measurements of population testing with those of wastewater analysis (ideally objective and insensitive to testing capacities).

As the Proverbio study articulates, this assumption can be highly variable, as levels of testing within a population are themselves prone to variation, yet this assumption is necessary to establish a quantitative link between levels of virus detected in wastewater and the number of likely individuals who shed that virus—which is a direct correlation to overall infectiousness. The Yaniv study makes no mention of this ratio whatsoever, which immediately invalidates the model’s conclusions on structural grounds even before the impact of COVID-19 vaccination on viral shedding highlighted by Vanden Bossche comes into play.

Ironically, the Proverbio study is actually cited by the Yaniv study, making the omission especially glaring and perplexing.

There is an additional assumption in the Yaniv study that must be questioned: the propensity and/or inevitability of Omicron displacing Delta. While displacement has certainly been the prevailing pattern among pre-Omicron SARS-CoV-2 variants, Omicron represents a significant divergence from other variants, so much so that it arguably presents a completely different serotype.

A serotype is defined as a variation within a microbial species, distinguished by the humoral immune response. The serotype classification of bacteria or viruses is based on their surface antigens and was established before the availability of other techniques, such as genome sequencing or mass spectrometry. Antibodies generated to one serotype do not usually efficiently protect against another serotype. Serotypes have been described in many viral species and generally correspond to genotypes. A classification by serotype is not unprecedented in the family Coronaviridae, for example, feline coronavirus (FCoV) has two serotypes.

The expectation that Delta would decrease in prevalance in correlation to Omicron increasing in prevalence necessarily presumes a single SARS-CoV-2 serotype. As the discussion by authors Etienne Simon-Loriere and Olivier Schwartz of the Institut Pasteur articulates, Omicron arguably is a second serotype of the virus. If Omicron may be fairly considered a distinct serotype, then the expectation of displacement that is at the core of the Yaniv study becomes problematic, as multiple serotypes of a virus can and do coexist in nature.

Flawed and problematic assumptions leave the Yaniv study on a highly unstable and unreliable foundation.

The Bad Testing In The Yaniv Study

The Yaniv study’s conclusions are further undermined by a potential and egregious flaw in the testing itself. For reasons unknown, the study authors opted for a cycle threshold of 40 to use with their RT-qPCR test assays (emphasis mine):

For analysis purposes, a few criteria were chosen. Samples considered “Invalid”, “out of range” or “Unique” were not taken for analysis. Samples deemed as “Invalid” were samples without a detection signal for the RdRP gene (83/376). Samples deemed as “Out of range” were samples with a detection signal for Delta or Omicron larger than Ct 40 (17/376). Samples deemed as “Unique” were samples with a detected signal for RdRP gene, yet no detection signal for neither the Delta nor the Omicron detection (9/376).

As I have discussed previously, this threshold is likely far too high to yield accurate test results.

Did Rochelle Walensky Just Demolish The Credibility Of PCR Testing?

At high cycle thresholds, the possibility of a PCR assay detecting viral fragments and nucleotides, rather than whole culturable virus, is greatly elevated.

This is why the CDC does not culture positive tests with a Ct value above 28. This is why the CDC, per Director Rochelle Walensky, dropped a negative PCR test from criteria for ending isolation.

Even Anthony Fauci has acknowledged that at high cycle thresholds one is merely detecting dead nucleotides.

The detection of whole virus versus dead nucleotides has obvious and direct impact on quantitative PCR testing. Having a cycle threshold set too high skews the test results and, as a consequence, further invalidates the modeling derived from those results.

At a minimum, authors Yaniv, et al, should have included a discussion of the use of the Ct value of 40. Better would have been to use a lower Ct value.

You Get What You Measure

Noted technology author and commentator Bob Lewis has summarized the importance of sound measurements in his “First Law Of Metrics”:

Last week's missive on stupid consultant tricks introduced Lewis's First Law of Metrics: You get what you measure -- that's the risk you take. Our help desk tale of woe leads to us to Lewis's Corollary to the First Law of Metrics: If you mismeasure, you mismanage.

As study authors Yaniv, et al, demonstrate, if you mismeasure you also mis-model. The importance of proper measurements and proper assumptions is as applicable to environmental virology as it is to information technology. With flawed measurements and flawed assumptions the end result is a flawed model.

What Yaniv, et al, have devised can only be described as a flawed study, based on flawed assumptions, using flawed testing, resulting in a flawed model.

As the 2013 Israeli experience with poliovirus surveillance illustrates, wastewater surveillance can be a powerful early warning mechanism for detecting infectious pathogens before they spread too far within a community. Wastewater surveillance is a far more reliable surveillance technique than mass diagnostic testing, being logistically simpler and less intrusive on people’s lives. For the purposes of broad disease surveillance, we are well advised to seek expanded wastewater surveillance as a superior alternative to mass diagnostic testing (with the caveat, of course, that the testing used itself be proven accurate and reliable).

Yet wastewater surveillance is just that—an early warning mechanism. To extrapolate from the relatively narrow data set provided by such surveillance into a model of viral evolution going forward is stretching the data farther than it logically can go. While the end results can be quite fascinating, they are neither reliable nor useful.