This chapter presents two major dynamics issues relevant to the
design and evaluation of screening programs. Both of these issues
demonstrate the inadequacy of the standard static approach to
evaluating screening programs through the use of test sensitivity,
specificity, and predictive values. The first is that screening
changes the prevalence of asymptomatic and undiagnosed infection
in the population that is being screened. In order to optimally
design and evaluate a screening program, the temporal course and
extent of that change must be evaluated with an analysis of the
dynamic system that gives rise to the change. The second is that
screening intervals are an element of a dynamic system and can
only be optimized through a dynamic systems analysis.
A simple approach to a systems analysis which can be used to predict
the course of changes in prevalence and can optimize the screening
interval is presented. This approach minimizes the parameters
and compartments needed by making many sweeping assumptions.
Many ways to elaborate the model in more realistic ways with more
compartments and parameters are suggested. The principles of
performing a cost-effectiveness analysis using a dynamics systems
model are illustrated using the simple model. One parameter that
can be adjusted to optimize cost-effectiveness is the screening
interval.
An additional issue addressed is the use of receiver operating characteristic curves to optimally choose the test cutpoint which maximizes the cost-benefit ratio. The standard approach to use of the receiver operating characteristics curve is shown to be incomplete and potentially deceptive. A final issue addressed is how to decide on changing a screening test when the existing test has already very successfully controlled disease. The example of cervical cancer is discussed.
Screening programs are the classic secondary prevention programs
of Public Health. Their purpose is not to keep a disease process
from starting, but to reverse or detain a disease process after
it has already started. The primary objective of a screening program
is to find and treat individuals in the early stages of disease.
Most screening programs employ a cheap, but imperfect, test to
"screen" a population. Those who are culled by the screen
have a high likelihood of being in an early stage of disease.
They then receive a more thorough and expensive diagnostic evaluation
as to whether or not they are in the early stages of disease.
A disease must have certain characteristics in order for screening to be a useful means of controlling that disease.
1 People must stay in the early stages of disease long enough so that they can be detected by a screening program while they are still in that stage. Diseases that progress rapidly from their onset to a symptomatic state that should occasion medical care are not candidates for a screening program.
2 There must be something that can be done for people correctly identified as in the early stages of disease so that they are either cured or their progression to more serious illness is stopped or slowed. It is not just the duration of the asymptomatic stage which is important. It is the duration of stage of disease which is both detectable and treatable that is important.
3 The screening test must a) detect individuals early in the asymptomatic state, b) be socially acceptable, and c) be economically acceptable. A test that picks up individuals only toward the end of the asymptomatic state will not be acceptable. Neither will a test that is too invasive or too expensive.
4 The incidence of disease must be high enough so that the yield of cases prevented by the screening program justifies its cost. If the incidence in the overall population is too low, then one might identify a sub-population with a high enough incidence to justify a screening program.
5 The consequences of a positive screening test for individuals who do not have the disease must not be too severe for those individuals or for society. Even when the overall benefit of a program to society may be great, society is often reluctant to support a program that harms individuals who would not otherwise be harmed.
6 There must be some therapeutic or preventive advantage to identifying
individuals in an early disease state as opposed to the disease
state where they would be identified without a screening program.
If the case detected in the early stage of asymptomatic disease
would have been detected and cured soon after the onset of symptoms,
then less has been gained than if the symptomatic stage is essentially
incurable. Likewise if there is no treatment regardless of the
stage of disease, then screening will not help.
Certainly many more diseases have the above characteristics than
those that are currently the target of screening programs. Several
factors are increasing the number of appropriate diseases. 1)
Modern molecular biology is greatly expanding our ability to detect
early stages of disease and thus expanding the opportunities for
screening programs. 2) Better therapies for early disease are
also increasing the range of appropriate diseases. 3) The marginal
costs of screening programs are being reduced as they are being
combined with other screening programs.
But we cannot jump at every opportunity to screen without considering
the consequences. It is quite possible to waste a lot of money
on disease screening that could be more efficiently spent in other
disease control programs. It is also quite easy to misjudge the
benefits of a potential program and thereby forgo a productive
disease control program if one does not understand the principles
of how screening programs affect disease dynamics. Thus epidemiologists,
health officials, and administrators need to understand the dynamics
of screening programs that can be learned from the exercise in
this chapter.
In introductory epidemiology texts, the quantitative principles
behind screening programs are commonly taught as static measures
referent to the interpretation of screening test results. The
emphasis in introductory courses is on the sensitivity and specificity
of a test, the receiver operating characteristics of that test
when it is used under conditions that vary its sensitivity and
specificity, the predictive values of positive and negative tests
when the test is applied in a population, and the yield of a screening
program. The classic introductory treatment of screening fails
to deal with how a screening program changes the prevalence of
asymptomatic and undiagnosed disease and thereby changes predictive
values of tests.. The dynamic rates of asymptomatic and symptomatic
disease development are not used in introductory analysis of screening
programs This exercise is designed to help you start thinking
about the dynamics of screening programs.
It is easy conceptually to see that prevalence of the asymptomatic
disease being screened for should not be treated as a static element
in a screening program. The incidence rate of asymptomatic disease
and the diagnosis rate of symptomatic disease generate a balance
of inflow and outflow of asymptomatic and undiagnosed individuals
in the well compartment subjected to screening. This balance determines
the prevalence. The screening program generates a new outflow
of diagnosed cases no longer in the pool of individuals subjected
to screening. That immediately sets the prevalence on a decreasing
trajectory until a new balance is achieved. The predictive values
of positive and negative tests and therefore the costs and benefits
of the screening program will go through a process of change until
the new equilibrium is achieved. Thus there are two endemic prevalence
levels to be considered in evaluating a screening program: 1)
the prevalence before the program is instituted, 2) the equilibrium
prevalence that is finally achieved some years after a constant
application of the program.
This leaves a number of questions: 1) Should one use the prevalence
at the time the screening program is begun or at the time the
new prevalence is achieved to calculate the costs and benefits
of the screening program? 2) Which prevalence should one use to
determine the point on the receiver operating characteristic curve
to set the cut point on a screening tool where there is an optimal
tradeoff between sensitivity and specificity? 3) When one wants
to consider adopting a new screening tool, should one use the
current prevalence or the prevalence before the screening program
was begun to make one's calculations?
The answer to all of these questions is that one should use neither the prevalence at the start of the screening program or the prevalence after the new equilibrium due to the screening program is reached. Instead of using a prevalence to calculate static values for the screening program, one should conduct a dynamic analysis of the program using the rates of transition into an asymptomatic disease state and out of it. One should use data on the natural history of infection to determine the transition out of the asymptomatic state as a function of time since entering it. Likewise one should use data on the particulars of the screening program that one is contemplating. Those particulars include not only the sensitivity and specificity of the test used, but the rate or frequency at which that test gets applied to individuals who do not have a diagnosis of disease. The rate at which individuals are screened again after their original screening is not even an element of the standard static analysis of screening. Thus the standard static analysis of screening cannot establish the optimal interval for screening. Since the rate of screening is an essential element of the dynamic analysis of screening, however, a dynamic analysis can establish that interval.
To make a cost-benefit analysis of a screening program, it is
insufficient to calculate the costs and benefits only at the outset
of a screening program. As the screening program is instituted,
the predictive values of tests will change and consequently the
cost-benefit balance will change. The costs and benefits of screening
at any moment in time are a function of a changing prevalence
of asymptomatic but undiagnosed illness. Rates of change must
thus be the input parameters for a cost benefit analysis rather
than prevalence, sensitivity, and specificity.
To decide what screening and definitive tests to use, to decide on the cut points of those tests to use, to decide on screening intervals, and to decide whether one wants to undertake a screening program at all, one must balance costs and benefits. Costs include:
1 the costs of administering the screening tests,
2 the costs of administering definitive diagnostic tests when the screening test is positive,
3 the time that it takes people to get the tests,
4 the costs of treatment for the asymptomatic cases that are detected,
5 the costs of what happens to people who are identified as false positives that wouldn't have happened to them otherwise, and
6 the costs of what happens to people who are falsely identified as negative when they are truly positive and might have been treated earlier without the screening program.
7 the anxiety generated by the concern about the disease raised
by the screening program.
Benefits include
1 The years of life added by identifying and treating a disease during its asymptomatic stage.
2 The peace of mind provided to those with a negative test.
3 The medical costs saved by not having to treat advanced cases
of disease
A cost-benefit analysis thus involves estimating the yield of
a program in terms of either the early cases identified or the
number of cases prevented and the costs in terms of all the previous
factors mentioned. A complete cost-benefit approach would include
using an age structured model with appropriate death rates by
age so that one could calculate an appropriate value for the first
item under benefits. Since we did not get around to age structured
models this semester, however, we will have to settle for a simpler
outcome which assumes that everyone is dying at the same rate.
For the sake of this exercise we will make a further simplification.
We will limit our cost considerations to the number of false positives
with their attendant costs and the program administration costs
for just the screening test. We will unrealistically assume that
the costs for a definitive diagnosis are the same for true positive
individuals on the screening test as they would be if those individuals
were diagnosed later. Similarly we will disregard the costs of
false negative test results.
The number of true positives detected depends upon the prevalence
of the asymptomatic disease in the non-diseased population. The
number of false positives also depends upon this prevalence. The
static cost benefit analysis approach is to calculate the cost
and benefit values at some observed prevalence level in order
to evaluate the utility of a screening program. This assumes that
the characteristics of the program will remain constant and the
prevalence of undetected disease will remain constant. Thus we
could call this a "static" analysis. It only evaluates
costs and benefits at one point in time. Instead of being based
on dynamic measures like rates, it is based on measures that disregard
changes over time, like disease prevalence, test sensitivity,
and test specificity.
One screening issue that is often not correctly addressed using
the static approach is determining what the interval for screening
tests should be. Here one key issue is how fast the disease progresses
from the time it is first detectable to the time when clinical
symptoms make it detectable without a screening program and treatment
becomes less effective. A second key issue on which we will not
focus is how much variation there is in that rate of disease progression.
This variation issue is quite crucial as it is a major determinant
of how much is gained and lost as the interval is lengthened and
shortened. We do not want our presentation here to give the impression
that this issue is not important. Within the ordinary differential
equation approach we present here, one could divide the distribution
into discrete segments and perform the type of cost benefit analysis
we will present within each of those segments and then sum up
the total costs and benefits. Alternatively there are partial
differential equation approaches but they are beyond the scope
of this course.
How the dynamics of the disease process and the dynamics of the
screening program interact to affect the dynamic state of disease
prevalence is the focus of this exercise. The importance of the
dynamics of the disease process in designing screening programs
is often acknowledged at the individual level. The amount of time
that an individual spends in the stages of a disease process where
that process is detectable and/or controllable before it produces
clinical symptoms is recognized as a key element determining whether
or not a screening program will be successful. But the influence
of the rate of progression to symptomatic or uncontrollable disease
and rate of developing asymptomatic disease on the reduction in
asymptomatic disease prevalence is not often considered.
The reason why the importance of the dynamic response of the population
prevalence of asymptomatic and controllable disease to screening
programs is very commonly not appreciated by those who design
and implement screening programs is that the common frame of reference
of epidemiological theory is static risks rather than the rate
of dynamic processes. Most epidemiologists just do not have the
training necessary to consider and perform a dynamic analysis.
Now that you have gotten this far in this course, however, you
are not one of those very limited epidemiologists.
Although screening program dynamics have been well analyzed in
the literature, for example by David Eddy, dynamic considerations
often do not come into play when policy decisions are made because
dynamic analysis is either inaccessible to those designing and
running programs or more likely, those individuals have just never
considered the importance of dynamic issues. Even when screening
program administrators take a dynamic point of view, the dynamic
complexities of a wholly mathematical dynamic analysis may quickly
overcome and confuse those administrators. The computer simulation
of such processes of such processes which we present here, however,
should be less overwhelming. Hopefully the type of computer models
that you learn here will help you better understand screening
programs and at the same time make you more effective in communicating
the implications of disease and program dynamics to program administrators.
We will first build a simple model of the development of a disease
that is controllable through a screening program. Next we will
consider a simple model of a screening program. Despite the simplifications
which we will make in each of these models, we will see there
is some degree of complexity when we put them together. To get
closer to realistic situations, we will have to add considerably
more complexity. We begin simply because it will aid our understanding.
But we begin with the understanding that we will have to add more
realistic complexity later. That is the path of all good modeling
and all good science.
The simplest possible disease progression model with a pre-clinical stage which makes it appropriate for a screening program is one which classifies individuals into one of three alternative states we will incorporate into our model:
1) Well and not in the process of developing disease.
2) In a pre-clinical state where the incipient disease process is undetected. We assume that anyone who has begun on a pre-clinical course to disease is asymptomatic. In this asymptomatic, pre-clinical state, we assume that the disease process is reversible given a medical intervention so that disease is controllable or curable. To maintain simplicity, however, we will assume that without intervention the disease will inevitably progress to the clinical state and there will be no spontaneous reversions. This later assumption is particularly unrealistic for diseases like breast cancer.
3) Clinically manifest disease. This we assume to be incurable.
Note that in this model we consider the development of clinical
disease to be a fate similar to death. Consequently once an individual
is diseased, we do not follow them any further. They have reached
the endpoint in which we are interested. If we mount a screening
program, our objective will be to keep people out of this clinical
disease state.
These three states can be appreciated in the STELLA® model Diagram 10.1. Other simplifying assumptions intrinsic to the model presented in diagram 10.1 are:
1) the population size is in equilibrium with a constant number of individuals coming into and leaving the population,
3) All individuals in the population die at the same rate,
4) all individuals run the same risk of progressing from the asymptomatic disease state to the symptomatic disease state regardless of when they entered the asymptomatic state, (You should now be able to use the lessons of Chapter 9 to relax this unrealistic assumption.)
5) the symptomatic disease state does not spontaneously revert back to the well state or the asymptomatic state,
6) as soon as disease becomes symptomatic it becomes incurable, and
7) death rates are constant regardless of whether or not individuals
have asymptomatic disease and regardless of when they entered
the population.
Our simple disease system has a only three basic parameters:
1) the rate at which asymptomatic (curable) disease is developed,
2) the rate at which that disease becomes symptomatic (and incurable),
3) the background death rates for well individuals and those with
asymptomatic disease.
This model could be elaborated to make disease progression more realistic. Such elaboration will increase the number of parameters in the model. These include:
1 increasing the number of stages within our current asymptomatic and clinical disease categories,
2 including spontaneous reversibility within both the asymptomatic and symptomatic disease states,
3 including detection of asymptomatic disease in the absence of a screening program,
4 specifying different rates of cure in the symptomatic and asymptomatic states depending upon how far disease has progressed,
5 specifying different rates of entering the disease development process as a function of risk group status
6 specifying different rates of detection in the absence of a screening program as a function of how far the disease has progressed (or as a function of other factors like risk group for the development of disease),
7 specifying different rates of progression of disease as a function of risk group,
8 specifying some correlation in the rates that individuals progress through the various stages of disease and how curable individuals are, for example the rate of passing through the undetected pre-clinical state of disease may be correlated with the likelihood of achieving a cure once the symptomatic disease state has been reached,
9 specifying the effects of stage in progression of disease on death rates,
10 specifying the effect of age on background death rates,
11 including competing risks of death from diseases other than those affected by the screening program,
12 specifying differently the shape of the probability distribution
for time spent in the asymptomatic state of the disease (for example
using a conveyor stock).
For the evaluation of cervical cancer screening programs I have
considered, each elaboration in the above list is relevant. In
addition for the cervical cancer screening program it is necessary
to elaborate the disease development process to include HPV infection.
Our simplest disease progression model without any screening model
added to it is shown in diagram 10.1.
DIFFERENCE EQUATIONS FOR STOCK VALUES:
Clin(t) = Clin(t - dt) + (UndetClinIncid) * dt
------------INIT Clin = 0
-----UndetClinIncid = UndetPreClin*SympDisRate
UndetPreClin(t) = UndetPreClin(t - dt) + (PreClinIncid - UndetClinIncid - BackDthPreClin) * dt
------------INIT UndetPreClin = Well*(AsympDisRate/(SympDisRate+DeathRate))
-----PreClinIncid = Well*AsympDisRate
-----UndetClinIncid = UndetPreClin*SympDisRate
-----BackDthPreClin = UndetPreClin*DeathRate
Well(t) = Well(t - dt) + (InFlow - PreClinIncid - BackDthWell) * dt
------------.INIT Well = InFlow/(DeathRate+AsympDisRate)
-----InFlow = 100
-----PreClinIncid = Well*AsympDisRate
-----BackDthWell = Well*DeathRate
PARAMETER VALUES
AsympDisRate = .05, DeathRate = .02, SympDisRate = .05
Homework 10.1
Present the algebra which confirms that the way we have set the
initial values in this model are the values of the model at equilibrium.
Note that we did not use a birth rate here as we have in some previous exercises, we just used a constant inflow into the population. For now, we are not interested in the effects of birth and death dynamics on this screening program. Consequently we just used a constant inflow of births without defining a birth rate. We assume that when we start our screening program our population is in equilibrium. We do not consider how our program could upset this equilibrium by increasing population size. The degree of model elaboration required to do that does not seem appropriate. If you had some strong justification for a particular program, such as an HIV screening programming, that the population was not in equilibrium with regard to the disease stage classifications, then you would want to add to the model the more realistic dynamics.
Homework 10.2
How would the prevalence of asymptomatic disease among individuals without symptomatic disease (i.e. undetpreclin/[undetpreclin + well]) change if
a The population inflow is doubled and we begin at the equilibrium values appropriate to that inflow.
b The population death rate is doubled and we begin at the equilibrium values appropriate to that death rate.
c The asymptomatic disease development rate were doubled and we begin at the equilibrium values appropriate to that disease rate.
d The symptomatic disease development rate were doubled and we begin at the equilibrium values appropriate to that disease rate.
Present comparative graphs that show the system running at the
current equilibrium and then undergoing a change that involves
the various doubling of flows or rates. Explain why the different
patterns you present occurred. To help with explanations, you
may want to present curves of variables other than the prevalence
of asymptomatic disease.
The elements of a screening program in this population:
Now let us consider a simplification of an actual screening program.
Our screening program will use a screening test to detect individuals
in the asymptomatic state. That screening test will have a given
sensitivity and specificity. This sensitivity and specificity
will not depend upon how long people have been in the asymptomatic
state or on the rate of disease development. Such dependence can
occur with many tests. Once individuals with pre-clinical disease
are detected, they will be subject to a treatment program that
will have two effects. It will return pre-clinically diseased
individuals to the well state at a certain rate, and it will slow
the progression of those individuals from the pre-clinical to
the clinical state.
The additional elements which we will add to the previous model in order to build a screening program model are:
1 an additional stock of individuals who have detected asymptomatic infection,
2 the rate at which the screening is applied to the population.
3 the sensitivity of the screening test.
4 the specificity of the screening test
5 the rate at which detected asymptomatic cases are cured.
6 the rate at which detected asymptomatic cases progress to clinical
cases
Note that the test specificity and the rate at which screening
produces false positive results do not affect the distribution
of disease in the population at all. Individuals who have a false
positive test result are still considered to be well. But if we
are evaluating the performance of a screening program, we will
want to keep track of the number of false positives generated.
Each false positive case will have financial costs in terms of
the special diagnostic procedures to which they have to be submitted
in order to rule out disease. In addition, being falsely diagnosed
might have serious emotional or physical consequences. Thus we
want to keep track of these false positives. Note that we do that
in the model shown in diagram 10.2 with a separate stock. The
flow rate into that stock is one minus the specificity times the
number of well individuals subject to screening times the frequency
at which individuals are screened. This stock is not part of our
compartmental system. It tabulates separately of a fraction of
the population which is in the compartmental system. Similarly
for program evaluation purposes we may want to keep track of the
total number of screening tests performed. Since there is a cost
for each test, keeping a separate tabulation of the number of
tests performed will help with a cost benefit evaluation. Again
we have done this with a separate stock. The flow into this stock
is simply the number of individuals in the population that is
being screened times the screening rate.
These two flows and stocks which are separate from the compartmental
system represent costs of the screening program. The flows represent
the current costs per unit time and the stocks represent the cumulative
costs from the onset of the program. Treatment represents an additional
cost. This cost would be proportional to the size of the stock
of detected, asymptomatic individuals if treatment must be continued
until cure. Or it could be proportional to the flow into this
compartment if treatment is a one time event. This latter approach
will be easier for us. There could in addition be start up costs
which we will not consider.
The benefits of the program are the additional years of life in
those cases which are detected and cured or detected and whose
fatal disease is delayed. When and how the benefits are to be
counted is an important issue which we will not deal with completely
until the next section. Many program administrators count the
benefits in terms of the number of asymptomatic cases detected.
But not all of these are cured. Moreover, at the moment they are
detected their life is not improved over what it would have been.
The action taken improves their life at some later time. We will
see that there could be important differences in program evaluation
depending upon when and how the benefits are counted. But to begin
with a simpler procedure, we will count benefits in terms of the
number of asymptomatic cases detected.
We do not distinguish in our model well individuals who have not
been screened before, well individuals who have been screened
and had a negative test, and well individuals who have been screened
and had a false positive test. If we decided that such individuals
would have different screening procedures in the future, then
we might want to divide the stock of well individuals into well
individuals who have not been tested, well individuals with a
negative test result, and well individuals with a false positive
test result. But for now let us keep things simpler.
We start our simulation model at the equilibrium distribution of well and asymptomatic but diseased individuals if there had been no screening program. Our model assumptions include the following:
1 Individuals are subject to screening at a rate that is independent of whether or not they have been screened in the past or when they have been screened in the past.
2 Screening rates are unrelated to the risk of disease development or the presence of a pre-clinical undetected state of disease.
3 Screening rates are unrelated to death rates.
4 Individuals who have been cured are indistinguishable from well
individuals.
MODEL EQUATIONS AND STARTING VALUES
ClinS(t) = ClinS(t - dt) + (UndetClinIncidS + DetClinIncidS) * dt
------------.INIT ClinS = 0
-----UndetClinIncidS = UndetPreClinS*SympDisRate
-----DetClinIncidS = DetPreClinS*.01
DetPreClinS(t) = DetPreClinS(t - dt) + (preclinDets - DetClinIncidS - Cures - DetPreClinDthS) * dt
------------.INIT DetPreClinS = 0
-----preclinDets = UndetPreClinS*(ScreenFreq*ScreenSens)
-----DetClinIncidS = DetPreClinS*.01
-----Cures = DetPreClinS*Cure_rate
-----DetPreClinDthS = DetPreClinS*DeathRate
FalsePos(t) = FalsePos(t - dt) + (FalsePosF) * dt
------------.INIT FalsePos = 0
-----FalsePosF = WellS*(1-Specificity)*ScreenFreq
Num_Screens(t) = Num_Screens(t - dt) + (Screen_Flow) * dt
------------.INIT Num_Screens = 0
-----Screen_Flow = (UndetPreClinS+WellS)*ScreenFreq
UndetPreClinS(t) = UndetPreClinS(t - dt) + (PreClinIncidS - UndetClinIncidS - BackDthPreClinS - preclinDets) * dt
---------------.INIT UndetPreClinS = WellS*AsympDisRate/(DeathRate+SympDisRate)
-----PreClinIncidS = WellS*AsympDisRate
-----UndetClinIncidS = UndetPreClinS*SympDisRate
-----BackDthPreClinS = UndetPreClinS*DeathRate
-----preclinDets = UndetPreClinS*(ScreenFreq*ScreenSens)
WellS(t) = WellS(t - dt) + (InFlowS + Cures - PreClinIncidS - BackDthWellS) * dt
------------.INIT WellS = InFlowS/(AsympDisRate+DeathRate)
-----InFlowS = 100
-----Cures = DetPreClinS*Cure_rate
-----PreClinIncidS = WellS*AsympDisRate
-----BackDthWellS = WellS*DeathRate
PARAMETER VALUES
AsympDisRate = .05, Cure_rate = 1, DeathRate = .02, ScreenFreq = .5
ScreenSens = .7, Specificity = .7, SympDisRate = .05
Note that there are many different elements now that may affect
the behavior of this system. These include the rate of asymptomatic
disease development, the average duration of asymptomatic disease,
the distribution of the duration of asymptomatic disease (which
is negative exponential in our model), the frequency of screening,
the sensitivity of screening, the effectiveness of treatment programs
for the asymptomatic but diseased individuals, the inflow rates,
and death rates in the population. Note again that the specificity
of the screening procedure does not affect the behavior of this
system. It only effects the costs of the screening program.
As commented on above, in our model screening program everyone
runs the same chance of being screened no matter how recently
they have been screened before. You should be able to say why
the way we have set up our first order difference equations accounts
for this characteristic of our simulation. This is one aspect
of our simulation that is unrealistic. An alternative would be
to screen everyone at one point in time determined by the screening
interval. One way to do this we would use the impulse function
of STELLA® II so that everyone is screened at exactly the
same time and the same interval. This would be analogous to using
a conveyor to determine screening time so that the distribution
to screening times would be a point mass function. We could define
distributions of times to screening that are intermediate between
the negative exponential and the point mass function using multiple
stages. We would do this in a manner analogous to the way we built
Erlang distributions for incubation periods. This approach has
time progressing in two dimensions, chronological time and time
since last screening test. But for class purposes, we will go
with the simpler model presented here.
Given our first order equations with only one stage of asymptomatic
disease, both the frequency of screening and the sensitivity of
the screening test are multiplied by the number of asymptomatic
individuals to determine the rate that asymptomatic individuals
are detected. We can increase the yield of the screening by exactly
the same amount if we cut the screening interval in half or double
the sensitivity. This would not be the case if the time between
screenings were a fixed interval, that is to say if we had used
the impulse function to establish the screening as discussed above.
Homewok 10.3
Describe the temporal pattern of the prevalence of asymptomatic disease after a screening program is initiated by running the simulation. Examine the sensitivity of this pattern to the frequency of screening and the sensitivity of the assay. Do this by using a comparative graph with the following sensitivity settings:
a) test sensitivity is fixed at a high value and screening frequency varies.
b) test sensitivity is fixed at a low value and screening frequency varies.
c) screening frequency is fixed and test sensitivity is varied.
d) different screening frequencies times test sensitivity have
the same value.
Homework 10.4
Examine the effect of doubling and halving the screening frequency 1) upon the proportion of the well population that has undetected asymptomatic disease, 2) upon the flow of newly detected asymptomatic cases, 3) upon the predictive value of a positive test (make your own derived variable), and 4) upon the rate of symptomatic disease in the overall population. Do this at a point in time soon after the program has begun so that the prevalence of undetected disease has not been much altered and then do it again once the system comes into equilibrium. This system is too complex to solve directly for the equilibrium values. You will have to run the system examining values at intervals in a table to be sure that you have reached equilibrium. Some systems may not settle to a single equilibrium value. They may cycle regularly or they may be chaotic and never settle down even to a regular cycle. This system, however, is well behaved and comes to a nice equilibrium.
a) Present your results in the form of a graph or table.
b) Discuss the marginal returns of increasing screening frequency at the beginning of a screening program and after the program has come into equilibrium. Use the test sensitivity and specificity values in the above model.
c) How is the rate of false positive tests related to the proportion of the well who have asymptomatic disease, the test specificity, and the screening frequency? Is there any difference in these relationships early in the program and at equilibrium?
d) How is the rate of detection of asymptomatic cases related to the test sensitivity, the prevalence of asymptomatic disease, and screening frequency? Is there any difference in these relationships early in the program and at equilibrium?
e) How is the rate of symptomatic disease related to the test sensitivity, the prevalence of asymptomatic disease, and screening frequency? Is there any difference in these relationships early in the program and at equilibrium?
Homework 10.5
Predict how doubling either the asymptomatic disease development
rate or the symptomatic disease development rate would affect
the marginal returns you examined in part b) of the previous question.
Check out your predictions by conducting appropriate model analyses.
A simulation to assess the costs and benefits of a screening program:
One tactic in assessing the potential effect of a screening program
with a simulation is to compare the simulation results with and
without a screening program. This approach will count the benefits
not at the time that an asymptomatic case is detected, but at
the time that detected case develops disease in the simulation
without the screening program.
In the case where the system without the program is at equilibrium,
that comparison can be made without running a simulation where
there is no program. One could just compare the values with the
program at any point in time to the equilibrium values without
the program. But when there are factors that would move a system
off equilibrium even without a screening program, you may only
be able to project the expected behavior of the system without
a screening program through a simulation. For example in examining
the effects of an HIV screening system, only the simulation approach
would be possible. We will use a simulation approach but will
not examine non-equilibrium conditions with the model. The simulation
program in Diagram 10.3 builds upon the two previous programs
we have presented, it adds some costs to the previous programs,
and some comparison variables and calculates both the instantaneous
costs per case prevented and the cumulative costs per case prevented.
MODEL EQUATIONS AND STARTING VALUES
Clin(t) = Clin(t - dt) + (UndetClinIncid) * dt
------------INIT Clin = 0
-----UndetClinIncid = UndetPreClin*SympDisRate
ClinS(t) = ClinS(t - dt) + (UndetClinIncidS + DetClinIncidS) * dt
------------INIT ClinS = 0
-----UndetClinIncidS = UndetPreClinS*SympDisRate
-----DetClinIncidS = DetPreClinS*.01
DetPreClinS(t) = DetPreClinS(t - dt) + (preclinDets - DetClinIncidS - Cures) * dt
------------INIT DetPreClinS = 0
-----preclinDets = UndetPreClinS*(ScreenFreq*ScreenSens)
-----DetClinIncidS = DetPreClinS*.01
-----Cures = DetPreClinS*1
FalsePos(t) = FalsePos(t - dt) + (FalsePosF) * dt
------------INIT FalsePos = 0
-----FalsePosF = WellS*(1-Specificity)*ScreenFreq
Num_Screens(t) = Num_Screens(t - dt) + (Screen_Flow) * dt
------------INIT Num_Screens = 0
-----Screen_Flow = (UndetPreClinS+WellS)*ScreenFreq
UndetPreClin(t) = UndetPreClin(t - dt) + (PreClinIncid - UndetClinIncid - BackDthPreClin) * dt
------------INIT UndetPreClin = Well*(AsympDisRate/(DeathRate+SympDisRate))
-----PreClinIncid = Well*AsympDisRate
-----UndetClinIncid = UndetPreClin*SympDisRate
-----BackDthPreClin = UndetPreClin*DeathRate
UndetPreClinS(t) = UndetPreClinS(t - dt) + (PreClinIncidS - UndetClinIncidS - BackDthPreClinS - preclinDets) * dt
------------INIT UndetPreClinS = WellS*AsympDisRate/(DeathRate+SympDisRate)
-----PreClinIncidS = WellS*AsympDisRate
-----UndetClinIncidS = UndetPreClinS*SympDisRate
-----BackDthPreClinS = UndetPreClinS*DeathRate
-----preclinDets = UndetPreClinS*(ScreenFreq*ScreenSens)
Well(t) = Well(t - dt) + (InFlow - PreClinIncid - BackDthWell) * dt
------------INIT Well = InFlow/(DeathRate+AsympDisRate)
-----InFlow = 100
-----PreClinIncid = Well*AsympDisRate
-----BackDthWell = Well*DeathRate
WellS(t) = WellS(t - dt) + (InFlowS + Cures - PreClinIncidS - BackDthWellS) * dt
------------INIT WellS = InFlowS/(AsympDisRate+DeathRate)
-----InFlowS = 100
-----Cures = DetPreClinS*1
-----PreClinIncidS = WellS*AsympDisRate
-----BackDthWellS = WellS*DeathRate
DERIVED VARIABLES
Case_Prev = Clin-ClinS
Cum_Cost_Prev_Case = Program_Costs/Case_Prev
InstantCasePrev = UndetClinIncid-UndetClinIncidS-DetClinIncidS
Inst_Cost_Prev_Case =((WellS+UndetPreClinS)*ScreenFreq*Test_Cost+FalsePosF*
False_Pos_Cost)/(UndetClinIncid-DetClinIncidS-UndetClinIncidS)
Prev_Undet_PreClins = UndetPreClinS/(UndetPreClinS+WellS)
Program_Costs = FalsePos*False_Pos_Cost+Num_Screens*Test_Cost
PARAMETER VALUES
DeathRate = .02, AsympDisRate = .2, False_Pos_Cost = 500
ScreenFreq = .5, ScreenSens = .7, Specificity = .7, SympDisRate
= .05, Test_Cost = 40
Note that we have calculated both a cumulative and an instantaneous
cost per case prevented. The difference here is analogous to the
difference between a risk and rate. The cumulative number of cases
prevented is just the number of clinical cases without the screening
program minus the number of cases with it. The instantaneous number
of cases prevented is the flow into the clinical case stock without
the program minus the flow into it with the program. Since the
screening program has costs right from the start but a case can
not be prevented at a time before which it would have developed
disease, the costs per cumulative or instantaneous case prevented
start off infinitely high and then come down.
The cumulative cost per case prevented and the instantaneous cost
per case are of course dependent upon the costs of a test. In
the model equations these were set at $40 and the costs per false
positive which were set at $500. The cost benefit ratios are not
constant. They change for several reasons. One is that the asymptomatic
disease prevalence changes over time. Another is, however, that
we do not count the benefit from a detected case right at the
time the case is detected. We only count the benefit when we reach
the point in time where that prevented case would have been expected
to have disease. This accounts for a rather immediate and dramatic
drop in the cost-benefit ratio which we can see below. You can
appreciate in graph 10.1 that there is a long term rise after
a short term drop.
The long term rise has to do with the population dynamics which
take a long time to come back into equilibrium. You can appreciate
some of these dynamics from graph 10.2. After an initial sharp
drop in the number of undetected pre-clinical cases, there is
a subsequent rise due to the fact that the size of the well population
rises. This rise then decreases the rate at which cases are being
prevented. If our time units represent years, as would be reasonable
if we were considering a cancer screening program, these extremely
long term trends in our simulation are of lesser interest. Therefore
in Graph 10.3 we just examine the short term effects over 12 time
units.
Homework 10.6
Explain the relationships between the cumulative and instantaneous
cost-benefit curves on the above graphs. Why do they cross?
Homework 10.7
Discuss how you would assess the biases that might arise from
using detected asymptomatic cases as a measure of program benefits
as compared to the benefits derived from cases prevented.
Note that by increasing the amount of money that we spend on the
screening test, we may be able to buy more sensitivity and specificity.
Various published works suggest that for the case of cervical
cancer screening, this may indeed be a rational thing to do.
Homework 10.8
Determine how much increase in sensitivity or specificity would be needed to keep the cost of a prevented case approximately the same if a test that cost $400 instead of $40 were used in the screening. Examine only short term effects through 12 years for this purpose and discuss how the timing of when benefits would come might affect your decision.
Repeat the same analysis but now with a cost of false positives set at $2500 a case. Discuss your results in terms of how value assumptions determine the results of decisions made on the basis of models.
_____________________________________________
In future years I will develop two additional sections of this chapter. The first will analyze the use of receiver operating characteristic curves to choose the optimal cut points for tests. A homework to be performed will bring out the point that using the prevalence at the start of the screening program can result in the use of very non-optimal cut points once the system comes into equilibrium. The second will deal with the issue of adopting a new screening procedure after one has been long in place. This is the situation facing cervical cancer screening. It is clear to me that HPV screening will be far superior to Papinicalou screening. The correct comparison of these two approaches, however, is difficult because they entail detecting different sub-segments of the diseased population. HPV screening will pick up some cases that Papinicalou screening will miss and vice-versa. In fact for HPV screening, each different type is best viewed as a separate disease. Given this overlap, if one uses the prevalence induced by the Papinicalou screening to set screening frequency and ROC cutpoints for the HPV screening, one will be quite far from optimal in setting the program parameters.
_____________________________________________
Review questions
1 In a population at equilibrium with a constant birth rate which exactly equals the death rate, describe the mathematical relationships between the prevalence of the asymptomatic stage of a disease and the rate of the onset of that stage and of the symptomatic stage given that one is already asymptomatic. Assume that the disease does not affect the death rate. Note that the simulation in this chapter achieved population equilibrium by the alternative method of generating a non-homogeneous compartmental model with a constant inflow of births and a fixed death rate.
2 Define two general strategies for seeking a description of how instituting a screening program will change the prevalence of asymptomatic disease in a population. (This is a review question for the course, not necessarily this chapter.)
3 Outline three issues in the design of screening programs that can be analyzed using the rates of asymptomatic disease development and rates of symptomatic disease developmant which determine asymptomatic disease prevalence that cannot be addressed using the asymptomatic disease prevalence directly.
4 What developments are making it possible to control an ever larger number of diseases by employing a screening strategy?
5 Consider the choice between two alternative screening tests, one has higher sensitivity than the other but lower specificity. Describe in detail a scenario where the cost benefit ratio favors one of the tests at the onset of a screening program but the other test later on. Use a mathematical example where you set different levels of sensitivity and specificity for the tests.
6 Describe the temporal pattern of decrease from the endemic equilibrium in the onset of symptomatic disease that would be expected after an effective screening program is instituted. Assume that the detected and cured individuals are indistinguishable from the well individuals as in the model in this chapter. What generates this pattern?
7 What factors determine the optimal screening interval for an asymptomatic and curable disease state? Describe how you would go about determining such an interval.