Carl Waldmann, Max Jonas, Liza Keating and lan J.

Rechner
The contribution of technology assessment

Introduction
“No system has sufficient funds to provide the best possible treatment for all patients in all
situations" [1]
Healthcare budgets cannot be infinite and the costs of healthcare have increased in parallel with
population demographics. Increasing age, combined with the public's expectation that the
healthcare system will be able to cope with an increasing array of diseases has led to a significant
increase in healthcare spending in relation to GDP. In summary there has to be some rational use
of these funds.
Critical care consumes a disproportionate amount of resource for relatively few patients.
Financial analysis, es pecially with current fiscal pressures, may make it difficult to justify
delivering prolonged intensive care for patients with a poor prognosis or who are discharged into
rehabilitation requiring continuing access to the healthcare system [2]. It is therefore manifestly
important, following recognition of a finite budget, that preventing inappropriate prolonged
treatment in patients who do not ultimately survive, and improving the processes for other
patients must be regarded as essential. Rationing of healthcare is a difficult area and quickly
focuses the mind particularly in critical care [3]. This issue confuses non medical personnel
particularly as there is a wide variation in patient care processes. This may happen on many
levels: between countries, between intensive care units (ICU) in neighbouring hospitals as well
as in the same ICUs when consultants do not agree on management protocols [4].
In 1811 the Luddite movement began in Nottingham under the fictitious character Captain Ludd.
The Luddites set about destroying stocking frames, - the new technology of the time, which they
considered would threaten their livelihood. They managed to cause a considerable amount of
damage before the rebellion was squashed. Today ‘Luddite' is used to refer to the individuals
who oppose technological change. One of the challenges of introducing changes in process
particularly where there is a use of new technology is the reluctant healthcare professional.
New technology presents the opportunity to reshape the way in which we deliver healthcare, and
in particular critical care. Whilst the adoption of new technology is critical to progress, its
objective assessment is critical to the development of a service. The costs associated with the
adoption of newer technologies must be balanced by the inevitable need for resource allocation
and hence a decrease in resources available to other areas of health (5].
As the technologies available to critical care have continued to expand, the need for more precise
evaluation of new technologies has arisen. The current economic climate has crystallised the
need for accurate assessments. The situation is likely to become increasingly more acute and
relevant as healthcare systems are working under increasingly tight financial restraints. However
it remains clear that economic evaluation of any new technology must be driven by valid and
relevant clinical outcomes that remain patient centred.
One recent example of the adoption of a new healthcare technology is that of the introduction of
analgosedation using remifentanil for cardiac surgery in Leipzig [6]. In this study for the cost of
purchasing what was considered an expensive drug remifentanil (16,500 Euros) the estimated
savings made as a result of improved patient flow through the unit calculated was 315,000 Euros.
Analysis of this study clearly implied that it was not just the drug that made the difference but
the whole process. The objective of the study was to speed up recovery times while maintaining
safety and improving costs. A total of 421 patients given remifentanil were treated in a fast track
area and were compared retrospectively with patients who had been treated by a standard
protocol. The primary outcomes were time to extubation, length of stay in the ICU and treatment
costs. The results showed that the times to extubation were significantly shorter with 75 min
(range 45-110 min) compared to 900 min (range 600-1140 min) in the standard group. Intensive
care unit stay and hospital length of stay were also significantly shorter (p<0.01). The reduction
of treatment costs of intensive care for patients was 53.5 % corresponding to savings of 738
Euros per patient. As a conclusion, the protocol for cardio-anaesthesia including the central
elements of switching opiate therapy to remifentanil and switching patient recovery to a special
post-anaesthesia recovery and care unit, shortened therapy times and showed itself to be safe and
economically effective.
As a result many intensive care units have reconsidered their policies and process of sedation.
Merely changing the drug has not been the recipe for success.
The need for analgesia and converting from a hypnotic based sedative regime to analgosedation
(with newer drugs such as remifentanil and importantly the involvement of the hospital pain
team) has been the crucial combination in improving patient throughput. The processes involved
and patient pathways have had to be revisited with the use of sedation holding, sedation scoring,
delirium scoring and the initiation of rehabilitation programmes as early as possible for an im-
proved outcome for patients. For this process to be suc- cessful it is important that the process is
introduced
carefully with a carefully planned educational programme for all levels of staff. Furthermore this
educational programme needs to be assessed by monitoring compliance with the process of
sedation and that the education programme is repeated appropriately so that the staff are kept up-
to-date.
What is health technology assessment (HTA)
'Health technologies' are broadly defined to include all interventions used to promote health,
prevent and treat disease, improve rehabilitation and long-term care. This may include drugs,
devices, procedures (surgical techniques, screening and counselling) and even the settings of care
such as general practice, hospitals and care homes.
The NHS Technology Assessment Programme was set up in 1993 and the findings and output
from this Programme directly influence key decision-making bodies such as the National
Institute for Clinical Excellence (NICE). Although the definition of HTA presents a wide
spectrum of interventions, with the exception of drug based technologies, most of the principles
involved can be considered with reference to medical devices.
“Health technology assessment is a multidisciplinary process that summarises information about
the medical, social, economic and ethical issues related to the use of health technology in a
systematic, transparent, unbiased. robust manner. Its aim is to inform the formulation of safe,
effective, health policies that are patient focused, and seek to achieve best value” [7].
Health technology assessment is one facet of healthcare research. Interestingly, whilst the
hierarchy of levels of evidence categorically states that for the evaluation of drugs and
therapeutics the randomised controlled trial remains the gold standard, the accepted level of
evidence for the evaluation of new medical devices is lower down the pyramid and is often based
on case control studies.
The current system in the United Kingdom is the National Institute for Clinical Excellence
(NICE). Although government-funded it remains , an independent authority. Its main purpose is
to
deliver guidance in several areas. It concentrates on public health guidance and health
promotion: the areas of relevance to critical care are clinical.
The contribution of technology assessment
practice guidelines and health technology assessment (8]. Thus it has the ability to direct the new
direction of travel of health and at the same time offer guidance on the tools required.
The aim of the process of technology assessment is to standardise access to healthcare across the
United Kingdom. Indeed following a HTA by NICE the NHS is then legally obliged to fund and
resource the medicine and treatments recommended. The technological assessments undertaken
by NICE represent an objective reproducible technique upon which evaluations are undertaken.
The recommendations from NICE are based on both clinical and economic evidence. The
process of HTA through NICE is essentially based on three stages:
 The initial phase is the scoping stage. Thisconsists of the call for proposals. The
proposals can be received from individuals, industry and organisations. An advisory
committee on topic selection coordinates the process and clarifies the timeline. A
scoping document is drafted and a decision is made by the health minister as to whether
or not this is an appropriate topic for review. This decision is made within the context of
health benefit that maybe derived, the potential impact to health, resources involved and
what this may add to health guidance [8].
 If accepted, the process of assessment is then started. The appraisals committee invites
all stakeholders thought to be associated with the topic. The stakeholders represent a
comprehensive group from a wide background: patient/carer groups, representation from
industry – both pharmaceutical and medical devices, relevant medical disciplines and
from academia. An extensive assessment is drawn up and this forms the basis of the
assessment report.
This is then passed onto the appraisal committee which considers all available evidence in a
transparent and reproducible fashion. The evidence is summarised and preliminary
recommendations are made and published in the form of an appraisal consultation document.
This is an extensive process which adheres to a tight pre-specified schedule. Inherent to this
process is a cost-effectiveness analysis in the form of an incremental cost-effectiveness ratio. A
relevant example of this process to critical care is technology appraisal : drotrecogon alfa
(activated) for sepsis (severe). A consistent approach is adopted for all appraisals [9].
Inevitably the economic aspects have predominated: the ethical aspects of health technology
assessment are also relevant. Although the process of health technology assessment incorporates
many aspects of the problem, the integration of the ethical aspects to the problem are at times
neglected [10, 11).
Whilst large scale application of HTA is integral to the day to day running of any healthcare
system, local application is also important.
The drug-based technologies and pharmaceutical industry are tightly legislated and require
assessments ranging from animal studies to large interventional clinical trials. In contrast,
medical devices have relatively little legislature surrounding clinical introduction and there
frequently may not be a consensus as to what constitutes adequate assessment. There is as a
result a need for further developments of the science relating to HTA. This is especially pertinent
when considering the escalating numbers of devices for which manufacturers make claims
relating to patient care. The ensuing HTA exercises stipulate study design and assessment of
these new technologies in an attempt to judge and evaluate short and long-term benefits.
In effect there are two questions to answer: does the device have a positive clinical impact, and
are the resultant economics affordable in the context of local/national healthcare funding. The
requirement for a balanced approach of science versus economics ensures that the assessment
design will draw on expertise from clinicians, engineers, statisticians, economists, ethicists and
of course lawyers.
The ASSESSMENT Process
The HTA process has 3 core areas:
 device safety,
 device performance
 device cost.

C. Processes
Device safety
This is a complex area where there may be safety aspects related to the using a device as well as
the clinical process and context in which the device is used. Safety also encompasses accuracy of
the device as inappropriate clinical decisions can result from inaccurate measurements.
Frequently overlooked, it is the obverse component of the assessment process which should
scrutinise the impact of not using the device.
Assessing the safety of the device or box itself is a statutory requirement and includes elements
of electrical safety, detailed multilingual instructions, packaging, conformation to environmental
specifications and disposal instructions. All of these issues are dealt with using a European
Standard, referred to as the CE mark (La Conformité Européenne), which certifies that technical
manufacturing standards have been met. It is not a functional guarantee that the device or process
will be of benefit or be safe in clinical use. In the US, the Food and Drug Administration (FDA)
fulfils a similar role but in addition, before granting approval for sale in the US, has a remit
beyond the technical qualifications of the CE mark and mandates some assessment of clinical
operation. The FDA uses clinical experts to review investigational studies relating to the device
to ascertain whether the product does what is claimed by the manufacturer and/or presents
unreasonable risks to the patient. The FDA classify devices into three classes according to the
risk of potential harm to the user, ranging from Class I devices that present minimal risk to the
user e.g. home blood pressure kits, to Class 3 e.g. Dialysis machines or ICU ventilators.
Device performance
Efficacy and effectiveness are used to assess device performance. The efficacy relates to the
device performance under ideal conditions and the effectiveness is the performance under normal
clinical working conditions. This is an important distinction as efficacy ratings may indicate the
necessity for ‘experts' to operate the device, whereas effectiveness relates to the performance
limitations under average clinical conditions. Overall, when reviewing technology, the new
approach should offer improvement over current technologies. This may be because the device
can obtain measurements with less risk to the patient i.e. is less invasive, or has some other
aspect which is advantageous.
Efficacy
Assessing efficacy usually means comparison of the device against a known 'gold standard.
When the measurement is well defined e.g. cardiac output, the efficacy becomes the
mathematical correlates of accuracy.
Accuracy is the closeness of the device measurement to the actual value and can be described by:
 a Correlation (degree of association between readings),
 Precision (the closeness of repeated measurements),
 Bias (the difference between the means of the measurements)
The repeatability and inaccuracy of the gold standard is important as if it has a varying accuracy
then the limits of agreement between it and the technology under scrutiny become wider, less
meaningful and makes definitive comparisons difficult. e.g. thermodilution cardiac output vs.
other technologies.
Where a gold standard reference does not exist then it is necessary to show that the technology
has a clinical utility, usually in relationship to patient outcomes.
Effectiveness
Assessment of effectiveness requires clinical trials in which the technology is used as it would be
under average conditions. Standard rules of engagement for conduct of comparative studies | are
important to produce valid results and it must be appreciated that there may be issues related to
study design.
The two obvious trials are a technology versus a control, or the technology with its measurement
guiding subsequent therapeutic management. For the former type of trial, the data are usually
comparative, whereas for the latter theThe contribution of technology assessment
results tend to be clinical outcome based. For outcome data the ideal measurement is considered
to be mortality, but because of the resource implications of numbers needed to power studies for
mortality, the use of surrogates such as length of hospital stay or number of complications is
common. If using a device to guide therapy it is important to consider the relative contributions
of the device being assessed as opposed to the therapeutic interventions of the protocol which
may be the important determinant of the results.
Device cost
The economics related to health technologies is a key driver for technology assessment. These
principles provide a safeguard for a finite and shrinking health budget and represent an attempt
to ensure that resources and benefits are spread and do not favcur one particular group. The area
of cost containment is complex as in many situations the only costs considered are direcť costs
relating to the technology and disposables. Frequently the indirect costs such as training and
storage are ignored and these may represent a considerable financial load. e.g. a clinical
information system may have a monetary cost per bed but staff training, education and
maintenance personnel may generate considerable additional on-costs.

Procurement
In the hospital arena the principles of HTA are used to enable decisions for device procurement.
When purchasing a device the functionality i.e. safety and performance, has to be considered in
relation to the cost. The selection criteria for the purchase and replacement of medical devices
depends on cost and numbers being purchased. The selection process should also include an
evaluation of the manufacturer/supplier in terms of track record, profile in the local and national
healthcare market, company financial stability/ survival and service capability and response
times.
Technologies have to understandably undergo rigorous safety checks before they can be used on
patients. Once a CE mark has been given to the technology, then the process of procurement to

the healthcare industry occurs. When these barriers are overcome and the technology arrives
within the Intensive Care environment, how does it fit into the processes which are involved in
the care of the acutely ill patient? Factors such as staff confidence, availability and “user
friendliness” of equipment, along with being able to withstand the rough environment and
perceived benefit of the end user, all play a role in successful implementation of technology.
Technological advancement is moving at such a pace that once the technology is being used at
the “coal face", there may be another more modern version available.
Examples of technology assessment and processes
Beds
Beds are probably the most frequently used medical device for individual patient care. The
majority of beds for standard ward care are not suitable for critical care. Frequently thought of as
passive devices offering little more than support and a surface on which a patient is placed, there
has been a failure to recognise that beds have therapeutic potential and enable the delivery of a
wide variety of therapies, hence the term “therapeutic bed'. These therapies include
thermoregulation, pressure related wound therapy, position therapy such as proning and lateral
rotational therapy. The procurement of beds however may not involve the clinical staff which
may result in a suboptimal solution. One should warn against the lack of clinical involvement;
with beds in ICU the ability to reach the airway is important and may be compromised with
certain beds: Figure demonstrates difficulty of access to the head to manage airway on a critical
care bed.
Clinical information systems (CIS)
Clinical benefit of health technology - the ideal scenario!
In any walk of life, if a device is robust, easy to | use, inexpensive, looks "sexy", saves money
through improved processes, and makes the life of the user easier – then it will be a sure winner.
C. Processes
Fig. 1
Demonstration of difficulty of access to the head to manage airway on a critical care bed
Looking at the mobile phone industry and how the media have advertised mobile phones along
with how many health care professionals use them, shows that we are living in an era where
infcrmation technology is part of our everyday life. Imagine a life without television, a laptop
(on which we write this!) and the internet. It is in this environment that technology within
healthcare will thrive.
There are many types of technology found within the intensive care providing clinical data. One
of the ways this is all coordinated is through the clinical information system (CIS), or in the
Leyton Orient football parlance the central midfield controller'. This records all the observations
that are made on the patients, the blood results and investigations along with the clinical history.
The CIS in addition has reporting functionality to assess the costs, quality and outcomes of the
intensive care unit. The necessity or mandate for developing the CIS comes from the reports of
the Institute of Medicine (IOM) - part of the National Academy of Sciences. The report from the
IOM "To err is human' suggested that up to 98,000 people die in hospitals in the USA annually
due to preventable medical mistakes, more than are killed from road traffic accidents, breast
cancer and AIDS (12).
One of the commonest errors is medication mistakes (13].
The IOM strategy for improvement in this form of error is by implementing the computerised
order entry, standardising processes for medication doses, dose timing and dose scales in a given
patient unit and ensuring reporting of errors. Only once the errors and error rates are known can
these errors be managed.
Medical informatics provides an answer to ensuring patient safety by ensuring standardisation of
patient care and processes, allowing physician order entry systems with online clinical decision
support. The electronic patient record should be seen as the 'enabler' of the process of patient
management before during and after critical care. It will be of particular importance to ensure
that the process of rehabilitation starts and is documented on the ICU and is continued
throughout the post intensive care period. Rehabilitation for the critically ill patient has been
mandated by NICE [14].
Procurement of CIS technology
A project initiation document is probably the most crucial part of the project in the procurement
and development of a CIS. This is the link between what the end user wants and what the
technology is able to provide. Within the intensive care there are many healthcare professional
who work together to deliver patient care. It is important to ensure the end user has what they
want and time inust be spent developing a project initiation document and the formation from
this of a scope document which will help the end user to understand exactly what to expect
during the course of the project.
Project initiation document
This process should involve all members of the healthcare team (medical, nursing, technicians,
physiotherapists, dieticians and pharmacist), the information technology department and support
services (radiology, pathology and microbiology). The final and probably most important
member is a previous ICU patient. The group then need to sit down and plot the patient's journey
through the ICU and what information is required from investigations, clinical information and
monitoring data. Pharmacy data and prescribing along with order communications such as
radiology and pathology need recording. Various different clinic scenarios need to be thought
about to ensure the different clinical scenarios are all accounted for in the CIS. This includes
factors such as paediatric patients, surgical and medical and ICU compared to HDU patients.
Once this statement of need is finalised this has to be placed in the project initiation document
which is then sent out to tender to the different companies that can provide a CIS for a critical
care unit. care unit.
Statement of need (SON)
Determining which company is granted the contract will depend on which company can match
the statement of need. This will relate to cost, matching of the statement of need and the time
scale over which the project can be delivered.
The statement of need and the project initiation document are then merged to form a "project
scope”, which will help the end user to understand exactly what to expect during the course of
the project.
Potential pitfalls of the project scope
Like any binding contract between two individuals, it is only with time and what the user sees as
the end product that potential problems arise. Thcse of us who have been involved in house or
hospital building projects provide good experience of such a scenario. As a project develops
there might be new technologies or treatments available. Project development might be delayed
so that the money originally put aside by the contractor does not cover the task that it was
originally set aside for. You may argue that this is the contractor's problem, but if the vendor has
caused delays by for example not giving the contractor access to the clinical area, problems
might occur.
Project development
Once the contract has been tendered and awarded, the project is moved from the sales team of
the company to the development side. Most companies are set up so that these two departments
are separate and as long as there is not too much deviation from the scope, a good relationship
starts to develop between the company developers and the ICU project administrator team. A
CIS project takes approximately 6 to 12 months to set up. The hardware for the servers needs to
be set up first and then the software is loaded onto these, which have to be mirrored and backed
up in a fire and flood proof place. Training for configuration of the system them starts. The
configuration is based on the original meeting of all the hospital employees who developed the
project initiation document.
Testing of data quality
All configurations and interfacing of devices such as blood gas results and monitoring data
which come across automatically to the CIS have to be tested. A parallel testing CIS system
which mirrors the CIS which is ultimately used to enter patient data on must exist.
To ensure the data quality data, you must have minimum and maximum bound for parameters.
This means that for example the range for something like central venous pressure should be -5 to
30 mmHg. If a value such as 250 mmHg is entered it will be rejected. In addition, normal
reference ranges can be entered and if these are exceeded then the data item can appear in a
different colour. Mandatory data fields can be entered to ensure data fields get populated. By
doing this data which is used to calculate scores or predictive tools becomes valid. These can
then be used in alerts to the clinical user that there is a potential adverse clinical situation
developing with the patient which needs to be addressed sooner rather than later.
Test data must include all interfaced devices (monitoring, laboratory, ventilator and infusion
pump information), prescribed drugs, fluid balance calculations and audit data such as APACHE
and SAPS scores. This involves entering actual patient data into the test CIS and then getting a
third party to check that these data are correct.
Staff training
Once the system has been developed and tested then the end users have to have training in the
use of the system. If the processes of care are not very different from those used before, then essentially
the training is mainly getting the staff used to a new CIS. Processes of care can be altered with the use of
information technology to benefit the patient and staff. Data can automatically be acquired from the
bedside monitors and ventilators. These data have to be verified by the user but will in addition come
under the scrutiny of the data parameters which were set in the original system development.
Advisories and calculations
Information systems have the ability to carry out calculations on data which is entered into the system. An
example would be if the patient's blood sugar is greater than 10 mmol/l then an advisory message
appears in the bedside computer screen. This message then has to be acknowledged and this leaves an
audit trail. Calculations can be configured to ensure for example that if the cerebral perfusion pressure is
below a certain value then an advisory is triggered.
Audit and reporting systems
Once the system “goes live“ following system testing and staff training then live patient data can start being collected.
This is one of the paradoxical issues with report and audit writing, which is that only when you have data can these
reports be looked at.
Quality of care reports
Technology within the CIS ensures that data are entered in mandatory fields and the quality is
good by putting bounds on these data. In addition with the use of advisories and calculations
data quality can be further enhanced and known clinical care which benefits patients such as
keeping the blood sugar below 10 mmol/l is achieved. With data it is hugely important to ensure
it is entered correctly in the first place, rather than having to correct it at a later date. After 48
hours it is sensible to ensure that certain data types such as vitals and ventilator parameters
cannot be altered. This stops the data from being corrupted by unscrupulous investigators. In addition,
most individuals are noi likely to remember data figures after 48 hours.
Reports can be automatically generated at monthly intervals to ensure that the ICU is practising to a
certain standard. Examples of this include factors such as care bundle adherents for sepsis, veno
thromboembolic prophylaxis and safe ventilation with low tidal volumes. In addition – as ICU is best
described as a complex interaction of a multitude of factors in a process of care which is delivered to the
patient - performance of this systems can be evaluated using tools such as cumulative summation of
mortality (CSUM) and standardised mortality ratios (SMR) against known outcomes.
Using technology to alert processes of care
Technology with the CIS ensures that data have to be entered; otherwise the user cannot progress to the
next screen or entry. Similarly advisories can appear which instruct staff to carry out certain tasks.
Provided the clinical user sees these advisories as aid memoirs rather than dictatorial instructions,
processes of care can be altered. Then with the use of the tools such as the CSUM and SMR mentioned
early performance of the ICU can be monitored.
Summary

1. Health care is expensive and budgets are notlimitless.

2. Traditional processes of care may need to be chalenged because they are not cost-effective.

3. Introducing a new process will only be effective if introduced carefully and constantly evaluated.

4. New processes of care are often associated with new technologies that would have undergone a
rigorous period of evaluation to ensure they do the job they are designed to do.

5. New technologies such as a CIS may have an important role to play in improving patient safety but
often require patient processes to be re-thought

The contribution of technology assessment

The authors
Carl Waldmann, MD, MA, MB, Bchir, FRCA, EDIC Max Jonas, MB, BC, FRCA? Liza Keating, MB, ChB, MSc, MRCP,
FCEM, DICM lan James Rechner, MD, BSC MB, BS, FRCA?
JICU | Royal Berkshire Hospital, Reading, UK 21CU Southampton University Hospital, UK
Address for correspondence
Carl Waldmann Intensive Care Unit Royal Berkshire NHS Foundation Trust London Road Reading RG5 4EU, United
Kindom E-mail: cswald@aol.com

References
1. Hope T, Savulescu) and Hendrick J. Medical Ethics and Law. The Core Curriculum. Oxford: Churchill
Livingston Elsevier 2008.

2. Stricker K, Rothen HU and Takala J. Resource use in the ICU: short- vs. long-term patients. Acta
Anaesthesiol Scand 2003;47:508-15.

3. Terblanche M, Adhikari NK. The evolution of intensive care unit performance. J Crit Care 2006;21:19-22. 4.
Suter, P. Some ICUs save more lives than others: we need to know why! Intensive Care Med 2005;31:301-
302

4. Sachdeva R. Measuring the impact of new technology: An outcomes-based approach. Crit Care Med
2001;29:N190-N195.

5. Hantschel D, Fassl J, Scholz M, Sommer M, Funkat A,

.