Validation of Omron HBP-1100-E Professional Blood Pressure Measuring Device According to the American Association for the Advancement of Medical Instrumentation Protocol: The PERSIAN Guilan Cohort Study (PGCS)

Introduction

Blood pressure measurement accuracy is critical to the diagnosis and management of hypertension.¹ The decrease in the traditional clinical use of mercury Sphygmomanometers with the increasing availability of automated blood pressure (BP) monitors increases the concern about the validation of automated devices.² Many population-based survey protocols use automatic digital devices for measuring BP since they require no strict training and have no observer error.^3–6

Medical device agencies, challenged with recommending suitable BP monitors as a replacement for mercury sphygmomanometers, are progressively relying on results of technology assessments in the form of clinical validations against published US, European or British protocols.^7–9 Validation compared to these protocols provides evidence of the degree of agreement between clinical readings from the mercury sphygmomanometer that extensively used as the gold standard for blood pressure measurement and those on the automated devices.¹⁰

During the last 10 years, various automated devices for BP monitoring have been developed.^11,12 Some BP monitoring devices have been validated according to specific protocols, such as the Association for the Advancement of Medical Instrumentation (AAMI) protocol.^2,13,14 A BP-measuring device that can fulfill the AAMI protocol criteria would be desirable for either research or clinic.¹⁵

The Omron HBP-1100 devices, an automated professional BP monitor (Omron HBP-1100-E; OMRON Healthcare, Hoofddorp, the Netherlands) that is widely used in the north of Iran,¹⁶ has never been validated previously according to the Association for the Advancement of Medical Instrumentation (AAMI) Collaboration Statement. The Latest Standard for validation of BP monitoring devise accuracy is the AAMI protocol.²

Therefore, this study aimed to evaluate the accuracy of Omron HBP-1100 devise based on the AAMI protocol at the beginning of the PERSIAN Guilan cohort study (PGCS) a prospective, population-based cohort study in the north of Iran, to determine whether it can be used in this cohort.

Materials and Methods

Device

The Omron HBP-1100-E (OMRON Healthcare, Hoofddorp, Netherlands) is an automatic digital device for professional BP measurement at the upper arm level.

It is designed to estimate BP values range of 0–299 mmHg and heart rate range of 40–180 beats/min. Heart rate, diastolic BP, and systolic BP are displayed on a liquid crystal digital (LCD) screen. The inflation is by an automatic electric pump, and the pressure release valve controls rapid automatic deflation. Standard cuff size (22–32 cm) was used in the current study.

Individuals

This is a sub-study of a population-based cohort study in the north of Iran (PERSIAN Guilan cohort study) that is a part of the Prospective Epidemiological Research Studies in Iran (PERSIAN) study,^17,18 to determine whether it can be used in this cohort. A total of 85 participants (the optimal sample size for a validation study base on AAMI protocol) aged 35 to 70 years were recruited according to the AAMI protocol. Selection of subjects was made according to the recommendations of the AAMI protocol, include at least 30% males and 5% of the reference systolic BP readings ≤100 mmHg or ≥160 mm Hg, and 20% with ≥140 mmHg and 5% of reference diastolic BP readings ≤60 mmHg or ≥100 mmHg, and 20% with ≥85 mm Hg.² The first 85 individuals to enter the PERSIAN Guilan cohort and met recommend criteria were included in the study (Table 1).

Table 1 Error Category Criteria and Mean Differences for Test Device and Observer Readings by Age, Sex and First Measurement Order

Ethical Consideration

The study protocol was approved by the Local research ethics committee of Guilan University of Medical Sciences (IR.GUMS.REC.1397.347). All subjects included in the study agreed to participate and signed written informed consent for study participation.

Procedure and Measurements

Two clinical technicians recruited patients and acted as observers, and a medically qualified expert provided clinical oversight and acted as a supervisor. All those involved as observers were trained in accurate BP measurement.

The participants were seated in a quiet room with back supported and legs uncrossed, and after a 10 mins rest period, BP measurements started. All measurements were made on the same arm at the heart level, and the cuff type was adapted to the arm circumference.

Two observers, who were trained in accurate BP measurement, measured the BP with calibrated mercury sphygmomanometer (reference devices), and one supervisor measured the BP with the test device. The agreement between the two observers was checked all over the assessment period in order to maintain the quality of the measurements. If observers’ disagreement > 4 mm Hg in systolic or diastolic BP, measurements have to be excluded, then another pair of BP readings (test and reference) is taken.

The two observers were blinded to each other’s results. Ten min interval was given between separating consecutive BP measurements to minimize venous congestion. Measurements were conducted in the following sequence:

BP1 Mercury (Observers 1 and 2)

BP2 Device (supervisor with test device)

BP3 Mercury (Observers 1 and 2)

BP4 Device (supervisor with test device)

BP5 Mercury (Observers 1 and 2)

BP6 Device (supervisor with test device)

BP7 Mercury (Observers 1 and 2)

The two observers’ readings were averaged before comparison with the reading by the automated device. Each of the test device, BP reading was compared against the average of the previous and next reference BP measurements (eg, BP2 Device versus the average of BP1–BP3 Mercury, BP4 Device versus the average of BP3–BP5 Mercury, BP6 Device versus the average of BP5–BP6 Mercury). Therefore, 250 paired readings were provided.

Data Analysis

According to the AAMI protocol, the measurements of 2 observers were averaged and then subtracted from the device reading. These BP differences were categorized into three error categories (within 5, 10, and 15 mmHg) for systolic and diastolic BP separately to determine the percentage with values ≤ 5, ≤10, and ≤15 mmHg. The required pass criteria for each category presented in Table 1. An additional validation criterion was requiring a mean device–observer difference (standard deviation) of ≤ 5 ± (8) mmHg.

The Bland–Altman plots were created to show the device–observer differences versus average observer and device values for right and left SBP and DBP separately. The statistics were analyzed using SPSS version 17.0 (SPSS Inc., Chicago, IL, USA).

Result

A total of 85 participants (40 males, 47.1%) were enrolled in the study, and there were 250 valid paired comparisons. The average age was 51.1 ± 9.3 years. The two observer’s readings difference was 1.3 ± 11.1 and – 0.5 ± 10.5 mmHg for SBP and DBP, respectively, and intra-class correlation coefficient (ICC) of two observer’s measurements for SBP and DBP was 0.81 and 0.45, respectively.

The validation criteria and the validation analysis results are presented in Table 1. According to the AAMI protocol criteria, the device failed to pass none of the criteria for both systolic and diastolic BP. The percentage of the device-observer differences falling in the specified error categories (within 5, 10, and 15 mmHg) is given for SBP and DBP (Table 1). The mean BP measurements were 130.5 ± 21.7/82.06 ± 13.5 mmHg with the Omron 1300-E device and 126.5 ± 19.3/80.9 ± 10.9 mmHg with the standard mercury sphygmomanometer (p-value = 0.001). The mean and standard deviation of the difference between the device and the reference method was 8.0 ± 13.1 and 2.2 ± 11.3 for SBP and DBP, respectively. The accuracy of the device was evaluated separately for the age category, sex, and first measurement order. Mean device–observer differences for systolic and diastolic BP were not significantly different in ≤ 50 vs >50 years (Table 1). Mean device–observer differences for systolic BP were significantly higher in male vs female (p-value = 0.03), but mean device–observer differences for diastolic BP were not significantly different (Table 1). Overall, the data showed that the validation criterion was not fulfilled in all subgroups.

The Bland–Altman plots for diastolic and systolic pressures by age category and sex group showed in Supplementary Files 1–8, respectively. These graphs do not show any systematic variation in the error, and there are no trends in the data.

Discussion

The results of this validation study revealed that the Omron HBP-1100-E device failed to fulfill the validation criteria of the AAMI protocol² for SBP and DBP. Among the Iranian adult population and cannot, therefore, be recommended for clinical use in this population.

Overall, the device tended to overestimate diastolic and systolic BP. This finding was In line with other studies.^9,19 The device accuracy in measuring diastolic BP did not seem to be affected by age, sex, and order of measurements. Inconsistent with this finding, another study was found No correlation between the accuracy of the device in measuring BP and age, sex.²⁰ Although the device measuring systolic BP was more accurate in female and mercury first measurement groups, these measurements were not within the accepted levels of the validation criteria.

Other studs

This study finding shows that with the Omron 1100E, the device–observer limits of agreement widened with SBP rather than with DBP. The mean difference of diastolic BP was 2.2 and achieved the required value (5 mmHg), but the standard deviation was wider than the AAMI criteria (8 mmHg). This result revealed in subgroup analyses, too.

Limitations and Recommendations for Future Research

It must be considered that the interclass correlation coefficient (ICC) of two observer’s measurements for DBP was low (0.45), and these variations may be related to human errors.²¹

One of the limitations to the current study is that the results are based on only one device, and the accuracy evaluation was done in only one center; however, the international protocol in order to enhance the heterogeneity of the study population does not specify the number of study sites or the number of devices to be tested recommended.^22,23 The AAMI protocol does not specify the number of devices required for validation but recommends more than one study site,²⁴ whiles the British Hypertension Society (BHS) protocol recommends using more than one of the models being tested to give consistent measurements, but does not specify any particular number of sites.^9,25 If considerable differences between instruments of the same device occur, further device validation is not recommended. By using a more accurate technique for automatic Systolic blood pressure measurement, the optimal blood pressure target can be achieved with a lower risk for hypotension and its adverse events.²⁶

Conclusion

Omron HBP-1100-E professional blood pressure monitor is not desirable for measuring the BP for Iranian adults as it overestimates blood pressure in this population.