Clinical Validation of the Spirohome Clinic Ultrasonic Spirometer in Child and Adolescent Patients

Introduction

Lung diseases are among the most common non-communicable medical conditions in the world. A study conducted between 1990 and 2017 showed that chronic respiratory diseases increased by 39.8%, and close to 545 million people had diseases in the world.¹ Spirometry remains the “gold standard” for assessing obstructive and/or restrictive airway diseases^2–4 in both adults and children.⁵ Spirometers are reliable and non-invasive tools that can characterize flow in both the large and small airways. They can be categorized technologically based on the type of flow sensor they contain. Although pneumotachometer-based⁶ and turbine-based⁷ spirometers are still used in clinical spirometry, current state-of-the-art spirometers utilize ultrasound-based flow sensors. Ultrasonic spirometers have become renowned for their accuracy, and their sensitivity to lower flow rates. Unlike turbine-based spirometers, ultrasonic spirometers have no moving parts and are thus low-maintenance devices that do not require routine re-calibration. Turbine-based spirometers also have difficulty reaching low flow sensitivity which is limiting in conditions like COPD where patients have extended periods of low flow exhalation in spirometry tests.

Spirometers are also classified by whether they are conventional desktop systems or hand-held devices. Hand-held spirometers offer portability, are generally space-saving and often less expensive than conventional desktop spirometers. Today, hand-held spirometers are designed with features like on-screen guidance, real-time feedback and session grading which aim to reduce the need for expert intervention and boost user confidence during spirometry sessions. With the rapid growth of digital health technologies and remote care through telehealth platforms, integratable patient-centric smart spirometers have become increasingly popular in the pulmonary health field. These modern spirometers are not only proving to help solve the underutilization problem seen with traditional spirometers in the clinic, but they are moving lung function testing to home settings where patients are liberated from the burden and costs of travel whilst still being able to provide lung function data to their health-care professionals for monitoring. Such technologies have become particularly relevant in recent times as the COVID-19 pandemic has led to the closures of pulmonary function laboratories around the world to reduce patient and personnel exposure to the virus. However, hand-held devices are still establishing their reliability in clinical spirometry as full-scale spirometers.

Due to their relative newness, validation of hand-held spirometers in comprehensive lung function testing remains a point of interest for the respiratory health community. There are several examples in literature comparing hand-held and conventional spirometers^8–15 and with each other.^16–18 The advent of spirometry platforms that combine portable spirometers with applications running on smart devices raises further concerns such as those regarding the device’s processing power, interface components, robustness, and reproducibility of its measurements.¹⁴ Whilst portable spirometry platforms aim to and seemingly achieve making spirometry more readily accessible and appealing to professionals and patients, it is still critical that they are rigorously validated through clinical validation studies.

This work discloses a two-phase validation study of the Spirohome Clinic ultrasonic spirometer using the EasyOne Air device as a contrast. First, devices were tested in the laboratory to assess compliance to the accuracy and performance criteria detailed in relevant ATS/ERS guidelines and the ISO 26782 standard. Then, a group of pediatric patients with pulmonary symptoms from Pediatric Allergy and Asthma at Hacettepe University Hospital who met the inclusion criteria performed pulmonary function tests with both devices during clinical visits. The accuracy and repeatability of certain spirometric parameters from both devices were then statistically compared.

Materials and Methods

Lung Simulator Analysis

The accuracy of both spirometers was assessed with a lung simulator as suggested in the ATS guideline and described in the BS EN ISO 26782 document. The spirometric parameters FEV₁ (L), FVC (L), and PEF (L/s) were measured thrice each with the Spirohome Clinic and EasyOne under a set of 13 different waveforms created by the PWG-33 lung simulator (Piston Medical, Budapest, Hungary). The repeatability (L) and accuracy (L) of the spirometric parameters measured by each device were calculated. Repeatability was calculated as the difference between the best and second-best measurements, and accuracy was the difference between the mean of three measurements and the target value (the target value is calculated by the lung simulator device).

Clinical Pulmonary Function Tests

The pulmonary function tests of participants were conducted with both the Spirohome Clinic and the EasyOne Air spirometers. EasyOne Air was chosen as a reference device due to its similarity in flow sensor technology and its history of clinical application.^19,20 Both spirometers were factory-calibrated, however, performance accuracy was also confirmed with laboratory lung simulator tests before participant use. Participants were required to perform at least 3 spirometry trials (maximum limit of 8) with each device with a 30-minute quiescent period between each instrument. Of the 48 participants that presented to the clinic, the first 24 used the Spirohome Clinic first and the EasyOne Air second, and this order was reversed for the next 24 patients to prevent potential bias. All measurements were guided in a standardized way by the same trained personnel in accordance with current ATS/ERS guidelines.

FEV₁, FVC, FEV₆, FEF_25-75, and PEF values obtained from each spirometer was recorded digitally for subsequent statistical analysis. The acceptability of FEV₁ and FVC measurements was assessed according to the ATS guideline grading system.²¹ To ensure consistency and validity of data, the results ranked as either a grade A or B for FEV₁ and FVC measurements were included in the study.

Study Design and Population

The present work presents a cross-sectional, prospective, single-center comparative study. The intended patient sample size was calculated with a priori power analysis performed based on FEV₁ and FVC measurements. Based on the power analysis and with reference to methods reported in a similar study,¹⁵ a total of 48 participants were required for detecting a mean difference of 0.25 L between the spirometers to achieve 80% statistical power at α=0.05, when the standard deviation is 0.6 L for FEV1 and FVC.

Healthy controls and individuals with asthma or allergic rhinitis presenting with a cough that were between the ages of 6–18 (pediatric patients between 6 and 11 years of age and adolescent patients between 12 and 18 years of age) and who previously demonstrated ability in performing spirometry tests were included in the study. Patients were selected from those presenting to the Hacettepe University Pediatric Allergy and Asthma outpatient clinic with asthma, allergic rhinitis, or another condition and who volunteered to take part in the study.

Participants (patients and parents/legal guardians) were provided with information regarding the purpose, content and significance of the study and written consent forms were collected (from both patients and parents/legal guardians) prior to enrolment. Participant exclusion criteria were as follows: history of syncope due to difficult expiration or cough, having cardiac arrhythmia, history of brain, thoracic or abdominal surgery in the last four weeks, eye, sinus, or middle ear surgery in the last week, presence of pneumothorax, uncompensated heart failure, cerebral aneurysms, hemoptysis, significant secretion, oral lesion, oral bleeding, nausea, vomiting, unconsciousness, dementia, muscle disease or bone disease and respiratory or systemic infections.

Patients meeting inclusion criteria and presenting to the clinic within a specified date range were evaluated by an experienced pediatrician. Participant eligibility was also dependent on their capacity to comply with the requirements of performing a correct pulmonary function test. Only participants who consented to participate were included in the study and only a couple of patients declined participation in the study due to time constraints. The study commenced on 12 August 2020 and concluded on 18 December 2020.

Ethics Approval and Consent to Participate

Ethics approval to conduct this study was obtained from the Ethics Board of Approval at Hacettepe University (Decision Number: 2020/06-39 (KA-20037) and the Turkish Ministry of Health (Decision Number: 68869993–511.06-E.148267)). Informed written consent was also obtained from each participant before inclusion in the study.

Spirometers

The two spirometers comparatively assessed in this study were the Spirohome Clinic (Inofab Health, Ankara, Turkey) and EasyOne Air (ndd Medizintechnik AG, Zurich, Switzerland). The Spirohome Clinic is a CE-certified ultra-portable ultrasonic spirometer that pairs via Bluetooth^® to smart tablets and desktop computers running the accompanying application. The following indices obtained from spirometry are stored through the application: forced vital capacity (FVC), forced expiratory volume in 1 s (FEV₁), FEV₁/FVC ratio, forced expiratory volume in 6 s (FEV₆), peak expiratory flow (PEF), forced expiratory flow at 25–75% (FEF_25-75)of forced vital capacity, back extrapolated volume (BEV) and Forced expiratory time (FET). The Spirohome Clinic also integrates to an accompanying cloud platform and local electronic health record systems to facilitate telehealth-based pulmonary care and scaled respiratory research. The EasyOne Air is a CE-certified and FDA-cleared ultrasonic spirometer that has an on-screen display. It can be used as a portable or as a PC spirometer. It produces the same set of spirometric data as the Spirohome Clinic. Table 1 shows key differences and similarities between the SpiroHome Clinic and the Easyone Air features.

Table 1 SpiroHome Clinic and EasyOne Air Features

Demographics and Anthropometrics

Participants assessed and approved by the pediatrician for inclusion in the study were taken to a designated room to perform the lung function tests. The age and gender of all participants were noted, and their weight and height were measured. No other demographic or anthropometric data was collected.

Statistical Analysis

The mean±SD and median (Interquartile Range) repeatability and accuracy values for spirometric data recorded over the 13 different waveforms produced by the lung simulator are given in both liters and as a percent ratio. The repeatability and accuracy of spirometric data were statistically compared between the Spirohome Clinic and reference device.

Correlation between spirometers was also assessed with correlation plots and the Pearson’s correlation coefficient. A simple linear regression line was fitted to facilitate comparisons. Bland-Altman plots and paired t-tests were applied to depict bias between mean differences for spirometric parameters recorded by each device. Finally, Kruskal Wallis and post-hoc Dunn’s tests were used to compare error rates between spirometers. The results were evaluated within a 95% confidence interval and at a level of p<0.05 significance. Spirometric data were also analyzed in age and gender subgroups. All statistical analyses were performed using R version 4.0.3 (R Development Core Team, 2010; www.R-project.org) and run in RStudio (www.rstudio.com). Figures were generated by using the ggplot2 package in R.

Results

Lung Simulator Analysis Results

The mean and median accuracy and repeatability values for FEV₁, FVC and PEF measurements from the 13 different test waveforms collected by each spirometer are given in Table 2 (please see the Supplementary Information for details). Values are presented both in liters (L) or liter per second (L/s) and percent (%). Both spirometers met ATS limits for measurement accuracy in all 13 ISO 26782 waveforms tested. There was a statistically significant (p<0.05) difference between spirometers with respect to FVC accuracy both in units of liters and percentage. The SpiroHome Clinic mean FVC accuracy value (0.03 ± 0.03 L) remains at the upper limit for accuracy as defined by the relevant standard, and the EasyOne Air mean FVC accuracy value (−0.02 ± 0.04 L) remains at the lower limit for accuracy.

Table 2 Lung Simulator Analysis Results. Differences in Repeatability and Accuracy Measures Between Spirohome and EasyOne Air

Study Population and Pulmonary Function Results

The study group consisted of 48 persons between the ages of 6–18 years. The mean age of the study population was 12.7±3.5 years. Thirteen (27.1%) of the participants were female. Other demographics and pulmonary function test results for both spirometers are presented in Table 3. Results were also presented separately for two age groups, children (6–11 years old) and adolescents (12–18 years old).

Table 3 Study Population Characteristics

Table 4 Mean Differences (with 95% Confidence Intervals) in FEV1 (L) and FVC (L) Measurements Between Spirohome and EasyOne Across Gender and Age Subgroups

FEV₆, FEF_25-75, and PEF measurements obtained with the Spirohome spirometer were higher than that of the EasyOne spirometer (Table 5). Significant differences in FEV₆ and PEF measurements between the two spirometers were observed in males and 12–18 years old volunteers. There was a larger difference in mean FEF_25-75 volume recorded by each spirometer in patients over the age of 12 and it was also larger in males compared to females.

Table 5 Mean Differences (with 95% Confidence Intervals) in Measurements Between Spirohome and EasyOne Air Across Gender and Age Groups

Correlation analysis of FEV₁ and FVC values (n = 48) obtained from two spirometers is presented in Figure 1. For both values, the Pearson correlation coefficient was 0.99, and p values were <0.001. The simple linear regression line was fitted with r = 0.99, p<0.001 for FEV₁ Spirohome and FVC Spirohome (Figure 1). The regression equations show that the Spirohome Clinic measured higher FEV₁ and FVC values than the EasyOne Air.

Figure 1 FEV1 and FVC correlation analysis between spirometers.

In Figure 2, the agreement between the measurements (FEV₁ and FVC) is shown using Bland-Altman plots. Figure 2A presents Bland-Altman plots for the differences between FEV₁ and FVC in all volunteers. Figure 2B shows Bland-Altman plots for differences between FEV₁ and FVC based on gender subgroups. Figure 2C shows Bland-Altman plots for differences between FEV₁ and FVC based on age subgroups. According to the Bland-Altman plots, the majority of the results were within the limits of agreement.

Figure 2 Bland-Altman plot showing agreement between FEV1 and FVC measurements. (A) Differences between FEV1 (L) and FVC (L) for all patients. (B) Differences between FEV1 (L) and FVC (L) based on gender. (C) Differences between FEV1 (L) and FVC (L) based on age groups.

Discussion

The Spirohome Clinic ultrasonic spirometer (Inofab Health, Ankara, Turkey) is a hand-held spirometer that connects to a smart device application via Bluetooth^® connection. The findings of this study support the accuracy and repeatability of measurements taken with the Spirohome Clinic spirometer. The results show a high agreement and concordance between the Spirohome clinic and the reference spirometer when used in a pediatric population which includes patients with different respiratory conditions and also healthy controls.

Lung simulator analysis is currently the only method of objectively quantifying the accuracy and repeatability of spirometric measurements collected by spirometers. The accuracy and repeatability of FEV₁, FVC and PEF values recorded by the Spirohome Clinic and EasyOne Air were examined with a lung simulator according to the methods and waveforms given in the relevant ISO document. The Standardization of Spirometry 2019 Update published by the ATS²¹ states that the maximum allowed error in volume measurements in spirometers is ±3.0% or 0.05 L (whichever is greater). Our lung simulator analyses show that the mean repeatability and accuracy errors in volume readings (FEV₁ and FVC) were less than ±3.0% or 0.05 L in both spirometers. The only statistically significant (p<0.05) difference between spirometers was found in the accuracy of the FVC, although the actual error rates were fairly small for both spirometers. We attributed the statistically significant difference to the clustering of results on different sides of zero and assume it to be a Type 1 error. Importantly, as both devices are within the accuracy limits set by the ATS, they are both deemed clinically accurate and therefore safe and effective to use in spirometry measurements. Lung simulator analysis results verified the reliability of both spirometers for further use with clinical subjects in phase two of the study.

The clinical phase of the study involved lung function testing with both devices in 48 volunteers under the age of 18. The mean±SD of a range of clinically important parameters were included in the study for a more systematic comparison of the study and reference device. Among FEV₁, FVC, FEV₆, FEF_25-75 and PEF mean values for the whole study group, only the mean difference in FEV₁ and FVC values between spirometers was found to be statistically significant (p<0.05). Overall, the mean difference between spirometers was 65 mL for FEV₁ and 59 mL for FVC. This is only a 2.5% difference for FEV₁ and 1.8% for FVC. Although both spirometers were compliant to the ATS limits for accuracy (demonstrated in the lung simulator analysis), the statistically significant difference between the measurements for FEV₁ and FVC can be attributed to the Spirohome Clinic consistently measuring slightly higher values than the EasyOne Air (seen in the lung simulator results and regression analysis).

The mean differences of FEV₁ and FVC between spirometers were statistically significant (p<0.05) for the whole patient group, and particularly in males aged 12–18 (p<0.05). Statistical significance was found in the mean differences of FEV₆ (p<0.01) and PEF (p<0.001) in the whole patient group and also in the male subset (p < 0.001 for both FEV₆ and PEF). Results suggest that there is influence of age and gender on FEV₁ and FVC mean differences between spirometers. Also, the correlation between age and height also suggests that larger lung volumes in patients over the age of 12 may have attributed to the greater mean differences in several parameters recorded by each spirometer. Other demographics or anthropometrics not accounted for in this work may also have played a role in these findings.

Correlation plots for FEV₁ and FVC values showed a significant (p<0.001 and Pearson's correlation coefficient of 0.99) agreement between spirometers for these values. The reproducibility of measurements with the Spirohome Clinic and EasyOne Air also highlighted in the Bland-Altman plots for FEV₁ and FVC which show that most measurements are located within the lower and upper agreement limits, revealing no systematic difference between the two devices. Bland-Altman representation of age and gender subsets also reveal no systematic difference between spirometers based on these variables.

The present study demonstrates the reliability of the Spirohome Clinic in pulmonary lung function testing of respiratory patients. Fair agreement to the EasyOne Air device in both benchtop validations according to internationally accepted standards, and furthermore in clinical results strengthens the case for adopting hand-held spirometers in clinical practice. These devices have proven to be effective in pediatric patients who often find performing breathing maneuvers for spirometry a challenge, even under the guidance of a health-care professional.

Future studies building on this work should address some of the limitations of the current work. A larger data set will increase the accuracy of sample means. Similarly, a longitudinal study involving a greater number of devices will be important for examining interdevice reproducibility and holds relevance for clinical and epidemiological research.²² Although not covered in this work, the bias of chronic respiratory disease types on spirometer performance can also be explored. It will also be valuable to confirm the repeatability of the results of this study in an adult patient population.

Conclusions

In conclusion, we evaluated two portable ultrasonic spirometers currently available on the market with two complementary studies. Our findings show that the Spirohome Clinic results in statistically significant but not clinically significant higher measurements than EasyOne Air, even though both meet the ATS/ERS criteria. The precision of the Spirohome Clinic ultrasonic spirometer in clinical application is supported through the findings of this study. Ultrasonic, portable, durable, and user-friendly spirometers are critical tools for respiratory clinics and may continue to contribute to respiratory health by lung function measurements from the clinic to the home through teleMedicine advances.