Satisfaction of telehealth implementation in a pediatric feeding clinic

1.   Introduction

Telehealth has been implemented in various forms since the 1960's, including telephone, internet, and video conferencing offered both synchronously and asynchronously. The COVID-19 pandemic necessitated a swift and sweeping adoption of this modality due to the need for social distancing [1]–[3]. Telehealth has become an increasingly viable, and at times preferred, option for families [1],[4],[5]. Telehealth provides an increased accessibility to healthcare services [6], reduced geographical barriers [7], a decreased visit length [8],[9], and a decreased cost [10],[11], all while maintaining patient satisfaction in care [9]. Although there has been a growing body of research on the feasibility and benefits of telehealth in a variety of settings, there has been limited research on the use of telehealth in multidisciplinary settings [12],[13], and even fewer studies specifically investigating the use of telehealth within pediatric feeding disorder populations.

Pediatric feeding disorders are increasingly common, impacting up to 1 in 24 children [14], and the gold standard of treatment includes multidisciplinary care [15],[16]. Most of the research on the use of telehealth in these populations has focused on the outcomes of telehealth follow-up visits following an initial in-person assessment by the multidisciplinary medical team [7],[10],[17],[18].

This study aims to address a gap in the literature by assessing both patient and provider satisfaction with multidisciplinary team visits conducted solely via telehealth in an outpatient clinic for pediatric feeding disorders. Additionally, the current study assessed the visit completion rates within the telehealth framework.

2.   Materials and methods

2.1.   Participants and study design

All participants were caregivers of patients seen by the Growth and Nutrition Program, which is a multidisciplinary pediatric feeding disorders clinic for children with or at risk for malnutrition up to the age of 7-years-old within an urban children's hospital. The caregivers of the patients were included in the study if the patient had at least one telehealth visit completed between April 1, 2020, and June 30, 2020, and the primary language spoken at home was English. There were approximately 300 unique patients seen by the multidisciplinary and behavioral medicine teams during the study period.

Providers who had seen at least one patient using telehealth platforms between April 1, 2020, and June 30, 2020, were asked to participate. Given the multidisciplinary nature of the program, a range of providers were invited, including the following: gastroenterology (MDs, NPs), nutrition (RDs), feeding (CCC-SLP, CLC), and behavioral medicine (Psychology PhD). There was a total of 18 providers who had seen patients utilizing telehealth platforms during the study period.

2.2.   Procedures

Patients identified as potential study participants were contacted by the Growth and Nutrition Program via email and provided with a link to an anonymous survey regarding patient satisfaction with telehealth using the REDCap (Research Electronic Data Capture) database [19],[20]. REDCap is a secure, web-based software platform designed to support data capture for research studies. If the link was not accessed within 7 days, the caregivers were emailed a reminder via REDCap with a prompt to complete the survey. The participants completed the survey between October 2020 and December 2020.

All providers who had participated in at least one telehealth visit during the study period were emailed by the Growth and Nutrition Program and provided a link to an anonymous survey regarding patient satisfaction using the REDCap database. Given the number of visits that the providers may have been a part of and to minimize the provider burden, providers completed the survey based on a general satisfaction over the course of the three-month study window. Study procedures were determined as Exempt by the Boston Children's Hospital Institutional Review Board (IRB-P00036158 and IRB-P00036926).

2.3.   Measures

Patient demographics and visit characteristics: As a part of the patient satisfaction survey, the patient demographics, including patient sex assigned at birth, ethnicity, race, and insurance type, were collected. In addition, the telehealth visit characteristics, including the platform, number of providers present, and disciplines of providers present, were requested for the most recent telehealth visit.

Visit modality: During the study period, telehealth visits were completed using a variety of technologies. At the time of the study period, the hospital had a hospital-based computer application (SBR Health) which caregivers could access through hospital portals. This did not have multi-user functionality and many visits were conducted using the Zoom application, in which caregivers would receive a link via email from administrative staff and could access using either a computer or a smartphone application. For families who had technical difficulties or could not access neither the hospital-based application nor the Zoom application, some visits were completed by video or voice call on a cellphone. All visits were conducted in-person during the comparison period.

Visit completion: Visit completion was assessed by counting the number of no-show visits, late cancelled visits, and completed visits. A visit was counted as a no-show if the parent did not cancel prior to the day before the visit and did not login to the visit. A late cancel visit was counted if the visit was cancelled within 24 hours of the visit, and completed visits were counted if the family logged into the visit and was completed (patient was present, parent was still available) for at least one provider. Total no show, cancel, and completion rates of virtual visits in the study period (April 1, 2020 to June 30, 2020) were compared to the no show, cancel, and completion rates of in-person visits during the comparison period (April 1, 2019 to June 30, 2019). The same time period was utilized to account for differences based on time of year.

Patient satisfaction: Telehealth satisfaction questions were adapted from the UCLA Pediatric Obesity Clinic Telehealth Satisfaction survey [21] and included the following assessments: caregiver satisfaction with visit, ease of experience, level of communication with providers, privacy concerns, and perception of appropriate quality of care. In addition, an open-ended question for additional feedback was included (Figure 1). The patient caregiver responses included 3 options (0 = no; 1 = yes, 2 = I don't know). Caregivers were able to provide additional feedback regarding their experience with telehealth visits in the form of open-ended questions at the end of the survey.

Provider visit characteristics and satisfaction: Providers were asked about which platform they utilized most often, how often they interacted with multiple providers, and which specialties were included in the telehealth visits. Provider satisfaction questions were adapted from the UCLA Pediatric Obesity Clinic Telehealth Satisfaction survey and included the following assessments: provider satisfaction with visit, effectiveness of visit, and perception of quality of care for the visit. Provider satisfaction questions utilized a 5-point Likert Response (1 = Strongly Agree... 5 = Strongly Disagree). Due to the small provider sample size, the provider demographics and identifiers were not collected to maintain confidentiality.

Figure 1.  Illustrative Quotes highlighting areas of satisfaction and need for improvement from caregivers of patients and providers.

DownLoad: Full-Size Img PowerPoint

2.4.   Data analysis

To assess patient and provider satisfaction with telehealth visits, we conducted descriptive analyses (frequencies, means, standard deviations) for the most variables of interest. Poisson comparisons were used to compare differences in the visit completion rates for the study and comparison periods. The frequencies of specific themed statements from the open-ended statements and illustrative quotes were included; however, the full mixed-methods analyses were outside of the scope of this study.

3.   Results

3.1.   Survey completion rates

A total of 269 emails were sent to potential caregiver participants; 40 caregivers started the survey and 36 surveys were completed. 18 caregivers completed the open-ended feedback questions at the end of the survey. 18 providers were invited to complete the survey; 13 providers completed the survey and 7 providers completed the open ended feedback portions.

3.2.   Demographics (Table 1)

Caregivers reported that patients were an average of 3.4 years-old (range 1 month to 8 years) and just over half were female (55%). The majority of patients with completed surveys were identified as White (80%), and most carried commercial insurance products (78%). Approximately a third of participants had state public insurance (Masshealth), either as a primary or a secondary insurance (33%). The provider demographics were not requested in order to maintain anonymity.

Table 1.  Demographics and visit rates.

Demographics	Age (mean; range)	3.4 (1-month corrected to 8 years)
	Patient gender (n, %)
	Male	18 (45)
	Female	22 (55)
	Race (n, %)
	White	31 (80)
	African American	4 (10)
	Asian	5 (13)
	Alaskan/Native American	2 (5)
	Other	2 (5)
	Hispanic	1 (3)
	Insurance (n, %)a
	Commercial	31 (78)
	State (Masshealth)	14 (33)
	Military	2 (5)
Visit completion rates
	April–June 2019	April–June 2020
Scheduled visits (N)	441	823**
Completed visits (N; %)	284 (64)	511** (62)
Late cancellations (N; %)	217 (49)	266 (52)
No-shows (N; %)	30 (7)	46 (6)
Patient visits
Visits per patient (mean; SD)	1.26 (0.61)	1.94 (1.68)

---

Note: ^aPatients may have both primary and secondary insurances. Comparison of visit rates between study period and comparison period. Using Poisson regressions, there were significantly more visits scheduled during telehealth period, though overall completion percentage was similar between the two time periods. *p < 0.05, **p < 0.01, ***p < 0.001.

 | Show Table

DownLoad: CSV

3.3.   Telehealth visit characteristics

Caregivers reported that their visits during this time period were conducted via several platforms, including the following: Zoom (51%), the hospital-based platform (SBR Health; 32%), telephone (3%), or a combination of platforms (14%). All providers reported that they most frequently utilized the zoom platform. Based on the caregiver report, there were most frequently 3 providers in the visit (46%), though there were also often 4 or more providers (27%). Similarly, the providers approximated that there were most frequently 3 providers present (M = 44% of time, SD = 17.3, range = 5–75% of time). The providers noted that they were less likely to have solo visits (M = 16% of visits, SD = 19.6, range = 2–75%) or with 5 or more providers (5% of visits, SD = 3.6, range = 0–10%). Based on the caregiver report, the most common providers that were in the visits were gastroenterology (89%), nutrition (89%), and feeding (SLP; 60%). Behavioral medicine was present for 38% of visits and social work for 14%. The providers noted that a GI provider was generally present for 80% of visits (M = 74%, SD 26.8, range = 20–100%) similar to Nutrition (M = 81%, SD = 28.2, range = 20–100%). The feeding team (SLPs) was present for approximately three quarters of visits (M = 75%, SD =27.1, range = 20–100%) and behavioral medicine for 37% (SD = 31.9, range = 0–100). Since the caregivers all spoke English as a part of the inclusion criteria, no interpreters were noted to be a part of the team visits. However, the caregivers reported that approximately a quarter of the visits included interpreters (M = 25%, SD 11.9, range = 10–50%).

Most respondents (84%) reported that they had enough time during their visits, though there were technical difficulties for almost a quarter of the visits (19% while logging in and 8% during the visit). Most often, if there was a technical difficulty, the visit was still able to be completed (74%). Most caregivers received written recommendations at the end of the visit either by reading the visit notes in the patient portal (62%), receiving a separate patient message through the portal (14%), or via a secure email (3%).

3.4.   Visit completion rates (Table 1)

During the study period (April–June 2020), there were 823 virtual visits scheduled, and 511 visits were completed (62% completion rate). Most uncompleted visits were late cancellations (N = 266, 52%), alongside a smaller number of no-show visits (N = 46, 6%). There were 263 unique patients seen in these visits, with an average of 1.94 visits during the three-month study period for families (range = 1–12; SD = 1.68).

During the similar 2019 period, there were 441 in-person visits scheduled and 284 visits were completed (64% completion rate). Similarly, the majority of uncompleted visits were late cancellations (N = 127, 45%), alongside a smaller proportion of no-shows (N = 30, 7%). There were 225 unique patients seen during this time period, with an average of 1.26 visits during the study period (range = 1–5; SD = 0.61). This was a significant increase in the number of scheduled and completed visits from 2019 to 2020 during the telehealth study period. However, there were not significant differences in the no-show and late cancellation rates.

3.5.   Telehealth satisfaction—caregiver (Table 2)

The majority of caregivers reported that they felt satisfied with the virtual appointments (91%) and found that it was easier than going to the hospital to see the specialists (83%). Overall, the caregivers noted that they were able to talk about their concerns for their child and that the providers heard their concerns. Almost all participants noted they would be interested in continued telehealth visits in the future (94%). There were a few areas where approximately 80% or fewer of the caregivers noted that they were satisfied, such as that the appointments were easy to make (81%) and that they felt virtual communication was as good as talking to the provider in person (78%). See Table 2 for the caregiver satisfaction rates.

Table 2.  Caregiver of patient telehealth satisfaction.

Question	Response	N (%)
Do you feel satisfied with the virtual appointment?	Yes	33 (92)
Was your experience of telemedicine appointment easier than going to Boston Children's Hospital to the see the specialists?	Yes	30 (83)
Did you feel the virtual communication was as good as talking to a doctor in person?	Yes	28 (78)
Did you feel that you could express yourself and your health concerns for your child?	Yes	35 (97)
Did you feel comfortable talking to a provider through telemedicine platform?	Yes	35 (97)
Did you feel providers heard your concerns?	Yes	34 (94)
Did you feel that providers understood the situation?	Yes	31 (89)
Did you feel your privacy was protected?	Yes	35 (97)
Did you feel that you received quality care?	Yes	33 (92)
Did you feel that it was easy to make the appointment?	Yes	29 (81)
Would you be interested in continued telehealth visits in the future?	Yes	34 (94)

---

 | Show Table

DownLoad: CSV

3.6.   Telehealth satisfaction—provider (Table 3)

All providers noted that they either agreed or strongly agreed that they were satisfied with telehealth visits, including the following conclusions: it was easy to set up the equipment for the visits, telehealth visits were more convenient for the patients/caregivers, the patients/caregivers seemed comfortable having telehealth appointments, the specialists listened to the patients/caregivers during the visit, they understood what the patients/caregivers were saying during the visits, the patients/caregivers effectively communicated, and that patients received high quality care. The majority of providers noted that there were technical problems during telehealth visits (42%). Additionally, most providers noted that seeing a specialist using telehealth was as effective as an in-person visit (67%); however, several providers reported either feeling neutral about this statement (17%) or disagreeing that it was as effective (17%).

Table 3.  Provider satisfaction with telehealth.

Question	Strongly agree N (%)	Agree N (%)	Neutral N (%)	Disagree N (%)	Strongly disagree N (%)
I feel satisfied with telemedicine.	4 (33)	8 (67)	0 (0)	0 (0)	0 (0)
It was easy to set up the equipment for telemedicine appointments	6 (50)	6 (50)	0 (0)	0 (0)	0 (0)
There were no technological problems during the appointments	0 (0)	3 (25)	4 (33)	5 (42)	0 (0)
For patients, seeing a specialist via telemedicine was as effective as seeing a specialist in person	0 (0)	8 (67)	2 (17)	2 (17)	0 (0)
Having a telemedicine appointment is more convenient for my patient than attending an appointment in person.	6 (50)	6 (50)	0 (0)	0 (0)	0 (0)
Patients seemed comfortable having a telemedicine appointment	6 (50)	6 (50)	0 (0)	0 (0)	0 (0)
I felt like patients' privacy was being protected in the appointments	7 (58)	4 (33)	1 (8)	0 (0)	0 (0)
I felt like the specialist(s) listened to what the patient told him/her during telemedicine appointments.	5 (42)	7 (58)	0 (0)	0 (0)	0 (0)
I felt like the specialist understood what the patient said during the telemedicine appointment.	6 (50)	6 (50)	0 (0)	0 (0)	0 (0)
I felt like the patient communicated effectively during the telemedicine appointment	5 (42)	7 (58)	0 (0)	0 (0)	0 (0)
The patient received high-quality care in this telemedicine appointment	6 (50)	6 (50)	0 (0)	0 (0)	0 (0)

---

 | Show Table

DownLoad: CSV

4.   Discussion

Both caregivers and providers were satisfied with telehealth services in the context of a pediatric feeding disorders clinic. This was not unexpected given the challenges of commuting to a major metropolitan area, including parking and associated time and financial costs. The audio-visual platform allows for clear communication and discussion, while also providing additional information for providers, primarily related to the feeding environment in the home. The multidisciplinary team could see highchairs, tables, feeding utensils, and drinking vessels, along with allowing visual confirmation of medications, supplements, and formulas, thus enhancing the visit.

Although we hypothesized that the visit completion rates would improve with telehealth, there was no significant difference in the visit completion rates between the study period with virtual visits and the comparative period with in-person visits. There was a significant increase in the number of visits scheduled and completed during the study period. This was likely due to virtual visits not needing physical clinic space, thus allowing for an increased flexibility in scheduling, as the room space did not limit the number of visits. This was particularly important during the pandemic lockdown period (during the study period), as it allowed providers to see patients and caregivers more frequently during a period of high caregiver and patient distress. Overall, the clinic added approximately 25% more openings by adding telehealth clinics on days when there had not previously been a clinic due to room availability. Additionally, between spring of 2019 and 2020, the clinic hired an additional behavioral medicine provider, which also further increased the number of visits completed, and the behavioral medicine capability increased from 4 patients per week to 20; however, the new provider's schedule was not full prior to the implementation of telehealth, reducing the impact of this change on the data shown.

General concerns noted in open-ended discussions (Figure 1) were associated with the availability of anthropometric data and measurements, which is a crucial part of these visits, as nutrition status is one of the most important assessments in children with feeding disorders. For visits without measurements, only general advice could be provided, and this necessitated sooner follow-ups or more numerous patient portal messages.

Technical difficulties were also a problem noted with virtual visits and continue to be a barrier to accessing care. While some caregivers were able to access the audio-visual platform via mobile devices or laptops, others had limited data on their wireless plans, lacked reliable internet connections, possessed outdated devices that were incompatible with the platform, or lacked understanding on how to use the platform. This highlights the importance of robust technical support and training for both providers and caregivers to enhance the overall telehealth experience and ensure its acceptability. Additionally, it highlights disparities in the care provided during the lockdown phase of the pandemic.

5.   Limitations

The current study highlighted areas of strength and areas that needed attention to maintain the sustainability of telehealth visits in a pediatric feeding clinic. However, there were several limitations. The findings may lack generalizability as only English-speaking families were included. Interpreters were available for every telehealth visit and could be easily accessed, but the hospital instructions for telehealth visits sent by email were initially not available in a wide variety of languages. If we had included non-English speaking participants, we hypothesize our visit completion rate may have been lower given previous literature that indicated that telehealth may not increase access to care across race/ethnicity, particularly for non-English speaking families [22].

Additionally, the response rate was low. Over 200 families were contacted, but only 40 (i.e., 20%) completed the surveys. While this number is low, web-based surveys are found to have lower response rates [23],[24], and it is consistent with satisfaction survey response rates during the time of the pandemic [4],[25], so was not unpredicted. We hypothesized a lower response rate due to the high levels of stress these caregivers experienced at baseline, so were ultimately satisfied with this response rate.

6.   Future directions

These findings indicate that caregivers and providers note high satisfaction with telehealth visits; however, several questions need to be addressed to ensure that telehealth continues to provide sustainable quality care. First, more data is needed on how many patients did or did not have accurate and updated measurements for telehealth visits. The accuracy of these measurements is particularly important. The program had several quality improvement initiatives around this, including the distribution of Bluetooth scales to families. These scales have Bluetooth functionality that paired to the users' cellphones and communicated data through the electronic patient portal to the providers, which allowed for more comprehensive home/virtual exams.

Although satisfaction is important from a consumer standpoint, quality of care is an important metric that was not reported in the current study. Future studies should include chart reviews and outcome measurements to ensure a quality of care beyond the perception of satisfaction/quality. This will provide a more comprehensive assessment of the effectiveness of telehealth services.

Finally, it is important to understand whether access to care with telehealth either improves or worsens health inequities. There was no difference in the completed visit rates with the use of telehealth platforms; other findings indicate that inequities may still exist, especially for those at the greatest need. In this same vein, there is also a need to expand this study in non-English speaking populations in the future.

7.   Conclusions

Both caregivers and providers of patients with pediatric feeding disorders indicated high levels of satisfaction with telehealth services, while also noting challenges and areas for improvement. Addressing these concerns, including access to accurate anthropometric measurements and ensuring equitable access to care, will be critical. Exploring additional metrics related to quality of care by using telehealth in healthcare delivery will also play a pivotal role in shaping the future of healthcare services in multidisciplinary settings.

Author contributions

The authors confirm contribution to the paper as follows: Ryan D. Davidson and Sarah Fleet: Study conception and design. Ryan D. Davidson: Data collection. Ryan D. Davidson and Sarah Fleet: Analysis and interpretation of results. Ryan D. Davidson, Rebecca Kramer and Sarah Fleet: Draft manuscript preparation. All authors reviewed the results and approved the final version of the manuscript.

Use of AI tools declaration

The authors declare that they have not used Artificial Intelligence (AI) tools in the creation of this article.