Evaluating a Web-based Adult ADHD Toolkit for Primary Care Clinicians

Natalia Y. Loskutova, MD, PhD; Cory B. Lutgen, BS; Elisabeth F. Callen, PhD, GStat; Melissa K. Filippi, PhD, MPH; Elise A. Robertson, MA


J Am Board Fam Med. 2021;34(4):741-752. 

In This Article


Sample Overview

Ninety-seven providers from 6 practices participated in the Toolkit evaluation and completed the baseline assessment. Five practices were described as large organizations with more than 15 practicing physicians in the clinic. Of these 5, 4 were affiliated with a University and 1 with a large state-wide health care organization. One practice was a small solo practice. All participating practices were located in urban areas in one of the following states: Georgia, Florida, New York, Indiana, Ohio, and Oregon, and geographically distributed in Southeast (n=2), Midwest (n=2), Northeast (n=1), and Pacific Northwest (n=1). Participating providers included 51 family medicine or general internal medicine physicians, 34 physician residents, 6 nonphysician clinicians, 5 mental and behavioral health specialists, and 1 pharmacist. Provider characteristics are presented in Table 1. All 97 participants completed the baseline survey, with 86 (87%) and 81 (84%) participants completing midpoint and end of study surveys respectively.

Care Patterns in Adult ADHD

Before beginning the study, participants completed a baseline survey where they reported that on average, they spent more than half of their time in direct clinical care (n=74, 87%). At the baseline, most respondents (n=64, 66%) estimated on average seeing approximately 1 to 6 adult patients with ADHD in a typical week. However, when asked to report weekly, a smaller number of providers reported seeing adult patients with ADHD (n=36, 37%) throughout the 17-week study period, averaging about 1 to 3 adult patients with ADHD in a week (range, 1 to 6). More than half of the study participants (n=61, 63%) reported that they did not see any patients with adult ADHD while in the study. The most frequent problems patients with ADHD had throughout the project as reported by the participants were related to prescription medication for ADHD and/or refills (39%), concentration/attention problems (9%), seeking a formal diagnosis of ADHD (8%), and other problems (19%).

Toolkit use and Utilization

Web analytics reports indicated that the Toolkit and all subpages received 394 page views, of which Toolkit Home Page (47%) and Assessment & Diagnosis (18%) were the most frequently visited sections. The printable/downloadable handouts were downloaded 158 times, with the top 3 being on diagnosis and assessment and patient self-management (Table 2). Most of the total Toolkit page views (66%) and total resource downloads (54%) occurred in the first 4 weeks of the evaluation. After the first month, the Toolkit visits remained stable averaging 8% of all visits over the remaining months.

An analysis of survey data indicated that 52 providers self-reported use of the Toolkit ("Users") and 45 self-reported they did not use the Toolkit ("Non-Users"). Postbaseline survey data were available for 45 Users and 26 Non-Users. On average, Toolkit Users reported they used it 1 to 9 times a week. Similar to the web analytics use, self-reported Toolkit use was the highest closer to the start of the study: 15 Users in the first week and 16 Users in the second week. The most frequent reason for using the Toolkit was self-education (29, 64%) and seeking guidance on diagnostic assessment tools and strategies (23, 51%). Having no patients with ADHD was listed as the top reason for not using the Toolkit.

Next, we examined the chartership of Users versus Non-Users. We hypothesized that seeing/having adult patients with ADHD, the number of patients with ADHD seen in a week at the baseline, and nonresident status in practice would be associated with the use of the Toolkit. The only significant variable associated with the use of the Toolkit was whether participants saw patients with ADHD over the evaluation period; if they did, they were 49.5 times more likely to use the Toolkit than participants who did not see patients with ADHD. The number of patients with ADHD seen in a week and the nonresident status did not contribute to the use of the Toolkit (Table 3).

Impact of Toolkit on Knowledge and Confidence

In both Toolkit Users and Non-Users, improvements in knowledge levels were observed over time (Table 4). When comparing Users versus Non-User, significant improvements in knowledge were observed in the following categories: "Monitoring for treatment effects, side effects and outcomes" (midpoint 3.6 vs baseline 3.0; P = .004); "Existing resources for patients and clinicians about adult ADHD" (midpoint 3.3 vs baseline 2.9; P = .03); and "Management of ADHD in patients with comorbid conditions" (midpoint 3.2 versus baseline 2.7; P = .01), and at the end of study (3.4 vs 2.9; P = .02). There were no differences between Users and Non-Users in any of these categories.

Improvements were observed over time in participants' confidence levels in all areas (Table 4). When comparing the end of study data to baseline data, the Toolkit Users reported significantly higher confidence levels than the Toolkit Non-Users in the following categories: "Mental health interview techniques and life history interview" (3.5 vs 3.0; P = .03); "Choosing treatment options for ADHD with comorbid mental health disorders" (3.2 vs 2.3; P < .001); and "Choosing treatment options for ADHD with coexisting substance use disorder" (3.0 vs 2.3; P = .003).

Perceived Value of the Toolkit in Addressing Provider Needs

When asked about the most pressing needs related to adult ADHD (Table 5), respondents indicated it is "better understanding for nonmedication management options, effectiveness and side effects" (n=42, 52%), followed by "better understanding diagnostic criteria and diagnostic process for adults" (n=33, 41%).

The Toolkit Users reported that in most cases the Toolkit addressed "some to most of their needs" related to adult ADHD (n=40, 78% at midpoint and n=47, 87% at end of study). Fewer respondents indicated that the Toolkit addressed "very few to none of their needs" (n=11, 22% at midpoint and n=7, 13% at end of study). At the end of the study, the majority of participants indicated they would benefit from additional adult ADHD resources easily accessible at the point of care and education related to adult ADHD (n=62, 87% and n=63, 86%, respectively).